At AWS re:Invent 2019, AWS CodeBuild announced a new test reporting feature which can help make diagnosing test failures in CodeBuild much easier. You can read more about it here.

I wanted to use this feature for .NET and after a little research I was able to quickly add support for .NET tests to my CodeBuild projects. Let’s take a look at how I made this work.

Project Setup

For demonstration purposes I have a test project with the following tests. I admit these tests don’t really do anything but they will give me 3 passing tests and one failing test for a malformed Uri exception.

using System;

using System.Net;

using Xunit;



namespace CodeBuildDotnetTestReportExample.Tests

{

    public class ExampleTests

    {

        [Fact]

        public void TestSuccess1()

        {



        }





        [Fact]

        public void TestSuccess2()

        {



        }



        [Theory]

        [InlineData("https://www.google.com")]

        [InlineData("fffaaa")]

        public void TestMalformedUri(string uri)

        {

            new Uri(uri);

        }

    }

}

To run this project with CodeBuild I first started with a buildspec.yml file that built my project and ran my tests.

version: 0.2



phases:

    install:

        runtime-versions:

            dotnet: 3.1

    build:

        commands:

            - dotnet build -c Release ./CodeBuildDotnetTestReportExample/CodeBuildDotnetTestReportExample.csproj

            - dotnet test -c Release ./CodeBuildDotnetTestReportExample.Tests/CodeBuildDotnetTestReportExample.Tests.csproj

The first step to making my test reports was making sure the dotnet test command logged the test run. To do that I need to specify the logger format and where to put the logs. I changed the dotnet test command shown above to use the trx log format and to put the results in the ./testresults directory.

- dotnet test -c Release <project-path> --logger trx --results-directory ./testresults

What to do with a trx format file?

To have CodeBuild use the trx files to create the test reports I needed to add a new reports section to the buildspec.yml file. This snippet below grabs all of the xml files in the ./testresults directory to generate test reports under the DotnetTestExamples report group in CodeBuild.

    

reports:

    DotnetTestExamples:

        file-format: VisualStudioTrx

        files:

            - '**/*'

        base-directory: './testresults'

By default CodeBuild will assume the test log files to be in JUnit XML format. To make sure CodeBuild understands the trx xml files I needed to set the file-format field in my buildspec.yml file to VisualStudioTrx.

Here is now the full version of my buildspec.yml that will run by my build, execute tests and collect the test logs for a CodeBuild test report.

version: 0.2



phases:

    install:

        runtime-versions:

            dotnet: 3.1

    build:

        commands:

            - dotnet build -c Release ./CodeBuildDotnetTestReportExample/CodeBuildDotnetTestReportExample.csproj

            - dotnet test -c Release ./CodeBuildDotnetTestReportExample.Tests/CodeBuildDotnetTestReportExample.Tests.csproj --logger trx --results-directory ./testresults

reports:

    DotnetTestExamples:

        file-format: VisualStudioTrx

        files:

            - '**/*'

        base-directory: './testresults'

Setting up the build project

A CodeBuild project can be configured with a variety of options, for example what source control provider should be used, or whether to run as a stand alone job, or run as part of a pipeline. For this post I’m going to keep it simple and create a standalone CodeBuild project pointing to a GitHub repository. Here are the steps I used to create the CodeBuild project.

• Sign in to the AWS Management Console and navigate to CodeBuild
• Click Create build project
• Set a project name
• Select the GitHub repository
• Configure the Environment image
  • Operating System = Amazon Linux 2
  • Runtime = Standard
  • Image = aws/codebuild/amazonlinux2-x86_64-standard:3.0
  • Service Role = New service role
• Click Create build project to finish