iamcosban

Reporting Accurate Unit and End-to-End Test Coverage in Golang

I write a lot of code in go. It has a nice testing suite, but one thing that has frustrated me is that I had previously been unable to generate an accurate coverage report for my unit and end to end tests. Sleuthing through other users’ solutions had been unsuccessful my use case, so I will document my findings here. The specific problem is that I want to include my package isolated end-to-end test coverage in with my unit test coverage report. After working and refining the process for which tests are run and coverage reports are produced, I have finally discovered a solution which works. My go query builder/ORM tool, persistence, will be the code-base I use to in this example and the end result will generate a badge like this: Coverage Report

prerequisites

Ensure the following are installed and in your path

  • go
  • make
  • tail

Golang has its own build/test commands which work great. With the amount of work I have in gitlab’s CI, it’s easiest to just specify that a make command should be run and have that command do all of the work.

code

The first step is to actually create an end to end test. This example does so in an e2e package which imports the root package. It is a separate package from the root package so that the project may be compiled without including any end to end tests which may make use of external packages, or packages that are not necessary for the main project to function.

package e2e

import (
        "gitlab.com/cosban/persistence"
        "testing"
)

func TestOneToManyJoin(t *testing.T) {
	if testing.Short() { // skip if -short flag is used
		t.Skip("skipping e2e test")
	}
        // actual testing occurs below
        // ...
}

Next, create a makefile. The following is a stripped down version with only the properties necessary for this example, the full makefile is available in the repository.

#makefile
PROJECT := persistence
PKG := "gitlab.com/cosban/$(PROJECT)"

.PHONY: test clean

test: clean
	@GO111MODULE=on go test ./... -short -coverprofile coverage.log
	@GO111MODULE=on go test $(PKG)/e2e -coverpkg $(PKG) -coverprofile coverage.e2e.log
	@tail -n +2 coverage.e2e.log >> coverage.log
	@go tool cover -func=coverage.log

clean:
	@rm -f coverage*.log

explanation

  1. the test rule requires that the clean rule is run first
  2. end to end tests have been skipped in the first pass because the tests include a check for the -short flag
  3. the end to end tests are then run and is told to generate a report for its coverage against the root package
  4. tail is used to concatenate all but the first line of the coverage.e2e.log file into the coverage.log file
  5. the coverage.log file is then analyzed by the go coverage tool and a report is printed out

further adjustments

If one wanted to expand the coverage beyond just one package for their end to end tests, they could do so by running the test against a separate coverpkg. As of speaking, the -coverpkg=all method does not work for this use-case because it causes all packages to be imported by the test. This is undesirable for persistence because only one dialect driver package should be imported at a time.

integration with gitlab

Within a project, navigate to settings > CI / CD > Test coverage parsing and then enter the pattern which will be used to determine test coverage. I use total:\s+\(statements\)\s+\d+.\d+%.