Automatic Unit Testing in .NET Core plus Code Coverage in Visual Studio Code
https://github.com/shanselman/dotnetcoreunittestingwithcoverageinvscode
I was talking to Toni Edward Solarin on Skype yesterday about his open source spike (early days) of Code Coverage for .NET Core called "coverlet." There's a few options out there for cobbling together .NET Core Code Coverage but I wanted to see if I could use the lightest tools I could find and make a "complete" solution for Visual Studio Code that would work for .NET Core cross platform. I put my own living spike of a project up on GitHub.
Now, keeping in mind that Toni's project is just getting started and (as of the time of this writing) currently supports line and method coverage, and branch coverage is in progress, this is still a VERY compelling developer experience.
Using VS Code, Coverlet, xUnit, plus these Visual Studio Code extensions
- Coverage Gutters - Reads in the lcov.info file (name matters) and highlights lines with color
- .NET Core Test Explorer - Discovers tests and gives you a nice explorer.
- Coverlet - The start of .NET Core Code Coverage
Here's what we came up with.
There's a lot going on here but take a moment and absorb the screenshot of VS Code above.
- Our test project is using xunit and the xunit runner that integrates with .NET Core as expected.
- That means we can just "dotnet test" and it'll build and run tests.
- Added coverlet, which integrates with MSBuild and automatically runs when you "dotnet test" if you "dotnet test /p:CollectCoverage=true"
- (I think this should command line switch should be more like --coverage" but there may be an MSBuild limitation here.)
I'm interested in "The Developer's Inner Loop." . That means I want to have my tests open, my code open, and as I'm typing I want the solution to build, run tests, and update code coverage automatically the way Visual Studio proper does auto-testing, but in a more Rube Goldbergian way. We're close with this setup, although it's a little slow.
Coverlet can produce opencover, lcov, or json files as a resulting output file. You can then generate detailed reports from this. There is a language agnostic VS Code Extension called Coverage Gutters that can read in lcov files and others and highlight line gutters with red, yellow, green to show test coverage. Those lcov files look like this, showing file names, file numbers, coverage, and number of exceptions.
SF:C:githubhanselminutes-corehanselminutes.coreConstants.cs DA:3,0 end_of_record SF:C:githubhanselminutes-corehanselminutes.coreMarkdownTagHelper.cs DA:21,5 DA:23,5 DA:49,5
I should be able to pick the coverage file manually with the extension, but due to a small bug, it's easier to just tell Coverlet to generate a specific file name in a specific format.
I should be able to pick the coverage file manually with the extension, but due to a small bug, it's easier to just tell Coverlet to generate a specific file name in a specific format.
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov.info .my.tests
The lcov.info files then watched by the VSCode Coverage Gutters extension and updates as the file changes if you click watch in the VS Code Status Bar.
You can take it even further if you add "dotnet watch test" which will compile and re-run tests if code changes:
dotnet watch --project .my.tests test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov.info
I can run "WatchTests.cmd" in another terminal, or within the VS Code integrated terminal.
NOTE: If you're doing code coverage you'll want to ensure your tests and tested assembly are NOT the same file. You might be able to get it to work but it's easier to keep things separate.
Next, add in the totally under appreciated .NET Core Test Explorer extension (this should have hundreds of thousands of downloads - it's criminal) to get this nice Test Explorer pane:
Even better, .NET Test Explorer lights up some "code lens" style interfaces over each test as well as a green checkmark for passing tests. Having "debug test" available for .NET Core is an absolute joy.
Finally we make some specific improvements to the .vscode/tasks.json file that drives much of VS Code's experience with our app. The "BUILD" label is standard but note both the custom "test" and "testwithcoverage" labels, as well as the added group with kind: "test."
{
"version": "2.0.0",
"tasks": [
{
"label": "build",
"command": "dotnet",
"type": "process",
"args": [
"build",
"${workspaceFolder}/hanselminutes.core.tests/hanselminutes.core.tests.csproj"
],
"problemMatcher": "$msCompile",
"group": {
"kind": "build",
"isDefault": true
}
},
{
"label": "test",
"command": "dotnet",
"type": "process",
"args": [
"test",
"${workspaceFolder}/hanselminutes.core.tests/hanselminutes.core.tests.csproj"
],
"problemMatcher": "$msCompile",
"group": {
"kind": "test",
"isDefault": true
}
},
{
"label": "test with coverage",
"command": "dotnet",
"type": "process",
"args": [
"test",
"/p:CollectCoverage=true",
"/p:CoverletOutputFormat=lcov",
"/p:CoverletOutput=./lcov.info",
"${workspaceFolder}/hanselminutes.core.tests/hanselminutes.core.tests.csproj"
],
"problemMatcher": "$msCompile",
"group": {
"kind": "test",
"isDefault": true
}
},
]
}
This lets VS Code know what's for building and what's for testing, so if I use the Command Palette to "Run Test" then I'll get this dropdown that lets me run tests and/or update coverage manually if I don't want the autowatch stuff going.
Again, all this is just getting started but I've applied it to my Podcast Site that I'm currently rewriting and the experience is very smooth!