Machine.Specifications
Machine.Specifications (MSpec) is a context/specification framework that removes language noise and simplifies tests. All it asks is that you accept the = () =>
. Keep up with the latest news and discussions or follow the maintainers, @agross, CTO of GROSSWEBER and @danielmarbach.
Installation
You can download the unsigned binaries (recommended) or the signed binaries directly from the TeamCity server. But, we recommended installing the NuGet package. Install on the command line from your solution directory:
cmd> nuget install Machine.Specifications
cmd> nuget install Machine.Specifications.Should
# or:
cmd> nuget install Machine.Specifications-Signed
cmd> nuget install Machine.Specifications.Should-Signed
Or use the Package Manager console in Visual Studio:
PM> Install-Package Machine.Specifications
PM> Install-Package Machine.Specifications.Should
# or:
PM> Install-Package Machine.Specifications-Signed
PM> Install-Package Machine.Specifications.Should-Signed
Usage
MSpec is called a "context/specification" test framework because of the "grammar" that is used in describing and coding the tests or "specs". That grammar reads roughly like this
When the system is in such a state, and a certain action occurs, it should do such-and-such or be in some end state.
You should be able to see the components of the traditional Arrange-Act-Assert model in there. To support readability and remove as much "noise" as possible, MSpec eschews the traditional attribute-on-method model of test construction. It instead uses custom .NET Delegates that you assign anonymous methods and asks you to name them following a certain convention.
Read on to construct a simple MSpec styled specification class.
Subject
The Subject
attribute is the first part of a spec class. It describes the "context", which can be the literal Type
under test or a broader description. The subject is not required, but it is good practice to add it. Also, the attribute allows ReSharper to detect context classes such that delegate members will not be regarded as unused.
The class naming convention is to use Sentence_snake_case
and to start with the word "When".
[Subject("Authentication")] // a description
[Subject(typeof(SecurityService))] // the type under test
[Subject(typeof(SecurityService), "Authentication")] // or a combo!
public class When_authenticating_a_user { ... } // remember: you can only use one Subject Attribute!
Tags
The Tags
attribute is used to organize your spec classes for inclusion or exclusion in test runs. You can identify tests that hit the database by tagging them "Slow" or tests for special reports by tagging them "AcceptanceTest".
Tags can be used to include or exclude certain contexts during a spec run.
[Tags("RegressionTest")] // this attribute supports any number of tags via a params string[] argument!
[Subject(typeof(SecurityService), "Authentication")]
public class When_authenticating_a_user { ... }
Establish
The Establish
delegate is the "Arrange" part of the spec class. The establish will only run once, so your assertions should not mutate any state or you may be in trouble.
[Subject("Authentication")]
public class When_authenticating_a_new_user
{
Establish context = () =>
{
// ... any mocking, stubbing, or other setup ...
Subject = new SecurityService(foo, bar);
};
static SecurityService Subject;
}
Cleanup
The pair to Establish is Cleanup
, which is also called once after all of the specs have been run.
[Subject("Authentication")]
public class When_authenticating_a_user
{
Establish context = () =>
{
Subject = new SecurityService(foo, bar);
};
Cleanup after = () =>
{
Subject.Dispose();
};
static SecurityService Subject;
}
Because
The Because
delegate is the "Act" part of the spec class. It should be the single action for this context, the only part that mutates state, against which all of the assertions can be made. Most Because
statements are only one line, which allows you to leave off the squiggly brackets!
[Subject("Authentication")]
public class When_authenticating_a_user
{
Establish context = () =>
{
Subject = new SecurityService(foo, bar);
};
Because of = () => Subject.Authenticate("username", "password");
static SecurityService Subject;
}
If you have a multi-line because statement, you probably need to identify which of those lines are actually setup and move them into the establish. Or, your spec may be concerned with too many contexts and needs to be split or the subject-under-test needs to be refactored.
It
The It
delegate is the "Assert" part of the spec class. It may appear one or more times in your spec class. Each statement should contain a single assertion, so that the intent and failure reporting is crystal clear. Like Because
statements, It
statements are usually one-liners and may not have squiggly brackets.
[Subject("Authentication")]
public class When_authenticating_an_admin_user
{
Establish context = () =>
{
Subject = new SecurityService(foo, bar);
};
Because of = () => Token = Subject.Authenticate("username", "password");
It should_indicate_the_users_role = () => Token.Role.ShouldEqual(Roles.Admin);
It should_have_a_unique_session_id = () => Token.SessionId.ShouldNotBeNull();
static SecurityService Subject;
static UserToken Token;
}
An It
statement without an assignment will be reported by the test runner in the "Not implemented" state. You may find that "stubbing" your assertions like this helps you practice TDD.
It should_list_your_authorized_actions;
Assertion Extension Methods
As you can see above, the It
assertions make use of these Should
extension methods. They encourage readability and a good flow to your assertions when read aloud or on paper. You should use them wherever possible, just "dot" off of your object and browse the IntelliSense!
It's good practice to make your own Should
assertion extension methods for complicated custom objects or domain concepts.
Ignore
Every test framework lets you ignore incomplete or failing (we hope not) specs, MSpec provides the Ignore
attribute for just that. Just leave a note describing the reason that you ignored this spec.
[Ignore("We are switching out the session ID factory for a better implementation")]
It should_have_a_unique_session_id = () => Token.SessionId.ShouldNotBeNull();
Catch
When testing that exceptions are thrown from the "action" you should use a Catch
statement. This prevents thrown exceptions from escaping the spec and failing the test run. You can inspect the exception's expected properites in your assertions.
[Subject("Authentication")]
public class When_authenticating_a_user_fails_due_to_bad_credentials
{
Establish context = () =>
{
Subject = new SecurityService(foo, bar);
};
Because of = () => Exception = Catch.Exception(() => Subject.Authenticate("username", "password"));
It should_fail = () => Exception.ShouldBeOfType<AuthenticationFailedException>();
It should_have_a_specific_reason = () => Exception.Message.ShouldContain("credentials");
static SecurityService Subject;
static Exception Exception;
}
Command Line Reference
MSpec, like other testing frameworks, provides a robust command-line runner that can be used to execute specs in one or more assemblies and allows a number of output formats to suit your needs. The runner comes in different flavors:
mspec.exe
, AnyCPU, runs on the CLR 2.0mspec-x86.exe
, x86, runs on the CLR 2.0mspec-clr4.exe
, AnyCPU, runs on the CLR 4.0mspec-x86-clr4.exe
, x86, runs on the CLR 4.0
Usage of the command-line runner is as follows (from mspec.exe --help
):
Usage: mspec.exe [options] <assemblies>
Options:
-f, --filters Filter file specifying contexts to execute (full type name, one per line). Takes precedence over tags
-i, --include Executes all specifications in contexts with these comma delimited tags. Ex. -i "foo,bar,foo_bar"
-x, --exclude Exclude specifications in contexts with these comma delimited tags. Ex. -x "foo,bar,foo_bar"
-t, --timeinfo Shows time-related information in HTML output
-s, --silent Suppress progress output (print fatal errors, failures and summary)
-p, --progress Print dotted progress output
-c, --no-color Suppress colored console output
-w, --wait Wait 15 seconds for debugger to be attached
--teamcity Reporting for TeamCity CI integration (also auto-detected)
--no-teamcity-autodetect Disables TeamCity autodetection
--html <PATH> Outputs the HTML report to path, one-per-assembly w/ index.html (if directory, otherwise all are in one file)
--xml <PATH> Outputs the XML report to the file referenced by the path
-h, --help Shows this help message
Usage: mspec.exe [options] <assemblies>
TeamCity Reports
MSpec can output TeamCity service messages to update the test run status in real time. This feature is enabled by passing the --teamcity
switch, but the command-line runner can auto-detect that it is running in the TeamCity context.
HTML Reports
MSpec can output human-readable HTML reports of the test run by passing the --html
option. If a filename is provided, the output is placed at that path, overwriting existing files. If multiple assemblies are being testing, the output is grouped into a single file. If no filename is provided, it will use the name of the assembly(s). If multiple assemblies are being tested, an index.html
is created with links to each assembly-specific report. You can use this option if your CI server supports capturing HTML as build reports.
XML Reports
MSpec can output XML test run reports by passing the --xml
option. This option behaves the same as the --html
option, in terms of file naming.
Selenium Reports
The MSpec HTML reports can show additional Selenium-specific information, like screenshots and debug statements. Instructions on how to integrate this feature into your specs is available on the web. There is also a sample implementation available.
ReSharper Integration
MSpec provides a batch file to integrate with the ReSharper test runner, custom naming rules, and code annotations. MSpec currently supports ReSharper 6.1, 7.0, 7.1, 8.0 and 8.1.
Code Annotations
By default, ReSharper thinks that specification classes (those with the [Subject]
attribute) and their internals are unused. To change this behavior in Visual Studio:
- Open the ReSharper Options (ReSharper -> Options...)
- Select "Code Annotations"
- Ensure that the namespace "Machine.Specifications.Annotations" is checked
- Click "OK"
Templates
The file, live, and surround templates can be imported from MiscReSharper.*.DotSettings
. The single file template creates a basic context. The single surround template wraps a Catch.Exception
call (more information how to use them). The live templates cover the major delegates:
mse
, anEstablish
delegatemsb
, aBecause
delegatemsi
, anIt
delegatemsf
, a failingIt
delegate, use in combination with theCatch
surround template
TestDriven.Net Integration
MSpec provides a batch file for setting up TD.NET integration. Newer versions (2.24+) support an xcopy integration that avoids the versioning issues arising from the registry-based scheme. If you use NuGet, you're already set. If you're not using NuGet, make sure to copy Machine.Specifications.dll.tdnet
and Machine.Specifications.TDNetRunner.dll
to your project's output directory.