前言
What's mocking and its benefits
Mocking is an integral part of unit testing. Although you can run your unit tests without use of mocking but it will drastically slow down the executing time of unit tests and also will be dependent on external resources. In this article we will explain mocking and different benefits that we can achieve by introducing mocking in our unit tests. - （Introduction to Mocking）
好文评鉴
Mocking and Isolation in Unit Testing - Vagif Abilov
Introduction
Rapid growth of test frameworks and tools accompanied by books and articles on various aspects of TDD does not only bring solutions to developers' needs, it also brings confusion. Dummy objects, fakes, stubs, mocks – what should we use? And when? And why? What complicates the situation is that most of developers can not afford to invest much time into test methodology studies – their efficiency is usually measured using different criteria, and although the management of modern software projects is well aware of how important is to surround production code with maintainable test environment, they are not easy to convince to dedicate hundreds of working hours into updating system tests due to a paradigm shift.
So we have to try to get things right with minimal effort, make our decisions based on simple practical criteria. And in the end, it is the compactness, readability and maintainability of our code that matters. So what I'm going to do is to write a very simple class with dependencies on a database, write an integration test that illustrates how inconvenient is to drag these dependencies with such simple class, and then show how to break the dependencies: first using traditional mocking with recorded expectations, and then using more lightweight arrange-act-assert approach that does not require accurate reconstruction of mocked object behavior.
The code that illustrates this article uses Typemock Isolator framework.
1. Code to test: calculating insurance price group
Our goal is to write tests for an algorithm that calculates a price group for car insurance. For simplicity the algorithm is based only on customer age: people under 16 fall into a Child group (with great chances to be refused), people between 16 and 25 belong to Junior group, age of 25 to 65 puts them into an Adult group, and anyone older is considered to be a Senior. Here's the code:
public class CarInsurance
{
public PriceGroup GetCustomerPriceGroup(int customerID)
{
DataLayer dataLayer = new DataLayer();
dataLayer.OpenConnection();
Customer customer = dataLayer.GetCustomer(customerID);
dataLayer.CloseConnection();
DateTime now = DateTime.Now;
if (customer.DateOfBirth > now.AddYears(-16))
return PriceGroup.Child;
else if (customer.DateOfBirth > now.AddYears(-25))
return PriceGroup.Junior;
else if (customer.DateOfBirth < now.AddYears(-65))
return PriceGroup.Senior;
else
return PriceGroup.Adult;
}
}
The code is simple, but note a potential test challenge. The method GetCustomerPriceGroup does not take an instance of a Customer type, instead it requires a customer ID to be sent as an argument, and database lookup occurs inside the method. So if we don't use any stubs or mocks, we'll have to create a Customer record and then pass it's ID to GetCustomerPriceGroup method.
Another issue is a visibility of Customer constructor. This is how it is defined:
public class Customer
{
internal Customer()
{
}
public int CustomerID { get; internal set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime DateOfBirth { get; set; }
}
Our tests will be written in a different assembly, so we won't be able to create a Customer object directly. It is supposed to be instantiated only from a DataLayer object that is defined below:
public class DataLayer
public int CreateCustomer(string firstName, string lastName, DateTime dateOfBirth)
{
throw new Exception("Unable to connect to a database");
}
public Customer GetCustomer(int customerID)
{
throw new Exception("Unable to connect to a database");
}
internal void OpenConnection()
{
throw new Exception("Unable to connect to a database");
}
internal void CloseConnection()
{
throw new Exception("Unable to connect to a database");
}
Of course, the real DataLayer definition won't throw exceptions: it will perform a series of well-known steps: obtaining connection strings, connecting to a database, implementing IDisposable interface, executing SQL queries and retrieving results. But for us it does not matter because the purpose of our work is to write and run test code without connecting to a database. Therefore it makes no difference if I instantiate and execute an SqlCommand or simply throw an exception: we don't expect a database to be reachable, we haven't created required tables and stored procedures, and any attempt to execute an SQL query will fail. So let's not focus on database code, we assume other developers will fix it.
2. Integration tests
So we are ready to test GetCustomerPriceGroup method. Here's how the body of a test might look:
var carInsurance = new CarInsurance(); PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(1); Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
No, that won't work. Notice hard-coded customer ID passed to GetCustomerPriceGroup method. We must first create a customer that would belong to an adult price group and pass the ID returned by CreateCustomer to a GetCustomerPriceGroup call. Data creation API often is separated from data retrieval API, so in a larger system CreateCustomer method might reside in a different assembly. Or even worse – not even available to us for security reasons. In case it's available, we have to learn new API, just for use in our test code. It shifts our focus from our main job – write and test CarInsurance class.
Here's integration test code:
[Test]
public void GetCustomerPriceGroup_Adult()
{
var dataLayer = new DataLayer();
int customerID = dataLayer.CreateCustomer("John", "Smith",
new DateTime(1970, 1, 1, 0, 0, 0));
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(customerID);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
If we compile and run this test we will receive the following output:
TestCase 'UnitTests.CarInsuranceTest1_Integration.GetCustomerPriceGroup_Adult'
failed: System.Exception : Unable to connect to a database
C:ProjectsNETTypeMockAAADataLayer.cs(12,0): at Customers.DataLayer.CreateCustomer(
String firstName, String lastName, DateTime dateOfBirth)
C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest1.cs(19,0):
at UnitTests.CarInsuranceTest1_Integration.GetCustomerPriceGroup_Adult()
Not very encouraging, isn't it? We spent some time learning customer creation API, expanded our code to create a customer record – only to find out that the database is not available. Some developers might object that this is not a bad experience: we need integration tests at some level. Yes, but should we write any test as an integration test? We are testing a very simple computational algorithm. Should this work include setting up a database and its connection strings and learning customer creation API that we haven't used until now?
I must admit that many (actually too many) tests that we have in our company have been written like the one above: they are integration tests. And this is probably one of the reasons the average test code coverage for our projects is still at 50-60%: we don't have time to cover all code. Writing integration tests requires extra effort, and fixing broken integration tests is even bigger effort over time. It's useful when hundreds of integration tests fail for different reasons. It becomes boring when they all fail due to a change in a single place. So we can draw a couple of conclusions:
- Writing integration tests requires learning additional APIs. It may also require referencing additional assemblies. This makes developer's work less efficient and test code more complex.
- Running integration tests requires setting up and configuring access permissions to external resources (such as database).
In the next section we will see how we can avoid this.
3. Unit tests with traditional mocking
Now that we know that writing integration tests when the actual purpose is to test a small unit of code is not a good idea, let's look at what we can do. One approach is to inherit DataLayer class from IDataLayer interface and then implement an IDataLayer-derived stub which would return a Customer object of our choice. Note that we won't just need to change DataLayer implementation, we will also have to change visibility of Customer constructor that is currently defined as internal. And while this will obviously make our design more testable, it won't necessarily make it better. I am all for use of interfaces, but not for relaxing visibility constraints without important reasons. But isn't class testability important enough? I am not sure. Bear in mind that making class instantiation public does not open it only for testing. It also opens it for improper use. Luckily, starting from .NET 2.0 an assembly attribute InternalsVisibleTo opens internal definition just for explicitly selected assemblies.
Anyway, we'll take another approach: we'll use mocking. No need to change implementation of other classes, no need to define new interfaces. Using TypeMock framework a new version of our test will look like this:
public void GetCustomerPriceGroup_Adult()
{
Customer customer = MockManager.MockObject<Customer>().Object;
customer.DateOfBirth = new DateTime(1970, 1, 1, 0, 0, 0);
using (RecordExpectations recorder = new RecordExpectations())
{
var dataLayer = new DataLayer();
recorder.ExpectAndReturn(dataLayer.GetCustomer(0), customer);
}
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(0);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
As you can see, we recorded our expectations regarding GetCustomer method behavior. It will return a Customer object with properties that we expect so we can use this object to test GetCustomerPriceGroup.
We compile and run the test, and here’s what we get:
TestCase 'UnitTests.CarInsuranceTest2_NaturalMocks.GetCustomerPriceGroup_Adult' failed: System.Exception : Unable to connect to a database C:ProjectsNETTypeMockAAADataLayer.cs(22,0): at Customers.DataLayer.OpenConnection() C:ProjectsNETTypeMockAAACarInsurance.cs(13,0): at Customers.CarInsurance.GetCustomerPriceGroup(Int32 customerID) C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest2.cs(31,0): at UnitTests.CarInsuranceTest2_NaturalMocks.GetCustomerPriceGroup_Adult() at TypeMock.VerifyMocksAttribute.Execute() at TypeMock.MethodDecorator.e() at TypeMock.MockManager.a(String A_0, String A_1, Object A_2, Object A_3, Boolean A_4, Object[] A_5) at TypeMock.InternalMockManager.getReturn(Object that, String typeName, String methodName, Object methodParameters, Boolean isInjected) C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest2.cs(20,0): at UnitTests.CarInsuranceTest2_NaturalMocks.GetCustomerPriceGroup_Adult()
Still the same exception! Unable to connect to a database. This is because we can’t just set an expectation on a method that returns a long awaited object, we have to record the whole chain of calls on a mocked DataLayer object. So what we need to add is calls that open and close database connections. A revised (and first successful) version of a test looks like this:
[Test]
[VerifyMocks]
public void GetCustomerPriceGroup_Adult()
{
Customer customer = MockManager.MockObject<Customer>().Object;
customer.DateOfBirth = new DateTime(1970, 1, 1, 0, 0, 0);
using (RecordExpectations recorder = new RecordExpectations())
{
var dataLayer = new DataLayer();
dataLayer.OpenConnection();
recorder.ExpectAndReturn(dataLayer.GetCustomer(0), customer);
dataLayer.CloseConnection();
}
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(0);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
And by the way, we had to add an assembly attribute InternalsVisibleTo to grant access to methods OpenConnection and CloseConnection that were not public.
Using mock objects helped us isolate the code to test from database dependencies. It didn’t however fully isolate the code from a class that connects to a database. Moreover, if you look at the test code wrapped within the RecordExpections block, you will easily recognize a part of the original GetCustomerPriceGroup method code. This smells code duplication with its notorious consequences. We can conclude the following:
- Mocking requires the knowledge of mocked object behavior which should not be necessary when writing unit tests.
- Setting a sequence of behavior expectations requires call chain duplication from the original code. It won’t gain you more robust test environment, quite opposite - it will require additional code maintenance.
4. Unit tests with isolation
So what can we improve in the above scenario? Obviously, we don’t want to eliminate a call to GetCustomer, since it is from that method that we want to obtain a customer with a specific state. But this is the only method that interests us, everything else in DataLayer is irrelevant to us in the given context. Can we manage to write our tests only referencing GetCustomer method from DataLayer class?
Yes, we can, with some help from mocking framework. As I mentioned earlier, our company uses TypeMock Isolator that recently has been upgraded with support for exactly what we’re trying to achieve. Here’s how it works:
- Create an instance of Customer object with required properties.
- Create a fake instance of DataLayer object.
- Set the behavior of a call to GetCustomer that will return previously created Customer object.
What is the difference with an approach that we used in a previous section? The difference is that we no longer need to care about additional calls made on DataLayer object – only about calls that affect states used in our test. Here’s the code:
[Test]
[Isolated]
public void GetCustomerPriceGroup_Adult()
{
var customer = Isolate.Fake.Instance<Customer>();
Isolate.WhenCalled(() => customer.DateOfBirth)
.WillReturn(new DateTime(1970, 1, 1, 0, 0, 0));
var dataLayer = Isolate.Fake.Instance<Datalayer>();
Isolate.SwapNextInstance<Datalayer>().With(dataLayer);
Isolate.WhenCalled(() => dataLayer.GetCustomer(0)).WillReturn(customer);
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(0);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
Note that the first lines that fake a Customer object are needed only because Customer constructor is not public, otherwise we could have created its instance directly. So the lines that are essential are the three lines where we create a DataLayer fake, swap creation of next instance with the faked object and then set the expectation on GetCustomer return value. And after the article was published I received a comment with suggestion how to completely eliminate creation of Customer instance: since the only expectation regarding customer state is his/her birth date, it can be set right away, without instantiating Customer object first:
[Test]
[Isolated]
public void GetCustomerPriceGroup_Adult()
{
var dataLayer = Isolate.Fake.Instance<Datalayer>();
Isolate.SwapNextInstance<Datalayer>().With(dataLayer);
Isolate.WhenCalled(() => dataLayer.GetCustomer(0).DateOfBirth)
.WillReturn(new DateTime(1970, 1, 1, 0, 0, 0));
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(0);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
But what happened to OpenConnection and CloseConnection? Are they just ignored? And what if they returned values? What values would they then return?
It all depends on how the fake is created. Isolate.Fake.Instance has an overload that takes as an argument fake creation mode. The possible modes are represented in Members enumerator:
- MustSpecifyReturnValues – this is default that was used in the code above. All void methods are ignored, and values for the methods with return values must be specified using WhenCalled, just like we did. If return value is not specified, an attempt to execute a method with return value will cause an exception.
- CallOriginal – this is the mode to make the mocked object running as if it was not mocked except for cases specified using WhenCalled.
- ReturnNulls – all void methods are ignored, and those that are not void will return null or zeroes.
- ReturnRecursiveFakes – probably the most useful mode for isolation. All void methods are ignored, and those that return values will return fake value unless specific value is set using WhenCalled. This behavior is applied recursively.
For better understanding of how this works, let’s play with our test code. We begin by removing expectation on Customer state:
[Test]
[Isolated]
public void GetCustomerPriceGroup_Adult()
{
var dataLayer = Isolate.Fake.Instance<Datalayer>();
Isolate.SwapNextInstance<Datalayer>().With(dataLayer);
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(0);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
What do you think should happen here? Let’s see. OpenConnection and CloseConnection will be ignored. GetCustomer can’t be ignored because it returns a value. Since we didn’t specify any fake creation mode, a default one, MustSpecifyReturnValues will be used. So we must specify return value for GetCustomer. And we didn’t. Here what happens if we run the test:
TestCase 'UnitTests.CarInsuranceTest3_ArrangeActAssert.GetCustomerPriceGroup_Adult' failed: TypeMock.VerifyException : TypeMock Verification: Unexpected Call to Customers.DataLayer.GetCustomer() at TypeMock.MockManager.a(String A_0, String A_1, Object A_2, Object A_3, Boolean A_4, Object[] A_5) at TypeMock.InternalMockManager.getReturn(Object that, String typeName, String methodName, Object methodParameters, Boolean isInjected, Object p1) C:ProjectsNETTypeMockAAADataLayer.cs(16,0): at Customers.DataLayer.GetCustomer( Int32 customerID) C:ProjectsNETTypeMockAAACarInsurance.cs(14,0): at Customers.CarInsurance.GetCustomerPriceGroup(Int32 customerID) C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest3.cs(42,0): at UnitTests.CarInsuranceTest3_ArrangeActAssert.GetCustomerPriceGroup_Adult () at TypeMock.MethodDecorator.CallRealMethod() at TypeMock.DecoratorAttribute.CallDecoratedMethod() at TypeMock.ArrangeActAssert.IsolatedAttribute.Execute() at TypeMock.MethodDecorator.e() at TypeMock.MockManager.a(String A_0, String A_1, Object A_2, Object A_3, Boolean A_4, Object[] A_5) at TypeMock.InternalMockManager.getReturn(Object that, String typeName, String methodName, Object methodParameters, Boolean isInjected) C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest3.cs(37,0): at UnitTests.CarInsuranceTest3_ArrangeActAssert.GetCustomerPriceGroup_Adult ()
It’s fair, isn’t it? Any call to a method with return value is unexpected – values must be assigned first.
But what if we do the same but set the fake creation mode to ReturnRecursiveFakes? In this case Isolator will have to create an instance of a Customer object that will be returned by DataLayer.GetCustomer. Let’s try:
[Test]
[Isolated]
public void GetCustomerPriceGroup_Adult()
{
var dataLayer = Isolate.Fake.Instance<Datalayer>(Members.ReturnRecursiveFakes);
Isolate.SwapNextInstance<Datalayer>().With(dataLayer);
var carInsurance = new CarInsurance();
PriceGroup priceGroup = carInsurance.GetCustomerPriceGroup(0);
Assert.AreEqual(PriceGroup.Adult, priceGroup, "Incorrect price group");
}
And here’s the output:
TestCase 'UnitTests.CarInsuranceTest3_ArrangeActAssert.GetCustomerPriceGroup_Adult' failed: Incorrect price group Expected: Adult But was: Senior C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest3.cs(55,0): at UnitTests.CarInsuranceTest3_ArrangeActAssert.GetCustomerPriceGroup_Adult() at TypeMock.MethodDecorator.CallRealMethod() at TypeMock.DecoratorAttribute.CallDecoratedMethod() at TypeMock.ArrangeActAssert.IsolatedAttribute.Execute() at TypeMock.MethodDecorator.e() at TypeMock.MockManager.a(String A_0, String A_1, Object A_2, Object A_3, Boolean A_4, Object[] A_5) at TypeMock.InternalMockManager.getReturn(Object that, String typeName, String methodName, Object methodParameters, Boolean isInjected) C:ProjectsNETTypeMockAAAUnitTestsCarInsuranceTest3.cs(49,0): at UnitTests.CarInsuranceTest3_ArrangeActAssert.GetCustomerPriceGroup_Adult()
That’s interesting. Why did we get a senior customer? Because Isolator created a Customer instance with default properties, setting therefore DateOfBirth to 01.01.0001. How can such an old customer not be treated as a senior?
Conclusion: write less test code, write good test code
We have looked at different methods of writing test code: integration tests, unit tests based on recording of the expected behavior, and unit tests with isolating the class under test from any irrelevant aspects of code execution. I believe the latter approach is a winner in many cases: it lets you set the state of objects that you are not interested to test in a lightweight straightforward manner. Not only it lets you focus on the functionality being tested, it also saves the time you would need to spend in the future updating recorded behavior affected by design changes.
I know I am at risk of oversimplifying the choice. If you haven’t read Martin Fowler’s article "Mocks Aren’t Stubs", you can find there both definitions of terms and reasoning behind the selection of it. But with all respect to the problem complexity, I think we should not ignore such simple criteria as compactness and maintainability of the test code combined with using the original code to test “as is”, without revising it for better testability. These are strong arguments, and they justify selection of a lightweight "arrange-act-assert" method shown in this article.
I think what we’re observing now is a beginning of a new phase in test methodology: applying the same strict rules to test code quality that we have been using for production code. In theory same developers should write both production and test code with the same quality. In practice test code has always been a subject to compromises. Look for example at developers’ acceptance of low code coverage: they may target 80% code coverage but won’t postpone a release if they reach only 60%. Can you imagine that they could release a product with 20% less functions without management approval?
Such attitude had practical and even moral grounds. On the practical side, test code was treated as code of a second priority. This code is not supposed to be deployed, it is for internal use, so it received lower attention. And on the moral side, when automated testing is so underused every developer’s effort to write automated tests should be sacred. Criticizing a developer writing tests for writing bad tests was like criticizing an animal abuse fighter for not fighting smart. But we must leave behind all romantics around unit testing. It has to obey well-defined rules of software development. And one of those rules is "write less code".
One of the statements that played positive role in early stage of test-driven development and that should be considered harmful now is “there can’t be too many tests”. Yes there can. Over several years we’ve been writing tests for systems consisting of several complex layers and external integration points that were hard or impossible to reach from test environment. We were trying to hit a logical error one way or another, and usually succeeded. However, speaking for myself I noticed that I developed a relaxed attitude about not catching an error in the first place: in the code that was written specifically to test a given function. This is a dangerous attitude. It lowers an effort dedicated to covering each class with unit tests that would validate all aspects of its behavior. "If unit tests for A does not expose all errors, then unit tests for B may expose them. In the worst case they will be exposed by integration tests". And I’ve seen many times that when developers had to find an error that occurred in production, they managed to write new integration test that failed (production errors often are easier to reproduce from integration tests), then they traced this test in a debugger, identified an offensive module and fixed the error there. After the fix they made sure that the integration test succeeded and considered the issue solved. They did not write a new unit test for the affected module. "Why? We already wrote one." After a while, the system is covered by thousands of tests, but it is often possible to introduce an error in a module without breaking its dedicated tests.
For a long period this was hard to blame. There was so much excitement around daily builds, nightly tests, traffic light-alike test runners that some developers even managed to wire to large industrial LED boards so every visitor could know who broke the last build. It was great, and it’s not over. We just have to start measuring efficiency of our test efforts. If unit test code written to test module A exposes a bug in module B, and this bug is not exposed by unit tests written to test B, then you have some work to do. And if a logical bug is exposed by integration tests, you also have work to do. Integration tests should only expose problems related to configuration, databases and communication with external resources. Anything else is subject to unit tests written specifically to validate respective functions. It is bad practice to duplicate test code. It is bad practice to bring many dependencies to a test module. It is bad practice to make test code aware of anything that is not related to states used by the code under test. And this is where latest development in TDD tools and frameworks can provide us with great help. We only need to start changing our old habits.
写在最后
If you want to go further and learn unit testing in depth using mocking frameworks such as Moq, FakeItEasy and Typemock Isolator, I highly recommend checking out The Art of Unit Testing: with examples in C#