Testing asynchronous code: async vs fake async

In the last post I explored implementing a mock which tested asynchronous code in a “fake” asynchronous way, and I promised to dive a little deeper into that concept and compare it with testing in an asynchronous way.

I say “fake” here because it’s still using async/await, but the way of testing is more of a step by step approach where the unit test ends up effectively waiting on each task or promise to finish before moving on.

Angular has really clear examples of each pattern, so let’s see the differences and compare the pros and cons for each.

Async

This is the patterns I think C# developers are more familiar with, although I’ve seen it used in TypeScript as well. It generally involves the three traditional sections: . Here’s an example of a test using this pattern:

it("should display some links when the user is not logged in", async(() => {
    // Arrange (some arranging was already done in beforeEach as well)
    let authenticationService = TestBed.get(AuthenticationService) as AuthenticationService;
    spyOn(authenticationService, "userInfo").and.returnValue(new BehaviorSubject(notLoggedInUser));

    // Act (detectChanges ends up triggering ngOnInit which actually does the work we're testing)
    fixture.detectChanges();
    fixture.whenStable().then(() => {
        fixture.detectChanges();

        // Assert
        let expectedLinks: { text: string, url: string }[] = [ /* Omitted for brevity */ ];
        let navItems = fixture.debugElement.queryAll(By.css(".nav-item"));
        expect(navItems).not.toBeNull();
        expect(navItems.length).toEqual(expectedLinks.length);

        for (let i = 0; i < navItems.length; i++) {
            let link = navItems[i].query(By.css(".nav-link"));
            expect(link).not.toBeNull();

            let expectations = expectedLinks[i];
            expect(link.nativeElement.innerText).toEqual(expectations.text);
            expect(link.attributes.href).toEqual(expectations.url);
        }

        expect(authenticationService.userInfo).toHaveBeenCalled();
    });
}));

I won’t go into detail about the test as it was pulled from a real example, but the pattern should look familiar. You start in the Arrange section mocking calls and setting up the test, then you trigger the code that’s actually being tested, then finally you assert your expectations on any results or side-effects.

A minor note for those unfamiliar with the use of async here, it’s equivalent to using the done construct and calling done after the last expectation. Or for any C# developers, it’s equivalent to returning a Task from your test. The test framework just waits for all async work to finish before the test ends.

Fake Async

This pattern is used pretty widely in Angular apps, especially when mocking http calls. It’s also how I implemented my Testing.HttpClient C# library. It differs from the usual “Arrange, Act, and Assert” and instead interleaves and combines all three. Here’s an example:

it("should return some data", fakeAsync(() => {
    // Act
    let response: IUser;
    let error: HttpErrorResponse;
    userService.getUser(userName)
        .then((r: IUser) => response = r)
        .catch((e: HttpErrorResponse) => error = e);

    // Arrange, with some implicit assertions
    let expectedResponse: IUser = { name: "someName" };
    let request = httpMock.expectOne({ method: "get", url: `/api/users/${userName}` });
    request.flush(expectedResponse);
    tick();

    // Assert
    expect(response).toEqual(expectedResponse, "should return the expected response");
    expect(error).toBeUndefined();
    expect(httpErrorHandlerService.logError).not.toHaveBeenCalled();
    expect(httpErrorHandlerService.getValidationErrors).not.toHaveBeenCalled();
}));

In this example, the test being called is made immediately (although there is some setup in a beforeEach not shown). It returns a promise, but the promise is unresolved at that point. Next there is an implicit assertion about what a request that the userService.getUser looked like and a mock response is provided. Then the promises are flushed, which is why these are fake async tests. Finally, assertions are made.

Comparing the two

Ironically, the Async pattern more closely follows how you should test synchronous code. Both tend to follow the “Arrange, Act, and Assert” structure. This pattern set up a bunch of mocks and rules beforehand to “wind up” the test, then lets the code under test run to completion, the expectations are checked. This pattern is good for testing how code behaves to inputs and the results it produces.

The fake async pattern is a lot more granular as you essentially interleaves the test code with the code being tested. It’s like stepping through a debugger and providing mocks and asserting as you go, line by line. This pattern is better at testing code that creates side-effects, or code that needs to be done in a specific order. Anecdotally though, the granularity can come at a cost of brittleness. Refactoring can easily break tests even though the code remained functionally correct, so you can end up wasting time fixing tests.

One downside of the fake async pattern is that it tends to require extra code to get the async parts of the code flushed. In Angular tests, the tick function does this magic for you, as Angular is able to wrap all Promises and so tick can wait for their completion for you. In C#, this is fairly cumbersome, as there isn’t a way to provide a replacement to the default TaskFactory, so you end up having to expose TaskCompletionSource objects from all your mocks to provide the flushing functionality.

Additionally, given the right tools, any fake sync test could be made into an async test. In Angular instead of using the HttpTestingController, you could provide your own mock HttpClient object and use spies to both match specific request patterns and provide responses.

Verdict

When writing up these examples, I actually had attempted to use the same test written in each pattern, but one would always feel a little awkward. There is definitely something to be said about using the right tool for the job, so in Angular tests if you find yourself testing code that makes http calls or uses timers, feel free to use the fake async pattern. The magic is provided for you, so you might as well use it.

However, I also feel that usage of fake async is fairly niche. It doesn’t work well in all scenarios, and it’s not even a very common pattern in some languages like C#. The async pattern on the other hand works well in almost all cases, and so it’s the pattern I’d suggest any well-rounded software engineer to master. It’s a commonly used, widely applicable, well-known, and well-understood pattern.

tl;dr if I was stuck on a desert island with only one unit testing pattern, I’d pick the async pattern over the fake async one.

HttpClient and Unit Testing

If you’ve written C# which uses HttpClient and tried to unit test it, you probably noticed that it’s not the easiest thing in the world to mock out. If this github issue is any indication, many developers don’t find it particularly easy either. It’s certainly not impossible, but it requires learning about some of the internals of HttpClient, such as HttpMessageHandler as the HttpClient is designed as just a wrapper around these things.

Using Moq

Over time I’ve personally solved this problem again and again for different projects by implementing a mock HttpMessageHandler, either explicitly or using a framework like Moq to setup the method calls and return values. The problem is that with these solutions, I found that it was either too loose in terms of ability to validate the code, or too complicated to get the level of strictness I wanted.

Here’s an example from the github issue above for how to mock a request using Moq:

var requestUri = new Uri("http://google.com");
var expectedResponse = "Response text";

var mockResponse = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(expectedResponse) };
var mockHandler = new Mock<HttpClientHandler>();
mockHandler
    .Protected()
    .Setup<Task<HttpResponseMessage>>(
        "SendAsync",
        It.Is<HttpRequestMessage>(message => message.RequestUri == requestUri),
        It.IsAny<CancellationToken>())
    .Returns(Task.FromResult(mockResponse));

var httpClient = new HttpClient(mockHandler.Object);
var result = await httpClient.GetStringAsync(requestUri).ConfigureAwait(false);
Assert.AreEqual(expectedResponse, result);

That sets up a mock which matches a call to a specific url and returns a mock response. It works well enough, but surely there’s a better way.

Patterns in other languages/frameworks

While working on some tests for an Angular project, I realized how much nicer their http testing library is and how easy it was to expect specific requests, do additional validations on the details of the request, and respond. Here’s an example:

let logInSuccessful = false;
let error: Error;

let httpMock = TestBed.get(HttpTestingController) as HttpTestingController;
let authenticationService = TestBed.get(AuthenticationService) as AuthenticationService;
authenticationService.logInWithPassword("someUsername", "somePassword")
  .then(() => logInSuccessful = true)
  .catch(e => error = e);

let request = httpMock.expectOne({ method: "post", url: "/api/auth/token" });
expect(request.request.body).toEqual("grant_type=password&username=someUsername&password=somePassword&scope=openid%20offline_access");

let response = createMockResponse();
request.flush(response);
tick();

expect(logInSuccessful).toEqual(true);
expect(error).toBeUndefined();

httpMock.verify();

The Solution

So with that, I decided to build a .NET library to implement a similar pattern as what I was using in the Angular test. It’s on Nuget.org as Testing.HttpClient (Github).

Here’s an example of a unit test using the library:

[TestMethod]
public async Task ExampleTest()
{
    using (var http = new HttpClientTestingFactory())
    {
        var worker = new Worker(http.HttpClient);

        // Make the call, but don't await the task
        var resultTask = worker.FetchDataAsync();

        // Expect the request and respond to it
        var request = http.Expect("http://some-website.com/some-path");
        request.Respond(HttpStatusCode.OK, "123");

        // Await the result and assert on it
        var result = await resultTask;
        Assert.AreEqual(123, result);
    }
}

Design tradeoffs

Because I based this library heavily off Angular’s http testing, the Expect call must be made after the request is made or else it won’t match a request and fail the test. This means that your test needs to separate the calls that start the http call and the one that awaits it. This design decision makes it easy to inspect the actual full request and simplifies the mocking logic.

However, this admittedly may be a little awkward for C# developers who haven’t seen this pattern which is heavily used in Angular testing. I describe this pattern as “synchronously testing asynchronous code”. For those familiar with Angular, this is the difference between using async and fakeAsync.

In a future post I plan on diving into the pros and cons of both testing approaches in more depth.