I read an interesting article by Steve Sanderson on Integration Testing your ASP.NET MVC Application. I have never really been a fan of what Microsoft have done with testing frameworks – I usually prefer NUnit to MSTest (the built-in Visual Studio testing framework) and the Microsoft presentations that I’ve seen on the other testing tools they provide I found rather… sub-par. Despite this I think it’s great that they are making an effort in this area.

I thought it would be worthwhile to take a look at the new Lightweight Test Automation Framework.  To do this I’m first going to take a look at some of the current alternatives and how they compare before trying out this new framework.

What is Integration Testing

Let’s first look at the difference between Unit testing and Integration testing:

• Unit testing is testing at the most basic level. Here the scope of the test is a single method on a single object. While there is no rule that says you can’t test multiple methods or objects at once with a Unit test, we usually stub or mock these dependencies and never touch the database. Unit tests are also at the core of Test-Driven Development (TDD).
• Integration testing is testing multiple objects and layers. We use the real database (which we usually setup with test data via scripts or backups) and test the entire software stack together. These tests are usually good at spotting any issues with different components not working together as expected.

There are multiple frameworks available for both types of tests. For unit testing the most popular ones are NUnit, xUnit, MSTest and MbUnit. For integration testing most developers simply use NUnit (or a similar framework) for testing against the database, or for web projects Selenium seems to be the most popular option. While using a unit testing framework is fine for writing integration tests, you tend to end up with large tests which can become nightmare to maintain. The problem usually lies in verifying large amounts of data – unit tests simply don’t allow you to visualize the results in a way that is easy to maintain.

In this post I will be taking a look at a little-known framework called Fitnesse, while the next 2 will focus on Selenium and the new Lightweight Test Automation Framework.

Fitnesse

Fitnesse is an Integration testing framework implemented as a wiki with a java web server. While the framework is web-based, your application doesn’t have to be – you can use Fitnesse on pretty much any type of application (including the web-based variety).

The biggest advantage Fitnesse has over other integration testing frameworks is that the entry point into the code stack is not the (web) frontend – fixtures are written in C# and call out to whatever part of the codebase you are testing. This allows us to test separate layers as well as testing the entire code stack working together.

I’m going to show an area where I found Fitnesse to be very useful – testing the repository layer. For those of you who haven’t used the repository pattern before, here’s a brief overview. The repository pattern has become very popular in DDD (Domain Driven Development) where the repository is used to access the data used by the domain model. It sounds very impressive, but it basically means we have 2 views of the data stored in the database – a data model, which is an exact representation of the database tables and is usually generated by an ORM like NHibernate or Linq-to-Sql; and a domain model, which is another representation of the data in the way we want to work with it in our application. The repository pattern handles the mapping between these 2 and also handles all persistence. You can find a much better explanation of the repository pattern on The NHibernate FAQ.

In this example I have 2 objects of interest – clients and accounts. There is a one-to-many relationship between clients and accounts and my repository needs to retrieve this for each client.

The code for the repository is a very simple implementation.

public class ClientRepository
{
    private readonly DataModelContext context = new DataModelContext();

    public IList<Domain.Client> GetClients()
    {
        var clients = from client in context.Clients
                        select new Domain.Client
                                   {
                                       Id = client.Id,
                                       Code = client.Code,
                                       Name = client.Name,
                                       Telephone = client.Telephone,
                                       Fax = client.Fax,
                                       Accounts = GetAccountsFor(client.Id)
                                   };

        return clients.ToList();
    }

    private IList<Domain.Account> GetAccountsFor(int clientId)
    {
        var accounts = from account in context.Accounts
                       where account.ClientId == clientId
                       select new Domain.Account
                       {
                           Id = account.Id,
                           Number = account.Number,
                           Type = account.Type
                       };

        return accounts.ToList();
    }
}

Right, so how do we test this? None of the integration testing tools can test this layer in isolation, although all of them can test this layer as part of the complete code stack. We could use a unit testing framework, but this will lead to a whole bunch of Assert statements and will make our tests very difficult to maintain. Fitnesse is ideal for this type of scenario, since it allows us to write our own fixtures and choose our entry point into the code stack.

For this particular scenario I’m going to create a C# fixture that calls out to the ClientRepository and returns the retrieved objects to Fitnesse. I need to write 2 fixtures – one for verifying the clients being returned and another for the list of accounts being returned with each client. To do this, we need to create 2 rowfixtures – Fitnesse uses a number of different types of fixtures, but the ones we use most often are rowfixtures and columnfixtures. In general, rowfixtures are used for verifying retrieved data while columnfixtures are used for performing actions.

public class ClientRepositoryClientsFixture : RowFixture
{
    public override object[] Query()
    {
        return new ClientRepository().GetClients().ToArray();
    }

    public override Type GetTargetClass()
    {
        return typeof(Client);
    }
}

public class ClientRepositoryAccountsFixture : RowFixture
{
    public override object[] Query()
    {
        var client = new ClientRepository().GetClients().Single(c => c.Name == Args[0]);

        return client.Accounts.ToArray();
    }

    public override Type GetTargetClass()
    {
        return typeof(Account);
    }
}

Notice that in the second fixture I will be passing a parameter to the fixture – the name of the client. So now we can start the Fitnesse web server and create the expectations using the wiki interface and run the test.

Here you can see the expectations I created via the wiki interface.  The values are pipe-delimited and we need to specify the full class names for the fixtures we are using.  Creating the expectations can be rather time-consuming, but once you’re done the results are a lot easier to view than a whole bunch of C# Assert statements (which is what would happen if we did this test using NUnit or a similar framework).  You can also create the expectations in Excel.

This is the result of running the test.  My expectations are correct (I am expecting the data I specified in the database), but my test is failing because there is actually a bug in my code – I’m not mapping the Surname field in the repository class. Pretty neat. After I fixed the bug and recompiled it all goes green.

Here you can also see how I passed the parameter to the rowfixture – simply add the value after the name of the fixture and Fitnesse will make it available as a parameter.   Now that we’re done we can also include these tests as part of our build process.

Conclusion

The repository pattern in this example is tied to the database and as a result cannot be tested with unit tests.  We can use a unit test framework, but we can’t stub/mock the database dependency in an elegant fashion.  Fitnesse allows us to choose our entry point into the code stack and test the repository layer in isolation.  It also allows us to setup our expectations in a format that is easy to view and maintain.

Since our tests now depend on the database (which is the usual case with all Integration tests) we need to be able to restore the database to a known state before running the Integration tests.

Advantages

• Ability to test separate layers in isolation
• Easy to maintain tests and view test results
• Expectations can be setup in Excel

Disadvantages

• Web interface can make setting up expectations time consuming
• Cannot explicitly test a web frontend

In my next blog I will take a look at Selenium.

Share on: Twitter, Facebook, Google+ or use the Permalink