Setting Up CRUD Test Conventions With Fixie

Unit Tests, conventions, and all their disagreements can be quite annoying. It’s like watching prime time cable news. So, when I wanted to set out and try something new, I feared the worst. To my surprise, it turned out a lot easier and better than I could have predicted.

The Goal

What I wanted to do is have my tests executed in CRUD order. That is…

  1. Create something
  2. Read that something
  3. Update that same something
  4. Delete that something for good

I’ve taken a liking to building up my CRUD services in this order. It helps me match what the consumer would experience and sometimes I catch things that might have been missing.

Given the following code…

public class MyTestClass
{
	protected Something myObjectFromStorage { get; set; }

	public void create_test() { ... }
	public void read_test() { ... }
	public void update_test() { ... }
	public void delete_test() { ... }
}

When running an update, I obviously need something to update. Also, if I’m going to be accurate with my test, I should probably obtain that something the same way my consumer would. Since I know I have a CREATE and READ test, why can’t I just leverage those outputs?

I can if my test runner would execute these tests in the correct order. During their execution they will update myObjectFromStorage. In the end, this saves me a lot of hassle.

The Problem

The real problem here is that I’m trying to break isolation conventions that a lot of smart people have said you shouldn’t. With NUnit, variables within a fixture are wiped after each test. XUnit was created to help enforce this by convention.

These frameworks have been designed to steer you away from this and for good reasons as Martin Fowler explains…

Isolation provides several advantages.

Any combination of tests can be run in any order with the same results.

You never have a situation where you're trying to figure out why one test failed and the cause is due to the way another test is written.

If one test fails, you don't have to worry about it leaving debris that will cause other tests to fail. That helps prevent cascading errors that hides the real bug.

So, Let’s Set Some Ground Rules

These guidelines around isolation are important to follow. If we’re going to steer from them, we better have some guidelines (convention) to keep us in check.

  1. Our CRUD tests are to be run in the order: CREATE, READ, UPDATE, DELETE.
  2. Our test class names will end with “CRUDTester”
  3. Our test names will start with the words “should_create” or “should_delete”
  4. There shall be no other tests other than CREATE, READ, UPDATE, or DELETE within a CRUD test class

Bring On Fixie

Fixie is a convention-based test framework. With it, we are able to…

  1. Define what a “test” class/method is
  2. Define how are tests are run
  3. Define stuff to do before/after our tests run

This gets us outside the box of the rules within a framework such as NUnit/XUnit. So, we then can define our example test class as follows:

public class Book_CRUDTester
{
	protected Book testbook { get; set; }

	public void should_create() 
	{ 
		var book = new Book() { Title = "Hello World" };
		var target = new BookStorage();
		var result = target.Save(book);

		// Assert on result

		testBook = result;
	}

	public void should_read() 
	{  
		// Assert that test_book is not null

		...
	}
	public void should_update() { ... }
	public void should_delete() { ... }
}

In our best effort to heed the warnings on isolation, we’ll make sure that each test only runs if the previous was successful. We can control this for now by just ensuring that the book is not null. If the previous test fails, we need to accept that the others will fail as well. So, our goal should be to make it as easy to figure out the failre as possible.

Defining Our Convention

public class CrudTestConvention : Convention
    {
        public CrudTestConvention()
        {
            Classes.NameEndsWith("CRUDTester");
            ClassExecution
                .SortCases(CrudComparison)
                .CreateInstancePerClass();
                
            Methods
                .Where(method => method.IsVoid());
            
        }

        private static int CrudComparison(Case x, Case y)
        {
            var xOperation = Operations().FirstOrDefault(o => o.IsMatch(x));
            var yOperation = Operations().FirstOrDefault(o => o.IsMatch(y));
            var max = Operations().Count() + 1;

            var xValue = xOperation == null ? max : xOperation.Value;
            var yValue = yOperation == null ? max : yOperation.Value;

            return xValue - yValue;
        }

        internal static IEnumerable<OperationType> Operations()
        {
            yield return new OperationType("should_create", 1);
            yield return new OperationType("should_get", 2);
            yield return new OperationType("should_update", 3);
            yield return new OperationType("should_delete", 4);
        }
    }

    internal class OperationType
    {
        private readonly string _convention;
        private readonly int _value;

        public OperationType(string convention, int value)
        {
            _convention = convention;
            _value = value;
        }

        public bool IsMatch(Case @case)
        {
            return @case.Name.ToLower().StartsWith(_convention);
        }

        public int Value { get { return _value; } }
    }

Breaking It Down

This tells Fixies that this convention applies to classes that end with “CRUDTester”

Classes.NameEndsWith("CRUDTester");

This tells Fixie how to sort our test execution order and that we want to same class instance used for all our tests.

ClassExecution
            .SortCases(CrudComparison)
            .CreateInstancePerClass();