Unit Test Setup

In many unit test frameworks, there is a feature of adding a setup function. There are two kinds of unit test setup: class setup and a single test setup. In class setup, you write a special function that setups up the class for testing. This class is executed once, then all the tests within that class are executed. On the other hand, a single test setup is a function that runs before each unit test.

I found that setup functions, though helpful at times, come with maintenance and readability cost. Often developers think that a setup method is a great way of reducing duplicate code, and hence developers put setup code in these setup methods. As a result, some tendencies arise:

In the first two cases, the setup function will be difficult to understand and read. It will be long, and it might include code specific to some tests and not others. Unit tests might become small, but it is only an illusion: setup method is large. In the last case, the unit test becomes difficult to understand since the setup needs to be understood first.

A year from now, when a new developer wants to fix a unit test, it will take them a long time to understand what the unit test does because they will have to parse out what the setup method does before touching the code, and why how all the code in the setup is relevant to the unit test in question. Then they need to understand all the magic values used in the mock, and how they are being used in the unit test. Setup function and unit tests are no longer easy to read or modify.

Here is an example:

setup() {
	people_db_mock.create()
	people_db_mock.given(id=12).return(james)
	people_db_mock.given(id=13).return(null) 
	people_db_mock.given(position=developer).return({james, john})
	people_db_mock.update(id=12,name="jack").return({id=12,name="jack"})
	// more of people_db_mock setup
	people_service.create(people_db_mock)
}

test_return_user() {
	assert(james, people_service.get_id(id=12))
}

test_return_null() {
	assert(null, people_service.get_id(id=13))
}

test_return_developers(){
	assert({james, john}, people_service_get_by_position(position=developer))
}

// more tests

The person who is going to read the unit tests needs to mentally map that 12 is James' id, and that James and john are both developers, and that 13 is an invalid id. If the developer is fixing test_return_user, the developer needs to know that many of the lines in the setup function are irrelevant, but that requires reading and understanding the setup function entirely (at least most of the time).

There are multiple ways to avoid this problem. Setup functions are not bad; they are abused, and that is the problem. Consider helping the developer how to evaluate the decision of adding any additional logic in the setup function. Code reviews are good time to do so. Setup functions ideally should contain setup that all unit tests share, and not some of them. Setup functions ideally should not include magic values (in our example, the values 12 and 13). Violating these can be of no cost if there are very few unit tests. The crux of the matter is: be pragmatic.

I myself avoid this by including all the setup I need in the unit tests:

test_return_user() {
	people_db_mock.create()
	people_db_mock.given(id=12).return(james)
	people_service.create(people_db_mock)
	assert(james, people_service.get_id(id=12))
}

test_return_null() {
	people_db_mock.create()
	people_db_mock.given(id=13).return(null) 
	people_service.create(people_db_mock)
	assert(null, people_service.get_id(id=13))
}

test_return_developers(){
	people_db_mock.create()
	people_db_mock.given(position=developer).return({james, john})
	people_service.create(people_db_mock)
	assert({james, john}, people_service_get_by_position(position=developer))
}

This way it is clear what setup is needed for which function. Sometimes you will need a complicated setup needed for each function. Suppose that you need to clean some database table for each test (I know what you are thinking: unit tests are not meant for that. This is just an example and it is up to you to do so), then I would do something similar to this:

clean_db(tablename) {
	sql.drop(tablename)	
}

test_return_user() {
	clean_db(people)
	people_db_mock.create()
	people_db_mock.given(id=12).return(james)
	people_service.create(people_db_mock)
	assert(james, people_service.get_id(id=12))
}

test_return_null() {
	clean_db(people)
	people_db_mock.create()
	people_db_mock.given(id=13).return(null) 
	people_service.create(people_db_mock)
	assert(null, people_service.get_id(id=13))
}

test_return_developers(){
	clean_db(people)
	people_db_mock.create()
	people_db_mock.given(position=developer).return({james, john})
	people_service.create(people_db_mock)
	assert({james, john}, people_service_get_by_position(position=developer))
}

I will still avoid a setup function, and instead create a small function that will have only one job: cleaning the table. This way, it is clear what clean_db does and you can add this functionality to only the unit tests that need it. If there is more setup, I will create another function and add it to the unit tests that need it.

In general, avoid setup functions if you can. Setup functions are not intrinsically bad. Rather, then encourage bad patterns because it is easy to misuse it. I suggest that you do all the setup in the unit tests, and if there is really a need to have a shared setup, then I suggest that you break that setup into small functions, and execute these small functions in the unit tests that need them.