How to write good tests (in any language)
In the last few years, I've read many many articles on how important testing is. In fact, I'd say that it's possibly the single most blogged about thing in programming. All of this advice is great, but what if you don't know how to write tests in the first place. You can be following all of the advice, and writing bad tests, and getting no results for your efforts. In fact, I suspect that there are PMs out there who have convinced their team to give up on testing because of this. So, let's talk about writing good tests.
As you can see, once the Given section is complete, we have everything we need to run our test. The only thing left is to call the method that we're testing.
The when clause is always the shortest. Good testing tests a single method at a time. If you wish to test what happens when one method is called after another, then you run the first method in your Given section. Only the method you're testing goes in the When clause, and it should always be just one method. This may sound a little like I'm advocating "Unit Testing" here and while I'm a big fan of Unit Testing, that's not what this is about. A single method can test your entire stack of code. A single method can be an integration test, a load test, a unit test, or even an end-to-end test. Good testing implies that you're only testing one thing at a time. Otherwise, how will you know what's broken?
So this example is pretty standard in my opinion. I'm not showing you the logic of our EmployeeManager, because you don't need to know it in order to test it. You only need to know what it outputs. If you write good tests, then you will not only know that your code works as expected, but you'll know when something changes, even without seeing what changed. You can do things like safely refactor thousands of lines of code in no time, or safely let other developers play in your code base sure that your tests will make sure they don't break anything. Testing may be challenging, but spending the time doing it is worth the effort.
As you can see, by following these simple steps we can keep our test code clean, organized, self-explanatory with thorough commenting, and even self-contained by running cleanup on our logic. Some testing suites offer you initialization and teardown logic for handling this, but I like to keep my code together in one place. However you do it, it's important that you test often, and test well, so let's see the whole code in one place.
With simple tests like this, we can clearly see what's being tested, and we have a scaffolding so that we don't forget what's being tested. If you think this test looks small, you're mistaken. I've written shorter tests than this and been happy with the result. That said, an integration test or an end-to-end test may have a much larger quantity of assertions in the Then clause, and possibly a larger section for cleanup.
All of this is a just a guide though. It's important that you remember the value of testing, and that you test often. But none of that matters if you don't test well. So do this for each and every method in your code, and you'll have a good start. Code coverage tools will tell you that's not enough though. You need to test each code path. That's just a fancy word for testing your if statements and your loops. If you have a switch statement, you need to have another test for each of the potential inputs to that switch. The same is true for any code path like if and else. If you can achieve 100% or nearly 100% code coverage, using a testing style like this, I guarantee you'll get phone calls like the one I got from one of my previous employers three months after I left.
"We finally deployed that program you wrote to production."
"Really? Cool."
"Yeah, it works great, thank you very much."
Production code that doesn't get rolled back 5 or 6 times? Man, that was a good feeling. You try it.
Scaffolding
When it comes to testing, I personally follow the Steve Sanderson methodology: Arrange, Act Assert. But this technical terminology just makes the comments hard to read, and in practice, they actually become Given, When, Then, so I skip the middle man. Here's a sample of how I write test code.
//Given: Some intialization logic //When: We call some method/function //Then: We expect these results //CLEANUP (optionally, unchange the things you've changed in your test)This scaffolding guides me in building my test to ensure that I write good tests. However, scaffolding is no good if you don't undertsand the purpose of each line, so let's go over their purpose. For consistency, I'm going to use a single example throughout this blog post, and give you the complete code at the end. The example is simple. We have an array of an object called Employee. We want to add a new employee to the array through a function we created called AddEmployee on a class we'll call EmployeeManager.
Given - Initialization logic
In the Given clause, you prepare your environment. You set up the variables and values throughout your code to get your code to exactly the point you want to test. This is not where you trigger the events that cause the test. If you're testing some code that should trigger at midnight, this is not where you set the time to midnight. This is where you create the events that you will trigger If you're testing code that should trigger at midnight, this is where you turn on the program and set your logic to get ready to control the time of day. In our example, this is where we initalize the EmployeeManager class and create the employee that we will add.//Given: an EmployeeManager and a new Employee
var employeeManager = new EmployeeManager();
var employee = new Employee{ FirstName = "John", LastName = "Doe" };
As you can see, once the Given section is complete, we have everything we need to run our test. The only thing left is to call the method that we're testing.
When - Call the method you're testing
In the When clause, you execute the method you're testing. If it returns some value, you'd save it to a results variable here. If you were testing some code that should trigger at midnight, this is where you set the clock to midnight. In our case, we'll assume that our AddEmployee() function will return an EmployeeID, a number to identify our new employee in the future. So we'll save that value to a results variable.//When: we call AddEmployee var result = employeeManager.AddEmployee(employee);
The when clause is always the shortest. Good testing tests a single method at a time. If you wish to test what happens when one method is called after another, then you run the first method in your Given section. Only the method you're testing goes in the When clause, and it should always be just one method. This may sound a little like I'm advocating "Unit Testing" here and while I'm a big fan of Unit Testing, that's not what this is about. A single method can test your entire stack of code. A single method can be an integration test, a load test, a unit test, or even an end-to-end test. Good testing implies that you're only testing one thing at a time. Otherwise, how will you know what's broken?
Then - Figure out what happened
So now we have to find out what happened. All this other stuff was just executing your code. The Then clause is where we really find out if everything is working. Steve Sanderson calls it "Assert" for good reason. This is where we call that famous testing function "assert" over and over again until we're sure we know what happened. Smaller tests, like unit tests, should have very few assertions. Larger tests, like end-to-end tests should have a much larger number of assertions. To be honest, I've seen the Then clause scare off more developers from testing than I care to admit. I find that the best way to approach a scary test where you're not sure what will happen is just to debug it. Check the values, and build your then clause based on those values. Remember, testing isn't about getting the right answer. It's about figuring out what answer the program is really giving you. You might find out that a massive piece of logic you thought was controlling everything was just for show and that the real code turned out to be a sneaky "return 12" for font size. Here's where you learn the truth. If you're testing code that should be called at midnight, in the Then clause you need to verify that your function was actually called. In our simple employee creation case, we need to test two things. First, we need to see if our result was an EmployeeID like we expect. Then we need to check that our employee was actually added to the array. Here's what it might look like.//Then: we should see our employee is added and get an EmployeeID back Assert.AreNotEqual(null, result); //verify the result is not null Assert.AreEqual(true, result > -1); //verify the result is a non-negative number Assert.AreEqual(true, employeeManager.Employees.Contains(employee)); //verify the employees array contains our new employee
So this example is pretty standard in my opinion. I'm not showing you the logic of our EmployeeManager, because you don't need to know it in order to test it. You only need to know what it outputs. If you write good tests, then you will not only know that your code works as expected, but you'll know when something changes, even without seeing what changed. You can do things like safely refactor thousands of lines of code in no time, or safely let other developers play in your code base sure that your tests will make sure they don't break anything. Testing may be challenging, but spending the time doing it is worth the effort.
CLEANUP - Undo the damage you've caused
Our final step, CLEANUP, is only needed if our test does something that needs to be undone. For our code that would need to run at midnight, it might alter some settings in the computer that we need to change back, or start some job that we may need to stop. For completeness, let's assume that our EmployeeManager also writes to a database. The test will not work if we run it a second time since our employee is already in the database. We need to remove him. Since the EmployeeManager's AddEmployee method added the employee to the database, we'll pretend that a RemoveEmployee method would do the opposite. If such a situation exists for you in your code, I would highly recommend testing the RemoveEmployee method as well, just to make sure that this cleanup procedure works. In a more security concious situation, I might run some more direct SQL logic to remove it from the database manually, rather than relying on another method, but this is a smiple example and I want to keep it language agnostic.//CLEANUP employeeManager.RemoveEmployee(employee);
As you can see, by following these simple steps we can keep our test code clean, organized, self-explanatory with thorough commenting, and even self-contained by running cleanup on our logic. Some testing suites offer you initialization and teardown logic for handling this, but I like to keep my code together in one place. However you do it, it's important that you test often, and test well, so let's see the whole code in one place.
//Given: an EmployeeManager and a new Employee
var employeeManager = new EmployeeManager();
var employee = new Employee{ FirstName = "John", LastName = "Doe" };
//When: we call AddEmployee
var result = employeeManager.AddEmployee(employee);
//Then: we should see our employee is added and get an EmployeeID back
Assert.AreNotEqual(null, result); //verify the result is not null
Assert.AreEqual(true, result > -1); //verify the result is a non-negative number
Assert.AreEqual(true, employeeManager.Employees.Contains(employee)); //verify the employees array contains our new employee
//CLEANUP
employeeManager.RemoveEmployee(employee);
With simple tests like this, we can clearly see what's being tested, and we have a scaffolding so that we don't forget what's being tested. If you think this test looks small, you're mistaken. I've written shorter tests than this and been happy with the result. That said, an integration test or an end-to-end test may have a much larger quantity of assertions in the Then clause, and possibly a larger section for cleanup.
All of this is a just a guide though. It's important that you remember the value of testing, and that you test often. But none of that matters if you don't test well. So do this for each and every method in your code, and you'll have a good start. Code coverage tools will tell you that's not enough though. You need to test each code path. That's just a fancy word for testing your if statements and your loops. If you have a switch statement, you need to have another test for each of the potential inputs to that switch. The same is true for any code path like if and else. If you can achieve 100% or nearly 100% code coverage, using a testing style like this, I guarantee you'll get phone calls like the one I got from one of my previous employers three months after I left.
"We finally deployed that program you wrote to production."
"Really? Cool."
"Yeah, it works great, thank you very much."
Production code that doesn't get rolled back 5 or 6 times? Man, that was a good feeling. You try it.
Comments
Post a Comment