Okay, what do bugs an unit testing have to do with each other? Well, a lot, usually. The first answer is that you use unit testing to prevent writing bugs in the first place. You can also use unit testing to find bugs after code has been written. But there is another connection that you may not have thought of: writing a unit test for every bug you find.
Here’s how the process should go:
- A bug report comes in.
- Your QA guy reviews the report, verifies that it is indeed a bug
- Your QA guy writes a unit test that reveals the bug. In other words, this test should be a test that you’d expect to pass if the bug weren’t there, but because the bug is there, it fails. A way to approach writing the test is to write a negative test case, but actually use CheckTrue instead of CheckFalse. You can change it later when the test starts passing.
- Along the way, if he sees other tests that he might write, he should do so. If they are known/expected to pass, great. If some other tests reveal bugs, those bugs should be logged and the process started again for that bug.
- The test name should include include the bug number for easy identification. The bug report should be updated to include a reference to the test that reveals that bug. This way, the whole system is self-referential.
- The know failing unit test should be included among the list of tests that are, well, known failures. In other words, your testing system should be fine with these tests failing, and actually notify you only when they don’t fail. That is, “failure” is the known, acceptable state.
Now at this point you have something. You have a good solid bug report that has been validated. You have a test that proves the bug exists. You’ve added it to a collection of tests that do the same. These tests all fail, and you know that they are “”supposed to fail”. You can safely ignore those tests for now.
That last part may seem a bit strange. Why happily ignore a bunch of failing tests? Aren’t failing tests bad?
You probably have guessed already – you are waiting for those tests to pass. You know those tests fail, and you are tracking them to know when they actually pass. Your developers will fix the bug, and then suddenly the test will pass, you’ll know the bug was fixed. And shoot, sometimes the test will start passing without the developer explicitly fixing the bug. That’s an extra bonus when that happens.
Now at this point you really have something. Once the test starts passing, you know for sure that that the bug is fixed without any more work. You can move that test over to the regular test suite, and it becomes a regression test. If that bug appears again, you’ll know it right away.
Now, not al bugs can have clean, simple unit tests associated with them – UI bugs come immediately to mind, but my recommendation is this: To as large a degree as possible, never accept a bug that doesn’t have an associated unit test that “reveals” the said bug. Enforcing this rule will have a lot of good benefits:
- Bug reports become easier to write because very often they merely need to be the test itself with a short description.
- The developer will know that bugs that come his way are really bugs, and can be easily reproduced and fixed.
- Working on the fix gets a lot easier because the developer just needs to debug the failing test case and make it pass. No need for a long list of steps to follow to recreate the bug
- Your QA team will be engaged in the development process at a higher level. They’ll be writing code within your frameworks and applications, giving them a deeper understanding of the codebase they are testing. Heck, it will even help groom them to move to the R&D team.
- You will know that a bug has been fixed without any need for further testing.
- You’ll find out when a bug is incidentally fixed as a result of other changes. (This should be examined a bit to make sure it’s for real, but it’s a nice surprise when it happens)
- You’ll have a built in regression test to make sure that if the bug pops up again, you’ll know right away.
And what development team wouldn’t want all of that to happen?