Test Driven Development
Date: May 28, 2017
There are a multitude of coding standards and methods available for writing modern day firmware and software. One of the more popular approaches that has been floating around for us and in the wider coding community is Test Driven Development (TDD). While this is more commonly used in software development, various tools allow the process to be implemented in firmware as well. A recent project has allowed us to revisit this approach in more detail, where we've implemented a few chip drivers using TDD on the CppUTest platform.
What is Test Driven Development?
When first learning to write code, it is very common to write a number of lines, and eventually test the code to see if what you wrote works. This process often sticks with us, where we write more and more complex and lengthy code only to test once it has been written. TDD is a shift in this mentality, where tests always come first.
The basic process of TDD:
Refactoring is an important component of the process as well, and simple to do. If you see a simpler method of doing things, the code should likely be changed to use this method. As most of the code would be protected by a multitude of tests any refactoring errors should immediately pop up in test failures. The code will shape itself as you progress, often deviating from an original design to make things simpler or more efficient. By following the process, and refactoring as you go to keep things neat, the idea is that eventually your code will be complete, readable, and fully tested.
For us the test process starts with simple unit tests which slowly grow into more application specific system tests as the drivers themselves grow. Making sure that these tests fail before writing the code seems like a trivial thing at first. However, there have been minor instances where we've caught ourselves not initialising a variable properly, leading to a surprise test pass with no application code actually being written. Test failure in a way is a small test on its own.
By following the process we end up with some basic chip drivers that can easily be expanded on in the future. While we believe that refactoring is an important part of the design process, getting together a full plan of what the code will look like should minimise the need for this. The danger with this in TDD is that you are tempted to write an extra few lines of code even though a test currently does not cover what you have written. Coming from a process of "Test Eventually Development", ensuring this doesn't happen requires a fair bit of discipline.
Agile Development and Feature Creep
We live in a world of agile development. Often times we are given a loose specification and it is up to us to see what we come up with. This loose spec becomes more and more defined over time, where the client will often request additional features during the project's lifetime, even after deployment or the start of production. Getting defect free code out fast often becomes the norm. Attempting to fit additional features into a plan is not always simple. However, TDD is definitely suited for this. An additional feature simply becomes another test or set of tests that needs to pass. Even if code needs to be refactored in order to get the feature into the plan cleanly, you can be fairly sure the rest of the code is unaffected as long as no other tests have failed.
Debugging vs. TDD
From small $2 LED drivers to the $2 billion Mars Rover, almost all firmware and software will have hidden bugs. Whether these pop up during development or later on in the life-cycle of a project, they will often need to be fixed. Debugging and TDD have very different amounts of effort involved. Debugging often involves inserting break points, miscellaneous print debug statements, and temporary variables. It is often a lengthy process, involving looking at large sections of code, where most of the effort is eventually wasted. All of these features are eventually removed when the bug is found, and any new programmer investigating further issues will need to do so with a fresh set of eyes. In contrast, TDD will have code surrounded by a suite of tests which can be run on hardware or software. In firmware, often times the bug can simply be a hardware fault, and TDD can point this out quite quickly when the tests are run on the actual hardware. TDD tests are often focused and can pinpoint the problem so that only a small portion of the code needs to be examined.
The problem is that TDD tests won't always cover the system as a whole. In situations where interrupts can disrupt processes at any stage, it may be necessary to move to the standard debugging steps to find the cause of the problem. However, at the least TDD gives some confidence once the fix is in place that other functionality is unaffected. Tests can be added to cover any further edge cases found through debugging.
Conclusions
Test Driven Development can be a powerful method of programming in agile projects. It can save time in debugging and additional feature development due to the thorough testing that underlies the whole process. We will likely continue to use this method in the future for quite a few projects, especially in the case of reusable drivers which we can continually improve on.
References:
Test Driven Development for Embedded C (Pragmatic Programmers) by James W. Grenning
https://www.versionone.com/agile-101/agile-software-programming-best-practices/test-first-programming/
https://www.agilealliance.org/glossary/tdd/