Estimated time for this notebook: 5 minutes
As we write code, we want to be sure that it does behaves the way we’d like it to - so we test it. Testing (and re-testing) our code is something that needs to be done regularly (ideally after every change to the code), comprehensively, quickly and reliably. In short testing is an task that is ideally suited to automation.
We write additional code to test the behaviour for our main code. We use these terms to distinguish between the two types of code:
“Production code” - the code that fulfills the purpose of the software, and is run by the end user.
“Test code” - additional code only used by software development team
For this module we are focusing on automated testing.
A few reasons not to do testing#
Code is just a one off throwaway
As with most research codes
No time for it
A bit more code, a lot less debugging
Tests can be buggy too
Not a professional programmer
Will do it later
A few reasons to do testing#
lazyness testing saves time
peace of mind tests (should) ensure code is correct
runnable specification best way to let others know what a function should do and not do
reproducible debugging debugging that happened and is saved for later reuse
code structure / modularity since the code is designed for at least two situations
easier to modify since results can be tested
Not a panacea#
“Trying to improve the quality of software by doing more testing is like trying to lose weight by weighting yourself more often.” - Steve McConnell
Testing won’t correct a buggy code
Testing will tell you were the bugs are…
… if (and only if) the test cases cover the scenarios that cause the bugs or occur.
Also, automated tests only test a narrow interpretation of quality software development. They do not help test that your software is useful and help solves a users’ problem. We will touch on this again in Module 06.
Tests at different scales#
Level of test
Area covered by test
smallest logical block of work (often < 10 lines of code)
Unit tests should run fast (eg ~1/100th sec) so that they can be re-run regularly (eg every git commit). To achieve this they should not invoke network access or substantial disk access.
several logical blocks of work together
These can be useful where you need to tease out the expected/useful behaviour of 3rd party libraries.
all components together / whole program
These can take longer to run, and can be run less often.
When writing new code (see below) always start by creating tests at the smallest scale (unit tests).
If a unit test is too complicated to write, then consider adjusting your production code (possibly by breaking it down into smaller, individually testable functions). Ensuring that your production code is easy to test is a healthy habit.
Legacy code hardening#
Very difficult to create unit-tests for existing code
Instead we make a regression test
Run program as a black box:
check output against expected result
Does not test correctness of code
Checks code is as similarly wrong on day N as day 0
fixture: input data
action: function that is being tested
expected result: the output that should be obtained
actual result: the output that is obtained
coverage: proportion of all possible paths in the code that the tests take
if energy > 0:
! Do this
! Do that
Is there a test for both
energy > 0 and
energy <= 0?