SWC2013TDD » History » Version 19

Chris Cannam, 2013-02-08 12:31 PM

1 1 Chris Cannam
h1. Test-driven development outline
2 2 Chris Cannam
3 19 Chris Cannam
*Note*: This section went very badly in the MAT workshop. The following has been reworked subsequently, as a possible basis for future workshops.
4 1 Chris Cannam
5 15 Chris Cannam
We assume that the "intro to Python" section has at least introduced how you would run a Python program and compare the output against an external source of "correct" results; also that the audiofile section has discussed what an audio file consists of.
6 15 Chris Cannam
7 6 Chris Cannam
h2. Motivation
8 6 Chris Cannam
9 2 Chris Cannam
We'll refer first back to the "intro to Python" example, with the text file of dates and observations.
10 3 Chris Cannam
11 3 Chris Cannam
<pre>
12 3 Chris Cannam
Date,Species,Count
13 3 Chris Cannam
2012.04.28,marlin,2
14 3 Chris Cannam
2012.04.28,turtle,1
15 3 Chris Cannam
2012.04.28,shark,3
16 3 Chris Cannam
# I think it was a Marlin... luis
17 3 Chris Cannam
2012.04.27,marlin,4
18 3 Chris Cannam
</pre>
19 3 Chris Cannam
20 3 Chris Cannam
We have our program that prints out the number of marlin.
21 3 Chris Cannam
22 3 Chris Cannam
<pre>
23 3 Chris Cannam
$ python count-marlin.py
24 4 Chris Cannam
2
25 3 Chris Cannam
$
26 3 Chris Cannam
</pre>
27 5 Chris Cannam
28 5 Chris Cannam
We can check this against some human-generated output, or the result of "grep" or something if the program is simple enough, in order to see whether it produces the right result. But what if we change the program to add a new feature -- will we remember to check all the old behaviour as well and make sure we haven't broken it? What if the program as a whole is so complex and subtle that we don't actually know what its output will be?
29 5 Chris Cannam
30 5 Chris Cannam
We need to do two things:
31 5 Chris Cannam
32 6 Chris Cannam
# automate the tests, and
33 1 Chris Cannam
# make sure we test the individual components that the program is made up of (so we can be confident of its behaviour even when we don't know what the program as a whole should produce)
34 1 Chris Cannam
35 15 Chris Cannam
h2. Simple unit tests with Nose
36 10 Chris Cannam
37 15 Chris Cannam
Imagine we're dealing with audio data, and we have a program that loads data from an audio file and then (like many spectral methods) chops it up into fixed-length frames (1024 samples per frame is popular).
38 1 Chris Cannam
39 15 Chris Cannam
Let's consider a function that tells us how many frames we can expect to get from a file of a given size.
40 9 Chris Cannam
41 1 Chris Cannam
h3. Stub function
42 1 Chris Cannam
43 1 Chris Cannam
Sketch the frame count function into @framer.py@:
44 1 Chris Cannam
45 1 Chris Cannam
<pre>
46 1 Chris Cannam
def get_frame_count(nsamples, hop):
47 1 Chris Cannam
    """Given the number of samples, return the number of non-overlapping frames of length hop we can extract from them."""
48 1 Chris Cannam
    return 0
49 1 Chris Cannam
</pre>
50 1 Chris Cannam
51 15 Chris Cannam
So we have a stub implementation that always returns zero.
52 1 Chris Cannam
53 15 Chris Cannam
h3. Unit tests
54 15 Chris Cannam
55 15 Chris Cannam
Now, before filling in the details, we go and write a test for it. The reason we do this first is so that we have the opportunity to think about what we really expect our function to do -- it will turn out that we have some decisions to make.
56 15 Chris Cannam
57 18 Chris Cannam
*Discussion*: What tests do we need?
58 18 Chris Cannam
59 15 Chris Cannam
In @test_framer.py@:
60 15 Chris Cannam
61 11 Chris Cannam
<pre>
62 12 Chris Cannam
import framer as fr
63 12 Chris Cannam
64 12 Chris Cannam
def test_get_frame_count():
65 13 Chris Cannam
    assert fr.get_frame_count(0, 10) == 0
66 15 Chris Cannam
    assert fr.get_frame_count(4, 2) == 2
67 15 Chris Cannam
    assert fr.get_frame_count(4, 3) == 2
68 12 Chris Cannam
</pre>
69 1 Chris Cannam
70 15 Chris Cannam
etc. Note that for the last of these, we have to start thinking about what to do with partial frames -- do we zero-pad them and return them? do we return them part-full? do we skip them entirely? (I would pick the first of these myself but it depends on the application.)
71 15 Chris Cannam
72 15 Chris Cannam
This is test-driven development -- we are using the phase in which we write the tests, to solve problems about what behaviour we really want.
73 15 Chris Cannam
74 15 Chris Cannam
We run @nosetests@, which runs all the functions it finds called @test_@-something in files called @test_@-something in the current directory and subdirectories (recursively searched), and it fails.
75 15 Chris Cannam
76 15 Chris Cannam
(Note -- it turns out that @nosetests@ will only pick up code from files that are not executable. On the network file system in the MAT lab, all files are executable it seems. Run @nosetests --exe@ to include those.)
77 15 Chris Cannam
78 15 Chris Cannam
h3. Implementation
79 15 Chris Cannam
80 15 Chris Cannam
*Exercise:* Implement @get_frame_count@ to satisfy the tests.
81 15 Chris Cannam
82 15 Chris Cannam
h2. Applying this to our marlin example
83 16 Chris Cannam
84 18 Chris Cannam
h3. Rearranging our files to make functions testable
85 18 Chris Cannam
86 16 Chris Cannam
The above example also shows how to call code in one file from another one.
87 16 Chris Cannam
88 17 Chris Cannam
We can rearrange our marlin counting program, then, so that it is simply a module of functions we can call from elsewhere. That is, replace the main loop (the bit whose core is something like @for line in source...@) with a function @count_marlin_in(file)@ that takes a filename argument and returns the number of marlin observed in that file.
89 17 Chris Cannam
90 17 Chris Cannam
Now if we run the marlin.py file, nothing happens -- it only defines functions, it doesn't actually call them any more.
91 17 Chris Cannam
92 17 Chris Cannam
But we can do e.g. at the IPython prompt,
93 17 Chris Cannam
94 17 Chris Cannam
<pre>
95 17 Chris Cannam
In [12]: import marlin as m
96 17 Chris Cannam
In [13]: m.count_marlin_in("data.txt")
97 17 Chris Cannam
6
98 17 Chris Cannam
</pre>
99 18 Chris Cannam
100 18 Chris Cannam
h3. Test data and unit tests
101 18 Chris Cannam
102 18 Chris Cannam
Now we can make a set of small data files for specific tests.
103 18 Chris Cannam
104 18 Chris Cannam
*Discussion*: What tests do we need? e.g. empty file, no observations, observations but no marlin, some marlin, etc.
105 18 Chris Cannam
106 18 Chris Cannam
For each, we can write a simple unit test into @test_marlin.py@:
107 18 Chris Cannam
108 18 Chris Cannam
<pre>
109 18 Chris Cannam
def test_empty_file():
110 18 Chris Cannam
    assert m.count_marlin_in("empty.txt") == 0
111 18 Chris Cannam
</pre>
112 18 Chris Cannam
113 18 Chris Cannam
etc.
114 18 Chris Cannam
115 18 Chris Cannam
*Exercise*: Write these tests.