I’ve been in meetings, often driven by the root-cause-analysis of a software fault found in the field, where the topic of code coverage has cropped up. I’m sure many of us have been in similar meetings. On occasions I’ve also been asked to justify some of my apparently poor line coverage figures, where the percentage has fallen short of what was perceived by the inquisitor as acceptable.

Rather than discus the merits or otherwise of such statistics I would instead like to tell you a story (originally drafted by Alberto Savoia). One that I hope you’ll enjoy as much as I did. One that I think explains the wisdom behind sensible test coverage targets very well.

---

Early one morning, a programmer asked the Great Master

I am ready to write some unit tests. What code coverage should I aim for?

The Great Master replied with the simplest of responses…

Don’t worry about coverage, just write some good tests.

The programmer smiled, bowed, and left.

Time passes and, later that day, a second programmer asked the same question. This time the Great Master points at a pot of boiling water and asks…

How many grains of rice should I put in that pot?

The programmer, looking puzzled.

How can I possibly tell you? It depends on how many people you need to feed, how hungry they are, what other food you are serving, how much rice you have available, and so on.

The Great Master paused to let the second programmer reflect on his own words before responding.

Exactly!

The second programmer smiled, bowed, and left.

Toward the end of the day, a third programmer came by and the same question was put to the Great Master.

Eighty percent and no less!

replied the Master in a stern voice, pounding his fist on the table.

The third programmer smiled, bowed, and left.

After this last reply, a young apprentice approached the Great Master.

Great master, today I overheard you answer the same question about code coverage with three different answers. Why?

The Great Master stood up from his chair.

Come get some fresh tea with me and let’s talk about it.

After they filled their cups with smoking hot green tea, the Great Master answered the apprentice.

The first programmer is new and just getting started with testing. Right now he has a lot of code and no tests. He has a long way to go; focusing on code coverage at this time would be depressing and quite useless. He’s better off just getting used to writing and running some tests. He can worry about coverage later.

The second programmer, on the other hand, is quite experienced both at programming and testing. When I replied by asking her how many grains of rice I should put in a pot, I helped her realise that the amount of testing necessary depends on a number of factors, and she knows those factors better than I do – it’s her code after all. There is no single, simple, answer, and she’s smart enough to handle the truth and work with that.

The young apprentice thought for a moment.

I see.

Thinking he’d caught the Great Master in an inconsistency he continued…

But if there is no single simple answer, then why did you answer the third programmer ‘Eighty percent and no less’?

The Great Master laughed so hard and loud that his large belly, evidence that he drank more than just green tea, flopped up and down. Once he’d settled he replied.

My young apprentice, that’s easy. The third programmer wants only simple answers – even when there are no simple answers – and then goes on to ignore the advice anyway.

The young apprentice and the grizzled Great Master finished drinking their tea in contemplative silence.