OK, this isn’t news, this study came out a couple of years ago and was covered by many people then. But, I find myself regularly referring to it trying to find the link. So I’m going to blog about it then I’ll always be able to find the link.
The study is by Nagappan, Maximilien, Bhat and Williams and is entitled: Realizing quality improvement through test driven
teams and is freely downloadable from Microsoft.
This is the key findings are summarised in this table:
MS Visual Studio
Defect density of comparable team not using TDD
Defect density of team using TDD
Increase in time taken because of TDD
The way to read this is: the researches looked at two Microsoft MSN teams, one team did not use TDD and had a defect density of Y. The second MSN team had a defect density less than a quarter of Y but took 15% longer.
To my mind that proves that which was to be proven, i.e. TDD reduced bugs. But, I’m also aware that other writers have disputed this and I’ve heard of studies which disprove it. (Anyone got a link? Thanks)
Most people who I’ve met, and who have practices, or understand TDD agree it is effective. However there are those who don’t believe it. It reminds me of the episode of Black Adder where Rowan Atkinson’s Black Adder hires a ship Captained by Tom Baker. When there is a problem it plays out like this:
Black Adder: “Someone in the crew will know... you do have a crew don’t you?”
Captain: “Arh, opinion is divided on the subject... all the other captains say you need a crew, and I say You Don’t”
At the end of the day Confirmation Bias will probably decide which set of results you choose to believe.