How Agile am I?
I’ve been having some thoughts about how we could measure the effectiveness of an agile test process.
One of the measures I think could work is ‘Time to defect’ (TTD). That is the time between a coder injecting a bug into the system and the time the official defect reporting the problem arrives back on their desk. The lower the TTD the better.
A low TTD has many benefits, not only does it indicate that the testers are finding defects quickly, but it also gives the coders the chance to fix the problem when the code is still fresh in their minds.
Automated unit tests that check code changes as soon as they are integrated into the driver are an excellent means of reducing TTD. You could potentially use your unit tests to gate the integration, so your TTD approaches zero as the code never makes it into a proper driver.
For function and system testing it is essential to have a good set tests that can be targeted at the latest changes. The goal being that if a single defect has been added to the system, then only a single test is required to find it. (As opposed to having to run huge regression suites that may take hours or days to complete).
Agile champions the use of continuous testing (unit, functional and system) throughout each iteration. This is a big change from the traditional phased approach to testing (as discussed in an earlier post). Measuring TTD will clearly show if the right testing is being done at the right time or not. Ideally your maximum TTD would be no more than the length of a single iteration. so if you are working on two week iterations then all the new code delivered in that two weeks will be fully tested in that time.
High TTDs will indicate areas of testing that have been left too late and this should be highlighted and fed back into future iterations or releases.