Who is responsible for quality?
If a team of people produce something, whose fault is it if the quality is poor? After all, a tester’s job is, ostensibly, to find defects in a product. A poor quality product must be the result of the testers not doing well enough, right? After all, test teams are often referred to as “Quality Assurance” teams. Well, I’d say that the testers aren’t the ones making the quality low, and everyone involved in the project is responsible for the quality, including designers, developers, testers and managers. By discovering a defect, a tester is merely saying that a problem exists, and is highlighting a risk. I think the question we should be asking, is who is responsible for the risk?
A typical development cycle involves finding a lot of defects – the early defects are usually high risk functional problems (i.e. it just doesn’t work), but as the project progresses, you start to find that more obscure problems are found. Shipping without fixing these problems is a risk, but it might be a reasonable one, depending on how and why the product is going to be used. Deciding when to stop testing has got to be one of the toughest things about software development. I think it’s the tester’s job to present the risk, rather than to be the judge of how much risk is acceptable.
I think that one implication of this is that testers need to document all the defects they find – by only documenting the “serious” ones, they are not presenting the risk, but making a decision about what they perceive the risk to be. Another important implication, is that it’s not the tester’s fault when everything breaks six months down the line – it was always broken. Sure, more testing might have found more defects, but at some point a decision was made to ship the product, which is always a calculated risk.
January 29th, 2008 at 17:13
Being able to make an informed decision about when a product is good enough to ship is critical and, as you say, documenting all the problems found makes it possible to judge the risks as accurately as possible. On the other hand deadlines can create enormous pressure to meet targets for the number of problems that exist.
I can certainly relate to the negative perception that discovering problems can have, which gets worse as the day of delivery approaches. Equally, it can be tough in development when there are problems that haven’t been resolved, but those problems are still better than the ones you don’t know about. Surely it’s better to find out before a customer does, even if the decision is to ship anyway?
January 30th, 2008 at 13:04
I agree. I think it’s a sadly common perception that defects are somehow the test team’s fault. Targets for the number of defects to find seem counter productive to me, and don’t take into account the risk associated with a particular feature not working. It’s the latter that allows you to make intelligent decisions about whether it’s an acceptable risk to ship anyway.