XKCD on writing good code: http://xkcd.com/844/

David Cummings has written an article for the Los Angeles Times about his experiences with testing on the Mars Pathfinder project, and how that might relate to Toyota’s recent problems:

If Toyota has indeed tested its software as thoroughly as it says without finding any bugs, my response is simple: Keep trying. Find new ways to instrument the software, and come up with more creative tests. The odds are that there are still bugs in the code, which may or may not be related to unintended acceleration. Until these bugs are identified, how can you be certain they are not related to sudden acceleration?

This brings us back, as ever, to the question of “when do you stop testing?”.  How do you answer that question when safety is involved?  At some point, you have to ship your product.  With software, we are often afforded the luxury of updating it afterwards, but it isn’t always possible (problems with installation programs usually fit into this category, as do many embedded systems).  We can never prove the absence of errors.

Here’s a fun take on unit testing achievements (in the XBox 360 sense of the word “achievement”): http://exogen.github.com/nose-achievements/

I discovered the Cobertura code coverage tool the other day.  It is an elegantly simple tool, which produces simple, easy to read reports.  My current project is using Apache Maven for its build system, and with the Cobertura Maven plugin provided, I simply added a couple of lines to my project file, and ran a build.  From start to finish, I spent less than five minutes to go from nothing to having some really useful code coverage data.  The report even includes a code complexity metric to help you identify the more complex areas of code.

I’m impressed by the simplicity of this tool – quite a few years ago, I spent months and months getting code coverage data for a large project (code coverage was relatively new then), and it’s great to see how easy it has become to use this technology.  I’ve previously used Emma for code coverage, which is also a great tool, but I think Cobertura has the edge when it comes to reporting.

I came across this site which has a lot of great information: 97 Things Every Software Architect should know.  I particularly like this quote from Michael Nygard’s advice in Skyscrapers aren’t scalable:

We often hear software engineering compared to building skyscrapers, dams, or roads. It’s true in some important aspects.  The hardest part of civil engineering isn’t designing a building that will stand up once it is finished, but figuring out the construction process.

Lots of great stuff for software engineers in general.

Every time we add a new configuration option, switch or dial to our software, I get a little edgy.  I work on a very small and outwardly simple software component, which has grown over the years, as more and more people have come to use it.  Without fail, every time someone new tries to use the software, or someone tries to use it for a different project than before, they ask for a new option to be added – “it would be great if it could just do XYZ”.  Every single one of these requests is completely reasonable, and would ostensibly make the software better.  Combine all of them, and you have a usability and (perhaps more importantly) a testability problem – every dial and switch that we add means a new combination of things to test.  What you can end up with is a slightly larger, and outwardly quite complicated piece of software.

This is a problem that plagues software, and there’s no good answer – when you get a customer who says “I need feature X before I buy”, then of course you want to add that feature – it’s just that eventually, you can end up with a mess.  Apple is a company that I think does well to resist this temptation, but probably reduces its potential market share as a result – that’s a tough one to call.

Interesting reading (see the section entitled “The Question of Preferences”): http://ometer.com/free-software-ui.html

The main reason that I can think of for putting testers and developers in different teams, is the desire for independent testing.  By this, I mean someone who can think about the software critically, and not be “polluted” by knowledge of the internal details.  This is a very valuable thing, and means that you can end up with a tester who is using the software in ways that the original developer never even thought of.  A side effect is that the independent tester is more likely to be measured by the number and quality of defects that they find, and can therefore be appropriately critical of the software under test.  This can be good.

I’m not convinced, however, that *all* testers need to be completely independent.  While independence is valuable, it can create a rift in a delivery team of developers versus testers – it can become a battle, with software and defect reports being “thrown over the wall”.  I think that there are levels of independence.  It may be good for some testers to be involved in high level design discussions, but not low-level ones, or actual coding.  There are times when having a tester with knowledge of the code can be advantageous – maybe that person could even spend the rest of their time actually developing the code.  There doesn’t have to be a one-size-fits-all solution.

In short: independent testing is a good thing, but I think there are advantages to identifying when independence is required, and when breaking down barriers between developers and testers is more useful.

Following on from Jon’s previous post on testing quotes:

“The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be impossible to get at or repair.” – Douglas Adams

“Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.” – Rich Cook

Merry Christmas to anyone who’s actually reading this, and even the people who aren’t.

Here’s a tag cloud (created by Wordle) of Wikipedia’s article on software testing:

I removed the words “software” and “testing”, as they swamped the picture.  The results are mostly what I guess I expected, though “agile” doesn’t appear very often.

I went to a presentation given by Graham Spittle today, and he used the analogy that watching salesmen was a lot like watching five-year-olds playing football – they all run to where the action currently is, instead of staying where they might be needed very shortly.  I like this analogy, and was thinking how it applied to software development teams.  You could say that the developers are the strikers, and the testers are the defenders.  Most of the time, it might be useful for everyone to stick to their positions, and not just run where the ball is.  Sometimes, however, you need everybody in the same half, because a.) you really need a goal; or b.) you really need to stop one.  It’s all one team though, and its a given that you also need midfielders who do a bit of both things.

Other sporting analogies will be welcomed.  Hit the comments (out of the park; for six; etc.)