The final production build of the software is ready. The last full run of testing starts. After two weeks, a potentially serious problem is found and the product manager decides it has to be fixed. The fix is rushed in, unit tested and built into a new production build.

Now, there are only two days left before the product must be shipped to customers. The three week final test phase now goes “out the window” and the test team do the best they can to verify that the product still works without any serious regressions.

This is an example of “Targeted Testing” being applied. Every test team has to do it at some stage, there simply isn’t enough time and resource to do an ideal job.  The effectiveness of that final test of the shipped product depends on the skills of all those involved in the decisions affecting every aspect of the life cycle of that last change – and probably a great deal of luck.

“What I need is a list of specific unknown problems we will encounter.”
(“Dilbertism” from Lykes Lines Shipping)

Time constraints like this are actually happening throughout the whole product development cycle. All the tests in the official plan may be run in several test phases, but by the end of each phase, the product has moved on, and in an ideal world, all those tests (and some not even dreamed of yet) need to be re-run.

Wouldn’t it be great if there were some tools and techniques to help do Targeted Testing so that at any time in the lifecycle of the product, we could know exactly which test is most likely to find a possible problem? All the available tests would be dynamically ordered based on this ever changing likelihood of finding problems. That way, we could be sure that the best possible testing was being done at any point in time in whatever time is available.

(more…)

The plan has been approved and the team is fully committed, great stuff! Hang on a minute, what if… How about… hmm, I need another test! While I’m at it, it would be good to tidy up a few loose ends and address a few concerns. I’ll fetch my cape, it’s time for some Hero Testing; it can only make things better.

Jon first introduced me to the term ‘Hero Testing’ and to some of the challenges this ‘I can do it quick & cheap’ approach may bring. For example:

• Employ different process
• Compromise test repeatability (manual v automated test execution)
• Lead to conflicting priorities
• Undermine the value of ‘core’ testing already in plan

If this stuff is important, shouldn’t it be evaluated, prioritized, owned and managed by the team from *their* backlog? Maybe this is what a true Testing Hero would do. Shame, I thought I made the cape look good.

Travelling to the Share conference (share.org) in Orlando I’ve achieved something I have never done before. I started, and finished a book during the flight. And proudly it wasn’t a small book of pictures, it was a book on Software Testing.

Having previously not given it much attention, the book was laying around my office for a while. Recently, I was on the hunt for some information and rather than go straight to the mother of all search engines, I decided to be old-school and check out the books on my shelf.

Flicking through a few I stopped on one that gave me exactly what I was after. In fact, it appeared to me that within its contents were lessons that every Tester should be aware of. It hit me, “Where has this book been all my life?”.

The answer was easy; “Sitting on a shelf”. After all, the book which was written by Glenford J. Myers has been in publish for the last 30 years.

It’s called “The Art of Software Testing”.

(more…)

It is widely recognised that testing software can be an enormously challenging task, but why is this? The answer to this question is not straight forward; software is complex, code paths are long and the numbers are not on the tester’s side…

(more…)