Just been having a chat with a new grad who is working in one of our System Test teams.
We were talking about load, stress and performance testing and I was trying to articulate the reasons behind having a workload with peaks and troughs during the day (in the same way that a customer would) rather than our usual approach to load which is run at a constant high level for a long period. However, I was struggling with a justification as to why this might be a good idea and how it would find a different class of defects.

Then I remembered the ‘Top Gear’ episode when Hammond was trying to drive a Formula 1 car around a track. Apparently F1 cars are designed to go flat out – all the time – and are pretty good at this. What Hammond discovered to his horror was that when you don’t drive them flat out then you can get into all kinds of trouble.  Cornering was a particular problem. Flat out the tyres are warm and stick to the track, the brakes responsive and the down force of the car helps it round. Anything less than flat out and as Hammond found you are all over the place.

So in this case if your system is the car and you only use Lewis Hamilton to test it then there are a lot of defects that will get missed. Fortunately few people drive F1 cars slowly on the road, but this would be a severe problem if the system under test was something like a mini.

So moral of the day, remember for every Lewis Hamilton tester you have you need at least one Hammond.

After weeks of bullying from my pod mate Russell, followed by a great  two pronged sales job from Scott and Dan, I thought I should at least have a look at the new version  of Ubuntu.

So armed with Russ’ 8.10 CD I gave it a whirl, booting from the CD without installing it properly. I tried it first on my old T42 Thinkpad and it was up and running without any intervention. OK, I’m partially impressed.

Now the real test, would it cope with my 5 year old Evesham desktop?

Simple – it went on a treat, recognised all the bits (external harddrive, wired internet) – I’m now very impressed.

Plan is to buy a Mac in the new year and then turn the Windows box into Ubuntu. My daughter is not impressed as Linux doesn’t have MSN (at least that’s what I’m telling her – and will keep it like that as I think MSN has been responsible for 90% of the viruses on our box)

So count me as a new Ubuntu fan…..

Happy Christmas to both our reader. See you all again in 2009

I’ve just been reading Mike Cohn’s ‘User Stories Applied’ for agile software development.  Excellent book – well worth a read and it’s helped me clarify my understanding of what user stories are, how to use them and how they relate to use cases.

I’ve pulled out some sound-bites that I liked:

“Story cards are the reminder to talk!”

“Just because users have the problem does not mean they are uniquely qualified to propose its solution”

Also liked his analogy for gathering requirements, where he suggests its like fishing with nets. We start with a net with big holes so we only catch the really big requirements, then over time the holes in the net get smaller. Traditional requirement gathering uses a very fine mesh net that attempts to catch everything first time round. Usually sinking the boat (I made up that bit – hope you don’t mind Mike).

I’d recommend this book to anyone involved in agile projects. There’s some good stuff in there.

Now I know you’re going to say ‘Jon, you haven’t mentioned anything about test in this post!’. Well don’t panic, I’m going to do a seperate post on my thoughts around how it releates to testing, especially system test.

I have just been sent a link to a company called UTest who are building a software testing community.

Their basic model is to build a community of networked testers, invite development projects to submit their projects and then match testers with projects. Simple.

I can see this working very effectively for small scale, web based projects as it brings incredible diversity to the testing. I wonder how we could benefit from it in Enterprise testing?

Back in May I posted these thoughts on Agile SVT and I’m please to find I wasn’t too far of the mark all those months ago.
I was going through a presentation on ‘User Stories’ and one of the slides (page 73 out of 89) was dedicated to ‘Writing Stories for SVT scenarios’. Here’s the gist of it:

I dug these up when we were first thinking of a tag line for this blog.
Thought it was a good time to share them with you all.

“Why go into something to test the waters ? Go into it to make waves.”

“The greatest test of courage on earth is to bear defeat without losing heart.”
Robert Green Ingersoll

“Courage doesn’t always roar. Sometimes courage is the quiet voice at the end of the day saying, “I will try again tomorrow.” ”
Mary Anne Radmacher

“Here is the test to find whether your mission on earth is finished. If you’re alive, it isn’t.”
Richard Bach

“Let’s just say I was testing the bounds of reality. I was curious to see what would happen. That’s all it was: curiosity.”
Jim Morrison

“The test is to recognize the mistake, admit it and correct it. To have tried to do something and failed is vastly better than to have tried to do nothing and succeeded.”
Dale E. Turner

Program testing can be used to show the presence of bugs, but never to show their absence!
Edsger Dijkstra

Courage is not simply one of the virtues, but the form of every virtue at the testing point.
C. S. Lewis

Testing leads to failure, and failure leads to understanding.
Burt Rutan

“If it ain’t broke – you’re not testing it properly”
Jon Tilt

Not much activity on here recently, must be stuff going on somewhere.

Anyway, I’ve been pondering the role of testing in an agile environment more and more recently as its something we need to get right. I’ve already touched on tester driven design in an earlier post.  Now I’d like to look at three basic concepts that may help us move on. The three concepts are:

1. Forget about test phases and replace with ‘user stories’
2. The coding, testing continuum for a user story
3. The roles in an agile team

The first concept is probably the most scary. Most teams are familiar with the Design, Code, Unit Test, Functionally verify, System verify approach to developing software. So the assumption is that in an agile environment you will still need an FV phase followed by an SV phase. This is what I want to challenge.

I’d like us to think of it more simplistically. We write some code, we do some testing. What code and testing we do should be clearly defined by the user stories that are driving the delivery. So, if for example we are delivering a new API, we would have user stories to describe how the API would be used in the single user environment (equivalent to FV type testing) and stories that describe how it might be used in a high load environment (equivalent to SV type testing). So the right type of testing still gets done, we are just driven by the user story deliveries rather than by the traditional test phases.

This then leads nicely into the second concept which I’ll cal the ‘code/test continuum’. When you have user stories as described above you’ll find that some have a large amount of coding and a relatively small amount of testing. (Delivering an implementation of an industry standard API that already has a conformance test suite would be a good example of this). At the same time you will also have user stories that have very little coding, but large amounts of testing required (using the API in a complex environment under heavy load for instance).

So the two extremes may be expressed like this.

CODE………test..

code….TEST…….

But in reality the user story can fit anywhere on the continuum.

This is an important step to realize, because I think traditionally use cases and user stories have been geared towards the functional or code end of the spectrum and we need to ensure that all aspects of a delivery are considered.

Now how do we deliver on this sort of continuum? Well that’s where my third concept comes in. The roles of the agile team. For too long we have had the split in roles between coders and testers. This has to change if we are to be more agile. What we need is skilled software engineers who are capable of coding and testing. If we had a one person project then  it is obvious that that one person would need to perform both roles. so why is it that when our project team s grow  we feel we have to split the roles across different people?

If you take an ideal sized project team (about 10) I’d expect to see a spectrum of people ranging from a small number of pure coders at one end to a small number of pure testers at the other, but with the bulk of the team sitting in the middle as ‘software engineers’ who can switch roles depending on the needs of the user story. The team would morph with each delivery making much more effective use of the team’s resources.

I’ll be carrying on this thread as its close to my heart at the moment. Let me know your thoughts.

Big business frequently looks to the sporting world for fresh approaches to improve teamwork, motivation and goal setting. All too often the focus tends to be on the high achievers and their results, whilst overlooking the need to plan, develop and perform more like their sporting heroes.

The modern business trend is to adopt an increasingly flexible or agile business model.

In this article we will examine some of the techniques used by athletes and their coaches and explore how these can be harnessed to support such a business environment.

(more…)

I went up to London this week to attend my first British Computer Society (BCS) event. The Specialist Group in software Testing (SIGIST) hold quarterly conferences open to testers around the UK. Over 250 testers turned up for a day of presentations and workshops. I opted for all the presentations as the theme was Agile testing (my hot topic of the moment), but I hear the workshops were good too. I’d like to attend more of these and hopefully get the opportunity to present.

I picked up a few useful links:

• The QAGuild (Quality Excellence) has some good tooling information

• A collection of Testing blogs called TestingReflections. Well worth a read.

• Another collection of useful Test information and blogs – The UK Tester Forum

I’d like to thank one of our senior developers for inspiring this post. Dave Vines recently sent out an excellent, thought provoking email on how we should be moving towards a tester driven development model. I’ve slightly reworded his email:
“One thought that I’ve recently had was that perhaps we need “Tester Driven Development” rather than “Test Driven Development”. Let me try to explain.

We are currently very much a “Developer Driven Development” organization:
We gather new requirements and (typically) farm them out to the development community to investigate
Once a work item is identified, it is typically a developer that

• Writes the Design document
• Calls the meeting to develop the use cases
• Identifies the interesting scenarios
• Gathers the development resource to write the code
• Decides which iteration in which the code will be written
• Hopes that test will have the resource to test all the use cases and “interesting edge cases” that have been identified and for which code has been written (or not written)

In effect the developer owns the work item

In a “Tester Driven Development” organisation I would see us:
Gathering new requirements and (typically) farm them out to the test community to investigate
Once a work item is identified, it is typically a tester that:

• Writes the design document
• Calls the meeting to develop the use cases
• Identifies the interesting scenarios
• Gathers the test resource to test those scenarios and use cases
• Decides which iteration in which the code will be tested
• Indicates to development what use cases and code will be needed and when

In effect the tester owns the work item

This would be a massive culture shift for a department, but one that makes it clear that the desired deliverable is actually requirements that have been tested, not requirements that have been coded and may have been tested if test had enough resource. “

(more…)