Just been having a chat with a new grad who is working in one of our System Test teams.
We were talking about load, stress and performance testing and I was trying to articulate the reasons behind having a workload with peaks and troughs during the day (in the same way that a customer would) rather than our usual approach to load which is run at a constant high level for a long period. However, I was struggling with a justification as to why this might be a good idea and how it would find a different class of defects.
Then I remembered the ‘Top Gear’ episode when Hammond was trying to drive a Formula 1 car around a track. Apparently F1 cars are designed to go flat out – all the time – and are pretty good at this. What Hammond discovered to his horror was that when you don’t drive them flat out then you can get into all kinds of trouble. Cornering was a particular problem. Flat out the tyres are warm and stick to the track, the brakes responsive and the down force of the car helps it round. Anything less than flat out and as Hammond found you are all over the place.
So in this case if your system is the car and you only use Lewis Hamilton to test it then there are a lot of defects that will get missed. Fortunately few people drive F1 cars slowly on the road, but this would be a severe problem if the system under test was something like a mini.
So moral of the day, remember for every Lewis Hamilton tester you have you need at least one Hammond.