Following the release of the first graphical interweb browser back in 1993 a whole new field of software development was opened up – that of the internet application.

Whilst we now take for granted the ability to enjoy our banking, shopping, socialising and entertainment online, back in the “early days” this future was by no means assured:

The proliferation of web applications and their increasing complexity pose huge challenges to us as testers. In this post we’ll take a *very high-level* tour of what to look out for, and how we might approach testing web applications.

The Challenges

Testing for a web application differs from testing in a traditional software application in a number of ways, not least because of the challenges faced:

The GUI problem

Almost every web application worth its salt has a strong emphasis on an engaging user-experience (or at least they should).

What this means in practice is a heavy-reliance on an easy to use GUI, and well thought-out visual cues aimed at the end user. For the tester, used to say a low-level API, or a traditional command-line based system this raises all sorts of issues when writing test-cases:

1. How do I know what to test ? The problem domain in the world of the GUI is usually an order of magnitude larger than a traditional command line environment – there are many more ways of doing the same thing.
2. How do I reliably automate my tests ? What tools are best of breed, and will they help or hinder ?
3. Are regression runs reliable ? What happens if a key button moves position ?
4. What about application versioning ? If the GUI changes between releases how does my test-case know if this is because of a bug or a new feature ?

Although tools such as Selenium and Rational Functional Tester reduce much of the heavy lifting when it comes to writing GUI test-cases, the experience can often be a largely manual one for testers.

Platform … what platform ?

An added complication for the Web GUI tester is browser selection. With Wikipedia listing dozens of active browsers or “platforms” for your web app how do you know which ones to test ?

Do you side with those who ignore the 20% market share of Internet Explorer 6 or test as many browsers as you can ?

And what of the operating systems on which they run ? Do you test on every single flavour of Linux ?

The browser and platform considerations are important in the pixel-perfect world of the web application where quirks mean that you cannot guarantee that a given UI component will always display in exactly the same place on screen when rendered by different browser DOM engines.

Added to these decisions are the shifting sands underneath your application – nothing on the internet is truly “finished” or “released” in the traditional sense of the word as we understand it in software terms. Browsers change, standards morph, and your app is expected to remain as shiny as the day you launched it.

Volume and load

No amount of testing can fully prepare you for this:

In the world of enterprise software and managed environments, application usage is often boringly predictable (i.e. usage is low !) but the web application world (particularly in the consumer sphere) is wildly unpredictable. Who’d have thought a 17-year old kid from Moscow would be the author of the latest runaway web hit ?

Testing your web application for load is a huge challenge, not least because nothing quite prepares you for the bizarre and unexpected ways users will find to break your app. At the very least you need to think about the mainline use cases for your app and exploit tools such as Apache JMeter or Apache Bench to let you know how far the app can be pushed before it starts to break at the seams.

Testing on a tight belt

So in the face of all these challenges how do you go about testing your web application efficiently on limited resources ?

Whole essays could be written about each of these areas, but you could:

• Hire cheap testers
  (A practice increasingly in-vogue amongst big old U.S. corporates):
  
• Plan carefully – If your development cycle can benefit from Agile practices then use them. Do test-driven development. Release early and often.
• Listen to your users – Especially Alpha/Beta users and early adopters – your customers can become a free test team !
• Use Automated GUI test tools – We’ve referenced them already. Use them but don’t forget that the object of your testing is a usable and performant web app, not a beautiful but useless regression suite.
• Practice Tactical Testing – Coined by a testing legend. Are all those different browser tests necessary ? Where do you get the “most bang for your buck” ?… but don’t use this as an excuse to not test the basics:
  
• Make use of testing communities – Both uTest and Amazon Mechanical Turk offer opportunities to engage with wider testing communities.
• Learn from the experts – Google place a high premium on having a solid testing rigor, with their mantra of “If it ain’t broke – you’re not trying hard enough“. They encourage testers to eat, sleep, breathe and even, er, meditate on testing. Even IBM has history of building hugely successful test communities such as their fabled “Black Team” of the 1960s.

Conclusions

The challenges faced in efficiently testing large-scale web applications can seem enormous, but with careful planning and prioritisation you can test them effectively. Hopefully this post has given you food for thought – we’d love to hear your views in the comments.

What do you make of, for instance, the commoditisation of testing – have the emergence of offshoring and cheap online testing resources diminished the role of the so-called “professional tester” or enhanced it ?

Tags: gui testing, testing blues, web application testing, web applications

This entry was posted on Wednesday, February 17th, 2010 at 18:16 by Chris Whyley. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.