This page contains a Flash digital edition of a book.


App Quality Alliance (AQuA)


Martin Wrigley, Chairman of the App Quality Alliance (AQuA) looks at mobile app testing and what impact the trend of testing in the cloud might have…

recent report from Research and Markets, Global Software Testing System Integrator Market 2012-2016, discussed the trend of vendor offerings geared towards cloud-based testing solutions. Now this may be all well and good for cloud-based enterprise systems, but what about mobile apps?


Once you agree that you need to test a mobile app (and frighteningly many available apps are totally untested until some poor unsuspecting user loads it on their device), your next question might be how to go about it and what tools are there to help and automate that testing.

Mobile app testing is a tricky topic, full of traps ready to catch the uninitiated. Basic functional testing is relatively straightforward, as it uses a single device and controlled conditions. But the reality of deployment is that there are many different devices that behave in different ways with different network connections, battery and performance characteristics.

The traditional approach was that a developer needed to physically functionally test their app on every device that they targeted. Clearly this was uneconomical. Add to this the need for speed of development and updates of mobile apps, and testing more than the basic function of the app starts to become a seemingly more complicated process.

In the ideal world, a developer would rely on the accuracy of a device emulator, and easily port an app across platforms (from Apple iOS to Android to Microsoft Windows Phone to Blackberry) but that is very rarely a reality. In some tightly controlled closed platforms, the issue is reduced as there is a single version of the hardware and OS from a single unified manufacturer. However, this is just one part of the market and the rest is still as fragmented and confused as ever.

With no magical solution in place, we are starting to see developers trending towards cloud-based solutions, which are becoming part of their answer. To begin with, there is no substitute to having a real target device in the development process. The app speed and feel is vital, and the emulator can only get you so far. Functional testing by the developers and then QA testing by an independent test team using a functional test script, combined with specified testing criteria, can get you to a good working app on one device.

Many apps have a continuous improvement cycle in place, and new releases are put out to the users every few weeks or months. One critical aspect to support this is the ability to re-run the functional testing scripts and check that only the

PAGE 30 intended

changes have happened. Such script automation is

feasible on a physical device, but if the original

functional test script is also implemented in a cloud-based tool then this can also be very effective.

Having achieved a well-behaved app on one device locally in the lab, there are a few other variables in the real- life scenario that need testing, to answer:

• Does the zz run well on other devices? • Does the app work in other geographies?

• Does the app perform well in variable network conditions, for example changing from Wi-Fi to 3G to 2G to losing the network altogether in a railway tunnel?

• Can cloud testing help here?

Cloud-testing really comes into its own when testing for the app’s compatibility on a wide range of devices. Having a bank of devices tethered in the cloud and available for a brief period of time at a fraction of the cost of owning the device is a great boon and clearly a sensible way to share such resources. Such cloud-based solutions can really help by allowing a set script to be run against a large range of devices to find any discrepancies. This is a major task that many developers face without such a tool.

However one aspect that is much tougher in the cloud is geography and location. The testing of, for example, a money transfer system on specific handsets overseas can really only be done with a real device on the ground in the right geography. Moving from location to location can be simulated in the cloud, but it can’t test the apps behavior as a GPS signal is gained or lost as the user moves indoors.

And then of course there is the network connection. Many apps go awry as the connection gets chopped and changed by the device. Whilst this can be simulated, the bank of devices is actually in a fixed datacenter somewhere. They’re not truly floating in the clouds so can’t really experience the everyday phenomena of losing signal.

There is no doubt that cloud tools for mobile testing are increasingly getting better. And although many challenges remain within mobile testing that cloud solutions inherently cannot address, a developer and test team armed with the right tools can successfully overcome such hurdles.

AUGUST 2013 |

Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44  |  Page 45  |  Page 46  |  Page 47  |  Page 48