QA AND TESTING | BUILD Lombardi’s colleague at VMC, Linda
Lemieux, director of development, integration and QA says that as well as the need for a good reputation, with the internet allowing unhappy customers to quickly come together and speak out, the plethora of options available means that demand for quality is higher than ever. “QA has always been important. However,
today the consumer has many more options,” she explains. “They are used to paying a dollar or two for a good product through online services. Because of this competition, price sensitivity and lower price points, if a customer is going to pay $50 or $60 for a major title, it better work right. The customer is demanding much higher quality.” Babel CEO Richard Leinfellner believes that the importance of outsourcing work to external games testers has also come from industry changes and tough economic times, with many companies restructuring with smaller output, leaving internal QA teams with a reduced volume of work. “With many publishers these days
downsizing, more often than not, they won’t have the throughput of titles needed to keep their in-house QA groups sharp, which means they may miss things leading to unwelcome delays just at a point where they should be getting much needed revenue in,” he says.
GOING MOBILE Whilst the importance of testing in triple-A titles for console and PC has become more prominent, the rise of mobile and app stores has offered up a mixed reaction towards the process. Whilst titles must be certified by Apple before they can reach the App Store, as explained earlier, how stringent these test are is up for debate. Developers working on Android meanwhile do not require any form of certification, meaning the quality of apps can vary widely. Leinfellner says that whilst it can be easy for
given the inherent risks in making a game? The focus on quality has only increased over time, although there is content out there which has clearly not gone through a QA process. It is as if we are now in an environment of extremes.” The QA process can start as early on as the
design phase, with specialists helping developers as the creative process begins, working out feasible aspects of a game’s development, identifying logic flaws in requirements and using their experience to highlight likely bumps in the road ahead. The process then accompanies the game’s creation through alpha, beta and certification stages, beyond to release and then even post-launch, with updates and the advent of DLC in recent years becoming ever more popular.
RISING EXPECTATIONS But the integration of a solid quality assurance practice throughout development has not always been recognised as integral to a game’s creation. Whilst its importance at some stage has never been in question, Testology’s managing director Andy Robson says the industry’s awareness of games testing has greatly risen over the past ten years.
DEVELOP-ONLINE.NET “Ten years ago, opinions on the importance
of QA were perhaps divided,” says Robson. “Having said this, Bullfrog and Lionhead Studios always regarded QA as a department
If a customer is to pay $50 or $60 for a
major title, it better work right. The customer is demanding much higher quality.
Linda Lemieux, VMC Labs
with high importance. I think the importance has always been high, but the industry’s awareness of this has developed.” Lombardi agrees that games testing has
become more prominent than ever, and highlights the example of a large publisher that demonstrated an in-development title, only for the demo to fail within the first minute of play. He says this resulted in word of the failure spreading quickly across the internet, with the title then shelved for 18 months.
devs to use the public to beta test its titles, this method is not conducive to long-term success, as the negative reviews begin to stack up. “As a small developer it must be tempting to see the customer as a free tester, however the novelty for the consumer soon wears off and buggy games end up dropping from the charts rapidly,” he explains. “Babel do understand the issues of cost though, which is why we invented our low cost, fixed price TestMyApp and LocaliseMyApp service. Triple-A go to market services on a fixed low price.” VMC’s senior program manager for mobile
services Jon Atkinson says that just five years ago, releasing content on mobile required passing certification programs, providing a baseline in terms of acceptable quality. This required developers to conduct testing of their products to ensure they could meet these standards. But Atkinson believes this has changed, although serious mobile developers can see the importance of testing. “The advent of the App store, and
developers being able to submit content directly to the store, in some cases with no quality assurance, has led to a reduction in the overall level of quality built into content,” explains Atkinson.
MAY 2012 | 53
Alastair Harsant of Testronic Labs (top), Alfonso Lombardi from VMC Labs (middle) and SQS India’s Gireendra Kasmalkar (bottom)
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76