24 | Agile testing
these high level metrics appear to be need to provide frequent interim
significantly higher than expected then performance test reports to ensure
further analysis of the other metrics
Types of
that any results are suitably escalated
can be carried out. This reduces the
performance
and dealt with in time for the planned
time spent on analysing tests and release date. The performance testing
targets time and resources on
testing
teams must be willing to challenge
problem tests. the release contents if there is
A performance test environment not sufficient time to test out the
checklist is produced and then updated performance risks.
Load testing determines the
after every release. This will reduce The performance modellers need
the amount of time lost due to
response time and throughput
to work more closely with the
environmental defects. performance testers in order to
Because of the frequency of change
during typical forecast load.
understand what the results of the
in an agile development process, a
Stress Testing determines the
performance testing will mean for the
significantly lower level of accepted production system. This needs to be
performance degradation per release
peak throughput, gradually
done rapidly during the performance
needs to be set. The performance testing process in order to make
increasing the load beyond that
degradation needs to be tracked judgements in ‘go’ or ‘no-go’ decisions.
in conjunction with the capacity
expected until the maximum
The capacity plan may need to be
and performance modelling work. reissued to understand the impact of
The diagram below shows how the
achievable load is reached.
the changes, eg what is the impact
cumulative impact in small changes
Volume Testing or Soak Testing
of a significant increase in database
in performance for every agile CPU time for a new feature on the
release can have a significant impact.
determines the problems that
production system?
Therefore the acceptable degradation
occur during long-term activity
per release should be three-times An agile result
lower for agile, if we assume each (eight hours to one week). The shift from more traditional
agile release is equivalent to a development to an agile development
‘traditional’ release.
Single User Testing determines
model requires a change to the
In order to assist the evaluation
the system resources consumed
performance testing methodology.
of the performance degradation, This includes the need to:
Capacitas has defined a standard
by a single transaction on an
•Work smarter in order to cope with
test in the performance test strategy
unloaded system, ensuring
the reduced timescales and higher
which is executed on every agile pressure environment;
release. This is called a ‘baseline queuing does not occur and
•
Have a proven methodology which
test’ and is executed before and will help to ensure that tests are
after the new code release. A more
allowing collection of service
designed well and meet their
detailed level of reporting is required
times (typically required for
objectives;
by the performance testers, who
•
Be more flexible in the approach
will formally raise all environmental
production of performance and
to resourcing;
issues as defects to ensure that they
capacity models).
•
Maintain a longer term view of
Paul Seaton-Smith
are attended to promptly, thereby performance rather than just
Principal consultant
reducing the loss of valuable test time. focussing on changes from one
Capacitas
www.capacitas.co.uk
The performance testers and analysts release to the next.
T.E.S.T | March 09 March 09 | T.E.S.T
Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44  |  Page 45  |  Page 46  |  Page 47  |  Page 48  |  Page 49  |  Page 50  |  Page 51  |  Page 52