Test data | 35
your development data up-to-date,
you can run an extract and import
•
Create a standard data object, for
the new data which will still makes example a Customer.
referential sense. You must, however
Before beginning a
•
Tokenise or parameterise those
from time to time change any offset objects so that you can vary them
values, hash keys or simple algorithms
scrambling project you
when you create them.
and refresh the test databases as users
will need to understand
•
Inherit objects so that you can make
will eventually work out the old and
your data. A request from
your own edits without affecting
new values.
management may be
the original data objects.
Offset values
as simple as “make sure
Library of functions
Managing offset values can be more
no one can recognise
Once you have completed your
complicated as data in one column/
a customer”; however,
analysis and built a library of standard
table must be related to the other
understanding what a
functions you can use these functions
column/table’s values for the system across the enterprise to scramble
to operate in the same way as
customer consists of is the
data in other systems and linked
production. Obviously understanding
first task before you can
applications.
where these types of relationships
begin scrambling.
For each table in the application
exist is crucial. The types of offset you must produce and store some
values include: level of documentation to verify
•
M icro aggregation – The values of beforehand, to create offset lookups is the scrambling is effective and
prior rows in a set of transactions an effective technique, especially with sufficient. Spend some time looking
refer to each other. date-type data. In addition, the ability at each column deciding on which
•
A pplication process driven to use portions of application code function or transformation should be
aggregations – Many systems may need to be incorporated into the applied to each column. You must
have application components scrambling routines to reset end of day also note any pre processing required
that calculate balances based on balances, customer totals etc. to build cross reference lookups
transaction throughput. These tend and any post-processing balance
to be separate processes which can Data creation aggregations etc.
be run as stand alones. For example, Many organisations have very strict
if you are adjusting transaction data usage restrictions and may well New builds and releases
amounts the customer balances need to create data from scratch to An often forgotten problem with
may not match. Running the balance use in their development and testing scrambling is that minor and major
adjustment process may be required environments. Having no access to new releases and builds may come
to reset these values. production data is common in many thick and fast so the scrambling
•
O ffset dates – If you adjust dates government departments as well toolset must be able to handle
then other values may well have to as more and more health care and the addition of new, possibly
be reset. An example of this is dates financial sectors. At first, creating sensitive columns, the addition
of birth; adjusting this by a few days data from scratch may seem like a of cross column relationships and
will alter the age and also the age difficult task, however when you offset values and any new functional
bracket so a person may move into a consider some of the difficulties with requirements which may need
different insurance premium. scrambling and the basic problem of new seed tables.
The aggregation and setting of offset developing with full size databases It is important that the scrambling
values can be the hardest part of any that are often out of date it may well
Huw Price
Director
methods are updatable quickly
scrambling project. In my experience, be an easier option.
Grid-Tools Ltd
and you do not have to spend time
www.grid-tools.com
the extensive use of dynamic tables The basic procedures to do this are: waiting, for example, for a DBA to
update some scripts. If this happens
the initial work will be quickly lost and
production data will begin being used
What a scrambling project should include
again in development and testing.
Scrambling projects often start
out as a request to the DBA team
• Understand your data – Its peculiarities, quality issues and interfaces.
to “do something to make sure
• Agree up front what data is sensitive – Don’t forget to include internal IDs.
• Discuss the different methods and how they apply to the application as a whole and in
production data is not being used in
detail at the column level.
development”. While the DBAs may
• Build up a library of functions – Consider reusing some of your existing code which may
be able to do a quick job to scramble
handle non standard situations.
some columns, the responsibility
• Consider buying in a toolset to scramble data.
for data ownership cannot live with
• Consider creating data from scratch – It is often easier than you think and has many
the DBA team. A scrambling project
benefits in terms of the amount of code covered in testing.
should come under normal application
• Document and audit everything you are doing.
control and be the responsibility of
• Use normal project control mechanisms.
the development team in conjunction
• Don’t forget that the scrambling has to be kept up to date – design your scrambling
with the application ‘data owners’. Too
application with this in mind.
many projects have failed to deliver
even basic data security by going
down this route.
T.E.S.T | March 09 March 09 | T.E.S.T
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52