BUREAUX SERVICES FEATURE
Is big data really such a big deal?
The buzz around ‘big data’ has reach epic proportions, but do marketers really need to get so excited and concerned about it in the UK, challenges James Lawson?
W
e hear a lot about “big data” and its associated storage, processing and analysis challenges at the moment: the
torrent of data produced by everything from web analytics to social media appears set to spread chaos through our systems. But does marketing really need to care about big data here in the UK? We talked to the experts to find out.
ELEPHANT IN THE ROOM First, some numbers. According to IBM, we create 2.5 quintillion bytes (1018 or 1 Exabyte) of data every day and 90% of the data in the world has been created in the last two years. With 24 hours of Twitter feeds equating to about 8 Terabytes, you can see why data volumes are ballooning at some businesses.
“Big data is an in-vogue term coined by the big consultancies,” says Ruaraidh Thomas, Managing Director of Data Lateral. “But it certainly is a challenge for marketing and it’s not just about volume.” The hype around volumes is not entirely unjustified. Machine-generated data is largely to blame for the big increase in new data production. Produced in much larger quantities than traditional formats, which tend to be relatively well-described and change slowly, data from everything from jet engine management systems to radio telescopes flows ever quicker and its volume trends remorselessly upwards. Companies involved in big data are collecting
it from many sources and using it for many reasons: running a mobile phone network, managing user-generated text and video, streaming stock market information, handling legal compliance or supporting research. “There’s a huge compliance element,” notes Thomas. “They have governmental responsibility to know what data they have in their business and store that compliantly.” So purely on volume, what sort of file sizes do MSPs commonly deal with these days? “We hold around 380m records on our combined file, which covers all b2b and b2c
www.dmarket.co.uk
reference data for the UK,” says Antony Allen, Managing Director of Data8. “The data is well structured and very tightly managed, with a minimum of 50,000 daily changes. The file size works out about 1Tb.” Darron Gregory, Director of Insight and Innovation at Celerity, estimates a “reasonably- sized” hosted client database of two to three
“Big data is an in- vogue term coined by the big consultancies but it certainly is a challenge for marketing and it’s not just about volume.” Ruaraidh Thomas, MD, Data Lateral
million customers would be in the “low Terabytes” today. Only four or five years ago, “it was rare for a marketing database to be larger than a few Gigabytes”. A three-orders-of- magnitude volume change shows that expanding data is a reality in UK marketing – but it’s still a far cry from big data.
So it’s not surprising that, though marketing use of big data is name-checked by vendors as a justification for system investment, real-life case studies are hard to find. Companies such as NICE are handling large data volumes to analyse call centre and web chat, Twitter feeds and other interactions in order to trigger offers or other
2.5 quintillion bytes the amount of data created every day
90%
the percentage of the world’s data that has been created in just the last two years
8 Terabytes
The amount of data created by 24 hours of Twitter feeds
actions, but examples tend to be more about “lots of data” rather than big data at the Exabyte-and-beyond level.
For marketing at the moment, it’s about “what’s in it for me?”; if new data is now available, how can we derive value from it? In the UK, it’s marketers themselves that are forcing volumes upwards as they demand more detailed information on customer and prospect behaviour. “The number of customers hasn’t changed, but the data around them has,” says Gregory. So rather than only using derived variables for lifetime value, date of most recent purchase, annual income by customer and so forth, there’s more interest in analysing every single transaction. Instead of basket value, each item is recorded. Where they might simply have logged a web visit or an online purchase, companies increasingly want page-level details, such as the products viewed, whether they were added to the basket and if a purchase resulted. Building a single customer view is where these larger data sets do challenge marketing today. Putting the four components commonly used to describe big data – volume, velocity, value and variety – into MSP hosting terms translates to storing and processing large amounts of structured and unstructured customer-related data and presenting it such that it can be swiftly analysed and exploited. “Volume is definitely in the top three challenges for any new client solution,” says Andy Grace, Technical Architect at Occam. “A simple rule of thumb for cost and effort is volume multiplied by complexity, where complexity is a measure of the number of data items to be integrated, the number and types of business rules that need to be applied to the data, plus the required transformation of data for final presentation.” To manage and analyse really big data quickly enough, companies like Google long ago moved away from relational databases to platforms such as MapReduce and Hadoop that can support work across multiple separate servers. Though most marketing database volumes may still be
May 2012 27
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32