data analytics ICT
traditional IT reporting solutions focused on historical data and shift them towards data modelling and governance solutions capable of predicting trends and correlations. The value of big data comes from the knowledge gained from it and what you do with it, as such, the promise of big data lies within its ability to make predictions – that’s what gets people excited.
Q A
As with most new developments, is the focus on the high end enterprise, or are the benefits of Big Data available to, and affordable for, SMEs as well?
If you were to believe everything you read, you would be under no illusion that ‘Big Data’ is, well, big. However, Big Data is a relative term and applies as much to SME’s as it does to the high end enterprise. Every company has a tipping point and most organisations – regardless of size – will eventually reach a point where the three V’s – volume, variety and velocity - of their data make it difficult for them to extract business value anymore.
Whilst it’s interesting to those who are technical to focus on size, the real focus should be on business value first. Let’s be honest, not all SME’s will take the time or energy to even gather the data surrounding their company, let alone analyse it. But those that do go through the process will potentially get a leg up on the competition and provide themselves with insights that others simply don’t have.
The key is to focus their efforts on a few business-critical sets of data rather than investing in an ‘all singing, all dancing’ big data solution. Companies, in particular SME’s, are not prepared to make speculative investments in expensive solutions just in case there is something to be found but instead can now turn to alternative hybrid emerging technologies which can provide a more cost effective solution.
Q A
Are there specific Big Data vendors, or how/where are data analytics solutions available?
The Big Data landscape is dominated by two classes of
technology: £ Operational – systems that provide operational capabilities for real-time, interactive workloads where data is primarily captured and stored; and
£ Analytical – systems that provide analytical capabilities for retrospective, complex analysis that may touch most or all of the data. These classes of technology are complementary and frequently deployed together.
Operational and analytical workloads present opposing requirements and systems have evolved to address their particular demands separately and in very different ways. Each has driven the creation of new technology architectures. Operational systems, such as NoSQL databases, focus on servicing highly concurrent requests while exhibiting low latency for responses operating on highly selective access criteria.
Analytical systems, on the other hand, tend to focus on high throughput; queries can be very complex and touch most if not all the data in the system at any time. Both systems tend to operate over many servers operating in a cluster, managing tens or hundreds of terabytes of data across billions of records.Technologies such as Hadoop have emerged to address Big Data challenges and to enable companies to leverage the capabilities of their operational systems by combining it with analytical systems to develop new types of products and services to be delivered by the business.
The latest Apache Hadoop framework consists of the following modules: £ Hadoop Common – contains libraries and utilities needed by other Hadoop modules
£ Hadoop Distributed File System [HDFS] – a distributed file- system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster.
£ Hadoop YARN – a resource-management platform responsible for managing compute resources in clusters and using them for scheduling of users’ applications.
£ Hadoop MapReduce – a programming model for large scale data processing
Hadoop and other Big Data vendors are available from specialist resellers such as S3.
Q A
What are the major issues to consider when looking at Big Data analytics and the potential benefits it offers to an end user organization?
It is essential that you think about the requirements and design of your big data analytics project from the start. As your data grows so do your IT requirements – often – the gap between the business need and the IT infrastructure gets even bigger. To overcome these challenges the major issues to consider when looking at big data analytics are: £ Skills – getting real knowledge out of data is not really an IT capability but a skills issue. Big data technologies require much more of a software development bent rather than an IT systems management skill set. You certainly don’t need to take a big bang approach in terms of implementation but instead leverage standard architectural principles and in-house skills to ensure you don’t box yourself in with either proprietary products or services
£ What data to collect – data is often collected and persistently stored, mainly for disaster recovery, but this may not be the most flexible way to maintain the data to make it have future value
£ Data Volumes – sampling data is not appropriate the whole data needs to be available to provide the proper insights for the business value
£ Data structures – data needs to be structured in a way that makes it accessible for ad-hoc analytics. Once structured it’s very hard to fix after the fact. Other factors will be how you can apply analytics to determine what to do with your data, determine which data is relevant and how or whether data should be stored
£ Technology – identify the business challenge or goal and then align this with the technical approach and solution size. Next decide whether batch mode processing versus real-time or
May 2014 I
www.dcsuk.info 31
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56