high-performance computing

Speeding up the storage stack

Robert Roe discusses the merits of the latest storage

technologies, including a push by storage providers to develop end-to-end platforms featuring intelligent data management systems

performance while introducing more efficient methods of managing data across large multi- petabyte storage platforms. Molly Rector, chief marketing officer,


executive vice president, product management and worldwide marketing at DDN explained that choosing the right system for a particular workflow is critical to getting the most out of your storage technology. ‘We have a very technical pre-sales engineering

team on site with the customers, just listening to their requirements’, said Rector. ‘Tere are a lot of subtleties there that would

lead to one product or another but there are four or five key things’. Rector explained that the key principles to

consider are based on expansion and upgrading requirements, the scale of the data, the number of sites that need to be connected and whether the system is going to be a read or write intensive environment. ‘In a lot of HPC centres, IO is not their

bottleneck but in others it is. Ten you go to things like object storage, which is great for low- cost, long-term storage of data but not great for connecting straight to a compute environment – it just doesn’t have the right performance attributes. ‘Tey each have their spot and it can be

difficult for a customer to choose without a bit of consulting work’, concluded Rector. Tese complex requirements mean that there is no one storage technology that is perfect for every


hile storage volumes continue to increase dramatically, storage providers are trying to meet demand by increasing storage


situation or workflow. Tis led to two types of file system – Lustre and GPFS – gaining popularity but, recently, Object Storage has also started to see some use in scientific or HPC environments – thanks to its cost and ability to keep contextual metadata alongside experimental data. Choosing the right storage technology can

be a difficult business, but Cray has developed an end-to-end platform based on its Sonexion storage platform, which it believes can support the majority of workflows because of its inherent flexibility. Barry Bolding, senior vice president and chief

strategy officer at Cray, said: ‘Over the last 10 years or so Lustre is what the market has been demanding. Te performance of Lustre is far better than GPFS for certain types of workloads. Now, GPFS can be better in certain situations as well but, for most of our customers, Lustre offers the highest performance.’ However, while Cray does favour Lustre,

Bolding stressed that Cray’s philosophy is to provide a flexible system that users can customise to suit their needs. ‘If you are developing a storage

product, you take a storage view of the world’ said Bolding. Cray takes a systems view, and from a system’s

perspective you need to have different tiers of storage – and they have to be visible to the user – because that is the only way to make sure that you can get the best performance out of your tiered storage.’ However, as Bolding explained: ‘we have a few

customers who use GPFS’; it is possible to remove the Sonexion Lustre file system and replace it with a GPFS based solution. ‘What drives us towards Lustre is market demand from what we see in the requirements from our customers,’ he concluded.

The use of Object storage in scientific computing In recent years, Object storage technology has started to gain traction in scientific computing industries. While it cannot offer the high- throughput performance of a traditional file system, it changes the conventional storage paradigm, storing data as objects with associated metadata rather than files in a specific format. Tis provides considerable options for data analytics, managing data more efficiently and even re-using data long aſter an application – with an associated file format – has been removed from the system. Matt Starr, CTO at Spectra Logic, said: ‘Lustre

➤ @scwmagazine l

Joe Techapanupreeda/

Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44