search.noResults

search.searching

dataCollection.invalidEmail
note.createNoteMessage

search.noResults

search.searching

orderForm.title

orderForm.productCode
orderForm.description
orderForm.quantity
orderForm.itemPrice
orderForm.price
orderForm.totalPrice
orderForm.deliveryDetails.billingAddress
orderForm.deliveryDetails.deliveryAddress
orderForm.noItems
HIGH PERFORMANCE COMPUTING


Storage advances for HPC and AI


ROBERT ROE LOOKS AT STORAGE TECHNOLOGIES BEING DEVELOPED TO SUIT BOTH AI AND HPC WORKLOADS


The storage market is in a unique position, in that there is demand


from both the traditional HPC and enterprise storage markets, such as media and entertainment, alongside new growing markets for AI and machine learning. This has created huge potential to increase market share for storage vendors, as long as they can deliver the necessary performance required by HPC and AI users. While storage volumes


continue to increase dramatically, storage providers are trying to meet demand by increasing storage performance, while introducing more efficient methods of managing data across large multi-petabyte storage platforms.


Choosing the right system


for a particular workflow is critical to getting the most out of storage technology. The traditional products associated with parallel file systems still persist, but now there is increasing competition from cloud and all-flash storage arrays, which are becoming increasingly attractive to users at opposite ends of the hardware spectrum. Many HPC users have


complex requirements, and this means that there is no one storage technology that is perfect for every situation


8 Scientific Computing World April/May 2019


or workflow. As the price of storage hardware drops and new technologies, such as 3D NAND, become commoditised, HPC users are readying themselves for the next generation of HPC storage technology.


While it is not completely clear just how these technologies will be used in large scale file systems, the consensus among vendors is that SSD and flash technology will be key to developing large- scale storage architectures – particularly as we approach initial targets for exascale computing. The classical techniques


for increasing computing performance typically revolve around turning up clock frequencies and increasing use of parallelisation, but this has created a disparity between storage, memory and compute, as huge amounts of data must be fed into increasing numbers of processing elements. To combat this, storage


vendors are moving towards much faster, lower latency storage architectures with many opting to move data


”Hyperscale AIRI is designed to bring supercomputing capabilities to pioneers of real-world AI without the complexities that often occur when scaling across multiple racks ”


as close as possible to processing elements. This helps to reduce the penalty of moving data into and out of processors or accelerators for computation.


AI generates new storage technology In March, Pure Storage announced that it had partnered with Nvidia to release a portfolio of storage products for AI initiatives, from early inception to large-scale production. The company announced


a new hyperscale configuration of AI-Ready Infrastructure (AIRI), which has been designed to deliver supercomputing capabilities for enterprise users. The technology is aimed at AI users who demand the highest performance, or have grown beyond the capabilities of AI-


ready solutions in the market. Built jointly with Nvidia and Mellanox, hyperscale AIRI provides multiple racks of Nvidia DGX-1 and DGX-2 systems with both Infiniband and Ethernet fabrics as interconnect options. In addition, Pure Storage announced FlashStack for AI, a product built jointly with Cisco and Nvidia to deliver storage performance to meet the demands for data generated by the DGX-2. The partnership with Nvidia has also enabled Pure to deliver software advancements that fit with the existing tools provided by Nvidia. Using Nvidia NGC software container registry and AIRI scaling toolkit, data scientists can begin building applications with containerised AI frameworks and rededicate time to deriving valuable


@scwmagazine | www.scientific-computing.com


Gorodenkoff/Shutterstock.com


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32