HPC 2014-15 | Soſtware tools


provisioning, and management of Big Infrastructure [for] fully configured Big Data clusters.’


Managing resources and managing workloads Van Leeuwen stressed the distinction between soſtware tools that are intended to manage the compute resources – hardware, operating system, middleware, libraries, security, etc – and those that are focused on optimising the way in which the application soſtware runs on the underlying resources. Bright Computing is distinctive in that it focuses on managing the resources, but offers more than that by ensuring that, once installed, Bright Cluster Manager (BCM) integrates with, manages, and monitors all common HPC workload managers. Bright Computing is already ahead on the


issue of the cloud, according to van Leeuwen: ‘Tis is a trend we have seen coming for quite some time. We have made it very easy to stand up a complete HPC cluster inside the cloud and manage it the same way as you would manage your own on-premises cluster.’ However, he believes that the soſtware is particularly innovative in the way that it has made it possible to extend an existing, on- premises, cluster into the cloud. For example, if a company has 50 nodes on premise but needs to double its capacity over 30 days to accomplish a particular task: ‘We make it almost trivial to add another 50 nodes inside a public cloud like Amazon. And to my knowledge, we are unique there – I am not aware of anyone else that can do that.’ And there is a demand for this, especially


in the life sciences, Van Leeuwen continued. ‘All the pharmaceutical companies are running some of their workload in the cloud. Most of them run it separately from their on-premises resources, so they need to take several manual steps to add capacity in the cloud, and to move applications from on-premises to the cloud,’ he said. Bright Cluster Manager (BCM), in contrast, gives them the advantage of running across on-premises and the cloud. Ease of use was key, he said: ‘Tey don’t have to move the workload over to the cloud manually – they can do it automatically, and even dynamically based on policies.’ While Bright Computing is well known


in the HPC market, van Leeuwen sees the recent capital injection as allowing it to develop further into areas such as Hadoop and Open Stack. ‘Our customers pulled us into those markets,’ he said. ‘People think that Hadoop and Open Stack install on


26


bare metal – on empty servers. But they are unpleasantly surprised when they discover that they first need to prepare the servers, install the operating system, configure the operating system, configure a network, and only then are they ready to install Hadoop or Open Stack. And then they discover that it is really hard to do. Open Stack is really a bunch of independent services that need to be configured individually and with each other. Tere’s millions of ways of getting it wrong.’ With Open Stack as an add-on on top of


BCM, however: ‘We have made it much easier to get started with Open Stack, while still


allow you to do any other types of workload anymore.’ He cites Cray as a solid HPC company that is also moving into the Big Data, Hadoop world. Cray is a significant Bright partner, using BCM to manage all the external service nodes.


One ecosystem for converging technologies Te convergence of Big Data, cloud, and HPC lies behind Moab 8.0, the new release of Adaptive Computing’s optimisation and scheduling soſtware tools. Speaking at the preview at ISC’14 in Leipzig in late June, Jill King stressed the extent of the innovation in the new release, with its focus on enabling companies to take ‘data-driven decisions for competitive advantage’. Te soſtware would allow users ‘to improve results without having to install new infrastructure,’ – by a factor of between two and three in terms of performance boost, according to the company. And Adaptive Computing too is integrating Open Stack into its package. Te new release is a single ecosystem, King


“We have made it easy to stand up a complete HPC cluster inside the cloud and manage it in the same way as you would manage an on-premise cluster”


allowing, aſterwards, any expert change that you would like – but most users don’t need these expert options.’ Hadoop is simpler to install and is not


as extreme as Open Stack but, according to van Leeuwen: ‘You have still to do all the preparation work yourself unless you use BCM.’ He views Hadoop as partially a form of HPC: ‘A lot of people attracted by Hadoop don’t need Hadoop; they need traditional HPC but they don’t realise it. When you do Big Data analysis, it is very close to traditional HPC data analysis. We can combine traditional HPC workloads with Hadoop workloads. If you want to do that on a cluster that is deployed and built with one of the management tools provided by Hadoop distributions, you are in trouble. If you take Cloudera Manager, for example, that is really focused on managing Hadoop clusters – it takes full control of your cluster, it’s very possessive, and does not


said, that provides the wherewithal to unify resources: in HPC and Big Data; across private and public clouds; and across virtualised and bare metal machines. As engineers and scientists want to process intensive simulations and carry out Big Data analysis to accelerate insights, this convergence of Big Data and better workflows leads to the concept of ‘Big Workflows’ which is at the heart of the new release, she said. In creating the Big Workflow solution, she continued, Adaptive Computing sought to understand the dependencies and thus remove log jams in the workflows. Te result is dynamic scheduling, provisioning, and management, of many applications running across HPC, cloud, and Big Data computing. Power consumption is directly addressed,


she continued. King said that the soſtware can slow the CPUs, by decreasing the clock frequency, and this can be done through a policy set out for all similar jobs. In addition, it offers power states and options including ‘suspend’ and ‘hibernate’ that go beyond simple ‘on/off’ control. Te company believes that Moab 8.0 can cut energy costs by between 15 and 30 per cent.


Scalability, reliability, and performance Power consumption is also one of the items on the agenda for Altair in developing the latest version of its PBS Works soſtware suite, according to Bill Nitzberg, the chief technical officer. ‘Power really resonates’, Nitzberg said.


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40