years ago and are sequential in nature. So just porting to GPU technologies and rewriting algorithms to take advantage of the parallel nature of the technology can provide a big boost in performance. ‘This is summarised in two main

reasons: one is with AI you have an easy on boarding strategy onto the GPU, almost like a porting strategy to get these algorithms onto the GPU in the first place. The second reason is that AI, like modern GPUs, are finely tuned to the right kind of workloads, so you get the full advantage of the hardware and software stack, including maybe mixed precision calculations and so on, that could be more difficult to achieve otherwise in the scientific domain,’ stated Angerer. An example of this can be found in work

by the European Centre for Medium- Range Weather Forecasts (ECMWF). They have been working to transform their weather reports by implementing DL technology in the prediction of weather effects from radiation transport. In this example, Angerer highlights how

DL can help to parallelise a previously sequential computing problem, which helps to provide a huge increase in the speed of the calculations. ‘You have this radiation transport which has multiple layers and each layer has some variables like moisture, aerosol concentrations, temperature and so on. Now they have an understanding of the underlying physics they want to model, they come up with an algorithm which is then fairly sequential. ‘Originally you go from A to B, from

layer to layer you go down once and then you go up again; it is somewhat like a tridiagonal solver in some sense - but that is a very sequential way of solving the problem. Now if you rephrase the problem as an AI problem, one thing that you have in an AI model is that you have significantly more parameters than you have in a handcrafted parametrised model, which could have 10 to 15 parametres that you tune. In an AI model, you could have millions,’ stated Angerer. ‘Now this could have the effect that

those models pick up on underlying processes that the original modeller may have not deemed important, or may not even be aware of. That can be one case but the second case is due to the fact that this parametrised model itself but of a certain fixed structure which makes it very amenable to acceleration. You have convolutions and fully connected layers for example, you could not really handcraft a neural network to solve this problem but if you train a neural network to solve this problem the structure of the neural network is way more parallelisable

6 Scientific Computing World October/November 2019

‘Those models pick up on underlying processes that the original modeller may have not deemed important, or may not even be aware of’

than it would be if you handcrafted the model to directly or indirectly model the underlying physics,’ added Angerer ‘From the way that you design an algorithm, a human could not design a neural network – the weights in a neural network – to achieve this performance, because that is not how our brain works. But if you teach the computer to come up with those weights, the algorithm that comes out of it is much more parallel, and that is where a big chunk of the speed-up comes from,’ Angerer explained.

Revolution in a neural network That is not the only example of a huge speedup but there are even more revolutionary effects that come from the implementation of AI. A clear example comes from the domain of image recognition, which was an area of interest for both scientists, enterprise and even

defence, long before AI and DL were used to solve this challenge. Whole careers were dedicated to the design of individual filters for edge detection, which were then combined to detect faces and skin colour. The introduction of AI was found to

solve these problems much faster than was possible before. While you need significantly more data to train the algorithm the fast increase in compute and data has meant this is now much cheaper to run on AI, than it is to try and create ingame detection algorithms manually. Other examples can also be found in

science, such as the use of language models to predict protein folding. A TU Munich paper essentially used a language model to train protein sequences. You feed a protein sequence into the system and it outputs another sequence which tells you about the secondary structures when this protein folds,’ said Angerer. ‘Talking with them, they said a similar

thing, one of the researchers dedicated their career under the assumption that human ingenuity was needed to design how these proteins fold but now it turns out that if you take an off the shelf language model, let it train for a couple of days - it can outperform 40 years of research,’ Angerer concluded.

@scwmagazine |


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36