Feature: AI
Whilst Nvidia claims 130TOPS of peak performance on T4 cards, the real-life AI models like SSD MobileNet-v1 can utilise 16.9TOPS of the hardware
Enabling AI productisation
By Nick Ni, Director of Product Marketing, AI, Software and Ecosystem, and Lindsey Brown, Product Marketing Specialist Software and AI, both at Xilinx
Keeping up with demand Demand on hardware in AI inference has skyrocketed, since modern AI models require a lot more compute power than conventional algorithms. Yet, as we already know, we can’t rely on gradual silicon evolution. Processor frequency has hit a wall with Dennard scaling (or Mosfet scaling): an algorithm can simply no longer enjoy a “free” speed-up every few years. Adding more processor cores has also hit a ceiling, thanks
to Amdahl’s Law: if 25% of the code is not parallisable, the best speedup is 4x, regardless of how many cores have been crammed in. So, how can hardware keep up with such increasing
demand? One answer is Domain Specific Architecture (DSA). Since each AI model is becoming heavy-duty and dataflow
complex, today’s CPUs, GPUs, ASSPs and ASICs can’t keep up. CPUs are generic and lack computational efficiency. Fixed hardware accelerators are designed for commodity workloads that don’t welcome further innovation. DSA is the new approach where hardware needs to be customised for each group of workloads to run at highest efficiency.
D
ata is exploding, innovation is exponential and algorithms are changing rapidly. Whilst artificial intelligence (AI) is increasingly adopted in many industries, most AI revenue comes from training AI models, by improving their accuracy and efficiency.
The inference industry is just getting started and will soon surpass training revenue with “productisation” of AI models, where a model can be bought as a product to fit an application’s requirements. Since we are still in the early phases of adopting AI
inference, there’s a lot of room for improvement. For example, most cars still don’t have advanced driver-assistance systems (ADAS); drones and logistic robots are still in their infancy; robot-assisted surgery is not perfect yet; and there are many enhancements needed in speech recognition, automated video description and image detection.
Customised for efficiency Every AI network has three parts that benefit from customising for highest efficiency: its data path, precision, and memory hierarchy. Most newly-emerging AI chips have high horsepower engines but fail to pump data fast enough due to these three inefficiencies. Every AI model will require slightly – or sometimes
drastically – different DSA architecture. The first part is a custom data path. Every model has a different topology (broadcast, cascade, skip-through, etc.) passing data from layer to layer. It is challenging to synchronise a layer’s processing to make sure data is always available for the next layer to begin its work. The second part is custom precision. Until recently,
floating-point 32 was the most prevalent precision in designs. However, with Google TPU leading the industry in reducing the precision to Integer 8, state-of-the-art has shifted to even lower precision like INT4, INT2, binary and ternary. Recent research confirms that every network has a different sweet
www.electronicsworld.co.uk September/October 2020 29
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68
