Embedded Technology
2020: Transitioning to a fully connected future
By Gideon Intrater, chief technology officer, Adesto Technologies Corp
2
020 will be an important year of transition, where many of the life- changing technologies and products
that are on the horizon will get a lot closer and begin to have an impact. 2020 will see the mass-rollout of 5G technologies, which will start to pave the way for Industry 4.0, fully autonomous vehicles, AI at the edge, and a host of new products and services that the higher speeds and bandwidths will enable.
The semiconductor industry is continuing to create innovative solutions to bring about this fully connected future. One area of innovation is in making AI inference at the edge a practical and scalable opportunity. The challenge lies in the fact that AI inference requires significant processing capabilities, consuming a great deal of power that edge devices can’t afford. The matrix multiplication at the heart of AI processing may begin at the first level with millions of operands, i.e. weights. While the matrices get smaller as the computation progresses, each operation requires iterating through multiple dot product operations on vectors reading two operands from memory, multiplying the operands and accumulating the result of the multiplication. This process
translates to relatively high-power consumption and potentially high cost.
Industry powered by innovative algorithms People are using a variety of solutions today. One option is off-the-shelf MCUs, which are available from multiple vendors, and in some cases a good option. The weights are stored in a fast, external memory – usually an octal flash device such as Adesto’s EcoXiP non- volatile memory, to enable fetching them at a very fast rate. In such a traditional computing model, data moves through a single combined bus, and typical solutions can reach up to 0.1 TeraOps per watt. For more demanding inference algorithms, dedicated hardware is required. And indeed, there are a good number of SoCs in the market today from numerous vendors that have dedicated AI acceleration hardware. The advanced process nodes and dedicated hardware make these designs relatively efficient. However, they still suffer from the fact that weights need to be fetched from an external memory, dissipating power as the weights are brought into the SoC.
A more intelligent future Looking ahead, algorithms will continue to grow in complexity, further pushing the limits of AI inference hardware at the edge. To address this, digital inference architectures are evolving beyond MCUs and SoCs with dedicated inference engines, taking advantage of deep sub- micron processes and new architectures including near-memory processing.
future AI edge inference engines through in-memory analog processing, where compute resources and memory reside in the same location.
In this paradigm, the matrices of the deep neural networks (DNNs) become arrays of NVM cells, and the weights of the matrices become the conductance of the NVM cells. The multiply operations are done through an analog multiplication of
By integrating the memory and the AI
hardware accelerator on the same die, near-memory processing can lead to higher performance with lower power. One commercially available example of such a solution claims to reach 9.6 TeraOps/watt. This is enabled by the higher bandwidth between the memory and the execution units and, also, because there is no need to fetch the weights across chip boundaries. However, there is a tradeoff between
efficiency and scalability because the number of weights is limited to what’s possible to implement in a silicon device. In addition, to limit the power consumption, a leading-edge silicon process is required. In such process nodes, area- and power- efficient non-volatile memory (NVM) cells do not exist, and expensive SRAM arrays are required.
Matrices get smaller with each step of the computation 16 March 2020 Components in Electronics
Balancing power efficiency and scalability We believe there is a need for even further innovation, and we see great promise for
the input voltages by the cells’ conductance. The resulting current of these operations on all the vector elements are added in an analog fashion to produce the MAC results. Since there is no need to move the weights, this model can achieve an unrivaled combination of power efficiency and scalability. One industry estimate for this solution reaches 28 TeraOps per watt. There are of course challenges to in- memory analog processing, and we won’t realise the promise of this technology in 2020. However, we believe the industry will make progress in this area during the year. Analog solutions are catching up, using single-transistor storage and in-place computation without expensive hardware multipliers. Hybrid analog and digital computing may be the only way to meet the challenges of AI inference at the edge as we rush into the fully connected future.
adestotech.com
www.cieonline.co.uk
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48
