Artificial Intelligence
Tools and IP facilitate low-power FPGAs for edge computing
By Hussein Osman, marketing director, Lattice Semiconductor E
dge computing’s development has been driven by the exponential growth of now ubiquitous smartphones and IoT devices, which send and receive information via the Internet. Some IoT devices, such as video cameras, generate enormous amounts of data during the course of their operations. Other IoT devices, like temperature sensors, individually create small amounts of data but since there are billions of these sensors, again there is a lot of data for the cloud to handle. Therefore, edge-based processing is essential to reduce the costs of the network communications and cloud storage costs, and to prevent the eventual overloading of links to the cloud.
Developers of these edge products and applications increasingly employ artificial intelligence and machine learning (AI/ML) algorithms for complex pattern matching and recognition to help analyse data and make decisions based on that analysis. AI/ML algorithms are now viewed as essential to the efficient processing of raw data because they identify and recognise complex, multidimensional data patterns that are increasingly difficult to isolate and identify using conventional algorithmic programming. Specific AI/ ML applications include the detection, recognition, identification, and counting of people and objects, asset and inventory tracking, environmental sensing, sound and voice detection and identification, system health monitoring, and the scheduling of system maintenance.
Many edge applications that can benefit from AI/ML deployment must function under extremely challenging energy constraints. Often, these widely distributed devices run on battery power. Such applications include factories, farms, office buildings, retail stores, hospitals, warehouses, streets, and residences. Increasingly, these devices are required to operate for long periods - months or years - on one battery charge
30 December/January 2022
or on harvested energy. Therefore, in such applications, devices will often spend a significant amount of the time in sleep mode until required to perform a function.
Low power FPGAs
Lattice’s newest FPGAs – the low-power, small-footprint, high-performance Lattice CertusPro-NX family of devices – are specifically tailored to meet the many design requirements of low-power edge devices. These FPGAs can support multiple sensors, displays, high-resolution video, networking, and edge AI/ML processing.
Meanwhile, the latest release of the company’s sensAI solution stack, version 4.1 (Fig 1) provides ready-to-use AI/ML tools, IP cores, hardware platforms, reference designs and demos, and custom design services that design teams need to develop and bring new edge devices to market quickly. The sensAI stack facilitates end-to-end AI/ML model training, validation, and compilation. Lattice has recently added sensAI Studio design environment, a GUI-based tool that helps developers build accelerated machine learning applications quickly. Configured with the tools in the Lattice sensAI 4.1 solution stack, edge computing designs can deliver real-time AI/ ML performance while consuming very little power – as low as 1mW to 1W – when based on Lattice iCE40 UltraPlus, CrossLink-NX, ECP5, or CertusPro-NX FPGAs. Support for the Lattice CertusPro-NX FPGA family in the sensAI 4.1 solution stack has allowed Lattice to add performance enhancements such as the ability to classify multiple objects simultaneously in real time, in addition to existing object detection and tracking capabilities. The sensAI 4.1 solution stack includes an updated neural network (NN) compiler and is compatible with other widely-used ML platforms, including the latest versions of Caffe, Keras, TensorFlow, and TensorFlow Lite.
IP cores in the Lattice sensAI 4.1 solution Components in Electronics Figure 1: The Lattice sensAI solution stack
stack include three types of convolutional neural network (CNN) accelerators – CNN, CNN Plus, and CNN Compact – and a CNN coprocessor engine. The CNN IP core allows developers to use many of the widely used CNNs published by others, such as Mobilenet v1/v2, Resent, SSD, and VGG, or to implement custom CNN models where needed. The sensAI 4.1 CNN accelerators simplify implementation of ultra-low power AI designs by leveraging the parallel processing capabilities, distributed memory, and DSP resources of Lattice FPGAs. The accelerator cores take advantage of the FPGA’s programmable logic to implement low-power NNs including extremely efficient binary neural networks (BNNs) to implement CNNs with extremely low power consumption, in the mW range.
Reference designs
Lattice FPGAs provide programmable I/O that can be configured to support many different electrical interface standards commonly used for sensor interfaces. The company also offers many hard and soft IP blocks to support different sensor communications protocols. Because FPGAs have long excelled at sensor fusion, the Lattice sensAI solution stack 4.1 is specifically designed to ease the development of AI/ML inference features based on multiple
sensors for edge devices, enabling the development of intelligent sensor fusion. The sensAI 4.1 solution stack includes many example reference designs that demonstrate intelligent sensor fusion use cases that can run concurrently to enable deeper context awareness. These reference designs include: • Hand Gesture Detection • Key Phrase Detection • Human Face Detection • Human Presence Detection • Object Detection, Classification, Tracking, and Counting
Conclusion
While the advantages of using AI/ML algorithms to improve the performance of many edge devices, such as autonomous robots, environment controls, and video security cameras, is fairly apparent, other types of edge equipment can also benefit – PCs and laptops, for example. Lattice is working with partners and customers to harness multimodal, intelligent sensor fusion and AI/ML techniques in the never-ending quest to enhance the user experience and to significantly reduce operating power and increase battery life by as much as 28 per cent in some applications.
www.latticesemi.com
www.cieonline.co.uk
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66
