ARTIFICIAL INTELLIGENCE FEATURE
Leaning over
the threshold AI applications on the network edge
Words by Hussein Osman, marketing manager, consumer segment, Lattice Semiconductor V
oice and other commands for connected applications on the
edge have typically been processed in the cloud. But there are a number of reasons why it is preferable to move that processing to the edge; developers are making that happen using FPGAs. More broadly, AI at the edge involves the creation of a trained model of how something works. That model is then used to make inferences about the real world when deployed in an application. This gives an AI application two major life phases: training and inference. Training is done during development,
typically in the cloud, whereas inference is required by devices as an ongoing activity to refine performance. Because inference can be a computationally difficult problem, much of it is currently done in the cloud. But there is often little time to process data and transmit. This need for real-time control applies to many application areas where decisions are needed quickly – applications such as smart home devices and industrial equipment. Because of this, there is a strong move to take inference out of the cloud and implement it at the edge – that is, in the devices that gather data and then take action based on the AI decisions. There are other benefits to local inference. The first is privacy. Data around the cloud is subject to hacking and theft. But if the data never leaves the equipment, then there is far less opportunity for mischief. The other benefit relates to the costs associated with cloud-based data analysis. Sending data up to the cloud for real-time analysis is costly, particularly when multiplied across thousands of edge devices: making the decisions locally can reduce costs.
There are two aspects to building an inference engine into an edge device: developing the hardware platform that will host the execution of the model, and developing the model itself. Execution of a model can, in theory, take place on many different architectures. But execution at the edge, especially when taking into account the power, flexibility and scalability requirements of edge devices, limits the hardware platform choices available for powering an inference engine – particularly for always-on applications.
MCUS, ASICS AND ASSPS The most common way of handling AI models is by using a processor. But the processors in edge devices may not be up to the challenge of executing even simple models; such a device may have only a low-end MCU available. Alternatively, using a larger processor may violate the power and cost requirements of the device, so it might seem like AI would be out of reach for such devices. This is where low-power FPGAs can
play an important role. Rather than beefing up a processor to handle the algorithms, FPGAs can act as a co-processor to the MCU, providing the heavy lifting that the MCU cannot handle, while keeping power within the required range.
/ ELECTRONICS
For AI models that are more mature and will sell in high volumes, ASICs or application-specific standard products (ASSPs) may be appropriate. But, because of their activity load, they will consume too much power for an always-on application. Here, FPGAs can act as activity gates, handling wake-up activities based on a spoken trigger phrase, or recognition of some broad class of video image, before waking up the ASIC or ASSP to complete the task with high confidence.
STAND-ALONE FPGA AI ENGINES Finally, low-power FPGAs can act as stand-alone, integrated AI engines. The DSPs available in the FPGAs take the starring role here. Even if an edge device has no other computing resources, AI capabilities can be added without breaking the power, cost, or board-area budgets. And they have the flexibility and scalability necessary for the evolving algorithms, common to applications like smart devices and industrial cameras, used to detect defects or count objects/ people. Designing hardware that will execute an AI inference model is an exercise in balancing the number of resources needed against performance and power requirements. To stay dominant in very competitive markets, developers must design computing solutions running at the network edge that are appropriate to relevant requirements. Systems based on small, low power FPGAs that enable inferencing close to the source of a command or IoT data, can help OEMs meet these stringent demands.
Lattice Semiconductor
www.latticesemi.com
ELECTRONICS | NOVEMBER 2019 31
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48
