EMBEDDED DEVICES
Figure 2: GStreamer pipeline for running the application
allows real-time decoding of up to eight 1080p streams or two 4K streams. Furthermore, the
i.MX 8M’s powerful quad core CPUs and GPU can handle the pre- and post-processing functions of multiple inference pipelines.
Choosing the right number of Ara-1 devices
functions. One example of post processing can be non-maximal suppression (NMS) logic, which is often used to interpret the output of object detection models. Although post processing is typically performed on the host processor, in some cases it can be offloaded to Ara-1. Either way, the post-processed results are then made available to the application so it can implement the desired business logic. AI applications must manage the execution of pre-and-post processing steps plus the inferencing on Ara-1. Most smart camera applications also require multiple models to be executed per frame or per stream, either in parallel inference pipelines with multiple models running independently of each other, or in a cascaded pipeline where one model executes first. Based on the result of that model, another one or more instances of another model is executed.
To simplify deployment of such AI pipelines, NXP platforms provide GStreamer and NNStreamer frameworks. That means that regardless of which
i.MX SoC is deployed and whether the pre-processing is run on CPU or GPU, GStreamer is the framework for creating streaming media applications, abstracting the hardware layer to allow any
i.MX SoC to be used without having to change the underlying vision pipeline software. As a library designed for the construction of media-handling component compute graphs, GStreamer makes it easy to assemble multiple filter pipelines to handle any multi-model scenarios.
The addition of NNStreamer plugins enables GStreamer developers to efficiently adopt neural network models into the vision pipeline that run on the CPUs, GPUs, or NPU present in the
i.MX applications processor. Kinara has developed a set of GStreamer compatible plugins that make it seamless to integrate Ara-1 into NXP inference pipelines.
Building edge AI appliances with
i.MX 8M SoCs and Ara-1
22 NOVEMBER 2022 | ELECTRONICS TODAY
Edge AI appliances can also be used, instead of smart cameras, for AI computations. In this scenario, cameras can be regular IP cameras without AI capabilities. The feeds from multiple cameras and other deployed sensors are routed to an AI appliance for inferencing. From a practical standpoint, each AI appliance can handle 4-16 camera and sensor feeds. In a large facility that requires hundreds of cameras, multiple appliances are deployed. This enables a far more scalable processing model compared to edge servers where feeds from hundreds of cameras must be routed to a set of central servers, incurring substantial infrastructure and power-hungry server costs. Unlike the direct sensor interface used in a smart camera, an Edge AI appliance typically uses Ethernet or Wi-Fi to receive video feeds over an IP network. Moreover, the incoming feeds are generally encoded so the appliance must decode each incoming stream before it can be processed. Finally, unlike smart cameras, an appliance must perform inferencing and associated pre-and- post processing tasks on multiple streams. Such an appliance would need multiple Ara-1 processors for higher AI compute capability as well as a more powerful host processor. One such design uses NXP’s
i.MX 8M Quad applications processor as the host (Figure 3). Its hardware-accelerated, video decoder
The maximum number of streams supported by an edge appliance is partly a function of the host processor and how many streams it can decode and perform pre- and post- processing on. Similarly, the number of Ara-1 devices in the appliance determines how many AI processing streams can be supported. A single Ara-1 device can handle multiple video streams, but the number of streams per device is directly linked to model complexity and size. For example, an eight stream AI edge appliance typically requires anywhere from 1 to 4 Ara-1 devices depending on the model complexity and desired processing rate.
Building Flexible AI Solutions Whether building smart cameras or edge AI appliances, a developer can integrate inferencing functions into a wide range of NXP
i.MX applications processors. As discussed above, different
i.MX SoCs serve different functions. One example is the
i.MX 8M Plus and its ISP and
i.MX 8M Quad that provides decode support for eight 1080p streams or two 4K streams. From a hardware design perspective, further flexibility and simplicity is achieved by combining any
i.MX SoC with Ara-1 because of the common PCIe or USB support. And finally, as we’ve seen, from a software development perspective, Kinara’s utilisation of GStreamer to easily add Ara-1 into the inference pipeline and eliminate major changes to code when switching
i.MX applications processors greatly streamlines the ability to readily expand the capabilities of smart camera or edge AI appliances.
NXP
www.nxp.com Figure 3: Edge AI appliance application flow
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44
