Supplement: Semiconductors
Cooper Foundation: for building and deploying ML applications at the edge
Figure 1: Block diagram of Ambarella’s N1 GenAI SoC series
using natural language to control robots; and a plethora of AI helpers that can complete tasks, including generation of text, code or images.
While these systems are diverse, they rely heavily on a combination of camera and natural language understanding. Here, on-device GenAI processing delivers speed and privacy, while reducing the overall cost of ownership. Local processing is also ideal for application-specific LLMs that are normally fine-tuned on the edge for each individual use case—as opposed to the server approach of using larger, power- hungry LLMs to cover all use cases. Based on Ambarella’s powerful CV3- HD architecture, initially developed for autonomous driving applications, the N1 SoC series repurposes all this performance for running multi-modal LLMs in an extremely low power footprint. For example, it can run Llama2-13B with up to 25 output tokens per second in single- streaming mode at under 50W of power. Combined with the ease-of-integration of pre-ported models, this new solution can quickly help OEMs deploy generative AI into any power-sensitive application, from an on-premise AI box to a delivery robot.
The Fourth ‘P’ – Productivity Cooper is Ambarella’s developer platform, providing a flexible and modular
www.cieonline.co.uk
development environment tailored for use with any of Ambarella’s AI SoCs – including both the CV7 family and the N1 series. Named for its emphasis on cooperation, it is suited to all applications, including robotics, AIoT, automotive and consumer video devices.
Cooper is structured around a suite known as the Cooper Foundry, which consists of the following four software modules: Cooper Core: Linux-based OS, compiler and SDK
Cooper Vision: containing building blocks for multi-modal sensor processing and fusion
Cooper UX: for analytics and development
The suite runs on Cooper Metal—a hardware layer based around AI SoCs, developer kits and modules, which also integrates with common third-party open- source software tools.
To demonstrate these capabilities, Ambarella developed a demo system on Cooper, using their latest AI SoCs. Here, the company’s industry-leading ISP with AI features is merged with a multi-modal
vision language model that takes into account image data from a video frame. As part of a separate demo available to developers, Ambarella has built a system (using an N1 SoC) that can handle up to 16 video streams. It allows users to search contextually in natural language for any anomalies or unusual events; or even for something specific, such as a particular make or model of car.
Smaller systems with 1-4 video feeds can be built with lower power consumption, based upon the CV7 SoC family. These systems run vision language models such as ‘CLIP,’ with the whole system consuming less than 5W of power.
Summary
GenAI is a transformative technology that was once confined to fixed, mains-powered servers, due to the energy required to deliver the necessary processing power. Based on highly efficient AI SoCs that require less power-per-inference than any other device available today, Ambarella is bringing generative AI to edge endpoint devices and on-premise hardware, across a wide range of applications such as video security analysis, robotics and a multitude of industrial automation use cases. Additionally, compared to GPUs and other AI accelerators, Ambarella provides holistic SoC solutions that are up to 3x more power-efficient per generated token, while enabling immediate and cost-effective deployment in products.
Figure 2: The comprehensive Cooper development platform accelerates time-to-market
https://www.ambarella.com/ Components in Electronics May 2024 41
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64
