ANALYSIS: EVENT-BASED VISION


Event-based vision rises to the challenge of IoT


Event-based vision sensors are evolving to address the internet of things and other low-power, small-form-factor applications, says Luca Verre


E


vent-based vision has come of age as a proven paradigm shift in the acquisition and processing of visual


information. Inspired by the functioning of the human vision system, this approach uses neuromorphic techniques to achieve greater efficiency and performance to improve safety, productivity and the user experience for a range of vision- enabled systems in consumer, industrial automotive and other application areas. To date, its adoption has largely been in industrial applications and use cases where characteristics such as power efficiency, sensor size and interoperability with other vision acquisition methods (e.g. frame-based cameras) are not necessarily the primary considerations – important, but other KPIs are more critical in large- scale machine vision systems. Camera performance, dynamic range and speed of data acquisition are key drivers for event camera use cases, such as high- speed counting, predictive maintenance and inspection. However, when considering use in


other areas, notably the internet of things (IoT), the needs are different and event sensors must adapt. Energy efficiency, package size and interoperability with other on-chip functions, interfaces and data processing are needed. Here, we will discuss an evolution of an earlier generation event sensor that is optimised for a greater range of uses and demonstrate how it can be used in a wider range of applications.


Event-based vision primer Unlike traditional cameras, event sensors do not use one common acquisition rate (frame rate) for all pixels, but each individual pixel defines the timing of sampling points by reacting to changes in the amount of incident light. Information about a contrast detection is encoded


‘Event sensors must be further optimised for low-power operation at levels practical for IoT systems’


in the form of ‘events’, data packets containing the pixel’s x,y coordinate and precise time of event generation. In the case of Prophesee’s event-based Metavision sensors, each pixel intelligently activates itself depending on the contrast change (movement) it detects. This enables the acquisition of only and all essential motion information, continuously at the pixel level. The advantages of event sensors include high-speed operation (10,000 fps time resolution equivalent), extremely efficient power consumption (down to 36µW), reduced data processing requirements (10- 10,000x less data-generated than frame- based), and high dynamic range (up to


120dB), all of which make it well suited to many applications and products.


Inherent advantages for IoT Event sensors’ advantages translate well to the IoT space. Power consumption is always a major consideration in IoT and other typically battery-operated systems. In this regard, event-based vision is very well suited since it can operate well below the power levels of systems using frame-based cameras. Event-based cameras are also


particularly well suited for challenging lighting conditions typical of many IoT uses, largely because they process information independent of light. As such, they feature a high dynamic range. This makes them ideal for environments with strong variations in lighting, such as outdoor scenes with bright sunlight and deep shadows, or even at night. Because the Metavision sensor supports high dynamic range (HDR) of >120dB, it can operate in environments where lighting can pose a challenge to traditional frame-based cameras. This could include in very bright situations, for example, a smart display or kiosk in a public space, a vehicle, or in a security camera in an outdoor location; or in dim to no lighting, as in a camera you might find in a dark factory at night. In addition, event-based cameras provide extremely low latency, as they


Event-based vision offers solutions for always-on applications such as fall-detection cameras


16 IMAGING AND MACHINE VISION EUROPE DECEMBER 2023/JANUARY 2024


@imveurope | www.imveurope.com


Prophesee


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32