EVENT-BASED VISION


Neuromorphic pioneer wins Vision Award


Greg Blackman speaks to Prophesee’s Luca Verre about the prospects for event- based imaging


V


ision Stuttgart has returned and with it the Vision Award, which this year went to Prophesee, recognising the


potential its neuromorphic – or event-based – approach to imaging has for the machine vision sector. Martin Wäny of the judging panel for


the award called event-based vision a ‘new paradigm’ in imaging technology, one that’s been worked on for 20 years, but is now coming to fruition through the efforts of Prophesee. Prophesee’s first Vision show was in 2016,


fresh off the back of winning best start-up at the investment conference, Inpho Venture Summit, the month before. Since then a lot has changed: recently


Sony announced neuromorphic sensors based on the firm’s technology, and earlier in the year Prophesee won investment from the Chinese AI venture capital firm, Sinovation – the first European company to do so – along with Xiaomi as a corporate investor, showing the technology has scope for use in mobile devices. Prophesee’s CEO Luca Verre explained


that machine vision remains an important market segment for the company. Prophesee began by targeting machine vision for the first three iterations of its sensor, in part because the form factor of those sensors was too large for consumer devices but a fit for machine vision. Now, its fourth-generation sensor, which has just been released with Sony, has an optical form factor that can fit easily into consumer devices, even mobile, Verre said. Nevertheless, machine vision is still a


key market, as the high-speed, real-time performance of neuromorphic sensing lends itself to machine vision tasks. Unlike conventional imaging, where all the information in the scene is captured for


Luca Verre (second from left) collecting the award in Stuttgart


each frame, event-based imaging records changes – or events – in the scene, similar to how the human eye records and interprets visual input. Tis gives specific advantages: the sensor can run at microsecond time resolution, or greater than 10,000 images per second time resolution equivalent. It is therefore ideal for applications like high- speed counting – counting and measuring the size of particles or objects moving at up to 500,000 pixels per second – or monitoring vibrations in manufacturing equipment at 10kHz for predictive maintenance. Because the sensor is only capturing


changes in the scene, it generates 10 times to 1,000 times less data than frame-based approaches; it offers 120dB dynamic range, imaging in light levels down to 0.08 lx, and power efficiency of 26mW at the sensor level. Te sensor developed with Sony uses


Sony’s 3D stacking technology and Cu-Cu interconnects to shrink the pixel to 4.86µm with 80 per cent fill factor; the previous generation of the sensor, Gen 3, based on a 180nm CIS process, has 15µm pixels with 25 per cent fill factor. Te two Sony sensors have a dynamic range of 86dB, and resolutions of 1,280 x 720 pixels (IMX636) and 640 x 512 pixels (IMX637). As in all imaging, resolution helps with


12 IMAGING AND MACHINE VISION EUROPE OCTOBER/NOVEMBER 2021


accuracy, for detecting small vibrations in a narrow field of view, for instance, although Verre said the objective is not to reach a multi-megapixel image. However, the bio-inspired nature of the technology has led to some clever tricks to generate high- resolution images. Te human eye is constantly moving,


making lots of micro-movements called saccades. Te eye does not have a huge physical resolution because the number of receptors is limited, but the brain reconstructs a high-resolution image using these saccades. In the same way, it’s been shown that


by putting Prophesee’s VGA sensor on a piezoelectric stage and then shaking it, a multi-megapixel image can be reconstructed from the stream of events produced by the movement. Verre said, while not a commercial


solution, a metrology system maker in Japan has been experimenting with the sensor to work on surface inspection. Te VGA sensor wasn’t able to identify scratches on the surface initially, but when some small vibrations were introduced, over time, the sensor accumulated enough information to identify these defects. ‘Tis is an interesting approach,’ Verre


@imveurope | www.imveurope.com


Landesmesse Stuttgart GmbH


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32