EVENT-BASED VISION


The event-based image (right) picks up the outline of the car as it is vibrated on a stage


said, ‘that showed our technology can use the super high time precision we have to generate spatial resolution – you can trade off time resolution to generate spatial resolution.’


Mobile magic Te new Sony sensor opens up consumer imaging markets for the technology, and the investment from Sinovation Ventures, Xiaomi, along with Inno-Chip, reinforces this. Speaking about the Sinovation investment, Verre commented: ‘Tey clearly see that our approach to AI is very original, potentially a technology platform that can serve applications from machine vision to IoT, mobile, automotive, drone, robots – all important segments worldwide.’ Sinovation was founded by Dr Kai-Fu Lee,


a pioneer in AI, and has more than $2.5bn assets under management. Te involvement of Xiaomi, Verre said,


is an investment from a strategic angle. Neuromorphic imaging has the potential to address some of the pain points found in mobile phone cameras, namely motion blur and slow motion video. Tere have been various research papers,


Verre said, on combining data from an event-based sensor with that from a frame- based sensor. Event data is not constrained by frame rate, by clock, and therefore it can give a better understanding of motion in the scene, and potentially correct for motion blur. Also, a frame-based sensor running at 10fps could be augmented with event-based data to generate slow motion video without huge amounts of data. Te event stream could be used to reconstruct a sequence of images in between each frame of a traditional rolling shutter sensor.


Connecting the dots ‘One of the key challenges for us since the beginning has been to connect dots in the ecosystem,’ Verre said. ‘We started with this sensor technology with fundamental


www.imveurope.com | @imveurope


benefits. Very quickly we realised that to make sure you convey these benefits to the end user you need to work with camera makers and system makers, SoC vendors, software partners and system integrators. We invested a lot of time and resources to bring all these partners together. We are glad to work with companies like Imago in Germany and Century Arks in Japan, as well as Lucid Vision Labs, Framos and Macnica ATD Europe. It’s important they help us to deliver a full solution to the market.’ Prophesee has released an evaluation kit for the Sony sensor; a software development kit with 95 algorithms, 67 code samples, and 11 ready-to-use applications; and a set of open source software modules to optimise machine learning training and inference for event-based applications, including optical flow and object detection. Te open source


‘Tere will not only be Sony; other companies like Samsung and CIS companies will enter the space’


tools have so far registered more than 500 unique users, Verre said, ‘which implies more engineers and inventors are taking on our technology to start evaluating it, experimenting with it and creating solutions.’ Cambridge Consultants developed an


automated system to look for contamination in cell samples using the firm’s evaluation kit, while Xperi built a driver monitoring solution using Prophesee’s evaluation kit and SDK. ‘Our effort will be to keep providing


evaluation kits, development kits, camera reference designs to camera makers to facilitate the integration work so more cameras will use an event-based sensor,’ Verre said. ‘We will keep enriching our SDK


with more fundamental algorithms, but also application examples, with models that are both commercial – part of the software is only accessible with a licence – but a lot is now available for free because we also want to have a wider community of users.’ Prophesee is also offering training to


system integrators and customers. It has put in place a field application team with almost 20 field application engineers worldwide, as well as a network of eight distributors and system integrators for industrial imaging – companies like Framos and Macnina ATD Europe. It has also reached an agreement


with SynSense to develop low power solutions for event-based vision on edge computing. SynSense, founded in 2017, provides neuromorphic computing with a line of asynchronous, event-based vision processors that have low power consumption and low latency. Te partnership will combine, in a single chip, SynSense’s vision SNN processor, Dynap-CNN, with Prophesee’s event-based Metavision sensors. Te aim is to develop a line of modules that can be manufactured at high volume. Verre said more functionality will be


integrated into the sensor, ‘working with more partners like Sony and other foundry and image sensor companies to make sure we can open up this market opportunity of event-based technology.’ Prophesee’s technology lends itself to


on-sensor processing much more so than a frame-based approach. Verre added: ‘Tere will not only be


Sony; other companies like Samsung and CIS companies will enter the space, which will open up event-based sensing to more markets. Our role as a pioneer is to keep innovating, stay ahead, make sure we connect dots at the software and system level and work with as many partners as possible, companies like NXP and Qualcomm, so everyone in the ecosystem is able to build a full application.’ O


OCTOBER/NOVEMBER 2021 IMAGING AND MACHINE VISION EUROPE 13


Prophesee


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32