BIO-INSPIRED IMAGING Chronocam was founded three years ago with


the initial objective of building a silicon model of the human retina and bringing the resulting sensor to market. Now the company produces such a sensor as well as its own soſtware, which performs new bio-inspired methods of computer vision and machine learning that are adapted to the way the sensor captures information. ‘Chronocam’s technology is leading to high


speed, real time machine vision,’ commented Verre. ‘Now we can effectively run at tens or hundreds of kiloframes per second equivalent, while producing very little data that we can process in real time.’ Inivation’s own Dynamic Vision Sensor, which


carries the same name as the type of sensor it represents, came out of a line of research that stretches back at least 30 years. According to Bamford, it was the first silicon retina model that overcame practical problems that were previously preventing them being viable for machine vision. While this sensor, being Inivation’s first-


generation bio-inspired offering, is purely a dynamic vision sensor, the firm’s second-generation sensor, the DAVIS (Dynamic and Active-Pixel Vision Sensor), has the ability of being able to produce a video stream alongside its event stream. ‘Fundamentally, this is a CMOS vision sensor


with a very different pixel design,’ Bamford explained. ‘In the second-generation sensors we’ve combined the standard APS global shutter circuit with the dynamic vison sensor circuit, so that you can have a global shutter exposure and, even while the exposure is happening, if there’s a fluctuation in the photocurrent, the very same pixel can produce a string of events.’ Hundreds of universities


and companies have already purchased prototypes of the new sensor and are currently working with Inivation to help develop algorithms and applications that will enable it to be commercialised. ‘Tere’s a huge amount of interest because of the unique advantages of the sensor,’ commented Bamford, who also said that a third-generation device is now being developed, one which uses CMOS image sensor processes, micro-lenses and colour filters, and can be used in an improved range and quality of applications. In addition to its sensor offerings, Inivation also


We can


effectively run at tens or hundreds of kiloframes per second equivalent while producing very little data


normal camera, and then there’s the event stream in parallel, which is completely novel,’ Bamford explained. ‘Most of our prototypes also contain an inertial measurement unit, and the data from this is also built into the event stream, which is helpful for positioning and robotics applications.’ While the highest resolution camera currently achieved by Inivation is VGA standard, the dynamic vision sensing technology has a temporal resolution of microseconds. ‘In certain applications a very high temporal resolution can compensate for low spatial resolution,’ said Bamford. ‘When tracking movements, a limited number of high temporal resolution


samples allow you to get a very accurate fix on a spatio-temporal plane.’ With their ability to optimise data acquisition


develops cameras, with outputs very different to standard imaging devices. ‘Tere’s the frame stream, which is the same as a


MANTIS SHRIMP-INSPIRED CAMERA COMBINES COLOUR AND POLARISATION


Many biomedical applications would benefit from being able to capture both the colour and polarisation properties of light, for example in cancerous tissue detection, which requires polarisation information to be superimposed onto anatomical features captured in colour camera images. Unfortunately, many of today’s colour-polarisation imagery technologies are complex, bulky, expensive and offer intolerably low polarisation performance, as well as limited translation to field experiments for the study of naturally occurring phenomena. Professor Viktor Gruev and graduate student Missael Garcia, from the University of Illinois and Washington University in St Louis, have designed a camera inspired by the eyes of a mantis shrimp. The camera can record both colour and polarisation imaging data on a single chip, while costing less than $100 to produce, according to the researchers.


The mantis shrimp has two


and processing, more conventional image sensor firms and computer vision companies are exploring bio-inspired approaches, which are becoming more accessible thanks to modern manufacturing techniques, such as 3D stacking wafers, according to Verre. Soſtware advances are also aiding the


Still frame of Odontodactylus scyllarus underwater with polarised antenna scales. Left: colour image. Right: degree of linear polarisation represented in a false-colour map, where red and blue indicate highly polarised and unpolarised light


apposition compound eyes, sensing 16 different spectral channels, four equally spaced linear polarisation orientations, and two circularly polarised states. It is one of the best-adapted predators in shallow waters, thanks to its visual system. The researchers stacked multiple photodiodes on top of each other in silicon to see colour without the use of special filters. They then combined this technology with metallic nanowires to replicate the portion of the mantis shrimp visual system that allows it to sense both colour


and polarisation. This created a high-resolution colour-polarisation camera with better than 30 per cent quantum efficiency in the visible spectrum and high polarisation extinction ratios. ‘By mimicking the space- and


energy-efficient implementation of the mantis shrimp visual system, we have designed a compact, single-chip, low-power colour- and polarisation-sensitive imager,’ Garcia commented. Testing revealed that the new sensor captures co-registered colour and polarisation information with high accuracy, sensitivity and resolution


(1,280 x 720 pixels). The technology can be used in a wide range of applications, including remote sensing, cancer imaging, label-free neural imaging and the study of new underwater phenomena and marine life behaviour.


In the future the two researchers plan to fully mimic the hyperspectral and


polarisation-sensitive capabilities of the mantis shrimp’s eye, in addition to extending the multispectral concept to be able to detect multiple narrowband spectra.


www.imveurope.com @imveurope December 2017/January 2018 • Imaging and Machine Vision Europe 15


Gruev et al


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44