3D VISION


Powering stereo vision with FPGAs


Dr Konstantin Schauwecker, CEO of Nerian Vision, describes the firm’s stereo vision sensor for fast depth perception with FPGAs


F


ast and accurate three- dimensional perception is a common requirement


for many applications in robotics, industrial automation, quality assurance, logistics and many other fields. Today, active 3D camera systems, which rely on emitting light in the visible or invisible spectral range, are widely used for realising such applications. Under controlled conditions,


these systems can provide very accurate measurements. In difficult lighting situations, however, they reach their limits. Depth perception using active camera systems is only possible if the emitted light can clearly outshine the ambient light. However, this is difficult to achieve in bright environments, like bright daylight. For applications such as automated logistics and mobile service robotics, where the prevailing lighting conditions often cannot be controlled, other sensors must be used. Another problem for active sensors is measuring over distance: the greater the distance, the larger the area to be illuminated. A possible alternative is


passive stereo vision. Te environment is captured by two or more cameras with different observation positions. Intelligent image processing can then reconstruct the spatial depth and thus the three-dimensional structure


of the imaged environment. Since stereo vision doesn’t use active illumination, the brightness of the environment is of no importance, and there is no fixed upper limit for the maximum measurable distance. Furthermore, only one image per camera is required, making stereo vision particularly suitable for dynamic applications. Despite these advantages,


stereo vision is rarely used in industrial applications. One of the main reasons for this is the enormous computing power required for image processing. Take two cameras


‘SceneScan can calculate depth data for 30 million pixels per second using the FPGA’


with a resolution of 720 x 480 pixels and a frame rate of 30Hz. If the maximum difference between the pixel positions of two matching pixels from both camera images is limited to 100 pixels, more than one billion pixels per second have to be compared with each other. To make things worse, if high-


quality results are to be achieved, pure image comparison is not enough. Modern methods of stereo image processing rely on optimisation methods that try


28 IMAGING AND MACHINE VISION EUROPE DECEMBER 2019/JANUARY 2020 Depth map from the SceneScan


to find an optimal assignment of matching pixels from both camera images. Tis gives a better image, but it also increases the computing load. If one leaves the image


processing to ordinary software, then one must inevitably decide between fast processing and exact results. Tis can be remedied by offloading image processing to graphics cards. However, these have a high power consumption, which prevents them from being used in mobile systems in particular. Nerian Vision has developed


a hardware solution for stereo image processing based on an FPGA. Mapping the image processing algorithms directly into hardware allows a massive parallelisation to be achieved, which leads to a large increase in performance compared to a purely software-based solution. FPGAs are also energy-efficient,


which allows them to be used on mobile systems. Nerian’s SceneScan 3D sensor


can calculate depth data for 30 million pixels per second using the FPGA. Tis corresponds to a resolution of 2 megapixels at 15fps, 0.5 megapixels at 65fps, or 0.3 megapixels at 100fps. Power consumption remains less than 10W. Tis makes SceneScan particularly suitable for battery powered mobile systems such as logistics robots. Nerian hopes that with this


technology, passive stereo vision will become more widely used in industrial applications. It makes stereo vision a very promising sensor technology for applications that require fast and robust 3D measurements. O


Dr Konstantin Schauwecker spoke at the Embedded Vision Europe event in Stuttgart, Germany in October.


@imveurope | www.imveurope.com


Nerian Vision


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40