Viewpoints
@imveurope
www.imveurope.com
Thor Vollset, CEO of Tordivel, argues that 3D stereoscopic imaging will advance robot guidance in manufacturing
T
here is an on-going revolution with regards to automation and robotics; machine vision is a critical part in
Google’s self-driving car, Amazon’s delivery drones and many industrial robot applications. Te application space is endless and the innovations are countless – drilling robots and bricklaying robots will change the construction industry, for instance. In industrial automation I have participated in projects where stereoscopic 3D machine vision has been used to implement generic systems for depalletising, box picking from containers, 3D gear picking, and sausage picking, among others. Depalletising parts is an important 3D
robot vision application where generic systems can be made that locate the objects based on 3D CAD data or physical dimensions. One example involves an ABB robot that empties 30 sausages from a bin within a minute. Te system is based on IR LEDs and a powerful IR random pattern projector (RPP) laser. Te two stereoscopic RPP images are used to generate the 3D image. Te application combines processing in the 2D and 3D images to locate the sausages in real-time in 3D – image capture and location is less than 650ms. Stereo vision is my favourite method
for creating 3D images or extracting 3D information from a scene. Te successful stereo vision soſtware implementations are based on the most accurate 2D sub-pixel feature location and have a true 3D reference system concept. With 3D references one can re-sample or work in any object plane in the 2D images. Tis means that one can seamlessly move back and forth between the 2D and 3D space and combine 2D and 3D machine vision algorithms. It is easy to verify how good a 3D solution is, just extract the values from the 3D model of the measured part. When everything is right the accuracy of the system is typically within 1.0mm to 0.01mm depending on the 3D field of view and system resolution. In addition, the best 3D machine vision
systems will never crash or fail, i.e. they will never output a result which is wrong. Based on the possible redundancy in a 3D vision system, there should be reliable quality data available. When the quality of the data is not sufficient the system will not output void data but indicate that something is wrong to enable the master system to act in accordance.
What is the cost of a stereoscopic system? Te cost of a stereoscopic system can start from €100 to €200 implementing an OEM system with low-cost VGA cameras using an open-source soſtware library like OpenCV. A professional stereoscopic solution might range from €4,000 to €50,000. Tere is a very big difference between putting together a solution based on bare cameras or commodity 3D sensors and the industrial solutions. A professional solution will outperform the simple 3D stereo camera in terms of mechanical integrity, selection of baseline, working distance and field of view, and projector type, power and patterns. Tese features and flexibility makes a big difference in 24/7 robotic system operation where the equipment must work for 5 to 10 years without interruption. Stereo vision uses the displacement between
two images to calculate the x,y,z position of a feature. By combining multiple features it is possible to calculate the 3D position of an object or the 3D object pose (x,y,z,rx,ry,rz). Te depth resolution is proportional to the 2D feature location accuracy. Tis means that
sub-pixel resolution feature location provides depth resolution gains by a factor of 5 to 30 times. Tis is critical in serious 3D robot vision. An even more sophisticated technique is
A professional
stereoscopic solution might range from €4,000 to €50,000
to create a complete 3D image or point cloud by combing two images. Te two images must have texture or intensity variations for the algorithm to calculate the displacements reliably. Te more accurate the 3D camera calibration is, the better the depth resolution in the 3D images. Te image processing technique is well-documented and known as block matching. Te two images are rectified along the epipolar lines – this process is called image rectification. Te output of the block matcher is a disparity map that can easily be converted to a 3D image and then a height map. On a standard computer or PC the 3D image creation is fast, typically from 50ms to 2.0 seconds. I think 3D shall be a
transparent and natural step from 2D machine vision.
With a 3D calibrated camera it is possible to extract 3D information directly from a 2D image, which forms the basis for the most advanced 3D machine vision solutions. With two or more images from 3D calibrated cameras stereoscopic images can extract sparse 3D information or generate dense 3D images or point clouds. Tese techniques are fast becoming an important method for 3D robot vision applications and will become even more important in the future. It will contribute to the evolution of the robotic revolution or the robotic age.
Robot pick-and-place of sausages. The left image shows an IR image of the tray; the 3D model is generated by a stereoscopic 3D camera
18 Imaging and Machine Vision Europe • Yearbook 2015/2016
Tordivel
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48