Digital & Communication Technology
Smart video stabilization takes the shake out of modern communications
By Johan Svensson, chief technology officer of Imint Image Intelligence T
hank goodness for the trusty webcam. For knowledge workers especially, video communication has been a mainstay of the pandemic years. To this day, “hybrid” work is the norm. For safety’s sake, not everyone in an organization can be together in-person; some still must be patched in via video.
But hybrid work means something different in hands-on industries like construction or manufacturing, which are less digital and where many workers don’t have a laptop and webcam for collaborating. Still, there’s often a need for video communication on-site or on the shop floor, whether to support social distancing or to connect experts who might work remotely even in non-pandemic times.
Last year, ABI Research described the growth of augmented reality (AR) usage in industrial markets—manufacturing; transportation and automotive; architecture, engineering, and construction; energy and utilities; and logistics. Using AR- and video- enabled mobile devices, as well as a growing selection of “smart” glasses and headsets, industrial companies are empowering more workers to collaborate visually. For some it’s about enabling hands-free video calls from a work site. For others, it’s about visually documenting inspections or patching in a remote expert to consult on a situation both parties can see.
But none of this works well without video stabilization. A construction worker or utility inspector with a video-enabled headset is no tripod. They move, shake, cough—all of which make the video they’re capturing impossible to view accurately. For remote experts who need to “see” what field workers are seeing, jumpy video leads to nausea.
Enter software-based video stabilization
Fortunately, there’s a new generation of software-based video stabilization that can support new mobile video uses cases. In fact, we’re reaching a point
34 March 2022
where stabilization is so good, it’s almost unnatural. At this point, we can reintroduce motion as appropriate and as it supports the unique use case. Imagine a surgeon with a video headset streaming an operation. We want to stabilize the effects of her head shaking, for example, but not the heartbeat of the patient.
Software video stabilization started as image-based, using optical flow methods to compare video frames, detect the motion of pixels from one to the other, and compensate for that flow to produce more stable video. The results were good but required more compute power than early mobile devices could afford.
As smartphones began including gyroscopes, stabilization evolved to be gyro- based, rather than image-based. Stabilization algorithms didn’t need to read actual images from a camera sensor to understand how motion was affecting the video. The tricky part was correlating the motion detected in gyro data to the actual motion of the camera sensor, which required a theoretical model of the camera system to ascertain real-world conditions. The stabilization software could
Components in Electronics
then send information to the camera’s rendering device on how to adjust and crop frames to stabilize the video. Gyro-based stabilization is very good and power-efficient, making it ideal for use in mobile devices like smartphones and AR headsets.
Not ‘more’ stable, ‘better’ stable For the most part, this is where we are today. But if we combine the two stabilization paradigms—image- and gyro-based— we can achieve even better, sub-pixel stabilization that future applications may require and benefit from. As camera resolutions increase, and, for instance, as industrial visualization applications exploit those higher resolutions, video stabilization will have to rise to the challenge. There are now software-based video stabilization solutions that overlay image-based stabilization on gyro-based stabilization, but they are compute-intensive. Flagship mobile devices that offer both are often designed to fall back to gyro-based at very high resolutions, like 8K. But the time is coming quickly when a smartphone or headset, with faster, dedicated processors,
can perform the most advanced video stabilization available.
That becomes the canvas on which we paint new, advanced video applications. As more and better video cameras enter the workforce for new and different uses, video stabilization will be critical—to a point. But the future of mobile video isn’t more stable; it’s better stable. And the way to get there is to first achieve highly stable video through software—video so steady it may appear unnatural—and then allow or reintroduce motion using artificial intelligence. As video and AR devices permeate industries and require advanced stabilization, there will be a need to discern unwanted motion from wanted motion. Call it smart video stabilization or intent-based video stabilization. Not all camera bumps, shakes, and sweeps are bad, and this new-generation technology senses the difference. The time is coming when video communication will permeate our lives in new and unforeseen ways. It shouldn’t make us nauseous.
https://weareimint.com/
www.cieonline.co.uk
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54
