Optimise your Augmented Reality solution using an All Programmable SoC

AR systems are becoming commonplace across several sectors from commercial to industrial and military. These devices bring with them the challenges of high performance, system level security and power efficiency, which are often mutually exclusive. These challenges can be addressed by using a Zynq SoC or Zynq UltraScale+ MPSoC as the heart of the processing system


esearch has shown that humans interact with the world visually, being capable of processing images many times faster than information presented in different forms such as written text. Augmented Reality (AR) like its Virtual Reality (VR) cousin, enables the user to experience an increased perception of the surrounding environment. The major difference is that AR adds to, or augments, the natural world with virtual objects such as text or other visual objects. This enables the user of the AR system to interact safely and with more efficiency within their environment. This differs from VR where the user is immersed within a synthetically created environment. Combinations of Augmented Reality and Virtuality are often described as presenting a Mixed Reality (MR) to the user.

One of the first things that comes to mind when AR and its applications are mentioned is the Heads-Up Display (HUD). These are used in aviation and automotive applications to provide the user with pertinent vehicle information without having to look down at the instrument cluster. HUD’s are one of the simpler AR applications available. Further AR applications with more advanced capabilities including wearable technologies often called smart AR, are predicted to be worth $2.3 billion by 2020 according to Tractica.

Augmented Reality applications and use cases Across the industrial, military, manufacturing, medical and social commercial sectors, AR is finding many applications and use cases which are contributing to its adoption. Within the commercial sphere, applications are focused upon social media providing, for instance, the ability to recognise who you are talking with and adding biographical information. AR also provides the capability to enable the consumer to see products which are sometimes hard to visit for instance cars, boats and buildings. Many AR applications are based around the use of smart glasses which are worn by an operative. These smart glasses can allow for increased efficiencies within the manufacturing environment, for example by allowing manuals to be replaced, showing the user how piece parts are assembled. In the medical field, smart glasses provide the potential for the sharing of medical records and of wound and injury details, enabling information of treatment both by the on-scene emergency services and later in the ER to be available. Independent of the application and use case, implementing an AR system comes with several competing requirements including performance, security, power and future proofing which must all be considered if the

Example of smart glasses used in an industrial setting

designers are to arrive at the optimal solution for the AR system.

Implementing AR systems These complex AR systems require the ability to interface to and process data from multiple camera sensors which enable the system to understand the surrounding environment. These camera sensors may also operate across different elements of the Electro Magnetic (EM) spectrum such as infra-red or near infra-red. Additionally, the sensors may provide information from outside the EM spectrum providing inputs for detection of movement and rotation. Embedded Vision systems which perform sensor fusion such as this from several different sensor types are commonly known as heterogeneous sensor fusion systems. AR systems also require high frame rates along with the ability to perform real time analysis, frame by frame to extract and process the information contained within each frame. Providing the processing capability to achieve these requirements becomes a driving decision factor in component selection. All Programmable Zynq -7000 SoC or Zynq UltraScale+ MPSoC are used to implement the processing core of the AR system. These devices are themselves heterogeneous processing systems which combine ARM processors with high performance programmable logic. Zynq UltraScale+ MPSoC, the next generation of the Zynq-7000 SoC, additionally provides an ARM Mali-400 GPU and select family members also contain a hardened video encoder which supports H.265 and HVEC standard.

Anatomy of an Augmented Reality System

These devices enable the designer to segment the system architecture optimally using the processors to implement the real- time analytics and transferring to the ecosystem traditional processor tasks. The programmable logic can be used to implement the sensor interfaces and processing; this offers several benefits namely: parallel implementation of N image processing pipelines as required by the applications, and Any to Any connectivity, the ability to define and interface with any sensor, communication protocol or display standard allowing flexibility and future upgrade paths.

Most AR systems are also portable,

untethered and in many instances wearable as is the case with the smart glasses. This brings with it a unique challenge which is implementing the processing required within a power constrained environment. Both the Zynq SoC and Zynq UltraScale+ MPSoC families of devices offer the best performance per watt, and can further reduce the power during operation by implementing one of several different options. At one end of the scale the processors can be placed into standby mode to be awoken by one of several sources, to powering down the programmable logic half of the device. Both these options can be possible when the AR system detects that it is no longer being used, extending the battery life. During operation of the AR system elements of the processor which are not currently being used can be clock gated to reduce the power consumption. Within the programmable logic element by following simple design rules, for example making efficient use of hard macros, careful planning of control signals and considering intelligent clock gating for device regions not currently required we can achieve a very efficient power solution.

Several AR applications, for example medical sharing of patient records or sharing manufacturing data, call for a high level of security both in the Information Assurance (IA) and Threat Protection (TP) domains, especially as AR systems will be highly mobile and could be misplaced. Information assurance requires that we can trust the information stored within the system along with information received and transmitted by the system. As such for a comprehensive Intelligence Artificial domain we need to make use of the Zynq secure boot capabilities which enables the use of encryption, and verify using the AES decryption, HMAC and RSA verification. Once the device is correctly configured and running the developers can use ARM Trust Zone and hypervisors to implement orthogonal world, where one is secure and cannot be accessed by the other. design-tools/embedded-vision- zone.html

Components in Electronics March 2017 17

Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44  |  Page 45  |  Page 46  |  Page 47  |  Page 48  |  Page 49  |  Page 50  |  Page 51  |  Page 52