TEST & MEASUREMENT FEATURE
Testing utopia: Simulating all the parts of a system
Jakob Engblom Product Line Manager at Wind River explores the importance of considering every real-world scenario in the simulated test environment to avoid costly design failures
E
stablished computer simulation technologies such as virtual
platforms are able to fully test a computer system, including all its processors and executing the complete code. However, the challenge is that the computer system itself is just one part of the larger system under test. There can be a multitude of other components, networks, sensors, and software, in addition to the real world that the system operates in or in which it interacts. Consider the control systems for an industrial production facility, a train or a tunnel-boring machine – it is not an isolated system, but a part of the overall whole. So, to reach what many testers consider the utopia, you need to simulate the control computer, the system it resides in, and the world around it. In this context, it is possible you need not just one simulation, but multiple instances all communicating with each other. Figure 1 illustrates the decomposition
into computer board, controlled system, and environment. The simulation of the computer running the control software, in this example using Wind River Simics, is interfaced to and integrated with the software simulators for the system and its environment. Sometimes, the system and the environment are contained in the same model (for example, the plant model in processor-in-the-loop or PIL testing), but more often than not, they are separate simulators provided by separate teams or even companies. It is also common to see each part being built from multiple separate simulators for separate subsystems. The essential element of any computer system simulation set-up is that the simulator can execute the unmodified binaries from the real target. Not all simulators can achieve this so the tester needs to carefully select their tools since many existing simulation products rely on various forms of shim layers or API simulations to make the software run. With unmodified binaries the software is compiled, linked, integrated, and run just
used. A change to the environment that a piece of software operates in is just as important as a change to the software itself. Sometimes it might seem that testing takes too much time – but it is the only effective way of safely mitigating the risks and associated costs involved. With simulation used in addition to hardware tests, more testing can be performed faster, saving time and allowing more what-if testing. It is thus critical to build the integrated
like it is on the real system – making the simulation more useful across more product life-cycle phases. As illustrated above, input and output values pass from the simulated system over to simulated input and output devices in the simulator. The values then reach the target software via device drivers, just like they will in the real system once it is built. In this way, the entire integrated software stack can be tested in the virtually real environment, making it possible to perform automatic testing and continuous integration even for systems that are deeply embedded and connected to their environment. If the environment changes you need to test everything again. A stark lesson taught by the Ariane rocket disaster in 1996 makes it all too apparent that the environment matters. The Ariane 5 rocket failed on launch due to a software failure. The software used for the guidance systems had been used in the earlier Ariane 4. However, the newer rocket had a completely different launch trajectory causing the software to crash. The development team learned the hard lesson that no matter how high quality the software may be, it has to be tested within the context where it will be
Figure 1:
Overall system to be simulated
simulation illustrated in Figure 1, and there are many ways to do that. When working with Wind River Simics as the simulation framework for the computer system, other simulators can be separate from Simics, or sometimes run inside of Simics, as shown in Figure 2. It is also possible to embed Simics into another simulator. The simulators might be spread across multiple machines, or run on the same machine. It all comes down to the situation and the requirements of the various simulation technologies. When using multiple simulators, some
Figure 2:
Alternatives to how multiple simulators can work together
solutions are peer-to-peer, even though master-slave set-ups are more common as they tend to be easier to retrofit to existing simulators. So long as your choice of simulator has the hooks and features needed, an integration can be built. There has been many Simics integrated with many different simulators over the years, using all the patterns shown in Figure 2. By far the most common solution is to have the simulators run side-by-side, communicating over network sockets or shared memory, and with one simulator being the master that asks the other simulators to run for a specified amount of time. In practice, existing simulators tend to be written to run as stand-alone programs, sometimes even requiring their own particular dedicated hosts to run. Simulation is almost a prerequisite to the successful design, development, and testing of a complex system. By combining computer simulation in the style of Wind River Simics with simulation of the physical system and environment, it is possible to reach tester utopia, where tests run often, fully automated, and with broad test coverage.
Wind River
www.windriver.com 0121 781 7240
/ ELECTRONICS ELECTRONICS | JULY/AUGUST 2015 17
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76 |
Page 77 |
Page 78 |
Page 79 |
Page 80 |
Page 81 |
Page 82 |
Page 83 |
Page 84 |
Page 85 |
Page 86 |
Page 87 |
Page 88 |
Page 89 |
Page 90 |
Page 91 |
Page 92 |
Page 93 |
Page 94 |
Page 95 |
Page 96 |
Page 97 |
Page 98 |
Page 99 |
Page 100 |
Page 101 |
Page 102 |
Page 103 |
Page 104 |
Page 105 |
Page 106 |
Page 107 |
Page 108 |
Page 109 |
Page 110 |
Page 111 |
Page 112