THE IoT
requirements that support many industrial domains. The ETSI document proposes and represents a generic minimum level of security for IoT devices, e.g. authenticator feedback, password lifetime, session lock and account changeability. The work of the ETSI MTS Committee testing
Figure 2: Sample test purpose for MQTT using TDL-TO
languages in MTS are the Test Description Language (TDL) and the Testing and Test Control Notation (TTCN-3). Both have been defined by TC MTS and are in use in many testing projects worldwide. Figure 1 illustrates the workflows proceeding from the system requirements to the executable tests. TDL definitions are the result of the test design phase and input to the test implementation and test execution level. The new ETSI standards for IoT have been
produced using the Test Description Language (TDL-TO) specified in part 4 of ETSI ES 203 119. TDL provides simplified and generic means to achieve the specification of test objectives using informal text. The TDL-TO extension introduces additional concepts to support the specification of (semi-) structured test objectives in a more formalised manner as well as a concrete syntax notation for representing the related concepts (formalised conditions, expectations, etc.). The implemented TDL-TO library modules of the new IoT test specifications corresponding to the test purpose catalogue are provided in an archive, which accompanies the standards. The TDL-TO developments have been achieved using the TDL Open Source Project (TOP), which is a free tool supported by ETSI members for the purpose of writing and validating TDL code. A sample test purpose definition for handling the MQTT message
CONNECT including an unexpected header flag value is provided in Figure 2. The new test specification catalogues
include, in particular, two sets of test catalogues: the ETSI TS 103 596 series provides an overall test suite structure and catalogue of test purposes for the Constrained Application Protocol (CoAP); the ETSI TS 103 597 series provides an overall test suite structure and catalogue of test purposes for the MQ Telemetry Transport (MQTT). Both serve as a reference for both client-side and server-side test campaigns. Part 1 specifies the conformance issues, part 2 the security issues and part 3 the performance issues. The part 1 document mostly contains a long list of formalised test purposes of conformance tests Parts 2 and 3 also provide an overview on how to retrieve and define security and performance tests. Relevant test techniques have been presented in relation to the target IoT protocols. Therefore, parts 2 and 3 also serve as a kind of guide for the developer of security and performance tests. However, they also include concrete test purposes, e.g. derived for the CVS database. In addition to CoAP and MQTT tests the ETSI
TS 103 646 specifies testing for selected industrial security requirements as known from the IEC 6244-4-2 standard. This IEC standard has been identified on account of the generic nature of its well-known security
group has also been considered by active Open Source projects currently hosted by the Eclipse foundation: IoT-Testware. It includes Test purposes in TDL-TO in addition to TTCN-3 test code developments, which are important for the execution of test campaigns in test labs. Thanks to this collaboration, organisations are now able to execute MQTT and CoAP test programs based on standardised test purpose catalogues released by ETSI. Figure 3 illustrates the workflow, initiated by ETSI, with the development of the standardised Test Suite Structure and Test Purposes. These documents have been used by the Eclipse project IoT-Testware to implement abstract and executable test suites ready to run against CoAP and MQTT implementations. Using the Open Source IoT-Testware,
developers are now able to test their implementations based on the ETSI test purpose catalogues. The test tool implementation of the IoT-Testware includes a set of free test tools and a user-friendly dashboard control panel and is illustrated in Figure 4. Based on these tools, different free open-source CoAP and MQTT implementations have been tested and a number of defects have been identified. Further details on the test programs and results have been presented in the scope of the IoT-T project at various conferences. In summary, it should be noted that the
formal testing methodology defined by ETSI can be understood as the reliable approach for IoT quality assurance and certification as it has been learned from fixed and mobile communications. The working approach constitutes a foundation for automated testing, reduction of human error and excellent maintenance procedures that are needed for the long operation phases of IoT systems and devices.
Figure 3: The use of ETSI test purpose catalogue for Eclipse projects
Figure 4: The Eclipse open-source test tools of the IoT-Testware NOVEMBER 2021 | ELECTRONICS TODAY 19
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46
