Measure Twice, Cut Once
members from the four Services, their Operational Test Agencies and Developmental Test Organizations, Science and Technology (S&T), Research and Development (R&D) and Joint Program and Requirements organizations, along with DHS, NIST, DOE, EPA and other federal organizations. Commodity area TecmipT sub-groups called Capability Area Process Action Teams (Capats) review existing test methodologies or develop new ones when needed. They develop community T&E capabilities requirements documents, and then review T&E capabilities validation plans and final reports to ensure compliance with the community requirements. After Capat member signed concurrences are obtained for a document, it is recommended for CBRND T&E Executive approval as a T&E Standard. There are eight Capats, with T&E subject matter experts (SMEs) in the commodity areas of bio surveillance, chemical detection, non- traditional agents, radiological/nuclear T&E, individual protection, collective protection, decontamination, and modeling and simulation. In addition to cross-fertilization of T&E Standards with other federal agencies through their direct participation in Capat activities, the Tecmipt is coordinating its T&E Standards and sharing information on DOD T&E infrastructure with international partners under various Memorandums of Understanding. The Capat chairs work directly with SMEs from these countries to develop multinational Test Operations Procedures (Tops), informed by Tecmipt T&E Standards.
SJ: There’s clearly a challenge approaching T&E for Cots as opposed to DOD developed acquisition systems? Yes, there is, but it is manageable. The CBRND T&E Executive recently identified a new strategic focus for the Tecmipt. Federal budget cuts and the increased procurement of Cots systems to meet DOD CBRND acquisition needs (as opposed to DOD-managed development of new systems) is anticipated to increase and represent a greater percentage of future CBDP acquisition. Cots
procurement already accounts for most CBRND equipment for DHS and many other federal, state and local agencies. In order to ensure reliable, reproducible, quality Cots test data, the Tecmipt Capats will design Standardized Cots T&E Programs for each commodity area, through which vendors may submit their systems for T&E. These will not be pass/fail programs, but will thoroughly characterize the performance of the systems via a graduated approach with multiple levels of compliance, providing agency decision makers with the information needed to make “best value” procurement decisions to meet their mission needs. The programs will utilize certified, accredited labs, both within and outside of DOD and will use Tecmipt/Capat developed T&E Standards. Independent panels of experts will review the test data and characterize the performance of the systems. The Standardized Cots Test Programs will also be available to provide feedback on the maturity of developing systems to vendors, as well as to R&D and S&T organizations. Inclusion of field feedback data is also being considered, and all data will be stored in centralized databases. These Standardized Cots Test Programs have great potential to reduce costly redundant testing and accelerate acquisition programs by providing earlier, higher quality data to decision makers.
SJ: What key successes have you had with designing a great T&E structure? It’s hard to pick one, but I have to say that the development of the Whole System Live Agent Test (WSLAT) fixture for the testing of biological agent point detectors definitely stands out as a success story, both from a technical point of view and because of the community agreement that was required to make it happen. WSLAT is currently being completed at Dugway Proving Ground West Desert Test Center. In the past, we have had to test bio detector components, and then put the data together to get the information on how the whole system will work. This technique does not account for what can happen to the agent as it makes its way from one component of the detector to the next. WSLAT will allow for testing of
the entire detector system in its actual configuration and with much more threat realistic representation of biological agent challenges.
SJ: Do you plan to share data and results, such as how the TSA allows a product to display a ‘passed TSA testing’ mark? Absolutely. Our goal, as we develop our T&E Standards through the TECMIPT, is to share data with our interagency and international partners. Standardization allows us all understand the test procedures used and what the resulting data means. This will allow for commodity area standardized test programs and “Underwriter Lab” type certification of systems. We are also looking towards using each other’s T&E infrastructure and labs when appropriate. The labs would, of course, have to be independently accredited to ensure data quality.
SJ: Data reliability and repeatability is essential but difficult. Are they luxuries? No, they really are necessities. Program Managers will tell you that once you get CBRND test data, you can’t always apply it. Right now, they are right. The technical complexities involved with the testing of chemical and biological agents are daunting, whether it’s for detectors, individual or collective protection or for decontamination systems. There is a wide variety of CBD systems, technologies and threats, all of which require a different test, and there are nuances which must be understood for each one. The smallest differences in the execution of a test methodology can result in disparate data from different expert testers in separate qualified labs, making it difficult, at best, to compare and interpret. Program Managers can find themselves spending more time and money than budgeted for problem-solving and repeating tests required to achieve confidence in the test data. The procurement of Commercial Off The Shelf (Cots) Systems is burdened with both government and vendor uncertainty in the other’s test data. In addition to redundant government testing due to lack of confidence in vendor data, vendors often lack confidence in government testing of their systems, which can result
CBRNe South America 2012, 13-14 March, Rio De Janeiro, Brazil. More information on
www.icbrnevents.com 58 CBRNe WORLD February 2012
www.cbrneworld.com
CBRNeWORLD
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76