advantages of automating vae detection 827
of automation. Here, we report the development and validation of an automated electronic surveillance system using real-time extraction of bedside physiological monitor data, with clinical and demographic data. The performance of the tool demon- strates the advantages of automation, as well as common failures of surveillance, which relies on human review of patient records. Automated surveillance provides opportunities for the imple- mentation of rapid-cycle quality improvement interventions among patients in real time.
methods Setting and Cohort Identification
This retrospective cohort study was conducted at Massachu- setts General Hospital (MGH), a 1,056-bed, tertiary-care hos- pital in Boston, Massachusetts. Patients admitted any of 4 intensive care units (ICUs) during January–March 2015 and January–March 2016 were included in the development and validation cohorts, respectively. The ICUs included a medical ICU (18 beds), a surgical ICU (20 beds), a neurosurgical/neu- rology ICU (22 beds), and a medical-surgical ICU (18 beds).
Surveillance Methods and Application of VAE Definition
certified infection control (IC) staff with a combined 30 years of experience, based in the MGH Infection Control Unit. Staff received a listing of all ventilated patients along with daily positive end-expiratory pressure (PEEP) and fraction of inspired oxygen (FiO2) data that had been entered into an Excel spreadsheet manually by respiratory therapy staff caring for ventilated patients. Respiratory therapy staff recorded theminimum values once every 12 hours, then they compared the values to those recorded during the prior shift. If the VAE criteriawere met, the datawere provided to the IC staff to review. Using these data, the IC staff applied the VAE definition to determine whether VAC criteria were met. If VAC criteria were met, the IC staff assessed IVAC criteria by reviewing the EHR forwhite blood cell count (WBC). If the case did not meet criteria based on WBC, the IC staff reviewed electronic progress notes to determine whether the IVAC temperature criterion was met. During the study period, temperature was not recorded in electronic flow sheets. The IC staff subsequently reviewed the electronic medication administration record (EMAR), which included all medications administered to patients (alongwith notations if themedications were held and if so, for what reason) to assess administration of antibiotics eligible for inclusion. If criteria for IVAC were met, the IC staff would proceed to review microbiological and pathology data in the EHR to determine whether the case met PVAP criteria. The EHR during the study period included a
Both manual surveillance and automated surveillance applied the 2017 NHSN VAE definition.8 The work flow for manual surveillance and automated surveillance with respect to the VAE definition is depicted in Figure 1, which provides the elements required to meet VAE criteria in the middle column. Manual surveillance. Manual surveillance was conducted by
combination of locally developed and commercial products inclusive of all progress notes, laboratory, radiology, pathology, operative notes, admission, and discharge documentation, for both inpatient and outpatient visits. At the end of the process, the IC staff entered the event into the CDC’sonline VAE calculator9 to confirm ascertainment, and they documented the final classification as well as the event date. IC staff were aware of the study during both the development and validation periods. Automated surveillance. The automated surveillance component of the study was accomplished using computer code developed in-house, written in Python version 3.5 software (
http://www.python.org) and Matlab version R2016b software (Natick, MA), as well as a proprietary software provided by Excel Medical (Jupiter, FL). The first step in automated surveillance was tracking patient entry and exit times from ICU rooms. This step was accomplished by continuous monitoring of the hospital’s admission–discharge–transfer data log. The second step was to determine which patients were on ventilators, which was accomplished by continuous monitoring of ICU monitor data over the hospital network, using BedMaster software (Excel Medical, Jupiter, FL). For this study, our teamhad direct access to the streaming BedMaster data. Patients for whom ventilator settings were available were identified as being on mechanical ventilation. For patients on mechanical ventilation, the algorithm monitored second-to-second ventilator settings: PEEP and FiO2. For each ventilator day, the minimum daily PEEP and FiO2 values were computed as the lowest value of PEEP and FiO2 during a calendar day thatwas maintained for at least 1 hour after any given change in PEEP or FiO2 setting, respectively. These daily minimum values were used to determine whether conditions were satisfied for a VAC event, and if so, to calculate the VAE window within which conditions for IVAC or PVAP events were subsequently checked. Having detected a VAC event and VAE window, conditions were next checked for an IVAC event. For this determination, time-stamped chemistry results and antibiotic administration records were extracted from the EHR. The WBCs within the VAE window were compared with leukopenia and leukocytosis thresholds (Figure 1). Antibiotics given were compared with the NHSN list of eligible antibiotics, and the timing of antibiotic initiation and the number of qualifying antibiotic days were determined. In events qualifying as IVACs, the algorithm further checked whether PVAP conditions were met. The PVAP conditions were checked by extracting microbiology results, including sputum specimens, lung histopathology, and urine testing for
Legionella.The algorithm further checked whether combinations of conditions were met regarding identity of microorganisms, specimen type, purulence, and amount of growth. Determination of gold standard. All “positive” detections
(detection of VAC, IVAC, or PVAP), either by the algorithm during automated surveillance or by the IC staff duringmanual surveillance, were manually reviewed by senior IC staff to determine the reference standard. Instances of discordance between the reference standard and either manual surveillance or automated surveillance were iteratively discussed and
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76 |
Page 77 |
Page 78 |
Page 79 |
Page 80 |
Page 81 |
Page 82 |
Page 83 |
Page 84 |
Page 85 |
Page 86 |
Page 87 |
Page 88 |
Page 89 |
Page 90 |
Page 91 |
Page 92 |
Page 93 |
Page 94 |
Page 95 |
Page 96 |
Page 97 |
Page 98 |
Page 99 |
Page 100 |
Page 101 |
Page 102 |
Page 103 |
Page 104 |
Page 105 |
Page 106 |
Page 107 |
Page 108 |
Page 109 |
Page 110 |
Page 111 |
Page 112 |
Page 113 |
Page 114 |
Page 115 |
Page 116 |
Page 117 |
Page 118 |
Page 119 |
Page 120 |
Page 121 |
Page 122 |
Page 123 |
Page 124 |
Page 125 |
Page 126 |
Page 127 |
Page 128 |
Page 129 |
Page 130 |
Page 131 |
Page 132 |
Page 133 |
Page 134 |
Page 135 |
Page 136 |
Page 137 |
Page 138 |
Page 139 |
Page 140 |
Page 141 |
Page 142 |
Page 143 |
Page 144
