BACKLOG MAINTENANCE MODELLING
Risk matrix methodology has ‘fundamental flaw’
Dr Melvyn Langford C.Eng, MIMechE, MCIBSE, who for 10 years ran his own consultancy, Healthcare Operational Risk, but previously worked for nearly four decades in the NHS (including as an estates and facilities manager) discusses the key factors that have long convinced him – and, he claims, many others – of ‘a fundamental flaw’ in the ‘5x5 risk matrix’ used for some years to establish the relative urgency of backlog maintenance in NHS healthcare facilities. In this fifth article to date on the subject for HEJ, (following publication of previous articles in the November 2009, September 2010, September 2011, and June 2013 issues), he explains, using mathematical computations, why he believes the existing 5x5 model is so ‘flawed and unreliable’.
This document advisers the reader of the mathematical fundamental flaws within the following two particular NHS guidance documents – NHS Estates. A risk based methodology for establishing and managing backlog1
, and HTM 00, Policies
and principles of healthcare engineering.2 Throughout the past 27 years, numerous scientists have undertaken research and developed case histories into the problems created from the matrix design recommended by these two guidance documents. These issues have been proven beyond any reasoned argument to generate information that will at best be misleading, if not completely meaningless, resulting in erroneous risk ratings. This evidence has been accepted and supported by the academic, industrial, and international standards community, which all warn against the use of this particular risk matrix. Against this backdrop, it is indefensible, in my view, to ignore the failings within the current national policy when known, particularly when the ramifications and scale of these problems are so large. I have
5 4
Uncalibrated Calibrated
3
1 Insignificant 2 Minor
Impact 3 Moderate 4 Major
5 Catastrophic
Table 1: The national guidance numbered 5x5 matrix. Probability of Failure
1 2
2 3 4 5
thus put together and circulated this article to explain and demonstrate the scale of this issue.
Not limited to FM activity The mathematical fundamental flaw described in this document applies to all risk assessments based on the NHS numbered 5x5 risk matrix, and is not limited to estates and facility management activities; in my view all judgements based on this matrix design should be re-evaluated. The references cited have
2 4 6 8
3
3 6 9
4
4 8
5
Rare Unlikely Possible Likely Certain 1
5 10
12 15
12 16
20
10 15
20 25
been kept to a minimum, both for the sake of brevity, and with the non-academic reader in mind; they are all readily available without resort to specialist academic databases and journals.
Going back to the 1980s To address the mathematical fundamental flaw discussed within this document it is necessary to go back in time to the 1980s, where the numbered 5x5 risk matrix was designed and introduced to the US Department of Defence (DoD), which adopted its concept, and indeed it is the design recommended within the two aforementioned NHS estates-related guidance documents.1,2
Once it appeared 2 1 0 Insignificant Minor Moderate Major Figure 1: Uncalibrated vs. calibrated rating scores for impact. Catastrophic
in DoD documents it took on a life of its own, and its use quickly spread throughout the industrialised nations. It was not until the early 1990s that the fundamental flaws were uncovered by Dr Edmund H. Conrow. In essence, he proved beyond any reasonable doubt the mathematical irregularities within the matrix with the use of uncalibrated rating (ordinal) scales in risk analysis. He also highlighted the problems in the use of verbal scales for eliciting estimates of risks and probabilities, together with the extensive research relating to peculiar human errors when assessing risks.
May 2019 Health Estate Journal 39
Rating score
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76
