Clinical engineering
You may start with an artificial intelligence algorithm that is very accurate; but if one thing changes, you can get technical drift. It can change the accuracy
of the algorithm overnight. Mark Hitchman, Canon Medical Systems UK
University Hospital and Aberdeen Royal Infirmary, a team of software engineers and architects was established – known as the Safe-Haven Artificial Intelligence Platform (SHAIP) team. Mark explained that ‘SHAIP’ anonymises patient data – taking the data out of the PACS and into a ‘safe data environment’. “When you train in AI, you have a lot of patient data that must be anonymised, organised and curated,” he commented. A range of AI projects have been underway
– from triage of normal and abnormal chest x-rays, a mammography screening programme (second reader), stroke decision support in hyper acute care, ankle and wrist x-ray fracture triage, and COVID triage and screening. From these projects, key risks have been identified. He cited the example of an AI application for breast screening. Two consultant radiologists are currently required to read each image. The project looked at the potential for AI as the ‘second reader’. Initially, the AI package proved to be very accurate. However, the mammography machine subsequently had a software upgrade during a routine service. “Nobody noticed for a few weeks, but the
accuracy went from beating the radiologists by around 8% to plummeting – it completely drifted,” Mark commented. “You may start with an AI algorithm that is very accurate; but if one thing changes, you can get technical drift. It can change the accuracy of the algorithm overnight and if you’ve been lured into relying
on it, you may send patients home, saying: ‘your mammogram is clear; come back in a couple of years’, by which time they are dead. That is the severity of the risk, if you do not keep an eye on this. I feel passionate that we have learnt this and that we should share it with you. “You can overcome this, but it is one of the risks, and it is becoming like the Wild West in AI. It is everywhere – I could fill this slide with the healthcare algorithm companies that have emerged in the last three years. Some clinicians, with the best interests in mind, are adopting these algorithms and I think these risks need to be brought to the surface more,” Mark asserted. Another myth, he pointed out, is that you can
train an AI algorithm in a safe data environment in Australia or America, and it will still be accurate when transferred to another geography. “It doesn’t travel well – if you bring the AI algorithm into Britain, where there is a different ethnic cohort, different disease processes, and other factors, it may not be accurate. If you believe that you are buying an algorithm that is very accurate in Japan or American, do not assume that it will be accurate in Birmingham, Cambridge, Coventry, or Edinburgh. We have learnt that this is something that you cannot assume,” Mark warned. Other myths include the assumption that AI in diagnostics improves efficiency. He explained that the radiologist will typically read the report on the PACS workstation, ask the AI for an opinion, then have to decide
whether or not they agree with the AI. “It creates an extra step, so do not assume that quick deployment of AI is going to be the saviour of efficiency. I think it will be, in a few years’ time, but not yet. Only when we have a system that can run autonomously, where we can trust it not to drift or be inaccurate, then we can start to improve efficiency,” Mark commented. He added that there are many variables that are a barrier to consistency with AI, at present, so there needs to be quality systems in place. For example, a system may have been calibrated with patients locally, but the AI company may bring out a new version from Japan, America or Australia. “They may upload the new version and assume it is going to be better than the last one, but this may or may not be the case. You need to prove that to yourselves before you adopt it,” he warned. He pointed out that quality systems are
paramount going forward. The healthcare sector needs to look at how it adopts AI with good practice. There is a need to ask the following questions: Is it as good as it is claimed to be in your location? Does it reach 100% accuracy? (Mark pointed out that this is unlikely). Is there bias? Is it accurate enough and how do you know how accurate it is? Does it bring value for money? Mark went on to point out that regulators
are currently overwhelmed by the volume of AI companies in the market: “It’s a complex space... It feels a bit out of control, at the moment. This led us to conclude there is a need for AI governance. It needs to be a system that measures and tracks AI performance continually and in real time in clinical practice – otherwise, we do not believe you can deploy AI safely. You are bringing a lot of risk into the equation,” Mark asserted. When buying AI, he advised that it is important
to consider: l How to select the best AI from the many available.
l What absolute accuracy can you expect on your use cases?
l Can you safely bring experimental or in- house AI into practice?
l Is each new AI version, or alternative, an improvement?
44
www.clinicalservicesjournal.com I September 2023
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76
