FEATURE · ANALYTICS/AI
advanced technologies, into clinician workflow—and that will take a while. He sees what Michael Hasselberg and his colleagues are doing at the University of Rochester Health as hope-inducing in that regard. The challenge, he says, is that, “Ultimately, if you go down the road of building something within the health system, now you’ve basically built a software product. Do you really have the resources to do that?” In other words, he says, the better-resourced health systems will move far faster to leverage ChatGPT and other advanced technologies into clinical workflows. Looking at the overall landscape nation-
wide, Mathaeus Dejori and Ryan Nellis see a natural evolution—with bumps along the way—ahead for patient care organization leaders. Dejori is chief data scientist and AI lead, and Nellis is vice president and gen- eral manager, at PINC AI Stanson Health, a subsidiary of the Charlotte-based Premier Inc. health alliance. PINC AI Stanson Health provides a solution that provides point-of-care clinical decision support. There’s a lot of work ahead, naturally.
“These models are very powerful, but they’re very biased,” Dejori says. “And when you start to measure how good they are—these models are trained on internet data, and aren’t necessarily aligned to your domain. So you have to make sure each large language model is tamed to your domain and works in your domain. You can use ChatGPT, but in terms of guaranteeing accuracy and safety, everyone is struggling now to figure that out. And these models are powerful, and the cost aspect is interesting, because you have powerful models with billions of parameters. But do you really need such powerful, expensive models? We do real- time decision support. And you cannot get a real-time response of hundreds of billions of parameters model. So you need for clinicians to play with these models.” In other words, Dejori emphasizes, it
will take a lot of thought and effort for the leaders inside patient care organizations to take large language models and train them on clinical and operational processes inside hospitals, medical groups, and health systems. “People will need to train these large language models on our domains,” he says. “So whether to recommend whether a patient should get a Chest CT, you need to know the patient’s smoking history, for example. And it’s highly complex and written in notes all over the place. And these models can do surprisingly well-ish
if you ask the right question, like what is the patient’s smoking history? But they’re not meeting our high-fidelity needs. It’s a matter of steering them into focus in the domains we need, and scaling them, and developing guarantees about the reliability of a model’s performance.” “We have a data science team and a
clinical team,” Nellis says. “And Mathaeus has taught me, and we’ve been building our own models for many years, and there’s this new breed now, with addi- tional computing horsepower, but we need clinical input and guidance. And ChatGPT, you can say, hey, write me a four-paragraph thank-you letter to Mark, and include these three things. And it will do it, but it’s not ready to go from here to there. And that’s where we are in health- care: these models are impressive, but you still need humans to get into high-fidelity production. And most folks are getting a little bit overhyped. One of our clients, a big Premier member, a big multi-state health system, organized all their clinical and business-line leaders to get together quarterly in person for a whole day. And they think AI is the future, that it will make HC faster, more sustainable, and of higher quality, for our patients, and they want to lead the charge. So they’re asking all the right questions. What are the high-value use cases? Mathaeus flew down to one, I did. And there has to be an active participant, using these things in a controlled, but meaningful way.” Neinstein notes that “There’s this
immensely powerful new capability with ChatGPT, and I think that having a very vibrant, broad ecosystem of trying new ways of using it, actually is of value,” he says. “People don’t necessarily know where the most value will be. I think of trying to do things in a diamond shape: do testing and experimentation, and then converge around where there’s traction and value. So I think it’s appropriate that we’re in a divergent state around this.” Neinstein says that what will be required will be several things, including strategy, intentionality, and “guardrails for safety and trust,” meaning, very conscious choosing of which areas to explore and which areas in which to attempt to build scale. “It can’t just be a thousand flowers blooming indefinitely.” In that regard, he predicts, “organizations with structure and strategy will do best; progress will be about rapid-cycle learning and innovation.” What should CIOs and CMIOs be thinking right now, how should they be
12
hcinnovationgroup.com | SEPTEMBER/OCTOBER 2023
planning? “For me,” Neinstein says, “a fundamental organizing principle around this is that technology and operations can- not exist separately here. This cannot be looked at solely as a technology problem or opportunity; there has to be tight inte- gration of technology with operations, particularly with workflow.”
What health IT leaders should be thinking about right now And in that regard, asked what health IT leaders should be doing right now, Hasselberg says that, “If you haven’t already, you need to set up AI governance within your health system. Who sits at that table, and what policies are you applying? There’s a ton of work involved in creating the governance and project prioritization processes that will lead organizations to success in this area.” And he adds that, inevitably, the leaders at many patient care organizations will wait until their electronic health record and analytics vendors develop off-the-shelf systems for use, while others will move to “work with the Microsofts, the Amazons, the Googles of the world, and will look to those big companies to provide those services to them.” But he and his colleagues at the University of Rochester have made the commitment to get out ahead of com- mercialization and develop use cases and solutions that make sense for them and will move their organization forward sooner rather than later. Nellis says, “First of all, be leaders.
And you want to achieve focus, so that this doesn’t become some side project.” Most of all, he urges, “Match a really important business opportunity and business goals, with the potential tech- nology. Pick things that matter,” such as improving workflow and the work lives of clinicians and others in the organi- zation, improving cost-effectiveness or financial performance, and so on, “and be realistic about them.” Meanwhile, for all the challenges ahead,
those who have plunged in see a world of opportunity ahead. “Six months ago, a year ago, given our own experience,” Hasselberg says, “I was very pessimistic about the speed with which AI was going to transform healthcare. Now, I’m really, really excited. This is going to transform healthcare for the good, and it really, really excites me. I think we’ll move from micro-researchers to opportunities at scale quickly, in the next six months to two years.” HI
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36
