Jiajie Zhang, PhD, SHARPC prin-
cipal investigator and dean of the UT School of Biomedical Informatics, says that at one point during its lifes- pan, SHARPC involved approximately 100 people working at 12 different in- stitutions. The researchers made sure physicians were an integral part of the process. “They were the main investigators
“We need to create better bridges between the physician community, the vendor community, and the IT community in general so we can actually have better participation in this process.”
or supporting investigators or consul- tants in each of the projects that we had,” Dr. Walji said. “So they were fully part of the team. Many of them participated in giving feedback as well as [being] participants in the research itself.” Kevin Hwang, MD, an associate
professor of medicine at UT Medical School at Houston, says he took part in some of the project testing involv- ing entry of prescription informa- tion into an EHR, such as medication reconciliation or change of dose. Re- searchers asked him to talk and think out loud as he reacted to performing different EHR processes. “Then they tracked my movements,
the mouse movements, and my eye movements with a camera,” he said.
“One of the things they were trying to look at was if there were wasted movements or if I was having to go back and forth between different screens or things like that.”
INVESTIGATING USABILITY Through SHARPC, researchers dis- covered where potential problems lie for physicians struggling with com- mercial EHRs. For example, the researchers used
an analytical process known as rapid usability assessment (RUA) to inspect and evaluate EHR systems and to identify challenges to each EHR’s us- ability. The Better EHR book details the results from evaluating five com- mercial EHR systems. (See “Measur- ing Usability,” opposite page.) RUA involved a three-step pro-
cess. First, researchers selected 12 EHR clinical tasks to evaluate, such as clinical summary, demographics,
40 TEXAS MEDICINE September 2015
vital signs, and e-prescribing. The researchers then used a task analy- sis method known as keystroke level model (KLM) to predict the per- formance of each EHR system us- ing completion times for each of the tasks, also known as “use cases.” As Better EHR explains, “KLM predicts the time it takes for an expert … to execute keyboard and mouse inputs along with the associated cognitive overheads (e.g., thinking time or time taken to visually acquire objects on the screen).” “If everything can be tracked, now
we can evaluate the actual perfor- mance of a user [versus] the theoreti- cal, optimal performance,” Dr. Zhang said. “Now we can say, ‘Wow, you’re actually far away from what it could potentially do. So this task can be done in 30 seconds, and you’re spending five minutes — something’s not right.’” The last step in RUA was an expert
review process, which involved evalu- ating each system using seven design principles, known as heuristics. Ex- amples of those principles include:
• Consistency, which measures whether the EHR product con- sistently presents information the same way and requires consistent navigation methods;
• Feedback and error, which mea- sures how well the product pro- vides the user with feedback about the actions the user performs and how well the system prevents er- rors from occurring; and
• Undo, which evaluates whether the system can correct errors.
Researchers ranked each problem
they encountered using a four-point scale. They assigned a ranking of one for a cosmetic issue, two for a minor usability violation, three for a major violation, and four for a catastrophic violation. (See “Usability Severity Rankings,” page 42.) The Better EHR book notes that re-
searchers could perform just six of the clinical tasks in all five EHR systems
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68