249: UC Davis Health's CHIO Dr. Scott MacDonald: Diagnosing EHR Frustration Like a Patient Exam, How AI Scribes Could Break Clinical Decision Support, and the Hidden Risk of AI's Errors of Omission

Subscribe on: RSS | SPOTIFY | APPLE PODCAST | GOOGLE | BREAKER | ANCHOR

On this episode of The Digital Patient, Dr. Joshua Liu, Co-founder & CEO of SeamlessMD, and colleague, Alan Sardana, chat with Scott MacDonald, MD, Chief Health Information Officer at UC Davis Health, about "Diagnosing EHR Frustration Like a Patient Exam, How AI Scribes Could Break Clinical Decision Support, the Hidden Risk of AI's Errors of Omission, and more..."

Audio:

Guest(s):

• Scott MacDonald, MD, Chief Health Information Officer at UC Davis Health
• Joshua Liu, MD (@joshuapliu), Co-founder & CEO at SeamlessMD

Episode 249 - Show Notes:

[00:06:30] How Scott diagnoses EHR frustration the way he'd assess a patient — he takes a good history and does a good exam, because health systems that take clinicians at their word and deliver the solution they ask for often solve the wrong problem. He lets frustrated clinicians vent so they feel heard, then uses the five whys framework. Sometimes the answer is a feature they don't know about, which his training team can show them. Other times the root issue is frustration with a medical assistant or clinic manager, or a misunderstood policy, and "the EHR is just the conduit."

[00:08:46] Why clinician frustration can expose outdated EHR design — organizations often use the EHR to meet payer or regulatory requirements, and sometimes that's been done badly. Frustration may reveal design that's no longer appropriate as the regulatory and technological landscape has changed. Problems that couldn't be solved five years ago may be solvable now, and Scott expects agentic AI to do much more over the next five years, "assuming we can do agentic processes safely."

[00:10:10] Why EHR training still matters in the age of AI — the AI capabilities Scott's EHR vendor recently presented aren't here yet and may not be robust enough to deploy widely and safely for a couple of years, so training will be needed for current functionality and for whatever workflows change. He doesn't expect everything to move to verbal workflows: "We are primates," with big visual cortexes, and glancing at a grid of results to see a hemoglobin A1C trend is far faster than waiting for an AI to describe it.

[00:12:22] How UC Davis governs predictive AI for bias — Scott frames predictive and generative AI very differently. Predictive models like sepsis risk or hospital admission risk are quantitative, so UC Davis can validate that they're calibrated across racial, ethnic, and demographic groups. When bias is found, the team looks at the entire downstream workflow (how the score prioritizes work, directs resources, and determines which patients get social determinants of health outreach) and mitigates it there, since models are "all trained on flawed human data."

[00:14:15] Why generative AI is harder to govern — without quantitative indicators that something is going wrong, governance is "squishier" and more important. The biases are subtle, and the models are "very confidently incorrect." Scott prefers the term "confabulation" over "hallucination," and says errors of omission are especially hard: unless you know the patient well and have done a thorough chart review, you won't recognize that a summary left out important data.

[00:15:46] How UC Davis is working on LLM-as-a-judge evaluation — the goal is using LLMs to evaluate other LLMs at scale for accurate and fair output. The team is "stuck in the mud" on validating the judge itself, which still requires a human with expertise focusing tightly on errors of omission and bias. Once a model is known to be safe, the next question is its impact on the people using it.

[00:16:45] What AI means for trainees and the physician's job — an experienced attending is very different from a PGY-1, 2, or 3 who is still forming mental models, and Scott cites concerns in the literature about never-skilling, mis-skilling, and de-skilling. UC Davis is working with graduate medical education colleagues on how to responsibly give trainees access, since graduates who can't use AI tools won't be prepared to practice. A big part of his job over the next five years will be deciding which tasks physicians should do themselves, which they should do with AI, and which they should leave to AI.

[00:21:50] Why clinicians must stay critical of AI output — Scott has told Claude to flag when it isn't confident and to ask clarifying questions, which has helped to an extent. But in areas where he has expertise, he still has to push it to consider things he knows exist in the medical literature, and it "sycophantically" agrees. He says everyone using these tools needs to be very critical: "It can be helpful, but it's gonna make mistakes."

[00:24:39] What decades of clinical decision support can teach AI implementers — the risks are being wowed by the bright, shiny object and believing AI maximalists, when humans are stubborn and many physicians have been trained to value autonomy. Whether a tool is AI-enabled or not, start with the problem you're trying to solve, engage stakeholders and leaders early, and win over naysayers so they become advocates: "If Dr. Smith likes it, then it's probably pretty good because... she hates everything." The five rights of decision support still apply.

[00:26:09] How agentic AI could earn back trust in alerts — many sensitive, specific CDS interventions haven't been possible because the chart lacks structured data, but agents may reliably extract data from notes to feed alert logic. Alerts that are often wrong train clinicians to override them, which trains them to blow past accurate alerts too. Alerts that are right most of the time could rebuild trust and engagement, supporting resource stewardship, standardization, and guideline-directed treatment.

[00:28:21] Why AI scribes could undermine intelligent visits — AI scribes have gotten Scott out from behind the computer; sometimes he doesn't log in to the EHR at all. "I can be a doctor again," but that removes the moments, like entering an order, when informaticists could intervene. Intelligent visits will need to deliver enough value to bring physicians back to the screen, such as a large monitor in the room to show X-rays and data, or preview patient instructions at a different reading level in real time. An AI-assisted agenda could combine the physician's, patient's, and AI-surfaced quality or revenue cycle concerns, with priorities leaning toward the patient's goals.

[00:31:39] How health systems should approach ambient AI note errors — UC Davis measured errors through a post-visit survey, and Scott says the numbers have wide error bars but are "certainly non-zero." The key point is that clinicians have to edit the notes; editing is far easier than writing, and the scribe surfaces things he forgot were discussed. Organizations should have clinicians with time evaluate new products critically, then monitor editing rates once live. A vendor's "100% effort reduction" metric, he notes, means the note wasn't edited at all, which is "probably not the goal."

[00:34:57] Why reviewing an AI-generated note may still have value — Scott still sees errors, so notes can't go unreviewed yet, and he's unsure whether LLMs will keep improving at the same rate. Writing a note has traditionally been a chance to think through the differential and synthesize data. Scott does much of that aloud with the patient during the visit, but reviewing the note can still prompt new ideas: "Oh, I didn't think about... it might be a vasculitis... Let me add that lab on."

[00:37:37] How UC Davis decides when residents get AI scribes — the health system relies on faculty judgment of each trainee's readiness, starting with PGY-2s and above, with some appetite for stellar interns. Use is limited to a resident's home specialty, so a pediatric resident rotating through pediatric surgery likely wouldn't use it there. Some PGY-3s still won't write strong notes, and they need to before getting tools that would "probably stunt their development."

[00:39:20] How predictive models help reach patients who need the most support — assessing social determinants like diet, food access, safety, and loneliness drives a large share of health outcomes, but the time to do it isn't reimbursed. UC Davis uses predictive models to identify patients at risk of ER visits and admissions, including "MVPs" (multi-visit patients), and nurses and social workers reach out to address what could be handled in clinic. With ERs and hospitals "packed to the rafters," keeping people healthy at home is "a win-win-win."

[00:41:52] Why not every patient outreach task needs AI — UC Davis is building an enterprise strategy with its population health team, but more frequent surveys, questionnaires, and vendor-supplied automated reminders may gather much of the data. "If the hammer will do just fine, we should just use a hammer. It's cheaper, and it uses fewer tokens, burns less electricity and less water." Scott also notes California bills that recently passed the legislature could, if signed, prohibit AI from doing tasks that require a licensed person, such as nurse assessments.

[00:44:31] What patient attitudes toward ambient AI reveal — an early survey, done before patients had experience with the tools, found skepticism skewed young, likely because younger people have seen how inaccurate AI can be, while older patients may trust the health system more. In deployment, explaining at consent that the tool is purpose-built, governed by data retention policies, and vetted by IT security allays most fears, and the vast majority of patients agree to its use.

[00:46:22] Why time may be the wrong measure for AI scribes — vendor time measurements track how long a note is open with mouse and scrolling activity, which doesn't separate writing from thinking about the patient and refining the differential and treatment plan. Scott now spends time he used to spend generating text on value-added thinking, which he calls "a real winner of AI scribes": "I think I'm a better doctor for it." Measuring that, he says, may take a stopwatch, eye tracking, or "an EEG."

Fast 5 Lightning Round:

1. What is your favorite book, or the book you've gifted the most?

Assembling California by John McPhee

2. If you could instantly master any skill, what would it be?

"Python"

3. Would you rather have super strength, super speed, or the ability to read people's minds?

"Speed"

4. What is something in healthcare you believe that others might find insane?

"A clinical note has six audiences, and that's why notes are so bloated"

5. What is the last movie or TV show you saw, and what did you think of it?

"Ted Lasso. I like the characters, but this season seems a little more preachy."

The Digital Patient has been recognized as Feedspot's #1 Patient Engagement Podcast of 2025. Thank you to our listeners for making this happen!