246: Stanford Children's CMIO Dr. Keith Morse: The Real Reason AI Projects Succeed Has Nothing to Do With AI, Why One Use Case Undersells Every LLM, and Why Building Your Own AI Is a Closing Window

Subscribe on: RSS | SPOTIFY | APPLE PODCAST | GOOGLE | BREAKER | ANCHOR

On this episode of The Digital Patient, Dr. Joshua Liu, Co-founder & CEO of SeamlessMD, and colleague, Alan Sardana, chat with Keith Morse, MD, MBA, CMIO at Stanford Children's Health, about "The Real Reason AI Projects Succeed Has Nothing to Do With AI, Why One Use Case Undersells Every LLM, Why Building Your Own AI Is a Closing Window, and more..."

Audio:

Guest(s):

• Keith Morse, MD, MBA, CMIO at Stanford Children's Health
• Joshua Liu, MD (@joshuapliu), Co-founder & CEO at SeamlessMD

Episode 246 - Show Notes:

[00:05:30] What Stanford Children's OpenAI for Healthcare partnership looks like today — announced in January, Stanford Children's is one of eight institutions and one of two pediatric institutions in the group. Because the paperwork took longer than anyone wished, use is still early and almost exclusively operational or administrative. That's partly structural, since physicians have access to other AI tools, and partly a safety choice to intentionally keep the tools away from frontline clinical decisions around diagnosis, prognosis, and treatment while the technology is still so new.

[00:07:25] Why the real impact of enterprise AI depends on integrations and user familiarity — the most common question is whether the tool can touch Epic, and right now the answer is no. It integrates with Google and Microsoft, but not with Epic, Workday, or ServiceNow. Keith expects the theoretical impact to arrive when those integrations land and when users develop expertise, noting that user comfort and familiarity are a big challenge for evaluating impact.

[00:08:56] How Stanford Children's approaches shadow AI — rather than rushing to full deployment, the organization already has a "pop-off valve": an internal HIPAA-compliant chatbot built on Microsoft infrastructure with OpenAI on the back end. Keith knows people use these tools on their own phones, so the goal is making the desirable action about as easy. The team hasn't blocked Claude, Gemini, and others because it feels like an unnecessary game of Whac-A-Mole: "We're never gonna get them all and why kid ourselves?"

[00:10:34] Why build versus buy for AI should look like the EHR decision — Keith draws on informatics' experience rolling out enterprise EHRs. Swap "AI" for "EHR," and the answer is obvious: "We are not a software company." Health systems need enterprise-grade tools, support teams, and security experts, and he wants billion-dollar companies building what Stanford Children's uses. He believes the window for locally developed AI in clinical practice is rapidly closing or already closed.

[00:12:36] How locally built AI still fits at Stanford — homegrown tools that are implemented are usually IRB-funded or IRB-governed research, a separate beast from the enterprise systems that run the hospital, or products on the path to commercialization with venture funding. For most DIY AI pitches, the answer is "like 95% sorry this isn't gonna fit." Keith doesn't want "the responsibility of maintaining a piece of bubblegum and duct tape code forever."

[00:16:39] Why AI vendor demos often solve only a trivially small piece of the problem — by the time people seek an AI solution, they've usually tried rejigging workflows, existing EHR functionality, and education, and that didn't work. AI becomes a seductive plan B, but vendors don't tell you their product is a trivially small piece of the solution. Teams get lulled into a sense of progress while the original workflow and education problems remain.

[00:18:14] How AI acts as a positive catalyst for change management — people show up to meetings and do things differently when there's an AI component, in a way they wouldn't for an Epic optimization. Often, when things actually improve, it's not the AI but the fact that it galvanized people to care about the problem; the real difference was "bread and butter change management." Keith compares this to serious safety events, which act as a negative catalyst. In sepsis reduction, for example, the benefit often comes from the protocol attached to the trigger tool, and a non-AI trigger would likely produce similar results.

[00:23:26] Why LLMs break the one-model, one-use-case approach — before LLMs, every model was built for a single endpoint, like OR scheduling or sepsis prediction, so choosing the right use case determined the project's success. Large language models are one-to-many, applicable to hundreds or thousands of use cases, so focusing on only one likely undershoots their impact. A research project using an LLM to support ADHD medication prescribing is worthwhile, but Keith says it should be considered at the level of system-wide medication clinical decision support. The approach now is to make tools broadly available and see which use cases "actually have legs."

[00:26:51] The governance gap between research and hospital operations — LLM research projects go through the IRB, which has learned to structure its review of these tools, and run on research infrastructure with consenting clinicians. Hospital operations don't get IRB approval for their work, and the IRB isn't concerned with the same issues. "It's almost like we are both worried, but we are on our own to manage those concerns."

[00:28:49] Why pre-LLM AI governance doesn't translate to large language models — governance teams used to ask meaty questions, like the racial breakdown of training data, and could ask vendors to retrain an algorithm for a more representative population. None of that is possible with LLMs, and applying pre-LLM framing "feels wildly disconnected." Keith says health systems need to be better at articulating the reasonable limits of their ability to intervene, because tech companies are happy for health systems to take responsibility for their tools. Governance should reflect that the emphasis lies mostly on the end user and the model developers.

[00:31:30] Why hands-on experience is the most important AI priority for the next two years — Keith says "there's no substitute for experience." Training videos and multiple-choice tests can't replace using a tool in a relevant application and seeing where it screws up. Asked how to ensure user trust, his answer is that "we don't want trust": tools are helpful to the extent their output can be confirmed. When the ambient scribe described an eight-year-old as a 44-year-old female, the team's answer was, "Nope. That's just kinda what happens," and egregious hiccups like that are really educational.

[00:34:24] Why health systems should bring AI to the data — healthcare data is fragile, expensive, and complicated to move from its original source. Requiring vendors to work where the data lives also serves as a vendor filtration mechanism, since startups find it easier when customers just send their data and skip "pesky Epic" and ServiceNow. Tools that need a chunk of data sent to them, like NICU growth charts or ICU ventilation data, are almost by definition single-use-case tools. "We are not gonna be sending anybody, like, all of our data."

[00:36:31] What adult-centered digital health leaders misunderstand about pediatric privacy — adolescence spans eight to 10 years, and each kid makes the switch to independence at a different time. Systems need to let patients and guardians choose when big chunks of the process transition to the child. Done poorly, a 16-year-old is treated like a six-year-old, leaving young adults, especially those with chronic health issues, with little experience managing their own healthcare just as they're going to college, moving out, and getting a job.

[00:39:00] How pediatric informatics improvements spread through Epic — convincing non-pediatric hospitals is hard when the issue is complicated, expensive, and affects a minority of patients "who generate very few RVUs." Keith says one of the best avenues is Epic, through communities of expertise led by Natalie Pageler and others that identify issues, pilot and publish solutions, and see them adopted into Epic Foundation build. One example separates parent and child phone numbers, allowing a child number only once the patient reaches 12, so teens' confidential results, like STD results, aren't delivered to a parent. The change applies going forward through the registration process.

[00:42:33] Why pediatrics can't wait for perfect pediatric AI data — "Yes, we cannot wait." Even if a pediatric version is 10% less valuable than the adult version, there's still immense value. Because 90% of the process is the same for peds and adults, Keith says the priority is identifying the tools useful across healthcare before fine-tuning for pediatrics: "The idea that we need a pediatric specific LLM feels silly."

[00:44:38] Why frontline providers should decide when an AI tool is ready for them — Keith says it's each provider's job, not his, to decide when a tool is ready; the organization provides the tool, training, and contacts. Health systems used to worry about a "one and done" phenomenon, as with early buggy versions of Dragon dictation, which big chunks of the organization never retried even though it's now "exquisite." He doesn't expect that with LLMs because a constant swirl of new features, capabilities, and success narratives will keep people coming back.

Fast 5 Lightning Round:

1. What is your favorite book, or the book you've gifted the most?

Having and Being Had by Eula Biss

2. If you could instantly master any skill, what would it be?

"The Chinese language"

3. Would you rather have super strength, super speed, or the ability to read people's minds?

"Super speed, because time is valuable"

4. What is something in healthcare you believe that others might find insane?

"When you swallow something, it's still outside your body until you absorb it"

5. What is the last movie or TV show you saw, and what did you think of it?

"HBO's DTF St. Louis. I actually really liked it."

The Digital Patient has been recognized as Feedspot's #1 Patient Engagement Podcast of 2025. Thank you to our listeners for making this happen!