245: UCSD's CMIO Dr. Marlene Millen: Is 20% Use of AI Draft Portal Responses Enough, Why Clinician Stories Beat ROI Spreadsheets for AI Funding, and How to Stop Shadow AI Without Punishing Anyone

Subscribe on: RSS | SPOTIFY | APPLE PODCAST | GOOGLE | BREAKER | ANCHOR

On this episode of The Digital Patient, Dr. Joshua Liu, Co-founder & CEO of SeamlessMD, and colleague, Alan Sardana, chat with Marlene Millen, MD, Chief Medical Information Officer at UC San Diego Health, about whether 20% Use of AI Draft Portal Responses is Enough, Why Clinician Stories Beat ROI Spreadsheets for AI Funding, How to Stop Shadow AI Without Punishing Anyone – and much more.

Audio:

Guest(s):

• Marlene Millen, MD, Chief Medical Information Officer at UC San Diego Health
• Joshua Liu, MD (@joshuapliu), Co-founder & CEO at SeamlessMD

Episode 245 - Show Notes:

[00:05:03] How to make the financial case for AI — Marlene says requesting funding for AI is not that different from any other request. She thinks of AI as a tool and steps back to ask what it is trying to accomplish, whether that's physician wellness with an AI scribe or revenue cycle coding. Data is the most useful thing to bring: as part of the University of California system, she asks colleagues what they're using and what happened, so she can say, "Someone else used this. This is how it worked." For brand-new tools without that evidence, she defines KPIs up front. "The worst they'll say is no."

[00:07:06] Why clinician stories can sell AI better than metrics — for soft-ROI tools, Marlene says framing depends on the audience. For the AI scribe, she didn't make the pitch herself; she sent pilot users directly to the CEO of the medical group and the health system CEO. Her own endorsement didn't resonate as much because she is tech savvy, but a non-tech-savvy clinician saying, "I'm getting home right away and my notes are closed," had a huge impact.

[00:07:58] Why you should never let a bad survey go to waste — Marlene treats negative results, like burnout surveys from a couple of years ago, as an opening to propose trying something new for a set period of time. UCSD runs system-wide surveys about every two years plus department-level wellness surveys, and she partners with the physician wellness group led by Dr. Sonia Ramamoorthy, using their support and feedback to strengthen the case for tools with a soft impact on physicians and APPs.

[00:10:27] How to keep the focus on the goal, not the tool — Marlene says AI isn't at the forefront of all of UCSD's work. Diabetic retinopathy screening took years to get off the ground and moved forward through the managed care department, which wanted to improve its diabetic screening metric, illustrating the value of asking who benefits and who should pay. She notes people now jump to the AI solution first, and sometimes her team has to say, "Actually, we already have something that works pretty well for this."

[00:12:27] How a chief health AI officer helps build AI literacy — Marlene credits Dr. Karandeep Singh, UCSD's chief health AI officer, with training staff and launching a "summer of AI" newsletter showing how existing tools can help people work more efficiently. She nudges colleagues toward him and loves success stories, like a non-tech-savvy faculty member who built her own AI agent to update her bio-bibliography for academic advancement instead of spending an hour formatting, all "without spending any more money."

[00:17:33] What the AI scribe rollout revealed about consent and planning — project managers now map out the whole process for new initiatives, and the team pokes holes in it. Because California is a two-party consent state, recording requires permission, so legal and compliance teams decided on an annual written consent plus a verbal consent each visit. That meant building an e-consent and a visible column, delaying psychiatry's go-live, and re-consenting patients when something changed. To build that flexibility, Marlene runs a "project management lite" course that cross-trains physician informaticists to think like project managers.

[00:22:00] Why UCSD kept AI-drafted message replies despite no time savings — Epic's AI-drafted In Basket replies were UCSD's first AI tool, piloted alongside Stanford and the University of Wisconsin. With no established evaluation process, they measured time because it was easy, though Marlene didn't expect it to improve. "If I had just done it on time, we would've probably turned it off." Surveys told a different story: clinicians said it wasn't perfect but helped, and asked, "Please don't turn it off." The tool expanded rapidly across ambulatory care, which she says shows the value of being flexible on KPIs.

[00:23:56] How AI drafts help after hours and what UCSD is measuring next — depending on the area, 10 to 20% of AI-drafted responses get used. Marlene uses them most after hours, when she doesn't have the energy to write even the greeting. UCSD is studying whether AI-assisted replies lead to fewer follow-up messages because answers are clearer, and she encourages clinicians to bill for medical advice requests that take more than five minutes.

[00:25:52] How to roll out AI tools with almost no training — Marlene pitches new tools as additive rather than a change in workflow, and aims for an out-of-the-box experience like picking up a smartphone. Early feedback to Epic stressed that the drafting tool needed to be seamless and in the workflow. Rollout required only a tip sheet and mentions in the physician and APP newsletter and EMR news; most people simply started trying it.

[00:27:49] Why the future of the inbox is answering patients directly — Marlene is satisfied with a 10 to 20% usage rate because the bigger issue facing ambulatory physicians and APPs is exponentially growing message volume, and better drafts for clinicians won't solve that. She believes the future lies in a back-and-forth conversation that gives patients answers in real time, starting with requests like switching to a video visit.

[00:30:07] How agents could resolve patient questions before they reach the inbox — based on daily requests to turn off messaging, Marlene believes many providers would welcome answers delivered to patients ahead of time. Patients are already using ChatGPT, Claude, and Google for health questions, so the goal is a health-centric, protected approach that tells patients when they need an appointment. Healthcare moves slower than airlines because of the risk of harm, but she says this is coming "whether we like it or not," and health systems should partner with the companies building it.

[00:32:54] How UCSD chooses between Epic-native and third-party AI — the first question is whether a tool is useful, then whether it should be integrated and whether it is HIPAA compliant. Embedding a tool like Doximity GPT in Epic gives it authority for patient data and discourages clinicians from using other tools. Rather than telling people what they can't use, Marlene asks, "Well, what are you giving them to use?" UCSD maintains a website listing approved HIPAA-compliant tools, and having two tools that do the same thing is fine.

[00:35:56] How to catch shadow AI without driving it underground — departments sometimes buy tools on their own and then ask IT to connect them, only for the team to discover "a lot of AI features." UCSD's IS ThinkShop gives people a frictionless path to bring ideas to experts, including security. Marlene says there's no way to get 100% ahead of it, so the goal is having as many places as possible to catch it. The issue is usually a security review or BAA, not the AI itself, and being punitive only makes people hide things more.

[00:39:06] Why UCSD discloses AI use even when it's not required — in California, human-in-the-loop AI use doesn't require disclosure, but UCSD discloses in many instances to build patient trust. Disclosure on AI-assisted portal messages helped patients understand the AI scribe when written consent was introduced. UCSD uses standard phrases for AI-drafted messages and scribe visits, and Marlene tells patients when she uses AI to write a letter. She notes several lawsuits stem from recording consent, not the AI itself: "The trust part is gonna be even more important as we roll out these agents."

[00:43:07] How dyslexia shaped Marlene's path into informatics — severe dyslexia meant Marlene didn't learn to read until about age 11 or 12, and she remembers how good it felt when she finally could. She still reads slowly and relies on tools like voice reading, which taught her to use technology to make her day better. Hearing that technology didn't make other people's days better drew her deeper into informatics, and she says dyslexia helps her think in a more "3D" way about whole processes and the big picture.

[00:45:23] Why Marlene goes through her own health system like any other patient — her experiences as a patient have informed changes like posted ER wait times, Express Care video visits, and online mammogram scheduling. She avoids asking for favors, calls the phone tree and triage herself, and encourages others not to skip the line: "You don't really understand what's happening in your system unless you do it." That perspective helped her advocate for referrals lasting two years instead of one, better access for UCSD's own employees, and wait lists. "They're all talking about AI, but don't forget about day-to-day, like calling people back for colonoscopies."

[00:50:07] Why informatics leaders need to embrace change — Marlene's parting advice is to keep iterating and not get stuck doing things the same way, even when it's easier. "If we all kind of, especially in the informatics world, embrace that this is a changing landscape, changing job, and try to help people with it, we're all gonna be really successful at it."

Fast 5 Lightning Round:

1. What is your favorite book, or the book you've gifted the most?

Ali Hazelwood's STEM-based romance novels

2. If you could instantly master any skill, what would it be?

"Skiing"

3. Would you rather have super strength, super speed, or the ability to read people's minds?

"Super speed"

4. What is something in healthcare you believe that others might find insane?

"The number of MyChart messages in my In Basket"

5. What is the last movie or TV show you saw, and what did you think of it?

"Crash Landing on You on Netflix. It's a lot of fun."

The Digital Patient has been recognized as Feedspot's #1 Patient Engagement Podcast of 2025. Thank you to our listeners for making this happen!