Close up of woman nurse or doctor working on tablet computer inside hospital
Voices

The Data Tsunami and How Artificial Intelligence Can Help

The Voices section is a place for physicians, staff and community leaders to share their perspectives on all things healthcare. Dr. Michael Scahill writes about the intersection of technology and healthcare and offers his perspective on a constantly shifting landscape

MADERA, Calif. – In just the modest span of my career, the volume of medical data has exploded. Well-intentioned folks in my line of work call this “bloat” and worry over cajoling humans to stem the tide. We never will. Fortunately, AI will save us from ourselves.

“… it will be necessary to develop a more organized approach to the medical record… and a positive attitude about the computer in medicine… Developments [in computers] are far more advanced and immediately applicable than many realize, and concern with them is neither premature nor impractical.”

– Dr. Lawrence Weed

Ahead of the Wave

That quote nails it. I love to ask my students to guess when it was written. Typical guesses range from mid–Bruce Springsteen to late–Radiohead. No one yet has guessed the correct answer – 1968, when the Beatles were at their peak.

Dr. Weed was a man ahead of his time. The first commercial microprocessor, the Intel 4004, didn’t hit the market until 1971, and even that was some eight orders of magnitude slower than the bog-standard hardware of 2025. An IBM 2311 disk drive could hold 7.25 megabytes, almost enough for two typical JPEG photos, yet it weighed more than 100 pounds.

Indeed, so far ahead of his time was he that by the time I started medical school, authors reacted to Weed’s farsighted prediction: “For more than 30 years, there have been predictions that the widespread clinical use of computers was imminent. Yet the ‘wave’ has never broken.”

We sensed then – rightly, as it turned out – that the wave was finally upon us.

Just as I was emerging from medical school coursework into clinical rotations, Stanford made the leap from a motley array of paper charts and siloed computer systems into Epic. As Meaningful Use requirements for the Centers for Medicare and Medicaid Services (CMS) loomed on the horizon – with penalties beginning in 2015 – more and more organizations followed suit.

The Wave Breaks

Turns out, we needed the government. CMS Meaningful Use started to bite around 2015. Ten years on, a paper chart is about as intuitive as a telegraph to medical students today.

And yet, the way we think about that data – and particularly the way we train new physicians to do so – barely changed. It needs to.

Behind the Wave

Even in my senior year of residency in 2015, nurses in the NICU at San Francisco General charted on actual flowsheets – specially printed paper about three feet wide, folded twice to fit in an 8.5-inch binder.

At the end of every shift, they’d splay it out over whatever table they could find to tally I/Os and vital signs. We’d wait for them to finish, hovering like vultures to get data for rounds. Pen on paper – not to mention manual calculations – limited flowsheet data to at most a few dozen variables per shift.

The keyboard-savvy like me might type and print H&Ps, but our progress notes were pen on paper and, as such, short. No one read them.

Those days are long gone and never coming back.

Après Moi, Le Déluge

Fast-forward 10 years. The average baby in our NICU accumulates more than 1,000 rows of flowsheet data every day. Those pens and folded sheets are long gone. A daily physician note can easily reach 4,000 words. As two-month stays are not unusual, especially in the NICU, the corpus of physician notes alone can easily stretch longer than Joyce’s Ulysses. And that’s not even to mention notes from nurses, social workers, therapists and case managers.

A complex but hardly unusual patient under my care recently had well over 750,000 rows of flowsheet data and 330,000 words (1.8 million characters) in notes. And that’s just at our hospital. They had probably as much data from two other institutions related to the same illness – and they stayed another month after that.

No human can process that much. I get the impression that few are even trying. The data explosion ship sailed when the modern electronic health record triumphed over the ballpoint pen and writer’s cramp, and it’s never coming back. Succinctly summarizing even a fragment of that much data is a real and complex skill that even experienced clinicians struggle with.

Medical informatics and compliance types, myself included, wring their hands about these stats and dream the impossible dream of winning the battle against “note bloat” and its less poetic cousins like flowsheet bloat.

A brave new world where doctors write short notes with just their impressions and medical decision-making – without the meaningless boilerplate and auto-copied data that became standard in the great EHR wave.

In short, a world much like the pre-EHR world minus the paper. Much as this appeals to my romantic inner Don Quixote, it’s hard to imagine we’ll ever turn back the clock.

It’s Only Getting Harder

AI scribes – software that listens to a doctor’s visit and automatically generates documentation – are all the rage in 2025, including here at Valley Children’s, where many of our physicians love them (and patients love that doctors are talking to them instead of typing in the precious minutes they have together).

But freeing doctors from the keyboard, awesome as it is, won’t help us win the quixotic battle against note bloat. Ambient AI nursing, which does the same for flowsheets, seems the next big thing. I’m frightened to ponder how many flowsheet types we’ll create when it’s so easy to fill them in. If we’re buried in the data wave now, worse seems sure to come.

The AI Surfer

AI may not be a panacea, but it’s the only solution on the horizon. A mere eight years since “Attention Is All You Need” transformed AI, large language models are excellent at summarizing and extracting relevance from the reams of data we produce in healthcare – and they’re still improving rapidly (for some perspective, it was 10 years from the Wright brothers’ first flight until the first paying passenger rode a plane).

Unwilling to wait for Epic to build such a feature, I built my own secure HIPAA-compliant LLM interface that extracts a patient’s hospital notes and automatically drafts a summary for their discharge paperwork. I make minor edits, but it’s better – and far, far faster – than I could do unaided, even on a patient I know well. Soon, it will be hard to remember when humans did such work. Writing a discharge summary will be to medical students today what writing paper prescriptions is to me.

Fighting note and data bloat by trying to generate less, more relevant data is a great concept… that we’re definitely not going to succeed at. Changing the way doctors write notes or hospitals design flowsheets is really hard. Sifting through masses of data with AI is far easier – and only getting more so.¹ Practicing medicine will be different from when I trained. And that’s OK.²

¹ AI writing notes for AI. If AI scribes are writing doctors’ notes and some other AI is reading them… where exactly are we going?

² OK for society. But our medical education system is in for an epochal challenge. I’m not sure it’s up for it.


Share

Contributions by

Michael Scahill, MD

Medical Director, Informatics & Digital Health
Read Bio

Related Articles

More articles from this category

Vaccine and plaster with a doctor and patient consulting in an appointment or checkup at home.

Why People Don’t Get the Flu Vaccine, And Why They Should

Toddler trying wheat toast with peanut butter.

Introducing Peanut Early: What Parents Should Know

Little girl in witch costume holding jack-o-lantern pumpkin bucket with candies and sweets.

Finding Balance This Halloween: A Pediatrician’s Perspective

How Primary Care Can Lead the Way in Pediatric Obesity Care

Join Our Newsletter

The latest from The Pulse straight to your inbox.

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Follow Us!

No results found.