Paging Dr. Chatbot …

Some people fear AI. Some resent it. And some can’t welcome it fast enough, having waited our whole lives to live the lazy, technicolor, robot-assisted existence we were promised on The Jetsons.

But whatever your stance, odds are good you’ve asked ChatGPT about a freaky medical symptom that plagues you.

Polls vary, but somewhere between a quarter and a third of Americans have sought health advice from Large Language Models (LLMs) such as ChatGPT. And it’s easy to see why.

They’re the same reasons people check out online porn: It’s fast. It’s free. And though we know intuitively they’re stockpiling our data, it feels miraculously, momentarily anonymous.

Who among us wouldn’t rather type constipation, back acne, or bad breath into a phone from the comfort of home than sit under fluorescent lights in a paper gown and confess those maladies aloud to a stethoscope-draped human stranger?

Perhaps most compelling, though, chatbot responses have an air of confidence and authority — just like a real doctor.

But (and I can’t stress this enough) it’s not a real doctor. A recent study found that, when asked about health symptoms, AI chatbots often produce faulty results. In fact, they led the study’s 1,200+ human users to incorrect diagnoses more often than not.

Of course, staring at a scary diagnosis on your laptop can induce its own disquieting symptoms: sweaty palms, racing heart, cotton mouth. And two of my own family members (I won’t say who, but I birthed one and one birthed me) chased some curious symptoms down a spiraling AI hole last month and emerged hours later fully panic-stricken.

Turns out they’re fine.

To be fair, though, LLMs are actually quite good at processing medical information. They’ve even successfully passed medical licensing exams, and been known to pinpoint answers missed by human doctors — who, let’s face it, have been known to get things wrong on occasion.

“My uterus was a medical mystery to my doctor and the hospital I was at,” says my friend Chelsey, who had complications after the recent birth of her daughter. “It was really scary, but ChatGPT guessed the right diagnosis.”

Tests later confirmed what the bot had surmised.

“It didn’t stop me from getting medical attention, and it never would,” she says. “But it helped calm me down about it, and relax.”

So, why’d the bots in this recent study botch so many diagnoses? Wellll, how do I say this nicely?

It’s us. Hi. We’re the problem. It’s us.

Researchers say that the way most of us ask our fretty, amateur questions leaves chatbots without a full picture of our problems. We omit important details that clinicians know are key to proper diagnoses — things such as frequency, duration, and severity of pain, for example.

So, the bad news is we’re still constipated and woefully uninformed — but now we also ask bad questions.

There’s good news, though! Turns out chatbots are exceptional when it comes to advice about … well, chatbots. Here’s what ChatGPT told me about asking LLMs for health advice: 

There’s a real risk in over-trusting AI for medical decisions. It doesn’t examine you, can’t access your full history, and may miss nuance or rare-but-important conditions.

Think of an LLM as a well-read assistant, not your doctor.

When seeking health advice, ask for multiple possibilities, how common or likely they are, and what symptoms would rule them out or signal you should seek care promptly.

Use AI as a translator and brainstorming partner on health issues, then bring what you learn to a real clinician for any decision-making.

I don’t know about you, but that sounds like a prescription I can swallow. At least until I can finally get an appointment with Dr. McGravity. 

You know, from The Jetsons.