Opinion: AI has created a shadow medical system
Every few weeks, headlines claim that AI has matched or beaten doctors, fueling the implication that doctors are slow, expensive, and optional. This narrative has helped build a parallel, industry-driven medical system outside hospitals: more than 40 million Americans ask ChatGPT a health question daily, often outside clinic hours, with no clinician to guide or critically appraise the answers. A study this year found that medical disclaimers have largely disappeared from chatbot responses, and leading models now ask follow-ups and attempt diagnoses, so patients can act without ever seeing a doctor.
Concrete consumer products illustrate the trend: Oura sells a 50-biomarker blood panel through Quest Diagnostics for $99; Function Health, valued at $2.5 billion in November, lets members order 160 lab tests a year, schedule a full-body MRI, and authorize ChatGPT to read the results; Ro and Hims can prescribe weight-loss or anxiety medications after an asynchronous intake; and Doctronic, which claims to be 'the world's #1 AI doctor,' has run 24 million consultations and now writes AI-generated prescription refills in Utah. The work once confined to clinics is becoming a set of consumer products patients can assemble on their own.
The transformation is happening faster in medicine than in law, finance, or software because of money: health care accounts for close to one-fifth of the American economy, creating enormous incentives for AI companies to capture part of a doctor's work or convince hospitals, insurers, and patients they can save money without sacrificing quality. However, while advanced AI systems perform impressively on selected clinical tasks in research settings, researchers are beginning to ask whether autonomous clinical AI should be licensed and regulated more like clinicians than conventional medical devices. The central risk, the author argues, is that evidence generated in bounded research settings is being pulled into a marketplace eager to claim that doctors are becoming optional, with a model that performs well on a benchmark potentially being marketed as a complete substitute for medical care.