Not too many years ago Geoffrey Hinton confidently predicted that radiologists would soon be replaced by AI. That didn't happen. But now...
The New York Times a small study (50 doctors, a mix of residents and attendings) in which ChatGPT-4 outperformed physicians in diagnosis based on a case report:
...doctors who were given ChatGPT-4 along with conventional resources did only slightly better than doctors who did not have access to the bot. And, to the researchers' surprise, ChatGPT alone outperformed the doctors.
"I was shocked," Dr. Rodman said.
The chatbot, from the company OpenAI, scored an average of 90 percent when diagnosing a medical condition from a case report and explaining its reasoning. Doctors randomly assigned to use the chatbot got an average score of 76 percent. Those randomly assigned not to use it had an average score of 74 percent.
The study showed more than just the chatbot's superior performance.
It unveiled doctors' sometimes unwavering belief in a diagnosis they made, even when a chatbot potentially suggests a better one.
Of course, reading an x-ray and analyzing a case report are very different activities. Still...
There's more at the link, including a brief look at INTERNIST-1, an old-school AI system developed in the 1970s for diagnosis. It's clear to me that AI his here to stay, in general, and certainly in medicine. What's not at all clear is just how it's going to be used. Obviously, that will change over time as AI capabilities develop. While thinking about that you might look at Hollis Robbins' post, AI and The Last Mile:
While we worry about AI replacing human judgment, the real story may be how AI is creating a market for that judgment as a luxury good, available only to those who can pay for the "last mile" of human insight. What do I mean by this?
The challenge of mail delivery from the post office to each home or from a communication hub to each individual end user is known as a "last mile" problem. In the paper newspaper era, the paper boy was the solution to the last mile problem, hawking papers on street corners or delivering papers house by house in the early morning before school. The postal carrier is a solution to the last mile problem. DoorDash is a solution to the last mile problem in the food business. [...]
What I'm calling "the last mile" here is the last 5-15% of exactitude or certainty in making a choice from data, for thinking beyond what an algorithm or quantifiable data set indicates, when you need something extra to assurance yourself you are making the right choice.
