Text: In recent years, the application of artificial intelligence (AI) in the medical field has garnered increasing attention. In particular, chatbots like ChatGPT are being sought by many hospitals as auxiliary tools to enhance doctors' diagnostic efficiency. However, a recently published study shows that the use of ChatGPT did not significantly improve doctors' diagnostic capabilities. This study was published in the journal JAMA Network Open, revealing both the potential and limitations of AI in medical diagnostics.
Image Source Note: Image generated by AI, image licensed from service provider Midjourney
In the study, participants included 50 doctors, comprising 26 attending physicians and 24 residents. They were asked to make diagnoses based on six real cases within one hour. To assess the assistive effect of ChatGPT, the researchers divided the doctors into two groups: one group could use ChatGPT along with traditional medical resources, while the other group could only rely on traditional resources, such as the clinical information platform UpToDate.
The results showed that doctors using ChatGPT scored 76% in diagnostics, while those relying solely on traditional resources scored 74%. In contrast, ChatGPT achieved a score of 90% when diagnosing independently. Despite ChatGPT's outstanding performance when working alone, its combination with doctors did not yield a significant improvement, which surprised the research team.
The co-first author of the study, Ethan Goh, a postdoctoral researcher at the Stanford Center for Clinical Excellence, stated that the design of the study was not conducted in a real clinical environment but was based on simulated data, thus limiting the applicability of the results. He noted that the complexities doctors face with actual patients could not be fully captured in the experiment.
Although the study indicates that ChatGPT outperformed some doctors in diagnostics, this does not mean that AI can replace doctors' decision-making. On the contrary, Goh emphasized that doctors must maintain supervision and judgment when using AI tools. Additionally, doctors may exhibit stubborn tendencies in diagnostics; established preliminary diagnoses could affect their acceptance of AI recommendations, which is an area for future research to focus on.
After the diagnostic process concludes, doctors also need to answer a series of new questions, such as "What are the correct treatment steps?" and "What tests are needed to guide the patient's next actions?" This indicates that the application of AI in the medical field still has broad prospects, but its effectiveness and applicability in real clinical settings need to be explored further.
Key Points:
🔍 The study shows that doctors using ChatGPT had only a slight edge over those using traditional methods, with no significant improvement in outcomes.
🤖 ChatGPT's independent diagnostic score was 90%, showcasing its strong performance, but it still requires doctors' supervision and judgment.
📈 More research is needed in the future to explore how to optimize AI's application in medical diagnostics to enhance its effectiveness.