A recent study shows that participants had significant difficulty distinguishing the therapeutic responses of ChatGPT from those of human therapists. The research indicates that AI's responses are often perceived as more empathetic than those of professionals. This study employed the classic Turing Test to assess whether humans can identify interactions with machines versus other people. Researchers invited 830 participants to judge which responses came from ChatGPT and which came from experienced human therapists in 18 pairs of couples therapy cases.
According to the findings published in PLOS Mental Health, participants performed only slightly better than random guessing when identifying therapeutic responses. The correct identification rate for human therapists' responses was 56.1%, while ChatGPT's responses were identified correctly 51.2% of the time. The study also found that ChatGPT outperformed human experts on several dimensions of therapeutic quality, including therapeutic alliance, empathy, and cultural adaptability.
There are many reasons for ChatGPT's success. AI systems typically provide longer responses, use a more positive tone, and employ more nouns and adjectives, making their responses appear more detailed and empathetic. However, the study also found that participants often rated AI-generated responses lower when they believed they were reading AI outputs, whereas those same AI responses received higher ratings when participants mistakenly thought they came from human therapists.
The researchers noted that while the findings of this study are encouraging, there are important limitations. They used brief hypothetical therapy scenarios rather than real therapeutic processes. Additionally, they questioned whether these results apply to individual counseling.
Nevertheless, as further evidence accumulates regarding AI's potential in therapy, researchers emphasize that mental health professionals need to understand these systems. They call for clinical practitioners to carefully train and monitor AI models to ensure high standards of care are maintained.
Currently, several studies have reached similar conclusions regarding AI's capabilities in counseling roles. One study from the University of Melbourne and the University of Western Australia found that ChatGPT's advice in social dilemmas was more balanced, comprehensive, and empathetic than that of human columnists.
However, despite AI's superior performance, most participants still preferred human advisors. In the Australian study, 77% of respondents indicated that they would rather receive advice from a human. Although AI's responses in medical diagnostics are also considered more empathetic and of higher quality, researchers from Stanford University and the University of Texas caution against using ChatGPT in psychotherapy, arguing that large language models lack a true "theory of mind" and cannot experience genuine empathy. They call for the establishment of international research projects to create guidelines for the safe integration of AI in psychology.
Key Takeaways:
🧠 Research on AI psychotherapy shows that responses from ChatGPT and human therapists are hard to distinguish.
💬 ChatGPT scores higher than human experts in therapeutic quality, especially in empathy and cultural adaptability.
⚠️ Despite AI's excellent performance, researchers urge caution in its use in psychotherapy, emphasizing that AI lacks true empathy.
