Recently, research teams from Stanford University, the University of Washington, and Google DeepMind developed a new type of artificial intelligence (AI) agent that can accurately simulate human behavior in social experiments. According to their research, this simulation technology is expected to provide a laboratory basis for theoretical testing in fields such as economics, sociology, organizational studies, and political science.
During the construction of these AI agents, researchers utilized interview data from over 1,000 American voters. The respondents' ages, genders, educational backgrounds, and political views represented the diversity of American society. The AI agents analyzed these interview records and utilized the GPT-4o model to reproduce the respondents' real reactions when users posed questions.
In terms of implementation, the research team conducted two-hour in-depth interviews with each participant and used OpenAI's Whisper model to convert the interview content into text. This method significantly improved the accuracy of the AI agents. In tests predicting human behavior, the AI agents based on interview data successfully predicted human responses in general social surveys with an accuracy of 85%, outperforming AI agents that relied solely on basic demographic information.
The researchers also conducted five social science experiments, and the results showed that in four of these experiments, the outcomes generated by the AI agents were highly consistent with the responses of human participants, achieving a correlation coefficient of 0.98. This indicates that the interview-based approach demonstrated higher accuracy and better balance in analyzing responses across different political ideologies and racial groups.
To facilitate further research, the research team uploaded the dataset of the 1,000 AI agents they created to GitHub for use by other scientists. To protect participant privacy, the team implemented a two-tier access system.
Scientists can freely access aggregated response data for specific tasks, while access to individual response data in open research requires special permissions. This system aims to help researchers study human behavior more effectively while protecting the privacy of original interview participants.
Project link: https://github.com/joonspk-research/genagents
Highlights:
🌟 The AI agents developed by the research team, based on interview data, can accurately simulate human behavior, enhancing the accuracy of social science research.
📊 The AI agents achieved an 85% prediction accuracy in social surveys, significantly outperforming agents that relied solely on demographic information.
🔍 The dataset is publicly available, allowing other researchers to access it via GitHub, promoting the study of human behavior while protecting participant privacy.