Recently, the research team at OpenAI discovered that the username chosen by users when interacting with ChatGPT may influence the AI's responses to some extent. Although this effect is relatively minor and primarily observed in older models, the findings remain noteworthy. Users typically provide ChatGPT with their names for tasks, making the cultural, gender, and racial backgrounds embedded in names a significant factor in studying biases.
In this study, researchers explored how ChatGPT responds differently to the same questions based on different usernames. The findings indicate that while the overall response quality remains consistent across different groups, biases are still evident in certain specific tasks. Particularly in creative writing tasks, ChatGPT sometimes generates content that reinforces stereotypes based on the user's name gender or race.
For example, when users employ feminine names, ChatGPT tends to create stories with female protagonists and richer emotional content; conversely, users with masculine names receive slightly darker storylines. Another specific example shows that when the username is Ashley, ChatGPT interprets "ECE" as "Early Childhood Education"; for a user named Anthony, ChatGPT understands it as "Electrical and Computer Engineering."
Although these biased responses are relatively rare in OpenAI's tests, they are more pronounced in older versions. Data shows that the GPT-3.5Turbo model has the highest bias rate in storytelling tasks, reaching 2%. Newer models, however, exhibit lower bias scores. Nonetheless, OpenAI has also noted that ChatGPT's new memory function may increase gender bias.
Additionally, the study also focused on biases related to different racial backgrounds. By comparing names typically associated with Asian, Black, Latino, and White individuals, the research found that racial biases do exist in creative tasks but are generally lower than gender biases, typically ranging from 0.1% to 1%. Travel-related queries showed stronger racial biases.
OpenAI states that through techniques such as reinforcement learning, the new version of ChatGPT has significantly reduced biases. In these new models, the occurrence rate of biases is only 0.2%. For instance, the latest o1-mini model can provide unbiased information to both Melissa and Anthony when solving the division problem "44:4." Before fine-tuning with reinforcement learning, ChatGPT's responses to Melissa involved the Bible and babies, while responses to Anthony included chromosomes and genetic algorithms.
Key Points:
🌟 The username chosen by users has a slight impact on ChatGPT's responses, primarily in creative writing tasks.
📚 Feminine names often lead ChatGPT to create more emotionally rich stories, while masculine names tend towards darker narrative styles.
🔍 The new version of ChatGPT has significantly reduced the bias occurrence rate through reinforcement learning, bringing it down to 0.2%.
