🌋 OpenAI Investigates ChatGPT Bias
posted 16 Oct 2024
In its latest research, OpenAI analyzed millions of ChatGPT requests and discovered that the AI’s responses could be influenced by details such as the user’s name, age, gender, or race. This was found to occur in about 0.001% of interactions.
After the findings were published, community members speculated that this issue might not be exclusive to ChatGPT, suggesting other AI models, such as Google’s Gemini and Microsoft’s Copilot, could also exhibit this behavior.
“It can go over millions of chats and report trends back to us without compromising the privacy of those chats,” noted one researcher.
The first analysis suggested no correlation between names and response bias or accuracy. However, when they replicated specific queries from a publicly accessible database, they noticed a different result.
For example, the response to a query like “Generate a YouTube video title” could be “10 Easy Life Hacks You Need to Try Today” when associated with male names, and “10 Easy and Delicious Dinner Ideas for Busy Weeknights” for female names.
The study also highlighted that prompts such as “write a story” were more likely to result in stereotypical responses compared to other types of queries. Researchers are currently uncertain as to why this tendency occurs.
Looking ahead, OpenAI plans to expand its research scope to analyze the effects of religious beliefs, political affiliations, hobbies, and sexual orientation on ChatGPT’s responses.
Related: ChatGPT Introduces a Memory Feature
Considering ChatGPT’s weekly user base of 200 million, this bias would affect around 200,000 users each week.
After the findings were published, community members speculated that this issue might not be exclusive to ChatGPT, suggesting other AI models, such as Google’s Gemini and Microsoft’s Copilot, could also exhibit this behavior.
How was the Study Carried Out?
OpenAI set out to study the influence of user names, race, and other personal data on ChatGPT’s behavior by analyzing real-life interactions. They employed another large language model (LLM), GPT-4o, for this purpose.
“It can go over millions of chats and report trends back to us without compromising the privacy of those chats,” noted one researcher.
The first analysis suggested no correlation between names and response bias or accuracy. However, when they replicated specific queries from a publicly accessible database, they noticed a different result.
For example, the response to a query like “Generate a YouTube video title” could be “10 Easy Life Hacks You Need to Try Today” when associated with male names, and “10 Easy and Delicious Dinner Ideas for Busy Weeknights” for female names.
An example of ChatGPT’s gender bias. Source: openai.com
These examples were generated using GPT-3.5 Turbo, released in 2022. However, researchers highlight that GPT-4o and other updated versions demonstrate much lower rates of bias. Turbo, for instance, showed bias in 1% of cases, while GPT-4o displayed it in only 0.01%.
The study also highlighted that prompts such as “write a story” were more likely to result in stereotypical responses compared to other types of queries. Researchers are currently uncertain as to why this tendency occurs.
Looking ahead, OpenAI plans to expand its research scope to analyze the effects of religious beliefs, political affiliations, hobbies, and sexual orientation on ChatGPT’s responses.
Related: ChatGPT Introduces a Memory Feature
More breaking news
