Turing Test: Do Machines Already Pass as Humans?

Alan Turing described this method as “The Imitation Game” because the person must ask questions to determine whether the communicator is a machine. On the other hand, the machine is designed to fool the interrogator into thinking it's human.

Ask tricky questions that are difficult for AI to respond to. That can be topical questions about the weather, a sudden change of topic, or questions to find out the communicator’s opinions. 

Since its creation, the Turing Test has been widely used by researchers to assess AI’s cognitive abilities. Over time, new versions of the test have been created.

For example, the Visual Turing Test examines if a judge can tell whether visual content is created by a human or AI, while the Linguistic Turing Test checks if a machine can understand emotional nuances, context, or conversational language.

A machine is considered to have passed the Turing Test if it convinces people it’s human for a certain period. The pass rate and process can vary across different cases.

Computers have shown they are getting better at the Turing Test. In 2014, a program pretending to be a 13-year-old Ukrainian boy named Eugene Goostman was the first to score a 30% mark, convincing 33% of judges it was a human. Earlier, programs like Eliza and Elbot came close to 30% but couldn’t beat it. 

A recent test found that 54% of the time, people couldn’t distinguish GPT-4 from a human. In a 2024 study by researchers at the University of California, San Diego, human participants had a 5-minute conversation with a human or AI to tell if they were interacting with a human. 

Besides ChatGPT-4, the judges interacted with actual humans, and 2 other AI systems - ChatGPT-3.5 and Eliza. The test was set up as a game and used a messaging interface, where human participants were randomly assigned to one of the groups to play a round of a game. 

During the study, researchers prompted both GPT models to behave as young persons who used slang, made occasional errors, were concise, and didn’t take the game very seriously. To prevent the models from responding at unrealistic speeds, their messages were sent with a delay. 

A part of the conversation with ChatGPT-4 went like this: 

For the last 60-70 years, AI systems struggled to score a 50% pass rate, and this research, along with other studies, shows the progress made in the technology.

Not all researchers are convinced that AI can pass the Turing Test, though. Critics mention the need to test a program’s ability across various contexts for more accurate results.

The biggest question perhaps is if the Turing Test is a reliable method to measure a machine’s intelligence. 

What different research and scientists seem to agree on is that passing the Turing Test doesn’t mean an AI has human-level intelligence. To measure this metric, researchers offer new testing methods. For example, a paper published in Intelligent Computing suggests testing if AI understands its own way of reasoning and how close it is to human beings.  

It depends on who you ask. 

Generally, scientists agree that the test is an important benchmark to analyze AI’s progress and abilities. But there’s another big question: 

There’s no correct answer to this question accepted by all, either. While studies like the one from UN San Francisco suggest ChatGPT passed the Turing Test, some scientists are skeptical.

Toby Ord, Senior Researcher at Oxford University, wrote on X that the results were mixed. In a thread, he says the research was done differently than Alan Turing’s initial vision, where a judge talked both with a human and a machine to decide which one is the human. 

Instead, participants held conversations one by one, and when talking to a human they rated them as human 67% of the time. According to Ord, the results showed that GPT-4 failed the Turing Test. The scientist added that the most accurate results would be possible with the participation of OpenAI and other AI labs. Meanwhile, they haven’t run public tests yet. 

Now that computers can pass as humans, humans constantly need to prove they are not robots. This is what the Reverse Turing Test is about. A widely used example of the Reverse Turing Test is CAPTCHAs (Completely Automated Public Turing Test to tell Computers and Humans Apart).

In the Reverse Turing Test, machines assess whether there is a human on the other side of the screen. For this purpose, we need to select the right images on websites, type letters, move the mouse, and complete other tasks that the verification system requires.

As bots are getting better at mimicking humans, the question is how long the Turing Test will remain relevant and what methods will measure a machine’s human intelligence.