By Timothy Tuttle, Founder & CEO of Expect Labs
For centuries, the four-minute mile was the height of athletic achievement. Roger Bannister turned that apex on its head in 1954 when he completed the mile in three minutes and 59.4 seconds. Similarly, the Turing Test has been the high-water mark in the artificial intelligence community for the past six decades. The latest development occurred on June 8th, when a chatbot disguised as a 13-year-old Ukrainian teen passed the test. 33% of judges were shocked to learn that this quirky guy named Eugene Goostman had a heart not made of gold, but computer code.
Another artificially intelligent program has entered the scene, leaving many computer scientists scratching their heads about its implications. Some were hung up on the technicalities of Goostman’s achievements, claiming that the event was too calculated to merit a passing grade. This analysis misses the mark. Today’s systems are intelligent, intricate, and overflowing with more advanced computing power than ever before. As a result, we now need to decide if Turing’s age-old assessment is still a useful barometer of a machine’s intelligence.
This is not the first time a computer program has baffled judges into thinking its alive — researchers are now able to routinely craft these kinds of cognizant systems. Back in 1972, 48% of a group of psychiatrists were certain that a chatbot named PARRY was actually a person with a very real case of paranoid schizophrenia. In 2011, a chatbot named Cleverbot passed with flying colors, convincing 59% of judges that it was a human. Both examples are way higher than Goostman’s meager 33% score — and they barely scratch the surface. In the past ten years, there has been an explosion of systems that can now outdo the once ironclad test.
For the first time since the dawn of computers, the deluge of machines that are capable of beating the test are making each individual breakthrough less and less remarkable. The fact that we’ve reached this point is astounding. Just twenty years ago, it was impossible to build systems that could top Turing’s gold standard, no matter how many limitations it was fed. Eugene Goostman symbolizes an important achievement in the field because we now have so many systems surpassing the threshold that it is no longer an accurate marker. With increasingly sophisticated systems entering the scene, what should we focus on instead?
Last year, A.I. researcher Hector Levesque proposed an alternative approach called Winograd schemas. The test is composed of two sentences that contain a double meaning that can only be decoded through commonsense reasoning. Terry Winograd invented the test in 1972 with the following example: “The city councilmen refused the demonstrators a permit because they feared/advocated violence.” If any of us were to read the sentence, it is fairly obvious that “feared” should be attributed to the city councilmen and “advocated” refers to the demonstrators. This is simple for a human to detect, but very difficult for a computer. Levesque argues that measuring machine intelligence with Winograd schemas is a more accurate assessment because it emulates the human ability to decipher ambiguities, something that the Turing Test currently fails to capture.
The controversy surrounding Eugene Goostman shows that we have entered a brighter era in artificial intelligence. In the past, no one could dispute the efficacy of the Turing Test because we were incapable of writing programs that could actually beat it. A more rigorous test is now needed because today’s computer programs are simply getting too good. Turing’s standard will always remain significant because it was the first test that encouraged scientists to build state-of-the art systems. Instead of fixating on the details of the Eugene Goostman news, we need to keep an important thought in mind: Chatbots like him are bringing us closer to a world full of machines that have the spark every human possesses, and every researcher still yearns to recreate.
