New Artificial Intelligence Challenge Could Be the Next Turing Test
A recently released
biopic of Alan Turing ("The Imitation Game") tells the story of the
British mathematician and cryptographer who built a machine to crack the
German Enigma code during World War II. But Turing is perhaps best
known for his pioneering work on artificial intelligence.
In 1950, Turing
introduced a landmark test of artificial intelligence. In the so-called
Turing test, a person engages in simultaneous conversations with both a
human and a computer, and tries to determine which is which. If the
computer can convince the person it is human, Turing would consider it
artificially intelligent.
The Turing test has been a helpful gauge of progress in the field of
artificial intelligence (AI), but it is more than 60 years old, and
researchers are developing a successor that they say is better adapted to the field of AI today. [Super-Intelligent Machines: 7 Robotic Futures]
The Winograd Schema Challenge consists of a set of multiple-choice
questions that require common sense reasoning, which is easy for a
human, but surprisingly difficult for a machine. The prize for the
annual competition, sponsored by the Burlington, Massachusetts-based
software company Nuance Communications, is $25,000.
"Really the only approach to measuring artificial intelligence is the
idea of the Turing test," said Charlie Ortiz, senior principal manager
of AI at Nuance. "But the problem is, it encourages the development of
programs that can talk but don't necessarily understand."
The
Turing test also encourages trickery, Ortiz told Live Science. Like
politicians, instead of giving a direct answer, machines can change the
subject or give a stock answer. "The Turing test is a good test for a
future in politics," he said.
Earlier this year, a computer conversation program, or "chatbot," named Eugene Goostman was said to have passed the Turing test
at a competition organized by the University of Reading, in England.
But experts say the bot gamed the system by claiming to speak English as
a second language, and by assuming the persona of a 13-year-old boy,
who would dodge questions and give unpredictable answers.
In contrast to the Turing test, the Winograd Schema Challenge doesn't
allow participants to change the subject or talk their way around
questions — they must answer the questions asked. For example, a typical
question might be, "Paul tried to call George on the phone, but he
wasn't successful. Who was not successful?" The correct answer is Paul,
but the response requires common sense reasoning.
"What this test tries to do is require the test taker to do some
thinking to understand what's being said," Ortiz said, adding, "The
winning program wouldn't be able to just guess."
Although the Winograd Schema Challenge has some advantages over the
Turing test, it doesn't test every ability that a truly intelligent
entity should possess. For example, Gary Marcus, a neuroscientist at New
York University, has promoted the concept of a visual Turing test, in
which a machine would watch videos and answer questions about them.
To address the need for alternatives to the Turing test, Ortiz, Marcus
and other AI researchers will convene at the 29th annual conference of
the Association for the Advancement of Artificial Intelligence in
January 2015 in Austin, Texas.
Follow Tanya Lewis on Twitter. Follow us @livescience, Facebook & Google+. Original article on Live Science.
- The 6 Strangest Robots Ever Created
- History of A.I.: Artificial Intelligence (Infographic)
- Top 10 Things that Make Humans Special
