Monday, February 14, 2011
If IBM Corp.’s Watson computer passes the Trebek Test, it doesn’t mean it will pass the Turing Test the next day. Or achieve world dominance.The Trebek Test will take place Feb. 14, 15 and 16 in the Jeopardy! Challenge. The computer plays the Jeopardy! quiz show against two of the best players in Jeopardy! history. Alex Trebek is the host of Jeopardy!
The computer must make sense of the tricky clues, search the tons of information it has digested, find a correct answer and decide how much money it wants to bet on its answer—in less than three seconds.
The Turing Test was proposed by Alan Turing, one of the pioneers of computer science. If you talk to a computer and a person (where you can’t see either one) and you can’t tell the difference from their responses, then it has passed the Turing Test.
Ray Mooney, a professor in the Department of Computer Science at The University of Texas at Austin, said that Watson is far from passing the Turing Test.Mooney, Bruce Porter and Ken Barker, also computer scientists at the university, were cited by IBM for their contributions to the development of Watson. IBM scientists have worked on building and training the computer for four years.
“There’s this science fiction hype out there that makes people nervous,” Mooney said about the capabilities of computers. “We know the problems are really, really hard. We love examples like this that show the progress we’ve made. But computers are not going to pass the Turing Test tomorrow. They’re not going to take over the world.”“They’re not going to wake up tomorrow and decide we’re useless,” added Barker, with a laugh. “That’s just not going to happen.”
Mooney, Barker and Porter emphasized just how hard it is for a computer to understand natural language.
A computer can think a sentence has 10,000 meanings, but a human can pick the right meaning right away.
“The real challenge in natural language,” Mooney said, “is to pick up that one meaning out of the combinatorial number of possibilities that the machine is actually confronted with that a human would assign to that. That requires all sorts of knowledge, of understanding the grammar of language, the meanings of words, the contexts in which words will have particular meanings, all sorts of things.”
The computer science professors are clearly excited about the contest as they talk about it. But they’re also a bit apprehensive.
If the computer wins in a walk, that might lead people to expect more than computers can deliver right now.
“There’s the success disaster,” Barker said. “Where Watson’s going to make it look too easy. I don’t think that’s going to happen. We want to see it sweat and indicate to the general public that it is a hard problem.”
So computers won’t take over the world tomorrow, or any time soon, apparently. But Watson and computer systems like it will be able to take on some of the tasks that humans can’t physically handle, such as keeping up with the large amounts of information flooding fields of all kinds, from medical and health research to intelligence.
Porter picked up on a point made by David Ferrucci, the leader of the IBM team building Watson, made in the Nova television program about Watson.
Ferrucci said a successor to Watson could be like the computer on Star Trek. The user addresses the computer by talking to it. The computer responds after quickly combing through its stored information.
Porter said the computer amplifies the intelligence of the human.
“It’s where you take a person who’s already smart like Capt. Kirk and augment their intelligence with this remarkable ability to sift through piles of data that no human could handle on their own,” he said.
How might that technology be used?
“Any area where performance is based on the ability to deal with a lot of information and act intelligently on it could benefit from this,” Barker said.
People, he said, are good at acting intelligently on what they know. Computers are very good at storing a lot of data,
A system like Watson would be able to “read” the data and cull out the information the human is looking for, he said.
“So any domain that’s dependent on these huge amounts of data and acting intelligently on it,” Barker said.
“Let’s say you wanted to keep track of a new sector of the economy,” Porter said. “And you wanted to monitor the news wires, the Internet, blogs and so forth for anything having to do this new business sector. That’s right up the alley for a system like Watson.”