University of Texas at Austin

Tuesday, February 22, 2011

Wrapping up Watson

Ken Jennings, the Watson avatar and Brad Rutter on the Jeopardy! set.

Ken Jennings, the Watson avatar and Brad Rutter on the Jeopardy! set.

It’s been a few days since the IBM Corp.’s Watson computer won big at Jeopardy! and still no computer overlords.

What we do have is a new sense of what computers and artificial intelligence (AI) can do and how they can be used.

Researchers, including some at The University of Texas at Austin, have worked for decades to get computers to understand natural language, the way people talk. Computers have a hard time dealing with nuance, allusions and puns.

Some of Watson’s technology makes progress in those areas.

“The sort of QA (question answering) technology behind Watson is the next major step in information management beyond Web search, allowing direct answers to specific questions rather than just ranked document retrieval,” said Ray Mooney, a computer science professor at the university.

IBM spent millions of dollars and four years of research by a team of high-level scientists to build Watson. The company bet it could deliver a computer that could make its way through language tricks used by Jeopardy! to come up with the right answers to beat past champions Ken Jennings and Brad Rutter.

It did, despite that Toronto answer in the category, “U.S. Cities,” in the first game’s Final Jeopardy! round. Watson, named after IBM founder Thomas J. Watson, won the two games played with a total of more than $77,000.

Computer Science Professor Ray Mooney, an expert in natual language processing.

Computer Science Professor Ray Mooney, an expert in natual language processing.

Research done by Mooney and Texas colleagues Bruce Porter and Ken Barker, and computer scientists at several other universities was used in the development of Watson.

The Texas professors had been occasional witnesses to Watson’s creation. They work with IBM scientists on other projects and were at IBM’s research center in Yorktown Heights, N.Y., several times while Watson was being developed.

At first, Mooney said, the professors were skeptical that the IBM team, or any other artificial intelligence researchers, for that matter, could build a successful Jeopardy! contestant.

Weighing in after the Jeopardy! games, Mooney assessed Watson’s strongest and weakest points.

“Watson’s primary strength is its ability to effectively combine many different AI methods by using machine learning to weigh and combine evidence for many competing hypotheses proposed by a multitude of different techniques all executing in parallel,” he said. “Its weakness is still a fundamental reliance on keyword matching in a large body of text while exploiting relatively little semantic understanding of the text or deeper reasoning.”

Mooney said the question-answering technology behind Watson could allow computers to make direct answers to specific questions, rather than ranked document retrieval as search engines do.

“This is achieved by searching a huge repository of text for relevant passages, extracting competing answers to questions from these passages and using machine learning to effectively combine the results extracted from multiple documents into an overall confidence in each potential answer,” he said.

Applications in medicine, law and finance are being developed.

The Watson-Jeopardy! games provided a jolt of publicity not only for IBM, but for the field of artificial intelligence.

That was true at the packed on-campus watch parties, where Mooney, Porter and Barker provided context and some behind-the-scenes anecdotes they’d picked up on visits to the IBM research center.

“It was a very fun and exciting event which attracted significant attention to the progress that AI has achieved over the last decade,” Mooney said. “I am hopeful that the experience will spawn a new generation of AI researchers who will continue to address the significant remaining challenges in developing truly intelligent computing systems.”

Share this post: These icons link to social bookmarking sites where readers can share and discover new Web pages.
  • Print
  • Digg
  • Facebook
  • Google Bookmarks
  • LinkedIn
  • StumbleUpon

Post a comment

» Wrapping up Watson



Comments are moderated. They will be posted if they stick to the topic and contribute to the conversation. They will not be published if they contain or link to abusive material, personal attacks, profanity or spam.