Last month, IBM supercomputer Watson competed on Jeopardy! and beat two human champions. What did it take to engineer this achievement, and what’s next for Watson?

In mid-February, a new competitor on the TV program Jeopardy! decisively defeated two returning champions: Brad Rutter, the game show’s highest-ever money winner; and Ken Jennings, who holds the record for the longest streak on Jeopardy! with 74 straight victories. Besting the iconic quiz show’s two most successful and celebrated contestants is a feat in itself — but the new competitor, named Watson, is also an IBM artificial intelligence (AI) computer system.

“For IBM, the showdown was not merely a well-publicized stunt and a $1 million prize, but proof that the company has taken a big step toward a world in which intelligent machines will understand and respond to humans, and perhaps inevitably, replace some of them,” according to the New York Times.

The show was the culmination of a long project for IBM technicians and programmers who were looking to successfully showcase their computer systems’ abilities at natural-language processing, information retrieval, knowledge representation and reasoning, and machine learning. Making a machine that can understand natural human language and even wordplay, such as puns, has long been a goal of scientists and engineers.

Following computer Deep Blue‘s defeat of then-reigning world chess champion Garry Kasparov in 1997, the IBM team behind the machine-over-man triumph looked for a new challenge. But it wasn’t until 2005 that the team began working in earnest: developing a computer system that could defeat human challengers on Jeopardy!.

Watson is a breakthrough achievement in the scientific field of Question Answering (QA). Its software is powered by a server optimized to handle the massive number of tasks that Watson must perform at rapid speeds to analyze complex language and deliver correct responses to clues. The system incorporates a number of proprietary technologies for the specialized demands of processing an enormous number of concurrent tasks and data while analyzing information in real-time.

The project began by testing extant IBM technologies at general knowledge QA, and initial results were dismal, with the computers failing to answer the majority of clues or taking several minutes to answer clues correctly. At times, the machines misinterpreted clues and gave bizarre responses. In preview matches, “Watson’s logic appeared to fall down some odd semantic rabbit hole, repeatedly giving the answer ‘Tommy Lee Jones’ — the name of the Hollywood actor — to several clues that had nothing to do with him,” according to a separate New York Times report.

However, after years of work, the IBM team determined that the best way to mimic human ability to respond to complex, witty and syntactically-eccentric clues like those on Jeopardy! was to supply Watson (named after IBM founder Thomas Watson) with not one algorithm, but hundreds. The team fed numerous databases of knowledge into Watson’s 16-terabyte memory, including multiple dictionaries, thesauri and even the entire text of Wikipedia, then armed it with the ability to reference, cross-reference and distill clues down to their bare essentials to determine the correct response.

“[AI scientists] wrote algorithms that could take any subject and automatically learn what types of words are, statistically speaking, most (and least) associated with it,” the Times explains. “Using this method, you could put hundreds of articles and books and movie reviews discussing Sherlock Holmes into the computer, and it would calculate that the words ‘deerstalker hat’ and ‘Professor Moriarty’ and ‘opium’ are frequently correlated with one another, but not with, say, the Super Bowl.”

After years of research and development and hundreds of practice Jeopardy! clues, Watson, along with its human counterparts, was able to respond correctly to the February 16th Final Jeopardy! clue: “William Wilkinson’s ‘An Account of the Principalities of Wallachia and Moldavia’ inspired this author’s most famous novel.”*

Watson, who had scored correctly on several Double Jeopardy! clues throughout the match, had wagered enough to finish with a score of $77,147 to Jennings’ $24,000 and Rutter’s $21,600, securing bragging rights and the $1 million top prize (which was donated to charity).

Now that Watson is a Jeopardy! champion, what’s next?

IBM plans to market the Power 750 servers that ran Watson and the DeepQA software that enabled it to perform so well. Watson’s team envisions the software performing tasks for the health care, finance and customer service industries. John Kelly, IBM research laboratory head, “imagines a hospital feeding Watson every new medical paper in existence, then having it answer questions during split-second emergency-room crises.”

Already, IBM has teamed up with speech recognition software firm Nuance, Columbia University Medical School and the University of Maryland School of Medicine to have the system help diagnose and treat patients.

“For example, a doctor considering a patient’s diagnosis could use Watson’s analytics technology, in conjunction with Nuance’s voice and clinical language understanding solutions, to rapidly consider all the related texts, reference materials, prior cases and latest knowledge in journals and medical literature to gain evidence from many more potential sources than previously possible,” an announcement of the research agreement explains. “This could help medical professionals confidently determine the most likely diagnosis and treatment options.”

However, concerns have arisen over the reliance on a computer for important information that can, for instance, affect health care treatment. Specifically, as some see it, there are two problems: 1) Computers can and do fail, either because of software bugs or hardware malfunction; and 2) there are unexplored legal ramifications that may prove thorny. For example, if a doctor recommends treatment gleaned from computer help and the patient suffers or dies, does the malpractice suit hit the hospital or the computer’s manufacturer?

Watson’s head engineer, Dave Ferucci, offers a calm warning to people getting too worried or too excited over the implications of the computer’s Jeopardy! win.

“When you deconstruct this, and look at the machine, is any part of this really understanding the question? No,” Ferucci told PC Mag. “We don’t want computers in my opinion making value judgments about what it means to be human. Only humans can do that.”

* _{The correct response: Who is Bram Stoker?}

Resources

IBM Watson

Watson Research Center

Computer Wins on ‘Jeopardy!’: Trivial, It’s Not
by John Markoff
The New York Times, Feb. 16, 2011

Smarter Than You Think: What is IBM’s Watson?
by Clive Thompson
The New York Times, June 16, 2010

IBM to Collaborate with Nuance to Apply IBM’s “Watson” Analytics Technology to Healthcare
IBM, Feb. 17, 2011

Where Next for IBM’s Watson?
by Tom Simonite
MIT Technology Review, Feb. 17, 2011

IBM’s Watson Wins Jeopardy! Next Up: Fixing Health Care
by Peter Pachal
PCMag.com, February 16, 2011

IBM’s Watson: “Think” of the Implications
by David Hill
Network Computing, March 1, 2011

IBM Watson Scientist: Speed Matters, But So Do Accuracy, Intuition
by Sam Gusti
Epicenter (Wired.com), Feb. 16, 2011

Noam Chomsky v. IBM’s Watson Computer
by Gavin C. Schmitt
The Framing Business