The Game’s Afoot!
Remember Sherlock Holmes’s famous line: “Come Watson! The game’s afoot.”
While some have argued that Holmes was referring to a soccer match, in fact this line almost always preceded Holmes going forth and solving the mystery.
This time, though, Watson was the brilliant one.
The news of Watson, the IBM supercomputer, winning Jeopardy has been all over the web lately. I was lucky enough to attend an event at IBM in Littleton where they explained a bit about Watson and how it was developed, followed by the final Jeopardy show.
Yesterday, I received an email from someone arguing that Watson was, quite possibly, just a publicity stunt. After all, doesn’t a computer have an innate advantage in buzzing in? And what”s the big deal about a computer answering questions? After all, can’t Google do that?
Here’s my response (although since I’m quoting myself, I get to add all the things I wished I’d thought of when I originally responded 🙂 )
An interesting post on Watson, but your questions are easily answered… just use Google 🙂
Seriously, as impressive as Watson’s question answering was, that wasn’t what made it so successful. Let me address your other points first, though.
The trigger finger point: all human players develop heuristics for training themselves to buzz in as quickly as possible without getting locked out. Watson has its own algorithms, based on how much confidence it has in its answer. There were times when the human players beat Watson to the punch. However, just as a human player will try to keep the questions in an area where he has greater knowledge, which translates to an improved ability to respond quickly, Watson does the same. Just as humans respond more rapidly when we have higher confidence in our answers, so does Watson.
Watson vs. Google: try typing a typical Jeopardy question into Google: “A city whose first airport is named for a WWII hero and whose second for a famous battle from the same war.” What you’ll get is a discussion of how Watson answered that question (Toronto???). Google forces us to ask questions in a way the computer understands; Watson answers questions the way we naturally speak. Although probably oversimplified, Google does keyword matching ranked by popularity. Watson is attempting to do semantic matching — in other words, answer based on meaning. That’s more like what we do, although Watson doesn’t necessarily mimic how we do it.
The real secret to Watson’s success, though, was less about its ability to answer questions as its ability to gauge the confidence of its answers. Watson bets small amounts when it has low confidence and large amounts when it has high confidence, just like a person (or at least how a person might wish to act). However, Watson is considerably more able than most people to accurately assess the likelihood of its being right or wrong.
Watson is also able to calculate with a high degree of accuracy where Daily Doubles are likely to occur. Apparently, it’s a statistical calculation based on past games, and Watson can run that calculation very, very fast. Faster than any human. Given the previous discussion on confidence, we can see that this strategy gives Watson a chance to really clean up.
In short, as impressive as is Watson’s ability to understand English and understand puns (yes, it can do that!), the real secret to Watson’s success is that it knows how to win big when it’s right and cut its losses when it’s wrong.
Now that’s a lesson we might all benefit from!
