Watson, the IBM supercomputer designed to play Jeopardy!, made his television debut yesterday, and while the game may not be over, it was a very auspicious start. Watson is currently tied with one of Jeopardy’s most successful champions, Brad Rutter, and is currently leading the most famous contestant ever, Ken Jennings.
This is pretty amazing on a lot of levels. Computers have been “smart” for a very long time (dating back to at least whenever we started teaching them chess), but Watson takes it to another level. For one, it’s worth pointing out that Watson is not connected to the Internet. In other words, Watson is storing all of the information it has for itself. More impressive, the computer can understand the intricacies of human language better than any other machine, since it has to not only generate information, but also generate the specific piece of information that the clue is looking for. Not to mention the fact that Jeopardy! clues are known for being full of wordplay and allusions that seem particularly troubling for a computer to digest.
While all this is impressive and important to the world of artificial intelligence, I’m more interested in another epistemological component of Watson’s thought process: How does Watson know what it knows?
This, after all, is far more crucial to playing Jeopardy! than it is to, say, playing chess. In chess, a player has to make a move when his turn comes up. Furthermore, a player knows all the most crucial elements of a game of chess before the game even begins (what the goal of the game is, the rules governing each piece’s movements, the relative value of each piece, etc.). With Jeopardy!, however, choosing when to answer is almost as important as choosing what to answer, and the windows for both of these decisions is about one to three seconds long.
So Watson’s ability to know stuff is less impressive than its ability to know what it knows.* Watson apparently works by using the language of the clue to generate lists of potential responses, and then running complex algorithms to select the best one,** all in a matter of seconds. As the processes are run, Watson’s “confidence” in each answer, represented by a percentage, changes until one hits the “buzz threshold,” at which point Watson rings in with that response (the buzz threshold appears to change throughout the game, presumably based on the scores and the round).
*After all, it’s hardly shocking in 2011 to see a computer that can contain massive quantities of information (particularly a computer as gigantic as Watson, which takes up an entire room and is the size of about three refrigerators).
** Of course, this is easier said than done: A clip shown of an early version of Watson showed the computer responding to a clue asking for the first nondairy creamer with “What is milk?” presumably since “nondairy” includes the word “dairy.”
It’s worth pointing out how different this is from how humans tend to think, and playJeopardy!. Watson’s thought processes seem to operate something like guess-and-check on a massive and rapid scale. Human brains, on the other hand, simply cannot process all that information at once. Instead, human brains use clues to get closer to the correct answer, rather than evaluating a multitude of choices at once. More important with regard toJeopardy!, Watson doesn’t play like most good contestants. Most contestants will buzz in when they have an inkling of the correct response, and finish their thought process before the time goes out. Jennings, in fact, has made an art of this—at one point last night he buzzed in and said, “Hmm, I don’t know,” before ultimately getting the correct response.
Helpfully, Jeopardy! included an answer panel at the bottom of the screen last night which helped illustrate just how different Watson’s thought processes actually are. For example, inthis video of a pre-show taping, we see that one of Watson’s possible responses to “This child star got his first on-screen kiss on the set of My Girl” was “Miley Cyrus.” Of course, no human would ever guess this—even if you had no idea who was in My Girl, the male pronoun in the clue would immediately eliminate Miley Cyrus from consideration. But since the clue contained “child star” as well as “kiss” and “girl,” Watson had Cyrus as the second most-likely choice. Of course, Watson ultimately did get the right answer (Macaulay Culkin, though Jennings buzzed in first), but not in the way any human ever would have.
Of course, the fact that Watson thinks differently than Jennings or another human shouldn’t matter, since its IBM designers have addressed the two crucial epistemological concerns: Watson generally knows what is right, and it knows that what is right is right. So it seems that Watson certainly does know things, but how it knows things is totally unique. When you consider that, until very recently, humans were the only things in the world that could be said to “know” things at all, this is pretty remarkable.
Whether or not it means that our future computer overlords will be friendly (or more like this), I can’t say. But I’ll be watching tonight to find out…
