Global Language Monitor, which follows and analyzes language trends around the world, said claims that Watson used "natural language processing" to defeat human competitors Ken Jennings and Brad Rutter during this month's "Jeopardy!" showdown appear to be misleading.
"Watson did not prove adept at processing language in a manner similar to humans," said Paul Payack, president and chief word analyst at GLM. "In fact, computers have dramatically failed at this task for four decades now."
In a statement, IBM Vice President Scott Brooks said Watson has not claimed Watson understands natural language in the same way humans do.
"However, Watson does indeed process natural language in its own way," Brooks said. "For example, it can figure out the difference between running for office and running a race and between being hit by a bat made of wood and a bat that's a mammal."
Payack called Watson's capability "an accomplishment" but "a far cry from natural language processing.
"Rather, what Watson achieved was a very close approximation of appearing as if it had acquired an acuity at understanding of the English language," he said. "This, in itself, is an accomplishment to be acknowledged. After all, Watson was designed from the ground up as a 'question-answering machine,' as IBM readily admits."
Payack said, that's "not quite accurate" because Watson was specifically built as a game-show answering machine for "Jeopardy!"
Brooks said Watson's natural language processing is not perfect but is a "step in that direction. After all, it did it well enough and fast enough -- in its own way -- to compete nicely against Jeopardy champions, including the best ever."