
IBM's Watson starts on Jeopardy Tonight - KevBurnsJr
http://www.jeopardy.com/minisites/watson/
======
brown9-2
You can watch the PBS Nova episode about the building of Watson for free on
their website (about 60 minutes long, aired Feb 9 2011):

[http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-
earth....](http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html)

One question I have about Watson that I don't recall being mentioned in any
videos or articles so far - what sort of interface does Watson receive the
questions over? Is Watson performing speech recognition or getting the text of
the question via some sort of interface?

~~~
lrm242
This was answered in the Nova special, in passing. The question is fed to
Watson as text at the same moment it appears on the display that the
contestants can see. Likewise for correct answers. Watson is not performing
speech recognition.

~~~
dstein
_The question is fed to Watson as text_

Aw this seems rather half-assed to me. I hope at a later time they do another
one of these with a new and improved Watson. I really want to see a humanoid
robot standing on the podium (connected by wifi to the supercomputer) and
using OCR and speech recognition. I also think Watson's voice should be more
authoritative. It's like they went to extraordinary effort to make Watson
sound unimposing.

~~~
lrm242
It can always be "awesomer". However, calling it half-assed I think is a bit
harsh. The truly novel aspects of Watson are that in its abilies in to answer
questions in a truly domian-free way. It is not trained to be an expert in a
specific domain, but instead to calculate what it believes to be the best
answer from any domain. This is truly novel and amazing. That it can do this
with Jeopardy questions, regardless of how it is given those questions, is
truly remarkable. Jeopardy questions are often times laden with puns or other
literay devices. Watson can understand and answer better than the vast
majority of humans on this planet. That it can't currently "see" or "hear" is,
IMO, less interesting given the advances it makes in deep QA.

~~~
dstein
I didn't mean to sound harsh - I understand the significance. I just meant the
technology involved in Watson being able to answer such questions is
undoubtedly orders of magnitude more complex than the OCR and speech
recognition required to process the question in the first place.

They're playing a "human" game after all, and they're not playing like a human
is fed into the computer electronically.

~~~
natural219
You realize that if it were only to recognize speech, other players would have
a significant time advantage because they can read the text in probably 1/10th
of the time it takes the host to speak the question out loud. "Feeding" Watson
the text is just another way of saying Watson "reads" the text displayed on
the screen, just like every human is allowed to.

~~~
bugsy
Well is the text "fed" in a word at a time, or the whole question as a block
that takes nanoseconds to parse. That's an unfair advantage if so because
because the computer is getting the data in a different format than the other
contestants, who are handicapped since they have to preprocess the data in a
way the computer doesn't because it has been preprocessed in advance for it.

~~~
thirtyseven
How are the two situations any different? Just because a program has an entire
block of text in memory doesn't mean that it can instantaneously build all the
data structures needed to process and make sense of it. We both "read,"
Watson's "reading" just takes place in its code.

~~~
bugsy
It's very different. The humans have to decode the visual and aural
representations before parsing for meaning can commence. The computer should
have to do this too, but that step has been done in advance for the computer.
This gives the computer a time advantage not because it was faster at
computing, but because necessary computing was removed in advance from its
task.

~~~
rbarreira
But it would be trivial to OCR the text in a few microseconds, so I don't see
how it makes any difference at all to give the text to the computer...

------
swanson
"a';DROP TABLE 'knowledge'; This type of attack is commonly used by hackers."

Game over, Watson.

~~~
zach
Ken should totally go for the Bobby Tables response if there's a Daily Double
he doesn't know.

That or the Star Trek self-destruct sequence.

------
waxpraxis
I'm currently finding out what information we're allowed to share about how
the avatar works and what went into developing it. The problem is we're so far
down the totem pole I probably won't know for a while yet. :-/

[http://automatastudios.com/automata-gives-ibms-watson-a-
face...](http://automatastudios.com/automata-gives-ibms-watson-a-face/)

------
kj12345
One thing I'm interested in is any skew in the questions from normal. In
particular I hope they ask linguistically tricky questions where you can't
even figure out what's being asked at first. I felt like they went a bit easy
on that front in the preview round:

<http://www.youtube.com/watch?v=WFR3lOm_xhE>

~~~
kenjackson
I doubt they are skewing the questions. That kind of defeats the point. I
suspect Jeopardy is constantly adding to a pool of questions. My hope is that
they simply grab from that pool just like any other episode.

Note, the hardest questions for Watson are short questions -- the reduced time
hurts Watson more than most human competitors.

~~~
random42
Also, smaller questions provide, much lesser context for Watson to work with.

------
ugh
It’s a shame that this is (necessarily!) such an insular challenge. Everybody
knows what chess is all about, I fear that the impact of this game will be
limited to the US or the Anglosphere. Just as an example, there has been no
Jeopardy on German TV since 2000, it’s not really a part of German pop culture
and because of all the puns it doesn’t translate well.

(Question for native speakers: When watching the practice round [0] are you
generally able to keep up and answer the questions? The speed with which the
game was moving made it nearly impossible for me to follow or enjoy the game.
I would like to know what the experience is like for native speakers.)

[0] <http://www.youtube.com/watch?v=WFR3lOm_xhE>

~~~
metageek
> _Everybody knows what chess is all about,_

Well, that's kind of the point. Chess is a hard game, but it doesn't require
broad knowledge of a human culture. If you're going to build a machine that
exhibits such knowledge, you've got to pick a culture.

~~~
ugh
That’s why I wrote “necessarily!” Jeopardy is certainly a great pick, I can’t
think of anything better.

------
chaosmachine
Is there somewhere to watch this online?

~~~
jackowayed
Looks like IBM will have it online in a few days
<http://twitter.com/#!/IBMWatson/status/37223337453158400>

~~~
andrewce
Thanks for posting this. I got called into work and missed today's Jeopardy!
episode, so seeing this link was definitely a relief.

------
shortlived
In case anyone is interested to know Watson's opponents: _Watson will compete
against Brad Rutter, the current biggest all-time money winner on Jeopardy!,
and Ken Jennings, the record holder for the longest championship streak_
[source: wikipedia]

------
sambeau
A valentine's day gift for us single geeks

~~~
jazzyb
Speak for yourself. My girl-friend is sufficiently geeky that she suggested we
watch it together on Valentine's.

------
3pt14159
[http://www-03.ibm.com/innovation/us/watson/what-is-
watson/co...](http://www-03.ibm.com/innovation/us/watson/what-is-
watson/countdown-to-jeopardy.html)

Has a nice little video too.

~~~
quickpost
I'm glad they touched on the idea that this is actually a "Human vs Human"
competition... Really good Jeopardy players vs. a team of humans that built
their own "Jeopardy Player".

Watson winning the tournament is a triumph for humanity, not just for
machines.

------
savrajsingh
Did anyone else notice that the vignettes about Watson's creation featured IBM
researchers using MacBook Pros? So much for "International Business Machines."
;)

~~~
bjg
As a former IBM employee, you can pretty much use what ever type of machine
you want for your work as long as it doesn't interfere with what you do. There
is a significant Mac subculture internal to IBM, and they love and suppor it.
However a huge percent of their 600,000 employee's still use IBM branded
laptop's and desktop's.

------
umjames
Much of success on Jeopardy is not just deciphering clues in the answers, but
your timing on ringing in to give the question. I'd imagine a machine could
get really good at getting the timing down. Does Jeopardy have a way of
varying who rings in first to keep things more fair?

~~~
cryptoz
The timing of when players are allowed to click in is controlled by a human.
Someone backstage decides at what moment Alex is finished speaking and then
opens the clickers. If you click too early, you get ~ 300ms delay penalty
which gives the other players a chance to click in.

Click timing is indeed very important but I do not think Watson has any
special advantage there.

~~~
jackowayed
But there's a visual cue letting the players know that clickers are open (a
light flashes, I think), so the computer probably gets some notification too,
at which point it could instabuzz.

~~~
cryptoz
> instabuzz

Nope, it has to press down on a physical button just like everyone else. Maybe
it can move it's finger a bit faster, but...not much.

------
powrtoch
Anybody know whether this will be live or if it's pre-taped like most jeopardy
episdoes?

~~~
metageek
I believe I saw a report recently that they had finished taping it.

------
icedpulleys
In case anyone missed it the first time around, the nytimes mag had a pretty
good writeup on Watson back in June -- might be worth instapapering and
reading later if you're going to catch the broadcast this week:

[http://www.nytimes.com/2010/06/20/magazine/20Computer-t.html...](http://www.nytimes.com/2010/06/20/magazine/20Computer-t.html?_r=1&pagewanted=all)

As an AI researcher I'm excited to watch this week. Even if it's not the most
elegant artificial Jeopardy player imaginable, it raises the public profile of
a lot of AI & ML topics and might encourage and inspire other groups to tackle
ambitious projects.

------
phren0logy
_SPOILER_ Link to final score for those of us who are curious, but not curious
enough to watch:

<http://twitter.com/robotwisdom/status/37200070742904832>

~~~
ars
Thank you for not spoiling it in the text of your message.

------
olalonde
Suppose Watson scales cheaply (it doesn't), would it be serious threat to
Google? My guess is that it would be but I'd be curious to hear HN's take.

~~~
bryanh
I think it's fun to think about how Watson type intelligence will be at the
average consumer's figertips (and affordable, to boot) in less that 20 years.

A quick look at the TOP500 supercomputers puts about a 10,000x increase in raw
flops since 1993. Using that as a rough benchmark (a very rough one), we're
looking at some impressive stuff in the next twenty years. It's not
unreasonable to think that:

    
    
        current consumer processor in FLOPS * 10,000 < Watson's grid's FLOPS
    

Naturally, that is a very rough estimate and has no scientific bearing at all.

~~~
whatusername
For Reference: The laptop I am typing on is more powerful (in terms of FLOPS)
than Deep Blue.

(For that matter - so is every server in Watson)

------
KevBurnsJr
Is there somewhere in Silicon Valley to watch this together in person?

~~~
kenjackson
The geek equivalent to the sports bar... There must be some website for doing
this, right?

~~~
metageek
Actually, that might be your answer: if 100 geeks show up at a sports bar and
ask them to turn one of the TVs to Jeopardy, maybe they'll do it.

------
gursikh
(This is just for kicks.)

Assuming the space required to run watson halves every two years and v:: of
the human brain is v1 = 1500cm³, and Σv of Watson's servers = v2 = 90 x
(17.5cm x 44.0cm x 73.0xcm) = 5.05e6 cm^3.

So Watson will be the size of a human brain in t = - ln(v1/v2)/ln(2).

We're looking at 11.7 years. So, 2023.

* Source: [http://en.wikipedia.org/wiki/Watson_%28artificial_intelligen...](http://en.wikipedia.org/wiki/Watson_%28artificial_intelligence_software%29#Technology)

"Watson is made up of a cluster of ninety IBM Power 750 servers..."

* Source: <http://www-03.ibm.com/systems/power/hardware/750/specs.html>

"175 mm x 440 mm x 730 mm"

------
jal278
I'm sure there are those that disagree strongly, but I feel like this isn't as
much an advance for AI as it is another interesting combination of filtered
human-structured knowledge and computation power. Just as Deep Blue brute-
forced chess, this is mostly a brute-force of another, albeit more open-ended
domain (i.e. Q/A).

I'm not arguing that this isn't an impressive accomplishment, but that the
statistical-learning stream of research is likely a conceptual local optima
that yields the best results in the near term but is probably unrelated to the
way we ultimately achieve a creative, general AI.

------
tremendo
<http://twitter.com/robotwisdom> has some results from the show, don't know
how he was able to see it already.

------
kirpekar
Thanks for the reminder. DVR set.

------
thought_alarm
And be sure to stick around for Wheel of Fortune, where two previous champs
will take on an Apple II+. Beep.

------
lorax
I hope someone writes down all the answers (questions?) and feeds them into
google so we can have a Google-Watson showdown. (Oh, we need a third
contestant, how about bing too).

~~~
pbhjpbhj
I was interested in this too -
[http://www.reddit.com/r/technology/comments/fl477/tonight_a_...](http://www.reddit.com/r/technology/comments/fl477/tonight_a_computer_will_compete_on_jeopardy_this/c1guc4q)
was the first few clues fed in to Google.

What I found was that the answer was in the snippets on the first SERP but
certainly for some of the questions it seemed pretty hard to parse it out.

Also I think Jeopardy got 1 or possibly 2 answers wrong

\- White City was the City of the 1908 Olympics, it was in London Borough or
Greater London but not in the City of London
(<http://en.wikipedia.org/wiki/City_of_London>). White City is a borough and
not a city, which makes this a really hard question for a computer to answer I
think.

\- Nagini killed whatever Harry Potter character was mentioned ; I'm not
familiar with the series and didn't see this question this was just based on a
comment on Reddit and a Google for "killed $CharName" (I have a really bad
memory).

------
Kilimanjaro
I'd like to see Watson as a web service.

~~~
Retric
As would I. However, due to the computer resources it takes per question it
would be extremely expensive to scale.

 _Watson is made up of a cluster of ninety IBM Power 750 servers (plus
additional I/O, network and cluster controller nodes in 10 racks) with a total
of 2880 POWER7 processor cores and 16 Terabytes of RAM. Each Power 750 server
uses a 3.5 GHz POWER7 eight core processor, with four threads per core._ and
it still takes ~15 seconds a question.

PS: Rough order of magnitude: 90 IBM Power 750 ~= 180 EC2 High-Memory
Quadruple Extra Large Instance * 15 seconds ~= $1.50 a question. (Edit: that's
probably still low, each of those 750's have 16TB / 90 = 180GB of ram).

Edit2: Ouch 90 times 32 core IBM Power 750 list price is 492,210$ a month or
2.84$ per 15 seconds
([http://www-03.ibm.com/systems/power/hardware/750/browse_aix....](http://www-03.ibm.com/systems/power/hardware/750/browse_aix.html))
and those only have 128GB of ram,

~~~
patrickgzill
That would be a cheap promotion for IBM, who presumably is getting everything
at wholesale price :-)

------
tocomment
Why does Watson need so much computing power? I don't know of many NLP
algorithms that are so intensive?

~~~
brown9-2
Because in order to answer a question Watson is really executing hundreds or
thousands of different algorithms in parallel to attempt to find candidate
answers, and then comparing and building confidence scores for each.

~~~
tocomment
Thanks. Is there anything I can read to learn more about how it works in
detail? All the mainstream media articles I've read are very hand-wavy.

------
AARC233
This is interesting. Way to mix it up Jeopardy

------
tpatke
I assume this is pre-recorded and IBM would have cancelled it if they didn't
like the outcome.

~~~
T-hawk
It was pre-taped a few weeks ago in line with Jeopardy's usual schedule. There
were a number of untelevised practice sessions beforehand against other
Jeopardy contestants and wannabes. IBM did their tweaking then and promised
that the actual competition rounds would be televised regardless of results.

~~~
radioactive21
Also if IBM were to cancel an episode that showed bad results, it'll
eventually leaked out, especially for project of this caliber. It would be an
instant negative publicity, since EVERY tech site would pick it up.

You'll have the scorn of all the geeks on the internet, which could be
interesting lol

------
Charuru
Shouldn't IBM start their own search engine if they have algorithms like this?

Or conversely, maybe Google should buy them?

~~~
yumraj
_Or conversely, maybe Google should buy them?_

I wish you had checked the market cap of IBM and Google before suggesting that
:). IBM > GOOG

~~~
Charuru
You're right. Looks like IBM is doing pretty well.

