Hacker News new | comments | show | ask | jobs | submit login
Watson Services (ibm.com)
362 points by harscoat on Oct 8, 2014 | hide | past | web | favorite | 104 comments

This is the first batch of APIs that my team - the Watson Developer Cloud team - is exposing through IBM BlueMix for everybody to try and use. We have many more APIs in the pipeline but we decided to come out quickly with a small set to start with. We are trying get some early feedback from the developer community on what they find useful, what they'd like to see, how the APIs are designed, the quality of the documentation and example code, etc. Comments and questions are very welcome!

I have some questions:

- It says, "During the BETA period, all Watson Services will be available for free on Bluemix." So what are the prices after the beta period? I can't put the time into learning this if it's not viable after the beta.

- What is Bluemix? Is it just the "place where all the services are?"

1. Right now we are working on a revenue sharing basis with the start ups that sign up as ecosystem partners (they also have access to more capabilities). http://techcrunch.com/2014/10/07/ibm-sees-salvation-in-watso... has a few examples of these start ups and you can sign up here: http://www.ibm.com/smarterplanet/us/en/ibmwatson/ecosystem.h.... We will likely come up with a pay-per-call model that's compelling for start ups as well.

2. BlueMix is the IBM PaaS - it has a lot of powerful services beyond Watson - http://bluemix.net. It allows you to create and deploy apps in the cloud.

This makes it sound like you are really only thinking about specific partners at the moment then? So it's not yet for the masses?

Like - I'm not really going to be using this in my next Android / IPhone app any time soon.

Presently IBM's just getting in the space. I was offered a chance to use BlueMix during the closed beta. I didn't for the same reason you mentioned: they didn't have a pricing plan. I too thought, "What's the point of building something if it will be really expensive later?" Fortunately, IBM seems to be reasonable when they came out with pricing.

I wouldn't build my company around the beta of Watson, but if you experiment during beta, you could have first mover advantage when the pricing comes out and you find it acceptable.

Our plan is definitely to allow you to use this on your Android/IPhone app once we are GA.

While it's not something that you'll have running completely on your mobile device, it's something that you can easily run on the backend server. Fortunately, BlueMix provides lots of very useful services for mobile devices, including automatic crash monitoring, user feedback, and cross platform push services. Feel free to find me on twitter (same name) and I can provide more details.

I dont think they were intending for this to go out on production applications just yet. You could just play around with the API in some personal or open source project for now. Whatever pricing plan they come out with should be reasonable.

Bluemix is IBM's implementation of CloudFoundry, an open-source Heroku-like platform-as-a-service. Presumably they're making this API available as an add-on to Bluemix applications.

That's correct. These APIs are some of the many services you can attach to your applications within BlueMix.

Looks great! One question: Any plan for additional client libraries / demos beyond java, nodejs, ruby/rails? e.g. - django (python) - flask (python) - obj-c - scala - go

* Plug for python - lots of ml / nlp / scientific computing done with using python.

Yes you can use Python with these services and BlueMix. See a tutorial here: http://thoughtsoncloud.com/2014/08/getting-started-python-ib...

What you guys do is awesome. I'm interested in machine translation. I know python and NLTK. I have a grasp of statistics, I'm learning everyday more. Will I be able to design a domain-specific machine translation system with Watson?

Yes but it's not yet available. The current service only provides a fixed domain but we will extend the API and provide tools to allow you to specify a custom domain. Offering the ability to domain adapt all these services will be a key differentiator of the Watson services. Stay posted!

It's great that you've released the functionality. And you asked, so I hope I don't come across too harsh. I'm looking at your examples, here for example:


To make a simple call with the Java example, you've managed to write 372 lines of code.

Three hundred and seventy two lines.

That's some bad code.

There's nothing I can point at and go, that, that's what's wrong, the whole thing just has a lot of code smells. The 'zZz's (what is that DIY escaping all about?). The multiple http calls that should have been abstracted into one function, but aren't. The over commenting. The api structure requiring you to do 3+ calls. The exponential call back time (this is not a good idea, have fun with weird bugs). That you've made your own json objects. That you've used PUT for a pretty tenuous reason. That you've put all the urls in some config file or something, so the code isn't particularly clear at what it's doing.

The ruby & javascript are more terse because those languages are, but they aren't that much better. It's often silly little things, like this:

    '/status?' + querystring.stringify({'jobid':jobid});
Could have been written simply as this:

    '/status?jobid=' + jobid;
There really is no good reason to use 'querystring.stringify' in that instance, it's not user input. The code's simply over-complicated.

The code reads like a sysadmin/amateur/uni professor's code, a scripter rather than a professional programmer's. It's like you don't have a senior dev on the team to cast an eye over it and get rid of the excesses.

The APIs themselves are not bad, as I mentioned there's the slightly tenuous PUT stuff, the only other annoyance I could see is that you'd have to ping it all the time to see if it's ready. I appreciate why, and you've made the right call in the callback vs ping, simply because so often when hacking setting up a callback would be hell.

It's better than a lot, like google or paypal, so it's not all bad.

NB: But now I just found this exploring github:


Which at first glance is better code than the examples on the site? Bit odd.

Since the sample Watson code is downloadable and intended to show how to use these services, I find it odd that you cite "over commenting" as a problem. Comments are a great thing in sample code, especially in code that you can download separately, without docs.

For the jobid example, the stringify is necessary because the jobid returned by the server includes a # symbol. Try the code you suggested and you get the result: "Error: ContextExpansion Error: jobid is in incorrect format."

This is a perfect example of why I don't like to take silly shortcuts based on tenuous assumptions like the GP suggests.

Assumptions, more often than not, (1) change, (2) are understood differently by others, and/or (3) (as in this case) are flat-out wrong. Code that takes "advantage" of such silly assumptions usually (1) are poor examples to coders new to the project, (2) complicate reviewing the code for correctness, and (3) waste brain-time figuring out when they can (and can't) be applied.

Just escape everything that should be, whether you think you know what it is/where it came from or not.

This rampant clumsiness is fairly typical for IBM code samples--doing things manually that should use a popular library, or choosing obscure or IBM-invented libraries for common things. For example, the Java example should really use Jetty or CXF for JAX-RS rather than mucking about with HTTP client calls directly...

It seems that to get a true "wow" moment out of this, you would need to use the services in combination, but there is no demo of this. Each demo looks like nothing more than a database search of the training data - no super amazing algorithms supplied.

Perhaps IBM could provide an example "wow" moment to inspire us?

The demos are really made to demonstrate some basic input/output usage. Combination apps are indeed where we think developers will find the most value. We plan to have an app gallery to show that soon. Thanks for the suggestion.

Could you describe a potential "wow" app, and how you would use the combination of services to achieve it? To get me salivating for more?

To give an example ... Here is what a retail partner is building with these services around specialized marketing: They have (or get) a user's twitter handle, feed the content authored by the user into the user modeling service and based on the characteristics, send appropriate coupons/offers (e.g. very adventurous or less so etc). Combining that with the machine translation & language identification services, they are able to broaden this to many more languages vs. just English.

The part that they are doing next is: with the community of their users, they want to check the potential resonance of their marketing messages to those users, which they do with the Message Resonance service. Based on that, they can pick the most likely to be resonant message before sending it to the user.

Was expecting people to talk about apps centered around delivering healthcare advice/information cheaper/more readily to people, turns out we're just going to keep finding ways to use tech to market ads to people.

The APIs can be used for all of the types of apps you mention -- there's nothing limited to, or even targeted to advertising. Here's an idea for an app: Imagine that you're a non-native English speaker, and you want to communicate with English-speaking friends on Facebook or Twitter. Language identification can detected your language, machine translation and translate it to English, concept expansion can automatically generate potential synonyms, message resonance can select the synonyms that best capture your message, and user modeling can tell you what kind of emotions your message is communicating. This technology can and will be used to bring people together, but ultimately it comes down to what types of apps people make.

create a persona that can have a conversation with people

see the movie "Her"


how many people will fall in love with Watson?

Let's call it Emma Watson then.

I already do, and don't think I'm alone...

Watson itself was the "wow" moment.

more like the "wow great PR" moment

Most commenters here find these demos very underwhelming and I must agree with them. Well known open source NLP packages like Stanford NLP or NLTK do much better on these (very academic) tasks.

I think the problem is that Watson's demos have been trained for very specific and narrow domains (like "cloud services" in the case of Resonance). What we really want to know is how easy (or difficult) it is to train Watson for our own domain.

If Watson has to make assumptions on my domain, then it will be really hard to build something useful with it.

I am not super familiar with Stanford NLP or NLTK, but I don't believe they overlap with more than two or three (relationship extraction, and maybe language id and some MT) of the services here. The value we want to bring is by providing all these services together in one place and provide tools to do fast domain adaptation. Right now to do adaptation you need to sign up to become an ecosystem partner but we will expose that adaptation tooling to everybody as well in the near future.

I'm by no means an expert, I'm just a linguist that's been learning python and NLTK for a little while. I would assume the demos don't show the potential of the technology. This kind of tech shines when you customize it and train it with domain-specific data, am I wrong?

For all the hype that surrounds this platform, the live demos are truly underwhelming...

Agreed. For kicks I dumped a bunch of dmr biographical material (Wired, NYT, his personal web page) into http://ibmlaser.mybluemix.net/siredemo.html

Some "PERSONS" it found, everything between '' is [sic]:

'C' 'Cobol' 'B begat C.' '”' (yes, a single fancy closing quote) 'engineer' '’ initials' '’70s' '“Jobs’ genius'

Some of it is understandable given how annoying single-character identifiers are to parse and others are roughly reasonable in context.

Even given some of those quirks I bet it would be useful if you had bajillions of pages (cough ediscovery cough) to search through instead of a few general-audience articles.

Yeah it's pretty bad, even inputing a well written news article like this one: http://www.theguardian.com/fashion/2014/oct/08/return-of-joh...

    ”              PERSON
    its            ORGANIZATION
    ’s Wear Daily  PERSON
    shoulder pad   DISEASE
The rest is mostly correct (although random punctuation seems included at times) but it's nothing you couldn't get by word matching using a big dictionary. It reminded me to get tested for shoulder pads though, so there's that.

We have a bug with quotes/punctuation that we are fixing now (that account for 1 & 3). "its" is not necessarily a mistake, it likely refers to the owner. "shoulder pad" is a mistake due to the probabilistic nature of the algorithms used.

FWIW I ran the article through AYLIEN's Text API and here's what I got: http://aylien.com/text-api-demo?text=http%3A%2F%2Fwww.thegua...

(disclaimer: I'm the founder of AYLIEN)

For anyone else who wants to know, 'dmr' in parent post is Dennis Ritchie.

(I knew this, but didn't make the association right away, and it's quite hard to search online for 'dmr')

You should've asked Watson. Then you would have learned the truth: dmr is 's wear daily.

Searching for 'dmr person' does the trick with Google. Top result, and a sidebar of Dennis Ritchie.

Interesting, this service is a lot like http://www.alchemyapi.com/products/demo/alchemylanguage

We have found a bug with character normalization (which messes up the parsing of the punctuation). We'll fix this today.

We fixed the bug with punctuation. Can you try again?

Took me a while but I retried. It's definitely better, to the point that I have to stretch a bit to find really bad errors. "B begat C" still makes it in as do a few similar ones but considering how pathological a case a single-letter identifier is I can forgive that.

I assume because the identifiers aren't unique that they're returned in the underlying api as text locations. Hmm now I want to throw EDGAR data at this.

I played with language recognition demo and learned that Polish is apparently German. Being born in Galicia[0], I guess I shouldn't be surprised...

[0] - http://en.wikipedia.org/wiki/Galicia_%28Eastern_Europe%29

Sorry Polish is not one of the supported language yet: Arabic; Chinese (Simplified); Chinese (Traditional); Cyrillic; Danish; Dutch; English; Farsi; Finnish; French; German; Greek; Hebrew; Hindi; Icelandic; Italian; Japanese; Korean; Norwegian (Bokmal); Norwegian (Nynorsk); Portuguese; Spanish; Swedish; Turkish; Urdu.

French is identified as EN_us -> http://imgur.com/VdDJVw4 Inadmissible !

AYLIEN's Text API does a better job :-) http://aylien.com/text-api-demo?text=vache%20cochon%20metal%...

(disclaimer: I'm the founder of AYLIEN)

I also imagine that distinguishing between many of these languages is not a trivial task for a computer to handle.

You'd get collisions if you're trying to determine the language of a single word, obviously. But a sentence or two should be no problem. You're just matching words in a dictionary.

Different languages are mostly quite distinct. It would be a more interesting challenge to recognize and understand, say, German dialects.

In fact you can already get very good results, even without a dictionary, just with a distribution of the trigrams in your input languages (which is immediate to compute from a corpus). Just compute the distribution of the sample and return the language that has the closest one.

Even a single letter frequency histogram is already a very powerful discriminator to detect language. Although I suppose scoring maximum likelihood on bi- or trigrams works better for very small (sub-sentence) bits of text.

Oh right, I'm sorry. It was actually written on the demo page, but I somehow missed it.

Odd that Farsi is supported, not Polish, given that Poland's GDP is close to Iran's and that Poland has no sanctions against it. Perhaps IBM is planning to sell lots of Watson-powered systems in Afghanistan & Tajikistan?

Incidentally, Poland's GDP is several times that of Finland…

Do you not think there might be a market for software for analyzing text in Farsi perhaps -outside- the region where it's actually spoken?

(disclaimer - work for IBM, but not on Watson, or anything related to intelligence analytics)

Polish as a language is more similar to other languages than Finnish or Farsi. Basically, it's easier to detect Farsi and Finnish because they're more distinct and in smaller language families than Polish, which is similar to many slavic languages.

Do you mind sharing your input text? Interested to see how our API performs (http://aylien.com/text-api-demo)

I don't understand what the heck IBM is up to. They have ads with a bunch of cool icons saying stuff about a smarter future world, but I don't know what to make of any of it. Like, "Wouldn't it be great if the doctor came to you before you were sick?" Um, I guess so?

It just makes me think IBM will never get the consumer the way other companies like Apple and Google do. There is nothing immediately actionable about how I can start using this from the get go. What the hell is bluemix ? Is that an internal thing I am supposed to know about ? How can I start being productive in 5 minutes vs. trying to solve a puzzle by piecing different pieces together ? It all seems very enterprise-y and indirect - an immediate turn off in this day and age of immediate usability and short attention spans.

The ads refer to our healthcare offering. See https://www.youtube.com/watch?v=8lGJ0h_jAp8 for an actual product. The services here are the components that allow us to build such products.

How much time before?

How do these services work, compsci-wise? Could someone link a few IBM publications explaining how the Question & Answer API works? That would be very helpful. The documentation pages explain nothing.

Some of these services definitely draw from the area of Natural Language Processing [1] and Semantic Analysis [2]

[1] http://en.wikipedia.org/wiki/Natural_language_processing

[2] http://en.wikipedia.org/wiki/Semantic_analysis_(machine_lear...

Thanks, however I was already aware of that, I was looking for something more specific.

The research group within IBM that's working on the question and answer portion of Watson is called DeepQA. You can check out lots of information about the team, including publications, at http://researcher.watson.ibm.com/researcher/view_group_subpa...

Thanks for the info.

Good suggestions. Most of these have published papers, we'll add that to the documentation soon. In the meantime you can read an overview of the Watson Jeopardy system here: http://www.aaai.org/Magazine/Watson/watson.php.

Thanks, that would be great.

The language detection failed to work (it identified it as tr-TR) with these words for Slovene: jabolko hruška sliva marelica drevo dež mleko malo preveč majhno veliko

This works perfectly on the first result on Google for "language identification", however: https://open.xerox.com/Services/LanguageIdentifier

Edit: I now see Slovene is not supported, apologies are in order.

Noticed it does not find Frisian, but then again neither does Google. It just gives random guesses every time.

As mentioned in other comments, this language is not yet supported. But it would be better to return "unknown" rather than guessing, so we'll look into that.

One challenge with trying to expose these kinds of services via an API is that much of their utility emerges from a building a model around data of interest. In order to efficiently build a customized capability, one would have to upload potentially large amounts of data to an engine for processing. As a general information service, you ask a question in text, and get an answer in text.

I 'made' this to analyze you HN posts: http://hn.mybluemix.net/

Okay, wow that's pretty impressive, actually.

How did you make this exactly?

The analysis (for myself) seems quite accurate, although I have to try a few different queries (for which I don't have time right now) to determine to what extent it's suffering from the "horoscope effect" (some classifications are quite general and nice to read about oneself, making it more acceptable as accurate).

Hey! So I just used the Watson Node sample project. It had everything ready to go :)

Oops; I typed in "patio11" and then after a reasonable delay got "502 Bad Gateway: Registered endpoint failed to handle the request.".

It came up with mine quite quickly, so guess it spends some time ingesting the data? (I haven't posted very much)

This is interesting!

I copied their own text from the top of the page at http://watson-mr-demo.mybluemix.net/ and selected "Cloud Computing." Then I changed all the words that were grey until they were green.


The Watson Message Resonance service analyzes the popularity of a given word within a specific community. By using this ranking system it is possible to recommend more resonant words to use when crafting messages targeted at specific audiences.


The Watson Message Acceptance service analyzes the popularity of a given word within a specific community. By using this ranking system it is possible to recommend more effective words to use when A/B testing messages written for specific audiences.


Which one resonates more with you? :)

Here's a version modified for "big data and analytics" :-)

The Watson Buzzword Resonance service gives the usage frequency of a test word within a target community. By using this ranking system it is possible to recommend more fashionable words to use when making messages targeted at different customers.

I'm heading up a project that will add a layer of text-based help to online courses (thats the short description...). I would love to explore the Watson APIs more. Aside from diving into the samples here, do you have other suggestions for learning more?

Thanks for the interest Tad. The docs/samples/demos collateral should be a good starting point. If you would like to have a deeper engagement, please consider joining the Watson ecosystem program: http://www.ibm.com/smarterplanet/us/en/ibmwatson/ecosystem.h...

There seems to be a lot of context missing. I can't for the life of me come up with a simple example that demonstrates these services' value to me.

See my response to philbarr above about a retail partner. Here is another example related to recruitment. Often we have a need to hire that we identify with another model employee (e.g. wouldnt it be great, if I had a dev like John from TeamX). With the User Modeling service, we had a Hackathon team create a beautiful app where you can ask questions like these and comparing the characteristics of candidates to John, it can find similar matches (of course done in the context of matching skills)

In the 30 second attention I was going to give, 20 seconds was spent on registering. And then 10s to figure our if there is any demo or any sort.

I couldn't find anything easy to find. I give up. Website is too slow too. And everything shows classic big corp red tapism.

That's odd, I see big links to live demos on each of the services, and no one asks you to register for anything?

For example, click on "view a live demo" here: http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercl...

You don't have to register to see the docs and the demos. Every demo is 2 clicks away from the front page. Registering is free but necessary, as for every other Paas to create your own app. Tell us where is the "big corp red tape" and we'll take it out. We are trying to make the experience as streamlined as possible.

First off, sorry I was rude yesterday. I tried three different demos and all of them were asking for either FB login or something and got annoyed I couldn't try it.

Few questions. I am interested in question and answer service. Is there any related demo ? Specifically, a pre-trained knowledge base. Even better if you can allow anyone to ingest a knowledge into your cloud, train and have it running.

Likely something else going on with your browser.. Demos have no logins whatsoever...

So I can feed Watson public info about someone and it can make recommendations...for travel like the example, maybe preference of beverage, or custom dictionary words/terms that would be associated with this relevant information. Like a custom dictionary customized to an individual with a little alpha/numeric replacement and some special chars thrown in...wait I have to go ask Watson a couple of questions.

Very impressed. I tried out the machine translation live demo, and based on a few samples I gave it, it appears to beat everything else I am aware of.

Thanks! The Watson research team has a long tradition with MT (this is the team that actually invented statistical MT http://en.wikipedia.org/wiki/Statistical_machine_translation) and always achieves top rankings in many of the public competitions. But we had never exposed MT to a broad market. So bear with us while we iron the wrinkles.

I tried the Language Identification service (http://ibmlaser.mybluemix.net/liddemo.html), put in Vietnamese from wikipedia and "en-US" returned.

I did sign up. Note that in incognito mode, I have access to that link. I am guessing others do too.

Here is a link to the docs that should get you up and running with an example application pretty quickly.


For anyone that would like to leave feedback about what we can do to improve the API's feel free to leave your input here. Have at it!:


Looks like there's some Ruby behind the scenes.

``` Error: undefined method `each' for nil:NilClass ```

On bluemix (the IBM PaaS) you can use Ruby, node.js, etc. So they probably chose Ruby as the language for the example.

More details? Which service? What were you doing with it?

happened to me on this page: http://watson-ce-demo.mybluemix.net/

i changed Corpus to "Social Media" and hit "Submit".

i refreshed the page and did the same thing and it worked.

Yep - that's where i got it too.

I tried the Relation Ship demo on the first paragraph of an HN frontpage story :


It obviously has some way to go .

Sorry we have a bug with character normalization which messes up punctuation parsing. That accounts for most of the mistakes here ("its" is not a mistake, it refers to the owner). Here is the fixed output:


Digital Editions ORGANIZATION

others PEOPLE

readers PEOPLE

mother- PERSON







Can I... Can I actually date Watson? Is that in the short-term plan? Anybody knows?

Yes, it is already available. You just have to close your eyes and try very very hard... Make sure you are all positive in your heart, else it fails (we are working to remove that limitation soon)

Interesting. I can see a market for maybe five of these 'Watson' things.

Applications are open for YC Summer 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact