

Ask HN: machine learning success stories? - cageface

There seems to be a bit of a buzz around machine learning these days. The combination of cheap compute clusters and lots of easy available, potentially mineable data from social networking, e-commerce etc seems to present some juicy new opportunities for these techniques.<p>So, are there some good examples of successes with this kind of approach lately or is this the overpromise and underdeliver of AI all over again?
======
bravura
[disclaimer: I'm an optimist.]

I personally think that ML is on the verge of some major breakthroughs.

In particular, I think that results in "deep learning" are very promising.
I've written about this approach in earlier HN comments.

"Deep learning" is the new big trend in Machine Learning. It promises general,
powerful, and fast machine learning, moving us one step closer to AI. Deep
learning has already made important advances in achieving state-of-the-art
accuracy in vision and language, but with _much_ less manual engineering that
competing methods.

In fact, I think the major success of the deep learning movement has been to
get the community to start focusing on figuring out how to get powerful
learning algorithms actually to work. A lot of people used to (but many still
do, sadly) work on making incremental improvements on learning algorithms that
are implausibly simple. "Sure, we know this model can't achieve human level
performance on vision or language or control (robotics) or planning, but with
this neat refinement I can get a paper out of it." Deep learning begins its
endeavor with the goal of AI, and rejects techniques whose upper bound isn't
high enough. Simply the fact that the community is setting its sights high---
not in terms of over-promising to the outside world, but merely in terms of
the learning machinery being explored---and is actually trying to achieve AI
is a step forward.

An algorithm is deep if the input is passed through several non-linearities
before being output. Most modern learning algorithms (including decision trees
and SVMs and naive bayes) are "shallow".

For intuition, imagine if I told you that your main routine can call
subroutines, and your subroutines could call subsubroutines, but you couldn't
have any more abstraction than that. You can't have subsubsubroutines in your
"shallow" program. You could compute whatever you wanted in a "shallow"
program, but your code would involve a lot of duplicated code and would not be
as compact as it should be. Similarly, a shallow machine learning architecture
would involve a lot of duplication of effort to express things that a deep
architecture could more compactly. The point being, a deep architecture can
more gracefully reuse previous computations.

Deep learning is motivated by intuition, theoretical arguments from circuit
theory, empirical results, and current knowledge of neuroscience. Here is a
video where I give a high-level talk on deep learning, and describe this
intuition: [http://bilconference.com/videos/deep-learning-artificial-
int...](http://bilconference.com/videos/deep-learning-artificial-int..).

Here is more detailed information about deep learning:
<http://deeplearning.net/tutorial/>

Another aside: My colleague Hoifung Poon published exciting work in semantic
parsing. It received best paper award at ACL 2009, the most prestigious NLP
conference. (<http://www.cs.washington.edu/homes/hoifung/papers/poon09.pdf>)
You read it and you're like: "Really? You're doing that? You're actually
trying to solve NLP using purely automatic techniques. Whoa. _I forget that
was the goal, I was too busy doing feature engineering!_ "

He achieves impressive results on question-answering, and beats other systems
in recall, giving answers to many more questions, at the same level of
accuracy at the competing methods.

The source code for his semantic parser is available
(<http://alchemy.cs.washington.edu/papers/poon09/>) and you can use it to
build a Q+A system. You can try a demo of it here, which I put up:
<http://bravura.webfactional.com/> He is about to talk about an updated
version of this work, in which he induces ontologies purely automatically.

~~~
cageface
Thanks for the leads. Looks like some interesting reading.

------
physcab
Handwriting detectors implemented by the Postal Service. My ML professor
worked on algorithms with USPS for over 20 years and gave our class a sub-
problem for a term project. He once told us a story that the computer
scientists were all up in arms because they wanted the post office to design
letter boxes on envelopes to encourage people to write their characters in
legible english. Obviously that didn't happen and thus it took quite a bit
longer to design software that would reliably recognize v's from u's and b's
from 6's and r's from t's and c's from a's under the ridiculous constraints
that sorting millions of packages of mail requires.

------
npp
Some fairly well-known consumer-facing things that use machine learning are
spam filters, recommendation engines, speech recognition systems (speech-to-
text or customer service stuff), internet advertising, news clustering (Google
News), related stories, handwriting recognition, questionable content
identification, automatic closed captioning, and machine translation. These
are not all equally successful or sophisticated, but are ML-based and mostly
good enough to use.

There are a number of examples that are not consumer-facing, like credit card
fraud detection, snail mail routing, quantitative trading, market segmentation
analysis, demand prediction for inventory control, and other things. It is
also used for scientific data analysis in several areas, with bioinformatics
being the really big one. There are other examples.

There are also applications that are not considered machine learning, but use
the same ideas for different purposes. An example would be modern codes, which
are used for things like compression and satellite communications, and are
based on the same `graphical models' pervasive in machine learning.

There is hype, and some applications need only a little bit, but it is at
least used in some real stuff.

------
helwr
google: <http://research.google.com/pubs/MachineLearning.html>

LinkedIn SNA: <http://sna-projects.com/sna/>

Microsoft MLAS: <http://research.microsoft.com/en-us/groups/mlas/>

Yahoo ML: <http://research.yahoo.com/Machine_Learning>

ATT & Netflix: <http://www2.research.att.com/~volinsky/netflix/>

------
woodson
I think there is a bit of a hype, and people specifically try to deemphasize
any relation to AI promises. IMHO, many things that were once associated with
statistics are now rather referred to as machine learning, and the range of
applications tends to become very broad.

One could refer to the unsupervised adaptation of new HMM-based speech
synthesis (HTS) voices as a promising machine learning application. It's not a
success story, but imagine being able to create new synthesis voices given
only a few minutes of speech by any person.

Want to try out how well that works? Go to [1] and select the voice "GWB (HTS
2007)", and enter any english text you want. Sounds familiar? And that's just
an academic demo page...

[1]
[http://homepages.inf.ed.ac.uk/jyamagis/demos/page35/page35.h...](http://homepages.inf.ed.ac.uk/jyamagis/demos/page35/page35.html)

------
shib71
FlightCaster (<http://flightcaster.com/>)

~~~
mahmud
I came to this thread to check if Bradford replied. Not yet. Hopefully he will
pitch in.

The uninitiated will benefit from digging into his blog:

<http://measuringmeasures.com/>

Also, the new ML Q&A site:

<http://metaoptimize.com/>

------
_delirium
At least basic machine learning seems to have become ingrained enough in a lot
of areas that it's not really a separate thing anymore, just part of how
people do things, which I suppose is one kind of success. E.g. lots of
business-analytics departments will use things that at least 10 or 20 years
ago would've been considered "machine learning", but now are just part of how
you analyze and visualize data.

------
pgbovine
i think a lot of very lucrative successes are not in consumer-facing
applications. i suggest reading Super Crunchers if you're looking for some
_very lucrative_ successes:

<http://www.randomhouse.com/bantamdell/supercrunchers/>

~~~
jgrahamc
Really? I thought that book was lame.

------
datageek
The best case study I've ever heard is Tescos. Apparently they were a
downmarket British retailer in the early 1990s. They teamed up wit a
statistical consultancy called dunnhumby and now they're the third biggest
retailer in the world.

Apparently one per cent of marketing mail is opened, while 98 per cent of
Tesco mailouts are opened (because they offer people targetted discounts). No
surprise large retailers and grocery chains are rushing the adding significant
ML capabilities.

------
tel
IF you round ML into "computerized statistics" you'll find endless success
stories in scientific research. Particularly interesting might be f/MRI image
analysis

------
drlisp
Bayesian spam filtering - machine learning 101

------
drlisp
Also clustering algorithms (k-means et al) used to do a preliminary analysis
of NMRI scans

------
dennisgorelik
Spam detection: [http://postjobfree.blogspot.com/2009/06/postjobfree-
automode...](http://postjobfree.blogspot.com/2009/06/postjobfree-
automoderator.html)

------
thesethings
Though techmeme is guided a by a human hand, I'm pretty sure much of its power
and intelligence is via machine learning (for determining related stories, and
their basic order).

