
Welcome to the New AWS AI Blog - phodo
https://aws.amazon.com/blogs/ai/welcome-to-the-new-aws-ai-blog/
======
niklasrde
We have recently set up a TensorFlow assessment function in AWS lambda, and
got _very_ close to the maximum allowed size of a lambda function (250MB) with
the trained model currently being 85MB, and the TensorFlow libraries and
binaries taking up another 140 or so megabytes by default (<\- which we
cracked down a bit, but that's quite a hack).

I feel like Amazon could do some work in this area to support users to use
their own engines and not be bound to AWS AI Platforms and Services.

This could be as simple as publicly documenting the time lambda's stay 'warm'
for and retain data in /tmp persisting through multiple invocations or some
other examples on how an AI workflow could be implemented with popular custom
engines such as TensorFlow.

Does anybody else have any experience in this regard?

~~~
tcas
I've hacked around lambda quite a bit (I think the compressed size of one
function is a tad under the max allowed). My hacks I remember are:

\- Run strip all .so libraries -- many aren't stripped fully

\- In Python I manually deleted sub packages of numpy/scipy I didn't need

\- If you're loading large models at initialize, numpy load routines are
_much_ faster than cPickle. Have it load at module initialization, not during
each invocation.

I should really write a blog post about my experience with it.

At a certain point I decided I was doing something that lambda really wasn't
designed for -- I'm looking at migrating off, but the current implementation
makes capacity planning super easy. Provisioning 1000 machines with 1GB of RAM
for 15 minutes every day to read off a queue isn't a trivial problem.

(Also, if anyone from AWS is reading, being able to limit the max concurrency
of a single function vs account level limits would be super useful).

~~~
jedberg
Just to throw this out there as another possible optimization, if you find
that you're putting a big fat library into every function, one possibility is
to run the library as its own lambda function. You'll be slowed down a bit by
the network but it might be made up for by not having to constantly initialize
the same thing.

------
nielmalhotra
Quick slightly unrelated question: Does anyone have a comparison of using
Google cloud services vs AWS for machine learning? I'm planning to pick one,
and I was leaning towards Google Cloud Services because of the TensorFlow
support and the fact that Google is big on ML, making it likely that it's
something that Google will support and be good at. With this blog post, I'm
not sure.

~~~
boulos
(Disclosure: I work on Google Cloud).

Both providers offer you raw VMs with GPUs and such so you can run popular
machine learning frameworks yourself by hand. After that the three providers
diverge a bit, and I've not seen a good writeup myself. Roughly:

\- Google has both a hosted TensorFlow (Cloud ML) as well as specific, pre-
trained models you can simply use (Cloud Vision, Cloud Speech, etc.). For an
easy to use interface, we have direct TensorFlow (and more) integration in
Datalab.

\- AWS also has some pre-trained services (Rekognition, Polly, Lex) but for
"obvious" reasons doesn't do hosted TensorFlow. Instead Amazon Machine
Learning is a bit more like Azure's offering: "Put data in, wire up stuff in
the console and hit go".

If you're really interested in ML, my biased opinion is that you'll be using
TensorFlow. And as you surmised, we're committed to making TensorFlow the
"best" ML framework and making sure it runs well on Google Cloud. Like
Kubernetes, we're not going to handicap it elsewhere, but having it managed
and accelerated for you, is extremely convenient.

[Edit for formatting. I also should have mentioned there will be lots of ML-
related talks at NEXT in San Francisco in two weeks!].

~~~
mza
GM for AI at AWS here: actually - we like TensorFlow quite a bit, too.

We provide a machine image with TF, MXNet and others pre-installed, along with
Keras, CPU and NVIDIA divers, and other libraries for deep learning. We just
added Ubuntu support too:

[https://aws.amazon.com/blogs/ai/the-aws-deep-learning-ami-
no...](https://aws.amazon.com/blogs/ai/the-aws-deep-learning-ami-now-with-
ubuntu/)

~~~
MrQuincle
The challenge is always to get a lot of training data.

Are there (artificial) datasets that can be used that showcase particular fit
for deep learning?

I think there is way to little research in building artificial datasets (using
domain knowledge of course).

It might even be possible to run these generative models and have this type of
data very soon.

~~~
boulos
ImageNet, GoogleNet, etc. are all image datasets for precisely this purpose.
There's also the recently announced YouTube dataset and Kaggle challenge [1]
and Google Research's datasets [2].

I agree though, the kind of _artificial_ / play-against-yourself datasets that
the folks at DeepMind created for say Alpha Go are an entirely different
beast.

[1] [https://cloud.google.com/blog/big-data/2017/02/google-
cloud-...](https://cloud.google.com/blog/big-data/2017/02/google-cloud-and-
youtube-8m-challenge-predict-youtube-video-tags-for-a-chance-to-win-up-to-30k)

[2] [https://research.google.com/research-
outreach.html#/research...](https://research.google.com/research-
outreach.html#/research-outreach/research-datasets)

------
techdragon
I find it frustrating for all the power they want to give me... that some
basic service design is lacking.

Polly is a stand alone component but the reverse is closely bound up into Lex
which is a conversational interface API.

Amazon has internally built an engine I could ask to convert an audio file in
S3 into a text content representative of the audio file... yet I can only use
Lex to drive a conversation via text and audio.

If AWS really want to give me the power of their AI tools. How about
unbundling them?

------
mark_l_watson
Very impressive. I am working in a cognitive computing book, and I am going to
add a chapter or appendix on Amazon AI. A little off topic, but even though I
self-classify as a Google fan and very much enjoyed working there as a
contractor, when a friend asked once which technology company impressed me
more I said Amazon.

A big win for Amazon is that so many companies already have huge data sets in
S3. Having AI APIs 'close to' existing data makes it easier getting started.

------
starik36
I am a complete noob to the AI space but I was wondering whether the following
is possible (in AWS).

I have a million scanned images of court documents. Some are briefs, some are
motions, some are court orders, etc... Given that I have images and their
types, could I "train" the AI with these million documents to recognize a new
image that might come in?

~~~
bhaavan
Principally, yes. However, the approach may be more nuisanced than that. If I
were you, I would first pick a character recognition engine (which might have
already been well trained) to convert the image to text. Once the text is
there, that might serve as a better feature to classify the content.
Furthermore, I had recommend converting words in the text to word-embeddings/
vectors using a suitable Glove or Word2Vec dataset similar to your content.

While there are many benefits of end-to-end training, I don't think it might
be best suited for this case. This is because we already know that the only
useful feature in the document is the text, and not the contours or textures.
Theefore wasting neurons in your neural network to learn the wastefulness of
this features is just waste of resources. Furthermore, you benefit from even a
larger corpus of learned data, which the char recognition engine has been
trained on.

~~~
starik36
OCR is my current approach. I am not really happy with it. The quality of
OCRing leaves much to be desired, probably due to the documents themselves
being haphazardly handled by the court personnel. OCR itself is a pretty CPU
intensive activity and takes a significant time to complete for many
documents.

Thus, I was looking for a more advanced approach.

~~~
jononor
What do you want to recognize?

OCR is a better understood problem than a general neural net, so I think it's
likely easier to improve its quality that to superseded the quality with
image-based recognition.

~~~
starik36
Ideally, I would like to get all the information from the page. Phone numbers,
who is suing whom, case caption, etc...

With OCR, you get bits and pieces of information, but because I don't know
what the type of the document is, it is difficult to determine where,
structurally speaking, this information resides on a page.

If I could use AI to determine the type of the doc, I would know the structure
of the document and I could then use OCR to pinpoint specific information on
the page.

Most court documents are created from templates.

~~~
jononor
If your OCR is unable to recognize the characters on a full page, I don't
think it when scanning a region either? Unless using full resolution of a full
image is somehow too much for the algorithm in use. But then I'd just
subdivide the entire image into regions, and scan them all independently. This
is also a trivially parallelizable task, so you can throw many servers at it,
if the time to get results is an issue.

------
LeicaLatte
Production ready AI services are few and far and but Polly is up there. I am
currently using it in a workflow as part of a IVR front end. Seeing good
results.

~~~
mza
Awesome - email me if you need any help: matthew@amazon.com.

------
campbelltown
Do we really need a Mark Cuban quote in there though.

~~~
LAMike
Dude he put radio on the Internet, show some respect. Plus, his doors open
like ___this_ __

------
saosebastiao
I know ML is the big cheese right now, but doesn't it seem like a bad use case
for the cloud? Consider:

1) Training ML models does not require network access, which is one of the
biggest competitive advantages of the cloud.

2) Training ML models is typically a batch process, which benefits minimally
from the scale-on-demand model of the cloud.

Since the cloud premium is a significant exchange for the value that it adds,
I don't see this being a big win for cloud providers. I can't help but think
that if I were making use of extensive machine learning with continuous
training, I'd have it training models on a local bare metal cluster statically
scaled to my application's demand with minimal network connectivity needs. And
then ship the serialized trained models to the cloud. The potential cost
difference is huge.

~~~
btown
> does not require network access

> benefits minimally from scale-on-demand

Neither of these is true when dealing with terabytes of data (or more, if
you're working with image/video corpora). Many AI/ML problems have stages that
are trivially parallelizable - if you can divide your problem into iterations
where a subgraph of nodes communicates internally, then sends/receives updates
to other subgraphs, it's very similar to an iterative map-reduce algorithm,
perfect for networked cloud systems.

And as you're tuning your hyperparameters, you don't know what the performance
characteristics are, and you will absolutely want to run experiments in
parallel, until you find the right settings that you'll use in production.
You'd need to invest in a LOT of redundant bare metal to have that capability.
As Netflix puts it in this presentation, the key to effective machine learning
is iterating often, and that means having a lot of parallelism to bring to
bear. [https://www.infoq.com/presentations/machine-learning-
netflix...](https://www.infoq.com/presentations/machine-learning-netflix-2014)

~~~
saosebastiao
Good point. I've worked with lots of local exploratory machine learning (R or
Spark on a single workstation), and lots of stable applications that use
machine learning processes that are no longer fiddled with, but I've never
witnessed the transition between the two.

------
kriro
How should I interpret their picture? Can I get an AMI with Keras
preconfigured on a p2 instance? Because that would be pretty useful. I
currently have a p2 instance (smallest possible one) that I spin up for
training and the like.

~~~
Rezo
Yes, the AWS Deep Learning AMI comes with MXNet, TensorFlow, Caffe, Theano,
Torch, and Keras installed and ready to use:

[https://aws.amazon.com/blogs/ai/the-aws-deep-learning-ami-
no...](https://aws.amazon.com/blogs/ai/the-aws-deep-learning-ami-now-with-
ubuntu/)

~~~
kriro
Thanks, somehow never saw this. I guess this can replace the scripts from the
"Practical Deep Learning" series for me then. I'll just use the vanilla Amazon
Deep Learning AMI + a p2 instance :)

Quick Heads up in case anyone wants to do the same. The only European region
that supports p2 at the moment is Ireland (I tried Frankfurt at first).

------
neom
Has anyone used Rekognition? We're thinking about pumping traffic cam feeds
into it in cities for vehicle counting but don't want to waste time if it's
junk.

~~~
ranman
If you want to email me I can connect you with the rekognition team: randhunt
at Amazon dot com.

You cannot currently count objects other than faces and bounding boxes for non
face objects are not currently provided.

For the time being existing solutions in the opencv/tensorflow/MXNet realm
might provide more of the result you're looking for.

~~~
neom
Hi Randall, long time no chat! Hope you're doing well. Good to know, it's not
really our core product mission right now, so we're not at the point of
building something (like a tensorflow implementation). I figured it
rekognition has what we needed out of the box we might just clip it on and see
how it performs, but wouldn't bother if it's not there yet, I'd imagine over
time it will improve. Thanks.

------
avoutthere
I want to know if any of the pictures from my trail camera include deer. It
sounds like Rekognition might be the answer for this.

~~~
dharma1
Does your trail camera capture video or image sequence? You probably only need
to analyse 1 frame every few seconds or less if you just want to get deer
alerts.

I would go with a DIY solution, since Rekognition will get expensive quick
($1/1000 images). If you did 1fps running continuously, you'd pay $87/day.

Nearly any ML framework with an ImageNet pre-trained model will work. Accuracy
will be better if augmented with more labeled deer images (ImageNet probably
only has frontal poses) but should be alright straight out of the box.

You could run the whole thing locally on a cheap android phone for free
instead of AWS cloud pricing, 1fps will be fine with TensorFlow or MXNet on
Android. Then just setup email alerts.

------
jmngomes
"Mark Cuban recently talked about it as the most important technology to ramp
up on, to avoid becoming a “dinosaur”"

I wonder if this impresses this blog's audience, or does exactly the
opposite...

~~~
dominotw
does cuban actively invest in A.I startups? wondering what makes him a
quotable authority on the topic.

~~~
kyledrake
> wondering what makes him a quotable authority on the topic

His wealth.

The obsession with cyber-malthusianism among the owners of tech informs me
more about their valuation of humanity than about the future of tech.
Fortunately there's a more level headed analysis that was published by the NYT
recently for those of us that aren't consciously or subconsciiously all-in
about the idea of making Snow Crash non-fiction
[https://www.nytimes.com/2017/02/20/opinion/no-robots-
arent-k...](https://www.nytimes.com/2017/02/20/opinion/no-robots-arent-
killing-the-american-dream.html)

------
ge96
The one time I actually want to subscribe/follow something there's no "Hey
would you like to give me your email address?" popup... hmm

~~~
kiliankoe
The link to the RSS feed is right down at the bottom ;)

~~~
ge96
I did see that, but that RSS feed (atom?) I don't think I've ever used that in
my life. I'll Google and see what/how to use it. I see it everywhere though.

edit: I did see that they have a podcast, nice, something to listen to while I
walk 2 hours in the middle of the night.

~~~
dankohn1
Ughh, you're making me feel old. RSS is (was) an amazing system for tracking
website updates over time. It lost out to Twitter and Facebook (which don't
remotely replace the use case), though it is still what makes podcasts work.

I highly recommend Inoreader as a great RSS reader.

~~~
ge96
Thanks for the recommendation. I tried one, I am not quite sure how it works
yet at least for that AWS one. One page was a full-article, another page was
several articles in one, it was odd but thank you.

Regarding Blogger back in like 2011 or so, I saw RSS feed all the time at the
bottom (usually still do).

~~~
dankohn1
Running a partial text RSS feed through here will often replace it with a full
text RSS feed: [http://fivefilters.org/content-
only/](http://fivefilters.org/content-only/)

