
AWS Lambda - cpenner461
http://aws.amazon.com/lambda/
======
jedberg
I’ve had a chance to use the service for a couple of weeks. My quick summary
review is that it’s a little tricky setting up the IAM roles and security
groups, but once you have that going, it works great! I see a ton of potential
here in transforming the way people use AWS.

I also put together the Netflix use cases in the keynote so if you have any
questions I’ll try to answer them!

~~~
joshuahutt
One thing I am struggling to understand is how much of my applications can be
replaced with Lambda.

I think it would be really awesome to make applications that do not have any
dedicated instances associated with them, but are entirely based on the
various AWS platforms.

For some things, it seems like Lambda could be a drop-in replacement, whereas
others, it would not quite fit the bill. Do you think it will be possible to
build a viable a "server-free" application with Lambda and the other services
(S3, DynamoDB)? Or is it probably going to be limited to the types of use
cases listed on the landing page[1]?

[1] [http://aws.amazon.com/lambda/](http://aws.amazon.com/lambda/)

~~~
krallin
It seems like you're limited to 25 concurrent lambda functions executing (at
least at this time).

This looks like it's more geared towards enabling you to automate /
orchestrate your AWS resources, that run customer-facing code.

~~~
jedberg
The 25 function limit is probably one of those small defaults that's easy to
change, like the 20 instance limit.

~~~
samstave
True, its likely just like all other limits; arbitrarily small for anyone by
default to prevent you from having an errant process spin up a ton of
resources that you'd be expected to pay for.

Just like how you can get specific increases on your spot bids so long as you
acknowledge that if you get a spot spike, and your personal limit is far
greater than the spike -- you'll be charged your rate.

------
MBlume
Right now the only language/runtime supported is js/node, but they intend to
include others.

(figured people would want to know this, and you have to scroll a ways to find
out, so)

~~~
notastartup
I saw it and was shocked that Python is not yet supported.

~~~
crucialfelix
Most of the Python things I would want to do have slow startup times.
Generally I keep my pythons running. But node is super quick to start.

~~~
mjfisher
That's an interesting perspective - what's a slow startup time in this
context?

A quick back-of-the-envelope test on my machine for both node and python
suggest similar (around 100 ms) start up times for the bare environments.

Do you just mean that you'd prefer to use node for smaller jobs anyway?

~~~
crucialfelix
Django is the problem. Even if I don't trigger the database to connect, the
startup costs are noticeable to validate models and load things that won't
ever be used.

so maybe lambda its not suitable for things that are less than a few seconds.

I could write python with no django and try to structure things so they don't
need database. but then I would have to take input from and put output data
back to a queue of some sort.

>> A quick back-of-the-envelope test

and actually python on mine was even faster than node when both are doing
nothing. so my assumption was wrong (but its based on mostly using django and
all my node stuff is small servers and tools)

» time node noop.js node noop.js 0.04s user 0.01s system 98% cpu 0.045 total

» time python noop.py python noop.py 0.01s user 0.01s system 90% cpu 0.026
total

but importing significant numbers of libraries would be the real test.

the main benefits to lambda would still be:

\- scaling for high demand \- no cost for idle time

and let's not worry about the micro pennies in startup times

------
makmanalp
I'm so torn - on one side this is a very neat thing that'll save a lot of
boilerplate, and on the other it screams of vendor lock-in.

~~~
vmarsy
as MBlume mentionned[1], they seem to support NodeJS.

From the FAQ :

 _" At launch AWS Lambda supports code written in Node.js (JavaScript). Your
code can include existing Node.js libraries, even native ones."_

so it seems that you can avoid lock-in if most of your javascript code is not
too coupled to Amazon's API

And they claim on the detail page:

 _" With AWS Lambda, there are no new languages, tools, or frameworks to
learn. You can use any third party library, even native ones. At launch, AWS
Lambda will support Node.js code, with support for other languages coming in
the future."_

[1] :
[https://news.ycombinator.com/item?id=8603059](https://news.ycombinator.com/item?id=8603059)

~~~
mooreds
Err, I think the parent's point wasn't lockin in terms of the code you write,
but in terms of your architecture. Setting up your own system driven off of s3
or dynamodb events (if you wanted to move the functions onto your own bare
metal) would be a pretty big project.

~~~
justincormack
ZeroVM lets you run code on Swift, Openstack's S3 clone. Although it is not
evented, and they mainly support Python, so it is a way off at present.

------
spitfire
I wish they hadn't named the units of computation "Lambda functions". Cause,
you know there's already something known as a "Lambda function" in computer
science.

But kudos for Amazon for furthering the datacenter-is-the-computer approach.
It is simply the right thing to do.

~~~
erichmond
I'm guessing they borrowed it from this : [http://lambda-
architecture.net/](http://lambda-architecture.net/)

~~~
turingbook
The Forbes article said it is named after Star Wars space craft:
[http://www.forbes.com/sites/benkepes/2014/11/13/more-
amazoni...](http://www.forbes.com/sites/benkepes/2014/11/13/more-amazonian-
announcements-aws-anoints-docker-and-makes-a-nod-towards-star-wars/)

But I think it is simply named after the functional language feature. The core
concept of this service is function.

------
marknadal
Holy mind blowing awesomeness, this changes everything, yet I feel like this
was such an obvious thing to do. So obvious that I can't believe it is real.
This, ladies and gentlemen, is why AWS defines the cloud, they are so far
beyond everyone else and still innovating, even and the most basic level.

~~~
erichmond
This idea has existed for years (see storm, samza), but it's great AWS is
wrapping the infrastructure piece for those who can't roll their own!

------
cperciva
Something I want: AWS Lambda code which responds to S3 PUTs by issuing
Cloudfront invalidations. Anyone want to write this?

~~~
ncallaway
I imagine it would be very with the S3 Event Notification stuff they also
released today: [http://aws.amazon.com/blogs/aws/s3-event-
notification/](http://aws.amazon.com/blogs/aws/s3-event-notification/)

~~~
cperciva
Yes, exactly. Someone needs to write the code, that's all; I'll do it myself
if necessary, but if someone else gets there first all the better.

~~~
donavanm
Watch out for the limit on the number of concurrent invalidation batches.
Youll probably want to scan for "pending" PUT objects and try aggregate them
in to an invalidation batch.

~~~
cperciva
Good point. I guess this would need to be "S3 PUT event triggers code which
adds object name to a list and starts a timer"; "timer expiry triggers code
which issues a batch invalidate for the list (and empties the list)".

I wonder if there's any good way to generate that timer expiry event...

~~~
donavanm
Hrm, I havent gone through the docs completely but object lifcycle managment
on the sentinel/list object?

------
tomcart
Seems like a much more interesting announcement than the container service.
Can see the architectures of dozens of our systms collapsing down with this.

~~~
derefr
At a guess, Container Service was built in order to build this. I.e. Lambda is
probably an ECS cluster creating and running Node.js-based Docker containers.
(A bit like Heroku's original Ruby stack, actually.)

It's a bit amusing to note that they could have built Lambda on top of their
own previous Elastic Beanstalk Docker offering instead, but chose to build ECS
presumably for cost reasons (which everyone who has evaluated Elastic
Beanstalk's Docker support will understand.)

------
mathgladiator
This is way cool especially considering S3's event notification will enable a
ton of interesting workflows:
[https://news.ycombinator.com/item?id=8602955](https://news.ycombinator.com/item?id=8602955)

------
numlocked
Should I think of this essentially as an abstraction that can replace my
asynchronous task queue + workers (e.g. RabbitMQ + Celery workers, obviously
assuming you aren't using MQ for additional messaging)? I hate managing those
pieces and would be happy to hand that infrastructure to Lambda, but are there
additional benefits or use cases that are opened up?

I guess I would have expected others to describe this the same way ("replaces
your distributed task queue"), but since I'm not seeing that description I
wonder if I've misunderstood.

~~~
jberryman
I guess maybe part of what you might be missing is that these "Lambda
functions" can (will be able to?) be kicked off by all sorts of different AWS
events, so it can replace queue + workers where you're generating tasks from
your own code, but it might also be able to replace the code that generates
those tasks too.

~~~
numlocked
Now that you say it, I'm sure we'll see wrappers to support this type of
thing. For instance, I could imagine a django library that provides a
declarative way to wire Django signals to AWS Lambda functions.

lambda_signal(Model, signal_name, lambda_fn_name)

When that signal fired on the model, the model would get serialized to JSON
and sent to the lambda function. It'd be fun to whip this up if no one beats
me to the punch.

lambda_signal(User, 'post_save', 'sync_to_mailchimp')

------
hackerews
You can rip out quite a bit of logic into separate services with this.

Will definitely be interesting to see how Lambda actually works.

Try out Blockspring, you can play around with your usecase now (and no vendor
lock-in).

------
dk8996
Is it me or AWS is releasing too many services... there is a service for
everything. I wounder if they are just throwing stuff out there see what
sticks... kinda like landing pages.

~~~
poxrud
To be fair, unlike google they are not cancelling any services. I think Amazon
first builds these services for internal use, and then releases them to the
public.

~~~
Apoplectic
To be fair, you have to pay per unit for these services, so as long as people
are using them, Amazon is making money. Putting a service out to pasture in
maintenance mode probably wouldn't cost that much.

~~~
mentat
Maintenance on all of their services is non-trivial in terms of people at
least. However if they are used internally, directly or indirectly then
they're already paying for it.

------
jameshart
Claiming that your code is a 'lambda function' makes it sound sexy, but..
isn't it really just a procedure? Unless I'm missing something and there is
some higher-ordered capability for composing AWS lambda functions together in
a way that permits the platform to perform lambda reductions or optimize
resource allocation...

~~~
gbrits
I guess the term 'Lambda' here is taken to mean 'a function without side
effects'. So no higher-order chainability, etc. afaik.

~~~
jameshart
Except that it can have side effects, can't it - it can trigger services,
cause stuff to be changed in a datastore. It doesn't have any local stateful
storage, but it can make use of remote state. In that respect, how is it
different from basically any web request processor?

------
debaserab2
I'm excited for this. This replaces what I wanted to use SQS for. SQS always
felt like too much vendor lock-in to me to justify not using something like
RabbitMQ or Beanstalkd.

With Lambda, the resource consuming the queue is managed for me - that is
huge. Also, the pay by time model is perfect for this - instead of managing
when to stop/start resource intensive instances, I don't even have to think
about the problem. I only get charged what I used, which can be crucial during
growth stages of a business or prototype.

The big penalty is the vendor lock-in, but this tips the scales for me for
certain things.

~~~
crescentfresh
> *the resource consuming the queue

Are you referring to kinesis as the queue here? Sqs/sns are not (yet?)
supported as event generators for lambda.

------
turingbook
Werner Vogels's blog about Lambda:
[http://www.allthingsdistributed.com/2014/11/aws-
lambda.html](http://www.allthingsdistributed.com/2014/11/aws-lambda.html)

------
cschmidt
> You can use any third party library, even native ones. [1]

I realize they are starting with node.js, but I wonder how this will work? It
sounds like they plan to support arbitrary dependencies. Will you upload a
Docker container with your necessary dependencies? They talk about
milliseconds until your code is running, and containers may be (?) slower than
that. Or am I hoping for too much.

[1]
[http://aws.amazon.com/lambda/details/](http://aws.amazon.com/lambda/details/)

~~~
tekacs
It's definitely possible to do this if you're not set on containers.

We do it at [http://wym.io/](http://wym.io/) by putting our runtime library in
the environment that the code runs in and keeping the runtimes around, keeping
the environment clean along the way.

We're working towards doing all that (and more than) Lambda does in a deploy-
your-own fashion...

------
amelius
One difficult part of doing event triggered processing is in the progress
reporting and keeping the code related to it simple. I wonder how they deal
with that.

------
ColinCera
I'd pretty much given up on AWS for compute and moved most everything to
Linode and some bare metal servers, but this service looks very compelling for
discrete compute tasks.

The ability to pay only for the fractions of seconds actually used, and the
ability to scale quickly without provisioning (or over-provisioning) EC2
instances, is awfully attractive.

Plus, Amazon has priced this pretty aggressively — i.e., it looks shockingly
cheap.

------
jlrubin
How is this different from CGI?

------
rajatchopra
Good stuff. Basically it seems to do the bind+listen for you if you are the
trigger subscriber. If you are the trigger generator, then it does
socket.write for you. But the big deal is that you dont pay for 'listen', just
pay for the function execution. The one thing that will surely happen with
this is that the code written will be 'locked in' to run only on aws
territory.

------
dkarapetyan
All these announcement are making me feel sorry for all the other players in
the cloud game. The other guys don't even come close.

~~~
rodgerd
I feel sorry for all the people in 5 - 10 years who discover Amazonis now in a
position to charge monopoly rents.

~~~
Apoplectic
And yet, they don't. Compare that to Oracle. Plus, when Amazon makes a new
cloud architecture that gets popular, others copy it. OpenStack will have this
if need be in a year or two and then you can port to OS in 5 years when the
rent goes up.

Oh, and FWIW, when has AWS raised a price? My servers have gotten cheaper
every year I've had them with AWS (3).

~~~
epowell101
Regarding OpenStack and something similar, there is the underlying workflow
service (not quite the same I know) called Mistral. A project called
StackStorm that is related but adds event handling and a rule engine launched
a couple weeks ago.

------
JoshTriplett
This looks quite interesting, and a lot more fun to work with than maintaining
a pool of servers ready to handle events and spinning up new ones based on
capacity.

Anyone know of any similar mechanisms for the OpenStack world, or more
generally for any cloud infrastructure other than AWS?

------
hendry
I tweeted that it's a PITA to transcoded uploaded media yesterday and today
AWS solve the problem!

I HAVE THE POWER

[https://twitter.com/kaihendry/status/532783437225025537](https://twitter.com/kaihendry/status/532783437225025537)

------
hcarvalhoalves
Full-circle back to mainframe era.

------
adelevie
Is this comparable to IronWorker?

~~~
carimura
[I'm with Iron.io]

Similar, yes. We're excited that developers are seeing the power of the
task/worker/lambda being the scalable unit of work rather than server or VM or
even container.

Haven't used Lambda much yet but off hand we see many IronWorker advantages:

\- Supports all languages \- Easily scale to thousands of parallel workers \-
Integrated logging, error reporting, alerts, auto-retries on failure \-
Integration with IronMQ our queue service to achieve stream processing and
pipelining \- A great dashboard to manage your workers/tasks

And many innovations/announcements coming soon that'll make using IronWorker a
great choice for developers both on AWS and inside the enterprise firewall.

~~~
jakemcgraw
I recently used iron.io to process around 750K images (generating pHashes for
duplicate detection). The service is fantastic.

------
james33
It sounds like this is essentially what Joyent's Manta is, which we've been
using in production for the last year and have found to be absolutely
fantastic. Are there differences that I'm not seeing?

~~~
nivertech
It's not. The name is confusing. Especially when you hear AWS Lambda and
DynamoDB/S3 in the same sentence.

------
riobard
How is it different from Google App Engine? Conceptually the two seem very
similar to me, that is, developers do not have to worry about the underling
infrastructure at all---just write code and deploy.

~~~
willyg302
As a tldr, not-quite-right distinction, GAE is more of a platform whereas
Lambda is more of a service.

I feel that GAE is a much more generalized solution for building/deploying
applications, e.g. it's very integrated with BigTable and has
caching/cron/queuing support. To a point you do not have to worry about the
infrastructure, but write any GAE app at scale and soon you'll run into
questions of caching some content and optimizing DB writes.

Whereas Lambda, at least right now, is a more specific use case: respond to an
event by running some code. It's too early to tell whether managing Lambda
functions will require fine-tuning the "advanced settings" (in all honesty it
probably will) but it seems like it's much simpler.

------
hyperliner
I finally have stored procedures and triggers for my DynamoDB database!

------
luminati
Haven't had much time to read the docs. Sorry if it's already evident, but
does it allow for running Lambda code on cron as opposed to listening to some
event?

~~~
numlocked
Yes. From the bottom of TFA:

Scheduled Tasks

AWS Lambda functions can be triggered by external event timers, so functions
can be run during regularly scheduled maintenance times or non-peak hours. For
example, you can trigger an AWS Lambda function to perform nightly archive
cleanups during non-busy hours.

~~~
luminati
That's great news. Thanks mate.

~~~
Rapzid
Reliability is my real question around not only this, but lambda as a whole.
I'll need to dig into the docs on that.

------
impostervt
I love that it uses Node.js to start, but does it support NPM?

~~~
impostervt
Looks like yes:
[http://docs.aws.amazon.com/lambda/latest/dg/walkthrough-s3-e...](http://docs.aws.amazon.com/lambda/latest/dg/walkthrough-s3-events-
adminuser-create-test-function-create-function.html)

Now, how to deal with node packages that require libraries not on the default
amazon linux ami?

~~~
bradyd
From that example it looks like it doesn't support NPM directly, instead you
download the dependencies locally and include the node_modules directory in
the zip file you upload. Interestingly the example makes use of ImageMagick,
which I don't believe is in the default AMI.

------
motoboi
AWS Lambda announcement video: [https://www.youtube.com/watch?v=9eHoyUVo-
yg](https://www.youtube.com/watch?v=9eHoyUVo-yg)

------
nivertech
Does AWS Lambda supports cron-like periodic triggers?

~~~
epowell101
Yes.

------
Zaheer
This reminds me of [http://www.Webscript.io](http://www.Webscript.io)

Similar single event -> function service

------
jbaudanza
It would be nice if a lambda could respond to web requests. Maybe an Elastic
Loud Balancer could be an "event-source".

~~~
tekacs
We support exactly this[1].

[http://wym.io/](http://wym.io/)

Incidentally I almost had a heart attack when I saw the Lambda announcement.
:P

[1]: We also provide the ability to respond to other kinds of events as does
Lambda.

------
dominotw
Would this be useful for build services like setting up a jenkins instance
using lambda compute resources as slaves?

~~~
willyg302
Using Lambda for the slaves is probably not the best idea because, as @saynay
mentioned, you couldn't run builds any longer than a few minutes.

You _could_ "wrap" a Jenkins fleet running on EC2 with Lambda functions to set
up on a build request event and tear down on a build finish event, and
theoretically this would reduce the amount of resources you consume if your
builds are few and far between. Think of GitHub webhooks triggering Travis CI
as an analogy.

Whether or not that would actually be useful is debatable.

------
davidw
From a casual glance: it's kind of like Heroku but with Node.js, and it scales
automatically?

------
Apoplectic
Huh? Shared server infrastructure? That's really what this sounds like.
Welcome to web hosting in 1999 guys. Most of the point of AWS was that you
have your own dedicated resources. Sure, this is a scaling solution, but
revolutionary?

~~~
ceejayoz
> Welcome to web hosting in 1999 guys.

Sorry, where were you getting servers in 1999 that existed for fractions of a
second?

~~~
Apoplectic
Ain't a 'server' boss. They're running a segment of code. When someone hits
your PHP page on an oversubscribed DreamHost box with 4000 other sites
contending for resources, it's pretty much the same thing. Here we're getting
some dedicated CPU/Mem, but it ain't a 'server.' Where's the dedicated disk
and ability to install/run whatever I want? Nope, just a lil segment of JS
running on a shared resource and billed in 100ms increments.

------
salimmadjd
Amazon Echo (voice interface of IoT) + Amazon Lambda (the cloud services of
(IoT) = Amazon disrupting home appliance IoT products.

------
GIFtheory
Sounds a bit like Mirage OS to me
([http://www.openmirage.org](http://www.openmirage.org)).

~~~
yohanatan
Nope, nothing at all like that.

------
squidcactus28
How to get this on private cloud?

------
sly010
IaaS -> PaaS slowly but surely

------
waitingkuo
How can it compare to zappier?

~~~
epowell101
Or StackStorm? Or [https://factor.io/](https://factor.io/) I think much much
more intra application wiring as opposed to inter app / tool chain wiring. Or?

------
notastartup
What about dependencies. What if you need a specific environment setup first
in order to process. Would you end up paying 1 minute for each request just so
that it can start installing bunch of stuff? Is it possible to just setup a VM
of some sort and use that environment each time?

If thats possible Lambda would be like PiCloud but without Python, and will
stick around (hopefully).

