
Writing Cloud Functions - smithclay
https://cloud.google.com/functions/writing
======
encoderer
At Cronitor we've just finished building out lambda infrastructure for a fan-
out where I would otherwise use a worker application and supervisord. Some of
this is lambda specific but I think it might be interesting nevertheless.

Lambda has a lot of strengths, and it gives you solid primitives like
immutable versions, version aliases/pointers and stateless functions. The
problem I had was that it didn't give me any more than that. Here are just a
few of the things we had figure out and then build ourselves:

1\. The development workflow. How can I get a REPL workflow going that doesn't
make me go crazy.

2\. Deploys and Rollbacks. How can we safely deploy and rollback, especially
in cases where you have the same lambda function in multiple regions and each
region is at a different published version (because the version number is
monotonically incrementing)

3\. Permissions and Policies. The broad strokes are clear. But you want your
S3 bucket or SNS topic to trigger lambda? Get ready to spend an hour trying to
figure out what you've done wrong and what's missing from the vague
directions. Hope you get that REPL flow solid first. In the end, we built
several scripts (create_build, publish_version, promote_to_prod, etc) and we
use these directly during development and from a Fabric-based deploy script.
When I have time I plan to release this tooling open-source.

If I had to do it all over again, I wouldn't. And I wouldn't use Cloud
Functions. I would just use a t2.small instances with a simple worker.

~~~
viraptor
My biggest issue is that (unless I'm missing something obvious), there's no
way to add any configuration to the lambda. If I want different queues to do
(slightly) different things, then I either need different code upload for each
of them, or the parameter to be sent on the queue.

I could not find any place where I can tell it - this is the code, this lambda
uses it with N=5, that lambda uses it with N=42.

~~~
glenjamin
I've spoken to an Amazon rep about this.

The best approaches they were able to suggest were:

    
    
      * Lookup config from dynamo db when the lambda runs
      * Lookup config from S3 when the lambda runs
      * Bake config into the artifacts
    

You can read the name of the alias you're currently running as from the
context data, and use this as a key into some external system.

Not ideal.

~~~
twagner
Suboptimal ;-). This is a common request, and we're looking at options for a
config feature that's independent of the code payload. Tweet or email me
detailed suggestions/asks for it: @timallenwagner or timwagne at amazon.com.

~~~
viraptor
Why not inject it to `context` parameter in some variable with known name? As
long as the field is exposed to CF and in the panel, it should be all we need.

------
iamleppert
As usual, they leave out all the really important details. Who cares if you
can run hello world "in the cloud".

Some questions if anyone at Google knows:

\- What are the CPU, memory and disk limitations? Lambda has caps on all three
(5 min, 1 GB, and 512 MB last time I checked). \- Pricing? \- Native module
support? And if so, what platform to target when creating our build scripts?
Possible to download npm modules with pre-build binaries?

------
jorgecurio
built my new project using AWS Lambda. Serverless. It's fucking awesome.

\- html and js files hosted on s3

\- api gateway talking to dynamodb

\- no servers and worries about scaling.

The only downside thing is that it's truly tough to figure out how to point
your domain to s3. I followed everything and DNS is okay too yet the domain
never resolves. Tested with route53 and after 1 month of trying gave up and
using digitalocean's DNS manager.

I implemented a token based HTTP Api, a fully functional Angular.js app that
talks to it. Allow user to register, change passwords, login, create a blog.

It definitely requires new type of thinking: throwing shit out. Seriously,
I've built apps with Flask, Node.js, variety of PHP frameworks on Apache,
Nginx, Java Web applications and I think that a serverless architecture blows
everything out of the water.

It's nice to know there's not a server anywhere. All of the HTTP requests are
sent to AWS, and AWS runs a script that modifies or does things to the
dynamodb. It feels me with so much excitement. I've yet to stress test it but
I expect it to be highly scalable as AWS ramps things up.

However, I don't think I will be using AppEngine because I'm so used to AWS
now it feels like a lot of cost to switch over to Google....I'm afraid
AppEngine is another Google+ moment.

Guys, I really think serverless architecture is the way to go, and is the
future!

~~~
skrebbel
A nitpick: Calling AWS Lambda serverless is like when Salesforce called their
software "no software" (back when SaaS was new). It's a nice marketing message
but don't you think we're a bit above it here at HN? Of course there's a
server, just not a Linux VM that you need to sysadmin by hand.

I mean, really how much is Lambda more serverless than Heroku has been for
years?

~~~
tyre
I disagree.

With Lambda, you're not getting a server. No OS setup, no ip-tables, no weird
compile errors, and no 5am pagers. You're paying for code to run.

With Salesforce, you're not getting software. No installation, no packaging,
no purchasing updates. You're paying for your team to centralize information.

While I despise Salesforce for many reasons, it was a game-changer in the way
enterprises thought about cloud software. GP's point was a similar mental
shift in how we write that software.

~~~
cakoose
I don't think "not getting a server" is enough justification. With Gmail I
don't get a server, but nobody calls it "serverless email". A server is still
involved.

It seems like an ill-fitting adjective even in a vacuum, but what's worse is
that "serverless" is commonly used to describe something that runs entirely
client-side (e.g. p2p).

Just because AWS Lambda is a "mental shift" doesn't mean that "serverless" is
a good way to describe it.

~~~
devonkim
Server-less is probably nomenclature that needs to be phased out like
"Application Service Provider" or "CORBA." It focuses too much upon the
implementation rather than the value. The value of not having a "server" in
most cases is that you don't have to operationally manage a new host and incur
all the cost of ownership of managing more infrastructure that does not
directly add value to your business. Perhaps if you're pretty efficient it is
better to take this on yourself, but for most companies the trend is
accelerating against infrastructure management since most companies are
decidedly not experts in managing compute, storage, and networks - I've seen a
lot of places in the Fortune 500 and they are inefficient and ineffective at
managing their IT as a rule because IT is a cost center normally.

As for a better term, perhaps operationally-opaque or "pure service" are
starting points? Service is correlated with stronger federation while p2p is
decentralized as a rule. I'd rather avoid SOA which does not say anything
about whether you have to manage the operations or dependencies - nobody is
calling microservices as "server-less" after all. When people in companies
talk about a "services" company they mean that the labor and overhead of
management are all outsourced and as a customer you don't really care about
how they operate as long as they don't violate certain OLAs and SLAs. AWS can
run Lambda on marmot-based computing on Mars for all I care as long as it
meets certain specs. Services compete on cost, SLAs (features are part of
SLAs), and OLAs, period.

------
orliesaurus
I am glad AWS lambda is getting rivals, when I gave my talk about AWS lambda
last fall only a third of the room had tried out lambda, a lot of people were
making funny noises, lamenting the lack of this or that functionality
(200/500, tests etc). This competition will only make Lambda stronger or
weaker. Altho last time I tried to use one of 'Google Cloud' products
(Pub/Sub) it it took 2 hours and got nowhere (auth issues for days) before
going to AWS SQS and 5 minutes later I was already testing a hello world - so
I hope this new product wont stink as much!

~~~
jganetsk
Google Cloud Pub/Sub engineer here. We will be delighted to discuss this with
you. E-mail cloud-pubsub-discuss@googlegroups.com

------
planetjones
I don't really get this - it says you can use any npm dependency as well as
access loca data. What do they mean by local data? Is there any persistency
library for this e.g. to access a cloud sql database or a google big table
instance?

~~~
waprin
I haven't actually tried it out yet (about to) , but just by reading the
instructions on deploying, it looks like you stage your code in a GCS bucket
or on a Git repository (Google Cloud Source), so I think your local data would
refer to any files in that bucket/repository along with your code.

*edit note: looks like you can definitely add more files via the package.json and import them as node modules, but I wouldn't rely on reading and writing from the file system.

As for persistency, if it's just Javascript code you can install node modules
to accomplish what you want including talking to those cloud APIs.
Unfortunately Bigtable might be the exception since I'm not sure if there's a
good nodejs client yet (Bigtable only supports gRPC, not REST), but you could
use Cloud Datastore instead.

Firebase will also be a very popular choice for persistence. Cloud Functions
naturally complements Firebase to complete Backend-as-a-Service. The hope of
Firebase is you don't need a server at all, but sometimes you have logic that
really doesn't belong on a client (background jobs, "master" logic). Cloud
Functions can fill in those gaps.

Firebase: [https://www.firebase.com/](https://www.firebase.com/)

hand-rolled clients nodejs clients for google cloud apis:
[https://github.com/GoogleCloudPlatform/gcloud-
node](https://github.com/GoogleCloudPlatform/gcloud-node)

auto-generated nodejs clients for google apis:
[https://github.com/google/google-api-nodejs-
client](https://github.com/google/google-api-nodejs-client)

disclosure: work for GCP

~~~
planetjones
So the local data comment implies the file system is writable? Or is it read
only? After a bad experience with Google cloud end points I think I'll give
this a mess - especially as Amazon have (what at first glance) seems a much
mature and feature complete offering with lambda.

I wasn't aware of those libraries for accessing Google cloud sql - so thanks
for those.

~~~
waprin
I agree the docs could use some improvements.

After playing around with it, I'm not sure if the filesystem works, which
makes sense given you just have a bucket or repo underneath and the code can
be called at unpredictable times. You can, however, add additional javascript
files via the package.json and import them, so if you can express your data as
JSON or base64, you're good to go. Maybe there is a way to add other types of
resources via the package.json but I haven't figured it out yet.

Generally you would interact with Cloud SQL using a regular node MySQL
library, since it's just managed MySQL. The API would only be used to automate
creating and setting up the CloudSQL instance.

------
kevindeasis
You know what I would like right now.

Give me something that competes with Heroku.

But with more noSql Databases like RETHINKDB and CHEAPER noSQL DB

and CHEAPER SSL per month for non-wildcard

~~~
gtaylor
> Give me something that competes with Heroku.

It's not 100% apples to apples, but Google Container Engine (hosted
Kubernetes) is young but already outstanding if you are comfortable with
Docker. If not, nevermind me.

------
jakozaur
Btw. has anyone built some high traffic service using AWS Lambda or friends?
By high traffic I mean at least 100 requests / second.

I see some people using it for small use cases and it worked fine. In
particular "I need to run this tiny integration function somewhere". However,
I would be really careful to do something that has huge traffic.

~~~
jedberg
I know someone who has, but I'm not sure if I'm allowed to say who. They're a
really big site and you'd probably know the name, but they're doing 1000's of
requests per second.

But other than them I don't know of anyone else.

My company is built totally on Lambda, but we haven't launched yet.

------
jmathai
This is interesting because of Google's suite of consumer products. Imagine
being able to run logic via events which get triggered when you receive an
email, a file is added to drive or someone is detected in a photo.

Not that I care to have that level of introspection into my data but it opens
up some interesting opportunities.

~~~
jganetsk
Google Apps Script has had this functionality for years.
[https://developers.google.com/apps-
script/guides/triggers/](https://developers.google.com/apps-
script/guides/triggers/)

Disclaimer: I work for Google.

------
scanr
Anyone know which version of NodeJS they're using? AWS is stuck on node
v0.10.36. Fortunately, it's easy to use babel to transpile back to it.

~~~
yowmamasita
I only tried using
[https://cloud.google.com/nodejs/](https://cloud.google.com/nodejs/) and it
supports any official version available in the site that you indicate in the
"engines" field. Tried 5.6 (App Engine) and it worked.

[https://docs.npmjs.com/files/package.json#engines](https://docs.npmjs.com/files/package.json#engines)

------
zitterbewegung
Where is the pricing for this API? I tried googling around and could find
nothing. I can easily find this for Lambda but if I would consider this a
competitor I would need to be able to compare pricing.

~~~
nikolay
Probably free while in alpha.

------
wging
What node.js version does this run? Site doesn't say.

~~~
yowmamasita
I only tried using
[https://cloud.google.com/nodejs/](https://cloud.google.com/nodejs/) and it
supports any official version available in the site that you indicate in the
"engines" field. Tried 5.6 (App Engine) and it worked.
[https://docs.npmjs.com/files/package.json#engines](https://docs.npmjs.com/files/package.json#engines)

------
alooPotato
What are some good use cases for this or aws lambda?

I.E. when would you use this over a plain old simple appengine app

~~~
robzyb
An AWS lambda function powers this Twitter account:
[https://twitter.com/actdigstream](https://twitter.com/actdigstream)

Considering that the lambda function was pretty easy to write, free to run,
and very portable: why would an Appengine app be better?

------
scprodigy
What about execution isolation? Any info?

~~~
yowmamasita
My guess: it runs on a NodeJS container along with your other "functions".

~~~
scprodigy
In my own VM, or share a VM with others?

------
yandrypozo
really nodejs ?? well done Google

~~~
karangoeluw
what's wrong with nodejs?

~~~
50CNT
Because .js.

I think the general argument is this. Javascript is a necessary evil, but
serverside, it isn't. Please, let us use something saner, or at least give us
the option to do so.

Javascript seems to be spreading into areas javascript has no business of
being in, ("node.js on microcontrollers? what?"), and that is a bit
disconcerting.

Granted, I'm fairly new to Javascript, but even the good stuff seems like
piecing together sane abstractions from wonky components. It's the horror
you'd feel if a nuclear reactor operator told you they get the components to
go subcritical by grounding that bare wire next to their coffee cup. Does it
work? Yes. Abusing function scope and closures to make variables private?
Write a function that checks whether a variable is an array because typeof
malfunctions? Does it work? Yes.

But then, is it sensible engineering? And even if you've gotten good enough at
javascript that the gotchas simply seem kinky to you, do you really need it in
every other area of your life?

I'm also be a little bit concerned that javascript will swallow everything
else, and then the only way to program is to engage in a kinky whipfest, and
whilst as a German I enjoy a good kinky whipfest, it's really something for
after work.

~~~
melted
It's not sensible if you're principled about your technology choices, but if
you just want to whip something up quickly and all you have are "full stack"
developers, Node gives them the ability to use just one language for
everything, as well as a pretty high performance runtime. So the value is
there, given those constraints, which may or may not apply to you.

~~~
50CNT
It does make sense in that context, but even then it seems like a dangerous
choice to make depending on your development culture, especially if it only
pays the minimums on technical debt. Javascript seems like the payday loan of
technical debt.

I think the fear and distrust towards it stems from the perception that it is
on its way to become the lingua franca of development, and the expense of
other much more suitable programming languages. That seems like a terrible
idea.

------
nikolay
Amazon leads and everybody else - copies! There can't be a better ad for AWS
than this!

~~~
lacker
Hmm I think someone else had "Cloud Functions" before AWS had lambda...

~~~
nikolay
Amazon made it mainstream, Google saw how popular it became and copied it. I'm
not having negative feelings about Google - I just have highly positive ones
about Amazon.

~~~
iofj
How does lambda differ from something like google app engine ?

