
Serverless Docker Beta - Rauchg
https://zeit.co/blog/serverless-docker
======
andrewtorkbaker
And so ZEIT, my favorite serverless provider, keeps getting better.
Highlights:

\- "sub-second cold boot (full round trip) for most workloads"

\- HTTP/2.0 and websocket support

\- Tune CPU and memory usage, which means even smoother scaling

And all that for any service you can fit in a Docker container - which is also
how you get near-perfect dev/prod parity, an often overlooked issue with other
serverless deployment techniques.

On top of all that, ZEIT has one of the best developer experiences out there
today. Highly recommend trying it out.

And for the perpetual serverless haters out there: this is not a product for
you, FANG developer. I think people underestimate how well serverless fits for
the majority of web applications in the world today. Most small-medium
businesses would be better off exploring serverless than standing up their own
k8s cluster to run effectively one or two websites.

~~~
symlock
I'm not a "serverless hater", but every company I've ever worked with had
backend processes that were not tied to HTTP requests. I still keep actual
servers around because the HTTP gateway is not the pain point. It's long-
running processes, message systems, stream processing, and reporting.

That said, I look forward to the company (or side project) where "serverless"
can save me from also assuming the "devops" role.

~~~
rhacker
I think all that is still possible in Serverless. I'm not a serverless
architect or anything, but that's typically handled by various serverless
queues and related event systems.

~~~
deskamess
Don't most serverless calls have a time limit?

~~~
bdcravens
Yes, but you typically build your functions to split up the work if necessary.
(Create additional queued events)

------
Sujan
Am I the only one having problems to follow .gif "demos"?

When I get to the image, it is in the middle of everything and I don't really
have an idea what is going on. Even watching it multiple times, I am not sure
where it starts, ends, what the individual steps are.

Or is it because I just don't know enough about this stuff?

~~~
Sujan
Oh, and to make this more than a simple rant/whine, some constructive
suggestions that could possibly solve the problem:

\- Normal video with playback controls?

\- Add a clearly distinguishable "this is where it starts" screen.

\- Is there maybe a way to control playback of a gif via JS/HTML? Add a
"restart" button.

\- Add a timeline into the .gif that shows where I am (similar to usual video
player).

~~~
piyh
Make it a webm. Then you can right click from pretty much any browser and
click "show controls". See Gyfcat.

~~~
marzell
You can view controls on GIFs in many cases as well, I believe.

Also, nobody seems to make use of these features, but GIFs do not have to
loop, nor do they have to loop by going back to the very beginning of the
animation as well. In particular, I find it extremely ineffective where there
are GIFs that only show a "final product/scene" for like 1 frame before
looping back to the beginning.

~~~
TheDong
> You can view controls on GIFs in many cases as well, I believe

I don't think that's true.

Twitter and imgur do convert gifs to mp4, and you can see controls there, but
that's because they're not gifs.

Do you have a citation or example of controls on actual gifs in any commonly
used web browser?

~~~
marzell
Yea I am probably mistaken, this is likely gifv

------
ransom1538
Yep. Pretty soon. You write code. You create a docker file. You find a place
to run the docker file with your code [cheapest!]. Run it through your tests.
Monitor it. The end. No vpcs,salts, puppets, sshs,chefs, horses,anisbles,
cats,ec2s,devops,noops, sysadmins, kubernetes or chaos monkeys required.

~~~
aequitas
Until you discover that the thing you are building requires more than a single
application running in a single container and you end up building an entire
"Operating System" around your containers and the circle starts all over
again.

Complexity is hardly ever in the solution, but mostly in the problem. Single
solutions to complex problems often ignore/forget important parts of the
problem and they come back to bite you, hard.

~~~
some_account
I don't know... My experience is the other extreme - tech teams that make
everything super complicated to support everything that can possibly happen.
As a consequence, the it environment requires six months of experience to even
understand. It's really not very fun to work in those environments. Lots of
unnecessary complexity.

~~~
jarym
Yup. I’ve seen this

\- customers who insist every package has to be installed in some special
place because /opt is ‘reserved’

\- have to have non-standard ports for everything because it might slow down
attackers

\- have to have an Apache proxy in front of everything, always - even internal
components. ‘Cos.

\- won’t invest in trusted SSL certificates for internal services.

\- every sql query has to be wrapped in a stored procedure - no exceptions.

... list goes on

~~~
laumars
There is actually a reasonable arguement for the stored procedure one. It
means you can have different permissions for tables as you do for stored
procedures so your application doesn't have permission to directly query your
passwords. The benefit there is if an attacker gains access to your
application and/or the DB authentication credentials they cannot then export
users passwords or other sensitive information.

~~~
phil21
Seems more efficient to me to use column-based permissions in this case, and
create stored procedures for the queries you need to interact with the
"sensitive" columns.

That of course does increase (dba) administration overhead, but it seems on
it's face simpler and far more "programmer efficient" than storing every
single query as a stored procedure.

~~~
laumars
It's a difficult thing to generalise but if you have a pretty large or complex
application and DBA (or several) then it does sometimes pay to have your DBAs
write the SQL because it's a different enough paradigm from normal backend
development that not all backend developers are skilled at writing perfomant
SQL. However few applications are that complex, not all businesses can afford
a dedicated DBA team, and there are clearly a lot of very talented developers
who can turn their hand to multiple different stacks. So it's a difficult
thing to generalise.

------
watty
Looks great for basic websites but it's missing the biggest and most difficult
piece of cloud infrastructure. The DATABASE!

Today you'd have to open up your cloud DB provider to the world since Zeit
can't provide a list of IPs to whitelist. This is a showstopper for me
unfortunately.

~~~
Rauchg
Most databases are quite unfit for the serverless world that's becoming a
reality, where the needs shift towards global replication, flexible horizontal
scalability (sharding) and vertical (provisioned QPS).

We like and use CosmosDB because it fits this criteria. We anticipate that
Google Spanner, CockroachDB and similar databases will become the go-tos in
combination with ZEIT Now.

~~~
skrebbel
So what do most of your current customers do for data storage? I mean, I doubt
they all use CosmosDB? (simply because it's not particularly mainstream)

~~~
arunoda
We don't have insight on what our customer's use mostly.

We use existing technologies. Anyone can use any cloud Database service. We've
datacenters on San Fransisco and in Belgium. So, based on those users can
choose where they need to deploy their DBs.

Usually we recommend to configure databases via env variables. Users can also
use our [now secrets]([https://zeit.co/docs/getting-
started/secrets](https://zeit.co/docs/getting-started/secrets)) service as
well to avoid hard-coding secrets.

------
orf
Shout out to the RCE I found in the zeit.now deploy button:

[https://github.com/zpnk/deploy.now/issues/27](https://github.com/zpnk/deploy.now/issues/27)

Hopefully someone from Zeit reading this can get my fix merged, it seems to be
quite a popular service

~~~
Rauchg
This is not supported nor maintained by us. Thanks a lot for bringing it to
our attention. We will do our best to reach out!

~~~
orf
Sure, but I reached out nearly a year ago to the maintainers with no reply. It
seems anyone can now execute anything on your infrastructure (for free!),
which isn't great.

~~~
simonw
That's the case already: anyone can sign up for a free Zeit account and deploy
code (wrapped in a Docker container) that then executes on Zeit's
infrastructure.

~~~
orf
And now someone can do it without signing up, using someone else's account,
possobly disabling a widely used and quite cool service.

------
newscracker
> “A very common category of failure of software applications is associated
> with failures that occur after programs get into states that the developers
> didn't anticipate, usually arising after many cycles. In other words,
> programs can fail unexpectedly from accumulating state over a long lifespan
> of operation. Perhaps the most common example of this is a memory leak: the
> unanticipated growth of irreclaimable memory that ultimately concludes in a
> faulty application.

> _Serverless means never having to "try turning it off and back on again"_

> Serverless models completely remove this category of issues, ensuring that
> no request goes unserviced during the recycling, upgrading or scaling of an
> application, even when it encounters runtime errors.

> How Does Now Ensure This?

> Your deployment instances are constantly recycling and rotating. Because of
> the request-driven nature of scheduling execution, combined with limits such
> as maximum execution length, you avoid many common operational errors
> completely.

Somehow this sounds very expensive to me (like restarting Windows 2000 every
hour just to avoid a BSoD, except that here it’s not that time consuming a
process) and seems to leave aside caching, state management and other related
requirements on the wayside for someone else to handle or recover from.

Or it’s likely that I’ve understood this wrong and that this can actually
scale well for large, distributed apps of any kind. Sounds like magic if it’s
that way.

~~~
agency
Really brings to mind that Rasmus Lerdorf quote:

> I'm not a real programmer. I throw together things until it works then I
> move on. The real programmers will say "Yeah it works but you're leaking
> memory everywhere. Perhaps we should fix that." I’ll just restart Apache
> every 10 requests.

~~~
gfodor
It's funny when I was reading this part of the article I thought "that sounds
exactly like how PHP works!"

------
tfolbrecht
Awesome! While I was at AWS Summit in NY, I asked a round circle of AWS
ECS/EKS users (Container orchestration products) about thoughts on a Docker
container service that could execute like a FaaS product and there seemed to
be none anyone knew of. I have a portion of a legacy application that's used
infrequently and too costly to decompose but works fine Dockerized.

Looking forward to using your product!

~~~
TheDong
I'm amazed no one knew of one. Off the top of my head, services which let you
run docker containers without dealing with any servers yourself:

1\. AWS Fargate

2\. Joyent Triton (which was as simple as DOCKER_HOST=<joyent-ip> docker run
back in 2015)

3\. hyper.sh (available since 2016 I think?)

4\. OpenFaaS cloud (though you have to go through a couple more steps, you can
bring a custom docker image as your 'function').

5\. A metric ton of hosted kubernetes offerings

~~~
tapsboy
Also ACI (Azure Container Instances)

------
robrtsql
I'm confused about pricing. I come from using AWS Lambda, where you pay for
the amount of memory allocated for your function, how many times it runs and
how long each run is.

Looking at Now, it looks like you are billed by the 'plan' that you choose,
and that decides how many deployment instances you are limited to. What does a
deployment instance mean for something that is 'serverless'?

EDIT: Whoops, I see that there are 'on demand' prices for deployment instances
too--now I just need to figure out how deployment instances map to serverless.

------
jgh
So I've been messing with Fn + Clojure + Graal Native Image and I'm seeing
cold start times around 300-400ms and hot runs around 10-30ms. TLS adds
something like 100-150ms on top of that. I was excited about seeing improved
docker start times, but it seems like you guys are pretty much at the same
place I am with it.

Here's my question, being relatively ignorant of Docker's internals: _is it
possible_ to improve that docker create/docker start time from 300-400 ms (all
in) to <100ms? 300-400ms is kind of a lot of latency for a cold boot still,
and people still do things like keepalive pings to keep functions warm, so it
would be pretty great to bring that down some more.

~~~
jacques_chester
There are a lot of low- and medium-hanging fruit in this space.

I take the view that there is a "warmth hierarchy" (an idea I spitballed with
other folks in Knative before Matt Moore took it and filled it out a lot).

You have a request or message. You need to process it. The hierarchy runs CPU
> Memory > Disk > Cold.

CPU-warm: immediately schedulable by the OS. Order of nanoseconds.

Memory-warm: in-memory but can't be immediately scheduled. Our reference case
is processes that have been cgroup frozen. Order of tens of milliseconds.

Disk-warm: the necessary layers are all present on the node which is assigned
to launch the process. Order of hundreds of milliseconds.

Cold start: nothing is present on the node which is assigned to launch the
process. It will need to fetch some or all layers from a registry. Order of
seconds to minutes.

Some of these problems can't be solved by Kubernetes (it doesn't know how to
freeze a process), some of them can't be solved by Docker (it doesn't know
about other nodes that might have warmer disk caches). Then there's the
business of deciding where to route traffic, how to prewarm caches and based
on what predictive approach, avoiding siloisation of processes due to feedback
loops.

As these are knocked down they will pay dividends for everyone.

------
tango12
So this is different from running docker containers on heroku because heroku
has 1 to n autoscaling but not 0 to n?

What are the other fundamental differences?

~~~
tfolbrecht
It's actually pretty similar to the Heroku free tier dyno "sleep" till there's
a trigger, but it's stateless.

Different pricing, different ways of conceptualizing but from a user
perspective that's a pretty good comparison.

~~~
collinmanderson
> it's stateless.

Can we use "stateless" instead of "severless"? That seems like a better term.

------
ingenieroariel
I am already running an API service on Zeit now using a golang container that
adds the binary and a csv file to a scratch image and re-reads the csv on each
request (all requests take less than 0.3 seconds so i have not optimized).

Currently I have to make sure to set min instances to 1 on the 'production'
version of the API and set min instances = 0 to older versions.

Will have to try before knowing for sure but I seems like switching to
serverless docker would mean I don't have to make the distinction between 'old
production' and 'new production' anymore and keep servicing requests to very
old versions of my API without an expensive (7 seconds) cold boot.

Nice!

~~~
arunoda
You can alias your deployment and try to add a scaling rule to the alias.

So, you don't scale up and down as you deploy. Once you call the `now alias`
it'll take care of the scaling.

------
gamegod
Finally, I can have left-pad as a service.

------
mirceal
I did not play with it, but this actually looks pretty cool. The documentation
seems to be sane/good quality.

A few questions: How would one coordinate between multiple nodes running an
app? (scenario in which the nodes cooperate to find each other. is there some
sort of discovery available?) For the docker case, does it support health
probes? (how you you know if the app is healthy?)

~~~
Rauchg
For the duration of the warm phase (i.e.: while the process is running), we
ascertain healthfulness via `SYN` packets as a baseline.

Because you give us your code under the constraint that it will expose data
over a port, we have a much better safety baseline than stock schedulers, that
operate under the assumption that the process describe by `CMD` is a black
box.

We will describe this in more detail in coming weeks. One of our priorities is
to minimize the cognitive load on the developer necessary to run a healthy
service at scale.

------
StanAngeloff
This is truly amazing and has so much baked in I find myself wanting to use
it. However most apps my team and I work on require some form of persistence -
uploads, log ins, API rate limits, etc. These scenarios don't fit well with
the serverless world. It is technically possible to offset most of this to S3,
a hosted NoSQL and perhaps even a service for dealing with a
images/thumbnails. By that time your monthly bill is in the three digits,
though. For a big web shop this is fine but your average freelancer with
multiple smaller and finite-resource projects it's something to consider
carefully.

------
danpalmer
> Serverless

> A new slot configuration property which defines the resource allocation in
> terms of CPU and Memory, defaulting to c.125-m512 (.125 of a vCPU and 512MB
> of memory)

Sure sounds like needing to be concerned about the hardware. That feels like a
leaky abstraction that the serverless design pattern claims or appears to take
care of, but seems like it doesn't in practice. Is "serverless" the right
level of abstraction? I'm not sure.

~~~
arunoda
Whether it's serverless of not choice of hardware is very important.

For an simple web app, you might not need special CPU and Memory requirements.
But for a video encoding/decoding serverless app may needs different
CPU/Memory requirements (and GPU).

That's what this `slot` configuration property does.

The default is good enough for all the general purpose apps. But if you need
more, you can control it.

~~~
danpalmer
I agree that one-size-fits-all doesn't work with hardware requirements, but I
still think this is a leaky abstraction. The promise of Serverless is not
having to care about servers, but this is an option to specifically control
that. A better abstraction would auto-optimise hardware spec for the
software's performance characteristics - that would be much more "serverless".

------
manigandham
Great announcement.

This development was inevitable, especially since most Functions-as-a-service
infrastructure was really docker (or some other) containers running in the
background, spinning up on demand using a pool of servers. Azure Container
Instances launched last year, and now Google Cloud Functions has serverless
containers in alpha, along with the serverless addon for GKE/Kubernetes, and
there should be a good bit of announcements this year as other providers
follow.

The max time limits still seem pointless to me, there should be an option for
unlimited uptime so the full spectrum of ephemeral function to entire app can
be managed and deployed using a single flow.

------
virtualritz
Serverless n00b here: I'm just playing with serverless Rust, via Apex, which
works great so far.

Why would I want to switch to this? What are pros/cons of having Docker in
there?

~~~
iampims
You don’t need a shim on top of Rust. You can run your docker image anywhere
it is supported, like Heroku for instance…

------
nickik
There is no 'hype' word in history I hate more then serverless. I was fine
with microservices and all those other hype words but serverless is terrible.

~~~
SamLevin88
mind elaborating a bit?

~~~
nickik
There are still servers involved. Serverless sounds like its peer to peer.
What it really is is VM less.

~~~
Drdrdrq
Well, as a developer I don't need to concern myself with servers, so in a way
the term is fitting.

~~~
nickik
The interface it gives you is the functionality that the server provides. A
server does not mean 'operating system'. you don't have to deal with the OS,
but you still have you app running on the server.

------
gitgud
A couple of questions for Zeit:

1\. Once your docker container is built, am I paying to store it on Zeit?

2\. How is a Dockerfile versioned? Can I update it? Or do I need to redeploy.

3\. Is pricing for containers granular by time or per request to a container?

4\. How can these Dockerfiles talk to each other? Is there an API or method to
fetch specific url's for each container?

Looks like a great idea, am curious to try it out sometime.

~~~
arunoda
> 1\. Once your docker container is built, am I paying to store it on Zeit?

No. You don't need to pay to store containers.

> 2\. How is a Dockerfile versioned? Can I update it? Or do I need to redeploy

Once the Dockerfile is built, you'll get a URL for that. It's a immutable URL
and you don't need to re-deploy again. It'll sleep if there are no HTTP
requests. If there are it'll start again.

> 3\. Is pricing for containers granular by time or per request to a
> container?

It's based on the container running time. But we'll have a another blog post
on this soon with more info.

> 4\. How can these Dockerfiles talk to each other? Is there an API or method
> to fetch specific url's for each container?

There's no internal API to communicate with each other. But every docker
container has a URL. You can use our
[alias]([https://zeit.co/docs/features/aliases](https://zeit.co/docs/features/aliases))
feature and communicate with each other via HTTP.

------
tapsboy
This is more of a question to @rauchg and others who have used now one. When I
use now within the AWS realm, are the AWS services when used in conjunction
with Zeit Now containers subjected to ingress/egress costs? As it does sound
like I am using a different cloud as far as AWS is concerned

------
fru2013
Is there support planned for using this in combination with docker-compose? If
so would be sweet to see this integrated with Prisma -
[https://www.prisma.io/](https://www.prisma.io/)

------
daveheq
How can something be serverless if it habdles http requests? Does it just
process requests and never serves responses, or is it just a hip-sounding
misnomer?

~~~
mildweed
Long story short, there's a server somewhere, but most of the configuration is
abstracted away. Instead of thinking of servers, they want you to think about
endpoints (which they manage somewhat efficiently for you).

------
unixhero
My biggest concern with serverless is pricing. How does it fare when compared
to regular Docker hosting and vps hosting?

~~~
Rauchg
We will be announcing very clear pricing when it goes into GA. Needless to
say, serverless makes for a much much cheaper (and robust) operation than VPS
and traditional clusters.

------
netcraft
My biggest problem with serverless functions is development and testing. Can I
run these locally? even better offline?

~~~
Rauchg
As we mentioned in the blog post, there is _zero difference_ between a
`Dockerfile` you execute locally and one that you give to us.

[One of the differences is performance. You'll find that we can build and
execute much more quickly :)]

There are many examples here: [https://github.com/zeit/now-
examples](https://github.com/zeit/now-examples)

~~~
e12e
So, I'm not running the same docker image, just something somewhat similar
built from the same Dockerfile?

~~~
arunoda
> something somewhat similar built from the same Dockerfile?

Yes. We build the container based on the Dockerfile.

------
gigatexal
What magic will they come up with next? Sheesh. Cool stuff, looking forward to
having some time to play with it all.

------
SamLevin88
Documentation is good. Post is informative. Interesting indeed.

------
lxe
I've never been more excited about a cloud platform than I am about Zeit/Now.
I wish you offered stateful volumes or key value stores or something where I
can persist my data so I can keep all of my stack on Zeit.

~~~
iampims
I suggest you use something like DynamoDB, it fits this paradigm quite well.

------
fake-name
"Serverless"

So where do your functions execute?

~~~
gregknicholson
I think “serverless” doesn't actually mean no server (a distributed system,
like Dat); it means “don't worry about the server; we'll take care of it for
you”.

…or maybe that's “Cloud”.

Is there a clear explanation of what “serverless” means, and how it's
different from “Cloud”, suitable for semi-technical non-developers?

~~~
dx87
From what I understand, "Cloud" basically means that you still have to deal
with servers, whether they be VMs or physical devices, but all of the
networking and loadbalancing aspects are abstracted away. "Serverless" takes
it a step further and makes it so you don't even have to deal with setting up
VMs, you just give the provider the code you'd normally run on a host, and
they'll automatically handle all of the infrastructure stuff, like spinning up
extra hosts under heavy load.

