
Fission: Serverless Functions as a Service for Kubernetes - ferrantim
http://blog.kubernetes.io/2017/01/fission-serverless-functions-as-service-for-kubernetes.html
======
ekidd
From the article:

> To optimize cold start overheads, Fission keeps a running pool of containers
> for each environment. When a request for a function comes in, Fission
> doesn't have to deploy a new container -- it just chooses one that's already
> running, copies the function into the container, loads it dynamically, and
> routes the request to that instance. The overhead of this process takes on
> the order of 100msec for NodeJS and Python functions.

100 ms of overhead is pretty steep. Google recommends a limit of twice that
for the entire server response
([https://developers.google.com/speed/docs/insights/Server#rec...](https://developers.google.com/speed/docs/insights/Server#recommendations)):

> You should reduce your server response time under 200ms

Still, I've heard that AWS Lambda may run around 200 ms, so perhaps Fission is
doing OK here.

But with modern progress in fast-starting compiled languages like Go and Rust,
I'd love to have some easy way to supply small static binaries instead of
source code. With compiled web server frameworks, it's feasible to handle over
250,000 (very simple) requests per second, or thousands of requests while
Fission is trying to figure out where to send your source code. It doesn't
always make sense to pay for interpreter startup overhead on every request
(EDIT: this is not the case according to the Fission developers), especially
not now that compiled code is so easy.

~~~
shanemhansen
Everything old is new again.

So in the old days there was a way to supply a binary rather than source code.
That binary could be fast starting. There was even an open standard for
connecting those binaries to different web servers. The standard used unix
patterns such as reading stdin and writing stdout.

Sometimes it seems like people have created an insanely high learning curve to
run their CGI programs. Sometimes it seems like most of the magic and value
add of some of these platforms is that they reduce some of the overhead they
added.

~~~
coredog64
Someone needs to start a "container global initiative", and then we can keep
all of our binary containers in "cgi-bin".

------
Mizza
Zappa author here, very excited by this project!

We've been planning on supporting other DIY/non-AWS offerings in this space -
particularly IBM's OpenWhisk[1]. If the Fission authors are here - can I ask
how the two stack up? OpenWhisk's ability to write your own custom even
sources seems like a leg up, as does their wider language support, but
Fissions's Kubernetes foundation seems preferable. Curious how the two
compare!

[0] [https://github.com/Miserlou/Zappa](https://github.com/Miserlou/Zappa) [1]
[https://developer.ibm.com/openwhisk/](https://developer.ibm.com/openwhisk/)

~~~
soamv
Hi there, it would be awesome to have Zappa support Fission, happy to talk
more about it!

We're working on a detailed comparison with Openwhisk, but basically -- we bet
on Kubernetes as a good foundation to build this on, and it seems to be going
well so far. It's true that we're at an earlier project maturity stage than
Openwhisk is. In particular, we're actively working on integrating event
sources and adding support for more languages.

~~~
Mizza
Awesome!

Can you talk more about Fission Router? This seems to be where most of the
hand-waving is happening right now :) AWS API Gateway doesn't get very much
love, but it is actually a very important part of the server-less ecosystem.

Why did you write your own Router and not make an nGinx plugin, for instance?
How well does it scale? How do you deploy it? What features does it have?

~~~
soamv
Agree on the importance of API gateway.

Feature-wise, fission router is very simple right now. It's a service based on
gorilla/mux. We had to write our own router because of cold-starts --
basically, the router needs to be able to talk to the pool manager and to get
an instance when one doesn't exist, while holding on to the incoming request.
(I'm not sure how much of this is possible to write as an nginx plugin.)

I don't have perf numbers just yet, but the router is stateless and you can
just run more replicas when you reach its limit. It caches results from the
rest of fission components, and most requests are on this fast path.

(We're also thinking about adding some existing feature-ful API proxy in front
of the router, to get stuff like authentication, rate limiting, etc.)

------
zeptomu
It might be the case that docker was just some enabling technology to go back
to static binaries even with dynamic languages that IMHO expect a more
complicated runtime setup.

So if one goes with statically compiled web services written in e.g. Go, Rust,
Haskell or any other language supporting static binaries (most languages
support this actually, but not all of them encourage you to do it), the only
thing missing is a deployment framework that distributes your binaries to your
(potentially virtual) machines. So am I correct that this is the point where
Kubernetes comes in?

So how to use Docker in production is a controversial topic and the consensus
seems to change rapidly, but in my opinion is mostly used as a weapon against
dependency hell, but this problem is mostly solved using static binaries.
Other use it for sandboxing, but I am not sure what security guarantees one
can expect here.

If one compares static binaries with Docker containers how do they compare
from a security point of view if one is hyperparanoid that the code executed
may harm the host?

~~~
majewsky
I would add another advantage: You get to escape the steaming mess that is
configuration management. Our legacy landscape uses VMs configured with Chef,
and our new landscape has containers on Kubernetes. I like the latter approach
much better since I don't have to care about idempotency or whatever. I just
scrap the container, start a new one, and its startup configuration is handled
by a (usually trivial) shell script that `exec`s into the application binary
at the end.

~~~
zeptomu
But if the shell script is trivial, why not just give the binary its runtime
constraints via arguments or environment variables?

IMHO configuration management overlaps with dependency hell, or could be seen
as a dependency. You do not have to bake the dependency into the binary, but
just have to setup the runtime environment.

~~~
majewsky
Putting a shell script in front of a third-party binary is easier than setting
up a build pipeline to build that binary with the modifications that you need,
and maintaining those patches across releases. (And that's implying that you
have the source code for the binary at all.)

Of course, the perfect solution is to upstream the necessary changes into the
binaries that you need, so that you can supply all configuration via args and
env.

------
lars_francke
I'm relatively new to the Serverless/FaaS space. Or to be more precise: I
missed AWS Lambda and only heard about it when OpenWhisk was donated to
Apache.

I have a couple of questions, maybe someone here has answers for me:

1) In particular I'm wondering about expensive one-time/setup operations (e.g.
opening a database connection) that are usually done once at startup of a
service. But that doesn't really apply here. How are these things handled?

2) As a Java guy I also wonder which of these tools support Java and how they
deal with dependencies and JVM startup times etc.

3) And my last question: How do these integrate into development workflows?
Continuous integration? Run this offline/local etc. Will I need a Kubernetes
instance on my machine?

~~~
soamv
1\. One time operations like that are run at module init time. Here's a
trivial example:
[https://github.com/fission/fission/blob/master/examples/pyth...](https://github.com/fission/fission/blob/master/examples/python/guestbook/add.py#L9)

2\. We don't support Java yet, but we'd like to. The JVM would already be
running by the time there's a request, so there won't be a JVM startup in the
request path.

3\. Dev workflows: Lots to do on that front. A big part of our roadmap is
improving the dev workflow, better versioning support, CI/CD.

For running locally, yes, the best way is to run a local Kubernetes instance
on your machine. minikube makes it very easy to do that, see the instructions
in the README: [https://github.com/fission/fission#setup-
kubernetes](https://github.com/fission/fission#setup-kubernetes)

~~~
rabbah
OpenWhisk actions (equivalent to functions in other frameworks) have two life
cycle events: initialization and execution. The very first activation of an
action (a cold start), will initiate both events. Future activations of the
action, if they're contemporaneous, will only generate execution events. This
is the warm execution, which can take advantage of previous activations. To
provide scaling, it is necessary to sometimes create new execution
environments for the repeated actions, and so that means new initialization
events.

While one cannot rely that all executions after a cold start are warm, it's
possible to for the action code to check if it's warm and forgo internal
initialization as needed. This is a convenient trick that's common to avoid
expensive setup.

For Java actions running in OpenWhisk, this can be done with static fields.

A nice flow of the lifecycle can be seen on slide 24 here
[http://www.slideshare.net/psuter/openwhisk-deep-dive-the-
act...](http://www.slideshare.net/psuter/openwhisk-deep-dive-the-action-
container-model).

~~~
lars_francke
Thank you both. That was very informative. Trying the OpenWhisk Vagrant demo
now.

I've seen a couple of issues and I agree that it would be fabulous to have
OpenWhisk integrated into OpenShift/Kubernetes.

~~~
rabbah
It remains to be seen if the abstraction that Kubernetes adds to manager
containers will have performance characteristics that serverless demands -
considering the need for container reuse, reclamation, pause/unpause, and
elastic scaling.

Should you need help with OpenWhisk please reach out to us on
[http://slack.openwhisk.org/](http://slack.openwhisk.org/) or GitHub.

------
chrissnell
This is handy. It would be great if you could add Go as a supported service
language.

~~~
koolba
While native support is always better, this seems to be implemented similar to
AWS Lambda sothe same approach should work there as well. You can have a husk
of a nodejs app exec the precompiled binary of your go program. You'd just
have to make sure the CPU arch matches up with the k8s infrastructure (i.e.
it's going to be Linux x86-64).

------
binocarlos
A serverless framework using k8s to hide the servers make a lot of sense

------
soamv
Hi all, Fission developer here. Happy to answer any questions.

~~~
snug
Hi, on the kubernetes blog[0], there's a command:

`fission function create --name hello --env python --code hello.py --route
/hello`

The `function` function, doesn't seem support route, and should be broken up
into two parts:

`fission function create --name hello --env python --code hello.py fission
route add --function hello --url /hello`

Taking a look at the help menu for function it's not there, I'm running:

`fission version 0.0.0`

[0][http://blog.kubernetes.io/2017/01/fission-serverless-
functio...](http://blog.kubernetes.io/2017/01/fission-serverless-functions-as-
service-for-kubernetes.html)

~~~
soamv
I think you're on an older version of the CLI, can you try updating it?
[https://github.com/fission/fission#install-the-client-
cli](https://github.com/fission/fission#install-the-client-cli)

~~~
snug
Tried it, still isn't working.

~~~
soamv
Oops, sorry, the option is supposed to be --url, not --route. I'll submit a
fix for the blog.

$ fission function create --name hello --env python --code hello.py --url
/hello

------
kozikow
I think it feels like a competition for celery. It fits into those tasks that
are too heavy to run on the primary web server, but too small to set up the
dedicated deployment/replicaset.

On slightly different note, I am still looking for a good job pattern for my
use case. I run jobs on very custom docker images (including deep learning and
computer vision stuff). I currently use redis queue with todo items, each pod
in deployment picks up items from the queue in the loop, but I dislike some
things about it. Fission is not the good fit, but I am wondering if HN
community would have some suggestions. I thought about kubernetes jobs, but
they have some things I don't like.

~~~
TheIronYuppie
Happy to hear more about what we can do around jobs!

Disclosure: I work at Google on Kubernetes.

~~~
bryanlarsen
Not the OP, but I'll chime in with what we're missing when we took a look at
it. AFAICT there is no way to tell k8s to retry a job X number of times; you
have to futz around with deadlines.

~~~
gtaylor
Job ordering/dependencies! Job1a must finish successfully before Job1b and
Job1c.

Alternatively, all Jobs matching this query must succeed before this job.
Failures can be OK if a configurable threshold of success is met. Exceeding
the thresh fails the dependent jobs before they get to run.

You can do all of this with supervisor pods, but since you asked...

------
ridruejo
Kubeless from Skippbox is another open source serverless offering for
Kubernetes
[https://github.com/skippbox/kubeless](https://github.com/skippbox/kubeless)

------
allcentury
Your article mentions the reason you chose client code vs docker images, which
I understand and can agree with. However, is there any plan to support docker
images in place of client code?

What you're suggesting [initially] is very similar to what the folks at
iron.io did but I'd really want to ship containers all around, not pieces of
code. Iron.io addressed that a little too late in my opinion and their feature
set around docker leaves a lot to be desired.

IMO, AWS Lambda and Google Cloud Functions are going to support docker in the
near future, they have to if they want their ecosystem to be adoptable by all.

~~~
soamv
So we've started by focusing on improving the whole workflow, which is why
we're operating at the source level. But we have occasionally heard the use
case for working at the image level, and it has pros and cons, so we're
thinking about it. I'd like to learn more about what you're trying to do --
did you consider just using Kubernetes Deployments and Services? Or are you
saying you'd like to combine the on-demand start with working at the image
level? Is this basically a resource optimization use case?

~~~
chaosagent
(not GP) I'm actually building something that would require running large
amounts of containers of varying usage (some almost never used, some
constantly queried) that serve up webapps and other network services, and
being able to run services inside of containers instead of functions from
fission would be awesome.

------
andyfleming
Isn't one of the big benefits of serverless functions being able to scale
transparently? Can a kubernetes cluster give you anywhere near the elasticity
of something like lambda?

~~~
TheIronYuppie
By and large, yes. However, the responsiveness is highly variable. Both pods
and nodes can automatically scale, though it is intended that this is somewhat
gradual, by design (you don't want to spin up 4000 nodes because you have a 30
second spike). Because the functions tend to be _very_ small, you can (likely)
pack many of them together without having to scale.

Disclosure: I work at Google on Kubernetes

------
alexellisuk
I've spent some time on a PoC FaaS project with Docker Swarm primitives and
Docker 1.13 features - exploring things like auto-scaling.

[http://blog.alexellis.io/functions-as-a-
service/](http://blog.alexellis.io/functions-as-a-service/)

Long version:

> FaaS is a platform for building serverless functions on Docker Swarm with
> first class metrics. Any UNIX process can be packaged as a function.

> Enabling you to consume a range of web events without repetitive coding.

------
_Marak_
Please see:
[https://github.com/stackvana/microcule](https://github.com/stackvana/microcule)

We've got better functionality, streams ( not a buffered context ), more
language support, and better response times.

------
valuearb
I'm a long time software developer, have recently implemented a series of web
application servers on Heroku using Node.js, and I have no clue what this
article is about. My suggestion is a better introduction might broaden it's
readability.

~~~
majewsky
I had the same problem with docker.com a few years back. I read through pages
of "successful developer stories" and shit, and was sitting there like,
"That's great and all, but what does it actually do?"

On the upside, this has led me to a good practice: When I do a website for one
of my apps, the first few sentences will state in the utmost clarity what the
software actually does.

------
johnhenry
This popped up a few days ago and it seems worth it to compare:
[https://github.com/alexellis/funker-
dispatch/](https://github.com/alexellis/funker-dispatch/)

------
geertj
How does this compare against funktion
([https://funktion.fabric8.io/](https://funktion.fabric8.io/))? Both projects
seem very similar.

------
brilliantcode
so basically a host your own AWS Lambda? very interesting wondering if this
has any of the cold boot time lags

~~~
seangrogg
According to the article it has ~100ms "cold boot" times for Python and
Node.js

~~~
brilliantcode
I wonder why someone would host their own aws lambda, isn't the whole point of
serverless so I don't need to mess with servers?

~~~
rpk09
I was talking to a financial company about aws lambda and they wanted control
over the underlying containers/VMs that is running the aws lambda,
unfortunately AWS lambda doesn't give you that control and 'fission' does

~~~
_Marak_
You should look at
[https://github.com/stackvana/microcule](https://github.com/stackvana/microcule)

