
Firecracker: Secure and fast microVMs for serverless computing - based2
https://firecracker-microvm.github.io/
======
mrkurt
Firecracker is great We use it to run fleets of fast booting vms at
[https://fly.io](https://fly.io).

It’s really the best OSS to come out of Amazon.

~~~
DenseComet
I played around with fly.io for a bit, it seems pretty interesting. It works
pretty well too, I went through the setup for the DoH proxy and the latency I
get is very similar to Cloudflare itself, so that's pretty awesome.

It seems that the autoscaling limits are only defined in the fly.toml with the
soft and hard limits? It might be useful to make this easily visible under
flyctl scale. Also if I delete the fly.toml, can I regenerate it easily?

As a sidenote, I was looking around for more information on the platform and
looking at old hn posts. I know the company pivoted a couple of times, but all
the old articles are 404ing because the blog url changed.

~~~
jeromegn
That's nice to read! Thanks.

We do need to cleanup our old blog posts and links. We created a lot of
content at various times. This content is not always relevant anymore.

As for your fly.toml question, you can get the config with `flyctl config save
-a your-app`. It'll create a fly.toml with the latest config we know about.

Concurrency limits are still being worked on. They should definitely be
visible in more places. The only way to know about them right now is from the
fly.toml, that's not ideal.

------
tofflos
I played with Weave Ignite the other day which is a Docker-like CLI for
Firecracker. Sure there were some rough edges but the overall experience was
pretty good. If you are familiar with the Docker CLI you will be able to get
some virtual machines up and running very quickly.

Two questions in case someone from Weave tunes into the discussion:

I got the impression that VMs needed an SSH server to be accessible. Is this
correct and if so will it be possible to implement something similar to docker
exec so that I won't need an SSH server on every VM?

> At the moment ignite and ignited need root privileges on the host to operate
> due to certain operations (e.g. mount). This will change in the future.

Is there a timetable and could you perhaps elaborate a bit as to why it
currently requires root? (I don't know anything about virtual machine
internals so this isn't a passive-aggressive question from my side. It's
genuine curiosity.)

~~~
imhoguy
Containers are provided by host kernel cgroups and namespaces, therefore the
kernel implements attach (exec) operation which is practically running a new
proces (e.g. bash) in a cgroup (container).

Virtual Machines are provided by software or hardware emulation which run
separate guest OS with own kernel. There is no standard way for a host to let
you run any process and interact with its stdio inside guest OS because the
host simply is not aware what you exactly run inside.

The solution is to have an agreed connectivity standard both on the guest and
the host. The guest can provide SSH server, telnet server serial terminal, irc
bot or some other kind of control capability. Then of course host needs a
tooling too, e.g. SSH client.

~~~
CameronNemo
Are there any real alternatives to SSH and/or sftp? E.g. a mutual TLS
authenticated HTTP server...

~~~
wmf
virtio-vsock

------
darren0
"Firecracker provides a rate limiter built into every microVM. This enables
optimized sharing of network and storage resources, even across thousands of
microVMs."

Probably the most interesting feature.

~~~
hinkley
One of the things you learn about multitasking is that in theory cooperative
multitasking is the most efficient, but the least reliable. It’s cheaper (for
the human) to use hard and fast rules that trade nasty surprises for vague
disappointment.

This sounds like a hybrid system. The intermediary is cooperative, the client
code is oblivious. I’m curioue to see how this plays out over the long haul.

~~~
ffk
This runs Amazon lambda, so presumably well enough?

~~~
hinkley
You don’t think 12 years from now we’ll be laughing about how lame Lambda is?

Coding practices change like the seasons. Every year is a little different,
maybe a little better than the last, some things feel brand new, but the
general patterns hold.

~~~
ffk
I'm sure we'll laugh about most things we are currently using and gasp when we
need to revisit them in legacy systems. :)

------
based2
[https://blog.acolyer.org/2020/03/02/firecracker/](https://blog.acolyer.org/2020/03/02/firecracker/)

------
asymptotically2
Can I run a single process in a Firecracker microVM, or do I need to bring a
full distro?

~~~
asymptotically2
Seems like you can, but probably shouldn't since you'd want to run services
like chrony to keep time inside the VM.

------
detaro
I'm curious about restarts and snapshotting: would it be feasible to reset a
microVM for each incoming request?

~~~
mrkurt
Snapshot / suspend isn't doable yet, but it sure would be nice.

We've booted some apps in Firecracker in <20ms, so if you build the app
properly you can absolutely do a new VM per request or TCP connection even.

~~~
scarface74
Yeah, this is what AWS does....

~~~
ec109685
No it doesn't. It pre-inits the Firecracker VM's before the first request:
[https://www.usenix.org/system/files/nsdi20-paper-
agache.pdf](https://www.usenix.org/system/files/nsdi20-paper-agache.pdf)

~~~
scarface74
I meant the part about one request per VM.

------
shubidubi
I'm happy to see AWS is giving back to the open source community.

~~~
swyx
if you had to guess.. what are your thoughts on _why_ this was open sourced?
everybody i listen to seems a little confused by the "why" of this specific
action

------
andrewstuart
Note Firecracker does not work on AWS cloud instances, apart from bare metal
instances.

~~~
tfwnonested
>2020

>Still no nested virtualization on aws

;( It should work on some azure instances and gce instances when you enable
nested virtualization.

------
nicklarsennz
Does anyone know if any integration with Kubernetes is on the cards? Something
like an operator with CRDs?

Or is this intended to be separate, a competitor?

~~~
mrkurt
There are a few ways to use k8s + firecracker, weave has one:
[https://www.weave.works/oss/firekube/](https://www.weave.works/oss/firekube/)

------
pimlottc
> Firecracker: OSS virtualization techno, creating and managing secure, multi-
> tena

This title is a bit awkwardly worded; in particular, "techno" in American
English is a genre of music, not an abbreviation for "technology".

~~~
dang
We've reverted it now. Submitted title was "Firecracker: OSS virtualization
techno, creating and managing secure, multi-tena".

Submitters: please don't do that—this is in the site guidelines:
[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html).
If a title is misleading or baity, please rewrite it, but please also make it
good English.

------
xrd
My skills are more on the developer side, this seems more like a devops tool,
right?

As such, I'm never really sure if this is something worth playing with. I've
played with docker and k8s, and generally understand how those tools help me,
but I'm unsure about how firecracker would help unless I'm building a PaaS
like fly.io.

It would be really cool if someone wrote a blog post which compared running a
service on the different comparable options. Developers have the Todo apps
written in dozens of frameworks to compare against. Is a similar type of
exploration not feasible here?

------
ausjke
docker is used for microservices, is firecraker designed for serverless
applications? what's the key difference between firecracker and docker? are
these two overlapped?

~~~
bennyz
Amazon uses it mainly for Fargate and Lambda (from what I've read). Docker is
a container technology (shared kernel), while Firecracker is an actual VM
manager so it provides better isolation. It is more comparable with QEMU.

~~~
solarkraft
Why does FaaS (that's what Lambda is, right?) need more full blown
virtualization? I thought you could maybe get away with even lighter
separation than Docker?

~~~
fiddlerwoaroof
Docker isn’t really designed to be a security boundary, so if you’re
colocating containers from different customers (e.g. in Fargate), you need to
separate them with a real security boundary like a VM. The same thing is true
for lambdas: a lambda is just an archive and the code in the archive needs to
run somewhere where one customer cannot intercept another customer’s data.

~~~
scarface74
To add on, AWS has never run Lambdas for different accounts on the same VM.
Before Firecracker, they would run multiple Lambdas for the same account on
the same VM. Now with Firecracker, they can run each lambda in its own VM.

------
rvz
> Intel processors are supported for production workloads. Support for AMD and
> Arm processors is in developer preview.

Firecracker looks very promising from a server-side technology stand-point but
the support for AMD, RISC-V platforms couldn't be stressed more enough.

Amazon better find a way of supporting AMD processors since Intel's CPU bugs
are being brought into the sunlight and exploited in all directions by
security researchers which have cataclysmic implications for users and server
providers these days. This is demonstrated by a ridiculous Intel vulnerability
which rendered Apple's FileVault encryption facilities completely useless
which is absolutely unacceptable to Apple. There are many other CPU
vulnerabilities waiting to be found and it could be the next Meltdown-like
candidate.

The sooner the move to AMD or RISC-V open technologies, the better for
developers and users.

~~~
karambahh
I am actually surprised by the absence of AMD support for a project born at
AWS. AWS has been offering AMD Epyc ec2 instances for quite awhile[0].

Missing arm support is also suprising but less so, as arm market penetration
is obviously lower than x64

[0][https://aws.amazon.com/fr/ec2/amd/](https://aws.amazon.com/fr/ec2/amd/)

~~~
mrkurt
AMD processors work fine with Firecracker, support is relatively recent though
so they’re being conservative about calling it production ready.

------
pbreit
What are some use cases?

------
airocker
Would it not be better to make docker vm aware? The tooling would not have to
radically change.

------
hestefisk
... and all the koolaid drinking Kubernetes fans are now going to refer to
Kubernetes as legacy and start creating a new cottage industry of Firecracker
migrations. Slightly cynical yes. But also a bit true.

------
christian7007
Will microVMs replace containers in short time?

~~~
swyx
i'd also like to know this..

------
m00dy
I believe it is still using virtio. Therefore, disk read/writes are not good
enough compared to other virtualization tech such as qemu

~~~
detaro
qemu setups often use virtio, afaik virtio was defined by qemu devs, so your
statement doesn't really make sense without more details.

Also, "not good enough" kind of needs a definition of a workload and what's
"good enough" for it.

~~~
bonzini
It's using virtio-mmio which is less efficient than virtio-pci. But I/O was
not a focus of Firecracker, for example it doesn't scale very well because it
doesn't do concurrent I/O operations.

