
Introducing Network Containers - pkcsecurity
https://www.zerotier.com/blog/?p=490
======
heavenlyhash
Userspace networking to dodge kernelspace ping pong RTT overhead? This looks
like it has the potential to be _awesome_. At the very least, it's a great
quick talk on how "kernelspace/userspace RTTs are not fast mkaaaay?" and it's
exciting that some folks are trying to think outside the box on this.

So, questions: Am I correct in thinking this approach would be limited to
advisory-only sorts of networking help, and cannot be used as a security
boundary? It seems like anything running within a container's network
namespace would still need a host interface exposed for its own virtualized
packets, and obviously LD_PRELOAD hijinx remain freely ignorable by any
program that decides not to go with the libc flow. Maybe it's possible to pick
up the host interface, and then jump into the container's limited network
namespace and thus be able to give no other options to the other contained
processes?

Alternatively, I wonder if this approach could also be plugged in via seccomp
filters, as well as LD_PRELOAD hooks? The docs at
[https://github.com/torvalds/linux/blob/5634347dee31373a8faf0...](https://github.com/torvalds/linux/blob/5634347dee31373a8faf084f4cdbf6d5ea0b03a4/Documentation/prctl/seccomp_filter.txt#L96-L111)
seem to suggest syscall capture and reroute might be capable of this. This
would still be causing a kernel/userspace bounce, which we were trying to
avoid, but it would still cut out the unnecessary trips through the host
networking stack that we're going to ignore anyway... and more importantly,
would actually be strong enough to be relied upon as a security constraint.

Good luck, ZeroTier folks! Looking forward to watching this continue to
develop.

~~~
api
Depends on what kind of security boundary. You could only allow zerotier
traffic, in which case the container only lives on the virtual net and no
"real" traffic flows. You could also have the preloaded intercept library
forbid any other traffic, isolating the process network wise. It's a total
socket API takeover.

And yes there are other mechanisms than LD PRELOAD. It can also be explicitly
linked into a binary if you do want to rebuild, or linked into libc, or linked
into everything via the dynamic linker configuration files. The latter us how
an entire container as opposed to a process can be placed into a virtual net.

------
lobster_johnson
This looks very cool indeed. I have a quick question unrelated to containers —
is ZeroTier something you can run on servers to create a private, transparent
cloud VPN?

For example, say I have a bunch of boxes on some cloud providers like Digital
Ocean and Linode. I'd like for them to communicate securely — across data
centers and providers — without having to set up SSL for _every_ individual
app in the stack (Memcached, PostgreSQL, etc.). At the same time I'd like the
boxes to talk to any open port among their peers, and not worry about having
to configure iptables separately for every service the box is running. In
effect, I want a private network layered on top of what the hosting provider
has.

Is this what ZeroTier can do for me? If yes, are a lot of people using it this
way? How's the performance? (I notice ZeroTier runs over UDP.) If not, what's
the appropriate software?

~~~
api
Yes, it does indeed and lots of people use it this way. Also runs on desktops
and phones.

------
otterley
The simplest solution of all, as far as I can tell, is to add a new virtual
interface to the host (all hypervisors provide this functionality; or use
macvlan on bare metal) and assign it to the container after obtaining layer 3
information about it (DHCP, static addressing, whatever). Then you don't have
to worry about the complexity of overlay networks or NAT.

What's the objection to this?

~~~
wmf
That's a great idea unless you run out of addresses or your cloud provider
only gives you 8 per VM or you want multitenancy etc.

~~~
otterley
Under what realistic scenarios would you want that many addressable containers
on a host?

~~~
wmf
Containers can add up fast with microservices and sidecars.

And on the cloud provider side, imagine how many 128 MB containers fit on a
host.

~~~
the8472
shouldn't lan-local v6 interfaces suffice for that?

------
MichaelAza
So what is, exactly, the current state of the art in container networking? To
the best of my understanding all current solutions (including this one) create
one big LAN where all containers can see each other, but I'm certainly no
expert.

How would one go about creating a network of containers more in line with
traditional physical networks, with virtual switches, routers etc. ?

~~~
wmf
Maybe hooking Neutron into Docker can get you there:
[https://developer.ibm.com/open/clampify/](https://developer.ibm.com/open/clampify/)

~~~
adjohn
Kuryr aims to achieve this:
[https://github.com/openstack/kuryr](https://github.com/openstack/kuryr)

------
neuroserve
I cannot state often enough, that networking Linux containers on smartos seems
so much more simple.

------
ThatMightBePaul
This is neat!

Has it been shared with any of the folks at the open container initiative?
opencontainers.org

I'm sure they'd love to collab.

------
z3t4
How do you hide the LAN from the container?

