This website is a proof of concept with a simple Memcached SaaS. It is my final project for cs50x that ended this sunday. Please tell me what you think :)
+ You can read the how-to on SlideShare: http://www.slideshare.net/julienbarbier42/building-a-saas-us...
+ You can see the source on GitHub: https://github.com/jbarbier/SaaS_Memcached
Giving a public facing application sudo powers (even limited ones) is a little scarey. If I understood your documentation I believe if I somehow got onto your box I could escalate to root just by editing your iptables scripts. You probably want to ensure www-data (or anyone) cannot write to these files :-)
If anybody with Mesos experience wants to lend a hand, I would love to give this a try. Say hi on the issue!
Update/clarification: SmartOS is running tens of thousands of VMs and virtual OS containers in production at Joyent -- and has for years. It is new in nomenclature and exposure, perhaps, but not in terms of core technology or production readiness.
Don't misunderstand me: FreeBSD jails are awesome if you're doing FreeBSD and want lightweight virtualization. For the rest of us, LXC is just as awesome, and docker lowers the barrier of entry considerably!
Personally I have used both in production and as far as I am aware Linux is light-years ahead already. Yes: Documentation is lacking. Yes: Some security issues remain. Yes: You need to be happy recompiling kernels and tinkering away. Yes: You need quite high baseline knowledge to figure out what's going on.
Perhaps I am wrong and FreeBSD has changed in the last three years to provide a stronger offering - but show me a policy that runs a program within a container, restricts it to certain syscalls, exposes only a whitelisted subset of devices and a virtualized network of arbitrary topology, bandwidth, and firewalling (at either layer 2 or 3), successfully limits block IO bandwidth to the container on a per-device basis through one of multiple schedulers and policy configurations, similarly both accounts for and limits cpu, memory and swap utilization via multiple configurable strategies (CPU affinity, time slices, hard/soft limits, etc.). Now show me it co-exist with multiple disparate options of kernel-level security toolkits for additional protection. Now show me it boot arbitrary Linux distributions within those containers, to effectively allow the container-based execution of the majority of modern 'nix software.
While we can umm and ahh about it, in truth, FreeBSD just doesn't have this, and realistically it never will at the container-based virtualization level.
Are broad software support and extreme configurability a requirement for most container scenarios? Probably not; in truth FreeBSD is practically comparable to Linux in most cases. However, to describe it as "better" seems dishonest.
(As for ZFS, OpenSolaris is the platform of choice there. Linux has some realistic alternatives appearing with more features, and well established stopgap measures like LVM2 snapshots... which allow the achievement of similar functionality without requiring additional complexity in the filesystem (or everyone to learn a new OS!), though they do lack some of ZFS' capabilities, they're good enough in practice for many situations.)
IMHO it boils down to this: design of any complex system (be it an organization or software) is largely about pragmatism and tradeoffs between various options. Being on an outlying OS doesn't increase your options, either immediately or in future. On the other hand, this is Linux's great strength.
Of course, this is highly subjective, as everything else is - its cool that you can run Steam in LXC and I cant run it in jails, but running steam is not something I want and need to achieve, its security and separation of architecture blocks, and in this world, jails are 'better' technology. Keep in mind, this is not religious - LXC was my technology of choice to start with, but I had to drop it due its immaturity and issues. Will LXC fix all of this, I will probably switch in no time (probably, because of the surrounding features FreeBSD provides).