

Sandstorm Personal Cloud Platform - durin42
http://sandstorm.io/

======
zellyn
I've thought for a while that an interesting question is, "What would be the
perfect way to deploy an open-source, distributed twitter/facebook/whatever
replacement, where you run your own server and own your own data?"

Docker seems like a great step: give people a docker container image, and they
can spin it up and hit a web interface to configure it with almost no extra
knowledge needed.

Sandstorm seems like a great step too: now you can imagine a network of
applications/plugins that know how to talk to each other, but all run on the
user's server. And it can be made very friendly to non-techies. Nice work.

~~~
techwatching
It would be even more compelling if it was packaged as a device, so that a
layperson could plug it in, and log into it to install apps etc. An inverse-
Steam box, if you will.

~~~
ersii
I would like to suggest taking a look at the "Freedom Box" project, their
foundation homepage is at
[http://www.freedomboxfoundation.org/](http://www.freedomboxfoundation.org/)
but there's also some stuff at
[https://wiki.debian.org/FreedomBox](https://wiki.debian.org/FreedomBox)

In short; It's a platform that started after Eben Moglen's talk "Freedom in
the Cloud" \- to establish a platform for running a decentralized network of
Free Software social/personal services. It happened at the same time that so
called "plug computers" (ie. small, somewhat cheap and decent System-on-Chip
computers (think Raspberry Pi but with casing)).

The thought was and is (but I'm not all too up to date on the project) was to
offer software and hardware in a easy package for whoever to setup easily and
get started with.

------
Geee
This is just incredible beyond imagination. I don't think even the author
realizes this. I don't think the marketing copy catches the real potential of
this thing.

This makes it easy for developers to distribute cloud apps without having to
manage users, billing, hosting, security or any of that SaaS boilerplate. Just
code the app and distribute it for your users. This changes everything.

~~~
wmf
_This makes it easy for developers to distribute cloud apps without having to
manage users, billing, hosting, security or any of that SaaS boilerplate. Just
code the app and distribute it for your users._

Of course, the users probably won't pay and you can't use ads, so there may
not be much business here.

~~~
MarkPNeyer
> Of course, the users probably won't pay and you can't use ads, so there may
> not be much business here.

No ads? Awesome!

No incentive other than 'build something cool?' Awesome!

No defective by design? Awesome!

Who cares about business? It's a means to an end, nothing more. If we can get
the end without the business, the paperwork, the ads, the politics - i say
good riddance.

~~~
conradk
Most software we use is either proprietary and uses ads/licenses/etc to make
money. Successful open source software (as in what non-technical people use)
seems to be at the core of something that generates revenue (Linux for
instance). No ?

------
mynegation
This is a wonderful effort! I very much hope that either Sandstorm or OwnCloud
or both start to gain more features, more power and more popularity at least
among the people who care about privacy and like to keep data under their own
control.

Let's face it, Linux should not try to win the battle for the desktop. Not
only it is already lost, its is pretty much irrelevant. I feel that instead of
producing another fork of desktop office package, efforts of open source
contributors would be much better spent on creating Open Source counterpart of
Google Apps.

I do a lot of things with my linux-based NAS already: it is backup server, git
server, hub for online and offsite backup, media storage, and much more. But I
wish more apps would be easily available for an easy installation, without
fiddling with database settings and ruby versions. I want my own pinboard.in,
I want my own RSS reader, I want my own mint.com installable on my NAS with
few clicks. I would definitely pay for good commercial apps too! I just want
them to run on my server with the data I control.

~~~
kentonv
Yes! "The open source counterpart of Google Apps" indeed. The thing that
motivated me to do this in the first place was observing that we have whole
open source desktop environments containing just about everything many of us
need on a day-to-day basis, but in "The Cloud" open source options seem
totally lacking. I use gmail. What else can I use? There are many open source
web mail clients but to use them I need to run my own server, which is too
much friction for me.

But if I could just install an e-mail client on Sandstorm with a couple
clicks... I'd totally do that.

~~~
OWaz
I was just thinking today about how cool it would be if I could easily deploy
an email service that was entirely under my control. I've allowed MS and
Google full access to my email only because I don't know how to setup an email
server with appropriate spam filtering and reliable security. I have my own
RSS aggregation server and I don't ever have to worry about it going away as a
service.

Brian K. Vaughan's The Private Eye [1] was on my mind when I was thinking
about an individual being entirely in control of all their data without
compromising on functionality or usability.

[1] [http://panelsyndicate.com/](http://panelsyndicate.com/)

------
monatron
I'd love to see these guys start using Docker. Not only would it be easier for
developers to containerize their apps to fit this platform, but it would be
quite a boost to the eco-system -- pushing development of Docker to
(potentially) giving Sandstorm the "fine grain" controls they're looking for.

Either way, cool idea! I definitely see web apps moving in this direction.

~~~
kentonv
There's a continuum here between "stay compatible with existing code" and "do
things in a new and interesting way". Docker stays firmly on the compatibility
end, and obviously for people wanting to use software that _exists_ , that's a
great thing. Sandstorm tries to strike a balance by making it relatively easy
to port existing apps, but also trying to do things pretty differently,
scaling things so that every document gets its own independent container and
the only way to talk to the rest of the world is through capability-based
Cap'n Proto RPC. It will take longer to populate the ecosystem, but apps will
be able to do different kinds of things.

Sandstorm does use the same underlying kernel features as Docker, so any
advances there should benefit both. Above that, I'm not actually sure if
trying to share userspace tools would benefit either project, given the very
different design decisions. It might just become a tug-of-war, like Chrome and
Safari sharing Webkit while going in different directions with their
sandboxing model.

------
flyinglizard
I find it hard to evaluate something with an open mind when the first thing I
see is a statement such as "The Web is Broken", when the web is successfully
used by billions on a daily basis, including that very webpage.

I'd rather skip the hyperbole and just get to the value proposition.

~~~
bishnu
Is it really that hard to read like, the very next sentence on the page where
he explains exactly what he means? "When you use a web app today, you usually
connect to its developers' servers. This is backwards, and leaves them in
control."

Also since when does "billions of people use it" mean something isn't broken?
Lots of ubiquitous things are very, very dysfunctional.

Anyways this all seems very pie-in-the-sky but Kenton Varda is an incredible
person. I'm very interested.

~~~
flyinglizard
Broken implies non-functional. It's broken, so it can't be used. The web isn't
broken. Sure it can be made better, though.

~~~
aranjedeath
If the function you're looking for is integrity or privacy, the statement
stands.

------
jerluc
I was quite intrigued at first by this concept, but was a little turned off by
the fact that the Sandstorm crew offers to "Use Our Servers..." to run
Sandstorm itself. IMHO, this offering basically reverses any of the real
benefits of Sandstorm. I'd encourage its authors to perhaps discourage the use
of their "managed instances" so as to actually convince people to stop doing
exactly that with other external software services.

~~~
kentonv
Yeah, it does come off as contradictory. Let me provide some rationale...

The idea here is that for many people, maintaining a Linux instance on which
to run Sandstorm just isn't feasible. It's too expensive (you don't really
need a whole VM just for yourself), requires sysadmin expertise, and is time-
consuming. So for those people who don't want to do all that, the next-best
option seems to be to get a cheap managed instance from someone else -- maybe
the Sandstorm group, or maybe a friend.

I don't want Sandstorm to be something that only developers and Linux
sysadmins are able to use, even if the early adopters are likely to be mostly
from these groups.

Another option might be to sell physical devices pre-loaded with Sandstorm,
although speaking for myself I really do want my cloud services to be in "The
Cloud", not on my home network.

In any case, I want to make it as easy as possible to transition from a
managed instance to a self-run instance, so if you ever decide you don't trust
your host anymore (whether that host is us or someone else), you can easily
leave. That's definitely not something you can do with most web apps today,
even with Google Takeout and whatnot, because they don't let you transfer the
software that makes the data useful.

Bottom line, the pieces needed to run your own personal instance will always
remain open source. But not everyone actually wants to do that.

~~~
Pxtl
THe reason you want it on the cloud is maintenance/setup... But with a
purpose-built device you could have Jobsian control over the experience and
make it really delightful.

To imagine the Cadillac product: double as a WiFi router and put a small
cellphone screen on the thing. The screen gets rid of the admin setup
bootstrapping pain - the screen is default admin user. Making it a wifi router
lets you consolidate your initial configuration into just two questions: a
name for the network and a password. Name becomes ssid, shared login, dyndns
name, etc. Password becomes remote shared login password, WiFi password, etc.
Admin panel can create other admin or regular users, also having a screen
right on the device would be convenient for backups - just plug in the
detachable storage and the backup wizard begins.

I've given this way too much thought. I just want a zero-config device.

~~~
kentonv
That actually does sound pretty awesome. :) Will have to think more about
this.

~~~
Dwolb
I love this. I want to flash this concept onto any number of Linux-based ARM
boards and own my own services.

~~~
kentonv
One catch with that: Since Sandstorm packages contain native code, they need
to target a particular architecture. I've defined x86-64 as the preferred
architecture. In theory all the code should build and run fine on ARM, but
you'll need to convince all the app developers to build separate ARM packages.
:/

------
twiss
I've recently been thinking about the same problem and here's what I came up
with: a bootstrap javascript script which takes the user's password and
decrypts another javascript file with it and runs that. That javascript file
then proceeds to decrypt other files and build an interface with them. This
way, the server could not change files without your consent. You could maybe
even use the Application Cache [1] to notify the user when the server changes
the bootstrap file, or some kind of mechanism where the browser, instead of a
bootstrap file, takes the user's password and decrypts the first file
(something like [2], perhaps).

[1]:
[https://developer.mozilla.org/docs/HTML/Using_the_applicatio...](https://developer.mozilla.org/docs/HTML/Using_the_application_cache)

[2]: [https://en.wikipedia.org/wiki/TLS-
SRP](https://en.wikipedia.org/wiki/TLS-SRP)

~~~
jamesaguilar
How does this help? If it's running on the server, you just insert a
breakpoint right after the decrypt function call, put whatever code you want,
then continue running.

~~~
twiss
Edit: I understand your comment now. No, I meant that the bootstrap script is
running in your browser.

Yes, you're right that it's very far from perfect without browser support for
decrypting files, but it reduces the amount of files on the server you need to
monitor to one (or a couple) and it would make targeting someone harder (by
using appcache). I'm not a security researcher, though.

------
fafwefwaefa
I love the concept, but have few questions: does it allow chrome-extension
like permissions (web, local storage + quota)? how are apps updated? does it
use zerovm? how are unresponsive apps managed? can apps use protocols like
websocket, webrtc? how are data backed up/restored?

~~~
kentonv
> does it allow chrome-extension like permissions (web, local storage +
> quota)?

Currently, there are no permissions; apps are totally isolated. Soon, it will
become possible to grant permissions to apps through a powerbox interface.
This is a bit different from -- but IMO superior to -- the Chrome Extension
and Android models.
[http://plash.beasts.org/powerbox.html](http://plash.beasts.org/powerbox.html)
[http://en.wikipedia.org/wiki/Capability-
based_security](http://en.wikipedia.org/wiki/Capability-based_security)

> how are apps updated?

All packages are cryptographically signed. Two packages signed with the same
key represent versions of the same app. You specify the version number in the
app manifest. When a user installs an updated package, the platform offers to
replace the old package.

Automatic/push updates will be implemented eventually.

> does it use zerovm?

No, Sandstorm has a custom sandbox implemented on the same Linux kernel
features underlying Docker. This means that existing Linux binaries can run in
the Sandstorm sandbox. In fact, many of the existing apps were created by just
copying over binaries from my local Linux installation.

Currently the sandbox is mostly based on chroot() and unshare(). In the future
we'll lock it down more with seccomp-bpf syscall filtering.

> how are unresponsive apps managed?

Currently, an app server is killed off after a few minutes of the app not
being open in any users' browsers. The server starts back up the next time you
open the app.

Perhaps it also makes sense to stick in a "reset" button for apps that are
wedged. That would be pretty easy to implement, but hasn't been needed yet.
(You can also just "killall sandstorm-supervisor" if you have shell access to
the box, then refresh the app. It should come back up gracefully.)

> can apps use protocols like websocket, webrtc?

WebSocket is implemented now. WebRTC is not, but certainly could be some day.

> how are data backed up/restored?

Nothing has been implemented yet on that front, but eventually it will be
possible to download a tarball of a particular app (or all your apps) easily,
and then re-upload it to the same or a different host later.

Of course, hosts should also implement a backup solution so that users don't
have to worry about data security. Any existing solution that just dumps the
hard drive should work fine with the current Sandstorm implementation (just
back up /var/sandstorm).

~~~
teleclimber
> This is a bit different from -- but IMO superior to -- the Chrome Extension
> and Android models.

Yes please don't emulate the Chrome/Android permissions functionality
(particularly Android).

Android apps often ask for broad permissions upfront just to do a one-off
thing, like inviting your friends to use the app. Then they get to keep that
permission for as long as the app is installed.

And it's "take it or leave it". If you're not happy you get to stop using the
app/service, or you just have to bite your tongue and install anyways. Ugh.

~~~
kentonv
Exactly. In the Sandstorm model, the app would ask for a list of people to
invite by telling the _platform_ to present you with a picker UI. The platform
knows all your contacts, but the app only ever receives the ones that you
select. Yet, from your perspective, the UI is effectively the same as what
you'd see on Android.

------
shliachtx
I was just thinking about this problem today. Specifically, I was wondering
how it would be possible to make a decentralized, hyper-context-aware service,
a la Google Now.

I can't wait to see where this goes, and if the above is indeed possible. That
would be truly awesome.

------
DVassallo
> Since every end user is running their own instance of your app, you never
> need to worry about scaling. You can pretend your app only runs on a single
> machine.

What about "big" users? Scaling is not just for multi-tenant systems.

~~~
kentonv
Yes, that is a good point. Let's get this out of the way: not all apps fit
into this model. If you as a single user have needs that require the power of
more than one machine, Sandstorm might be the wrong platform for it.

But a lot of what users need from day to day fits quite well.

Also, Sandstorm does potentially have an answer for things that do really need
multiple machines: you can have multiple instances of the app that talk to
each other through Cap'n Proto RPC. But obviously, at that point, it's not so
much simpler than the status quo.

------
gaddygaddy
If you own a Raspberry Pi then you can try our open beta
[https://www.gaddygaddy.com](https://www.gaddygaddy.com) where you can install
and run apps on your own Raspberry Pi. Everything is managed via a web user
interface and you have full control of the applications and the data. We are
already offering 10 different applications including cloud storage(Seafile),
NAS, photo web servers, minecraft, audio etc. We really believe in the concept
of let the user take full control of the services and provide an easy to use
experience including configuration management, status of their applications
and troubleshooting

------
dwrensha
I can personally attest that it is straightforward to port Linux-based web
apps to work on Sandstorm. Over the weekend, I ported a game of mine. You can
play an instance of it here (warning, makes sound):
[https://alpha.sandstorm.io/grain/ozfn8bSxa6Ygt74YA](https://alpha.sandstorm.io/grain/ozfn8bSxa6Ygt74YA)

If you have a Sandstorm server, you can install your own instance of this game
by grabbing "Duoludo" from the Sandstorm "app store":
[http://sandstorm.io/apps](http://sandstorm.io/apps)

------
pjungwir
Wow, I'd love to pay for an email app. This also seems great for any SaaS
company that wants to sell to customers who insist on self-hosting. I hope
this really takes off!

------
habosa
This seems great, but I'm having some trouble thinking of a particular example
where I'd want to use it.

Can you give a brief example of a (real or imaginary) app that would run on
Sandstorm and how I would benefit from the Sandstorm model in this case? I'm a
programmer but I'd appreciate a "layman's terms" explanation if possible.

~~~
lovemenot
There's Vendor Relationship Management, which appears to be a group of fellow-
travellers promoting technologies with a first-person first perspective. A
recent blog post by Doc Searls mentions some existing technologies that are
somehow affiliated to ProjectVRM.
[http://blogs.law.harvard.edu/vrm/2014/03/19/why-we-need-
firs...](http://blogs.law.harvard.edu/vrm/2014/03/19/why-we-need-first-person-
technologies-on-the-net/)

------
incision
Very cool.

I was trying to wrap my head around how to create something like this with
CoreOS, but found myself lacking the know-how to turn it into more than a
half-baked toy/concept for myself.

Looking forward to trying this out and learning something from the repo.

------
ent
The site isn't really clear on what this actually is. "More than just a VM" \-
so it's a vm? A linux distro geared towards hosting "cloud" services? All I
can really tell is that it has (a few) apps.

~~~
kentonv
It's a server program you can install on your Linux box, which then allows you
to easily install other web apps through a web interface, each of which runs
in a separate secure sandbox.

------
whitehat2k9
How would this work for apps that are multi-user, like email, social
networking, or marketplaces? The Sandstorm data model doesn't really work when
you're dealing with the data of many users besides yourself.

~~~
kentonv
Federated networks! Your server will be able to connect to other people's
servers through Cap'n Proto RPC interfaces (or perhaps other protocols, if you
grant them general network access). So your social network app would host just
your profile, and people would connect to it to get your feed, or your server
would push your feed to servers of people subscribing to you. "Friending"
someone would mean obtaining a Cap'n Proto capability to read their feed (but
the user doesn't have to understand that).

There have been attempts to build federated social networks over the past few
years, such as Diaspora, but they have not really taken off primarily because
it's too hard to get users to run their own servers. Sandstorm potentially
fixes that.

------
blakesterz
Can anyone comment on how this is different/better/worse than owncloud?

~~~
kentonv
As the author of Sandstorm I'm biased, but this is my take:

OwnCloud is a great idea and has a lot of great features, but as a platform
for "apps" it has some big limitations:

\- AFAIK, there is no sandboxing. All apps can access all data, apps can
"phone home" without permission, and one bad app can break your server. Under
Sandstorm, each app instance runs in a separate sandbox and can only talk to
other apps (or the internet) with permission.

\- OwnCloud apps must be written in PHP, whereas Sandstorm can support any
Linux-based tech stack.

That said, it would totally make sense to see OwnCloud as an app running
inside Sandstorm.

------
retr0h
From the readme:

At present, Sandstorm's sandboxing is incomplete. Malicious code probably can
escape. Malicious code definitely can DoS your server by consuming all
available resources.

~~~
kentonv
Yes. But we know how to fix it, it's just work.

------
sbt
Looks great, but I had some trouble building this (Clang 3.5 threw stack
traces all over the place). Would be great if there was a Vagrant box
available to try this.

------
ThinkBeat
uh... soo....

Its good to run our own server for control, freedom, security and so on. Makes
sense. I have been doing that for decades but ok.

So now we add the term "cloud" to it, and wrap it up. Some great ideas that I
like sure.

But.. I can rent my own private server on their cloud platform...... HOw is
that different from almost every other vendor.

Inside my house/business sure.

~~~
kentonv
It's different in that you get to choose your host and your apps separately.
If you want to use some particular app, but you don't trust that developer,
you can run it on a different host that you do trust. If you don't trust the
Sandstorm team to host your server, then by all means, don't get hosting from
us. You can run your own (the code is all open source), or get your friend
whom you trust to share the instance they set up, or perhaps there will even
be competing paid hosts some day.

------
russell_h
I don't get it, is this just native apps in some sort of sandbox?

Creative marketing though, "Personal Cloud Platform".

~~~
kentonv
Sort of.

The apps talk to the world in a different way that normal native apps would.
All communication goes through a Cap'n Proto RPC connection that is handed off
to the app at startup. An executable is provided that knows how to bridge this
to HTTP, making it relatively easy to port in existing apps, but the real
power of the platform will eventually be exposed through that RPC connection.
Inter-app communication will be through Cap'n Proto RPC, which is capability-
based, allowing apps to expose APIs to each other and letting the user easily
and security control which apps can talk to which other apps. Compare to
Docker, where you can open ports, but don't have fine-grained control over
capabilities. (Sorry, this really deserves a lot more explanation...)

Also, the idea is that you don't isolate apps, you isolate individual objects.
So for Etherpad, each document is in a separate sandbox. This means that the
platform can implement sharing for you, by doing it at the instance level. It
also means that if I share a document with you, I don't have to worry that a
security bug in the app will allow you to get access to all my other
documents. Only a platform bug could allow that.

Finally, there's a nice web-based UI for installing and managing apps, so even
non-sysadmins can actually run their own server.

~~~
senand
Can you elaborate on what you mean with "capability-based". How do developers
define capabilities? How does a data-interchange format support them?

~~~
kentonv
Sure!

A "capability" is just an object reference, like a pointer in your favorite
programming language, except that a capability not only identifies the object
it points to, but also confers permission to use the object. So if you receive
a capability, you can use it. If you don't have the capability, you can't use
it.

Cap'n Proto is a capability-based RPC system in that capabilities are a first-
class type. That is, any message can contain capabilities pointing to remote
objects, and thus the recipient gains access to those objects upon receipt of
the message.

In terms of Sandstorm, an app will be able to say "Hey platform, I have an
object here that implements this Cap'n Proto interface 'Foo'. Here's a
capability (pointer) to it. The user may redistribute this capability at their
discretion." Later on, another app may say "Hey platform, I need a capability
that implements interface 'Foo'." Then, the platform presents the user with a
UI listing all of their matching capabilities and asking them which one to
use.

The brilliant bit here is that when the user makes a choice, the platform then
hands that capability to the app, and all of the access control _just works_.
No need for an OAuth dance, no need to ask the user "Do you want to permit
this?" \-- obviously they do, otherwise they wouldn't have made the choice.
From the point of view of both apps and the user, the whole interaction is
trivial. And, critically, the permissions transferred were only for the
specific object the user chose, as opposed to broad permissions for the second
app to manipulate the first. No "I need access to your contact list, so that I
can display a picker for you to choose which friends to invite", instead it's
just "Please use the system picker to choose which friends to invite, and I'll
never see anyone you didn't select in the first place."

See also: [http://en.wikipedia.org/wiki/Capability-
based_security](http://en.wikipedia.org/wiki/Capability-based_security)

------
quarterwave
"Sandstorm makes it easy to run web apps on your own server."

Could that server be one's mobile phone?

~~~
kentonv
In theory, on Android, you could probably get it to run. There are some
practical obstacles, though: \- Your carrier probably puts you behind NAT, and
your IP address probably changes all the time as you move around. \- Your
server would be inaccessible whenever you lost signal. \- Sandstorm apps are
built for x86-64, but your phone is probably ARM. \- You of course would need
to root your phone.

------
danielweber
So, how can I war dial with this?

:)

