
Ask HN: How can I create a decentralized GNU-social-compatible website? - rayalez
Hi! I&#x27;m working on this project, it&#x27;s a reddit-like open source social network, written in django.<p>Beta is almost complete, and now I really want to make it decentralized and compatible with GNU Social. I&#x27;ve never done this before, and some advice would be really useful to me at this point.<p>Where do I start? Which technologies should I pick? Are there any great resources&#x2F;tutorials about this stuff?
======
paroneayea
So I have quite a bit of bias, since I work on a specification in this area.
So yes, if you want to make sure it's GNU Social (formerly StatusNet)
compatible specifically, use OStatus, which is a number of protocols combined
in a specific way.

The most unfortunate part of all this though is that all these different
applications, GNU Social, Diaspora, Pump.io, etc all speak different protocols
and aren't all really interoperable. This leads to a "fractured federation",
which really isn't great. I've been participating in the W3C Social Working
Group to try to standardize federation. Here's our spec, ActivityPub:

[https://www.w3.org/TR/activitypub/](https://www.w3.org/TR/activitypub/)

Some background: it's derivative of the "Pump API", which is sort of the
successor of OStatus, in the sense that the primary author of StatusNet (now
GNU Social) wrote the Pump API in response to some perceived shortcomings in
OStatus. We've done a lot of analysis to make sure that the protocol can
handle all the kinds of things that you expect from the "big player" social
networks, but in a federated setting.

Interested? Now's a great time to research and give feedback. We're looking to
move to Candidate Recommendation status shortly, which means we're looking for
feedback and implementations.

PS: I'm also involved in MediaGoblin, and we definitely intend to implement
there. If you're interested in a bit of news from that end, here's a
moderately fresh blogpost, including info on how to give your feedback:
[http://mediagoblin.org/news/tpac-2016-and-review-
activitypub...](http://mediagoblin.org/news/tpac-2016-and-review-
activitypub.html)

We're planning on using it for MediaGoblin.

------
chriswarbo
Not directly answering your question, but does anybody know of any server
software I can run to take part in such decentralised/federated/etc. networks,
which isn't complicated by multi-user, Web-frontend bloat? GNU Social,
pump.io, etc. and even Matrix, XMPP and IRC, etc. all seem to assume I want
thousands of simultaneous users. I don't; I'm the only user, but I want to
integrate into the network.

Signing up for an account on someone else's server seems to defeat the point,
yet I haven't found anything which I can run persistently on my server, access
in a simple way (e.g. via SSH, a shared secret, a named pipe, etc.), and
doesn't make me uncomfortable allowing it online (i.e. dynamic Web stuff).

I used to use Laconica (AKA StatusNet AKA GNU Social) a lot via XMPP, but then
identi.ca switched off their bridge, and then switched to pump.io which
doesn't seem to work with anything other than a Web frontend.

How would Usenet compare? I never got into it, but hear it's filled with spam
:(

~~~
mxuribe
If I'm not mistaken, with Matrix you do not have to run a front-end per se. I
_believe_ you can simply setup a matrix home server, and then interact via an
open source mobile/desktop/web application. This is not unlike old school
email, where you can host your own email server, but manage email via any
mobile/web/desktop email clients (of which you don't need to host the client
side) .

For example, I'm running a private (matrix) synapse home server for _ONLY_ my
family; as a test. I only interact via the mobile (open source) riot
application (formerly named vector.im), though they do also offer a web
application at [https://riot.im](https://riot.im) . I simply point the client
to my server's (appropriate matrix) url, and am off to the races - again
internally for my family and I only at this point.

My plan eventually is to keep running my own matrix home server, and open it
up so that my family and I keep using any old client (like
[https://riot.im](https://riot.im) or whatever) _BUT_ begin to interact with
other users using matrix clients/servers, etc.

My understanding is that what you wish to do ("...I'm the only user, but I
want to integrate into the network") is actually quite common for folks
beginning to use matrix. Give this a look-see:
[http://matrix.org/docs/projects/try-matrix-
now.html](http://matrix.org/docs/projects/try-matrix-now.html)

Alternatively - less ideal of course but - you can always lurk around on
certain matrix-related rooms on the first/original matrix.org home server, and
ask questions (on how you can set up your own server, etc.). Because I'm still
in my own test phase, that's what I'm doing. Here are a few channels (via
riot.im): *
[https://riot.im/app/#/room/#matrix:matrix.org](https://riot.im/app/#/room/#matrix:matrix.org)
*
[https://riot.im/app/#/room/#riot:matrix.org](https://riot.im/app/#/room/#riot:matrix.org)
*
[https://riot.im/app/#/room/#homeowners:matrix.org](https://riot.im/app/#/room/#homeowners:matrix.org)

Good luck!

~~~
chriswarbo
> If I'm not mistaken, with Matrix you do not have to run a front-end per se.
> I believe you can simply setup a matrix home server, and then interact via
> an open source mobile/desktop/web application.

If by "front-end" you mean Web server, then as far as I'm aware none of these
networks (GNU Social, pump.io, Matrix, XMPP, email, etc.) require one per se.
The problem is finding an implementation which doesn't bundle one.

Looking at the servers listed on matrix.org, it appears that Pallium makes
this separation: you can run Pallium stand-alone, and connect to it using
client applications (which may or may not be Web-based). Unfortunately,
browsing the code for all of those servers, they _all_ seem to include
facilities for user registration, authentication, etc.

Maybe that's a requirement of the way Matrix works? I know that, say, an IRC
server which only allows a single user would be a lonely place, since
chatrooms are hosted by a particular server which all participants must
connect to (logically, at least; I know there are ways to implement redundancy
like FreeNode does).

~~~
mxuribe
Head on over to: [https://github.com/matrix-org/synapse#synapse-
installation](https://github.com/matrix-org/synapse#synapse-installation)
...Then scroll to the last paragraph of this section, "...By default,
registration of new users is disabled." The little command line method is what
i used to manually register the (3) accounts for my family on my private
matrix server. If you plan on using it only for yourself, it might be best to
keep registration disabled, and only add yourself manually via the command
line...and then you would be done.

As far as "web server", I think i know what you mean now, and admittedly I
have my private matrix home server running behind nginx...but that's _only_
because I've set it up via DigitalOcean, and simply picked one of their pre-
set "LEMP" server images. I believe others have installed/run a matrix home
server without nginx or apache, etc. Again, since this is all just a test for
my family, i literally went into my DigitalOcean account, generated a new
droplet (what they call their virtual servers), and picked a "LEMP" pre-set
server. From there, i went installing the synapse server. Because nginx was
already installed, I did have to forward/proxy stuff from nginx to synapse...I
fully expect to kill this test matrix server off. Now having lived this, I
would not have picked the LEMP image, instead just use a basic linux server
from DigitalOcean _and then_ install synapse (and any associated
requirements/dependencies), that way it doesn't have nginx/apache, etc.

I have no experience with Pallium, but I suppose because there are several
different implementations (python-based, Go-based, etc.) of the matrix
homeserver, there could be different nuances for implementation.

 _Side note_ : The DigitalOcean droplet that I'm using for my private test
matrix home server is the $5/month version (w/512MB memory, 20GB storage).
Once I create the production matrix homeserver for my family, I was thinking
on using the next level up $10/month version (1GB memory, 30GB storage).

~~~
chriswarbo
> "...By default, registration of new users is disabled."

It's good to know that it can be disabled, but I'm mostly concerned with
attack surface, minimalism/hackability and not getting caught on an upgrade
treadmill. At the moment my personal Web site is static, the Web UI to my git
repos is static and read-only, pushing to those repos is done via SSH, I
accept patches by email, etc.

I'm hesistant to run anything dynamic at all, let alone something more complex
than I need (e.g. a Web frontend) with the capability to perform destructive
actions which I don't need (even if they're marked 'disabled').

It would be nice to have an Internet-facing server which only deals with
untrusted data, accumulates incoming updates to a file (e.g. sqlite), pushes
out posts from a queue (e.g. a directory) and serves the necessary public info
when requested, maybe using static files for stuff like my profile. I can then
read and write those files over SSH, maybe transforming them into some easy to
consume format like maildir.

Of course, the first rule of security is don't write it yourself, but this is
beginning to sound like a nice weekend project :)

------
chriswarbo
AFAIK GNU Social uses OStatus, which uses WebFinger (for identities), Salmon
(for replies), ActivityStreams (for posting), etc.

Some relevant links from Google:

[https://en.wikipedia.org/wiki/OStatus](https://en.wikipedia.org/wiki/OStatus)

[https://github.com/andstatus/andstatus/issues/378](https://github.com/andstatus/andstatus/issues/378)

I'd look for some libraries which implement those specs. If you can find some
in Python, then integration would mostly be a case of marshalling data between
your representations and those standards. If there's no Python support, then
you could either make a library yourself or use a supported language to make a
standalone translation server to convert between the standard formats and some
some simple raw serialisation of your data.

Edit: also, security/encryption might be a hurdle. I played with Salmon years
ago (using it as a plugin for the ocPortal CMS) when it was first announced;
the example code was trivial to get up and running, but it required some not-
so-straightforward crypto/certificate stuff to properly implement the "magic
signature" part. IIRC this couldn't be done up-front by a developer; it would
require some work for each deployment.

------
rogerbraun
I looked into this a while ago. Here are some notes I made:

\-
[https://git.heldscal.la/roger/the_wired/wiki/OStatus](https://git.heldscal.la/roger/the_wired/wiki/OStatus)

\-
[https://git.heldscal.la/roger/the_wired/wiki/Salmon](https://git.heldscal.la/roger/the_wired/wiki/Salmon)

Hope it helps. Essentially, you can basic functionality going by implementing
webfinger and an atom feed. This old guide might be helpful:
[https://web.archive.org/web/20120306210855/http://ostatus.or...](https://web.archive.org/web/20120306210855/http://ostatus.org/2010/10/04/how-
ostatus-enable-your-application)

------
jajern
You'll probably want to start with the GNU Social source code and also take a
look at mastodon to see how they hook into GNU Social.

As for decentralizing, my thought would be that it should have been a
developed around that from the start. I'm not an expert in it but I think that
projects would be built around the decentralized idea and not later modified
to be decentralized.

------
clacke2
You're on Python, but it might make sense to look at the choices made by
[https://github.com/hotsh/rstat.us](https://github.com/hotsh/rstat.us) and
[https://github.com/gargron/mastodon](https://github.com/gargron/mastodon) ,
to get some ideas. They're in Ruby, but the language is more similar to Python
than PHP is, and some of the tooling and libraries in Python and Ruby are
inspired by each other too.

It always helps to look at those who came before you.

------
marknadal
Is GNU social a spec?

You might be interested in general P2P decentralized tools like
[https://webtorrent.io/](https://webtorrent.io/) and
[http://gun.js.org/](http://gun.js.org/) . There are tutorials that go with
them, but nothing specific to GNU.

~~~
chriswarbo
> Is GNU social a spec?

It follows
[https://en.wikipedia.org/wiki/OStatus](https://en.wikipedia.org/wiki/OStatus)

> You might be interested in general P2P decentralized tools like
> [https://webtorrent.io/](https://webtorrent.io/) and
> [http://gun.js.org/](http://gun.js.org/)

Would client-side solutions like these be useful for a django app? GNU Social
uses federated servers (like email), so I think it would make more sense for
the poster to do federation server-side too.

------
dethos
I've been thinking about building some reusable django app that could be used
in other projects to integrate and make them compatible with gnu social, but I
never put the idea into practice (I still think it is a good idea).

Recently someone posted here on HN a compatible server
([https://github.com/Gargron/mastodon](https://github.com/Gargron/mastodon)),
it isn't what you are searching for, but these 2 posts about the topic in a
short period of time shows there is an certain interest.

Maybe its the push I need to start building that reusable app. Regarding your
question, you should start by looking at
[https://www.w3.org/community/ostatus/](https://www.w3.org/community/ostatus/)
and the wiki.

------
mxuribe
If you are interested in continuing to leverage your python skills (I assume
this because of your ref. to Django), then perhaps building a bridge between
Matrix and Gnu Social would be something _BOTH_ communities would benefit
from: [http://matrix.org/docs/projects/try-matrix-
now.html#servers](http://matrix.org/docs/projects/try-matrix-now.html#servers)

------
dorfuss
Have a look at Diaspora, it could be at least an inspiration:
[https://en.wikipedia.org/wiki/Diaspora_(social_network)](https://en.wikipedia.org/wiki/Diaspora_\(social_network\))

------
cabalamat
I wonder whether it would make sense for work in this area to (like Git)
distinguish between plumbing and porcelain. So once the plumbing is written
(with a reference implementation in Python or similar) it would hopefully be
an easy matter for people to write porcelains to top of it.

------
teekert
Use IPFS [0] and take a look at Identi.ca [1].

[0] [https://ipfs.io/](https://ipfs.io/)

[1] [https://identi.ca/](https://identi.ca/)

~~~
chriswarbo
[http://identi.ca](http://identi.ca) runs on [http://pump.io](http://pump.io)

It used to run on (what is now called) GNU Social, which is what the poster is
asking about.

