
Software design mistakes that Diaspora needs to fix. - imaginator
http://buddycloud.com/cms/node/204
======
joe_the_user
The basic argument is "design a protocol (damn-it)".

I agree with this point, I've made it here and I don't think I'm the only one.

But everyone knows the Diaspora folks went the "product first, protocol later"
route and a fair number of people agree this approach. So it's a disingenuous
little to act "oh, they made a basic mistake so it just won't work." They went
a route, at least, that lots of people think will work. It's not like they
used hard coded eight-bit node identifiers, had each node multi-cast its
information or some other design decision would _guarantee_ things couldn't
work.

~~~
joe_the_user
The other thing with protocols is...

Naively, on first glance, to me Facebook looked like just a nice interface on
top of standard Internet functionalities. And there are standard protocols for
all of these.

* Messengering we have * Micro-blogging we have. * Selectively sharing information is pretty simple. Give your "friends" a password to your website and let them connect via SSL or Oauth or something. ...

Nothing _seemed_ that new. My first thought was, "why couldn't you just slam a
UI/Application on top of this existing stuff and call it an open Facebook
implementation"? Ah, but there is a funny thing I realized looking at the
situation (and discussing it here). The _unique selling point_ of Facebook and
its open competitors boils down a "share and revoke" function. The umpteen
Facebook users want to imagine they are "safe" when "privately" sharing their
photos. They don't look too carefully into what "safety" or "privacy" involves
and they probably don't want to look that carefully but they really want to
have it. Facebook has flattered them into thinking they can just carry their
real world intuitions into virtual space. Of course, this isn't quite "real
world privacy" as a number of fired employees can atest.

Thus the "competing Facebook protocols" aim to provide something like
"personal DRM". Without going into the details of the inherent problems, I'll
just ask, "why would you want to?".

I would argue that rather than trying to simulate something that's impossible,
that Facebook doesn't and can't give you, why not educate people. Allow secure
sharing of information but start people learning about what makes sense to
share or not. And sure, that's a big job but the Internet itself is kind of
doing that.

Looking at the situation as a whole, Facebook is leveraging people's lazy,
foggy and "intuitive" notions of privacy taken from the real world. Anything
Facebook-like is going to "leak-like-a-sieve" and the more people learn about
the Internet, the more they will realize this.

------
mikesofaer
I don't agree that it makes sense to "design a protocol" before building the
application. You can try to imagine what you want the protocol to look like at
the end of the day, but you are likely to get it wrong if you don't have an
application that's actually using it. It's important to keep your eyes open
and try to avoid going down blind alleys, but there's not much value in
building infrastructure before it's needed.

The questions raised in that post are good, and we intend to address them as
they become relevant to what we have build. Discovery, namespacing and routing
are areas where we are starting to feel tension on in the product, and will
probably be tackled quite soon. The point about affecting interface design is
good, but we look at it the other way. We want our interface design to drive
the protocol design, so we want to make protocol choices as late as is
reasonable.

Finally, it's fantastic that people are reading and forking the code. The more
people out there trying things, the better.

~~~
michaelchisari
As someone who has been working on this for a very long, my advice is to write
the software to be protocol agnostic. Be able to switch out protocols easily,
and support multiple protocols concurrently, because we still have no idea
which protocol will win out. It may be something that hasn't even been built
yet.

~~~
kenjackson
I agree, but this is harder than it sounds. Why? If this is your first time
building such a system you don't even know what is protocol and what's not.

There's an old saying that you need to build three apps on your framework.
That first app shows that your framework works for something. The second app
shows that its more general than just the first app. The third app shows that
you weren't lucky.

Curious, how was Facebook designed? Are there any post-mortem docs on it? I
know that MySpace seemed really ad hoc. I wouldn't be surprised if Facebook
was more ad hoc than we imagine it to have been

~~~
jacobolus
Who doesn’t imagine Facebook’s early design to be “ad hoc”? It was programmed
as a spaghetti mess of PHP on Zuckerberg’s tiny laptop, and only worked
because the user base was inherently limited to a few thousand people. Then
the spaghetti mess was grown and added to over the next year or two, and the
system scaled because each college’s network could be contained in a single
server that only had to support a few thousand users (these servers used to
use domain names like harvard.thefacebook.com or mit.thefacebook.com, IIRC).
At some point they completely rewrote it with an architecture that would scale
and not waste hardware, but it definitely didn’t start out that way.

~~~
kenjackson
_Who doesn’t imagine Facebook’s early design to be “ad hoc”?_

Maybe just me. I don't know much about the history of the project, except Zuck
started it in college.

------
joshfinnie
I think Diaspora could do itself a huge favor by writing up a blog post
explaining how the $200k was distributed. Everyone keeps harboring on the fact
that they have that much money, but everyone forgets that they have quite a
bit of liability with what they guaranteed on Kickstarter.

I think once that is done, people won't be so fast to suggest to them to hire
a $100k/year engineer from Google or other things. Just my $0.02...

[edit: spelling]

~~~
michaelchisari
Yeah, they have to send out 3,296 t-shirts alone, and even if they get a great
deal, and end up paying $5 each (incl. shipping & handling), that's $16,480
just in t-shirts.

------
jdp23
Maybe the old software engineering maxim "build one and throw it away" is the
best way to look at what Diaspora's done so far.

Yes, it would be great to have a well-thought-out protocol and secure
software. But then again there's also huge value in having an implementation
that people can play with. We hear all the time about how important it is to
get your best guess at an minimum viable product into users' hands, and that's
pretty much what Diaspora has done.

~~~
wdewind
IMO what they have released is not an MVP.

The article made the point about federation algo, and I think that's the most
important: proper federation/decentralization IS the product. As of yet it is
an unsolved problem in their camp (and many others). This isn't to take
anything away from them, because its a Big Problem, but if they can't solve
that, a slightly polished CRUD social networking app is just fluff.

The product is the protocol, not the app. That's the only potentially
innovative thing they are doing. Sorry guys, aspects are a product made for
privacy concerned software devs - normies don't care.

~~~
mkramlich
great comment, I had only one quibble or expansion on what you said:

> Sorry guys, aspects are a product made for privacy concerned software devs -
> normies don't care.

I'd argue that normals care about this too. Teens might want to keep their
friend/school/personal stuff private from their family/friends-of-
family/adults-they-know stuff. And adults often want to keep their "adult
adult" stuff hidden or distinct from their family and/or coworkers. The whole
"my mom wants to friend me on Facebook, ew!" or "my boss wants to friend me on
Facebook, noooo!" factor.

~~~
wdewind
Thanks!

Agreed 100%, but this feature alone isn't worth leaving facebook due to
network effect. That may change, but if you have to boil it down to a black
and white, it still leans far to the side of "not worth it to switch." Also,
should this change, I find it highly unlikely facebook would find itself
unable to pivot. Regardless of what you think of fb (I personally hate it) you
really can't deny they have a world class engineering team that has a finger
on the social pulse more than any other company right now.

I have tried building apps around these kinds of features, they don't really
work (you end up building a new interface to email _awfully_ quickly).

------
michaelchisari
I think a huge requirement for success is that distributed social networking
needs to be a framework, not simply an application (as Diaspora is now).
Hopefully they're considering refactoring in that direction, and that
definitely falls into point #1 (Coding before designing). You don't want to
haphazardly code a framework, you need to design it on some level beforehand.

The killer feature that moves people over to distributed social networking is
not distributed social networking itself. But third party developers building
onto a distributed social networking framework is what will allow the next
killer feature to be built.

~~~
nostrademons
Actually, no. Virtually all successful frameworks (Rails, Django, Tornado,
MapReduce, etc.) grow out of an application or set of applications where
common pieces are incrementally refactored out of the application and into the
framework. When you start with the framework and design it upfront, you end up
with an overdesigned mess.

~~~
michaelchisari
Actually, I agree. My framework was built out of Appleseed the application,
and was then re-factored into a framework, and most of the structure was based
on what I had learned from building the app.

I don't think anyone should start by building a framework without any
conception of what that framework is meant to do. But Diaspora already has an
app of sorts, so they wouldn't exactly be doing that.

~~~
kunjaan
Which framework was that you created?

~~~
michaelchisari
The Appleseed social framework. =)

<https://github.com/appleseedproj/appleseed>

------
trotsky
I have really bit my tongue since the beginning of diaspora - I think really
they are simply victims of their own success. If they had a normal amount of
publicity, or if they had gotten the 10 they asked for instead of 200 I think
they'd be much better off. Much hungrier, scrappier. Less expectations to live
up to. Way less people looking over their shoulder as they learn to code and
manage a project. It's got to be a bitch to learn to dance on live TV as well.

I am not saying they are doing a bad job. Just that it's a very hard job to
do. Most rock stars of today that built the next big thing had the luxury of
not having to show anyone the code. And I'd say 90% of the time, that code is
fucked, at least in parts. That's pretty recoverable - your product isn't the
code it's the service or executable. Many opensource projects are similar -
they can be hacky at first because no one even knows they exist - as they get
traction they can fix things as they become issues.

But now with the whole internet breathing down their necks, celebrity status
and a bunch of money burning a hole in their pockets they have no such
luxuries at all.

I am inclined to agree though that they should really consider closing back up
and rebooting both the design philosophy and the code base after folding a few
key contributors into the core team.

Not that I'm right or that I know what I'm talking about, but just my opinion.

------
imaginator
Incidentally I see Ben Nolan has released his xmpp port of Diaspora today.
This goes some way to assuaging my worries about their architecture.

<https://github.com/bnolan/diaspora-x>

~~~
michaelchisari
I think there's two issues with this, though. One, it's somewhat dangerous to
have an incompatible fork so early on. Of course, the code is so immature that
this may not be as big of a deal at this point.

The other is that I'm not sold on XMPP as the best protocol for distributed
social networking. I think it can be overly verbose, and a simple protocol
over HTTP/HTTPS makes more sense to me, and I've had good luck with that so
far. That said, I'm not completely sold _against_ XMPP, either.

~~~
imaginator
There's two things here:

Protocol and the architecture. Protocol-wise I have no problem with http -
it's well understood and any RoR dev can pick it up an use it. And indeed you
could even, with a lot of work, build a federated network off http(s).

But http is not a a federation architecture. XMPP provides this out the box
and makes bootstrapping a secure federated social network much easier.

~~~
michaelchisari
I'm not talking about _extending_ HTTP, though, I'm talking about a protocol
_over_ HTTP. My project uses a custom protocol called QuickSocial:

[https://github.com/appleseedproj/appleseed/blob/master/_docu...](https://github.com/appleseedproj/appleseed/blob/master/_documentation/quicksocial.txt)

Although the software is protocol-agnostic, and hooks could be written to
support XMPP, as well.

I think my biggest issue with XMPP (beyond it's verboseness) is that it
requires a whole separate XMPP server. This is less of an issue for Diaspora,
which already requires a series of services running, but for my project,
Appleseed, the goal is to get everything running under a LAMP stack which can
be run on any shared host.

~~~
imaginator
Don't worry about the verboseness of XMPP - TLS with the deflate option drops
that down to 10% of the original size. Or you can do what google did on
Android and write your own binary protocol representation.

If I setup a new appleseed node and how do you really know it is
me/example.com? I've yet to see that done well in http and would love to be
proven wrong.

~~~
michaelchisari
Right now, all requests require a callback with a token. user a@a.com sends a
request to b@b.com with a generated token. b@b.com then sends a verification
request back with that token to a@a.com, and a@a.com sends back true, and
b@b.com completes the request.

Which, if I'm not mistaken, is very similar to how XMPP does it.

The protocol also tries and minimizes the amount of actions that a third party
node can take. You only trust your own node to take actions on your behalf.

------
SoftwareMaven
"My concern is that hype can also bite when you don't have something to ship
or there isn't a big value-add over existing solutions."

This is critical. Diaspora could spend a lot of time in the architecture tower
with nothing to show, and the early hype will bite them if they have nothing
to show. I think they've done a good job making sure that doesn't happen.

Now that they've shown they can produce _something_ , they need to slow down
and make sure they are producing the _right thing_. When you are building a
simple application, you can get away with design-by-coding; if you are trying
to build a federated, global, social wonder, you are going to need to spend
some time thinking about it.

------
JonnieCache
Just to add to the chorus of voices: yes, Disapora team, please let your
protocol spec define your application code, not the other way around. This one
factor will make the most difference to the long term success of the project.

------
blutonium
FTA: "...build this on XMPP."

[http://www.reddit.com/r/programming/comments/ehhbz/i_made_a_...](http://www.reddit.com/r/programming/comments/ehhbz/i_made_a_diaspora_branch_that_uses_xmpp_instead/)

------
jchonphoenix
At this point, I don't think diaspora will work. The project has lost its
hype, so it has no more fuel. Its heading towards the deadpool. It doesn't
even look like it'll live on as an open source project.

The main issues with the diaspora project, besides buggy code, was that there
was never a good reason to build it. People were not angry enough at Facebook
to start learning how to run their own servers (which costs money) or pay
diaspora to do it (which costs money)

------
roryokane
This is one case where I think the number "Four" in the original title would
have been better left in. Without the number, the title made me think this
site might be a community bug tracker for Diaspora design mistakes. With the
number, it would be clear that this is one person's many ideas.

------
samratjp
Is it me or can't Diaspora just make it easier to run your own Seed server,
even for playing around. Come on guys, stick in on a EC2 AMI and make it
easier for us to play.

(Before you retort, yes, I am capable of spending the hour or so build the
stack, but it's nice to have plug and play sometimes).

------
ams6110
Technical considerations aside, another problem is the name itself. Sounds
like a gastrointestinal ailment. Yes I know what it really means.

------
lhnn
Four software design mistakes that Diaspora needs to fix quickly

FTFY

~~~
steveklabnik
> If the original title begins with a number or number + gratuitous adjective,
> we'd appreciate it if you'd crop it. E.g. translate "10 Ways To Do X" to
> "How To Do X," and "14 Amazing Ys" to "Ys." Exception: when the number is
> meaningful, e.g. "The 5 Platonic Solids."

<http://ycombinator.com/newsguidelines.html>

------
mkross
Because Diaspora is open-source, it attracts this sort of "look at how bad the
code is" attention. I'm willing to bet that the first prototype of
yahoo/google/facebook all had terrible code that may have lasted a long time.
However, by not being open source, no one was able to critique them on their
software design mistakes and instead had to focus on the product itself.

Since it is open source, Diaspora could probably use the writers of these
critique pieces (bad design, security holes, etc.) to actually make a better
product.

(Tangentially, why are there so many blog posts about Diaspora's code? Is the
code actually unforkable? Or is it just because Diaspora painted itself into a
corner promising a secure social network on a relatively quick timeframe and
now everyone wants to prove them wrong?)

~~~
michaelchisari
_Since it is open source, Diaspora could probably use the writers of these
critique pieces (bad design, security holes, etc.) to actually make a better
product._

The writer of this critique, as well as myself, are both working on what we
feel are better products, respectively.

Take that as you will.

