

Intro to Yada Project - An Open Source p2p Social Network (proof of concept) - pdxwebdev
http://yadaproject.com/

======
jonpaul
_"Yada is a patent pending method for implementing a peer-to-peer based social
network without the use of a ANY central server."_

Is that a joke? Seriously?

~~~
pdxwebdev
No, definitely not. Please read below and go to yadaproject.com to see why you
should help out with this open source project.

------
gdberrio
Pardon what may seem to be a silly question (amateur "would-be" engineer
speaking), but I have a doubt, which is the following:

Data needs to be stored. You either store it locally, and as such it is only
accessible when the device is on, or it's stored on a central repository or
index, meaning it's always on even if one of the devices is off.

How do you tackle this? How do you make sure that user A tries to access news
feed of user B, and fails to get it because user's B device is offline?

Maybe I'm being slow or noobish, but it seems to be the same problem one has
with bitorrent: user A downloads from user B if user B is online. If data is
common, I have multiple seeds, but social network implies that nobody has the
same data. Unless you copy my graph to my friends "device" so one has multiple
seeds, which could be quite unpractical on mobile devices if I have a lot of
friends.

How do you solve this "intermittent issue" and the synchronization and
persistence problems it generates? Without a central "clearing"/indexing
server? Even Skype has "Super nodes" to handle this issues...

This aside from the fact that "IPV6 deployment" is not going to be a easy cake
to bake, one has to fix the problems faced by GPG trust model to ensure that
two parties can verify (with some degree of trust) that the other party with
whom they're communicating is, indeed, the person they expect, dynamic IPs,
and so on.

~~~
pdxwebdev
You can have your data hosted if you don't believe your device will be online
all the time. This could open up some ad-based services.

~~~
gdberrio
I don't think that answers my question on intermittence, persistence and
syncronization.

I can have my date hosted, but if my friends don't (that's why they use your
service, so not to have to do that, or else they remain with Facebook,
LinkedIn, etc. That's your USP), I can only access their data when my friends
devices are online. Or, alternatively, you store my whole friends graph on my
device, which presents scalability and performance issues. Or, 3rd route, you
store my friends data on multiple devices on the network, doing "multiple
seeds". Again, we're back to "opps my data is all over the place", when your
Unique Selling Point was P2P = privacy, no need to centralize.

Even Skype Supernodes don't solve this issue. They solve some of the routing
issue. Not the "address book/data issue". You don't seem to solve it either.

~~~
pdxwebdev
"Or, alternatively, you store my whole friends graph on my device, which
presents scalability and performance issues."

With all due respect, this is where you are incorrect. Devices today
completely capable of handling a large amount of compressed plain text data.
Media would will be links to external sources as storage capacities play catch
up.

The CPU of the device would be able to map your entire social graph and do
things that are not possible today being that calculating a social graph on a
central server for each user is EXTREMELY expensive and basically impossible
with high-load services. Kind of like the concept behind folding @ home.

------
mayank
There appear to be a number of problems with this proof of concept:

1\. A requirement seems to be the ability to make and receive socket
connections. How would you address the typically restrictive home
firewalls/NATs that most people are living with?

2\. In the responses, a commenter raises this issue: _With the above example
of two users and 56 connections it seems like there could be scaling issues
and or perormance issues. They may be less obvious on computers but mobile
devices with more limited bandwidth and sometimes quotas may suffer for those
users with muilple devices and their multi device highly connected friends._

The response from admin is: " _Correct, this technology will have to scale
along with hardware. I don’t believe we are too far off with 4g, more
inexpensive memory and the like._ "

That doesn't inspire much confidence in the idea as a proof of concept.

You're also claiming to solve a far more fundamental problem: that of creating
and maintaining a reliable distributed peer-to-peer network without any
central servers at all. If you do have a _practical_ solution to this problem,
then a bunch of other difficult problems (like tracker-less Bittorent, for
example, which currently relies on distributed hash tables) reduce to yours.
Are you sure you're able to make that claim?

~~~
pdxwebdev
"1. A requirement seems to be the ability to make and receive socket
connections. How would you address the typically restrictive home
firewalls/NATs that most people are living with?"

IPv6 is already being deployed from top tier mobile carriers and that allows
globally rout-able addresses even behind NATs.

"That doesn't inspire much confidence in the idea as a proof of concept."

This technology is a little ahead of its time, but not by much. And by the
time an end-user ready core is developed, we'll we be there.

"Are you sure you're able to make that claim?"

yes, the technical hurdles are only time consuming, not insurmountable.

~~~
mayank
> IPv6 is already being deployed from top tier mobile carriers and that allows
> globally rout-able addresses even behind NATs.

With all due respect, if your solution depends on the following developments
in the real-world:

(1) Everyone upgrades their cheap DSL/cable home modems to something that
supports IPv6 (2) Cell carriers allow incoming mobile data connections (can
you say mobile DoS?) (3) People modify their firewalls (again, nothing to do
with IPv4 or IPv6) to allow incoming connections (4) You devise some way of
allowing static or otherwise identifiable IPv6 IPs, even when the ISP doesn't
give you a static IPv6 IP

then I would say you have a number of challenges ahead of you. Assuming that
device X and connect to device Y on the Internet, for consumer devices, is a
very difficult proposition.

~~~
pdxwebdev
Thanks for the reply, but take Skype for example, they have gotten over these
technical hurdles. I could give you a huge explanation involving relay points
and NAT traversal etc. but at the end of the day, these challenges have been
defeated.

~~~
mayank
Skype uses a number of super-nodes that provide some guarantees on service
level and has a set of central login servers:
<http://www.mjalali.com/blog/?p=10>

~~~
pdxwebdev
Yes, through mutual friends and indexers with persistent connections, these
challenges are mitigated.

------
michaelchisari
Have you spoken with the Diaspora team? This was their exact idea, until they
started implementing, and the practical concerns moved them back to the
client-server, node-based model.

~~~
pdxwebdev
There model is very different, though I don't know their history.

~~~
michaelchisari
Their model is very different now, yes, but their original presentation was
based on this exact concept. I would suggest looking at their history to see
why they moved away from this design, and if possible, speaking with them or
discussing on their mailing list the pitfalls they ran into.

~~~
pdxwebdev
I'll speak with anyone willing to help solve technical challenges associated
with this model. But until someone comes up with something "damning" then I'm
going to keep moving forward.

~~~
michaelchisari
I wish you the best of luck, and I admire your ambition (and hubris) but as
someone who has been working on distributed social networking (Appleseed) for
over half a decade, I'll present this:

You don't always need a singular, damning situation in order to prevent a
model from being implemented. Sometimes dozens of purgatorial ones are plenty.

~~~
pdxwebdev
It just takes time and dedication, the motto of any true software developer.

Using the CPU power of a device that is capable of playing video games to
calculate your social graph (that is stored on that same device) is extremely
powerful and well worth the time and dedication.

------
nyellin
Sorry, but I don't understand how this works without a central server.

edit: Namely, how do you securely add a friend without prior knowledge of his
public key?

~~~
pdxwebdev
Checkout the how-to page and watch that video. It gives more detail.

But in short, you can make friends with your friend's friends or you can be
friends with an indexer, which will list a bunch of people you may or may not
know that you can request to be friends with as well.

When you send a request through a friend, your connection is done securely
through that already established relationship. Then that friend passes the
request to the friend you want to be friends with that they are already
friends with. Once that request reaches that friend and they accept, they will
have the public / private key pair you originally generated. They will also
have your IP address information and begin making secure connections directly
to you without the need for the friend or indexer which you sent the request
through.

~~~
dinedal
How do you intend to tackle NAT, dynamic IP addresses, and devices that aren't
always on, or devices only accessible to some machines but not others?

How do you intend to solve the chicken and the egg problem this network will
encounter?

~~~
pdxwebdev
IPv6 is already being deployed from top tier mobile carriers and that allows
globally rout-able addresses even behind NATs.

If you have no social graph on yada, then your very first connection to a
friend may have to be unencrypted, but not in every case.

~~~
dinedal
> IPv6 is already being deployed from top tier mobile carriers and that allows
> globally rout-able addresses even behind NATs.

You're making a lot of assumptions:

1) That IPv6 will assign a unique and constant IP to every device. Many ISPs
provide this as an up-sell feature, not the default, and I don't expect them
to change their behavior even if the cost of it approaches 0.

2) That IPv6 will solve all your network problems. It doesn't solve the issue
of a bad link or a restrictive firewall. Consider Hosts [A,B,C] and Firewall
F. F will block all connections from B. How will yada connect A and B in this
arrangement: A <-F-> [B,C] ? Will is use C to route the connection? How will
it determine that C can act as an intermediary, and how will it protect A's
data such that C can not be used as a MTM (Man in The Middle) attack vector?

~~~
pdxwebdev
1) Each client checks for internal and external IPs and saves them. That list
is transmitted to your friends every time you update them with other data. It
automatically manages this list of possible places to reach you.

2) There will be indexers for these cases which can be used as relay points. C
would merely be a match-maker, telling each host which IP address and port is
currently open. No data is sent through C.

Skype does this.

~~~
dinedal
Skype doesn't do this. Please read: <http://www.mjalali.com/blog/?p=10>

>1) Each client checks for internal and external IPs and saves them. That list
is transmitted to your friends every time you update them with other data. It
automatically manages this list of possible places to reach you.

According to the link, "supernodes" route connections and provide matchmaking
between Skype callers. How do you find the "supernodes"? "If this file is not
present, SC tries to establish a TCP connection with each of the seven Skype
maintained default SNs IP address on port 33033."

>2) There will be indexers for these cases which can be used as relay points.
C would merely be a match-maker, telling each host which IP address and port
is currently open. No data is sent through C.

"If caller and receiver are behind a UDP-restricted firewall they will need a
relay (node) in between to establish TCP connection to and then the traffic
(including media) will go through from one side to the other."

Skype's protocol would use host C as an intermediary.

~~~
pdxwebdev
That's true, its not exactly the same as your example. But the method I have
developed does work.

~~~
dinedal
You didn't address my point about initial discovery being maintained by Skype.

Also, could you please point out to me the method you have developed for
initial discovery? I've yet to see it in your source code. This is a very
tough nut to crack and if you've got it done already then I'm impressed.

~~~
pdxwebdev
And if you're impressed, what happens? Do you help me? Because I'll fight
tooth and nail to impress you if you'll help me.

Seriously.

~~~
dinedal
Show me one place in your code right now where you have done something
innovative with peer discovery, that no network engineer, or researcher has
tried before, doesn't use a central meeting point, and is successful at
discovering peers in a reasonable amount of time. I'll start work immediately.
I'm pretty sure more people will join in on it too if this is the case.

~~~
pdxwebdev
That is not coded at this time but you shall see, sir.

