
Ask HN: Recommendations for Peer to Peer Frameworks? - zilchers
I&#x27;m looking for peer to peer framework recommendations to build a distributed app. In particular, a framework that supports seeds and peer discovery, message passing, maybe firewall &#x2F; UPNP support, etc.<p>I&#x27;m not particular about language (Java, Go, Javascript, etc), since it&#x27;s for a new project. I found this thread from 2016 (https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11145691), it had some interesting recommendations, but quite a few were out of date, and with the big push to distributed apps nowadays, I thought it might be a good time to readdress the topic.<p>Some of the solutions form the other thread or that I&#x27;ve found:<p>- Smokesignal (Javascript &#x2F; Node, not updated in &gt;10 months) https:&#x2F;&#x2F;github.com&#x2F;marcelklehr&#x2F;smokesignal<p>- ZeroTier: https:&#x2F;&#x2F;www.zerotier.com&#x2F;product-netcon.shtml<p>- Maki: https:&#x2F;&#x2F;maki.io<p>- Telehash: http:&#x2F;&#x2F;telehash.org&#x2F;<p>- A whole host of blockchain projects, Bitcoin, Ethereum, HyperLedger, Scorex (Scala blockchain)<p>Bitcoin is an interesting option, it certainly has a large user base and is well tested and secure code, but forking it has it&#x27;s own set of issues.
======
0XAFFE
Maybe Scuttlebutt? [https://www.scuttlebutt.nz/](https://www.scuttlebutt.nz/)

Scuttlebutt is advertised as a social network, but in core it is an eternal
distributed log. You can pass either public or private messages and assets
around.

~~~
freshhawk
Scuttlebutt and Dat ([https://datproject.org/](https://datproject.org/)) are
definitely looking like good bets right now.

~~~
yoshuaw
Also perhaps fun to note is that we're currently hacking on a Rust version of
Dat. Hopefully we'll be able to bring the Dat protocol to more languages and
platforms soon! [https://github.com/datrs](https://github.com/datrs)

~~~
robotmay
Oh man that makes me happy. I just posted about that this morning. Hopefully I
can find a way to contribute!

------
rauljordan
Look into libp2p created by the Protocol Labs team (created IPFS). We're
thinking of moving to this framework internally at Ethereum.

~~~
no_identd
>We're thinking of moving to this framework internally at Ethereum.

Why not GNUnet? Check the recently released 181 pages long dissertation on it:

[https://grothoff.org/christian/habil.pdf](https://grothoff.org/christian/habil.pdf)

~~~
tlrobinson
GPL license is probably a non-starter.

~~~
thomastjeffery
Why?

~~~
tlrobinson
> The core of Ethereum includes the consensus engine, the networking code and
> any supporting libraries. For C++, this includes libethereum, libp2p,
> libdevcore, libdevcrypto, libethcore, libevm and libevmface.

> The core of Ethereum will be released under the most liberal of licenses.
> This reflects our desire to have Ethereum used in as many diverse
> environments as possible, even those which, for various reasons can require
> modifications or augmentations to the software which cannot be released to
> the public.

[https://github.com/ethereum/wiki/wiki/Licensing](https://github.com/ethereum/wiki/wiki/Licensing)

------
indescions_2018
Awesome List of P2P resources:

[https://github.com/kgryte/awesome-peer-to-
peer](https://github.com/kgryte/awesome-peer-to-peer)

Personal bias tilts to web tech. WebRTC will require you to centrally
implement all your peer broker and logic. But take a look at something like
Peer5 Video CDN to appreciate scalability:

[https://www.peer5.com/](https://www.peer5.com/)

~~~
inglor
Hey, Benji from Peer5 here - our product is really cool and helps you scale
your video delivery but I don't believe it works in this case.

We solve the problem of distributing HLS/DASH delivery over large p2p swarms
and scaling video. Doesn't sound like it could work for OP.

Thanks a lot for keeping us in mind though :)

~~~
adam_ellsworth
Hey Benji. I saw you guys setting up at NAB yesterday. Would there be a good
day this week to come by and talk shop?

~~~
inglor
Hey Adam, I'm not attending NAB this year - Shachar and Hadar would love to
talk (shachar@peer5.com or hadar@peer5.com) in terms of technical people
attending :)

------
shpx
[https://urbit.org/docs/](https://urbit.org/docs/)

The idea with urbit (amongst others, it's a big system) is for everyone to
have a personal server that's easier to administrate than a linux server. It's
been posted a bunch of times
[https://hn.algolia.com/?query=urbit&sort=byPopularity&prefix...](https://hn.algolia.com/?query=urbit&sort=byPopularity&prefix&page=0&dateRange=all&type=all)

I've also heard about [https://zeronet.io/](https://zeronet.io/) and
[https://beakerbrowser.com/](https://beakerbrowser.com/)

------
petegordon
I’m curious what your goal is. Do you want the complete immutability of
transactions like bitcoin/blockchain; or do you want the peer/seed capability
like BitTorrent and IPFS with a DHT (Kademlia)? I’d be interested in seeing a
good list of the technology choices that can make up a P2P project if anyone
knows of one, please share.

But you definitely should checkout IPFS and Kademlia.

------
mcjiggerlog
Maybe check out Dat[0] - especially useful if you need some kind of
distributed data store. The main implementation is in Javascript.

[0] [https://datproject.org/](https://datproject.org/)

~~~
robotmay
I really wish there was an additional implementation in something like Rust
(personal preference, but C would be good as it's fairly easy to bind to) for
DAT, because what they're doing is rather cool and the experience is tidy.

~~~
jondashkyle
That’s in the works: [https://github.com/datrs](https://github.com/datrs)

~~~
robotmay
Awesome. I'm really excited for that, I'll have to try to contribute.

------
no_identd
The 181 pages long GNUnet dissertation very recently got released:
[https://grothoff.org/christian/habil.pdf](https://grothoff.org/christian/habil.pdf)

I'd strongly recommend going for GNUnet, nobody takes me serious when I say
this, but it WILL surplant IP eventually.

~~~
yid
> but it WILL surplant IP eventually.

The painful move from IPv4 to IPv6 suggests that this is unlikely. More likely
is an overlay over IP, TCP, or even HTTPS.

~~~
no_identd
>The painful move from IPv4 to IPv6 suggests that this is unlikely. More
likely is an overlay over IP, TCP, or even HTTPS.

GNUnet can function as an overlay and an _underlay_ simultaneously, quoting
from the dissertation:

> GNUnet is an overlay network, as it is initially supposed to primarily
> operate over the existing Internet network. However, GNUnet does not assume
> that this is always the case. For example, GNUnet can also operate directly
> over WLAN or Bluetooth to create an ad-hoc wireless mesh network, and with
> the GNUnet “PT/VPN” subsystems we can run TCP/IP over GNUnet. So in this
> configuration, GNUnet would be an underlay network, a bit like MPLS or
> B.A.T.M.A.N. [NAL07]. In reality, we in fact expect to see both at the same
> time: some peers will run over the existing Internet, while others may
> connect to GNUnet on Layer 2.

------
JamesLeonis
The WebTorrent Project [1] has a whole suite of tools for decentralized
applications based around BitTorrent. It's not a framework per-se, but it
breaks up the protocol into libraries for your use.

If you're interested in Ethereum, it has other non-blockchain technologies
that you can use for your apps. It has the Swarm network for storing bits of
data, the Whisper network for chat and messaging, as well as Public-Key
signing to verify messages. I'm using the latter functionality for a demo to
replace password as the security mechanism.

[1]: [https://github.com/webtorrent](https://github.com/webtorrent)

------
jeswin
Secure Scuttlebutt. It's probably the only one seeing everyday use to get
things done. Patchwork, the flagship app is as good as Facebook/Twitter.
Scuttlers often use git over SSB, even.

What SSB gets right and the others don't, is the incentive to mirror or
replicate. Virtual currency is a weak incentive in an early stage network -
but friendship isn't; you mirror your friends' data.

The basic API to program the dapp can be seen here -
[http://scuttlebot.io/](http://scuttlebot.io/) But for more advanced uses see
[https://youtu.be/f_baWUW4R8Y](https://youtu.be/f_baWUW4R8Y)

------
alexmorley
Maybe checkout the dat project
[https://datproject.org/](https://datproject.org/) which is whats used by
BeakerBroswer etc.

------
DyslexicAtheist
Kademlia DHT:

[https://en.wikipedia.org/wiki/Kademlia](https://en.wikipedia.org/wiki/Kademlia)

[https://pdos.csail.mit.edu/~petar/papers/maymounkov-
kademlia...](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-
lncs.pdf)

------
Dowwie
How about IPFS? OpenBazaar is an IPFS case study. They recently released a
revamped v2 built with Go.

~~~
xendipity
Yeah, I'd look at IPFS's networking library, libp2p. It's the peer-to-peer
backbone of IPFS and has a big community and active development
[https://libp2p.io/](https://libp2p.io/). It's also implemented in both go and
js (as most of IPFS is)

~~~
jjallen
We moved away from libp2p at the project I work on because of major
instability problems in browsers. Our dapp that used libp2p would crash after
a few minutes and was generally not easy to use.

EDIT: I'm referring to the JS version here.

~~~
ComputerGuru
Did you try to investigate the stack trace or file a bug upstream? No snark
intended!

~~~
jjallen
Yes, of course, and it was WebRTC. We ended up with a better solution
(encoding the metadata we were storing in IPFS in a bytes32 stored in the
Ethereum blockchain).

There were extra scripts and setup. Overall moving off simplified our stack,
so it was probably a good move despite libp2p's crashing.

Could be totally fixed by now for all I know.

That said, it was believed that most Protocol Labs resources were/would be
devoted to Filecoin, which is probably a safe bet.

~~~
lgierth
We have no intention of letting libp2p stagnate - blog post with announcements
coming soon!

------
povilasb
Another project you might find useful, that we at
[https://www.maidsafe.net](https://www.maidsafe.net) are actively working on,
is [https://github.com/maidsafe/crust/](https://github.com/maidsafe/crust/).

It's a generic peer-to-peer networking library written in Rust.

    
    
      * it connects two peers together
      * various NAT traversal techniques are implemented: hole punching, UPnP/IGD, etc.
      * it supports both TCP and UDP. In case of UDP it uses uTP: http://www.bittorrent.org/beps/bep_0029.html
      * all messages are encrypted
      * it implements automatic peer discovery on LAN
      * it caches it's previous connections and is able to reuse them in the future
      * etc.

------
monocasa
The virtual synchrony protocols don't get enough love. Conceptually, they're
sort of like if you performed PAXOS to determine ordering for a distributed
finite state machine. Let's you build all sorts of distributed applications
where you trust the nodes.

[https://en.wikipedia.org/wiki/Virtual_synchrony](https://en.wikipedia.org/wiki/Virtual_synchrony)

[http://www.jgroups.org/](http://www.jgroups.org/)

[https://archive.codeplex.com/?p=vsync](https://archive.codeplex.com/?p=vsync)

------
billylo
I am using [https://hypelabs.io/](https://hypelabs.io/)

Pretty good for small payloads.

------
wslh
Re blockchain technologies: while there is an obvious intersection between the
world of blockchain and P2P, blockchain technologies are slow because BFT adds
many requirements to the game. Basically blockchain technologies use P2P to
reach a common state and this could not be necessary in your use case.

You can take a look at OpenBazaar and Particl for P2P implementations.

------
sova
ZeroMQ [http://zguide.zeromq.org/py:all](http://zguide.zeromq.org/py:all) ...
"In One Hundred Words: ZeroMQ (also known as ØMQ, 0MQ, or zmq) looks like an
embeddable networking library but acts like a concurrency framework. It gives
you sockets that carry atomic messages across various transports like in-
process, inter-process, TCP, and multicast. You can connect sockets N-to-N
with patterns like fan-out, pub-sub, task distribution, and request-reply.
It's fast enough to be the fabric for clustered products. Its asynchronous I/O
model gives you scalable multicore applications, built as asynchronous
message-processing tasks. It has a score of language APIs and runs on most
operating systems. ZeroMQ is from iMatix and is LGPLv3 open source."

------
doomjunky
Peer Name Resolution Protocol (PNRP)

\- Developed by Microsoft

\- Open Specification

\- DHT-based

\- IPv6 only

\- Windows support since Windows XP SP 3

\- .NET support since .NET Framework 3.5 (System.Net.PeerToPeer)

[1]:
[https://en.wikipedia.org/wiki/Peer_Name_Resolution_Protocol](https://en.wikipedia.org/wiki/Peer_Name_Resolution_Protocol)

[2]: [https://msdn.microsoft.com/en-
us/library/aa371699.aspx](https://msdn.microsoft.com/en-
us/library/aa371699.aspx) "PNRP Namespace Provider API"

[3]: [https://msdn.microsoft.com/en-
us/library/bb726971.aspx](https://msdn.microsoft.com/en-
us/library/bb726971.aspx) "Peer Name Resolution Protocol"

~~~
zilchers
Microsoft had a few cool things, but it all seemed .net specific. Check out
this in WCF [https://msdn.microsoft.com/en-
us/library/cc297274.aspx](https://msdn.microsoft.com/en-
us/library/cc297274.aspx)

------
hapnin
Blockstack looks like it would fit here:

[https://github.com/blockstack/blockstack/blob/master/README....](https://github.com/blockstack/blockstack/blob/master/README.md)

~~~
abreckle
+1 for Blockstack

------
smaddock
If you plan on having any collaborative editing features, Yjs [1] is a p2p
framework utilizing CRDTs with support for IPFS.

[1] [https://github.com/y-js/yjs](https://github.com/y-js/yjs)

------
devxpy
I recently discovered zyre. Its from the same guy that brought us the
wonderful zeromq library

While it does a lot of the connection and communication part, it lacks a way
to bypass nat routers.

Here is a book on zyre
[http://zguide.zeromq.org/php:chapter8](http://zguide.zeromq.org/php:chapter8)

I also found this thing called pwnat which bypasses nat routers without the
need for UDP hole punching !

[https://github.com/samyk/pwnat](https://github.com/samyk/pwnat)

I feel like if someone made a high level wrapper combining these two, we can
have a very good framework.

------
tribler
> supports seeds and peer discovery, message passing, maybe firewall / UPNP
> support, etc.

Our audacious 'IPv8' work is now deployed live. Might fit you needs. Includes
above features in a low-complexity approach. [https://github.com/Tribler/py-
ipv8](https://github.com/Tribler/py-ipv8) Our fresh IETF Internet Standard
draft of this work: [https://tools.ietf.org/html/draft-pouwelse-
trustchain-00](https://tools.ietf.org/html/draft-pouwelse-trustchain-00)

------
tomp2p
TomP2P ([https://tomp2p.net/](https://tomp2p.net/)) is a library handling
firewall / UPNP support, however, the focus is academic/experimental.

------
mayamatrix
Worth having a look into the Holochain framework:

[https://holochain.org/](https://holochain.org/)

Depending on the application(s) you have in mind.

~~~
PkBuzios
Holochain is interesting for having a different approach then the blockchain-
based frameworks, which are inefficient

------
mempko
How about something [completely
different]([http://firestr.com](http://firestr.com))

------
martindale
Maki 0.3 will include components from Fabric:
[https://fabric.fm](https://fabric.fm) — automatically available to anyone
upgrading from 0.2. :)

------
dnomad
I2P works well enough.

------
w3news
Check this great talk:
[https://youtu.be/izQFMADw70w](https://youtu.be/izQFMADw70w)

------
cjbprime
WebTorrent.

------
adamnemecek
Look into crdt it’s the shit for collaborative editing (idk if that’s what you
ares looking for tho).

~~~
malloryerik
Do you happen to know of a decent and easy-ish crdt library in Go?

~~~
adamnemecek
Check github there’s plenty.

