
Erlay: Bandwidth-efficient Bitcoin transaction relay protocol - wyldfire
https://github.com/bitcoin/bitcoin/pull/18261
======
nullc
Our implementation of the underlying set reconciliation primitive is probably
of more general interest to HN. I submitted it previously (
[https://news.ycombinator.com/item?id=18645790](https://news.ycombinator.com/item?id=18645790)
) but it got no love.

Erlay itself is not terribly interesting except in the context of the specific
robusness requirements such as exist in Bitcoin.

When robustness against malicious parties isn't a major concern O(N) gossip
can also be achieved by more obvious protocols such as computing a spanning
tree between nodes and forwarding along it. Erlay is interesting because it
achieves the low bandwidth usage while preserving the robustness of "send
everything to everyone you're connected to".

------
kanzure
Gleb has presented on this a few times, here are some transcripts:

[https://diyhpl.us/wiki/transcripts/scalingbitcoin/tel-
aviv-2...](https://diyhpl.us/wiki/transcripts/scalingbitcoin/tel-
aviv-2019/erlay/)

[https://diyhpl.us/wiki/transcripts/london-bitcoin-
devs/2019-...](https://diyhpl.us/wiki/transcripts/london-bitcoin-
devs/2019-11-13-gleb-naumenko-p2p-erlay/)

paper: [https://arxiv.org/abs/1905.10518](https://arxiv.org/abs/1905.10518)

------
sneak
I still don’t know why the bitcoin p2p protocol doesn’t opportunistically
encrypt, even unauthenticated. Bittorrent has been doing this for ages.

It would be a huge privacy benefit, both for those running a p2p node, as well
as to those broadcasting transactions (the government snoops passively
monitoring would not know the location/ip from which a given new transaction
originated).

Raising the stakes from a passive splitter to active MITM has value.

~~~
nullc
I think it's an unambiguous ethical failure that Bitcoin's traffic isn't
opportunistically encrypted. As protocol developers we have an obligation to
provide for users at least a basic level of protection and because of known
threats on the modern internet that protection almost always needs to include
encryption.

In particular, due to material published by Edward Snowden we know for a fact
that nation state attackers have exploited this weakness to compromise the
privacy of Bitcoin users, but even if before we knew that-- it's a reasonable
assumption for any protocol which handles private information.

Unfortunately, Erik Voskuil (maintainer of "libbitcoin", a bitcoin node
implementation used by virtually no one) has aggressively argued against any
implementation of encryption with arguments that it would create a false sense
of security and ultimately convert Bitcoin to a permissioned network. (e.g.
[https://lists.linuxfoundation.org/pipermail/bitcoin-
dev/2016...](https://lists.linuxfoundation.org/pipermail/bitcoin-
dev/2016-June/012848.html) and the surrounding thread; note that this was in
_2016_. The same fruitless discussion repeated in subsequent years.)

I don't think any of Bitcoin's regular contributors have found his arguments
persuasive in the slightest. Personally, I believe the arguments are outright
incoherent. Never the less, these arguments are repeated every time its
discussed.

After years of vicious abuse many Bitcoin contributors simply no longer have
sufficient will remaining to proceed in the face of even more drama. It would
be perfectly possible for Bitcoin Core to deploy encryption and for software
like libbitcoin to not do so-- it doesn't have to be universally supported...
but even so, the drama remains and no one wants to deal with it.

As a result little progress has been made in bringing the encryption
functionality to completion and users are left needlessly exposed as a result.

~~~
qertoip
How is it possible that a single person, however vocal, can affect the morale
and peer review process so much? I would say, ignore and happily move on with
the implementation.

~~~
sneak
I emailed the primary bitcoin maintainer Wladimir J. van der Laan when the
Snowden slides came out, asking specifically for this. The hand-wavey
rationale then for not doing it was basically along the lines of “well, anyone
can MITM it and people who care about privacy should already be using it over
Tor, so it isn’t important”.

I disagreed then and I disagree now, but my C isn’t up to the task (I can read
it well enough, but I doubt I could securely implement a network crypto server
the first try). Perhaps I will look into adding this to the golang bitcoin
implementation, although I doubt it will go much of anywhere without
bitcoind/bitcoin-core (the main canonical implementation) supporting it.

~~~
nullc
FWIW, Wladimir was fine in support of the work, see e.g. this discussion from
a year ago: [http://www.erisian.com.au/meetbot/bitcoin-core-
dev/2019/bitc...](http://www.erisian.com.au/meetbot/bitcoin-core-
dev/2019/bitcoin-core-dev.2019-03-07-19.01.log.txt)

I didn't become aware of those snowden docs until a year and a half after the
article about them... :( but the need was obvious enough before they came out
(as you can see from the 2016 discussion on the subject).

Sitting down and coding is probably not the limiting reagent in getting the
work completed. Review, herding cats, building positive momentum, etc. would
all add a lot if you're willing and able to provide it. Collaborating on a
second implementation would probably also be useful (though not so much in
isolation, but it would be useful for testing and validation). (Heck, even
just cheering work along to distract from drama and haters would probably
accomplish a lot).

~~~
sneak
The Snowden drop was in 2013.

A functioning implementation goes a long way toward herding cats and building
positive momentum. Amongst hackers, working code trumps a lot of objections;
moreso if it is simply opportunistic and doesn't break anything for old
clients, and can be disabled by the fearful pedants.

It's also a great starting point for other practical objections: with a
working prototype, you can always counter "but what about edge case $x, huh
smart guy?!" with a "well, here's a small patch to address $x, it's in the PR
now", which is a great way to achieve forward progress.

~~~
nullc
The edge cases should all be addressed by the spec. :P

I don't recall any objections raised that would be addressed by code. (though
also, there have been at various points in time implementations, just not
merge ready ones...)

In 2013 there were more serious privacy problems in the bitcoin software which
probably would have made the encryption less relevant. Most of the known ones
were addressed by 2016. I still didn't hear about the bitcoin tie in until Oct
2019, though, sadly.

------
SilasX
Because transaction-bandwidth is Bitcoin's dominant inefficiency?

~~~
Taek
Transaction bandwidth is the primary inefficiency for continuously running
nodes, yeah. Especially if you are using EC2 pricing.

~~~
BubRoss
$5 a month can buy a VPS that runs a node with 1% of all of its resources.

~~~
RL_Quine
Barely. The resource requirements for running a meaningfully performant node
outstrip what you're getting on digitalocean. Most Bitcoin nodes as a result
are horrendously slow compared with running on reasonable hardware.

~~~
BubRoss
Slow in what way? A computer from 15 years ago or old rasberry pi can run a
node. What can't handle verifying one tiny block every 10 minutes while
sending out data?

