
Bitcoin's ASICBOOST Problem Explained [pdf] - jlrubin
http://rubin.io/public/pdfs/Asicboost.pdf
======
jcoffland
This is excellent. I've been trying to find a technical explanation of
ASCIBOOST for some time. Most of the articles and discussions gloss over the
details.

It is possible that this is part of the reason some miners are blocking SegWit
but there is more to it. Core (the main Bitcoin development team) promised to
provide a block size scaling solution and then reneged. This pissed a lot of
people off and led to the blocking of SegWit. I don't see ASCIBOOST as a real
problem for Bitcoin.

~~~
AgentME
>I don't see ASCIBOOST as a real problem for Bitcoin.

It often incentivizes miners to reorder or drop transactions until they hash
to a specific value. (A certain mining pool is publishing a significant number
of empty blocks.) Miners exist to verify transactions, and Bitcoin was
designed with transaction fees in order to incentivize miners to include
transactions. A competing force pushing miners away from including
transactions that can be trivially fixed is just a bug.

Imagine a mining "optimization" which involved miners making only empty
blocks. That goes against why Bitcoin has mining in the first place, and would
delay transactions eventually to a halt as more took advantage of the
optimization. A trick that only often encourages miners to make empty blocks
isn't categorically different, and it's something suboptimal about Bitcoin
that can be fixed.

~~~
mrb
Reordering or dropping transactions is a non-issue. All you need to covertly
implement Asicboost is dropping 2 random tx on the left side of the tree, and
2 on the right side (out of ~2000!) That's enough combinations to find enough
partial merkle collisions. So at worst these 4 dropped collisions will be
included in the next block because the next miner, even if he also implements
Asicboost, will statistically almost certainly drop 4 different collisions.

And it's not true that a "significant" number of blocks are empty. Only ~2% of
them are.

Furthermore the reason some blocks are empty is not due to Asicboost. It's
because after a new block is found, there is a small delay until a miner can
assemble the new set of txs, and during this period of time the hardware has
to mine on _something_ (empty block).

Above all, it would be in an Asicboost miner's interest to include as many tx
as possible in order to gain a ~10% increase in revenues from tx fees!

So, no, I disagree that Asicboost is a "problem". It's not. It looks like
Gregory Maxwell either misunderstands/misanalyzed the "risks" or is simply
dramatizing the situation for his own personal (political) reasons which would
be part of his plan (Blockstream's plan?) to attack and discredit large-scale
miners in the overall debate of Segwit vs. Bitcoin Unlimited.

Asicboost could hypothetically become a problem only if (a) mining profit
margins became very thin caused for example by a multi years period of BTC
losing value or remaining flat, (b) if somehow the 3 Asicboost patents
(Bitmain's, KnC's, Hanke's) were never licensed to anyone else, (c) if the
patents owners actually did evil things by abusing their patent rights, and
(d) while all this chaos is happening everyone else sits on their hands
instead of ignoring the patent and adding the optimization to their ASICs for
the benefit of Bitcoin's health or survival if it came to that...

It's very unlikely that (a) (b) (c) (d) all occur at the same time.

~~~
mrb
Hah! There is actually a 4th patent for asicboost:
[https://medium.com/@vcorem/the-real-savings-from-
asicboost-t...](https://medium.com/@vcorem/the-real-savings-from-asicboost-to-
bitmaintech-ff265c2d305b)

4 patents (applications) from 4 different holders! This provides enough
competing interests that it is even less likely that asicboost will ever turn
out to be a "problem".

------
needs
Just to point out that SegWit is everything but "already adopted in the
industry". It turns out that SegWit is the solution promoted by the developers
of bitcoin core to allow for bigger blocks, and solve some issues like
transactions maleability. It's more than 10 000 lines of code highly
controversial because they require... A soft fork, and will change bitcoin in
a fundamental ways.

Not to add that a company called "Blockstream" have paid almost every
developers of bitcoin core to push Segwit without a clear consensus from the
community. Worse, this company actually support the huge censorship happening
in bitcointalk and r/bitcoin. Blockstream received more than 75 millions from
AXA, for obscure reasons.

The paper only focus on Segwit, without mentioning that Segwit has nothing to
do with ASICBOOST. The community and a growing majority of miners is actually
switching to bitcoin unlimited, a fork of bitcoin core.

Sorry for the little off topic comment but I love Bitcoin and I hate that a
few dishonest people took control of it and try to destroy it.

~~~
futuravenir
I want to say that I've been on the anti-censorship boat and the 'AXA money is
sketchy' and the unlimited train for as long as I've heard of it.

Just recently, someone in the ecosystem that I've known since the beginning of
my time with Bitcoin came out in support of Blockstream and Segwit with a
backing of other Canadian support.

[https://medium.com/@francispouliot/canadian-bitcoin-
economic...](https://medium.com/@francispouliot/canadian-bitcoin-economic-
nodes-unite-against-bitcoin-unlimited-412786fb4bb6)

I don't know what to make of it all. I spoke with him a bit and he seems to
believe that those pushing against segwit are paid by ASICboosters in China
because they stand to make $100,000,000/yearly from their (now not so secret)
advantage.

In any case, I just wanted to add that I'm more confused than ever and things
are very unclear.

~~~
AgentME
>because they stand to make $100,000,000/yearly from their (now not so secret)
advantage.

If anyone is confused how the 30% advantage that ASICBOOST gives translates
into this huge number, remember that the 30% boost applies to revenue, not
profit.

The revenue of mining in the long run approaches the costs to mine (basically
the cost of electricity), leaving extremely slim profit margins. Let's say the
usual revenue from mining over some period of time is $1.02, and the cost is
$1.00, giving a profit of 2 cents every time unit. A miner using ASICBOOST
could have a revenue of $1.32 every time unit, giving a profit of 32 cents
every time unit, which is at least an order of magnitude more profit than
anyone else. This miner then has more resources to spend on buying hardware
and scaling up.

(The above paragraph goes for any type of optimization and isn't necessarily
nefarious. It can be a problem if one miner effectively keeps an optimization
secret for too long, because they'll continue to grow in size and could get
more than 50% of mining power, which is a specific point that causes huge
problems with Bitcoin. But the real unique issues about ASICBOOST are about
how it encourages empty blocks, and how it has been secretly incentivizing
some groups to argue against any incompatible protocol change.)

~~~
midmagico
Your arithmetic is off. Revenue doesn't increase. Costs decrease. Effectively
the increase in profit is similar, though, to your _result._ Just for
different reasons.

Ignoring depreciation or hardware costs, the arithmetic is like this:

Pre-ASICBoost: Revenue: $1.02 Costs: $1.00 Profit: $0.02

Post-ASICBoost: Revenue: $1.02 Costs: $0.70 Profit: $0.32

The closer to costs that revenue for non-AB mining is, the closer to infinity
the comparative profit margin is.

That is, AB doesn't increase hashrate. (That would be a very, very much more
complicated equation since the hashrate reward pie is essentially static.) All
it does is decrease _the amount of power required to do the same work_.

------
tlrobinson
This might not be the place for this question, but can someone explain why we
can't / shouldn't have _both_ unlimited block sizes and SegWit?

Block sizes will supposedly be constrained by bandwidth / propagation times.
SegWit will allow more transactions to occur off chain, reducing the need for
larger blocks.

They seem complementary to me.

~~~
shea256
SegWit does in fact represent a block size increase. It results in 1.7x the #
of single-signature transactions per block and 4x the # of multi-signature
transactions per block.

Unlimited block size? There are many reasons that's a very bad idea but here
are a few:

1\. It would put immense pressure on the network in terms of latency between
miners, leading to less stable mining, a higher rate of reorganizations, and a
massive advantage for the larger miners, resulting in increased mining
centralization.

2\. It would result in a substantial increase in the amount of bandwidth that
each node has to handle. Modest increases are OK but unlimited blocks means
the vast majority of nodes will die off. Remember that nodes have to check
blocks as they come in, so this would be an insane DDoS vector. Imagine
putting a funnel in someone's mouth instead of a straw and being able to force
whatever you want in there.

3\. It would result in a substantial increase in the amount of data that each
node has to store. Remember that in Bitcoin 100% of data must be stored by
100% of nodes. Imagine "unlimited emails" with Gmail meant that you had to
store all of the emails in the world for EVERYONE. Not a good idea.

~~~
Kinnard
That's not an increase in the size of the block per se, as much as it is an
increase in the efficiency of encoding off-Blockchain information, no?

~~~
jlrubin
No, it is a space increase! Witness data is counted with a discount. Blocks
can be larger than 1MB under SegWit.

~~~
Kinnard
Perhaps this is semantic?

~~~
jlrubin
It's not semantic, it is substantive. A maximally utilized SegWit block is
larger than a maximally utilized current block.

I.e., if you were planning out how much disk space you needed years in
advance, you would have to increase that figure non-negligibly if SegWit
activates.

------
wmf
Some context: [http://hackingdistributed.com/2017/04/05/bitcoin-drama-
respo...](http://hackingdistributed.com/2017/04/05/bitcoin-drama-response/)

~~~
AgentME
I find this article very misleading.

>2\. If ASICBOOST was actually used, we'd see ample evidence on the
blockchain.

The covert form of ASICBOOST (where they don't use the version field, the form
that was only recently publicly discovered) would only show up as a higher
than usual number of empty blocks or blocks with reordered or missing
transactions (depending on how the attacker implemented it; it's not
necessarily both).

>3\. ... As you can also examine for yourself, the transactions are ordered
essentially by fee-per-byte, which is not what we would see if they were
shuffled to create collisions.

I find it extremely disingenuous that the author only focuses on looking for
reordered transactions and ignores the often-noticed unusual number of empty
blocks that Antpool mines[0], and the article reaches absurdity when the
author claims that the lack of reordered transactions means there's absolutely
nothing to worry about.

>4\. Building more efficient mining chips is what miners are supposed to do,
as it secures the blockchain. Framing an optimization as an attack is
disingenuous. Going from 28nm to 16nm was not an attack on the network. Better
mining algorithms have never been considered an attack.

If for example a mining "optimization" encouraged every miner to mine only
empty blocks, then calling it an attack doesn't sound out of place. An
optimization that only often encourages miners to make empty blocks isn't
categorically different.

>7\. Segwit is being offered as a solution to ASICBOOST. Yet Segwit has at
best a tenuous connection as a response to ASICBOOST mining. There are
countless other, non-controversial solutions that can disable ASICBOOST,
should one choose (wrongly) to do so.

Greg Maxwell's very post announcing the issue introduces a fix for the covert
form of ASICBOOST that does not involve implementing SegWit!

[0] [https://bitcoin.stackexchange.com/questions/50184/why-
does-a...](https://bitcoin.stackexchange.com/questions/50184/why-does-antpool-
mine-so-many-empty-blocks) [http://www.livebitcoinnews.com/antpool-generating-
large-amou...](http://www.livebitcoinnews.com/antpool-generating-large-amount-
empty-bitcoin-blocks/)

~~~
kylebenzle
Either I am crazy and don't understand bitcoin or some well funded group is
spending a lot of money to support SegWit and push Core's agenda.

The few sensible posts like yours are downvoted to hell and back while pro-
SegWit rocket to the top. Every point you make is spot on and there is no
sensible reply to any of them, just a bunch of garbage about no one supports
BU and BU will never happen...

~~~
AgentME
I'm not really sure you replied to the right post, or you might be mixing up
some things.

The hackingdistributed article I was criticizing was framing Greg Maxwell as
making up the whole ASICBOOST attack as pro-SegWit propaganda. There seems to
be a group that's extremely anti-SegWit that thinks that the ASICBOOST news is
made up to push SegWit through, and the article is following that pattern.

I wouldn't necessarily call my criticisms "pro-SegWit", but it's definitely
not on the mentioned anti-SegWit "side" if someone were to group things by
side.

>or some well funded group is spending a lot of money to support SegWit and
push Core's agenda.

The operator of Antpool has been one of the people speaking out against
SegWit, and it's just been revealed that he most likely has had a huge secret
monetary incentive against SegWit and not for the good of the network.

------
RichardHeart
Anything that makes mining empty blocks more profitable is bad. Empty blocks
are an attack on the network. Someone is going to say "Empty blocks give
people a reason to raise fees." DDOS attacks give people reasons to buy more
expensive routers too.

------
cashmonkey85
I genuinely believe most of the comments on this page are paid trolls. Which I
didn't think was a thing on HN

People should read up of the Chinese miners disinformation campaign. Really
crazy stuff. [https://medium.com/@WhalePanda/the-extended-extension-
block-...](https://medium.com/@WhalePanda/the-extended-extension-block-
story-5bc3d888bdde)

------
fpgaminer
Besides the technical issues involved here, there's been a lot of political
quarrels as well. Here's my run down of them. Note that I was heavily involved
in the Bitcoin community a few years ago, but have been on the side lines
recently. As with all things political, take my interpretations with a grain
of salt:

1) The Bitcoin network began experiencing congestion due to rise in popularity
driving large numbers of transactions. This resulted in slower transaction
verification and higher transaction fees.

2) Users started asking for larger blocks, so that the network would have
higher transaction bandwidth. The defacto Bitcoin developers at the time
pushed back. Increasing block size is simple in terms of code change, but
requires a risky hard fork. It's also only a temporary fix. Bitcoin will grow
and require another increase, and thus another hard fork. They wanted a
better, long term solution.

3) After a lot of discussion the developers came back with Segregated Witness.
It's a soft modification to Bitcoin which meant no hard fork. It doesn't
explicitly allow larger blocks. It does, however, upgrade Bitcoin to allow
side chains. The argument is that faster, more abundant transactions can be
implemented in a side chain system. A sort of second layer on top of Bitcoin.
Much like how HTTP is implemented on top of the underlying TCP/IP protocol.

It should be noted that, while this enablement of side chains is the biggest
feature of SegWit, there were a few smaller improvements. It _does_ increase
effective block size slightly; ~2x. And it fixes a few minor annoyances in the
Bitcoin protocol (e.g. malleability).

4) The community was disappointed by the lack of larger blocks. While
developers were busy putting together their idea and specification for SegWit,
the congestion issues on the network continued to grow worse.

5) Eventually the debate became "on-chain" versus "off-chain". Some of the
community argued that they wanted to keep all transaction on the Bitcoin
blockchain. In other words, they still wanted bigger and bigger blocks. I
believe the root of this argument is that Bitcoin has served us well for many
years, so why try to build another, potentially weaker system on top of it?

The other parts of the community were in favor of SegWit and off-chain
transactions. Side chains would allow for faster transactions and
significantly more of them, and they would consume far less disk space (a
growing concern in the community is the disk space required to run a Bitcoin
node). Side chains achieve this by compromising on some of the strengths of
Bitcoin, either by reducing security or requiring more centralization.

6) Eventually the debate in the community shifted from just on-chain versus
off-chain to SegWit versus bigger blocks. Now, to be clear, SegWit doesn't
preclude the possibility of larger blocks. Obviously larger blocks can be
added regardless of SegWit. But, for whatever reason, the dialog shifted.

7) When SegWit was finally released as part of the Bitcoin Core client (the
defacto standard for Bitcoin clients/wallets) a sort of battle amongst the
community began. Mining operations that ran the SegWit enabled clients started
getting attacked; DDOS, etc. There was a lot of strife, hate, and fear. The
way SegWit is implemented in the code, it requires a large majority of miners
to support it before it officially activates and can be used. This is by
design, to ensure the community agrees with the feature and to ensure almost
all miners are able to understand the new blocks and not get left behind after
activation. Because of this majority requirement, the battle in the community
and between miners was very important. If half the miners didn't agree with
SegWit, it would never activate.

8) This "battle" went on for a long time and is, currently, still on-going.
Bitcoin continues to remain congested. Transaction fees have continued to
rise. SegWit is still not activated. During this time a couple factions of the
anti-SegWit community split off in attempts to fork Bitcoin. They modified
versions of the Bitcoin client to enable larger blocks. So far, none of those
forks have been successful, in terms of taking away any majority of users from
the main Bitcoin network.

9) Recently it became public that a large mining operation may have been using
a technology called ASICBOOST during this time. It allows mining chips to be
more efficient, which means those chips make more Bitcoins for less money.
Obviously an advantage. This technology has been known about for awhile now,
but as far as the community knew no one was using it. It was covered by
patents, and there's a sort of gentleman's agreement in the community not to
use it as a result of the patent and potential ill effects on the network.
(The patent means not all miners would be able to use it, so it presents an
unfair playing field).

ASICBOOST, the way this mining operation has supposedly been using it, is not
compatible with SegWit. That's explained in the OP.

The accused mining operation was also involved in a lot of the anti-SegWit
activity in the community; promoting the alternative forks of Bitcoin. They
had been accused in the past of using sock puppets to drive anti-SegWit
agendas, drive character assassinations, etc.

They are in control of a large percentage of the total Bitcoin mining power.

The public revelation of them using ASICBOOST painted a dark, but enlightening
picture.

As I mentioned before, SegWit ultimately has nothing to do with the on-chain
versus off-chain debate. It merely _enables_ off-chain possibilities. It
doesn't force them. So it was odd to see the discussion shift from on-chain
versus off-chain to SegWit versus Bigger Blocks. The theory now, given the
accusations against this large mining operation, is that they are responsible
for the majority of the anti-SegWit movement. Because the activation of SegWit
would force them to stop using ASICBOOST the way they've been using it, it was
in their financial interest to prevent SegWit from activating.

That's basically my summary of events.

I think that theory, that this mining operation has been basically using anti-
SegWit propaganda and other attacks to prevent SegWit from activating, so they
can keep using their secret version of ASICBOOST and profit (to the tune of
$100 million a year), makes a lot of sense. Why else would anyone oppose
SegWit? I've read through the SegWit specs. It enables a lot of really cool
tech for Bitcoin. Side chains are not just about trying to get more, faster
transactions into the Bitcoin network. They're also about allowing other
technologies like smart contracts to tie themselves in Bitcoin.

You know all those cool features that various alt coins are testing out?
SegWit enables those features to become side chains so that you can gain all
the security that Bitcoin offers. Side chain versus altcoin is like browser
addon versus a whole new browser.

At the end of the day, people in favor of bigger blocks could still campaign
for bigger blocks. SegWit doesn't prevent that. A mining operation being
financially incentivized to block SegWit is the simplest explanation I've
heard yet for why there would be significant opposition to it.

~~~
stale2002
Hey, the big reason to oppose segwit is mostly as a negotiation tactic.

Many of the bitcoin developers , small blockers, and the blockstream people
really, REALLY want segwit to activate.

They are so desperate to have it, that although they may not explicitly agree
to a "compromise" 2MB HF + segwit proposal, they might at the very least not
go freaking nuclear or something in their attempts to oppose it.

Segwit is nice, sure. But us big blockers know that if it gets activated now,
then we are never going to see big blocks, regardless of how much support we
get.

Don't believe me? Just check out some of the stuff that lukejr (the lead
bitcoin developer) says.

They explicitly say that it could be decades before any more block size
increases come around.

~~~
fpgaminer
I'm happy to have some big blocker perspective, as I know my comment perhaps
paints "Big Blockers" in less than favorable light. Not intentionally. It's a
bit hard to be completely fair in a short summary, given the events that have
occurred, even though those events are likely not the fault of real big
blockers.

That said, good technology should _never_ be used as a bartering chip.

I think it's hypocritical of any Bitcoin user to be against SegWit for
political reasons. From day 1 Bitcoin has suffered of accusations of being
used for illegal and evil activities. And from day 1 we, as a community, have
had to fend off those claims saying "Yes, it _can_ be used for illegal
activities. But that's not what it's about. It's about a revolutionary
technology that, for the most part, will be used for good."

To block SegWit is to go against that ethos, an ethos so embedded into
Bitcoin. It's about technology; about changing the world. SegWit enables so
many cool new uses for Bitcoin. Can it be used to create off-chain
transactions; something some big blockers are against? Sure. But that's no
reason to stop good technology. Just like it was no reason to stop Bitcoin
itself, just because it _could_ be used for illegal activities.

> 2MB HF + segwit proposal

I don't understand that proposal at all. SegWit _is_ an increase in block
size, by on average 2MB. So why have the additional hard fork if the block
size is already increasing?

> Segwit is nice, sure. But us big blockers know that if it gets activated
> now, then we are never going to see big blocks, regardless of how much
> support we get.

The one good thing to come out of this years long battle in the community, is
that ultimately users and miners are in charge. If big blockers wanted big
blocks, all they have to do is offer them. Offer real specifications and real
code with real tests and real study. To date, I've never seen that from the
big blocker camp.

I'm not in either camp, really. Small or big. Personally, I think Bitcoin will
need to increase block size eventually anyway, whether directly or indirectly.
1MB forever is silly. But nobody has offered a reasonable, long term solution
to increasing block size, besides SegWit. Hard forking is very risky. Emergent
consensus is risky.

All the big block supporting altcoins I've seen are cobbled together messes.
And I'm not speaking as a user; I'm speaking as someone who writes code for a
living, implemented the foundation of all modern mining ASICs, and have dealt
with the Bitcoin Core code specifically as well.

~~~
stale2002
2MB HF + segwit (2MB average) adds up to 4 MB total. 4MB forever is better
than 2Mb forever.

If this whole debate really is because of soft fork is hardfork, that is easy
to solve.

You can scale by soft forking using extension blocks, OR just by changing the
segwit discount parameter that segwit does.

If segwit can do it, then so can 4MB segwit.

------
mjevans
Wasn't the entire point of picking SHA-256 to both check the integrity of the
existing ledger (why it's a hash) AND as a proof of work lottery?

The "problem" being that instead of being a proof of actual work, shortcuts in
the work were found.

It seems then that a good solution would be to force many different types of
proof of work. Different basis of hashes are obvious, but maybe some type of
actual work every so often as well.

~~~
wyager
> The "problem" being that instead of being a proof of actual work, shortcuts
> in the work were found.

The same "problem" occurred when people invented GPU mining, then FPGA mining,
then ASIC mining. As long as SHA256 isn't broken, it doesn't matter what kind
of speedup miners manage to get; whenever someone finds a big advantage,
everyone else uses it soon enough.

> Different basis of hashes are obvious

"Use more hash functions" is sort of a reflexive response you get a lot from
non-cryptographers, but the intuitive reason this is a bad idea is that if
your hash function isn't broken, your cryptosystem shouldn't be either. It's
better to fix the cryptosystem than to half-assedly patch it over by
obfuscating any problems with more hash functions.

> but maybe some type of actual work every so often as well.

What do you mean by "actual work"? In all likelihood, whatever you're thinking
of has none of the properties that make PoW a viable mechanism for sybil
resistance.

~~~
petertodd
> everyone else uses it soon enough.

Unfortunately in this case, ASICBOOST is both patented, and adds a fair bit of
complexity to mining hardware. Both these problems have the undesirable effect
of increasing the barriers to entry for new asic manufacturers - an
undesirable thing from the point of view of Bitcoin users.

You're quite correct that shortcuts are irrelevant, but the reality of
manufacturing makes the playing field a lot less level than we'd like.

~~~
shea256
Exactly, I'd go further and say that if a particular technique is patented and
cannot be matched by a related technique, giving a permanent unfair advantage,
it is up to the community to change the proof of work and work around the
advantage imposed by a state monopoly.

~~~
jcoffland
Then Bitcoin has to hard fork every time someone comes up with a new
patentable idea. I don't see why people think mining must be fair. It only has
to not become completely centralized.

~~~
Taek
Someone who can control the majority of the hashrate can enforce a transaction
whitelist. Bitcoin's independence from the state requires the world
superpowers are not able to control a majority of the hashrate.

~~~
midmagico
> Bitcoin's independence from the state requires the world superpowers are not
> able to control a majority of the hashrate.

Too late. :-)

