
Tendermint 0.10.2 - djm_
https://jepsen.io/analyses/tendermint-0-10-2
======
deforciant
Looks like these guys (Tendermint team) actually care about design, testing
and scalability in their project compared to Ethereum where people are
concentrating on hype rather than product (I am not rude, used Ethereum from
the client's perspective) :)

It would be interesting to hear from people who actually use (if there are
any) Tendermint: * How easy/hard is it to create bindings to the blockchain
from your own backend application (as I can see there is a GRPC option). * How
many nodes you are running? * What's the use case?

~~~
coinculture
Thanks for the compliments! We do really care about quality software design
and testing, and we're looking to hire folks that share our values:
[https://tendermint.com/careers](https://tendermint.com/careers)

Lots of people are using Tendermint. The biggest users would be Hyperledger
Burrow
([https://github.com/hyperledger/burrow](https://github.com/hyperledger/burrow))
and our own Ethermint
([https://github.com/tendermint/ethermint](https://github.com/tendermint/ethermint)).
We're also working on an SDK for building apps in Golang
(github.com/cosmos/cosmos-sdk) but it won't be ready for another month or two.
There are a variety of apps built by members of the community, and ABCI
servers in different languages - you can see a somewhat up-to-date list here:
[https://tendermint.com/ecosystem](https://tendermint.com/ecosystem)

~~~
wslh
Good to have you here. What is your view around private blockchains and smart
contracts regarding Hyperledger Fabric vs Ethereum based ones? I mean, in my
perspective there is no need to have smart contracts ala Ethereum because in a
private blockchain you can simplify the problem to a workflow with states
where the parties sign the different transitions and the logic can be outside
a VM. Since private blockchains rely on a lot of oracles, those oracles are
the weakest link in the security chain and the VM does not add significant
security like in a public blockchain.

~~~
jaekwon
Private networks could grow to become public. It would be nice to be able to
use the same system for scaling from 4 to 10B.

Private blockchains are incentivized to adopt public blockchain architectures
because public blockchains are most economically vetted.

~~~
wslh
No, private blockchains can scale better than public blockchains for obvious
reasons: they are trustful.

Regarding private blockchains growing to public trustless blockchains, that is
not a trivial problem. There are new blockchain models that are being promoted
(e.g. IOTA, EOS, NEM) which have not being studied enough by consensus
researchers.

~~~
jaekwon
Tendermint works great for 4 to 300 validators, and with the Cosmos delegation
model, many more stakers can participate. So yes, it's not a trivial problem,
but Tendermint is solving it.

~~~
wslh
Tendermint is not solving the problem I pointed before because it is not
trustless, you are trusting the validators.

~~~
sunnya97
In PoW, you are trusting the miners.

~~~
wslh
I am not talking about PoW specifically but about the new research that
Bitcoin opened, including new consensus systems in trustless networks. The
community knew how to build distributed systems in trusted networks before
Bitcoin.

~~~
jaekwon
Yes, there were many designs.

I don't know of anyone who really applied traditional BFT networks into public
blockchain architecture, however, until Tendermint.

For example, who spoke or wrote of BFT fault accountability/attribution, and
how it pertains to proof-of-stake models, before Tendermint?

There was also the 2011 paper that lamented the lack of good open-source
middleware for BFT algos. Tendermint solves that engineering problem as well.

------
__erik
I first tried to use Tendermint a year ago and honestly it was rough. Their
codebase had very few comments, including things a go linter would harass you
endlessly for like function comments, and the docs were barely existent. The
codebase was also not exactly what you would call "idiomatic" go.

I checked back in at the Cosmos fundraiser, and then again a few weeks ago and
their documentation while still needing work, as well as their code quality is
definitely improved. I like their idea and I want the project to succeed, and
things like independent third party audits are the way that happens.

Kudos to the team for not being afraid to look for the warts, better to get
them now then later.

~~~
coinculture
Thanks! A year ago we were just two guys working on this project on the side
without any funding. Now we're a full time team of talented devs swinging in
full force. The comments and docs continue to improve, and we're excited that
folks like yourself continue to take a look.

If you have more specific issues/comments/suggestions, please open an issue!
[https://github.com/tendermint/tendermint/issues](https://github.com/tendermint/tendermint/issues)

------
runeks
Is the intended purpose of the Tendermint protocol to solve the double spend
problem, thus allowing the creation of digital tokens?

And, if so, wouldn’t the market value of these tokens be limited to whatever
it costs to bring _2n_ dishonest nodes, where _n_ is the current number of
nodes in the network (thus reaching 2/3 majority), and hereby allowing you to
define the transaction history (and assigning all tokens to yourself and sell
them in the open market — which will be profitable if the market value of the
tokens exceeds the cost of bringing up the dishonest nodes)?

~~~
zmanian
In a BFT protocol based cryptocurrency, an attacker who acquired 2/3rds of the
tokens in the network would be able to double spend at will and censor
transactions to gain control of more of the network.

They would not be able to rewrite past history.

The only response to an attack would be hard fork.

~~~
runeks
If I want to bootstrap my node and I join a network where 2/3 of the nodes
present a newly-created (false) transaction history, and the remaining nodes
present the original transaction history, how would I be able to tell which
chain to follow without trusted/special nodes?

In other words, the plan of attack wouldn’t be altering the current chain
(which indeed would require owning coins on it, because of proof-of-stake),
but rather presenting a completely different chain to new nodes with a 2/3
node majority.

~~~
coinculture
That's right. The first time you sync up, you're particularly vulnerable. You
need to authenticate first against a trusted source.

~~~
runeks
> You need to authenticate first against a trusted source.

If the system depends on a trusted source, why not just have this trusted
source sign blocks, thus solving the double spend problem without further ado?

~~~
coinculture
It's not some single centralized trusted source. It's a local trusted source.
Like a friend or a shop or a website you make payments on that uses the
network and has been keeping up to date. Ideally folks should check multiple
sources to ensure they agree.

In the same way folks need to figure out which software to download when they
join the Bitcoin network.

~~~
sunnya97
Also, two other comments regarding weak subjectivity:

When you join the bitcoin blockchain, you need some trusted source to tell you
the hash of the correct genesis block.

Also, if you want to follow a shorter fork chain like Ethereum Classic, you
also need weak subjectivity to tell you the first block immediately after the
fork, otherwise you might be tricked onto the longer malicious fork, Ethereum
:P

~~~
runeks
> When you join the bitcoin blockchain, you need some trusted source to tell
> you the hash of the correct genesis block.

No, you don’t. There’s nothing special about the Bitcoin genesis block, it’s a
block like any other. Whether you follow a chain that builds on top of this
block or some other block has no bearing on the security of the system. It
contains no keys that get to decide anything later on.

------
denisnazarov
How does Tendermint consensus compare to Nakamoto consensus? Are 2/3 of the
nodes expected to be honest? How is validator membership established/enforced?

~~~
sunnya97
Adding on to what coinculture said, the traditional BFT nature of Tendermint
consensus gives it a lot of benefits over Nakamoto consensus such as 1-block
finality and safety guarantees in asynchronous environments.

Traditionally, these BFT algorithms did not work in public settings, and only
worked with fixed validator sets. However, with the conceptual invention of
Proof of Stake, we realized we can use cryptoeconomics to facilitate validator
set changes in a public network.

And yes, 2/3 of nodes are set to be honest. However, just in case you weren't
aware, it's pretty not well known, but Nakamoto consensus also has a 2/3
threshhold due to the game theoretical vulnerabilities caused by a process
known as selfish mining.

------
lostmsu
So, to summarize, instead of trusting physical hardware owned by different
people we are expected to trust that keys, that were used to sign genesis
block will never leak?

~~~
coinculture
Not quite - see other comments about needing a trusted local source the first
time you sync up. Otherwise, forks are never tolerated. If the keys leak, and
there's a new history, it will be ignored by all nodes who have been online.

------
ansek
Great work! I just started using Tendermint to write a cryptocurrency for
supporting open source development and have several simple questions. Would
appreciate it if you could answer them:

1) What's your estimated time to be able to run in production?

2) What do you think of using something like PostgreSQL to keep the state?

3) Is it a good approach to write cron, for example, to recalculate some
values once in awhile and update the state by sending transactions?

4) What is the best way to handle random numbers?

~~~
coinculture
Thanks!

1) We're looking to get the Cosmos Hub up before the year is out, and to roll
out production support in 2018.

2) Great, if you like. But if you want to support light clients you will need
a scheme for producing Merkle proofs. Streaming data out of the app into
Postgres to better serve clients, sure. But Postgres within the hot paths of
your app? Not sure.

3) Sure. Or you can use BeginBlock or EndBlock messages to have the blockchain
automatically run code even without transactions :)

4) We're still working on this. For now you can use an external random number
oracle, or the block hash, which is available in the Header passed into
BeginBlock. The block hash is currently highly manipulable.

~~~
sunnya97
Some other random generation techniques include commit-reveal schemes, BLS
Threshhold Relays, VRFs, Private Hashchains, etc.

------
sandGorgon
Hi guys, I asked this question on Twitter as well. Is there a way to model
proof of work vs proof of stake using something like Jepsen/Merkleeyes.

I'm looking forward to see some Jepsen style testing of the different
blockchain protocols, but doesn't seem to be anything out there that can model
these aspects.

In building Tendermint, did you figure how to test and validate these aspects
of blockchains ?

~~~
coinculture
Tendermint doesn't use Proof of Work (PoW). PoW is hard to test with Jepsen
because it only supports probabilistic safety, which means the chain can
constantly reorganize. If I'm not mistaken, Jepsen would consider PoW's
routine reorgs as linearizeability failures. That said, Jepsen could
potentially be modified to support this better, for instance only considering
something accepted if its buried X blocks deep.

Tendermint, in contrast, provides deterministic safety, and serves as a
building block for Proof of Stake networks.

~~~
denisnazarov
> serves as a building block for Proof of Stake networks.

Can you elaborate on this, as there is almost no mention of PoS in the post?

From a high level, your response to my previous comment can be interpreted as
the validators being "hardcoded" upfront, which would be very different
decentralization properties from PoW systems.

What assumptions does Tendermint make in contrast to PoW systems, and what
applications do you see it being most useful for?

~~~
coinculture
Good points. Tendermint aims to be a general purpose consensus system - how
the validators are actually chosen and changed is outside Tendermint's domain.

All Tendermint wants to know is an initial set, and leaves it up to the App to
tell it how the validators should change with every block.

You can build PoS at the application level, if you need it. This is what we're
doing with [https://cosmos.network/](https://cosmos.network/). With every
block, the application can decide, according to its particular PoS design, how
the validators should change.

There's actually no reason you couldn't build a Tendermint app that determines
who the next validators are based on PoW they did and submitted as a
transaction. Not quite the PoW we're used to, but it shows Tendermint is quite
flexible. That said, a 2/3 coalition of the current validator set always has
complete control on what goes in the blockchain, and a 1/3 coalition can halt
it - we can't bail out of that without a hard fork.

Note PoW is also only safe in a synchronous network, though it hedges this
with an economic random lottery. Of course PoW guarantees a thermodynamic
immutability that nothing else can.

Tendermint isn't inherently economic, and is safe so long as <1/3 of the
validating power is malicious, even in asynchronous networks. That's what
Jepsen shows here (well, fails to disprove).

Arguably, Tendermint is most useful as an analog of Paxos/Raft but in a multi-
stakeholder, or otherwise more adversarial, setting.

We aim to prove it's also useful for structuring global financial systems and
scaling cryptocurrencies :)

------
Myztiq
I'm getting an out of memory error when loading the page.

java.lang.OutOfMemoryError: unable to create new native thread

Cached version:
[http://webcache.googleusercontent.com/search?q=cache:https:/...](http://webcache.googleusercontent.com/search?q=cache:https://jepsen.io/analyses/tendermint-0-10-2)

------
misterbowfinger
> Tendermint is a distributed, byzantine fault-tolerant consensus system

what's "byzantine" mean in this context?

~~~
jcassee
The Byzantine generals problem:

[https://en.m.wikipedia.org/wiki/Byzantine_fault_tolerance](https://en.m.wikipedia.org/wiki/Byzantine_fault_tolerance)

~~~
lobster_johnson
Non-mobile link:
[https://en.wikipedia.org/wiki/Byzantine_fault_tolerance](https://en.wikipedia.org/wiki/Byzantine_fault_tolerance)

------
Yuioup
java.lang.OutOfMemoryError: unable to create new native thread at
java.lang.Thread.start0(Native Method) at
java.lang.Thread.start(Thread.java:714) at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at clojure.core$future_call.invokeStatic(core.clj:6687) at
ring.middleware.gzip$compress_body.invokeStatic(gzip.clj:70) at
ring.middleware.gzip$compress_body.invoke(gzip.clj:70) at
clojure.lang.AFn.applyToHelper(AFn.java:154) at
clojure.lang.AFn.applyTo(AFn.java:144) at
clojure.core$apply.invokeStatic(core.clj:648) at
clojure.core$update_in.invokeStatic(core.clj:5950) at
clojure.core$update_in.doInvoke(core.clj:5939) at
clojure.lang.RestFn.invoke(RestFn.java:445) at
ring.middleware.gzip$gzip_response.invokeStatic(gzip.clj:87) at
ring.middleware.gzip$wrap_gzip$fn__12777.invoke(gzip.clj:95) at
aleph.http.server$handle_request$fn__9026$f__5083__auto____9027.invoke(server.clj:156)
at clojure.lang.AFn.run(AFn.java:22) at
io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62) at
manifold.executor$thread_factory$reify__4975$f__4976.invoke(executor.clj:44)
at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745)

~~~
softwarelimits
This is something I do not understand with java as a webserver - doesn't such
behavior show a major design problem?

"Lots of traffic" can be expected by any webserver software - handling "too
much" of it in a not too painful way could be a design goal. Just spitting out
an error on such a condition and making the site disappear is a little bit too
simplistic for my taste.

I have seen this error on sites with java webservers many times - is this an
accepted limitation in the java world?

I am not trying to be snarky here, I have lots of respect for aphyrs work, but
I am really wondering about this "normal behaviour" of java webservers - how
can an "enterprise platform" get away with this?

~~~
aphyr
The jepsen.io server is literally something I hacked together in a few
weekends. I wouldn't really point to it as an exemplar of high quality
production-facing engineering. :)

That said, it _does_ push 50,000 reqs/sec on my local box, did just fine on an
AWS m3.medium, then fell over today at 5 reqs/sec on a heroku "pro" dyno,
so... this is something I'll have to address.

~~~
sandGorgon
You should switch over to using Hugo and generating static html for most of
your site. Deploying it to Firebase CDN or Netlify should cost you a few
dollars every month.

------
sandGorgon
Why did you create your own data store - Merkleeyes.

I mean, my question is - could you not have wrapped postgresql jsonb or redis
using application logic to give you the same kind of semantics as Merkleeyes,
yet not have to reinvent data storage ?

Would you not have to resolve all the problems that Redis has had to...Also
that commercial infrastructure for Redis (e.g. Elasticache) is widely
available.

~~~
coinculture
You're probably right, we could have.

The primary reason to use our own is to support Merkle proofs of the state.
For the Jepsen tests, it wasn't really needed, and may have been smarter to
just wrap something else. But we didn't really reinvent any data storage, we
just used goleveldb.

~~~
sandGorgon
Leveldb is great - but we are already seeing an explosion in system
requirements of blockchains. For example, Monero is already at 25gb. What ends
up happening is that people who are serious about this, end up having to port
the data structures to a scalable solution. For example take a look at all the
questions around "how do I make Bitcoin work with MySQL for larger websites"
\-
[https://www.google.co.in/search?q=bitcoin+mysql+site:bitcoin...](https://www.google.co.in/search?q=bitcoin+mysql+site:bitcointalk.org&client=ms-
android-
google&prmd=nvi&sa=X&ved=0ahUKEwjQ473s7Y_WAhWLsY8KHXevAboQrQIIOigEMAM&biw=412&bih=660)

It would be so great if you guys could use a datastore that is setup for scale
from the get go.

------
ceocoder
Can someone explain what this,

> simple byzantine faults

means? Isn't "simple byzantine" an oxymoron?

~~~
aphyr
To be specific, our "byzantine" faults are limited to very specific types of
behavior: running multiple copies of the same validator on different nodes,
and modifying on-disk state of the validator. We make no changes to the
validator source, don't inject arbitrary messages, don't attempt to invalidate
already signed blocks, etc etc.

------
jaekwon
Tendermint is hiring.

We have a hubs in Toronto and Berlin, and we're building one out in the Bay
Area.

If you have what it takes to design and implement protocol standards for the
blockchain/cryptocurrency industry, reach out. We want to hire you!

If you have significant open-source software development experience in
distributed systems design, operating systems design, database systems design,
or language design, reach out. We want to hire you!

We're redefining the definition of money while we build our future's financial
infrastructure.

email: careers@tendermint.com

~~~
sctb
Please post this here instead:
[https://news.ycombinator.com/item?id=15148885](https://news.ycombinator.com/item?id=15148885).

