
TiDB 2.1 GA: Battle-Tested to Handle an Unpredictable World - shenli3514
https://pingcap.com/blog/tidb-2.1-ga-Battle-tested-to-handle-an-unpredictable-world/
======
mdcallag
This post inpsired a lot of great discussion about the Spanner family of
distributed databases. Someone asked about TiDB there, but there was no follow
up. Can someone answer that question?

[http://dbmsmusings.blogspot.com/2018/09/newsql-database-
syst...](http://dbmsmusings.blogspot.com/2018/09/newsql-database-systems-are-
failing-to.html)

~~~
morgo
I'll pass this onto the engineering team. Thanks!

For a quick answer, I believe Daniel Abadi's followup comment addresses it:

> This post did not discuss a central oracle. It discusses the difference
> between partitioned vs global consensus.

In TiDB the pd leader is a central oracle.

------
matt2000
Anyone here who is running TiDB in production that can speak to upsides and
downsides? I find I only trust a database after a couple years of real usage,
so it would be great to hear from someone who is currently using it.

Thanks!

~~~
morgo
To your point: I agree that it pays to be conservative with new filesystems
and databases alike.

However, I want to mention mitigating circumstances which have helped TiDB
stabilize:

\- We use RocksDB as the underlying storage library, which was developed by
Facebook, and has been battle-tested by many others.

\- The architecture itself has been inspired (and proven) by Google Spanner

\- Correctness has been proven with TLA+ [https://github.com/pingcap/tla-
plus](https://github.com/pingcap/tla-plus)

\- We have 300+ customers storing 15 PiB+

~~~
mountainview
You can find some case studies here: [https://www.pingcap.com/success-
stories/](https://www.pingcap.com/success-stories/)

Specifically full case studies: [https://www.pingcap.com/success-stories/tidb-
in-iqiyi/](https://www.pingcap.com/success-stories/tidb-in-iqiyi/)
[https://www.pingcap.com/success-stories/tidb-in-
zhuanzhuan/](https://www.pingcap.com/success-stories/tidb-in-zhuanzhuan/)
[https://www.pingcap.com/success-stories/tidb-in-
eleme/](https://www.pingcap.com/success-stories/tidb-in-eleme/)
[https://www.pingcap.com/success-stories/tidb-in-
mobike/](https://www.pingcap.com/success-stories/tidb-in-mobike/)
[https://www.pingcap.com/success-stories/tidb-in-
yuanfudao/](https://www.pingcap.com/success-stories/tidb-in-yuanfudao/)

------
Rafuino
Hey can someone from TiDB help me understand the benchmark results you posted
comparing 2.0 with 2.1?

[https://github.com/pingcap/docs/blob/master/benchmark/sysben...](https://github.com/pingcap/docs/blob/master/benchmark/sysbench-v3.md)

The main question is this: how large is the dataset created in Sysbench? 16
tables with 10,000,000 pieces of data per table is interesting, but how big
are those individual pieces? Morgan said "the expected data volume for TiDB is
larger than memory," so does this benchmark actually test the underlying
storage, or is it all in-memory?

Thanks for any help! Would like to try replicating this if possible in our
lab.

~~~
morgo
Hi Rafuino, send me an email at [firstname]@pingcap.com and I'll connect you
with the benchmark author.

Just to expand a little bit on this comment:

> The expected data volume for TiDB is larger than memory.

There is both total data volume, and working set (workload dependent). This
point was in regards to some databases use data structures which require the
total data to be memory resident.

The working set is a longer discussion - it could just be a few percent of the
total volume, and some working sets have a "long tail" of cache misses vs.
short tail. And it can vary for a B+tree vs. an LSM tree too. RocksDB uses
compression which helps a lot.

It is not clear in this benchmark if the working set fits entirely in memory.
Thanks for bringing it to my attention - I will follow up and make sure we do
a better job of communicating what is being tested.

------
morgo
Morgan from the TiDB team here. Happy to answer any questions you have.

~~~
shodan757
On [https://www.pingcap.com/docs/sql/mysql-
compatibility/](https://www.pingcap.com/docs/sql/mysql-compatibility/) it
mentions "FOREIGN KEY constraints" under unsupported features. Is that right?
Isn't that a rather big problem for an OLTP DB? Or am I missing something?

~~~
gregwebs
Greg from the TiDB team here. I do share your sentiment, and at the moment you
can probably best track or progress on this issue here:
[https://github.com/pingcap/tidb/issues/8484](https://github.com/pingcap/tidb/issues/8484)

The explanation is just that TiDB is being developed with tight feedback from
our customers that have many TB of data. The feedback from that scale of users
is overwhelmingly that they do not want to take the performance hit of foreign
keys. It is worth mentioning though that you can declare foreign keys and that
on master we do properly check DDL statements (but there is no DML
enforcement).

I am trying to figure out a design that will satisfy users with large and
small data alike and even let users use foreign keys for documentation
purposes when they are not enforced for performance reasons. It would be great
to have more community input on this.

