

High Concurrency MySQL @ Facebook #mysqlconf - igrigorik
http://docs.google.com/present/view?id=dgjzt2ms_40gjxrjkdx

======
superjared
Those slides are incredibly dense and it's a little difficult to tell exactly
what they mean. The OP's twitter stream has some good info though:

"facebook is carving out entire chunks from InnoDB to scale up concurrency: no
deadlock detection, no read-ahead.."
<http://twitter.com/igrigorik/status/12117662626>

"myisam has single mutex, will fall down after 3+ cores try to access it
concurrently. stick with innnodb"
<http://twitter.com/igrigorik/status/12116958816>

~~~
igrigorik
Yep, just to provide some context / quick review of the talk:

It seems like the primary concern for Domas (presenter) is concurrency within
InnoDB. Just to get it out of the way, he dismissed MyISAM, and went right
into the low level optimizations & tweaks they're working on at Facebook. High
level points are:

\- they're using XFS, and he stressed understanding the block math and how it
relates to your record sizes, etc.

\- using "correct" scheduler has a big impact

\- swappiness = 0, etc.

For InnoDB itself, it seems like they're carving out entire chunks of
functionality that he claims "never worked correctly" anyway. For example,
deadlock detection is something they disabled outright because it gave them a
large perf boost. Likewise, for read-ahead functionality within InnoDB. Those
were the two he went into a lot of details.

Concurrency + threads: there is no definitive formula, but he suggested
setting thread concurrency to be the min concurrency of your lightweight
requests (claimed: threads are cheap).

For ops / day-to-day, they focus on slow servers and dive right in: custom
tools + gdb + freeze the server. They are playing with the idea of providing
stored procedures interface to app developers since it'll avoid the round trip
time between different queries (TBD). Also, they sometimes go very aggressive
(timeouts of 3s, or even 1s), to force internal app devs to respect the DB
layer.

Also hinted that they're using "smart caches" which are capable of buffering
counters / updates and doing one update instead.

All in all, their biggest problems seems to be thousands of (app) servers that
are always connected, hence the need for high concurrency...

(I'm sure I'm forgetting a whole lot, but that's off the top of my head)

~~~
spudlyo
Thanks a lot for the summary. Wish I was there.

