
Sysbench for MySQL 5.0, 5.1, 5.5, 5.6, 5.7 and 8 - tkyjonathan
https://smalldatum.blogspot.com/2017/06/sysbench-for-mysql-50-51-55-56-57-and-8.html
======
kator
It would be interesting to do this same comparison with MariaDB.

I wonder if that project has managed to continue progressing while MySql seems
to have regressed.

It's super easy for all of us on HN to say "Oh look Oracle ruined another
thing". But it would much more interesting to have quantified results to
compare from.

~~~
mdcallag
Blog author here. I think that Oracle has been the best corporate owner for
MySQL, much better than Sun and standalone MySQL.

I write these posts to document & market the problem and motivate upstream and
my team to make things better. I don't think they are ruining MySQL.

------
wcarron
Hmmm, consistently deteriorating performance over time is rather suspicious. A
move by Oracle to kill off competition for their implementations? I can only
speculate, though.

It seems odd, though. I guess it's all PostgreSQL all the time for me, from
now on.

~~~
emfree
I think the author is specifically evaluating low-concurrency in-memory
workloads here. The previous post describes why regressions for those
workloads might be "not a surprise":
[https://smalldatum.blogspot.com/2017/05/the-history-of-
low-c...](https://smalldatum.blogspot.com/2017/05/the-history-of-low-
concurrency.html)

~~~
kijin
Yes, it seems that Oracle does care about performance in high-concurrency
applications: [https://www.mysql.com/why-
mysql/benchmarks/](https://www.mysql.com/why-mysql/benchmarks/)

Even their own benchmarks show MySQL 5.7 dipping below previous versions at
the low end of the concurrency scale.

~~~
mdcallag
Yes, context is everything when looking at benchmark results. I am showing the
worst-case for this performance regression. Thanks for mentioning that. Note,
that is my blog.

I also showed the impact for IO-bound workloads, and even there the regression
is larger than I want. But not as bad as the in-memory workloads.

------
spotman
tldr: mysql is getting worse for small, in memory workloads with low
concurrency.

for the average medium to high traffic mysql install, i find 5.7 to be much
faster than 5.6 or mariadb where concurrency and varying workloads prevail.

ymmv, do your own testing to see which db is a good platform for your specific
workload. maybe it's mysql 5.6 maybe it's mariadb, maybe it's postgres.

for me mysql 5.7 brings much awaited niceness, saner defaults and even though
not all workloads are better on it i think oracle is really trying to iterate.
likely they will fix this regression and there would be little reason to use
it over 5.6 or older.

------
gweinberg
It looks like overall things are getting worse as we "upgrade".

~~~
colonelpopcorn
Oracle buys MySQL, slowly kills it by a thousand small cuts to performance...

~~~
mdcallag
Regression is expected with new features. Oracle has been a great owner for
MySQL. I think we, including Oracle, can make progress on this problem.

------
jd007
Is there any reason nowadays not to deploy one of the more performant forks of
MySQL (e.g. Percona, MariaDB)?

~~~
viraptor
Are they more performant? It would be great to see the comparison on the same
benchmark.

One reason not to move to mariadb could be slight differences in schema syntax
edge cases. But if you're starting from scratch, it doesn't matter.

~~~
jd007
Typically the forks include many performance tweaks over vanilla and so
generally perform better. Here is an old test showing 5.5 differences in
performance:
[http://vbtechsupport.com/657/2/](http://vbtechsupport.com/657/2/)

Percona's fork is also much closer to vanilla (than MariaDB), so there
shouldn't be any issues even for existing databases. Unless maybe if the
system/cluster set up is very complex so there is additional cost for
migration. For new projects though, I can't think of any reason to use vanilla
over one of the forks.

------
devoply
Would Oracle slowly degrade performance so they could sling their enterprise
offerings better.... hmm. However, this is just going to kill MySQL altogether
rather than actually achieve its implied goal. I guess you could say,
competition always finds a way.

------
jacquesm
That's why there is MariaDB.

~~~
mdcallag
Has anyone run similar tests for MariaDB? That is my blog, but I have been too
busy with upstream MySQL to test MariaDB.

------
chx
WTF, MySQL is not doing performance regression testing on each commit?? WUT.

~~~
gtirloni
I find it unlikely that Oracle doesn't have extensive performance testing
infrastructure.

If they have it, how can they miss this? Is it too specific?

If they haven't, did they throw it away after acquiring Sun? Or Sun never had
it?

------
douche
In a world where Postgres exists, and Microsoft SQL Server Express is
supported on Linux, what's the advantage of using MySQL?

~~~
SwellJoe
Lots of reasons why I still use it:

\- MySQL/MariaDB are much better tested for almost every OSS web app and
framework I use. It may be the only database that is official and thoroughly
supported. If I want to interoperate on the same DB across many applications,
MySQL/MariaDB is the lingua franca.

\- It's fast enough. The vast majority of applications never need to scale
beyond a few hundred requests per second. Any database on nearly any hardware
with sufficient memory will do this.

\- Aurora. Scale up automagically (not infinitely, but pretty easily) when you
do find yourself with an app that needs to scale a lot and fast.

\- It's fine. Sure, PostgreSQL does a number of things better (though I'd
wager some of the reasons MySQL is getting slower for some workloads is
because it is now doing some of the things PostgreSQL has done for a while,
and they require more resources; features have a cost). If I were weighing out
the pros and cons in isolation, I'd pick PostgreSQL. But, I'm not. I'm
building things on top of other things, and those other things were built with
MySQL. when MySQL was faster than PostgreSQL I didn't choose it because of its
speed (I used to use PostgreSQL for my company website, but migrated to MySQL
about ten years ago).

MySQL/MariaDB may not be better, but they're good enough. MySQL and PostgreSQL
have converged in terms of features and performance to be reasonable
substitutes for each other.

~~~
elmigranto
> MySQL and PostgreSQL have converged in terms of features and performance

Is there CTEs in MySQL yet?

What about binary JSON and indexes on that?

How is implementing arrays and aggregate functions beyond simple arithmetic
going?

Or maybe there is a way to write triggers in Python or JS if I want some
performant number crunching with ACID guarantees?

Can DDL statements be rolled back, so I wouldn't have to pull my hair out
during development after running migration I'm working on, if it happens to
have a typo?

~~~
EwanToo
CTEs are in MySQL 8

[https://dev.mysql.com/doc/refman/8.0/en/with.html](https://dev.mysql.com/doc/refman/8.0/en/with.html)

