
Show HN: EDB – A framework to make and manage backups of your database - RoxasShadow
https://github.com/RoxasShadow/EDB
======
dchest
Filed issues related to crypto:
[https://github.com/RoxasShadow/EDB/issues/created_by/dchest](https://github.com/RoxasShadow/EDB/issues/created_by/dchest)

~~~
RoxasShadow
Thanks for the good points raised!

------
falcolas
The mysql driver needs additional work. In particular, it needs a '\--single-
transaction' flag, or a global lock, to ensure that the dump is consistent -
particularly if you want to dump multiple databases concurrently.

Doing one dump per table with chunking (each file has N rows) would help with
both speed and disk sizes of backups by allowing S3 or some other program to
implement de-duplication between incremental backups.

It also wouldn't hurt to capture the binlog position, if available, to enable
point in time recovery.

Have a look at mydumper for an idea of how another tool implemented these:

[https://launchpad.net/mydumper](https://launchpad.net/mydumper)

~~~
RoxasShadow
I will open an issue with your suggestion. Thanks.

------
inquist
Simple, clean, straightforward. I've done similar things in bash before but
not bothered to publish :) It's a good idea though to have an ecosystem of
simple but flexible tools like this.

Here's another idea we implemented alongside a backup system like this. Say
you run a backup daily; you can run another script to prune the archives,
keeping D most recent daily archives, W most recent weekly archives, M most
recent monthly archives, and all yearly archives.

~~~
RoxasShadow
Thank you for your feedback.

That would suite well as module when used alongside the FTP module. I'll keep
it in mind :)

------
mickeyben
I've recently created a similar tool at our company.

We found out that dumping the database table by table in a parallelized way is
much faster than a full database mysqldump.

There's also mysqlpump soon available in mysql 5.7 to replace mysqldump and
works in parallel.

~~~
falcolas
Until then, have a look at "mydumper", which does both parallel dumps at a
table level, but also chunking, which makes de-duplication possible.

