
ElasticDump – Import and export tools for ElasticSearch - evantahler
https://github.com/taskrabbit/elasticsearch-dump
======
evlapix
I haven't used this yet, but it seems like it should make updating mappings
much easier. See the "using aliases for greater flexibility" section in this
article: [http://www.elasticsearch.org/blog/changing-mapping-with-
zero...](http://www.elasticsearch.org/blog/changing-mapping-with-zero-
downtime/)

I had to write the scripts to implement this concept myself and it wasn't a
quick and easy task. It would had gone along a lot easier if I was able to
abstract away some of those queries with a tool like this.

~~~
evantahler
We use index mappings at TaskRabbit, and it does make things a lot easier. It
helps with 0-downtime mapping changes (as you point out), and it also lets us
combine large indexes. We segment large data collections by time (ie:
events_year_month) which we roll up in an alias. This allows us to delete/add
large collections without downtime as well.

What's cool about the elsasticsearch api (and therefore the elasticdump tool)
is that with aliases, you can read/write to and from an index just like you
can an alias.

------
andrewvc
This is a neat tool for moving data around, but for serious stuff I'm not sure
why you would use this over the official snapshot / restore API
([http://www.elasticsearch.org/guide/en/elasticsearch/referenc...](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-
snapshots.html)).

While the API there is a bit more cumbersome, it's one less dependency, and
should be more efficient.

For the specific use case of backups, the official API is even less
cumbersome, esp. if you're using the cloud plugin and the S3 export option.

~~~
schmichael
snapshot/restore doesn't allow applying new mappings.

Also I use our similar custom tool* for dev purposes where you might want to
kill the copy after a while and just work with a subset of the data... or
explicitly apply a filter to copy a subset of data.

* [https://github.com/lytics/escp](https://github.com/lytics/escp)

~~~
Xylakant
on top of that you can apply transformations to the data if you use scroll and
bulk, but not with dump/restore. On the flip side, dump/restore allows
incremental transfer which is a bit of work to do if you're implementing your
own copy script.

------
Argorak
It really needs some cleanup and some extractions for public use, but we have
a set of rake tasks to do all that and a bit more (e.g. reindexing into a new
index and flipping the associated alias)

[https://github.com/asquera/elasticsearch-rake-
tasks](https://github.com/asquera/elasticsearch-rake-tasks)

------
schmichael
While we don't have a spiffy logo, we made a tool for copying indexes around
for testing/dev:

[https://github.com/lytics/escp](https://github.com/lytics/escp)

------
rhoml
So whats the difference from this and the embedded backup restore tools on ES
1.0.1?

~~~
lukeholder
snapshot/restore doesn't allow applying new mappings.

------
coolsunglasses
Replicating this could be a good way for me to exercise the (okay) index and
(very poor at the moment) mapping support in
[https://github.com/bitemyapp/bloodhound](https://github.com/bitemyapp/bloodhound)

