
Ask HN: What tool do you use to sync MongoDB with Elasticsearch? - anikdas
Currently I am performing full text search on MongoDB but I would love to use Elasticsearch for the same. My primary data source is MongoDB. Right now I am not able find a suitable&#x2F;reliable sync tool for MongoDB-&gt;Elasticsearch.<p>There used to be a tool called elasticsearch-river-mongodb [1] which is deprecated now.<p>Also, there is no official logstash input plugin for MongoDB<p>[1] https:&#x2F;&#x2F;github.com&#x2F;richardwilly98&#x2F;elasticsearch-river-mongodb<p>What tool do you use to reliably sync MongoDB collections with Elasticsearch?
======
FBISurveillance
Take a look at Kafka. When your backend pushes data to MongoDB, have it
produce a Kafka message that contains serialized Elasticsearch JSON documents
and then use either a bespoke consumer, or kafka-connect-elasticsearch, or
logstash-input-kafka to consume those documents and bulk-insert them into
Elasticsearch.

Bonus points: it will let you have a standby Elasticsearch cluster in another
region you could failover to when needed, perform Elasticsearch cluster
upgrades/maintenance without losing data, and more.

Depending on your use case this could be an overkill so YMMV.

~~~
anikdas
I could definitely do that. But my original intention was to use a standalone
application which can get the job done. For example when the data is
updated/deleted instead of insertion I need to write my own logic.

Any way great suggestion. Will definitely keep in consideration.

------
pictur
[https://github.com/indatawetrust/mongo-to-
elastic](https://github.com/indatawetrust/mongo-to-elastic)

