
Elasticsearch 6.0.0 GA released - haakon
https://www.elastic.co/blog/elasticsearch-6-0-0-released
======
eric_b
The new sequence IDs are the most interesting things in this release I think.
Having an officially supported cross datacenter replication strategy would be
real nice.

Lots of folks will be mad, but removing multiple mapping types is a nice
change too. It was a feature that never really made sense. Index-per-type was
always the better strategy, even going back to the 0.23 days.

As others in this thread will no doubt point out though - the ES folks are
moving awfully fast. I still support 1.7.5 clusters that will take heroic
efforts to update. I'd love to use the new features on those clusters, but
there simply isn't a valid business case to take on the risk. This isn't like
small NPM packages that you can update on a whim - some of these systems
require terabytes of re-indexing to upgrade :/

~~~
naiv
Cross data center replication is really a much needed feature.

The way Elasticsearch is going though looks promising. With sorted indices,
single mapping type and the other changes we might give it another try after
switching to Algolia.

Is there a safe way now to query Elasticsearch directly without the need to go
via proxy scripts on the server? This just adds so much overhead to the
queries compared to Algolia.

~~~
jasontedor
Jason from Elastic here. We are actively developing cross datacenter
replication (internally we are calling it "cross cluster replication" so you
will likely see it referred to this in the future but of course this is
subject to change).

I can not give a timeframe, but it is one of the top features on the ES
roadmap. :)

Source: I am an engineer working on this feature.

~~~
bullen
Maybe you could be inspired by my async-to-async database:
[https://github.com/tinspin/rupy/wiki/Storage](https://github.com/tinspin/rupy/wiki/Storage)

------
sidi
My biggest gripe with the 6.0.0 GA is the removal of multiple mapping types
per index. This creates a significant breaking change that will hurt the
community tools adoption to 6.0. Their initial plan was to deprecate it and
only remove it v7 onwards, which imho would've been a better balance.

~~~
pudo
The removal of mapping types really kicks the bottom out of the app we're
making; it's some serious docker-style "break all the things" behaviour.
Seriously losing trust on this one.

Are there any good alternatives to ES? Has Solr moved?

~~~
milesokeefe
Can you not use a separate index for each mapping type and just combine the
indexes by aliasing them all to one "virtual" index? I think that would
effectively be the same but I may be forgetting something.

~~~
notimetorelax
I believe single document updates fail if alias refers to multiple indices.

------
Qerub
I'm excited about the official Docker images now being available (again?) in
an OSS flavor without X-Pack. Quote from
[https://www.elastic.co/guide/en/elasticsearch/reference/6.0/...](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/docker.html)
:

> The images are available in three different configurations or "flavors". The
> basic flavor, which is the default, ships with X-Pack Basic features pre-
> installed and automatically activated with a free licence. The platinum
> flavor features all X-Pack functionally under a 30-day trial licence. The
> oss flavor does not include X-Pack, and contains only open-source
> Elasticsearch.

~~~
praseodym
The pre-installed X-Pack basic license sounds great as well: this at least
allows one to use features like monitoring, dev tools and the upgrade
assistant out of the box, without the 30-day trial kicking in as was the case
for the 5.x Docker images.

------
jdhawk
I really wish they'd included the ElasticSearch SQL in 6.0, but I guess that
feature still isn't fully baked.

Even if it didn't have the full power of the Elastic JSON Queries, for simple
SELECT COUNT() ..GROUP BY, it would have been a nice addition...oh well, back
to counting open and closed brackets...

~~~
djhworld
We're just in the process of looking into ElasticSearch for an analytics use
case, most of the queries we will be doing are simple group by aggregations
with count/sums etc

A SQL interface would help a lot, even better if it came with a JDBC driver

~~~
alexott
There is some ES integration in the Apache Calcite. And it has JDBC driver...

P.S. Writing this level of SQL for ES that you describe isn't very difficult -
in my project we got working in implementation in 2-3 weeks. Take Calcite.
(Recommended but complex, imho), Facebook's Presto SQL parser (not
recommended, but simpler)

------
jordanab
In a large internal project we still make heavy use of Groovy for scripting.
All scripting languages besides the new Painless language were deprecated in
version 5, and now in version 6 they are removed. This hampers us in our
migration efforts from 5 to 6. Does anyone know if it is still possible to use
Groovy through a separate (third party) plugin? I played around a bit with
Painless, but can't say that I really like it. Documentation was/is kind of
poor, and it seemed to me it somewhat assumed familiarity with Java and it's
framework/APIs.

~~~
milesokeefe
Might be worth trying the Groovy plugin that was needed prior to ES
integrating it:

[https://github.com/elastic/elasticsearch-lang-
groovy](https://github.com/elastic/elasticsearch-lang-groovy)

~~~
jordanab
Thx. I already came across that repo, and seems very outdated. I might have to
check out creating my own plugin, and use this as a start.

------
sbr464
Very much appreciate the improvements to handling docs with sparse fields!

------
lclarkmichalek
Does Lucene 7.0 have a better story around G1 GC? ES has always warned against
it, would love to see if that had changed

~~~
dakrone
We revisit the G1GC recommendation every once it a while. In fact, I am doing
benchmarks and testing for G1GC versus CMS with Elasticsearch 6.0.0 right now,
so that we have a better idea of where we stand.

Disclaimer: I'm an Elasticsearch dev employed by Elastic.

~~~
lclarkmichalek
Cool, I've a pretty big cluster with some GC issues (p90 - 15s, p99 - 60s)
during node failures, and would be super interested in those results! If
there's anything a user can do to help, my email is on my user page :D

~~~
notimetorelax
We observed in past that long GC is the cause of node failures. When long GC
happens node doesn’t respond, master node decides that this node had left the
cluster :\

~~~
lclarkmichalek
Ya, we often see a node die of natural causes, and then the garbage produced
from recovering the node and relocating the data ends up bringing down the
rest of the cluster via long GC pauses.

------
sschueller
Off topic but how do you guys Unit Test Elastic Search queries? Do you start a
full node and dump test data in it?

In case of SQL I can start an in memory sqllite and run my tests (Symfony
PHP).

~~~
gerbal
We've had good results using docker-compose to run ES test suites. Run the app
in one container and an ES instance in another.

Test data can be loaded from fixtures or captured/snapshotted using `docker
commit` to create specific test images.

------
chatman
Not as exciting as Solr 7 release.

~~~
liveoneggs
I've been using solr since 1.3 and found trying to use elastic is pretty crazy
annoying. I'd love to swap the ELK stack for a solar-based setup.

~~~
joking
lucidworks made banana, a kibana port to solr, but never take off like kibana.
I dont find the diferences between solr and elasticsearch so big, and always
prefered the solr way of configurating things instead of the json api of
elasticsearch. The solr data-import-handler has been also a big timesaver.

