
Show HN: PyEQS – query Elasticsearch like a Django Queryset - andrewgross
https://github.com/Yipit/pyeqs
======
spencera
We are currently working on an official QuerySet-like python library here at
Elasticsearch:

[https://github.com/elasticsearch/elasticsearch-dsl-
py](https://github.com/elasticsearch/elasticsearch-dsl-py)

~~~
honzakral
Yes, the goal is similar - to provide a nice API without limiting any of the
possibilities. It's much less restrictive (doesn't whitelist parameters) and
tries not to force you to learn both Elasticsearch's query language and a DSL
on top of by staying close to the original api.

It's currently work in progress but you can already see in tests some of the
features like chainable API, composing queries and filters and aggregations:

[https://github.com/elasticsearch/elasticsearch-dsl-
py/blob/m...](https://github.com/elasticsearch/elasticsearch-dsl-
py/blob/master/test_elasticsearch_dsl/test_search.py#L110-L118)

I try to work closely with the fine people at Mozilla currently maintaining
ElasticUtils to make sure there is a nice migration path and I don't miss
anything obvious. Any feedback would be more than welcome (github, twitter,
email, irc #elasticsearch-py, carrier pigeon, ...)!

~~~
andrewgross
Thanks for sharing. Nice to see an alternative approach for defining these
sorts of things.

~~~
honzakral
Happy to - note that the syntax in the search test is mostly just shortcut
(though I fully expect that to be the main interface). You can also use the
objects for individual query types:
[https://github.com/elasticsearch/elasticsearch-dsl-
py/blob/m...](https://github.com/elasticsearch/elasticsearch-dsl-
py/blob/master/test_elasticsearch_dsl/test_query.py#L52-L55)

------
kevinastone
Take a look at ElasticUtils from Mozilla. It also provides a queryset-inspired
interface for elasticsearch.

[http://elasticutils.readthedocs.org/](http://elasticutils.readthedocs.org/)

~~~
andrewgross
Thanks, checking this out.

------
akoumjian
Hi Yipit folks! It's Alec, from the Seattle Django group. We also developed
something like this internally at Nimia due to the exact same use case:
needing to access certain types of queries that didn't fit for haystack. Looks
like we should have open sourced it so we could have collaborated.

------
coolsunglasses
If you'd like something that tackles the challenge of making the Elasticsearch
JSON API explicit:

[http://github.com/bitemyapp/bloodhound/](http://github.com/bitemyapp/bloodhound/)

(Intended to act as a guide as well as a practical ES client)

~~~
andrewgross
Thanks for sharing this, didn't know about it. Always nice to see how other
people tackle the same issue.

~~~
coolsunglasses
Putting together and composing queries in ES is error prone and I have grown
quite allergic to libraries that just write off the query/filter API ES
provides as "json blob lol lol".

You can lose massive amounts of time debugging a broken query in ES, due in no
small part to a lack of an explicit spec.

To respond to the mis-aimed comment:

It's not about just beginning learning - it's a problem the moment you want to
properly compose queries and not just have dumb templates.

I am not new to ES, I've been using it for years.

I understand wanting directness and the full breadth of the Elasticsearch API
(thus moving away from Haystack), but not actually supporting anything _in_
the API (which is the actual hard work, not wrapping an http-client) is
problematic.

~~~
andrewgross
Are you referring to the drive to use more of a languages' built in operators
to build queries, instead of my approach? I think the Mozilla library uses an
approach like that.

[http://elasticutils.readthedocs.org/en/latest/](http://elasticutils.readthedocs.org/en/latest/)

~~~
coolsunglasses
Yeah that looks to be closer to what I'd like, although it's still not type-
safe.

------
zmsmith
I work with Andrew at Yipit. As he mentioned in a comment below, we're using
features of Elasticsearch that aren't (and probably shouldn't be) exposed by
haystack, and some of the query building started to get a little messy.

I'm excited Andrew decided to work on this during our last hackathon and that
now we're able to share it with the community.

We've been really happy with Elasticsearch. Hopefully this library will help
more people leverage Elasticsearch in their python projects.

------
jaegerpicker
I'll have to play with it, currently we use the SearchQuerySet api provided by
django_haystack to query Elasticsearch and it works pretty well for us.

~~~
andrewgross
We used to use Haystack, but found it a bit too opinionated for us once we
wanted to do some custom stuff. It is a bit more faithful to the Django
queryset API, something we had to abandon to let us use more of the complex
Elasticsearch query features.

------
sjaaktrekhaak
Is the precommit hook[1] a custom one or is it an off-the-shelf Django
recommended hook? Looks like it has some neat tricks and code style checking.

[1]: [https://github.com/Yipit/pyeqs/blob/master/pre-
commit](https://github.com/Yipit/pyeqs/blob/master/pre-commit)

~~~
andrewgross
It is a modified version of what we use at Yipit. I stripped out mostly things
related to working with Django/Flask apps. It comes in very handy when making
sure we maintain style and push fewer broken commits. Feel free to re-use.

------
lqdc13
bad coding style in some places (e.g.:
[https://github.com/Yipit/pyeqs/blob/master/pyeqs/dsl/range.p...](https://github.com/Yipit/pyeqs/blob/master/pyeqs/dsl/range.py)
where he redefines python range)

~~~
andrewgross
Fixed. Completely missed that.

------
jacquesc
I use Ruby and am massively jealous of this.

Anyone know of good Ruby alternatives?

thanks!

~~~
sprobertson
There's Plunk, a "human friendly" Elasticsearch DSL:
[https://github.com/elbii/plunk](https://github.com/elbii/plunk)

------
andrewstuart
Python 2 only?

~~~
andrewgross
Good point, I will have to swap out some testing libs but should be possible.

