

Crate: Elastic Data Store - panarky
https://crate.io/overview/

======
watson
Have anybody any experience with this DB they would like to share here?

~~~
mping
I took a spin a few weeks ago, it seemed like it would be a great fit to our
needs: \- we store alot of metrics for custom KPIs \- we don't do joins \- we
don't need transactions \- we're write once read heavy, sometimes re-write \-
crate's sharding, partitioning and replication, as well as "sql" api is great
for our use case

I even implemented some custom aggregation functions to check out the code,
and it was easy. But what I liked the most was the responsiveness of the devs,
the project seems to be moving at a good pace and the devs were very helpful
over github.

I wished the site made clear that it's using ElasticSearch under the hood,
though. I don't know a thing about ES so I can't really comment on how the
sharding, partitioning and replication are achieved. What I can remember from
aphyr's blog (jepsen series) is that it's not recommended as a primary
database though.

------
otterley
Previous discussion on HN, 6 months ago:
[https://news.ycombinator.com/item?id=7609323](https://news.ycombinator.com/item?id=7609323)

------
adrusi
OT: the domain name crate.io confused me into thinking this was about Cargo,
rust's package manager, which is documented on crates.io

~~~
forsaken
This domain used to be Python's new package manager as well.

It's still live under /packages/:
[https://crate.io/packages/requests/](https://crate.io/packages/requests/)

~~~
joshfinnie
They mention that here: [https://crate.io/thanks-
dstufft/](https://crate.io/thanks-dstufft/)

This confused me greatly as well...

------
Kequc
Looks great, really. I'm pretty tired of Postgresql all I want is a simple
modern data store that can handle arrays and blobs. Relationships and data
validation is all handled at the application level for me. I'm very interested
to try this out as soon as there is an implementation for active record.

Thanks for the heads up.

~~~
sanderjd
> all I want is a simple modern data store that can handle arrays and blobs

I'm curious in what way Postgresql doesn't fit that, for you. Is it not
"simple"?

~~~
Kequc
It has a lot of features that aren't needed in modern application development.
I just need indexible columns. Arrays were added later so you have to include
a module, I mean maybe it's just a feeling but it all feels dated.

More importantly it isn't elastic. Constructing a heavy load or large database
server cluster is difficult. More complicated than it should be. If there is
on offer an elastic solution with simple arrays built in, I'm all over it
right now.

Blob support in postgresql is seen as something you shouldn't do as a best
practice. If it's a data store then what I want is to be able to use it to
store data easily without a fuss. Postgresql is the king right now and I use
it but I'm very open to looking at alternatives.

~~~
sanderjd
Your last two paragraphs make a bunch of sense - clustering should be easier
and blobs definitely aren't the "happy path".

I don't relate much to the first paragraph though. You don't have to include a
module to use arrays. I'm mystified by the implication that data integrity,
normalization, and ad-hoc querying are useless for "modern application
development".

~~~
Kequc
That stuff is handled at the application level a data store should just be
there to store and retrieve data, I think databases used to need to do a lot
more than they do now. I thought Postgres required one to include a module in
order to gain array support.

Looking into it again now it seems I was confusing Arrays for Hashes, with
regard to the hStore extension.

The application is in charge of what data goes into the database. It is
performing sanitisation, validation, returning errors, all of that before
persisting data. I don't want to have to deal with errors from the database
nor do I want to have to manage data integrity from two separate locations.

I want to be able to tell the database to store something and it stores it
modern development frameworks can handle data integrity stuff.

~~~
AlterEgo20
"modern development frameworks can handle data integrity stuff". Wrong. They
cant. If you have 2+ applications working with the same db, there will be
problems with data integrity that no "modern development framework" can solve.

~~~
Kequc
If you have two applications making use of the same database, then build a
shared ORM layer and have the applications interact with that. Modern
applications do not interact with the database directly there is an ORM.

Separate that out and share it between however many applications you want you
should be able to swap out data stores not be confined to them.

~~~
sanderjd
Your solutions to these problems (like "do validation and integrity in the
application layer" and "build a shared ORM layer for multiple applications")
are definitely _possible_ solutions to the problems, but you state them as if
they are obviously the _best_ solutions. They actually sound a lot harder to
me and seem to be driven by some theoretical purity in having a dumb data
store that I don't really understand. Building a shared ORM layer for multiple
applications to use is just a hack around having a shared data store layer
doing validation and integrity, which is what a smarter database already does.
It's reinventing one of the really hard things that databases are already good
at. What is the advantage?

I've also never made an application that doesn't eventually want to do
something that the ORM I'm using doesn't know how to do, at which point it
makes a lot more sense to interact directly with the database than to shrug my
shoulders and say "I guess I can't do that".

Maybe my applications just aren't "modern" enough.

