Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Unfortunately, there are no open source alternatives to kdb+. Effortlessly handling hundreds of terabytes of data on a single machine, while consisting of only a 300 kB (yes, kilobytes)... Nobody even came close.

Market adoption could be better, but the license costs around $100,000/year (probably the most expensive software per kilobyte). Fintech can afford it, other industries can’t.



You'll also end up spending at least that much on consultants. While KDB is remarkable in many ways -- the first time I saw it I thought the demo was fake, the performance so great -- it is very difficult to develop in.

They have a an SQLish interface, that allows non-specialists to get work done, but anything serious needs to move beyond that.

I haven't used it in a few years, but queries had to be carefully optimized -- swapping the order in a where clause could cause order of magnitude differences in performance. Also it is append-optimized. If you need to update or insert data, it is a nontrivial exercise.


Regarding being append-only — these days, where reactive programming, immutability and FP are all in vogue, this is a feature. As for the rest, yes — K/Q are idiosyncratic, but they are simpler than it looks at first sight (and second.. and fifth). :)


> Effortlessly handling hundreds of terabytes of data on a single machine

Single machine, because it can't handle parallel queries, at all, and there are no options to scale it.

Every financial institution I've worked at is busily unwinding their investment in q/kdb. It's legacy tech.


What are they moving to?


Other TS products like OneTick, RDBMSs for TAQ (esp. if you need non-TS indexes), streambase for UI backends

I even met a programmer who hacked something together in Java using the same columnar kdb layout but, you know, multithreaded, so different basket optimization jobs could run simultaneously on the same store.


Now there is shakti, https://anaconda.org/shaktidb

This seems pretty much next generation k.

We can see how things goes.


kdb is frequently described as an in-memory database, or with great emphasis on how efficiently it utilizes a ton of RAM. I've no idea how much does its performance actually drop when the amount of data goes past, say, 2x the amount of available memory, but I'm pretty sure hardly anyone can afford multiple terabytes of RAM in addition to the software license these days…




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: