
Kdb+ and Python: EmbedPy and PyQ - gricardo99
https://kx.com/blog/kdb-python-embedpy-pyq/
======
osrec
I've never quite understood why banks still opt for kdb when (a) hiring staff
is ridiculously difficult/expensive and (b) there are so many other cheaper
alternatives available. They literally have a strangle hold on every eFX desk
in London, and I cannot really figure out why!

~~~
geocar
Maybe you're wrong about (b)?

Sure, I could cobble something about as fast and functional out of postgres,
C, awk, and a bunch of other things, but it'd certainly cost more _just in
developer time_ than kdb.

~~~
osrec
Your comment made me chuckle. In fact, made me immediately think you were
employed by Kx at some point; your profile confirmed this was the case!

I think Kx has built a wonderful business, but as an ex MD at a bulge bracket
bank, I would not sanction its use any more. It is expensive, and it's
difficult to hire good people to work with it (not every guy with kdb on their
CV is great). The product is good, but so are many open source offerings,
which are now matching it both in terms of performance and flexibility. Plus
hiring good people is significantly easier and cheaper.

~~~
oper8or
What are the open source offerings that you referring to? The ones that
provide 1) no-copy analytics 2) i/o stack bypass i.e. memory-map.

~~~
oper8or
Thank you for your answers. To summarize, we have 1) Hobbes 2) hacked LMDB 3)
C++ memory-mapped store of arrays.

Given that options #2 and #3 require some (non-trivial) work, they are not
really options.

We left with #1,-- hobbes, which was uploaded to GitHub about 5 months ago and
has a whopping team of 2 contributors, both employed by Morgan Stanley.

This is more than nothing, but not much.

I do not have experience with KDB, and looking at the language syntax, not a
fan. Integration with Python (depending on implementation) may push KDB
towards larger acceptance.

So far I was mostly relying on a variation of the option #3.

~~~
kthielen
You must be able to come up with better criticism than that. Number of people
who work on the project, time it's been on github, contributors' employers ...
these are completely irrelevant to the question of whether this project is a
viable alternative. There's not a single technical argument here! :)

For what it's worth (not much), from a purely superficial standpoint, kdb
itself started out as a one or two person project at Morgan Stanley! :D

We've managed to get this thing right in the hot path (not just for analysis
off on the side, though that use-case is important too) where a significant
portion of global trading happens, in one of the biggest investment banks in
the country, and we've had it working in production for four years doing this
(before recently open sourcing), having had to make the technical case to many
people who are very aware of kdb and what it can do (as far as kdb goes,
Morgan Stanley is Mount Doom!).

I mean, I take your point that it's not ubiquitous in the world yet, but in
terms of the OP proposition that there are free and technically superior
alternatives, it's proof positive.

------
bkeroack
I like it when kdb is mentioned because then I can post this link:
[https://github.com/KxSystems/kdb/blob/master/c/c/k.h#L96](https://github.com/KxSystems/kdb/blob/master/c/c/k.h#L96)

It's one of my all-time favorites. A window into a certain type of mind.

~~~
chc4
I'm surprised you didn't link an actual c file; everything related to k seems
to trigger an exorbitantly high number of "what the fuck"s.
[https://github.com/kevinlawler/kona/blob/master/src/0.c](https://github.com/kevinlawler/kona/blob/master/src/0.c)
for example.

[http://archive.vector.org.uk/art10501320](http://archive.vector.org.uk/art10501320)
is one of my favorite articles, though

~~~
RodgerTheGreat
That article was enough to inspire me to write a K interpreter, and eventually
land me a job working with K. If I ever meet Stephen Taylor in person I
imagine it'll be an interesting story to tell.

~~~
kthielen
That's great that your interest produced that result! When I made a K
interpreter, Kx threatened to sue me and everyone I had worked for.

~~~
beagle3
That’s likely because yours was fast enough to threaten their sales, whereas
RodgerTheGreat’s is JavaScript and can not.

Nick Nickolov’s one also disappeared off GitHub, though Kevin Lawler’s Kona k3
implementation and Andrey Zholos’s jitted weird dialect are fast and still up;
also nils holm’s klong.

------
mclovinit345
the idea that I could seamlessly move data between python and kdb+ is making
me salivate. Please, make this work well and lower the cost-barrier to using
kdb+ and I think you'd see this become the defacto "stack" for many database
setups.

~~~
ah-
All this needs now is a free (at least commercial use allowed 64bit version,
ideally OSS) kdb+.

I'd love to use this, many things are so much more straight forward in q than
they are in pandas. But if it means noone can run my software unless they pay
for kdb+ it's a non-starter.

I wonder if this would work with Kona.

