What was the total token count? The output is impressive (if accurate). I'm also curious if doing things like omitting function bodies would alter the output (obvious makes the process cheaper and would enable larger projects, but may lead to worse analyses).
From the Anthropic logs looks like 1827 input tokens and 1038 output tokens.
I'm still on the free trial API plan, but at Opus price of $15 per million input tokens and $75 per million output tokens that comes to about 10.5 cents.
Do you think it's possible to play a game of 20 questions and go from not knowing what HNSW is and in what context it is used, to become a sophisticated user of any library that implements this?
Thanks for the notes, I thought it would be a more detailed prompt :) Any reason why you choose Opus instead of other LLMs, was it because of the 200k context window?
I'm defaulting to Opus at the moment partly because it's brand new and so I need to spend time with it to get a feel for it - but also because so far it seems to be better than GPT-4 for code stuff. I've had a bunch of examples of it writing mistake-free code that GPT-4 had generated with small bugs in.
From a quick survey of the implementation probably not very well since, for example, it is using dynamic dispatching for all distance calculations and there are a lot of allocations in the hot-path.
Maybe it would be better to post this repository as a reference / teaching implementation of HNSW.
This is in pretty early stages. Might consider instant-distance which I wrote a few years ago and which is in production use at instantdomainsearch.com:
I can answer #3. HNSW will allow for incremental index rebuilding. So each additional insert is a sublinear, but greater than constant time, operation.
I can answer how it would be in Qdrant if interested. The index will take around 70GB RAM. New vectors are first placed in a non-indexed segment and are immediately available for search while the index is being built. The vectors and the index can be offloaded to disk. Search will take some milliseconds.
If people are interested in fast, scalable HNSW implementations which have solid rust bindings then I’d highly recommend checking out usearch https://github.com/unum-cloud/usearch
This project is a library so it's actually relevant? The language libraries are written in is one of their most important features because it determines if/when you can use them...
I strongly disagree. If I'm looking for one of these libraries then the language is written in is extremely relevant in almost all cases. If I'm writing a Rust program then this is very relevant but if it said "in Java" then I would only use it if I had literally no other choice.
> As a Rust dev myself, can we stop the trend of adding "in Rust" to titles?
tbf it's the headline of the Readme.
More generally I largely agree for software. I don't really agree for library code like this though... I actually care about what language a library is implemented in when I'm coding up a project that might use it.
https://gist.github.com/simonw/9ff9a0ab8ab64e8aa8d160c4294c0...
You don't have a license on the code yet (weirdly Claude hallucinated MIT).
Notes on how I generated this in the comments on that Gist.