Hmm, does this just use the traditional term frequency search bars (scholar, arxiv, etc) under the hood with query expansion for the prelimary search?
Without chunking the papers I'm skeptical the prelim search would be all that useful.
Also, using GPT4 as a cross encoder seems really wasteful both in terms of compute and latency.
Using GPT4 as a cross encoder also seems very wasteful.
Might try it anyways, but damn 3-6mins is brutal. Traditionally research has shown that low latency is more important than relevance for search because it allows users to reformulate queries.
While the time/cost of using GPT-4 is not ideal, GPT-4-level classification is absolutely crucial for the entire adaptation process to succeed. With 3.5 guiding the adaptation, we find that errors quickly accumulate. It can't identify complex ideas correctly.
3-6 minutes for results takes getting used to, but we've found most people don't complain if it solves a problem that is actually impossible to solve without hours of digging, ie if you use it on something truly hard. Low latency is more crucial for public search engines like Google (0.5s delay -> 20% traffic loss) where there are convenient, fast alternatives.
Preliminary search is a blend of semantic embeddings on 100M+ papers and keyword search, citation links, etc. Reasonably accurate, but full of noise for complex queries.
Without chunking the papers I'm skeptical the prelim search would be all that useful.
Also, using GPT4 as a cross encoder seems really wasteful both in terms of compute and latency.
Using GPT4 as a cross encoder also seems very wasteful.
Might try it anyways, but damn 3-6mins is brutal. Traditionally research has shown that low latency is more important than relevance for search because it allows users to reformulate queries.
Maybe this approach is worth it though.