Yeah, they seen how their product is used and now expanding the use case. Most of their userbase don’t know these tools, and looking for a better search than google.
Does it add much? I currently include "don't search web unless I ask" in my profile instructions. Then, when I want it to search the web, I just include "please search web" in my prompt. It works every time.
> We’re launching to a small group of users and publishers to get feedback. While this prototype is temporary, we plan to integrate the best of these features directly into ChatGPT in the future.
This is interesting. Right now there is a big distinction between how I use an LLM chat bot and a search engine. I'll have to get used to them being merged into one thing
Yeah it would suck if they decided to add search features into ChatGPT without a way to turn it off. There are lots of use cases for a language model on its own without all the extra noise. I really hope they keep them separate.
I presume that in terms of the OpenAI Assistants API (i.e. what you’d be using if you have a specific “use case”), search will be just another built-in tool (like the code interpreter) that an Assistant can be told to invoke.
(And “SearchGPT” is probably just such an Assistant, with the currently-hidden tool forced on.)
They seem to be slow-walking it with a select few users like with that Toys'R'Us ad. Probably to figure out how to sell it without risking regulation when it's inevitably used against politicians.
I would assume any product built on llm that has 10 million MAU, openai will go after. On the LLM side the gap is close between openai, anthropic and now meta. OpenAI might start to focus on products that’s built on its technology even if it’s UI/UX frontend.
Phind too maybe. Though Phind can be used without log in, it starts giving you a fake “at capacity” modal and forces you to sign in after enough queries. Funny how they are no longer “at capacity” when I clear my cookies.
Lol more like, Perplexity has a terminal diagnosis.
Doing RAG using like.. prompt hacking and text embeddings + vector store when you have no access to the underlying model, nor ability to fine tune the generation for RAG, will fail. It will fail in an epic fashion compared to doing RAG the right way.
What do I mean by RAG the right way? The RAG term has been overloaded.
There's RAG that's just kind of bolted onto the LLM after it's been fine tuned for instruction following, and then there's RAG where the document/fact retrieval is a part of the LLM itself that is differentiated and optimized.
Almost everyone is doing the first "hacky" kind of RAG, but Meta published in 2020, the "correct" way to do RAG, where you include a neural retriever in the feedback loop.
Almost no one is doing this because it's more expensive (requires fine tuning the model), but will produce much better results than doing "bolted-on" RAG.
I'm not sure why the assumption of "will produce much better results". The fine tuning is not that predictable. Maybe some documents are remembered, maybe not. Maybe the document markers are preserved, maybe they fail. And adding anything new risks destroying existing data and is expensive.
Compare that to the vector + graph search, which is almost free to add to (if you're searching the internet, you're adding N documents per minute, not per days of training), repeatable, not affecting existing data. It would be cool to have a neural search, but how realistic is it without making it extremely fuzzy, forgetting and expensive?
Actually I disagree. I think the barrier to entry on a decent RAG system is very low. Embeddings have gotten so good, that retrieving chunks is going to be commoditized. Neural search was needed when embedding models were not good enough.
The reason they aren't dropping new models, is they are using their LLMs to embed in products, not as the end product. The APIs are a race to the bottom. They are going after Google.
