If anyone is interested in trying local AI, you can give https://recurse.chat/ a spin.
It lets you use local llama.cpp without setup, chat with PDF offline and provides chat history / nested folders chat organization, and can handle thousands of conversations. In addition you can import your ChatGPT history and continue chats with local AI.
Locally it's actually quite easy to setup. I've made an app https://recurse.chat/ which supports Llava 1.6. It takes a zero-config approach so you can just start chatting and the app downloads the model for you.
Not really. VS Code does have some performance optimizations where even the web browser optimization wouldn't suffice, for example it implements its own scroll bar instead of using the web native scroll bar. But for the most part the browser render optimizations is the crucial factor. After years of optimization you can't easily beat a web browser.
A user's personal data really does not have that much scale. Worst case they can cache everything locally. I've imported thousands of chat sessions into a local AI chat app's database, total storage is under 30MB. Full text search (with highlights and all) is almost instant.
Check out https://recurse.chat (I'm the dev). You can import ChatGPT messages. It has almost instant full text search over thousands of chat sessions. Also supports llama.cpp, local embedding / RAG, and most recently bookmarks and nested folders.
I have been using local LLM as a daily driver. Built https://recurse.chat for it.
I've used Llama 3, WizardLM 2, Mistral mostly, and sometimes just trying out models from hugging face (Recently added support for adding it from Hugging Face https://x.com/recursechat/status/1794132295781322909)
It lets you use local llama.cpp without setup, chat with PDF offline and provides chat history / nested folders chat organization, and can handle thousands of conversations. In addition you can import your ChatGPT history and continue chats with local AI.
reply