Are there any good LLM writing apps out yet where it’s not just “writing/code on the left, chat on the right”? I feel like there could be much smoother UX where you’re able to zoom in on sections of your document, conversate about it with the llm and iteratively work through changes, then zoom back out and hop into another area. Almost like if you were peer writing/programming with someone who is observing your writing and offering suggestions, or able to take a second look at a part of something and help you optimize it in context.
I feel like with locally hostable LLMs improving so rapidly, it will become tenable to just be constantly querying the AI as you’re working, allowing for proactive engagement by the AI, which is obviously cost prohibitive today when using API-based LLMs
I’ve mentioned it before on here but I’ve been using someone’s tool called aider for a more integrated experience for coding: https://github.com/paul-gauthier/aider
It’s got a few QoL improvements it could use, especially with larger code bases, but otherwise works pretty well.
If you wanted to use a local LLM you could combine it with something like FastChat to expose an OpenAI compatible API and just point the tool at that URL instead
Are there any good LLM writing apps out yet where it’s not just “writing/code on the left, chat on the right”? I feel like there could be much smoother UX where you’re able to zoom in on sections of your document, conversate about it with the llm and iteratively work through changes, then zoom back out and hop into another area. Almost like if you were peer writing/programming with someone who is observing your writing and offering suggestions, or able to take a second look at a part of something and help you optimize it in context.
I feel like with locally hostable LLMs improving so rapidly, it will become tenable to just be constantly querying the AI as you’re working, allowing for proactive engagement by the AI, which is obviously cost prohibitive today when using API-based LLMs