This kind of reminds me of the old web IM interface Meebo, but instead of having your chats with humans, you have AI bot chats.
One stylistic thing that I like about ChatGPT vs Bard's web interface for code is that way it progressively types compared to the instant display. I find that I am able to read the code and get some learning of what it is doing as it is typing compared to reading a big block after the fact.
Huh. I hate the streaming mode of ChatGPT. In my experience, any UI that adopts it ends up feeling slow and sluggish for no apparent reason. Plus, I often find myself looking at the text as it appears, and wondering if I should press the "Stop" button, or will it say something useful in the two paragraphs I expect to still follow.
Alas, I use the streaming mode, because it gives an indicator that the connection to the model is still live, and you can kind of estimate how much longer you'll have to wait for the reply to end. With "batched" mode, you just sit there for anywhere between few seconds and few minutes, wondering if it's still generating response, or if the API connection broke again.
When it streams too slowly it is also quite annoying, but when it is the right speed I really enjoy it.
It can also be annoying when I made request a small change and it rewrites the entire block of code compared to only displaying the section I want. I think I could prompt it better in those cases by saying "only show me the code changes" though.
I'm working on relevant prompts. I've managed to get it to output edits a few times, but most of the time it does ignore my requests to just give me the changes.
I think it's mostly my laziness. I found GPT-4 with default system message good enough for most use cases, so I tend to just switch over to it, start a new conversation, and blurt my request. Setting specialized "characters" for my requests? Too much hassle. Typing in a proper, explicit request to just give me diffs? Too much writing, breaks flow.
Ironically, I'm both too lazy for giving GPT-4 this kind of editing guidance, while I'm also wasting too much time catering to conversational style.
Sad part is that I think it would be quite easy to make a handful of UIs with the built in system prompts to achieve exactly what you want... And someone is going to do it and charge money for it.
> And someone is going to do it and charge money for it.
This already happened. It's called TypingMind.com; I'm happy paying user. It's main selling point is that it isn't dog slow and isn't trying to take down my browser with its bloat, the way OpenAI chat interface is. "Character" creation and selecting is lovely, but half the time I'm too lazy to switch away from the default.
It sounds like you just want a faster response, not a batched one, given that you don't use the batched one for the same reason we prefer the streaming one?
Just imagine it's a human on the other side typing or speaking, and you're getting to hear / see what they type as they type it.
It is, in fact, extremely strange for humans to have the experience of sitting waiting for a complete utterance before processing: it feels more like sending a letter than talking.
There were even chat systems that worked like this, including (I think?) the Unix talk command and one of the popular ICQ-era chat apps (maybe it was ICQ itself?).
> Just imagine it's a human on the other side typing or speaking, and you're getting to hear / see what they type as they type it.
Yes. I imagine it and I shudder from suddenly experiencing how annoying that would be.
> There were even chat systems that worked like this, including (I think?) the Unix talk command
Correct. I've used `talk` in high school (and a colorized variant called `ytalk.qq`). It was a fun experience, and I miss it a bit - since then, the only software I used that kind of reproduced it, was Etherpad.
Now, the way `talk` conversations and real-life conversations are tolerable, whereas watching someone (a human or a bot) typing 'till completion is frustrating, is in the speed of the feedback loop. When talking (or `talk`ing), I can interrupt you when I see you saying things I already know. You can interrupt me too. This happens at the level of phonemes/characters, and allows the conversation to flow dynamically - one moment, we'd have a rapid back-and-forth, another moment, I'd listen to you very carefully, etc.
With normal messengers of today, if I were to see you typing (instead of having the - also annoying, but for other reasons[0] - pencil icon), but had no way to interrupt you - I'd be frustrated all the time, being able to know with good confidence what you want to say while you're 1/3rd through your message. Natural language, especially in conversations, is stupidly redundant. I'd just Alt+TAB to HN or something, and wait for you to finish[1].
This is my experience with GPTs. I often want them to just stop typing, but if I hit "Stop", I might just break the response in the wrong moment, creating cleanup work for myself. The side effect of the streaming chat UIs being dog slow makes this even worse: there's no telling how many seconds it'll take for my "Stop" to register. For some reason, this gets both worse and less predictable as the conversation gets longer.
--
[0] - All kinds of interpersonal nonsense, but a big factor is that, with a bot, you know it's actually generating a message and will finish soon (unless the connection fails). With humans? Who the hell knows. They may type you an essay. Or type two sentences for 5 minutes. Or get distracted by cat. Or just be screwing with you. With humans, half the time I'm considering looking for ways to disable typing indicators.
[1] - Which is exactly what I do most the time with people and typing indicators. And GPTs.
Why is everyone so obsessed with "moat"s every time a new GPT-type service or more general AI product pops up on HN?
Is that just a meme response now or is there more to it?
The market (and technology) is very immature. We have no idea what potential usecases, integrations, or models will ultimately dominate. Even if there is a single one-model-to-rule-them-all there is still going to be potential for smaller/almost-as-good models to be succession by doing direct integrations with various business or B2C usecases.
The chatbot UI is just one potential usecase that's popular now. That might have dominance in pure brand recognition and userbase, but it's not like they have network-effects or other lock-in either (besides maybe custom API integrations).
> Why is everyone so obsessed with "moat"s every time a new GPT-type service or more general AI product pops up on HN?
Because of the second element of HNs domain name. Its a startup-community focussed forum, moat is an important business consideration, but especially so for startups.
This is what I’m talking about. It’s such a limited perspective on what LLMs mean as products/services. Everyone is so focused on the ChatGPT consumer interface they can’t see beyond it to the thousand niche business usecases. It’s as much an enterprise platform (meaning heavy custom usecases), in addition to a broad consumable API system for developers, and then the Google style search/chat Q&A UI.
Just because Poe/Claude is experimenting with an early consumer mobile app doesn’t mean that’s the business they ultimately plan to be in.
The network/monopoly effects of Google/Chrome/IE for pure consumer plays might make sense but we have no idea what the technical underpinning might look like in a decade. It could be just WebKit plus a few others or it could be more like Databases where you hit a baseline plateau for performance but there’s a hundred businesses built off all the variations.
It’s like debating whether “the internet” is a moat, in 1990.
Tech is never a moat. That’s not new or interesting. Business processes can be, brand can be, scale can be. And it is far far far far far far too early to even think about moats in the AI business.
quite the opposite. they haven't existential risk where they might get blocked by one of the providers that I used to sign up, but I still am opted into a monthly paid subscription
It grinds my gears a little bit when people pretend that Google came out of nowhere and invented a new paradigm for accessing information and making money by selling ads there.
It was a well recognized problem they solved with Page rank!
i think the best way to consume these things is a menubar app that can be brought up quickly with a shortcut. i recently released https://github.com/smol-ai/menubar which i personally use!
It reminded me of the same thing. I remember how back before Google dominated the market I used Metacrawler (which surprisingly still exists, but it's a completely different site now)
One stylistic thing that I like about ChatGPT vs Bard's web interface for code is that way it progressively types compared to the instant display. I find that I am able to read the code and get some learning of what it is doing as it is typing compared to reading a big block after the fact.