Like several other commenters in this thread, I also wrote[0] something recently on a related topic: Google's AI Overviews and ChatGPT harm the discovery of long tail information - from a product builder's perspective. Basically, users are having a tougher time finding accurate info about your product (even if the correct answer to their query is in Google's own search results). And I also found the basic tier of ChatGPT hallucinated my app's purpose in a way that was borderline slanderous. AI can make it tougher (at scale) for creators trying to break through.
AI is going to solidify "convention wisdom" and "common sense" as whatever the AI says. That's why there's such a fight over what assumptions and biases get baked into that.
So ChatGPT seemingly guessed its purpose just from its name. Its name is CurrentKey Stats, and it inaccurately described it as an app that kept stats on the current keyboard keys you were pressing, so essentially a key logger, which is again completely wrong. I was actually somewhat hesitant to make this comment out of the fear that the next AI models will train on it and reinforce the false idea that that's what my app is. Sad times
FWIW, one thing they're pretty competent at is sentiment analysis, so if they read your comment, even in isolation, what they'll probably learn that reading that kind of thing into a name is really bad in general.
It's not like the bad old days where sentiment analysis was a bag of words model, add up all the "positive" words and subtract from that total the number of "negative" words — back then, they would mis-identify e.g. "Product was not as described, it did not come with batteries, and the surface wasn't even painted" as "this review favours this product" because they couldn't handle "not" or "wasn't".
You may be happy to learn that I just asked ChatGPT (the free tier that you can access without signing in) "What does the CurrentKey Stats app do?" and got this back:
----
CurrentKey Stats is a macOS application designed to enhance your productivity by offering detailed insights into your app usage and providing advanced management of your Mac's virtual desktops, known as "Spaces."
Key Features:
Time Tracking: Unlike macOS's Screen Time, which tracks total app usage, CurrentKey Stats monitors the time each app spends in the foreground, offering a more accurate representation of active usage.
Room Management: The app allows you to assign custom names and unique menu bar icons to each Space, facilitating easy identification and navigation. You can switch between these "Rooms" using the menu bar icon or customizable hotkeys.
Automation Support: For advanced users, CurrentKey Stats supports AppleScript, enabling you to automate actions when entering or leaving specific Rooms. This feature can trigger scripts to perform tasks like launching applications, adjusting system settings, or controlling smart home devices.
Privacy-Focused: The application prioritizes user privacy by keeping all data local on your device, ensuring that your usage statistics are not shared externally.
By combining detailed app usage statistics with enhanced Space management and automation capabilities, CurrentKey Stats aims to help you use your Mac more efficiently and deliberately.
----
But more generally, I think people often look at LLMs and assume that the current flaws will be around forever and this will be horrible. But all technologies have big, obvious flaws when they're first introduced and these get fixed because there's a strong incentive to have a less bad product.
o1 says "I’m not aware of a widely recognized or universally known product called “CurrentKey Stats,” so it may not be a mainstream or highly publicized app. There are a few possibilities, though:" and then some guesses.
o3-mini is similar: "I couldn’t find any widely recognized information about an app specifically called CurrentKey Stats. It’s possible that:"
When I turn on its shiny new search capability, it correctly identifies and summarizes the app.
o3-mini-high, search turned off, asks for clarification.
Which means the default free 4o is quietly doing retrieval-augmented generation behind the scenes. I thought o3-mini would search if it didn't know the topic, but I might be misremembering.
Fascinating. I would have thought the fancier models would have definitely known about it. The app is six years old, has ~120 ratings globally with ~4.5 stars, with several articles written about it etc.. It should be pretty widely available in the training data
That's good. When i wrote the article, I was able to get a similar response when i used chatgpt "search", but not with the basic tier default prompt with the prompt "i own a Mac, would currentkey stats be good for me?". Were you using the default chatgpt prompt here or chatgpt "search" or a better model?
> Basically, users are having a tougher time finding accurate info about your product (even if the correct answer to their query is in Google's own search results).
That's a gigantic "even if".
In my experience, I'm able to find stuff much easier with LLM's that Google search couldn't surface.
If I'm looking for a product that does exactly X, Y but doesn't Z, keyword search can be pretty terrible. LLM's actually understand what I'm looking for, and have a much higher probability of pointing me to it.
Yeah, i have also found LLMs useful, and sometimes with broad search queries. Which makes it quite the paradox when you own a product on the other side of bad LLM results
[0] - https://keydiscussions.com/2025/02/05/when-google-ai-overvie...