The idea that we can solve "language" by breaking down and understanding sentences is naive and funny with the benefit of hindsight, is it not?
An equivalently funny attitude seems to be the "natural language will replace programming languages". Let's see how that one will work out when the hype is over.
That grammar doesn't necessarily convey 100% of semantics is a problem of natural language. Or rather, of people being poor at communicating unambiguously.
Programming languages can also be ambiguous sometimes, but that ambiguity is resolved before execution by essentially following a priority list or throwing an error if no combination of rules fits.
Entropix will get it's time in the sun, but for now, the LLM academic community is still 2 years behind the open source community. Min_p sampling is going to end up getting an oral about it at ICLR with the scores it's getting...
> the LLM academic community is still 2 years behind the open source community
Huh, isn't it the other way around? Thanks to the academic (and open) research about LLMs, we have any open source community around LLMs in the first place.
Have you been paying attention at all these past few weeks? Google is crushing it with releases. Gemini 2.0 is great, Veo2 is crushing Sora, live video conversation from aistudio... 12 days of OpenAI turned out to be 12 days of Google.
My respectful counterpoint is that most people aren't paying attention to tech releases at all, ever, unless they go viral like ChatGPT did.
I have very nontechnical coworkers get excited about cool new things ChatGPT can do, but I'm not certain any of them even know we _have_ Gemini in our Google Workspace.
This would hardly be the first time Google has produced innovative technology which eventually fizzles because it never captured much mindshare outside of the tech news circles
Google's recent launches have been technically impressive (especially Veo 2), but given the company's past track record on creating new products, I'm not very bullish that they can turn those launches into products with the same excitement and sense of direction as OpenAI at least appears to have. Google has the benefit of having platforms that span billions of devices and people, but with the looming threat of antitrust regulation, I'm not so sure they'll have the benefit of the last thing for long. Granted, I doubt that 1-800-ChatGPT will be a significant source of users for the product, but it does signal some of the creativity from the company that seems to be escaping Google regularly (see: NotebookLM's leads leaving to form their own startup).
Google search also was good at the beginning. Now it occasionally gives results that contain none of the keywords and have nothing to do with what I searched for.
Thanks! I am collecting all "test prompts" which appear at HN and Reddit. I plan to create a hugging face dataset. I will soon publish "Vojta-1B", which is gonna pass all of them.
It's like with drug dealing on the streets. If you force dealers off the streets, the deals will take place in private apartments that are much harder to surveil. If you force dealers from large social nets to private social nets or even specialized "underground" apps, the police work is going to be harder, not easier.
An equivalently funny attitude seems to be the "natural language will replace programming languages". Let's see how that one will work out when the hype is over.