Seems like an ideal compression method for LoRa/Meshtastic-style communication. An LLM wouldn't run on an ESP32, but there are several that could run on a raspberry pi.
It's not just natural language that could be compressed this way, either. Code (HTML, JS, etc) could be compressed with the same technique/models. I bet that the same general idea could work for image compression as well, using an image/diffusion model (or perhaps a multimodal model for everything.)
This could lead to an entire internet of content using just a few bits.
The key insight is that the larger the shared context between parties, the more efficient communication can be, as communication tends towards a purely relational construct. The limit of this is two parties that both share the exact same context and inputs, the inputs should produce the same hidden state within both parties and communication is not even necessary because both parties have the same knowledge and state.
That's not new to anyone familiar with compression or information theory, but the novelty here is the LLM itself. It's absolutely plausible that, given an already highly compressed relationally-encoded context like a trained LLM, very few bits could be communicated to communicate very abstract and complex ideas, letting the LLM recontextualize information which has been compressed across several semantic and contextual layers, effectively leveraging a complete (but lossy) history of human knowledge against every single bit of information communicated.
LORA is also pretty slow, like the 'long fast' mode that most meshtastic users use is about a kilobit per second... and presumably a small percentage of the traffic at any time is traffic in channels that you're monitoring.
Probably decoding few tokens per second is fast enough to deliver more goodput than the existing uncompressed usage.
Interesting idea, though I doubt they'd ever offer a reasonable amount for it. But doesn't it also change a sites legal stance if you're now selling your users content/data? I think it would also repel a number of users away from your service
No, because the price they'd offer would be insultingly low. The only way to get a good price is to take them to court for prior IP theft (as NYT and others have done), and get lawyers involved to work out a licensing deal.
What mechanism would make it possible to enforce non-paywalled, non-authenticated access to public web pages? This is a classic "problem of the commons" type of issue.
The AI companies are signing deals with large media and publishing companies to get access to data without the threat of legal action. But nobody is going to voluntarily make deals with millions of personal blogs, vintage car forums, local book clubs, etc. and setup a micro payment system.
Any attempt to force some kind of micro payment or "prove you are not a robot" system will add a lot of friction for actual users and will be easily circumvented. If you are LinkedIn and you can devote a large portion of your R&D budget on this, you can maybe get it to work. But if you're running a blog on stamp collecting, you probably will not.
It was a response to the approach of carefully planning and specifying every aspect of the software up-front.
The problem with that "waterfall" approach is that, in the real world, we often work in fast moving/evolving problem domains, or domains where we discover more as we build. Changing requirements are disastrous to the waterfall approach and can kill projects easily.
If we're to be pragmatic at all, we need some flexibility/adaptability in the process.
Yeah, I understand that it sort of sits in between. But what are those "additional" features compared to a more spartan DEs? More included apps / accessories? More access to the machine's properties (like, more GUIs for various options)? Something else?
There's a lot of nuance in how AI is used vs. the fact that it is used at all. Which specific AI tool is used matters a lot as well.
For example, LLMs can generate a ton of slop, whether in writing form, code, or anything else. But if used thoughtfully, they can lead to higher quality code bases with less tech debt and allow higher quality products to be built faster.
If someone is using the right tools for the right reasons, I see it as a good thing.
Are you finding building MCP integrations to be worth it? We've been using agents (e.g. langchain), which are pretty good at bringing in context and taking actions. Tool results become part of the context --it just works.
Good thing to me (besides being an open spec) is their simplicity, with libraries such as FastMCP you can just bring stuff you already have implemented into Claude (or any MCP client).
It's not just natural language that could be compressed this way, either. Code (HTML, JS, etc) could be compressed with the same technique/models. I bet that the same general idea could work for image compression as well, using an image/diffusion model (or perhaps a multimodal model for everything.)
This could lead to an entire internet of content using just a few bits.
reply