> Open models keep getting bigger, but smaller open models also keep getting smarter.
Not only bigger, but smarter and more capable. From what I can tell, smaller are only getting smarter in very specific areas. There is a subtle difference there, that is extremely important.
> What I can do with an 8b used to require a 32b.
What exactly do you do with an 8b? I usually ask this question and either get no response or it is something that doesn't generate anything of value. So, please surprise me.
Smaller models have improved in every regard, not just domain specific challenges. You can see this in the benchmarks or by just testing one yourself side by side with an older release.
I'm using it at work in multiple data classification and redaction pipelines. It's replaced tedious manual labor and opened that staff up to focus on the parts of the work that requires their human intuition, rather than spending time on tedium.
Presumably part of the reason you don't get a response is your hostile approach to asking.
They have not improved in the one way that I use them today, which is coding. It is like talking to a halfwit, and I always end back up with codex.
Asking a direct question is not hostile. I'm glad to hear you've found a use for a smaller model that generates value. It gives me hope for the future, that said, I think we are still a long ways away from needing HPC in DC's.
Different people have different requirements. I use my local model for coding regularly. It doesn't help the conversation that things like quant, config, harness, and even prompting are equally as important to your outcomes as the choice of model.
I pay $200/month and I generate 10000000x more value of it than that. As you're saying, the actual barrier isn't just the model, it is the set up. As long as the walls are too high, $200 is going to dominate.
The internet was great before AOL, but the simplification is what brought the masses to it.
Not only bigger, but smarter and more capable. From what I can tell, smaller are only getting smarter in very specific areas. There is a subtle difference there, that is extremely important.
> What I can do with an 8b used to require a 32b.
What exactly do you do with an 8b? I usually ask this question and either get no response or it is something that doesn't generate anything of value. So, please surprise me.