I think the weakness of this argument is that “domestic American drones” will just be using parts, or entire drones, made in China
“Ukraine are producing two-and-a-half drones a day now.“
What is that supposed to mean? A drone has a lot of parts. I can make 2 1/2 drones a day if I have the parts.
It seems like there’s a misunderstanding as why this happened. They’ve been baking this model for months. long before deep seek came out with fundamental new ways of distilling models. and even given that it’s not great it’s its large form, they’re going to distil from this going forward .. so it likely makes sense for them to periodically train these very large models as a basis.
I think this framing isn't quite right either. DeepSeek's R1 isn't very different from what OpenAI has already been doing with o1 (and that other groups have been doing as well). As for distilling - the R1 "distilled" models they released aren't even proper (logit) distillations, but just SFTs, not fundamentally new at all. But it's great that they published their full recipes and it's also great to see that it's effective. In fact we've seen now with LIMO, s1/s1.1, that even as few as 1K reasoning traces can get most LLMs to near SOTA math benchmarks. This mirrors the "Alpaca" moment in a lot of ways (and you could even directly mirror say LIMO w/ LIMA).
I think the main takeaway of GPT4.5 (Orion) is that it basically gives a perspective to all the "hit a wall" talk from the end of last year. Here we have a model that has been trained on by many accounts 10-100X the compute of GPT4, is likely several times larger in parameter count, but is only... subtly better, certainly not super-intelligent. I've been playing around w/ it a lot the past few days, both with several million tokens worth of non-standard benchmarks and talking to it and it is better than previous GPTs (in particular, it makes a big jump in humor), but I think it's clear that the "easy" gains in the near future are going to be figuring out how as many domains as possible can be approximately verified/RL'd.
As for the release? I suppose they could just have kept it internally for distillation/knowledge transfer, so I'm actually happy that they released it, even if it ends up not being a really "useful" model.
You could only create a land value tax for land that is already owned outright if there’s a mortgage for that land, then the owner will default and society will colander.
Because they owe the bank $1 million for property that suddenly worked much less than that so they’re gonna default on their debt and then their bank is going to sell it and attempt to recover the loss
“Ukraine are producing two-and-a-half drones a day now.“ What is that supposed to mean? A drone has a lot of parts. I can make 2 1/2 drones a day if I have the parts.
reply