The cat is out of the bag. This is the landscape now, r1 was made in a post-o1 world. Now other models can distill r1 and so on.
I don’t buy the argument that distilling from o1 undermines deep seek’s claims around expense at all. Just as open AI used the tools ‘available to them’ to train their models (eg everyone else’ data), r1 is using today’s tools.
Does open AI really have a moral or ethical high ground here?
Plus, it suggests OpenAI never had much of a moat.
Even if they win the legal case, it means weights can be inferred and improved upon simply by using the output that is also your core value add (e.g. the very output you need to sell to the world).
Their moat is about as strong as KFC's eleven herbs and spices. Maybe less...
Agree 100%, this was also bound to happen eventually, OpenAI could have just remained more "open" from the beginning and embraced the inevitable commoditization of these models. What did delaying this buy them?
It potentially cost the whole field in terms of innovation. For OpenAI specifically, they now need to scramble to come up with a differentiated business model that makes sense in the new landscape and can justify their valuation. OpenAI’s valuation is based on being the dominant AI company.
I think you misread my comment if you think my feelings are somehow hurt here.
> It potentially cost the whole field in terms of innovation
I don't see how, and you're not explaining it. If the models had been public this whole time, then... they would be protected against people publishing derivative models?
> I think you misread my comment if you think my feelings are somehow hurt here.
Not you, but most HNers got emotionally attached to their promise of openness, like they were owed some personal stake in the matter.
> I don't see how, and you're not explaining it. If the models had been public this whole time, then... they would be protected against people publishing derivative models?
Are you suggesting that if OpenAI published their models, they would still want to prevent derivative models? You take the "I wish OpenAI was actually open" and add your own restriction?
Or do you mean that them publishing their models and research openly would not have increased innovation? Because that's quite a claim, and you're the one who has to explain your thinking.
I am not in the field, but my understanding is that ever since the PaLM paper, research has mostly been kept from the public. OpenAI's money making has been a catalyst for that right? Would love some more insight.
I don’t think there is any ethical issue here, but I don’t think it’s good for the industry to remove all incentives for companies to spend lots of money solving hard, novel problems.
Why would anyone go through the effort of training the next groundbreaking model if they know they can just wait for someone else to do it and leverage that work?
> Why would anyone go through the effort of training the next groundbreaking model if they know they can just wait for someone else to do it and leverage that work?
Why would anyone write, work or research anything if they know it would be consumed by AI and sold on a $xx/month subscription?
I don’t buy the argument that distilling from o1 undermines deep seek’s claims around expense at all. Just as open AI used the tools ‘available to them’ to train their models (eg everyone else’ data), r1 is using today’s tools.
Does open AI really have a moral or ethical high ground here?