Yeah a lot of people here seem to not understand that PyTorch really does make model definitions that simple, and that has everything you need to resume back-propagation. Not to mention PyTorch itself being open-sourced by Meta.
That said the LLama-license doesn't meet strict definitions of OS, and I bet they have internal tooling for datacenter-scale training that's not represented here.
Source available means you can see the source, but not modify it. This is kinda the opposite, you can modify the model, but you don't see all the details of its creation.
> Source available means you can see the source, but not modify it.
No, it doesn't mean that. To quote the page I linked, emphasis mine,
> Source-available software is software released through a source code distribution model that includes arrangements where the source can be viewed, and in some cases modified, but without necessarily meeting the criteria to be called open-source. The licenses associated with the offerings range from allowing code to be viewed for reference to allowing code to be modified and redistributed for both commercial and non-commercial purposes.
> This is kinda the opposite, you can modify the model, but you don't see all the details of its creation.
> Please read through their "acceptable use" policy before you decide whether this is really in line with open source
I'm not taking a specific posiion on this license. I haven't read it closely. My broad point is simply that open source AI, as a term, cannot practically require the training data be made available.
Open source requires, at the very least, that you can use it for any purpose. This is not the case with Llama.
The Llama license has a lot of restrictions, based on user base size, type of use, etc.
For example you're not allowed to use Llama to train or improve other models.
But it goes much further than that. The government of India can't use Llama because they're too large. Sex workers are not allowed to use Llama due to the acceptable use policy of the license. Then there is also the vague language probibiting discrimination, racism etc.. good luck getting something like that approved by your legal team.
Note that Meta's models are not open source in any interpretation of the term.
* You can't use them for any purpose. For example, the license prohibits using these models to train other models.
* You can't meaningfully modify them given there is almost no information available about the training data, how they were trained, or how the training data was processed.
As such, the model itself is not available under an open source license and the AI does not comply with the "open source AI" definition by OSI.
It's an utter disgrace for Meta to write such a blogpost patting themselves on the back while lying about how open these models are.
You are definitely allowed to train other models with these models, you just have to give credit in the name, per the license:
> If you use the Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” at the beginning of any such AI model name.
> you can't meaningfully modify them given there is almost no information available about the training data, how they were trained, or how the training data was processed.
I was under the impression that you could still fine-tune the models or apply your own RLHF on top of them. My understanding is that the training data would mostly be useful for training the model yourself from scratch (possibly after modifying the training data), which would be extremely expensive and out of reach for most people
From what i understand the training data and careful curation of it is the hard part. Everyone wants training data sets to train their own models instead of producing their own.
Oh, wow, I came here for the same thing. The tone of the post got more and more annoying as it progressed.
Posting quotes from reddit as if they're intellectual discussions was a red flag, but this specific statement was just too much to ignore.
Although I work on Linux almost daily, I'm happy to say it's possible to simply avoid those kinds of people. Fostering a healthy environment where people can discuss and disagree respectfully is incredibly important for volunteer-led projects like Linux distributions.
> On the third hand, I can't deny some amount of schadenfreude looking at Apple having to deal with the same "do the work first, we'll judge if it meets our review standard afterwards" treatment as they do to developers.
That's not really how the process was supposed to work. Apple has had a lot of discussions with the EC over the past year about this law and its implementation.
This is common and the goal is to help the company figure out how to implement it.
However, from the EU side, it seemed as if Apple completely misunderstood what the discussions with the EU were for.
Apple was mainly focused on trying to lobby against all restrictions, not realizing the EU already made up their mind. The EU was trying to prepare Apple, but instead Apple dragged their feet, remained in denial, and were then suddenly surprised that yes, indeed, they really can't lobby their way out of compliance.
