I don't think you understand the scale of large creative projects like games. The current (only medium sized) Unreal Engine project I'm working on has about 300k files in the head alone, and a fresh sync for the stuff only required to build/cook the game is about 350GB. If you add the raw content from things like Maya, substance Painter, brush, etc, it is well in excess of 1TB.
Vanilla git just does not scale to this. Microsoft git and Scalar get much closer, but require a bunch more setup and are a giant foot gun if someone tries to use the repo with vanilla git. Add to that the lack of permissions control and remote management, and it is just not a good fit for an industry where say 75% of the contributors are non technical.
Git is a fantastic tool and works for the vast majority of software projects, but there are non trivial amount of projects out there that it just doesn't suit.
Perforce, as horrible as it is, is the game and VFX industry default for a reason - think of it this way, given how notoriously cost driven and penny pinching games companies are, don't you think they'd be using a free/cheaper alternative in hosted Git if they could? But they don't. This is why.
Good luck to the Diversion team, more competition in this part of the market can only be a good thing.
Despite all the questions I have about storing un-versioned large media files in Git instead of EC2 and your lack of good management about modularizing the code base and lack of build tooling....thats the use case I specifically stated in my post.
The monstrosity that is Diversion could have great benefits for niche use cases like yours. i.e. large media file Version Control for teams who aren't knowledgeable about how to manage large code bases.
If for some reason you want to pay an additional nine person team with a huge infrastructure instead of just using EC2 and a build script..feel free.
EC2 and a build script? Are you implying that the parents 1TB of assets are "un-versioned large media"? If so, I'm sorry you are being very dismissive about something you don't fully understand.
git can handle large repositories there's no reason you should have a one terabyte repository unless youre Google or something with the money to write your own VCS.
but if you want to pay for a nine developer team with a massive cloud infrastructure for something like version control.... go for it!!!
I'm going to check back a year from now and see if products still exists
sigh So, in GameDev, art assets are as important to the final game as code. And is, just as important to be version controlled as code.
And sure, you can split it up between many different repositories, and call it modular But you're just adding complexity, and not really solving the core problem which is that a single version of an art assets can possibly be > 1G, and be a poor format to be stored delta-encoded, so each revision can possibly be nearly a full duplicate. (Yes, we can talk about the file formats being a poor fit for VCS, which is true, but GameDevs are just trying to make their product) And again, it's reallyimportant for game assets to be version controlled. And maybe Git isn't the best place for the assets to exist in version control, but we are in a thread about Diversion and not Git.
Now, don't get me wrong, I'm uncertain that Diversion will exist in a year when you check-back, But I can guarantee you that Perforce will, and if Unity doesn't continue it's trajectory of destroying everything it touches, Plastic should continue to exist also.
I don't even completely disagree, Diversion, P4, and Plastic appear to be a bad fit for code in comparison to git, and even in our studio we've taken a modular approach (gasp) and store our code in git, while using a better suited VCS for assets. But it doesn't change the fact that you are being very dismissive for something that you very clearly know nothing about, and clearly have far too much ego to even be interested in a legitimate dialog about.
It's not rocket science to know that if you have a 1 TB git repo filled with massive binary files that all need to be version controlled, youre doing something VERY wrong or you are using the wrong tool.
Literally the very first result on google search says almost every major game studio on earth uses perforce due to this specific problem because perforce allows you to check out binaries on a file to file basis and mark them with a change list. This appears to be a solved problem with numerous solutions.
Theres also git-annex mentioned in the thread and a number of other options.
I love that you're saying I have a big ego, but you're doing exactly one of the things I'm suggesting in your studio. lol
This Diversion seems doomed if they're competing with git, but if they're competing with perforce, that's a different story. They're probably still doomed, but at least their product fit makes sense.
> Perforce, as horrible as it is, is the game and VFX industry default for a reason
...
> or you are using the wrong tool.
You were literally responding to a comment talking about how git isn't suitable for this problem, and seemed to be suggesting that ec2 and a build script should be sufficient...
Edit: on a second full read, it's really unclear actually what your point was in the first place, And perhaps you were in agreement with the original poster and me all along, and just being confrontational and hostile for no reason?
Vanilla git just does not scale to this. Microsoft git and Scalar get much closer, but require a bunch more setup and are a giant foot gun if someone tries to use the repo with vanilla git. Add to that the lack of permissions control and remote management, and it is just not a good fit for an industry where say 75% of the contributors are non technical.
Git is a fantastic tool and works for the vast majority of software projects, but there are non trivial amount of projects out there that it just doesn't suit.
Perforce, as horrible as it is, is the game and VFX industry default for a reason - think of it this way, given how notoriously cost driven and penny pinching games companies are, don't you think they'd be using a free/cheaper alternative in hosted Git if they could? But they don't. This is why.
Good luck to the Diversion team, more competition in this part of the market can only be a good thing.