
Ask HN: How do game companies keep their files organized for a large-scale game? - mfcecilia
What are the best practices for naming? What is the hierarchy for folder organization? How do you name assets built in Maya versus those that are built in the game engine? What about code? Is it saved with its corresponding asset? Or in a dedicated code folder? Is there a template for this?
I&#x27;ve been working on a team for a year developing for VR and web. When I first arrived on the scene, I saw that there was absolutely no organization in the 3D models, textures, scripts, etc. It actually took a full 2 weeks to figure out what pieces were being used in the current working build. Now I am able to navigate and do my work no problem. However, we just added two new members to the team and they spent an hour talking about how disappointed they were that they were expected to just know what to do after having been given that jumbled mess. I&#x27;ve been looking around online for examples of folder organization and naming conventions for the variety of files used in a game, but with no avail. I was playing Skyrim the other day and all I could think about was how they named all the NPC&#x27;s locally, where they saved all their textures, animations, ai scripts, etc. Even for all their environment assets. Not all rocks are the same, so I am pretty sure they didn&#x27;t name it &quot;rock01&quot; &quot;rock02&quot; etc. Surely there&#x27;s a code or a method for naming everything as well as a folder hierarchy. I know they are unique for every game, but I think it would help to just see a few templates&#x2F;examples.
======
jtolmar
The best way to do it that I've found is to arrange a folder hierarchy of
"where" rather than "what." So there's no one place with all the game's rock
models, but rather firelands/towerofeternalflame/courtyard/rock01.3ds and so
forth. "Where" doesn't have to mean a literal location; it could be
playerbases/horde/rock.3ds or something. Though for most games, everything not
tied to the player really can just be its literal location. It's crucial that
these directories have a mix of models, textures, and whatever else you need,
instead of a mass of parallel structures.

If there are automated tools, they should run on the entire asset directory
and dump to a new directory in whatever hierarchy makes sense to the tool.
When that turns out to be slow, check "file modified" timestamps in the tool
instead of dumping required knowledge on the artist.

Getting artists to actually use source code control instead of having personal
folders full of things like rock01_retouched_new_final_final.3ds is a bit of a
pipedream, but you can at least put the asset directory under source control
and make that the only way to see things in the engine.

~~~
solipsism
_Getting artists to actually use source code control instead of having
personal folders full of things like rock01_retouched_new_final_final.3ds is a
bit of a pipedream_

BS. Just hire better artists. It sounds like hand-waving, but I've worked with
both kinds of artists. You're building games, so if you focus on only hiring
artists based on their portfolio you might end up with a bunch of stubborn
artists who need to be hand-held and worked around constantly. If instead you
focus on hiring artists who are good at art AND who can demonstrate a
propensity or desire to learn new workflows and and to be flexible, then
you'll end up with a bunch of team players.

This is a common complaint about artists, but it has more to do with culture
and expectations than anything else.

~~~
blablablame
Agreed. While a few artist keep a few _01 _02 files around, mostly as
backup/undo files, my last few years in the industry, they all used some kind
of version control (maybe not git, but perforce and svn with their visual
tools)

~~~
dev360
Sounds like perforce is common when there are lots of binary files - is this
mostly due to storage concerns?

~~~
vvanders
Bunch of factors:

    
    
      - Really good binary performance(light years ahead of git).
      - Proper locking for files that can't be merged(basically everything in the art dir).
      - Proxy support for co-located teams.
    

Game repos can easily approach multiple TB just for the art assets. In that
area Perforce is once of the few VCS that scales appropriately.

~~~
malkia
+1. Also easy graphical interface - P4Win/P4V, and easy to understand the
system of changelists. It also helps that older changelists mean things
submitted in the past, newer in the future - e.g. in contrast with
digest/fingerprint schemes like other systems use.

~~~
vvanders
Yup forgot about that but a good point as well.

------
egypturnash
Not quite games but back when I was doing Flash cartoons in the 90s we had an
elaborate scheme for naming art assets. It was something like:

3-4-dd-rhand-07 - episode 3, scene 4, dirty dog, right hand, from keyframe #7

(there were naming conventions for background elements too but they were
somewhat looser IIRC)

We had a cheat sheet for this with standard abbreviations for all the
recurring characters, and a list of body part names. Everyone had a copy stuck
to the wall or floating around their desk. I think that was an important part
of what made it work. _Don 't_ expect new hires to pick it up by looking at
the results of the naming scheme; codify it into a document you hand to them
as part of orientation.

(We also accessed a lot of this stuff visually; the file handed to an animator
would have all of the keyframe drawings in each scene laid out in sequence,
with the raw art separated out into individual parts and turned into named
symbols right there on the working canvas.)

All of this was something we hashed out over time; early stuff had a horrible
mishmash of naming schemes. I'd advocate having a discussion about
Organization as soon as possible, agreeing on a standard and sticking with it
for new assets. Get the old stuff reorganized when you have a chance, maybe
throw that task to an intern if you have one.

------
corysama

        Code
        	Game
        		UI
        		Gameplay
        		GFX
        		Physics
        	Editor
        	ThirdParty
        SourceAssets
        	Levels
        		Fire
        		Ice
        		Swamp
        		Volcano
        		Sewer
        	Characters
        		Player
        			Models
        				playerInital.fbx
        				playerAwesome.fbx
        				playerGodlike.fbx
        			Animations
        			Materials
        		Merchant
        		Thug
        		ThugFat
        		ThugTall
        	Objects
        		Urban
        		Indoor
        		Outdoor
        	UI
        		Menus
        			Boot
        			Main
        			Options
        		Icons
        ProcessedAssets
        	Pak1
        		(UI stuff)
        	Pak2
        		(Level 1 stuff)
    

Everything goes in source control. Directories are organized to group together
similar work. Names are a mixture of searchability for the artists and
conventions needed by the code.

------
cableshaft
They don't. At least artists are really bad at it when naming their files.
I've had to constantly rename assets received from artists to give them some
sort of consistency, even after telling them how to name it.

But yeah, Perforce version control is pretty important. It's almost designed
to help manage assets, especially large ones. And it's free for under 20
users, I think.

Beyond that, you might want to include the level, screen, and/or date for the
assets in your naming structure. Pretty much every company, and sometimes
every project, uses a different structure, though.

~~~
cpeterso
I worked on a commercial MMORPG project for a while. The game assets were
hosted on an unversioned asset server instead of in Perforce. The same asset
server was shared by every branch and release of the game client, which made
backward and forward compatibility a major hassle. None of the artists
understood what the big deal was. I feel like I should write a post-mortem of
that project some day.. :)

Kotaku has a great post-mortem on the development of Bungie's _Destiny_ game,
including fantastic horror stories about the design tools:

    
    
      “Let’s say a designer wants to go in and move a resource node two inches,” said
      one person familiar with the engine. “They go into the editor. First they have 
      to load their map overnight. It takes eight hours to input their map overnight. 
      They get [into the office] in the morning. If their importer didn’t fail, they 
      open the map. It takes about 20 minutes to open. They go in and they move that 
      node two feet. And then they’d do a 15-20 minute compile. Just to do a 
      half-second change.”
    

[http://kotaku.com/the-messy-true-story-behind-the-making-
of-...](http://kotaku.com/the-messy-true-story-behind-the-making-of-
destiny-1737556731)

~~~
vvanders
I don't know how much I trust that source. Maybe they were talking about
lightmap baking but any system that incurs more than a 2 minute cost for
testing a tweak is going to kill any chance build a meaningful game.

Game design is built upon rapid iteration and anything in the way of that will
kill your title.

~~~
jhasse
Have you played Destiny? :D

------
groby_b
First, you need to distinguish between in-game assets, and the source models
in Maya, Photoshop, etc.

For the latter, most large games use an asset management system. Either
homebrew, or something like Alienbrain. (Well, people used Alienbrain when I
left. Not sure what's hip these days)

For in-game, there's usually a fairly strict organizational scheme. It depends
on the game, but usually folders are about "location" \- which map, which part
of the world, which episode, etc. Asset names indicate a name, and various
other bits like left/right hand hold, color, or whatever matters for that
asset.

It's usually the lead artists job to come up with said scheme, unless you have
a tech lead for the asset pipeline.

If you look for assets, you do a search in the asset management system, which
also contains the in-game name and location.

For smaller games, you make do by manually organizing, but as things grow,
you'll start hand-rolling tools.

------
solipsism
Organizing assets is hard because the dependencies are complex and change
often. We solved this at a previous company using in-game editor tools
(material editors, level editors, character editors, AI script editors, etc)
that all interfaced with a database back-end. The database didn't store the
files, those were stored in Perforce (which the tools interfaced with as
well). The database just stored metadata for each version of each asset along
with the relational links between assets. So, for example, a particular 3D
model referenced a particular material (storing just its id/foreign-key in the
DB), which in turn referenced a set of textures and perhaps a shader.

An asset browser allowed you to perform queries (e.g. show me all the
characters from level X which reference materials which reference texture Y).
We had tags/labels, comments, everything you could want.

When you do it like this your folder organization becomes irrelevant. You can
store everything in a single directory with a GUID for a filename if you want.
Or store all textures in one place and all 3d models in another. The down side
is it's quite opaque, everything must be done within your toolset, so you need
great tool support.

------
bsamuels
It's hard. I personally believe that designing large scale video games is one
of the most complicated kinds of programming projects out there - so don't try
to reinvent the wheel.

Like daodedickinson recommends; you want to look at how AAA game designers
design their systems, then imitate them. Your best avenue to figuring out how
games are designed is by understanding how game modding is carried out in AAA
games. In many games, almost all of the content and mechanics within the game
are defined using the same scripting interface that mods are created through.

I recommend using games like Skyrim, Morrowind, Mount and Blade, and The
Witcher (i forgot which one shipped with all the game's scripts in the install
folder, might have been 2).

------
shorodei
> Not all rocks are the same, so I am pretty sure they didn't name it "rock01"
> "rock02"

I wouldn't be so sure...

~~~
cableshaft
Also 'rock01_final', 'rock01_final_final', 'rock01_new_colors1',
'rock01_new_colors2', 'shirt_long_AC07D4' (different item textures using hex
color code as an id number), 'rock01_george' (for george's version), etc. All
the bad naming you can imagine I've seen with asset naming.

------
fitzwatermellow
Does anyone else see the similarity between designing Redis keys and game art
asset filenames?

Being able to mine as much metadata from a file name and path is a plus. As is
having a logical naming convention that can be queried from the string
programmatically. Not to mention fast lookup and asset name generation merely
from string concatenation...

This is a good problem to have, mfcecilia! You are on the path to making great
games. Its stuff like this that gives you the confidence to say my solution is
just as good as anyone else's and I am just going to do it and learn from my
mistakes. Time pressure is actually your best ally. If you want your game to
be playable by the end of the decade you will solve this challenge and level
up to the next big one. Such as figuring out how to organize your assets in a
single compressible pak that is laid out to support fast and efficient
patching via binary diffs!

Good luck! And if you are still stuck check out the Unity and Unreal engine
docs and forums for further tips on pipeline production. And of course, post a
link when you build a playable alpha ;)

~~~
tomphoolery
> Does anyone else see the similarity between designing Redis keys and game
> art asset filenames?

I always thought Redis keys use filesystem-like names just for ease of use.
Pretty sure you can name them whatever you want though, using colons or dots
as separators if you don't want to use slashes. But yeah, I do see your point.
Think it's just the most straightforward way to name something honestly.

------
chipsy
You'll never get around the source of the problem, which is that it's not the
priority of the artist to figure out how to name and organize things. It's the
same as bothering developers to "update the wiki". Pick a convention and
structure the culture around that by providing a spreadsheet of assets or an
equivalent document. Maybe the artists "own" their assets, so you lead with
their username. Maybe you're building the environments collaboratively, so you
lead with the environment first. Maybe you envision the project being a long-
term thing like an MMO, so you start with the year and month. These are things
that a good producer will keep their eye on, because they reflect how the team
is communicating.

The most important thing, from the technical standpoint, is that your asset
pipeline is robust and can handle weird things. The engine shouldn't consume
files haphazardly, it should deal with "built" assets, where you can trace
things from the in-game content back to the source file. A configuration layer
usually appears to give an in-engine ID to a loaded asset, because files and
assets do not map 1-to-1. You can choose to fight the organization battle in
that layer, rather than in the file system. When you have a slick IDE like
Unity, this is all built into the GUI, but a custom engine can just use a
batch process to do the same.

Version control is useful - not a DVCS, because it doesn't scale to binary
assets, but SVN or Perforce. Artists will groan about waiting around for the
VCS to resolve things and the inevitable problems with locking, renaming, and
deleting stuff, but their lives will be better off overall if it's in their
workflow loop. It's "10 hours waiting on SVN Updates" vs "30 hours debugging
project history".

Here's an article on "import" vs "export" based pipelines, where the thing
being "imported" to the engine is a common, standardized format that every
tool can export - rather than one proprietary tool the team happens to use:

[http://people.cs.aau.dk/~bt/GameProgrammingE09/Pipeline.pdf](http://people.cs.aau.dk/~bt/GameProgrammingE09/Pipeline.pdf)

------
vvanders
Perforce + Diligent lead tech artist.

Also most of what you see in the final game output is packaged and minified to
a high degree in order to improve performance/memory/etc.

Usually the lead tech artist just sets a standard and everyone conforms to
that standard. Art assets(.psd, .ma, etc) are usually stored in a separate
section than in-game assets.

------
daodedickinson
Skyrim's dev tools (the mod kit) should give you a hint. At least I know with
Morrowind you could get a good idea of their naming conventions.

------
thenomad
You can get a snapshot of how big game studios organise their art assets by
using various modding or decompilation tools to have a look inside their
package files.

For example, you can use MPQ Explorer to look inside the package files of
World of Warcraft, and see how they organise their tens of thousands of
assets.

Generally, as others have said, it's by "where" rather than "what", or
sometimes "what"/"where".

Beyond that - Perforce is the usual asset control system for these things. It
offers the ability to tag files, which makes searching easier. When you're
dealing with a huge pile of art, searching is basically the only way to go, so
good tagging is very important.

(Source: I'm not a game designer, but I've worked with the complete art
libraries of multiple triple-A games over the last two decades creating
Machinima films, either working directly with the studio or on a modding
basis. I know far more about the internal file structure of WoW, for example,
than is probably healthy.)

~~~
TheCams
I think there's a difference between the files layout on a disc/install folder
and a repository. The layout on the disk can be optimized for loading (even
duplicating the assets sometimes). OP is asking for a repository hierarchy I
think, since he's talking about sources files too.

~~~
thenomad
Oh, yeah, there definitely is.

OP, my bad if you were specifically looking for repo hierarchies. That's not
the side of things I'm usually involved in :)

------
DonHopkins
This may seem obvious, but some people just don't get it, and it can be easily
taught: I like to purposefully and consistently use "big-endian" names for
assets, classes, functions, variables, events, files, directory structures,
etc.

The desired result is that related names are convenient grouped together in an
alphabetical sort.

A rule of thumb is: the most important part is at the beginning, and most
specific part is at the end.

But what is convenient, important or specific is context sensitive and
debatable, since there are many ways to name the same thing in different kinds
of hierarchies.

Sometimes it's ambiguous what's most important, like when you're classifying
things along two or more dimensions, like x y and time.

Should it be (MouseXCurrent, MouseXLast, MouseYCurrent, MouseYLast) or
(MouseCurrentX, MouseCurrentY, MouseLastX, MouseLastY)?

In that case I'd put the most tightly bound dimension last (x and y, which are
usually used together as a 2d vector, so keep them adjacent).

------
wink
id software's were usually not too bad:

Quake's pak0.pak:
[https://quakewiki.org/wiki/pak0.pak](https://quakewiki.org/wiki/pak0.pak)

Quake 3 Arena's pak0.pk3:
[http://openarena.wikia.com/wiki/Pak0.pk3](http://openarena.wikia.com/wiki/Pak0.pk3)

~~~
striking
But that's because John Carmack et al. were geniuses that were ahead of their
time, who put effort into making fun out of programming games.

It became especially important as people started buying Quake's engine and
modding it to be different games entirely.

[https://en.wikipedia.org/wiki/Game_engine#/media/File:Quake_...](https://en.wikipedia.org/wiki/Game_engine#/media/File:Quake_-
_family_tree_2_Simplified.svg) for example

------
liamkf
From what I've seen, and as you guessed, each game handles it differently and
no matter what conventions you have, someone will be grumpy. But if you're
just looking for inspiration, I would look at the content for
UE4/UnrealTournament which is about as good as it gets for 'industry
standard'. It's free to get access to and you should be able to get some ideas
by poking around.

------
bitwize
The smart ones have asset management systems, big databases that track what
gets used where how.

The Scheme hacker Shiro Kawai has, as one of his claims to fame, having
written an asset management system for Squaresoft (now Square Enix) -- not for
any of their games but for the movie _Final Fantasy: The Spirits Within_.

------
gavanwoolery
What you are looking at is a classic problem in all of programming: making
data machine-readable vs human-readable. Relying on file names for any sort of
data is bad practice, IMO. This info should be contained within metadata
within the file. So in the file system "object_0001.obj" might contain
metadata like "label" "tags" etc - that said you may want two versions of the
data - let artists organize their files according to a human-friendly system,
but when they are imported into the engine the appropriate metadata must be
filled out and the file gets automatically saved to a specific location
determined by the engine or editor.

~~~
gravypod
Doesn't that make it pain when debugging or heading down to the directory
level? I think rememberable and commonsense file naming makes sense.

~~~
gavanwoolery
It totally depends on the scope of your project, IMO. If you are a one man
team making a small indie game, then it probably makes the most sense just to
use a simple file system with the naming convention you suggest. The reason I
would avoid such a thing in a bigger project is that inevitably such a system
will break or become complex if it is not managed by a machine (even I, as a
one man team, have hit this problem and I had to go in and rename hundreds of
files for a simple change). A simple analogy is an SQL database - you could
depend on naming stuff precisely by certain key values to make sense of it,
but it becomes more powerful if you provide it with some simple data for each
entry and let the computer make sense of it (select all objects with tag
"static" and "rock" and "desert" for example). A given convention will force
organization inherently. Every time you need to update an object you simply
look it up in the editor/engine and reload it from a file you select.

------
CM30
Well, in quite a few cases they don't. For example, I've seen the file
structures for quite a few 3DS games via the modding community, and the files
are often just named complete gibberish. Like 'file1music.brr' to
'file2639music.brr'.

But from games which do have somewhat logical structures, it's usually that
the files are seperated by purpose. So the music and sound effects in one
folder, the textures in another, some level files in another folder, etc. And
the names would be somewhat descriptive of the content, like
'course_desert_1.lvl' If you're lucky, this is also backed up in a git
repository somewhere and has version control.

------
moron4hire
There is a cultural problem in game development where they think the
performance imperative makes them the epitome of software development. They do
not understand how anyone in any other field of software development could
ever touch them in terms of pure, unadulterated, programming prowess. As a
result, tools, lessons, and best-practices learned in other fields are nearly
completely ignored. Not-invented-here syndrome is strong.

There is nothing particularly special about game development. Yes, there is an
emphasis on real-time operation that changes the specific design of
applications. But there is no reason to believe that should have any impact on
project _management_.

~~~
babuskov
Well, there is one specific thing not common to other types of programming:
large amount of small-to-large binary files to handle. Most of those files are
in proprietary formats which makes it hard to automate stuff. I believe OP was
asking exactly about managing these files.

I have to agree with the rest you said. Discussions I had with game devs show
that most of them are clueless about databases, web technologies (aside from
HTML5 JavaScript) and tend to invent their own sub-optimal solutions. What's
the real problem is that some of them share those and get adopted as standard
in the gamedev community - because they don't know any better.

------
sly010
They mostly don't:
[https://www.facebook.com/10153270300786451](https://www.facebook.com/10153270300786451)

------
malkia
Assuming Perforce, or perforce like system. Couple of things.

//depot/main/game - all your code + exported (but not yet converted) data
lives there. You can give this to QA, Build manager, scripter, coder and they
can build a level, compile the game, tools, etc. if needs so. By exported here
is what I mean: \- Exported textures, models, sound, etc data ready to be
further on converted to platform specific formats. The original data was kept
in //depot/source

Keep several department oriented //depot/source p4 views. Each specific
department would get mapped this in their view, others would not.

//depot/source/sound - For example big huge 24-bit sound files + whatever
other extras ProTools or other software writes. Suitable for sound folks to
keep their project settings, experiments, settings, but not suitable for
everyone else to see.

//depot/source/textures - Your big ass huge PSD, or whatever else source files
- containing all layers that you've built your textures, or whatever else a
texture/material department might see fit (there might be more than one -
character, environment, etc. departments with different organizations). Not
suitable for the main team, great for whoever works on textures.

//depot/source/animations - For example big huge FBX MotionBuilder or
something else files. Suitable for animation people to edit, but not suitable
for the game tools to read them and convert them every time.
//depot/source/models - etc. - your big ass again .3DS, .MA/.MB, even .FBX
files. Great for modelers, not great for everyone.

//depot/source/something-else

You might even have external people working for you, you don't put their stuff
directly with your stuff (legal maters, payment, review process, etc.)

//depot/outsource/BlamStudio/textures

//depot/outsource/BlamStudio/models

//depot/outsource/CoreySounds/sound

//depot/outsource/OrchestraYou/music

You also don't want them to have access to everything you do, and you need to
establish way to review their assets and how to communicate work - what needs
to be done, requirments, etc.

.....................

So a typical game developer would only need

//depot/main/game

some studios would even submit back converted data, for example:

//depot/build/game/data/ps4

//depot/build/game/data/xbone

//depot/build/game/data/wiiu

//depot/build/game/data/pc

others prefer to convert while building. For example all huge ass PSD files
with lots of layers were probably exported as .TIFF, .PNG, even .JPG for
insanely big backdrops, and from then on these would get converted to swizzled
optimized textures - for example on all iPhone/iPads you get PowerVR chip, and
it has PVRTC texture compression, unlike ETC, DXT for PC.

It all depends also on your team.

So a WiiU only programmer, might just sync

# Not ideal locations, but something that might work:

//depot/main/game/... c:/blah/game/

//depot/build/game/data/wiiu/... c:/blah/game/build/data/wiiu

------
1812Overture
Desktop/Bob's Files/New Project/Misc/Stuff for Jim/Important First Draft FINAL
v2(1).doc

