

Doom 3 source released - padenot
https://github.com/TTimo/doom3.gpl

======
dgallagher

        $ sudo perl cloc-1.55.pl --unicode doom3.gpl/
            2014 text files.
            1907 unique files.                                          
             476 files ignored.
        
        http://cloc.sourceforge.net v 1.55  T=40.0 s (36.6 files/s, 22123.3 lines/s)
        -------------------------------------------------------------------------------
        Language                     files          blank        comment           code
        -------------------------------------------------------------------------------
        C++                            517          87051         113094         366423
        C/C++ Header                   615          29640          26891         110991
        C                              170          11407          15566          53520
        Bourne Shell                    36           4529           5476          33717
        m4                              10           1079            232           9025
        HTML                            55            391             76           4142
        Objective C++                    6            709            654           2605
        Perl                            10            523            411           2380
        yacc                             1             95             97            912
        Python                          10            108            182            895
        Objective C                      1            145             20            768
        make                            22            160            253            579
        DOS Batch                        5              0              0             61
        Teamcenter def                   4              3              0             51
        Lisp                             1              5             20             25
        awk                              1              2              1             17
        -------------------------------------------------------------------------------
        SUM:                          1464         135847         162973         586111
        -------------------------------------------------------------------------------

~~~
JonnieCache
Question 1: Why did you run cloc as root?

Question 2: Why does Doom 3 involve 33,717 lines of bash? That's a lot of
bash. I realise it's used heavily for build scripts, but thirty three thousand
lines seems excessive.

~~~
dgallagher
_Question 1: Why did you run cloc as root?_

Hmmm, because of a really bad habit I need to break. ;)

~~~
frobozz
If your habit is what I think it is, I have the opposite habit - I never use
sudo until it tells me I've tried to do something I'm not allowed to do - even
if I know it's not going to let me. Then I use "sudo !!"

~~~
DavidMcLaughlin
> Then I use "sudo !!"

Wow. For six years I've been doing "UP + CTRL^A + sudo + SPACE" instead. This
is much better.

~~~
tome
Off-topic:

I use Ctrl-P Ctrl-A sudo <space> which is exactly the same number of
keystrokes as "sudo !!" and a much more general solution in my opinion.

------
metashurick
I wonder why the depth fail shadow rendering algorithm had been explicitly
excluded from the open source release. It's been around for years now, after
all.

~~~
TeHCrAzY
There was a patent issue; this delayed the release slightly whilst it was
removed.

~~~
daeken
I give it a week before someone forks and adds it back in, just to spite
Creative.

------
afhof
I was under the impression that most large projects end up having to write a
custom memory allocator, but after a few random probes into the source I
didn't see anything that stood out. Did I just miss it, or did they use the
built in one?

~~~
ars
Why do you have that impression?

That makes very little sense to do - every single project? Don't you think
someone would have released a library to implement the "new and improved
allocator" if everyone needs to use it?

~~~
vilya
That has been borne out by my experience so far too.

There have been several successful efforts at writing better performing
general-purpose allocators:

    
    
        * TCMalloc (http://goog-perftools.sourceforge.net/doc/tcmalloc.html)
        * jemalloc (http://www.canonware.com/jemalloc/)
    

There's a great Facebook Engineering blog post about jemalloc which describes
why a fast allocator is needed & why the standard malloc is not always
sufficient: [https://www.facebook.com/notes/facebook-
engineering/scalable...](https://www.facebook.com/notes/facebook-
engineering/scalable-memory-allocation-using-jemalloc/480222803919)

~~~
forrestthewoods
I've worked with several projects where a TCMalloc based allocator (threaded)
was used to great success. A simple switch to DLMalloc (non-threaded) can also
provide a significant performance boost.

Moderate usage of STL can do nasty, nasty things to memory performance. In one
case a game I worked on ran at 5-10fps and a simple, fast change to DLMalloc
instantly took perf to 60fps.

~~~
caf
It's worth noting that at least on some platforms the system malloc _is_
dlmalloc (presuming that DL is Doug Lea).

~~~
iso8859-1
Which platforms?

~~~
caf
glibc's malloc has long been based on Doug Lea's malloc.

------
metachris
The source for the physics engine
([https://github.com/TTimo/doom3.gpl/tree/master/neo/game/phys...](https://github.com/TTimo/doom3.gpl/tree/master/neo/game/physics))
looks rather interesting. Might contain valuable additions for other physics
engines to pick up.

~~~
barrettcolin
I'm not sure where I picked up the info, and I haven't looked at the code to
confirm it, but I believe the physics engine implementation is tied tightly to
both the collision detection system and the entity update, both of which are
quite specific to Doom (in particular, each entity steps physics
independently, rather than a whole simulation advancing at once as is common
in 'pluggable' physics engines like ODE, Havok etc).

Also, and this is subjective I suppose, I don't remember Doom 3 doing anything
that 'felt' or looked better than even what the contemporaneous engines did
(such as the version of Havok used in Half-Life 2).

So I guess, there probably aren't valuable additions, but I imagine there is
much to tinker with.

~~~
barrettcolin
For the sake of completeness, here is where I read that:
[http://www.ode.org/old_list_archives/2004-December/014735.ht...](http://www.ode.org/old_list_archives/2004-December/014735.html)

From looking at the game code, I see odd things like rolling barrels animating
their rotation as though the physics engine itself doesn't handle rolling
contact.

------
gcv
Any advice on how to dive into reading this codebase?

~~~
corysama
My favorite technique is to run the code, place breakpoints in interesting
locations and manually type out psuedocode descriptions of each function up
the callstacks. That's the fastest way I know to get an understanding of the
structure of a new code base.

Barring running the code... I prefer to start with a quick scan through the
foundation code. What does the math library look like? How about the file and
network I/O? Next, locate the top-level main loop, then the top-level Update
Everything/Render Everything functions. At that point, you'll understand the
general theme of the code and you'll have spotted lots of interesting avenues
to explore.

------
zeratul
196 clones, 171 watchers, 23 forks; that's just in the first hour from
release. Not bad, not bad at all.

EDIT: Doom 1 source code seems dirty when compared to Doom 3.

~~~
waterlesscloud
Skimming over the D3 source, it all looks so very readable. Which is
impressive in itself.

~~~
vilya
A bit off-topic, but I'm curious: why do you think that's impressive? I
thought it was pretty commonly accepted these days that readable code saves
more time than it costs over the life of a project...?

~~~
waterlesscloud
Because it may be commonly accepted, but it's rarely actually done so well.

~~~
wladimir
Game code specifically is usually the exception to this rule, because it tends
to be written to very strict deadlines, ad-hoc requirements, short project
lifetime (no need to maintain or support it for years after), and not much
expectation of re-use.

This does not all hold for Doom3 as it was also an engine sold to other
companies for other games.

------
shocks
Only took 8 minutes and 22 seconds to compile this, I was quite surprised.
Going to have some fun over the next few days that's for sure! Thanks Carmack,
heh.

~~~
akacd
Can you give me some advice how to add the gamedata to the code?I got compile
errors for I had no doom3 on my computer(windows 7),I'm downloading it now.

~~~
shocks
Sure. To compile you only need to get the DirectX SDK (from here
[http://www.microsoft.com/download/en/details.aspx?displaylan...](http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=6812)).
Then open Visual Studio (I'm using 2010 Ultimate, but Express should be fine).

(these are the steps that worked for me)

Open doom.sln Go to View -> Property Manager Expand Debug | Win32 Right click
"Microsoft.Cpp.Win32.user" and choose Properties Select VC++ Directories

You need to add the following directories C:\Program Files (x86)\Microsoft
DirectX SDK June 2010\Include C:\Program Files (x86)\Microsoft DirectX SDK
June 2010\Lib\x86

Do the same for Release | Win32.

Doom3 should now compile. Note that 2 projects might fail and 1 could be
skipped.

You should find Doom3.exe under doom3.gpl\build\Win32\Release From your Doom3
installation copy the "base" folder, if you have the expansion copy that too.

Doom3 should now launch.

If you have any more problems, feel free to ask!

~~~
kunqiana
hi I got his error, on ubuntu 11.04. I moved the base folder to the neo folder
but it still didn't work. Is it because it's missing the game data?

    
    
      DOOM 1.3.1.1304-debug linux-x86 Nov 24 2011 00:27:04
      found interface lo - loopback
      found interface eth0 - 10.0.0.19/255.255.255.0
      ------ Initializing File System ------
      Current search path:
      /home/xx/.doom3/base
      /home/xx/doom3.gpl/neo/base
      game DLL: 0x0 in pak: 0x0
      Addon pk4s:
      file system initialized.
      --------------------------------------
      ----- Initializing Decls -----
      ------------------------------
      ------- Initializing renderSystem --------
      idRenderSystem::Shutdown()
      Sys_Error: _default material not found

------
exDM69
This seems to be pretty hard wired to build 32 bit binaries only, I'm having a
little trouble build on Ubuntu w/ amd64. First you need to install multilibs
and 32 bit gcc/g++. But I got stuck when I'd need some other libs in 32 bit
versions (zlib, curl and probably openal and others). Ubuntu doesn't seem to
ship with 32 bit versions of some libraries as some other distros do (like
Arch, for example). Or am I wrong, what packages and what repositories I need
to get e.g. zlib 32 bit binaries?

Has anyone had success building this on a 64 bit machine?

Is my only option to set up a virtual machine with a 32 bit distro?

~~~
sparky
Fix SConscript.curl to look in /usr/lib32 for zlib (package name: lib32z1-dev
), and to pass -m32 to your compiler.

    
    
      /media/d2/sw/doom3.gpl/neo$ git diff
      diff --git a/neo/sys/scons/SConscript.curl b/neo/sys/scons/SConscript.curl
      index 4d889b9..7271dfc 100644
      --- a/neo/sys/scons/SConscript.curl
      +++ b/neo/sys/scons/SConscript.curl
      @@ -14,7 +14,7 @@ class idBuildCurl( scons_utils.idSetupBase ):
       
              def Compile( self, target = None, source = None, env = None ):  
                      self.TrySimpleCommand( 'cd curl ; make clean' )
      -               cmd = 'cd curl ; CC=\'' + env['CC'] + '\' ./configure --enable-shared=no --enable-static=yes --enable-http --enable-ftp --disable-gopher --enable-file --disable-lda
      +               cmd = 'cd curl ; CC=\'' + env['CC'] + '\' CFLAGS=\'-m32\' ./configure --enable-shared=no --enable-static=yes --enable-http --enable-ftp --disable-gopher --  enable-fi
                      if ( self.debug ):
                              cmd += '--enable-debug'
                      else:
      @@ -37,5 +37,5 @@ else:
       
       g_env.Command( target_name, None, Action( build.Compile ) )
       
      -curl_libs = [ target_name, '/usr/lib/libz.a' ]
      +curl_libs = [ target_name, '/usr/lib32/libz.a' ]
       Return( 'curl_libs' )
    

Linked for me on Ubuntu 11.04 64-bit.

~~~
exDM69
Thank you, this made it link.

I probably would have figured it out by myself, I solve this kinds of things
for living. But I was trying to quickly build this before going to work and
ran out of time.

------
huskyr
It will only be a matter of time before somebody converts this to Javascript
:)

------
goodweeds
I can't seem to find references to this anymore, but I seem to remember that
the doom3 source code was stolen and released in like '03 or '04. So if that's
true then my response to this article is, "Again?".

~~~
wmil
It was in '02. Id gave alpha code to ATI so they could tune their drivers. One
of their employees leaked it.

[http://forums.gamershell.com/showthread.php?407-About-
Doom-3...](http://forums.gamershell.com/showthread.php?407-About-
Doom-3-Leaked-Beta)

------
kitsune_
I'm impressed by the cleanliness of the code base, very readable. From my
experience, this is not the case with many C++ open source projects.

~~~
beaumartinez
...That's because Doom 3's engine is not an "open source project" as we know
them ;)

~~~
burgerbrain
Nor was it closed source as we knew it. I _wish_ I got paid to work on code
that nice...

------
ortatherox
git clone <https://github.com/TTimo/doom3.gpl.git>

and just browse around locally in your favourite editor.

~~~
seiji
Tip: they added direct URL cloning a while ago, so you can copy/paste without
adding .git:

    
    
        git clone https://github.com/TTimo/doom3.gpl

------
hiena03
How do I compile the map editor in linux? I guess it's in neo/tools/radiant
but I haven't found a way to build it.

~~~
barrettcolin
You don't. It's only implemented for Win32 (see neo/tools/edit_stub.cpp).

------
kunqiana
Anyone got this error?

    
    
      DOOM 1.3.1.1304-debug linux-x86 Nov 24 2011 00:27:04
      found interface lo - loopback
      found interface eth0 - 10.0.0.19/255.255.255.0
      ------ Initializing File System ------
      Current search path:
      /home/xx/.doom3/base
      /home/xx/doom3.gpl/neo/base
      game DLL: 0x0 in pak: 0x0
      Addon pk4s:
      file system initialized.
      --------------------------------------
      ----- Initializing Decls -----
      ------------------------------
      ------- Initializing renderSystem --------
      idRenderSystem::Shutdown()
      Sys_Error: _default material not found

------
bengl3rt
Hmm... Looks like this is an Xcode 3 project file. I can still get Xcode 3
from Apple's developer downloads, but it looks like it only includes the older
iOS SDKs, not the OS X 10.5 SDK required to build this.

~~~
forbes
Try: [http://stackoverflow.com/questions/5378518/how-to-add-
base-s...](http://stackoverflow.com/questions/5378518/how-to-add-base-sdk-
for-10-5-in-xcode-4)

------
smackfu
It's kind of sad to see the expressions of surprise that it actually builds
and runs. Implying that most source code releases fail test case #1.

~~~
mdaniel
In my experience, it is usually an _environmental_ assumption that is
mismanaged rather than the "source code" itself being broken.

Whether that is failure to check for a dependency, failure to document a
dependency or failure to clearly specify which specific version of a
dependency is required. And the pain of autotools is a completely separate
chapter in describing how "my" environment can differ from "theirs."

All of that just goes to show that state management is _the_ hardest part of
software.

~~~
smackfu
True. Generally the failure of the packager is that they never do a clean
install and build. And yes that is tedious but it finds so many issues, and
isn't so bad with VMs.

------
kunqiana
I am getting the error, "scons: __* [build/debug/core/sys/scons/doom] Source
`/usr/lib/libz.a' not found, needed by target
`build/debug/core/sys/scons/doom'. scons: building terminated because of
errors." Anyone knows what I should install?

~~~
enduser
zlib

or zlib-dev if your distribution separates out development packages

------
admg
Anyone else having trouble trying to run it? got the assets in place, get to
the menu fine, hit new game then get a black screen with an empty box in the
middle. Suspect its the copy protection?

------
jigs_up
Are there any parts of the quake source code in here?

------
MtrL
Heard that the experimental render path with more advanced features is
included in the source.

Can anyone here confirm/deny that?

------
baby
Does that mean we don't have to pay to get the game now? Compile the source
and playable?

~~~
Splenivore
No. “This source release does not contain any game data, the game data is
still covered by the original EULA and must be obeyed as usual.”

You should be able to play a total conversion that does not rely on the
original game’s assets, however.

~~~
beza1e1
Suggestions? The Dark Mod looks interesting.

<http://www.thedarkmod.com/>

~~~
AlexMax
Requires Doom 3 game data to play. It uses many Doom 3 resources, and other
resources are derived from retail game data.

------
maranas
thanks for this. i haven't even fully gone through quake 3's source yet.

~~~
atomicdog
I'm still on Doom :/

------
jc-denton
Installed Doom3 using the CDs and updated to the latest release. Then opened
the sln file in VS 2010 and changed the command line args for debugging to
point to the data of the game installed from the cd (instead of steam).

Now what is the matter with default.cfg and what about "Unknown command
'vid_restart'"? I am not familiar with the structure of idgames.

    
    
      ------ Initializing File System ------
      Current search path:
      C:\Program Files (x86)\DOOM 3\base/base
      game DLL: 0x0 in pak: 0x0
      Addon pk4s:
      file system initialized.
      --------------------------------------
      Unknown command 'vid_restart'
      idRenderSystem::Shutdown()
      Shutting down OpenGL subsystem
      ...shutting down QGL
      Couldn't load default.cfg

~~~
jc-denton
ah.. it works w/o the base at the end! So the correct path is "C:\Program
Files (x86)\DOOM 3"

~~~
kunqiana
I got a similar error on linux. How did you get it to work?

------
dbbo
I hope this means that there will be an actual Doom package for Debian in the
near future.

------
JonnieCache
Any gambling types fancy a little wager? Which will come first, a stable JVM
port of doom 3, or a stable build of Textmate 2? The only problem is, I really
can't think which side I'd come down on.

NOTE: For those primed to accuse me of hating: this is as much a compliment to
certain hackers and their fixated attitude towards porting iD code dumps, as
it is a snide reference to textmate2's status as vaporware.

