
Doom 3 Source code review - vorador
http://fabiensanglard.net/doom3/index.php
======
Arjuna
Worth noting here regarding the patent issue with Creative.

John Carmack said [1]:

 _"The patent [2] situation well and truly sucks. We were prepared to use a
two-pass algorithm that gave equivalent results at a speed hit, but we
negotiated the deal with Creative so that we were able to use the zfail method
without having to actually pay any cash. It was tempting to take a stand and
say that our products were never going to use any advanced Creative/3dlabs
products because of their position on patenting gaming software algorithms,
but that would only have hurt the users."_

[1] [http://newenthusiast.com/carmacks-reverse-still-an-
issue-200...](http://newenthusiast.com/carmacks-reverse-still-an-
issue-20090409489)

[2] The patent number 6,384,822, "Method For Rendering Shadows Using A Shadow
Volume And A Stencil Buffer", can be read here:
<http://www.google.com/patents/about?id=Om0LAAAAEBAJ>

Here is the patent work-around in the Doom 3 source code. Check out the
_RB_T_Shadow_ method in _draw_common.cpp_ at line 1151 [3]:

    
    
      // patent-free work around
      if ( !external ) {
        // "preload" the stencil buffer with the number of volumes
        // that get clipped by the near or far clip plane
        qglStencilOp( GL_KEEP, tr.stencilDecr, tr.stencilDecr );
        GL_Cull( CT_FRONT_SIDED );
        RB_DrawShadowElementsWithCounters( tri, numIndexes );
        qglStencilOp( GL_KEEP, tr.stencilIncr, tr.stencilIncr );
        GL_Cull( CT_BACK_SIDED );
        RB_DrawShadowElementsWithCounters( tri, numIndexes );
      }
    

[3]
[https://github.com/TTimo/doom3.gpl/blob/master/neo/renderer/...](https://github.com/TTimo/doom3.gpl/blob/master/neo/renderer/draw_common.cpp#L1151)

Further reading on the general topic can be found here:
<https://en.wikipedia.org/wiki/Shadow_volume>

------
aw3c2
Attention readers: For some reason i cannot comprehend the pages use
javascript to load some images. There are no noscript elements notifying you
about this. So enable javascript to read through this.

~~~
mwyvern
The images are not loaded until you scroll them into view, so bandwidth is
saved if you don't read the whole article.

~~~
aw3c2
In that case the noscript fallback should be loading all immediately.

~~~
mwyvern
That's true. It seems like javascript disabling is just not on the minds of
web developers. You have to be a special kind of masochist to use noscript
these days... I used it for years, but ghostery covers most of what I liked
about it.

~~~
aw3c2
I don't use noscript (if you mean the Firefox addon). I was talking about the
<noscript> tag. ;)

I use Opera with Javascript and Cookies disabled because most things work fine
and are much less annoying with that setup.

~~~
Retric
Well I guess the overall improvement must be worth images not loading on this
page and whining on HN. So, what's the problem again? O you think your setup
should be added as an edge case, sorry non standard setups are often not worth
catering to. If you want it to work just enable scrips, it's not that hard.

------
dkersten
_We only just found out last year why it was problematic – on windows, OpenGL
can only safely draw to a window that was created by the same thread. We
created the window on the launch thread, but then did all the rendering on a
separate render thread_

I actually thought this was well known and documented as I've known it for
years. Guess not :)

AFAIK you _can_ use OpenGL in other threads, but you must lock the context to
the current thread, which I believe means you're still only rendering from one
thread at a time and possibly this is OpenGL 3+ ???

In my own multithreaded OpenGL code, my main thread always became the render
thread after starting up the other subsystems. Well, render thread and input
gathering thread (as that often needs to be done in the main thread too - at
least in SDL).

~~~
dkarl
It may be not so much "well-known" as universally observed without thinking
about it. I made the same mistake id did, using SDL in fact, spawning a
separate thread for SDL calls. I beat my head against it for much longer than
necessary and finally ended up in the SDL irc channel, where someone firmly
informed me to stop making calls from multiple threads. When I told them I was
making _all_ SDL calls from my rendering thread, including SDL_Init, as
advised by the documentation, their response was basically that nobody else
had ever had this crazy idea before, every game does rendering and UI
interaction from the main thread (never mind that I wasn't writing a game and
had no user interaction,) and I shouldn't go around trying weird complex stuff
until I learned the basics.

They didn't argue that the restriction was obvious or well-known, only that it
didn't need to be documented because good programmers never thought of
violating it. (I feel more than a little vindicated with Carmack in my corner
:-) ) Strangely enough, they also implied that it might be fixed in the
future, and now the SDL docs imply that what I did would work [1], though I
wouldn't bet on it because they have a FAQ item that seems to say otherwise
[2].

It's something for people to keep in mind when they're writing documentation:
document the limitations of your software even if you can't imagine why
someone would violate them. People coming from a different background, such as
a non-game programmer picking up a game library for some simple animation,
might approach your software with different assumptions.

[1] [http://www.libsdl.org/cgi/docwiki.cgi/Multi-
threaded_Program...](http://www.libsdl.org/cgi/docwiki.cgi/Multi-
threaded_Programming) [2] <http://wiki.libsdl.org/moin.cgi/FAQDevelopment>

~~~
dkersten
Both of the links you posted say that video/event functions should not be
called from different threads:

 _Don't call SDL video/event functions from separate threads_

 _most graphics back ends are not thread-safe, so you should only call SDL
video functions from the main thread of your application_

The second one specifically says from the main thread, while the first one
only says "separate threads". I was always under the illusion that it didn't
matter _which_ thread, as long as its the one you cal SDL_Init from. Your
experience shows otherwise...

I agree - documentation should be clear about limitations and assumptions.

------
38leinad
I really like Fabians source code reviews. Would love to see engines like
Unreal being open-sourced to compare how they work internally. Would be a nice
perspective. I.e. Unreal was much better in out-door scenes and I think the
maps where not heavily preprocessed like in Quake et.al. What was their way of
tackling the problem?

In case there are any resources that describe the technical details, would be
nice If someone could post a link.

------
stack0v3erfl0w
> the variables highlights and the "Command-Click" to reach a definition make
> the experience superior to Visual Studio.

Seriously ? There may be reasons to think Visual Studio is an inferior
product, these are not among them.

~~~
taligent
He mentioned just for code browsing so don't panic. Nobody in their right mind
would think XCode compares to Visual Studio. I mean it's 2012 and there is
still no refactoring support.

~~~
mwyvern
Can you refactor c++ code in visual studio?

~~~
lewispb
Yes. <http://www.wholetomato.com/>

~~~
mwyvern
Thanks for the link. It's a shame that functionality isn't built into VS
(especially when it costs half as much as VS pro).

------
Havoc
Very respectable of id to release it. I bet the could still get some sold cash
from it if they kept it closed.

~~~
veyron
They dont open source the latest version. Tech 5 already came out.

~~~
Havoc
>They dont open source the latest version.

I never said they did?!?

Doesn't matter though. Even the version before the current is still pretty new
tech & interesting to study.

------
ralfd
I liked especially the 2012 Q&A with John Carmack:

<http://fabiensanglard.net/doom3/interviews.php>

The blog author goes full nerd, like (paraphrasing) "You did this genius thing
making a frontend/backend pipeline for rendering. Was this inspired by LCC?
What are the advantages to a monolithic renderer?" and Cormack answers
practically "Oh, didn't help that much, because of an unexplainable quirk of
OpenGL it only worked well on my developer machine."

:-p

------
mindblink
This is great. I love reading stuff like this to gain other's insight, and
just to give me incentive to look at cool open source code.

Does anybody have additional recommendation of similar reviews of interesting
open source code?

------
marshray
I get the feeling the reviewer kind of looks up to John Carmack a little bit.

I think it must be a little bit hard to be John Carmack. He probably has a
hard time getting balanced feedback on his stuff.

------
Muzza
>In some part of the code (see dmap page) there are actually more comments
than statements.

>Dmap source code is very well commented, just look at the amount of green:
There is more comments than code !

You know, in my experience that's not a good thing. I work on similar,
heavily-commented code and find it extremely painful. At some point it becomes
a burden to see the code behind the comments. (And just so no one
misinterprets me: I am not against comments /per se/.)

It's like when you read code written by someone who simply loved whitespace
and who appended a useless "banner comment" after each real comment[1]:

// If x is less than 3, do stuff 50 times.

// -----------------------------

if ( x < 3 )

{

// While i goes from 0 to 50.

// --------------------------

for ( int i = 0; i < 50; i++ )

{

// Do stuff.

// ---------------

doStuff ( ) ;

}

}

So what would've fitted in one screen of text, if written in a sensible
fashion, now requires one and a half screen and lots of scrolling.

[1] The comments in the example are actually both crappy and pointless. Sadly,
the program I work on is riddled with them. Please don't write out what the
programming language constructs do in English.

~~~
mappu
At least in that case you can attribute it to well-intentioned stupidity.
Here's my favourite short example you must instead attribute to malice: [1] a
closed-form implementation of fibs(n). Follow along with the comments!

1\. <https://gist.github.com/eb02e9546102594e8bf7>

~~~
DeepDuh
That's really a fun example. It took me a while to even find the definition of
f after having been distracted by all the comment clutter. Point taken.

~~~
gjm11
The actual algorithm is rather nice, despite the reader-hostile
presentation...

------
cheatercheater
Sadly, no crazy genius advancements like the fast inverse sqrt. Is Carmack
going out of form, or did I just miss something?

------
cldrope
Great find, thanks for sharing. Also just idle curiousity, where'd you nab
your username? LoK series?

