
John Carmack's Comment on Hardware Ray Tracing - lispython
http://arstechnica.com/gadgets/2013/01/shedding-some-realistic-light-on-imaginations-real-time-ray-tracing-card/?comments=1&post=23723213#comment-23723213
======
electrograv
Excellent response by Carmack.

"Ray Tracing" has always been an overhyped or misunderstood technology, at
least in my experience. Because of the impressive lighting effects it's famous
for producing, people view it as some "holy grail" of superior computer
graphics technology that we just need to optimize a bit for use in games.

As Carmack described, those highly realistic ray traced renders come at a
price: _billions_ of ray calculations. You don't get all that pixel-perfect
refraction, etc. for free (and probably never will - at least not with ray
tracing.) He also explains that for most people, this (e.g. pixel perfect
refraction) really doesn't matter, when rasterization techniques exist that
achieve extremely high quality approximations at a fraction of the
computational cost.

Conversely though, ray tracing and related concepts (ray casting) are not at
all without value. Many modern video games today actually use a sort of hybrid
rasterization / ray casting approach. Macrostructure objects are rasterized
the normal way, while smaller high resolution detail patterns (like a brick
wall, or stones in the ground) are "added" to surfaces via per-pixel
displacement mapping, which is ray casting at its core. This is one of the few
cases where you can take advantage of the "log2" efficiency Carmack mentioned
-- in a highly specialized implementation without a huge time constant.

~~~
ChuckMcM
Pretty much agree, the only thing that we might disagree on is the 'billions'
number, while I expect good ray tracing to trace billions of rays, if
transistors are cheap enough this becomes more interesting. The typical HDMI
1080p display is 2 megapixels (or megatexels) if you have a rendering engine
with 2,073,600 cores each of which is looking at a billion ray 'view' based on
where it sits in the scene, its easier to set up the scene and light it. That
presumes that you can do those cores like DRAM instead of like current
processors of course. My point is that cheap transistors keep amazing me and
whenever I say "Oh that will never happen" some fool goes and shows me I'm
wrong.

I'm trying to imagine how complex a 'core' would be that computed the incident
rays on a single pixel. Then figuring out how big that is in a 18nm process
technology and then trying to see if I can fit 2M on a reasonable size die. My
head exploded sadly.

~~~
yk
Throwing more ( or finer etched) silicon at an algorithm will not help the
algorithm, compared to a superior algorithm. For almost all scenes you get
better results with rasterization, the two exceptions I can think of is scenes
where the geometry of light rays is not flat, so for example involving lenses
or black holes, or where there is a lot more optical complexity than you can
reasonably use. ( A forest where each of the leafs is an object, you will
never be able to notice that the advantage of this over an forest where the
trees in the background are just textures.)

~~~
Tuna-Fish
> Throwing more ( or finer etched) silicon at an algorithm will not help the
> algorithm, compared to a superior algorithm

Actually, it will when the complexity classes of the algorithms differ. Which
is true in this case -- the time taken to ray trace a scene rises as the
log(n) of the size of that scene, while the time taken to rasterize rises
linearly with regards to the scene.

There exists a threshold of computing power/scene complexity above which
raytracing beats rasterization in speed. However, like Carmack pointed out,
the constant factors are massive, so this won't be reached in the near future,
if ever.

The reason the constant factors are so huge is that complexity of raytracing
rises linearly with the count of pixels to be drawn, while in rasterization a
lot of the work can be shared by neighbouring pixels.

Intriguingly, this means that if you can reduce the pixel counts, you can
vastly improve the value of raytracing.

Notably, if you can do eye tracking and rendering in less than 15ms, you can
reach the same visual quality as a full-screen, high-resolution render by
rendering only the areas you are actually looking at in high resolution, and
rendering the rest of the scene at a progressively lower resolution farther
from the focus point. The cone of high-precision vision is surprisingly small,
something like tens of pixels when looking at a screen from a normal viewing
distance. If you did this, you should be able to cut the amount of rays you
need to send by at least two orders of magnitude, which would bring raytracing
to real-time quality on modern hardware.

~~~
James_Duval
Naive question:

Does your last paragraph have any implications for VR? Could something like
the Oculus Rift benefit from raytracing technology? Or have I misunderstood
what you are saying?

~~~
duaneb
The idea is that raytracing can degrade gracefully, so there's a sliding scale
between rendering time and rendering quality for every pixel. And yes, this
means that, theoretically, the computer could naturally degrade pixels that
you're not currently looking at.

I suspect this would look highly unnatural in practice, though, as even static
scenes would flicker and change, especially with reflections and any surface
or technique that uses random sampling methods (which is virtually every
algorithm that is both fast and looks good).

~~~
Tuna-Fish
> I suspect this would look highly unnatural in practice

In pair tests test subjects were unable to see a difference between the
foveated image and the normally rendered one. The flicker can be removed by
blurring the image to the point where flicker no longer exists. Because your
brain is used to a very low sampling rate outside the fovea, it actually helps
in hiding the artifacts, because they occur naturally in your vision.

------
jamieb
Here's a commercial, realtime raytracing tool. Runs happily on a dual-core
macbook pro. No custom card needed. Yes, it runs better on a $20,000 64-thread
Tigerton, but it will run usably on a laptop.

<http://www.youtube.com/watch?v=yUiLBmioRI4>

Carmack is right about games (surprise!). I can't imagine "Imagination
Technologies" is pushing the Caustic R2500 at games.

~~~
samwillis
We use KeyShot at work and its an amazing package, its the industry standard
now for rendering product design CAD. The demo video showing the Caustic card
running is about equivalent to KeyShot running on a $3000 PC. They have a way
to go until they convince me that they have anything better than what KeyShot
is capable of.

------
chii
Having no real knowledge of ray tracing (or rendering for that matter), can
someone tell me why we can't be moving towards a massive cluster of gpu (in
the order of 10,000) ray tracing a scene in parellel? I imagine each ray trace
call is quite independent of any other. SOooner or later, cpu (and gpu) will
have many cores available for such form of rendering. Is it because of the
chicken/egg problem?

~~~
InclinedPlane
This is indeed ray tracing's huge advantage, of being highly amenable to
parallelization. However, the same thing is more or less equivalently true for
other rendering methods, such as rasterization. Modern GPUs have thousands of
rendering sub-unit "cores". And with CUDA you can use those cores for almost
anything, including ray tracing. But for a given scene and a given amount of
finite computing power you're almost always going to be able to get superior
results using a rasterization technique than ray tracing.

~~~
yk
I think that rasterization is easier to parallelize than ray tracing, since
raster engines essentially multiply each vertex with the same matrix, while
for ray tracing you have to traverse an octtree for each step in each ray. (
Both are of course easy to parallelize, I just think that in general you will
run into memory bandwidth problems faster with ray tracing algos.)

~~~
ericmj
Yes, memory bandwidth is one of the biggest slow downs of ray tracing.
Rasterization access memory very linearly which makes it very easy to cache
and easy to optimize in hardware on the GPUs. But ray tracing access memory
very randomly, two rays originating from pixels next to each other easily
diverges and so hits objects far from each other in the scene. This means that
ray tracing will spend most of its time waiting for memory.

------
hayksaakian
Its so cool that someone as high up in the games industry as carmack actually
knows his technical shit.

~~~
damian2000
He might be high up now, but he started off as a programmer. He co-founded id
software and (along with others) developed Wolfenstein 3D, Doom, etc.
<http://en.wikipedia.org/wiki/Id_Software>

~~~
enneff
He still is a programmer. He's the lead programmer at id and writes code for
Armadillo Aerospace.

~~~
damian2000
yeah, he can probably do whatever he wants, the fact he's still programming
means he obviously enjoys it.

------
ohwp
I think path tracing is one of the most promising methods for realtime ray
tracing.

Brigade is one example: <http://igad.nhtv.nl/~bikker/> Here an in-game
example: <http://www.youtube.com/watch?v=6_DrgiwLABk>

And a nice blog with posts about Brigade, Octane and others:
<http://raytracey.blogspot.nl/>

------
ianb
"For example, all surfaces that are shaded with interpolated normal will have
an unnatural shadow discontinuity at the silhouette edges with single shadow
ray traces."

For some reason I _really_ want to understand what this sentence means. I
don't know why it jumped out at me, maybe because it seems both accessible and
arcane, I want some path to even just tour the arcane concerns of someone so
deep into a (this) particular domain.

~~~
daeken
In English: if you're only using one ray for shading, then you'll have areas
that are completely in shadow directly next to areas that are completely not
in shadow; what we call hard shadows. Soft shadows are a better approximation
of reality, because they show the shades of grey between "covered" and
"uncovered", but they require more ray intersections (unless you're using
sphere tracing!).

~~~
dualogy
Sphere tracing is awesome. So many effects can be so easily and cheaply
approximated, apart from soft shadows also subsurface scattering, bloom/glare
etc. Downside, while great for geometry describable by distance fields, it's
quite useless with polygonal assets...

------
gavanwoolery
I think he is not wrong at all, but personally, I've been on both sides of the
debate, and then I kind of formed the opinion that it's not worth debating
about. The one thing I would strongly, strongly push for is _general purpose_
hardware (we are kind of headed in that direction with OpenCL and CUDA, but
industry-wide interest in these things is really low). Don't force me to use
rasterization or raytracing (or even triangles), just give me a really
powerful, flexible parallel processor...even if that means sacrificing a bit
of speed. I think above all Carmack would appreciate such hardware...there was
a kind of beauty in his old code you don't find any more -- you can do lots of
tricks with GPUs, but overall it is a much more limited, less creative
programming experience.

~~~
duaneb
> just give me a really powerful, flexible parallel processor

Isn't that exactly what a GPU is? They aren't terribly fantastic at
traditional computation, but then again we're talking parallelism here.

~~~
gavanwoolery
Yes, GPUs are becoming increasingly flexible, but they are still limited by
CPU-GPU bottleneck, GPU memory (which for the most part has been limited to 2
GB per card), really poor branching performance and limited ability to modify
data in any other way than 1:1 correspondence. They are first and foremost
designed to pump out polygons at high fill rates, which is fine unless you
want to do some untraditional rendering or computation. :) Again, CUDA and
OpenCL are paving the way for a new path, but I still think we should be using
general purpose silicon underneath it instead of hardware designed to to do
things like MSAA, interpolation, triangle setup, etc.

~~~
duaneb
Sure, but I still believe it's far closer to the ideal parallel hardware than
this n-core-shared-memory business that is really hard to program well.

------
Keyframe
As anyone who ever wrote a raytracer would know (or find out soon): Primary
rays cache. Secondary rays trash! Brazil guys wrote a decent raytracer
(entirely based on Glassner's book) and are really competent devs in general
(both of them). Current state of the art, performance oriented, offline
renderer is Arnold by SolidAngle, with more smart guys behind it. I think we
will see convergence of offline and online RT soon enough. There are already
steps being made in that direction, in offline world, towards RT previews.

------
MrJagil
This reminds me of <http://en.wikipedia.org/wiki/Euclideon> ... Not trying to
draw a comparison, i just find these "future of gaming" technologies
interesting, vapor or not.

------
krosaen
Within his comment is a great example / reminder to consider constant factors
when doing your 'big-oh' analysis of different approaches to a problem.

""" Because ray tracing involves a log2 scale of the number of primitives,
while rasterization is linear, it appears that highly complex scenes will
render faster with ray tracing, but it turns out that the constant factors are
so different that no dataset that fits in memory actually crosses the time
order threshold. """

------
javanix
I wish I could sit down and talk with him for an hour or so.

I'm a Unix programmer but I suspect I could learn as much from him about
software design as from weeks of talking with just about anyone else.

~~~
jerf
Here's a link you may enjoy: <http://news.ycombinator.com/item?id=3388290> ,
which goes to [http://www.altdevblogaday.com/2011/12/24/static-code-
analysi...](http://www.altdevblogaday.com/2011/12/24/static-code-analysis/) .

