
John Carmack starting port of Wolf 3D in Haskell - bobfunk
https://twitter.com/id_aa_carmack/status/331918309916295168
======
Arjuna
In case you were not aware, the iOS source code for Wolfenstein 3D Classic
Platinum is licensed under GPL. It is available here:

[http://download.zenimax.com/idsoftware/src/wolf3d_ios_v21_sr...](http://download.zenimax.com/idsoftware/src/wolf3d_ios_v21_src.zip)

I thought this story was cool:

 _"[...] They were using the software rasterizer on the iPhone. I patted
myself on the back a bit for the fact that the combination of my updated
mobile renderer, the intelligent level design / restricted movement, and the
hi-res artwork made the software renderer almost visually indistinguishable
from a hardware renderer, but I was very unhappy about the implementation.

I told EA that we were NOT going to ship that as the first Id Software product
on the iPhone. Using the iPhone's hardware 3D acceleration was a requirement,
and it should be easy -- when I did the second generation mobile renderer
(written originally in java) it was layered on top of a class I named TinyGL
that did the transform / clip / rasterize operations fairly close to OpenGL
semantics, but in fixed point and with both horizontal and vertical
rasterization options for perspective correction. The developers came back and
said it would take two months and exceed their budget.

Rather than having a big confrontation over the issue, I told them to just
send the project to me and I would do it myself. Cass Everitt had been doing
some personal work on the iPhone, so he helped me get everything set up for
local iPhone development here, which is a lot more tortuous than you would
expect from an Apple product. As usual, my off the cuff estimate of "Two
days!" was optimistic, but I did get it done in four, and the game is
definitely more pleasant at 8x the frame rate.

And I had fun doing it."_ [1]

[1] [http://www.idsoftware.com/iphone-
games/wolfenstein-3d-classi...](http://www.idsoftware.com/iphone-
games/wolfenstein-3d-classic-platinum/wolfdevelopment.htm)

~~~
ksec
It would properly take two months, which is properly optimistic, they would
get it done in four months instead.

And JC got it done in 4 days....

This isn't 10x programmer anymore, this is 30x!!

~~~
CmonNoReg
Well, you better know the code you have written...

~~~
ido
...many years ago!

------
wting
Carmack has been thinking about functional programming for a while and posted
his thoughts on applicable lessons for C++ a year ago:

[http://www.altdevblogaday.com/2012/04/26/functional-
programm...](http://www.altdevblogaday.com/2012/04/26/functional-programming-
in-c/)

He's a great developer and has always pushed boundaries. I look forward to his
postmortem after this project is finished.

~~~
blt
_Maybe if all of the objects just referenced a read only version of the world
state, and we copied over the updated version at the end of the frame… Hey,
wait a minute..._

This sounds like a game development reference that I'm missing. Can anyone
explain?

~~~
chipsy
He's referring to the utility of immutable data for solving certain
parallelism issues - rather than attempt to coordinate all the code that uses
a data structure, you can double-buffer it and queue up the write events for
the "next frame" instance.

This is a hugely successful pattern throughout a number of aspects of gaming,
graphics being one of the most classic examples. Double-buffered graphics
don't suffer as much from tearing and other display artifacts.

~~~
laureny
> This is a hugely successful pattern throughout a number of aspects of
> gaming, graphics being one of the most classic examples.

Not really, no. Immutability comes at the cost of performance compared to
mutability. The gap is shrinking between the two, but it's still wide enough
that using pure immutable structures for frame buffers, shaders and other
graphical concepts is simply not an option to write games.

Haskell is interesting in the sense that it doesn't prevent you from using
mutable structures (e.g. Lenses, Writer) but it encodes this information in
the type system. I'm really curious to read the conclusions that Carmack will
draw from his experience but I wouldn't be surprised to read that at the very
low levels, mutable structures are just unavoidable for high performance
games.

Also, mutable structures accessed by concurrent threads is a problem that's
much less difficult than most people claim, and it's often much easier to
reason about locks and semaphores than about immutable lazily initialized
structures.

~~~
DanWaterworth
I don't know where to start.

> using pure immutable structures for frame buffers, shaders and other
> graphical concepts is simply not an option to write games.

Seeing as people have written games in Haskell, this is clearly not true.

> Haskell is interesting in the sense that it doesn't prevent you from using
> mutable structures (e.g. Lenses, Writer)

Lenses and Writer both only use immutable data. It is possible to use actual
mutable data in the ST and IO monads.

> but it encodes this information in the type system.

This is true of IO, but not of ST. With ST, runST :: (forall s. ST s a) -> a,
hides the effects.

> it's often much easier to reason about locks and semaphores than about
> immutable lazily initialized structures.

I don't know what you mean by this. In terms of functional correctness,
immutable data-strucutures, lazy or otherwise, are much easier to reason
about. If you are talking about resource-usage, sure, it's a little harder to
reason about lazy data-structures than strict ones, but give me a space leak
over a race condition to track down any day.

~~~
evincarofautumn
It’ll be interesting to see how the performance issues play out, no? In order
to get reliable memory behaviour, you still have to go through a certain
amount of voodoo to appease the gods that govern the interplay of laziness and
GC. There are comparatively few people who really know how to optimise Haskell
code from top to bottom—in part because there is such a distance between top
and bottom.

~~~
jerf
"It’ll be interesting to see how the performance issues play out, no?"

Not really. There's no question whatsoever that GHC can run a fine Wolf3D on
fractions of a modern hardware setup. You could do it in pure Python with no
NumPy. There's tools to help with the laziness stuff and a 3D rendering loop
will fit those perfectly.

~~~
marshray
Sure, Wolf3D is almost a quarter of a century old by now.

But the performance _limits_ of immutable structures for simulation and
graphics are certainly interesting to me.

------
thezilch
_I also had to make one last minute hack change to the original media -- the
Red Cross organization had asserted their trademark rights over red crosses
(sigh) some time after we released the original Wolfenstein 3D game, and all
new game releases must not use red crosses on white backgrounds as health
symbols. One single, solitary sprite graphic got modified for this release._

I always wondered about color choices for some games' health packs.

~~~
octo_t
As much as thats annoying, it does make a lot of sense. If you can assert that
everything which bears that logo is going to be medically related (and not
just something random), especially in wars and the like, means you can be more
certain of someones intentions.

~~~
marshray
Wars observe trademarks? I thought the Red Cross was established by
international treaty. <https://en.wikipedia.org/wiki/First_Geneva_Convention>

~~~
dllthomas
No, but if there is a red cross on white on a sign, it's good to know it's not
just advertising from back before the region became a warzone.

~~~
mzs
How long before?

[http://upload.wikimedia.org/wikipedia/commons/0/0d/Knights_T...](http://upload.wikimedia.org/wikipedia/commons/0/0d/Knights_Templar_Cross.svg)

~~~
vidarh
Both the cross of the Knights Templar, and the similar Maltese cross used by
the Knights Hospitalier are quite distinct from the straight cross the Red
Cross uses. I very much doubt anyone anywhere where variations like that are
likely to be seen would confuse them.

------
kleiba
Here's a question for my fellow HNers.

In the article linked from his tweet, Carmack describes how he ported
Wolfenstein3D to the iPhone. He apparently didn't start off with their own
original code base, but used the open source project "Wolf3D Redux"
(<http://wolf3dredux.sourceforge.net>) as a starting point. This was possible
because id open sourced their original game, and "Wolf3D Redux" is distributed
under the GNU GPL v2.

Carmack also states: "I think the App Store is an extremely important model
for the software business."

Therefore my question: is it possible to publish GPL'd games in the App Store?
I seem to remember that this was not possible, since ToS of the app store
impose further constraints which is forbidden under the terms of the GPL.

~~~
nglevin
It's possible, though it depends on the wishes of the original authors.

Stockfish Chess [1] is the one GPL v3 licensed app that I know of which has
been available for... quite some time, now. I've seen several other Chess apps
build on that engine, which go as far back as when I was just starting iOS
development.

Now, VLC was the one case where one volunteer for the project invoked the GPL
to get Apple to take an iOS port of the app off the store [2]. Based on that
situation, it seems that if the original authors of a GPL licensed codebase
want to pursue a claim against an app that uses that code, they can, and Apple
will take it seriously.

I don't believe that situation has changed.

EDIT: For your own sake, it's probably best if you approach the original
author to see if they can make an exemption, in writing, for the DRM
situation. IANAL, but that seems like it would be the cleanest way to go about
handling GPL licensed code without issue.

[1] - <http://stockfishchess.org>

[2] - [http://www.tuaw.com/2011/01/08/vlc-app-removed-from-app-
stor...](http://www.tuaw.com/2011/01/08/vlc-app-removed-from-app-store/)

~~~
rmc
Remember, you'll need to contact all the authors and they all have to consent
to (essentially) relicense their code. Any open source project that accepts
patches may have dozens of actual authors. You can't just ask the project
maintainer/chief contributor.

~~~
nglevin
Ideally, a well-organized project would have a copyright assignment process to
resolve those sorts of issues.

However, as far as I know you're right. If there's no clear authority as to
who retains ownership and licensing rights to the code, and the contributions
made to it, it's going to get messy.

~~~
Tuna-Fish
> Ideally, a well-organized project would have a copyright assignment process
> to resolve those sorts of issues.

A lot of projects, notably including Linux, intentionally avoid copyright
assignment to make it impossible for anyone to relicense the codebase. Making
sure that there are thousands of copyright holders from hundreds of
jurisdictions, many of who are not easily reachable or even knowable, all
bound by common license terms protects the project from situations where some
project participants would do something not agreed by the rest, either
willingly or because they were forced to (eg. through bankruptcy).

~~~
nglevin
That's a good observation, some inefficiencies are quite intentional. I didn't
even know that about Linux until just the other night, when I was looking for
non-GNU GPL projects that didn't require this process.

LWN.net has covered both sides of this subject quite well, in recent months,
with the tedious process of relicensing VLC [1] and the GnuTLS copyright
assignment controversy [2].

The politics of OSS make for some excellent reading.

[1] - <https://lwn.net/Articles/525718/>

[2] - <https://lwn.net/Articles/529522/>

------
Legion
His post about the iOS port of Wolf3D was a great read, and also a crystal
clear reminder of why I never play 3D games on a touchscreen-only device. The
means of input are so truly terrible that I do not understand why anyone
bothers. But, clearly, there's an audience that does not care, and it was neat
to read about all the things Carmack tried to figure out how best to pull it
off.

~~~
adventureloop
You should try Galaxy on Fire 2.

The trend seems to be towards difficult 3D controls, but you cannot write off
the entirety of 3D games because you haven't seen good samples.

------
ineedtosleep
What would be the immediate benefits of this? Would it be mostly related to
multithreading?

And on a side note: As a NoScript user, I'm surprised that I've had to give
Javascript permissions to Twitter just so I can visit that idsoftware.com link
inside of the tweet. Obviously I don't visit Twitter often, but that should
never be the case for it or any site.

~~~
dubcanada
It's fun... He's a developer who enjoys doing stuff, it doesn't matter if it
has a purpose. To him it does, and he has enough followers that he felt he
should share it with people, which makes sense considering I can't wait to see
the result.

And I'm sorry, but if you turn off javascript this day and age you should
expect more sites to not work then to work. You're turning off part of the
browser, and you do not like having to turn it on to use the browser? Seems a
little backwards to me... considering the extremely small population of
noscript users.

I also understand that I'll probably get down voted into oblivion as every
single noscript users is probably on this site. But it's true...

~~~
SquareWheel
No, I'd agree. I've stopped building <noscript> tags into my sites as
virtually every browser supports Javascript now. If folks are smart enough to
turn off Javascript they're just as capable of turning it back on.

I personally block Flash by default (mostly to stop audio ads), and I have no
problem turning it back on when a site requires it.

~~~
blub
Folks don't want to turn it on, because it's not needed for most cases and
it's the perfect way (after flash and java) to get malware.

I don't know what's happening to web development, but since this app fever
developers are fighting for eye candy and LESS accessibility. Sometimes I see
a page that fails to work without JS, open the source and see just JS code.
Where is the content? Web apps are often times also walled gardens, and
completely break the functionality of the browser (based on linking, rendering
text & images and using the back button).

Now even simple websites completely disable access to content just to show
some silly animation.

~~~
the_gipsy
I can't think of a way of getting malware from JavaScript anymore than from a
simple link, that's just plain false.

The move to "web-page-apps" is not about eye candy, it's about speed,
responsiveness and yes, usability. About not trying to awkwardly force an app
down the http/html way.

A broken website/app is simply broken: if things do not work as expected,
that's not the fault of webapps per se or JavaScript - it's all doable and not
a big deal anymore (the back button thing).

~~~
papsosouid
>The move to "web-page-apps" is not about eye candy, it's about speed,
responsiveness and yes, usability

No, it is about following fads, just as web development has always done.
Creating sites that are slower, less responsive, and offer terrible usability
is clearly not done for reasons of speed, responsiveness and usability.
Enhancing a page with javascript can increase responsiveness and usability.
Replacing the page with javascript is moronic.

~~~
mercurial
There are cases where a "full Javascript" page provides clear benefits (eg,
GMail).

~~~
papsosouid
Perhaps I should have been more clear. I'm just talking about web pages,
basically just information exchange. Twitter being the example here. Browser
applications are a different thing, and obviously have to be delivered in
javascript. There's nothng wrong with browser applications, but pretending web
pages should be built as browser applications is insanity.

~~~
petsos
GMail is just some text and links too. Why does it get a free pass, but
twitter doesn't?

~~~
FreeFull
GMail does have a no-javascript fallback, that has all of the important
functionality.

------
taeric
I'm more than a little curious at what folks are thinking this will prove. I
mean, to a large extent this feels like it would be akin to watching a master
photographer switch from film to digital. The vast majority of the craft is
hard to see at the implementation layer.

I mean, consider everyone's favorite sort method. Seeing it implemented in any
language does little really show how amazing the original insight was.

I rush to say that I am highly interested in seeing this. Nor do I question
what Carmack is looking to see. It is strictly the talking heads around this
that have me somewhat... off.

~~~
kayoone
I think its more like watching a master photographer experimenting with a new
way of taking pictures (think HDR or something), to keep your reference.

In the Clang world Carmack usually lives in, the tools for game development
are extremely mature. There is decades of knowledge and experience that went
into the current generation of 3D Engines/Games.

Game Development in Haskell on the other hand is extremely new, as in never
been done on a larger scale. Of course alot of the knowledge and experience
can be reused, but it is starting from scratch to a large degree.

So id see this as an experiment of a very talented programmer who wants to see
if he can translate his immense knowledge of graphics programming into a
working game prototype built with a functional language like Haskell.

~~~
taeric
In my head, I was thinking of someone like Ansel Adams when I made my
comparison. So, I was thinking of pioneers in the field. I believe I see what
you mean, that at this point there is plenty of knowledge in digital
photography. I was just trying to invoke the idea that the algorithms and
thought processes that go into creating a project are not necessarily
reflected in the actual program that is written.

Now, I have little doubt that a lot of this is because he used to have to
program so close to the metal. The abstraction was the computer, to a large
extent. The hope now, I suppose, is that he can focus on creating abstractions
with the aim of a maintainable and flexible engine.

~~~
kayoone
Yep, but even then its still just Wolf3D which basically is a 20 year old
game/technology. Certainly a good first step into graphics programming in
functional languages but i doubt we will ever see something like Unreal Engine
4 written in Haskell. All the effort put into C-based Engines has such a
legacy that starting from scratch would be an immense effort for questionable
benefits.

------
b0rsuk
A bit off-topic: has John Carmack made any comments on Go programming language
? It sounds like a language that should appeal to him. The name makes it not
easy to google for (I've tried golang, etc). I couldn't find anything, is it
because Go has a far way to go before it's mature enough to use in games ?

~~~
Tuna-Fish
As of right now, Go is essentially completely uninteresting to game dev
because it's not possible to guarantee steady frame rates in it. (The GC
solution is bad for latency.)

Also, I doubt Carmack would be interested in Go -- based on his previous
talks, he seems to be going for a more functional approach. Go is imperative
to the bone.

~~~
gillianseed
Well I would agree if we are talking about very demanding current gen style
games (which id software indeed typically produce) but if we look at indie-
style games then having garbage collection doesn't preclude a language from
being used in games development, or having steady frame rates.

There's no technical reason I can think of why Go wouldn't work just as well
as C# (XNA/XBLA/MonoTouch/Android) or Java (Minecraft etc) for game
development.

Garbage Collector sweeps can indeed be a performance problem but there are
obvious ways of minimizing it's impact during gameplay.

Overall I think (read guess) that unless you are doing some sort of physics
simulation, typical game logic requires relatively little cpu power and for
the graphics there is hardware acceleration doing the heavy lifting.

~~~
Tuna-Fish
Go will likely eventually have a GC that will not be that terrible. However,
right now it's a simple parallel mark & sweep. If you allocate more than a
1000 objects, and do any allocation during simulation, there _will_ be very
noticeable GC pauses. The current Go GC is not anywhere near the capability of
C# or Java GC, and would _not_ be suitable for minecraft-level games.

> Overall I think (read guess) that unless you are doing some sort of physics
> simulation, typical game logic requires relatively little cpu power and for
> the graphics there is hardware acceleration doing the heavy lifting.

Well, it of course depends on the game. AAA games spend all the CPU budget
allocated to them.

~~~
gillianseed
I realize that it doesn't compare to Java's GC (which is likely state-of-the-
art) but is it really terrible?

Granted I haven't used Go myself (leaning towards Rust) but I've read that Go
is already being employed in production scenarios with good results.

~~~
Tuna-Fish
It's perfectly passable for web serving and other such high-latency tasks. In
a GC, the goals of throughput and latency are diametrically opposed --
optimizing for one makes the other worse. The present simplistic GC is a
typical throughput-oriented design.

~~~
gillianseed
I just remembered these (I knew I'd seen some game examples in Go):

<http://www.youtube.com/watch?v=BMRlY9dFVLg>
<http://www.youtube.com/watch?v=iMMbf6SRb9Q>

which are game examples as part of a game engine framework written in Go,
seems to move smoothly with lots of objects including 2d physics.

------
sunwooz
Can someone tell me what exactly a 'port' is? Is it just rewriting all the
code base in a target language?

~~~
brooksbp
Rewriting the code base in another language would be considered a 'rewrite'.
Usually, ports are in the same language but targeted at a different platform
whether that be an OS or architecture, or even something like a different
graphics library (OpenGL game ported to DirectX).

~~~
vidarh
Referring to total rewrites in different languages as "ports" has a history in
the game development world going back to the beginning, when those doing
"ports" often did not even get access to original source or assets, and the
games would be written in asm for the target platform anyway.

------
bwooceli
My heart goes out to the poor Redux project maintainer. Can just imagine him a
few months from now checking on his old email address that he's forgotten
about and finding John's invite. Friendly reminder to everyone to keep your
project contact info current.

------
GrandTheftR
This is encouraging, have been working on my game in Haskell for last few
weeks, it is fun. Hope to see more insight & progress from John soon.

------
Aardwolf
So how do you keep state like health points in a functional language? :)

~~~
ufo
One way you could do it is by using mutable state. Impure languages, such as
Scheme or Ocaml, let you use mutable like normal imperative languages do and
in Haskell you can use mutable references inside some special monads (and you
even ave more than one kind of mutalbe reference, depending on what monad you
are on: IORef, STRef, etc)

The other way is to not use state. If you squint a bit, you can see that if
you explicitly encode your "state" as function arguments you can kind of
update it by calling the function recursively:

    
    
       go 0 acc = acc
       go n acc = go (n-1) (n * acc)
    
       fact n = go n 1
    

In the previous example, the code I wrote does exactly the same thing as the
usual imperative loop might have done, but the accumulator is a parameter on
the helper function instead of being a mutable variable.

In general, if you are OK with this kind of non-destructive updates that I
used here, you can encode all your state as extra parameters that you thread
around your functions. You can do this by hand in most cases but in some
situations the state is very pervasive and correctly threading it around can
be complex and error prone. In that case, you can look into use things like
the State monad (not to be confused with the ST monad!) to do that implicitly
pass that parameter around for you.

~~~
Aardwolf
But for a game your state needs to update at specific time intervals,
otherwise you walk too slow or too fast. How can that ever be purely
expressed?

Personally, I would love to see more functional possibilities in imperative
languages, but have regular imperative possibilities as well. E.g. running the
game and keeping state seems pretty suitable for imperative code. But doing
certain updates or calculations could be expressed better with functional
code.

~~~
ufo
> But for a game your state needs to update at specific time intervals,
> otherwise you walk too slow or too fast. How can that ever be purely
> expressed?

In this case you want ot have a system that reacts to an event that fires on a
regular interval (as well as other sorts of input events). If you search for
Functional Rdeactive Programming you will find some example libraries out
there that try to do this in a pure manner (although I would personally have
to say that this is all still a bit on the experimental side of things).

That said, Haskell still lets you do things the imperative way if you want!
All you need to do is put the impure code in the IO monad, where it belongs.

You are only forced to be purely functional if you want to or if whoever is
calling you must be a pure function. So basically, the idea is that your
`main` function is impure code in the IO monad and it can call either more
impure code or pure "helper" functions. Increasing the percentage of your code
that is pure is a nice thing but its not mandatory.

------
ctdonath
Any word on a Quake port to iOS?

------
the1
it's gonna be another perl 6.

~~~
pjscott
John Carmack is actually pretty good at writing Wolfenstein 3D.

