
Diablo III Economy Broken by an Integer Overflow Bug - minimaxir
http://minimaxir.com/2013/05/stones-of-jordan/
======
archgrove
I remember a similar bug in the original Asherons Call (amazingly, still
running with regular content updates!). Three or four years in, they added a
new, very expensive object to the game (a "Pyreal Scarab" for anyone who
cares). It cost a lot, so the idea was that people wouldn't be (at that point
in the economy) buying many.

Now, in Asheron's Call, the world is _huge_. There are hundreds, if not
thousands of vendors. And three of these vendors were set to sell their goods
in stacks of up to 1000. Unfortunately, Cost of Pyreal Scrab * 1000 > 2^31,
which wrapped. I can't remember if you either just got the goods for free
(which you then sold back for huge profit), or if you actually got paid to
take these things. Either way, overnight, the economy was destroyed. The
entire game state had to be reset from backups; a dreaded _rollback_. Worse,
the developer took a few days to do this.

Trust me, out of all the customers whose data you _don't_ want to muss with,
it's hardcore MMORPG players. Even though I was just a player, I can still
remember the outrage all these years later. It taught me to always use
appropriate types for objects with "value", and I've never accidentally used
signed or floating point storage for currency again.

~~~
jdk
Haha - I was the one who was responsible for adding the Pyreal Scarab. I
remember freaking out that night when I was playing and went to Teth's vendor
and saw what was happening. I called up the producer at the time at 2am and
said there was a "problem".

"Oops." --Devilmouse

~~~
dxhdr
Awesome man! Asheron's call was an inspirational game for me growing up, nice
to see one of the original devs :) (Sorry for being totally off-topic!)

------
tikhonj
This is what happens when we let machine constraints trump semantics.

When you say int, you usually want an actual integer, not an integer with an
arbitrary limit. In this day and age, having that limit there is simply
premature optimization.

I think having a nice bignum type--one that looks and feels just like a normal
numeric type--is very important. It should also probably be the _default_ ;
you should only switch to a machine type if you have a good reason. With gmp,
big integers perform well enough to be used widely.

~~~
vinkelhake
> In this day and age, having that limit there is simply premature
> optimization.

You say this with certainty. Do you know of studies of real-world programs
where machine-sized integers were replaced whole-sale with bignums?

~~~
stouset
All Ruby code does this. Mathematics can still be performant because of tagged
pointers. If the MSB is set, the value is a Ruby object. If it's not set, the
value should be considered a native data type.

~~~
vinkelhake
Perhaps I should have been more specific. I know that there are languages that
have this property. My question is if there are studies on the performance
impact. Preferably in a language that is suitable for high-performance
applications (like games).

~~~
mcherm
The python example is good here, because originally the Python language had
separate types for unbounded-integer and machine-sized-int. This happened as
part of the move from Python 2.x to Python 3.x, right along with changing
strings from byte-arrays to unicode.[1]

There has been lots of fuss and even pushback about the change to use unicode
instead of byte arrays, but I have never heard anyone complain about semantics
OR performance of the switch to a single integer type.

[1] - <http://www.python.org/dev/peps/pep-0237/>

------
lmm
Just another reminder that it's never worth bypassing the normal deployment
process. Every year or two I learn a similar lesson myself: it's so tempting,
the fix is so small, it couldn't possibly break anything (heck, I once had a
data-gathering script that made a bunch of read-only calls to our system's API
cause a live issue). Just say no.

~~~
Guvante
To be fair, I am not sure if anyone would try posting 6 billion gold onto a
PTR equivalent of the RMAH or whether the RMAH is even available on the PTR,
which might explain why it wasn't tested.

~~~
MBCook
If you set the limit to 10 billion of something, you need to test that 10
billion actually works.

------
bloaf
Kingdom of Loathing had an integer overflow bug way back in 2004. It let
players set their currency (in this case: meat) to the max value of a 64-bit
integer. The game spent the next several months setting up meatsinks in an
attempt to reduce the amount of meat in circulation.

<http://kol.coldfront.net/thekolwiki/index.php/Black_Sunday>

<http://kol.coldfront.net/thekolwiki/index.php/Bugmeat>

<http://kol.coldfront.net/thekolwiki/index.php/Meatsink>

~~~
russellsprouts
It's nice that they resolved that mostly in-universe. They didn't go and ban
everyone, but instead changed the gameplay to compensate.

------
tlarkworthy
Reminds be of a bug waaaay back. In the original Elite, you could obtained a
missile lock on a space station, then dock and sell all your missiles, launch,
and finally fire a missile (you still had the missile lock). Suddenly you have
255 missiles! 0-1 = 255 in unsigned 8 bit integer math!

You could then sell all your new shiny missiles for loads of money. Made a
hard game a bit easier.

~~~
mickeyp
Those games (Frontier Elite and First Encounters included) had loaaads of bugs
like that.

Another one in FE had you put in passenger holds, fill them with passengers,
then sell the holds -- this would obviously not work as you had to evict the
passengers first, however the game logic credited you with the cash anyway
because the check came _after_ the money had changed hands.

~~~
Luyt
The amount of cash you had was stored in savegame files as 4 bytes. A few
changes with a binary editor and suddenly you had millions of cash ;-)

~~~
mickeyp
Yeah but that's true of most things that're not obfuscated. I used to use the
Amiga Action Replay on my Amiga to alter memory on the fly in games like that.

------
eduardordm
Hard to understand why people spend so much time on a short game like Diablo
III. Even before I finished it on inferno the game didn't make much sense, I
just finished for the sake of it, which I regret.

Most Super Mario games requires way more abilities than that and less time.

Don't waste your limited time on earth playing consumption-driven games. I've
been trying Eve online for a few days, It does not looks promising, it seems
that Eve also is also driven by item accumulation and not actual playing.

~~~
vyrotek
It's basically a very pretty slot machine. I played quite a bit of D2 and D3.
The thrill comes from grinding for hours (and hours and hours) and finding
that one item that earns you hundreds of dollars. (Yes real dollars) You can
thank the the Real-Money Auction House for both destroying the game and being
the one thing that keeps so many people coming back for more.

~~~
dangoldin
I loved Diablo 2 and probably spent a good portion of my high school hours
playing it. I was very excited for Diablo 3 and tried to get into it but just
couldn't.

I felt that they just took a ton of ideas from WoW and implanted them into
Diablo. Sure it may have made them more money but it made it a lot less fun.

~~~
srumple
If you liked Diablo 2, Torchlight 2 was made by the same people, and is the
spiritual successor to Diablo 2.

~~~
mattreaver
Marvel Heroes is made by David Brevik - He had the original idea for diablo 1
and played a bigger role in diablo 2 than the guys that made torch light 2.
Lets no mention his failed project with bill roper on hellgate london though.

------
danceonfire
> ... where players volunteered to tested the patch to ensure that there were
> game-breaking exploits ...

Is this an error? :) Although I assume the players would very well enjoy game-
breaking exploits, as long as they are to their advantage.

------
seanalltogether
The fact that the diablo economy has reached the point where users are running
around with 6 billion gold shows its been broken for much longer.

~~~
apetresc
Why? It's just a matter of scale. Until very recently, Romania's currency was
such that most middle-class families had 8-10 digits in their bank account at
any given moment, and the economy is relatively healthy.

Good items sell for hundreds of millions. The number of zeroes doesn't matter,
as long as the balance between items and monetary value is stable.

~~~
falcolas
"The number of zeroes doesn't matter, as long as the balance between items and
monetary value is stable."

And you're not just starting the game. The way that D3 drops work, the first
few times you're visiting the auction house, it's to buy, not sell (unless you
get very lucky).

The game was also balanced around the auction house - meaning fewer high
quality items drop, with the belief being that you're selling those few high
quality items & kitting yourself out from the auction house.

When I first played, the difference between playing with just the drops I got
versus kitting myself out from the auction house were vast. Some people might
enjoy the challenge of just playing with dropped gear, but just as many do
not.

~~~
zplesivcak
" The game was also balanced around the auction house - meaning fewer high
quality items drop, with the belief being that you're selling those few high
quality items & kitting yourself out from the auction house."

AFAIK, Blizzard collects interest on each real money transaction, so if
they're optimizing for maximization of that interest (I'm assuming they do)
they aren't minimizing occurrence of the rarest of the items. They should
behave like diamond cartel...

------
Glyptodon
I feel sorry for the players who were amused by this and will now likely get
banned. I don't know when video games started to be like a bad elementary
school where you get punished for experimenting or finding a loophole, but it
seems like it's punishing one of the fundamental joys of games. Or at least
one of the joys I remember being particularly rewarding as a child.

~~~
gebe
I agree to some degree but MMO games in general are serious business and a
special breed, especially when real currency is involved. It was most
definitely against the ToS and I am sure that most of the players who
participated in the exploit knew that. Also there were people who were using
this bug to make real life money through the real money auction house, those
people in particular can't be surprised that they were banned.

~~~
Glyptodon
I don't know... wasn't there an article on here a few days ago celebrating how
EVE online doesn't ban people for doing the exact same kinds of things?

~~~
omegaham
I think the difference is that this completely broke the economy, while the
exploit in EVE was much more fixable. All they had to do in EVE was take away
some people's loot and they were good.

------
doublec
Bitcoin has also suffered from an overflow bug, requiring a fork to fix it:
[https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposu...](https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures#CVE-2010-5139)

------
astrodust
It is odd that they're using 32-bit numbers when you'd be hard pressed to find
a 32-bit only CPU and machines with over 4GB of memory are the standard.

Good luck overflowing a 64-bit unsigned.

~~~
mckilljoy
I don't find it that odd. 32-bit applications are still very common, if not
the standard. Most processes don't need more than 4GB.

More importantly, blindly increasing all of your 32-bit integers to 64-bit is
going to double your memory usage, and ultimately just mask the real issue
(i.e. improper bounds checking).

~~~
kabdib
The actual money fields are probably small compared to the infrastructure used
to track them. It's not going to double the amount of memory usage. I'd be
surprised if memory use, network traffic and database I/O went up by more than
a percent or two.

~~~
mckilljoy
I agree for this one integer, they obviously should have been using 64-bit. I
just take issue with the implication that Blizzard is somehow at fault or
unusual for (presumably) not running a 64-bit stack.

They made a mistake for sure, but 32-bit vs. 64-bit architectures should not
be on trial.

------
sown
I remember World of Warcraft had a similar issue. The total amount of copper
and in turn gold a player could have was the positive half of a signed 32-bit
integer.

~~~
minimaxir
That didn't cause an exploit though. It just meant that the user couldn't get
more than that amount of gold.

Although back in those days, if you had that much gold, you were controlling
the economy by yourself anyways.

------
login1234
sad mirror: <http://pastebin.com/YYPM4uQK>

------
mikevm
The scary thing is that integer overflows are considered rare so unlike things
like null-pointer dereference no one really checks for them (heck, it seems
impractical checking for it).

In this case, how should they defend against an overflow? Impose an arbitrary
limit on gold?

~~~
Jabbles
Use 64 bit integers everywhere.

~~~
TwoBit
Why not 128 bit integers.

------
meerita
I told this in their forums many times: you cannot have both real world money
and one digital currency at the same time. Farming gold is hillarious. You
need to implement instead something like bitcoin and it's the only way to stop
the inflation.

------
pilif
Something like this used to work in Sim Farm too: buy and sell a piece of land
and watch taxes grow until they flow over and you get a bunch of money instead
of paying. If only this worked in real life :-)

------
nsxwolf
Is this sort of hyperinflation intentional? Or a sign of economic ignorance? I
don't get it. Gold is as common as dirt, and players are pumping huge amounts
of it into the economy on a constant basis.

------
ebbv
420,081,335,014 is 420 billion not 420 trillion.

~~~
unwind
English is handy enough to have two "scales", where these words have different
meanings.

In the short scale, you'd be right, but the author probably uses the long
scale.

See <http://en.wikipedia.org/wiki/Long_scale#Comparison>.

It's one of those things that makes English ... interesting to learn as a
foreign language. :)

 _EDIT_ : Actually no, I mis-read the table on the Wikipedia page. The author
was simply mistaken, and has corrected that. Thanks to everyone who pointed
this out.

~~~
quarterto
Also, basically no-one uses the long scale anymore. Which is a shame. -illiard
is a fantastic suffix.

~~~
adlpz
Well, if no-one is pretty much all the non-english-speaking world, then yes.

~~~
yen223
The Chinese don't use the long scale either, at least not in this form - we
count in magnitudes of 10,000.

~~~
adlpz
I wasn't aware of that. To be sincere, I also made the mistake of looking too
close home. I was mostly talking about the non-English-speaking 'western
hemisphere'.

------
yekko
Coming soon to the real world with Ben's continued money printing. Maybe we
can rollback the bank accounts to pre-1999!

------
WhoIsSatoshi
is the link down? Can't access content - keeps loading here..

------
kabdib
I hear the D3 Auction House was written in Oracle.

This explains /everything/.

------
yoster
Heh, this reminds me of when I played Diablo 2. I duped so many sojs/items
that I had multiple accounts just filled to the brim. I rarely play Blizzard
games nowadays, but as long as there are games with any type of currency that
can be sold, it will always be exploited.

