
Finding a Video Poker Bug Made These Guys Rich - relham
http://www.wired.com/2014/10/cheating-video-poker/
======
colmvp
> At 1:30 pm on October 6, 2009, a dozen state and local police converged on
> Andre Nestor's split-level condo on a quiet, tree-lined street in Swissvale.
> He was dozing on his living room couch when the banging started. “State
> police! Open up!” The battering ram hit the door seconds later, splintering
> the frame and admitting a flood of cops into the house.

> Nestor says he started toward the stairs, his hands over his head, when he
> came face-to-face with a trooper in full riot gear. “Get on the floor!”
> yelled the trooper, leveling his AR-15 at Nestor's face. Nestor complied.
> The cop ratcheted the handcuffs on Nestor's wrists, yanked him to his feet,
> and marched him into the kitchen.

That seems perfectly reasonable for a non-violent crime.

~~~
discardorama
Read till the end. The government seized all his money, and even though he was
acquitted (or never tried), they still kept it. And the IRS is after him too.

If you go against the government, you almost never win, regardless of how
innocent you are.

~~~
grecy
Would he have any grounds to charge the government and/or the casino of theft?

He won that money in a fashion deemed not-illegal, the IRS even says he
legally won it and want to tax him for it, and then they took it off him.

Sounds like theft to me.

~~~
CocaKoala
> the IRS even says he legally won it and want to tax him for it

The job of the IRS isn't to pass judgement on the legality of income, just to
collect taxes. If you earn money, it's taxable; that's true wether you're a
computer consultant, a video poker gambler, a hitman for hire, a drug lord, or
Al Capone.

~~~
grecy
Right, but the distinction here is they believe he earned it.

If they believe he earned it, and someone else took it away, isn't that theft?

~~~
CocaKoala
The belief of the IRS is entirely separate from the issue of theft and
bringing it up in this context is just confusing the issue. It's like asking
"If they believe he earned it, and the sun is rising, is it before noon?"

------
bmmayer1
Yup. Sounds about right.

Read about Phil Ivey, who used a similar "hack" to advantage play in Baccarat.

Casinos really can't have it both ways. If they want to make money from people
gambling money, they need to assume the risk of losing the gamble through
their own incompetence.

If they released a table game with a negative house edge (due to some
mathematical miscalculation) and most of the people who played it won money,
they would figure out their mistake, pull the game and everyone would keep
their winnings. How is this any different?

~~~
specialp
Casinos make money by providing games with various house edges to gamblers.
Gamblers and casinos know this, and if it is unfair you have the choice to not
gamble. Suppose there were a glitch with the machine that caused it NOT to pay
out as it should... The casino would be liable to pay back that money to
gamblers under numerous gaming laws. Would you then say that they should not
have to pay it back as gamblers should assume risk when they gamble? Also, if
it were found that the casino knowingly had a glitchy machine out there that
did not pay out there would be people going to jail for that.

If there were a vending machine that gave me all of the money it had in it as
change after hitting a combination of buttons, and I travelled to every
machine in town and did this, I would be stealing. This is no different. If
they calculated the house edge wrong and launched the game to players that
would be a different story as the players played in good faith and it is the
casino's fault that they did it wrong.

~~~
joelberman
It is highly likely that software-driven games could have a bug. Therefore the
casino could have decided that electronic machines are too risky. Or they
could have paid for some kind of insurance. I think the casino, just like many
banks and retail stores, are using technology that they do not understand and
are unwilling to pay or learn. They want more profit. Let the seller beware.

~~~
specialp
Risk assessment != allowing someone to steal from you. Surely the casino knows
that any game can be fallible and they could take a loss from it. I guess
banks know that their computer systems could have problems and they can lose
money from it. But what if I noticed if I exchanged a certain amount of money
between my checking and savings account a few times, it doubled my balance?
And then I repeated this many times knowing that there was an error in their
system?

Now the bank if I had gotten away with it and could not be detected would then
chalk it up as a loss and part of the calculated risk of having computerized
banking. However if they knew I did this on purpose and took their money
should they chalk that up as a loss and allow me to get away with it? No they
would want that money back from me and possibly have me arrested for fraud.

------
bradly
Three friends and I found a broken slot machine an a Casino a few years ago
the night before Thanksgiving. It was an older style slot where coins actually
still paid out and the darn thing would just spit out more money they you it
was supposed to. Obviously we were blown away. We would put twenty bucks in,
play one or two times, and then hit payout and get between $50 and $100. So we
started cashing out. A lot.

So here's the sketchy part... The machines only hold so much coin, so we kept
having to have the attendent call to get the machine filled. I had seen enough
movies to know how Casinos feel about this sort thing, so I would go to the
bathroom and stuff the cash in my sock. The attendent started to make comments
like, "wow, lucky night, huh?". This really spooked us so we eventually took
off.

The four of us came in two cars so when my one friend was dropping me off at
my house, we looked at other, and then took off back to the casino. When we
got there there was someone else playing the machine; doing what we were doing
but at a smaller scale. We probably sat and waited three hours for him to
finish as he gave us a nod as he walked off.

We cashed out a few more times and then took off around 6am with a great story
to tell the family at Thanksgiving dinner.

~~~
dmayle
Slot machines are programmed with a table of payouts based on a rate,
typically between 80 and 99%, such that the statistical expected payout
overtime will be compounded at that rate. In some jurisdictions (e.g. Vegas),
it is legal for the casinos to program a machine at greater than 100%. They do
this to attract the players, who will attempt to find the winning machine on
the floor. Random numbers still work, so not every payout wins, but over the
course of the day you can get a payout. So you may have just walked away from
free cash.

Source: I used to work in casino gaming (I wrote the Megapot server used for
multi-casino progressives in France)

~~~
bradly
Interesting. Thanks for the info!

Re: Megapot server. I've always wondered if those incrementing jackpot banners
are actually tied to a real server of if it just increments based on an
average. Are those incrementing numbers actually coming from a server in
realtime?

~~~
dmayle
In the case of most progressives, there is just a single bank of machines, and
they are communicating in realtime (wire delay) between them. When I quit the
industry, the Megapot was the first server distributing money across sites in
realtime (network delay). There was a similar product offered by IGT at the
time, but the incrementing counters were entirely based on play within the
casino; casino jackpot sums were batched and shared across the casinos
according to a fixed schedule. Our displays were certainly based on real
money, and I suspect anyone else in the industry would have to do the same to
be legally in the clear.

------
discardorama
As I read the story, I shook my head in disbelief. How stupid could these
people be? Hitting the same machine again and again and again, in sequence?

And once again, talking did them in. If Kane had kept it a secret and just did
the Vegas circuit, hitting one casino per day for about $10K, he could still
be milking the bug.

~~~
PhantomGremlin
Nah.

I'd hit one casino per day for $1000, no more. No IRS paper trail that way.

I could probably manage to survive in Vegas on $250,000 tax free per year,
working a few hours a day, 5 days a week.

~~~
sejje
I imagine him feeling some pressure that the bug could be patched at any time
(even accidentally).

I would have traveled to hit machines everywhere, as quickly as possible, but
only once per location.

------
jrockway
As programmers, we sure have a good position in society. If someone uses our
buggy code to steal credit cards, the evil criminals are blamed. If someone
uses our buggy code to win infinite money from a casino, the evil users that
figured out the bug are blamed. If our autopilot software decides that a pilot
is trying to land an A320 when he's trying to climb and the plane crashes into
a forest, the pilot goes to prison, not the programmers!

And, we get this complete autonomy and immunity without any education or
certification!

With great power comes great responsibility, folks.

~~~
josho
I like your point, but you've drawn the wrong conclusion.

Those with power are the ones that make the rules. Programmers aren't the ones
with power, it's the casinos or the plane manufacturers with power, so it's
their rules as to who the losers are. If, somehow it was in the casino's best
interest that the programmer was made the villain then I'm sure a reason would
have been found.

Case in point Goldman Sachs screwing over Sergey Aleynikov, a programmer that
served time because Goldman didn't want his expertise in a competitors
company.

~~~
jrockway
The Goldman issue wasn't about defects in the work product, though. If Goldman
sent everyone to jail that wrote buggy code, there would be no programmers
left in New York!

------
mikeash
It struck me how the cooperation of the casino often comes up in these
stories. Here, the trick ended up relying on the casino enabling the Double Up
option on request. There was a story making the rounds a few months ago on HN
about a famous poker player who took a casino for millions of dollars (playing
a different game) by relying on a subtle asymmetry in the back sides of their
cards, and that scheme required the casino to cooperate by providing a dealer
who spoke Mandarin and allowed the cards to be turned around ahead of time.

I wonder why casinos don't just refuse _any_ sort of unusual request related
to the game. Sure, it _sounds_ like an innocent request that won't affect the
payout. But how sure are you about that? Wouldn't it be safer to just set the
rules and never change them on request?

I assume the answer is ultimately that legitimate high-rollers sometimes have
crazy requests, and you make more money from accommodating them than you lose
from cheaters. But the response from the casinos (getting this guy thrown in
jail, suing the poker player I mentioned above) seems to say that they aren't
really comfortable with that tradeoff.

~~~
specialp
"I assume the answer is ultimately that legitimate high-rollers sometimes have
crazy requests, and you make more money from accommodating them than you lose
from cheaters. But the response from the casinos (getting this guy thrown in
jail, suing the poker player I mentioned above) seems to say that they aren't
really comfortable with that tradeoff."

They may be making the tradeoff but that does not imply that when the negative
consequence happens (cheating) that they are going to let them get away with
it. Suppose a store placed products outside in front of the store. There is
tradeoff of increased sales of that product, but increased theft rate of that
product. If they caught someone stealing they would still make them return the
stolen item and possibly prosecute. Just because a negative risk is
calculated, that does not mean that a detected negative event is _accepted_

~~~
mikeash
That's a good point, but taking advantage of a casino after a player _asks_
the casino to help them cheat, and the casino _obliges_ , is a much grayer
area of morality than simply stealing stuff.

To make another retail analogy, it's like a guy walking into a store, picking
something up, and asking the clerk, "Hey, is it OK if I just take this home
now?" And the clerk says "yes" with the _assumption_ that the person intends
to pay for the purchase later.

~~~
bobbyhotpockets
"That's a good point, but taking advantage of a casino after a player asks the
casino to help them cheat, and the casino obliges, is a much grayer area of
morality than simply stealing stuff."

FWIW, the commenter wasn't referring to the morality of the practice. He
mentioned what casinos _would_ do, not whether it was right or wrong.

~~~
mikeash
It's obvious enough what casinos actually do, I'm just commenting on the
psychology behind it.

And given that the casinos are willingly taking this tradeoff, _and_ that the
losses aren't what I see as immoral actions, it then becomes rather nasty that
the casinos are dragging people to court and putting them in jail for it.

In short: you and I generally work within the framework of the law and make
tradeoffs accordingly. The casinos appear to be bypassing that and are bending
or modifying the law so they don't have to make tradeoffs.

------
josu
It's nice to see a real life example of the prisoner's dilemma going against
the Nash Equilibrium:

>Prosecutors had a weak hand, and they knew it. As a December 3, 2013, trial
date approached, the Feds made Kane and Nestor separate but identical offers:
The first one to agree to testify against the other would walk away with five
years of probation and no jail time.

>The old gambling buddies had one more game to play together. It was the
Prisoner's Dilemma. Without speaking, they both arrived at the optimal
strategy: They refused the offer. A few months later, the Justice Department
dropped the last of the charges, and they were free.

~~~
fineIllregister
I don't think the Nash Equilibrium applies here. The consequence of defecting
has to be better than cooperating, regardless of what the other player does.
Here, you have to consider the probability that prosecution would be
successful, which did not seem likely.

~~~
josu
So the scenario is:

If you don't defect -He defects first: You may or may not go to prison -He
doesn't defect: You may or may not go to prison

And if you defect first -He defects second: You don't go to prison -He doesn't
defect: You don't go to prison

Still seems to me that the Nash Equilibrium is to defect. Not going to prison
is better than probably not going to prison. According to the article, they
still don't talk to each other, so it's not like the outcome would have
changed much.

~~~
lovemenot
That was probably the logic the Feds used when designing this game. Actually,
kind of surprising that Nestor did not go for it, based on my assessment of
his character from this article.

~~~
james1071
More of a bluff than a prisoner's dilemma.

------
giarc
>The concept was proven in 1995, when one of the GCB's own staffers, Ron
Harris, went bad. Harris modified his testing unit to covertly reprogram the
EPROMs on the machines he was auditing. His new software commanded the machine
to trigger a jackpot upon a particular sequence of button presses—like a
Konami Code for cash.

This is actually a really interesting story that I saw on one of those
"Vegas's Biggest Criminals" type shows. This Ron guy was given the job and
created a device that would plug into the slot machine and check for
tampering. He programmed it to check the device, mark it as clean, then insert
his own code. This code allowed him to instantly win jackpots. He would have
to bet a specific combination to open the jackpot. For example he had to bet 1
credit, bet 3 credits, bet 4 credits, bet 1 credit etc. I believe the sequence
was like 50 steps long. He would obviously have friends play the machine to
win.

Benefits to this system was that his team could spread out among many
machines. If the casino wanted to check the machine, they would send Ron in
and the machine would come up as clean (he developed the software to check
it). The sequence was so long that it would take someone a lot of work to
recognize it, and no one was likely going to stumble upon it by accident. It
was really genius, however I can't remember how he got caught.

~~~
CanSpice
According to Wikipedia, Harris's slot machine hack wasn't discovered. He
turned to Keno, finding a flaw in the pseudorandom number generator, and when
his accomplice went to cash in a high-value winning ticket in New Jersey, the
casino got suspicious, alerted investigators, and everything came out after
that.

~~~
giarc
Interesting. Sounds like a lot of these people get caught when they either get
too greedy, or when they try to expand too much.

------
vinhboy
As much as I hate how casinos deal with people who find loopholes in their
games, sitting down and winning "seventh jackpot in an hour and a half" is
pretty silly.

Everything in moderation... But I guess you can't really expect that from a
gambling addict.

~~~
joshdick
I couldn't help but think that I would've handled this so much better than
these two knuckleheads.

You don't need tens of thousands of dollars a day. Just go in to win one
jackpot, lose money on some other games to lessen suspicion, and then go home
with a few thousand dollars in profit. You could probably keep that up for
ages without anyone connecting the dots.

~~~
seanalltogether
The article made it clear that most casinos don't enable the double up feature
by default. If you tried to play the long game, you'd end up interacting with
the same high roller employees week to week to enable the feature, and
eventually arousing suspicion.

~~~
swang
You underestimate the weird things high rollers demand to order to "increase"
their luck. Turning on some feature that is usually turned off is pretty low
on the list of suspicious things high rollers would do.

------
S_A_P
Maybe this is a moral failing on my part, but I am still trying to figure out
what was "wrong" about this. They found a bug. They exploited it. They asked
other casinos to enable a function that allowed them to exploit it, but it was
still a valid configuration for the game.

To me this sounds like IGT should have lost their shirts, but instead the NGC,
local police and the IRS can recoup the loss from the individual and emerge
unscathed?

I still cant see how this is wrong. But then again I think if you are good
enough to count cards in blackjack you should be able to profit as well...

~~~
MBCook
If he found a flaw in the poker simulation, I would agree with you. Cases like
that have come up.

But instead he's retroactively changing the size of his bets. He plays the
machine so he wins $50 then uses the bug to change the payout to $5000. It's
like writing extra 0s on a check someone gave you.

It's an interesting bug, but I don't think he deserves his winnings. He was
clearly playing outside the game and cheating.

That said he did _NOT_ deserve to be SWATed. That's a massive overreaction.

~~~
S_A_P
Excellent point, and I do see that as dishonest somewhat. I'm actually amazed
they didn't exceed statistical payouts on just dumb luck alone.

------
d4vlx
I worked on online casino software for several years and we had a number of
bugs like this. They would usually involve someone writing a client to hit our
API directly and fiddling with the xml messages. The operators would usually
catch the issues within hours, turn off the game and often deny the players
their winnings, if it was a legitimate win. Several people wound up in jail
for exploits.

In the early days of the company it was a fairly significant problem with
incidents pretty frequently. The lack of QA and a release process really bit
them a few times as well. A game was put into production with the result hard
coded to a win. Goes to show that just because a company makes software that
deals with money and is financially successful it does not mean they are at
all competent. It took 9 years for them to turn things around and transform
the company into a highly effective dev shop, at least by industry standards.
I am proud I was part of that. Unfortunately the industry had some major
problems in 2009-2010 and they ended up having to find a buyer who even more
unfortunately does not appreciate or understand developers or software
development.

We did integrations with many other gambling software companies and not a
single organization was what I consider competent. I would love to see a new
company with serious technical chops break into the world of online gambling
and school the crusty behemoths. Hard to do however because of the network
effects and importance of reputation and deal making.

------
cesarb
The article is missing the most interesting part, for me at least: how the bug
worked. Not from the outside (the article has the sequence of steps to exploit
the bug), but from the inside. What change in the control flow the "Double Up"
feature did which caused it to remember the previous payout and multiply it by
the new denomination?

~~~
minimaxir
Hypothetically, if pressing DoubleUp forces a recalculation of the payout
immediately, that would create a vector for a timing bug.

~~~
cesarb
From what I could gather from the article, the bug happened even if the user
never used the Double Up feature. But the bug disappeared if that feature was
disabled.

My guess would be that, for the Double Up feature to work, the program saved
some state (or delayed clearing that state), so it could double the payout
(I'm guessing that the "Double Up" feature is something used after a win,
something like "flip a coin to choose between 2x or 0x payout"). Some other
part of the program expected that state to be cleared. Without the Double Up
feature enabled, the code skipped saving that state (or cleared the state
earlier), which is why disabling that feature prevented the bug.

That is, something like the classic "MissingNo." bug in Pokémon, where there
was a strip of terrain which didn't setup the random encounter table, and the
space for that table was also used as temporary space to copy the player's
data during an event in the game. Trigger that event, go to that strip of
terrain, walk until you get a random encounter, and what was on the player
data is used as the encounter data.

------
samsolomon
It will be interesting to see what happens to casinos in the coming years.

Most people under 40 don't play slot machines, which account for a large
portion of casino's revenue. Free-to-play games such as Candy Crush are far
more interesting to younger players.

Vegas casinos will be fine, because gaming isn't the leading source of revenue
for them. However, regional casinos depend on slots for revenues. As gambling
become legal in more and more states those regional casinos will also have
more competition.

~~~
JoblessWonder
Wheel of Fortune slots are __insanely __popular with college-aged women. Or at
least were 5 years ago.

The barrier to entry for slots is significantly lower than games like craps or
poker. That is also why you see so many novice gamblers playing roulette. You
can figure it out right away.

------
GigabyteCoin
His only mistake was slaying the golden goose. That and immediately telling an
aquaintance what he was up to.

What was he thinking winning 8 jackpots in under 2 hours? He of all people
should have known that was impossible and incredibly suspicious.

He could have made millions and probably got away with it if he just spaced
out his winnings between a few years rather than a few hours.

~~~
enraged_camel
Addiction tends to get in the way of rational thinking.

------
rodly
These guys did this all wrong. They could have kept this going indefinitely if
they found a team of people who all agreed to go in and play these machines at
random times, at random casinos and never together. Better yet, rotate the
people so they never go to the same casino within 6 months of their last visit
so they don't get flagged for winning a jackpot every month.

Just send someone in, play for a bit and then win a jackpot and leave. $1.8k
to $10k jackpots so ~$6k/casino per session per person.

You could easily get everyone to an annual $50-100k/year salary with very
little play and be under the radar forever.

You would have to use people you trust since it's naturally a greedy endeavor.

I'm sure there are holes in my system, but those would be filled with a group
of 10 people analyzing and iterating on it every week quite easily.

~~~
jessaustin
The larger a conspiracy, the more doomed to failure. There is no way a couple
of schlubs like these two could recruit eight other people to follow their
directions precisely and not get greedy. The first guy couldn't even recruit
_one_ person he could convince to do that!

Perhaps a charismatic religious leader could inspire enough loyalty among
brainwashed acolytes for such a scheme, but that's about it. Normal people are
greedy.

------
Scuds
IIRC there's an episode of the .NET Rocks podcast where games in F# (roughly
oCaml for .NET) is being used as a backend for these gaming systems.
[http://www.dotnetrocks.com/default.aspx?showNum=846](http://www.dotnetrocks.com/default.aspx?showNum=846)

That bug probably wouldn't have existed if the code had been written in a
functional language where state management was an inherent part of the whole
deal.

Then again, some of these video poker systems look like a glorified Sega
Genesis, and probably run under similar hardware, too.

------
j_s
The last discussion of this case on Hacker News (May 2013):

 _Use a Software Bug to Win Video Poker? That’s a Federal Hacking Case_

[https://news.ycombinator.com/item?id=5638894](https://news.ycombinator.com/item?id=5638894)

------
jredwards
I love the unexpected last line.

~~~
genwin
The SWAT team is on its way.

------
smaili
Is there a reason why these guys couldn't sue the casino, the machine maker,
or both?

------
lifeformed
Casinos should have bug bounties.

~~~
MBCook
I wonder if they would have. If he had contacted the company (via a lawyer)
and said "I can show you a bug where X happens for a nominal fee" I wonder if
they would happily given him some fair sized sum of money.

But he seems to have been a gambling addict, so used it himself.

~~~
FireBeyond
Possibly.

Or they'd have the FBI come after him for attempted extortion.

------
serve_yay
Play to win! (Unless you win -- we can't have that.)

------
tvhiggins
you can be arrested for 'suspicion of theft?' Am i missing something or is
this illegal? Security in casinos have the legal right to detain you?

~~~
mcherm
Nearly all arrests are for "suspicion" of some crime -- until you have plead
guilty in court[1] it's always suspicion. And anyone has a right to make a
citizen's arrest, while many security guards have limited police powers just
within the private property they guard. With any of these, they can only hold
you long enough to hand you over to police.

[1] Or, I suppose, found guilty by a judge. But that only happens with a
miniscule portion of our legal cases in the US courts -- it is ignorable as an
obscure special case.

------
obamasupporter
yikes

