
Update on Christmas Issues - dom96
http://store.steampowered.com/news/19852/?snr=1_550_552&utm_source=twitterfeed&utm_medium=twitter
======
whizzkid

       "There are only two hard things in Computer Science: 
        cache invalidation and naming things."
    
       -- Phil Karlton

~~~
calpaterson
This is a famous quotation but in my experience it is not true. I have
implemented web caching a few times and while some RPC-style forms/APIs have
been difficult I've never put a cache in and had it go seriously wrong. Naming
things is sometimes tricky but never profoundly difficult.

The hard problems are probably things like doing something concurrently or
maybe implementing the right feature...

~~~
adiabatty
It sounds like some people forgot a `Vary: Cookie` header while they were all
running around with their hair on fire because of the DDoS.

~~~
mst
I, and many other ops-ish people I know, cringed at the obvious caching error
that seemed like the most likely culprit.

The "in the middle of a DDoS" makes it make a lot more sense, and while this
is still a cringeworthy situation, the cringing at least on my part is now
more sympathetic than horrified.

------
Blackthorn
This is a pretty solid public postmortem. Props to Valve. Usually when
something bad happens in the gaming realm, it just gets swept under the rug.

~~~
pnt12
Valve is (was?) known for being silent too often - at least concerning Dota 2
(the diretide event 'scandal'). Maybe they're trying to correct that, after so
many complaints about their poor communication, which is nice.

~~~
CaptSpify
I think this was big enough that they _had_ to comment. Publicly leaked
personal information might be enough to start a lawsuit (IANAL). But I agree,
props to them for going against their SOP, and actually communicating.

------
Zikes
I really hope this catalyzes Valve to create a proper customer support team,
one that can publicly address these issues in a timely manner as well as
handle day-to-day customer support problems. It's ludicrous that a company as
big as Valve has got away with it for as long as they have, and that's due in
large part to an extremely devout fanbase that's only now starting to get fed
up with them.

~~~
prezjordan
I don't think the Valve fanbase is even close to being fed up with them,
unfortunately.

~~~
CaptSpify
Unfortunately, there's just no competitor big enough to force them to care

~~~
Zikes
gog.com is becoming really popular, and I actually had a really good
experience dealing with EA Origin's support earlier this week. Sure they're
not as big as Steam yet, but they're getting big enough to be competitive.

------
danso
Nice to see the update and I'm very interested in how many people they
determine deserve compensation, and why them (if there is any distinction
between severely affected and no-big-deal effects). The error seems
"basic"...in the sense that everyone who knows devops knows how to prevent it,
but according to their account, the circumstances were extreme (plus, it was
the holidays) and so mistakes were made. Just like how only a seemingly moron
of a surgeon could operate on the wrong limb of the wrong patient...but such
catastrophic mistakes often happen when the ER is in a state of chaos.

------
vive-la-liberte
>Valve is currently working with our web caching partner to identify users
whose information was served to other users, and will be contacting those
affected once they have been identified.

How will they be able to do that?

~~~
sciurus
Analyzing log files, I presume.

------
grogenaut
They're glossing over over this a bit. Many sites use the last 4 digits of
your cc number as validation so the last 2 digits + email + phone suffix setup
is valuable information and is definitely an addition to any cc database for
liveliness of credit cards or accounts to go after on steam.

------
nso95
Do people get fired for making these kind of mistakes?

~~~
batiudrami
Probably not, unless it's one of a series of similar issues. Hiring and
training people is expensive, and everyone screws up sometimes.

Firing people for a single mistake is for fast food restaurants, not for
billion dollar tech companies.

------
minimaxir
While an expected response, it's hard to forgive Valve for taking 5 days for
_any official acknowledgement of the issue_ (not even a Tweet on the Steam
Support Twitter accounts, which you would expect to be the primary source of
information), especially since personal information leaked.

To quote a NeoGAF moderator:

> To be honest, I think people would have been willing to give them 5 days
> (irrespective of what the professionally appropriate timeframe was) if their
> initial response wasn't so tone-deaf.

~~~
aeturnum
Valve did make an official statement to the press on the day of[0].

[0] [http://www.pcgamer.com/warning-steam-is-revealing-private-
ac...](http://www.pcgamer.com/warning-steam-is-revealing-private-account-
information/)

~~~
Zikes
Call it petty, but hearing their official statement secondhand makes it feel
disingenuous to me. They have more appropriate channels in place for that sort
of thing, including email, their two official Twitter accounts, and their very
own web site, which would communicate directly to the customers affected.

