
Costco's Thanksgiving Day Website Crash Cost It Nearly $11M - thomk
https://www.thestreet.com/technology/costco-thanksgiving-day-website-crash-cost-it-nearly-11million-15185344
======
fma
The key part to this article is at the very end, and not addressed in the
title (of course)....they extended their sales till the next day. Anyone who
couldn't place an order (like me) did it later. So I'd argue it did not cost
Costco $11 million...

~~~
purple-again
Most likely only a small fraction of the people who were going to make a
purchase heard this news and only a fraction of them decided to follow through
and purchase the next day.

~~~
arcticfox
That's not actually true, since the website had a message saying it while
suffering. I also finished my purchase the next day.

~~~
orev
You can’t really say “it’s not true” since you need facts to make that kind of
statement, and I doubt you have access to the data one would need to come to
that conclusion.

It’s very easy to postulate that people would go to another site (e.g. Amazon)
immediately instead of waiting around to see if the Costco site would recover,
especially since most BF deals are limited by both time and quantity. That
alone makes the loss estimates feasible.

~~~
arcticfox
> Most likely only a small fraction of the people who were going to make a
> purchase heard this news

This is the part I was referring to as "not actually true". Fact: the site
itself, which was loading although not working entirely, displayed a big
banner saying this during the problem periods. It was hard to miss.

The rest is indeed just speculation either way. I doubt the losses are as high
as estimated, but there's no way to say without inside data.

------
tedunangst
So 0.00755% of their annual revenue? Do you think they'll ever recover from
the loss?

------
PretzelFisch
Costco's site has never been a role model in fact it can hardly handle most
weekend traffic. This annoys me on multiple levels since it puts a bad name to
asp.net's ability to scale as well as the fact weekends are my main shopping
window and I just can't use the site.

~~~
paranoidrobot
Does ASP.NET have a bad rep for poorly scaling? I'd have thought it was just
the licensing/overhead costs of running Windows that turns folks off.

This should, hopefully, change for the better with .NET Core getting wider
adoption.

~~~
gabrielrdz
It should not. StackOverflow.com, for example, runs on ASP.NET MVC and is one
of the websites with most traffic in the world. Also blazing fast and rarely
goes down. Even in Black Friday!

~~~
flavius29663
also, once you open a web browser with the website, it keeps an open
connection to the server. Their servers handles all the open concurrent
connections

------
ram_rar
Would love to see some postmortem from the tech folks @ Costco. What really
got them down. Also, does anyone know if they use any cloud services or run
their own DC?

~~~
9nGQluzmnq3M
Costco is an Azure customer, but I'm not sure where the site in question is
hosted.

[https://www.reuters.com/article/brief-microsoft-says-
costco-...](https://www.reuters.com/article/brief-microsoft-says-costco-
recently-cho-idUSFWN1N11XX)

------
llamataboot
I would bet that quite some of these people came back later. When people want
Costco prices, they will come back. So prob not that harmful. Still never a
good day to be an engineer and imagine there will be some difficult post-
mortems.

~~~
_nickwhite
Small sample size of my household, be we bought nearly everything we were
going to buy at Costco elsewhere due to them being basically offline all of
Thursday AND into Friday.

Seems like they host all of their web infrastructure in-house and need some
major help on scale-ability.

------
maaaats
One of the biggest electronic stores in Norway knew their web page couldn't
handle the load, so they had some kind of queue system im front. When I
checked, my spot in line was estimated to have to wait more than an hour. They
got a lot of flak for it, all other web shops worked fine.

~~~
fpgaminer
I wonder how much engineering effort went into building that queuing system,
versus what it would have cost them to just fix their site so it could scale
properly...

~~~
hombre_fatal
On the other hand, how much hype and commitment does it build for people who
decide to sit in queue?

Maybe by the time you sit in queue for an hour, you're more likely to make
more purchases than you would have otherwise. Or people feel like they're
missing out by shopping on Amazon when they can claim a spot in queue.

We're not all that rational.

------
drak0n1c
During one of the sudden logouts on Costco's website yesterday there was a
complete stranger's email address prefilled in the login screen.
Disconcerting, but at least there was no password.

------
friedman23
I was browsing Costco.com when I saw this happen and the thing I was most
thankful for on Thanksgiving was not being on-call on the responsible team at
Costco.

------
mogadsheu
$11M isn’t trivial, but for a company that’s got billions in turnover it’s not
significant either. Top brass will say “Let’s fix that” and move on.

~~~
hinkley
How many people do you suppose can all get a promotion off of figuring out to
make Costco an extra $11M?

------
vinniejames
$11 million based on what metric, they can't exactly quantify all the people
who tried to shop when the site was down, then simply came back later to make
the same order

------
badrabbit
TIL: people buy stuff from costco's website.

I alwayd thought since you buy in bulk shipping would be insane or not at the
value point of amazon.

~~~
OrgNet
Bought my mattress online from Costco and I'm not even a member...

~~~
badrabbit
Nice! I gotta check it out sometimes

