Hacker News new | past | comments | ask | show | jobs | submit login
Costco's Thanksgiving Day Website Crash Cost It Nearly $11M (thestreet.com)
107 points by thomk 7 days ago | hide | past | web | favorite | 54 comments

The key part to this article is at the very end, and not addressed in the title (of course)....they extended their sales till the next day. Anyone who couldn't place an order (like me) did it later. So I'd argue it did not cost Costco $11 million...

Anecdotally, my step mom was trying to buy 2 apple watches from Costco and gave up after an hour of trying to get through the cart. I realized that Amazon had them for the same price, so she bought it from there instead.

So there are at least some lost sales by people shopping elsewhere. It's rare that a specific price is only offered by only one retailer for a product.

Problem is Costco high price products came with 5 year warranties so it’s a nobrainer to just first try going to the store then ordering online. I suspect a lot of people did that prompting even more sales (I bought house stuff I didn’t even need, I was just mad they didn’t have my tv and I drove all the way out there)

When I’m doing Christmas shopping I have no tolerance for malfunctioning websites, I just jump over to some similar service with similar pricing

> When I’m doing Christmas shopping I have no tolerance for malfunctioning websites, I just jump over to some similar service with similar pricing

I suppose the difference with Costco is the sunk cost (membership fee) that means you're probably more likely to want to get your money's worth.

It's not just sunk cost fallacy. Everything I've bought from Costco has been at least "good" with most of the products pushing into "great" territory. The curation is one of the benefits of the membership.

Especially in the era of Amazon shoving counterfeits every which way for every conceivable product, it's nice to have at least one retailer that isn't just hawking shelf space to anyone with a dollar and crap to sell.

I certainly resonate with this. In California the CostCo membership pays for it self with gas purchases so there isn't as much 'sunk cost' as you might think. Their product quality, and return policy has consistently impressed me as Amazon has gotten worse at this.

If you need to replace your car tires, you sink that membership several times over in one shot.

Everything afterward is just a plus.

Ehh their tire selections suck for some cars though. If you’re buying dirt cheap with okay performance tires, sure they’re great. But discount tire wins the cheapest for oem tire shop (sorry y’all, I love my pirelli’s)

Kirkland Batteries are a win once again if they have your car size stocked

Costco members are pretty loyal and known to be so. They offer cash back as an incentive.

Exactly. Stories about websites losing $N per minute tend to be examples of how to lie with statistics.

It's a cost model, they probably have a coefficient for what % of people they think will wait vs. go elsewhere while the website is down. When you're running a retail company, you have to at least attempt to quantify lost sales in order to model your business.

There's no way to measure lost sales with high accuracy, but that's not really the point, because it's most useful for modeling decisions and policies in a forward-looking context. Future revenue forecasts, the other term in the equation you're optimizing, are also inaccurate.

These functions are also generally "bathtub" shaped, and with the amount of money at stake, you can save quite a lot by being a little closer to the optimum, so they're worthwhile.

Yeah some people determined like me probably tried over and over until it succeeded, it was a bit scary clicking refresh during the actual payment (although was monitoring my credit card charges, was prepared to cancel extra orders and at worst return extra stuff) , fortunately it went only once despite multiple refreshes until I saw the confirmation page.

Other people probably just went directly to stores.

Since they had the same issues running the site the next day as well It may still be close to that 11 million.

Perhaps. But I'd argue they lost more. That is, next time, many of those people will go elsewhere first and then Costco as a last choice option.

I suspect a lot of problems are like that.

Store too crowded? Website slow or down? Something out of stock? Business not lost, people just come back later.

Most likely only a small fraction of the people who were going to make a purchase heard this news and only a fraction of them decided to follow through and purchase the next day.

It took a lot of patience and persistence to get something added to my cart and complete the checkout, so I can easily see many people abandoning the process. Though they did send a reminder email about the abandoned shopping cart, so that might have helped get some people back.

I was looking at one of their fridges that was on Black Friday sale and couldn't get the page to load. I had no idea they extended until today.

That's not actually true, since the website had a message saying it while suffering. I also finished my purchase the next day.

You can’t really say “it’s not true” since you need facts to make that kind of statement, and I doubt you have access to the data one would need to come to that conclusion.

It’s very easy to postulate that people would go to another site (e.g. Amazon) immediately instead of waiting around to see if the Costco site would recover, especially since most BF deals are limited by both time and quantity. That alone makes the loss estimates feasible.

> Most likely only a small fraction of the people who were going to make a purchase heard this news

This is the part I was referring to as "not actually true". Fact: the site itself, which was loading although not working entirely, displayed a big banner saying this during the problem periods. It was hard to miss.

The rest is indeed just speculation either way. I doubt the losses are as high as estimated, but there's no way to say without inside data.

this might be true for products where costco was offering a much better price, but I'm sure there were also plenty of annoyed customers who just bought the item somewhere else for a slightly lower discount. presumably there is some way of approximating this ratio and using it to adjust these kinds of loss estimates. I guess lovethesales.com might choose not to adjust the figures in order to report a juicier story.

There doesn't even have to be a 'the item'. If someone spent their budget, or spent enough to feel guilty about spending more, they're a lost sale.

yeah, i always think the same thing when i see loss figures in situations like this...

So 0.00755% of their annual revenue? Do you think they'll ever recover from the loss?

Costco's site has never been a role model in fact it can hardly handle most weekend traffic. This annoys me on multiple levels since it puts a bad name to asp.net's ability to scale as well as the fact weekends are my main shopping window and I just can't use the site.

Does ASP.NET have a bad rep for poorly scaling? I'd have thought it was just the licensing/overhead costs of running Windows that turns folks off.

This should, hopefully, change for the better with .NET Core getting wider adoption.

It should not. StackOverflow.com, for example, runs on ASP.NET MVC and is one of the websites with most traffic in the world. Also blazing fast and rarely goes down. Even in Black Friday!

also, once you open a web browser with the website, it keeps an open connection to the server. Their servers handles all the open concurrent connections

Would love to see some postmortem from the tech folks @ Costco. What really got them down. Also, does anyone know if they use any cloud services or run their own DC?

Costco is an Azure customer, but I'm not sure where the site in question is hosted.


I would bet that quite some of these people came back later. When people want Costco prices, they will come back. So prob not that harmful. Still never a good day to be an engineer and imagine there will be some difficult post-mortems.

Small sample size of my household, be we bought nearly everything we were going to buy at Costco elsewhere due to them being basically offline all of Thursday AND into Friday.

Seems like they host all of their web infrastructure in-house and need some major help on scale-ability.

The problem was that is was a one day sale. And even though they extended the sale, that message was only posted on the website the next day.

One of the biggest electronic stores in Norway knew their web page couldn't handle the load, so they had some kind of queue system im front. When I checked, my spot in line was estimated to have to wait more than an hour. They got a lot of flak for it, all other web shops worked fine.

I wonder how much engineering effort went into building that queuing system, versus what it would have cost them to just fix their site so it could scale properly...

On the other hand, how much hype and commitment does it build for people who decide to sit in queue?

Maybe by the time you sit in queue for an hour, you're more likely to make more purchases than you would have otherwise. Or people feel like they're missing out by shopping on Amazon when they can claim a spot in queue.

We're not all that rational.

I’ve seen queue-it.com a few times. It’s a managed system that’s like Cloudflare: it sits in front of your site so you don’t need to integrate.

Now I see why microsoft is building so many Nordic data centers.

During one of the sudden logouts on Costco's website yesterday there was a complete stranger's email address prefilled in the login screen. Disconcerting, but at least there was no password.

I was browsing Costco.com when I saw this happen and the thing I was most thankful for on Thanksgiving was not being on-call on the responsible team at Costco.

$11M isn’t trivial, but for a company that’s got billions in turnover it’s not significant either. Top brass will say “Let’s fix that” and move on.

How many people do you suppose can all get a promotion off of figuring out to make Costco an extra $11M?

$11 million based on what metric, they can't exactly quantify all the people who tried to shop when the site was down, then simply came back later to make the same order

TIL: people buy stuff from costco's website.

I alwayd thought since you buy in bulk shipping would be insane or not at the value point of amazon.

No one is buying 50 rolls of toilet paper on black Friday LOL. They have electronics, tools, appliances like any other store...for great prices and lifetime return period (for non electronics).

But I do buy diapers and formula in bulk from online if I can anticipate I need it in a few days. The price is $1 or $2 higher online, but cheaper than elsewhere and more convenient if I don't need anything else from the store.

Bought my mattress online from Costco and I'm not even a member...

Nice! I gotta check it out sometimes


Amazon being down would be extra PR. Costco’a instore credit card machines went down. Costco’s website, nowhere near a leader in e-commerce, and you already have to be a Costco customer - their site doesn’t generate new customers. The site going down, doesn’t generate enough press for the name recognition increase to increase amongst of new customers (infact it does the opposite) offset the negativity around dedicated customers not being able to access Costco accounts.

Doesn't really affect the point you are making but you don't have to be a Costco member to purchase most items at their website. You will pay 5% more than non-members which can sometimes still be worth it.

The website says that, but I've never seen them actually charge the 5% for me.

They list the public price unless you're logged into your member account.

I've cross checked using a member account before; the price has always been the same.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact