
Etsy Announces Move to Google Cloud - xref
https://investors.etsy.com/news-and-events/press-releases/2017/12-14-2017-130416690
======
foobarbazetc
I love GCP but honestly have no idea how these big internet properties can
make the $ case for “the cloud”, unless they’re also planning to fire 90% of
the Ops/SysAdmin team (ala Evernote).

We run on bare metal and to move to GCP would triple our hosting cost, not
including bandwidth (which is free* for us).

Do people get discounts for doing PR? I’m really curious.

~~~
geofft
I don't have very detailed knowledge, but my impression is that this move
doesn't make long-term financial sense for the company, but it's being pushed
by activist investors / board members who want to appear like they're good
stewards of money because investors think cloud is cheaper / etc. (Note that
this is on investors.etsy.com, not their tech blog; you can also find a call
for public cloud in the Black & White Capital letter, in the NYTimes story
from a couple weeks ago, etc.) I heard some rumors about this when I was
applying for a job right before the layoffs happened, and I didn't hear anyone
expressing interest in moving out of their datacenters _before_ the layoffs.

~~~
foobarbazetc
Yeah. The exact same thing happened at Evernote. They hit a sales slump, new
“corporate” management, cuts etc -> GCP.

The Ops people there were pretty proud of their infrastructure, but they took
it to a pretty extreme level. I think they were racking their own stuff and
even running their own routers etc.

~~~
lykr0n
Racking your own servers and running your own routers is extreme? I don't know
about you, but I haven't set foot in the DC in two years after initial setup.

~~~
foobarbazetc
Yeah I meant to put it in quotes. :) Hard to find B2C type SaaS companies
doing that though (with the exception of Pinboard, who’s crazy :)

We actually haven’t had a hardware failure (apart from HDD that have all been
replaced with SSD) in something like 7 years.

We don’t even pay big bucks for Dell, HP etc. Just cheap SuperMicro boxes.

~~~
amigoingtodie
What does your hardware and software stack look like?

------
gregmac
Found some interesting contrasting statements from 2015 [1]:

> The company performed big data jobs early on in the cloud, using Amazon's
> Elastic Map Reduce service.

> “That [EMR] got us to a place where we think about what data we collect. At
> the time it was really inefficient from a cost perspective, but there
> weren’t any creative constraints,”

> The team eventually decided to bring it in house. “Our Hadoop cluster wields
> a huge amount of flexibility," said Rembetsy. "We can get a lot done on bare
> metal, and we know very much what we’re looking for. Cloud services, that’s
> where efficiencies fall away. Spin up and spin down doesn’t have the same
> value proposition.”

[1]
[http://www.datacenterknowledge.com/archives/2015/05/11/how-e...](http://www.datacenterknowledge.com/archives/2015/05/11/how-
etsy-optimized-data-center-infrastructure-for-efficiency)

~~~
interskh
EMR is really expensive, especially you are using spot instances. It is
roughly $2 for the EMR tax on $1 instance

------
code_duck
Sounds good. I was an early Etsy seller, as a glass artist and bead vendor, As
well as a developer who worked with them as closely as I could for a few
years.

They have always seemed a bit disconnected from their customers. The first
regime, Rob Kalin, Haim and Revolving Dork, had an age and culture disconnect
from most of their customers. The next group, headed by Chad Dickerson and
Allspaw, seemed like they wanted to focus on technology for technology’s sake,
more than what Etsy was actually doing with that as a company. They even made
some statements to the effect of how they wanted Etsy to be known as a
technology company, like Facebook, and be a platform, like Amazon.

All fine and good, but so far, what all of the leaders of Etsy have had in
common is that they don’t really understand what it’s like to be a micro
business person selling handmade art online. If I was an investor, I wouldn’t
want to be paying for a bunch of guys to experiment with Mongo DB search
engines and shards. As long as they’re still not understanding their core
business, it’s a distraction.

Etsy should be focused on one thing: connecting buyers and sellers of handmade
art with each other.

~~~
BlanketApple
Disclaimer: I worked at Etsy, a long time ago.

> The next group, headed by Chad Dickerson and Allspaw, seemed like they
> wanted to focus on technology for technology’s sake, more than what Etsy was
> actually doing with that as a company.

I think you're missing quite a bit of this story. I was around during the
transition of Chad from CTO -> CEO, and the truth is that Etsy needed to focus
on the technology badly during that time. The site would go down constantly.
We had very little insight into what was going on with sellers or buyers.
Every busy day was basically us crossing our fingers that the site wouldn't go
down.

We needed someone focused heavily on the technology. We could never have
continued on that path and grown as much as we did.

> Etsy should be focused on one thing: connecting buyers and sellers of
> handmade art with each other.

Deciding what potential buyers are interested in and what items to display to
them is a pretty difficult technology problem.

Detecting mass-produced goods in a way that's more subtle than "ban all
Chinese sellers" is a pretty difficult technology problem.

If anything I would argue that Etsy didn't focus nearly enough on that, and
focused way too much on "low-hanging fruit" (minor workflow adjustments for
sellers and buyers).

~~~
code_duck
Agreed, there is a whole lot more to the story that sadly this comment area is
too small to contain. Dickerson and Allspaw definitely did a great job of
modernizing Etsy from the relatively ragtag team that operated it prior, and
it seems pretty robust now.

That’s great – for their engineering department. What about the guiding vision
for their core business? It’s not enough to make IT top notch when their
overall aim is floundering. As a customer and developer, I frequently felt
that their communications and decisions missed the mark and heard this from
fellow customers continually. Etsy needs to be run by artists and retailers,
not technologists.

Part of the problem is that they want to address the things like eliminating
mass produced goods as technology problems, to be eliminated with the right
algorithm. I think it risks turning the cozy marketplace into some automated
hell like Google or Amazon.

~~~
BlanketApple
> That’s great – for their engineering department.

Well, no - it's great for you too. Nobody can buy anything if the site is
down. First-time buyers are especially sensitive to that. If their first
experience with Etsy is a timeout page they won't be back (we actually had
numbers asserting as such). Amazon famously wrote that 100ms of additional
latency cost them 1% of sales, and Etsy is very similar in that regard.

> What about the guiding vision for their core business? It’s not enough to
> make IT top notch when their overall aim is floundering.

The guiding vision was, and likely still is, to have Etsy be what it is today
(but with much better execution, obviously). Amazon with a more hand-made
slant to it.

I saw this frequently when I was there - sellers want Etsy to be more like
ArtFire or ArtYah is now, but that's not what Etsy's leadership and funders
want Etsy to be. They want Etsy to be Amazon, but for unique items.

Now sure, that's not what Rob Kalin wanted it to be. But that vision was lost
years ago, way before Chad et. al took over.

> As a customer and developer, I frequently felt that their communications and
> decisions missed the mark and heard this from fellow customers continually.

No argument there. The decisions usually made sense internally, but the
external communications were poor. I would not be surprised to find out it's
part of why the C-suite was gutted. I'm really sorry you had to be a customer
during that phase. Seriously.

> Etsy needs to be run by artists and retailers, not technologists.

They tried that. Rob Kalin is an artist, through and through.

Turns out artists don't always make great business leaders. And let me tell
you, definitively, Rob Kalin would have ruined that company had he stuck
around for much longer than he did.

> Part of the problem is that they want to address the things like eliminating
> mass produced goods as technology problems, to be eliminated with the right
> algorithm.

It is a technology problem. Etsy can't possibly afford to hire enough people
to sift through items manually looking for mass-produced goods. They'd go
bankrupt. The marketplace is just way too big for that.

Etsy can remain a small marketplace and do that, which is I'm guessing your
preference, but that's not the preference of the VCs who backed Etsy.
Unfortunately, money has the final say.

~~~
code_duck
I realize that snappy pages help, as an engineer myself (who often depended on
Etsy's API responses...). I appreciate the great job they did taking Etsy from
the substandard heap it was circa 2009 to where it is now. My point is that
Etsy needed more of a guiding light than their engineering department
providing effective technology.

Rob Kalin was not an artist - what was his art, Swimmy? BS? A couple wooden
computer cases? In my perception he was a con-man hipster who dabbled in
craft. I have no praise for him or anything he did, and am not suggesting his
vision, whatever it was, or certainly his management, would have been better.
I believe he is the one who imbued Etsy with a toxic culture from which that
they are still suffering. One inept conman doesn't disprove 'artists' being
management, and that's not what I called for - someone with retail experience
would be nice, ideally online as well as the typical setting where Etsy
members sell their wares offline like art galleries, craft fairs, jewelry
stores, boutiques, and so forth.

All the time I've dealt with Etsy, I've heard the excuse that it's impossible
for them to police their market place in any sort of pro-active way. But the
sellers in each category have no problem finding obvious violations in their
categories every day, and reports have never been dealt with logically and
promptly. I, and others, gave up reporting mass production and resellers after
it seemed futile due to lack of response, or even Etsy would claim items that
seemed like clear violations were not violations.

I don't have a preference for Etsy's future, as I gave up attempting to use it
years ago and have recommended to my friends and clients for years that they
buy a domain and set up a shop they actually control, using shopify or big
cartel to make it simple.

~~~
BlanketApple
> My point is that Etsy needed more of a guiding light than their engineering
> department providing effective technology.

They had that, though. Amazon for unique items.

I think that's just not the guiding light you wanted for Etsy, but it's been
there for a long time. They were missing the technology ~2009 up until a
couple years ago. It was absolutely the most important problem at Etsy at that
time. Now? I'll agree, not so much.

> Rob Kalin was not an artist - what was his art, Swimmy? BS? A couple wooden
> computer cases? In my perception he was a con-man hipster who dabbled in
> craft.

I have no love lost for Rob Kalin as a CEO, and I would never work in a
business run by him again, but this is an extremely unfair characterization of
him.

He's an artist. Sorry. To call him otherwise is to pull a "no true scotsman."

> someone with retail experience would be nice, ideally online as well as the
> typical setting where Etsy members sell their wares offline like art
> galleries, craft fairs, jewelry stores, boutiques, and so forth.

Etsy had plenty of people like that in senior leadership roles. Not the C
suite, sure, but high up there.

I also don't see any reason why Etsy needs people like that, to be honest.
Bezos had little (if any) retail experience before starting Amazon. Clearly
hasn't impeded them any.

> All the time I've dealt with Etsy, I've heard the excuse that it's
> impossible for them to police their market place in any sort of pro-active
> way. But the sellers in each category have no problem finding obvious
> violations in their categories every day, and reports have never been dealt
> with logically and promptly.

You're grossly unaware and underestimating of the volume of support emails
Etsy got (and probably it's even worse now).

Sellers outnumber Etsy employees by a godawful amount. Reports outnumber Etsy
employees by a godawful amount. Not sure if it's still the case, but we used
to have so many reports that we needed employees to all go on rotations where
they answered emails and went through reports.

I personally must've gone through close to 1,000+ support emails and reports a
year during my time there just doing these support rotations. The full-time
community staff probably handled that much each day, if not more.

Even with all that, I bet we answered _maybe_ 25% of all reports and emails.
And this was way, way before the IPO.

> I don't have a preference for Etsy's future, as I gave up attempting to use
> it years ago and have recommended to my friends and clients for years that
> they buy a domain and set up a shop they actually control, using shopify or
> big cartel to make it simple.

That's fine, but I think it's naive to assume Etsy's future would look
anything other than this. Actively managing a marketplace of Etsy's size is a
technology problem.

You can bring up Myspace/etc. all day, but don't forget that in addition to
monitoring for resellers and mass produced goods Etsy also has to watch out
for:

1) Illegal activity e.g. money laundering 2) Pornographic/inappropriate items
3) Scammers 4) Legitimate support complaints (missing items etc.) 5) General
complaints (I don't like this new feature, etc.)

Support staff monitoring all of those numerous issues would bankrupt Etsy,
even if they did farm it out to a developing nation for pennies.

~~~
code_duck
Thanks, I’d love to respond to this when I have the chance.

As far as Rob Kalin, you didn’t answer the question - what is his art? What
medium? I’m a member of a group that doesn’t have to stretch, claim or pretend
to be artists, since we make things out of hot glass all day and it’s really
obvious.

I find it odd that you’re convinced he’s an artist, but I’ve never seen any of
his art. Are you referring to his personality? Does he actually create craft
or art? Was he ever self employed to trying to sell it for a living?

A couple of times here, you are using outliers to try to prove broad
principles: Kalin proves that artists wouldn’t know how to run Etsy, and Jeff
Bezos proves that a retail background doesn't help to build a retail website.
Hmm, I would be hesitant to draw conclusions like that.

Overall, Etsy is irrelevant to people like me, who were the people that it was
nominally created for originally. Obviously, it was taken over by investors
who don’t care at all about the original mission. What I’m still judging at by
is the original intent, which they never managed to achieve.

------
stareatgoats
Not much meat for the technically interested here unfortunately. The
respective business departments were clearly in charge of this press release.
Which was a business mistake in my opinion: these press releases are primarily
interesting for the CTOs and similar - the fluff presented here would serve to
remind them of how they are kept at arms length i many board rooms even in
their areas of expertise, more than anything. My guess is both Google Cloud
and Etsy would dip in their opinion as result of it.

------
josephhurtado
It begs the question why Google Cloud and not MS Azure or Amazon's AWS?

Could it be a price break they got, and also the AI advantages of Google
Cloud's offerings? I am sure that played a big part.

~~~
bckygldstn
Many retail companies avoid AWS because they see Amazon as a competitor.

Source: work for a retail company, we went with Azure.

~~~
TheReveller
I think that certainly comes into it. Seems perverse to take the money you
make from sales and give a cut of it to your competitor, who can then reduce
the prices in their own store.

~~~
billmalarky
That and there's a trust issue I'm sure, regarding hosting all your data with
your competitor.

------
codingdave
> Etsy's transition to the Google Cloud Platform is expected to be complete
> within the next two years.

I have no idea what Etsy looks like internally, but that sounds like an
awfully long time in this day and age. Anyone have an inkling as to why this
is a multi-year project?

~~~
sitharus
Having worked for two companies that have shifted to cloud providers, multi-
year timescales is fairly usual even just for like-for-like migrations. You
can’t just shut down the servers, image them and start them in the cloud, you
need to have a plan to progressively migrate services while maintaining site
operation.

~~~
spyspy
Truth. This[0] is how we did it for the NYTimes Crossword platform. It took a
year and a half, but was managed with a team of only 2 people (myself being
one of them).

[0] [https://open.nytimes.com/play-by-play-moving-the-nyt-
games-p...](https://open.nytimes.com/play-by-play-moving-the-nyt-games-
platform-to-gcp-with-zero-downtime-cf425898d569)

------
clintonb
Did they use another provider prior to the move, or self-host?

~~~
whitepoplar
Last I checked, they hosted their own bare metal machines.

~~~
jasonlotito
Correct. I've seen their cages.

------
_Codemonkeyism
I would love to see a cost analysis of a company moving to GCP/AWS.

We were a 6M customer flash sale company sold for $200M. We had 3 devops
engineers mostly helping developers with designing the system and pager duty -
mainly because 24/7 was to demanding for only 2 people. We were SOX/PCI
compliant.

Hosting costs all in all by year were around the same as the salaries I'd say.

People who moved to AWS/GCP, would I save costs in this scenario moving to
AWS/GCP?

------
vpol
I’m not even sure Etsy will exist in two years.

~~~
Someone1234
Based on this alone?

Etsy seems to be fine for the most part. The only criticism I have of Etsy is
that they've moved away or diluted their core business (small business/home
business sales) by allowing mass manufactured goods to be sold on the site.

Etsy still has a place, and I've ordered several items from Etsy vendors this
last year. They just need to keep a check on what they want to be otherwise
they'll be subpar eBay clone.

------
damm
Okay so how much did they get for free by moving to Google Cloud? That's the
question you should be asking.

Etsy has been hurting; why would they move infrastructure now unless t hey got
kind of kickback for the PR?

This isn't an easy move; so there has to be a huge benefit to them.

------
CaptSpify
Another DC to cloud story! I love these!

It's like watching a play in 3 acts:

Act 1) Yay, we did it, we're gonna save so much money!

Act 2) [a year later] Wait, we're spending how much money?!?

Act 3) It's going to cost us how much to leave our current cloud?!?

Haven't people figured out yet that cloud companies make a profit? Sure,
companies like Etsy can get a _huge_ discount, but everyone else who isn't a
big name is going to pay out the ass.

~~~
btian
That's not how any of this works.

Boeing makes a profit, but how much money will it cost to design and make a
plane from scratch?

~~~
CaptSpify
> That's not how any of this works.

Weird then, considering that I've seen >5 companies go through that process.
Something that you say doesn't work one way clearly does.

Planes are _vastly_ different than data-centers. Depending on what you need, a
brand new data-center can run as little as a couple hundred dollars for a
server + parts, and <$30 / month on everything else. What is the cheapest
plane you can buy?

------
merb
wasn't etsy the company that was using amazon and shifted back to their own
infra because they said cloud is not as cheap as own infra?

~~~
illumin8
No, they've always self-hosted on bare metal.

~~~
stfwn
What's the difference between self-hosting and self-hosting on bare metal?

~~~
Someone1234
Self-hosted could mean running on a rented server managed by a vendor (e.g.
Rackspace) often marketed as a "private cloud." The servers they're using are
individually assigned to that client, the vendor manages the physical, the
company manages every other layers.

Self-hosted on bare mental usually means the company owns the physical
servers, storage arrays, and so on. No renting or leasing. No subscription
fee-like service. They may still be renting space in someone else's data
center, but it is their hardware.

The distinction isn't super important to be honest.

