
How Stack Overflow upgraded from Windows Server 2012 - nsoonhui
https://stackoverflow.blog/2019/07/22/how-stack-overflow-upgraded-from-windows-server-2012/?cb=1
======
plq
I will be honest and just say this: As a Linux guy who has negligible
familiarity with the Windows Server technology, this post doesn't make much
sense to me.

What is the Linux equivalent of this operation? apt-get dist-upgrade? Now that
Windows 10 onward is rolling release (is it?) is this the last time SE staff
is going to have to go through this?

This caught my attention:

> Putting the 2019 server into the existing 2012 cluster – as expected, this
> fails due to the operating systems being different.

How can a website be this tightly coupled with the operating system it's
running on? Shouldn't Microsoft be blamed that their operating systems are not
backwards-compatible? To my knowledge, Microsoft goes to extreme lengths to
assure backwards-compatibility of the userland apps. What went wrong here?

~~~
madiathomas
Windows 10 was designed for desktops, not as a server software. Windows Server
2003/2008/2012/2014/2019 etc is for servers and they don't subscribe to
Windows 10 rolling release. They have their release and support cycles. People
rarely upgrade.

I have upgraded a website from Windows 2003, to Windows 2008, to Windows 2012
and now it is running on Windows 2019. I jumped 2014 and 2016 because upgrades
weren't necessary for my needs. Each time, I only spent 1 hour maximum to
migrate the system. I am not even a server guy.

~~~
amaccuish
You mean 2016 right, not 2014? Or are you talking about SQL versions?

~~~
madiathomas
Yes. I meant Windows Server 2016. Thanks for the correction.

------
eternalny1
I like how Stack Overflow, in a market that is ripe to bash everything and
anything, has no qualms about discussing running .Net and SQL Server.

I've been a software engineer for over two decades now and I've seen stuff
come and go. The products Microsoft is shipping today are impressive bits of
technology and, and the end of the day, do what you want them to do and do
that very well. And I've been through the more difficult years where I wish I
had thrown SQL Server off a cliff.

Why am I bothering to write this? Because I'm still in technology and I wish
people would stop jumping on the next bandwagon or pointing to the next shiny
object. There are tried-and-true technologies out there that shouldn't be
ignored based on prejudices. Stack Overflow is a great example of a company
who understands this.

------
trilila
Personally, I am not a huge fan of Windows, .NET and SQL Server, but this is a
good example of a company using the right tool for the task at hand, rather
than copy pasting popular stacks. It would be really interesting to see some
stats, benchmarks, or any interesting metrics to help understand the benefits
or drawbacks of using this tooling. There is way too much focus on Linux, to
the point where I am worrying that Linux itself is becoming a monopoly, and it
might end up not benefiting from good competition.

~~~
larzang
Does the word monopoly even have meaning when the "product" is open source?
It's not like a single company controls linux, it is contributed to and grows
from input from many people and many companies worldwide, all of whom have an
interest in seeing it flourish.

Is HTML a monopoly? Is JS a monopoly?

~~~
trilila
IMO it does have meaning, in the a technical sense, not necessarily
commercially. JS does monopolize browser scripting. Now that you mentioned it,
it's quite interesting that there are basically no competitors to JS, in the
browser.

~~~
SenHeng
> _it 's quite interesting that there are basically no competitors to JS, in
> the browser_

Because the next best way to change a standard is add another one?

For all the moaning about javascript over the past decades, it has evolved
into a fairly formidable language since 2015.

I’m not even sure introducing a new language would help anything at all. It
would simply be quickly adopted by those the very same abusers.

Maybe the solution instead is to have better browsers and implement better
privacy policies.

~~~
trilila
I like JS, actually. It would be great to have some diversity tho. Also i
agree on having better browsers.

------
Accacin
I'm a big Linux fan, and use it as my daily driver but I can't understand why
so many people are talking about Linux when it's been very clearly explained
why the chose an MS stack.

They had already got experience with it! That's why they chose it. In my
opinion go with the stack you have experience with, don't just do what
everyone else is doing.

------
number6
Best part is the second part, where they stop upgrading to 2019 cause it
destroys their SSDs:

[https://www.tarynpivots.com/post/how-stack-overflow-
upgraded...](https://www.tarynpivots.com/post/how-stack-overflow-upgraded-
from-windows-2012/#phase-2-the-part-where-i-broke-dev)

~~~
thefarseeker
I'm Taryn's coworker I was involved in that part of the process very closely.

That statement is not entirely accurate. The SSDs certainly did die, but not
_because_ of the 2019 upgrade, it just happened at the same time as the 2019
upgrades.

The major 2019 SSD issue was however that Intel and Dell kept passing the buck
on who was to blame for not having 2019 drivers for the NVMe SSDs. Eventually
I ended up having to bully Intel into giving us the drivers. We decided that
"bullying Intel" was not an appropriate future tactic for getting the next
version of the drivers, so that's another factor for staying on 2016 (which
has drivers we've tested for years)

~~~
znpy
Interesting, how does one bully Intel? I wanna bully Intel too (/s)

------
zen_mike_eu
It would be interesting to get a guesstimate for the licensing costs of all
this.

~~~
nikanj
Around a dozen servers at
[https://stackexchange.com/performance](https://stackexchange.com/performance)
. Around $1k for Windows per machine, around $5k per SQL * 4 servers.

All in all, well under $50k in licensing costs. Basically a rounding error, if
you compare to something like AWS billing for a website of the same scale.

~~~
BrentOzar
> around $5k per SQL * 4 servers.

I wish. Microsoft SQL Server Enterprise Edition (which is required if you want
to use over 128GB RAM for caching data, or servers with over 24 cores) costs
approx $7,000 per CPU core.

~~~
nikanj
Yikes! For contrast, Buffer burns through $20k per month in managed hosting (
[https://open.buffer.com/transparent-pricing-
buffer/](https://open.buffer.com/transparent-pricing-buffer/) ). Running a
site at scale is an expensive ordeal

~~~
romanovcode
What does this SAAS has anything to do with MS server pricing

~~~
nikanj
It gives perspective on how expensive running a company on the internet is,
from a hosting-and-servers perspective. MS server pricing is steep, but the
cost of licensing over the useful lifetime of a SQL server is not a big cost
factor.

------
acqq
The full text is here:

[https://www.tarynpivots.com/post/how-stack-overflow-
upgraded...](https://www.tarynpivots.com/post/how-stack-overflow-upgraded-
from-windows-2012/)

------
EnderMB
I'm surprised by all the negative comments here surrounding how Windows Server
or SQL Server makes this more complicated, or whether the license costs are
prohibitive.

I worked as a .NET dev for nine years, and although I've never had to deal
with upgrades at this level before, I can appreciate that at the kind of scale
that Stack Overflow operate, and considering they run on physical boxes (a
rarity in any stack nowadays) any kind of upgrade between major database and
OS versions is going to result in its own issues.

While I don't work on the Microsoft stack any more, I've always appreciated
the way that Stack Overflow build their software. Their engineers and DBA's
seem to be a who's who of talent, and they both build great services and put
out great content, at scale. It's doubly-surprising when so many people used
to give Jeff Atwood so much shit for not delivering anything of note when
Coding Horror was one of the top programming blogs on the web.

The stack might be very different to anything done in 2019, but the fact that
I can't think of many times where Stack Overflow went down shows that they're
doing a great job, and hopefully they'll continue to document the journey for
years to come.

------
dvdbloc
Anyone know why systems like this aren’t Linux based? Seems rather ironic when
stack overflow seems to be a staple of the open source/sharing community.

~~~
danielovichdk
Why should it be? Honest question.

~~~
callesgg
Cause it is more work to manage windows servers than it is to manage Linux
servers. Less work is better than more work so Linux is better for server
implementations.

But I guess that is something people will argue over for a long long time.

Well... my opinion, based on my experience...

~~~
teh_klev
> Cause it is more work to manage windows servers than it is to manage Linux
> servers.

This isn't true. I managed and supported a fleet of Windows and Linux servers
for a web hosting company for 14 years. Both OS's have their time sinks and
gravity wells of failure.

~~~
callesgg
So both have time sinks... how does that permis effect my statement? I said
one is worse. You say both suck time, which is obviously true, work takes
time.

You can disagree with me ok, but if you want to convince me that I am wrong it
would be nice with substantiated material.

~~~
wayneftw
You hadn't offered any substantive material in support of your own opinion
either.

As a sysadmin with 30 years of experience running Linux and Windows servers
though, I'll offer my own anecdote - I'd say that Windows is far easier to
manage for many common scenarios. It's all point and click and you can
basically figure out what to do without looking at any documentation anywhere.

It's so easy, you can teach a monkey how to configure a workgroup, domain,
users, dns server, firewall, etc. on Windows.

Also, which Linux distro are you talking about? What with all the
fragmentation, nothing is ever the same on every distro.

~~~
AnIdiotOnTheNet
That's putting it mildly. A lot of the time things aren't even the same on the
same distro 2 years later, which is incredibly annoying when the only
documentation you can find is for the wrong version.

------
dmhimanshu
nice one

------
raverbashing
Whenever I read something like this (and I'm not blaming MS here, most tools
are like this) it seems that configuration and setup of systems is still in
the chipped stone age... No services (or versions of services) talk smoothly
with each other, setup is very prone to mistakes and it's just a big pain
overall.

(Windows probably makes it a bit hard though)

