
Raymond Chen explains what the Y2K was like at Microsoft [video] - douche
https://www.onmsft.com/news/semi-official-windows-historian-raymond-chen-explains-what-the-y2k-was-like-at-microsoft
======
GrinningFool
One thing that I found interesting was the various media coverage of the time
describing Y2K as a 'non-event'.

The media - and consequently the public - was generally unaware that it was a
non-event _because_ of the massive resources poured into making it so. Instead
they wrote it up as a bunch of hype and paranoia over nothing.

IMO it's one of the greatest unsung engineering success stories of our time.

~~~
thesimp
Yes, indeed! As someone who also sat in a room (with many other people)
watching the clock tick until the stroke of midnight I still get pissed off if
people remember y2k as the event that "did not happen". Nothing happened
because most of 1998 and 1999 was spent running around customer sites checking
& patching software.

I work with industrial control systems and the oldest code comment that was
found about the year 2000 problem was in code from the early 80s. The
programmer long retired but that code was still running oil refineries.

There were very little large chemical installations in the pacific so we
waited for updates from sites in New Zealand. After those sites rolled into
the new year without problem everybody relaxed and we knew that our software
installed in middle east oil&gas sites would work ok so people would have
petrol and diesel in the new millennium. Then the rollover in the EU and US
region were easy after that.

~~~
all_usernames
"I still get pissed off if people remember [it] as the event that "did not
happen""

Welcome to the life of any operations engineer, my friend. If we're doing our
job right, no one notices that the insanely complex and down-to-the-wire
maintenance goes smoothly.

~~~
fillskills
I run OPS and Tech at a startup. A very thankless job I must say.

~~~
Moru
Ever tried cleaning a school? Students can even be heard saying things like
"It's their job to pick up after me so it doesn't matter that I throw garbage
on the floor".

------
mikestew
Y2K at Microsoft was boring as hell. What _wasn 't_ boring as hell was the
long period of time leading up to it. But everyone here knows that. New Year's
Eve 1999 and the hours following was probably the most Age of Empires I've
ever played in one sitting.

EDIT: "for an estimated issue programmers were not taking into account when
applying the Gregorian calendar rule to software."

First, what the hell does that even mean? Anyway, no, it was taken into
account. What, you think programmers didn't know what would happen when 2000
rolled around? What _wasn 't_ taken into account was that the software would
still be running ten, twenty years later.

~~~
TheCoelacanth
And most of the programmers who assumed that their software wouldn't be around
that long were probably right. It was just a few who had the misfortune of
creating successful software that ran into the problem.

------
slyall
I worked Y2k by myself in the Noc at an ISP in New Zealand ( aka UTC+1300 )

* Got paid $1000 bonus

* Called up by local reporter asking if any problems - Nope

* Called by several of our vendors/providers in the US just to say "Hello", and to casually ask how things were going.

* One ISP in Australia took themselves offline for the evening

* Rumors were going around Australia that all the power and the phones in NZ had died.

* The one little problem with a provider on the evening got a lot of people closely looking at it.

* There was fog and I didn't get to see the fireworks down town.

~~~
lostlogin
The weather was truely awful. I went up One Tree Hill and just saw clouds
light up with the fireworks.

------
CJefferson
Raymond Chen is the tech writer who has influenced me the most. His blog has
really helped me understand how large long-running software projects work (or
at least, how they work at Microsoft), and how apparently strange features can
come into existence in a sensible way.

One interesting evolution over time is how Windows moved from basically
trusting all applications (which they had to do in Windows 3.X anyway, as any
badly behaving app could crash the whole system) to treating applications as
bad actors.

~~~
ikeboy
I haven't seen anything from him on the whole Windows 10 forced update fiasco.
I'm curious how that played out internally.

~~~
chadgeidel
I think he writes his blog posts a year or so in advance.
[https://blogs.msdn.microsoft.com/oldnewthing/20090227-00/?p=...](https://blogs.msdn.microsoft.com/oldnewthing/20090227-00/?p=19003/)

------
user837387
About Y2K: during that time my family kept hearing that really bad things
could happen at midnight with the coming of the new millennium. One of the
main things that could happen, I think, was the lost of electrical power. So,
a couple of seconds before the new year I strategically positioned myself next
to the light switch and when the clock hit midnight and everybody started
yelling "happy new year" I turned the light off and yelled "we lost power". I
wish I could have recorded the faces of shock and fear in some of my family
members. It was funny as hell.

------
acqq
I'm sure it's not how the whole Y2K process was running, it's just how the
exact New Year's Eve event was planned.

I was personally also on ready-to-go-to-office stand-by, even not working in
Microsoft, but in much, much smaller company. And my team did do a serious
work in making the code we were responsible be Y2K proof, we've spent good
part of 1998 with that.

In short, we've worked for almost a year to make that "nothing serious
happened" result possible, in our own domain of responsibility. The highest
management understood the problem and the process was properly planned. That
"nothing serious happened" for our code is a success story of the quality of
the tests we've did and used to fix the real problems. And they did exist. I'm
sure there are other HN readers who can tell the similar stories.

The computer-related risks are a serious subject.

I remember following the news on the New Year's Eve to see what's happening in
Japan and Australia. As also "nothing serious" happened there, I was ready to
bet that everything will be fine, and that enough other companies also took
the subject seriously and acted early enough.

~~~
keithpeter
I'll admit to a pause for reflection around 9pm UK time (Midnight Moscow
time). After that, no worries. ADA has robust time and date libraries.

~~~
AnimalMuppet
I hadn't thought of that as a serious possibility, but a friend and I
considered writing a novel on it.

The basic idea was that a bunch of US weapons systems wouldn't work due to Y2K
- things like ballistic missile navigation. So the US was frantically trying
to get all this patched so that they would have a credible defense after Y2K.
The Chinese knew this, and launched on New Years Day...

... and completely missed, because they had used borrowed Russian code for
their ballistic missile navigation. The Russians had stolen US ballistic
missile navigation code, which had the Y2K issue in it.

~~~
acqq
> I hadn't thought of that as a serious possibility

Not serious?

[https://en.wikipedia.org/wiki/List_of_military_nuclear_accid...](https://en.wikipedia.org/wiki/List_of_military_nuclear_accidents)

[http://arstechnica.com/tech-policy/2013/12/launch-code-
for-u...](http://arstechnica.com/tech-policy/2013/12/launch-code-for-us-nukes-
was-00000000-for-20-years/)

"Launch code for US nukes was 00000000 for 20 years"

[http://foreignpolicy.com/2014/01/21/air-force-swears-our-
nuk...](http://foreignpolicy.com/2014/01/21/air-force-swears-our-nuke-launch-
code-was-never-00000000/)

So it was not actually eight zeros but apparently 6 zeroes and the "key under
the doormat" (in the safe, but not really something you needed the president
to access, the opposite of what was claimed then).

~~~
AnimalMuppet
That's all _very_ different from "launch because of a Y2K bug", though...

~~~
acqq
No technology is perfect, it's always a process.

And effectively every computer-related technology has undiscovered bugs.

[https://around.com/ariane.html](https://around.com/ariane.html)

""The board wishes to point out," they added, with the magnificent blandness
of many official accident reports, "that software is an expression of a highly
detailed design and does not fail in the same sense as a mechanical system."
(...)

(...) really important software has a reliability of 99.9999999 percent. At
least, until it doesn't. "

The statistics is against that generous number of nines.

------
tyingq
I wonder when the wind up for the 2038 problem (32 bit signed int and unix
epoch) starts.

~~~
tyingq
If you're thinking it's already largely resolved, this is from 64 bit mysql. I
assume similar issues exist in other software.

    
    
      mysql> SELECT UNIX_TIMESTAMP('2037-11-13 10:20:19');
      +---------------------------------------+
      | UNIX_TIMESTAMP('2037-11-13 10:20:19') |
      +---------------------------------------+
      |                            2141742019 |
      +---------------------------------------+
      1 row in set (0.00 sec)
    
      mysql> SELECT UNIX_TIMESTAMP('2039-11-13 10:20:19');
      +---------------------------------------+
      | UNIX_TIMESTAMP('2039-11-13 10:20:19') |
      +---------------------------------------+
      |                                     0 |
      +---------------------------------------+

~~~
raverbashing
Well, we still have 21 years to solve it

~~~
tyingq
A bit pedantic, I suppose, but Y2K problems showed up well ahead of Y2K.
Things like credit cards with expiration dates well in the future, and
validation code making comparisons.

~~~
Moru
Yes, our system doesn't work on some operating systems because we for some
reason use dates after 2050 as a flag for something. And that breaks together
if we testrun on windows but works fine on Linux.

------
baus
My first professional programming job was to fix a Y2K bug in 1995 in an
embedded system that hadn't shipped yet

------
CurtMonash
I was born on January 1, 1960.

So January 1, 2000 was going to be either my 40th birthday, or else my -60th.
:)

------
Spooky23
Where I worked after college, Y2K was the land of honey. It was a .gov that
essentially was able to get unlimited overtime funds for Y2K. Everyone
participated and my understanding is that people were "working" 18-20 hours a
day in preparation for Christmas 1999.

------
harry8
y2k was a massive deal that was totally solved by great engineering. The full
range of engineering approaches, the full range of talents, the full range of
management interference brought to bear in every country, company, department
etc. and _all_ of it worked out equally well so there was _no_ problem. Us
engineers, we're amazing, we should take a bow.

Or maybe, just maybe it was just a teensy weensy bit overblown so consulting
companies could charge really big fees especially to government departments
and banks. Easiest way to sell is to scare the s __t out of people then show
them "the solution." I bought an invasion like that once, regret it now...

------
stevewilhelm
We spent the two years prior to Y2k replacing our client's old mission
critical servers and services with new 'Y2k certified' hardware and software.

I personally think this contributed significantly to the dot.com era growth.

~~~
mSparks
Yep, and subsequent bust.

Many many companies getting not significant cash inflows in the run up to 2000
which made their balance sheets look great, until the end of the 2000 tax year
in 2001 when they didn't look so great any more.

~~~
firebones
That's a great point that I had forgotten.

In terms of y2k effort, it seemed to me to be a small amount of code change
and a disproportionately high amount of testing. The ramp up in hiring for
those changes and certification, combined with the lost opportunity cost, did
contribute to a later contraction.

Here's one thing that puts it in perspective though. The amount of code in the
world that was Y2K-affected was considerably less than the total amount of
code that exists today. And stepping outside the HN startup bubble (where code
is relatively young and modernized) and into the types of companies affected
by Y2K, there are _many_ efforts that are as bad or worse occurring all the
time. Stuff like regulatory changes (SarbOx or various banking reforms),
technology uplift (e.g., 32bit --> 64bit, Win32-->.NET, platform ports, etc.)
are as bad or worse based on the amount of legacy code it affects.

But what Y2K did is teach a lot of these companies how to solve sweeping,
codebase-wide problems.

------
raverbashing
Now that I realize, problems caused by Y2K were much fewer than problems
caused by leap seconds or "unexpected" leap days

~~~
acqq
Only because the former was widely known, discussed and the measures taken,
and the later was typically badly understood and almost nobody cared.

Quick, do you know, does your computer system ever display the 61st second?
And do you think it should?

~~~
TazeTSchnitzel
It most likely does, but blink and you'll miss it.

~~~
acqq
The answer is: it _is_ complicated.

POSIX:2001 specifies:

"As represented in seconds since the Epoch, each and every day shall be
accounted for by exactly 86400 seconds."

And that's typically no problem for normal use. The specialists know that
there are different time standards and that for "real" number of seconds one
has to use TAI, not UTC.

The problem in Unix world with NTP and the datetime algorithms was that some
programmers believed that they have to actually _see_ the leap second on their
own computers _in the kernel timestamps_ , up to the kernel intentionally
producing discontinuities for kernel times (behavior which never had sense for
timestamping purposes but implemented as such anyway). So now we have the
configuration variations like this:

[https://access.redhat.com/articles/15145](https://access.redhat.com/articles/15145)

and, to avoid Linux kernel discontinuities:

[https://developers.google.com/time/smear](https://developers.google.com/time/smear)

In fact, the smoothing of UTC and using TAI for those who need the "real
number of seconds" since point x was known as the reasonable approach long
ago:

[https://www.cl.cam.ac.uk/~mgk25/time/utc-sls/draft-kuhn-
leap...](https://www.cl.cam.ac.uk/~mgk25/time/utc-sls/draft-kuhn-
leapsecond-00.txt)

Now it's clearer why it's complex: too much people locally "assumed" what was
not to be assumed and didn't understand the effects of their local decisions
and the global context.

Hopefully "smearing" will get standardized and accepted and nobody will have
to care, except the specialists who really need TAI. The leap second
corrections should be invisible for normal uses, just like nobody cares for
correcting the clocks for much bigger differences.

------
divbit
This hacker-fiction y2k is fun (if quite dated now):
[https://www.amazon.com/Wyrm-Mark-
Fabi/dp/0553578081](https://www.amazon.com/Wyrm-Mark-Fabi/dp/0553578081)

