
GitHub was down - stevekrouse
https://www.githubstatus.com?
======
colinbartlett
If you’re wondering if GitHub has been down more since the Microsoft
acquisition, I actually just recently did some analysis on that using their
status page data (the answer is yes):

[https://nimbleindustries.io/2020/06/04/has-github-been-
down-...](https://nimbleindustries.io/2020/06/04/has-github-been-down-more-
since-its-acquisition-by-microsoft/)

~~~
tempay
It's not surprising. I don't have any data to prove it but it feels like
they're shipping improvements and new features at a much faster rate.

~~~
stupidcar
Or it could be some of the older employees decided to cash out after the
acquisition, and they've lost some institutional knowledge about how to keep
things running smoothly?

~~~
user5994461
The addition of interns and new members typically cause more outages, as they
make a lot of changes and discover how things work or not.

Retaining older employees has little effect to prevent that, because they
typically don't/can't review much of what's happening from a new team,
sometimes a whole department, who joined in on the project and has its own
agenda.

------
rvz
Again? This is like the fourth time GitHub went down. Last time this happened
was almost a week ago. [0]

Seriously, just consider self-hosting [1] rather than 'centralizing
everything' [2]

[0]
[https://news.ycombinator.com/item?id=23604944](https://news.ycombinator.com/item?id=23604944)

[1]
[https://news.ycombinator.com/item?id=23572532](https://news.ycombinator.com/item?id=23572532)

[2]
[https://news.ycombinator.com/item?id=22867803](https://news.ycombinator.com/item?id=22867803)

~~~
dx034
I set up a self hosted Gitlab where on-prem was a requirement and it's
surprisingly easy. Obviously another service that needs to be maintained and
updated but as code repositories and connected services are a crucial part of
infrastructure it makes sense to have control over that.

Not the biggest fan of Gitlabs UI but I got used to it quickly.

~~~
dna_polymerase
> Not the biggest fan of Gitlabs UI but I got used to it quickly.

It gets more attractive now that GitHub starts changing theirs. Might as well
adapt to GitLab now.

------
erikig
So many services rely on github that it has become a single point of failure
for online infrastructure as we’ve come to know it. I remember seeing this
shared many years ago [1] and not much seems to have changed in the interim.
If anything the MS acquisition has only exacerbated matters.

[1] [https://www.brandonsavage.net/github-your-single-point-of-
fa...](https://www.brandonsavage.net/github-your-single-point-of-failure/)

~~~
navanchauhan
> Gitlab is written on Ruby on Rails; you should understand the security
> implications of Ruby on Rails before using Gitlab

Can someone ELI5 the security implications of Ruby on Rails?

~~~
breatheoften
I'm sure its 'less bad' now -- but here's an example of an absolutely
horrendous security-bug-as-a consequence-of-design-deeply-baked-into-rails ...
[https://github.com/rails/rails/issues/13420](https://github.com/rails/rails/issues/13420)

Interestingly -- the rails developers decided to put in a really horrendous
hack to mitigate the common paths through which this design-flaw might lead to
unexpected security outcomes ...

In a way, one could argue that the willingness to put in a horrendous hack to
'mitigate' a security flaw provides an example which demonstrates some amount
of 'security reasonableness' in rails ...

In reality tho -- I think that this example serves as evidence more for the
fact that rails is deeply flawed and very unlikely to be secure in practice --
for reasons of design complexity alone.

~~~
navanchauhan
Thank you. I agree with you, even though most of the tech world is run on
bodges, that does not mean it is the right thing to do.

------
developuh
Has anyone here tried self hosting Phabricator?

Some of our private repos are Mercurial so it would be nice to have both git
and hg repos on a single platform.

~~~
fart32
Years ago, it was too complex for me. I recently switched to Gogs and I'm
satisfied with that. But that's Git only.

~~~
developuh
Thanks. I need something that can handle our hg repos.

~~~
wilsonthewhale
Sourcehut (hg.sr.ht) is supports Mercurial and can also be self-hosted. Not
sure how complicated it is to do so.

------
protomyth
Well, at least Microsoft hasn't had a Sidekick like event happen to GitHub.
[https://en.m.wikipedia.org/wiki/2009_Sidekick_data_loss](https://en.m.wikipedia.org/wiki/2009_Sidekick_data_loss)

------
YetAnotherNick
99.95% SLA means less than 66 minutes of downtime is acceptable per quarter. I
guess they will be offering credits to enterprise customers this time for the
first time.

Edit: Correct a mistake but the time calculated is correct.

~~~
Nemo_bis
Nice. I hope GitHub is down 67 minutes every quarter, just so that Microsoft
loses some money.

~~~
aparsons
You must have a fulfilling life with lots of friends

------
agustif
C'mon microsoft GH is down it's becoming a weekly trend here on HN..

~~~
dx034
After Facebook bought Whatsapp it also started having outages at a scale never
seen before. Whatsapp used to be a service that was practically never down,
after the acquisitions it sometimes had outages of up to a day. Rumours were
that it was related to features that FB wanted to add (e.g. video chat). Maybe
there's something similar at play here, that Github integrates MSFT services
without proper testing?

------
jonny383
Sounds like a good time to run something like this
[https://github.com/clockfort/GitHub-
Backup](https://github.com/clockfort/GitHub-Backup)

~~~
pantalaimon
Git itself is not the issue - the problem is Review Comments, Pull requests
and issues.

~~~
coronadisaster
Are you saying that Git should implement those features?

~~~
sirodoht
Fossil [1] has those integrated, along with a built-in web interface (which
git kinda has)

[1] [https://www.fossil-scm.org/](https://www.fossil-scm.org/)

~~~
developuh
I wish fossil had become more mainstream. It seemed such a cool tool when I
tried it 8 to 9 years ago.

------
st_goliath
Browsing through the status page makes me wonder how they manage to have
issues like this so frequently. Are there publicly available post-mortems for
previous incidents? From a first glance I can't find any on the status page.

Having an electronics engineering background, my personal pet theory is that
the convoluted layers upon layers of automagic container management, load
balancing and scaling mechanisms act like nested control loops in respect to
each other and sudden load increases (e.g. Monday morning load spike) cause
the system to essentially produce a step response and it starts
overshooting/oscillating. Just a thought tough.

~~~
jlgaddis
Earlier this year, they were blaming several of the outages on "the database".
Not sure if that's still their scapegoat or if they've come up with a new one.

------
Santosh83
GitHub Pages seems to be up though. My website is still online.

~~~
mcintyre1994
I'm guessing the core functionality of Pages is quite isolated from the rest
of the site since it's just serving static sites over a CDN.

------
sbahr001
It feels like this is happening a lot more(since quarantine) . It's really
weird for how big this company is, how important, and have microsoft's
backing. It is a little concerning given how much my work depends on its
stability.

------
Lapz
Its back up for me

------
soulchild37
Monday blues...

------
erkanerol
Unicorn!

~~~
sbahr001
I get the falling off a cliff.

~~~
anilakar
No, you fall of a cliff in that very specific manner only when ACME fails you.

------
pbexe
It's back up for me now

------
rawoke083600
Ag nee ouens !

------
ramnique
+1 down in India

------
h0ek
Looks like something went wrong!

------
pnako
Can we make this thread a sticky?

------
iamdual
I hate Microsoft. I've seen a lot of unicorn lately.

~~~
iamdual
I love downvotes.

------
Yarduza
The prophecy, it's starting...

------
tony-allan
We had issues producing the response to your request.

Sorry about that. Please try refreshing and contact us if the problem
persists.

Contact Support — GitHub Status — @githubstatus

------
corytheboyd
It drives me insane that they don’t localize the times on their status page,
is there a real reason they are displayed as UTC? If you are looking at a
status page chances are anxiety is already high, to throw time zone conversion
into the mix too is just obnoxious.

~~~
jlgaddis
I'm the exact opposite.

I hate it when sites use their local time, instead of just using UTC
everywhere. I got accustomed to using UTC ~25 years ago.

At worst, it'll take you two seconds to open up a terminal and run

    
    
      $ date -u
    

if you can't handle the conversion "internally".

------
coronadisaster
works for me... loaded this file no problem:
[https://github.com/washingtonpost/data-police-
shootings/blob...](https://github.com/washingtonpost/data-police-
shootings/blob/master/fatal-police-shootings-data.csv)

------
oars
First time I've ever seen it down, I guess there's a first for everything. Now
I understand why many people are very paranoid when it comes to taking
backups.

Is it worth backing up my GitHub repos somewhere else? What do other people
use as an alternative source of truth for their code?

~~~
huseyinkeles
Not sure if you were being sarcastic but Github goes down quite often :)

[https://hn.algolia.com/?q=github+down](https://hn.algolia.com/?q=github+down)

~~~
s9w
I think this implied that it never was down for him before when it was
reported to be down. Which is exactly my experience, too. I've seen a lot of
GH down posts here - it never was for me. Now it's the same.

~~~
arghwhat
Possibly region/cluster specific? Not loading for me from Denmark.

