
GitHub was down - vickyonit
https://status.github.com/
======
jclulow
We just started using Gerrit, but using Github authentication. Can't sign in!
Not doing our own account management appears to have been a poor decision.

~~~
bdcravens
Every time a service like Github goes down, I always swear to myself I'll
replace that dependency with a project that takes days, weeks, or months to
implement will ensure I'll never have hours or minutes of downtime ever again.

~~~
roblabla
I run a gogs[0] instance, which is a GitHub clone with great performance
characteristics. I use it for both company and personal projects, and we
haven't had any downtime. No more productivity drop when github is down :D.

I recently discovered another huge dependency I have when coding though :
StackOverflow. That's going to be a lot harder to replace though...

[0]: [https://github.com/gogits/gogs](https://github.com/gogits/gogs)

PS: Yes, I realize the irony in using a github link here...

~~~
dmitry-k
>No more productivity drop when github is down

It could have been the motto of RhodeCode
([https://rhodecode.com](https://rhodecode.com)). Self-hosted repository
management not only has a benefit of having one's uptime under control, but
also is much more secure.

------
isuckatcoding
I've heard some good things about GitLab but seems like the majority of the
OSS community is still on GitHub (...for now).

~~~
mkagenius
Why would one assume that their self-hosted git will never go down or will be
up more than github?

~~~
hueving
At least when it's self-hosted, it's your responsibility and you can fix it.
When you are anything larger than a tiny company, this is much less risky.

Remember that 'the cloud' just means 'someone else's computers' so you are
subject to their infrastructure management practices and uptime guarantees (or
lack thereof).

~~~
morgante
Even for a company much larger than tiny, you're better off with GitHub from a
reliability perspective. Most medium companies (up to hundreds of employees)
do not have operations staff anywhere near as responsive as GitHub's are.

Look at their status history. The vast majority of companies could not boast
such a record for their internal operations.

~~~
Kudos
Not to mention if it's one person who decided to set up the self-hosted
solution and then that person leaves. How reliable is it then?

------
luhn
I'm looking forward to the postmortem—GitHub usually does an excellent job
with those.

~~~
yAnonymous
I'm looking forward to the post looking forward to the postmortem - HN usually
does an excellent job with those.

------
antoineMoPa
I had just posted a link to an issue in github and when I tried to click it to
see if it worked, I was wondering if I did anything wrong. Github has so much
activity that many people could be thinking the same right now.

I also guess hundreds of persons are wondering if the commit they just pushed
killed github's servers.

~~~
oceaniity
I did the same. I spent a solid minute trying different things and googling to
figure out what was wrong. Checked the status page and bam!

It's still down. :(

~~~
antoineMoPa
Now there is a human Denial of Service attack because everyone is refreshing
their pages like theres no tomorrow.

~~~
wckronholm
All DOS are because of humans?

~~~
antoineMoPa
When used as an attack, the efficient method is to buy servers or to have a
botnet to do it for you.

~~~
wckronholm
But a human sets it in motion.

------
davidgerard
Has anyone got a good Github backup script that's actually robust? Something
that will reliably go through all your company private repos and clone the
lot. I've found a few, but they're all clunky and in serious need of
adaptation. (e.g.
[https://gist.github.com/rodw/3073987](https://gist.github.com/rodw/3073987)
which is the best I've found so far.)

Hard part appears to be finding one that does an organisation account well,
not just a user.

What are other people here actually using?

~~~
0xmohit
You could try github-backup [0]. Quoting from the description:

    
    
      It backs up everything GitHub publishes about the repository,
      including branches, tags, other forks, issues, comments, wikis,
      milestones, pull requests, watchers, and stars.
    

And did I mention that it's written in Haskell!

[0] [https://github.com/joeyh/github-backup](https://github.com/joeyh/github-
backup)

~~~
davidgerard
Sounds good :-) Are you actually using it? What's your experience using it to
try to back up an organisation account?

edit: "github-backup does not log into GitHub, so it cannot backup private
repositories." So, looks like that won't do it for us, sorry :-(

------
sriharis
Strangely, pulling from and pushing to github works. Just that no page on
their website loads.

~~~
lobster_johnson
The git hosting is fairly independent of the web site. It happens fairly
frequently that one is down and the other isn't. The git hosting has a
resilient distributed back end designed to maintain redundant replicas and be
highly available. The web site is, I believe, a huge Rails app.

~~~
ipedrazas
Not only the website, all the github pages sites were down also

------
0xmohit
Latest update [0] says:

    
    
      Service is recovering and we are continuing to monitor.
    

[0] [https://status.github.com/messages](https://status.github.com/messages)

------
pandada8
[https://status.github.com/messages/2016-07-21](https://status.github.com/messages/2016-07-21)

by the way, why the status from July 15, 2016 to July 21, 2016 is red ?

~~~
cliftonc
It looks like they've lost a chunk of data - the last items in my news feed
are from the 14th of July.

~~~
ocdtrekkie
My activity is missing on my activity page, but the issue I just filed like an
hour ago is there on my contributions page. I suspect that perhaps some of the
feeds and lists have to repopulate, but nothing is really missing.

------
typpytyper
Github was good while it lasted. Back to SourceForge.

~~~
antoineMoPa
Then if SF goes down we'll go back to emails & patches. Issues will be tracked
with post-its & todo lists and communicated by voice.

~~~
ben_jones
Young blood here. Can we go further?

~~~
wckronholm
There's only so far tin can and string will go.

------
livus
This is the new "It's compiling!"

------
ben_jones
I would like to take this moment to remind everyone the new episode of Mr
Robot is out.

------
source99
Are there companies out there that include Github as a critical part of their
infrastructure such that if the web front end or GIT hosting goes down that
their production servers are affected?

I'm sure there are but how many people would actually have been affected by
this specific outage which only seemed to affect the web front end?

~~~
happyslobro
I've been planning to do a static company page, deployed as a GH organisation
page. Maybe I should have a fail over plan that can run off of S3 or
something, and a short TTL on the DNS records...

------
minton
Well that was scary. I pushed some changes and noticed the CI server didn't do
anything and it reported no changes. So I checked the PR and sure enough it
doesn't show any of my latest commits although git seems to think all is well.

------
lmontrieux
Looks like it's up again.

~~~
akerro
and down again.

------
Sylos
Yeah, I noticed.

Saw that Syncthing had an update for the desktop-version and wanted to check
that the Android-client was on roughly the same version number, so that the
protocols wouldn't be incompatible.

And because GitHub was down, I couldn't check the changelog, meaning that I
had to go to such crazy measures as opening the Syncthing-app to check what
protocol-version it displayed, which turned out to be a hundred times quicker
than rustling through some changelog, but uh, something something, ramble
ramble.

------
mcs_
This is why you keep your code in a GitLab self hosted machine with 2 cheap
disks in raid. Github should be used as mirror.

------
theaustinseven
I'd be interested to see what exactly caused this. They are usually very good
about preventing downtime.

------
ztratar
Probably a DDoS from a foreign actor.

~~~
kornish
What are the main motivations for foreign actors to DDoS GitHub?

~~~
laksjd
There have in the past been some ddos attacks that were most likely by a
foreign actor, but they were directed against individual repo's that contained
material that the attacker wanted to suppress.

See [http://arstechnica.com/security/2015/04/ddos-attacks-that-
cr...](http://arstechnica.com/security/2015/04/ddos-attacks-that-crippled-
github-linked-to-great-firewall-of-china/)

------
typpytyper
Looks like github lost two weeks of its public activity data.

------
Swizec
Obligatory "If only Git was a distributed system" snarky comment.

But this is seriously not a good time. Using gh-pages, hard deadline tomorrow
morning.

~~~
qwyxzy
Might [https://surge.sh](https://surge.sh) help?

~~~
Swizec
Sure, but the stuff I need is already hosted on github. As is most of the API
docs I need.

At the very least I need URLs for my stuff so I can put them in things I'm
writing. But I need to check github to find them.

Basically, every 3rd thing I looked up in the last 20min was hosted on github.

------
HappyTypist
GitHub needs to move slower and break less things. When was the last time
Google was down for more than a minute?

~~~
0xmohit
The world ought to move faster, so that the red on status page would appear
green (Doppler effect [0]).

[0]
[https://en.wikipedia.org/wiki/Doppler_effect](https://en.wikipedia.org/wiki/Doppler_effect)

------
minton
They say everything is operating normally but my latest commits are still
missing. Have they lost data?

------
denfromufa
Everyone should have a mirror on gitlab or bitbucket

------
aaronpk
Don't worry, it will come back soon

------
maurandk
Github downtime is pizzabreak time

------
fagnerbrack
All the company stopped, yay!

------
jacktang
Major service outage...

------
fagnerbrack
Time to procrastinate.

------
xfour
Seems straightforward.

~~~
googleir
500, everything I ever built is on github including my webpages, I am screwed.

------
ibloging
my blog is hosting on github pages.

------
obasa
back to normal now

------
dmlittle
And it's up!

------
vishweshh
I guess its up

------
divbit
github pls

------
seletskiy
How it's possible in the modern world with all software development practices
that project as github.com can be down for more than 10 minutes?

~~~
crawlerie
It's pretty amazing it wasn't down for more than 30 minutes.

