
GitHub is down - napsterbr
https://status.github.com/?
======
justsaysmthng
I hope the world realizes that clouds are not just fluffy white things in the
blue sky, but can create destructive storms too, with wind, rain and so on.

Make sure your infrastructure is ready to weather such storms - have local
backups and fallbacks, make sure your code is available even if github (or the
Internet) goes down.

The more I hear 'cloud', the more I try to depend only on tools that work
offline.

~~~
eric001
Metaphors like that only makes it even more confusing. It's a marketing scam,
let's call it for what it is. Servers. They are servers.

~~~
blowski
There _can be_ difference between hosting on bare metal boxes and hosting on
cloud providers.

If you are connecting different services (message queues, databases, storage,
transcoding), and spinning up and down instances based on current demand and
spot prices, then cloud hosting is totally different. On the other hand, if
you are just renting a couple of dedicated cloud instances, then yeah -
they're just (virtualized) servers.

~~~
TeMPOraL
Yeah. The problem is, "cloud" now means both of the things you described and
then some. Probably the best definition corresponding to present usage of this
word would be "not on localhost".

(Because yeah, if I host a Dropbox clone on my Raspberry Pi sitting on the
wardrobe, I have my files "in the cloud" now.)

We need a new word for automated, dynamic allocation/deallocation/management
of remote instances for various services. Something that would differentiate
it from buying a VPS.

~~~
blowski
Agreed. Amongst non-technical people, 'in the cloud' just means 'somewhere on
the internet'. For example, if I backup my iPhone onto iTunes on my laptop
then that's not on the cloud, but if backup to iCloud then it's on the cloud.

I guess it's frustrating that cloud means quite different things depending on
the context, but I guess that's true of a lot of language.

------
mserdarsanli
Phew, I can't develop anymore but [http://left-pad.io/](http://left-pad.io/)
is still up so my systems continue operating.

------
colinbartlett
It's interesting to me to see all the services that depend on or integrate
with GitHub in some way, as they all have problems at times like this.

My side project [https://statusgator.com](https://statusgator.com) shows
currently a dozen different services all posting warnings or down notices on
their status pages. It's mostly CI services and the like, all which break in
some way when GitHub is down.

~~~
TeMPOraL
> _It 's mostly CI services and the like, all which break in some way when
> GitHub is down._

It boggles my mind. If you're not adding a new dependency to the project, your
CI server should _never_ hit GitHub when doing builds. Who sets those things
up without local cache?

~~~
WillAbides
Why shouldn't your CI get the source code it's building from GitHub?

~~~
TeMPOraL
It should, but only once. Redownloading the same sources over and over and
over again is a stupid waste.

------
napsterbr
And so is all Github-dependent deployment...

~~~
diegorbaquero
Got here because of that haha, can't deploy. EDIT: Why downvote fellas?

~~~
dtech
Because your comment adds nothing.

~~~
random55643
It adds something. It shows an example of an actual person affected by this.
Yours here, however, I'm not sure what that adds :)

~~~
dtech
Someone was apparently confused as to why his comment was downvoted. I
answered that question.

------
Bino
Haha, I always feel guilty when this happens, I'm doing something and suddenly
start getting errors, which just escalates on refresh and now everything is
down :P

------
chtoric
You can also follow githubstatus on twitter:
[https://twitter.com/githubstatus](https://twitter.com/githubstatus)

~~~
oneeyedpigeon
Interesting that one of the tweets uses identical wording to one of the status
messages, and one doesn't. I would have assumed the tweets would be automated,
but maybe there's a thinking that the message for the status bar isn't
necessarily appropriate for Twitter, so it can be 'overridden'?

------
dvhh
updates :
[https://status.github.com/messages](https://status.github.com/messages)

------
kome
SourceForge is doing fine ;)

~~~
zodiakzz
And so are my free private repositories on Bitbucket. :)

~~~
akerro
and my two independent gitlab instances.

~~~
rootlocus
I'm guessing your gitlab instances see as many visitors as SourceForge

~~~
akerro
That's why I call it private.

------
Jean-Philipe
Once the office router was down, we just pushed our commits to a USB stick.
Worked pretty well.

------
maccard
The common complaint is that when github goes down so do issue
trackers/project management/docs etc. What are the equivalent alternatives for
these systems that are distributed, so tolerant to downtime.

If you are in a situation that you're using a DVCS but relying on centralised
everything else, and saying that you're unable to work because your Issue
tracker/CI server is down, have you really gained anything over the
alternatives?

~~~
akerro
You can setup gitlab with clones, if one server goes down, second has a copy.
You know, git is a distributed system, unlike github.

~~~
mostlystatic
But if two systems are identical, wouldn't it be likely that both would break
at the same time?

~~~
Piskvorrr
No. Unless you do something dumb like "don't test that the failover system is
fully functional before taking the primary offline;" this applies to any
maintenance, which also includes upgrades. Usual HA tactics apply - don't rely
on one power source, one network link, or just about any SPOF. (Which means
that requirements start breeding like rabbits, which gets expensive rather
quickly - whereas Github is free-as-in-beer, which is _good enough_ for most
people)

Perhaps the only case breaking both at once would be deliberate action (DDoS
etc.).

------
Ianvdl
The Web UI is still accessible for me, just extremely slow.

------
romanovcode
GitLab to the rescue!

~~~
voctor
Momentarily, yes. But it's just another central hub.

~~~
Sir_Substance
You can host your own.

~~~
jbrooksuk
Which can go down too.

~~~
romankolpak
Still, a lot less risky. Self-hosted hub is something you have more control
over and is less likely to be DDoS'ed by some Chinese harm-wishers.

~~~
zo1
Not just that, but if it's self-hosted, that also implies that it might be
local to your work-area. I.e. It's on-premise, so even the entire internet
going down wouldn't necessary stop local operations.

------
paradite
I pinged GitHub and it seems to be responding but the web UI is taking a few
minutes to respond.

------
jimjimjim
Just a friendly reminder to store source locally.

Generally try not to put external services in a critical path.

------
vacri
Amazon oregon is down for me - can't get the EC2 status page... and the
service outage page also has trouble loading...

[http://status.aws.amazon.com/](http://status.aws.amazon.com/)

------
tiernano
Status page[1] is now saying everything is operating normally... now waiting
for the post mortum...

[1]:[https://status.github.com/messages](https://status.github.com/messages)

------
joncampbelldev
Taking bets on if this is the chinese government testing out GreatCannon
v2.0.0

~~~
tankenmate
Panama Papers anyone?

~~~
zdkl
If only someone started selling tinfoil hats with "I told you so" logos.

~~~
TeMPOraL
Ask the [http://shieldheadwear.com/](http://shieldheadwear.com/) guys. They
actually managed to kickstart stylish tinfoil hats...

------
tlrobinson
git is decentralized, but a bunch of the supporting infrastructure GitHub
provides, like Issues and PRs, isn't. It would be neat if that stuff were
tracked in a git repository as well.

------
qznc
Is HN traffic significantly above average at the moment?

------
kchoudhu
If only we had a version control system that didn't rely on a central hub...

~~~
cytzol
We do, it's called Fossil! [http://fossil-scm.org/](http://fossil-scm.org/)

Fossil tracks issues (tickets), documentation, and wiki pages at the same
level as it tracks code, so when you clone a Fossil repo, _everything_ gets
downloaded to your local computer. You can then respond to tickets and perform
repo maintenance offline, syncing everything to the server when you're back
up.

Edit: A few weeks ago I got a job a one-hour train ride away. I can commit,
close tickets, and write docs on my way in, sync everything to the internet
when I get to work, then do the same on my way back - even though I don't have
a net connection during my commute. It works great. (Ok, not as good as not
having to travel at all, but great nonetheless!)

~~~
thecatspaw
the parent commentator was joking that git _is_ decentralised, but that we
made it centralised again by relying on a small number of hosters (github,
bitbucket and others)

~~~
cytzol
Heh, don't worry, I got the joke. But I also got the joke the last time this
was posted, and the time before that, and the time before that...

This just _keeps happening_. It's not GitHub's fault that they keep getting
DDoSed or have a network outage every now and again. But it's not a problem
with no solution.

~~~
popey456963
To be honest, I can cope with Github's downtime, it is probably less than if I
was hosting a Git solution myself and the service is much more reliable than
all the other solutions I've tried (which are either lacking features or have
bugs or have massive downtimes).

Github is trying their best as far as I can see, they manage to fix the
majority of issues within three hours and even during those three hours its
not the end of the world because you can still code.

------
catslave01
Does it affect my work if I am using Waffle? My company is planning to use a
kanban service.

------
Matt3o12_
The webpage works fine for me. Can you tell me which part of the
infrastructure is down?

~~~
oneeyedpigeon
Back up now, apparently, but GitHub themselves were reporting "Major service
outage" in bold red letters. It looked very serious, but it seems they've done
a great job getting everything back up fairly quickly.

------
sureshn
has any one tried setting up a fossil-scm locally and work with that instead

~~~
rcarmo
Yeah. I've been using it on and off for a while, but git works fine locally,
so it was mostly for private bug tracking.

------
darkcl
Sorry github, I know my code is unbearable to watch :(

------
pythondz
Maybe it's time to use Git with Storj ?

------
gregdoesit
Facebook and Twitter seems to have sorted their once frequent outages pretty
well over the years. Come on, GitHub - you can do better!

------
vladimir-y
Azer Koçulu did you kill Github? )

------
hathym
we need a gittorrent.

~~~
masklinn
No. Git is easy to distribute, that's pretty much the point, and even if your
central repository is down you can generally work: you can easily commit
locally and can even use ad-hoc servers for P2P collaboration. Anything based
on your VCS (CI, deployment) can be rerouted or handle multi-master sources.

The problem is all the shit that is _not_ in the VCS and not directly based on
it: issues, code reviews, discussions, … and the community around them if you
have external contributors.

"Gittorrent" attacks a problem which by and large is already solved, it might
further improvement on it but it's improving something _which is not the
bottleneck_.

------
llomlup
No issues for me, just merged a pull request.

------
Gurrewe
GitHub seems to be back up again now.

------
chtoric
well, it seems it is working now, even if the error persist on the status
page.

------
J3remyD
it's "working" now , still very slow

------
progx
And? Work offline.

------
devsonfire
final status -
[https://twitter.com/Insping/status/717271742964568065](https://twitter.com/Insping/status/717271742964568065)

------
ddon
I like how they write there "Everything operating normally." :) when it is
down...

------
touristtam
You'd think by now GH would have a more resilient system in place ... I mean
it isn't like it is the first time it happens to them.

Thanks for the downvote, my hearts goes to you. :/

~~~
starefossen
I guess you will get what you pay for... which is nothing for most of their
users.

~~~
cbd1984
> I guess you will get what you pay for...

1\. There is literally no amount of money which will get you 100% uptime. You
may be able to buy _guarantees of_ 100% uptime, however...

2\. ... which will not change the fact that things still go down, and that if
something goes down in a way which violates your signed contract, the most you
can hope for is some kind of monetary compensation and, maybe, an apologetic-
sounding employee to scream at until you feel a bit better. If you expect to
buy someone's _care_ , you're woefully naïve.

My point is that the paying customers don't necessarily get a better service.
All they get is limited compensation for enduring the same bad service all of
the rest of us get.

~~~
adwf
It's true that you can't get 100% uptime. And yes 99.999% is more of an SLA
agreement/goal than a generally achievable metric. But Github aren't even
managing two nines - they only have 98% uptime over the past month. That's bad
for _any_ service.

