
"Major service outage" - yogipatel
https://status.github.com/messages
======
templaedhel
Github is great, they're working on some serious problems, and every service
has downtime.

But this is getting pretty serious.

This is the 3rd time in these last few weeks there has been a "significant
service outage"

So much of a typical dev workflow is based around github, but more importantly
a lot of new package managers use github as a base. Not being able to pull
dependencies is a fairly big problem.

I will continue to use github, because they're awesome. At the same time,
we're going to have to start building around it to ensure our uptime isn't
reliant on their (less than optimal) uptime.

~~~
PommeDeTerre
I've found it somewhat odd how, although advocating the use of a decentralized
version control system, much of the git community has ended up heavily
centralized on GitHub.

~~~
jeremyjh
It is a distributed system. You still have to have shared remotes in order to
integrate branches.

~~~
snogglethorpe
Yup, when github barfs, you can still happily keep on hacking.

You're also not limited to one shared repo by any means, so even
synchronization between multiple developers can continue when github is down.
To avoid problems it's of course a good idea when one of the repos is a
"master", and if using a secondary repo for synchronization when the master is
down, you'll probably want to switch to unique temporary branches for the
purpose.

~~~
yakiv
One possible backup would be Bitbucket
(<https://bitbucket.orghttps://bitbucket.org/>).

(I'm not affiliated Bitbucket in any way. I don't profit from them doing well,
aside from them sticking around to keep providing free hosting for me.)

~~~
bitcartel
If you want to use your own servers, Atlassian (who own Bitbucket) recently
released Stash, a competitor to Github Enterprise. Pricing is very good and if
you have a commercial license, you also get the source code.

~~~
lflux
I've evaluated Stash a couple times, and I feel that Gitlab still has it beat.

------
DigitalSea
I get outages are a way of life with a widely used web application, but Github
have really dropped the ball lately. This is one of many service outages
lately and as a paying customer it's disheartening and worrying because I use
Github in my day-to-day workflow, I and many others have come to rely on it.
Don't get me wrong here, I love Github and couldn't live without it, but they
really need to sort out these problems and it's not like they don't have the
funds to address the issues anyway. My knowledge of distributed computing is
somewhat limited, but I would have thought they'd just spin up a few extra
virtual machines to handle the database spike (maybe it's not that simple with
Github's setup, I'm not sure).

~~~
hunvreus
Github have their own datacenter and hardware, they are not relying on any
cloud provider out there. It makes it harder to handle spikes of load, but
they actually have an I/O intense service which justifies that choice.

~~~
druiid
Incorrect:

<https://github.com/blog/493-github-is-moving-to-rackspace>

~~~
DigitalSea
My favourite line from that linked blog post from 2009 is this: "We're aware
of the current stability and performance issues, and we want to let you know
what we're doing about it." - issues they were having nearly 4 years ago are
still happening unless the problems they've faced lately are completely
different.

~~~
bobfunk
The move really did fix their problems back then.

There were a period where they were really having visible scaling problems,
with response times often getting painfully slow - apparently especially due
to slow I/O. These problems completely disappeared after their move to
Rackspace's managed hosting.

Now it looks like they might be hitting a new barrier that might require
architectural changes to overcome.

------
s1kx
I suppose this is related to the Rails vulnerability
(<http://news.ycombinator.com/item?id=5028218>) and everyone
updating/deploying their applications

------
outside1234
Correlated with thousands of redeploys for the Rails security issue?

------
mikec3k
I use BitBucket for most of my projects since they offer free private
repository hosting.

------
willvarfar
Good to see that they've upgraded/patched their Ruby On Rails attack surface
then... :)

------
emperorcezar
Looks like it was a whole 10 minutes. EVERYONE GRAB THE PITCHFORKS!

~~~
rtkwe
2:13 if you count from the first message about high exceptions to the all
clear "everything is working normally."

------
cagenut
If they keep this up I'm gonna have to buy github-enterprise!

wait a minute...

~~~
jonursenbach
Github Enterprise runs on your own servers. Not theirs.

~~~
X-Istence
That's the joke ...

~~~
jonursenbach
There was a joke in that?

------
gbog
Would it be possible to build a github mirror with only the repos and nothing
of the web candies? Then maybe the reliability would be increased.

~~~
Doih
We use rhodecode to mirror our hg and git stuff it auto pulls stuff also.

------
darkarmani
This is why I should deploy using git (via github), right? I can always
rollback if i break something...

------
Nux
Just run your own shit, people.

------
yogipatel
Update from GitHub: 23:49 UTC Everything operating normally

------
rauar
Is this an automated status page ? The messages are so ... focused. If yes, I
love it. Need to build one too.

------
cmwelsh
Mirror your projects to BitBucket to reduce SPOF. That's what I'm doing with
my private repositories.

------
gjvc
i've downgraded to free

------
Doih
Just use rhodecode as a backup and don't worry anymore

