

We are currently taking a DDoS attack and are working to mitigate - orrsella
https://status.github.com/?2013-03-24
"We are currently taking a DDoS attack and are working to mitigate. The site may be slow to respond, and you may struggle to pull/push code via SSH - we apologise for any inconvenience."
======
hp50g
This is one reason us corporate software pushers can't use github and the
likes. We need stuff to be hosted on a network we control. Each hour we lose
is potentially multiplied by the number of staff as we would push and pull
hundreds of times a day. Someones head will roll if were down for more than a
couple of minutes at a time.

It becomes just as much of a PITA importing projects github and maven for
example as well.

We actually ended up sticking with svn and trac and doing manual mirrors of
every source dependency we pull in to protect our asses. SVN is kept only due
to externals support (even though all our externals are internal!). Our head
rev is over a quarter of million revs, 37gb of data and the repository has
been online since 2006 to give you an idea.

I'm aware of github enterprise btw - its actually easier to build our own
using trac as we can support it end to end, its a crap load more configurable
and we can scale it up and run failover nodes easily using svnsync and pgsql
replication. We have our own plugins and reports plugged in using reportlab as
well. And its not atlassian's crap either.

~~~
Volpe
You know you can run git internally as well?

Using github as an excuse to stay with svn seems rather ill informed.

You wouldn't lose hours if github is down. You might lose your ability to
deploy (if you are deploying from github) but you could setup any other remote
you like, and deploy from that.

I'm not sure what the 'size' of your repo has to do with anything...

~~~
huherto
To me github is the killer feature of git.

I think svn is easier and more appropiate for a corporate environment.

~~~
FuzzyDunlop
I think the killer feature of git -- which applies to any other DVCS -- is
that you're not dependent on github if you're well prepared. There's
absolutely nothing stopping you adding redundant remotes so you always have
somewhere to push/pull changes and deploy from.

    
    
        git remote add <remote_name> url

~~~
hp50g
Yep and that's like managing dns with host files...

------
benatkin
Somewhere else on the Internet: "We are currently giving a DDoS attack and are
working to..."

~~~
aspensmonster
"...keep it that way."

------
nemoto
If you have trouble cloning/pushing on git@github.com right now, you could use
these config[1] for ~/.ssh/config

    
    
        Host github.com
          Hostname ssh.github.com
          Port 443
    

Cloning and pushing to github now works in my machine.

[1]: [https://help.github.com/articles/using-ssh-over-the-https-
po...](https://help.github.com/articles/using-ssh-over-the-https-port)

~~~
xxbondsxx
Wow! Huge thanks, this actually works (for those of you still having trouble)

------
Shank
I really wish they'd release geographic information on who's launching these
attacks. On the bright side, a Saturday @ 7 eastern isn't near as annoying as
something during the workweek.

~~~
SG-
While 'neat', it won't reveal who's actually controlling the botnet and
launched the attack.

~~~
Shank
It's really quite a shame at how little can be done to find the perpetrator. I
suppose contacting law enforcement wouldn't do much either, unless it's a
significant persistent threat that would warrant a large scale investigation.

~~~
trotsky
Based on the FBI taking 50 doors related to the operation payback ddos and yet
not charging anyone it appears the US Attorney's office isn't sure its even a
crime.

------
spikels
Why is anyone DDoSing GitHub?

~~~
becojo
A lot of people relies and uses Github on a very frequent basis. It's their
way to get attention I guess.

~~~
spikels
Possibilities mentioned so far: (1) Fame for attacker (2) Excuse for other
problems at GitHub (3) Competitors

Not sure I find any of these is more convincing than the next or very
convincing at all.

GitHub seems to be one of the least offensive businesses around but still gets
attacked on a regularly. It's popularity might make it a target but don't
those running botnets have better targets? And wouldn't they publicize their
exploits?

And I can't believe GitHub would use it as an excuse. A DDoS attack has clear
evidence that their entire devops team and many of their suppliers would have
first hand knowledge of. If word leaked out it would be pretty embarrassing.

Lastly would a competitor risk destroying their entire business if found out.
People are irrational and stupid but that would be crazy.

Why is this happening?

------
niggler
Anyone counting how many DDoS attacks github saw this year? I don't doubt they
are having issues, but at this point it seems like they are blaming every
outage on a DDoS

~~~
bdg
> at this point it seems like they are blaming every outage on a DDoS

Related: <https://news.ycombinator.com/item?id=3576964>

~~~
AYBABTME
Was about to post asking why would somebody want to DDoS Github. Your link
seems like an "understandable" motive.

~~~
badgar
Extortion is one of the better possible reasons for their DDoS. Eventually,
when the money doesn't show, its in their best interest to stop attacking.
While new extortionists will keep showing up, they won't likely come back.

Ideological or government attackers, on the other hand, can't be reasoned with
and can only be expected to escalate in the future. If that's who Github is
dealing with, then they'd better figure out how to actually deal with DDoSes
sometime soon, because they will only increase in size and frequency.

------
gfodor
In light of this it would be nice if github offered a feature to push repo's
to s3 or something every time they receive a push, so you could use that repo
until they get back up.

~~~
aprescott
You know what I want? Transparent failover somehow baked into git remotes.

    
    
        you <--> (X <--> Y)
    

If you push to and pull from X, and X should be unavailable, automatically
start using Y. When X is pushed to, check with Y first to keep in sync. There
are ways to push to multiple remotes by defining multiple remote URLs, but
then it's a harder problem of dealing with downtime.

~~~
jaegerpicker
It's really easy to push to two remotes. I have a tfs service account setup
and using git and also a github account. I like tfs's project and issue
tracking better but want the public repo available on github.

git add remote <name> <url> git add remote <name> <url> then create a shell
script that checks both remotes for the latest if one is greater than the
other merge the two if they are the same then push to both. Then just always
push with that script.

Just how I've handled it so far. That said a autoMagical way of handling it
built into github/git itself would be cool.

~~~
aprescott
As I said, it's doable with scripts and whatnot, but transparent and built
into git would be awesome. I see it as something you just set up on a remote
repository server. No need to make N developers aware of it, or aware of the
script, or resolving the script breaking, or having a ton of implementations
of said script across different teams and companies.

Doesn't help if you use GitHub for issues, of course, but hey!

------
benatkin
It's back. I was able to watch the graph change and see it reflected in my own
experience of trying to load a GitHub page. Great job on the Status site,
GitHub!

I first noticed the top graph change, and a minute or two after it went back
up towards 100%, they updated their status message to "Minor service outage".
Now it's been moved to the history: <https://status.github.com/messages>

~~~
trumbitta2
maybe... but I still can't push my stuff from Italy :-/

~~~
gisenberg
I'm still having problems pushing/pulling as well.

------
guiambros
Oh that explains. Seems it started half hour before they posted the update.

I was trying to make pulls just a minute ago and it was incredibly slow, but
the status.github was just showing "we're investigating the high rate of drop
packages".

Now, why would anyone want to DDoS GitHub? Gee...

------
kmfrk
We need to agree on a date format for this kind of submissions:
<https://news.ycombinator.com/item?id=5430129>. :P

------
hkdobrev
What can WE do about it so GitHub stops getting DDoS-ed?

~~~
superuser2
Stop letting computer-illiterate people believe it's okay to run Windows when
they aren't going to be responsible about updates/safe computing?

------
tel
Does anyone have insight into that repeated spike train pattern in the
exception percentage graph? That seems so predictable it ought to be fixable?

------
argc
No wonder.. I was wondering why the hell I couldn't push and then I thought,
"well... I'm bored guess I'll read hacker news... "

------
account_taken
This is the reason I love git---distributed. I can keep on keeping on despite
these silly attacks (if they really are DDoS).

------
brador
How about whitelisting known genuine users?

~~~
kbuck
The attackers would just find such a user and spoof their IP. DDoS is a hard
problem to solve, and it's a shame that so many ISPs and datacenters don't
work harder to prevent spoofed traffic. On top of this, they'd still need
routers and switches in front of their machines big enough to handle the
traffic from the attack plus the load of trying to filter out the good traffic
(this kind of hardware is quite expensive).

------
leothekim
Kudos to GitHub for showing their latency and error stats. That is grace under
pressure.

------
daGrevis
Everyone on HN opening GitHub to read the message will surely help! :)

------
DigitalSea
Wow, Bitbucket must be getting desperate... Haha. This is annoying, even
though it's a weekend, I've got work to do and the fact I can't pull down some
changes I made on a machine at work to my home machine is highly annoying.

