
GitHub hit by DDoS attack - theyeti
http://status.github.com/
======
dengnan
This is an article [0] summarizes what happened. It is however in Chinese. So
let me put a simple summary here:

Baidu has Baidu Analytics, a service similar to Google Analytics. In short, a
website includes a javascript file from Baidu and Baidu will report some basic
analytics to the site manager like how many visitors per day, how much time
they spent on average per page etc.

Someone in the middle between a client outside China and Baidu, allegedly it
should be the Great Fire Wall, changed the javascript file from Baidu and
added some code so that any client executing the javascript file will
periodically access
[https://github.com/greatfire/](https://github.com/greatfire/) and
[https://github.com/cn-nytimes/](https://github.com/cn-nytimes/). This means
any user who is accessing a site using Baidu Analytics will be an attacker to
github.

Here is a simple solution: Block any javascript from Baidu if you do not use
it. For chrome users, add the pattern [*.]baidu.com. See here[1].

Edit 1: Added a solution.

Edit 2: Format.

Edit 3: Oh, it's not only Baidu Analytics. Baidu Ads' javascript is also being
hijacked and changed [2]. Imagine that all sites containing Google Ads use
their visitors as attackers to attack github. Now it is literally what is
happening to Baidu and its customers (and their customers' visitors.) The
javascript is only changed for visitors outside China. This is why people
believe that is done by Chinese government --- the only entity who has total
access to all out-going routers in China. Since many Chinese users use VPN or
other types of proxy to access Internet, they are all considered as visitors
outside China.

0\. [http://drops.wooyun.org/papers/5398](http://drops.wooyun.org/papers/5398)

1\. [http://www.howtogeek.com/tips/how-to-block-javascript-and-
ad...](http://www.howtogeek.com/tips/how-to-block-javascript-and-ads-for-a-
single-site-in-chrome/)

2\.
[http://www.solidot.org/story?sid=43489](http://www.solidot.org/story?sid=43489)

~~~
orf
According to this[1] post GitHub (or someone else in between) started changing
the responses to alert("Malicious Script Detected")[2]. That's an awesome
counterattack - this stops the script from looping indefinitely and annoys the
users.

1\. [http://insight-labs.org/?p=1682](http://insight-labs.org/?p=1682)

2\. [https://github.com/greatfire/](https://github.com/greatfire/)

~~~
dengnan
For github, this is a smart move.

But, really, you can hardly negotiate with Chinese government. I'm pretty sure
that they will deny this attack and re-emphasize their so-called Internet
policy.

If I were github, instead of a warning message, I would redirect the workload
to some Chinese government's website and let them suffer what they've created.
Let's face it, they are waging a war on the Internet first.

Edit: Disclaimer: I know that my post is quite biased, especially this one.
I'm not suggesting that people should wage a war to Chinese government. Please
take my words just as a (biased?) sample from an ordinary Chinese citizen who
is really tired of the government's censorship.

~~~
tedunangst
How would github redirect the load?

~~~
ninjaoxygen
[http://en.wikipedia.org/wiki/HTTP_301](http://en.wikipedia.org/wiki/HTTP_301)

~~~
eli
I think you are underestimating the volume of traffic. Simple generating that
many 301s would be an issue. And... where would you redirect to?

~~~
kevinchen
Generating a 301 is certainly less work than rendering the entire user's
profile page.

~~~
spartas
Generating a 301 is likely more work. Profile pages are simple database hits,
and they may be dynamically or even statically cached (for popular pages).
You're probably severely underestimating how much traffic China can produce
[1].

1\. [http://furbo.org/2015/01/22/fear-
china/](http://furbo.org/2015/01/22/fear-china/)

~~~
true_religion
A 301 avoids a database hit, and avoids having to go through the cache
framework all together.

It just says given this URL, we return an header that tells the browser to
redirect. The only thing faster would be just dropping the connection as soon
as its given to you.

~~~
rookieljw
Excuse me, Why no 403 or 404?

~~~
pbjtime
If I scroll up a bit, I see the reason is:

> redirect the workload to some Chinese government's website and let them
> suffer what they've created.

------
zhufenggood
Baidu's javascript cdn is being Hack by national firewall, inject these JS
attack script. If other webseit include some javascript library from Baidu's
javascript CDN will automatically run JS script that will DDOS attack Github.
The attack JS script is here:

[https://gist.github.com/zhufenggood/7bb040b1effb71d14bcc](https://gist.github.com/zhufenggood/7bb040b1effb71d14bcc)

Here is deobfuscate version using
[http://jsbeautifier.org/](http://jsbeautifier.org/)

[https://gist.github.com/zhufenggood/6a38c2a2b2185977b3cb](https://gist.github.com/zhufenggood/6a38c2a2b2185977b3cb)

Github notice that, it replace that DDOS http request respond with a
alert("WARNING: malicious javascript detected on this domain"). That is why
some Chinese guy gets a weird pop-up with English text when visiting Chinese
websites.

~~~
talnet
clever move we should say ? or any better idea ?

------
jdsnape
Looks like another case of Chinese traffic being tampered with to load
resources from another domain - in this case Baidu searches: [http://insight-
labs.org/?p=1682](http://insight-labs.org/?p=1682)

~~~
nickysielicki
This is far more interesting than the OP. Thanks for sharing.

~~~
plq
You mean TFA.

------
sgarrity
This is a reminder that having a single service, like Google Analytics or
Google Fonts, injected into just about every major site on the web might not
be a great idea.

~~~
omgitstom
This!

Even though I don't think this would have proactively helped this DDoS, but it
can't be closer to the truth.

CDNs hosting libraries / fonts / resources for the web are going to be
targeted more and more, it is just too attractive to malicious people.

~~~
kalleboo
This is why I wish we could have a file hash attribute added to certain tags
(such as script). It could improve caching across domains and validate the
content you're serving up. I proposal was posted here a while ago.

~~~
JeremyBanks
_Subresource Integrity - W3C Editor 's Draft_
[http://w3c.github.io/webappsec/specs/subresourceintegrity/](http://w3c.github.io/webappsec/specs/subresourceintegrity/)

------
rdl
Someone turning a widely-used third-party-hosted JS into "evil" seems like an
incredibly difficult layer 7 DDoS to address. Assuming you have great capacity
to filter on the edge (CloudFlare, being Google, etc.), but a limited backend,
it's still very hard to identify legit vs. non-legit traffic and do filtering.

(Obviously if the attack is against, say, Chinese users, and your site's
legitimate users are mainly in Estonia, you can do filtering, or if the attack
only hits an obscure URL, but the attack doesn't have to be weak in that way.)

There are a bunch of potential ways to address it, but they all work best if
you have a site with a defined community of users. If you're a large public
site, without login, it's hard. Some of the better techniques are in-browser
challenges (JS, CAPTCHA, etc.), but it's conceivable with enough endpoint with
real browsers and real humans on them, these could get defeated.

~~~
nickodell
GitHub seems to have done just this. Both attack URLs return

>alert("WARNING: malicious javascript detected on this domain")

They can probably serve that without hitting their database servers.

------
jhildings
A good warning sign for companies that only have their codebase at Github
which seem more common to me nowdays. If you run your own server at least you
can like physicaly restrict the access to local network only.

~~~
onion2k
First of all, physically restricting access to a local network pretty much
makes the benefits of using Git redundant. Ignoring that though..

I don't think there are many companies who have the depth of info security
knowledge that Github can draw upon. You might believe that running your own
server and locking it down to your local network ( _as best your team can do
that_ ) is better, but I'd rather trust Github even if that means my code is
available online. While the chance of being victim to a non-specific attack is
far higher (Github is a much bigger target; I'm affected if they're attacked),
the chance of someone targeting my code and actually getting it are far, far
lower because Github is better at making things secure than I am, and they
have people who are paid to make sure things stay that way.

A policy of only having your code on Github has it's flaws but making sure
it's secure isn't one of them.

~~~
flyinglizard
> I don't think there are many companies who have the depth of info security
> knowledge that Github can draw upon

That's true and I believe in that as well. However, GitHub also poses a much
larger attack surface than any single company and it's safe to assume that
once someone is in, they're going to get _everything_. Apart of that, you're
also vulnerable to having GitHub disgruntled employees accessing your data,
and the inherent vulnerability in having a remotely accessible repo in the
first place.

~~~
ddorian43
I think google protected your data from employees. Can't github do the same ?

~~~
flyinglizard
It makes sense that Google protects your data from most employees, but there's
always a core of employees that have everything accessible. It's probably a
small operational core in Google (still probably way bigger than the entire
headcount of Github).

~~~
mauricemir
It depends I know that old school telcos use security Vetting for people with
wide access to systems and this is DV Developed Vetting or TS (in American
usage)

And our Internal security team (BT Security) was bad news if you where
investigated - they have a ferocious reputation

------
tsheeeep
We use bower and npm for our project. Every couple of months github is under
attack by a DDoS or not working correctly leaving us with broken deploy
scripts. What is the best way to fix this? We don't like the idea of commiting
the node_modules or bower_components folder. Is there a tool which will cache
the npm and bower sources so they only have to be downloaded if something
changes?

~~~
kowdermeister
Committing downloaded packages is not a bad practice. Yes, it can be a bit
big, but otherwise I don't see much problem with it. You will be always sure
that the installed packages are compatible with each other.

~~~
tsheeeep
To be sure everything that we know works together we use things like npm-
shrinkwrap files. We don't like it because it makes the git changelogs a lot
bigger and almost unreadable if you want to compare a pull request.

~~~
__david__
You could commit them to their own repository so they don't taint your main
repo. Then use a submodule to pull that repo in to the main repo...

------
alexchamberlain
Making the not unrealistic assumption that the data is _not_ normally
distributed, mean is a useless average here. You should be looking at median,
which will be much less distorted by the long tail.

~~~
hayd
Would be interesting to see the 10/90% too.

~~~
TeMPOraL
This, or if you are evil and know that your competitor has a stupidly-designed
build process that _depends_ on GitHub being available - by DDoSing GitHub
itself you'll make your competition unable to work.

~~~
MollyR
Some smaller companies depend on github. I think its stupid, but I know of at
least one ivy league university subgroup who depend on github for everything.
Management wanted to outsource everything they could, they just saw developers
and IT as cost sinks.

------
Ethan_Mick
What we really need is a free and open source distributed version control
system.

------
mirekrusin
Could github whitelist ip addresses who did commit to protect normal users
from DDoS effects (splitting traffic to two sets of servers during DDoS etc)?

~~~
zer0defex
Seems like a reasonable strategy to me, but probably very infeasible for an
attack already in progress if this tactic weren't planned and ready to go in
advance. It would be something I'd investigate post-attack however to see if
it's a viable strategy for mitigating future attacks.

------
andao
maybe the point is to scare people into preemptively blocking Chinese IPs so
the Chinese gov doesn't have to swat flies with the Great Firewall. it's good
marketing too: "look at all those foreigners who refuse to let us access the
free internet!" "anti-Chinese prejudice!" etc etc

~~~
dEnigma
If that is their plan it seems to be working, judging by some of the comments
here.

------
yAnonymous
They should redirect the attack to the server hosting the script as a friendly
encouragement to use encryption.

------
ptr
Still getting lots of instability in spite of the status page.

------
r3bl
Why the hell would anyone launch a DDoS attack against GitHub? Seriously, the
only point I see in DDoS-ing GitHub is to prove yourself that you can DDoS it.

~~~
dagw
Given that the DDoS wasn't targeted at
"[https://github.com/"](https://github.com/") but rather
"[https://github.com/greatfire/"](https://github.com/greatfire/") and
"[https://github.com/cn-nytimes/"](https://github.com/cn-nytimes/"), two
projects that can reasonably be described as not being pro Chinese government,
it seems that github was targeted for hosting anti-Chinese 'propaganda'.

~~~
r3bl
Thanks for the clarification!

------
mahouse
I wonder what's the positive effect of sharing the same Internet with the
chinese. I never visit nor I know someone who visits sites from China.

Major ISPs from the west should definitely consider blackholing all traffic
coming from there to avoid DoS attacks, spam, etc. – From my experience, this
would mitigate spam by a 50% or even more.

~~~
chrisBob
So you agree with the Chinese government and think we should just censor the
Internet for all Chinese citizens?

------
sreya
If someone wouldn't mind explaining, what could the motive possibly be for the
Chinese government to be doing this?

~~~
touristtam
Maybe because the github repo is for this website:
[https://zh.greatfire.org/](https://zh.greatfire.org/) ?

------
bitinn
I have written a brief summary of issues, as a tweetstorm:
[https://twitter.com/bitinn/status/581350026217013248](https://twitter.com/bitinn/status/581350026217013248)

------
Irish
Anyone in europe having trouble with github this morning? I cant get bower to
install and it fails with cannot connect to github error, status page seems to
suggest everything is working

------
b123400
Did Baidu or its employee said anything about their script being used to
attack Github? Would like to know how they think about it

------
TACIXAT
The internet is so cool. I hope no one ever fixes the ability for shit to go
crazy online. It makes me so happy to be alive in the age of data leaks,
ddosses, and malware. It's all the more awesome that it isn't just individuals
but entire nation states fucking shit up. This is a really neat attack. I
hadn't thought of a MITM being used on a such a massive scale, and to leverage
uninfected computers as a botnet is pretty great.

Props to China. 很好!

------
jmakov
What I think is most interesting of all is not that a foreign country is
attacking a US company nor that the company has no support from the wast pool
of three letter agencies but the fact that github as a company designed their
architecture in such a way that a sub site is allowed to eat all of the
resources bringing the whole company down. Kudos to all the engineers and
architects with +100k salary over there.

------
dataker
Is it confirmed the Chinese government is behind this? Could it be possible to
also be a competing company?

~~~
mikekchar
Given Snowden's recent allegations that the Canadian government is engaging in
false flag operations (causing havoc and placing the blame on other nations),
it could even be another country that just wants to make China look bad. To be
honest, I would expect an attack from China to be a little bit more subtle...
Of course it could be a double bluff... but then... Basically, it's pretty
hard to know what the heck is going on.

------
jeremybass
Question, way can't `code.jquery.com` help here? or has the attack moved
passed this MIM attack?

------
mwadams
Apologies - it seems my earlier comment was made during a brief respite.

------
teknologist
And we're down again

------
edwintorok
the page says that everything is operating normally, but the main github.com
page doesn't even load for me...

Edit: works again now

------
_RPM
def noticed this like 5 minutes ago. DNS completely failed for a second.

------
hokutosei
oh boy, think we have to go home early today and its tgif.

------
mitkok
How is this news ?

------
vinceyuan
Fxxx GFW!

~~~
vinceyuan
To the guys who gave me -1: If you cannot visit Google, Facebook, Twitter,
Pinterest, and many other websites, you will say "Fxxx GFW!" too.

~~~
ramchip
I didn't downvote, but I can understand this getting pushed down as it doesn't
bring anything to the discussion.

------
mwadams
However, I still prefer [http://gitlab.com](http://gitlab.com) :-)

------
mwadams
It seems fine for me - and
[http://www.downforeveryoneorjustme.com/github.com](http://www.downforeveryoneorjustme.com/github.com)

