
Weak passwords brute forced - olefoo
https://github.com/blog/1698-weak-passwords-brute-forced
======
fphilipe
Considering that the code in a GitHub organization or in a repo with
collaborators is only as safe as the weakest link in the chain of users that
have access to it, it would be nice if the admin of that org/repo could set a
policy that all members need to have 2-factor authentication activated before
they get full access to the codebase. Enforcing strong and unique passwords is
hard, but enforcing 2-factor is easy and the bump in added security is high.
At least I would sleep better knowing that, for an attacker to access the
codebase, brute forcing would not be possible.

~~~
csmuk
Enforcing unique and strong passwords is not hard. We do it.

~~~
mtrimpe
Unique across all other services the user has accounts at? If you did that it
would be both creepy _and_ impressive. :)

~~~
csmuk
We try to make sure it is by enforcing password expiry, no password reuse and
complex passwords. We also have a massive shitlist of passwords from various
leaks.

~~~
RobAley
Because of which, your users probably write their password down on a post-it
note stuck to their monitor...

~~~
daigoba66
That's a lot safer than using a weak password. I don't have a citation handy
but security researchers often support writing down passwords to encourage
using stronger and more frequently changed passwords.

~~~
RobAley
That depends, if it's on a system that's accessible internally only, then it's
likely not safer.

------
voltagex_
Good reminder to add 2FA via
[https://github.com/settings/two_factor_authentication/config...](https://github.com/settings/two_factor_authentication/configure)

~~~
allochthon
I think two-factor auth should be the new basic auth. Every site should
provide it, either directly or through an integration with a trusted OAuth
provider.

I'm waiting for the day that the security people work out a solid federated
single-sign-on system that gains widespread adoption.

~~~
Goopplesoft
Shameless plug: to get alot of sites using 2FA in a standardized manor is
exactly why I made GAuthify
([https://www.gauthify.com](https://www.gauthify.com))

------
forrestthewoods
How is this possible? I would expect repeated failed login attempts would
force a timer making a brute force attack impossible? Or at least more
difficult. Is that not the case or am I missing a key piece of info? (Serious
question, not trying to troll or be smug.)

~~~
pavs
From the blog post:

"While we aggressively rate-limit login attempts and passwords are stored
properly, this incident has involved the use of nearly 40K unique IP
addresses. These addresses were used to slowly brute force weak passwords or
passwords used on multiple sites. We are working on additional rate-limiting
measures to address this. "

~~~
berlinbrown
Seems a bit excessive for a open source repo site.

~~~
jakobe
Actually not. I'm the maintainer of a popular Open Source project that is
hosted on Github. If someone stole my credentials, they could replace the
current release with a binary containing a Trojan.

Looking at the security history page I see a lot of failed login attempts.
Makes me glad I enabled 2-factor-authentication!

~~~
yukkurishite
You put compiled files on github?

~~~
jakobe
Github recently introduced a "Releases" section where you can tag releases and
provide binaries:
[https://github.com/PostgresApp/PostgresApp/releases](https://github.com/PostgresApp/PostgresApp/releases)

It's a great way to distribute Open Source software. (Previously Mattt hosted
it on a personal Amazon S3 account which he paid out of his own pocket; now
bandwidth is generously paid for by Github)

------
null_ptr
Interesting, my account's Security History shows a few failed login attempts
in the last 3 days from Eastern Europe and Southeast Asia, and my account is
not even popular.

~~~
Wingman4l7
A friend and I both showed 6 login attempts from unique IPs and neither one of
us are very active. Seems like they hit a lot of people.

~~~
oneeyedpigeon
I've seen a small amount of suspicious activity (possibly 3 failed logins that
weren't mine); I'm wondering how relevant various attributes of the username
are. I'm starting to move to unique usernames for various services: not
anything vastly secure, but something that's at least harder to automatically
cross-reference. Does anyone know if using much longer usernames is a
worthwhile investment?

------
Wingman4l7
I wonder why Github didn't publicly post the details behind the attack --
namely, that is was an attempt to game a Ripple _(cryptocurrency)_ giveaway.
They mentioned it in their "heads up Github user, your account was
compromised" emails, apparently:
[https://news.ycombinator.com/item?id=6766293](https://news.ycombinator.com/item?id=6766293)

~~~
nilved
That wasn't in my email.

------
nilved
The attack must have been done over Tor. My account was flagged and locked but
it's not reasonable to say the password was cracked. However, I do my day-to-
day browsing over Tor and the matching IP addresses caused a false positive.

Its worth mentioning, though, that the rate limiting is at the account level
and not the IP level. When I was trying to get access to my account before the
email was sent out, my account was locked and remained locked even after
developing a new Tor circuit. How were the attackers able to circumvent the
aggressive rate limiting? Three password attempts a minute is a generous
estimate and it would take longer than the expected life of the universe to
crack even a moderately secure password at that rate (my old password had 704
decillion permutations.) Perhaps some part of GitHub isn't being rate limited?

~~~
ushi
>> The attack must have been done over Tor.

Nope, none of the failed attempts on my account originate from a tor exit
node.

~~~
nilved
I only have Tor logins recorded and nothing indicates that my account was
compromised, so one of the exit nodes I used must also have been used by the
attackers. I guess it doesn't necessarily mean that the entire attack was done
over Tor. Perhaps a zombie in the botnet was running it. :)

------
captn3m0
If these attackers had tried targeted brute-force on some limited accounts,
they might have been successful without tripping over the GitHub security
team. Nonetheless, kudos to GitHub for handling this perfectly.

------
Wingman4l7
Wow, ~40,000 unique IP addresses were used to circumvent rate-limiting.

Thankfully that even with that many unique IPs and presumably a few tries per
IP, you still only have enough tries to crack _weak_ passwords.

------
arasmussen
I'd love to know just how weak "weak" is. Something like [A-Za-z0-9]+ and <= 6
characters, or [a-z]+ and <= 8 characters?

~~~
FiloSottile
Here, it is weak as in "password", "qwerty" or "{your Lifehacker leaked
password}"

~~~
codygman
my password was \w{6}\d{2} and it was compromised.

~~~
mh-
was the \w{6} _github_?

~~~
codygman
LOL, luckily it was not. It wasn't a dictionary word.

------
cik
I'm very impressed by GitHub's response - what a prime example of how you
should handle an intrusion attempt! 11/10 for style from an IT governance and
disclosure point of view. They've successfully: identified the nature of the
attack, notified customers in a polite manner, publicly disclosed the goings
on, and exercised added paranoia (which in this case == security) by disabling
related accounts. Kudos! The only thing they missed is disclosing that they're
working with local law enforcement - something I'd presume they're doing.

More interesting is the attempt. I frequently scan customer systems on their
behalf, by spinning up clouds of resources to hammer their login APIs. By
using Digital Ocean $0.05/hour boxes, and massive threading, tests tend to
cost me ~$50/hour (yes, that's 1000 boxes orchestrated).

Given the real world cost of cloud computing, and the ease of libraries such
as libCloud (or ruby's: FOG), the cost of doing this in terms of time and real
money has dropped dramatically.

------
Damin0u
I wish the weak password blacklist was public, so we can all implement
password check like this easily, and maybe get rid of them forever (even if
banning "chicken" as password is only a way to have "egg" popping up).

------
imdsm
Looking at my security log, it looks like they gained access over two weeks
ago. No failed login attempts, just a successful login. No brute-forcing about
it.

oauth_access.create: GitHub XRP Giveaway - 16 days ago

user.login: Originated from 23.29.121.166 - 16 days ago

------
eliteraspberrie
_Only allow logins from country /continent: X_

With a feature like that, if an IP address from country C attempts to log in
to accounts of users in separate countries A and B, they are quickly spotted.

~~~
itchitawa
Gmail does something a bit more sophisticated than this but it's a real pain
trying to check your mail from the airport in another country when your phone
doesn't work either to confirm that way. Also annoying for VPN users who may
appear to be in another country.

------
Wingman4l7
What was the purpose? Were the repositories private? Was the attack targeted
toward known accounts working on new projects -- i.e., industrial espionage?

~~~
patio11
I'm going to guess it was simple spray-and-pray. (My account took its fair
share of compromise attempts, too, probably because I use the same username
everywhere so it's going to be in many modern dev-heavy password dumps.)

Github is, for better or worse, a high-value target these days. Compromise of
a Github account will often get you credentials which you can use to
compromise high-value targets, for example production machines.

(Example from the Rails world: if someone gets read access to your git
repository, expect them to figure out your secret token which you use for
HMACing sessions to prevent forgery. If an attacker can forge sessions, they
have remote code execution on your production servers. You can assume they
will go directly from code execution to root on those servers, and from root
on those servers to a systemic compromise of your entire network (if they want
to do that).)

Use your imagination on what a mass compromise of accounts gets you. The
simplest possible example is grepping for /bitcoin/, finding the full source
code and credentials for a Bitcoin exchange, and rooting them then emptying
the hot wallet. That results in fairly obvious economic advantage, right? You
could also use scriptable attacks to root thousands of big-n-beefy production
machines on fairly trusted IP addresses, then add them to a botnet, which
you'd use for spamming or various other nefarious activities.

Then, even farther down the list, you have a dedicated adversary actually go
through every repository they got access to and look for something uniquely
fun to do with that particular target.

If you were looking to do industrial espionage on a particular target, you'd
probably pick a way that was less likely to be detected and cause a company-
level security response. (Use passive recon to identify one or a few likely
target email addresses, find one or a few likely passwords for them, and try
them first. Heads you win, tails you just left evidence in a log of _perfectly
normal user behavior_. You might follow up this perfectly normal user behavior
with a social engineering attack.)

~~~
tedunangst
Is there a setting I can change to turn off this "session -> code execution"
feature?

~~~
patio11
You could, hypothetically, move from CookieStore to a different session store,
but I am not positive that this will mitigate the vulnerability (by making it
more difficult for the attacker to get write access to the session, not by
making it more difficult to turn a session into code execution), and do not
have a few hours to reacquaint myself with the code to give you a firmer
answer there.

I am weakly confident, on the basis of some spelunking in the Rails internals
in February, that if you get attacker chosen data into the session you're
uniformly hosed on (at least) Rails 2 and Rails 3.

~~~
rmc
_if you get attacker chosen data into the session you 're uniformly hosed on
(at least) Rails 2 and Rails 3._

I wonder if you could expand on this? I'm not too knowledgeable about Rails or
security, so I'd wonder how that works?

~~~
patio11
Long story short: If a cookie is valid (HMAC signature matches what you'd
expect given the cookie contents and your secret key), Rails (Rails 2) or Rack
middleware (Rails 3) will attempt to use Marshal.load(txt) on the cookie
contents to populate your session hash. You might naively assume that this
creates a map of strings into other strings, but surprise, Marshal.load(txt)
can create arbitrary object payloads and, through a code path which the
margins of this comment are just slightly too thin to contain, allow you to do
remote code execution.

See: [http://www.exploit-db.com/exploits/27527/](http://www.exploit-
db.com/exploits/27527/)

This can be done in an entirely automated fashion and requires no knowledge of
the application other than the session's secret key.

------
shubhamjain
This reminds me the #1 tip of web development: Never trust the user. I can't
imagine there is a person out there who has all the time in the world to brute
force accounts with common phrases with 40,000 proxies (or whatever).
Seriously, is there anything worth gaining if you get hold of even say 100s of
accounts.

~~~
Wingman4l7
Well, first of all, it's very likely they used a botnet and some automated
attack software. Secondly, there was a financial incentive[1] -- my back-of-
the-napkin calculation was ~$15-25 per compromised account. That of course is
ignoring secondary values like API keys, reused passwords allowing access to
accounts on other sites, etc.

[1]:
[https://news.ycombinator.com/item?id=6766293](https://news.ycombinator.com/item?id=6766293)

------
welder
> This is a security log of important events involving your account.

> 18 hours ago user.failed_login: Originated from 178.245.129.47 (Istanbul)

> 3 days ago user.failed_login: Originated from 180.250.45.186 (Indonesia)

> 3 days ago user.failed_login: Originated from 123.119.141.184 (China)

Is this a distributed brute force attack against my account?

~~~
dbaupp
I see similar a thing; I'd guess that they just tried a few common passwords
against millions of accounts, rather than targeting some specific accounts.

------
benaiah
Had five attempts over the last 2 days, four from Venezuela and one from
Bangladesh. All unique IPs.

~~~
samweinberg
Same here. 5 attempts over the last 2 days all from unique IPs. Four from
Venezuela and one from Beijing.

------
dm2
Along with 2-factor auth, what about the option to block all IPs from outside
of a certain country (if you live/work in the US, only allow US logins).

This obviously won't work for everyone, but might prevent a decent amount of
brute force successes.

~~~
rythie
Facebook detects this when you move countries and makes you do a quiz about
your friends pictures - though it can be pretty annoying [though I haven't
seen it recently so maybe they've stopped].

~~~
krlkr
I just dealt with this today with my own account using a proxy, quite a clever
security approach IMHO

------
andyhmltn
Logging into my account, I can see quite a few failed attempts from a variety
of IP addresses. All of them are from the same ISP in Venezuela. Quite scary,
but my password is 40+ characters long and I have 2FA enabled. How weird!

------
ismail
Single failed login attempt from china. Though could this be any way related
to the adobe leak? If you have the list of emails and password, you could try
a bunch of commonly used services and see if that works.

~~~
objclxt
GitHub do say in their post:

> _These addresses were used to slowly brute force weak passwords or passwords
> used on multiple sites._

..."multiple sites" could possibly refer to a compromised list, like Adobe? Of
course, it doesn't have to be Adobe - Macrumors was compromised recently, etc
etc.

------
uladzislau
Can Fail2ban [http://www.fail2ban.org](http://www.fail2ban.org) help in this
case?

~~~
nobodyshere
Not really. Some ISP provide same external IP to multiple users and you have
to take that into account. I think otherwise a good part of github's users
would never be able to access it.

------
cft
Just show captcha randomly on login, with the frequency proportional to login
failure rate for an account.

------
ya
ripple.. you bastard,.

~~~
Wingman4l7
It wasn't Ripple, it was someone trying to game their giveaway:
[https://news.ycombinator.com/item?id=6766293](https://news.ycombinator.com/item?id=6766293)

