

Main GNU source repository server compromised - there
http://savannah.gnu.org/

======
Sandman
I see a lot of angry comments on their webpage, people asking why somebody
would do this. Well, the real answer is probably - simply because he/she
could. And to all of those wondering how anybody could do this to a project
that hosts free software, well, the attackers don't really care. They saw the
exploit and used it. Personally, I expected that a project such as Savannah
wouldn't be vulnerable to an attack as simple as SQL injection.

~~~
dkarl
Maybe someone had an idea for putting a vulnerability into a GNU utility.

 _Personally, I expected that a project such as Savannah wouldn't be
vulnerable to an attack as simple as SQL injection._

Seconded. But then again, the GNU source code is mostly C, isn't it? They're
used to doing things that most developers would consider intractable and
therefore impossible to do safely: such as comprehensively sanitizing inputs
so you can safely construct a query string out of them.

"Hey, those web guys say sanitizing inputs is effectively impossible. Should
we really be doing this?"

"Snort. That's what they say about manually deallocating memory, too."

"Oh yeah! What a bunch of wimps! I'll get right to work on the query builder."

~~~
rryyan
Going further: how about a vulnerability in every program compiled with gcc?
I'll just leave this here:

<http://cm.bell-labs.com/who/ken/trust.html>

------
raphman
From the comments on the linked page:

 _Date: Tue Nov 30 01:34:15 2010 Pseudo: mjflick - Savannah Hacker Comment:
re: HappyCrow,

Only one project was targeted in this attack.

Second, a postmortem will be forthcoming, as well as more information shortly
from the FSF staff, who are planning on making an official announcement about
this._

------
pluies
Ouch. This is bad.

It illustrates how much of a pain SQL injections are, if they can affect the
GNU project which has some of the most incredibly talented hackers worlwide...

Props to them for having a working backup strategy though.

~~~
petrilli
Actually, no... you have to be pretty ignorant to be subject to them. Use bind
variables in your queries and you are done. That it's still one of the top 2-3
exploits tells you how horrible most developers really are.

~~~
tptacek
No. You are _not done_ with SQL injection just because you use "bind
variables". Plenty of things can't be parameterized that are nonetheless
subject to user influence. Click the top of any reverse-sortable column in a
web user interface to see one obvious example.

Parameterized queries are a good thing, and you should use them, but I feel
like I've had to be a broken record about this over the last week: they are
not magic anti- SQL injection fairy dust. Our whole team spends most of its
time looking at smart people's code, and we routinely find SQL injection.

Plenty of people who aren't ignorant of SQL injection manage to let SQL
injection slip through. Knowing how SQL injection works and being able to
devise and implement engineering procedures to reliably prevent them are two
very different things.

~~~
btilly
_Click the top of any reverse-sortable column in a web user interface to see
one obvious example._

People don't do those in JavaScript?

That's what I did a decade ago. Well, then I moved it back to the server after
we got tired of the performance problems that JavaScript had back then. But
today it wouldn't be an issue. And when we moved it back, we were careful not
to have an SQL injection attack. If memory serves we actually did the resort
in Perl. (In our defense, much of the data we were serving lived in flat
files, or were generated on the fly from a compute server, instead of coming
from a database.) However at another company I had the same problem, and I did
the obvious "process CGI parameter, insert appropriate ORDER BY statement".
Where the definition of appropriate was by column position, from which I
worked out the field to sort by, so I didn't have to trust the client for the
name of that column.

I should back up. We didn't have any SQL injection attacks that I knew of and
were reasonably careful. But that code base did not get audited, so I can't
really know that. However after the next company that I worked for got bought
by eBay, they did a penetration test on us. The worst thing that they found
was an open redirect that could be used to let a spammer construct a link to
any web page with us as the referer.

I'm happy to use this as a testament that programmers really can avoid SQL
injections. However their surprise that they didn't find any SQL injections in
our code supports your claim that most teams fail to do so successfully.

~~~
tptacek
I'm not sure what you think I'm arguing; that it's impossible to avoid SQLI?
Of course not. All I'm saying is that parameterized queries aren't the end of
the story; the teams that believe they don't have SQLI because they use
parameterized queries are the ones that are going to lose their apps to SQLI.

~~~
btilly
What is the full story?

It seems to me that two tricks nail it. First use parametrized queries. And
secondly if you have information you need to send/receive from the client that
isn't easily parametrized, have a limited list of possible things that can be
accepted back, which is checked in code while building the query.

If you're doing those two things, I don't see how much work it is to avoid
SQLI. Furthermore if you're using a reasonable ORM, then you should get both
of those pretty much for free. (Well you have the overhead of learning the ORM
itself.)

~~~
tptacek
There's almost nothing wrong with your second "trick" (when we write
recommendations, we usually suggest people never allow users to directly write
syntax, and that they structure their app so that the parameters the user sees
couldn't possibly land in SQL; use "1" for "ASC" and "0" for "DESC", etc).

But it's not nearly as powerful a statement to say "use parameterized queries
and then do everything else right" as it is to say "just use parameterized
queries and you won't have this problem", is it?

~~~
Dylan16807
How about "only pass user-sourced data in parameters"? So anything that isn't
a parameter is coming from the app, possibly as a response to user data, but
not actually _from_ user data. This statement seems just as powerful as the
original. Anything misleading about it? Somewhere you can't use parameters
_and_ can't simply use app-supplied query pieces?

~~~
tptacek
You see how we're converging on "just use proper input validation", which is
the all-time least useful piece of advice in software security?

~~~
Dylan16807
Because input validation is hard to do. A rule to not pass input EVER (outside
of parameters), even in a supposedly validated form, is something different.

------
viraptor
Ok... I hope I misunderstood this, but does anyone see a problem in the order
they're doing this in?

    
    
        [X] Reset passwords
        [/] Fix SQL injection and look for potential others
        [ ] Implement crypt-md5 support (like /etc/shadow, strong and LDAP-compatible) hashes
        [ ] Implement password strength enforcement
    

So they're basically forcing password resets, even though people can use
"password123" in plaintext on a site that's not completely audited for
vulnerabilities?

~~~
lukeschlather
You eliminated step one, which clearly states:

[X] Put services online using backup, __except for password-based ones __(e.g.
the web interface)

The other step you omitted is the one where they allow people to log in after
fixing the vulnerability.

As for the audit, there's no such thing as a complete security audit, and
there's no reason to keep the site down in fear of hypothetical holes.

~~~
viraptor
Sorry, my mistake. I got the website<->other services restoration time
backwards. Thanks for correcting :)

Re. audit - of course you cannot be sure, but once you get taken over and your
system is offline anyways, it might be a good idea to at least grep the
sources for queries and quickly check for obvious stuff. Which is what I
assume they're doing.

------
sandGorgon
Question - if they had been using a DVCS which checksums every commit, would
they have been safer ? Presumably, you could build a tool that stores a list
of checksums and if you had to do a restore, that tool could validate the
backup.

Or is it that, with a little effort, DVCS' can be hacked/corrupted
fundamentally ?

~~~
JoachimSchipper
I don't think DVCS would have helped in any way. It's not the repository
server itself that was compromised, AFAIK, but "only" the web interface. This
allows password recovery and thus allows people to e.g. make malicious commits
from a trusted account, but that would be no less true if there were a DVCS
under it all.

~~~
sandGorgon
agreed - but my question still stands: Take SVN for instance - if I managed to
hack into a SVN server, can I change the source code (under SVN management) in
a way that would be undetectable ?

Repeat the same question for git, mercurial and CVS.

~~~
JoachimSchipper
For CVS and svn: you can hide it until someone actually looks at the relevant
code. For git: not really (as long as you assume you change only the _data_ \-
clever attackers could serve one repository to the developers and another to
.mil/the rest of the world/...) I don't know about mercurial, but I'd wager
it's the same as git in this respect.

In either case, an attacker could add arbitrary commits, and the system
administrator would roll back to the last known-good backup as soon as the
compromise was detected. I don't think DVCSes help much here.

~~~
sandGorgon
_For CVS and svn: you can hide it until someone actually looks at the relevant
code. For git: not really_

 _I don't think DVCSes help much here._

Those statements sound contradictory. Let me try to understand - the
_distributed_ aspects of DVCSes would not have helped here. But the incidental
fact of checksums in DVCSes (maybe necessitated by the nature of
"distributed") does help. Right ?

~~~
JoachimSchipper
Not all that much. If I hack your SVN repository, I put something like "if
($password == "1337h4xx0r") $admin = 1" into an old revision and hope you
don't notice it when inspecting code. If I hack your git repository, I make a
commit with text "spelling fix: invole -> involve" and slip in something like
the above. Yes, it's a bit easier to detect, but not all that much - most such
breaches seem to be found by system administrators before the developers
notice them. (The kernel.org source code breach was detected because some
mirror failed to line up with the main repository, for instance)

Of course, a smart hacker would do something less obvious. See the obfuscated
C contest for inspiration.

And again, this assumes that I do not have any other access. In reality, I
could change your $PATH to invoke a trojaned git binary, or somesuch.

------
konad
Headline missed a word : "again"

Original issue date: August 13, 2003

<http://www.cert.org/advisories/CA-2003-21.html>

ftp://ftp.gnu.org/MISSING-FILES.README

    
    
                                  Moving Forward
    

All releases after the 2003-08-01 date will have checksums GPG-signed by the
GNU maintainer who prepared the release. This assures automatic certification
of the integrity of all GNU source from that date onward.

~~~
tjr
Slightly different, as that compromise was of the FTP servers, which store
packaged releases. Savannah is the development system.

------
linuxaddict
I thought more eyes on the source means that OSS is impenetrable. Guess not.

~~~
technomancy
Nobody really cares about helping out with Savannah. On the emacs-devel
mailing list every time Savannah is mentioned, it's in the context of "Yeah,
we would like to do that but lack the manpower. Any volunteers?" followed by
crickets.

------
SriniK
What a moran whoever did this.

~~~
konad
“But he has nothing on at all,” said a little child at last. “Good heavens!
listen to the voice of an innocent child,” said the father, and one whispered
to the other what the child had said. “But he has nothing on at all,” cried at
last the whole people. That made a deep impression upon the emperor, for it
seemed to him that they were right; but he thought to himself, “Now I must
bear up to the end.” And the chamberlains walked with still greater dignity,
as if they carried the train which did not exist.

