

Ask HN: Site hacked, where to begin?  - markstahler

I was just contacted by someone I know who came to own a well visited site in the past few years. I was asked by him to do a little bit of work for him: to switch hosts and perform some minor updates to their existing site. No problem I thought. Well, I have since learned the cause for moving hosts. The site had been hacked over the holidays and now has a threat warning from Google. A backup has been restored by the host (which still includes some badware redirects) and there is no source control or revision history.<p>The site is very large and I am still trying to get organized. Media files, multiple copies of pages with php.bak etc everywhere, horrible horrible mess. He complained about their last 'webmaster' being a nut. Yeah... Can you guys please recommend site scanners or any tools you would use to find any threats on an unfamiliar code base? I am hooked up with Google Webmaster Tools which is alright but I have found threats on pages they don't have listed. Any tips would be very appreciated. Thanks
======
cd34
rsync and take a snapshot, don't work from the snapshot, just keep it around
in case you screw up a sed/rpl/awk rm, etc. During portions after verifying
that your mass replace/delete/etc was successful, take another checkpoint
snapshot so that you have a known good to go back to rather than working
through everything once again.

things to look for '<iframe' '<script src'

You're likely to find some script src that refer to javascript loaded from the
site, check the very end of each of the existing .js files

Check the .htaccess and make sure they look sane and haven't added the ability
to run other extensions as cgi scripts or php files. Check your .htaccess for
any php prepends, odd mod_rewrites.

Clean up obviously dead files, i.e. the .bak files, etc. Look in the
image/uploads directories for any .php files, consider putting in mod_rewrites
or remove mimetypes to make sure code can't be executed in directories that
should be static content.

Dump your database, again search for iframe/script that is attached to fields
that shouldn't have html, i.e. category names, etc.

get webmaster tools running, it will give you a little more information about
what to look for.

Now that you've gotten the easy stuff out of the way, you need to look for
ways that they did this. Since you've moved hosts, you've lost some of your
forensic ability since file ownerships and permissions may have changed. Now
you've got to look for scripts that contain potential badware.

(r57|c99|c100)shell passthru, exec, system, eval, popen, base64_decode, rot13

Hopefully remote includes are disabled, globals should be disabled, etc.

You're going to get a lot of false positives. Once you have that list, you can
weed through and probably catch them quickly. If you believe there are remote
shells and you're not really sure what they are running, consider installing
suhosin, look for the line in execute.c around line 1588:

goto execute_internal_bailout;

and comment that out, run it in simulation mode,

    
    
        suhosin.simulation = on
        suhosin.executor.func.blacklist = include,include_once,require_once,passthru,eval,system,popen,exec,shell_exec,mail
    

Check your syslog for things that are reported.

When you look for code, you'll see things like:

    
    
        eval($_COOKIE['blah']);
    

which allows them to execute commands, but, using a GET rather than POST so
that you don't see the requests as easily in the logs. Since you're mentioning
malware, it was installed somewhere, remote shells are probably the main
reason, but, a lot of malware that is appended to a number of files is done
through FTP. I would suggest that they change the FTP password. Definitely
grep the logs for any POSTs just in case.

Not that you want to hear it, but, typically a webhost's backups are for
disaster recovery, i.e. hard drive crash. If it hasn't been detected quick
enough, their nightly or weekly backup could get overwritten before it is
noticed.

Most of the exploits I see modify php files and add some code at the bottom
through FTP, or, do a global search and replace with a remotely executed
script.

------
markstahler
I found this link on Delicious which is helpful:

<http://25yearsofprogramming.com/blog/20071223.htm>

It directed me to <http://www.unmaskparasites.com> which has helped me uncover
that there is a 302 redirect on one of my pages somewhere.

------
JoachimSchipper
I'd try to get older backups from the host and diff them; hopefully _someone_
has at least one clean backup that you can try bringing up to date.

Otherwise, you can try grep'ing for known bad stuff (base64, if the attackers
like encoding their stuff in that), but it's _very_ hard to recover if the
attacker is good. Fortunately, most of them are not; but take a look at, say,
the Obfuscated C contest for some "inspiration".

Oh, and you may want to figure out how the hack happened, too. I'm not too
familiar with software for this kind of things, but I seem to recall that RATS
([https://www.fortify.com/ssa-elements/threat-
intelligence/rat...](https://www.fortify.com/ssa-elements/threat-
intelligence/rats.html) ?) has a PHP mode.

------
iuguy
If you're not sure what to do, then I'd suggest backing off. While I
appreciate you want to find the cause you might be out of your depth. There's
plenty of professional companies that could deal with something like this to
audit the code for badware.

Disclaimer: I run one of said companies in Europe, but am not putting it
forward here. If you're in the states you might want to give a company like
Trustwave or Matasano a call.

