
How my Apache server became a malicious free internet proxy - MariuszGalus
http://blog.atrament.net/how-my-apache-server-became-a-malicious-free-internet-proxy/
======
SignMeTheHELLUp
"How my Apache server became a malicious free internet proxy"

tl;dr: Negligence, and failing to RTFM.

What really horrifies me is the author doesn't seem to understand the
magnitude of their error. The final quip at the end illustrates this. "Ha!
someone searched manslaughter over my proxy! I had a lot of fun reading my
open proxy logs..."

I wonder how many stolen credit card transactions were done over his proxy,
causing headaches for many innocent people? Or worse?

~~~
Laforet
I've had similar issues with an open source project in which a simple proxy
was established with FiddlerCore to tap traffic to a web browser, pretty tame
stuff and nothing malicious.

Problem is that by default it was configured to listen on 0.0.0.0:80, making
it an open HTTP proxy that everybody on the same LAN could connect to. The
only real threat so far is that somebody could send in a large volume of
traffic to crash the proxy, but wait and behold....

...some users were running it from hosts that are either a) directly connected
to the public IP space without a firewall and b) behind NAT, but with lazy
DMZ/port forwarding configuration that exposes their port 80 to the internet
anyway. For about a year people have been obliviously hosting open HTTP
proxies from home.

Eventually somebody found out and it took another couple of months of back and
forth issue reporting and PR tugging battles to get it properly patched. Opsec
is hard.

------
bognition
This is a perfect example of why most people should not run their own
hardware. Don't get me wrong its really fun to build and configure your own
server and I openly encourage people to learn but I also remind them that its
extremely difficult (for a novice) to do securely.

Additionally connecting a misconfigured server to the internet doesn't just
hurt the server owner but the entire network is affected, as you are providing
another piece of hardware that malicious actors can use to execute their
attacks.

~~~
codezero
I was going to say the opposite. It's awesome that we can spin up boxes and
host our own servers, and on top of that, learn from our mistakes. I doubt
there's a person here who's never been host to malware, spam or some other
malady as a result of some of the experimentation they've done as they learned
more about computer systems.

~~~
bognition
This is exactly my point. One of the major reasons that malware and malicious
actors have been able to do as much as they have is because of the large
number of misconfigured devices on the internet.

The internet has evolved beyond a network cobbled together by a bunch of
academics and engineers -- its a critical piece of infrastructure.

~~~
rlpb
The Internet is kept running by a bunch of people who experimented in this
way. If they didn't do it any more, in a couple of generations we wouldn't
have any competent Internet engineers any more.

~~~
codezero
Thanks, this puts what I was trying to say in a much more concise and clear
context :)

------
wiradikusuma
Honest question from a developer perspective: Why there isn't any "best
practice/hardened by default" wizard-style configuration, something people can
do right after they install their OS? E.g.:

Welcome to Best Practice Linux. Click Next to continue. Which http server you
want (httpd/lightttpd/...). Click Next to continue. (you get the idea).

Something like apt-get but with best-practice defaults.

~~~
AgentME
Apache doesn't default to acting as an open proxy. It already has safe
defaults! He specifically configured it this way.

------
userbinator
As someone who's used open proxies to get around geo-IP-
tracking/restrictions/censoring, I get the point about excessive bandwidth
usage (you can apply per-IP ratelimiting for that), but it does make me a bit
sad that open proxies are now considered "malicious"...

~~~
MariuszGalus
I used the word malicious because I saw people and have a list of credentials
now from compromised accounts and spambots. Mainly from Russia. Also, all the
sport betting websites that were being hit. I think there must have been
something shady there. I was also used for ad-click fraud. :|

------
nostalgiac
So you got to the end solution of... uninstalling fail2ban to fix it? You
didn't bother to check WHY it was maxing out the cpu?

Glad you got the issue resolved though and didn't fork over the $10 because
you would've just run into the same issue in the future if you didn't get to
the root cause of it (misconfigured Apache).

~~~
rtkwe
At this point fail2ban wasn't really needed anymore since the author installed
it to stop the people using the server as an open proxy. After they solved the
proxy issue fail2ban was just causing problems and wasn't needed.

~~~
nostalgiac
I understand it wasn't needed after the fact. But the entire point of the
original post/resolution was to understand what was causing the problem (page
time outs) - yet when faced with a second problem, he chose the 'easy' route
of just uninstalling it (unalike his resilience to pay for his IP change or
wiping the server).

------
jawshie
Any idea what the actual vulnerability was?

~~~
Sanddancer
It's really, really easy to misconfigure mod_proxy and set yourself up as an
open proxy. The ProxyRequests directive sounds like it should be needed for
any sort of proxying, but is only really needed if you're allowing your apache
instance to act as a forward proxy, not as a reverse proxy. For reverse
proxying, which is what you want most of the time, you really want ProxyPass
and ProxyPassReverse .

~~~
scintill76
The phrase "[my blog] was being hosted on another port because apache was
taking up the internet http port 80" sounds like the reason they were trying
to set up a reverse-proxy.

Apache docs have an obvious warning about ProxyRequests and security:
[https://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxyre...](https://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxyrequests)
.

This config snippet looks like it was copied/modified without understanding:

    
    
            <Proxy *>
                    AddDefaultCharset off
                    Order deny,allow
                    Allow from .example.com
            </Proxy>
    

Example.com? If you read the docs on Order
([https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#or...](https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order)),
you see that Deny,Allow defaults to allow, so that's why it's an open proxy.

Above that, there is a comment "turning ProxyRequests on and allowing proxying
from all may allow spammers to use your proxy to send email", so I guess it
was somewhat safe originally, until ProxyRequests was changed to On without
reading and understanding the comment.

~~~
MariuszGalus
I made the mistake of thinking it was harmless to enable. Also, with the
solutions I've found online for enabling 'ghost blog with apache
virtualhosts'. I guess someone trolled me.

~~~
jrochkind1
It's a good idea to always look up the docs on directives in apache configs
you are copy-pasting from the internet, to make sure you know what they are
doing.

For that matter, this probably applies to just about anything you copy paste
on the internet. Understand what you're pasting, look up the docs if you don't
or aren't sure or are using something you haven't seen before.

But apache httpd configs can be especially tricky. The accidental open proxy
is definitely something that gets lots of people, you are not alone. The
apache httpd directive names have a lot of 'legacy' in them, and probably
should have been named more clearly in retrospect (i assume the apache httpd
forward proxy feature came first, and reverse proxy was only added later; but
in 2015 reverse proxy is a lot ore common a thing to want).

(But the solution to an accidental open proxy, if you didn't mean to be
forward proxying at all.... is turning off the forward proxy in apache httpd,
not other weird workarounds).

------
mkhpalm
As a long time Apache user I've never understood using its proxy modules for
stuff like this. I've always felt like its much cleaner to just use a small
daemon process built solely for the task of reverse proxy or balancing. e.g.
haproxy, pound, etc.

