This works great, except that some sites seem to block requests from EC2 hosts. StackOverflow and Yelp are two that come to mind immediately although I'm sure there are others. If I remember right, StackOverflow only lets you access via the API if you're on an EC2 host.
On the other hand, I can see where they're coming from by banning the whole netblock. Otherwise you could scrape until your IP get banned for blowing a rate limit, then tear down that instance and spin up a new one.
I get this same message about once a week since I frequently use my Linode VPS as a proxy for my web traffic, but it's never lasted for more than a half hour or so. I've always wondered why their heuristics only seem to notice me so infrequently. Do you get it consistently?
Yes, I have never been able to access Google from my Slicehost VPN IIRC. I also used a Linode VPN for a while and got the message infrequently with the possibility to enter a CAPTCHA to complete my request.
I also connect a lot through an EC2 machine that I use as a proxy. I do this because I do not live in the U.S. and many websites require U.S.-based IPs.
It's a pity that StackOverflow doesn't allow me to do this (so I end up turning the proxy on and off and back on again). It's not like you couldn't rent a cheap Linode instance (or from another provider, take your pick) and do whatever you want with SO, if you really wanted to.
SO needs more sophisticated tools to block the access for bots/crawlers. IP blocking just doesn't cut it and tends to discriminate against legitimate users with special needs.
I'm kinda baffled how this recipe for using ssh, rudimentary sysadmin skills and scripting is deemed news. Maybe back in the heyday of Slashdot, this would be half-interesting -- like, uhh, when OpenSSH actually introduced tunneling, or when Iodine was introduced, or whatever... At best, a "protip" everyone could've discovered himself through a few minutes of Googling.
Not so much 'news' but perhaps a quick hack, nice reminder, a quick intro for the less experienced (who may not know what they are missing in the first place). Or perhaps someone who has set this up before in a different way might want to change how they do it.
Because not everyone knows everything, and the entire point of being HACKERS (I assume the uppercase is useful) is to promote learning and sharing knowledge.
So whilst this is probably not front news worthy of LEET HACKERS it's probably made someone go 'hey thats cool' and that's all it needed to do.
No, it implies that you think that the other person is stupid AND they use some form of Unix. Any way you put it, your response is unfriendly and rude. And Hacker News should not be known for arrogance. In fact, the person reading this article might not know much about tunnelling, but maybe writing the next greatest app. You won't know who you are displaying your arrogance to, so don't do it.
… because we are not all doing the same kind of hacking. The variety of news here is proof for that.
Back to the topic: I my own router to create a tunnel similar to the one described by the OP. It is not as fast as an Amazon instance but free wi-fi is neither.
Sadly as HN has become more popular it really ought to be renamed Larval Stage Hacker News. Recently some pretty bad articles have made the front page, like The 5-minute Guide to C Pointers which embarrassingly conflates arrays and pointers, or my personal pet peeve, the endless stream of puffy $EDITOR advocacy posts, for those folks who are still on the fence about what tool they're going to use to edit a fucking file.
Anyway, as much as I complain about all the puff pieces, there are still plenty of articles with real technical meat on HN, you just got to wade through some crap to get to them.
Actually, a lot of us come for the articles but stay for the conversation. That pointer article may have had an error, but it was discussed in the comments.
That's the hacker ethos - show what you know, if you are wrong someone will point it out and others (and hopefully youtself!) will learn from it.
The sort of attitude you are display is the sort of hypercritical one that prevents people from submitting "Show HN", which is a real pity.
Agreed, I'm also not sure why he didn't just use the SOCKS proxy in ssh as opposed to sshuttle. ssh -D8080 and then point your network config to localhost 8080 as the local proxy and everything is encrypted.
My trick is usually just to setup a socks5 tunnel to a remote machine running OpenSSH. The nice thing about this is that you don't have to install anything on your system, and most people have access to a remote Linux/BSD-based server.
Nifty one-liner to set it properly up: "ssh user@host -D 8080". Then you just point your system wide proxy to localhost:8080 and magically see all your data be tunneled.
I have read the documentation, and it's pretty nifty. But I rarely have to use it anyway, so I've stuck with a solution that is not dependant on any other software than what ships nativly with my OS.
Came in to talk see if anyone had already posted this. Hadn't heard of the sshuttle command mentioned a couple times in this thread. Like you mention below it's nice to just have everything built in ready to rock and roll.
Used the command for years to my own boxes. The amazon spin up version I guess is cool. I'll spin one up at some point just in case. Never know.
The typical environment variable for socks proxies is SOCKS_SERVER, so: export SOCKS_SERVER=127.0.0.1:8000 in conjunction with ssh -D8000
Unfortunately many utilities don't support SOCKS or have non-standard flags, so you would need a wrapping program like tsocks to force TCP through your proxy.
DNS is an application protocol, TCP is a transport protocol. You can't be tunneling TCP over DNS, DNS (when not using UDP) actually uses TCP port 53. Are you sure that you are just getting SSH to listen on port 53?
I have to interject here - not to blow my own trumpet, but as a 'point of fact'. (Oops... once you start correcting misinformation on the internet, you are going to be busy for a long time.)
Before that there was a long standing existing technique to tunnel data through UDP packets that simply pretended to be destined for the DNS port (53). That stopped working if the network admin filtered outbound UDP and forced people to use their local DNS server instead. My method still works in that scenario though.
(If anyone knows of an earlier reference to the method I posted about, please let me know.. for all I know it was a well-known tactic in the underworld before I posted to bugtraq.)
From Julian's post, it's not possible to see which of the two methods his code used, since the rb file seems to have disappeared. I suspect it was "my" method.
I do like the ppp interface through - mine just tunneled bash commands + responses.
Oskar, I didn't know you were an HN'er. Thanks for posting. I have studied malicious use of DNS for a long time and have not yet found any reference prior to your post in 1998. In fact, I used your original bugtraq post as a reference to kick off a whitepaper detection solution approach for enterprises: http://armatum.com/blog/2009/dns-part-ii/ I'd sincerely welcome your feedback.
HN'ers -- Despite Kaminsky's ego, all signs indicate Oskar invented DNS tunneling.
Typo. I searched for "tcp-over-dns", the first sentence of my comment said TCP-over-DNS, and tcp-over-dns is what several other comments on this thread are describing.
Fair enough :-) I'm actually quite amazed this is even possible, but quite happy to have been proven wrong! You learn something every day on this site...
Back in 2005, Google offered "Google Secure Access", a short-lived service to provide secure VPN access across insecure Wi-Fi networks. I'm unsure why Google dropped the service so quickly, but there is still probably a good business opportunity in this space. Business travelers stuck using airports' insecure Wi-Fi and hotels' captive portals might be eager to pay for turnkey VPN tunnels. Like this blog post suggests, you could spin up EC2 proxies as on demand.
I use a service called Cloak (https://www.getcloak.com/), which I love and recommend to people all the time. It does something similar, but it automatically detects when you're on an unsecured network and automatically does everything for you.
Highly recommended. The author of the program hangs out on HN sometimes too.
Another Cloak user here - I was able to stream Netflix on my flight this week due to running it through their proxy. No infrastructure to run, and fully integrated with OS X's wifi selector. It runs on multiple cloud platforms to boot, so if someone is blocking EC2 you might have better luck.
Nifty trick, but some services kill ssh on port 443, and I guess they do so by limiting connection lifetime. In these cases, I find GNU httptunnel very useful.
This is interesting but incomplete. It may not survive to deep packet inspection or rate limiting. Even worse - your only S3 machine could get blacklisted, and I don't see why operators wouldn't share blacklist the way it's done to fight spam.
So my suggestion is to have multiple alternatives instead, as in lines of defense.
First, a simple SSH server also running a socks and a http proxy is useful. Easy to reach and configure - some browser work better with http proxy (or was it sock? Opera only supports one but I can't remember which at the moment)
In the second line, you want to run the SSH server on a nonstandard port as the author does. I suggest multiplexing the port : simply run SSLH on both port 80 and 443, with the patch published on http://rutschle.net/pipermail/sslh/2012-April/000202.html
But that will not look like a standard SSL connection. Even worse, if you use your own certificate - operators could deny all self-signed certificates.
So get stunnel running on port 443 with a "usual" certificate instead, and pass the traffic on port 80 where you'll be happy to have SSLH or whatever you prefer - like Openvpn which can do port sharing or which can be put "behind" stunnel and sslh, so that you will use a standard 443 port, and do a genuine SSL connection on that port. Much better. Have some fun with burst.net - they offer you 2 IP for like $7/month : you can do some routing and give the 2nd IP address to your laptop on the other end of the tunnel. Now that is freedom :-) Every port will be opened in both ways :-)
Still, I do not believe anything of that will be enough to resist to statistical analysis. I've seen interesting tools, but if I down to that I'd better use a standard DNS tunnel (iodine, dns2tcp, ntsx, whatever).
Once again, multiplexing port 53 could be good to have them on different subdomains (you never know which tool you'll have on hand, or can easily recompile on the go) or to run a standard DNS server too (named, maradns, ...).
It will be a good idea to "rate limit" one of them because having too much traffic flow on port 53 to a given IP is something that stands out. (And I have seen DNS tunnels broken by a simple rate limit rule)
I found something interesting to multiplex DNS request in C, which name I don't remember at the moment. If anyone is interested I can dig that up.
Finally, deploy all that setup on at least 2 machines, using different domains and different subnet so that if one of your "lines of defenses" goes down (ie get blacklisted) you can safely move to the next one.
Total time it takes : 2 hours, then you'll always have a way out of a locked connection.
The need for "defense" implies an adversarial game with an opponent. I'm betting that the people who set up this security policy did it out of negligence rather than actively looking at port 22 and deciding not to allow it.
You could identify it as ssh vs SSL because they have different unencrypted headers. So systems which check that you are speaking SSL on port 443 would block ssh.
With HTTPS you get a handshake Client Hello packet sent in the clear, which has a TLS version identifier and a plaintext session ID among other things. With SSH2, you get a literal "SSH-2.0" as part of the protocol identifier which appears before the key exchange.
Instead of using an Amazon machine, why not use a low-power ARM device? I have been running a small ARM computer at home for doing sshuttle, iodine, and samba over ssh (with port binding) for the past few weeks and it works very well. I know tcp over tcp is usually not good but it still works quickly for some reason. This costs less than a VM on Amazon and gives you more flexibility.
A nicer sshuttle command for VPN may be:
$ ./sshuttle -v --remote=server_USERNAME@serverIP:port_number --dns 0/0
If you sshuttle to your home router you may be able to samba or nfs to the ARM on your home network (using tcp and udp, I think).
Alternatively, SSHFS or something more normal could be used for mounting the ARM filesystem without a VPN but if you want to try Samba over SSH you could try:
$ ssh -C -c blowfish -L[host_bind_port]:localhost:445 server_USERNAME@serverIP
$ mkdir /Users/username/mount_spot
$ mount -t smbfs //server_USERNAME@localhost:[host_bind_port]/server_drive ~/mount_spot
Tools like rsync over SSH and UFTP are also nice for moving large files to and from the ARM server.
The sshuttle docs explicitely mention that it doesn't do TCP over TCP (rather as I understand, it's a transparent TCP proxy). I wonder how it handles UDP, my main interest in a VPN is for VoIP (SIP); I guess turning UDP into TCP still won't fly for this, especially over 3G. I guess I'll have to try it.
If you run an openwrt/dd-wrt firmware on your home router you can bounce of it too. Just configure ssh access on it and you have pretty much the same thing.
Instead of setting up a standard proxy, you set up a standard ssh server, which can act as and/or tunnel to an internal proxy (e.g. by using the ssh -D option to create a SOCKS proxy).
I've been doing something similar for almost 8 years now. I wrote a Ruby script to do the heavy lifting of tunneling all my TCP traffic through an HTTPS proxy.
I first wrote it to circumvent the firewall at a summer job to be able to connect to IRC, and I recently got it out of the dust to do something similar at my current job. I couldn't use anything pre-existing, because I was on a tightly locked down Windows system. This approach only required some kind of portable Ruby runtime.
Corkscrew does pretty much the same thing, but my script doesn't require an SSH server to run on port 443 and does away with SSH encryption for most connections. The latter can be either a good or a bad thing, depending on your use case.
You can tunnel out that way just fine, but even with setting up dynamic SOCKS forwarding (ssh -D [bind_address:]port) it's less effective and more work. You're dependent on being able to set proxies per-application, the applications supporting it through some central mechanism (generally flaky), or using proxychains to force an application to use the SOCKS forward (insanely flaky IME).
What sshuttle buys you is transparent interception and forwarding of tcp with firewall rules (no proxies), more reliable DNS tunnelling, packet reassembly that avoids the worst tcp-over-tcp performance nightmares, and control of any or all networks that should be tunneled to. There are a couple of nice bonuses like better traffic reporting, toggle for latency vs. throughput, and the ability to offer tunneling to other local machines, but really it's a decent alternative to a vpn for anything that only uses UDP for DNS.
TL;DR Less tinkering needed, more reliable, more pleasant to use, supports more applications.
Also if an application does not support SOCKS, there is tsocks[1]. tsocks is a library to allow transparent SOCKS proxying. It wraps the normal connect() function.
You can also configure any OpenSSH Server/Client (v4.3 or greater) into a fully-fledged VPN (as opposed to port tunnelling). Granted there are more steps involved in connection than say, OpenVPN, however it is still relatively simple and you get a nice little VPN.
I'm pretty sure that you could also setup an OpenVPN server. It can be configured to listen over TCP port 443, so it can also get past proxies. It may be a bit more setup on the server-side, but it should be a lot easier on the clients.
In my experience, it is way more complicated on the server side and client support isn't that great either (no native iPhone/Android support last I checked). pptpd on the other hand is pretty easy to setup on server side and has excellent client support. The only drawback is that it is less secure than OpenVPN.
The PPTP encryption scheme is less secure, but the other problem is that it might be blocked at the firewall. If I remember my VPNs correctly, PPTP uses a different port and protocol (GRE), so it can easily be blocked by overly restrictive firewalls.
OpenVPN, on the other hand, can go through TCP 443, which is all but guaranteed to be unblocked.
But, you're right that there isn't any iPhone support for non-jailbroken phones. I'm not sure about Android, but I see less of a problem getting it working on an Android phone.
Single-user OpenVPN has become much simpler to set up than it used to be, because you can just use preshared keys and not worry about creating a certificate authority.
Well I'll throw my solution in as well. However, it is only intended for web traffic, nothing else (useful for getting around firewalls, filters, etc..)
I have squid running on a vps, but NOT exposed externally (and it has basic auth). Then just set up an ssh tunnel "ssh -f user@mydomain.com -L 3128:mydomain.com:3128 -N" and now you have a proxy server available on localhost:3128. Then its as simple as pointing a single proxy supporting app, your OS wide proxy settings, etc.. to localhost:3128 with the correct basic auth credentials and you are in business.
I use this on linux and windows (cygwin) every day, works beautifully
I use this setup on my laptop (mac specific). I have an ssh server listening on port 443 on a linux machine in the basement of some university somewhere.
.ssh/config
Host bouncebox
Port 443
/usr/local/bin/prox
#!/bin/bash
if [ $1 == 'off' ]; then
echo "Disabling Proxy..."
networksetup -setsocksfirewallproxystate "Wi-Fi" off
else
echo "Enabling Proxy on port 12345"
networksetup -setsocksfirewallproxy "Wi-Fi" localhost 12345
fi
While it's not as comprehensive as full traffic redirection, I host a shellinabox[1] over HTTPS on my server that I use when SSH is disabled. It's quite convenient when I want a shell from anywhere quickly since there's nothing to setup.
I also have an openvpn for when I want full tunneling, but that takes more time to setup properly.
I've written a script[1] to do the setup for me, using preshared keys transparently. I'm sometimes running into an issue where the UDP packets stop reaching the server, but that may be my ISP.
Alternative this is much more sneaky: run an obfuscated tunnel. Most firewalls will allow egress DNS for example, so tunnel your TCP over DNS. This will also allow you to sneak out of pay-for wifi setups like at the airport. http://analogbit.com/software/tcp-over-dns
I've done this with my Windows Home Server. You just turn on VPN, and then on my ThinkPad with WiMax, I have it automatically VPN in whenever I switch antennae. Never have to worry about what content is being restricted, what kind of shaping the hotel wifi is doing, or what snooping fellow coffee-shop goers are engaging in.
I long ago learned that effective public wifi means a VPN over SSL/443.
My concern is that this knowledge -- about VPN's, much of the "for dummies" versions using 443 -- is becoming mainstream. Whereupon there will be further escalation.
(As long as it was a niche, it was missed or ignored by many "public wifi" providers.)
Ah the classic SSH tunnel. I used to use something similar to bypass the blocks in highschool. I used PuTTY to set up an SSH tunnel to my home server and it would host a local proxy port for the browser to use.
This won't work with a variety of even quasi intelligent filters. In fact, it's shameful that this works anywhere near as much as it does.
This is way overkill anyway.
Change the SSHD port (since the filters are naive enough to just whitelist ports apparently), then run `ssh -D 8080 user@ec2-instance:80` and voila, you have a SOCKS proxy that will proxy any traffic, including DNS requests. (and good OSes will easily allow you to utilize that proxy system wide)
Except that port 80 is often intercepted by transparent proxies and the non-HTTP traffic will get dropped. Using port 443 is more reliable as except in the most restricted environment presumptive HTTPS is passed straight through.
The tool needs an update or two regarding a few features (notably it only supports a single tunnel at once), but it pierces through literally any HTTP proxy, since it's really HTTP and not some CONNECT trick over SSL.
Then you just use your favorite OpenVPN over that, and make all traffic (including DNS, and except your httptunnel endpoint) go over it.
Not "literally" any HTTP proxy, as IDS systems have very little trouble distinguishing tunneled traffic from real web sessions. If someone wants to block you, they will. My point was more: If you're going to pick a port other than 22 to avoid networks that block "ssh", 443 is probably the best choice.
On the other hand, I can see where they're coming from by banning the whole netblock. Otherwise you could scrape until your IP get banned for blowing a rate limit, then tear down that instance and spin up a new one.