This a nice summary; I was actually thinking about doing my own writeup of this very thing. I have a few thoughts beyond what was covered in the blog post:
For many protocols and modes of communication, UDP is really lousy for the developer -- we end up re-inventing TCP (congestion control, retransmission, etc.). Unfortunately, if we want to traverse NAT, we often must rely on UDP hole punching and thus use UDP. There's a TCP hole punching technique called "TCP simultaneous open", but it seems too flaky to expect to work in the real world. I've been thinking about trying UDT [1] as a user-mode TCP-like connection layer over UDP.
In addition to the ugly UPnP technique for asking a router to automatically port-forward, there have been attempts to standardize a much simpler solution in NAT-PMP and it's successor, PCP. (PCP even works for IPv6 stateful firewalls.) Many people do have security concerns about this capability. I don't know to what extent NAT-PMP/PCP has made inroads into the router market -- it would be interesting to conduct some sort of survey.
You don't need to reinvent TCP. You can simply put TCP over UDP. Works like magic :)
TCP hole punching works just fine in practice, but it's certainly a part of the "long tail" of empirical hacks that allow bumping NAT traversal rates into 95+% range. Detecting NATs that decrement ports is another. Ports that are incremented/decremented by more than 1 is yet another. And there's also UPnP that gives you another percent or so.
I was doing this as a part of the Hamachi project [0], almost 10 years ago, and this long tail was absolutely instrumental to being able to serve hundreds of thousands of clients off just few measly Dell servers for $300/mo in hosting costs.
Its somewhat OT, but thanks for Hamachi! It made many a late night gaming session possible in my teens and early twenties, and was always an astoundingly simple piece of software to use considering everything it did.
Cool to get a ping from Hamachi's author. I deliberately did not use Hamachi while designing ZeroTier since I didn't want to pollute my headspace with something so similar. :)
ZeroTier right now is sort of an OSS Hamachi, since I figured this would be a good starting application, but I've got different longer-term goals in mind. (Won't be abandoning that application though.)
Also, the article frequently mentions a 120-second timeout interval, within which the connection needs to be refreshed. In Linux (and thus most consumer routers?), I believe the timeout is actually 30 seconds for UDP. :(
The 120 second interval is the minimum from RFC4787 Sec. 4.3 REQ-5. (There is an exception that permits shorter intervals but only for specific ports, in particular port 53 for DNS.) If Linux is using 30 seconds then it's in violation and someone should submit a bug report.
I just took a look at Linux's nf_conntrack_proto_udp.c to see what's happening. It looks like Linux starts the UDP timeout at 30 seconds, but extends it to 180 seconds when it sees traffic both ways, assuming it's "some kind of stream."
Privileged ports were a broken security model to begin with.
Consider a a distributed system that relies on that minimum 120 second interval when determining whether to send a keepalive. Failure to send one within this interval would create an otherwise easily avoided network partition.
Consider someone adapting this system to operate across a restricted network which only permits traffic on privileged ports to traverse its border. Suddenly idle connections can be purged well under that previously safe two minute window. Instant partition.
Whether Linux implements the conditional to distinguish between privileged and unprivileged ports is irrelevant. Any serious systems designer must consider the timeout interval to be 30 seconds.
It isn't about privileged ports. It's about the protocols those ports are assigned to, and it's really only referring to port 53 for DNS because it's the only protocol that requires it. Using a shorter mapping timeout for port 22 or port 80 is not permitted.
The reason for DNS to be special is that DNS has to use a random source port for every query to reduce vulnerability to the Kaminsky attack. Most other UDP applications use a single socket (and thus port and thus NAT mapping) for all peers. So if you have many DNS clients behind a NAT you're going to run out of source ports if you try to keep mappings for 120 seconds. Meanwhile the DNS protocol only requires the mapping to persist until the query response is received which is almost immediately.
So DNS gets a special rule. Which isn't a 30 second timeout. There are real NATs in production that use 5 second mapping timeouts on port 53, which is completely reasonable and necessary if your site is making 10,000 DNS queries a second.
The sensible way to deal with all of this is to assume the 120 second timeout initially and if you encounter a hard disconnect that allows an immediate reconnection then reduce the keepalive interval. And don't run non-DNS protocols on port 53 unless you're prepared for the consequences.
Any NAT deployment that large will included local DNS caching servers. Out of the box consumer NAT configurations (WRT-54G, etc) typically run a DNS cache also. Source port randomization in response to Kaminsky's paper in 2008 is definitely the sort of thing that would fill NAT tables faster. But RFC4787 was published in 2007.
The behavior chosen by the implementers of the Linux netfilter code (see simmons' post) elegantly covers most cases without getting as specific as REQ-5 in the RFC, and not nearly as specific as you're describing. Network operators just manually configure a faster timeout for port 53. That's what the RFC permits: tuning in response to real conditions.
A hard disconnect is a major event that usually goes all the way back up the stack. A systems designer wants to avoid such a thing, and the sensible thing is to look at the source (of the most commonly deployed implementations).
> Any NAT deployment that large will included local DNS caching servers
A DNS cache only reduces the number queries by answering some from the cache, it still sends queries and requires source ports proportional to the number of client queries being made.
> Source port randomization in response to Kaminsky's paper in 2008 is definitely the sort of thing that would fill NAT tables faster. But RFC4787 was published in 2007.
The weakness of the 16-bit DNS query ID has been known since long before Kaminsky's actual paper. Sensible DNS caches like Daniel Bernstein's have been using source port randomization since around the turn of the century. Kaminsky just set a fire under the ones that weren't by publishing a devastating proof of concept, so now they all [had better] do it.
> The behavior chosen by the implementers of the Linux netfilter code (see simmons' post) elegantly covers most cases without getting as specific as REQ-5 in the RFC, and not nearly as specific as you're describing.
What Linux is actually doing seems reasonable. There is no requirement that the DNS timeout has to be short, only that it can be. For smaller networks it shouldn't make any real difference.
> Network operators just manually configure a faster timeout for port 53. That's what the RFC permits: tuning in response to real conditions.
Your point was that you have deal with systems that exist in the wild. If some significant percentage of large sites are using very short mapping timeouts for port 53 and you want to handle that, it doesn't much matter if it's a result of manufacturer defaults or manual configuration.
> A hard disconnect is a major event that usually goes all the way back up the stack. A systems designer wants to avoid such a thing, and the sensible thing is to look at the source (of the most commonly deployed implementations).
Sometimes all you have is a dog's breakfast. If it's a common scenario that DNS is the only open UDP port and the mapping timeout for the DNS port can be as little as 5 seconds then you can either always assume that you need a 5 second refresh interval or you can default to a longer one and then have to detect when you're wrong. Do you see any better alternative?
This has zero to do with the ports being "privileged" and everything to do with their state as a "well-known port range" used by "IANA-registered application(s)".
The idea being you can adjust the timeout because you can infer the way the port is being used.
So true in the kittens dept. The worst part is that while this situation has been developing, Stockholm syndrome has taken hold and half the battle is convincing people to be receptive to having real IP addresses again! Few people know the difference between a NAT box and firewall anymore, and "router" now means NAT box to many people.
In the case of many consumer/desktop machines, being behind a NAT is a very useful first tier security barrier. Every machine being fully reachable -- say via IPv6? I foresee many IT headaches and the Nortons/McAffees of the world raking in more money.
NAT != firewall. You can have a firewall without NAT.
Probably the sensible thing to do for IPv6 is to make the default to block the ports of specific applications people don't canonically expect to be visible from the internet, like Windows file sharing or telnet or HP JetDirect, but allow everything else so that VoIP and other P2P apps will all work properly.
Yes, NAT != firewall, but most consumer machines with static/world routable/reachable IPs have been particularly prone to attack over the years. Public access points (hotel, airport, etc) offer a similar level of risk -- yes Firewalls are important, but I'd pretty much make sure most machines/devices under my control in my home network (v6 or v4) not be world reachable. I'm just oldskool paranoid that way.
The only way you can make a device "not be world reachable" is to not connect it to the internet. It's silly to think that a vulnerable application invoking listen() is more dangerous than a vulnerable application invoking connect(). As soon as you connect to some cloud server you're exposed to any malformed data anyone can ask it to relay to you.
Of course you can do without a NAT. But the NAT has the advantage to make the firewall rules trivial to write.
Put "DROP all packets with destination 192.168.0.0/16 coming from the external interface", and a few other boilerplate rules of the same kind, and poof: you just secured 99.9% of the NAT-ed networks. No configuration required.
To do the same without a NAT, you need at the very least to adapt the rules to the IP addresses of the machines behind the firewall, which add complexity and increase the likelihood someone will screw up the configuration.
And, hu, please don't open ports by default. That's just a recipe for disaster. If apps really want to open a globally accessible port, make them explicitly signal that to the firewall so it opens access.
You know you could just drop all packets with destination of any valid address coming from an external interface. This isn't BGP, you're not going to really be routing the entire internet.
> being behind a NAT is a very useful first tier security barrier.
Misconception. being behind a NAT without a firewall is a very useful nothing in terms of security, because one is basically a static route away from being reachable. It just happens that masquerading is coupled with packet dropping in most appliances and configurations. Only the firewall protects you.
Even if I don't have a firewall, pretty much any competent ISP will configure its switches to drop any packet pointing to/from something that looks like private IP address space. Not to mention they wouldn't route it to me anyway.
So even if I don't have a firewall, you would probably need to physically plug yourself directly on the WAN port of my NAT box for your static route to work.
Not to say the firewall is useless, but even by itself, the combination NAT + private IP space prevents a lot of attacks. That's very different from what happens when you have no NAT and no firewall, where you are basically reachable by any kid on the other side of the earth.
Not a problem - all of the enterprise IPv6 networks I've seen (not that many, but some) have deployed their IPv6 in RFC 4193 and they use IPV6 NAT at the border.
Your average IT Enterprise director/VP has little to no interest in having end-end routing of IPv6 to their internal workstations.
I'm just wondering about how much will actually change with IPv6. I already have native IPv6 at home and it seems that my home router (a FritzBox) has a stateful firewall enabled for all IPv6 connections. I can manually exclude individual addresses from firewalling, but that's about it.
At first glance that looks like the right solution: People keep saying that instead of the NAT hack you should have native IPv6 with firewalling for the same or better level or protection. But doesn't a stateful IPv6 firewall introduce the same problems as NAT? Won't I still have to use keepalive packets or protocols like PCP to actually use P2P?
Of course real P2P is easily possible with IPv6 by simply opening up everything. But what are our options for having both security by default in home networks and no need for those workarounds we have in NAT now?
At least with IPv6 we won't have the mess of translation, requiring p2p apps to perform public address determination (and possibly port guessing). But you're right, the central problem remains.
> But doesn't a stateful IPv6 firewall introduce the same problems as NAT?
Depends on what it does and how it's configured. If the whole internet ends up running with default-deny stateful IPv6 firewalls, then that's a setback, but you still have global addressibility if not reachability.
> Won't I still have to use keepalive packets or protocols like PCP to actually use P2P?
Even IPv6 + PCP is a real improvement.
Default-allow on the consumer routers + host based firewalls would be much better though.
I remember the time of 56k modems. Port scans were fun back then. Sucked, would not dial-up again.
Host based firewalls: for some reason this sends the networking neckbeards into rage mode, god knows why. I'd say its slightly weaker than blocking everything on the router.
I worked in infosec for a while. Nearly every real-world threat we saw came in via HTTP, IMAP/POP3, or some other "pull" mechanism. Firewall might as well not have even been there. Anyone who thinks "firewall == security" is terribly out of date.
I'd say the only thing firewalls still provide protection against is pure remote vulnerabilities in common system services. Those still appear occasionally, but are more rare than they used to be. With better service isolation, coding standards, and authentication they'll become even rarer.
> Some NAT devices support various methods of intentionally opening ports from the inside. The most common of these is called Universal Plug-and-Play (UPnP).
UPnP should be avoided whenever possible. It's a hideous protocol. NAT-PMP (originally from Apple, now RFC6886) is dramatically simpler and ideal for small networks. Its successor NAT-PCP (RFC6887) is only slightly more complicated in order to better deal with large enterprise networks.
> "Lots of people think NAT is a show-stopper for peer to peer communication, but it isn’t. More than 90% of NATs can be traversed, with most being traversable in reliable and deterministic ways."
All the traversal methods require coordination with a 3rd party (ie: centralized) server so - yes - this is a show stopper for P2P.
As public addresses become more scarce, and carrier NAT becomes common, the problem of finding that intermediary will only get worse.
IPv6 should be a solution, but it won't get off the ground if carrier NAT gets priority, for example. (Or if ISPs just put firewalls everywhere, and other "best practices"...)
> All the traversal methods require coordination with a 3rd party (ie: centralized) server so - yes - this is a show stopper for P2P.
Third party and centralized aren't the same thing. Any peer with a real public address or even a manual port mapping or a router that supports NAT-PMP or PCP can play the role of the "server" in this context.
In order to boot strap a connection to a P2P network, one must contact a well-known server. It doesn't matter if the well known server is a "peer" that happens to be running the same software, or if it's the BitTorrent DHT bootstrap servers, what matters is that that peer has a disproportionate amount of authority and influence over the network, amounting to a single point of censorship or failure.
NAT (and firewalls anywhere except the networked computer - even at the subscriber's router) contribute to/create this asymmetry, where one side has to beg to connect to another, and so everyone winds up settling on the most-memorable 3rd party. Everyone has heard of the king (or Verisign), and so the rich get richer...
It's not a requirement, it's just that it tends toward it naturally due to the asymmetric addressability. As long as there are two or more global addresses available to the public on which to run STUN, UPnP, etc, there will be "competition" but it is immeasurably weak compared to what would be possible with direct (non-NAT) addressability. In an environment without those obstacles, systems are naturally designed in a P2P way - simply from the need for scalability.
Case in point A: Skype leveraged an initial P2P design at a time where direct addressability was the norm (and there were many freeware alternatives that allowed direct dialing)... Now that Skype has become dominant, it has switched to a centralized infrastructure 1) because it's owners can (it makes administration, censorship, and surveillance easiest), and 2) because a P2P model no longer makes sense with most users relying on their centralized bootstrap servers.
Case in point B: Dropbox and similar services have replaced self-hosted FTP, I would argue, simply because noone wants to maintain static port mappings and Dropbox is easier.
Even without other incentives, the presence of NAT is a centralizing force that - taken to the extreme (such as with carrier NAT) outright precludes P2P - and that is undesirable. In an Internet with NAT (or any other violation of the end-to-end principal) all systems suffer the same fate: centralization (the antithesis of P2P).
The idea is that it doesn't actually eliminate the horrors of NAT traversal, it just makes it my problem instead of yours.
The current solution is to use other nodes as relays using DHT-style routing and then I put a VM on AWS to bootstrap. The interesting thing is the bootstrap peer is only required for the first connection. Once there is an existing path A-B-C-D, it doesn't matter that zero of them have a public address, you can still use it to send a hole punch message from A to D.
The real problem is that trusting random peers to relay messages allows them to DoS you by filling up the network with Sybils and then not forwarding your messages. So I'm in the process of coming up with solutions to that, probably something along the lines of allowing particular nodes to be designated as trustworthy and preferring those.
Thanks for the link and thanks for taking a stab at a hard problem! Snow looks very promising so far... (I can ping nodes on my LAN over it too, which is usually a sticking point for traversal-oriented software - one is doubly-NAT'd, and there's an SSH server with an Ubuntu banner reachable from afar with UDP packets cutting through both brick walls nicely.) I'm impressed! :)
(FYI, building snow on a fresh Debian Squeeze 686-pae (packages: make g++ libssl-dev libminiupnpc-dev libnatpmp-dev) fails for me at dht.cpp line 220 (ambiguous function call) though; I'll have to read the source more to find the right cast or ::namespace to fix it but it compiles fine on amd64 with an identical set of packages.)
I can see the bug: The function is overloaded as taking uint64_t or a pointer and I'm passing "0UL" to it, which on 64-bit is an exact match for uint64_t but when 0UL is 32-bit it doesn't know whether to convert it to a uint64_t or a NUL pointer. It probably just needs a cast to uint64_t.
Unfortunately, that does not provide a solution. Since public IP addresses are becomming scarcer, and since at least one side needs a global address, all of the workarounds for NAT will tend to centralize the 3rd party coordination/proxy role.
Even IPv6, which should provide direct addressability in the long term (assuming ISPs provide it on the wire), may wind up increasing the centralization in the short term (creating a single point of failure and censorship) if the only way to connect to the IPv6 Internet is to tunnel into a major tunnel broker; rather than hundreds of ISPs, there may only be a handful - easy targets for mandatory kill-switches, censorship, and surveillance - and what started with more addresses than stars in the universe, will have degenerated into a global hub-and-spoke network.
> Another difficult situation arises if two peers are actually on the same local network behind the same NAT router.
...
> It might be tempting for peers to encode their private IP address and send it to the intermediate server and/or to the other peer. I thought about this when writing ZeroTier. On the plus side, this would work even on large segmented networks where UDP broadcasts don’t make it everywhere. But the problem is that this exposes internal configuration details about the network to potentially random external peers. Many network security administrators are not going to like that, so I decided against it. I tried to think of a way to anonymize the data but couldn’t, since the IPv4 private address space is so small that no form of hashing will adequately protect against exhaustive search.
I wonder if it would be possible to encrypt the local address with the internal MAC address of the router/gateway. Then only nodes seeing the same router could decode the information. I also wonder how Dropbox does this (it can tell when two connected computers are on the same LAN, and transfers files directly).
Neither the IP address nor the MAC address contain enough entropy to make that secure. Even assuming they were completely random, 32 and 48 bit keys can be bruteforced in no time. Additionally the MAC address contains mostly static or easily guessable values and the public IP address is, well, public. Thus even key strengthening along the lines of PBKDF2/bcrypt/scrypt is hopeless.
But if you brute force the key, when do you know that you got the right one? Its hard to tell a correct IP address from four bytes of garbage. Depending on the encryption scheme, you are bound to get a lot of fake local IPs - potentially all 256^4 of them.
Of course this assumes that you don't include a checksum or another way of verifying the key. If you do, then you're right, you can just brute force the key. If you don't, then you have another problem: For every node that you connect to, you have to try the local IP you "decrypted" (well, at least if it makes sense in your subnet). This could be a lot of overhead if you have a lot of nodes.
There might also be other sources of entropy that are common to all nodes behind a NAT, for example routing information, or identification strings of the router (most customer routers identify themselves via UPnP). Also you could try to piggyback on other networks, like Windows' workgroups or Apple's bonjour.
Finally, since the public IP is public anyway, you could send the (poorly) encrypted local IP only to nodes that have the same public IP as you do. I haven't thought this to the end and this probably won't work when an ISP gives multiple users the same IP, though.
So, I think it would definitely be possible to implement this, the question is how much effort you want to put into this. In a real application, I would probably just try the broadcast, and if it doesn't work, give the user the option to manually connect over LAN (e.g. by entering a code word on all nodes).
It's much easier to tell a correct private RFC 1918 IP from four bytes of garbage. There are only 17,891,328 valid values.
The odds move further in your favor when you have multiple private addresses to work with -- a valid key will most likely result in addresses that are all in a single RFC 1918 block, and very possibly all in the same /24.
Actually this is entirely achievable right now, with very little code. You can use WebRTC in chrome / firefox, which implements all the nasty details of peer-to-peer communication for you (and uses TLS-esque security). All you need is a STUN / TURN server and something to do the initial handshake and you're off to the races.
I've not gotten around too it, but someone should setup a webserver that speaks the protocol (perhaps with this: https://code.google.com/p/libjingle/) then you serve your site as a static HTML file on AWS or similar, spin up WebRTC, and then communicate with your server that way. Now you can serve a whole website sitting behind a firewall.
I think the vast majority of my IPv6 traffic is to sites who also care about security (or care about networking in general?) and consequently use TLS where possible.
Anyone know on any stats regarding popular sites reachable over IPv6? For example, what percentage redirect the requests to TLS by default?
I recommend reading nanog mailing list for having a glimpse on the added complexity of dealing with IPv6 in real life.
Just peak random mails, it pops up once a month.
Like for instance provider turning to LS (Large Scale) NATing or Carrier Grade NATING rather than going IPv6 for rationnale reason: IPv6 engineers are hard to find and LSNAT (+dual stack eventually) since China has proven valid has both a way to handle the scarcity of IPv4 addresses and the need to control user's traffic.
IPv6 is not bad, it is kind of bloated and requires a tinge more training than IPv4. Plus there are the concerns about privacy (but some IPV6 guru will come up with ULA or another obscure trick. It is IPv6 realm: everything is clear on the paper but is seems confusing to quite a few real life engineers since it requires a lot of RFC stacking in the brain)
IPv6 has privacy extensions that allow address randomization, and its address space is so large it makes it easy to just make up addresses. Besides, tracking IPs is really only one of hundreds of ways of tracking you on the Internet.
This doesn't change with the vast majority of residential IPv6 deployments. Many people conflate having a public IP address with not having a firewall. This is not the case at all. Yes, internal clients will get public, routable IPv6 addresses. They are still behind a firewall, though, and you have just as much control over ingress and egress traffic as you did before.
The problem is that all NAT related issues are not because of NAT itself, but because of the required stateful firewalling for NAT.
To be able to NAT and un-NAT, you first need to classify traffic (that NEW, ESTABLISHED, RELATED,... stuff in Linux netfilter), changing the destination or source ip addresses is only the second part of that process.
Protocols don't break only because of NAT, but mostly because of stateful firewalling, you'll face the same problems with IPv6 if you enable it.
The difference is in the isolation factor: there is absolutely NO way a packet with an internal IP can get anywhere on the public Internet, and likewise a packet with a public IP will never be accepted by a device that has been configured with an internal one. The failure modes are different - although firewalls can be configured to provide an "outgoing connections only" default like NAT, they are also software and thus not immune to bugs; a buggy firewall letting packets through which shouldn't be is far worse than a buggy NAT.
there is absolutely NO way a packet with an internal IP can get anywhere on the public Internet
That very much depends on your definition of "public internet."
My service provider doesn't properly filter RFC1918 space at their border. From my home on the west coast, I can hit 10.0.0.0/8 devices on the east coast in a separate AS.
Is a reachable printer web interface 3000 miles away in New York on the public internet? The "Internet" is a very tough entity to define succinctly, but I'd argue yes.
Why would you need to filter an internal IP as a destination IP, routing should handle that.
I understand filtering the source address as internal IP because that could be malicious spoofing of internal host (and destination can't legitimately reply.)
I used to feel this way but I've come to the opposite conclusion. Most homes have a rarely updated, always on computer connected to the public internet with increasing numbers of known vulnerabilities.
Agreed; despite what many seem to think I don't believe that every device on a network should have a public IP by default. Even if the whole Internet becomes IPv6-only I'll probably still keep my home network behind a NAT router on a private subnet.
As the recent series of articles on "smart" appliance security shows, many are exploitable through services listening on ports which the user normally isn't aware of. I would certainly not want a bigger number of those exposed to the public Internet than there already are.
This might be a totally off-topic tangent. I apologize if it is.
It seems to me that if consumer routers are configured to refuse to forward inbound IPv6 packets to machines on the LAN, we would be in almost exactly the same situation vis a vis home service hosting as we are with IPv4 addresses, NATs, and automated port forwarding protocols. Frankly, this would be a giant step backwards from IPv6 as she is envisioned.
From what I understand, the "average" home user has little to no idea of what a port is or why one might want to forward it. They are also very likely to be incapable of classifying machines that need not be globally reachable. So, either we have to write smart software that can determine whether or not a network device requires protection from inbound traffic, or we have to make a forwarding policy decision for this user when we send them their router.
If we're serious about IPv6, we shouldn't configure our gateway devices to require the software that we write to assume that a globally-reachable address is likely to actually not be globally-reachable.
It would be slightly less evil without NAT. You'd still need the three-party handshake but it would always work. You'd no longer have the symmetric NAT craziness.
You could also dispense with the need for frequent keepalives, a boon to mobile battery life.
Getting rid of NAT is step one. Step two is deperimeterization: getting rid of in-line firewalls. Step two is going to have to wait on OSes having better service encapsulation and app isolation models and for programmers to remove their heads from their behinds and stop writing code that is vulnerable to stack-smashing and buffer overflow attacks.
The last part will be tough. How much longer will the Sun be a main-sequence star? :)
A sends packet to B, which gets dropped by B's stateful firewall.
B sends packet to A, which gets interpreted as response and accepted.
A sends packet to B, which gets interpreted as response and accepted.
The third party is only necessary for telling B about the outgoing port chosen by A's NAT. With IPv6 where there's a stateful firewall but no NAT, a direct connection on a well-known port is possible.
Although in practice, you still need some third party to exchange each other's IP addresses.
> It seems to me that if consumer routers are configured to refuse to forward inbound IPv6 packets to machines on the LAN, we would be in almost exactly the same situation vis a vis home service hosting as we are with IPv4 addresses, NATs, and automated port forwarding protocols. Frankly, this would be a giant step backwards from IPv6 as she is envisioned.
I don't think it is such a giant step back. Devices still get to have addresses in a way that makes sense. Routing still gets to make sense. If two people in the same house want to play a peer-to-peer multiplayer game with a third person not in the same house, under IPv6+automated-firewall-exceptioning they can, whereas under IPv4+automated-port-forwarding they can't.
It's kind of dumb that we need automated port forwarding at all - opening a port should already be a deliberate indication from an application that it wants to accept external connections. If you want to listen only for connections from the same machine, bind to 127.0.0.1 - an equivalent for which will still exist under IPv6. If you want to listen only for connections from the same LAN, think again about what you really want - when the user goes to college, do they really want to be accepting connections from 400 students with poor security practices? If you can stand that, you have nothing to fear from the open internet.
But in the meantime we have the software we have rather than the software we want to have; for home routers, blocking traffic and requiring uPnP or the like is a reasonable default. Even if we do that, IPv6 is still worth it.
I think you're ignoring (or are maybe just innocently ignorant of) one key topic: state.
Stateful firewalls are exactly that - when an internal client initiates a TCP socket, for instance, the firewall knows to expect a series of replies back from whatever system the client contacted and is able to allow that traffic through.
In netfilter (iptables) land, this is handled by the "RELATED" qualifier, so you instruct the firewall to allow any related packets back through to the client.
IPv6 doesn't mean everything has to be globally reachable (except for certain ICMP messages that should always work). By default, I think blocking general inbound traffic is the right way.
When you want to run a service (i.e. something peer to peer or a web server or something), there are protocols for applications to tell the firewall to open certain ports, such as PCP - http://en.wikipedia.org/wiki/Port_Control_Protocol
It is, but mostly because popular operating systems have historically had poor application isolation and vulnerable services.
My personal opinion is that having to have a firewall everywhere in front of everything is a hack that was put into place because of awful OS security, and will become less important as OS security improves.
And/or you just substitute a NAT that happens to also function like a firewall with a proper firewall. The router could do the same sort of filtering as NAT would entail, but not require a hacky NAT table.
Or rephrased its not possible to implement "working" NAT without a stateful firewall, but the lack of NAT doesn't imply its impossible to deploy a stateful firewall.
(It would be interesting to research where this "can't have a stateful FW without NAT" meme came from. It is VERY popular among everyone but network guys)
It's surprising that the number of non-traversable hosts is only 4-8%. Aren't most cellular (3G/4G) users behind symmetric NAT, or is this considering only PC users?
As of at least summer 2013 Sprint no longer offers public ips with new phones. It's all carrier-grade NAT. T-Mobile is the same but they provide ipv6, though with just a single ipv6 address which doesn't make any sense.
I didn't mention carrier-grade NAT, but it's almost always full-cone and traversable. ZeroTier works behind tethering behind carrier-grade cellular NAT.
Their product doesn't do mobile, that's why. My 3G in Australia is definitely NATed (and the source address the other end sees seems to change with each cell!).
NAT isn't why it doesn't do mobile. It doesn't do mobile yet because there are issues around battery life that have to be solved. Mobile will take the addition of a sleep mode and the use of mobile push notification busses.
Its lots more complicated than ZeroTier would have you believe, especially for Enterprise clients. That can add wrinkles due to VPNs, firewall filtering, non-symmetric packet forwarding. And there's the case of two clients inside the same subnet - to go P2P using the described algorithm requires something called Router Hairpinning which has spotty support.
There are ways around all that, and Sococo supports them all. (Caveat: I work at Sococo).
Sure; but again assumes a simple subnet configuration. Lots of enterprise customers have multiple subnets with gateways, internal routers etc. They probably won't respect/forward LAN locator beacons. We (Sococo) gave up on that approach, particularly since Enterprise IT guys like to get excited about broadcast UDP wandering around their subnet.
A straightforward approach is, when talking with the 'rendezvous' server, to include in the message your discovered local subnet address(es). Then the server can inform your peer of the entire set of candidate IP addresses you might be available at (NATted address + subnet addresses). P2P connection pinging then works the same way, with the additional constraint that the 1st P2P message must include a unique id to identify the peer. Because non-routable subnet addresses can be re-used (10.x.x.x or 192.168.x.x style). You don't want to try to connect to Alice and discover that XRay has the same subnet address, XRay is available on your subnet, and XRay is also running the same P2P app.
Anyway like I mentioned, its complicated and full of wrinkles.
I guess they dismissed it too soon. None of our enterprise customers have a problem with it. And some demand it - the alternative is to have their P2P traffic travel outside the firewall and back in again, 'hairpinned' through the public router. Talk about a security risk!
What can that possibly mean? Of course I read it; they addressed a small fraction of the cases involved in P2P link negotiation.
Ok, I re-read it, missed that sentence the 1st time. The guy does dismiss 'private' IP sharing as mentioned elsewhere in this thread. This has not been an issue with any enterprise client in our network. In fact, its a little weird to be consider sharing IP addresses a problem in a protocol designed to share IP addresses. Especially unroutable ones that can be guessed anyway (there are only 65535 192.168.x.x addresses).
And NOT doing this makes some attempts fail, have worse latency, or worse yet NOW you're sharing your private traffic (voip? video? company IP?) over the PUBLIC internet. In fact several of our more careful customers explicitly demand that we route P2P traffic exclusively inside their firewall, and this is the feature that makes it work for them.
I found this twitter post interesting - the dude did a writeup of how to use metasploit when both the attacker and the victim are behind nat and you are able to get code execution some how on the victim: https://twitter.com/b1tripper/status/383085600040947712
And that will be their choice, and any problems caused will be their's (or their customers') to face.
With the current state of IPv4 there is between little and no choice so NAT can't be directly thought of as "doing it wrong", with IPv6 that would not be the case.
I'm also anxiously awaiting the rise of IPv6, but I'm guessing that -- in a sane world -- consumer routers will still default to a stateful firewall for IPv6, thus necessitating the continued need for hole punching. :/
Hole punching will be very reliable though - each end picks a port and starts talking to the other end at roughly the same time. Once they both start talking to each other, the flow will be fine.
> Ziggy then sends a message to both Alice and Bob. The message to Alice contains Bob’s public IP and port, and the message to Bob contains Alice’s.
Is this done by modifying the source address on the outgoing UDP packet? If so, wouldn't modern filtering prevent the spoofed packets from getting to their destination?
no, ziggy just acts as a directory of public ip addresses. Once alice and bob know each other's public ip they can start communicating each other. Suppose alice is at A.A.A.A:1234 and bob is at B.B.B.B:5678. Alice sends
A.A.A.A:1234 -> B.B.B.B:5678
and bob simulataneously sends
B.B.B.B:5678 -> A.A.A.A:1234
When the packets arrive to the routers the NAT already has an opposite rule because of the sent packet. As this is UDP there is no SYN-ACK handshake or other state tracking information in the packets and they traverse the NAT.
For many protocols and modes of communication, UDP is really lousy for the developer -- we end up re-inventing TCP (congestion control, retransmission, etc.). Unfortunately, if we want to traverse NAT, we often must rely on UDP hole punching and thus use UDP. There's a TCP hole punching technique called "TCP simultaneous open", but it seems too flaky to expect to work in the real world. I've been thinking about trying UDT [1] as a user-mode TCP-like connection layer over UDP.
In addition to the ugly UPnP technique for asking a router to automatically port-forward, there have been attempts to standardize a much simpler solution in NAT-PMP and it's successor, PCP. (PCP even works for IPv6 stateful firewalls.) Many people do have security concerns about this capability. I don't know to what extent NAT-PMP/PCP has made inroads into the router market -- it would be interesting to conduct some sort of survey.
[1] http://udt.sourceforge.net/