
IPv6 Performance – Revisited [video] - okket
https://ripe73.ripe.net/archives/video/1539/
======
drewg123
The host side of V6 can be noticeably slower as well. Mostly due to lack of
use. Here's one example:

I recently fixed a performance bug in our (FreeBSD based) TCP stack where
having just 10% of connections be V6 on a busy CDN server would result in
performance issues due to massive lock contention on network related
hashtables. The issue was that when an ICMP (like must fragment, for path mtu
discovery) was received targeted at a connection, the V4 code would look up
the appropriate connection (with a read lock), and do the work for that
connection. However, the V6 code would take a _write lock_ on the global hash
table (shared with v4) and iterate over the entire table while holding the
write lock. So the entire system would come to a screeching halt, with readers
spinning on that write lock while V6 bumbled along. This is something most
people would not notice, but on a busy CDN server with ~100K connections
serving 80-90Gb/s, it was pretty deadly.

The cause of this bug was primarily because people have been optimizing the V4
code for years, but the V6 code "scary", unused, and often forgotten about. So
people mostly forget about it when making optimizations.

Looking at some data from last night, one of the biggest sources of lock
contention on one of these 100G boxes is still in the v6 code, even at a site
that's serving only 15% of traffic as V6. Sigh.

~~~
feld
Can you please report your findings if you haven't already?

[https://lists.freebsd.org/mailman/listinfo/freebsd-
net](https://lists.freebsd.org/mailman/listinfo/freebsd-net)

~~~
drewg123
[https://svnweb.freebsd.org/base?view=revision&revision=30362...](https://svnweb.freebsd.org/base?view=revision&revision=303626)

~~~
crest
This is the best kind of bug report.

------
e12e
This seems like a good place to ask: has anyone got experience with going ipv6
only?

I am (or effectively have) inheriting (-ed) an aging small campus network. No
external services, and no core internal services (beyond DHCP). We'll have to
get all new core switches (still on 100mbit...). If my isp gives me a /64 ipv6
net, and I firewall off windows filesharing (and take other measures
appropriate for placing user devices on the routable Internet) - can I expect
most things to just work?

I'm fine with demanding windows 10, recent Linux, iOS or Android for
wired/WiFi access - and plan on moving to/requiring 802.1x for access (which
means setting up some form of auth/authz back-end, possibly Azure Ad - more
likely a Linux directory server). Can i live the dream and just go for ipv6?
Am I crazy because I'm considering it?

~~~
seiferteric
most websites won't work with v6 only, so you will need to be dual stack. For
your internal services, going v6 only should be mostly fine I think. If you
are running a campus network, your ISP should give you a /48 though,
definitely not a /64 only since you will not be able to subnet that.

~~~
e12e
By dual stack you mean something beyond nat64?

~~~
winthrowe
If you've got nat64 and dns64 setup, you should be ok for most things, but in
many environments it's easier to dual stack - do both 4 and 6 on end hosts,
rather than setting up nat64.

~~~
e12e
Right. I suppose the time still isn't right, and I'll have to stick with
private ipv4 subnets and NAT :/

(granted, nat64 is also nat - but on paper, it would be so nice to just stick
to ipv6...)

~~~
catdog
You also need DNS64 which basically adds DNS NAT on top of the IP NAT making
it an even worse hack (also breaks DNSSEC if you care about that). NAT64 may
be a valid option for large, complex networks but on a smaller scale simply
add v6 on top of the NAT setup you already have.

------
jlgaddis
Looks like there's some broken 6to4 relays out there.

~~~
p1mrx
6to4 has always been broken, but that doesn't really matter because client
preference for 6to4 has been <= 0.01% since 2013:

[https://www.google.com/intl/en/ipv6/statistics.html](https://www.google.com/intl/en/ipv6/statistics.html)

------
abpavel
It is sort of dissapointing the industry is stubbornly pushing ahead, even
after decades, without admitting critical flaw of IPv6 being incompatible with
IPv4, and just writing new version.

And I'm not talking about the difference such as absence of default gateway
concept, replaced by router advertisement. I'm talking about having to build a
new parallel Internet, beside the old one, and having to support both networks
indefinitely. IPv4 sunset is an illusion. We're stuck with it to the end, and
no incompatible protocol will replace it.

~~~
deathanatos
> _critical flaw of IPv6 being incompatible with IPv4_

What other choice do we/did we have? I've responded in the past to a different
commenter, who lamented IPv6 not being backwards compatible. You can read my
response[1] to that comment; essentially, I don't see a way to make IPv6
happen, aside from the introduction of a new, separate network, mostly due to
the pigeonhole principle: IPv4 only has 2 ^ 32 addresses, so how do you fit
the 2 ^ 128 IPv6 addresses into that scheme?

You can introduce a legitimately new protocol, like v6 did. You could perhaps
add a special flag, or special block of addresses, or a special option — but
whatever mutation you perform on IPv4 packets to "upgrade" them, the
intermediate routers would still have to understand that flag in order to
properly route packets — no? And thus, since they must understand that flag,
you're still upgrading the entire infrastructure, and building out a
completely separate network, even if it is just IPv4.1.

I do think the IPv4 sunset is far, far out in the future. There will be
stragglers for a long time to come, and I'm not sure it's really worth
considering when IPv6 isn't deployed to a majority of people yet. But I do
think it's _possible_ ; at some point, I hope someone realizes that the
maintenance of that code is benefiting so few people as to sunset it, perhaps
at an ISP-ish level first. No doubt this will break _somebody 's_ workflow,
though.

[1]:
[https://news.ycombinator.com/item?id=12168656](https://news.ycombinator.com/item?id=12168656)

~~~
drewg123
I actually think that a minimally extended v4, with no changes other than a
different version number and 64-bit addresses, would have gone a long way
towards increasing adoption. Having IPv6 as an entirely new protocol has lead
to all kinds of misery. And I'm not even talking about application layer
stuff, I'm talking about OSes and hardware.

Hardware vendors hate it, because it is another protocol they have to deal
with, which doubles ASIC testing matrixes. And don't even get me started on
how evil extension headers are from a hardware point of view. The only good
thing about the new V6 proto from a hardware perspective is the elimination of
the header checksum (which was trivial anyway). This lead to hardware vendors
delaying support for V6.

V6 is miserable in OSes because everything has to be done twice because you
have two stacks (see my other post here regarding the PMTU locking issue in
FreeBSD), and changes are applied unequally. Even if you want to say that
FreeBSD sucks, pretty much every innovation has some to V6 later than V4 on
every OS I can think of. From simple things like TSO and checksum offload to
RSS. There are examples of this on Linux, OSX and Windows.

I've always said that if I had a time machine, rather than shooting Hilter or
John Wilkes Booth, I'd concentrate on trying to make the ethernet header 16
bytes and trying to argue for an IPv5 that was just an extended v4 without all
this added complexity.

~~~
catdog
> And don't even get me started on how evil extension headers are from a
> hardware point of view. > trying to argue for an IPv5 that was just an
> extended v4 without all this added complexity. > V6 is miserable in OSes
> because everything has to be done twice because you have two stacks (see my
> other post here regarding the PMTU locking issue in FreeBSD),

On the other hand that flexibility may save us from going through that "we
have to upgrade the whole Internet at once" pain again. In the end widespread
v6 adoption is driven mostly by v4 address space exhaustion. At the time this
got really severe some years ago the implementations in both hard and software
were already pretty reasonable and widespread. Would we really be at a higher
adoption rate right now with no additional reason to upgrade? (also IPv5 was
already used in RFC 1819 for sth. different, that's why they jumped to 6 ;))

> The only good thing about the new V6 proto from a hardware perspective is
> the elimination of the header checksum (which was trivial anyway).

What about fragmentation which is entirely left to the hosts in v6?

> Even if you want to say that FreeBSD sucks, pretty much every innovation has
> some to V6 later than V4 on every OS I can think of. From simple things like
> TSO and checksum offload to RSS. There are examples of this on Linux, OSX
> and Windows.

I don't think such optimizations coming a bit later had a significant negative
impact in v6 adoption (and if v6 adoption would have been higher it would have
been implemented more equally). Also bugs get found and sorted out a lot
faster if someone is actually using the code.

