

Routers treat HTTP and HTTPS traffic differently - mmaunder
http://markmaunder.com/2009/routers-treat-https-and-http-traffic-differently/

======
aminuit
This article is based on a false premise.

 _Because HTTPS clients and servers set the DF or Do Not Fragment IP flag on
packets and regular HTTP clients and servers do not._

There is nothing anywhere in any standard that requires DF to be set for HTTPS
traffic. There are certain links on the Internet that will set DF on your
traffic in transit because it is more expensive to forward and even more
expensive to do deep inspection on fragmented IP packets, but there is no
reason this would be done for HTTPS and not for HTTP.

EDIT: For completeness: links to RFC 2246 for TLS 1.0 and one of the draft
specs for SSL 3.0 (it was never an official standard). There is nothing about
setting the DF bit in the IP header in either document.

TLS 1.0: <http://www.ietf.org/rfc/rfc2246.txt>

SSL 3.0: <http://tools.ietf.org/html/draft-ietf-tls-ssl-version3-00>

~~~
mmaunder
Here's how you set dont-fragment on a socket:

<http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html> See:
IP_MTU_DISCOVER

This premise isn't taken from an RFC. It's based on my own real-world
observations of what is crossing the wire. Grab wireshark, fire up tcpdump and
form your own opinion - and please do share what you find with us.

Mark.

~~~
aminuit
On Linux (at least recent kernels), there are a number of variables that can
determine whether or not DF is set on outgoing packets. I ran a few tests with
(Ubuntu Hardy) firefox and it seems largely to vary based on the value of
/proc/sys/net/ipv4/ip_no_pmtu_disc. If 1, no DF. If 0, DF set. But it has this
behavior regardless of whether I'm using HTTPS or HTTP.

I understand your objection: standards do not often reflect the real world.
But the title of your post is misleading. Saying that routers treat https
differently than http is just incorrect. When I read it, I thought you were
trying to imply that routers were doing some sort of deep inspection, or at
least looking at the TCP port numbers.

What is true is that you can't expect PMTU discovery to work reliably over the
Internet (another case of standards not reflecting the real world). I work at
an ISP and it's a constant source of frustration. In general though, it's
something that needs to be fixed on the client side (or at least on the
client's ISP side), rather than on the server side.

~~~
mmaunder
So you're saying that with ip_no_pmtu_disc set to off (the default value),
your HTTPS server does not set the DF flag on packets?

What I've seen is that all HTTPS servers (that I've observed) set DF on their
data packets. No HTTP servers (that I've observed) set DF for their data
packets.

What I would like to know is why SSL servers set DF on their data packets. As
you've observed there's nothing in the FAQ and I've googled the socket options
and couldn't find any recommendations. I'm curious if an underlying API like
openssl is simply so widely used that it's a de facto standard.

~~~
aminuit
Also, when you say "HTTPS servers set DF," are you observing from the server
or from the client? Over the Internet, it's likely that you'd see different
values for DF on opposite sides of the connection. Any router can set or clear
the DF bit on your packets for any reason.

------
wmf
This is setting off alarms in my head. He's talking about path MTU discovery,
but for some reason he doesn't actually use that term.
<http://en.wikipedia.org/wiki/Path_MTU_discovery>

Also, I've never heard of a system that sometimes uses PMTUD and sometimes
doesn't. I can imagine that those SSL offload boxes may be doing weird stuff;
maybe that's where the author saw this problem.

~~~
mmaunder
Thanks wmf I should have included the reference and have added it. There's a
lot more that I don't cover e.g. PMTUD black hole routers, etc. But the
article is targeted at an intermediate level audience, so leaving those out
for now.

------
axod
I just tried tcpdump of http vs https, and both had the flag unset :/ (As I'd
expect)

Would have been nice if the OP had included some example logs showing the
issue.

~~~
mmaunder
Hi axod, on request from a commenter on my blog I've described how I captured
packets and analyzed. I'm interested that you saw the DF flag for non-HTTP
traffic. Can you tell me which OS and client or browser you used? I used
Ubuntu 9.04 and curl, but have seen the same df/non-df behavior with a variety
of other clients on the same OS.

Mark.

~~~
blasdel
You don't need to greet the user you're replying to, and you sure as hell
shouldn't sign your comments.

~~~
ComputerGuru
Yeah, but you don't need to be so harsh about it..

------
noonespecial
Tinkering with your network cards MTU just to try to access a web site is
probably a bad idea in most cases!

