

TCP is UNreliable - LiveTheDream
http://www.cliffc.org/blog/2013/08/15/tcp-is-unreliable/

======
tptacek
I voted this up (tried to submit, actually), but is this article correct, or
is something in this person's stack not actually programming sockets
correctly?

When you call shutdown() to arrange a FIN for a socket, especially after
you've sent data and received no application-layer acknowledgement, you're
supposed to verify that the connection closed properly. You do this by
select()ing on the socket and then read()ing 0 bytes.

The error condition (the RST segment they're getting) is signaled by the
"Connection closed by peer" errno --- and -1 from read().

The reliability signals this app needs are built into both TCP and the socket
programming interface, from what I can tell. The application-layer
acknowledgement it wants appears duplicative.

~~~
abofh
Without looking at the code, it's hard to say, typically errors like the one
described won't be caught by return values of read/write, but by return values
of close() -- which many many people don't bother to check. (Not saying that's
what happened, but that's my bet)

~~~
tptacek
Did they close() the socket? If you write data to a socket, and receive no
signal that it's been processed, shouldn't you shutdown(2), select(2), then
read(2) before you close(2) it?

------
notacoward
This doesn't sound like a TCP problem.

Mistake #1: The sender optimistically sends a first packet before getting a
SYN reply. That implies that the sender issued a non-blocking connect, and
then wrote without waiting for the connect to complete. Otherwise, the sender
(kernel) wouldn't have any data to send right away.

Mistake #2: The sender sends a FIN, but doesn't block because all data has
already been acknowledged. This is supposedly a problem because the app never
receives the data, but a TCP ack has _never_ meant that data got beyond the
receiving kernel to an app.

So yes, it's true that neither side ever got an error and unfortunate that
it's possible for the receiving app to get no data in that case, but that
doesn't make it a TCP problem. Somebody at a higher level - JVM or the app
itself - over-optimized their "connection and first data" code without
understanding the actual behavior of the TCP protocol and socket API. Then
they "fixed" it by working with their own protocol instead of against the
standard one.

Nothing to see here. It was just a false alarm. Move on.

~~~
gonzo
It's not a TCP problem. It's a problem with the linux implementation of TCP.

~~~
noselasd
No, it's not even that. The linked article
([http://www.cliffc.org/blog/2013/08/15/tcp-is-
unreliable/](http://www.cliffc.org/blog/2013/08/15/tcp-is-unreliable/)) at the
bottom tells you what's actually going on.

The only thing slightly fishy is that the peer application didn't receive any
connection attempt, but if you're anyway quite overloaded, e.g. running out of
file descriptors, that may well happen.

~~~
dap
I still don't get how this can happen as described. If the client is using a
blocking connect and checking for errors (as the author says it is), I don't
see how that can succeed without the remote side at least seeing the
connection. As you say, the receiver could be overloaded and ignore the
connection, but the client should get a connect failure in that case.

~~~
noselasd
The remote kernel sees the connection fine. It accepts it, receives the data,
the client closes its end. What might not happen is, in a very resource
strained condition, is for the remote application to get the data. There's no
relation between you doing a connect() and the remote doing an accept() in
TCP, it's handled asynchronously by the kernel. (Traditionally the backlog
argument to listen() controlled this behavior, but that backlog argument is by
and large ignored on modern operating systems)

The caveat here is that you close() the socket when you're done. close() does
not, by default, wait until the remote end also closed its end. In the
scenario described above, the remote end would eventually signal an error, but
if you shoved the data into the remote kernel buffers, close and forget about
that connection, your app will never see that error.

------
noselasd
Note that he links to [http://blog.netherlabs.nl/articles/2009/01/18/the-
ultimate-s...](http://blog.netherlabs.nl/articles/2009/01/18/the-ultimate-
so_linger-page-or-why-is-my-tcp-not-reliable) which more or less explains
what's going on.

------
songgao
Interesting! One thing that I don't understand (which might be a little off-
topic): If it's an RPC call, shouldn't there be a response for each RPC
request to ack that the RPC request is well processed by the application
layer? Otherwise how do you make sure the application layer won't screw up?
How do you notify the client that the request is not invalid?

