
Hanging S3 downloads - duggan
https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/
======
Rjevski
Yet another problem caused by shitty consumer-grade network equipment.

~~~
matthewmacleod
I think you've maybe been unfairly downvoted there!

Almost all of the issues I experience with Internet connectivity day-to-day
are down to bad consumer networking equipment. APs start failing, or routers
stop routing, seemingly at random.

~~~
deathanatos
For example: my router/AP/modem combo-box — which is the standard one that
Comcast has been giving to all subscribers — when reset or power cycled, will
forget the WiFi key. It just gets reset to the factory default. Every, single,
time. And it's _just_ they key: the SSID _doesn 't_ get reset. It's _not_ the
first modem I've had, either; its predecessor was the same make, had the exact
same behavior.

I suspect it's because I'm using an actual key, instead of a passphrase, and
nobody else does that. (Or, at least, not enough for Comcast to notice that
the problem is them.)

The old one got returned b/c it was routinely dropping IPv6 packets. It took a
while — multiple vists from multiple technicians — to convince Comcast that
the symptom of "100% packet loss over IPv6, 0% packet loss over IPv4" was
_not_ a problem with the coax cable.

The predecessor of _that_ modem got replaced when it had extreme difficulty
forming IPv4 connections: almost all SYNs never got a SYN ACK, but in the rare
case that they did, that connection proceeded with 0% packet loss. I highly
suspect there was something going horridly wrong with its NAT processing, but
that's a guess.

~~~
Rjevski
The worst is that the Linux kernel (and any BSD) would be able to make a
perfect router, but manufacturers insist on using their own shitty
implementation. If they would've just used used the default Linux/BSD they
would've been fine.

------
coldcode
We take network transmissions for granted sometimes, TCP/IP has been around so
long we assume nothing strange can still happen. But implementations still
matter, it only take one strange piece of software or hardware and you wind up
with craziness like this.

~~~
moduspwnens14
Yep. Honestly I probably would have assumed it's not a protocol issue if both
HTTP and HTTPS downloads from S3 behaved the same way. Both of those protocols
are on TCP, though, so that would have thrown me off!

------
the_mitsuhiko
> "But Juho, what does any of this have to do with S3?", you ask. Well, S3 is
> one of those rare services that disable timestamps.

Interesting. Every once in a while I have issues on my cable provider (UPC in
Austria) with S3 and S3 only. Next time that happens I gotta have a closer
look at it.

~~~
manmal
I'm also on UPC in Austria, and yes, I'm suspecting the same issue now. I will
switch to Bridge mode now because of this.

~~~
mrkrabo
Maybe it's an issue with the cablemodem? There aren't many different
manufacturers of cablemodems. What's yours?

~~~
manmal
Actually I received a new one yesterday. I have to check again whether the
issue persists. The new one is called "Connect Box" and is jointly
manufactured by Compal und Arris.

~~~
the_mitsuhiko
Also using Connect Box. No interest in switching to bridge however as I have
no device capable of connecting by itself.

------
deathanatos
Huh. The Factorio community also ran into hanging downloads, and it also
seemed to the devs to be caused by checksums[1][2].

I think they had much the same solution, mix a random number into the packet.
(They use UDP, so they didn't have the benefit of TCP timestamps.)

[1]:
[https://www.factorio.com/blog/post/fff-176](https://www.factorio.com/blog/post/fff-176)

[2]:
[https://www.reddit.com/r/factorio/comments/5rvkyp/friday_fac...](https://www.reddit.com/r/factorio/comments/5rvkyp/friday_facts_176_belts_optimization_for_015/)

------
smudgymcscmudge
Way to put a spoiler right in the title. I'm happy to have read this before
seeing the post here.

~~~
JCharante
but the title of the HN post has the same content as the title of the post?

~~~
jsnell
It's a bit unfortunate that there's no history of the title edits on HN, as it
makes it hard to understand the context of any complaints about titles.

The post was originally submitted with the same title as the original post,
"The mystery of the hanging S3 downloads". That was apparently judged to be
clickbait, so the title was changed to something along the lines of "Hanging
S3 downloads due to NAT router", and the story was also very heavily penalized
at the same time. This title was indeed giving away most of the ending. But
luckily it was a bit later edited again, to the current form of "Hanging S3
downloads".

------
fumigator
Your cloud service does everything except for file backups, load balancing and
can't handle huge file copies. And for reliability and responsiveness,
requires a hybrid private-public cloud. Which begs the question as to what's
the point of moving to the cloud in the first place.

~~~
Rjevski
Is this a troll or you are really blaming Amazon for a fault in a shitty
router?

~~~
joshribakoff
I had the same reaction. If I need to support users with shitty routers should
I stay away from amazon? My users will just blame me not their router, or
switch to my competition who is not using Amazon since it works. My users
aren't going to read a long explanation of TCP/IP. Some users have no choice
like a school or work firewall/router. If my website doesn't work and the
competitions does what do you think happens next if I don't stop using s3? The
real question is how many users have a "shitty router".

~~~
Rjevski
Well S3 is popular enough (you don't realise it but a lot of sites use it
under the hood for static assets, file uploads, etc) and this is the first
time I've heard of this issue, so the number of routers so shitty that they
break S3 (and only S3) would be pretty insignificant. There will be more
routers that break the general web (no matter which site) than those only
breaking S3.

