Fair queuing seems to have been an idea that was used for a while and then forgotten. It's been an obscure option on Cisco medium-speed routers for years, but only on by default for 2mb/s links and below. Most of the attention on congestion control went into the TCP side, not the router side. There was too much attention to packet dropping, rather than packet reordering. In a router, you can do three things about congestion - drop, reorder, and delay. For decades, most of the action revolved around dropping, with Random Early Drop being popular. Reordering means you get to make decisions about what to reorder. That leads to policy issues, traffic shaping, quality of service, administrative interfaces, tuning knobs, and customer service/help desk calls. Which is why QoS never went much of anywhere outside of some tightly managed networks.
A big contribution of the bufferbloat guy is that he promotes the original dumb fair queuing - each source IP/source port/destination IP/destination port is one flow, and you service each flow equally. Nice and simple. No tuning. If that was in the middle of the network, you'd worry about people getting more bandwidth by gaming port numbers or something. But what's the point of gaming your own personal DSL router? In practice, the things that need really good latency on home DSL uplinks are acks, voice over IP, and game user action traffic, all of which are low-bandwidth. So the simple approach is good enough.
Early routers had barely enough CPU time to copy the bytes, let alone do fair queuing. So it wasn't in early routers. It's also a bit complex for a FPGA, so it's not in the big ones which route in hardware. (Nor is it that helpful there, because backbone routers don't buffer much.) That's how we got here.
It's embarrassing that this is still necessary to talk about. There's no excuse for big DSL operators deploying millions of routers with giant dumb FIFO buffers. Come on, AT&T.
A fair queuing add-on box is a definitive win only if the next link has a fixed data rate. DSL yes, cable maybe, cellular probably not. Cable modems at DOCSIS 3.1 and above are supposed to have DOCSIS-PIE , which is supposed to fix this.
Now will someone please do a startup that builds a little box? As a hobbyist thing this goes nowhere. As a little box you buy at WalMart, it does.
Or, AT&T guys, get your act together and ship this as standard. Thank you.
> The only catch is... what upload/download speeds should we give to cake? Okay, I cheated for that one. I just asked my dad what speed his DSL link goes in real life, and plugged those in.
So the critical problem remains unsolved. If the ISP ever increases their speed cap, it goes to waste; if they decrease the speed cap, the fix stops working. I use a similar solution, but I have to turn it off and retest a few times a year, because Comcast sometimes changes the speed without notifying me.
I think the blog really overdid it with the analogies.
If there is a huge buffer, it will take a while for packets to start dropping, so the congestion control algorithm won't function optimally--it won't respond fast enough and lower the rate it sends packets until the buffer is full and packets start dropping. Once the buffer is full, packets have to wait in line in the buffer until they can be sent along so they take longer to get to their destination.
Eventually the buffer empties, but then the TCP congestion control algorithm starts increasing the rate it's sending packets and starts the problem over again. You end up with a very variable latency--depending on where in the cycle you are.
This is why uploads interfere with downloads, even though they shouldn't.
That depends on your metric. It isn't necessarily true if you're targeting stable latency for whatever reason.
I got my Eth port 1 to bridge to WAN but it's not doing any shaping - my port 4 to Wan standard dual-nat shaping is working, however.
He mentioned that over 20Mbps you really don’t notice better browsing speed. He’s right. I have gigabit internet - usually around 950Mbps up and down. Nothing takes advantage of it in the real world. The best real world throughout I’ve gotten is through my work VPN in the middle of the night or transferring something to AWS. Even then it’s only 100Mbps. BackBlaze tops out at around 70Mbps with multiple threads running.
Video streaming at its highest but rate is only 15Mbps.
Worth noting this experience is from Phoenix, AZ (which is especially relevent because backblaze servers are in Sacramento and Phoenix).
The only time that it is tedious is when (for example) Overwatch ships a multi-GB update, and it hasn't auto-updated because my PC has been off. That used to be a "go downstairs and get a can of coke and it'll be pretty much done when you get back", but is now "go do something else for half an hour".
I expect you meant "bit rate", unless you're streaming a particular kind of video ;)
The dslreports speed test is pretty good, and the one from fast.com should give you a pretty accurate picture of what you could see from the Netflix infrastructure.
Beyond that, you need to run your own CLI tools and be in control of both ends.
So I went to fast.com and it’s showing 150/73 using WiFi. Speedtest is showing 249/66. I am on the second floor. When I am right next to my router, speedtest maxes out at 371/354 wirelessly to the closest server.
Dslreports is 319/94.
This is on my iPhone 8 Plus. Usually my iPad is a little faster.
Web browsing specifically has marginal improvements as the speed increases towards Gb. Latency has a larger impact as a percentage.
The most bandwidth intensive thing I do is probably “download Linux ISO’s” on but torrent
Worked like a charm. I remember that it was specifically designed to get rid of buffer bloat, with traffic prioritization for streaming, VOIP, etc. I honestly think it was config-less though, so it doesn’t make sense unless it was also testing the speed on a regular basis and adjusting itself.
Edit: found it: Hawking Broadband Booster (HBB1) https://www.amazon.com/dp/B0009PLO0Q/ref=cm_sw_r_cp_api_jEiB...
(I used to work on TCP acceleration boxes that among other things eliminated bufferbloat in satellite networks. But it's a far easier problem to solve when you're on the correct side of the bottleneck link.)
 The nearest town is https://en.wikipedia.org/wiki/Hillman,_Minnesota
Funny how that works out.
I'm currently using an EdgeRouter Lite iii which reduces the download speed from gigabit to 130 Megabits per second when SQM is turned on.
apt-get install wondershaper
Specify your upload/download speed. All done.
It works really well if your upload and download speeds are predictable, as they were when I had cable. Now I have ADSL, and upload/download speeds vary with the weather (yes actually), and phase of the moon (probably), so it doesn't work so well, unless you set it to the lowest speed you are likely to see. (Don't do this on ADSL - it isn't pretty.)
However this is too much editorializing - I don't bother reading any more with this style of writing.
>Okay, new analogy: oil pipelines! Wait, those are unfashionable now too. Uh... beer taps... no, apparently beer is bad for diversity or something... chocolate fountains!
You installed rate limiter that drops packets and doesn't buffer.
You don't need to write it like you're writing ad copy "This one little secret".
If I'm reading something to learn something new, I want to end up learning about it by the end of it or I hit the limit of time I have to spend on learning about it.
- The device is a D-Link DIR-825 running openwrt 18.06.0 with a pair of ports bridged so it can be dropped into place w/o having to reconfigure anything on the network.
- Cake is being used for rate limiting.
- If you only care about the details of the setup without all the prefatory text skip to https://apenwarr.ca/log/?m=201808#openwrt
This style was more enjoyable to read and gave more insight into the author and their perspective. I appreciated it and read the whole thing.
Post the situation, the plan, and details/documentation.
I do not have the time to read a literal 6,000 words.
How about that the format for this post was bad? Dont use blog formats when teaching a lesson.
Very quickly you will lose feedback if you downvote honest advice.
Don't read it then. It wasn't written for you or anyone else specifically.
The plan: Write about what they know and how they were able to improve their parent's rural internet experience using layman's terms and analogies so that more people that suffer this situation might benefit from the improvement.
Details/Documentation: Read the submission.