
What Can I Do About Bufferbloat? - velmu
https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/
======
Arie
An awful lot of home broadband connections suffer from bufferbloat and even
for the ones that don't, a single host can easily hog all the bandwidth. If
you're used to getting lag in your VoIP or gaming when a house mate starts a
stream/download/torrent, this can be fixed :)

The cake traffic shaper in OpenWRT is amazing for fighting bufferbloat in your
home network and it can also do almost perfect fairness in dividing the
available bandwidth per LAN host with very little configuration. Just get it
as part of the SQM tools in OpenWRT and enable it. For the per-host-fairness
take a look at the "Make cake sing and dance" from this link:
[https://openwrt.org/docs/guide-user/network/traffic-
shaping/...](https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-
details)

If you use an Edgerouter, you can get the cake traffic shaper but you'll have
to do without the easy web interface OpenWRT has:
[https://community.ubnt.com/t5/EdgeRouter/Cake-and-FQ-PIE-
com...](https://community.ubnt.com/t5/EdgeRouter/Cake-and-FQ-PIE-compiled-for-
the-EdgeRouter-devices/td-p/1679844)

~~~
bdelay
Parents live in a smaller town with two awful ISP selections. They had a bunch
of WiFi devices on an ISP router and the connection quality and latency was
just terrible when more than one device was in use and any bandwidth intensive
services were being used. (Low quality Netflix is intensive on small-town
monopoly internet.)

I purchased them a Netgear R7800 and installed hnyman's LEDE build [1] to
enable SQM. Night and day difference in latency response. No more staring at a
white screen for 3 seconds per URL click.

The build has been stable for several months. I wouldn't recommend this for
non-technical users or anyone not willing to spend time troubleshooting, but
it has been a great improvement. I couldn't find any other device capable of
doing this without running x86 hardware or something else silly.

A few other people mention it, but yes, this is only going to work on slower
connections on current SOHO hardware. I think the R7800 can do software SQM at
up to 150mbps or so. Plus, if you have a gigabit symmetric connection,
hopefully you aren't having bufferbloat issues.

Just wish a popular manufacturer would release an easy-to-use router with SQM
so I could install it for non-technical users and forget. Ubiquiti is somewhat
close to that, but I believe their prosumer hardware (USG) is running a slow
processor at the moment and doesn't even support SQM without installing custom
kernels.

[1] [https://forum.lede-project.org/t/build-for-
netgear-r7800/316](https://forum.lede-project.org/t/build-for-
netgear-r7800/316)

~~~
pedrocr
>I couldn't find any other device capable of doing this without running x86
hardware or something else silly.

Was the internet speed so high that you couldn't use a normal supported router
like a TP-Link Archer C7 at half the cost? I need to do more testing but it
seems my C7 can handle my 100/100Mbps fiber connection doing SQM without too
much issue.

~~~
thermodynthrway
I haven't seen a router under $400 that can do fair queueing in hardware
faster than ~200 megabits. If you have fiber it's cheaper to setup a beefy
Linux box and run PFSense on it. Hardware offload is usually disabled when you
turn on QoS so doing slow will often slow down gigabit LAN links as well

~~~
pedrocr
I'm at 100/100 though, so wouldn't something simpler be enough? The wired side
of the router is gigabit but that's just an integrated gigabit switch, it
doesn't even touch any CPU. I'll be doing more testing to make sure but my ISP
doesn't seem to have too bad a buffer bloat anyway.

~~~
thermodynthrway
Sometimes turning on QoS disables offload on all ports including lan. Try a
download test between local machines

~~~
pedrocr
My understanding of these routers is that the gigabit switch is independent
from the router. They're physically on the same board but the router is just
another machine on the switch. If the switch table says portA->portB it
doesn't matter what the router on portC has decided to offload or not.

Edit: Maybe you mean Wifi to wired may have a disabled offload? That path does
go through the router and not directly through the switch. For bigger
installations I end up having one of these with wifi disabled as the router
(firewall, dhcp, etc) and individual ones connected through ethernet as dumb
access points (same SSID on all and straight bridge from Wifi to Ethernet).
That should also avoid any issues and is a good setup to get more wifi
coverage with a simple config.

~~~
thermodynthrway
Good to know, would explain why there's a phantom eth port on some of these
routers, must be used to connect between router and switch chips. Sounds like
you're right about wifi->wired transit though if this is the casr

~~~
wtallis
The typical architecture for routers these days is that the main SoC has two
ethernet interfaces, each of which is connected to a 7+ port managed switch.
One of the host CPU's interfaces is on the WAN VLAN, and the other is on the
LAN VLAN. Some older routers used to have just one ethernet link between the
switch and the CPU, with the CPU's other interface exposed directly as the WAN
port. That made it easier to avoid bloat or bugs in the ethernet switch
itself, but was fundamentally incompatible with the NAT offload those switches
provide, so that configuration is now almost impossible to find.

~~~
pedrocr
This also makes these little routers extremely powerful. Since those switches
have VLANs as well you can create very interesting topologies that would
require much more expensive managed switches to achieve. I run an extra VLAN
from my router, to one of my APs, to a dedicated wifi SSID to another wireless
router to ethernet to a TV box so I can have the TV signal in a place I can't
run ethernet to. Doing it through the normal Wifi would be a bad idea because
the provider uses multicast IPTV and if you put that on your wifi every
connected devices receives it. And this is all done with 3 50€ routers running
LEDE, each with different VLANs and wifi SSIDs configured. They make for a
really flexible setup.

------
dtaht99
Recently I ranted at the ISPs and at the network neutrality people. fq_codel
(RFC8290) is now nearly ubiquitous in as the default queuing mechanism in most
linux distributions, and it is long past time more ISPs supplied it in the
gear they give customers.

[http://blog.cerowrt.org/post/net_neutrality_customers/](http://blog.cerowrt.org/post/net_neutrality_customers/)

sch_cake (derived from fq_codel with an integral shaper) is about to enter the
Linux mainline after years of baking in openwrt.

[https://lwn.net/SubscriberLink/758353/05c20f25115c852d/](https://lwn.net/SubscriberLink/758353/05c20f25115c852d/)

We can end bufferbloat, now, on everything.

~~~
pja
I enabled sch_cake on my router (an R7800 with Openwrt 18.06 on it) recently.
Worked like a charm. Would happily recommend.

I imagine the main thing that's holding back ISPs from using it on their
consumer routers is the (minimal) configuration the user has to do to set to
upload bandwidth limit. If ISP routers were also using decent queuing
algorithms and not buffering like crazy then that wouldn't be necessary of
course, but it doesn't seem like that's going to change any time soon.

~~~
dtaht99
Well, many isp-born devices are provided with that info by the ISP at connect
time, so the hope has always been that during configuration they'd just pass
(for example) "bandwidth 10mbit docsis" to the network setup routine. DSL
modems also typically get this info at startup.

------
dtaht99
I am immensely cheered up by the progress reports contributed by so many on
this thread. But, can I ask that if you grok it, go fix it for two friends? Go
fix it for a local small business, a coffee shop, or a hotel. And ask your ISP
to fix it in their default gear?

There's probably over 2b routers without bufferbloat fixes installed, and only
if we work together to get them deployed will the internet as a whole get
better, more capable of handling web, games, videoconferencing, and other
applications that demand consistent low latency.

Thanks.

~~~
doctorsher
Thanks for everything that you do, Dave. Your bufferbloat and WiFi work is
excellent, and I'm excited for the airtime fairness work with Toke to make it
into more 802.11 drivers than ath9k. Cheers.

~~~
dtaht99
I was essentially in a overwork-induced coma for the last 18 months. The work
was carried forward by (many, many) others.

I was happy to wake up a few weeks back, and find RFC8290 published, and
sch_cake being readied for mainline, and multiple commercial products finally
shipping what we'd worked on all these years.

------
vanburen
I recently bought an Ubiquiti EdgeRouter X [1]. Which has a smart queue
function that utilises fq_codel.

Results from the DSLReports speed test [2], using the same router, the only
difference being turning smart queue on:

Smart queue off:
[https://i.imgur.com/zeY4rTd.png](https://i.imgur.com/zeY4rTd.png)

Smart queue on:
[https://i.imgur.com/jfHpiFb.png](https://i.imgur.com/jfHpiFb.png)

Notice the improvement in bufferbloat score from D to A+.

Subjectively, I have noticed the connection seems more responsive and there no
longer seem to be latency spikes when utilising all the upload bandwidth.

It’s certainly worth considering bufferbloat, if you suffer from latency
spikes when using all your upload bandwidth (I used to suffer from this a lot
more, when I had an ADSL connection, with only 1 megabit upload).

[1]
[https://www.ubnt.com/edgemax/edgerouter-x/](https://www.ubnt.com/edgemax/edgerouter-x/)

[2] [http://www.dslreports.com/speedtest](http://www.dslreports.com/speedtest)

~~~
blattimwind
I'm currently behind a Linux router with basically nothing but firewall rules
configured; I still get an A on that test (meaningful?). I guess this sort of
thing is just the default nowadays?

~~~
dtaht99
What router? what ISP? What link technology? What bandwidth? A pointer to your
dslreports result? There are plenty of small ISPs that have adopted this
stuff... and a few router makers.

I am always happy to hear of a bloat free connection.

------
treis
Holllllleeeeeyyyyy sheeeeet!!!

I went from a buffer bloat of 1.2s UP and 0.6s DOWN to effectively 0. It's
like I have a whole new internet connection.

OP and everyone else involved gets a million internet points from me!

------
thermodynthrway
If your network is all Linux hosts, configure BBR TCP congestion control and
you won't ever have to worry about bufferbloat again (unless you use a lot of
UDP) . There's a ton of research in this area, but to summarize, there's two
main ways of controlling outgoing packet rate.

Measuring RTT(round trip time) or packet loss rate. Unfortunately all the
early (and still common) TCP protocols control their send rate by measuring
packet drop. Sending faster and faster until upstream routers somewhere start
dropping traffic. This has the effect of completely filling the outbound
buffers of the slowest link in the chain.

To combat this, most bufferbloat fighting algorithms focus on dropping traffic
before buffers fill, namely RED (random early drop) and CODEL.

Newer algorithms like TCP Vegas and BBR measure RTT and lower transmit rate
when they detect buffers down the line filling, preventing bloat.

In most cases, you still need a router configured to prevent bufferbloat
because even a single naughty protocol on your network can fill outgoing
buffers.

The most important thing to know about controlling bufferbloat with QoS is
that you MUST have an accurate estimation of your max upload/download rate.
This is because you can only control bloat if you are in control of the
slowest link where it will build first. All effective bufferbloat solutions
rely on artificially making your router the slowest link, usually by limiting
bandwitdh through them to ~90% actual.

Once you have control of the slowest link you can pick which packets get
dropped as the buffers fill. And using something like FQ_CODEL you can assign
equal bandwitdh to all IP's on the network. The nice thing about controlling
bandwidth this way vs hard speed limits per user is that it allows users to
use as much bandwith as they want until staying below line rate requires
sharing

Also notice I keep saying outbound. You actually have far less control of
bufferbloat on the inbound end. Best you can hope for is that dropping inbound
packets coming in over the configured rate will stop whatever server from
sending you more as quickly, but this isn't always the case. Luckily, in my
experience, the vast majority of "lag" is due to outbound buffers filling, not
inbound. So setting up bufferbloat fighting bandwidth sharing is usually
extremely effective

~~~
dtaht99
some comments:

1) BBR is currently not something I'd recommend at home.

2) The hope has always been that the core two bufferbloat-fighting algorithms
(BQL, and fq_codel) would end up in the cable, fiber or dsl modem hardware, so
that no shaping would be required, as there would be sufficient backpressure
from the link itself to regulate the link intelligently. The cpu costs on this
are nearly 0! BQL is 6 lines of new code in the device driver. fq_codel has
been shipping in linux for 6 years. It's just a matter of turning it on...

But: lacking that support from the ISP-supplied gear, we shape with htb +
fq_codel, (as you say), to ~90% of the link rate... with another box - or even
in the same box if the device driver can't be fixed and is overbuffered. We
are painfully aware of how much cpu shaping costs but modern cpus usually have
enough oomph to handle it.

btw: We've come up with a new deficit shaper (sch_cake) that lets us get to
~100% of the isp bandwidth (so long as you get the wire framing _exactly_
right), while providing vastly better queue management in the sqm system.

4) fq_codel is fair to flows, not devices. This works well in the general
case, but has edge cases where abusive apps that open a lot of flows gain
priority. Adding per host fq (while retaining per flow fq), even through nat,
was the number 1 request from the users for sch_cake, and one of the main
reasons why cake exists.

~~~
thermodynthrway
You can configure FQ_Codel to match on different parts of of the packet.
Default matching "tuple" for flows includes source/dest port along with
source/dest IP. If you set this to source IP only it should fairly distribute
bandwidth based on LAN IP alone. If cake is a normal QDisc you might be able
to enable flow-per-ip without adding any code. For an example with FQ CODEL
see:

$TC filter add dev $IF_WAN parent 11: handle 11 protocol all flow hash keys
nfct-src divisor 1024

At [https://openwrt.org/docs/guide-user/network/traffic-
shaping/...](https://openwrt.org/docs/guide-user/network/traffic-
shaping/packet.scheduler.example4)

Unfortunately every modem I've seen, including "business" models, doesn't use
CODEL.

------
josteink
As someone who has meddled with QOS in the past, this page didn't really feel
very concrete to me.

Especially regarding SQM... Is SQM just a new name for QOS or am I missing
something?

~~~
dtaht99
There were long debates about continuing to use the term QOS for what we did
with htb + fq_codel. From:
[https://www.bufferbloat.net/projects/cerowrt/wiki/Smart_Queu...](https://www.bufferbloat.net/projects/cerowrt/wiki/Smart_Queue_Management/)

“SQM” is shorthand for an integrated network system that performs better per-
packet/per flow network scheduling, active queue length management (AQM),
traffic shaping/rate limiting, and QoS (prioritization).

“Classic” QoS does prioritization only.

“Classic” AQM manages queue lengths only.

“Classic” packet scheduling does some form of fair queuing only.

“Classic” traffic shaping and policing sets hard limits on queue lengths and
transfer rates

“Classic” rate limiting sets hard limits on network speeds.

It has become apparent that in order to ensure a good internet experience all
of these techniques need to be combined and used as an integrated whole, and
also represented as such to end-users."

~~~
Rapzid
QoS Classic.. A total branding excercise. All of these strategies were
deployed in the past under the umbrella term "QoS".

------
magoon
I cured bhfferbloat with a cheap DD-WRT capable router (NOT Puma 6 chipset). I
found it important to limit the WAN bandwidth on my router to about 15% below
the max speed I’m paying for. What a difference!

Netgear R6300V2

Using fq_codel because the others use more CPU than I’m confident these
devices can handle.

I find that most people suffer from bufferbloat unknowingly, having bought
fancy routers 5-10 years ago.

------
theogravity
If you use Opnsense, I've written a tutorial on how you can configure it to
reduce bufferbloat.

[https://forum.opnsense.org/index.php?topic=7423.0](https://forum.opnsense.org/index.php?topic=7423.0)

------
berkeleyjunk
If you are interested in the background for this work I would greatly
recommend the following two articles from ACM queue

BufferBloat: What's Wrong with the Internet? \- discussion with Vint Cerf, Van
Jacobson, Nick Weaver, and Jim Gettys

[https://queue.acm.org/detail.cfm?id=2076798](https://queue.acm.org/detail.cfm?id=2076798)

Bufferbloat: Dark Buffers in the Internet \- Jim Gettys & Kathleen Nichols

[https://queue.acm.org/detail.cfm?id=2071893](https://queue.acm.org/detail.cfm?id=2071893)

~~~
dtaht99
That's good basics, but it's all fixed now. Jim's latest blog post goes into
exausting detail.

[https://gettys.wordpress.com/category/bufferbloat/](https://gettys.wordpress.com/category/bufferbloat/)

------
gwillen
I have a home internet connection that is a resold AT&T U-Verse connection. I
doubt any of these fixes are available to me -- the extremely user-hostile
equipment provides essentially no user-configurable options, and I have been
told it has no switched mode, not even a secret one. So there's no way I can
introduce my own router hardware unless I want to be double-NATted.

(Also, it has a broken caching DNS server, and forces that broken server into
its DHCP responses; the server it returns is not user-configurable.)

So what's an ordinary consumer to do?

------
ra1n85
>Twiddling with QoS might help, but a faster internet connection probably
won’t help at all.

The key issue here is contention, or a busy egress interface - that's when
buffering occurs. I do not understand why adding capacity, or reducing the
probability of a busy egress interface, "won't help at all".

~~~
tfha
The torrenting app will just consume all the extra space, leaving just as
little for gaming as there was before.

~~~
dtaht99
totally untrue. fq_codel manages normal torrents just fine in the presence of
gamer-style traffic. We tested against torrents in fixing bufferbloat a lot!
Pure aqm systems like pie (in docsis 3.1) do pretty well also. cake (per host
fq) can make even insane amounts of torrenting (or slashdotting!) bearable.

~~~
tgb
I think you misunderstood the above poster who was talking about the case
without SQM (or I'm misunderstanding things because I've never heard any of
these terms before now).

------
meta_AU
Anyone trying to work out if Mikrotik has some equivalent of this SQM, looks
like no. Well, not explicitly fq_codel or any extensions. Though using the sfq
queue system with traffic shaping seems to give the same improvement to
bufferbloat.

------
dsiemon
It really is amazing the difference that modern AQMs like FQ-CoDel make. This
is a bit of self promotion but we leverage FQ-CoDel in our product
([https://www.preseem.com](https://www.preseem.com)) to help ISPs provide a
better experience to their subscribers. Our customers regularly pass along
anecdotes from subscribers who are very happy that they can now do big
downloads or heavy streaming without breaking interactive applications like
gaming.

------
snikeris
News to me. dslreports reported a C for bufferbloat. My router (eero) has a
"labs" section that had SQM disabled. After enabling, dslreports is reporting
an A.

------
amelius
Curious: is it possible already in Linux to give the highest bandwidth to the
currently active application? And will routers play along?

~~~
supertrope
Careful, logically this means giving BitTorrent #1 priority when brought to
the top.

------
stuaxo
Is there a list of routers and their support for the different queuing methods
(and how they are setup by default) ?

~~~
dtaht99
DOCSIS 3.1 devices mandate pie, which helps a lot, but it's not as good as
fq_codel, nor do they do shaping from the isp, which is kind of needed for all
the cable links in the USA I've tried. Get one if you can, though, they are
better across the board in many other ways.

pfsense has fq_codel.

Anything (1000s of routers) from lede/openwrt has the most advanced
bufferbloat-fighting stuff in it, followed by dd-wrt, tomato, etc. If you need
high bandwidths the multi-core arms are the best. fq_codel is pre-configured
on all links (ethernet/usb/fiber/wifi/whatever) but if you need shaping to the
ISP provided rate you need to configure it. All the research that went into
fixing bufferbloat queuing problems everywhere landed in openwrt and lede
first. Most of the research that improved tcp everywhere came out of google.

Most gaming routers now sold commercially have some variant of fq_codel in
them in their trade name ISP "qos" system.

Also fq_codel derived anti-bufferbloat work has landed in many commercial wifi
routers on the wifi side (eero, google wifi, some ubnt products, meraki, many
others). The paper behind all that was:
[https://arxiv.org/pdf/1703.00064.pdf](https://arxiv.org/pdf/1703.00064.pdf)
\- happily that work was "good enough" to enable by default, and boy, does it
make a difference if wifi is your bottleneck.

The current premier dsl router with cake is evenroute. I think there are
several new models from several manufacturers that are going to get it right,
soon.

Not tracking FIOS (gpon fiber) closely at the moment. Yes, fiber networks have
bufferbloat, but it's harder to hit, and generally smaller than on dsl and
cable technologies. I configured cake on sonic fiber recently and got 60ms
back. (going from 60ms latency under load to 3ms )

Regrettably shaper setup is finicky and requires a few minutes of testing with
a site like dslreports or a tool like flent.org to get right. If more ISPs
published their shapers' bitrate and burst rate settings, life would be easier
here... but the hope has always been they'd just ship a router with this stuff
on and remotely configured to be "right".

In some ways, what are you doing about bufferbloat is

~~~
dtaht99
Let's see. The turris omnia is a very good router (but only available in
europe). For oomph (gbit shaping) people often leverage lede on a pcengines
apu2 or run a full distro of pfsense or linux on it.

------
doctorsher
In my experience, changing the default DNS servers, enabling sch_cake, and
minimizing shared spectrum interference are the most significant improvements
for a home WiFi connection. Can anyone think of an additional dimension for
improvement, besides upgrading the link itself?

~~~
wtallis
> Can anyone think of an additional dimension for improvement, besides
> upgrading the link itself?

Depending on the hardware, you can upgrade the link itself with newer, smarter
WiFi drivers. After pretty much solving the bufferbloat problem for wired
connections, many of the same developers moved on to fixing WiFi, and some
fruits of that effort are currently available in OpenWRT and LEDE.

~~~
doctorsher
By 'enabling sch_cake', I meant that I use sch_cake on my wireless router
(which I flashed with LEDE). Sorry, I was a bit light on the details.

------
philg_jr
Anyone have any suggestions for pfSense? I played around with the traffic
shaper, setting the scheduler type to CODELQ and limiting bandwidth to 95%,
but it doesn't seem to do much from what I can tell while testing with the
speedtest on dslreports.

~~~
dtaht99
85-90% is a better starting point.

------
tgb
Wow the manual test is very convincing: running fast.com's speedtest while
pinging google makes the time increase from <30ms to over 1000ms! It's taking
1.5 seconds to ping google? I had no idea this could be happening.

~~~
dtaht99
It's been happening to everyone. Spread the word.

------
scruffyherder
Residential internet in Hong Kong can go as high as 10gig.

I don't see anything about links anywhere around 10gig or higher, if I wanted
this on my 100gig backbone or 40gig metro links.

~~~
dtaht99
Bufferbloat happens on high speed links like those but amount of bloat you see
is in the 30ms - 60ms range (vs seconds(!!) on home links). Bloat happens
mostly (aside from microbursts) on overloaded links - and high speed backbones
are typically overprovisioned so the problem only shows up when there's an
outage or fiber cut. Example of what happens today on a fiber cut:
[http://blog.cerowrt.org/post/bufferbloat_on_the_backbone/](http://blog.cerowrt.org/post/bufferbloat_on_the_backbone/)

IF things like fq_codel were deployed on those we'd not see latencies climb
that much at all, we'd see bandwidths decline to the actual capacity available
- and only the biggest flows would be hit to do so.

FQ_codel is lightweight enough to fit directly in high speed hardware and it
does indeed run on 40gigE plus devices on linux, ddpk, BSD, in software. But
it takes a long time for new chipsets to adopt new algorithms even if they
incorporate support for deeply desirable features like ECN.

That said... 10Gbit to the home.... ooohhhhh. it's really hard to bloat that!

------
petermcneeley
How is priority distinguished between two TCP streams? I thought that QoS
basically didnt exist at the internet level of the network.

~~~
dtaht99
Priority is the wrong way to think about it. Given all the sources of bursts
on the internet today, fair queuing (or "flow queuing") has become the way to
turn flows back into packets.

there's an awful lot of lit on FQ, what we do with fq_codel is to not only
interleave packets better but apply congestion control signals at the right
time so competing tcp flows don't overwhelm the link (with under 10ms of
buffering (v seconds common on fifo ISP links)).

[https://en.wikipedia.org/wiki/Fair_queuing](https://en.wikipedia.org/wiki/Fair_queuing)

Of course, being perfectly fair to flows is sometimes undesirable, but making
something strictly higher priority[1] is fraught with peril as you end up with
a classification nightmare.

Having fq gives you the best shot at smaller flows completing sooner, and of
big flows sharing better with each other.

Having vastly reduced buffering improves the responsiveness of competing TCP
flows a lot, grabbing bandwidth whenever available, faster.

My take on folk that want "prioritization" is ask them to try some variant of
sqm with just fq and codel and get back to us. being fair with well managed
buffers works really well.[2]

[1] making something lower priority than best effort is actually a good idea.
[2] but if you really want some flows or devices prioritized, see the sch_cake
work mentioned on this thread. I still tend to think per host FQ is what many
want rather than attempting to raise the priority of certain flows from
certain services.

[https://arxiv.org/abs/1804.07617](https://arxiv.org/abs/1804.07617)

~~~
petermcneeley
I have been away from this field from some time. The Cake seems to solve the
problem with machines but not the problem with the same machine having streams
with different latency/bw requirements. The priority way of solving this
problem is round robin on packet priority and priority limit for machine/user
(different queues per discritized priority level meaning no bufferbloat for
high priority). The primary issue with this solution is that it would require
the packets to be labeled.

~~~
dtaht99
yep. solved - for 6 years in the sqm-scripts and now in cake. (not solved, in
docsis-pie)

We use diffserv for this, for apps willing to use it. Example: ssh sets the
imm diffserv bit for interactive use. cake respects that (I've cited the
relevant paper elsewhere, another place is
[https://www.bufferbloat.net/projects/codel/wiki/CakeTechnica...](https://www.bufferbloat.net/projects/codel/wiki/CakeTechnical/)
but after extensive testing we settled on 3, rather than four tiers of
priority)

stuff derived from the sqm-scripts use the same method (using htb + fq_codel)
but the problem has always been that diffserv is not respected end to end.
However, within your network, you can make your intention known and have it
work, if you have the bottleneck.

Also, we have always made the latency/bandwidth tradeoff explicit - if you
want less latency, you must want less bandwidth. It's the only safe answer to
apps gaming the diffserv markings.

~~~
petermcneeley
Ya in gaming we usually have 2 packets. Synchronization that must occur and
then visual fidelity sync. First one is small bw and less latency. The others
can even be dropped with some minor visual desyncing.

------
mekhami
Can someone explain what I can do about this to a networking/hardware layman?

~~~
vanburen
This video gives an overview of setting up the EdgeRouter X:
[https://www.youtube.com/watch?v=o-g2P3R84dw](https://www.youtube.com/watch?v=o-g2P3R84dw)

------
k__
Hm, I only get A and A+ results.

But I'm on cabel, not DSL.

------
isostatic
Do you want to have jitter, or do you want loss?

In reality I don't see buffer bloat on the internet adding jitter of more than
a few milliseconds. I do see loss though or 20, 50, even 100ms. I'd rather
have 50ms of jitter than 50ms of loss, but that's just my application.

~~~
Arie
Why would you have to choose either?

fq_codel and cake use a tiny bit of packet loss to get a sending host to back
off, for example to keep a large download flow within the limits of your home
link. Other flows aren't affected.

Bufferbloat regularly adds hundreds of ms on home internet connections, you
can get an indication of your bufferbloat on
[http://www.dslreports.com/speedtest](http://www.dslreports.com/speedtest)

~~~
isostatic
Not on my connection. I would rather my UDP packet arrive 30ms late than not
arrive - especially on high latency links where I want to process the packet
before a 300ms round trip nack/retransmit has a chance to work.

I don't see any buffer bloat or excessive jitter on my home internet (at least
on wired connection) on BT ftth.

~~~
tgb
Note that the bufferbloat choice for me is whether or not I want my packets to
arrive 1.5 _seconds_ late, not 30ms.

