
How we test Wi-Fi gear - Tomte
https://arstechnica.com/gadgets/2020/01/how-ars-tests-wi-fi-gear-and-you-can-too/
======
zamadatix
Cool of them to publish their testing methodologies, I agree with them very
well. I just got done designing 3 wireless testing cages for the company I
work for so I've been looking a lot into what makes good testing methodology
for wireless. Very nice to see them dive into airtime right away and they
actually cover it pretty well. I wouldn't extrapolate their conclusions on
mesh and why it's fine to anything beyond low density home environments but
that's what 95% of the readers fit into so perfectly reasonable.

One grave conflation they make is they keep referring to web streaming as
actually performing like a stream of data. In reality web streaming (outside
of WebRTC) is actually very bursty chunks being sent usually in the range of
every 1-5 seconds. In the Wi-Fi world the two types of streams can behave very
differently. Not that their test is bad but it isn't necessarily testing what
they say it represents.

The HTTP workload is pretty representitive of what it says it's testing. The
rise of HTTP/2 changes the way 16 connections to a single site would work but
in reality loading a single site often connects to multiple servers anyways.

I was very happy to see them point out the large swings in jitter/latency
under load as well!

Unless you're in a rural area and don't have too many legacy wireless devices
I'd really caution putting anything in 2.4 that doesn't need to be there. It
does penetrate farther but it's also only 20 MHz of air space that almost
always has to operate with legacy compatibility (damn you Nintendo devices and
cheap IoT wireless N chips). If possible try to keep everything new on 5 GHz
only and if you have a lot of devices or use a mesh utilize 2 40 or 80 MHz
5GHz channels depending on your wireless environment.

Overall very good and much better testing than I expected. If there were
anything I'd add to such an article it'd be a section in any wireless testing
article pointing out "you'll almost always be able to get better improvements
by replacing your clients instead of comparing wireless routers of the same
generation". Obviously more harder to do/more expensive but I think a lot of
the time home users think the access point is the be all end all of how good
your wireless will be, forgetting it's only 1 device out of a dozen in the
house and just knowing this can set more realistic expectations and a better
understanding of what you're actually trying to improve.

------
zwieback
Another interesting thing to test is compliance with WiFi standards. We had a
situation where our devices were crashing the customers APs and they blamed
our radios but it turned out it was because our devices and antennas were
weaker and more mobile so they ended up disassociating and reassociating a
lot, which sometimes confused the WPA layer in the AP and caused it to crash.
We had to show that their existing clients were doing it too when you moved
them to the perimeter of the coverage area. In the end the AP vendor had to
supply new FW but it was a long drawn out battle.

