Hacker News new | past | comments | ask | show | jobs | submit login
High-Speed Internet at a Crossroads (news.harvard.edu)
53 points by fortran77 4 months ago | hide | past | favorite | 100 comments

I can only imagine how much the Pandemic stretched the average home Internet. We had pretty good Internet, ~300Mbps in, maybe 15Mbps out, $100/mo, and when my wife and I and both the kids were on video meetings, it just couldn't keep up. I'd have to turn off video on my meeting to keep from losing audio. Upgraded to gigabit (30Mbps upstream?) and that helped, but it still struggled. Might have been at Xfinity...

About 6 months into it, we got hooked up for the municipal fiber network, we were not in the first batch deployed but were fairly early. Symmetric gigabit for $60/mo, and it's been working amazingly!

But, as the article says, "haves and have-nots". Our entire city should have the municipal network in a year or two. So the whole city will have it, and I think they'll be offering some incentives for people who can't afford the $60/mo. But nationally, and even in the rest of Colorado, there are still a lot of "have-nots".

Non fiber upload is a farce. I assume when they sell you more upload, they’re just placing you in a higher priority in the 30Mbps they have allocated for the whole neighborhood.

It is a hardware limitation as far as I know, so unless you see utility workers running new fiber, I would not expect much.

Most cable providers allocate spectrum to prioritize download. Even the DOCSIS standard is built around that asymmetry. Much depends on the level of over-subscription in a neighborhood.

In my old neighborhood, Comcast appeared to have a relatively low over-subscription ratio. My promised 1000/40 service pretty much always hit exactly that and I was a pretty happy camper.

In my new neighborhood, the Comcast over-subscription ratio is far, far worse. My neighbors complain that realized speeds never approach promised speeds. And I was unable to get service at all (despite all of my neighbors on the same street having service). After a year of tooth-pulling, I got an F-U quote from Comcast at $210,000 to connect my house.

So I banded together with neighbors in the community on a couple of adjacent un-served streets and we trenched and laid fiber. Now I have 10000/10000 service and it's awesome.

Edit: I actually can't measure how close my service is to meeting the promise. I run out of CPU first. That's OK.

You literally cannot find the upload for Comcast/XFinity until after you have gone through the sign up, and even given your credit card. There is one more confirmation before your card gets charged, and that is where it is.


I have tried to see if upgrading from 200Mb down to 400Mb down would change my upstream, and havent found a thing. One site suggests i might get 5Mb/s more upload. Sucks for pushing docker images.

I got 1gb from them just to rise my upload speed.

> So I banded together with neighbors in the community on a couple of adjacent un-served streets and we trenched and laid fiber. Now I have 10000/10000 service and it's awesome.

That’s fantastic! One might even scale this community up to a few more blocks and call it a government. Too bad it’s such a bad word for many.

It's not a government until you get the guns out and force people to join up and pay dues whether they want to or not.

What zbrozek described is just a co-op, or something less formal along the same lines: an organization designed to provide benefits other than financial profit for its (voluntary) members, who are also its shareholders. Co-ops are great. I wish people would use them more often rather than clamoring for unnecessary government involvement.

Co-ops are good for a farm. Or maybe in a rural area without lots of easements. Logistically, everyone will want fiber, and there’s no reason it cannot be dealt with like water/sewer/gas/electric. I do not see a volunteer based co op making progress in an urban area.

The water/sewer/gas/electric service you refer to is frequently supplied by a co-op, even in urban areas. I get my own electricity from a co-op. Natural gas is provided by a private company but there are also co-ops offering it in the area. Water and sewer are traditionally handled by the city, but are funded through service fees rather than taxes—there's really no practical reason why they couldn't be provided the same way. I can see the attraction in bundling everything together in hopes of cutting down on administrative overhead, but having the same organization empowered to impose taxes and regulations also supply utility services creates a huge conflict of interest and stifles competition.

Can you share more details about how you managed to get that done? Costs, obstacles, resources?

I'm stuck in a Comcast monopoly neighborhood in western Washington.

The mutual benefit corporation is https://lahcommunityfiber.org and I'm now on the board. It's basically the same corporate structure as our water company and so far is treating us well.

Costs are all over the map. My neighborhood ended up being about $12.7k to build out and $155/mo, but we're on a steep hillside and with large parcels in an expensive metro area. Another neighborhood in a flatter area and with slightly smaller (but still quite large) parcels and higher uptake was around $4k.

We were at ~50% uptake. Those that didn't either were outliers with Comcast, or weren't really serious internet users. Our alternative is AT&T DSL at 18/0.75 for $50/mo. We're expecting rebates on the initial build as uptake rises and additional nearby infra gets built, as well as a reduction in monthly cost as our backhaul gets better utilized. Getting enough commitment to be able to nail down costs and avoid the dilemma of a cost death spiral as people pull out of the project mostly boils down to peer pressure. We did OK on this in our neighborhood because the DSL alternative was so miserable that there was strong will to continue.

Construction is hard and the contractors were incompetent and constantly screwed things up. Folks suffering that incompetence got pissed. Getting power from PG&E is a miserable sufferfest and won't be resolved for another year. In the mean time we're mooching power from one of the subscribers and paying him back. Luckily we didn't need a lot in the way of permits because it's all private property, including the road.

If we were constructing in the public right of way we'd either need special permission from the town to do so, or we'd need to register as a CLEC with the public utilities commission. In practice we're working on both, but becoming a CLEC is so far away in our future that it won't be viable for any currently-planned expansions. Usually getting special permission from the town is easier for getting started.

I have 1000/100 cable and my upload is consistently 110-120

I have never seen a cable internet provider even advertise upload bandwidth, much less actually deliver it. They don’t even bother promising 2Mbps. Check Comcast/Spectrum/Cox/etc websites.

I assume the only reason they do not is because that is how little upload capacity they have.

I haven't had Comcast in a couple years but they did advertise upload speeds at one time and for the majority of the time (over 15 years) that I had them the speed was at or better than advertised.

The last speed teir I had with them was 150/75 in 2019.

Recently signed up with Comcast, they did promise an associated upload speed with the no frills Internet package I selected (200/5, $35/month) and listed it in the description when I chose it. I typically get 5-6 mbps up (although I rarely use it).

I don’t live in America? So maybe that’s why. I was advertised 1000/100 and I get that.

Welcome to botched network upload congestion handling. 15Mbps should be plenty if there's fair queuing at the choke point for the transition from LAN to WAN. See "bufferbloat".

Without that, you need huge amounts of bandwidth so that the FIFO queues never fill.

Back when I had crappy internet I used wondershaper to mitigate this sort of issue. It worked great and was very simple, it just throttled all connections a bit to eliminate the throttling.


I suspect that some major ISPs want bad queuing in routers to force people to upgrade and buy more bandwidth. Then they want to charge more if you use the bandwidth.

Not in my case it seems. I had done the bufferbloat tests I ran last year. IIRC, it was found off this site: https://www.bufferbloat.net/projects/bloat/wiki/Tests_for_Bu...

How much bandwidth did you actually have? I have symmetric gigabit shared with ~50 people and everything works great, but my router shows that video calls on all platforms take less than 10Mbps and somehow Zoom manages to have the best quality with around 3Mbps.

One big complaint I have about Zoom is that the user has little control over bandwidth usage. For upload, all you control is SD vs HD. For audio, no control. And seemingly no control over download, but it will use less if you run it in thumbnail mode.

I wish I could suppress download to thumbnail mode except screensharing.

As a Canadian, I’m used to having to deal with software developed in an environment with unlimited bandwidth and without caps…

I have 200/5 with Xfinity which is mostly OK. But it's hardly POTS levels of reliability. Went out completely on me during a call yesterday and I had to fall back on my (marginal) cell phone service. I sometimes regret dropping my landline. I had more reliable phone service in the 1970s.

> I had more reliable phone service in the 1970s.

I've considered getting a landline again, but it would be nothing but spam calls and surveys and scammers.

I grew up with landlines, and it was possible to have long, nuanced conversations on them without stepping on each other like happens with modern cell phones, and someone pointed out a year or two back that the issue is the latency. Cell phones are high latency, often with some variances in the latency - for everything. Landlines, across town, were both lower latency and far more consistent. Even across the country didn't seem so bad as a modern cell phone to your neighbor.

Talking on the phone didn't used to be painful - but it is now.

Talking on the phone didn't used to be painful - but it is now.

Indeed. I have a "land line", but it's just VoIP.

I miss ISDN voice. Rigidly time-synchronized, no stutter, no transmission noise.

Syonyk >"Talking on the phone didn't used to be painful - but it is now."


I ceased using a cellphone when I retired. Three days ago was conscripted into a cellphone conversation that had terrible audio and communications drops. If anything, cellphones have grown worse IMO.

My landline has ISDN to the local office but the audio quality drops b/c of the other party's cellphone. Good news: it worked through the recent Texas cold weather power blackout.

Wishing for the old days, when not only could speech be understood but also the emotions and inflections of the speaker were transmitted. All those nuances are lost "like tears in the rain":

"I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain.

Blade Runner - Final scene, "Tears in Rain" Monologue (HD):


I dropped mine last year (along with my cable TV) because it was nothing but junk. I'd have turned off the ringer and just kept it for backup but I don't use the phone a lot and $40/month or so was too much to pay for something I wouldn't really use.

I realized a few weeks ago that the house I've lived in for 5 years, I don't even know where a single phone jack is in the house. Except for the one that I tucked up in the attic when I remodeled the kitchen.

Our house literally doesn't have any. It was built new (manufactured home) in 2016, and I'm not even sure if it was an option in the build - I'd have to look over the list of options again.

In 2012 when our house was built, the builder tried to convince me to drop the RJ11 jacks. I guess I'm old fashioned, I told them to put them in anyway. Haven't used them for anything, oh well. They're not useful for anything more than actual POTS.

I'd recommend pulling the face-plate and checking the wiring. In my experience, builders use CAT5e for phone jacks (and for other low-voltage systems, like alarms.

Oh I'm sure it's at least cat5, but phone jacks are usually wired in series (in my experience, at least), which puts a damper on what you can do with them for data.

Yes, but they're usually terminated in an accessible panel located somewhere inside the house (I've mostly seem them in closets, look for a metal plate on the wall)

Once inside, it's a relatively simple process to cut the CAT5 wires at the punchdown block, and re-terminate them with male/female ethernet connectors, and connect them up to a network switch. If you're lucky, the panel will have a power outlet inside.

Lastly, connect one of the LAN ports from your ISP modem/router into the nearest ethernet jack and you should be able to use any other jack in the house for internet, assuming you terminated all the plugs in the panel into a proper switch.

Sadly, in my case they definitely did it in series. I documented it during the build and kept the pictures of all my in-wall wiring. It doesn't even terminate inside the house except at the jacks, it runs from jack to jack all the way back to where the electrical meter is, and then out a hole in the side of the house to where it would presumably be connected by Centurylink if I bought landline service from them.

One of my great blunders when this house was built was telling the electrician no when he offered to wire the house for ethernet for a few hundred bucks. He wasn't low voltage licensed, but he said he could still run the wire and just coil it up in boxes with blank plates for someone (me) to come along later and finish. I was in a rush that day and said no thanks, we're just going to use WiFi anyway. DOH. I still want to slap myself in the head almost 10 years later when I think of it.

The closest I can get to having wired network in the house using existing wiring would be using the coax which is installed to the same plates in every room as the RJ11. That wire does come back to a common point, and only one of those runs gets used for anything. But the gear to convert coax to ethernet is mildly expensive for what it is, and I haven't pulled the trigger.

What you can do, which is a little meh, but still usable is reterminate your cat5 as two rj45 jacks. For rooms with nothing with ethernet, put a small patch cable; for rooms with wired devices, put a switch with a cable to both ports plus whatever devices.

You should be able to run at least gigE on cat5, and shared 1gbps is probably better than what you get on wifi.

I've heard good things about MoCA, but it certainly costs more than regular ethernet.

It's worth double checking, home running POTS to a central point or at least the demarc got pretty common by 2000 or so. Makes it easier to add lines and what not, for your fax machine or computer modem.

You should call Xfinity about that. Up until my recent switch to fiber, I've had Xfinity/AT&T BI/@Home for ~20 years, and 7+ years at work, and it's been pretty reliable. The times I've had reliability issues have all been due to signal issues. Some have been damaged cable that got water into it, and would come and go based on rain or humidity. Some were bad splitters, or marginal cable that removed/replaced splitters had resolved.

It seems like the sort of thing where they'll be "Looks fine to us." It does get up to rated speeds. It's just that it sometimes is quite slow and now and then drops entirely.

I've had several experiences at home over he years where they said "If we come out and find the problem doesn't exist at the demarc, you will be charged X for a service call." Maybe that number was $75. In those cases, I just said yes, because it was worth $75 to me to have them test it. And in both cases they identified external wiring issues and rand replacement at no charge.

The most recent case was at work, with Comcast Business service, and they identified marginal signal power at our modem, and removed an extra splitter that was in the ceiling of another business. Took them around 4 hours to track down and repair.

I switched to fiber in 2019, and 2020 would have been a mess without it.

cries in satellite broadband

Seriously, we had to get a cell booster and root our phones to be able to use them as wireless hotspots just to survive and stay employed last year. It was an absolute trainwreck, the entire time.

Once talked with someone who lived in the country.

Satellite or drive to nearest town to use the public library during their hours.

I wish municipal Internet was an option by me, but much of the US is purposely broken:


I have symmetric fiber through Sonic here in SF for $50/mo and I think life would have been hard without it. I'm casually uploading things (Classic `docker push` a non-slim image) over while on video calls without a stutter. Amazing.

I noticed what you describe at work, too: families with kids in school and both parents working had the most difficulties. In places where home internet speeds are marginal for pre-pandemic usage, it was absolutely garbage.

Get yourself a router that support good AQM such as fq_codel or cake.

You will likely discover that 15MBps upstream was sufficient.

"And quite frankly, I hope I don’t have to give another live lecture ever again. I’ll tape it; I’ll have them watch it — I know most of them are going to watch it at 1.5 or 2x speed. I’ll sound like Alvin and the Chipmunks, but that’s OK. Then we can spend the time in class actually working on problems or discussing some of the issues that I brought up."

Beside the joke, I find the concept of ingesting content at your pace and then talking about it in class very interesting. No idea if the knowledge will stick in mind as well as classic lectures, but I would definitely have been seduced by this proposition when I was a student.

", I find the concept of ingesting content at your pace and then talking about it in class very interesting."

It's called a "flipped-classroom".

The teacher gives you material to read or watch at home before the class, and then in the class it just asks if anyone had any questions, and they can go over these together.

As a teacher it is super useful because you get to learn first hand which concepts the student didn't understand.

And as a student you can read or watch something whenever you want at the speed that you want, and if you don't understand something you can ask about it in class.

With some experience, teachers can also use the flipped-classroom model to focus only on the hardest part of a particular lecture. You can rely that some student is going to have trouble with it at home, so you can prepare before hand to the lecture to how to address that particular question (and once you have done the class a couple of times, you get pretty good at it).

There is no need to waste teacher and student time during 1:1 interactions on "easy" stuff that everyone can just read at home.

How do you deal with shy students, or ones who don't like to participate? This model sounds interesting, but I can't imagine exclusively using this method.

> How do you deal with shy students, or ones who don't like to participate?

With students who aren't participating in the current model? They will (have the opportunity to) do better than they are now, as when they watch prerecorded lectures they can skip back to parts they didn't understand. As opposed to their current behavior of not speaking up and, therefore, not being able to go over the content again.

> How do you deal with shy students, or ones who don't like to participate?

This method allows students to learn at home at their own pace, and use face-time with experts to asks questions.

Just learning at home at your own pace is a net win for all students.

Those who are a little shy, come to class, and learn something from the questions of others.

Those who are more shy, don't even come to class. They can read the stuff at home, and have ~2 hours extra per week to do other stuff.

We have 1:1 tutoring 2 per week (students have to get an online ticket), and they can also ask us privately per email, but in my experience, students that are "too shy" to go to class and ask questions in front of others are also often "too shy" to asks questions privately.

The other aspect of these is homeworks. Our homeworks allow groups, but they are thought of as "individual homeworks". We recommend people do them on their own. This means that non-shy people as a group get them done faster and often better, but it also means that shy people doing them on their own, end up learning more and performing better in the exams.

We don't grade homeworks, just pass or not pass, but passing them is required to be allowed to take the exam.

Still, we do internally grade them, and people that do them well alone do perform better in the exam.


In my opinion, the only "downside" for "shy" students, is that since we give them the freedom to not attend _any_ classes, they might decide not to do so. That means, no access to new friends, new colleagues, in person learning groups, isolation, and well understanding the hardest parts of the material might cost them more time if they are "on their own".

Most still make it. Some do find learning groups or new friends in the class forum, some do better than many students that do attend classes, etc.

Having taught many flipped and non-flipped (classic) classes, I never had the feeling that forcing people to attend class fixed any issues related to "shy-ness".

Students are how they are, I'm neither their parent, nor counselor, nor psychologist, nor am I qualified to do any of these things. I'm their teacher for a specific kind of content. I want the content to be useful and applicable to them, so that they can use it in their careers to be successful, whatever successful means to each of them. I also want them to enjoy the content, so that they can learn it easier.

For me this means giving them as many options as possible, so that they can master the content in whatever way makes the most sense to each of them.

Anyone speaking improvisationallly (not from a script) can be listened to at 1.5x at least. I get through a lot of tutorials at 2x.

I may be in the minority but I really don't care for listening to things sped up. If it's really just about maximizing data transfer, I'd probably rather read a good transcript at that point.

Unfortunately, as I'm sure you've discovered, "good transcripts" don't exist unless someone puts in the effort. :(

I don't mind audio, but I definitely don't understand the value of "talking head" style videos. They're, if not the worst possible information density possible, pretty darn close. A couple hundred megabytes to see someone speaking a few dozen kilobytes of text, and I still can't search it or skim it to see if it covers what I want.

A good transcript isn't really that hard. You just need to pay someone--the ML ones just aren't good enough to publish--and do some light editing. I usually do them for my podcasts. (Though the ML transcripts are good enough for skimming content and I use them if I'm just going to be using some excerpts from the audio.)

I actually find that, for talks with slides, I do like having the inset speaker video. What I tend to do is to start off with a full screen talking head intro with mostly the slides/multimedia and a small speaker video.

Totally agree that a good transcript is way better in almost all cases (except where actual moving visual demo actually helps).

However, are there any good automated transcription tools that can make a good transcript from an imperfect YouTube video? Most of what I've seen is good for humor, and I've even seen some used in a legal deposition context that are downright dangerous (inserting most likely word or phrase in place of what was actually said, so actively decreasing and corrupting info content). This might not be as disastrous for a lecture where there are also office hours, but... anything good out there?

>are there any good automated transcription tools that can make a good transcript

Automated? No. But if you're willing to pay $1/minute and the audio is good/accents aren't too heavy/etc. there are a bunch of options.

Any you'd recommend?

Maybe you'd like to try us out at https://3scri.be. Like I offered above, if you'd like to sign up and drop us a line at support@3scri.be mentioning this thread I can add a hour on top of the 30 minutes trial.

Well, we're new and we think we've the most accurate transcription engine on the market. Why not try us out at https://3scri.be. We give half a hour for free but if you sign up and drop us a line at support@3scri.be I can add on an extra hour for you.

I have had maths lectures where that just doesn't work, so I'd question the "anyone". You need the extra time to fill in the steps the prof left out, because they are obvious to him.

(And yes, I usually also go through presentations at 2x).

There is a forcing function to in-person lectures; I can't tell you how many events I've basically skipped in the past year because I can catch up on the videos anytime--and, of course, I mostly don't. But, in principle, depending upon the size of the class there's a lot to be said for watching the lecture on video and using class time for discussion, project work, etc.

There's also something ritualistic about in-person lectures -- in a good way, like having a morning cup of coffee. We know that humans are not good at multi-tasking, so there is tremendous value in a dedicated time and place to think about one thing only in the presence of other people who are also thinking about that one thing.

Of course, it's easy to squander that opportunity by delivering a rote, feelingless lecture. And if you're just going to lay out the facts, why not record them? In that case, videos have many advantages. But a live lecture is also an opportunity to get people excited about what you're teaching.

Post-pandemic, my plan is to take a hybrid approach. Technical details, like proofs, belong in pre-recorded videos watched out-of-class. But the main conceptual thread should be delivered in live lecture, where I can give it the energy and life it deserves.

Pre covid, we had one professor who did lectures with videos (of him presenting) combined with in-person tasks and discussions. Each week you had to take a 10-minute test on the content of that weeks videos. It worked quite well.

The biggest disadvantage was that with a live audience the professor got cues from the audience if we understood him or if he had to slow down. In his videos he constantly assumed he had to repeat everything slightly differently, so you had to watch at 1.5x speed to get to something bearable.

Yeah, particularly with a smaller audience that can definitely be a problem. I'm better at it than I was a year ago but I still find doing a video without an audience can be a bit challenging. And I definitely can't course correct the way I can if I see a bunch of puzzled expressions in the front row of a room.

On the flip side, you can redo sections and easily insert multimedia and just mix up the talking head format.

The best class I ever had was one where the teacher explained things for half the time and then we worked on homework for the other half.

I think that is probably the best model. Letting students watch lectures on their own time is hardly better than expecting them to read on their own time.

Years ago I heard of experimental elementary/high school classrooms where students were expected to watch lecture videos at home and what would be “homework” would instead be the focus of the whole time in the classroom.

Students were expected to help each other with the material and only go to the teacher if they were collectively stumped.

I can imaging all kinds of bad scenarios were someone tried to switch a disruptive class to this routine overnight. But, with good onboarding and ramp up, that plan sounded like a dream to me!

College classes are often split into lectures and precepts/discussions/office hours. I think the thing holding back your idea is the expectation of being taught in-person by a professor. For large classes, that’s only really possible in a lecture format.

My personal favorite method of learning was a TA that went through the material, meticulously calling on each person in order. You could skip, but it was still obvious if you were playing a game or browsing Reddit. Eventually you learned that you really didn’t have all the answers and that paying attention was probably a good idea. It also got people comfortable giving a wrong answer and that really helped everyone with the learning process.

Most folks don't realize this, but many issues are actually due to bufferbloat. Get a powerful router that runs CAKE and you'll be amazed at how little jitter will be seen while maximizing available bandwidth.

Also, if you live in a dense urban area, you'll also likely suffer from issues with available wireless spectrum and have to compete with everyone's router blasting as loud as possible.


But isn't the problem at every level, including the modem? Having a powerful router wouldn't help if the modem it's connected has bufferbloat as well.

I said powerful router because running CAKE/fq_codel means your router's CPU will be processing each packet. Which is not normally the case for a router processing unicast traffic without an AQM algorithm.

For instance, much of Ubiquiti's lineup will choke on anything over 500mbps w/ fq_codel (aka Smart Queue) enabled.

If you've profiled the system and rate limit bandwidth to what's below the "steady state flow" of your network (including modem), then you never have buffers. It's usually only one or two layers you have to worry about before you get into some higher bandwidth backbones and the problem largely goes away. If L3's main links are buffering, the internet has some larger problems.

As a concrete example here, I run networking for our church, and we've had to make a facility that really wasn't designed for livestreaming into something that can toss out a tolerable stream.

We've got... no idea what we pay for, actually, but it reliably measures about 70/15 on Sunday mornings with nothing restricted. However, due to some various quirks of network naming, there are a lot of other devices on the network, some of which only get woken up for Sundays, and they like installing updates. Plus cell phones that recognize "Oooh, wireless!" I need to split the network up, but I also hate rejoining "things" to networks, and we have a few more of those than I really care to deal with.

Experimentally, while our max upstream is about 15Mbit, things start to get erratic beyond about 10 - latency starts getting inconsistent and we start having stream issues. I've capped the livestream bandwidth at 8Mbit (we hardware transcode on site from the Main profile h.264 coming out of the switcher to High profile, at a lower bitrate), and that gets first priority - I'm using Mikrotik queues, so any packet coming out of the server on Sunday goes first. I played around with what everyone else gets, and eventually settled on around 2Mbit - that can mix in with our livestream and still not impact anything outbound.

But I also had to cap our download. While the connection can do 70Mbit, I reliably saw upload dropping (even from 10Mbit) if something had pegged out download in the morning. So that's capped to a conservative 20Mbit, which is enough for most things, and is low enough to not interfere with the higher priority upload.

This is all on timers, and the limits kick in Sunday before the services, and drop off afterwards.

You can also find a lot of gains if your router prioritizes acks outbound. DSL modems were often so badly asymmetrical (think 25Mbit down, 768kbit up) that your upstream acks would end up in queues and not get through for a while (buffer bloat, though I didn't know that term at the time). You could radically improve a DSL modem's connection by putting some queues in to manage upload. Prioritize acks, and then limit your total upload to about 750kbit (on that example 768k modem) so that you weren't buffering in the modem.

And all of this is entirely in the scope of the router, even if it's working around issues further upstream.

So you capped your connection from a theoretical 75/15 to 20/8? I guess it makes sense if you value latency above all else, but for situations where throughput matters (eg. streaming or downloading game patches), it's unacceptable. Dynamically setting your speed cap would get rid of this problem, but would be huge timesink to get it just right.

I leave 2Mbit for other uploads, so it's 20/10, but, yes. During times when we care about latency and upload packet loss with a "realtime" set of requirements (there's no mechanism to retransmit lost packets for livestreaming with what we're using, so lost packets are just lost and glitch), the connection is heavily restricted.

However, if you're trying to reduce buffer bloat and random latency, the general concept of restricting to what your connection can actually tolerate works quite well. I've done it plenty on various connections over the years.

>Get a powerful router that runs CAKE

It doesn't even need to be powerful if you're on a normal copper line for home use. I'm saying that running a 20€ Xioami router and having machines torrenting, an Android box watching stuff, phones watching YT etc. without hiccups.

> Get a powerful router

Any guide on how to do this properly? My ISP's hardware is garbage but I'm not sure how to replace it. Feels like it was easier in the DSL days.

Key takeaways

- asymmetric broadband (high download speeds, low upload speeds) won't work in a world where people are doing real work "in a world of Zoom conferences, we need something that is more symmetric in download and upload — or at least upload speeds need to be a lot faster than they were when it was mostly Netflix and HBO Max"

- rural/urban divide is real and a severe problem "for the most part, cities have done reasonably well, and rural areas have done reasonably badly. So this divide is really economic and population density"

- the free market has few, if any, incentives, to build out to rural and low-density areas "If you are running a wire, whether it be cable or fiber, it costs the same amount per mile pretty much no matter where you put it. So if you can put it someplace where you can service 100,000 people, it’s a lot more economically advantageous than if you’re going to be serving 20"

- there isn't any real free market anyway, since most locales have a single provider, maybe two "while there may be multiple internet providers, there are very few internet providers at any single place. So we essentially have, at least in regions, de facto monopolies that have very little reason to increase their offerings, at least from the sense of competition.

- jobs will not be completely unconnected to location "It’s still going to matter that you are in the same area so that on occasion, you can come into the office and meet physically together. But I think there’s going to be much less of the five days a week on campus sort of work. People will be able to work two to three days a week at home, without any loss of productivity or loss of culture within their group."

I'm constantly baffled at how the US can be stuck in this situation, being a country known for high tech businesses like Google etc. Especially when well-tested and proven solutions are readily available if politicians were interested. For example the problem with only one ISP is easy to fix (in theory, I know it isn't politically): Force infrastructure providers to open up for competitors. All electricity companies can be picked as a provider on the local infrastructure where I live and the same with internet. Even very rural areas have fibre (though there's nothing as rual as rual America of course). In my opinion all these problems stem from the two party system where nothing really changes so everything becomes inefficient in the long run.

> For example the problem with only one ISP is easy to fix (in theory, I know it isn't politically): Force infrastructure providers to open up for competitors.

The problem with this is that the US already did this, but with bad rules in the telecom act of 1996. After a few years of litigation, the FCC decided the line sharing requirements didn't actually apply to anyone, because there was 'enough' competition between cable, dsl, satellite (hah!), and powerline (double hah!). I don't know how we would get the 96 telecom act back, but this time with rules that make sense when the issues are difficult to explain to people.

As the article points out, there are (or were) many ISPs, and the FCC managed to finagle that fact into "there is actual competition" even though most markets were only served by one, or perhaps two. The country ended up such that actually trying to enter a market where one of the big telecoms is dominant essentially can't happen, but the FCC continues to pretend that competition is possible and the markets are free.

I'm probably kind of ignorant here, but isn't there a difference between ISP and electricity providers though? When it comes to electricity, the infrastructure is used to bring the product (electricity) to the customer. When it comes to Internet, the infrastructure is the product.

Say the infrastructure is now open for all companies. What is now the differentiator between Comcast and Fios? The infrastructure and the bandwidth it can handle is what they are selling.

edited for clarity

>What is now the differentiator between Comcast and Fios?

Price, support, included services, etc.

For example the infrastructure owner of my fibre connection is exactly that: an infrastructure company. The infrastructure is really good but their internet support is really bad. On the other hand my ISP is running a very tight ship with no bloated services at all. No TV packages available, no phone option, no bundled streaming services, only direct payment with card transactions (because it is cheaper), only support by mail and so on. In the end I get a less bloated service meeting my exact needs at a lower price but my mother can pick another ISP that can provide her with the services she needs and only one company needs to run cables in the area.

There are other differences too if going into details. Like the hardware they supply the end user (infrastructure owner supply the typical crappy WiFi router I'm sure you know and hate while my current ISP just gives me two external IPv4 and IPv6 addresses directly in the fibre box and that's it, I gotta take it from there). The hardware between the fibre into my house and the long haul fibre out in the streets is also supplied by the ISP. The infrastructure providers hardware is old 1 gigabit per port while the ISP's is... well I don't actually know but faster than 1 gigabit. This makes a small difference as my connection is 1 GB/s and 1 GB/s hardware cannot in practice deliver 1 GB/s so while one promise 950mbit the other promise the full gigabit.

All in all I get a cheaper connection but I would use my current ISP even at the same price. To me it makes a big difference that the person I can contact isn't some fresh out of school boy who reads from a list but someone who can tell me "this is why your IPv6 temp. addresses setup isn't working. Try this init string..." :)

How is the infrastructure the product? If I buy a cable modem and plug it into the wall but don't cough up money to Comcast/XFinity, I get nothing, same as if I plug my coffee pot into an electrical outlet but don't pay the electric company for service.

Electric cables can be shared by different companies because they still have the differentiator of how they get the electricity they sell you. Either they make it, or buy it and negotiate prices, etc. The electricity is the product, not how they get it to your house.

What would be a differentiator for ISPs if the infrastructure was completely shared? The internet would be the same.

oh, you have NO IDEA.

How about: not hijacking your DNS to redirect domains? not using traffic shaping to favor content the ISP makes money from over other content? Not blocking torrent ports? Not having monthly usage caps? Not filtering or blocking content based on country of origin? Not blocking Tor? Not requiring you to rent the router/modem from the ISP and have them control it?

In other words, all those Net Neutrality things that the FCC took away. https://en.wikipedia.org/wiki/Net_neutrality

You do realize most of these things have very little impact on cost and are barely understandable for random normies. Tor? DNS? Like they're gonna put that in their marketing campaign?

I just happened to mention ones that would resonate with HN readers. Usage caps, filtering sites, and requiring modem rental are things that ordinary users care about. Remember that Comcast was rated "worst company in America" twice at Consumerist, and was runner up twice, and appeared in the final four two more times.

As far as impact on cost, do you mean "cost to the consumer" or "cost to the company"?

Comcast is the most hated company in America mostly because it charges random things that customers never authorized, because of how it structures its TV packages and because its support usually sucks. At least that's what I got from a quick google search. Nobody mentions data caps or filtering sites. As for modem rentals, again, most normies don't even know you can buy a modem at BestBuy.

As for costs, I meant costs passed to the consumer. If we assume that the infrastructure is shared by 5 ISPs and they all contribute to its maintenance/upgrades. All ISPs offer the same down/up speeds and the reliability of the infrastructure is identical. What are the knobs to adjust costs to the customer and have them pick you? Pay support people less, have them work in shitty offices, do shady stuff with customer data. Maybe I'm not creative enough but other than reducing your internal cost of operation what can you do?

It's ridiculous that this is still an issue- cheap fiber optics have been around since the 80s. The blame rests on the ISPs for their monopolistic behavior.

The problem isn't the ISPs, it's the regulation and the arcane onion of overlapping jurisdictions. Places where infrastructure owners are legally bound to share with third party network operators don't have that problem.

One of the interesting things about the infrastructure bill that has been proposed in Congress is it talks a lot about universal high-speed broadband access in the same way that, back in the ’30s, the recovery bills from the Great Depression talked about telephone access for everybody.

Does anyone know about how this legislation worked out? Did it really get more people telephone access?

I'm skeptical of any praise toward the New Deal, because it's almost universally praised by just about everyone, but then you dig into it and you find out that agencies like the NRA[0] were so bad that even their proponents had to admit they were a failure, and the "successful" programs that lasted, like Social Security, pale in comparison to more modern programs like Australia's Super.

I don't hate the idea of the government delivering internet, but more like they deliver water or electricity, at the municipal level, rather than some set of national, overarching subsidies and regulations that will probably create perverse incentives all over this very large country.

[0] https://en.wikipedia.org/wiki/National_Recovery_Administrati....

Yes, I think the Communications Act of 1934 (and the Rural Electrification Act of 1936) was successful. The Communications Act established a Universal Service Fund that used long-distance charges to subsidize phone service for low-income households and expanding service to households that would not be profitable to serve.

The Telecommunications Act of 1996, meant to be sort of a digital update, has clearly not been successful.

We decided almost a century ago that universal telephone service was something that we wanted to have. We need to make that decision about the internet now.


Yeah, sure.

Then Bell System had to be broken down, and even that wasn't enough.

The whole mess with internet access in US is due to local monopolies held up by idiotic regulations. More regulations won't ever solve the problem.

Regulation is what connected every home to electricity and telephone service, it could do the same for quality broadband service.

Rural America would still be without phones and electricity without subsidies and universal service mandates. Privately owned businesses are not going to invest $50,000/mile to gain a customer who pays $50/mo.

That's the point - why it costs $50K/mile? Regulations, that's why.

It would be really nice if those designing technology systems for the web would spend some time using the stuff they design on lower speed connections. There's no excuse these days - the Chrome web inspector lets you simulate network profiles if you want (though I wish they'd add "Rural WISP" options by default - say, 5/1 with some random latency spikes and a few dropped packets every now and then).

I've been on a 25/3 (that mostly delivers 20/2 during the day, and "a connection" during the evening that... varies wildly) for about six years now, and I've been working 100% remotely that entire time. It means I have to think about some of my transfers and let them run for longer periods, but I've had no trouble doing video conferences and other things on this connection. I just have to pay attention to what I'm doing, and I might need to cancel or pause a transfer if I'm jumping on a video call. If it works for the meeting, I tend to use a cell phone for audio - most video conference bridges support this method.

But we've also generally designed our life around the fact that we're not on super fast connections. We cache content locally (I tend to rip DVDs to the server so I don't have to deal with the physical disks), and while streaming works if we let it, I'm certainly not streaming high bitrate 4k content... but we also don't have a large enough TV that it really matters at our viewing distances. Also tend to cache other stuff - I'm running a local Ubuntu repo, because I have a lot of VMs that do various things, and sucking down updates over the LAN is far faster than running over the WISP.

None of us play extensive computer games, and while I do game a bit, it's things like Minecraft or Kerbal Space Program that are offline/local, or at least not latency sensitive. I hear lots of complaints about how bad consoles with online-only games are on slow connections (especially the update game), so we just don't have one. We game quite a bit, but it's board games, tabletop games, etc.

If stuff were designed for "something slower than gigabit," it would certainly improve the experience on slower connections, but literally making a living on something a lot of people barely even consider broadband, I've not found it to be an actual issue if you just work within the limits.

Now, the usual answer for this is "But Starlink!" - and I have it. It's a secondary connection, because it's not actually usable as a primary connection for much beyond casual web browsing and file downloads these days. I know it's a beta, and I'm certainly putting traffic through it to help with their testing, but it's just not a good primary connection if you have alternatives yet. The bandwidth is fine, the latency is good, and the rest of it is just annoying. It's really prone to breaking long running connections (CGNAT and it seems like my public IP changes halfway often), so SSH tunnels and VPNs get broken constantly. They tolerate some packet loss just fine (just pause for a moment and then resume when the packets start again), but over Starlink, they get reliably broken. I've not poked around enough to figure out why. The bandwidth is also hugely variable (both upload and download). Sure, you see 300-400Mbit at peaks, and I often see north of 200Mbit... briefly. Then it slows down, drops to less than 10Mbit, pauses, goes back up, etc. It doesn't show in random speed tests as much, but if you scp or rsync a big set of files over, you can watch the speed rise and fall. I can't imagine bandwidth estimation algorithms are able to make much sense of this for things like video calls, and the quality of video calls on Starlink isn't great yet. It's also prone to plenty of "micro-drops" where it simply stops passing traffic for 5-10 seconds. Not normally a huge deal, but very, very disruptive for video calls. I still use my 25/3 for any sort of video conferencing, because it's slower, but more consistent.

(One may safely assume I'm pretty rural)

Very interesting to hear your experience.

Hopefully increasing satellite density fixes the speed reliability.

And I guess I understand why your IP would shift and break connections but they really need at least an option to not do that even if it adds a few more milliseconds of latency.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact