More power to you, NYC Mesh.
There are bazillion reasons that this is nothing more than a cute side project that will fall on its face when its time to scale in any meaningful way. Unless they have discovered a new technology they are not taking on ISPs by any meaningful way. Especially in NYC.
There is a difference between, just "connecting to the internet" and having a "reliable connectivity to the internet", the difference is a massive investment in infrastructure. Wifi p2p mesh networks don't work on magic. They are very unreliable and hard/impossible to scale to 100,000 users, let alone millions of users. In some places, wifi p2p connection hardware actually costs more money than a traditional wired network.
Mesh networks have been around for a while, technical and reliability aspect of this is well understood within the networking industry. There is a reason why it's not widely used, especially in cities. In remote areas, this kind of network makes sense, if you don't have a choice or an alternative.
Don't brush of educated criticisms from people within the industry. I own an ISP for 5+ years (mostly wired). I am not in US and I am not at all worried if this kind of network comes in my country/area. We are not worried because we have tried and deployed this kind of network in cities and they are extremely unreliable and costs real money.
I also don't understand the "Anti-authority" angle of the argument, unless I live in a different universe, they still have to get license/permit from the local government to do this business (non-profit or not), they still have to follow the same rules enforced by local and federal government when it comes to log/customer information sharing to LEO. So not sure how they are Anti-authority? Unless they mean anti-establishment?
Any ISP (if big enough), can host their own Google, Akamai, Facebook, Netflix nodes on their premise.
as 'known to be the largest community network in the world ... and is still growing while successfully evading the collapse of the tragedy of the commons. Undoubtedly, this is due to the governance model that the foundation has developed....'
Since it's been around for a while, and still viable, that says something about the idea worth attending to.
More info: http://www.dtic.upf.edu/~jbarcelo/tmp/guifi.net.pdf
These things don't scale well and all those radios can be universally jammed by a single drone over the city. Not a great backup plan if you're anti-establishment.
I'd love to partake in such a project but keeping your expectations grounded is very important.
The jealousy amongst peers for non-recognition.
"More power to you, NYC Mesh." is the correct answer. Thumbs up tlrobinson.
Also have a matrix chatroom on this topic: https://riot.im/app/#/room/#startyourownisp:matrix.org
I'd like to imagine this would be competition on price and service. Sadly, that won't be the case. They can expect everything from spurious CALEA requirements to rooftop lease shenanigans to unwarranted radio interference claims.
Eventually, though, this mesh model will take over the world, from both a network perspective and a commercial one. We just have to squelch our authoritarian impulse.
Of course, technical challenges may still exist, but those certainly don't discredit the model.
I don't see how this is true for any reasonable definition of "edge". It's certainly not the case that every node directly communicates with every other node; if they did, you wouldn't need a mesh.
Let me approach your math with a reasonable argument:
Assume a 1 sq mile area (1mi x 1mi).
Assume each node has a 50 yard diameter reach.
If you space a node every 50 yards, you will have 35 nodes on a side, for a total of 1,225 total nodes.
Each node will only be able to physically connect to 3 other nodes on the edges, and 4 other for interior nodes.
That means the node connectivity is only .00326 for an interior node.
The longest path is diagonal across the square. Given the 35 modes on a side, and A^2 +B^2 = C^2, that means 35^2 = 35^2 = C^2. And c^2 = 2,450, meaning C = 50 (rounding up).
So I’m not sure how O^2 even applies in this example. Yes, it’s theoretically a max, but theory >< the real world.
Is this not correct?
At least with the standard ISP model there is some accountability and privacy rules could be implemented/enforced.
Edit: I think I might have been misunderstanding what mesh network was referring to - this looks like a wireless access point?
Also, individuals who are concerned about this would be free to purchase a direct line via an ISP.
That seems to suggest otherwise? Unless I don’t understand what network traffic within the mesh means.
Certainly there are logistical differences between these things, but I don't understand why one would assert with confidence that such a project (that already seems to be working) couldn't work.
Doing > Wanting to do in nearly every instance. So to borrow from popular culture, Just Do It, and Haters Gonna Hate.
At least that was my read on this community.
...but why Dropbox? We already have google drive/OneDrive/WebDAV.
...but why an Airbnb? We already have BnBs, hotels, and hostels.
Compared to most funded startups, this idea seems extremely practical and feasible (1.6 million people in Manhattan alone). Wireless internet is already a successful business model (Monkeybrains in SF), and extending it with mesh may add latency but I'm not going to pretend to know more about it that the guy who is currently doing it...
This is the opposite direction. Mesh NYC is saying that ISPs are useless because anyone can string up some APs and get access to the Internet.
Which explains while there is always a caste of "professionals" who for the sake of "security"(job) - defend against the relentless wave of people who unleashed could undo there little safe space in life. To be fair- most of them have familys, so they are not doing that for greed- but because out of vulnerability.
You want WISP competition (co-op business model or not), then lobby the FCC to free up some spectrum for it. Cheap gear is here, spectrum is the scarcity.
So here is the plan- we jam a frequency of a local TV-station, nobody views, who is just sitting on the spectrum for license worth.
After proofing for one year and one day, that nobody listened to it and nobody cared- we pronounce that this spectrum is unclaimed and should be returned to the government to be auctioned off.
Under normal circumstances this should not go through legally- but hey, there is booty,
there is a government in debt and
there are interested parties.
Its the perfect storm and just having legal rights - didn't help the native Americans either, didn't it?
Come to think of it- maybe planning the heist in public was - ill advised.
I wonder how much it's because so many Silicon Valley salaries are dependant on people assuming that replicating what they do is way harder than it really is? What happened to Britannica or BitKeeper or AltaVista might just happen to the ISP industry - or any other Start-up-de-jour...
To contrast, those with negative presumptions have plenty to say, and incentive to say it (arguments are a form of discussion).
This leaves us with a skewed representation of attitude.
I think an unconscious bias may be that a bad but free connection that can negatively impact the perception of web solutions from casual users, which in the end is the revenue base for everyone here involved in B2C business models.
Unreliable but redundant commodity grade hardware pretty much allowed "cloud computing" to eat 99.9% of "big iron" sales.
I strongly suspect what Google does internally with software defined networking looks a lot closer to what these people are doing than the traditional "buy a small number of powerful-but-expensive entrenched vendor network hardware" approach.
This mesh is kinda the network equivalent of autoscaling or autorepairing cloud computing strategies that are so common these days. Once this grows to 2 or 3 IXP connections it won't surprise me much to find they stomp the traditional ISP model reliability-wise, and if they can build that "virtually", where they don't need much more than a bunch of somewhat transient mesh participants with a few medium reliability "super nodes" which are mostly completely remotely configurable...
Also, I'm guessing that users share the same IP address, which can cause problems. Plus, if one user is doing something illegal and causes the supernode to get shutdown then no one on the mesh network will have access to the internet.
Addition: quotes like "The internet doesn't really cost you anything, it's just the connection [that has a fee]... Nobody owns the internet, there's no one to pay." Are shockingly naive. Who does he expects pays for the fiber that runs across the country and along the ocean floor? The marginal cost of a new user on the internet is effectively zero, but that doesn't say anything important. The infrastructure that makes the internet work is enormously expensive.
All networks, including us, add to this expensive infrastructure. The difference with our infrastructure is it is owned and shared by members.
So the main thing holding you back is time, money, resources, access, customers, and logistics?
A bunch of businesses and nonprofits offer free wifi to their customers, and over time you build up a big enough blanket that pretty much anybody can walk into a coffee shop or library and check their email.
Tons of security risks, of course, but... I mean, I've used public wifi before during personal emergencies, and I was pretty grateful it existed. If you're gonna pick a comparison to be derisive with, maybe don't pick something that's widely useful and appreciated?
Comcast has even turned this into a selling point (I think somewhat unethically) by turning all of their customer access points into semi-public routers for other customers. It actually seems to scale pretty well.
Comcast provides its wifi by allowing you to connect to any other customer's router. (Roughly) a mesh network provides its wifi by allowing you to connect to any of the other nodes within the mesh.
What's your point? Is the money/infrastructure less legitimate because it was donated? Is the public wifi down at the library fundamentally worse because its cost wasn't bundled into the price of a coffee?
Hardware is hardware. If it works, who cares where it came from?
Far fetched you mean like getting instantly connected to the internet for a fraction of the cost and with better bandwidth in major city?
You realize that mesh is just for the access network correct? There's still fiber between them and their transit providers. Mesh makes a lot economic sense compared to the costs of trenching and laying fiber miles.
I don’t see why that would be the case - you can give every user their own public IP directly. I would imagine they’re giving everyone an IPv6 /64 and a carrier-grade NAT for IPv4 (as it’s really hard for any new player to get usable amounts of IPv4 space).
A spine/leaf arch would be great for this imo. The leaves can be local cells of wifi networks. The spine can be operated by payment collecting individuals that maintain a high speed(10g+) and low latency uplink to the internet and network gear that can handle the traffic from the leaves. In spine leaf, leaves connect to multiple spine nodes as well (don't have to worry about spine operator reliability too much,could potentially load balance over spine)
That being said,I hope any modern wifi network assumes wifi encryption is not reliable and instead implements a layer2 or 3 tunnel (macsec/.11ae and wireguard respectively)
1. The internet has been captured by the telcos, who operate the internet like telcos have operated the circuit switched voice network.
2. We are all overlooking the fact that smart radios make spectrum ownership obsolete. That means that the potential of mesh wireless connectivity is orders of magnitude larger than can be realized in current unlicensed spectrum.
3. The assumption that anyone is owed a business model in internet service is going unquestioned. Of course it costs a lot of money to run the internet the way the Bell System ran phones. But there are plausible alternatives, not all of which are compatible with investor-owned near-monopoly "markets."
This is an economic tragedy and a case of local optimization holding a global optimum hostage. ISPs are a sweet business, in a way that costs the rest of the economy dearly. Kill the incumbents and reap an across the board boost in GDP.
So with line of sight I know you could 'beam internet' from either a plain old ISP connection OR from an internet exchange.
This was consumer grade equipment too, I am sure there is much much stronger out there.
As far as the potential of mesh in rural areas, I think one of those two options should be workable. Of course there are costs and geography that may hinder.
> While there are mesh networks dotting the U.S., she says the best working example of what mesh technology can do is in Spain. Guifi.net has more than 34,000 nodes covering an area of roughly 50,000 square kilometres across the Catalonia region.
The ultimate benefit, I would think, would be to cause downward price pressure on the establishment ISPs. But wasn't that also Google Fiber's quest? Which we know didn't last long. Profits or not, there is still a certain level of cost to maintain sustainability of the model.
I hope I don't sound cynical. I honestly just curious and will to share my thoughts out loud.
This is a trend that will definitely take off and removes the huge barriers of entry that laying cable requires. 1ms latency too.
Oh, on average these next generation networks have much lower latency than traditional cable and DSL lines.
That's a LiteBeam Gen1 installed in the east village on 5th St connected to 375 Pearl St downtown. The gen1 actually has a stronger signal than the gen2, so this is a bit better than our usual connection.
That makes no sense. I doubt they're a tier-1 backbone themselves, they have to be purchasing transit from an ISP. It feels like lots of the docs are written by marketing-type folks. Also there's a weird diagram that insinuates NAT happens before bgp peering with the not-ISPs they buy internet from. Weird shit.
“We don’t have any use cases that show how nice it can be if you have enough users.”
They are going to need software. The supernodes are still a symptom of the thinking that you need the signal to go to another stage in order to communicate locally.
I think there should be more software that works on a local network, like things used to work before broadband.
This is what I’m talking about:
I mean: good for them.
(correction to original post: I did not at first see an ASN for them, or announcements of their own IP space. They have an ASN and announce two /24s, and peer at one physical location in metro NYC).
Real five nines network infrastructure takes significant capital investment.
This part is so much unadulerated bullshit:
""The internet doesn't really cost you anything, it's just the connection [that has a fee]. So however you can get plugged in — then you're on the internet. Nobody owns the internet, there's no one to pay.""
Actual, reliable internet costs real money, both in the salaries of people to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, equipment, salaries and expenses for field technicians to build it. And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each (do you really want to deploy something in the year 2018 to take several full BGP tables that doesn't have a 4 million FIB capacity?), etc. The Internet is a significant construction project at OSI layer 1, whether you're putting PTP radios on rooftops, running aerial fiber, or underground fiber. Otherwise you're just piggybacking on something that another, larger entity has already built.
I give them an A for enthusiasm and effort. It just needs to be channeled the right way so that they can figure out what it actually costs to run a reliable ISP. I'm all in favor of new startup ISPs.
Our uptime when connecting to supernode1 is significantly better than Time Warner Cable as many of our members have reported.
We are already established. This is something we have already done! It isn't impossible.
Our IXP is DE-CIX, the worlds biggest IXP. We are currently in one data-center/IXP and talking to a few more colocs.
Yep, DE-CIX is great and 32 Ave of the Americas is one of the most wired buildings in NYC. Great choice. Kudos on you folks, this is a great initiative. Reading this article made me happy. And I'm glad you are getting some press!
They are cheap as hell and awesome.
Actually - what you should do, is have a customer pay a fee of $100 to connect to the system - and take that $100 and buy another mesh was with each customer on boarding.
You announce a whopping total of two v4 /24.
Kudos for having enough clue to know that you needed to establish a presence at a major IX, and actually doing it, because it looks like you're adding peers. But your actual network presence is minuscule.
How do you intend to compete with the six NYC based companies I can think of off the top of my head that are putting fiber fed, $9,000 to $20,000 5 to 10Gbps 71-86 GHz PTP links on rooftops to build their own backbones, when you're playing around with 1000BaseT to the roof and AF24s?
You have high uptime? Do you have any of the following, because some of your much larger competitors sure do:
sites with parallel A and B side power systems
-48VDC rectifier + battery systems sized for 24 hour runtime at load
generator resupply contracts
chassis-based routers with hotswap fan, N+1 power supplies, dual redundant routing engines
pair of identical core routers
singlemode fiber to the roof
-48vdc power to the roof
ironclad rooftop lease agreements with building owners, drawn up by professional telecom/real estate lawyers that run for 5+ year terms
a 24x7x365 NOC staffed by live humans
This follow up comes across as though you've drawn a conclusion and are now arguing towards that predetermined conclusion while ignoring the additional information provided. Plus your argument has kind of devolved from talking about specific concerns to throwing criticisms at the wall to see what sticks.
You don't seem to be arguing in good faith.
FOSS zealots like to point to the success of Linux over Windows (which is restricted to the server market, I might add), but there's little evidence that the FOSS philosophy is effective when it comes to physical infrastructure. Software requires bytes and labor. Bytes are cheap and labor can be donated or paid for by companies. Hardware requires fabs and factories, which are expensive.
So I guess the points you brought up don't matter.
We compete on price (donation only)
What is your problem??
What ISP are you affiliated with?
So then you must intimately understand the depth of the lies and anti-consumer business practices that spur such networks.
> Real five nines network infrastructure takes significant capital investment.
5-9s is 5 minutes of downtime a year. I've never had a commercial, let alone residential ISP hit that mark.
> Actual, reliable internet costs real money, both in the salaries of people to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, equipment, salaries and expenses for field technicians to build it. ...
Going back to my first point, almost no one is unwilling to pay for their internet connection.
They're unwilling to pay less to get TV service they have no use for and internet than they are internet alone.
They're unwilling to look at the cost of network access in other countries and be happy when comparing that to the cost of internet in a major, densely populated area in the US.
They're unwilling to accept transfer caps and accept the lies they they help the network by making people share when they know that transfer is irrelevant and bandwidth is the limited commodity.
They're unwilling to accept lies of bandwidth speed that isn't attainable even under the best of conditions.
They're unwilling to accept shady and underhanded business practices such as random, "accidental" charges.
> It just needs to be channeled the right way so that they can figure out what it actually costs to run a reliable ISP. I'm all in favor of new startup ISPs.
However, as an "actual ISP network engineer", you must understand the (very arguably unfair) regulatory issues that the big players have put in place to make this impossible to do. Google failed at starting an ISP, and it's not like they are tight on either cash or lawyers.
As an enterprise customer, I only get 99% commitment, and frequently get credits. They’ve improved it in the last 5 years, but our whole region had an outage when the single fiber connecting to their core routers was cut by a house fire.
Starting a small ISP based out of extreme dissatisfaction with huge, evil incumbents is something that I encourage any plucky upstart to do. If you don't like Comcast, Charter, RCN, etc, if you have sufficient clue, go do it yourself. Just don't try to do it with $20,000 and some good wishes, because something with much deeper pockets like Webpass will come along and obliterate you.
The point that I am rather bluntly trying to get across to these folks is that if they're going to invest significant amounts of their personal time/engineering resources into it (unpaid labor?), that they should treat it like a serious business and not a nonprofit. Raise sufficient funds, somewhere in the $500,000+ range to start, to build a network that actually does meet five nines through sufficient diversity in topology and routing.
Compared to random YC companies that seem to raise $1m to $5m to do some vaguely defined web2.0 SAAS thing, the amount of money that a group of people need to raise to build the core of a serious ISP is not huge. Elsewhere in this thread I threw out a figure like $750,000 in beginning CapEx if they're serious. For some good reasons, however, it's easier to get $5m of series A funding from a VC for a pure software development business than a wholly-owned-facilities-based startup ISP.
Even if you completely ignore things like colocation costs, IP transit costs, IX/peering costs, office rental, office utilities and overhead, running a serious ISP with live, clued in humans watching the network takes a lot of money per year for salaries.
3 to 5 NOC persons for 24x7x365 NOC phone coverage. At a certain point if you peer with the big boy ASNs they will expect a live human to answer the phone at 3am when something unusual happens in traffic between your mutual networks. That live human needs to be somebody who has the equivalent of 'enable' on your peering router and sufficient clue to wield it.
2 network engineering/architecture positions, multiply by expected fully loaded yearly salary and benefits package for each person.
at least 2 field technicians
vehicles, vehicle insurance, tools for technicians
At NYC salaries? In a greenfield scenario, I could easily spend $900,000 a year in just payroll and benefits, just to begin to attempt to duplicate what some other serious ISPs already have in place for manpower.
And again that is before you spend one dollar on equipment.
I kind of don't understand what you're getting at here. They're a nonprofit. The majority of their work is coming from volunteers. They aren't paying New York salaries.
This is like going to Linus and saying "You can't possibly expect Linux to be sustainable - do you have any idea how much money Microsoft pays its developers?"
Nonprofit/open models aren't less work than commercial ventures; the difference is that people and companies donate their time/money because they want the project to succeed. If they end up with a competitive enough product, then businesses will start sponsoring more development, because it'll be a cheap alternative to dealing with Comcast.
At that point, large businesses operating in the city will pay/sponsor New York salaries to maintain the parts of the project they find useful. Exactly like every other Open Source or community run project on the planet.
I have plenty of doubts about volunteer mesh networks, but I'm not so skeptical to say they literally can't work. I think you're looking at one particular model of the market and assuming that anything deviating from that model is broken, rather than just targeting different segments or trying a different strategy or even just accomplishing different goals.
> the difference is that people and companies donate their time/money because they want the project to succeed.
And that volunteering time and energy are probably valuable then, especially if they can do the work of a real ISP.
I think one real question to ask though is what is actually more efficient—donating volunteer time and energy to a nonprofit, or donating money and having that nonprofit hire professionals instead?
GP doesn't sound like he is saying a volunteer mesh network literally cannot work, just pointing out that a network is inherently expensive to run, whether it is 'fully accounted for' on a payroll, or merely hidden in the cost of 'free' volunteering effort. It's an open question of whether asking for people's time or their money is better in the long run. Capitalism tends to suppose the latter.
That's a good question. Judging from software development (which may or may not map perfectly to hardware infrastructure) both seem to work pretty well. Depending on who you ask you'll get different answers about whether closed (purely paid) or open (either volunteer or mixed) is "winning".
But certainly they seem to both work, on the whole. Neither strategy is so radically inefficient that it's safe to dismiss a company that relies on one over the other.
One response/workaround: Guerrilla networking.
Why, in part? Because there's nothing much to target, financially. In some ways, legally. Instead, the incumbents have to elicit suppression through regulators and law enforcement, if they can.
Google has deep pockets. But it could spend years trying to get access to poles. Incumbents don't just pay to oppose; they pay to slow things down as much as possible. Good luck deploying, when you have to wait 6 months for a court case, that's then going to get appealed.
Stay under the radar. That's a whole different approach.
And communities are getting desperate enough to try anything that might actually work.
While I cheer them on, I simultaneously hope the press coverage doesn't bring the corporate mercenaries down on their heads.
A non-profit organization can be as "serious" as any for-profit business. The only difference is the non-profit might, for example, be required by their charter to re-invest income back into the business instead of paying it out as profit to the owners/shareholders/etc.
The level of "seriousness" of an organization should be determined by their business plan (is it profitable? And sustainable?), not their profit motive or financial footprint.
> if they're going to invest significant amounts of their personal time/engineering resources
First, it's their time and resources to spend as they wish.
Additionally, a suggestion I've heard is that pursuing anything that requires a serious investment of your time and resources (e.g. a job/internship, school, non-paid opportunities for "exposure") is worth pursuing if and only if it at lest two of these are true: [pays well/profitable, will improve future opportunities, you really enjoy it]. (If you love doing something and you're learning a lot or gaining experience that looks great on a CV, it can be worth a serious pay cut)
> build a network that actually does meet five nines through sufficient diversity in topology and routing
That is a good goal, but it isn't strictly necessary at first. Limit growth to what is possible at the time, and grow into a larger organization if it's successful.
> Ambitious – and expensive – high-speed internet program will stop plans to expand to new cities as business reportedly under pressure to cut costs
Although it looks like they've started expanding again as per https://www.techrepublic.com/article/the-rise-and-fall-and-r.... So perhaps I counted them out too early. However, even with the backing of Google their success isn't guaranteed and they've had many non-technical, not-purely-financial problems along the way.
I still think it's a good example of why a "plucky upstart [with VC funding]" isn't able to waltz in and build a large, traditional ISP in the current environment.
Yea, google paused expansion for ~3 months but building a nationwide network is a ~100 billion dollar investment. The only option to get there quickly is a LEO satellite constellation, otherwise it's going to be a really long slog with lot's of local governments getting involved.
~Kind of sounds like it failed.~ Apparently Google Fiber is starting to expand again. I still think it serves as a good case against why the current climate makes it near-impossible for a "plucky upstart [with vc funding]" to start a large, traditional ISP.
> Starting a small ISP based out of extreme dissatisfaction with huge, evil incumbents is something that I encourage any plucky upstart to do. ... Just don't try to do it with $20,000 and some good wishes, because something with much deeper pockets like Webpass will come along and obliterate you.
You can't be a plucky upstart while simultaneously having pockets deep enough to fend against the current major ISP players.
I'm not going to sit here and reply line-by-line to your post. The issue isn't that people want a free network. The issue is that people are fed up with the large ISPs and that the large ISPs have made it all-but-impossible to start an ISP.
You seem to think that VC-sized money is the only thing people need to enter the market. You're entirely wrong. Entrenched players have made it next-to-impossible to compete in terms of capital and regulatory hurdles.
They just realized it makes no sense for them to be in a low ROI industry: https://www.mckinsey.com/~/media/McKinsey/Business%20Functio...
that's the thing, they haven't. Not in the service territory this new nonprofit is operating in. There's something like 14 different ISPs which overlap service area with them in their part of New York City, none of which are big nationwide Comcast or Centurylink sized entities.
What they're trying to do on a shoestring is on super hard mode, they've picked the single most competitive market in the entire USA to try to compete with independent, non-LEC, non-major-telco companies that do have a few million dollars to throw at their own facilities-based network infrastructure.
I'm also just not sure why you're acting like this. You're being simply dismissive of the project. There are many successful meshnets.
You've yet to give a specific complaint about this project and have simply ranted about how they're just small, underfunded, and not a real ISP without ever actually substantiating any of that or specifically what they're doing wrong.
Do they need to run their own fiber - is there a way they could rent fiber space from an existing line? At least for a little while?
Don't be worried that a new player has entered the space; be worried that the existing ones haven't improved the space. The total market space can be increased by creating new connections, and someone has to do it.
Get large enough, and you can start carrying other traffic too, and eventually start buying/laying fiber. But you know this, it's probably part of your business plan.
And that's true even assuming just bare costs without markups. Obviously when you take into account how much profit & graft is attached to every single level of that stack, it really starts to add up. Throw in the absolutely, inexcusably bad, awful experience nearly everyone has with the big telecom companies and their employees, and that's enough ill will to propel many an alternative DIY network.
Metaphorically speaking, you just told a bicycle owner what a terrible car he has. Or rather, warned him how much the car is going to cost that he doesn't want in the first place. Is it possible the bicyclist can actually tell the difference between a bike and a car? Maybe he even sees some advantage you don't? (As opposed to the implausible but apparently favored hypothesis by car-owners that no, he just likes punishing himself while unsuccessfully pretending the bike is a car. When we don't know a stranger's rationale we glibly tend to assume there is none, and they're just cray-cray.)
To make yet another metaphor, when all you know is hammers, the whole world looks like it's full of 1) nails, 2) hammers, and 3) things that suck at being hammers.
A phone network that objectively sucked compared to AT&T and Verizon but was boatloads cheaper seems to have worked out just fine for T-Mobile. I can see the same thing working for this. $20/mo for an internet connection that works most of the time would have plenty of customers. I'm not familiar with the wireless tech involved so no clue how well this scales outside of high density cities but within some place like NYC there's plenty of people that would pay for this over $50+/mo (min!) to the local cable companies.
>"Actual ISP network engineer here: I'm sure these people are having fun and all, but it's insignificant in the larger scheme of things"
No it is actually very significant. In a past life I was also an "Actual ISP network engineer" at a large ISP. And what I can say is that large ISPs aren't exactly wellsprings of innovation. However these folks are hacking on things, solving an actual problem and seem to be having fun in the process. It's exactly this ethos that usually ends up moving the needle. Sometimes by accident. It's curious that you are so keen to poo-pooh them.
>"Real five nines network infrastructure takes significant capital investment"
There is no ISP that provides customers 5 9's of uptime. Not even for business customers. With a Time Warner or Comcast you are lucky if you get 3 9's from their network. This is what you get from "the duopoly."
>"And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each (do you really want to deploy something in the year 2018 to take several full BGP tables that doesn't have a 4 million FIB capacity?), etc."
There a clue in the article about which IXP they're - 32 Ave of the Americas(DE-CIX.) It's one of the 3 of the most "wired" buildings in NYC. You can peer with everyone there. These people seem to have done their homework. And you don't need to buy $15K Cisco ASR any more. You can take full tables on commodity hardware running Linux and ExaBGP or BIRD. There are many CDNs and IXP's doing just that.
I imagine consumers would be fine with 95% uptime of a gigabit connection compared to 99,99999% uptime of 16mbit.
I can guarantee that nobody would be okay with 95% uptime. That means that your Internet access is down for over an hour every day. Given that people go berserk when their Internet is down for even a few minutes, 95% uptime will be unacceptable to consumers.
(Actually, if I were an evil ISP lawyer/marketer, I'd try to claim a "Maintenance window of 1hr downtime sometime between 4am and 3am", then ignore any downtime of less than an hour during the day and if you complained about not getting yur expensive five nines I'd laugh and say "published schedule maintenance not included!"...)
"Real five nines network infrastructure takes significant capital investment."
No, it doesn't. Overly complex, corporatized infrastructures designed by committees in meetings - that takes significant capital investment. Simple infrastructure designed by hackers can achieve bullshit metrics like "five nines" without breaking the bank.
"And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each"
Establishing colo at a "major IX point" is a few emails and phonecalls away. Your salesperson will fall all over him or herself and waive the NRCs to get you in there. It doesn't cost much, even for a full rack with 20 or 30 amps. If you're just doing networking, you won't even need that. Then you get a fiber cross connect to he.net for 10 or 20 or 40 gbps and you're in business.
You can buy very, very nice used routers and since you're a hacker and not Mr. Enterprisey IT guy, you can run it and support it yourself.
We had many iterations of this exact critique in our early days - explaining how we couldn't possibly deploy this volume of storage without fancy bullshit enterprise solutions from EMC or Compaq and how we couldn't possibly run a global storage network without fancy routers (we currently own zero routers) and fantastically complex clustering and failover ... and 17 years later we're still making it all work just fine.
Fuck those people. Fuck those school trained, passionless, non-hacker, no-talent-pussies.
Good luck with your mesh network!
> In the mid-90s, I got a call from some friends at ATT, asking me to help them research the nascent web-hosting business. They thought ATT’s famous “five 9’s” reliability (services that work 99.999% of the time) would be valuable, but they couldn’t figure out how $20 a month, then the going rate, could cover the costs for good web hosting, much less leave a profit.
> The ATT guys had correctly understood that the income from $20-a-month customers wouldn’t pay for good web hosting. What they hadn’t understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn’t very good.
> For a century, ATT’s culture had prized—insisted on—quality of service; they ran their own power grid to keep the dial-tone humming during blackouts. ATT, like most organizations, could not be good at the thing it was good at and good at the opposite thing at the same time. The web hosting business, because it followed the “Simplicity first, quality later” model, didn’t just present a new market, it required new cultural imperatives.
Oh, and they'd need peering: more $$$$$
I don't think this is a helpful reply.
Since it sounds like you're associated with the project, can you address some of the specific concerns mentioned, like reliability? Do you disagree that hundreds of thousands of dollars are required for five nines connectivity? Do you disagree that that level of reliability is required for your customers etc.
Do you know what the cost per user is currently?
What frequency do the point-to-point links run at? Have you considered using some sort of lower frequency packet radio for longer distance links, perhaps to other nearby nodes?
DE-CIX, our IXP, donated bandwidth to us, and we have transit also donated from Packet Host and WebAir. We actually pay nothing for bandwidth. Probably one day we will pay but it is not that expensive if you do this at an IXP and use peering.
We have an AirFiber pair that is 24Ghz, but all of our sectors are 5Ghz wifi.
What is the incentive for them to donate bandwidth/transit to you? Your non-profit status? Do they get something else in return?
DE-CIX is the biggest IXP and they measure their network in terabits/sec. It is not a big deal for them to donate us a 1 gig connection. (it's a big deal for us!) Eventually we may upgrade it to 10 gig.
How often do you see full saturation on that link? I'm guessing if all 600 of your users all tried to stream one stream of $TV-show-de-jour at once they'd barely get 16Mbps each? Can one user's home office full of Apple gear flood the bandwidth downloading a dozen Apple software updates simultaneously?
Cable companies consider these high ratios to be quite embarrassing actually, as it shows how crummy their networks are: https://www.dslreports.com/forum/r31258251-Speed-Issues-Char...
We've never come anywhere near saturating our gig connection. We monitor it. Each building is limited to their own rooftop connection which is an average 100Mbps, so they can't do more than that if they tried. The thing is not everyone is downloading a file at the same time, and streaming uses much less bandwidth than downloading. From Netflix site-
0.5 Megabits per second - Required broadband connection speed
1.5 Megabits per second - Recommended broadband connection speed
3.0 Megabits per second - Recommended for SD quality
5.0 Megabits per second - Recommended for HD quality
25 Megabits per second - Recommended for Ultra HD quality
I have to assume another cost that's not noted is roof rights. That's not free - most people either have to pay or give the property management a kickback.
NYC is an interesting place for this - if you live in Manhattan, you clearly can afford the $80 for 1Gb/s FiOS or $40 for 100Mb/s FiOS or whatever Spectrum is charging. It would be way more interesting to plop this in a rust-belt city where people are on a paycheck-to-paycheck salary...
Not a single one of them has ever signed up for anything more expensive than three nines. Only a few have even discussed the differences between 3 and 4 nines solutions.
I'm reasonably sure I'd choose the same for my home internet connection - if offered representative pricing based on costs of providing 3, 4, or 5 nines, or possibly even 2 nines - I'd choose the least expensive because losing a few minutes or even occasionally 15mins a day of connectivity at home (or maybe more realistically an ~8 hour outage per month) really isn't going to bug me greatly - not if it's two or more orders of magnitude cheaper than a four of five nines connection.
Similar projects exist in Austria (FunkFeuer) and Switzerland (Openwireless).
- Gluon is the most popular: https://gluon.readthedocs.io
- Libremesh is more flexible: https://libremesh.org/
Like NYC Mesh both are based on OpenWRT.org a wonderful distribtion for wireless devices from 4mb flash / 32mb memory up to 64bit x86 support.
God forbids somebody uses the word "anarchist"...
Have to now do something with it.
I'm all for this kind of tech solutions, but for crying out loud pretending that this is a replacement for ISPs in NYC really discredits techies.
I had at&t in one of SF's typically old buildings and I can tell you, the internet there barely worked - i had to reset the router about once a week, plus hour long downtimes not uncommon. Even now, when I'm in a brand new 3 year new townhouse, I still have a reset the router once or twice a year and I get half the speed I'm paying for on a DSL broadband plan, not to mention that I get stuck in a 12 month contract anytime i want to switch plans.
Considering that there are currently no other options than the att/comcast douopoly, I would think people would give a little more credit for providing alternatives, even if it's not perfect yet.
The reason why I'm negative about it is that it gives people incorrect perception. The perception is that my neighbor does not need to pay Optimum or Time Warner or Verizon for internet and he or she can watch Netflix. And that's simply not the case because as soon as five of my neighbors get this WiFi chokes because in a regular Brooklyn block it is going to add another 40-50 wifi access points. Negative perception from folks who own their houses is a death to community ISPs.
This kind of wireless is not a solution. A solution is to wire as much as possible, including wiring between the buildings of my neighbors who own their houses and if needed terminating a single high bandwidth wireless or fiber link into a node that carries entire block.
P.S. I reached out to these guys about a year ago offering to help them when I read another piece written about them. Maybe it even was on HN. I did not get any response.
Does VoIP work?
What if I want to run a server at home?
Actually scratch that, I tried getting a 1 gig cross manhattan link. Cost more than 1g to London.
Unfortunately, the building owner where I used to live wasn't interested -- it turned out most of the building was empty, people keeping apartments for occasional visits or as an investment :-(
https://www.hyperoptic.com/how-hyperoptic-works/ or https://www.hyperoptic.com/map/
And the sewers a) aren't human-travelable like the old ones in Paris or parts of London; and b) overflow in heavy rains. You probably don't want to run electrical wiring through them.
But in general, go to a decent building and order Cogent. That gets your building on the net. Cogent eats fiber extension charges left and right. Plus you actually get 1Gbit/sec for $500/mo. Now you take that 1Gbit/sec and wire to neighboring buildings by running ethernet or standard yellow fiber. That's how ghetto ISPs get stuff done. It was done for years because it works. There's incentive for those who own building to form a community ISP or provide an access to a community ISP because those that own builds want the same product and they know they can share some of the costs.
However GTT offered me 1gig from New York office to London Docklands for $2400! Under $2k for p2p to office in Washington DC.
If local internet is $500pcm , get two lines into the building for $1k/month, charge 50 people $30 a month and give them 1 gig and 25:1 contention ratio, plus the cost of dropping the fibre and a router, leaving $6k a year. It's not going to give you a full time job, but will give you some decent internet and a bit of pocket money to look after it.
Order an SDI video circuit from VZ to one of the interconnect buildings. That's $1.5K/mo but it is delivered as a point to point single mode fiber with VZ gear being plugged in. After they install it and leave unplug VZ gear. Plug in your 10G. You just got yourself a 10G transport for $1.5K/mo.
It's truly superior to centralized control by ISPs/Governments and the world will be grateful for this after this whole net neutrality debacle recently.
You guys are brilliant and doing an amazing job and I'm grateful people like you guys exist!
Thank you for the response!
"NYC Mesh will comply with all federal laws in the countries it operates, however, as policy, no data is collected and therefore no data exists to provide requestors."
This is cute, but if you aren't hunting down people abusing the mesh, the whole connection to the internet will go away for a ToS violation.
Over this same period, I have given out a customer's identifying information zero times (if more than 24 hours have passed, there is a very small chance that I am able to identify the customer). In every instance, it was "sorry, I am not able to identify the customer".
Zero is also the number of times I have heard anything from any of my upstreams regarding any ToS violations or similar.
Hmm ??? Could u please explain.
I always assumed that all ISPs will store all DHCP (PPPoE? PPTP? IANANE) allocations by IP & time.
That's just not how IXes and peering agreements work.
I mean, this isn't true. I'm not even sure to begin. Perhaps specific points instead of nebulous nonsense would help us better understand you.
Running fiber is expensive but less complicated and less error prone. And unless I missed something, there aren't any initiatives by big companies (that obviously don't want extra expense and headache) to go mesh. Big ISPs use their customer nodes as wifi hotspots but that is obviously not mesh.
I think you have an overly narrow view of what meshnets can be.