
Visible signs left after Google Fiber abandons Louisville - gumby
https://gizmodo.com/when-google-fiber-abandons-your-city-as-a-failed-experi-1833244198
======
yingw787
When I interned at AT&T a few years ago, I asked a manager I respected whether
they were worried about Google Fiber coming in and eating their lunch.

"No", he said. "We'll sue them for every mile of fiber they lay down".

I can understand why they would make that play. It's kind of the only play
they have. AT&T and Verizon innovate enough to remain...okay...but they're not
exactly Google. The only way they would win is to get dirty and turn fiber
laying into Google's very own Stalingrad. And between starting a new, massive,
low-margin (by advertising's standards), and capital-intensive business, and
keeping salaries high and employees happy, Google chose the latter and bowed
out because honestly, dying on this hill isn't a smart move.

I doubt Google wants to do this. Pulling out at this stage, when Fiber hadn't
actually provided the "nudge" to internet speeds and throughput that would
make Google and Internet companies more powerful, would not place Google at a
place of negotiating strength. It makes Google look bad and cements the ISP
monopolies' notion that Google pulled out because they physically can't bear
the pain and so they remain safe, not because Google _chose_ to, can do this
whenever they want, and have the ISPs at their mercy. It probably also burned
a few bridges, if there were bridges at all, and makes weighing favors in a
(hopefully not long-term) post-net-neutrality world swing towards the other
major power players like Amazon or Facebook.

~~~
yingw787
I do think the significance of this is cities realizing there is no Disney
prince riding in on a white horse to save them, and taking their own futures
into their own hands. I'm hoping there's enough "anchor cities/regions", like
Nashville/RTP/Kansas City, that have contingency plans and referendums ready
for an infrastructure push at the local level if Fiber pulls out there, to
prevent all white-collar talent from drifting towards the coasts and leave a
complete vacuum everywhere else. It'll take a lot of organization, money,
guile, and community trust to push through ISP brainwashing, but the cities
that succeed will hopefully become the few "shining cities on a hill" that
could save an entire state from economic irrelevance. Hopefully, when the
pendulum swings the other way on urban centralization, they'll emerge as good
alternatives to raise children, work remotely, and live a good life.

~~~
sesutton
I think people here in Raleigh would hardly notice if Google Fiber pulled out.
They only ever offered it in a very small area and AT&T has been offering
gigabit to to large parts of the area for several years now.

~~~
dwighttk
Yeah, in Durham, I've seen a lot of AT&T guys running fiber... I've only
gotten a Google T-Shirt and seen an apparently empty office space downtown
with a Google Fiber sign in the window.

------
ilovecaching
Let’s not forget that the real issue here is that American ISPs are a complete
monopoly by Comcast and AT&T, companies made of pure greed. Small end sub ISPs
have to claw for every strand of dark fiber. Google really shouldn’t be the
answer though. We need to demand that politicians address the issue in the
upcoming election before they start looking at the Silicon Valley giants.

~~~
colechristensen
The best ISP in the country is in Minneapolis. $70/mo for symmetric 1 Gbps

[https://fiber.usinternet.com/](https://fiber.usinternet.com/)

They lay their own fiber terminating at your home and have been steadily
covering the metro area.

~~~
scruple
Does this scale to other metro areas, if other initiatives could be funded and
started? If so, why isn't it happening in those other metros? If not, what
makes the Twin Cities special?

~~~
colechristensen
It went relatively slowly and started in sections of the city with the right
customer demographics. The local government which is a big part of the
equation seems to be generally a little more reasonable and less corrupt than
is average.

I am speculating it is capital intensive and not enormously profitable, and it
is slow.

~~~
rayiner
> It went relatively slowly and started in sections of the city with the right
> customer demographics. The local government which is a big part of the
> equation seems to be generally a little more reasonable and less corrupt
> than is average.

That is also how Stockholm built out its fiber network. And, by the way, it is
illegal in almost every city, because it would result in wealthier
neighborhoods getting fiber first. That is the reason, for example, why
Baltimore doesn’t have FiOS whole almost the whole rest of Maryland does.

~~~
njepa
> That is also how Stockholm built out its fiber network.

Stokab also had various mandates to connect or cooperate with schools,
government, student housing, public(ly owned) housing and other entities. You
would often get fiber in the equivalent "the projects" before someone in an
affluent co-op. Without this I am not sure how well the approach would have
worked.

~~~
rayiner
Connecting a few government-owned buildings is a far cry from requirements in
the US, where typically every area above a very low population threshold must
be connected.

~~~
njepa
I wasn't a few government-owned building so much as 25% of the housing stock,
around a hundred thousand apartments, which were actually owned by the city at
the time. (And that is just the publicly owned housing). As far as I know it
wasn't subsidized as such, but paid for by profit from rents and rent
increases (these are essentially rent controlled apartments).

------
jly
I'm a happy Google Fiber customer (Austin), and my neighborhood was deep-
trenched. The results have been excellent, with near-zero reliability issues
and a clean look on the street.

Unfortunately, not everyone is having the same experience. The perception here
is that they are operating a shell of what the previous ambitions were.
Service has been active in my area for close to 3 years now, but I have
neighbors who have been on waitlists for months or years to get their house on
the network, still being told to call back every few months.

It seems like any houses that missed the initial sign-up (now many years ago)
are a crapshoot to ever receive service in a timely manner. Some streets that
didn't receive enough sign-ups were never trenched, while adjacent streets
were. Sometimes fiber is installed all the way to the house with months-long
delays connecting to the network (splicing issues, conduit clog, etc). I've
loved my service and it's been rock-solid, but there are certainly a lot of
obstacles compared with traditional telecom providers.

~~~
EpicEng
Same here in the burbs. Would love me some fiber, but it's never going to
happen.

------
robocat
At many traffic light intersections, you can see where the trenches are used
for inductive loop car sensors, with wire in them.

[https://www.traffic-signal-
design.com/vehicles_detected.htm](https://www.traffic-signal-
design.com/vehicles_detected.htm)

The trench is narrow, cut with a diamond saw, and the filler (tar?) doesn't
come out.

The example photo of a failure in the article, showed a relatively wide
trench, presumably width required for the microduct for blown/jetted fiber.

So maybe they used a trench that was too wide, and a filler that was
unsuitable.

I have also seen microtrenchs used in New Zealand across private driveways
(low traffic use).

I wonder how good the trials were of the filler? I sometimes see paint tests
on the road (multiple lines of different product and different thicknesses,
being tested by heavy, real traffic use).

------
esmi
I wish they provided some details on why the San Antonio and Nashville small
trials didn’t scale to Louisville. If the issues were really so glaring and
obvious it seems those efforts should have revealed them.

Edit: Looks like it didn’t actually work in those cities either. One wonders
why the Louisville representatives missed this.

[https://www.wsmv.com/news/repaving-streets-reveal-google-
fib...](https://www.wsmv.com/news/repaving-streets-reveal-google-fiber-buried-
too-close-to-surface/article_9af6a069-1466-582a-90d2-514b4eeef7ea.html)

~~~
KennyFromIT
Louisville's representatives likely missed that because they were wearing
rose-colored glasses. They were thinking "big picture" about would a
successful project could mean for the city and were ignoring the details. They
were probably (rightly?) assuming that an organization like Google would be
able to learn from their mistakes and "this time would be different". Really,
it's hard to fault them for thinking this way, but just like every other
situation in business, if it seems too good to be true, it probably is.

~~~
dwighttk
hoping to get something for their $400K investment.

------
WhuzzupDomal
“The company also wanted to cover the trenches with epoxy, rather than the
typical asphalt mix.”

Along with the rest, this outlines a basic problem that modern Google and
other Silicon Valley companies have with technologies. The Google people have
massive talent, capabilities, and funding. Yet the use this almost as a
handicap nowadays. Because they have talent doesn’t mean they have a monopoly
of it.

We have extensive testing of every conceivable paving and repair mathod done
over 100+ years. Engineering colleges around the U.S. (and the world of
course) have a massive legacy of tests. To go in and just guess your new idea
is going to work for a new project just isn’t necessary in such basic
technologies.

~~~
setquk
Indeed.

Epoxy even sounds like a stupid idea. I mean how are you going to repair that
next time. It’s impossible to dig that stuff up.

~~~
kkarakk
it's only a couple of inches deep, it sounds like "if there's a problem we'll
tear it out and replace it" kinda deal

------
gz5
Would it be surprising to see Google pivot to 5G for the tech solution? I
believe their business goal is still to control the access. Anyone see a
realistic unit cost analyis for 5G in urban environments at maturity?

On the fiber side, I think cities need to build the infrastructure (and are
often in a position to finance via low interest rate bonds etc), and then
enable _any party_ to lease/operate/compete on that infrastructure.

~~~
eeeeeeeeeeeee
I have WebPass (webpass.net) in Denver, which is a company that Google
acquired. It's mainly for residential multi-unit apartment buildings, though.
I've had it for several years and it works extremely well. It uses point-to-
point radios from their main node in the city to the top of your building,
then you get an ethernet hookup in your unit.

I almost always get in the ~700-800Mbps up and down range on ethernet
connections, regardless of time of day. Never had any issues during snow,
rain, fog, etc.

My guess is that Google realized that fiber-to-the-home is the wrong strategy,
and too costly, just as Verizon FIOS halted their rollout years before. And I
believe even ATT only does fiber to a neighborhood and then coax to the home,
right? It's sad to see, but the cost is astronomical and it's a lot harder to
get cities/municipalities to let you dig things up.

Especially for America with it having so much sprawl, a wireless solution will
probably work better and the rollout will be quicker.

~~~
ocdtrekkie
I know "fiber to the neighborhood and coax to the home" is how both Comcast
and WOW operate. Note that coax can provide some fairly decent performance,
with DOCSIS 3.1, you can do gigabit down a coax cable, though the networks are
still built asymmetrically, so you're only going to see 50 Mbps back up on a
connection like that.

~~~
eeeeeeeeeeeee
Agreed. DOCSIS 3.1 is nice. I think most of the bad news around coax is
because a lot of cable operators overload their nodes.

~~~
ocdtrekkie
Indeed, and the lack of upload is a design choice, not a physical limitation
of the medium, as far as I know. Coax is fairly durable, is less challenging
to install and much easier to repair, and as long as they can push the speed
over it, I really see no reason to run fiber into the home.

------
aksss
As the article says, the problem set is multifaceted. Nanotrenching doesn't
work, and laying infrastructure in any case is super expensive. In a green
field implementation in your privately owned parking lot, it would be an
expensive undertaking in terms of material and time & motion, tooling. Now add
in permitting requirements from the city to do the exact same mechanical
exercise, plus coordination to shut city streets temporarily, enviro-impact
studies, etc, etc, etc. Then add project managers, regulatory experts &
lawyers to deal with the city and navigate it all. The costs are huge. It's
annoying to me that some people gloss over the role of civil administration as
a market barrier, but by its nature Telecom as an industry has high capital
and expense requirements from installation to O&M. A lot of folks don't fully
appreciate this until they work in the industry or like Google, try to
revolutionize it.

------
paxys
The failure of Google Fiber paints a very bleak picture of the future of
internet infrastructure, or even all communication in this country. If
_Google_ cannot compete with the status quo, who can?

~~~
vb6lives
They tried to lay fiber in 2in trenches instead of typical 6in trenches. Then
left when it didn't work. This isn't Goliath beating David. This is David
throwing a rock, missing, and going home.

~~~
AareyBaba
Is there any technical analysis or reports available online on why the
trenches didn't work ? I've always wondered if we could use our network of
roads to bring internet to the doorstep by laying lines just below the road
surface.

~~~
drenvuk
You don't need much technical analysis here. Shallow trenches break more
easily, leading to higher maintenance costs. 2 inches is nothing.

------
dgacmu
This was announced a month ago:
[https://fiber.googleblog.com/2019/02/louisville_7.html?m=1](https://fiber.googleblog.com/2019/02/louisville_7.html?m=1)

And discussed here at the time:
[https://news.ycombinator.com/item?id=19106998](https://news.ycombinator.com/item?id=19106998)
(207 comments)

~~~
dang
Good catch. But the current article has new information. I've attempted to
give it a more specific title.

If anyone suggests a better one (i.e. more accurate and neutral), preferably
using language from the article itself, we can change it again.

------
neom
Grace Simrall the CTO over there is no n00b. They're a customer of ours and
they're probably one of the most advanced cities we work with in terms of
thinking about a "city stack". Google Fiber didn't abandon them, as far as I
can tell they just have better plans that don't include being strong armed by
google.

~~~
duxup
>that don't include being strong armed by google.

What does that mean exactly?

I would think getting more competition would be welcome.

~~~
peteradio
Perhaps their terms were outrageous. That's not the kind of competition you'd
want.

------
sodosopa
How bad were their roads prior to Google? If it's like a lot of other states,
roads get little funding for improvements

Louisville should have looked at the model set by Chattanooga's EPB. Fastest
internet in the country and a highly rated customer service team. Their roads
suck too but it's not the ISP's fault.

------
shmerl
Very irresponsible behavior. If they have commitments, they should have stuck
to them. The way they behaved with Google Fiber is atrocious. Networks can't
be built without long term investments and Google knew well about that when
they started.

~~~
908087
The "Google never actually meant to be an ISP, they just wanted to kickstart
competition" spin I hear pushed on this reminds me of the "they were just
trying to start the conversation" spin when people get caught faking
racist/sexist/homophobic attacks on themselves.

------
lizknope
I live in the Raleigh-Durham, NC area and we heard Google Fiber was coming to
the area around 2015.

A couple of months later Time Warner upgraded my cable modem speed from 15Mb
to 50Mb download for the same monthly price.

Then 2 years ago AT&T came through my neighborhood installing fiber and turned
it on 3 months later.

Google Fiber is concentrated in a few small areas. AT&T Fiber is expanding all
over the area. I know 1 person with Google Fiber and about 15 with AT&T Fiber.

I would prefer Google over AT&T but the AT&T Fiber works and is reliable and
Google is so slow at expanding that nobody cares anymore.

------
ggm
Long time ago, the campus I worked at was offered "the deal of the century"
from Dec to buy FDDI cards for all campus. A really sweet deal, only $5000 per
host..

The moral of the story: beware geeks bearing gifts. Google should have been
put into a contract to only field-test shallow trenches, if they committed to
remediation into deep trenches at their own buck.

"they would have gone somewhere else" is the voice of Amazon city bidding
nonsense: this is public utility infrastructure. They ballsed it up, they
should have been contractually obligated to get it right.

If it was a testbed freebie, different story. You let somebody in to test
something for free? Sure. Public Liability? sure. Disappointed customers?
Sure.

Hang on.. who are they disappointed in? the elected Lousville Officials?

------
endofcapital
"But Google Fiber got something out of its time here. It learned that
nanotrenching—the cost-saving process of burying fiber optic cables just two
inches underground—was a bust. “We currently do not have plans that call for 2
inch trenches, our primary specifications are focused on going deeper,” a
Google Fiber spokesperson said in an email."

It's so weird to just use cities as A/B tests and just disregard all the
people and plans built around a failed case at the drop of a hat. Are they
going to start A/B testing countries against eachother next?

~~~
paxys
Trying new products in small markets as an experiment has been a thing
forever, and is something literally every company does regularly. There is
nothing weird about it. As for the fallout, IMO the city is to blame for it
more than Google, since the company is obligated to do exactly what the
contract entails.

~~~
tzs
Yeah, but when a soda company, say, tries out one new flavor in my town, and
different new flavor in your town, and mine flops and yours succeeds, they
don't just stop selling soda in my town. They start selling the flavor from
your test in my town, too.

It would have risked much less bad publicity if they had done the 2"
experiments in a city they were willing to use 6" trenching in. Do part of the
deployment with 2" and part with 6", and if the 2" part doesn't work redo that
part with 6".

~~~
908087
Soda companies also don't leave deteriorating trenches behind on the roads in
town when their tests fail.

------
greedo
My city negotiated a fiber contract with Allo that was very favorable to the
city. Allo basically provides free internet to municipal buildings for the
right to sell fiber to residential areas. The entire city is almost covered
(300K people), and rates are very affordable ($90/month for 1gig symmetrical).
Allo also sells tv and phone service as well. I've been extremely happy. Allo
provided the CPE at no cost, doesn't require a contract, and has support
people who seem to be close to the metal.

~~~
dwighttk
I thought Google just canceled Allo the other day...

~~~
greedo
Different Allo. www.allocommunications.com

------
rdiddly
Anybody else notice the symmetry of how Google's competition is described as
"entrenched" and how Google's fiber lines were literally, insufficiently
entrenched?

------
Criper1Tookus
I really don’t think Google knows what they really want to achieve with Fiber
- otherwise they’d have figured out the right way to roll it out years ago.
They also bought a separate company, Webpass, that does “fixed wireless”
(wireless to the building using a pretty big antenna, about the size of a
stadium light, building is wired from there), and that has also been rolling
out very slowly.

------
paul7986
Google is a terrible company!

------
spullara
Fiber is probably a dead end and will ultimately be replaced by 5G point-to-
point transmission for the last mile.

~~~
Someone1234
Fiber offers better reliability, lower latency, higher bandwidth (right now),
and it can be reused for even higher bandwidth later (e.g. 1 Gbps fiber can
handle 10 Gbps with new transceivers on the same run).

~~~
eeeeeeeeeeeee
They could still do fiber to some central node, then point-to-point radios for
the last mile. I have WebPass (webpass.net) and it's point-to-point, gigabit,
and latency is awesome, much better than Comcast ever was for me at the same
location, or any location for that matter. My ping to 1.1.1.1 is consistently
around 2-4ms. And yes, this is obviously an anycast address, going to a
datacenter in my city, but still impressive:

    
    
      traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
     1  192.168.1.1 (192.168.1.1)  0.394 ms  0.383 ms  0.470 ms
     2  * * * (*)  2.460 ms  1.754 ms  2.599 ms
     3  10.5.1.9 (10.5.1.9)  1.518 ms  1.424 ms  1.614 ms
     4  206.51.46.41 (206.51.46.41)  1.926 ms  2.240 ms  2.138 ms
     5  one.one.one.one (1.1.1.1)  1.756 ms  1.945 ms  2.015 ms
    

I was skeptical of this over something wired, too, but it's clear that the
technology has greatly improved.

~~~
phonon
And in bad weather...?

~~~
eeeeeeeeeeeee
No issues whatsoever. That was definitely my concern. Snow, fog, rain, large
storms. It just works.

