
Google Chrome experiment crashes browser tabs, impacts companies worldwide - pasharayan
https://www.zdnet.com/article/google-chrome-experiment-crashes-browser-tabs-impacts-companies-worldwide/
======
Slartie
I chuckled at this:

> "Do you see the impact you created for thousands of us without any warning
> or explanation? We are not your test subjects," said an angry sysadmin. "We
> are running professional services for multi million dollar programs. Do you
> understand how many hours of resources were wasted by your 'experiment'?"

Did you, dear sysadmin, pay anything for Google Chrome? No? Are you in any
contractual relation to Google that covers your use of Chrome? No? Well, there
you have it. You are not a customer, hence you aren't really in any position
to demand anything. You basically agreed to take whatever Google shoves down
your throat for free, and if that includes "experiments", then that's what it
is.

If your multi-million dollar programs move so much money around, maybe take
some of that to either invest in the necessary software - including the
browser - so you are a paying customer and may actually demand anything, or
pay some people to be up to date on the intricacies of Google Chrome and test
them under your environment or disable them if undesired. The experiments are
not exactly a secret program and have existed for a while. Firefox has a
similar thing going. The Firefox one can definitely be disabled. The Chrome
one unfortunately not (at least I don't know of a way at the moment, there
might be one, maybe it's also only available for the Enterprise subscription
that I have never personally heard of any business to be using, but for which
money is actually paid, hence there might be leverage in that case to force
Google to offer an opt-out).

Or ultimately you could also just compile yourself a Chromium from scratch and
update it regularly. I've done it, it's not that hard, and that gets you the
ultimate level of control over that nice free piece of software that you
depend your business upon.

~~~
jve
Yeah.

> We are running professional services for multi million dollar programs. Do
> you understand how many hours of resources were wasted by your 'experiment'?
> \-- Angry sysadmin

Well, do they pay Google for testing in their environment?

Chrome is a free-to-use product. Their rollout strategy is good. Not that they
experiment in prod - the flag was in beta for 5 months. And then they turned
that on in prod for %1 users, still no reports.

Well what better could they do?

~~~
vanadium
Making experiments opt-in seems like a good first step, no? As it is, there
wasn’t even a clear opt-OUT.

~~~
jacquesm
Was there any opt-out at all? Even an unclear one would do, then at least
users can point each other to the option.

------
omh
I've been dealing with this bug for the last couple of days and it's been
hugely frustrating.

This was not a new version of Chrome or a software update. None of our
software was updated at all (and we spent a long time checking!). But
apparently Google have an ability to change a setting and globally affect the
behaviour of all Chrome browsers by enabling experimental features.

We carefully manage our software updates and patching so that we can test it
and roll back if it impacts the business. Google had been good at
understanding "enterprise" requirements - disabling automatic updates, setting
policies etc. But this shows that they're really focused on consumers and
business users will always be an afterthought.

~~~
staktrace
As an admin why would you not also run the beta version of the browser in a
limited setting? That would have helped you find out about this issue sooner,
since the experiment was enabled there for a while.

~~~
yabadabadoes
I'm sure plenty of admins do, that doesn't mean you will test with the same
flags chosen in an experiment it only means you will see changes going into
stable permanently enabled a bit early.

My own experience with ChromeOs is that I switch to beta, report bugs via
interfaces provided (by Google) and then the chromium team acts surprised when
the regression hits stable.

If you ignore signals don't ask for them.

Edit- clarity

------
voiper1
>Many didn't know that Chrome engineers could run experiments on their
tightly-controlled Chrome installations, let alone that Google engineers could
just ship changes to everyone's browsers without any prior approval.

Uhm, isn't that the entire auto-update feature, that google ships changes
without you even being aware?

~~~
omh
This change didn't come from a binary update.

We were affected and our version hadn't changed (in fact we weren't quite on
the latest version - we were still testing it). We have updates disabled and
are very much aware of how to manage this.

Google changed a feature flag that was automatically picked up by existing
copies of Chrome and changed their behaviour.

~~~
voiper1
Whoa, that wasn't clear in the article. Thanks for enlightening me!

~~~
dudus
That was crystal clear on the article. Be honest you just read the title.

~~~
voiper1
Right, I put a quote from the end of the article but I just read the title. /s

------
cimnine
If something is critical infrastructure for you, better make sure you have a
backup.

I've seen this in a Swiss government agency that has Firefox installed exactly
for this purpose. Their main browser is the built-in one, but if it can't or
must not be used (e.g. because of a zero-day threat), they can quickly roll
out a change to make Firefox the default browser until the problem/threat is
over.

~~~
lordfoom
>they can quickly roll out a change to make Firefox the default browser until
the problem/threat is over.

...why not just make firefox the default browser full-time?

~~~
lotu
What if Firefox has a problem?

------
transreal
This is the trouble with Google's aggressive auto-update policy. "Everyone is
always on the latest" is mandated because everyone should have the latest
security updates, but it works great except when it doesn't.

Another example is a recent issue where a Chrome update broke WebView based
Android apps and stopped them from being able to make certain types of network
requests. It was fixed in 2 weeks, but that 2 week period was full of unhappy
customers and lost revenue.

I'm hoping the upcoming Chromium based Edge from Microsoft will allow IT
Admins to control when a browser update is rolled out and give them more
control over the update process.

Chrome auto-updates can be disabled for networks behind a firewall by blocking
the update server address, but that's a very crude way, and doesn't allow for
updating a test machine to see how the new version works, or updating to the
latest minus 1 version.

~~~
senderista
This is not about auto-updates. Organizations which had disabled auto-updates
were still affected. In this case Google changed Chrome behavior _without
changing the release version_. That’s absolutely unacceptable.

------
chaz6
With Edge being based on Chromium I expect many corporations will start
switching away from Google Chrome, but it's just swapping one corporate
overlord for another.

~~~
pluma
It's switching two corporate overlords for one, so I think they'll still see
it as a net win.

------
chris_wot
I think we are now seeing that the attitude of Google in the way they treat
Chrome users is seriously worrying. There are now a litany if issues - from
adversely affecting ad blockers, linking Google sync to logons to Gmail and to
now rolling out “experiments” unannounced... surely by now it is time to
seriously consider Firefox?

Of course, Mozilla needs to make sure they never enable extensions like the
way they did for the Mr Robot one many years ago...

------
slenk
So when Firefox did this earlier in the year for the expired cert, everyone
was freaking out that Mozilla had that ability.

Where is all the freaking out at Chrome? My theory is that is was mostly
Chrome users (the majority) complaining last time when they had no personal
experience.

------
asah
There's lots of browsers out there, why are these environments single-sourced
to Chrome? Hope is not a strategy.

~~~
rchaud
Because for years, IT managers have been fending off requests from users to
give them something other than IE6. Since 2008, Google Chrome has been the
"winner" of the desktop browser wars, with IE falling away completely and
Firefox's word of mouth impact plateauing against Google's relentless
marketing of Chrome.

------
Shorel
The problem I see here is that companies are not moving to Firefox en masse.

------
BuckRogers
I’ve seen Chrome in so many corporate environments and never understood why.
Native browsers work fine and are crafted to work reliably in the environment
they’re used in. My guess is some admins simply like Chrome and thus decided
everyone should. I like Firefox but I keep it at home, at work I use Microsoft
Edge.

~~~
IggleSniggle
Perhaps easier to manage Chrome than managing a different browser for each
operating system?

~~~
BuckRogers
Not really, that would be an excuse if someone said that to you. Most
enterprises use Windows Server and manage this through Group Policy, which
supports all browsers for all operating systems. If you're not using Windows
Server it can be more difficult.

------
a3n
Except for the intent, it sounds like an attack.

"The attack is only possible through a carefully crafted browser ..."

------
CJefferson
What's the alternative?

Google thought this feature was polished, finished, and ready for release.
What used to happen was it would be pushed out in an update, and then everyone
on a server would have seen the crash. This "experiment" is just slow
releasing.

------
user740372
Can we stop the "not paying, no demanding" BS? First it's unfair to silence
people just because they are not paying. Second, we are always paying. If not
with money with our privacy.

~~~
m45t3r
The problem is the tone of voice. One thing is you asking, humbling, why did
this happen, if it can be reverted since I had a problem in my specific setup
and how I (as the non paying customer) can disable it. The other is to be
arrogant saying that I am a multi zillion company, you're idiotic for enabling
this feature and I demand you to disable it ASAP and never do this again.

The quote from this article is clearly from the later case, even if I am sure
that other people reported it more humbly.

------
sjustinas
While Google might have fucked up, what could possibly be your workflow where
a single crashed tab costs you one and a half day's of work?

~~~
micael_dias
Well it wasn't just one tab. At my company all work is done on the browser
(Web app) and the browser would just crash and there was no indication what
could be the problem. So although it didn't cost the entirety of one a half
days work per user it severely diminished productivity all around.

------
Animats
Enterprise environments don't click on ads, so they don't matter to Google.

Enterprise IT management should realize this.

~~~
andybak
> Enterprise environments don't click on ads,

Not sure where you've worked!

~~~
Animats
The article seemed to be about an enterprise where people in some job used the
browser mostly to access in-house systems. That generates no revenue for
Google.

------
hieloz
I came arcoss the loading circle tab suspending on Chrome 78.0.3904.97.Even if
I close the tab,the circle was still on browser,Finally I had to restore down
the chrome in order to clear it up.

------
lightedman
Its about time Google had a CFAA charge brought against them. I'm pretty sure
most network admins didn't know Google had such a whopper of a backdoor
deliberately snuck in through their security measures.

------
lessname
Even though it might break sometimes, it is one of the reasons why I choose
Chrome over Firefox: The updates on Firefox take just too long (mostly before
starting) on Chrome they just happen... However, it would be nice if they
asked before.

------
rbdeveloper
Thank god Brave has field trials disabled by default [1], among many other
Google "services" [2].

[1] [https://github.com/brave/brave-
browser/issues/4283](https://github.com/brave/brave-browser/issues/4283) [2]
[https://github.com/brave/brave-browser/wiki/Deviations-
from-...](https://github.com/brave/brave-browser/wiki/Deviations-from-
Chromium-\(features-we-disable-or-remove\))

