
Google Optimize now free for everyone - gmays
https://www.blog.google/products/marketingplatform/analytics/this-is-not-a-test-google-optimize-now-free-for-everyone/
======
rightbyte
"Optimizing" UI for session length and clicks is the same thing as finding the
local maxima of bad, right?

~~~
uberstuber
The effects of this are easiest to see with online recipes; the highest
ranking recipes are all thousand word ramblings with a recipe tacked on at the
end. Google sees you spent more time on the site (i.e. wasted scrolling) and
thinks you were more 'engaged.'

~~~
fancy_pantser
I noticed this too and made a plugin that saves me a lot of time; it copies
the recipe into a container and puts it in a modal at the top of the page with
original formatting intact.

Chrome extension: [https://chrome.google.com/webstore/detail/recipe-
filter/ahlc...](https://chrome.google.com/webstore/detail/recipe-
filter/ahlcdjbkdaegmljnnncfnhiioiadakae)

Source: [https://github.com/sean-public/RecipeFilter](https://github.com/sean-
public/RecipeFilter)

Video demo & explanation:
[https://youtu.be/3Xq1p10f3v4](https://youtu.be/3Xq1p10f3v4)

~~~
GunlogAlm
This is awesome and will save me a lot of time. Thanks!

------
maxpupmax
This is a freemium product. There's an enterprise upgrade that costs about 10k
per year I believe. It's nice though, I use it on my site.

~~~
AstralStorm
Additionally a freemium product where Google can steal all your ideas because
you just gave them the license.

(not necessarily true, but it has happened before with YouTube)

------
GunlogAlm
'Optimize' has been around for quite some time, just in case anybody thinks
this is new. In fact, I believe this feature has been around for over a year,
now.

------
curo
Does anyone have a rule of thumb on when A/B testing becomes important for
startups?

We have a few thousand visitors a month and are starting to convert, but my
guess is A/B testing language and buttons would be premature optimization for
us. Just curious at what point that's no longer the case.

~~~
roberttod
In order to be useful, you probably want to see AB tests reaching a conclusion
in under 30 days. I'd say for a conversion rate goal, this is going to be when
you have around 100k visitors a month.

There are a few variables to consider

\- What goal would you like to AB test? Conversion rate is an end-of-funnel
goal that needs a lot of traffic, you can use upper funnel goals like product
views, add to bag etc to get quicker conclusions (not as accurate but often a
good approximation)

\- The stats engine/AB testing tool you are using. More simple tools might
conclude quicker but in my experience they can be so inaccurate they are
counter productive. Usually a long time to conclude = reliable results. I've
never used Google Optimize so I'm not sure where it stands.

\- How many people are being exposed to the AB test, for example is it all web
traffic or just mobile?

\- How much of an affect the AB test has on behavior. A button color/text
change will normally take long to conclude than a feature that's really
helping your users.

\- How confident do you want to be before reaching a conclusion? I'd recommend
looking for 95% confidence in uplift before concluding an AB test.

~~~
citrablue
Google Optimize offers MAB testing, which is awesome. Personally, I don't see
it as a "when do results become reliable" problem -- it's a "what is the cost
of a false positive, false negative, or 'do nothing' decision?"

~~~
roberttod
Multi Armed bandit tests need even more traffic, as there are more variations
being tested. I think you have to be careful with false positives in AB
testing - drawing conclusions too fast can nullify it's usefulness.

~~~
citrablue
The different between "traditional" A/B testing and Multi-Armed Bandit has
nothing to do with the number of treatments. A/B is really shorthand for
A/B/n.

In an A/B test, the probability of selecting an "arm" (a treatment to show to
the visitor) is equally distributed. Google Optimizer differs by adjusting the
traffic distribution, sending more traffic to better performing options,
what's usually called MAB approach.

The difference is that MABs will more quickly converge on the "winning"
variation, but are more likely to get stuck in a local minimum. EG, if an
"worse" option performs better right off the bat, the MAB might send most
traffic to it. This would take a while for the algorithm to "recover".

The major advantages of the MAB approach are minimizing opportunity costs and
the ability to capture seasonality because you can continuously run tests.
Traditional split testing runs for a while, gets a result, and moves on. With
a MAB, you can assign an "explore" budget that keeps tests running in the
background to capture the seasonal/periodic change in conversions.

That comes with a cost: every visitor who doesn't see your best page is lost
revenue.

------
indiesolver
I am wondering if it is based on their own Google HyperTune / Vizier [1, 2]
which is modified to better deal with uncertainties or it is an absolutely
independent in-house development.

[1] [https://cloud.google.com/ml-engine/docs/tensorflow/using-
hyp...](https://cloud.google.com/ml-engine/docs/tensorflow/using-
hyperparameter-tuning)

[2]
[https://ai.google/research/pubs/pub46180](https://ai.google/research/pubs/pub46180)

------
matt4077
"Published Mar 30, 2017"

(I'm not complaining–it's interesting and I'll give it a try. But it's not
brand new)

~~~
na85
Where is the requirement that we only see "brand-new" stuff on HN?

~~~
yjftsjthsd-h
Well, it should probably be tagged (2017).

------
hartator
I just hope they won't kill Experiments in Google Analytics.

~~~
subpixel
"Google content experiments to be deprecated"

[https://github.com/dwyl/learn-google-
optimize/issues/8](https://github.com/dwyl/learn-google-optimize/issues/8)

~~~
hartator
Yeah, I know, we'll use it until they actually kill it. Nothing really
compelling in the Google Optimize offer make me want to switch, but maybe I am
wrong.

------
jwatte
So Google just killed crazyegg and optimizely?

------
oh-kumudo
How long will this promise be kept, I am curious.

