
How we scaled Google Meet during Covid-19 - caution
https://cloud.google.com/blog/products/g-suite/keeping-google-meet-ahead-of-usage-demand-during-covid-19
======
parhamn
It's crazy how little Google's marketcap benefits from things like this.
Zoom's here sitting on 75B (~1/15th Googs) as we speak and Google Meet
probably isn't even legitimately used in pricing Google. Wonder what Google's
market cap would be if each individual service got the VC future growth
pricing.

~~~
bhauer
Not only that, but modern Google Meet is significantly more useful to my day-
to-day remote conferencing needs than Zoom. Modern Google Meet's advantages:

* It's nearly instantaneous to use. You can just bookmark any prior meeting ID you created and reuse it indefinitely. You just click the meeting on your bookmark toolbar and join the meeting. Creating a new meeting is also nearly effortless.

* Works right in your web browser. It doesn't default to pushing you to use a native executable.

* IMO, the video and audio both are slightly better than Zoom.

Google Meet or Hangouts from years ago was quite bad, but it has improved
nicely.

~~~
hitekker
> IMO, the video and audio both are slightly better than Zoom.

Firm disagree.

Three days ago, I presented a video in a google meet stream for 50+ people.
For three minutes the stream dropped frames , displayed visual artifacts and
lagged behind my narration.

I was on gigabit internet and so were my colleagues.

I never had this problem with zoom, and I would never use google meets for a
streaming setup again.

~~~
ariwilson
Did you use the newish "Present a Chrome tab" feature to do this?

[https://gsuiteupdates.googleblog.com/2020/04/high-quality-
vi...](https://gsuiteupdates.googleblog.com/2020/04/high-quality-video-audio-
meet.html)

Normal Google Meet Present is optimized for slides/docs which is a very
different problem than streaming video.

~~~
giovannibajo1
Well, it’s still a joke that Google Meet will stream desktop / windows at 4
FPS. I would be curious to know whether they really think the feature is good
enough.

~~~
londons_explore
This is mostly a limitation of Chrome and other web browsers.

When streaming the desktop, the browser doesn't get notified when some region
on the screen changes. That means it has to retrieve a bitmap of the screen
and 'diff' it with every frame, which is computationally heavy.

~~~
giovannibajo1
FWIW, AirPlay can send a H264 video of my desktop at 60FPS with ~0% CPU impact
(hardware accelerated), good enough to play many games on TV.

~~~
londons_explore
I believe Airplay has special OS integration - a third party app wouldn't have
access to the screen buffer with zero-copy.. A third party app would at a
minimum have to copy the data to system memory and back to the GPU, using a
lot of CPU time for that alone, before any encoding.

------
jtwaleson
To Googlers working on this: thank you! My work life shifted from 5% Meet to
80% Meet and it only got better during the crisis. Never noticed deteriorated
performance.

------
ocdtrekkie
Literally adding Google Meet URLs to Zoom meeting invites in Google Calendar,
so half of a meeting's members went to the wrong room is kinda a bad way to
scale your product: [https://mspoweruser.com/googles-latest-audacious-growth-
hack...](https://mspoweruser.com/googles-latest-audacious-growth-hack-sending-
google-meet-links-along-with-zoom-invites/)

~~~
gav
There's a setting you can turn on called "Automatically add Google Meet video
conferences to events I create"[1].

I tested on a new account and the default was on, which you might argue is
bad, but when I tried to create my first calendar event there was a popup that
told me that it was on and gave me the option to turn it off.

Plus when you create a event the option to include a Google Meet is right
there and it's one click to remove.

[1]
[https://support.google.com/a/answer/9898950?hl=en](https://support.google.com/a/answer/9898950?hl=en)

~~~
ocdtrekkie
Question is, what's it doing when you use client software that doesn't support
the spammy popups? It appears that for a third party client user, it auto-
injected URLs the sender didn't expect or even see. So someone creating a Zoom
invite had no idea their coworkers might see a Meet link.

Also, Google could/should recognize that a meeting invite with a competitor's
room URL in it is somewhere it shouldn't inject its own. Especially since it
opens up yet another avenue for an open and shut anticompetitive case against
them.

~~~
londons_explore
Blacklisting some competitor URL's is discriminatory against smaller
competitors or people's on-site solutions at private URLs

~~~
SahAssar
The parent didn't say to blacklist the competitor but rather to not inject
googles own when a competitors URL was already in the invite.

------
jeffbee
I'd love to hear the war stories about the network capacity plan violations
that were involved here. There's no way they had 30x headroom in their 90-day
capacity forecast. The organizational ability to reach in and shuffle up the
network capacity allocations may be even more interesting than the technical
items discussed here.

~~~
justicezyx
Nah, this is considerably minor scaling problem for Google.

Youtube bandwidth is there. Google meet is just a rounding error. Of course
(most likely) there will be dedicated planning going forward, but these type
of traffic ramping up is daily operation at Google.

At Google, application developers rarely pay much attention to scalability,
that's how the TI org tries to achieve, and they are probably the best AFAIK.

~~~
jeffbee
It's clear that the Google where you worked existed in a parallel universe
from mine. On my home planet, google netcap required a forecast per-product,
per region pair, years in advance, and the product group got charged back for
the capacity requested, not for the usage, so nobody was requesting extra.

~~~
londons_explore
Hah - except there were overestimates at every level, like where teams were
confident of there 50% growth per quarter for the next 2 years, and that never
materialised. Know the right people, and that unused capacity can be
reallocated to you in a few hours.

Capacity planning done like that is mostly a waste of time...

------
j4mie
Every web-based video chat I’ve used (including Google Meet and Slack) makes
the fans on my decently-specced 2018 MacBook Pro spin up like it’s about to
take off. I’ve tried Firefox, Chrome and Safari. Zoom (via the native Mac app)
doesn’t do this. Does anyone know why browser-based video is so resource
intensive?

~~~
giovannibajo1
Use Safari. Turn on the developer settings, and find the setting to disable
VP9 in WebRTC. That will force Meet to stop using VP9 and switch to H264 for
which MacBooks (and Intel CPUs in general) have a hardware accelerated decoder
_and_ encoder.

~~~
deadmutex
Note that H264 is not as efficient as VP9, so you may end up using more
bandwidth or have lower quality because of this.

~~~
samoa42
true but this is a favorable tradeoff.

i also recommend forcing mpv->ytdl to h264 for the same reason; fan keeps
quiet while watching youtube.

------
guillemsola
I think the point of this article is how the Meet team handled the increase
usage, not whether if this product is better of worst than competitors.

I bookmarked the article that can be seen as an engineering approach to solve
a problem like this.

------
tasssko
When did Google meet get better? I used it actively in April had a some really
bad customer meetings on it (it seemed to be bandwidth heavy though that is
anecdotal). I decided on Zoom and haven’t really looked back. I also was
irritated at the Google Meets browser compatibility issues as I mostly use
Firefox.

------
CubsFan1060
I used Google Meet once, and immediately ended up back at Zoom. I was not
particularly impressed.

~~~
exclusiv
It's gotten better but the video quality is definitely lacking.

~~~
Aachen
Can't say I notice a difference, most platforms just take your webcam data and
forward it so I don't get why people report different qualities? Never had to
use zoom luckily but between Wire, Jitsi, Hangouts, BigBlueButton, and two
others whose names I don't remember right now, it always depended on people's
mics, cams, and WiFis (and sometimes available CPU if my crappy work laptop
had too much to do in the background to en/decrypt the streams).

~~~
exclusiv
A lot of factors but Zoom is superior to Meet in video quality from my
experience. Meet looks grainy. Zoom has various native clients.

~~~
Aachen
> Zoom has various native clients.

Yeah that's precisely what I'm afraid of given their track record.

------
ramshanker
30x growth only? My wild gutted is that at least 1 billion Gmail App
installations must have been updated to integrated Google Meet versions. So I
expected much higher jump.

Just yesterday, we were asking people in our school WhatsApp Group to install
Teams app. As for Google Meet, just update!!!

~~~
tpmx
> 30x growth only

srsly?

~~~
m3kw9
To be fair 1 to 30 is 30x growth. It really depend on the original size.

~~~
jeffbee
It says they ended up at 100 million daily active users. The fact that they
could deploy that capacity in 60 days should terrify their competitors.

~~~
oblio
Google Search had 1.2 billion users back in 2013, now it probably has 3
billion.

Youtube has 2 billion users.

Gmail has 1.5 billion users.

Android/Play Services has 2.5 billion users (well Android has 2.5, Play
Services probably has at least 1.5 billion users).

For Google a service with 100 million users is a drop in the bucket :-)

~~~
jeffbee
1.5 billion monthly active users sending short text messages is a really
different network capacity problem compared to 100 million _daily_ active
users sending high-definition video streams to each other.

~~~
londons_explore
But it's still small compared to YouTube video content served.

Also, the WebRTC stuff all browser based videoconferencing apps are built on
supports peer to peer Comms in most cases, so worst case they could have moved
all the users over to that in an overload situation.

~~~
jtwaleson
Not sure about the comparison with YouTube. YouTube is one to many, and can be
heavily cached. Meet is many to many with no caching.

~~~
jeffbee
Right, and YouTube doesn't need high priority traffic because it has a deep
buffer on the receiver, whereas videoconferencing will be disrupted by even a
single dropped frame.

