
Jitsi Meet – Improving Scale and Media Quality with Cascading SFUs (2018) - lioeters
https://webrtchacks.com/sfu-cascading/
======
steveharman
We did our daily standup call with it yesterday (mostly audio). 12 people, on
a $3 Scaleway Debian instance (2GB RAM). Worked perfectly.

Obviously as we stopped using Zoom we have to manually upload all our personal
data to Facebook now.

~~~
Certhas
We tried to but with most of our team on Firefox it simply doesn't work
unfortunately. Zoom has been the only thing reliable for us.

It's a bit ironic that somewhere, someone is switching from Firefox to Chrome
because of data concerns with Zoom.

~~~
ooboe
That's odd. My experience with Firefox on Windows is that I can't get audio to
work with Zoom. Jitsi, on the other hand, works perfectly though there is a
small yellow box that warns me that I should use a "fully supported browser"

~~~
fulafel
Anyone know what the reason for the yellow box is?

~~~
Certhas
Because Firefox in Jitsi is seriously not working unless you have small groups
and resources to spare. But they are working on it:

[https://github.com/jitsi/jitsi-
meet/issues/4758](https://github.com/jitsi/jitsi-meet/issues/4758)

[https://github.com/jitsi/jitsi-
meet/issues/5439](https://github.com/jitsi/jitsi-meet/issues/5439)

~~~
fulafel
Does it need more bandwidth or more server resources (or both?)

------
hirako2000
It's a fantastic open source solution. Can be self hosted.

It is to note that traffic does go through the server, hence need decent
bandwidth. the server afaik does have the keys to decrypt traffic

~~~
SparkyMcUnicorn
The Jistsi team says the server needs roughly 5.5Mbps per Chrome user. Firefox
uses a lot more bandwidth, system resources, and degrades the room capacity.

Just something to keep in mind, and after some testing I saw the same results.

My AWS bill was projected to be over $1k/month, so I put it on Linode where
it'll cost between $100-200/month. Just about any decent VPS provider would be
good options compared to AWS due to bandwidth.

~~~
toohotatopic
Is there cheaper bandwidth available than Hetzner's 1.19euro +vat per TB?
(Linode seems to be at 10$ / TB).

~~~
capableweb
Or Hetzners dedicated hosts that have zero cost for bandwidth and unlimited
traffic as well, haven't been able to find anything cheaper than that.

------
moron4hire
I've been playing with a custom install of Jitsi Meet for the last few days.
It was very easy to setup.

I'm kind of fearing trying to deploy it in my company. This has nothing to do
with Jitsi Meet and everything to do with the shitshow that is WebRTC in
browsers.

I can't find any desktop browsers that use the available GPU for hardware
accelerated video encoding. Chromium (Google and Edge) says it's "only
available on Chrome OS and Android". Firefox has a flag with no scare-
provisos, but I've been unable to tell if it's had any effect.

By default, every browser on my system was setup to prefer the Intel UHD GPU.
I've got an NVidia GeForce RTX 2080 in this laptop. Why not use that?

One of the reasons is that all the browsers have a software-rendering
blacklist for certain combos of OS/hardware/drivers. There are rare cases
where errant WebGL code can cause a full system crash on (checks notes)
Android. So if you're one of the unlucky many who have these combos, but also
an operating system smart enough to put graphics drivers into userland, you're
taking the fast-train to turning your laptop into a blow-dryer. So they tend
to take the Intel GPU over the NVidia GPU because apparently Intel's
hardware+drivers isn't as buggy.

You can override it in the hidden settings, which means nobody overrides it.

All of this is to say, you could have a very powerful computer and still have
very poor WebRTC performance.

~~~
quadrangle
Does this explain why I get poor Jitsi experience on my laptop and better
using the Android app on my phone??

I suspect you are hitting on the exact issue I'm having. I have a Thinkpad
with just the Intel 5500 GPU, nothing superb. So, how do I figure out if at
least that is being used or not?

~~~
moron4hire
Windows Task Manager will show you GPU utilization on the Performance tab.
Start Task Manager, then start your browser and see if there is an uptick.
Then start a meeting to see if there is another one. There probably won't be.
Then, disable the software rendering override in your browser: (Firefox:
[https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drive...](https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drivers#How_to_force-
enable_blocked_graphics_features)) (Chrome:
[https://superuser.com/questions/836832/how-can-i-enable-
webg...](https://superuser.com/questions/836832/how-can-i-enable-webgl-in-my-
browser)) and test again. Probably need to restart the browser after the
change in settings.

~~~
quadrangle
Why would you assume I'm on Windows?

I'm on GNU/Linux here.

But anyway, yes, I see specifically an uptick in the CPU dramatically with
meetings, distinct from other browsing.

------
SparkyMcUnicorn
Something that isn't mentioned in the docs is room (and server) capacity, or
at least rough estimates.

The Jitsi team gave some specific numbers on room capacity in the forums. Each
room should reliably handle 20-35 Chrome users (Firefox uses roughly double
the resources), and has a cap at 75. Apparently if you're only using audio,
then rooms of 70+ people will work fine.

They're working on upping this number to 500 for configurations that have
multiple bridges, as well as improving the UI to account for larger meetings.

~~~
toohotatopic
Where does the difference between Chrome and Firefox come from? Is Firefox
using a different video format?

~~~
l3s2d
See here for details: [https://github.com/jitsi/jitsi-
meet/issues/4758](https://github.com/jitsi/jitsi-meet/issues/4758)

------
gidam
Jitsi-meet is very easy to install and to setup. "apt install jitsi-meet" and
you have almost all done. The documentation is very good. It works great and
it is super userfriendly. You have no excuses.

~~~
UncleMeat
> You have no excuses.

Billions of users who aren't using linux distros with aptitude have an excuse.
This isn't remotely close to the "just works" that you need to get wide
adoption.

~~~
omnimus
Yet it is probably the easiest and most "just works" solution to self host
conferencing.

Of course if you don't selfhost anything it's not option for you.

------
ofrzeta
Alternatives to Jitsi are Janus (mentioned in this thread) but also OpenVidu
(formerly Kurento) and Mediasoup which provide building blocks to roll your
own. All of these also have demos that you can use out of the box for our own
conferences.

~~~
j1elo
Just a nitpick: Kurento still exists on its own, and provides a generalist
framework of components to build a service that can handle media.

OpenVidu is one such service, it builds _upon_ Kurento, and itself can be used
to make videoconference rooms very easily. Otherwise, Kurento is just a media
server that handles media but doesn't have the concept of "rooms",
"publishers", "consumers", escalability, reliability, etc. so you would have
to develop all those things on your own.

------
cf
Are there any good tutorials for how to setup Jitsi to work for 30-50 people?
The discussion forums are hard to navigate.

~~~
bsima
consider using mumble

~~~
LinuxBender
mumble / murmur is great, I host it for family and friends to use. That said,
it's only voice and text. No video or screen sharing yet.

------
jteppinette
I was told that Matrix/Riot is a good alternative to Jitsi (can share a
meeting URL and offers E2EE). Does anyone have a data point here?

~~~
overthelake
Riot isn't an alternative to Jitsi so much as a complement to it. Matrix is
for text chat, and Riot uses Jitsi for calls; while you can embed a Jitsi
client into Riot rooms, and it auto-shares the URL, AFAIK group calls are not
E2EE even if the room is.

One-on-one calls through Riot _are_ encrypted, but only if made in a one-on-
one direct chat.

~~~
jteppinette
The riot features page makes it seem very "built-in"
([https://about.riot.im/features#voice-
video](https://about.riot.im/features#voice-video)).

~~~
Half-Shot
riot.im hosts its own Jitsi instance which is built into the UI, so hitting
the call button in a room will spawn an embedded Jitsi call in the room.

Recently we added the option for people to use their own self hosted Jitsi
instances, if they prefer.

(1:1 calls at the moment are normal WebRTC, directly over Matrix)

~~~
dedward
I might be breaking the rules, but can you point me towards any information
about how to integrate my existing self-hosted jitsi server into my self-
hosted matrix install? Finding this seems elusive.

~~~
omnimus
I tried and i don't think its easily possible.

------
enriquto
I would love to use this instead of zoom. Any news about firefox support?

~~~
m-p-3
I use it for quick meetings on Firefox and so far it works with the limited
set of features I need, despite the warning.

~~~
enriquto
Last week I could not manage to use it for a three-way conversation (after 1h
of trying everything, and re-initializing the stuff several times). Sometimes
the sound of one participant was missing, sometimes the video of one. When we
got it to work for a subset of two people, the CPU of everybody was at 100%.

------
kodablah
I have been considering developing a simple video conferencing solution using
this approach with WebRTC. Basically, I was planning on doing something like
this example [0] of server-side peers or just a set of forced TURN servers,
whichever approach being deployed on servers at the edge.

I figured using anycast IPs, and having server to server communication across
regions before then sending back down to the client would be ideal. Each offer
and answer signaling would be done via distributed pubsub (e.g. embedded nats)
and general persistence also distributed (e.g. something simple like dqlite).
Has anyone had success with distributing WebRTC channels like this? Are there
any concerns with redistributing, say, raw h.264 and opus as is without
special concern for buffering or transcoding? How do slower consumers handle a
fast set of UDP packets?

Also, I doubt there's a way, but anyone familiar with an end to end encrypted
approach with WebRTC and DTLS when a server is in the middle? I figure not
since offer/answer is for a single peer instead of broadcast and I see no
approach in the browser to doubly encrypt media streams.

0 -
[https://github.com/pion/webrtc/tree/master/examples/broadcas...](https://github.com/pion/webrtc/tree/master/examples/broadcast)

~~~
supermatt
Using your first solution (a custom pion SFU relay), and selectively routing
the streams between "edge" servers would be similar to Octo, as shown here.

Using TURN servers instead would not have the benefit of SFU (i.e. clients
will have to upload to each peer) - i.e. its still a full mesh network.

Using TURN servers with SFU would work similar to your pion solution, however
it would also use more bandwidth, as it would be forwarding the same streams
multiple times for each peer routed through that TURN server (instead of once
per stream with SFU)

As for e2e encryption over webrtc via an SFU - yes, this is possible, but its
currently very messy (wasm video encoding and encryption streamed over an SFU-
bound datachannel with full mesh distribution of the encryption key). There
are plans to implement "Insertable Streams" which you will be able to
transform (e.g. encrypt) which will allow this to work without the hacks.

~~~
wildduck
>As for e2e encryption over webrtc via an SFU - yes, this is possible, but its
currently very messy (wasm video encoding and encryption streamed over an SFU-
bound datachannel with full mesh distribution of the encryption key). There
are plans to implement "Insertable Streams" which you will be able to
transform (e.g. encrypt) which will allow this to work without the hacks.

So currently Jitsi meet the one on the web site is NOT e2e encrypted?

~~~
supermatt
Yeah, not e2e encrypted when it goes through videobridge (as per this
article).

I haven't checked, but its possible for 1-to-1 or small meetings they may go
full mesh, which would be e2e encrypted - a few platforms do this.

edit: just checked and jitsi is "full mesh" for 2 participants - if you have 3
or more (video) participants, it switches over to SFU.

------
dergachev
I've been excited to try it with our company for a few months, but is it as
"user friendly" as Zoom for non-technical users?

~~~
dsr_
Yes. Installed it yesterday, ran a 12 person meeting with no instructions to
the users other than "go to this URL". No issues.

~~~
ehsankia
I would even say it's more friendly than Zoom. I only tried the hosted version
for both, but the experience with Jitsi was orders of magnitude simpler and
faster than Zoom.

For creating a room, you need an account on Zoom and it's generally
complicated to get to from the homepage. With Jitsi, right on meet.jit.si, you
simply pick your url name and you're in.

For joining, Zoom kept trying to push me to download the app, and had to press
two tiny buttons to get it to open in the browser. But by default you have no
audio or video, so I had to spend 10m teaching my parents how to enable
theirs. With Jitsi, it opens right in the browser, and video/audio work with
no issue once they accept browser permissions.

At every step, zoom tried getting me to sign up or download their app, whereas
Jitsi works fully in the browser with no account. Creating the room is a
single click and I get to pick the url too.

------
AnonC
I like Jitsi because it doesn't require any signups and it's simple to create
a channel/room. But the quality of video and audio seems a bit lacking. Does
anyone have suggestions on how to improve Jitsi Meet's video and audio quality
when working with the official smartphone apps and platform (not self-hosted)?
Sometimes (on a broadband connection), for some participants, the video from a
broadcaster freezes once in a while, and worse, the audio is many a times not
clear even when the video seems fine. This is with just three or four people
in the call with one person turning on the video and audio (like a presenter),
and the rest of them with no video and with their mics muted.

~~~
lioeters
Are any of the participants using Firefox by chance? There are known issues
being worked out ¹, some in collaboration with Firefox itself ², to improve
performance and compatibility.

For now, it seems the best audio/video quality can be achieved by all
participants using a Chromium-based browser such as Chrome and Edge.

\---

¹ 100% support for Firefox (and other non-Chrome browsers) -
[https://github.com/jitsi/jitsi-
meet/issues/4758](https://github.com/jitsi/jitsi-meet/issues/4758)

² Bugzilla keyword jitsi-meet -
[https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_t...](https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=substring&status_whiteboard=jitsi-
meet)

~~~
AnonC
Thanks. I know about the Firefox issue, but all participants are on mobile —
mostly Android and some iOS — using the official Jitsi Meet app.

------
dvduval
What about the cost in terms of per person per hour?

~~~
habi
It’s free and opensource.

~~~
gpm
Server time still costs money, I'd be curious to see how much at scale.

~~~
majkinetor
If you have commodore 64 you are OK.

------
eagsalazar2
I wanted to evaluate it but was very put off when trying to connect to my
google calendar they asked for full access to my Youtube profile. I get there
is some youtube integration they have but binding those things together
without any mention of why was enough for me to click cancel. Sucks because I
really want to ditch Zoom.

------
pojntfx
Jitsi has really been kicking it at my school lately; I wrote a chat bot for
out Nextcloud that creates meetings and now up to 150 people can easily join
into one conversation!

------
cryptonector
Can someone expand the SFU acronym? I chased the link to sfu-tagged blogs at
TFA and an expansion is nowhere to be found.

EDIT: Thanks!

~~~
ianmcgowan
Looks like it's a WebRTC acronym:
[https://webrtcglossary.com/sfu/](https://webrtcglossary.com/sfu/)

SFU stands for Selective Forwarding Unit.

At times, the term is used to describe a type of video routing device, while
at other times it will be used to indicate the support of routing technology
and not a specific device.

An SFU is capable of receiving multiple media streams and then decide which of
these media streams should be sent to which participants.

------
rebataur
We started to use it at our startup. If anyone needs to set it up, we can do
so in the cloud for them.

------
corwin7
I tried installing this on two different Ubuntu machines using their Ubuntu
repo and it did not work. Worked fine on both Android and Ios. This is about
what I expect from a java application - the java ecosystem is co convoluted
and fragile that the only real options are Android, Ios, and Docker. I wish
someone would write something like this in Go or Rust.

~~~
mrpurple_
Docker4thewin :)

------
mwcampbell
Does anyone know how the audio quality and latency of Jitsi Meet compare with
Zoom?

~~~
elagost
I've used it several times to chat with family members all over the globe, and
I use Zoom at work. Jitsi's great - I don't perceive a difference.

------
hyperluz
Too bad, like most other solutions, Jitsi, by default, shares screen at 5 FPS.

~~~
ygra
Is that possible to change, perhaps if self-hosting? Video quality of screen
sharing and frame rate are absolutely atrocious, to the point that you have to
wait ten seconds until text becomes readable.

~~~
hyperluz
Apparently, yes: [https://community.jitsi.org/t/screen-sharing-
fps/25132](https://community.jitsi.org/t/screen-sharing-fps/25132)

