
Building Firefox's Picture-in-Picture Support - primogen
https://hacks.mozilla.org/2020/01/how-we-built-picture-in-picture-in-firefox-desktop/
======
prook
At first I didn't even realize this was FF feature (I suspected something like
gstreamer, but was lazy to investigate). Now that I know it is indeed FF, I
just cannot wrap my head around it...

1) I can pause the video, but cannot navigate it. I guess the navigation would
be there if it was technically possible, but from practical standpoint this
makes it useless even if it appealed to me in other ways. Popping a video into
PiP mode and then go diving for the original tab to seek 10 seconds back to
rewatch an interesting moment seems utterly bizarre.

2) I've grown to truly hate this kind-of-PiP thing many websites (news outlets
mostly) do: Autoplay a video and then shove it to lower right corner when I
try to scroll away. Along with timed "subscribe to newsletter" popup, this is
an insta-close for me; Now why would I want to replicate this behavior
elsewhere?

3) I'm literally unable to read through even the first words of that blogpost
because of that gif. Now, thanks to this PiP feature, I can make any text
unreadable by placing a distracting video on my screen.

While I'd welcome features that fight distraction (and not support attention
fragmentation), there obviously is (significant part of?) FF userbase that
will make use of this feature. Good for them.

Luckily for us grumpy farts, that blue PiP button can be turned off in
preferences, we can mutter something about bloat, but generally go on with our
lives.

~~~
fiddlerwoaroof
I use this all the time to watch Netflix while programming: put Netflix in PiP
mode, then put the resulting window in the corner and switch to other
applications to work.

~~~
gbrown
It blows my mind that you can watch Netflix and program at the same time.

~~~
LandR
Sometimes coding is just mindless typing some boiler plate. It's easy to pay
attention to something else. I see it like listening to podcast or something
while mindlessly grinding in a game.

------
dthul
I have to say I love this feature and have so since the first time I
encountered it in my Firefox Nightly installation a few months ago (on
Windows).

Glad to also have it available on Linux and Mac now. It's perfect for on-the-
side Youtube videos.

~~~
lioeters
Yeah, I've been using the picture-in-picture feature regularly since I learned
about it recently. Love it for running lectures and presentations on the side.

------
Jestar342
People using tiling window managers like i3 or sway should get friendly with
this feature. It allows me to "full window" (note: not screen)
netflix/amazon/YouTube videos in a tile which was impossible to do without
PIP.

~~~
darau1
Either awesome or i3 used to allow me to fullscreen a video, then pop that
fullscreen video out into it's own floating window, that I could then stick on
top of all other windows, and make it stay on every workspace. Really cool way
to get a hacky picture in picture mode way before firefox did this.

~~~
Fnoord
VLC has a feature called "float on top". Together with resizing, it allows
basically something akin to picture in picture mode. You can use VLC to play
YouTube videos by downloading them with youtube-dl. That is what I used before
Firefox PiP mode was available in stable.

~~~
aasasd
VLC can play YouTube directly―at least if YouTube didn't change the format
again. Just ‘open’ the URL as a network source.

~~~
anticensor
They changed the playlist format. It has been failing again.

------
maxfrai
On mac os its mostly unusable, because this PIP window stays on same
workspace. My browser is always fullscreen and in safari I can open small
youtube window and move to other workspace (code, for example) and window
still exists on each of them.

~~~
dhritzkiv
This would be one advantage of the system's built-in PiP mode employed by
Safari. I don't find the limitations (maximum size, single video) to be
hinderances.

~~~
jorvi
Can't you trigger that in Firefox also by right clicking on a video (or twice
on YouTube) and then choosing PiP?

~~~
sercand
Problem is Firefox PiP don’t move other desktops on macOS. It is only
available at where your window is.

------
Wowfunhappy
> To work around these problems, we rely on the fact that the web page is
> being sent events from the browser engine. At Firefox, we control the
> browser engine! Before sending out a mouse event, we can check to see what
> sort of DOM nodes are directly underneath the cursor.

This triggered some mental alarm bells when I read it.

I understand what Mozilla is trying to accomplish, but this feels like it
could become a serious WTF moment for a developer. "There is absolutely no way
in hell this element should be able to receive mouse events right now, what is
Firefox doing?"

Is this a problem? Perhaps the web is so far down the hacks/exceptions route
that adding one more legitimately won't make a difference?

~~~
detaro
I don't think they are talking about mouse events the website can observe.
Firefox needs to figure out where the video is and make it's UI on top of that
work, there is no observable interaction with the site DOM or its event
handlers.

~~~
hoten
The post says that the video element is told it's being hovered, although it
is unclear if that means the rendering context has that element set to hover
(thus CSS changes even if pointer events is set to none), or that some
internal-to-the-browser state is set to hover for the video.

------
sdan
Picture in Picture is great for lectures or follow-along tutorials and I'm
glad people are taking it more seriously (initially was using Youtube on
Safari purely because of this feature, but later on Chrome had it... albeit a
bit wonky, I started using it on Chrome normally).

~~~
jpalomaki
I would love to have an external controller for start/stop/fast forward/rewind
when watching tutorials on Youtube. For example a controller [1] people are
using to edit videos. Quite often I would like to pause the video for a while,
repeat last 30 sec or by-pass some boring segment.

[1] [https://www.amazon.com/Contour-Design-Shuttle-Xpress-
Multime...](https://www.amazon.com/Contour-Design-Shuttle-Xpress-Multimedia-
Controller/dp/B0001DBEKG)

~~~
alexis_fr
That is an awesome idea. Would be extremely useful for guitar and
transcription too, while writing the text in Word.

It typically would be a usecase for the Macbook touch bar, does anyone has one
and can tell us whether it has an integration with web videoplayers?

~~~
daxelrod
Videos and audio in Safari use the touchbar as a scrubber with controls.

------
Merg
Everything old is new again it seems. Anyone used Maxthon Browser before? It
was multiengine browser, which used both Webkit and Trident under the hood. It
has that feature for ages, just for flash content back than.

If only someone would built good email client into Browser like old Opera,
that would be godsend. Opera and Maxthon were ages ahead in terms of certain
features.

It is little weird for me, they ended up putting video within shadow DOM
container.

~~~
girvo
Wow that’s a blast from the past! I also used... damn I can’t remember. It
used Gecko as its engine, implemented mouse gestures for navigation among
other nice features Firefox didn’t have at the time.

------
sverhagen
With (some) television sets in mind that popularized (for, um, me at least)
the picture-in-picture term as one channel of video on top of another channel
of video, it feels weird that the term is here used when only a minimum of 1
videos is involved...

~~~
aasasd
You're witnessing etymology in work:

\- the mechanism is about the same: the video stays in the corner

\- the term ‘picture in picture’ is already familiar to people, from TVs

...so the term is generalized. Happens a lot. If the feature gets popular and
the term sticks around, decades later people might search the web for ‘why is
it called PiP when it's not what it does’. We can even imagine that zero
videos may be involved by then (though unlikely―there are other terms for
that.)

------
cheez
Main problem I have is captions from Netflix don't work which is important for
me. Would have been nicer if we could have popped out a div instead.

------
LEARAX
To be perfectly honest, I didn't realize this was a Firefox feature. I thought
it was just a new part of YouTube.

~~~
2038AD
YouTube also recently added a similar feature that's more limited

~~~
LEARAX
I was aware of that, since it isn't very intuitive within the UI I figured
that Youtube was attempting to push it a bit more aggressively.

------
drannex
This is honestly one of my favorite features in firefox.

~~~
hombre_fatal
fwiw you can get this feature as an extension in Chrome and Safari.

for example, this is published by google:
[https://chrome.google.com/webstore/detail/picture-in-
picture...](https://chrome.google.com/webstore/detail/picture-in-picture-
extens/hkgfoiooedgoejojocmhlaklaeopbecg?hl=en)

------
huxflux
I've been waiting for this feature for years, and in my eyes, this seals
Firefox as the number one browser out there. Previously to that, I was using
Opera exclusively for this feature.

------
jancsika
I wonder if an attacker could leverage this to play twenty-questions with a
user.

Video is typically so-and-so dimensions in the bottom right-hand corner.
Clicking the video (I assume) pauses it. Pausing a video (I assume) can change
the request pattern for what is being streamed. Most sites use a white
background (let's say).

So an attacker who controls video content can stream a still image of some
site or browser security "statement" with a picture of a button to click if
the statement is untrue. If the user clicks the fake button they stop the
video and signal to the attacker their response to the statement.

Not super useful given an arbitrary page behind the video. But what if the
video's content is about, say, how to change settings on your bank site to get
a better APR? I can imagine someone assuming that the video has turned into a
small "quiz" from the bank. Maybe even a "click here to proceed" button to get
them to unpause the video.

~~~
chrismarlow9
the entire concept reminds me too much of iframes, embedded flash and Java
applets. i can see many security problems coming with this in the future.
especially if they try to make it a real window that can jump tabs and stay
pinned, which seems to be what most people here would really want and an
obvious next feature.

think about click jacking, address bar spoofing, and similar mosaic style
hijacking. that's not even getting into potential CORS and origin spoofing
potential.

------
rasz
I much rather they would provide video stream hook so I could execute my own
player as the external one (VLC, Mplayer, etc). As it is I have a bunch of
custom userscripts injecting my own hook into selected websites.

------
warpech
This is the Picture-in-Picture API it implements:
[https://w3c.github.io/picture-in-picture/](https://w3c.github.io/picture-in-
picture/)

I am surprised that the API is modelled to only allow watching videos in PiP,
while the old and mature Fullscreen API allows to put any element in full
screen. These should be analogous. I don't see a reason why I can run a
<video> in a small window, but not a <canvas> a <picture>, etc.

~~~
mconley
As the article mentions, Firefox is not currently implementing the Picture-in-
Picture Web API that you linked to. Firefox's implementation is a browser
feature only, and not exposed to the web at large.

------
GeneticGenesis
Mozilla's choice to add their own pip button is a massive pain for anyone
developing video applications - anywhere over the player can't be considered a
safe space to add our own buttons anymore, it really feels like Mozilla didn't
talk to any video service developers when deciding how to approach this
problem.

Pip needs to be common API driven (as Chrome is) to make building portable
video applications possible.

A terrible misstep from Mozilla. (And yes, this has been repeatedly raised on
the bug tracker with no answer)

~~~
mrpopo
> Pip needs to be common API driven (as Chrome is)

Ah yes, Mozilla should follow the steps of Google's web experiments
playground, the Chrome browser. Every month we hear about a new self-serving
feature implemented by Google/Chrome engineers.

The web is slowly drifting to the IE days again, and of course this move by
Mozilla doesn't help it, but let's not forget the bigger culprit.

------
dingaling
Well it'd be handy if I could PiP another non-video tab.

------
tommica
Firefox PiP needs a few additions - first of all show subs, second allow me to
change the shortcut position in videos - sometimes next/prev arrows are on top
of the element, making it impossible to click :/

------
tilolebo
Really awesome feature! Now I understand why I always see this little blue
icon in my Udemy course.

Too bad it is "under" the button to switch to the next video, so I can't
actually click it...

~~~
jaharios
You can right click to get the menu option, if needed try right clicking two
times.

------
atulvi
Spotify web player has a feature where it shows the album art as a picture in
picture. For some reason, it's disabled in Firefox.

------
megavolcano
I've been using PiP in Opera for years, glad to see other browsers starting to
implement useful features.

------
Flott
I love this feature. I wish it worked with Mixer. At least it work on Twitch!

------
Quai
Obligatory comment about Opera: Afaik, the video pop out feature was published
in Opera 37, May 2016, a few months before it was added to Safari.

------
alexis_fr
> Given that it would be available on any video content, the feature needed to
> be discoverable and straight-forward for as many people as possible.

This is the designer’s nightmare: Every feature needs to be available to
everyone, therefore be visible on the screen. Yet they only cite rare
usecases: « Watching sports event while continuing to browse the web ». In the
meantime, the blue tone isn’t generally consistent with the websites’ colors.

I would have supposed moderatelt-used features belongs in the right-click menu
like the rest, or they need to find a way to merge it with the video controls
(I guess an Apple designer would imagine a new kind of special press to detach
the video, which is fine too).

~~~
Wowfunhappy
I feel like this is the type of feature that should be a browser extension. It
could be an officially supported one from Mozilla.

~~~
AlphaSite
Safari implements this natively and it’s super handy.

I imagine if you intently watch videos it’s not useful, but for multitasking
it’s great. Or note taking etc.

------
Jach
I thought this might have been some Steam tie-in or otherwise not a Firefox
built-in feature, never clicked it.. and now I can't even try it as the
button's not appearing, it seems pretty unstable. Not that I really want to...
This solves a problem I've never had. What's so hard about moving the video to
a new window, or if your screen is too small moving the window to a new
monitor? In Linux you can also make the window smaller to just have the video
visible and then make it "always on top" of whatever else you're doing.

------
frankharv
For the life of me I can't see why anyone would need picture in picture for
browsers. More features most people don't need. Please come down to earth
Mozilla. You have lost your way. Please revisit NCSA Mosaic and witness your
roots.

~~~
Zarel
I use PIP a lot in Chrome while browsing Reddit. If I see a video link, I can
play the video while continuing to scroll through Reddit comments.

(YouTube PIP is kind of difficult to discover; you have to right-click, move
the mouse off the menu, and right-click again to get the native right-click
menu, which is where the PIP button is.)

~~~
Mirioron
Why not just open it in another tab?

~~~
detaro
You mean open in another window, arrange and resize that window to not get in
the way while making the video visible, hoping that the website layout doesn't
make that close to impossible, and still not easily being able to have it stay
on top of the window you interact with... instead of a single click that does
the right thing?

