> So does Mac already technically have a native way to play a video as your background, it just does not give the option?
Not really. It may have changed in Sequoia, but afaik in Sonoma this is how it worked :
- ScreenSaver plays the video
- You press a key/unlock your mac
- ScreenSaver does a XPC to the new Wallpaper system with the exact timing/position of where the screensaver was
- Wallpaper system starts a new video player and syncs it exactly
- Your mac wakes up, screensaver is hidden (it's complicated), and the second video player starts playing and slows down
- Eventually it gets replaced by a static screenshot of the last played frame seamlessly
- Screensaver exits (maybe, sometimes)
It's fairly complicated.
> what the performance hit during those few seconds of video playing when unlocking
Minimal, it's all hardware accelerated (videos are 4K/h265).
> is the video just "paused" or is it taking a frame and setting that as the background for better performance
It takes the last decoded frame eventually, but it's not instantaneous.
There are tons of other interesting tidbits about the system, I tried to untangle it because Apple introduced some really annoying bugs in the screensaver system as they changed the way screensavers exits because of the XPC handshake, and it threw a lot of issues in Aerial and other 3rd party screensavers.
Also if you use Spaces or multiple monitors, because you were able to have a different wallpaper on different Spaces/monitors, Apple now lets you set a different screensaver on each. That broke the ability to set a default screensaver which is a pain for 3rd parties.
Note that the "screensaver/wallpapers" videos are 4K/240FPS (likely interpolated frames for many) just to get that smooth slowdown effect. And they backported pretty much the entire catalog of existing Aerials to that format, even if only a few are available in macOS.
I do use these (the whole collection) in Aerial and for a few years I've also had a desktop version if you want to give it a try, it's in the installer. By default I use the 240FPS videos nowadays, and the desktop tool (Aerial Companion) will play those back at slow speed too by default (there are playback controls) : https://aerialscreensaver.github.io
That does seem like a lot of steps to make it work, but at least the screenshot part makes sense from a performance stand point.
However one part that confuses me:
> Your mac wakes up, screensaver is hidden (it's complicated), and the second video player starts playing and slows down.
I would think that it technically means that Mac does have the capability of playing a video in the background. Maybe right now it is hard coded to slow down and eventually be replaced, but something is still playing and moving in the background.
I did not realize until you mentioned it that you could have different screensavers on different screens. I guess logically it makes sense given that you can have different wallpaper, but I am kinda surprised they took it that far.
I thought all of the videos are available on Mac. I can't think of any that I have seen on my Apple TV that I am not currently seeing as part of this. I am looking in system preferences and I see 61 landscape, 30 cityscape, 21 underwater, and 22 earth
> I would think that it technically means that Mac does have the capability of playing a video in the background.
I mean, under the hood, it's just windows that are stacked on top of each other at different Z-index.
The wallpaper itself is a window, it sits at the lowest z-index. There's another (transparent) window a few levels above, where your icons sit. Dock/menubar sit on top of those again a few levels above. You can yourself put a window there at pretty much any level.
The screen saver is its own window that's put "on top" of everything else. The XPC I mention starts the player in the desktop window below. Makes sense ?
> I did not realize until you mentioned it that you could have different screensavers on different screens. I guess logically it makes sense given that you can have different wallpaper, but I am kinda surprised they took it that far.
They had too with the way they implemented it, for each screen if you want the "continuity" effect you have to be able to set a different one as both saver/wallpaper.
> I thought all of the videos are available on Mac. I can't think of any that I have seen on my Apple TV that I am not currently seeing as part of this. I am looking in system preferences and I see 61 landscape, 30 cityscape, 21 underwater, and 22 earth
Are you running Sequoia ? I double checked and it now looks like they use the full collection indeed. I'm fairly sure it wasn't the case in Sonoma at the beginning. Maybe they added them in a point update at some point and I missed them. They always had the full manifest of videos, but only a subset was available when they launched the feature, that I'm sure. Some are fairly old and not very sharp, so it made sense to be selective, but Apple does create new versions of the same videos to clean them up fairly frequently.
Behind the scene there's a SQLite database that used to contain only which ones were shown in the UI, while the manifest of videos (which is similar to, but not the one used on tvOS, which has HDR versions encoded with Dolby Vision) is pulled online periodically.
Not really. It may have changed in Sequoia, but afaik in Sonoma this is how it worked :
- ScreenSaver plays the video
- You press a key/unlock your mac
- ScreenSaver does a XPC to the new Wallpaper system with the exact timing/position of where the screensaver was
- Wallpaper system starts a new video player and syncs it exactly
- Your mac wakes up, screensaver is hidden (it's complicated), and the second video player starts playing and slows down
- Eventually it gets replaced by a static screenshot of the last played frame seamlessly
- Screensaver exits (maybe, sometimes)
It's fairly complicated.
> what the performance hit during those few seconds of video playing when unlocking
Minimal, it's all hardware accelerated (videos are 4K/h265).
> is the video just "paused" or is it taking a frame and setting that as the background for better performance
It takes the last decoded frame eventually, but it's not instantaneous.
There are tons of other interesting tidbits about the system, I tried to untangle it because Apple introduced some really annoying bugs in the screensaver system as they changed the way screensavers exits because of the XPC handshake, and it threw a lot of issues in Aerial and other 3rd party screensavers.
Also if you use Spaces or multiple monitors, because you were able to have a different wallpaper on different Spaces/monitors, Apple now lets you set a different screensaver on each. That broke the ability to set a default screensaver which is a pain for 3rd parties.
Note that the "screensaver/wallpapers" videos are 4K/240FPS (likely interpolated frames for many) just to get that smooth slowdown effect. And they backported pretty much the entire catalog of existing Aerials to that format, even if only a few are available in macOS.
I do use these (the whole collection) in Aerial and for a few years I've also had a desktop version if you want to give it a try, it's in the installer. By default I use the 240FPS videos nowadays, and the desktop tool (Aerial Companion) will play those back at slow speed too by default (there are playback controls) : https://aerialscreensaver.github.io