Really hoping epic takes the opportunity to give Bink a more open license, to be honest. Much as I appreciate how advanced Bink was at the time, i don't know any dev that still use it given that av1 and such are just Good Enough.
Bink isn't the codec, though that's good too, it's the software stack. How do you render AV1 on a PS4 or a Nintendo Switch, synchronize it with the audio stream? No, seriously, tell me.
GStreamer with its ridiculous threading architecture and lock-heavy approach? You think that's going to run on a console CPU? ffmpeg / libavcodec? Yes, we can hook that up, have it crank out YUV buffers and display those on the GPU, or we can just use Bink. Yes, it costs a ton of money, but based on when I evaluated it last, it was almost twice as fast as libvpx on our target platforms, it was well-suited for games (worked well with custom allocation strategies, didn't start a bunch of threads by itself, integrated well into our renderer), was easy to integrate, the toolchain was lovely, and when we needed help with a bug, we got an email back the same day from Jeff. RAD's support is top-notch.
Multimedia is not too big of a problem that a company like Epic can probably spend a few days integrating libvpx and opus to make a custom video solution, but big enough that most smaller studios with custom engines don't have time to spend on that, and those are the sort of problems that RAD likes to tackle.
PS4 and Switch both have hardware decoders (though neither would be using AV1 in particular due to age). I'm not sure about how it works on PS4 but on Switch games just use NVDEC like a PC with Nvidia hardware would and the functionality is exposed in the SDK. You can use h.264, h.265, VP8, and VP9 (the hardware supports more but it's not exposed).
When Bink was released and the PS2 and Gamecube were on the horizon this kind of thinking was needed. When you get to PS4/Switch you're just running consumer PCs. The exact same Nvidia SoC used in the Switch is actually used by Nvidia for their Shield Android TV boxes whose literal purpose is to play audio/video streams - it's quite capable of doing so.
I can't go into more detail because these platforms do not have any public documentation, but this is not true, you cannot use NVDEC. And it doesn't cover audio or synchronization.
"The Switch uses NVDEC along with VIC (Video Image Composer) in the GPU for video decoding. Games are exposed to higher level APIs by Nintendo which abstract the configuration of these modules."
If you're using a custom engine which does not want to use these APIs you're of course free to do it yourself (and deal with how you're going to syncronize it and the higher resource utilization) but most major titles use the built in hardware decoder provided via the API.
Just curious, is there any reason MS/SIE console (AMD custom SoC) won't implement hardware RAD decoder? From what I read from this submission, their codec is good and used everywhere so it looks like it's better to have HW decoder.
SDL has always been the stack of reference when implementing your own engine, as far as I've seen in game dev. Now that is probably not the case for AAA games like cyberpunk etc, I don't follow as much tech there. But yeah, I've never seen gstreamer used for that. Curious if you can elaborate bout your experience especially wrt. the switch like you were mentioning downthread, I'm super curious.
Gstreamer and FFMpeg would certainly run on a Switch, a PS4, or whatever else. We know this because it runs on devices with the same processor.
That said, yes, Bink certainly is a more turnkey solution, which can definitely make business sense to use. This is a byproduct of consoles being closed platforms.
When I say "run", I mean "runs at target performance". Yes, sure, Turing completeness technically means that software raytracing "runs" on the Switch, we all know that, but it's not a viable solution for a shipping product.
This is not a biproduct of consoles being closed platforms, multimedia is a challenge everywhere; cross-platform, usable multimedia playback lacks a high quality product even on open PC platforms.
