One non-obvious piece of advice I have for developers delving into ffmpeg's filter_complex is to try and keep the filter graphs as short and simple as possible. Creating complex filter graphs programmatically (as ffcms does in this case) is super powerful, but I've also found in practice that larger filter graphs often lead to random, impossible to debug errors. This could be due to anything from some combination of VM resource constraints, internal ffmpeg leaks that compound at scale, etc etc.
My advice is to think carefully about what you want to use ffmpeg for. It's excellent at transcoding and doing minor filter graphs but trying to get too crazy with complex filter graphs will lead you down a dark path that imho ffmpeg wasn't really meant for. If you can break up your complex filter effects into smaller, isolated ffmpeg commands it may be slightly slower but your rendering pipeline will be significantly more robust.
Aaand of course, thanks for the advice. Personally, I've never been in a situation where `filter_complex` caused problems. But, I've never done some weird filtering there too, so you may be right.
I'm not saying that now, with ffcms, you should write JSONs that produce an A4 ffmpeg command. If you're doing so, something is not right.
For now, it's just a tool that helps me with timelapse creation for my fiancée. Let's see how it evolves.
"-f lavfi -i testsrc - declaration of an input. The input format is lavfi, the input file is testsrc. This is duplicated four times to imitate four different inputs."
You can feed a single input multiple times in a filtergraph, so you can make do with only one. In this case, it hardly matters, but the input packet will be demuxed and decoded only once, unlike in the examples.
I spent quite a while trying to replace the syntax of FFmpeg's filter-graphs with a tree-structure of nodes. Then about a week in I remembered why you should never build a DSL over a DSL.
I personally find the FFmpeg syntax to not need further abstraction. There's more than a few projects that try and replace the graph and they'll be stuck trying to keep pace with version creep.
Thank you. Very welcome addition. You could of course use a script for that, but that requires more effort / is more difficult for a lot of people to understand.
I hope there will be a similar structure (or DSL) for imagemagick, as it's it has the same difficulties.
I can say from experience of my own and others, that people usually keep these close to their chest, I consider mine a bible. In fairness, its a tool, so what you do with it is up to you. It's really hard to organise a standard library for even video work as each source is always a bit different, given the amazing community aroudn questions and answers I suspspect nobody wants to have a standard the same as ImageMagic.. It really is a different beast.
Edit : With certain frames, certain 'standards' settings would not result the same, so.. It's really impossible to have 1.
You might have 1 clip which requires 2 entirely different settings.
Love it. FFmpeg is so powerful, but so opaque. I think there could be a whole vertical of startups dedicated to tooling built to simplify using it. Looking forward to seeing this develop.
I started building a tool that simplifies video scripting [1] and the first version worked from JSON. Although this was aimed at a technical audience, people really struggled with it. it's difficult to add comments, and ended up being very error prone. My second choice was YAML, but this ended up even more error prone due to whitespace issues.
My lesson is that JSON and YAML are great for machine-consumption when people just need to do something once and leave it (eg config files), but far from ideal for stuff people need to edit over and over (such as script files).
I converted the input to markdown (with some small extensions) and it made a huge difference. it's less fiddly, much easier for humans to edit, and the parser can point out errors much better. for some nice examples check out https://github.com/videopuppet/examples/
In conclusion, YAML and JSON are great for machine-to-machine communication or human-to-machine for things that do not change often and aren't complex. for human-to-machine that is more frequent, we should be kinder to our users.
"Consume YAML, emit JSON". I've been using this pattern for the last several years on most projects, ever since someone else on the internet recommended it. Since YAML is a superset of JSON, people can input either JSON or YAML (or mostly just JSON with JS style comments, like I do most of the time) and have simple JSON output.
I thought about TOML for a while. Went with JSON because I just wanted to start working.
Like I said in one of the comments, final decision has not been made yet. JSON is not written in stone, plus ffcms could support more than one language, e.g. JSON and TOML.
For representing a graph of processing nodes, none of these are great. Maybe something like DTS format used by the Linux kernel, where the language parser itself would be able to check connections between processing nodes.
I built and sold a company built largely around FFmpeg and this type of tool would've been really useful for understanding how FFmpeg's filter chain works. If you're interested in some non-trivial OSS examples, check out https://github.com/transitive-bullshit/ffmpeg-gl-transition and https://github.com/transitive-bullshit/ffmpeg-concat.
One non-obvious piece of advice I have for developers delving into ffmpeg's filter_complex is to try and keep the filter graphs as short and simple as possible. Creating complex filter graphs programmatically (as ffcms does in this case) is super powerful, but I've also found in practice that larger filter graphs often lead to random, impossible to debug errors. This could be due to anything from some combination of VM resource constraints, internal ffmpeg leaks that compound at scale, etc etc.
My advice is to think carefully about what you want to use ffmpeg for. It's excellent at transcoding and doing minor filter graphs but trying to get too crazy with complex filter graphs will lead you down a dark path that imho ffmpeg wasn't really meant for. If you can break up your complex filter effects into smaller, isolated ffmpeg commands it may be slightly slower but your rendering pipeline will be significantly more robust.