
Non-Rectangular Video Cropping with FFMpeg - lipeltgm
https://dragonquest64.blogspot.com/2019/11/non-rectangular-video-cropping.html
======
the8472
Note that a few video codecs also support alpha these days, which could be
used to preserve the mask and overlay the video over some background.

It even works on the web:
[https://files.catbox.moe/uu2ze0.webm](https://files.catbox.moe/uu2ze0.webm)
(needs a non-black background)

~~~
izietto
here with a yellow background (edit: changed to a yellow-to-green circle-
shaped radial gradient because why not):
[https://codepen.io/mdesantis/pen/RwNZEZJ](https://codepen.io/mdesantis/pen/RwNZEZJ)

~~~
pbhjpbhj
Maybe,
[https://codepen.io/pbhj/pen/dyPzwEj](https://codepen.io/pbhj/pen/dyPzwEj)
instead - centre the video and put the gradient on the body to emphasise the
matting?

------
rejschaap
It has some aliasing artefacts around the edge of the circle, you can see it
very clearly during the fade-in. When the video is playing you don't really
notice it. My guess is that this is caused by the way the mask is created. I
can't try it out right now, but it could probably be solved by setting the RGB
channels of the mask image to black.

------
_Gyan_
See
[https://superuser.com/questions/1270950](https://superuser.com/questions/1270950)
for a way to do it without needing to create a masking image.

~~~
lipeltgm
Agreed. The 'gex' filter can readily accomplish the masking if the shape can
be represented as an X,Y equation. This method does however work for a more
comprehensive list of shapes, titled wrt circular since that seems to be a
highly desired form of <i>cropping</i>. Thank you for pointing it out and
likely would address the aliasing another reader referenced.

------
vernie
I think it's more appropriate to call this matting. I came in thinking they'd
figured out how to coax FFmpeg into encoding non-rectangular frame geometries.

------
batt4good
Is there any way to provide a dynamic mask? For instance, if I wanted to blur
a dynamic portion of each fram, like a face or logo?

I've been trying to find higher-performance methods of doing this with live
footage that don't require openCV. Seems like FFmpeg doesn't have a great way
to do this or I just haven't found it yet?

I guess I'm also looking for frameworks or tools that can handle live video
"manipulation" like rendering shapes or graphics. OpenCV is painfully slow for
most of this stuff, everything else is seemingly completely proprietary. But
nVidia seems to have some incredible tools baked into their latest stream
manipulation API's.

------
rambojazz
Some day someone will write a simpler replacement for ffmpeg's user interface.
A tool that you have to read the manual every time you want to write (or even
just understand) a command is not a good tool.

~~~
_Gyan_
The CLI syntax design reflects the N-inputs --> N-outputs capability and no.
of options reflects the number of components e.g. there are 300+ demuxers and
400+ filters available.

This is not to say there isn't scope for improvement. A couple of areas for
improvement is the consistency of options naming across filters that implement
the same feature within those filters, and a clearer syntax to identify the
target of an option i.e. protocol, demuxer, decoder..etc

~~~
the8472
perhaps a declarative config file instead of CLI arguments would help. With a
config file you could lay things out in a more tree-like manner, have lists
etc.

And it's not just N inputs, M outputs. You also have a processing graph for
the intermediate filters. So ideally you want graph nodes represented by some
identifier which can then be referenced by other nodes.

~~~
_Gyan_
_You also have a processing graph for the intermediate filters. So ideally you
want graph nodes represented by some identifier which can then be referenced
by other nodes._

Unless I misunderstand you, this is already the case. Simple or complex
filtergraphs are declared as a string* by the user as an arg to -vf/-af (per-
output) or -filter_complex/-lavfi (global). Filter outputs can have link
labels, which allow their consumption by other filters.

*can also be read from a file

~~~
the8472
Yeah, but that's just part of the overall parameters. with a structured file
format (toml, yaml, json... pick your poison) the filter arguments could also
be structured themselves instead of having to be a single string, just another
nesting level among the other params. The shell only so many ways of
structuring things until you run out of seprator chars. And already has to
deal with the impedance mismatch of translating space-separated args into an
array of zero-terminated strings which results in its own escaping rules.

~~~
_Gyan_
Filtergraph declarations in files* can have whitespace so you can format it
logically. Each option of a filter can be on different lines, if you like.

*via shell too, if the arg is quoted/escaped.

~~~
the8472
You may be missing the forest for the tree. I am not talking about the filter
graph in isolation. I am talking about specifying all command line arguments
(inputs, seeking, outputs, filters, codec options) etc. in a structured way.
Filters would only be a part of that.

I think this might make use of ffmpeg less daunting than people attempting to
battle a single-line shell input, ordering-sensitive parameters and escaping.

Well, maybe it's redundant, there already is vaporsynth after all.

------
ArtWomb
My new years resolution might just be to gain mastery of ffmpeg. It's
incredibly versitile. Combine with python bindings, image magick and opencv
and you have a powerful weapon in your arsenal.

But I keep running out of memory. I like that the process just dies. And can
usually be solved by sequential batching. But how can I automate via
predictive instrumentation?

~~~
pselbert
Learning ffmpeg can be daunting but it is a highly useful tool to know. The
option intricacies and peculiarities make it extremely hard to “master”
though. It requires so much experimentation and experience (trial and error)
that achieving mastery in a year seems nigh impossible!

------
GJR
Fill and key is _basic_ video manipulation. As to the aliasing, they also need
to apply some filtering...

