Hacker News new | past | comments | ask | show | jobs | submit login

You can also apply this to each frame in a video for a rather interesting effect.

My brother has a YouTube channel full of content-aware scaling videos:

https://youtu.be/a8k3b-QNbhs




Wow, that's incredibly interesting!

My first reaction is how actor's faces look surprisingly like traditional caricatures that illustrators do -- e.g. shrinking foreheads and chins which are detail-light but keeping eyes and ears which are detail-heavy.

But my second thought is that the extreme jumpiness in frames occurs because each frame is processed separately. But if you considered each seam not to be a "jagged line" from point A on one edge to point B on the opposite edge of a single frame, but rather a "jagged plane" cutting through a series of frames -- all frames in a single shot -- you could eliminate the jumpiness entirely.

You might need to build a bit more flexibility into it to allow for discontinuities generated from object movement and camera panning, but I wonder if anyone's tried to do something like that?

Though I imagine it might be quite a lot of programming for a tool that might only ever be used as a kind of video filter for entertainment purposes -- I have a hard time imagining a cinematographer ever using it for serious purposes.


That's quite an insight!

Actually the authors of the seam carving paper went on to do just that [0]. From the abstract: "We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes."

[0] https://faculty.idc.ac.il/arik/SCWeb/vidret/index.html


Son of a gun, this is why I love HN. Thank you! And it turns out the results are shockingly good, far better than I expected. They have demo videos at:

https://faculty.idc.ac.il/arik/SCWeb/vidret/results/video_re...

My favorite is:

Original: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w...

Narrowed: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w...

Widened: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w...

Just wow.


Gladly! And yeah, the results really are quite good. This is why I like optimization problems - if you can formally capture what you want as an objective, and if you can find a way to optimize it, you can get surprisingly good results. Of course these are two very big IFs...


Interesting.


>You might need to build a bit more flexibility into it to allow for discontinuities generated from object movement and camera panning, but I wonder if anyone's tried to do something like that?

easier solution would probably be frame interpolation between the two seperate frames.


Interesting. A couple thoughts.... this might be a lot smoother on cartoons since there's usually less colors and noise overall.

It seems as though there's an additional effect (for extra... effect) when they scream. Not sure if that is a natural result of the content of the visual scene being processed, or if there's some sort of audio input into the visual processing, or if they manually/intentionally applied some sort of parameter change (at 0:29 and 0:38 in the video) that causes the video to get all chaotic.


Oh that's horrifying/skin-crawling. I don't know what's wrong with me, but I can't stand to watch it. :)


Very interesting, but it is hard to watch for me because of extreme shaking. Maybe some morphing would smooth the transitions.


Really cool. Do you know what he’s using to do the scaling?


In the video comments he includes a link to a tutorial but essentially he is dumping all the frames and then running a script to content aware scale down the frames 50% and then merges it all back together.


Damn that's trippy




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: