Hacker News new | past | comments | ask | show | jobs | submit login

For training, would it be useful to stabilize the footage first?



Stabilization appears to be a subset of a literally wider, but more rewarding, challenge: reconstructing the whole area that is scanned by the camera. It could be better to work on that challenge, not on simple stabilization.

That's similar to how the human visual system 'paints' a coherent scene from a quite narrow field of high-resolution view, with educated guesses and assumptions


https://vidpanos.github.io/

There are other recent ones that do a new camera from any vantage point, not just rotation+fov changes like the above as well. But they still might want stabilized video as the baseline input if they don't already use it.

Besides saccades and tracking, your eyes also do a lot of stabilization, even counter rotating on the roll axis as you lean your head to the side. I'm not sure if they roll when tracking a subject that rolls, I would think not common enough to need to be a thing.


Thanks - that link is very interesting. You can see some distortion and 'hallucination', which would be a risk with my suggestion. Their video output is great work, but the far end of the fence at the right hand side glitches and vanishes at about 4-5 sec mark, for instance


I guess yes. Having worked on video processing, it's always better if you can stabilize because it significantly reduces the number of unique tokens, which would be even more useful for the present method. However, you probably lose in generalization performance and not all videos can be stabilized.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: