
Comixify: Transform video into a comics - hardmaru
https://comixify.ii.pw.edu.pl
======
latexr
Site is slow right now, but got results after some waiting.

From this video[1] (one of their given demos), it generated this result[2].
From this other video[3] (another of their demos), it generated this
result[4]. All used the default settings.

Was frankly disappointed. They mention “neural style transfer” and whatnot,
but the results looks only slightly better than what you’d get by applying
default photoshop filters from a decade ago, probably longer. ImageMagick can
also achieve something similar[5].

The extracted frames, including their sizes and positioning, don’t look like
anything special. The final layout was the same for both tries, meaning
they’re probably not doing anything to detect/convey a better sense of the
story. In that sense, the result is as much a comic as any other random
sequence of photographs (comics don’t need to be drawings) divided into
squares.

It’s a neat demo, but I’m failing to understand why it is part of a larger
academic paper. Seems like the whole effect could have been achieved over a
decade ago with a bash script glueing ffmpeg and imagemagick.

[1]: [https://youtu.be/CvvAftMZYKM](https://youtu.be/CvvAftMZYKM)

[2]: [https://imgur.com/a/rbrQBbx](https://imgur.com/a/rbrQBbx)

[3]: [https://youtu.be/pvAhRcUofDk](https://youtu.be/pvAhRcUofDk)

[4]: [https://imgur.com/a/kZm2M5w](https://imgur.com/a/kZm2M5w)

[5]:
[http://www.fmwconcepts.com/imagemagick/cartoon/index.php](http://www.fmwconcepts.com/imagemagick/cartoon/index.php)

~~~
andrusza2
Hi, I am one of co-authors of the paper.

Thank you for your interest in our demo.

Regarding the points you raise in your comment:

1\. Layout

You are right that we use static layout for all comics we generate. Changing
that has a high priority on our todo list, we are currently working on more
dynamic layout generation that takes into account what we know about the story
and which frames are more important than others.

2\. Style transfer

The method we use gives the most pleasant results of all obtained with various
style transfer methods applied to comics - see our previous work on comparison
between them here:
[https://arxiv.org/abs/1809.01726](https://arxiv.org/abs/1809.01726).

3\. Story sense

(>probably not doing anything to detect/convey a better sense of the story<)
We use Video Summarization model in our pipeline, however as mentioned above,
we use static layouts and therefore this is not always apparent.

Overall, this demo is only a starting point. We are working on adding
automatic speech recognition and text generation to introduce dialogs into our
comics. And we have many other improvements planned. Stay tuned and if you
have any other comments feel free to send us an email to any of the addresses
from the paper: 243678@pw.edu.pl, 280814@pw.edu.pl, 261406@pw.edu.pl,
t.trzcinski@ii.pw.edu.pl.

------
neotek
The site seems to be suffering from the HN hug of death, unfortunately.

------
liquidmetal
Something similar we did for an internal Microsoft hackathon in 2017.
[https://utkarshsinha.com/blog/microsoft-oneweek-
hackathon-20...](https://utkarshsinha.com/blog/microsoft-oneweek-
hackathon-2018/)

------
ZeroGravitas
This reminded me of this:

BBC iPlayer for Kindle

[http://www.frisnit.com/iplayer-for-kindle/](http://www.frisnit.com/iplayer-
for-kindle/)

------
hardmaru
link to their paper:
[https://arxiv.org/abs/1812.03473](https://arxiv.org/abs/1812.03473)

~~~
lsh
doesn't have much more than a link back to the site.

google cache:
[https://webcache.googleusercontent.com/search?q=cache:c8dVMl...](https://webcache.googleusercontent.com/search?q=cache:c8dVMlkJspYJ:https://comixify.ii.pw.edu.pl/+&cd=1&hl=en&ct=clnk&gl=au)

edit: my mistake, it has the full PDF there

------
vackosar
no subtitles or audio integration yet

