
Depixelizing Pixel Art (2011) - doppp
http://johanneskopf.de/publications/pixelart/index.html
======
maggit
I was excited about this when it was all the rage. It speaks to me as a
programmer. Some years later, I stumbled upon this excellent article by an
artist considering the problem from the art perspective:
[http://www.dinofarmgames.com/a-pixel-artist-renounces-
pixel-...](http://www.dinofarmgames.com/a-pixel-artist-renounces-pixel-art/)

It points to the problem with these upscaled versions: They aren't any good.
In the end, the artist demonstrates how to get good results, draw it again:
[http://www.dinofarmgames.com/wp-
content/uploads/2015/04/yosh...](http://www.dinofarmgames.com/wp-
content/uploads/2015/04/yoshi-hand-drawn.png)

Of course, the algorithmic approach lets you upscale any number of preexisting
games. However, I have concluded that I like the original art better.

~~~
yoz-y
In their comparison
[http://johanneskopf.de/publications/pixelart/supplementary/m...](http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html)
the Vector Magic results appeal the most to me precisely for this reason. For
most images they look more like an artists rendition than an attempt to trace
the image. Which is good.

~~~
LoSboccacc
when it works it's beautiful but worse cases like mario face are ruined. hq4x
isn't as pretty but seems more consistent over the whole gallery of examples
[http://johanneskopf.de/publications/pixelart/supplementary/m...](http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html)

------
corysama
Meanwhile, over in r/gamedev, someone just released a tool that does a good
job of pixel-arting regular images.

[http://pixelatorapp.com/](http://pixelatorapp.com/)

[https://www.reddit.com/r/gamedev/comments/7ylvt9/pixelator_a...](https://www.reddit.com/r/gamedev/comments/7ylvt9/pixelator_a_gamedev_tool_i_wrote_to_convert_any/)

~~~
donutmonger
I'd be interested to see the result of iterating on an image using these
tools. Like go from high res to pixel art, back to high res etc.

~~~
esotericsean
I remember back in like 2010, someone uploaded and ripped a YouTube video 1000
times and then edited the natural degradation of the compression. Went from
perfectly watchable video to horrible static-y mess.

Here it is!
[https://www.youtube.com/watch?v=icruGcSsPp0](https://www.youtube.com/watch?v=icruGcSsPp0)

~~~
citrusui
Thanks for the link!

This may sound crazy, but I wonder if there's a way to "automate" this
deconstruction with ffmpeg...

~~~
citrusui
Update: here's a one-liner that will do exactly that

 _ffmpeg -i input.mp4 -strict -2 -crf 51 output.mp4 && rm input.mp4 && mv
output.mp4 input.mp4_

For better results, run the above command ten times:

 _for ((n=0;n <10;n++)); do ffmpeg -i input.mp4 -strict -2 -crf 51 output.mp4
&& rm input.mp4 && mv output.mp4 input.mp4; done_

------
ktpsns
The longest discussion about this article on hacker news so far:
[https://news.ycombinator.com/item?id=2578706](https://news.ycombinator.com/item?id=2578706)

I wonder why Microsoft payed him to do develop this kind of algorithm.

~~~
jjcm
Probably as a research project to see if it's worth using in HD conversions of
old games, such as the Age of Empires remake that's dropping tomorrow. The
algorithm seems to work well on standalone sprites, but kinda merges in a
gooey way when two sprites are next to each other. The super mario example
from the previous discussion shows this slightly:
[http://files.jjcm.org/upscaling.mp4](http://files.jjcm.org/upscaling.mp4)

Very curious to see this as it progresses however.

~~~
c12
I'm still impressed with the super mario example, I can't personally see
anything negative about it and would enjoy playing the game with that "art
style."

It's not perfect but seeing the alternatives it is a huge step in the right
direction.

------
blt
waifu2x ([http://waifu2x.udp.jp/](http://waifu2x.udp.jp/)) is another often-
posted superresolution approach, focusing on upscaling low-res anime images
instead of extremely tiny pixel art. It uses a totally different learning-
based approach - however, their approach would not work with pixel art because
it depends on having a set of high resolution ground truth images, and
downsampling them to generate training data pairs. No such high-resolution
examples exist for pixel art.

~~~
murukesh_s
waifu2x seems to be based on Deep Convolutional Neural Networks, while the
article looks like a pure handwritten algorithm and at least in this case the
algorithm looks like a winner..

I am wondering whether in the future we may forget about writing algorithms
altogether and instead rely on ANNs to do tasks that could have done better
using hand written algorithms?

~~~
blt
You mean tasks like computing the rank of a matrix?
([http://oneweirdkerneltrick.com/rank.pdf](http://oneweirdkerneltrick.com/rank.pdf))
;)

Image super-resolution is an ill-posed inverse problem: many possible high-
resolution images would be reduced to the same low-resolution image. However,
some of the possibilities are more realistic than others. Consider a grey
pixel in a photograph from real life. It's more likely that it was downsampled
from a 2x2 block of grey pixels than from a 2x2 black-and-white checkerboard
pattern. We apply such knowledge by adding regularization to the problem, or
using a prior distribution in a Bayesian formulation.

Deep learning is very good at memorizing those priors by looking at real world
data. I think, in problems where the prior is important and complicated, data-
driven approaches have a big advantage over hand-engineered approaches. That
does not mean they will take over every kind of problem.

------
bhouston
Something incredibly similar is already built into the latest versions of
Snes9x.

hq4x and the #xBRZ algorithms are amazing and fast and look nearly identical
to these results.

~~~
freshyill
They compare it to hq4x in their supplementary materials. Theirs is better.

[http://johanneskopf.de/publications/pixelart/supplementary/i...](http://johanneskopf.de/publications/pixelart/supplementary/index.html)

~~~
lathiat
hq4x looks a lot better in the video comparison:
[http://johanneskopf.de/publications/pixelart/supplementary/v...](http://johanneskopf.de/publications/pixelart/supplementary/video_multi_comparison_4x_h264.mp4)

This method still wins out though.. but interesting never the less.

~~~
cooper12
I think hq4x has a slight edge because it doesn't go all the way trying to
make everything smooth, still preserving some pixelation. There's actually an
analog to this in denoising: if you denoise too heavily you end up losing
detail. Smart denoising manages to preserve edges, giving a better result. In
this case, pixels are the same as noise, undesirable, but containing useful
information.

------
0x00000000
Video is quite neat.
[http://johanneskopf.de/publications/pixelart/supplementary/v...](http://johanneskopf.de/publications/pixelart/supplementary/video_ours_4x_h264.mp4)

Reminds me of some old newgrounds flash animations and games

------
kelukelugames
This page is where all the fun is. But it's also very slow.

[http://johanneskopf.de/publications/pixelart/supplementary/m...](http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html)

~~~
leni536
Their method change too much on eyes and facial expressions in general. Also
that broad sword is really blunt.

When comparing only the 4x results hq4x wins half the time. On the other half
hq4x is a close second and generally looks alright. However in some cases the
vectorized ones are pretty bad.

------
bluedino
It looks...weird? Cartoony and almost Picasso-y.

Boo and Bowser look great. Mario and Yoshi look terrible. The detailed pixel
art and gradients just don't carry over. The shapes of the outlines do,
however.

Scanlines IMO look the best for video games, because it also tries to simulate
the medium they were originally presented in, not just the graphics data
behind the images

[https://imgur.com/wtxyA2d](https://imgur.com/wtxyA2d)

------
tomlogic
A real-world example of this technique shows up in ColorDMD, an LCD to replace
DMD displays used in pinball machines from the early 90's until just recently.
Most of the displays were 128x32 with 4 (and later 16) brightness levels.

ColorDMD first had artists create color overlays for the images and
animations, and then emulated the display with colored dots instead of the
red/orange color from the original DMD.

A later firmware release included upscaling, with some impressive results.
This thread on Pinside (a pinball discussion site) shows some examples from
the game "The Simpsons Pinball Party". I've linked directly to a comment that
shows an original rendered DMD frame along with colorized versions with and
without upscaling.

[https://pinside.com/pinball/forum/topic/colordmd-unveils-
new...](https://pinside.com/pinball/forum/topic/colordmd-unveils-new-hd-color-
upscaling#post-2218723)

------
jcims
I like the aesthetic of hq4x in a lot of them, it’s a little less loopy.

------
FreakyT
Every time I see this paper I'm always disappointed that there doesn't seem to
exist a usable (open source or otherwise) implementation of it anywhere.

------
malkia
My favourite all time game is Heroes of Might and Magic III. It was released
in 1999, and the art is still good, but that maybe because I'm so used to it.
Ubisoft did a great job, enhancing it for the HD release in 2015, there is
video showing it:
[https://www.youtube.com/watch?v=AX_h3RIbSKk](https://www.youtube.com/watch?v=AX_h3RIbSKk)

Though, one lesson I've learned (as a game developer myself), was that
enhancing may introduce more visual problems. On a port of a game I worked
(Playstation1 -> PC) we hired an artist to enhance the eye textures, but once
the team behind the original game saw this, they told us that since there was
no EYE animation in the game, and having such high-fidelity texture (for it's
time, back in 2000) made all the characters look like toys.. Now if we left
(and we did) the eye textures blurry, then your own eyes can't focus, and
accept this better (without eye animation). I'm not an artist (just a coder),
but this taught me "less is more".

------
oceanghost
Is there an implementation of this anywhere?

------
jehlakj
The biggest advantage of pixel art is that it takes up less resources to
create a solid retro type game. Animations suddenly become a lot easier as it
relies on the abstract nature of putting the imaginative efforts to the users.

By removing these artifacts, you essentially defeat that purpose. Animating
vectors would take a lot more time to create something that seems
professional, mostly because you have to be far more detail - analogous to
moving from 2d to 3d. I personally don’t see the use cases for this in gaming.

------
larsiusprime
How does this compare to XBR? [https://github.com/Treeki/libxbr-
standalone](https://github.com/Treeki/libxbr-standalone)

------
TruthSHIFT
This site has a great page that compares various algorithms:
[http://johanneskopf.de/publications/pixelart/supplementary/m...](http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html)
While none of them really produce usable results, I love the artistic quality
of the Vector Magic algorithm.

~~~
MaxBarraclough
> none of them really produce usable results

I don't know about that.

In my opinion the images that _aren 't_ game characters look best with their
vectoriser/upscaler. Cursor 1 and Setup, for instance.

~~~
mrguyorama
The chest blew my mind with how it seemed to accurately capture the shading
and reflectivity of everything

------
galago
Many type designers deal with this as part of their workflow if they work with
traditional drawing tools like pen and ink. My understanding is that for
classically designed fonts its often easier to redraw vector objects over the
scan than to clean up what the automatic tools produce. However, for certain
types of "distressed" fonts its a kind of necessity.

------
Tommakx
Does anyone have an open source implementation of this algoritm? Or a just a
demo? I would love to play around with it!

~~~
phnofive
[https://github.com/vvanirudh/Pixel-Art](https://github.com/vvanirudh/Pixel-
Art)

can't vouch for it yet, but will be giving it a try later.

~~~
cknight
A quick play around and I find myself suffering from this same issue:
[https://github.com/vvanirudh/Pixel-
Art/issues/4](https://github.com/vvanirudh/Pixel-Art/issues/4)

A shame, I'm really keen to spruce up some old 50x50px forum icons from where
I used to hang around back in the early naughties.

------
snvzz
As long as using the pixel art as-is remains an option.

So many classic games have been destroyed in this process.

------
tptacek
Is there something that gets a really good result going the opposite
direction?

~~~
tjr225
It seems pretty trivial but I could be way wrong as this is in no way my area
of expertise...but- pixelization filters in photoshop and other image editing
softwares have been around for a very long time.

Even logically it is an easy leap as they're(the software/filters) most likely
choosing the most common color in a given area and then converting that area
to a square of that color..

~~~
michaelmior
Just pixelating an image doesn't really give high quality pixel art. No
examples handy but I'm confident you'd see this if you tried to reverse the
images in the post with the approach you outlined.

------
microcolonel
Gotta say, I really like those PhotoZoom 4 results, aside from some of the
fringing artifacts. They seem to preserve a lot more of the detail, whereas
the presented approach focuses more on eliminating noise.

------
InternetUser
For the like pixel art, click here and scroll way down:
[http://noirlac.tumblr.com/archive](http://noirlac.tumblr.com/archive)

------
lwansbrough
Could you take this a step further? For example, a human knows this thing
isn't supposed to be lumpy. What would it take to make this look more like it
was interpreted by a human?

~~~
Nition
It would take an understanding of what the image actually represents.

------
thriftwy
Is it included into DosBox already?

hq4x is included I guess, but unfortunately when you get your games from e.g.
GOG, you have to tune DosBox manually every time.

------
ttoinou
I love this paper ! Has anyone come up with an interactive pixel art tool
using this algorithm ?

------
z3t4
Take an image, Resize 500% so you see the pixels. Save. Run it through the
algorithm. Repeat.

------
zhamisen
Is this work been already used in emulators filters?

~~~
danbolt
Filters have achcieved similar results, but not quite with the same technique!

Or, this algorithm wasn’t intended for real-time processing and creates an
adjacency graph of similar pixels. Once it’s done that, it will try to
fit/optimize splines over the regions. Emulator filters like hqx determine
each pixel’s “upscaled shape” by the colours of its neighbours, usually
indexed with a table.

------
knkhere
I want to go through the research paper but am unable to load or download it.
If anyone has managed to do the impossible please share a drive link. BTW
which journal is this published in?

------
drcode
A fantastic classic paper (but ancient history)

------
samstave
Isnt that called "anti Aliasing" :-)

(just kidding)

------
MBCook
(2011)

------
EgoIncarnate
(2011)

