
Colorizing black and white photos with deep learning - saip
https://blog.floydhub.com/colorizing-b&w-photos-with-neural-networks/
======
vwcx
As a professional photo editor and historian, colorized photos really agitate
me. I'm all for the creation of new ways to get people to engage with
historical primary documentation, but the nuance that these colorizations are
interpretations gets lost immediately.

Do an image search for "D-Day in color" and try to tell me which results are
original color negatives and which are colorizations made by teenagers.

I'm also a little confused as to why colorizations always aim to restore color
to the equivalent of a faded color negative, with muted tonality and grain.
Human logic is funny.

~~~
WalterBright
I like colorized historical photos. It brings them to life in a remarkable
way. I'd like to see the old B+W movies colorized (the Ted Turner ones don't
count, as they were done very poorly).

Sure, the colors will never be exact, because we don't know what the original
colors were. But that doesn't matter in any material way.

~~~
anigbrowl
Having worked on movies shot on video, color film, and B+W film, the lighting
and design choices are very different. Colorizing BW movies makes some sense
for historical footage where that was the only available option, like
newsreels or very ear;y films, but for anything with aesthetic choices
involved (German expressionism onwards), it's no more appropriate than trying
to figure out what Picasso's models 'really' looked like. Artists are not that
concerned with fidelity, but with pushing the salient characteristics of their
available or chosen medium in support of a particular artistic vision. We are
not merely recording; as soon as there is more than one option available
(whether that's paint on canvas or different kinds of film stock) we start
playing with the possibilities. When you begin colorizing you risk
compromising that.

------
terrabytes
This is refreshing. I’ve been learning machine learning through Kaggle.
Recently and I’m a bit tired with the “tuning hyperparameter” culture. It
rewards people that have the pockets to spend on computing power and the time
to try every parameter. I’m starting to find problems that don’t have a simple
accuracy metric more interesting. It forces me to understand the problem and
think in new ways, instead of going down a checklist of optimizations.

~~~
wrangler99
Ha, it reminds me of what Andrej Karpathy‏ said "Kaggle competitions need some
kind of complexity/compute penalty. I imagine I must be at least the millionth
person who has said this." It would be interesting to collaborate/compete on
more creative tasks and have different metrics for success.

[1]
[https://twitter.com/karpathy/status/913619934575390720](https://twitter.com/karpathy/status/913619934575390720)

~~~
ReDeiPirati
So true. Another reason to put constraints in Kaggle competition is due to
production environment. How many winner models have been used in production? I
suspect this number is near zero. High accuracy with a delayed time makes a
ML/DL artefact not usable in production, because from users point of view
speed is much more valuable than the difference between 97% and 98% in
accuracy.

------
iluvmylife
The averaging problem in colorization is interesting. If it learns that an
apple can be red, green and even yellow - how does it know how to color it?

A HN user in an earlier thread suggested to use a fake/real colorization
classifiers as a loss function. [1] But I still feel that it would not solve
the averaging problem. It would hop between different colors and probably
converge to brown. I haven’t come across a plausible solution so far. [1]
[https://news.ycombinator.com/item?id=10864801](https://news.ycombinator.com/item?id=10864801)

~~~
zardo
>But I still feel that it would not solve the averaging problem. It would hop
between different colors and probably converge to brown.

At least to the extent that GANs work, it works. They will alternate between
the observed colours based on the noise vector. They do not simply converge to
averages, because the discriminator easily recognizes brown apples as fakes.

------
strin
Live demo!
[http://beta.moxel.ai/models/strin/colorization/latest](http://beta.moxel.ai/models/strin/colorization/latest)

------
flsantos
Interesting! If nowadays, pictures are colorized by hand in photoshop, it
wouldn't be practical to colorize a full black and white movie. I guess this
deep learning approach would solve this problem and colorize old black and
white classic movies.

~~~
saip
Agreed. I imagine this has applications in compression as well. You could
stream a movie (or a football game) in black and white and enable each device
to color it on the spot. A similar technique could also be done for HD/3D/VR.

~~~
zardo
Yes, you provide a handful of full data keyframes and reconstruct the details
of the stream from the middle out.

~~~
kiliankoe
That middle out compression has some fantastic Weissman scores I believe.

------
whatrocks
I'd like to train this on color comic strips and then run something
traditionally black and white like xkcd through it. Seems like it could make
the colorization part of hand drawn animation much easier.

~~~
web007
You'll probably want something closer to a GAN like pix2pix -
[https://phillipi.github.io/pix2pix/](https://phillipi.github.io/pix2pix/)

An example implementation would look something like edges2cats
[https://affinelayer.com/pixsrv/](https://affinelayer.com/pixsrv/)

~~~
whatrocks
edges2cats is already way too fun. Thank you for the link!

------
quotemstr
The suggestion of using a classification network as a loss function is
brilliant!

I love how we can, in general, elevate the sophistication of ML models by
having different models interact and train each other.

------
bootcat
Thank you, really good read and great product idea !!

------
ReDeiPirati
It could be really interesting if it could return different coloured versions
and provide a way to explore this different style.

~~~
houqp
What's even cooler is adding support for human annotations so users can
selectively give colorization hints for different parts of the image to
customize the output.

------
doppenhe
colorization applied to video: [http://demos.algorithmia.com/video-
toolbox/](http://demos.algorithmia.com/video-toolbox/)

