
Show HN: Colormind – Color schemes via Generative Adversarial Networks - Jack000
http://colormind.io
======
huula
Looks cool! I recently posted HuulaTypesetter
([https://huu.la/ai/typesetter](https://huu.la/ai/typesetter)) which infers
font sizes for web pages based on DOM context and CSSRooster which infers CSS
class names based on the context. I'm really happy to see that there are more
intelligence happening in the design world! The dream of replacing Web UI
design with AI will one day come true!

All (deep) learning models suffers the issue of 'garbage in garbage out', so
one way that could make the palettes more related to real world web designs is
to learn the colors used in web designs directly instead of from photographs
and movies since those data will has much more noise than good web designs
(with video and images stripped out of course).

~~~
agumonkey
I'm not 100% fond of the layout suggestions, but I have to admit there's a
peaceful balance in them.

~~~
huula
Yeah, statistical models just give you the most probable result (in terms of
the data we train it on), so if you are a creative person, you probably won't
find it useful.

~~~
vanderZwan
Sounds like it would be ideal for situations where you want the layout itself
to be as neutral and out-of-the-way as possible, since these are judgement
biased by normalisation.

------
wiradikusuma
sorry for the stupid question, but how do you "apply" color palettes? i mean,
sure they look great as colorful columns, but how do you put them into UI?

let say Bootstrap. i tried using 1st color for button, 2nd color for "success"
label, etc. it ended up ugly.

~~~
michaelbuckbee
This is what I'd want as well. Bootstrap is a good example of color
application as they define SASS variables to action states:

By default:

primary == dark blue success == green warning == orange info == light blue
danger == red

Then the rest of the styling for the theme is calculated from that, so you
have other elements defined as: whatever the primary color is but 40% darker.

What I'd love is a color palette picker that let me put in a particular color
orange and then would spit out:

"Hey, here's the green, dark blue, light blue, and red that match that orange
you chose."

~~~
crucialfelix
Warning danger info success each carry some specific meaning. I would keep the
colors standard (yellow red blue green) but match the saturation to the of the
pallette.

Primary and secondary are usually for backgrounds of major elements, detail
elements to draw attention to. The other 3 colors should be boring but
pleasant. For subtle variation without drawing attention to the elements

------
forlorn
I personally love color palettes generated by Coolors
[https://coolors.co/](https://coolors.co/).

~~~
component
I just tested coolors out using a screen capture of Spotify desktop on MacOS
and it didn't pickup the green color. I guess it's only picking out dominant
colors

Colormind picked the green color on all attempts

Edit: after several more attempts coolors started picking up the green color

Test Image: [http://i.imgur.com/6pMMV4W.jpg](http://i.imgur.com/6pMMV4W.jpg)

------
whistlerbrk
I do love these color scheme generators, and perhaps this is a suggestion for
improvement, the color vs. dominance of the color seems very important as
well. That is, proportionally showing the palette based on how dominant that
color should be in an overall end design

~~~
Jack000
the issue I see is that most sites have a white/beige background, which could
get repetitive for the purposes of design inspiration. Maybe train a separate
model for bootstrap/direct insertion.

------
vanderZwan
If you take into account various forms of colourblindness, could this be used
to create colourblind friendly palettes that are still easy on the eye for
everyone?

~~~
Jack000
yeah definitely, it would just have to train on a (relatively large) sample of
such palettes. I'll look into it!

~~~
vanderZwan
I have protanomaly myself, so you'd definitely be making my day!

Look up "cubehelix" if you haven't heard of it already, it might give some
inspirations.

------
SuperPaintMan
Damn, this has become my new tool for palette generation. Good work!

Have you considered using fine art as training data, perhaps broken down into
movements/motifs? In your blog posts you mentioned that not all photographs
make for well chosen palettes, this could get around that as artists should
have a great understanding of theory.

As far as the pixel-level errors in your palette training, would
downscaling+blur solve this?

------
r0muald
Really good results! Is there a quick way to copy and paste the hex values for
the palette?

------
Despoisj
This is a very naive question, but considering the artifacts you have on the
GAN-generated images, you are trying to generate the whole "palette image" at
a certain resolution.

For which reasons did you not use a 5x1 pixel RGB image as the target, that
would have been orders of magnitude faster to train?

Thanks!

~~~
Jack000
that's a great question! The main reason is that pix2pix works at a set
resolution by default, and changing it is non-trivial. I'm actually not sure
how to structure the Unet for a 5x1 image - if you have any ideas let me know
(I'm a beginner at this)

Speed improvement also doesn't scale linearly with input size. Eg. there's
only a 5x improvement going from 256x256 to 256x1

------
fgGAMI
Can you get a suggested color palette by API call?

~~~
Jack000
sure, you could just use the same API the main website does, it's
unauthenticated:

    
    
      curl 'http://colormind.io/api/' --data-binary '{"model":"default"}'
    
      curl 'http://colormind.io/api/' --data-binary '{"input":[[44,43,44],[90,83,82],"N","N","N"],"model":"default"}'
    

keep in mind this is hosted on a single Linode 4096, and there's no SLA or
anything :] It's handling the HN spike pretty well so far though.

also the models are rotated daily (ie. different ones are available each day),
with the exception of "default"

------
imh
Why GANs for this?

~~~
jcoffland
My question as well. There are several other great implementations of a color
pallet generator which use color theory rules. My bet is that the GAN, in this
instance, boils down to little more than a pseudorandom number generator.

~~~
Jack000
I think color theory alone isn't enough to produce good color palettes. eg.
try going to [https://color.adobe.com](https://color.adobe.com) and choose one
of the color rules, then go to
[https://color.adobe.com/explore](https://color.adobe.com/explore) and compare
with user-contributed palettes. There's a huge difference imo.

As for why a GAN specifically, I talk a bit about that in one of the blog
posts. Neural nets trained with L1/L2 loss tend to produce "averaged" colors,
dull greys and browns. The learned loss from a GAN allows the output to take
on more extreme values.

~~~
imh
Why neural nets though? It seems like total overkill compared to simpler
statistics.

~~~
Jack000
GANs are good at generating plausible images given a set of examples, which is
exactly what we're doing here.

A statistical approach might work, but you'd be essentially interpolating your
existing samples. A GAN is capable of generating novel solutions.

The crux of the issue is that despite just having 5 colors, the solution space
is huge (256^15), and most of the search space is junk. The difficult part is
identifying what looks good, and classically this is just called color theory.
The problem is that color theory is a leaky abstraction that doesn't capture
what intuitively "looks good" and is largely used as a starting point for
ideas rather than something that gives useable palettes. Hence the popularity
of user-submitted and curated sites like coolors and the old kuler site.

~~~
jcoffland
You are right that 256^15 is a large search space. This number implies a 24bit
color scheme.

    
    
        (2^24)^5 = 256^15
    

Many of the colors in the 2^24 range are similar which is why most of the
_linear_ search space is boring. Our eyes don't care much about #f67368 vs
#f67468 but we do care about #f67368 vs #f6e668. Why?

    
    
      0x73 x 2 = 0xe6
    

Instead of blindly incrementing RGB color values, look at colors which differ
by powers of 2. For example, choose 15 random values between 0 and 8. Let's
call them c0 to c14. Then assign those numbers to your color pallet as 8-bit
RGB values as follows.

    
    
        Color0: R=2^c0, G=2^c1, B=2^c2
        . . .
        Color5: R=2^c12, G=2^c13, B=2^c14
    

Rounding 2^8 down to 255. You will find non-boring color schemes because the
search space better fits how our eyes see color. This new search space is only
9^15 which is just less than 2^48, far less than 2^120 and a lot more
interesting.

You could also search the HSV color space in this way and you don't have to
only look at powers of two. Consider for, example powers of 1.25. The point is
that by organizing your search, you will find interesting colors pallets
easily.

I'm betting that your GANs already encode some sort of exponential search
based on how you trained them vs your initial attempt using L1/L2.

------
vtange
I can see a lot of utility in this. Great job!

Is this limited to generating palettes of 5 colors? Sometimes all 5 won't be
needed, wouldn't that make the output a little less useful since the algorithm
considers how each of the five interacts with the others.

If I only needed two colors, I wouldn't need those two colors to work against
three other colors.

------
chias
This looks great!

Did you have any thoughts with regards to releasing it as open source so that
we could run our own instances with the various training sets? I note that you
appear not to be monetizing it at all, so I figured I'd pop the question :)

~~~
Jack000
the GAN is actually just pix2pix:
[https://github.com/phillipi/pix2pix](https://github.com/phillipi/pix2pix) I
just resized the convolutional layers to 1px height since the vertical
dimension isn't that relevant in this case.

I do have a few bash scripts for color extraction that I could upload to
github, although they're pretty quick and dirty.

------
auganov
Would be super cool if you could pick your own ground truth data. Or
predefined sets.

~~~
Jack000
The torch7 models need to be loaded in memory, which is the main bottle neck.
I've got about 30 or so trained models on disk and they take turns on the
homepage.

Maybe there's a better way to do this that I'm not aware of though (Floydhub?)

------
notheguyouthink
These look cool! Is there a way to get.. more colors? Eg, i'd like to use this
for some themes editor themes, but i need a couple backgrounds, foregrounds,
etcetc. Maybe 10 colors would do, hard to say.

Thoughts?

~~~
Jack000
it's possible, but the main challenge would be to make a sufficient sample of
10-color palettes to train on.

I am making something that disambiguates between foreground and background,
but still 5 colors for now.

------
kr0
I'd be nice if I could use the color locking thing on an uploaded image. I'd
like to start with some base colors from a movie/game/etc and start exploring
the color range with the AI.

------
mgalka
Really great idea, and very practically useful! Thanks for sharing.

------
thwd
Very cool! One thing I noticed is that if you lock all colors except one and
press 'generate' repeatedly, each call produces a different color. Why is
that?

~~~
Jack000
the model isn't totally deterministic, but aside from that the main palette is
actually an ensemble of 12 separate models trained on slightly different
hyper-parameters.

Colors are kind of subjective, and this is just meant to add some variety. eg.
If you make a gradient out of 4 shades of the same color most of the models
will complete the gradient, but a few of them would give you a contrasting
shade.

~~~
jcoffland
> an ensemble of 12 separate models trained on slightly different hyper-
> parameters

Sounds like a PRNG to me.

------
maxmcorp
Using deepmind for generating color palettes is somewhat of an overkill. It is
really not that hard a problem.

It is very easy to programatically generate several color palettes from a main
color.

It is also wasy to select one or more main colors from a picture.

~~~
huula
Good point, just blindly select several "harmonious" colors are not hard at
all. But as some folks pointed out, if we want to also consider the context
(where the colors will be used, is it a button? or background? or text?), deep
learning is definitely a good fit.

~~~
jcoffland
Do you have any evidence that ML is a good fit? You seem so sure.

~~~
Jack000
the choice of a CNN makes more sense if you think of a color palette as an
extremely low-resolution image rather than a basket of colors.

positioning in a palette matters because colors are perceived relatively.

~~~
jcoffland
But all of this can be easily modeled statically and has been. If you desire
randomized results, consult a PRNG. This is more, "ML is really cool, what can
we apply it to," than "ML provides a better solution to this problem." IMHO.

