Hacker News new | comments | show | ask | jobs | submit login
Show HN: Colormind – Color schemes via Generative Adversarial Networks (colormind.io)
399 points by Jack000 on Mar 26, 2017 | hide | past | web | favorite | 52 comments

Looks cool! I recently posted HuulaTypesetter (https://huu.la/ai/typesetter) which infers font sizes for web pages based on DOM context and CSSRooster which infers CSS class names based on the context. I'm really happy to see that there are more intelligence happening in the design world! The dream of replacing Web UI design with AI will one day come true!

All (deep) learning models suffers the issue of 'garbage in garbage out', so one way that could make the palettes more related to real world web designs is to learn the colors used in web designs directly instead of from photographs and movies since those data will has much more noise than good web designs (with video and images stripped out of course).

I'm not 100% fond of the layout suggestions, but I have to admit there's a peaceful balance in them.

Yeah, statistical models just give you the most probable result (in terms of the data we train it on), so if you are a creative person, you probably won't find it useful.

Sounds like it would be ideal for situations where you want the layout itself to be as neutral and out-of-the-way as possible, since these are judgement biased by normalisation.

Hmm, a model could also give you multiple very different designs to help you see more possibilities

sorry for the stupid question, but how do you "apply" color palettes? i mean, sure they look great as colorful columns, but how do you put them into UI?

let say Bootstrap. i tried using 1st color for button, 2nd color for "success" label, etc. it ended up ugly.

This is what I'd want as well. Bootstrap is a good example of color application as they define SASS variables to action states:

By default:

primary == dark blue success == green warning == orange info == light blue danger == red

Then the rest of the styling for the theme is calculated from that, so you have other elements defined as: whatever the primary color is but 40% darker.

What I'd love is a color palette picker that let me put in a particular color orange and then would spit out:

"Hey, here's the green, dark blue, light blue, and red that match that orange you chose."

Warning danger info success each carry some specific meaning. I would keep the colors standard (yellow red blue green) but match the saturation to the of the pallette.

Primary and secondary are usually for backgrounds of major elements, detail elements to draw attention to. The other 3 colors should be boring but pleasant. For subtle variation without drawing attention to the elements

currently the colors don't encode context, so you have to be a bit creative in applying them.

I am working on something that does encode context though.

You typically want to choose 1 or 2 of the colors to base 90% of the theme around, and then use use rest for accents. You usually can't just distribute them evenly or you'll end up with a mess.

If your web page consists of distinct "rows" or sections, try setting these colors to be the background colors of those rows.

I personally love color palettes generated by Coolors https://coolors.co/.

I just tested coolors out using a screen capture of Spotify desktop on MacOS and it didn't pickup the green color. I guess it's only picking out dominant colors

Colormind picked the green color on all attempts

Edit: after several more attempts coolors started picking up the green color

Test Image: http://i.imgur.com/6pMMV4W.jpg

Colormind interface is simpler and more user friendly I think, I was able to start generating color schemes right away, in coolors tho I had to watch a tutorial to learn what those buttons are.

I do love these color scheme generators, and perhaps this is a suggestion for improvement, the color vs. dominance of the color seems very important as well. That is, proportionally showing the palette based on how dominant that color should be in an overall end design

the issue I see is that most sites have a white/beige background, which could get repetitive for the purposes of design inspiration. Maybe train a separate model for bootstrap/direct insertion.

If you take into account various forms of colourblindness, could this be used to create colourblind friendly palettes that are still easy on the eye for everyone?

yeah definitely, it would just have to train on a (relatively large) sample of such palettes. I'll look into it!

I have protanomaly myself, so you'd definitely be making my day!

Look up "cubehelix" if you haven't heard of it already, it might give some inspirations.

Damn, this has become my new tool for palette generation. Good work!

Have you considered using fine art as training data, perhaps broken down into movements/motifs? In your blog posts you mentioned that not all photographs make for well chosen palettes, this could get around that as artists should have a great understanding of theory.

As far as the pixel-level errors in your palette training, would downscaling+blur solve this?

Really good results! Is there a quick way to copy and paste the hex values for the palette?

This is a very naive question, but considering the artifacts you have on the GAN-generated images, you are trying to generate the whole "palette image" at a certain resolution.

For which reasons did you not use a 5x1 pixel RGB image as the target, that would have been orders of magnitude faster to train?


that's a great question! The main reason is that pix2pix works at a set resolution by default, and changing it is non-trivial. I'm actually not sure how to structure the Unet for a 5x1 image - if you have any ideas let me know (I'm a beginner at this)

Speed improvement also doesn't scale linearly with input size. Eg. there's only a 5x improvement going from 256x256 to 256x1

Can you get a suggested color palette by API call?

sure, you could just use the same API the main website does, it's unauthenticated:

  curl 'http://colormind.io/api/' --data-binary '{"model":"default"}'

  curl 'http://colormind.io/api/' --data-binary '{"input":[[44,43,44],[90,83,82],"N","N","N"],"model":"default"}'
keep in mind this is hosted on a single Linode 4096, and there's no SLA or anything :] It's handling the HN spike pretty well so far though.

also the models are rotated daily (ie. different ones are available each day), with the exception of "default"

Why GANs for this?

My question as well. There are several other great implementations of a color pallet generator which use color theory rules. My bet is that the GAN, in this instance, boils down to little more than a pseudorandom number generator.

I think color theory alone isn't enough to produce good color palettes. eg. try going to https://color.adobe.com and choose one of the color rules, then go to https://color.adobe.com/explore and compare with user-contributed palettes. There's a huge difference imo.

As for why a GAN specifically, I talk a bit about that in one of the blog posts. Neural nets trained with L1/L2 loss tend to produce "averaged" colors, dull greys and browns. The learned loss from a GAN allows the output to take on more extreme values.

You could solve the averaging problem by applying the L1/L2 trained net to exponential color gradients rather than linear ones. By doing so you are telling the NN that humans care about exponential changes in color because that's how our eyes work. E.g. we notice if you double the amount of green but probably not if you just increment it.

I think there are rather simple solutions to these types of problems. But sometimes everything looks like a nail.

Why neural nets though? It seems like total overkill compared to simpler statistics.

GANs are good at generating plausible images given a set of examples, which is exactly what we're doing here.

A statistical approach might work, but you'd be essentially interpolating your existing samples. A GAN is capable of generating novel solutions.

The crux of the issue is that despite just having 5 colors, the solution space is huge (256^15), and most of the search space is junk. The difficult part is identifying what looks good, and classically this is just called color theory. The problem is that color theory is a leaky abstraction that doesn't capture what intuitively "looks good" and is largely used as a starting point for ideas rather than something that gives useable palettes. Hence the popularity of user-submitted and curated sites like coolors and the old kuler site.

You are right that 256^15 is a large search space. This number implies a 24bit color scheme.

    (2^24)^5 = 256^15
Many of the colors in the 2^24 range are similar which is why most of the linear search space is boring. Our eyes don't care much about #f67368 vs #f67468 but we do care about #f67368 vs #f6e668. Why?

  0x73 x 2 = 0xe6
Instead of blindly incrementing RGB color values, look at colors which differ by powers of 2. For example, choose 15 random values between 0 and 8. Let's call them c0 to c14. Then assign those numbers to your color pallet as 8-bit RGB values as follows.

    Color0: R=2^c0, G=2^c1, B=2^c2
    . . .
    Color5: R=2^c12, G=2^c13, B=2^c14
Rounding 2^8 down to 255. You will find non-boring color schemes because the search space better fits how our eyes see color. This new search space is only 9^15 which is just less than 2^48, far less than 2^120 and a lot more interesting.

You could also search the HSV color space in this way and you don't have to only look at powers of two. Consider for, example powers of 1.25. The point is that by organizing your search, you will find interesting colors pallets easily.

I'm betting that your GANs already encode some sort of exponential search based on how you trained them vs your initial attempt using L1/L2.

I can see a lot of utility in this. Great job!

Is this limited to generating palettes of 5 colors? Sometimes all 5 won't be needed, wouldn't that make the output a little less useful since the algorithm considers how each of the five interacts with the others.

If I only needed two colors, I wouldn't need those two colors to work against three other colors.

This looks great!

Did you have any thoughts with regards to releasing it as open source so that we could run our own instances with the various training sets? I note that you appear not to be monetizing it at all, so I figured I'd pop the question :)

the GAN is actually just pix2pix: https://github.com/phillipi/pix2pix I just resized the convolutional layers to 1px height since the vertical dimension isn't that relevant in this case.

I do have a few bash scripts for color extraction that I could upload to github, although they're pretty quick and dirty.

Would be super cool if you could pick your own ground truth data. Or predefined sets.

The torch7 models need to be loaded in memory, which is the main bottle neck. I've got about 30 or so trained models on disk and they take turns on the homepage.

Maybe there's a better way to do this that I'm not aware of though (Floydhub?)

These look cool! Is there a way to get.. more colors? Eg, i'd like to use this for some themes editor themes, but i need a couple backgrounds, foregrounds, etcetc. Maybe 10 colors would do, hard to say.


it's possible, but the main challenge would be to make a sufficient sample of 10-color palettes to train on.

I am making something that disambiguates between foreground and background, but still 5 colors for now.

I'd be nice if I could use the color locking thing on an uploaded image. I'd like to start with some base colors from a movie/game/etc and start exploring the color range with the AI.

Really great idea, and very practically useful! Thanks for sharing.

Very cool! One thing I noticed is that if you lock all colors except one and press 'generate' repeatedly, each call produces a different color. Why is that?

the model isn't totally deterministic, but aside from that the main palette is actually an ensemble of 12 separate models trained on slightly different hyper-parameters.

Colors are kind of subjective, and this is just meant to add some variety. eg. If you make a gradient out of 4 shades of the same color most of the models will complete the gradient, but a few of them would give you a contrasting shade.

> an ensemble of 12 separate models trained on slightly different hyper-parameters

Sounds like a PRNG to me.


They are a distinct art style that the algorithm was able to identify. I don't have a problem with it being used as an example. Why isn't it ok?

How are they not ok? Is the color palette used in propaganda eternally tainted in some way? The algorithm distilled the content into a representative palette, nothing more.

It's just showing the results when Mao-era propaganda posters are used as input.

Not sure I understand the concern. I guess "Communist" is a bit broad, but I don't see it as offensive.

Using deepmind for generating color palettes is somewhat of an overkill. It is really not that hard a problem.

It is very easy to programatically generate several color palettes from a main color.

It is also wasy to select one or more main colors from a picture.

Good point, just blindly select several "harmonious" colors are not hard at all. But as some folks pointed out, if we want to also consider the context (where the colors will be used, is it a button? or background? or text?), deep learning is definitely a good fit.

Do you have any evidence that ML is a good fit? You seem so sure.

the choice of a CNN makes more sense if you think of a color palette as an extremely low-resolution image rather than a basket of colors.

positioning in a palette matters because colors are perceived relatively.

But all of this can be easily modeled statically and has been. If you desire randomized results, consult a PRNG. This is more, "ML is really cool, what can we apply it to," than "ML provides a better solution to this problem." IMHO.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact