

Color Detection - Peroni
http://developers.lyst.com/data/images/2014/02/22/color-detection/

======
raarky
Nice.

I'm building something similar in my spare time.

It performs multi-color image searching on street wear. You can select a bunch
of colors and adjust the ratios.

Here's a _very_ early in-development version.
[http://www.inthatstyle.com/womens?colors=73a1d3,e84b34&ratio...](http://www.inthatstyle.com/womens?colors=73a1d3,e84b34&ratios=74,26)

(I'm a little worried about posting that on HN since it's unoptimized and will
probably crash.)

I'm currently working on skin detection & exclusion during the color detection
phase and am looking at using basic machine learning techniques. The key
challenge I'm facing is differences in skin tones.

~~~
IanCal
> I'm currently working on skin detection & exclusion during the color
> detection phase and am looking at using basic machine learning techniques.
> The key challenge I'm facing is differences in skin tones.

Try looking at the chromatic colour rather than the RGB values. You can get
extremely far with just this, most skin colours fall into one of two peaks
[0], no machine learning needed.

Once you've got this, edge detection & a few other bits should give you pretty
reliable skin blocks. I've used it a few times before. Here's a presentation I
did some years ago that I apparently still have on my desktop:
[http://files.figshare.com/1409002/1.pdf](http://files.figshare.com/1409002/1.pdf)
[1]

[0] [http://www-cs-
students.stanford.edu/~robles/ee368/skincolor....](http://www-cs-
students.stanford.edu/~robles/ee368/skincolor.html)

[1] Calvert, Ian (2014): Finger pointing detection. figshare.
[http://dx.doi.org/10.6084/m9.figshare.953171](http://dx.doi.org/10.6084/m9.figshare.953171)

EDIT - I'm sure there are many good approaches for this, and many fancy ones.
This is very simple and was researched/written purely for fun in a couple of
weeks.

EDIT 2 - The final slide shows the more interesting part, where you use edge
detectors to guide your estimation of what is inside or outside a shape. That
plus an adaptive threshold (designed to stop if the number of pixels included
jumped rapidly) got some good results, but I've not got the code any more.

~~~
raarky
awesome stuff :D Thanks for this. Will definitely look into some of that in
more detail.

Another tricky part of skin detection is false positives. ie, what if the
actual product is that color?

Some things I've noticed and will be taking into account are: Skin areas tend
to clump around the same locations in photos. The product is usually the focus
and skin is near the edges. Product types also tend to share similar photo
layouts. So with that, skin color in those zones score higher.

~~~
IanCal
> awesome stuff :D Thanks for this. Will definitely look into some of that in
> more detail.

No worries, hope it helps, it was just a quick project back in the day at uni
that ended up working a lot better than I expected.

Give me a shout if you want any work done on it (my email address is in my
profile).

> Some things I've noticed and will be taking into account are: Skin areas
> tend to clump around the same locations in photos. The product is usually
> the focus and skin is near the edges. Product types also tend to share
> similar photo layouts. So with that, skin color in those zones score higher.

This kind of thing will really help you, small bits of knowledge about the
specifics drastically simplify the problem. For example, you can estimate the
skin tone by roughly segmenting the image into possible skin/not skin with the
approach above, then look at segments which are more likely to be skin because
of their positioning you can narrow your accepted parameters and hopefully
help distinguish between the two.

Identification of unusual edges/shapes can help too, to classify regions as
skin/not skin.

Beyond that, starting to look at estimations of pose to help guess the
underlying shape (since you know it's on humans you can make a lot of
assumptions).

Also, since you're detecting colours, mistaking very similarly coloured skin
as the product wouldn't change your results much :)

visual_cat posted a really nice site with some of the state of the art:
[http://clothingparsing.com/](http://clothingparsing.com/)

------
strebler
Color is actually much more difficult than most people realize! Even CIE2000
color distance still has fundamental "perceptual distance" problems,
especially comparing non-saturated with saturated colors (although that's a
blog post in its own).

Here's our engine linking "real world" photos into the Macys catalogue (based
on color, shape, texture):
[http://www.pcsso.com/demo/macys.htm](http://www.pcsso.com/demo/macys.htm)

~~~
jacobolus
It is impossible to create a three dimensional color space in which Euclidean
distance corresponds directly to perceived color difference. There are dozens
of papers on this topic, for anyone who wants some fun evening reading.

~~~
strebler
Yes, absolutely, which (I'm assuming) is why CIE2K difference is not-quite-
euclidean. My opinion is the we still need to find a better difference metric,
rather than improving the color space.

On that front, it's amazing to see how much of an effect "switching color
spaces" can have on many algorithms.

------
vision_cat
Nice post. I'm a researcher in computer vision, so I'm glad to see that people
find these problems interesting. It's often easy to underestimate how
difficult vision problems are, since they seem so natural to us.

I think the recent posts by lyst (this color naming post, and the previous
background subtraction post) are great introductions to problems in computer
vision. However, there is a lot more sophisticated work out there, and the
techniques used in these posts are decades old (and contain some errors).

If you liked this post, you should check out this more recent demo
[http://clothingparsing.com/](http://clothingparsing.com/)

If you have a computer vision problem in mind, you can gauge the state-of-the-
art, by searching recent papers on Google Scholar. Even if you're unfamiliar
with the jargon, the introduction and conclusion of a paper can give you an
intuition about the problem being solved and the steps the authors are
proposing.

If you want to learn more about computer vision and its details, I highly
recommend checking out online materials such as

[http://cs.brown.edu/courses/cs143/](http://cs.brown.edu/courses/cs143/)

[http://www.cs.cornell.edu/courses/cs4670/2013fa/lectures/lec...](http://www.cs.cornell.edu/courses/cs4670/2013fa/lectures/lectures.html)

[http://szeliski.org/Book/](http://szeliski.org/Book/)

~~~
ejlb
Thanks for all the great resources. I think it must be obvious that we aren't
image processing researchers! My background is combinatorial graph theory. But
these HN comments have given us so much valuable material for improvements.

------
TheZenPsycho
"To speed up the color difference calculations we decided to vectorised
deltaE. The standard deltaE functions are already implemented in the colormath
python package. We ported these to numpy and added a function to take a numpy
array of Lab coordinates. When used with large data-sets the vectorised
implementation is 25-180 times faster depending on which distance function is
used. The vectorised delta E functions are now available in the colormath
package and can be used in the following manner."

Uhm.. wouldn't it have been easier to just run a voronoi algorithm on the data
set in the labspace? then you have a lookup table/cube. For a paltry 6mb of
space, lookup becomes instantaneous. It's what voronoi is _for_.

~~~
jacobolus
Even better would be to take the full data set of all the survey responses,
and use it to generate the partition of CIELAB space, instead of just using
the single set of coordinates per name that Randall generated from those. But
it would be more work.

Edit: do note they’re using a more complicated distance definition such that a
straight Voronoi diagram in CIELAB space is not quite the same as their
result. But the difference is so slight as to not matter, so your suggested
solution would be substantially better than theirs.

------
jacobolus
I’d recommend using the ISCC-NBS system of color designation in cases like
this. You’ll have to convert your colors to Munsell color space, and then use
a lookup table to get the associated name. But it’s a pretty reasonable
system.

Ideally you could get some kind of actual color measurements from clothes
directly, instead of relying on (often very inaccurate) digital images, but
maybe the’s too much to hope for.

------
Bjoern
Nice post!

Made me think we should write something similar for our approach, which is a
bit more complex though leveraging semi-supervised learning, dynamic
estimations and normalized color calculations with statistical bias.

Here is some simple demo if you are interested, where we translated it back
also into color names to allow text search for estimated parent and real
color.

[http://text.clothesnetwork.com/](http://text.clothesnetwork.com/)

[http://text.clothesnetwork.com/tags](http://text.clothesnetwork.com/tags)

------
tsumnia
Really cool techniques (this is the background removal one). I'm curious on
how well this would work on hair color detection. My thoughts are that you
could run face detection, and since detection almost always (anecdotal) find
just the face, not the outline of the face, you could just grab around the
face for hair.

Does someone at Lyst have a background in image manipulation? If so, I can't
wait to see what else you guys are working on.

~~~
ejlb
Most of the guys here have more experience with text processing but we are
expanding into image processing. I think we could definitely detect hair
colour but it isn't that useful for Lyst. Gender identification on the other
hand...(watch this space)

~~~
adyus
That's a very sensitive area to be heading into. Any assumptions you might try
to make will find themselves with counter-examples quite often.

A better direction to move in might be to minimize gender differences as much
as possible (although that's hard for marketing to stomach).

I mean, would one try not to recommend pants for girls or skirts for boys
(kilts)? It's tricky...

------
netw1z
haha image is labeled pretty white girl.

[http://developers.lyst.com/images/color_detection/pretty-
whi...](http://developers.lyst.com/images/color_detection/pretty-white-
girl.png)

racists! saved for later.

~~~
leeoniya
the only thing that's white is her sweater

------
kimonos
Very interesting! Thanks for sharing!

