
Smartcrop.js – content-aware image cropping in JavaScript - rayshan
https://github.com/jwagner/smartcrop.js/
======
vortico
Really awesome, and the test cases look just as good as I could do.

I would warn web designers to not blindly apply this to everything though. It
scans all the pixels of an image, which can take up to 100ms each, especially
on mobile devices. A good use case would be a file upload box with a
suggestion to crop the image upon upload.

~~~
nacs
It can be used as a node.js module on the server-side which I suspect would be
the better place to run this kind of thing.

~~~
BinaryIdiot
I'm not so sure you'd want to throw away all of that processing power on the
client though. Running some performance testing on different types of devices
and figuring out which ones can handle it fine to good enough (especially if
you can do it in the background with a web worker) while sending really
horribly performing devices right to the backend.

------
zappo2938
Cropping images is a massive problem for social media. Here is a talk from
2013 by Christopher Chedeau a front end engineer at Facebook describing some
of the problems with their image layout algorithms.[1]

Initially, they tried to solve the problem by getting users to tag people
inside the image and then use location of the tags as parameters to crop. If
someone is tagged in a photo, Facebook makes sure that person is always inside
of the cropped version.

Here is the write up from Christopher's blog.[2]

Was Instagram only using square images at one time? That would have been a
brilliant way to have solved this problem.

1\. [http://blog.vjeux.com/2014/image/image-layout-algorithms-
htm...](http://blog.vjeux.com/2014/image/image-layout-algorithms-
html5devconf.html)

2\. [http://blog.vjeux.com/2012/image/best-cropping-
position.html](http://blog.vjeux.com/2012/image/best-cropping-position.html)

~~~
chipperyman573
>Initially, they tried to solve the problem by getting users to tag people
inside the image and then use location of the tags as parameters to crop. If
someone is tagged in a photo, Facebook makes sure that person is always inside
of the cropped version.

Do they still do this? A lot of the times people aren't tagged by where their
face is, but just tagged off to the side or something. I feel like this
wouldn't work in a lot of cases.

~~~
zappo2938
Now when I upload an image to Facebook it automatically uses facial
recognition to tag my friends. In the video from 2013 he mentions they had
already solved the problem using heuristics to find faces in images. Before
that, circa 2012, they might have been relying on people to tag their photos.

------
vjeux
If you are interested in a similar approach by a Facebook developer:
[http://blog.vjeux.com/2012/image/best-cropping-
position.html](http://blog.vjeux.com/2012/image/best-cropping-position.html)

------
emehrkay
Damn this is cool. It's kinda amazing that it is all done in a few hundred
lines of code. I see there is a skinColor method and setting defined as

    
    
        skinColor: [0.78, 0.57, 0.44]
    

I was curious how it worked with darker skin (I admittedly don't understand
what the numbers mean without further analysis), and It came out pretty well
(it may default on lighter skin, i don't know)

[http://imgur.com/a/CKUIo](http://imgur.com/a/CKUIo)

images found on google

~~~
mchahn
Almost all skin colors, e.g. black and white, have the same chroma. Only the
brightness changes. I have worked on video projects that took advantage of
this to have skin-color auto-adjustment.

------
jessedhillon
This is called "salient region detection" and some current approaches (there
are many) include detecting the contrast between each pixel and the global or
regional average color or luminosity. Areas of high contrast are likely to be
regions which are considered interesting. Once you have those regions, you
would have to have a separate algorithm which maximizes the placement of a
rectangle (the crop) to get the greatest coverage of "interestingness".

You could also combine this with face detection, so that a picture of someone
in a bikini doesn't end up cropping just to their midsection, since going by
surface area, the torso could have more high-contrast pixels than the face.

Here's one approach which has open source C++ code:
[http://mmcheng.net/effisalobj/](http://mmcheng.net/effisalobj/)

------
sam_goody
Two others in this space: [http://thumbor.org](http://thumbor.org)
[http://magickly.afeld.me](http://magickly.afeld.me)

But neither seems to get as much love as they need, so always good to have
more players.

~~~
thedz
thumbor is an on demand service that hosts and responds to image requests.
this is a JS based smart cropper. they're not quite the same thing.

~~~
slig
thumbor has a similar feature built in.

------
RichWalton
Awesome work.

As it happens I'm working on an image cropping front end (using CropperJS [1])
- I'm going to integrate this so that the initial crop selection is set using
the results from SmartCrop.

Thanks again.

[1]
[https://github.com/fengyuanchen/cropperjs](https://github.com/fengyuanchen/cropperjs)

------
rateofclimb
Very cool. The Ken Burns effect application of the algorithm is particularly
impressive.

------
IgorPartola
I've actually been looking for something related. I need to quickly classify
if an image contains a face and also if it contains any text. The former seems
to be relatively straightforward, but I haven't found anything for detecting
text, only OCR'ing it which I don't need. Anyone seen anything like this?

~~~
cokernel
I haven't tested this, but the package [https://github.com/subokita/Robust-
Text-Detection](https://github.com/subokita/Robust-Text-Detection) apparently
implements methods described in the paper "Robust Text Detection in Natural
Images with Edge-Enhanced Maximally Stable Extremal Regions." by Chen,
Huizhong, et al.
([http://web.stanford.edu/~hchen2/papers/ICIP2011_RobustTextDe...](http://web.stanford.edu/~hchen2/papers/ICIP2011_RobustTextDetection.pdf)),
which sounds like it might be the sort of thing you want.

In particular, the figure on the first page of the paper shows detection (not
OCR) of a textual logo in a photo.

~~~
IgorPartola
Awesome, thanks! This should be fairly easy to wrap for Python I think.

------
sirtastic
Worked well with the pictures I dropped in, very nice.

Wish someone would make a solid angular dropzone+cropper.

------
dheera
If Imgix could implement this server-side, that could be awesome.

------
zachrose
How long until this has native support in CSS?

------
donmb
Exactly what I was looking for. Will give it a try! Thank you.

