
Show HN: ObjectCropBot – Interactively Crop Objects from Photos with AI - andreyk
http://www.andreykurenkov.com/projects/hacks/objectcropbot/
======
andreyk
I post this now (the project is a few months old) purely inspired by this
post: [https://blog.photoeditorsdk.com/deep-learning-for-photo-
edit...](https://blog.photoeditorsdk.com/deep-learning-for-photo-
editing-943bdf9765e1)

They basically took the same idea and instead of just producing a hack
finished a nice full version of the idea - very nice write up!

But, they don't have any web UI for it yet, so I am getting tempted to revive
my dormant side project and likewise make a proper polished version of it with
a web interface. If anyone on here is a talented web dev with an interest on
working on this as a side project (for free, and for fun, though we could
explore monetization if it works), feel free to get in touch! (ps extra get in
touch if you are near south bay area/Stanford)

PS Incidentally, I independently came up with the same idea as in Deep
Interactive Object Segmentation
([https://arxiv.org/abs/1603.04042](https://arxiv.org/abs/1603.04042)) and
implemented it for my Stanford CS 229 (Machine Learning) project first - the
hack came later. The ObjectCropBot hack allows only cropping and not clicking
because it was faster to hack in that, but I think it is ideal to allow both
cropping to constrict around target object and clicking (clicking alone leaves
too much ambiguity , eg do you want to crop the person, or their shirt, or a
subset of that shirt).

~~~
jboy
Nice work! My startup [http://objectai.com/](http://objectai.com/) has just
launched something similar, Pizza Photo Editor:
[https://pizza.pics/](https://pizza.pics/)

Pizza Photo Editor is still early-stage -- we're still adding features and
increasing the coverage of different object classes in our training data set
-- but the core tech is there, and it does have an interactive web UI, so it's
already fun to play with. :)

Looking at the example inputs & outputs for ObjectCropBot, it's evident that
the challenge is not just detecting the object and clipping it approximately,
but tracing a tight precise boundary around it. We've found that DNN/ConvNet-
based approaches don't offer the necessary precision, so it's necessary to
perform some pre- or post-processing using other Computer Vision techniques.
At Object AI, we've developed an "object boundary deduction" pipeline that
combines ConvNets with other tech. It's interesting that the blog post that
you've linked to makes the same observation!

~~~
andreyk
Wow, this is great! Maybe i'll shelf my efforts a bit , if your online photo
editor is this good... I did Google around and find some of these GraphCut-
esque refinement techniques, but have yet to find an easily usable one that
gave you a good crop with just an outline or tap (or both) - kudos for making
one!

As to the lack of precision of ObjectCropBot, I think a lot of that is due to
it running based on DeepMask and not SharpMask; Facebook's insight to run the
final low-res features back through increasingly less downsized images with
sort of skip connections is a good one, and I'd bet the resolution of the
results would be a lot better if I just hacked it to use that instead of
DeepMask (I think the thing I linked just uses that, pretty much).

------
delinka
Reading the title I was interested to see how this thing filled in the
background after cropping an object from a photo... Alas, it's detecting an
object in the selected region and providing a _cut-out_ of that object to use
elsewhere.

Still impressive, just needs a better name. There are two problems in Computer
Science...

------
eriknstr
Neat. However I think the proper term would be cut-out. Cropping is just
cutting of edges vertically or horizontally.

------
shimon_e
Looks nice. May I recommend getting free credits from OVH.
[https://www.ovh.com/us/dlp/](https://www.ovh.com/us/dlp/)

Are you available for consulting? I have a project in this domain that you may
be interested in.

~~~
jboy
Can you provide any more info about your project? Maybe my startup
[http://objectai.com/](http://objectai.com/) could be of assistance.

(My email address is my HN username @ my startup domain, if you'd prefer to
discuss over email instead.)

