
Show HN: Using deep learning to detect objects from your webcam - mikeshi42
https://modeldepot.github.io/tfjs-yolo-tiny-demo/
======
mikeshi42
Hey Everyone!

After seeing the announcement of Tensorflow.js, and being a lover of both JS
and ML, I couldn't help myself from creating a demo with a non-trivial ML
model. YOLO is definitely one of the most interesting/useful ML models out
there (as we saw on ModelDepot) and thought that'd be cool to get working.

As far as I know, there hasn't been any prior work with getting YOLO object
detection working in JS, I assume mainly because the post-processing code to
transform the model's outputs to bounding boxes is a whole heap of math.
Luckily tf.js makes it a lot easier to translate from Python and so here it
is!

You can drop this into your project with just a couple lines of code (and no
ML required) using NPM! (via npm i tfjs-yolo-tiny)

If you want to help with perf/bug fixes, PR's are more than welcome @
[https://github.com/ModelDepot/tfjs-yolo-
tiny](https://github.com/ModelDepot/tfjs-yolo-tiny) !

------
locusm
Hope some of the entry/mid level security cam vendors are working on this type
of detection as motion detection currently leads to so many false positives
once you want notifications.

~~~
mikeshi42
Given the proliferation of ML and easy to access pretrained models, I imagine
they would! Though still the computational cost might be a bit high, but
certainly could act as a second level filter for motion detection.

------
gormz
Interesting. I'm about to start using Pixy sensor to track people for a
project I'm working on.

~~~
mikeshi42
That's definitely interesting, from the quick demo I just saw, doesn't it only
use colors to detect? If so, you'll definitely need something more powerful.

Depending on how accurate you need your model to be, (and what platform you're
looking to build on), you could use tfjs-yolo-tiny if it's a browser-based
environment. Otherwise I'd recommend doing processing in the cloud using the
full YOLO model which is more accurate than Tiny YOLO but slower
([https://modeldepot.io/mikeshi/yolov2/overview](https://modeldepot.io/mikeshi/yolov2/overview))
or FSSD which is more accurate than YOLO but slower
([https://modeldepot.io/lzx1413/fssd/overview](https://modeldepot.io/lzx1413/fssd/overview))

Oh and if you might be doing something self-driving car related, a
segmentation network might be useful as well (ex.
[https://modeldepot.io/hellochick/pspnet/overview](https://modeldepot.io/hellochick/pspnet/overview))

~~~
gormz
yes, I haven't done all of my preliminary research yet, but from what I know
it uses color matching to track things ie. clothes people are wearing. It's
for a robot that will follow a person. I'm going to be doing a lot more
research on this in the upcoming weeks.

Thanks for all this information, it seems I have a lot to learn! :)

~~~
mikeshi42
Ah yeah in that case color might make sense :) I'm not familiar with
persistent tracking of people for sure, but I hope your project works out!

------
rajeevk
Seems very very slow on iPhone (safari)

Here is what it detected for me
[https://imgur.com/a/DSIZQ](https://imgur.com/a/DSIZQ)

~~~
mikeshi42
Unfortunately iPhones just don't have a ton of computation power, but I've
been given some suggestions and hopefully will have a good perf boost coming
out soon!

As for that detection, this model is using Tiny YOLO, it's a simpler ML model
made to run on more constrained hardware (end user devices) and therefore does
not perform as well as the full YOLO or SSD. The other issue is that usually
these models take context heavily into account, a fun experiment is to raise
anything (ex. wallet) up to your ear and it'll interpret it as a cell phone.
I've read an interesting paper recently that forces models to focus more on
the object itself than the context to classify things.

------
nonbel
Has anyone ever scraped HN and checked number of exclamation points per post
and compared to story rank (eg time spent on front page)? On this page there
are currently 6 exclamation points and 5 posts. This seems higher than usual,
and actually I have noticed more than usual in ML-related threads before...

Does HN use the rate of exclamation points in their ranking algo?

~~~
mikeshi42
I'm honestly just very excited to share this! ;) but certainly! if
exclamation! points! help the ranking! then I'm all for it!

~~~
nonbel
I've noticed it in other threads too so don't think it is just you. It just
could be a coincidence...

