
Show HN: OpenCV based multi-object-tracker with various object detection models - adipandas
https://github.com/adipandas/multi-object-tracker
======
mendeza
I work a lot with object detection at my current role. Would be cool to add
Detectron 2, seems that this library that has some of the state of the
art(SOTA) models in object detection.
[https://github.com/facebookresearch/detectron2](https://github.com/facebookresearch/detectron2)

Also the TF Object Detection API has a suite of pretrained SOTA object
detection models
[https://github.com/tensorflow/models/tree/master/research/ob...](https://github.com/tensorflow/models/tree/master/research/object_detection)

What makes object detection really valuable is being able to finetune a model
to detect objects that are relevant to your problem. However collecting,
labeling, takes a lot of time since the best way is to manually label. But
30-50 hrs of manual labeling can get you some great results when you are
leveraging pretrained models. Building a tool to manage data labeling and
training for finetuning object detection models would be very valuable.

~~~
deppp
Check out our open-source tool called Label Studio,
[https://labelstud.io](https://labelstud.io) and we have a commercial offering
that does exactly this, finetuning the models.

~~~
mendeza
I actually checked out your github repo and saw your framework that enables
integration of trained models into your tool for active learning. Looks
awesome!
[https://github.com/heartexlabs/pyheartex](https://github.com/heartexlabs/pyheartex)

------
alexcnwy
I've built several object detection systems and you definitely want to fine-
tune your model to get good results, even if the object class you want to
track is available in a pre-trained model (often it isn't).

For example, pre-trained models don't work too well at detecting people in
surveillance footage when the camera is mounted on the ceiling and people are
at an angle in the footage. Off-the-shelf object detection models also don't
work well in poor lighting conditions but can work well if you fine-tune one
of the pre-trained models for your dataset and lighting conditions.

Here's a great and fun tutorial on how to fine-tune an object detection model
using TensorFlow (it's a surprisingly fiddly process but worth the headache):
[https://towardsdatascience.com/how-to-train-your-own-
object-...](https://towardsdatascience.com/how-to-train-your-own-object-
detector-with-tensorflows-object-detector-api-bec72ecfe1d9)

I suspect you can use your fine-tuned model with this repo by replacing the
`get_ssd_model.sh` compiled graph with your fine-tuned model graph.

It's also a fun exercise to implement the tracking yourself - here's a solid
tutorial to get you started: [https://www.pyimagesearch.com/2018/08/13/opencv-
people-count...](https://www.pyimagesearch.com/2018/08/13/opencv-people-
counter/)

~~~
adipandas
The top-down view problem is what I was facing when I was trying to use a
detection model for a project with quadcopters. For this I had to train my
object detector with the data we collected from that view. YOLO was used for
this work.

Yes, replacing the fine-tuned model with compiled graph should work. I tried
to keep it simple and intuitive to change the model so as to allow the use of
any other custom models.

------
martinshen
Have you tried using object attributes/features to enable tracking instead of
IOU-based tracking? I suspect this tracker falls apart when there are
significant obstructions. Thoughts on a Recurrent YOLO or DeepSORT based
tracker?

~~~
kevinsundar
I've found that IOU based tracking is surprisingly reliable (my dataset was
primarily surveillance footage with a static camera) if you take measures to
smooth the data and predict object motion using a kalman filter.

~~~
adipandas
Kalman filter is also one good way to reliably keep track of objects under
significant obstruction.

------
nullbyte
PyImageSearch is a fantastic resource for learning this stuff. Highly
recommend it.

------
mbpqd
Question for HN experts: what would be the best approach for object-detection
when the object I am trying to detect is a specific image (not a class of
objects) printed on a 2D surface (therefore it could be partially folded /
curved, but still more of a distorted 2D object than a complex 3D one)?

~~~
mendeza
I would check out template matching or using OpenCV feature extractors such as
SIFT, SURF, ORB

Template Matching scans an image with your template, smaller image and finds
the location with highest cross correlation: [https://opencv-python-
tutroals.readthedocs.io/en/latest/py_t...](https://opencv-python-
tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_template_matching/py_template_matching.html)

Template matching assumes the example image wont deform or take on different
orientations.

Popular Feature Extraction finds and derive more robust features that help
find objects in the scenarios of distortion, orientation, and scale.

Link for feature detection and matching: [https://opencv-python-
tutroals.readthedocs.io/en/latest/py_t...](https://opencv-python-
tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_matcher/py_matcher.html)

Pyimagesearch does a cool method where they enable template matching to work
at different scales. It ignores color information (by using edge detection),
which could be a critical feature if the object has specific color:

[https://www.pyimagesearch.com/2015/01/26/multi-scale-
templat...](https://www.pyimagesearch.com/2015/01/26/multi-scale-template-
matching-using-python-opencv/)

------
binarysaurus
There arw thousands of these repositories all over the place...

~~~
adipandas
Okay, I am not sure about others. But I was working on this as a fun project.
And I wanted to share my work. Hope you find it useful and please let me know
your suggestions and any additional improvements I can make.

