
Current state of the art in objects classification - ZeljkoS
http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html
======
dmm
I see object detection systems often compared by accuracy percentages but
rarely by speed.

I ask because I'm building a surveillance system that uses object detection to
answer simple questions, like: "Is there a squirrel on my bird feeder?", "Is
my car in the garage?", "Is there a package on my front step?".

Right now I'm playing with Darknet and yolo from pjreddie because he
emphasized performance and I'm getting about 69fps on my 1080ti.

Is speed not considered important by researchers at this time?

Is there any system that provides decent performance on CPUs? Dedicating $700
gpus to detect squirrels on the bird feeder is fine for crazy people like me
but others may find that excessive.

~~~
bitL
BTW, try SSD, it was even a bit faster than YOLO2.

~~~
aaroninsf
Is there a Docker makefile available for playing with SSD if you're not
working in the domain?

I have some 100Ks of pix I'd like to put through classifiers but could never
get YOLO2 properly working in a container, in part because even reading the
code I couldn't figure out how to simply point a trained model at a directory
of images and get classification out... :/

Definitely PEBKAC but if there are any dead-simple tutorials available I'd
love a pointer–classification speed not a factor, it would all be CPU non-
real-time and could be leisurely. Just interested in seeing what the state of
the art is in classifying a personal image set without training against it..

~~~
joshvm
There are some wrappers around YOLO3 in Python which may help:
[https://github.com/qqwweee/keras-yolo3](https://github.com/qqwweee/keras-
yolo3)

It's not Dockerised though, but you could grab a container with the usual
machine learning packages and install that on top.

Also look at NVIDIA Digits
([https://github.com/NVIDIA/DIGITS](https://github.com/NVIDIA/DIGITS)) which
has Docker images.

~~~
aaroninsf
Thanks Josh!

------
yorwba
> Last updated on 2016-02-22.

Given the speed of development in machine learning, the numbers are probably
outdated.

------
Dzugaru
Many years of "human optimization" on the same old datasets must lead to
severe overfitting, so these results should be taken with a grain of salt.
Related research:

[https://arxiv.org/abs/1806.00451](https://arxiv.org/abs/1806.00451)

~~~
bonoboTP
Have you read the linked paper? It finds no evidence of such overfitting
happening, despite common fears of this phenomenon.

Quote: "This shows that the current research methodology of “attacking” a test
set for an extended period of time is surprisingly resilient to overfitting."

------
dontreact
This is two years out of date. Current state of the art for CIFAR has about
1/4th of the errors

[https://arxiv.org/abs/1805.09501](https://arxiv.org/abs/1805.09501)

------
jamessb
Similar data (but for a wider range of problems - not just object
classification) was tabulated by the EFF's AI Progress Measurement project
[1]; I made an alternative interface to visualise it [2].

[1]: [https://www.eff.org/ai/metrics](https://www.eff.org/ai/metrics)

[2]: [https://jamesscottbrown.github.io/ai-progress-
vis/index.html](https://jamesscottbrown.github.io/ai-progress-vis/index.html)

------
spott
Anyone know of something like this for face recognition?

------
bitL
Must be very old, DenseNet-BC is nowhere mentioned...

