

Ask HN: I want to detect when arbitrary things come into a video stream - eof

I have seen some javascript implementations of &#x27;object detectors&#x27; but they all seem to be limited to just faces or hands or some specific type of gesture.<p>Is there anyway I can just show &quot;it&quot; say.. some little statue I made when I was a kid and have &quot;it&quot; tell me if it&#x27;s in an video?  I want to basically: have a video stream loaded (for example, the webcam on my laptop) and then, if some _thing_ that I have predefined comes into the video, then do something with it.. basically make an HTTP call to some other service I have written.<p>Is there any &quot;it&quot; that can help me do this?
======
CookWithMe
What you are looking for is an "object recognition" algorithm. These
algorithms take one or multiple images of an object (e.g. your little statue)
and then try to locate it in another image.

Generally, they are computationally expensive, so performing them on a video
stream, especially inside the browser, will most likely not work, even for
rather low resolutions.

I have some experience with the SIFT algorithm. Takes some time to grok it as
the algorithm consists of several sub-steps.

You may want to look into OpenCV, they have a couple of algorithms
implemented.

------
byoung2
I would imagine it would be pretty slow, but you could have a video stream
write to a canvas element at regular intervals. You could analyze the pixels
in the canvas to look for a pattern. This would be the hard part, since you'd
have to write an engine to detect edges and create a hash of all detectable
objects in the frame, then you'd have to match each hash to a known hash of
the desired object. To get the known hash, you'd likely have to feed in many
example images of your object and create hashes from these.

------
jlengrand
Well, what you describe is extremely difficult to do, and people have been
trying to do with for decades now without success.

It could work if you place yourself into similar conditions over all the
videos. By having a good understanding (precise model) of what you want to
recognize, you could make some pattern recognition / object detection to be
able to find it in other videos. But the huge problem in this field is that
the conditions keep changing. Light, luminosity, color, shape, distance. All
of the makes pretty much every method of recognition of "it" completely
useless.

There is one thing you can currently do though. It is target tracking. Without
knowing the target before, and with minimal user interaction to follow it, it
is currently possible to track it quite accurately.

Hope this helps.

------
xauronx
I've seen people do this with a Kinect I think. Obviously that's more info
than just a video stream but I don't know exactly what your limits on your
solution are.

------
michaelmior
Are you restricted to running in the browser?

~~~
eof
not necessarily, but preferably

------
beachstartup
i'm no expert, but i believe machine learning / classification algorithms can
help you on this. you just need to train the algorithms on what you want to
find.

'finding' objects means feeding the algorithms a series of 'features' that you
extract from video and normalize into data structures - color, corners, edges,
velocities, relative distances, etc, which are then fed into the algorithms to
produce a classification or a probability.

analyze every Nth frame and try to classify found objects. if a known object
is found, do something.

