
Live Computer Vision Coding - turrini
http://livecv.dinusv.com/index.html
======
arnioxux
There are a lot of rapid prototyping tools for computer vision usually done
via some dataflow interface:

[http://imageplay.io/](http://imageplay.io/)

[https://www.adaptive-vision.com/en/software/](https://www.adaptive-
vision.com/en/software/)

[http://www.adrianboeing.com/improvCV/](http://www.adrianboeing.com/improvCV/)

[https://github.com/utwente/parlevision](https://github.com/utwente/parlevision)

I don't use any of these but I see a few of these reinvented every year with
not much differentiating them from each other. I wonder why none of them
caught on yet?

I think they are useful compared to the tedious workflow I am stuck on
(prototype with python notebook to dump visualizations and play with
hyperparameters, then port to C++ later). But the fear of getting trapped into
some unsupported edge case and having to deal with a messier port usually
stops me from changing.

~~~
BugsJustFindMe
> _I wonder why none of them caught on yet?_

Because the data pipeline is never the hard part in computer vision.

~~~
ggg9990
What is the hard part? What types of inputs/outputs are we talking about here?
Fairly ignorant of this area :)

~~~
BugsJustFindMe
The hardest parts with computer vision, like with any advanced field, are

1) making the algorithmic bits in the first place (see opencv, vxl, itk, or
insert any other vision library here where many of these bits are made for
you)

2) knowing how to achieve the desired result using those bits

Many of these data flow tools show you boxes and arrows, and pretend that
connecting bits with arrows instead of function calls is where the magic is.

But if one of those boxes is called, say,
"ComputeOpticalFlowBetweenTheseTwoInputImages", how is that any different from
calling the ComputeOpticalFlowBetweenTheseTwoInputImages function in the
underlying algorithmic engine that they're building on top of?

The new layer doesn't solve either of the two hard problems for you.

Giving a live result is cute, but it's not something that you can't already do
with a Python IDE without having to learn some new arbitrary modeling
language.

~~~
dakerfp
That's not so arbitrary, that's plain QML.

~~~
BugsJustFindMe
QML is 100% arbitrary. WTF does a UI markup language have to do with CV? WTF
are they using a UI markup language for this? WTF thought that QML would be a
good fit for thiss? (sorry for using WTF in three different ways)

------
source99
I've never seen a live CV before. its pretty cool.

I have a flag that dumps out each stage of my CV pipeline when i need to debug
something. I imagine this would be useful but its nice to be able to use my
actual code base vs a separate system.

~~~
seanmcdirmid
The first coined live system (at least identified as much) was actually a CV
system. See Tanimoto, S. VIVA: A visual language for image processing, J. Vis.
Languages Computing, 127-139, June 1990.

------
fartcannon
Forgive me if I'm wrong, but several vital plugins seem to be missing in the
Linux download. Lcvcore for example. Compiling from source seems to properly
populate the folders.

------
spookyuser
I had the perfect use case for something like this last year.

I wanted to make a (very) short movie with a similar feel to Waking Life,
except instead of rotoscoped images I just wanted to do varying degrees of
edge detection on each frame. It came out looking pretty good but it took a
lot of tweaking individual numbers in a python script and repeatedly rendering
the same clip before I could even get a set of choices to decide between.

Having something like this with me when I was filming it would have made a
huge difference!

~~~
CyberDildonics
That would take about 10 minutes to set up in touch designer and it would work
in real time.

------
kalal
This can never work in practice. First, it seems super simple to do CV in the
video, but only because the author already knows what to type there. It will
never be so smooth in real life. Next, if you accept that you have to think
what you are doing, why not to fall back to OpenCV in combination with
ImageWatch visualizer. The good news is that you save yourself one layer of
abstraction which hides important details from you, not to mention learning
yet another API.

~~~
taneq
Basically this, although I wouldn't say "it can never work in practice", just
that "a boxes-and-arrows UI doesn't actually make CV (or anything else hard)
significantly easier".

It's great to start out with and play around with but eventually you're going
to want the GUI to get out of your way and just let you edit the code
directly. At which point something like shadertoy is (IMO) the best of both
worlds.

~~~
kalal
You are right, there is good application for this tool in education or people
who want to quickly test something new. Shadertoy is great and I like it. It
is at the right level of abstraction that I need for my current CG knowledge.
I can imagine that there are many people who find Live CV very useful and then
move downwards.

------
tw1010
I wish this didn't force us to use pre-existing lego pieces by abstracting
away the math. So so so many cool things could be built in the computer vision
sphere if engineers weren't afraid to go a level deeper, ignoring all the
opencv stuff, and just implemented the math themselves. There's no reason we
can't get creative on a level further down. With all the clever math people
that are enthusiastic about abstract algebra in security, you'd think there'd
be some spillover of that math confidence into the CV world.

~~~
dakerfp
It's Qt based, it's easy to create your own blocks with C++.

------
supermdguy
I wonder how hard it would be to create deep learning based plugins, for
example YOLO or Faster R-CNN for object detection.

~~~
lovelearning
A lot of the groundwork's already been done from the OpenCV side.

OpenCV has a dnn[1] module that can import models from other frameworks like
Caffe, TF and DarkNet (YOLO's native framework) into OpenCV's own DNN object
model, and run inference on them. Both Yolo and R-CNN families have been
implemented as examples[2].

What's missing is that OpenCV does not abstract all these behind a single
object detection interface if that's what the LiveCV side expects. So such a
plugin would have to define such an interface itself and provide an adapter
per model.

[1]:
[https://github.com/opencv/opencv/tree/master/modules/dnn/src](https://github.com/opencv/opencv/tree/master/modules/dnn/src)

[2]:
[https://github.com/opencv/opencv/tree/master/samples/dnn](https://github.com/opencv/opencv/tree/master/samples/dnn)

------
Larrikin
Any word on a macOS version?

~~~
jdlyga
They provide the source, and it's Qt based. So it should be buildable for Mac.

