Pretty amazing piece of code, the loss of the C API is sad since a number of embedded systems started there and are skittish about going to C++11. I used version 3 to build a simple license plate reader that I had stuck in an upstairs window for a time. Combine that with the simple IMSI catcher that was referenced here a while ago and you've got your own little neighborhood watch station with some pretty grown up capabilities.
From the article: "A lot of C API from OpenCV 1.x has been removed."
That doesn't sound like they removed all of the C APIs, unless the only C API was the one that came with 1.x? I'm not very familiar with OpenCV, but since they homepage still talks about offering Python and Java APIs, it seems likely that they're going through a C interop layer somewhere. Did they never introduce new C APIs in the last three major versions?
If there really is no C API, that's sad and kinda limiting. Everything binds to C, but almost nothing can bind to C++.
> a number of embedded systems started there and are skittish about going to C++11
I'm curious to know what embedded systems are powerful enough to do real-time computer vision, but don't support a C++11 toolchain? C++11 is pretty old at this point, and between gcc and clang, almost everything with a 32-bit or better processor is probably supported, as well as probably many 8-bit and 16-bit architectures, but I'm skeptical about how much CV you can do on the kind of processors that are built as 16-bit processors.
It isn't a question of power, I use a lot of Cortex M machines (32 bit ARM) which can compile C++11 just fine, it is that there is a lot of infrastructure (like you really have to have a working heap system, startup files that know how to build the necessary infrastructure, Etc.) that is intimidating to embedded programmers. Not to mention screwing up the symbol names :-).
I used to like the simplicity of the C API and knowing where buffers were allocated and discarded. Then at one point I took the dive and learnt the C++ API and its memory management scheme. I did not look back. Having all these things automated while still being reasonably predictable is really comfortable!
mbed makes it pretty easy to get going with C++ on Cortex-M, last time I tried it, but that was over a year ago.
Still, I'm skeptical about how much CV you could actually do on a Cortex-M processor. A few dozen to a few hundred kilobytes of RAM is really limiting when we're talking about just loading up an image from a camera, let alone doing computer vision on that image, unless you pay for a Cortex-M7 to get 1MB of RAM or something really expensive like that... at which point, an entire Raspberry Pi 0 SBC with 512MB of RAM is actually cheaper than just your microcontroller alone.
Of course, the microcontroller boots up instantly and supports incredibly useful sleep modes and a variety of other peripherals... but for raw performance per dollar, the M7s are just so costly.
Ah, a Cortex-M7 with 1MB of RAM... just like I mentioned. haha, the earlier editions used an M4 with 256KB of RAM it looks like, but yeah.
Do you think you could run the license plate reader you wrote earlier on one of these? What are the advantages you see for this over a Raspberry Pi 0 with a Raspberry Camera module? Or the new, powerful Raspberry Pi 3A+ with a camera module? This is certainly more expensive than either Pi option, with a lower resolution camera.
I'm reasonably certain it can be run on the H7 but not sure about the H4. It is one of my test cases :-). Note that these chips also include a DDR memory controller so you can put 16MB of RAM on them pretty simply, some of ST Micro's 'Discovery' cards come with this populated, but that is not used on the H4 or H7 camera.
Going out on the C++ support side, embedded have to bad C++ support, for many bad reasons.
People tend to think C++ have much higher runtime needs that C, but used right it's usually lower, specially if you use C++-11 or even C++14. There compilers can make smart stuff that allows you to do much more in compile time and gains much in runtime. There is a few nice talks about this at CppCon including Michel Caisse.
Imho the main reason is that embedded hardware chip vendors have two goals that kinda makes only supporting old C, e.g. pre c99 viable.
1) They get money on there own, or a partners toolchain, that isn't evolving as alternatives, keeping up with standards are hard.
2) They optimize the time for code to first boot time. The demo factor, not how the products work in CI/CD chain, the preformance of the code on the final product. How easy it is to get started in the sales meeting is more important.
Oh! G-API seems nice! It looks like something I had to implement imperfectly at one point (and I suspect many OpenCV developer also did it). The ability to describe a pipeline and having it run automatically it really a good thing to have.
You might also find Halide[1] interesting then. It too is a computation graph that does a good job of separating core algorithm and optimizations.
OpenCV's neural networks module actually uses Halide as an optional backend. I found it to be a practical approach to run TF/Torch/Caffe/Darknet models on CPUs and OpenCL/AMD.
There was an OpenCV proposal to port all algorithms to Halide API [1]. I think that's still the long term plan, and G-API is a kind of stopgap till then [2].
Yes, it is sad. OpenCV has a code-gen for Java/Python but they are hard to work with. A number of projects in other programming languages (Rust [1], Go [2]) has to use custom build wrappers.
Perhaps that last one works on mobile, although I wouldn't get your hopes up.
I haven't really used OpenCV since 2.4.x, however assuming the same holds true, performance enhancements are on a feature by feature basis. Meaning that if you enable CUDA etc. then it's not going to work for everything. Not GPU, but a common compile time enhancement (at least it used to be) was to compile with Intel TBB (thread building blocks) as a lot of the APIs included optimisations.
Thanks, but iOS supports neither CUDA, OpenCL, Vulkan nor TBB. It's only Metal and higher-level frameworks like Core Image for GPU, and vImage for SIMD image manipulation.
You can use MoltenVK to run Vulkan via Metal (https://github.com/KhronosGroup/MoltenVK). Apparently performance is quite good for regular 3D rendering, compute support is officially listed, but I'm not at all sure what performance is like.
> OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
It sort of does in the first sentence "OpenCV (Open Source Computer Vision Library)", but they could really be a lot clearer about that. I really wouldn't give a crap about the license and implementation details until I had a fair idea of what the project could do for me.
