
How I failed to replicate an $86M project in 1 line of code - rodgerd
https://medium.com/@ryanfb/how-i-failed-to-replicate-an-86-million-project-in-1-line-of-code-615048a1f9d0
======
new299
The post starts with a false premise that:

"[a medium post has been] doing the rounds the last few days, describing how
an automated license plate recognition (ALPR) system being developed for the
Australian Victoria Police could just use the open-source ALPR system OpenALPR
instead"

The original post was pretty much a tech demo, and noted a bunch of issues. I
think the point was that an ALPR project shouldn't really cost $86M.

ALPR is a well studied problem, there are a number of commercial solutions.
I've no doubt there are even a number of solutions already deployed in
Australia. Such systems are routinely used, for example, to monitor the speed
of traffic (see Traffic master).

And as the original post showed, there are even open source solutions to the
problem that work pretty well.

So yea... $86M seems like a lot when there are probably off the shelf systems
you can buy at a much lower cost...

~~~
jacalata
The Victorian police system was first running in cars on public roads in 2012.
No doubt there are plenty of solutions available today that have been
developed since then, but they probably weren't considered at the time. That
$86m has covered development of the software and hardware and at least six
years of running costs for the program. So yea, I'm sure $86m sounds like a
lot when you compare it to some random piece of software that does a small
portion of the project.

~~~
new299
The first ALPR systems were deployed in the late 1970s.

Fixed camera systems were rolled out in Australia in 2005...

So.. ALPR is not new. In 2012 there were multiple systems available. 86MUSD
seems like a lot to adapt existing research or existing systems. Even if that
includes acquiring/maintaining several vehicles.

Maybe the whole project is being mischaracterized. Perhaps it included
requirements completely unrelated the ALPR (face recognition, driver
identification? don't know...).

But with the information available, 86MUSD seems like a lot.

~~~
jacalata
For a start, it's not USD (why on earth would an Australian organization talk
about costs in USD?). And of course we know it includes requirements
completely unrelated to ALPR, it connects to a bunch of existing databases and
systems to do live tagging of stolen vehicles, vehicles owned by people with
an active warrant or suspended license, etc.

~~~
new299
In 2012 USD a Australian dollar were basically 1 to 1 exchange rate. So it
doesn't really matter does it? As to why I might expect it, well... USD is
pretty much the international standard, so on an international website I often
expect things to be quoted in USD...

Right... "connecting to a bunch of existing databases". Does not sound like
it's worth 86MUSD to me. But there we go, I guess we just disagree on this
point.

------
perpetualcrayon
What if they created the 57 line project, and spent $86 million to improve
OpenALPR?

~~~
jacalata
Then it would be useless to the people who spent the money, because they
wouldn't have the cars and cameras and ongoing support and research that they
wanted to spend that money on.

~~~
perpetualcrayon
Sounds like you're arguing a different point. My assumption was that 'x' ==
"86 million" AND 'x' == "the amount it costs to develop the software". So, I
would just replace "86 million" with "actual cost of developing the software".

~~~
jacalata
Yes, I was arguing that your assumption was wrong.

~~~
perpetualcrayon
"straw man" \- an intentionally misrepresented proposition that is set up
because it is easier to defeat than an opponent's real argument

------
zhanwei
I agree with this article more. Yes, $86M is sure expensive with lots of
efficiencies. But the first one is oversimplifying the technical aspect, which
is missing the point on the sources of inefficiencies. The project can't be
using openALPR at its current level and it is hopeless to use it as a starting
point to get decent accuracy.

------
alex_g
The first article points out the inefficiency and does a good job of
replicating some basic functionality. It's basically saying "with this
starting pointing, the performance is not as good, but we could probably make
it a lot better without spending $86 mil"

IMO, this one is saying "I'm gonna push the starting point even further back
and complain about someone making a suggestion about inefficiencies."

But that's the internet I guess.

~~~
sverige
A little digging seems to indicate that the author makes a living as an
engineer whose livelihood relies on customers paying for solutions provided by
the company he works for, so naturally it is offensive that someone would
criticize inflated costs associated with a third-party, closed source solution
to a problem.

------
kerkeslager
Two problems:

1\. You've shown the success of OpenALPR isn't perfect, but I'm sure the
closed source alternative isn't perfect either. It might be better, but is it
$86 million better?

2\. If we spent $86 million on developing OpenALPR, it would be an incredible
product that everyone could use, without paying so much to the overhead of an
executive structure.

~~~
jacalata
To even begin to answer your question, you'd need to know what the project
included besides the software that OpenALPR could theoretically replace. Like
how much hardware has it bought? How much of that money went to paying police
officers for time spent developing requirements, testing the prototypes, and
learning the eventual system? How much of it went to the research needed to
develop this system some five years before openALPR was available? How much
went to paying the software developers a normal first world wage and benefits
for five+ years?

------
andrewchambers
I'm pretty sure you could make it realtime with more than 10x improvement:

Scan a thinner section of the video because the sky/ground is useless, and
process every 10th frame.

~~~
Izmaki
This requires a vehicle to be right in front of the (police) car. Not very
practical as most of the license plates you get to scan will be in other
locations of the video - you know, since the car in front of you may be there
the next 10 minutes whereas cars drive past you and are parked to the side all
the time.

~~~
andrewchambers
you could narrow it 1/3rd at least without really affecting results at all,
then include the 10x improvement from skipping frames, since you really don't
need that many. The world doesn't move that fast.

~~~
hikarudo
Actually, the world (the part of it you're interested in, i.e. the license
plate) does move that fast, relative to the camera, in terms of pixels/s. You
need 10fps for patrol car ALPR to work well.

Source: I've worked with ALPR for 7 years.

~~~
naikrovek
You just need a short exposure, blur is not related to framerate, it is
related to exposure time.

------
taitems
Original author of the original article this person is responding to. Saw this
response earlier today which made me a bit sad, but mostly excited that people
are having a crack at it themselves.

It's currently Father's Day here in Australia so I don't have a chance to
respond right now. On the road all day.

I'll have to do a follow up next weekend, I'm absolutely flat chat both in and
out of work at the moment.

------
coldtea
Whether the actual project is worth exactly $1 or $86 million is not the
issue. Not even whether a $1/67 lines of code can replicate a $86 million
project.

The real substance -- and the first posts hints at it -- is that government
(and private sector) projects are more often than not many times more
expensive than they should be.

Sometimes because they are overengineered (in 2005 it would be 200 Java
programmers building a EJB/SOAP/XML monster that requires 10 workstations to
run), other times just because they can (charge more). The same way health
related projects and vendors can charge a ton for trivial stuff (even plain
plastic syringes).

In many countries, it's also because the bigger the project, the more greedy
intermediaries can get a bite.

------
jonathankoren
I don't understand the hate that's being directed at this article. This author
isn't the person basically said, "LOL. I'll string together some python in an
afternoon and do it." This author tried to replicate the "How I replicated an
$86 million project in 57 lines of code" article, and unsurprisingly came to
the conclusion, "No, he didn't."

~~~
naikrovek
Well, YouTube-encoded video is a severe mistake and it was his assumption that
this would be representative of video out of a commercial camera.

It is not.

The camera is also not calibrated. Text recognition does not work well on
video whose distortion characteristics are not known.

I am not at all surprised that this is his result.

This morning I calibrated my dashcam and used OpenALPR to replicate this, and
I got FAR better results than this blogger.

At the end of a 30-minute section of heavy traffic, I had many hundreds of
license plates with 95% confidence or better. I do not have a way to verify
the plates automatically, so I don't know how many of those are actually
correct.

I filmed at 2560x1440 at 30fps.

~~~
jonathankoren
And all of this is true of the original boastful post, but yet no one was
suspicious of those results.

------
gcb0
can't tell if incompetence or malicious.

everyone knows dashcam, even on 1080p, simple can't be used fir license
plates! humans can't read most of them. specially after youtube compression!
this is such a known fact that the more serious youtube reviews show a warning
about all that before dashcam reviews.

------
ww520
Often time the devil is in the detail.

------
knguyen0105
OpenALPR has a premium service ([http://www.openalpr.com/cloud-
stream.html](http://www.openalpr.com/cloud-stream.html)). Has anyone tried it?

------
klondike_
I think this is a good case for why all government software should be open
source.

How many police departments around the world are spending absurd amounts of
money for this technology? It would make much more sense both cost and
productivity wise to have one big project that anyone can improve on instead
of multiple inferior projects.

Not to mention that all of this is paid for with taxpayer money, so it is
reasonable for those same taxpayers to have access to the code.

~~~
lifeisstillgood
Please join my campaign at
[http://www.oss4gov.org/manifesto](http://www.oss4gov.org/manifesto) \- this
is exactly why we want all government software to be open source

(Shameless plug)

~~~
chii
i think there's some gov't contractors that survive off these inefficiencies.
They will fight tooth and nail against OSS in gov't, and since they are often
more connected, it's quite likely they can win.

~~~
enraged_camel
Full disclosure: I'm one such contractor. We work with small and local
government. Cities and counties mostly.

I personally don't have a problem with open source software and use it
extensively outside of work. However, the big problem with it is technical
support. If you are a government organization and the OSS you are using has a
bug, who do you contact to get it hotfixed ASAP or find you workarounds?

That's one of the biggest advantages of commercial software: it is often sold
with support contracts that entitle customers to certain levels of support,
which can be critical to get things fixed and minimize downtime.

I know there are companies that focus on supporting OSS deployments, but from
what I have seen in my own niche, they charge a ton of money that often
surpasses what the equivalent commercial package would cost.

~~~
lgas
> Full disclosure: I'm one such contractor. We work with small and local
> government. Cities and counties mostly.

...

>If you are a government organization and the OSS you are using has a bug, who
do you contact to get it hotfixed ASAP or find you workarounds?

You, or someone like you, presumably?

~~~
nradov
That's a nice theory about OSS but it seldom works in practice. It can take a
very long time for even experienced and skilled government contractors to
learn enough about a code base to fix major defects. And even then there's no
guarantee that the project maintainers will accept that patch into the
mainline, so if they don't that puts you into configuration management hell.
Governments need a "single neck to choke" along with some reliable assurances
of support levels and time to resolution.

~~~
dikaiosune
It seems to me that govt agencies could still pay contractors to build these
things, just with the requirement to make them openly available as part of the
contract.

If others step in to offer support contracts as well, great! Competitive
environments for that would probably be good.

------
Namidairo
I wonder how much better it would work if either article specified the
Australian training data that is included, instead I believe it is defaulting
to the US set.

As for speed, there appears to be an OpenCL option that should be
investigated.

