
Reverse OCR - mrtbld
http://reverseocr.tumblr.com/
======
albertzeyer
I was thinking about this:
[http://www.cs.toronto.edu/~graves/handwriting.html](http://www.cs.toronto.edu/~graves/handwriting.html)

~~~
DINKDINK
Unfortunately it does not take acute accents. Such as the sentence: Ceci n'est
pas écriture humaine.

~~~
geon
Nor umlauts, like åäö.

------
praptak
This is similar to the project where images of clouds were fed to face
recognition software:
[http://ssbkyh.com/works/cloud_face/](http://ssbkyh.com/works/cloud_face/)

~~~
stefs
there's also
[http://iobound.com/pareidoloop/](http://iobound.com/pareidoloop/), a project
that uses a genetic algorithm for breeding (random) polygons into a shape with
a face detection algorithm as the fitness function.

~~~
nihakue
That is totally cool. It feels like watching an oil painter slowly work from
something very abstract (layers of brushstrokes) to something very
recognizable (a human face).

------
jparishy
Not strictly related, but reminded me of the exercise in genetic programming
by Roger Alsing: [http://rogeralsing.com/2008/12/07/genetic-programming-
evolut...](http://rogeralsing.com/2008/12/07/genetic-programming-evolution-of-
mona-lisa/)

It's a rather cool attempt to draw the Mona Lisa using random, semi-
transparent polygons

~~~
darkFunction
I did this recently, the results were surprisingly good!
[https://github.com/darkFunction/PolygonPainter](https://github.com/darkFunction/PolygonPainter)

Edit: Roger Alsing's implementation was a single entity population (mutated
then reverted if the mutation was no good). I copied this approach in my first
implementation, but found that much better results could be achieved with a
breeding population of genes.

~~~
PavlovsCat
> I copied this approach in my first implementation, but found that much
> better results could be achieved with a breeding population of genes.

Ohhh, interesting, thanks for posting this! I just started playing around with
this myself a few days ago in Javascript (it has no UI so no link yet, but I
uploaded some samples [0]), and it also uses the original simple approach. I
wondered about an "actual" gene pool and cross-breeding, but shyed away from
the additional effort for uncertain benefit... so this helps, greatly :)

One thing I intend to try is to get the fittest (in terms of likeness to
target image), and then calculate the fitness of the other genomes by taking
the difference (in terms of variables that determine the shapes) to that
"champion". I see you take the two most fittest as is, maybe this could be
useful for picking the second one?

Also, when the Mona Lisa thing was posted on HN, someone suggested marking
areas of the target image as "more important", to maybe make facial features
etc. more recognizable. I'll also see if making such a mask automatically,
e.g. influenced by contrast, helps any.

[0] [http://imgur.com/a/4duom](http://imgur.com/a/4duom)

~~~
darkFunction
> I see you take the two most fittest as is, maybe this could be useful for
> picking the second one?

Yep, we take the two most fittest unchanged, and breed the rest randomly along
a non-uniform distribution tending towards the top.

> Also, when the Mona Lisa thing was posted on HN, someone suggested marking
> areas of the target image as "more important", to maybe make facial features
> etc. more recognizable.

This is a really good idea and would definitely make a difference. In my
penguin example, with too few polys, sometimes we reach a local maxima before
the eyes (small details) look any good. I combatted this somewhat by
encouraging new polys to be (a) small and (b) regular. But I like the idea of
a more guided approach.

There's an awesome javascript example you might find helpful:
[http://alteredqualia.com/visualization/evolve/](http://alteredqualia.com/visualization/evolve/)

Edit: Your images look amazing. How do you make the vector shapes?

~~~
PavlovsCat
I already knew the altered qualia link, and actually meant that one when I
talked about the post on HN (the link to which I got from the bottom of that
page) xD

Thanks! I just use what the HTML canvas has to offer: circles, n-gons, and
n-gons made of bezier curves. Those can get drawn both filled and as outline,
which in turn aren't drawn with plain colors, but gradients made of 3 HSLA
colors (alpha ranges from 0.05 to 0.85, to make sure everything matters at
least a bit and doesn't just get covered up or disappear), with the position
of the middle color stop, as well as the coordinates that define the gradient
direction, being variable.

The linewidth is variable, and it can also use dashed lines with a randomized
pattern, which looks funky but I haven't gotten a good result with it yet.
Another thing I'll do is add allowing picking a random compositing mode (and
in the spirit of the expedition, do that for both fill and outline), but then
it will be too unpredictable to look like being made of shapes, I expect.

The background also has a gradient consisting of no less than 6 colors, with 4
variable color stops.. what can I say, I like enums and copy and paste haha.
Though of course adding more degrees of freedom nilly-willy might not be the
best idea.. I have to improve the whole evolution/mutation stuff a _lot_
first, and then I intend to throw just about anything at it I can find, at
least as an option. Right now I added words, though that doesn't seem very
promising. But having one basic common function to setup fill and outline
gradients for any and all shapes I might come up with makes experimenting with
this a breeze.

I kind of feel I should be doing this with WebGL, but then I wouldn't have all
those convenient drawing functions.. but still, whenever WebGL 2.0 comes
(which will make a lot of compute-ish things a lot easier AFAIK), I want to do
at least a polgon/circle/bitmaps thing with it, because I feel a speedup
factor of a gazillion might just make up for that :)

~~~
darkFunction
Do you have a Github link? Or can you post back here when you're finished? I'd
like to see the final result :)

------
bane
This could be a cool way to visually "encrypt" messages. They're readable, but
only by the correct tool. I wonder how these squiggles might be creatively
arranged steganographicly in an image and still be "read" by the OCR tool.

~~~
SilasX
Correct me if I'm wrong here, but that just seems like reinventing crypto with
a large key, and requires you to implement a counterparty-provided algorithm,
which could be malicious.

~~~
sp332
The point of steganoraphy is that no realizes there's a message.

~~~
SilasX
And? It still has key sizes, the enemy still knows the system.

~~~
dragontamer
Steganography is solidly a "security through obscurity" thing. Sure, we comp-
sci people don't care about that, but spies do.

There was that Russian Spy who was transmitting data for years on her Facebook
account through steganography pictures on her Facebook account.

[http://www.technologyreview.com/view/419833/russian-spies-
us...](http://www.technologyreview.com/view/419833/russian-spies-use-of-
steganography-is-just-the-beginning/)

The FBI didn't know about it until after she was caught. So believe it or not,
Steganography _works_. If you're trying to hide _the fact_ that you're a spy,
encrypting all of your messages over TOR is a bad idea.

On the other hand, if you pretend to be a normal person and embed secret
messages in your Facebook posts, you can be a spy for years and not get
caught.

~~~
marcosdumay
> Steganography is solidly a "security through obscurity" thing.

I never really understood why is it so.

Encrypted data must be indistinguishible from random, thus, if you replace any
random projection of a file with your data, the result should be completely
unrecognizable. It shouldn't really matter if your algorithms are public.

Is the problem that it's hard to get random projections from modern data? If
so, why not use older formats?

~~~
cjg
People don't typically exchange randomised versions of their data.

~~~
dllthomas
I think "random projection", as used by the parent, can be things like "low
bits of the pixels in this image". If the color depth provides greater
resolution than the sensors, then you can expect to have some random data
implicit in the image that it would be possible to change in ways that _could_
be provably undetectable.

A tremendous caveat is that when we find ourselves shipping around lots of
meaningless random bits, we often quickly reach for lossy compression that
doesn't faithfully reproduce those bits, and that can break the scheme.

------
kitd
Could be used for automated printing of doctors' prescriptions ;)

------
mrtbld
Perhaps this could lead to a new kind of captcha that only bots can solve. I
doubt it would be efficient, though.

~~~
medmunds
[http://ssbkyh.com/works/fadtcha/](http://ssbkyh.com/works/fadtcha/)

~~~
pimlottc
Nice, a captcha to block colorblind users!

~~~
arjie
That's a subset of the objective. The objective is to detect all humans
because they wouldn't spot the 'face' in the coloured circles. Whether or not
it's coloured doesn't change that.

Unless you're concerned about the rights of colour-blind computers for some
reason.

~~~
pimlottc
Ha, oops, I guess I didn't read it closely enough. As someone who's partially
colorblind, just seeing those dot patterns pisses me off... :P

------
carsonreinke
Looks like he has written tons of very creative bots. They are all very
interesting ideas (e.g.
[http://randomshopper.tumblr.com](http://randomshopper.tumblr.com))

~~~
mturmon
Agreed:

[http://www.bostonglobe.com/ideas/2014/01/24/the-botmaker-
who...](http://www.bostonglobe.com/ideas/2014/01/24/the-botmaker-who-sees-
through-internet/V7Qn7HU8TPPl7MSM2TvbsJ/story.html)

------
sgentle
It would be pretty interesting to see one degree of abstraction up from this -
what sets of lines are close enough to match a certain word?

If you averaged over all those sets, would the resulting blobby heatmap
resemble the original word in a legible form? Or something else?

------
userbinator
I can imagine generating a few pages or even an entire book of this, and some
future generations attempting to figure out what sort of language it was
written in... reminds me of this:

[http://en.wikipedia.org/wiki/Voynich_manuscript](http://en.wikipedia.org/wiki/Voynich_manuscript)

------
cosarara97
I couldn't get that OCR to read my mouse-written E. It's a nice experiment
nevertheless.

~~~
amelius
Indeed, the underlying OCR seems to need lots of work still. It's no wonder
that the "reverse" operation results in such messy line art.

The demo is nice, though.

------
klausa
I highly recommend watching talk Darius Kazemi (author of Reverse OCR) gave at
this years XOXO:
[http://www.youtube.com/watch?v=l_F9jxsfGCw](http://www.youtube.com/watch?v=l_F9jxsfGCw)

------
emhart
It has been fantastic watching Darius' myriad experiments over the past few
years. His work always has a great mixture of whimsy and serious
experimentation.

------
MrBra
Nice. Finally computers approached the age of writing. :)

------
lucb1e
I can already imagine the innovation:

> Type over this text to prove that you are a computer.

> Human detected. Shoo, shoo!

------
Aaronneyer
Looks like my handwriting

------
z3t4
I can't believe OCR has not been solved yet. The only one even close is
OmniPage.

~~~
Tepix
Isn't OCR pretty good these days?

------
driverdan
Here's the source code on github:
[https://github.com/dariusk/reverseocr](https://github.com/dariusk/reverseocr)

------
jostmey
A generative model, although computationally expensive, would not suffer this
problem. Essentially a generative model can run in reverse, which means that
if you feed values into the output you get inputs that could explain the
output. Check out "Boltzmann Machines" for an example. There are plenty of
examples for the MNIST dataset of hand written digits.

------
k_sze
I think one of the problems is that the OCR assumes the images to be (English)
letters.

To be really really useful, the OCR would need to consider at least all
characters in the Unicode Basic Multilingual Plane. And then it needs to be
able to reject an image as containing any word, and then it needs to solve the
halting problem.

------
zwass
This reminds me of an experiment I played with using random search to "teach"
the browser how to draw characters:
[http://zwass.github.io/Learn2Write/](http://zwass.github.io/Learn2Write/)

------
bmh100
This actually seems like a great program for automatically generating
adversarial examples to improve OCR. A human could rate this text as being
illegible or legible. Each example can then be added to the training data to
improve its quality.

~~~
mng2
The Letter Spirit project (a Douglas Hofstadter thing) is sort of in that
vein, but less prosaic in its objectives.

------
eurleif
It would be neat to see the same thing, except using two OCR libraries instead
of just one, and requiring both libraries to be able to read the message. I
imagine the letters would start to look a bit less insane.

------
shangxiao
This is pretty cool, although it makes me wonder what the real world
applications could be. It does, at the very least, tantalise my curiosity and
gets me thinking.

------
achr2
Could this be used in a pseudo reverse CAPTCHA by showing a series of words,
and asking the user to say which is not human readable?

------
methyl
I wonder what would happen if you run this program letter-by-letter, possibly
the readability could increase.

------
mslot
I love algorithmic art.

------
Applico
very cool idea.

------
jdimov
What (if anything) is this saying about the quality of the OCR process?
Especially since none of these seem human readable.

~~~
pavel_lishin
I'm not bothered by it any more than I'm bothered by our brains' propensity
for seeing faces where there aren't any.

~~~
driverdan
That's a pretty interesting way of looking at it, machine pareidolia.

