
Extending Van Gogh’s Starry Night with Inpainting - lelf
http://blog.wolfram.com/2014/12/01/extending-van-goghs-starry-night-with-inpainting/
======
maho
Mathematica demos sure look pretty [0]. But I recently tried to get actual
work done with Mathematica and found the experience utterly ridiculous.

The high level functions, while nice for demos, never quite do what you need.
Want to display data vs. time with errorbars around the data? Sorry, no
function for this completely exotic use case, please hack a DateListPlot on
top of an ErrorListPlot [1].

The WolframAlpha integration "transparently" interprets code for you in the
background. This interpretation can change any day [2] and leaves you guessing
why your notebook doesn't run anymore.

The bugs I encountered were severe: One of the output function swallows the
"-" sign if the result is between -1 and 0 [3]. Currently I am battling file
corruption of notebooks that are saved on network drives and open while the
drive is disattached/reattached.

Had it not been for the awesome support by the mathematica. stackexchange.com
community, I would have quit Mathematica completely. Now I vowed to use it
only for small hacks. And, if I want to impress someone, for demos.

[0]
[https://www.youtube.com/watch?v=EjCWdsrVcBM](https://www.youtube.com/watch?v=EjCWdsrVcBM)

[1]
[http://mathematica.stackexchange.com/questions/18962](http://mathematica.stackexchange.com/questions/18962)

[2] The input

    
    
        Quantity[1, "Kelvins"]
    

is interpreted by Mathematica itself and yields an object equivalent to "1
Kelvins". The input

    
    
        Quantity[1, "Kelvin"]
    

however is interpreted "transparently" by Wolfram Alpha. A few weeks ago it
evaluated to "1 KelvinsDifference", but now has changed to "1 Kelvins". Don't
get me started on the KelvinsDifference vs. Kelvins thing, which are both
represented by the same symbol to maximize your confusion when your unit
conversions are failing...

[3]
[http://mathematica.stackexchange.com/questions/65600](http://mathematica.stackexchange.com/questions/65600)

~~~
leephillips
Really interesting, detailed comment. The problem is not that Mathematica has
bugs - all complex software has bugs (except TeX). The problem is that you
can't fix it, or even see how it works, because the source is secret. Nothing
will get fixed until the company decides to fix it. Access to source is not an
ideological imperative, it's a practical necessity.

~~~
cpach
I’m not convinced open source would really help. I get the feeling that
Mathematica’s underlying philosophy just isn’t a good fit for practical
programming.

~~~
JacobEdelman
I disagree. If it was open source I think enough people would write libraries
for it (or add-ons or whatever they would be called) that most if not
everything could be done with functions found online. In the rare event that
they could not be done you could just create your own. This is similar to my
annoyance over Wolfram Alpha being server side when some sort of client side
solution would be way better but would involve the source code being
available.

~~~
cpach
Sure, some limitations could be mitigated this way. But being open source does
not in itself make it a good language/platform.

~~~
ics
This is a point that many people seem to skip when discussing Mathematica. I
think it would be great if they open sourced all of the code but I doubt that
that alone would bring a surge of new package developers. If anything, I would
expect it to bring IPython, Sage, etc. people to pick around for useful bits
to reimplement in other projects.

------
jwise0
I wish that the Wolfram articles would describe a little more about what their
algorithms do, rather than the mechanics of how to use their primitives. The
articles remind me some of the era of web engines optimized for a specific
task -- there were plenty of MVC frameworks that work really really well for
writing a todo list in 25 lines, but doing anything more complicated than a
todo list meant dissecting the guts of the framework.

It'd be great to hear how they did an efficient implementation of texture
synthesis. But if you want to read at least about the basics, the dissertation
they linked [1] gets into the details on page 18, anyway, in section 2.2 (and
beyond).

[1] [http://www.logarithmic.net/pfh-
files/thesis/dissertation.pdf](http://www.logarithmic.net/pfh-
files/thesis/dissertation.pdf)

~~~
devilsdounut
Wolfram has a habit of creating blog posts about the test cases for their
algorithms.

~~~
jordigh
He also has a habit of not talking about how things work. All the time. Source
code, mathematical proofs, citations, all of these things are relatively
unimportant in Wolfram's world. He even thinks _you_ should not try to find
out how things work:

[http://reference.wolfram.com/language/tutorial/WhyYouDoNotUs...](http://reference.wolfram.com/language/tutorial/WhyYouDoNotUsuallyNeedToKnowAboutInternals.html)

~~~
JacobEdelman
Its a They, not He. The post isn't written by Wolfram himself :)

~~~
geofft
The Principle of Computational Equivalence says that Wolfram and his employees
are equivalent, right?

------
andrey-p
Worth checking out the rest of Yarin Gal (the PhD student whose work prompted
this post)'s extrapolated works of art:

[http://extrapolated-art.com/](http://extrapolated-art.com/)

Some of these look so seamless it's staggering.

------
thu
That's really neat. This makes me think to image analogies[0], something that
impressed me enough that I keep remembering it after 13 years!

[0]: [http://www.mrl.nyu.edu/projects/image-
analogies/](http://www.mrl.nyu.edu/projects/image-analogies/)

~~~
adamb_
Thanks for sharing this. I was really impressed by this example:
[http://www.mrl.nyu.edu/projects/image-
analogies/potomac.html](http://www.mrl.nyu.edu/projects/image-
analogies/potomac.html)

------
spot
afaik this technique was first implemented in the early 90s:
[http://draves.org/fuse/](http://draves.org/fuse/)

~~~
sp332
FTA: _The “TextureSynthesis” method is based on the algorithm described in
“Image Texture Tools,”[1] a PhD thesis by P. Harrison. This algorithm is an
enhanced best-fit approach introduced in 1981 by D. Garber in “Computational
Models for Texture Analysis and Texture Synthesis.”[2]_

[1] [http://www.logarithmic.net/pfh-
files/thesis/dissertation.pdf](http://www.logarithmic.net/pfh-
files/thesis/dissertation.pdf)

[2]
[http://dl.acm.org/citation.cfm?id=910609](http://dl.acm.org/citation.cfm?id=910609)

~~~
spot
is Garber's thesis (or better yet, a paper summarizing it) available online?

~~~
sp332
There have been lots of advances since, under the term "content-aware fill".
Lots of links from here [https://www.adobe.com/technology/projects/content-
aware-fill...](https://www.adobe.com/technology/projects/content-aware-
fill.html)

There's an oldish plugin for the GIMP called Resynthesizer, but there are
better algorithms out now. G'MIC has one called "inpainting".
[http://gmic.eu/repository.shtml](http://gmic.eu/repository.shtml)

------
mistagiggles
From the comments on the article:
[http://imgur.com/a/BfcK4](http://imgur.com/a/BfcK4)

Some more tests of the algorithm with less impressive results (though still
good looking)

------
thenomad
Extension algorithms are absolutely amazing - and incredibly useful if you're
a practical working visual artist.

I've personally found Photoshop's Content-Aware Fill to be so useful on films
and comics that it now gets its own credit, as:

"Matte extension: Content-Aware Phil".

------
special_ghost
Such an approach might be useful to the print industry for generating seamless
"bleed" around images, which is required to allow some margin for error when
trimming the physical prints.

------
trop
Not to be totally grumpy here, but this is classic mathematicians-think-art-
is-images-looking-like-famous-art. The expanded "starry night" isn't an
interesting painting, of course. And it adds nothing to Mr. V. V. Gogh's work.
It just mimics aspects of the extant work, in a spatially naive manner (e.g.,
what more do we learn except that the new rendition exists on a black mesa
surrounded by more village-with-church?).

SIGGRAPH-style research shouldn't validate itself merely because it teaches
the computer to adopt a cliched mode of image-making.

~~~
infinitone
Have you looked at the other examples? [http://extrapolated-
art.com/](http://extrapolated-art.com/)

Some of these are pretty impressive and do more than just pattern repetition.

~~~
TheOtherHobbes
I'm going to agree with the critics - I think you may be confusing technical
problem-solving with artistic awareness.

They're all interesting as technology demonstrations, but none of them improve
on the original compositions, and some of them are almost completely
unsuccessful as visual art.

------
djloche
Not to make this any less impressive, but doesn't the latest versions of
Photoshop have these types of features built in?

------
soperj
Only problem is, I've been to the village where Starry night was painted, it's
not that big.

------
JoeAltmaier
Its tiling the painting seamlessly, which is cool. Not really Van Gogh; but
Van-Gogh compatible?

