

Photoshop CS5's new "Content-aware Fill" - swombat
http://www.youtube.com/watch?v=NH0aEp1oDOI&feature=player_embedded

======
hkuo
With all the crap Adobe gets for Flash, this is where their primary area of
expertise lies. I've been using Photoshop from when there were no layers and
very limited number of undos. They never fail to deliver something incredibly
useful and time-saving with each iteration.

~~~
fnid2
I'm still using Photoshop 4 that I got in 1996 or so. There are layers, but
only one undo. I don't use it enough to justify the upgrade, but I have done a
lot of everything in this video from time to time and it is extremely time
consuming. This is quite an advancement. I usually give Adobe a lot of crap,
but this is really very nice.

But I wonder... are tools like this to manipulate reality making really less
and less interesting. Are losing our love of reality because of the dreams we
can so easily create?

~~~
jacobolus
> _it is extremely time consuming._

Most importantly, it’s not especially interesting. Figuring out what needs
fixing takes a human eye, but actually cutting a distracting pole or bit of
lens flare out of a picture is work for a patient robot.

> _Are losing our love of reality because of the dreams we can so easily
> create?_

Um... no?

How do you define “reality”? If it depends on what can be shown in
photographs, then the time to have this discussion was around 1850.

~~~
brfox
Lens flare is kind of nice sometimes. I like how CG movies add it in to make
it look more realistic.

------
nso
The demo is totally mind blowing. It's so good it almost seem fake (not saying
it is).

~~~
TNO
The bottom right area of the last image example makes me wonder...

~~~
pbhjpbhj
At 4:44 it just looks like they masked off an area and then switch off the
mask. Looks fake because it's so good - without actually using it one couldn't
tell.

~~~
dutchflyboy
Do you mean the fact that it seems to load in two parts? That's quite normal
behaviour, as the images that are used probably have quite a high resolution
and the downscaling takes some time. Just try taking any picture with a nice
big resolution and then applying an effect. It will load gradually and not at
once.

If you meant something else, could you explain it a bit clearer?

~~~
pbhjpbhj
It looks potentially fake because its such a good final result. In short it's
almost unbelievably good (in the video).

Elsewhere in the thread I repeated the experiment on one of the images with
resynthesizer GIMP plugin and got very good results too, but not quite as
good. Of course I could probably have chosen an image which would have
appeared to produce equally good results...

------
pkulak
You have got to be kidding me. Amazing. Every photograph will have at most 50%
original area after this gets into the average photographer's hands.

~~~
potatolicious
And this is why I shoot mostly film... with digital it's too easy and tempting
to resort to parlor tricks instead of taking a genuinely good photograph.

~~~
frio
I just scored a semi-nice film SLR the other day; it used to belong to my
parents but they've gone digital and it hasn't been used in yonks. It's
certainly nowhere near as programmable as even a cheap digital SLR, but
whatever, it's fun to use.

The difference, in terms of taking shots, is night and day - with my digital
camera, I'll snap 10+ shots of the same thing to try and find the right one;
with this, I spend five minutes angling around and trying to find the right
shot off the bat.

I'm not a good photographer by any means; in fact, I'm substantially below
average if I'm honest about it. But raising the bar to entry certains makes
you focus on what you're doing a lot more.

Which I guess is a long and winding way of saying I agree. It's more fun when
there' more effort involved.

~~~
potatolicious
I feel the same way - there's a greater sense of accomplishment when you allow
yourself only a single opportunity to capture what you see in front of you.

In photography parlance we call it chimping - i.e., jumping around like a
monkey and clicking the shutter like crazy, trying to get the right shot. Even
if you get the right one, you have no idea if it's your skill, luck, or just
law of large numbers...

When I shoot film I feel more purposeful - I rarely ever take more than a
single exposure of any one particular thing - it also teaches you to be
patient, and hold the shutter until you know you have the right shot. On
rapid-fire you will often get a good picture, but no idea what makes it good,
whereas with film I'm aware that this picture works because, say, I waited
until there was no one in the background.

My current favored camera is a Leica R4s - only manual and aperture priority
modes, manual focus, manual aperture control, and only 2 metering modes (you
only ever need one - center-weighted). When you strip a machine down to the
bare minimum it opens a lot of creative doors.

~~~
elpuri
Usually chimping just means checking your shots from the LCD.

~~~
angusgr
Yeah. I've heard "clicking the shutter like crazy, trying to get the right
shot" referred to as "spray and pray", though. ;)

------
MHordecki
That made my day. I'm kind of worried though that the examples are hand-
picked, and by default it doesn't look so nice. I would _love_ , though, to
know the algorithm behind this.

~~~
noidi
<http://www.youtube.com/watch?v=dgKjs8ZjQNg>

~~~
noidi
Whoops, I got beat by wallflower. Should've refreshed the comment page before
posting...

~~~
jsonscripter
Don't worry, I like the direct link to the video better :)

~~~
wallflower
As do I.

Seriously, I love the Google-foo/showdown challenges that sometimes a HN
reader poses. This search was easy.

------
froo
I was pretty impressed just by the first part, but when it got to the desert
and finally the cloud, my jaw literally was open by the end. Totally
gobsmacked.

Unbelievable.

~~~
eru
Though the patched up desert image looks a bit boring. I guess there's a
reason they went over this so quickly.

~~~
gridspy
Still a huge start.

~~~
eru
Indeed. Especially compared to the lack of effort on the user's part.

------
sev
Unbelievable. My head is hurting from trying to imagine the concept behind the
algorithm to do this. The desert and sky recreations are especially mind
boggling.

~~~
jcnnghm
Mine too. My initial (naive) thought on developing something like this would
be to have advanced users perform the tasks that I wanted to automate, and
recording the series of actions that they took to perform the task so I could
analyze it and try to find the patterns.

How is something like this actually developed?

~~~
joshd
noidi's yotube link explains it pretty well. It searches the rest of the image
for sections that are similar to the edges of the deleted area and
extrapolates from there. I imagine for large deleted areas this would have to
be done multiple times to build up the deleted area. It woudl be kind of lie
markov chains, but for image data.

The current patch tool in Photoshop can already adjust contrast and hue to
make a patch source match the patched area, so when the algorithm is searching
for similar areas to use as a source it can just focus on the image details
and not the lightness or actual colours.

------
stinky613
This can only be described one way: devil magic

~~~
sbhat7
"Any sufficiently advanced technology is indistinguishable from magic."

------
WildUtah
Wow. Now I need this in the GIMP.

~~~
patrickas
This is amazing and all, but this technology has been available for GIMP users
for years. I have been using something very similar since 2004 or 2005.

<http://www.logarithmic.net/pfh/resynthesizer>

This page <http://slacy.com/blog/2005/04/cool-image-processing/> shows you how
to use it to remove an object which will be automatically replaced by
surrounding texture (similar to first example in video)

This page <http://schwarzvogel.de/resynth-tut-sa.shtml> shows an example of
growing and image by auto-filling the missing edges around a photo (similar to
last example in the video)

I suppose the new algorithms in PS are better, but still it's been over 5
years now :-)

~~~
danudey
Interesting. Resynthesizer looks a lot like the technology (and end results)
behind the spot healing brush tool, introduced in Photoshop CS2 in 2005.

------
ansonparker
I'm pretty sure this isn't going to be in CS5. The engineer says as much and
John Nack, who just posted it, says it'll be in a "future" version of
Photoshop.

Which, if I'm right, makes this a horrible way to steal CS5's thunder!

~~~
superchink
Any confirmation on this?

------
aphyr
This is absolutely amazing. The PS guys are really pushing the envelope of
image editing. I think it's really cool that the editing process is more
structural, more high-level, than working with the basic tools. The lens flare
removal, especially, is an incredibly common and tricky problem.

At the same time, I am glad that the effects are still immediately visible,
even in a low-res youtube video. It's nice to have hints of what the original
actually looked like, some times.

~~~
illumin8
It is indeed a great algorithm. I think similar algorithms will be used to
"create" music in the future, although I shudder to think of how terrible pop
music will be when they can just create "content aware" music by selecting a
few bars and generating the rest.

------
Zakuzaa
Only about 7 days left to the April Fools Day.

------
aero142
This should make it easier for Iran to have successful missile launches. As
long as they can afford the upgrade license.
<http://dvice.com/galleries/iranphotoshop/iranmissile_1.jpg>

------
bho
wow. i'd definitely be interested in seeing the full image and seeing if there
are any weird artifacts going on at the transition area between the selected
border and the generated "content-aware" fill. from what i can tell , this is
close to magic.

------
Sindisil
Outstanding! On a quick look back when it came out, I didn't see anything in
CS4 to get me to upgrade from CS3.

I'm thinking I need to reevaluate with CS5. If nothing else, perhaps the total
diff from CS3 -> CS5 will now be worth the upgrade.

------
Tichy
It reminds me of that other algorithm that went around a couple of months ago,
where they do similar things by combining photographs on the web. Might have
been from Google, Microsoft, or some independent researchers.

~~~
sparky
Was it Sketch2Photo from Siggraph Asia 2009 (
[http://cg.cs.tsinghua.edu.cn/papers/SiggraphAsia_2009_sketch...](http://cg.cs.tsinghua.edu.cn/papers/SiggraphAsia_2009_sketch2photo.pdf)
)? If so, the authors are academics.

~~~
Tichy
Yes, exactly - thanks!

------
wenbert
The only thing that is keeping me from using Linux fulltime is Photoshop.

I had to literally get my officemates to see that video. It is unbelievable.

------
ThomPete
I guess it wont work with complexity behind complexity.

Two people, one in front of the other for instance.

But it's damn nice for most situations.

------
elblanco
The panorama example at the end is fantastic. I hate cropping all of my
panoramas down, I loose so many great things on the edges.

------
voidpointer
This is going to be really nice when cleaning up old photos with dust and
scratches

------
rsully
This is an amazing feature. The tree was removed perfectly. Cannot wait to see
CS5

~~~
jcl
That was actually one of the less impressive things the demo did, since it
replaced the tree with what is essentially a smooth gradient. This is
something that can be done by early image inpainting algorithms, such as this
one from SIGGRAPH 2000:

<http://www.iua.upf.es/~mbertalmio/restoration.html>

~~~
jerf
Yes, but you have to give it some credit for _deciding_ to use a smooth
gradient, when it is clearly not the only choice. (Also, it's hard to tell in
this video exactly what it really replaced it with, at the pixel level.)

------
dutchflyboy
I think I just lost my jaw!

------
benologist
... content-aware fill, or MAGIC?

------
quadyeast
hope they do away with the buggy/frustrating flash panels

------
makmanalp
Thinking how many so called "graphics designer"s have just been replaced by a
machine makes me smile and shudder at the same time.

~~~
ThomPete
Not graphics designers.

The hours graphic designers have do spend one something they hate so much has
been replaced by a machine. And for that we are very grateful.

~~~
thorax
These kind of problems for graphic designers are like software bugs to
developers.

Now they'll get to spend more of their time "coding features" instead of
debugging.

