

Thoughts on Image Processing in JavaScript? - ivankirigin

I intend to work on javascript tools to do image processing at DevHouseBoston.
<a href="http://devboston.pbwiki.com/Dev3Projects" rel="nofollow">http://devboston.pbwiki.com/Dev3Projects</a><p>The goals are to build an open framework, to learn more JavaScript, and eventually to implement that crazy dynamic image resize method that's been floating around the internet.
<a href="http://www.faculty.idc.ac.il.nyud.net:8090/arik/IMRet-All.mov" rel="nofollow">http://www.faculty.idc.ac.il.nyud.net:8090/arik/IMRet-All.mo...</a><p>I thought I'd ask this group if they knew of any existing tools to use. A cursory google search found little.<p>This might be easier to do in Flash, but I want to learn more JavaScript and get the sense that it will be more portable this way. Any thoughts on this issue?
======
waleedka
JavaScript in a browser doesn't give you enough low-level access to manipulate
images. A good compromise is to build the UI in javascript, and have the image
processing on the server. Then, use AJAX to give the user a nice and quick
experience that feels as if the processing is happening on the client. Of
course, you won't have as dynamic a behavior as if it's all on the client, but
that's probably the closest you can get with JavaScript. Other options include
using a JavaApplet that runs on the client, or a .NET component (IE on Win
only).

~~~
ivankirigin
Right, I've been told a number of times now that I can't get access to the
image buffer in JavaScript. That's too bad.

I don't believe it can be made asynchronous and real-time. I suppose
JavaScript could be used to control which resize scheme is used. While the
bounds are changing, just scale and warp the image as a browser would today to
make it fit a size. Once it stops changing, do an asych call to get a resized
version of the image, with whatever fancy resize is needed.

I'd like to avoid .NET and Applets. Flash might be a reasonable solution.

~~~
ratsbane
I think you're right. I think the options for image processing on the client
side are currently limited to VML ( <http://en.wikipedia.org/wiki/Vml> ) for
IE or for Safari/Firefox/Opera, SVG or Canvas (
<http://en.wikipedia.org/wiki/Canvas_>(HTML_element) ) and from the little bit
of tinkering I've done with both, I don't think you can read the values of
pixels of image buffers with any of those options. If anyone can disprove this
I'd be very glad to hear it. You can with Flash or Java applets but both of
those ideas have other problems as well.

It certainly would be useful if it were possible.

I've also been trying to figure out how to control a scanner from in the
browser. I'd like to have something on a page which would scan a document,
display the image in the browser, and upload it to a server. I was hoping I
could do it with Flash, because Flash does have a TWAIN interface, but I've
not been able to get Flash to recognize scanner devices, only cameras. I'm
starting to think applets may be the answer to that, but that seems like a
long road to start on.

Does anyone have any ideas about controlling scanners within browsers?

------
henryw
I don't know if you can actually modify the image, but you could do things
like change image width and height properties, or use css in ways to show
manipulated image.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"<http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd>"> <html
xmlns="<http://www.w3.org/1999/xhtml>"> <head> <meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1" /> </head> <style> body { margin: 0 }
</style> <body> <script src="<http://particletree.com/scripts/prototype.js>"
type="text/javascript"></script> <div id="image_container" > <img
src="<http://www.google.com/intl/en_ALL/images/logo.gif>" id="image" style="
position:absolute; " /> </div> <input type="button" onclick="resize1()"
value="double size" style="position:absolute; top: 260px;" /> <input
type="button" onclick="resize2()" value="morph right side"
style="position:absolute; top: 300px;" /> </body>

<script> var i = $('image'); var oldWidth = i.width ; var oldHeight = i.height
;

function resize1() { i.width = oldWidth + oldWidth ; i.height = oldHeight +
oldHeight ; i.style.clip = 'auto'; if ($('image2')) {$('image2').remove();} }

function resize2() { i.width = oldWidth; i.height = oldHeight; i.setStyle( {
clip: 'rect(0px 138px 110px 0px)'} ); new Insertion.Bottom( 'image_container',
'<img src="<http://www.google.com/intl/en_ALL/images/logo.gif>" id="image2"
style=" position:absolute; " /> '); i2 = $('image2'); i2.setStyle( { clip:
'rect(0px 552px 110px 276px)' , left: '-138px' } ); i2.width = oldWidth +
oldWidth ; i2.height = oldHeight; } </script> </html>

You could use overflow:hidden instead of clipping. Then find the math formulas
for the resizing, attach drag handler to image edges, overlay semi-transparent
layers to emulate effects. To save it in the end though the user has to take a
screenshot or your backend script has to process it for the user.

------
ivankirigin
I'm told they've filed a patent on the method they used. Often a component can
be changed in the actual implementation to avoid infringement, but that means
more work.

Also you can find a paper for the work here:
<http://www.faculty.idc.ac.il/arik/>

------
alex_c
My first instinct was "no", but then I remembered this:

<http://www.abrahamjoffe.com.au/ben/canvascape/>

I don't know what kind of image processing you want to do, but the demo above
definitely pushes the boundaries of what I thought JavaScript can do. I
haven't had the time to dig in and learn how it works.

Really simple stuff - moving, resizing, cropping - is trivial to do with
JavaScript and server-side with something like ImageMagick.

No matter what, beware of IE6. I've only recently come to truly appreciate how
slow and buggy it really is. No idea if IE7 is better. So I'll change my
answer to "Maybe, but not in IE".

------
fauxto
Image processing is not possible with JavaScript.

~~~
ivankirigin
What do you mean not possible? You can't touch pixels in an image buffer?

~~~
fauxto
Is it possible that you are referring to Java rather than JavaScript?

~~~
ivankirigin
No, I just didnt know before I asked whether browsers let JavaScript have
access to the low level image information.

I mention below the best options: server side or flash.

------
ed
Hey Ivan, JS is a seriously crippled language when it comes to doing low-level
processing. Like others have mentioned, cropping and resizing are trivial;
however, to do anything even slightly sophisticated I'd suggest looking into
flash.

Flash 8's image API -- specifically the BitmapData class -- can provide some
low-level access to an image buffer (however the image must be round-tripped
from localhost to the server if you want to edit local images).

Alternatively, IE provides some proprietary image filters which can be used to
process images (but obviously not in a cross-browser way).

~~~
ed
Ok, to counter my bummer parent post here are some things you COULD feasibly
implement in javascript:

\- color tinting (overlay semi-transparent colored elements on an image)

\- image cut-and-paste (like cropping, place some bounded area of an image in
a div with hidden overflow [the copy] and allow that to then be placed over
the original image [the paste])

\- image gradients, drop shadows, reflections (but these have been done to
death and are usually too sluggish to use in a production app)

\- some neat UI tricks like scroll wheel zooming and gesture-based panning
(mootools has some demos which show what's possible - demos.mootools.net)

\- if you really wanted to kill yourself, you could use flash (degrade to a
server-side script) to build a pixel by pixel representation of the image
using floated divs and background colors. Then create a JS singleton with your
desired methods to be the image buffer's API. Thus, you're granted pixel-level
access to the image and could (slowly) do whatever sort of processing you
desire in JS.

~~~
ivankirigin
Wow. Those options are bleak. I'd certainly learn more JS in the process, but
it wouldn't be reusable or scalable.

