

Ask YC: Best approach to processing uploaded images - akon

How would one go about manipulating uploaded images on dedicated image server?  Every upload, do you use ftp to pull the images from web server to image server?  Any other alternatives that is available?
======
st3fan
Give them a unique ID, correlate it to the upload in your database. Then
upload the image to S3. Also drop a description of what needs to be done in an
Amazon Simple Queue.

Your Image Processors poll the queue. Receive the work order, fetch the image
from S3 and then store it back there or do whatever you need to do with them.

Loosely coupled. Scalable. Fault tolerant. Oh and simple to implement.

Ping me if you need help with this :-)

------
brianm
Solution will change as size and needs of the deployment change. If accepting
images from the wild, dedicated image processing is a Good Idea because there
are some images out there that to nasty things to any/all of the image
manipulation libs available.

For giving the image to the image processor, I kind of like just using NFS.
Simple, works up to a pretty big load, and when it does you will have a better
feel for your operational needs -- everying _is_ different.

In terms of libraries to use, I think ImageMagick (via C!) and JAI (Java) are
probably the best available. There are more, and YMMV, but I'd start with one
of them. In terms of designing the app, well, that depends on what exactly you
need!

~~~
st3fan
/* NFS dorks */

------
gojomo
If the volume and processing effort is high and variable enough that you
_sometimes_ need multiple processing machines, Amazon EC2 + SQS + S3 may be of
help. I think that's what SmugMug uses.

~~~
mace
Here's a recent and very good blog post by the Chief Geek at SmugMug:

    
    
       http://blogs.smugmug.com/don/2008/06/03/skynet-lives-aka-ec2-smugmug/

------
logainablar
this is for plurk?

