

Solving the problem of multiple writes to the database - amrithk

Hi all, 
This might perhaps be a simple problem but I was wondering if people had any solutions for this.<p>I have a form which allows people to upload images. When a user uploads an image, I save to a database, 
1) a modified name for the image, 
2) a caption for the image, 
3) the id of the user who uploaded the image, 
4) Some other information that are used for image retrieval purposes<p>The form allows users to upload up to 30 images. I do not want to write to the database each time 
the user wants to upload an image. Rather, I want to store each upload request (plus all the associated information I need like captions etc) and make a large write when a user has confirmed that he/she is satisfied with all of their choices.<p>How can I do this easily? 
1) I have considered storing the attributes and information associated with each image in cookies and retrieving and processing cookie information after the user has confirmed his/her choices. However, I am not comfortable with this option as I cannot control what happens to the cookies when they are stored in the client machine.<p>2) Another option is to obtain many hidden fields (more than 30) in the HTML form and fill these up with the associated information I need. I could then process the information in these hidden fields before writing to the database. However, this seems pretty clumsy.<p>3) A third option I was considering was using Amazon's Simple Queue Service to queue image upload requests and then unqueue and process the messages after a user finished making his/her choices and hit a confirm button.<p>Which solution among these 3 is the cleanest and most flexible implementation? Are there other solutions for this problem?<p>Thanks all.
======
olefoo
This is what sessions are for, maintaining volatile state that may or may not
be written to a more permanent storage.

If your application stack does not have a session implementation that 'Just
Works!'(tm) for you, you will have to implement one yourself.

Generally it will look something like a cross between your solutions 1 and 3,
you create an opaque session key which is stored as a cookie and is used to
retrieve the data from the session store. What you use to store session data
is going to vary depending on how valuable the data and on how tragic failure
is.

Avoid hidden fields in the html, they are a pain to maintain and will cause
you problems because the temptation is to trust that their contents are what
you put into them... it also means that you are repeatedly processing the same
data. In sum; hidden form values make baby jesus cry.

~~~
amrithk
Thanks. It seemed to me in the beginning that HIDDEN fields were a very clumsy
solution. Now I understand why they indeed are.

We use PHP as our server side scripting language. I was thinking of using the
$_SESSION variable but attributes and pointer information for 30 image uploads
per user * no of concurrent users seems like a huge drag on server memory.

I'll probably just write the pointer information to the database for now after
each upload. It might make the web app slower by a few milliseconds but it
reduces unnecessary complications

~~~
olefoo
PHP uses files for storing sessions; you do want to restrict where on your
site you are using sessions as they do have side-effects. Most notably any
page served in a path configured for sessions will have an expiry time in the
past and a bunch of no-cache headers; for a dynamic page that is the behaviour
you want, make sure it's not being applied where it should not be.

------
hs
this is what i did:

I have a form that: user can upload as many as s/he wants user clicks submit
the browser then sends a stream of data one time

on the server: regex find . _multipart/form-data;._ boundary=.* from
CONTENT_TYPE to make sure it's the right stream save the stream into temp-
file, read the temp-file

split file-stream based on . _filename=\"(._ )\".* into chunks afterward,
"calibrate" the chunk (determine start & end write positions) then loop and
write the chunks into specific filenames provided in the stream

so there are 1 upload stream and 1 load n+1 write operations

I wonder if it helps since I don't use php (hint: newlisp)

~~~
amrithk
Thanks for the suggestion. So I would basically write the pointer information
(that would have been written to the database) to a temporary file instead? Do
you think this improves performance significantly?

------
gaius
Why not write to the database and just not COMMIT until ll the images have
arrived?

~~~
amrithk
Not sure what you meant. Could you elaborate? The form basically accepts input
until a user reaches the 30 image limit. A user can then hit the confirm
button if he/she is satisfied and wants to commit all the changes.

~~~
amrithk
Also, I didn't want to write to the database each time someone uploads an
image. I just wanted to queue all these writes and do it at once in the end
when a user commits to make all the changes.

That would save some processing time right? Thanks.

~~~
mechanical_fish
_That would save some processing time right?_

If premature optimization is the root of all evil, I'm afraid you are halfway
to hell.

This is not a good reason to make your app more complicated than it needs to
be. For one thing, it isn't even easy to tell if your statement is true, let
alone _how_ true it is. (i.e. it may be true that you save a few milliseconds,
but is that even going to be significant?)

If there is a _UI_ reason why you have to have the user specify 25 uploads on
one HTML page and then do them all at once, that's one thing. (I'd use the
big-pile-of-hidden-HTML-fields approach; that has the fewest moving parts,
it's simple to conceptualize, and I don't think the other methods will make a
big difference from the user's perspective.) But don't try to work around your
DB until you _absolutely have to_. Use it. That's what it's there for.

~~~
amrithk
That's helpful. Thanks. Maybe I am overthinking too much about optimization.
It doesn't seem like updating a bunch of columns with text will result in
significant time savings.

