Ask HN: HTTP compression from client to server?

[aristus]

I never noticed before, but it seems you can't send compressed data from an HTTP client back up to the server. You can't set the enctype to 'deflate' or 'gzip'. If you compress it yourself, say with lzw in Javascript, the %FF url-encoding that happens before data goes on the wire defeats the purpose. Since most clients have a much slower upstream than downstream, compression seems like it could be a nice win for webapps.

So, clever hackers, can you think of a way to do it with existing protocols and implementations? I offer a frosty beer for a 2:1 or better solution. Points off for using Flash.

[cperciva]

I never noticed before, but it seems you can't send compressed data from an HTTP client back up to the server.

Not true. It is perfectly valid for an HTTP client to send an entity (as part of a PUT or POST, for example) with Content-Encoding: gzip. RFC 2616 specifically mentions this case, in fact.

That said, in most situations this isn't worth the effort. Until you've established a connection to an HTTP server, you have no way of knowing if it will accept any particular Content-Encoding; a server which doesn't understand the encoding of a provided entity SHOULD return a "415 Unsupported Media Type" response code, but trusting that every server you'll ever encounter will respond correctly is probably overly optimistic.

Of course, there's always the ultimate in portability: Send your data with a Content-Type of "application/octet-stream", and decompress at the back-end.

[aristus → cperciva]

You're on to something here, but I've never been able to get a browser to send a form POST with a custom Content-Type. Maybe something AJAXy gives you more control over the headers.

[briansmith → aristus]

No, browsers won't do it. But, you can do it in Java and I think Flash 10 lets you do it too.

[bprater → cperciva]

So the big question is: can you compress this data using Javascript?

Has anyone seen a packing function?

[aristus → bprater]

lzw-compressing the data then base-64 to armor it might give you 1.5:1 on stuff over 1KB or so. I ran into trouble with Javascript getting clever about bytes and Unicode characters.

But, yeah, some efficient compressor and then an efficient url-safe encoding might do it.

[wmf → cperciva]

Of course in an AJAX app you know exactly what the server supports.

[yan]

How much data are you submitting? Usually, the overhead of everything else that goes on outside of your browser is a magnitude slower than the data transmission itself. Unless you're submitting hundreds of kilobytes, I wouldn't worry about compression.

[aristus → yan]

Assume it's 50KB going up over a 50kbit uplink. But even turning a 5-packet transmission into a 3-packet transmission could be a win in some situations. I don't have a burning need for this, but I did run into a situation where it might be nice, and I couldn't figure out how to do it.