

Improving Dropbox Performance: Retrieving Thumbnails - lowe
https://tech.dropbox.com/2014/01/retrieving-thumbnails/

======
herf
Mostly you shouldn't have to hide backend latency by reordering, and if you
don't reorder, you can just use JPEG image strips/sprites, which don't need
double compression. JPEGs also stream, but in an order determined by the
client, not the server.

In my experience, Dropbox is quite a bit slower (by 10-20x) than services
optimized for serving images - using 1500ms to deliver a 25k file is very slow
on the web, but it is very common when requesting files via the Dropbox web
API. (I can only speculate about the reasons, but Amazon's s3 has big
latencies too.)

~~~
ziga
We thought of spriting the thumbnails on the server, but this introduces a few
problems:

\- All thumbs would have to be available on the server before the sprite can
be encoded, which defeats progressive rendering. (Progressively _encoding_
JPEG might be a possibility, but this isn't something we explored.)

\- The additional cost of encoding/decoding a large image on the
server/client.

\-- Ziga (author)

~~~
herf
Yes, you can stream JPEG pretty easily as long as you know the output size.

Also to combine JPEGs into a sprite, you can avoid all DCT/iDCT/color
operations and use only the Huffman portion of the codec (decode, concat,
encode). Since Huffman throughput is a considerable multiple of gzip (~10x?),
I think you'd be quite a ways ahead, for memory and time.

------
erikpukinskis
I don't really use Dropbox, but I was randomly browsing through mine looking
for a photo the other day and it was a "holy shit" moment. It's really fast to
scroll through a huge number of photos. I had been scrolling through Flickr a
few minutes before and it was a massive difference. Flickr was very much a
scroll-and-wait experience.

~~~
rhizome
For me, Flickr is basically unusable. Slideshows are impossibly slow,
information is positioned outside the browser viewport, and simply bringing up
an image-page is reminder of dialup. I can only blame inertial lockin for
their usage numbers.

------
drcross
On the functionality end of things I think dropbox should be stop new feature
development until heavy UX is done with large unbiased test groups. The
application is simple and it works pretty well but recently I've noticed
different barriers to use, notifications that I'd prefer not to see just
getting in the way. It's a result of Zawinski's law, it's happening to gmail
where they think adding more bells and whistles is needed but it's not, in
many case user experience declines, for example playing a youtube clip now
blacks out the rest of your gmail page while you watch it. Application
developers need to learn that less is more.

~~~
dropboxer
hi there, I work at dropbox and am curious to learn more. what things did you
run into that were unexpected and retracted from the UX? hopefully we can make
it better!

------
cstuder
What about a sprites-like system?

Since you pretty much know which pictures the user will request, just glue the
next 100 thumbnails together into one big picture and then take it apart again
on the device.

~~~
elwell
[https://news.ycombinator.com/item?id=7133035](https://news.ycombinator.com/item?id=7133035)

------
b0b0b0b
Were you already serving thumbnails over https? Or were they served over http?

edit: I was also wondering whether you can skip downloading some thumbnails
based on the velocity of the scroll.

~~~
ziga
We were already serving over https (we've implemented HSTS some time ago).

And good observation about scrolling -- we do in fact queue, prioritize, and
skip thumbnails based on scrolling.

------
zaptheimpaler
Basically, they chunk thumbnails - client asks for the next 10 thumbnails in
one request, server sends back 10 thumbnails in the response.

------
zoren
Base64 and then gzip just to make it easier on javascript? Seems hard on
dropbox servers.

Why not use a multipart binary response?

~~~
acdha
gzip is basically free on modern server hardware – if it makes life easier on
mobile devices, this is a pretty easy tradeoff.

------
WhitneyLand
I think they could still do better without using SPDY. Going to try and make
time to test a different approach.

------
cmicali
tldr: use SPDY, but until more universal support a similar custom hack is
proposed

Neat hack though.. frustrating how much perf is lost due to HTTP sometimes

------
tinganho
I just see one problem with this solution. You can't create Retina images out
of base64 strings on any web browsers today.

~~~
madeofpalk
> You can't create Retina images out of base64 strings

Why not? There's no such thing as a 'retina image', they're just a larger
resolution. There's nothing stopping you from requesting a @2x set of
thumbnails

