
Does CloudFlare's Railgun really work? (Metrics) - dknecht
http://blog.cloudflare.com/railgun-gives-our-ecommerce-sites-the-edge
======
youngtaff
Railgun's an interesting idea but unfortunately the post doesn't really give
us enough information to really judge it's effectiveness.

For example we don't know enough about the 'test' environment such as how much
the pages vary from user to user, or from page to page which will have a large
impact on deltas.

There's also no detail on how other optimisations might perform e.g. serving
all dynamic content through CDN so reusing CDN to origin connections, whether
it might be possible to specify cache lifetimes for the dynamic pages etc.

There's a short mention of some performance numbers at the start but then the
article focuses on the reduction in outbound traffic (which is probably a good
thing) so we've no idea whether Railgun, the other optimisation services that
CloudFlare offer of just using a CDN is making these differences.

------
chrisholland
I can answer some of these questions:

1) outbound traffic reductions are 100% driven by railgun. If you use
cloudflare with railgun disabled, you're using it much like a traditional
dynamic acceleration product by Akamai or EdgeCast, and it does not reduce
your outbound traffic, because the entire response payload is proxied out of
your data center to the CDN's edge. With railgun, the response body is
crunched down to a diff. My post goes in good details into the figures we've
seen with regards to off-peak organic long tail browsing and peak traffic
driven by campaigns. Off-peak I'm down to about 1/3rd to 1/5th of my previous
outbound consumption, and on-peak gets down to 1/10th.

2) you can browse the site to see the various compositions of our page types:
[http://www.luxurylink.com/](http://www.luxurylink.com/)

Performance-wise:

3) I cannot apply any "full-uri" caching rules to any of my pages for various
reasons: for one, any URI might send a completely different payload based on
your device. You can bookmark a uri on your phone and open it up on your
laptop after bookmark synchronization, and it always looks as beautiful as it
should look for that device. Second, we've started to introduce dynamic
merchandising tailored to every single user at every page load. Third, this is
tied to our BI framework which does need every single request to fully go-
through such that every page view continues to feed our learning algorithms.

2) with railgun disabled, performance-wise, I've measured an average 25%
reduction in overall time to download a dynamic HTML document from new york
and it speaks to the traditional WAN optimizations you've mentioned, which is
on-par in features _and_ performance gains with similar products I've used
from Akamai and EdgeCast, for, you know, several grand a month base platform
fee plus bandwidth metering. Except that cloudflare before railgun was only
setting me back $20/month plus $10/month for 2 more sites (familygetaway and
vacationist)

3) with railgun enabled the figure is now closer to a 35% reduction in
download time from New York and 45% from London.

4) But as I alluded to at the end of the post, as far as what is felt by the
end user, no railgun vs with railgun means perhaps a 100-200ms difference on
the base HTML document total load time. And if you look at overall page load
time, you're going to spend several whole seconds downloading and rendering
scripts, CSS, images. So perceived end-user benefit is likely limited. If they
baked their compression into browsers I could imagine a performance
improvement on-par with compression achieved, because the "diff" would travel
all the way down to the client, instead of stopping at the CF proxy nearest to
the client.

Still there are strong correlations between reduced HTML load time and
increased Google crawl-rate on your site, which is a generally good thing.

Overall: Railgun has allowed me to insta-drop one whole bandwidth tier at my
ISP, which is financially _material_. By the time our growth gets me back into
the higher tier, i'll be serving 5 times the audience I used to serve for that
same amount of outbound consumption. We're not cloud-hosted, but if we were,
these bandwidth cost benefits would of course be a lot more linear.

