

Making the Switch from Amazon Cloudfront to Fastly - mayop100
http://blog.firebase.com/post/28352448484/making-the-switch-from-amazon-cloudfront-to-fastly

======
jconley
Wow, Firebase needs a new static asset management system. The fact that entire
set of static assets have to be purged and can't be served with a 1YR cache
timeout is a big red flag. Independent file versioning, whether through a
version number or hashing, should be put into play.

~~~
vikrum
That sounds like an interesting approach. We push out breaking changes to our
server and client libraries several times a week. Since we invalidate our CDN
on each deploy and set headers indicate as such, we can quickly and seamlessly
migrate our end users without having them modify their <script> include.

At present, our library is unlike other mature javascript libraries that can
make use of a very high cache timeout. How would the 1yr cache timeout work
for the library that is changing several times a week?

~~~
moonboots
Steve Souders blogged about how to serve scripts with far future expires
without updating the client's script tag [1]. This technique initially loads
the (potentially stale) js from the client's cache and checks for updates in
the background, updating the client's cache if a new version is detected.

[1] [http://www.stevesouders.com/blog/2012/05/22/self-updating-
sc...](http://www.stevesouders.com/blog/2012/05/22/self-updating-scripts/)

~~~
jconley
Yes, this is a great approach for javascript code. Usually, like in this case,
there is something that will be faster than a round trip to check for if-
modified-since or the ETag, which I would assume is used to do the
invalidation in this case. Of course, YMMV, and if fastly is close enough to
the edge and there aren't very many requests, this kind of optimization won't
be required.

There might also be an opportunity to modify what is served from the
customer's sites, depending on how all that is architected.

------
bjt
We use CloudFront at work. It works well for us. We do a couple things
differently though:

1\. Instead of trying to purge all old files and upload new ones, we use
cache-busting hashes in our static file names. This solves the stale-cache
problem even for browser caches, not just CDN edge servers. Luckily, Django
makes it easy to do this
([https://docs.djangoproject.com/en/dev/ref/contrib/staticfile...](https://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#django.contrib.staticfiles.storage.CachedStaticFilesStorage)).
2\. Where we care about gzipping (CSS and JS primarily), we use our own server
for the origin instead of S3.

------
alinajaf
Is there a good reason you don't append content hashes to compiled asset names
(images, stylesheets, js)? That would remove the need to do cache
invalidations manually, and comes out of the box with web frameworks like
Rails.

Also, CF supports custom origins so you can point it to your web application
and save all that screwing around with S3.

~~~
vikrum
The main reason we can't easily use hashes to bust the cache is that the
includes for our static assets (the Firebase JS lib) are happening inside our
customer's websites. We could hypothetically insert a shim that pulls the
include down the hash appended resource, but could still have the same problem
with the shim itself.

The custom origin has alleviated many of our pains but wouldn't have taken us
all the way. As people have begun to use Firebase to power Facebook apps
(mandatory SSL for the canvas) and other SSL only environments, we needed the
CDN to support it. We had a stop gap for some of our user (using Amazon's
domains) but it would've added too much friction in the docs to explain the
domain swaps for justify keeping this hack in place.

------
saurik
The primary feature advertised on fastly's website is a feature every real CDN
(as in, "not CloudFront") offers: an API to immediately purge your content. It
is not at all clear what differentiates them.

Looking at their feature plan breakdown doesn't help with this either: none of
the CDNs I've used have limited the number of backend domains I've had, and
all offered 24/7.

To be clear: that is not because I'm already paying the same $5k/mo minimum:
my bandwidth costs have almost always been much lower than that over the last
four years I've used CDNs; this is just not a common restriction.

Whether you are dealing with Akamai, CDNetworks, or EdgeCast, every level in
the CDN world offers this feature. I believe CacheFly even had this mechanism.
(Looking into t, I remember correctly.)

Meanwhile, their bandwidth pricing is insane (albeitimilar to CloudFront):
their $/GB is a few times what I'm paying for a "real CDN", and is about what
you will get if you call Akamai and then _don't_ negotiate.

The real question is: how many points of presence do they have? CDNetworks has
over a hundred, and Akamai has over a thousand. Are we talking "even smaller
than CloudFlare" here? (Apparently, the answer is "yes: even smaller, they
only 7".)

~~~
tyler
Alright. Let's go point-by-point.

 _The primary feature advertised on fastly's website is a feature every real
CDN (as in, "not CloudFront") offers: an API to immediately purge your
content._

Every CDN offers a mechanism to purge content, but they are not immediate.
Edgecast takes up to 15 minutes, CDNetworks I've seen take 20, Cloudfront can
take as much as 30. When we say immediate, we mean really immediate. Generally
speaking, it takes about 150 milliseconds.

 _Meanwhile, their bandwidth pricing is insane (albeitimilar to CloudFront):
their $/GB is a few times what I'm paying for a "real CDN", and is about what
you will get if you call Akamai and then don't negotiate._

Obviously, we will negotiate as well when we're talking about significant
amounts of traffic. And good luck getting Akamai to call you back if you don't
have significant amounts of traffic.

 _The real question is: how many points of presence do they have? CDNetworks
has over a hundred, and Akamai has over a thousand. Are we talking "even
smaller than CloudFlare" here? (Apparently, the answer is "yes: even smaller,
they only 7".)_

Yep. That's true. We're a rather young company and are actively expanding.
However, what is most notable about this is that despite having far fewer
pops, we're still significantly faster than most other CDNs, especially in
major population centers. We've put a ton of work into reducing latency inside
our servers so as to make better use of the pops that we currently have.

~~~
saurik
_And good luck getting Akamai to call you back if you don't have significant
amounts of traffic._

Look, I realize this notion sounds right and fits with the common dogma about
Akamai, but it is actually a lie: I have personally had long conversations
with Akamai negotiating deals where I would have had no minimum commitment
(although there were other totally reasonable non-monetary concessions
involved), and their prices still beat the ones on your website (albeit only
by a sliver).

(By the way, I am going to explicitly point out that if you had stopped after
your first paragraph about how your API is different, I would now have just
apologized and been interested to learn more about why people needed that, but
this obvious and totally incorrect FUD about peoples' abilities to negotiate
workable deals with Akamai is really bothering me. I wasn't actually "anti-
fastly" before: I just found it expensive and confusing... but now?)

~~~
sintaks
Also, that "2 billion requests per day" might sound silly to someone who
serves that in an hour...

~~~
tyler
We like to think that the exact number of requests is less important than
exactly how they're handled. While it would be cool to go "we serve a billion
requests a second", we're still an early stage startup. We're spending more
time making our responses even faster (< 1ms on the 99th percentile) and
trying to provide things that no one else does (for instance, instant purging
and surrogate key purging).

~~~
sintaks
I must say - that kind of perf is superb, though top percentile and averages
aren't super representative of the average customer's experience. What are
your tp50 and tp90 like?

Your sub-150ms invalidation is equally if not more impressive, especially if
you're talking about multi-region invalidation.

~~~
tyler
TTFB at the 50th hovers around 175 microseconds, 75th is at 250 microseconds,
95th around 450 microseconds.

As for the purging stuff, I do mean cross-region. So, it depends upon which
node receives your purge request. 150ms is average, but really it's "network
latency plus a millisecond or so".

~~~
sintaks
Wait, so 95% of your customers experience a TTFB of less than 500 microseconds
(tp95 of 450 microseconds)? I just want to make sure I'm understanding you
correctly. Because that's awesome.

~~~
tyler
95% of requests. But yeah! And thank you. :)

------
Hikari
1$ per gb for ASIA, only singapore and japan pop, simply way too expensive to
be considered an alternative cdn.

~~~
onetwothreefour
I agree with you (and $1 per GB in AU? Really?) but they are just starting
out. Maybe to American/European companies that don't care about the rest of
the world, fastly would be ok. :)

That said, you can get much better (and just as fast) deals from EdgeCast and
(previously) Cotendo.

~~~
tyler
Actually, tests from us and several of our customers have shown us
significantly faster than Edgecast throughout the US and Europe. (I work at
Fastly.)

------
DaveSwift
CloudFront's "Origin Pull" method seems to do almost everything you are
looking for with almost zero setup and management. No manual uploads and it
matches all of your HTTP headers from the origin (eg, cache control and
encoding). I've been using it for a few years and I love it.

Plus, CloudBerry Explorer PRO makes everything even easier and well worth the
$40 (<http://goo.gl/SM8dk>).

------
Apreche
Seems like their problem was that they used S3 behind CloudFront. Do they not
know that Cloudfront supports custom origins?

~~~
vikrum
Yep, that was precisely our problem :) We wound up using a custom origin with
Fastly. We made the switch off of Cloudfront since they don't support custom
SSL domains and the Cloudfront dashboard left a lot to be desired.

~~~
mayop100
Also, the invalidation on Cloudfront is pretty painful.

------
grandalf
What about cloudfront's dynamic buckets? Those seem to work great for us so
far.

