Hacker News new | past | comments | ask | show | jobs | submit login
Finding the right CDN for our startup (userapp.io)
65 points by typerandom on Nov 6, 2013 | hide | past | favorite | 62 comments

I've been extremely happy with CloudFlare. 4chan probably wouldn't be around today without them. CF has helped us by mitigating every DDoS since we started with them two years ago, and have kept operating costs in check by offering flat pricing. Even at their highest tier (Enterprise), we're saving money versus what we'd be spending with our colo.

Another point for Cloudflare here. Seriously boost in site responsiveness after we went with them on our site. They also have a CDN of JS and CSS libs that we use extensively at http://cdnjs.com/

Just adding my personal experience: my sites went from a 30ms response time (measured by Google's webmaster tools, as Googlebot sees it) to over 250ms on CloudFlare, for static (cacheable) content.

Turning it back off gave a very clear improvement both in my own experience and according to Google's chart.

I can not recommend CloudFlare enough. They offer a LOT of services for FREE! I have never had any problems and they use 2 factor auth.

It was due to 4chan's success with CF that I chose it for the projects I was the lead system architect for.

I created a browser based test a few years ago that captures CDN latency from real users and correlates the results regionally using MaxMind (http://cloudharmony.com/speedtest). CDNs set themselves apart outside of the US where infrastructure is more complex and costs are higher. I posted a spreadsheet that provides a breakdown on CDN latency globally based on this testing (tens of thousands of tests from thousands of ISPs):


How can you do a fair test across so many different providers? Presumably you did not host the exact same content with every single service.

Also, I'd be interested in seeing the results for Fastly.

I could write pages about managing CDNs, but unfortunately have to run. As an abbreviated version, Malwarebytes (company I work for) balances several CDNs around the globe to try to get the best performance possible (as well as to deal with some security issues inherent to hosting an antimalware tool). We have relationships with quite a few (not all of those relationships positive).

In my opinion this article skipped two of the best CDN's around, Edgecast and Highwinds. Edgecast in particular is damn fast (ssd boxes, high ram, most stuff served from memory). We're running multiple SSL'd domains through them without issue. Highwinds is more of an up and comer, but they have amazing service (as long as you avoid using them in South America).

The reason I avoid Cloudfront (AWS) is that it's way to expensive. I avoid CloudFlare because of the interactions I've had with their security team and CEO.

MaxCDN is good as long as you have no plans to go global. Let me be very explicit about this- if your CDN doesn't have a POP in Australia then they're not ready for prime time. There are only a few fiber lines going into the country and performance is absolute crap if you aren't hosting your files there. They also have nothing in South America, the MidEast or Africa.

As a seperate note, most DNS management companies will offer DNS based "CDN Managers", which will allow you to set priority rules for each CDN by region. This is an amazing tool that allows you to really take control of your traffic. I've never seen a CDN that was perfect everywhere, and when you have to deal with things like China (who don't really allow external CDN's in, forcing you to use one of theirs or host traffic out of Hong Kong) it's a life saver. I know that Edgecast, Dyn and UltraDNS offer these types of services (with the Edgecast one being the cheapest by far).

We switched from Cloudfront to Fastly and couldn't be happier. These guys are awesome and the product is top notch. I highly recommend Fastly.

We weren't too impressed with Cloudfront. The performance was all over the place and they provide no insight for cache hits/misses.

I actually looked at Fastly just a few days ago. Unfortunately their custom SSL fees are a bit high. But seems really nice and will definitely do a write-up if we try them.

On price Fastly has copied every CloudFront price to the cent. I seems they differentiate on statistics and rapid purging (their purported competitive edge). However, for most any application, with proper version naming of resources, purges should not be an issue, unless you've got hard coded references to those resources.

Though they have a $50 minimum also (unlike CloudFront), presumably as customers spending less than that are too expensive to support.

Except that the service they offer is much superior and are open to a conversation about pricing if you're a big enough client.

The rapid purge is probably inspired by the requirements for a large, popular wiki. The Fastly team comes from Wikia.

When the entire page is static, but changes must be communicated everywhere instantly, rapid purging is the answer.

I'd highly recommend Fastly too. Great product and great support. The guys there are doing an impressive job.

I've not used Fastly, but I know how competent the guys behind it are, and I'd start any search there...

We had the same experience: CloudFront to Fastly.

Fastly has a great CDN service, beutiful origin hosting for a Rails or Python app, and customizable output filters (gzip? yes, please) and customizable cache config.

Can't recommend them enough.

We use MaxCDN. It seems to work well, but I find all CDNs fairly opaque. How can I actually measure latency? I can't with web browsers being the way they are, so I really have no way of knowing. (There is http://www.w3.org/TR/navigation-timing/ but it is very limited, and not suitable for us.)

Clarification psated from a comment I made below:

Sure, I can test my latency, but I'm more interested in what my customers are experiencing. For instance, we have a big customer over in east Asia and I have very little idea what latency they get. When we were using Cloudfront a customer in Australia complained that they were seeing timeouts. It seems there was a bug in Amazon's routing table as Cloudfront has a POP in Sydney but their requests were going somewhere much further away.

Google has this thing which lets you essentially choose a location (and browser) and run a test with output from the page speed addon: http://www.webpagetest.org/

I'm using Cloudfront with the origin option for a subset of our users. I've found a few clients with so restrictive or broken firewall rules that I had to add an option to default to getting the files from the source if users come from network X or Y. While cloudfront lets you download the log files, you get one S3 object per edge location per 15 minutes or something like that, so 1000s per day, which made things difficult to troubleshoot (I suspect the problem with one user was partial cache due to Apache's deflate sending responses as chunked encoding as default and Amazon's origin spider sometime dropping such connections).

Pingdom's response time report is really nice. We also used LoadImpact to test it against "flash mobs" :)

All major browsers allow you to profile each load time of every single resource. Perfect for trying to compare relative latency at a particular location for a particular CDN. Chrome'a is especially good for this sort of thing, look at "network" in the developer tools.

Sure, I can test my latency, but I'm more interested in what my customers are experiencing. For instance, we have a big customer over in east Asia and I have very little idea what latency they get. When we were using Cloudfront a customer in Australia complained that they were seeing timeouts. It seems there was a bug in Amazon's routing table as Cloudfront has a POP in Sydney but their requests were going somewhere much further away.

You could try a tool like http://cloudmonitor.ca.com/en/checkit.php

I agree, I use it all the time for quick tests, but for a bit more accurate results other tools would be better IMHO :)

The major problem is SSL Pricing. Which i have never looked at.

But purely in terms of speed, Akamai Rules. Closely Followed by ( and sometimes even exceed ) EdgeCast.

From my experience, MaxCDN is the next bet since they are from NetDNA. Not saying CloudFront, or Other OnApp based CDN not good, for large files transfer or video streaming they are perfectly fine. But for small files, fast response and low latency those three are the way to go.

Haven't tested Fastly yet purely because i think they are very expensive against some other established players. And I guess they have far few PoPs. Would love to see some detailed comparison though.

Also worth looking at OVH: http://www.ovh.co.uk/cdn/

£8.99/month including SSL.

(Note, I haven't actually tried it myself!)

I briefly tested OVH's CDN, it wasn't great.

First, you use rules (patterns) to set the TTL, rather than headers. Second, I just couldn't get the SSL working. I'd upload the certificate, it would say that it would show up shortly, but days later, nothing. Third, the routing seemed messed[1]. Their support normally isn't horrible, but it took weeks to get anyone to acknowledge the support request that I opened on this, even after escalating it.

[1] I know people like to point out that internet routing has nothing to do with geography. But I squarely refuse to believe that, Singapore users should get routed to France (they have a POP in Singapore, HK and Japan).

I do not understand why a custom domain with a custom SSL certificate was necessary. Why not serve SSL off of the Cloudfront domain at normal cost?

I am not sure what the added benefit is (apart from the looks of using your own domain) when using your own custom SSL? Isn't the standard https connection through Cloudfronts wildcard SSL https://somethingsomething.cloudfront.net enough to be on the safe side? Really curious.

Depends on the situation.

Our product (Myna; mynaweb.com) provides a JS client that our customers embed on their web sites. If we give them a Cloudfront URL we have just created a big legacy problem for ourselves if we ever want to move off Cloudfront (and we did and we have!)

If you're creating a pure JS client connecting to an API on a different domain you have to worry about CORS support to get cross site requests. It's not a huge problem but it slows your site down a bit (you have to make two requests to check CORS permissions and then send data, where you could make one without this issue) and you can't support legacy browsers.

So there are a few reasons why using a Cloudfront URL might not be 100% suitable.

DNS provides an abstraction. If you use your own domain, you can change the underlying provider without breaking existing links.

This is probably setting the requirements a bit too high for the average site - requiring a unique SSL certificate from their CDN provider.

Most sites will probably be fine using SSL for their main site using their main servers - requests going direct to the site, with static/media files being served from a shared SSL certificate with their CDN.

I made the switch from Rackspace Cloudfiles + CDN to Amazon S3 + Cloudfront a couple months ago. I was regularly getting emails about my app erroring when someone uploaded a file via my app to Rackspace, since Rackspace was down (or responding slowly).

Now I'm happy with Amazon S3 + Cloudfront, it's cheaper and more stable.

I wish we could use CloudFront. Unfortunately they charge a bit for custom SSL ($600/zone/mo). Not that fun for a startup. Are you also using that? /R

I don't understand this. You have so much traffic that you need CloudFront, but cannot earn enough to spare $600/mo? What kind of startup is it?

From what I could tell talking with some people so far, is that they only think they need CDN, while a couple of regular dedicated servers would be just fine. But, then again, most of my customers are in Europe so I can serve files cheaply. In US, I'm currently using FDCServers, which are 3.3x more expensive than equivallent Hetzner server. And this was one of the cheapest I found. I find most Amazon services to be overpriced. IMHO, you're just paying more for the brand name.

The reason we're using a CDN is not because of the traffic - that would be easily solved with an AWS server and a lightweight http server such as nginx. We decided to go with a CDN because of two reasons:

1) The website and web app should load fast all over the world. Not only in the US or Sweden (where we live).

2) We don't want to host or put any time into hosting or own website and static files. We rather pay a little extra to don't have to think of that ever again ;)

Well, ok... spin up a few static web servers in cloud zones all over the world and run squid (caching web proxy) on them backed by an S3 bucket.

1. premature optimization 2. you don't need a CDN to host stuff virtually (AWS, GAE, rackspace, Azure, linode etc.).

Premature optimization or not. We care about the experience and want it to be as good as possible. /R

Negative, I don't use a custom SSL.

Since it's not all about the pricing for SSL pricing, I can recommend this site:


They provide a pretty nice overview about edge locations and other important options for considering to use a certain service.

Are the high SSL costs because they need a dedicated IP per domain at each of their POPs?

Would it be possible for them to offer an SNI-only SSL option for far cheaper if you know beforehand who your clients are (say, if you're hosting content for an iOS app)?

CloudFront is the same, $600/mo/cert. They claim it is not so much the cert as the need for a dedicated IPv4 address at each edge node (45 edges and counting), and the paucity of addresses left in said pool.

CloudFront has been asking a lot about a hypothetical SNI only offering in their user surveys lately, so that is likely the route they'll pursue shortly for far cheaper SSL options.

CDN.net offers the Subject Alternate Name option for $199/year. I've been doing research but we haven't tried them yet.

Edit: Just recalled they use Subject Alternate Name, not SNI. This is where they add your domain to their SSL certificate.

Prospective Rackspace customers - Avoid the cloudfiles offering from Rackspace at all costs. Their support is good, but we used to average out atleast one user who were getting access denied to store their files on Cloudfiles (We're kind of a file upload service in the most basic context). Switching in progress (to S3) and so far so good. I guess you get what you pay for, after all.

You're having problems with uploads only? What about downloads?

Sometimes they don't resolve properly too. And for vide, it's worse, they don't set the Mime type sometimes, which is quite irritating, for example say Mp4, because the video tag needs this to be set for it to work consistently :(

They don't set the mime-type on upload, or you set the mime-type on upload and they fail to set it when you download it via the Akami CDN?

I was planning to use Cloud Files and it's CDN. Thanks for sharing.

Their docs say they do it automatically, but for me sometimes they don't, which sucks and breaks my video player (flowplayer) on my sites. :(

We were using Amazon CloudFront and bleeding money. Approximately spending about $50 every 2 days. I researched to find a cheaper alternative. And found cdn77.com. Very good and reliable. The costs came down to approx $100/mo. I am not sure about custom SSL pricing though.

Bleeding money due to frequent cache invalidations, or traffic?

Try www.instartlogic.com which is much better if you have dynamic content or image heavy apart from letting you not buy new servers thanks to the offload while providing great user experience

typerandom: if you have time I'd be interested in understanding the performance problems you reported on CF. My email is in my profile.

Thanks. Would be happy really if we got it working with CloudFlare! /R

Please don't go to Cloudfront, it's truly horrendous (wearing my end user hat) to see so many sites have problems and timeouts because of them.

For example Ninite.com moved to them and now the site is virtually unusable because of it, over 90% of the time I get a "403: Forbidden" error when trying to do anything...like send them feedback on how the website doesn't work...

I even see more HN problems this week and I'm sure its because of them.

Do you mean CloudFlare rather than CloudFront?

Perhaps re: HN, and if so I apologise for confusion, but for Ninite it's CouldFront (d3doxs0mwx271h.cloudfront.net).

I spent a short while investigating if there was an easy way to get it working but ended up just giving up on the site.

Yes, the recent HN change is to introduce CloudFlare.

GitHub switched from Akamai to Fastly a little while back. I'd love to see a comparison of all these services.

What is your experience? :) What CDN are you using for your startup today? /Robin

I have been using CDN77.com, you should check them out. Custom SSL is 699 dollars but the performance is really good.

any experiences with Google's PageSpeed?

Applications are open for YC Summer 2021

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact