
Cloudflare’s CDNJS vs. Google Hosted Libraries - xxdesmus
http://www.baldnerd.com/make-your-site-faster-cloudflares-cdnjs-vs-google-hosted-libraries-shocking-results/
======
DannyBee
I'm not sure what using wget is supposed to show. Using it 25 times in a row
on a completely stable, completely idle 25mbps connection, I get speeds/times
that vary by 100% consistently

(IE min is 496k/s, max is 895k/s, average is about 600k/s)

Using wget is a completely useless benchmark, from what I can tell. Using
apache's little benchmark tool I get about 255ms vs 180ms average for 100
requests to each.

The interesting part is that for ajax.googleapis.com I get:

    
    
      Connection Times (ms)
                    min  mean[+/-sd] median   max
      Connect:      175  228  28.2    227     306
      Processing:     0   12  15.1      8      90
      Waiting:        0    0   0.0      0       0
      Total:        189  240  29.8    235     350
    

and for cloudflare I get:

    
    
      Connection Times (ms)
                    min  mean[+/-sd] median   max
      Connect:       19   28  13.1     25     125
      Processing:   125  155  28.0    146     246
      Waiting:       21   28   6.5     27      65
      Total:        144  182  30.7    175     271
    

IE for google, _all_ the time is in actually getting a connection and getting
the bits, whereas, for cloudflare, there is actually some time waiting for
their servers.

Using 5 concurrent requests actually gives me a _massive_ advantage for google
(cloudflare takes roughly the same time, google goes 4 times faster)

~~~
afhof
Could you post the exact commands you used? That high connection time for
google looks like SSL setup.

~~~
DannyBee
Non concurrent:

ab -n 100
[http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min...](http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js)

ab -n 100
[http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.mi...](http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js)

Concurrent case:

ab -c 5 -n 100
[http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.mi...](http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js)

ab -c 5 -n 100
[http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min...](http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js)

------
anonfunction
Interesting results, but let's think about a few things before we all switch
to Cloudflare CDNJS.

1) Time to first byte. In my experience the biggest lag is the connection, not
actually downloading the file (for small js, images, etc...) I don't know how
to test this.

2) Caching, by using google for common libraries like jQuery your chance of
the end client already having a local cached copy are much greater.

3) Reliability. I know google has been recently killing off it's products, and
even I tweeted how they could break the internet by shutting off their hosted
library API, but it's probably not going to happen. Can you say the same about
cloudflare?

I'm not saying I like one more than the other, but these are some things I
would like to address before switching my and my clients sites over.

One thing I will give a +1 to Cloudflare is the ability to add js to the CDN
via github. <https://github.com/cdnjs/cdnjs>

~~~
rgbrenner
Using wget for a test is just terrible. Here's a better test (and I'll put it
here, so everyone can criticize it):

    
    
      1) start chrome in incognito mode.
      2) press f12, and then click on network
      3) load each js
    

I'm in Denver, this is what I got:

    
    
      Google:
      DNS Lookup: 14ms
      Connecting: 24ms
      Sending: 0
      Waiting: 46ms
      Receiving: 42ms
      TOTAL: 128ms (86ms latency)
    
      Cloudflare:
      DNS Lookup: 15ms
      Connecting: 43ms
      Sending: 0
      Waiting: 46ms
      Receiving: 49ms
      TOTAL: 154ms (105ms latency)
    

I can't take a test run with wget seriously when there are better tools so
easily available. It's beyond lazy.. it's incompetence, or worse, dishonest.
He couldn't even be bothered to run it more than once from his own computer(!)

------
jmathai
Doesn't this have network effects? The CDN used by the most sites becomes the
most valuable since a user visiting your site will have the library cached.

I've always felt the benefit was that loading assets could be a networking no-
op.

~~~
Kudos
You're absolutely right.

------
VeejayRampay
It probably varies enormously depending on your location.

Here in Vancouver, I get 1.25MB/s for Google's CDN and 1.15MB/s for CloudFare.
Unless we have a distributed tool to test that on a massive scale, it will
only reflect the greatly-varying differences between people's connections,
"distance to CDN", DNS metrics and whatnot which I guess greatly impact the
speeds observed.

Nevertheless I really like that this actually "exercises" the different
services and provides some form of measuring and poking that improves the
overall transparency and accountability.

------
ominous_prime
Another point to make is server response time. I'm not making a claim one is
better overall than the other, but at my location (boston) cloudflare's
servers respond a bit faster. Just grabbing a couple different files between
the two shows up to 300ms difference, but usually <100ms.

[edit] oh yeah, and this is all moot if your user already has it in cache,
which is currently more likely from google.

------
inafield
Disclaimer: I'm not offering a perfect solution to the ideal test.

1.) Depends on how everyone is routing. The people replying directly to the
OP's page are checking from different servers with god-knows-what kind of
routing tables that the average web browser on the net does not have access
to.

1b.) A better test would be to have people from different ISP's and different
geographic locations testing this out. Routing is everything. In other words,
let's see what results are seen by mom-and-pop from India, UK, Texas, Rhode
Island, Montana, Nunavut, Brazil, China, and Nigeria.

2.) Already said, but worth pounding into the OP's head, Google is the most
widely used for jQuery because everyone else is using it and harddrive cache
is 1.21 jiggawatts faster than any networked CDN.

3.) Latency and overall speed need to be measured. What use is 88Mbps if
you've got horrible latency? Sure, there are times when a courier on a bike is
faster than FTP, but for small files that latency is a bigger deal.

------
trotsky
That is impressive, but you've basically just checked whether cloudfare has a
POP in Ontario, or atleast within 2-3ms of your datacenter.

They both have POPs local to me:

    
    
       eggplant:~ tsl$ wget http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js
       --2013-03-22 19:29:21--  http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js
       Resolving cdnjs.cloudflare.com... 141.101.123.8, 190.93.240.8, 190.93.241.8, ...
       Connecting to cdnjs.cloudflare.com|141.101.123.8|:80... connected.
       HTTP request sent, awaiting response... 200 OK
       Length: unspecified [application/x-javascript]
       Saving to: ‘jquery.min.js’
           [ <=>                                   ] 92,629      --.-K/s   in 0.04s
       2013-03-22 19:29:21 (2.03 MB/s) - ‘jquery.min.js’ saved [92629]
    
       eggplant:~ tsl$ wget http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
       --2013-03-22 19:42:05--  http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
       Resolving ajax.googleapis.com... 173.194.75.95, 2607:f8b0:400c:c01::5f
       Connecting to ajax.googleapis.com|173.194.75.95|:80... connected.
       HTTP request sent, awaiting response... 200 OK
       Length: unspecified [text/javascript]
       Saving to: ‘jquery.min.js.1’
           [ <=>                                   ] 92,629      --.-K/s   in 0.05s
       2013-03-22 19:42:05 (1.62 MB/s) - ‘jquery.min.js.1’ saved [92629]
    

Just to drive home how much locality matters:

    
    
       tsl@beast:/volumes/fast$ wget http://nas.***.com/jquery.min.js
       --20:00:05--  http://nas.***.com/jquery.min.js
               => `jquery.min.js'
       Resolving nas.***.com... 10.10.10.2, fd2b:2048:4633:10::2
       Connecting to nas.***.com|10.10.10.2|:80... connected.
       HTTP request sent, awaiting response... 200 OK
       Length: 92,629 (90K) [application/javascript]
       100%[====================================>] 92,629        --.--K/s
       20:00:05 (63.00 MB/s) - `jquery.min.js' saved [92629/92629]
    

Looks like my single core ARM6 nas is the best in my neighborhood.

~~~
kevinburke
Never seen that before - what is nas. __*.com?

~~~
cincinnatus
He censored the domain name with __* :-)

------
pm24601
You need to double-check your methodology:

<http://sworddance.com/blog/2013/03/22/always-check-twice/>

The difference is almost certainly an initial poor DNS resolution the first
time for which ever service had the slower time.

That said, a more complete js cdn is always nice.

------
X-Istence
I ran the test, did it five times for Google, five times for Cloudflare and
for each and every single one Google won out (for the current location I am at
... work), the script I used is here:

<https://gist.github.com/bertjwregeer/5225202>

This way it will also output how long it spent in each step, including
connect, appconnect (if you try the SSL version), how long it spends in
pretransfer and how long it takes to start the transfer at all. Followed by
the size of the download and the header size.

This way you can get a much better idea as to why it is taking longer for one
service over the other. The files themselves are relatively small so are not
an accurate indicator of transfer speed.

------
iza
For small files like this the latency is much more important than throughput,
not to mention more users will have Google's version cached already.

~~~
d0vs
Indeed. Past 5 Mbps, bandwidth is basically irrelevant, and Google knows
it.[1]

[1]:
[https://docs.google.com/a/chromium.org/viewer?a=v&pid=si...](https://docs.google.com/a/chromium.org/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDoxMzcyOWI1N2I4YzI3NzE2)

------
WithTeeth
Don’t forget that one of the advantages of CDNs is that the user may already
have the file cached in their browser. This is why I use Google’s CDN for
jQuery and CDNJS for everything else – more web sites use Google’s CDN for
jQuery, so the user is more likely to have that cached. Not to mention the
HTTP limitation of 3 simultaneous requests from one domain…

------
dustingetz
the point isn't how fast your CDN is, the point is how likely it is that the
user already has the asset in browser cache.

~~~
dminor
On that note: [http://www.stevesouders.com/blog/2013/03/18/http-archive-
jqu...](http://www.stevesouders.com/blog/2013/03/18/http-archive-jquery/)

------
maledale
network effect of caching aside, I decided to run tests from 95 nodes around
the world. Most of these have above average connectivity, however some are
true "last-mile" nodes. The following data set consists of just over 2000 runs
over one hour of testing.

    
    
                      #tests avgTotal  avgDNS    avgPingRT
      CDNJS             1105  187ms    58ms      27ms
      GoogleHostedAPIs  1120  219ms    30ms      38ms
    

<https://gist.github.com/dneufeld/5226469>

