

Researching a new form of HTTP caching optimization - ohcoder
http://blog.phusion.nl/2015/01/06/researching-http-caching-optimization/

======
samarudge
> the Varnish HTTP cache has been used very successfully to speed up
> WordPress. But Varnish doesn’t help a lot with logged-in traffic > This is
> caching that Varnish and other “normal” HTTP caches (including CloudFlare)
> could not have done

Varnish supports the ESI (Edge-Side Includes) standard, which allows it to
cache fragments of a page, and for the cache server to build them again. It
also allows you to completely bypass the cache for certain fragments. This is
also supported by a number of CDNs (Fastly, Akamai). I've used the ESI
technique several times and have been able to achieve a >98% cache hit rate on
Fastly for a site with dynamic per-user content. Even the cache misses are
only responsible for rendering a small component of the page

~~~
FooBarWidget
Good to know. Using edge side includes may be easier than trying to change the
app to a semi single page app. But that only solves half of the problem. The
other half is varying the response based on the value of a specific cookie.

I've updated the blog post with information regarding edge side include.

~~~
samarudge
Varnish also supports custom cache keys via the VCL function `vcl_hash` as
documented in [https://www.varnish-
software.com/static/book/VCL_functions.h...](https://www.varnish-
software.com/static/book/VCL_functions.html#vcl-vcl-hash)

I couldn't (quickly) find documentation on how to get the value of a specific
cookie, but the server could send a user ID in a header or something Varnish
can easily access to be used in the above function.

~~~
Argorak
You will probably have to write a VCL plugin if you want to parse the cookie,
but that's not a huge hassle.

~~~
acdha
You can handle cookies in pure VCL – the code's not particularly elegant but
it's manageable for light usage:

[https://www.varnish-
cache.org/trac/wiki/VCLExampleRemovingSo...](https://www.varnish-
cache.org/trac/wiki/VCLExampleRemovingSomeCookies)

~~~
Argorak
I'm not sure whether I would call "munging the cookie header with regexps"
"cookie handling" ;).

~~~
acdha
Agreed – it's possible and for simple tasks such as stripping an analytics
cookie it's workable but for anything more serious you'd want something like
[https://github.com/lkarsten/libvmod-
cookie](https://github.com/lkarsten/libvmod-cookie)

------
markcampbell
I've left some comments on the Disqus thread on the blog, but I'll reiterate
my concern about the security of the cookie being set.

The cookie being set is unsigned according to the documentation[0] on rails,
so a user could modify it and send it back to get a different cached response.
Say that they saw that the user level was being set in there (like in the blog
post) and they change the value to the 'staff' value to get the staff cached
response. Probably not a good idea!

With that said, I don't think that this technique is adequate right now when
you have user access level concerns and you're relying on that piece of
unsigned information to not be tampered with.

[0]
[http://api.rubyonrails.org/classes/ActionDispatch/Cookies.ht...](http://api.rubyonrails.org/classes/ActionDispatch/Cookies.html)

------
codingjester
I think it's been said enough here but Varnish can certainly do almost
everything that they are saying it can't do, including some other storage
optimizations like storing the gzipped response and serving a non-gzipped when
requested (as of Varnish 3.0).

You can use Vary for tons of caching optimizations via varnish, such as
caching mobile web-pages vs non-mobile web pages or just a particular header.
It's all about just flexing a little bit of VCL (which I'll admit sometimes
can throw people off).

They had me until this part:

> This is an HTTP cache built directly in Passenger so that it can achieve
> much higher performance than external HTTP caches like Varnish.

And since they have no benchmarks to really back up these claims, I'm
skeptical they did much research against Varnish to tune or set it up. I'd
love to see the numbers on varnish vs their turbocache. Without numbers, I
have to take a lot of it with a grain of salt.

Either way, seems like it could be an extra handy thing to have in your
toolbox, as long as it fits your stack.

~~~
FooBarWidget
You're reading too much hostility in it. The article does not claim to be
better than Varnish. Performance is achieved by not implementing many
features. For example, Varnish is a full-blown programming language and has an
infinite size. The Passenger turbocache has almost no configuration options,
does not support any sort of custom programming and only supports 8 entries at
most. The fact that the max size is so small allows it to be implemented in
very compact data structures, but its usefulness is also severely limited.
It's merely designed with a different set of tradeoffs than Varnish.

It isn't that difficult to make a web server that's faster all the production
servers out there. For example H2O is faster than Nginx... because it has
infinitely less features. Ditto for Passenger's turbocache.

The rest of the article describes some ideas, some of which cannot be
implemented using pure Varnish and require cooperation from the app. In my
opinion you're missing out on a lot of interesting ideas if you stop reading
only because you _think_ Varnish is being slammed.

~~~
Argorak
I think the complaint is just that the approach is in no way as novel as you
make it seem.

It can and has been implemented with Varnish and a few lines of VCL (with
support of the app as well, of course).

What is "pure varnish" anyways? Proper varnish use always needs a tuned VCL,
that's the whole core of the product.

Also, if I might say, I find your edits very handwavey. Please prove why that
is impossible in VCL.

~~~
FooBarWidget
Fair enough. But even claiming that it's truly novel is not the point of the
article. The point of the article is to research HTTP caching and to converse
with the community about possibilities. It is not to present a new product.

In hindsight I see that the title may have been badly chosen. Before
publication, none of the test readers have made any fuss about this.

> Please prove why that is impossible in VCL.

Where do you get impression that I claimed it's impossible in the VCL? It's
pretty clear that the single page app approach can be replaced by edge side
include, while the cookie parsing can be done with the cookie vmod.

The article claims that despite being the cache being able to do these things,
application support is still required. The app has to be modified to output
certain cookies in a certain format, e.g. the vary-on-user-permission-level
thing. It's the combination that is important.

Have these things been done before? I'm sure they have. Most ideas in computer
science are 30+ years old, and it's rare to truly find something new. But
again, the point of the article is research. What we're after is not to
present a new thing, but to present an idea, and if it's a new thing then we
want people to help us to test; if it's not a new thing then we want to hear
experiences.

------
stephenr
The part about inability to speed up authenticated page loads fails to take
into consideration things like ESI.

If the majority of your page is still the same for logged-in users, but they
see some pieces of content personalised for them, breaking them out into
individually request-able components means you can let software like Varnish
or a CDN rely on it's cache of the main page content, and make a very small
(and ideally simple to process on the backend) request for the per-user
content.

It took me some hunting to find it (no docs for v5 yet apparently) but if your
stack already includes a caching layer (e.g. Varnish or a CDN) you may want to
disable this extra cache using the config described here:
[http://blog.phusion.nl/2014/11/25/introducing-phusion-
passen...](http://blog.phusion.nl/2014/11/25/introducing-phusion-
passenger-5-beta-1-codename-raptor/#mc_embed_signup)

------
WimLeers
> What if the cache can parse cookies and vary the cached response by the
> value of a specific cookie, not the entire header?

> This is caching that Varnish and other “normal” HTTP caches (including
> CloudFlare) could not have done.

This is false. I'm not at all very familiar with Varnish, but I know this is
_easily_ possible, and has been used for many, many years.

E.g. for Drupal + Varnish, i.e. to only keep Drupal's session cookie, I found
these examples, in less than a minute of googling:

\- [https://www.varnish-
cache.org/trac/wiki/VarnishAndDrupal](https://www.varnish-
cache.org/trac/wiki/VarnishAndDrupal)

\- [https://www.lullabot.com/blog/article/configuring-varnish-
hi...](https://www.lullabot.com/blog/article/configuring-varnish-high-
availability-multiple-web-servers) (grep for "inclusion")

Everything in this article has been well-known for at least half a decade, yet
is being presented as major technical breakthroughs. _Too much marketing,
IMO._

~~~
FooBarWidget
The first example looks nothing like parsing a specific cookie. It merely
sanitizes the cookie headers a little, but it doesn't extract out a specific
cookie to use as cache key.

And both examples you link to _remove_ cookies. That's not what we're after.
We're after the extraction of a specific cookie without removing anything.

It's also not marketing for a commercial product. The research is for an open
source project, and the code is public and open source. The entire point of
the blog post is to call for research participants who could not only test our
ideas, but who could also point out anything we might have missed.

~~~
WimLeers
> The first example looks nothing like parsing a specific cookie. It merely
> sanitizes the cookie headers a little, but it doesn't extract out a specific
> cookie to use as cache key.

You say you don't parse a specific cookie, but doesn't extract a specific
cookie as a cache key? The blog post says the opposite:

> We modified Passenger to parse cookies, and to vary turbocache responses
> based on the value of this user_id cookie. We invoke Passenger like this:
> _passenger start --vary-turbocache-by-cookie user_id_

In other words: parse the _user_id_ cookie.

The mentioned commit even adds a ~250 LoC file
(ext/common/ServerKit/CookieUtils.h) to do cookie parsing:
[https://github.com/phusion/passenger/commit/a760649cd79fde43...](https://github.com/phusion/passenger/commit/a760649cd79fde4363ace40dd0e353c445449d04)

\---

Anyway, what you're getting at is that Phusion only parses a certain value
from the cookie and then uses it as a custom Vary header, whereas the examples
I linked to clean up the Cookie header and then varies on that cleaned up
Cookie header. That boils down to exactly the same thing.

P.S.: downvoting? Not very nice.

~~~
FooBarWidget
I am from Phusion. No idea who downmodded you, but this happens often on HN.

But what you mentioned is not the same thing. In the examples you linked to,
Google Analytics and other cookies are _removed_ , leaving only a single
cookie. But that's not what we're after. We want to vary by a specific cookie,
_without_ removing the other cookies.

For example, Rails stores the session data in a session cookie. We advocate
introducing a user_id cookie _in addition_ to the session cookie. Using your
approach, the session data would be removed, which breaks the application. Our
approach leaves all original cookie data intact while still varying on a
specific cookie.

Furthermore, another idea that we've described in the blog post is to vary
based some user property, e.g. the user's permission level, in order to
increase the cardinality of cache entries. This is not something you can do
with only Varnish: it requires cooperation from the app.

------
chris_overseas
I had a contract at a company about 13 years ago where I was working on a web-
based CMS that had been built entirely in-house. Because each rendered page's
content was built up in a hierarchical manner, I added a caching layer that
allowed arbitrary portions of the rendered page "tree" to be cached all the
way up to the entire page and HTTP response if possible. Each cached portion
could be located quickly based on its dependencies (template ID, content ID,
CSS etc). If any edits were made to the site, only the relevant portions
needed to be flushed and re-rendered. User-specific portions could be cached
in the user's session rather than site-wide. (Thinking about it now, each
portion could have been rendered in parallel too, though this was back in the
days when multicore machines weren't very common and it wasn't something that
occurred to me.)

I built this without giving much thought to whether anyone else had attempted
something similar. Presumably they had though I do remember being disappointed
when researching the various open source caching libraries, they didn't offer
much help at the time. Overall I was pretty happy with the way it all worked
and the performance boost it provided was like night and day.

Sadly the company is no longer operating, presumably the CMS codebase is long
lost.

~~~
jacquesm
Most major frameworks include something like this but it's usually harder to
use, 'partials' caching as it is called is a pretty effective way to speed up
the server side of things when you need that.

Varnish has a similar facility (which allows you to abstract it out of the
framework code entirely).

A dependencies ('make') like automatic approach to this would be quite nice to
have, maybe your approach could be retro-fitted onto one of the existing CMSs?

~~~
chris_overseas
Thanks, I hadn't heard of the names "Russian doll" or "partials" before. I
haven't worked with a CMS (or even web apps at all really) for over a decade
now but I'm surprised you seem to imply that automatic dependency management
isn't standard stuff these days already. It was certainly very useful to us,
though admittedly tricky to implement robustly.

------
rcarmo
Hmmm. I might be missing something here, but I routinely clean out cookies in
"public"/unauthed URLs in Varnish, as well as hashing the cache based on
_part_ of a specific cookie (the bit that defines, say, the site's theme, or a
generic user role).

They do mention that they didn't investigate how to do this in Varnish, but I
recall having picked up the basic technique from one of the author's posts.

~~~
michaelmior
This was my first thought too. You can vary by any subset of cookies in
Varnish if you simply normalize the Cookie header by stripping out irrelevant
cookies. Another trick I've used is to parse the cookie to look for the user
ID and set a User-Id header on the backend response and then set Vary: User-
Id.

------
MichaelGG
> Normally, non-cacheable page fragments would make every page uncacheable.

AFAICT, ASP.NET supported partial output caching since 2003. And, it can store
multiple versions of a cached item via a user-defined parameter (a property on
the cached control class). It seems it might be possible to simply create a
property that returns the user ID and let the cache vary on that.

Of course, this uses the Web Forms controls-based system, which isn't very
popular. But calling this type of caching _new_ seems a bit of a stretch.

[http://msdn.microsoft.com/en-
us/library/k4he1ds5%28v=vs.71%2...](http://msdn.microsoft.com/en-
us/library/k4he1ds5%28v=vs.71%29.aspx)

Edit: And here's a link talking about VaryByCustom, complete with example code
that uses the version of the browser to generate unique per-browser-version
caches.

[http://msdn.microsoft.com/en-
us/library/5ecf4420%28v=vs.71%2...](http://msdn.microsoft.com/en-
us/library/5ecf4420%28v=vs.71%29.aspx)

------
Argorak
Fragment caching of non-anonymous things on a cache is nothing new. Varnish
supports it with ESI, other caches support that with other strategies.

I've successfully build SPAs using that strategy. It's basically Rails russian
doll-caching on a dedicated process.

------
hannibalhorn
Even ignoring the fact that edge side includes are the way to go when
confronted with this problem, it sounds like they're basically saying they'd
like to include the userid in the Vary header, but cannot because the Cookie
header includes a bunch of other stuff.

Instead of parsing the Cookie header and doing a bunch of additional work in
the cache layer, why not just add a separate custom header (X-User-Id, say)
and Vary on that?

~~~
WimLeers
You're talking about response headers. But this deals with sending responses,
you only get requests. Request headers can't contain a custom header that you
control (unless your JS is making the requests of course). Hence you have to
transform the Cookie header into whatever you need.

------
megaman821
I almost always configure Varnish not to cache on Vary Cookie and pair that
with some simple changes on the back-end to keep most my pages cacheable.

One, don't blindly set Vary Cookie on every single page. Two, when a page only
needs minor variation like a username; store the username in a cookie and use
JavaScript to display it on the page.

~~~
lstamour
Yeah, the biggest advantage ESI offers is not requiring JS. If that's
important to you, configure it and do extra rendering on the server-side. If
not... :)

------
jcase
When you mentioned discourse serving 19k req/sec, does it hit the ruby stack
at all? If no, serving 19k req/sec of cached HTML doesn't seem that
impressive. What am I missing?

------
sandGorgon
Sorry to barge on this thread - but does have a _reasonable_ varnish config
that works for a wordpress site behind https nginx and W3 Total Cache ?

It seems strangely hard to configure something like this.

~~~
lstamour
I used (and modified) one based on Dreamhost's
[https://github.com/dreamhost/varnish-vcl-
collection/blob/mas...](https://github.com/dreamhost/varnish-vcl-
collection/blob/master/wordpress-example.vcl) but there's a bunch of others on
Github if you search for them. Note that this isn't a shortcut so you don't
have to learn VCL... and that the way I did it was to put Varnish on port 80
and the web server on another port. I've seen other configs where a load
balancer asks Varnish when it knows varnish should have the answer and
otherwise sends traffic directly to the web server. And sometimes your config
will vary based on asset type or domain name -- e.g. static.example.org assets
might be cacheable forever while your site's pages might only require a 2
minute cache. You can invalidate cached-forever pages with commands for
Varnish or by varying the URL using a "cache buster" hash or string in the
asset filename. Oh and don't forget to set the storage type Varnish uses to
malloc instead of file.

~~~
sandGorgon
thanks for this !

I guess you dont have an SSL termination point in front. What I was thinking
of was nginx -> varnish -> nginx.

~~~
lstamour
Ah, actually we do, but that runs on the load balancer. We currently use nginx
on the LB but you could use haproxy alone these days.

~~~
sandGorgon
When you use nginx as ssl termination loadbalancer - do you just do a proxy
forward to varnish, or is it anything complicated ?

------
est
uWSGI has a powerful cache mechanism, and it's clustered.

~~~
ddorian43
and you should be using it:

[http://cramer.io/2013/06/27/serving-python-web-
applications/](http://cramer.io/2013/06/27/serving-python-web-applications/)

------
marvy
Switching to a single page app seems invasive. Why not just have an iframe for
the username and stuff?

