
Varnish 4.0.2 released - lkarsten
https://www.varnish-cache.org/lists/pipermail/varnish-announce/2014-October/000698.html
======
latch
My experience with Varnish was when we tried to use it for something that had
relatively complex caching rules. The config/VCL became spaghetti: attaching
values to req and restarting the flow. This appeared to be the normal way to
write VCL.

We ended up writing our own system, with our own high-concurrent LRU cache,
that was more tightly coupled with our application servers (and thus able to
figure out what the cache key should be). It ended up being trivial to then
add things like ESI, purging, grace and saint mode.

Point being, since then, I've had a hard time seeing where Varnish fits
between proxy_cache for simple url+query caching, and rolling your own.

~~~
bkeroack
Agreed. Varnish is great if you have an existing web application that needs a
quick-and-dirty caching layer built in front of it.

Varnish gives you enough configurability to do just about anything, but the
end result will be a spaghetti mess of VCL, inline C and possibly custom
varnish modules. The more sustainable route is to build your own caching front
end, however I think it's a bit unrealistic to assume that all startups have
the engineering talent (or time) in-house to implement that sort of thing.

~~~
spacemanmatt
Wow, this (and parent-post) really colors my understanding about how far one
can/should go with Varnish before building something more tailored. Thanks.

------
sandGorgon
Does anyone have a fairly basic varnish config that deals with a) non-www to
www redirects, b) works with https certificates (with the Cloudfare config) ,
, c) allows CORS for fonts on cloudfront and d) switches off caching for all
/admin pages?

Varnish has proven to be very hard to use with nginx in the above setup.
Especially, trying to understand the ssl bit is getting to be really hard.

Can it also work with spdy?

~~~
megaman821
a) I just let Nginx do the redirect. Varnish can cache 302's so after the
first one it won't hit Nginx again until the cache times out.

b) You have to terminate SSL connections before hitting Varnish. There is very
little chance SSL termination will ever be part of Varnish. On AWS you can
terminate with ELB.

c) Varnish can add HTTP headers and it will respect the headers your backend
sets.

d) Varnish can disable caching for a specific path or again it will respect
the No-Cache headers a backend sets.

~~~
sandGorgon
for b), we dont use AWS - we are hosted on Softlayer cloud. Does that mean
that - if you use Varnish with SSL, then you NEED to use HaProxy ?

Or is it nginx (SSL termination) -> varnish -> nginx (server) -> Rails

~~~
megaman821
Yes, I usually set up Nginx just to listen to port 443 and forward to Varnish
on the same machine. You could do the same with HaProxy.

It is not as bad it seems since using Nginx to terminate SSL and forward is
just a really simple config file. If you have a lot of cached resources then
the path is mostly Nginx -> Varnish. Varnish is probably over 100 times faster
than Rails at serving anything cacheable.

------
imaginenore
Varnish left a sour taste in my mouth. We have used it on one of our high-
traffic sites, and had some truly bizarre hard-to-reproduce problems with it.

The major bug it had was that it would work normally for hours, but then it
would randomly let the flood of requests through basically killing our
servers. It happened over and over and over again. We had multiple talented
sysadmins look at it, and none of them could give us any explanation. The only
solution was to restart it, and warm the cache all over again. We couldn't
figure out what sets it off, it looked just so random.

~~~
phkamp
That sounds like a "hit-for-pass" scenario: Something from your backend told
Varnish "Don't cache this" and varnish stopped doing so.

~~~
imaginenore
Except we didn't have that. This was 99.9% content pages dynamically generated
and nearly static. They all had roughly the same headers that haven't changed
really, and they were not even aware of Varnish.

Response headers is the first thing we checked, there are only a few of them
that affect Varnish:

[https://www.varnish-software.com/static/book/HTTP.html](https://www.varnish-
software.com/static/book/HTTP.html)

~~~
seanp2k2
Are you sure it wasn't an issue with cache evictions?

~~~
imaginenore
All pages at the same exact time?

------
brokentone
Varnish and caching in general can be a complete mind-freak the first few
times you experience it, however after working with it nearly daily for the
last 3 years, I've come to love it. It has allowed me to scale a number of
WordPress websites far beyond where they had business being (34mm UV/180mm PV
per month on a handful of servers). Super excited to see them continuing to
build in great features.

~~~
sandGorgon
I have never tried but failed miserably in trying to setup a Varnish front-end
for a wordpress setup - especially with pretty urls. The hard part was getting
varnish to work with the admin interface - especially image uploads - and
comments.

Do you know of any good starting config file that can be used ?

~~~
ruben_varnish
A couple of resources for Varnish 3:

* [https://github.com/mattiasgeniar/varnish-3.0-configuration-t...](https://github.com/mattiasgeniar/varnish-3.0-configuration-templates) (fairly well maintained with CMS specific VCL examples)

* [https://github.com/slashsBin/nuCache](https://github.com/slashsBin/nuCache) (library for different programming languages, not so CMS centric)

Both of these are listed in our utilities directory:

* [https://www.varnish-cache.org/utilities](https://www.varnish-cache.org/utilities)

Other Varnish (VCL) extensions can be found in the VMOD directory:

* [https://www.varnish-cache.org/vmods](https://www.varnish-cache.org/vmods)

If you have ideas on how to make both these resource more visible and
accessible for our users, please let me know.

~~~
sandGorgon
@ruben_varnish - this is great ! thanks.

I think it would be great if varnish had an official repo with different
configuration templates for different stacks that could be used as "drop-in".

For example, my question about a fairly basic Rails config
([https://news.ycombinator.com/item?id=8431927](https://news.ycombinator.com/item?id=8431927))
had a couple of different answers disagreeing about stuff.

Also, things like SSL termination are something that the documentation skips
over entirely - I understand completely that Varnish does not implement it,
but Varnish necessitates the introduction of certain other players in the
stack, which it should document.

For example, a canonical example of varnish setup as Nginx (SSL termination)
-> Varnish -> Nginx (reverse proxy) -> App would be very useful.

------
Jgrubb
> If you are using Varnish Cache for business, consider attending the Varnish
> Summits

If anyone from Varnish AS is reading this (OP?), please consider having a
Varnish Summit on the east coast of the US sometime.

Soon.

Please.

~~~
lkarsten
Good news, I'm told they are planning an east coast event in the spring.

~~~
Jgrubb
Thank you very much!

------
ochoseis
Two questions: (1) Anyone use Varnish in front of Rails? Is it better to just
set up some memcached and/or redis and let the app handle it? (2) To deal with
session-specific stuff, how common is it to render some common plain html
views, then fill in the particulars with JS/Ajax? If you need to support no-JS
(heaven forbid) do people use iframes?

~~~
seanp2k2
Check out ESI blocks: [https://www.varnish-
cache.org/trac/wiki/ESIfeatures](https://www.varnish-
cache.org/trac/wiki/ESIfeatures)

Varnish will very likely be an order of magnitude or two faster than sending
requests to your app. You can get going with a simple config in a few minutes
and without breaking anything (just have varnish listen on a different port
and use :80 or your app server as the back-end), so I'd strongly suggest
playing with it if you're curious. It's pretty impressively fast software, and
VCL gives you an awesome abstraction for controlling how the cache handles
different routes.

------
nasalgoat
Has anyone documented how to convert a 3.x config to 4.x config yet? The
documentation on the site assumes a level of understanding of the internal
workings I don't have.

I'm still running 3.x because they broke a bunch of stuff. I discovered this
when I updated a couple of dev boxes and varnish stopped working.

~~~
ruben_varnish
You can use these:

* Upgrading notes: [https://www.varnish-cache.org/docs/4.0/whats-new/upgrading.h...](https://www.varnish-cache.org/docs/4.0/whats-new/upgrading.html)

* A script by Fed that will do 70-90% of the job for you: [https://github.com/fgsch/varnish3to4](https://github.com/fgsch/varnish3to4)

Let us know how it went :-)

------
krat0sprakhar
We've been long vacillating on using Varnish to power our E-commerce store but
from what we've read, Varnish is not a good solution in cases where cookies
are part of most of web traffic. Is this true?

~~~
lkarsten
Every site has something that can be cached. Images, js, headers, popular
product lists, etc.

If a specific page item can be cached is really up to if the backend
application takes cookie state into account when the page item is made.

The Varnish default of not caching anything when cookies are present is
because we have no idea if the backend writes "Welcome back, krat0sprakhar!"
when it sees the username in an incoming cookie. Sending that item to every
user would be unfortunate.

Other than that, I'd recommend you evaluate your needs and not just pick a
technology. If your page response times are low enough already, and your
backends/appservers scale to as many concurrent buyers you think you need,
you're good without Varnish/caching.

~~~
jpp
I'd add that edge side includes ([https://www.varnish-
cache.org/trac/wiki/ESIfeatures](https://www.varnish-
cache.org/trac/wiki/ESIfeatures)) can be amazingly useful. Even if you've
offloaded all of your static content to a CDN, the ESI features of varnish
alone can be a huge win.

------
dgerges
I had to click three links to discover what Varnish is. So either I'm dumb or
blind either there is no single description of what the product does on the
home page copy.

~~~
brokentone
It's probably assumed to some degree that you know what it is at this point.
It's the industry standard HTTP full-page cache daemon.

------
beaknit
I love varnish announcements. Their messages seemingly have a 78% likelihood
of being from the machines themselves.

And, frankly, I think the machines _are_ happy to make the announcement.

