

Introducing Triton: A static cache-enabled web server in Go - cjslep
http://cjslep.com/blog/triton

======
lunixbochs
To clarify: do you watch filesystem for template changes, then store the
parsed template in memory, or do you execute the template every time?

I'd be curious to see benchmarks against nginx with the equivalent html files
already generated.

> The droplet hosting the former site only accepted its key, so losing the
> hard drive meant I lost access to maintain the site.

Droplet as in DigitalOcean? As long as you still have access to your
DigitalOcean account, you can "Reset Root Password" and use "Console Access"
to re-establish ssh.

~~~
cjslep
> To clarify: do you watch filesystem for template changes, then store the
> parsed template in memory, or do you execute the template every time?

I watch the filesystem for template changes and re-read the template into
memory. However, upon serving a request the template is re-executed. The
rationale is to, in the future, expose the data portion of Go's
template.ExecuteTemplate so a third-party content manager can pass in
information such as date of publication, published by whom, tags, etc.

> As long as you still have access to your DigitalOcean account...

I had no idea that was possible, but now I am kicking myself. Fortunately, my
previous site had very little. Thanks for pointing that out!

~~~
lunixbochs
What do you mean by "third party content manager"? That sounds quite a bit
like a database. Is this a static site or not? :)

------
mcosta
Why manually cache plain files if the kernel does it for you?

~~~
cjslep
Good question!

In theory, the disk cache uses spare memory that applications are not using,
so it is not guaranteed to be present if I am hosting it on a server with
little-to-no free memory. However, if an application contains the cache then
it will not be eliminated in a memory-starved environment.

In practice, this will probably rarely happen on my server.

Edited to add the "in practice" bit.

~~~
lunixbochs
On a memory-constrained system, forcing files into memory will increase memory
pressure. You have roughly the same amount of cacheable data either way, but
many VPS providers don't configure a swap partition by default, so the kernel
can't page out your idle memory to make room for something else and you can
run into OOM problems.

If you're not hitting the bounds of your machine's memory, the kernel disk
cache will do a pretty good job. If you _are_ , then you're more likely to
cause problems by forcing more active data into memory.

For what it's worth, the kernel is free to evict your unused memory pages to
disk (on a system with swap), which is almost an identical access model to
disk caching:

Virtual memory: We need this previously-idle section of memory, so let's
restore it from disk until it becomes idle again.

Disk caching: We need this previously-idle section of disk, so let's cache it
in memory until it becomes idle again.

~~~
cjslep
So if I am understanding this right, to really get any appreciable cache
improvements would I need to pre-execute the templates once and store that
generated output in a temporary file to leverage the disk cache and repeatedly
fetch (cache hit) that file upon request?

~~~
lunixbochs
The basic statement I was trying to make was "due to disk caching, putting
your cache in memory or disk is roughly equivalent, except it's easier for the
kernel to make decisions about an on-disk cache."

As far as re-executing templates, let's break down some hypothetical costs,
including pretty optimistic network latency.

1\. Accept, read, and parse a request from a socket: >50ms

2\. Read a template from (solid-state) disk: 16µs

3\. Parse and execute a template: 100µs

4\. Serve the response over TCP: >20ms

If we replace (2) with a memory cache (whether it's your app or the kernel),
the cost goes down to maybe 1-5µs.

Your time-to-response is effectively the same with template caching. So what's
the point of a generated static site? Reducing CPU and memory pressure. Your
page load times are almost completely bottlenecked on network, so you can
optimize this using a CDN/external caching service (like Cloudflare) to bring
the content closer to the user, or you can optimize by removing CPU cycles per
page load to increase the number of concurrent connections you can handle.

There's a point where the userspace to kernel latency in serving a file is
also a bottleneck, but nginx can use the sendfile() [1] system call to copy a
file directly to a socket (from the kernel disk cache) without ever touching
userspace, which is _much_ faster than anything you're going to do with in-app
caching, unless you also put the TCP stack in-app.

[1] [http://linux.die.net/man/2/sendfile](http://linux.die.net/man/2/sendfile)

