"When you have the users on your server, you don't have to rely on benchmarks, for example. Benchmarks are simulated users. With server-based software, you can watch actual users. To decide what to optimize, just log into a server and see what's consuming all the CPU. And you know when to stop optimizing too: we eventually got the Viaweb editor to the point where it was memory-bound rather than CPU-bound, and since there was nothing we could do to decrease the size of users' data (well, nothing easy), we knew we might as well stop there."
Couldn't you have done better if, instead of keeping all hash entries in memory, you kept them on disk and used memory as an LRU buffer? It seems like you could have fit more users on a server that way.
The OS isn't very smart about it because it quantizes everything along page boundaries and has no semantic understanding of the data. That works fine in the typical desktop-system case where you have 100 processes using a few megabytes each and at any given time only one or two are active. When you're running one server process that uses all the system's memory, you can be a lot more efficient by handling it yourself.
"When you have the users on your server, you don't have to rely on benchmarks, for example. Benchmarks are simulated users. With server-based software, you can watch actual users. To decide what to optimize, just log into a server and see what's consuming all the CPU. And you know when to stop optimizing too: we eventually got the Viaweb editor to the point where it was memory-bound rather than CPU-bound, and since there was nothing we could do to decrease the size of users' data (well, nothing easy), we knew we might as well stop there."
Couldn't you have done better if, instead of keeping all hash entries in memory, you kept them on disk and used memory as an LRU buffer? It seems like you could have fit more users on a server that way.