

Geoff Wozniak: Tinkering with jump tables - fogus
http://exploring-lisp.blogspot.com/2009/07/tinkering-with-jump-tables.html

======
jsnell
Unfortunately the numbers are rather misleading here, since the benchmark
scaffolding he uses has very different performance characteristics on
different implementations. He's passing the symbols naming the functions to
test to time-tests, but funcalling symbols (instead of functions) is very
expensive on sbcl. So it's pretty much a benchmark of the speed of symbol-
function, not of different case implementations :-)

~~~
gjm11
Surely that doesn't have any effect on the intra-implementation results. So,
for instance, it still seems he's shown that case has O(1) performance on
CLISP but some sort of (apparently quite severe) number-of-cases dependence on
SBCL and ACL, whereas ncase and switch are pretty much O(1) on all three
implementations.

~~~
jsnell
When 95% of the runtime is spent in the benchmarking infrastructure rather
than the benchmarked code, interpreting even the intra-implementation results
is hard. Here's the results I get (benchmark, original, symbol-function lifted
out of time-test). Times in milliseconds:

    
    
      CASE-10 240 12
      CASE-100 244 52
      CASE-1000 632 432
    
      NCASE-10 212 20
      NCASE-100 220 24
    
      SWITCH-10 276 80
      SWITCH-100 272 76
      SWITCH-1000 268 72
    

For example SWITCH looks a lot more reasonable for a small number of cases
with the original benchmark than it does when the benchmarking overhead is
reduced.

(Edit: try to reformat the table)

~~~
gjm11
I'm assuming those are SBCL numbers, given who's posting them :-). So, it
seems to me that these figures confirm that CASE gets expensive for large
numbers of cases whereas SWITCH doesn't. (You're missing the NCASE-1000 entry.
If it's also quite small then you're also confirming that NCASE doesn't get
expensive for large numbers of cases.) Which I thought was the most
interesting finding here. You've also confirmed that SWITCH is a bit more
expensive than CASE and NCASE except when the former blows up.

However. Geoff Wozniak's results seem to show (1) CASE-100 being more
expensive than CASE-10, where you find it costing the same; and (2) CASE-1000
being more expensive than CASE-10 by a much larger factor than you've found.
The curious thing here is that a constant extra bogus overhead from an
expensive SYMBOL-FUNCTION implementation ought (I'd have thought) to bias his
results in the opposite direction to that. Are you and he using different
versions of SBCL, or something?

