Small correction: as of today, SoLoad has rudimentary support for callback functions. Although I have only tested it on a single example function so far, it seems to actually work pretty well.

Continuations, on the other hand, are not likely to be forthcoming any time soon.

I didn’t mean to imply that the benchmark guys *should* go out and write tests for all the various implementations, that’d likely turn into a full time job just for Scheme, let alone the various flavors of all the other languages too. And then keeping them up to date… it might be a fun job, but not for free =)

As for the profiling, I’m sure the implementation devs already do that sort of thing when they’re working on performance, and would be much more apt at such a task.

In the end I agree with Oisín’s description of benchmarks being a “grainy, distorted lens”.

And the benchmark game will not attempt to measure every Scheme implementation – that’s a chore for the Scheme community

Having said that I did install Larceny and did try to find someone interested in implementing the programs – without success; and I did install Gambit-C and did try to find someone interested in implementing the programs but they only provided a couple, which didn’t seem to work.

When you write “heavily numerical tests” I’d think of tests like n-body or spectral-norm, the sort of thing Fortran made its name doing.

pidigits is about arbitrary precision arithmetic, not hardware floats and ints.

fannkuch is about loops and vectors – can you profile the code and find where one of the Schemes spends time?

fasta certainly spends a chunk of time on random number generation – can you profile the code and find where one of the Schemes spends time?

binary-trees isn’t what anyone would think of as “heavily numerical” but none of the Scheme implementations are accepted because they use a more compact leaf-node.