Perl Advent Calendar 2010-12-21

Robot Santa asks, "Is your code nice enough?"

by Jerrad Pierce

Even when your hardware's rated at 50 megachecks per second, it is sometimes necessary to streamline your code.1 Alas, due to the vagaries of processor scheduling, disk access, et cetera, benchmarking is an inherently futile activity, fraught with uncertainty not dissimilar to that experienced in quantum mechanics …to paraphrase the fine manual of today's feature module. But things are a little better if you apply statistics. Therefore Benchmark::Timer uses Student's t-Test to determine how many times it should execute your code so that it can return an accurate run-time, within limits you define, rather than you having to choose and wait for some arbitrarily large number of times to loop.

% perl
69 trials of factorial (532.050ms total), 7.711ms/trial
Error: +/- 0.00038 with 95 confidence

Other features of Benchmark::Timer include the abilities to time your code inline without having to roll it into subroutines, and to benchmark arbitrary chunks of code including subsets of the statements included in other "tags." The module also allows you to control for slow and expensive hits incurred by initial runs, such as loading files into the system's buffers, by discarding the data from initial runs with the skip option.

Some misfeatures of Benchmark::Timer are the need to explicitly specify a minimum greater than one in order to ensure adequate statistics are gathered. In addition, it is sometimes necessary e.g; when only one test is run; to sleep or perform some other timely operations before requesting a report from Benchmark::Timer so that internal checks the method relies upon are cleared.

   1 use Benchmark::Timer;
   3 my $T = Benchmark::Timer->new( confidence=>95, error=>5, minimum=>2 );
   5 while( $T->need_more_samples('factorial') ){
   6   $T->start('factorial');
   7   factorial($_) foreach 0..123;
   8   $T->stop('factorial');
   9 }
  11 print $T->report;
  13 sub factorial{
  14   $_[0] > 1 ? $_[0] * factorial($_[0]-1) : 1;
  15 }

1. Just be sure to stay on the nice list and avoid the temptation of premature optimization.

View Source (POD)