My most recent post has a few comments, but because I’ve been on holiday I didn’t reply to them in a timely manner. Sorry about that folks, I did read all of them.
And time has moved on so if I reply to those comments in the comments section, I’m guessing the original commenters won’t read the response. Having said that, they might not read this post either, but I think that is less likely.
Therefore this is a post to respond to the commenters, and to keep up with the IronMan schedule even though I’m not sure I’m being measured.
First of all, Roman, thanks for the comment. The figures you provide are interesting and yours is a second complaint about opaque error messages. That would bother me as well – I get enough of them from emacs macros and I have at least got tools (well, a tool –
macroexpand) to debug those.
@Max – Hi. I’m not sure how you got the impression that I was interested in object creation time. We have primarily been talking about Moose start-up overhead here. I think I’m measuring the right thing.
@Anonymous – when you say "Run each benchmark item for 5 seconds or more." I think you missed this bit of the post:
Benchmark: timing 200 iterations of 1just-moose, 2all-includes, 3nocreate, 4create... 1just-moose: 100.196 wallclock secs ( 0.05 usr + 0.25 sys = 0.30 CPU) @ 673.40/s (n=200) 2all-includes: 279.252 wallclock secs ( 0.08 usr + 0.34 sys = 0.42 CPU) @ 475.06/s (n=200) 3nocreate: 318.256 wallclock secs ( 0.06 usr + 0.28 sys = 0.34 CPU) @ 581.40/s (n=200) 4create: 320.979 wallclock secs ( 0.06 usr + 0.27 sys = 0.33 CPU) @ 611.62/s (n=200)
Not one of my benchmarks runs for less than a minute and a half. Most of them are around the five minute mark. And I have deadlines!
So where is this message coming from? I had a look in Benchmark.pm.
print " (warning: too few iterations for a reliable count)\n" if $n < $Min_Count # 4 || ($t->real < 1 && $n < 1000) || $t->cpu_a < $Min_CPU; # 0.4
My iterations is 200 so $Min_Count is not the problem. $t->real is at least 100 so that leaves $Min_CPU. Hmmm… yeah, I would have had to double the number of iterations to be sure. Look at 4create – 321 wallclock seconds of which 0.3 is CPU. No worries, I’m good – the warnings were spurious as I originally thought.