For a long time now I have suspected that calling perl subroutines is slow. And I couldn’t figure out from the language shootout which benchmark tested subroutine calls (did it use to be Ackermann?) so I made up my own benchmark.
Perl in comparison to its closest rival – CPython.
I tested a few things:
First, a loop that does nothing, to see how much is loop overhead
Next a zero parameter function
Then a function called with two integers
And finally, declaring those two integers inline.
I’m assuming that an optimiser doesn’t come along and remove code that does nothing. From the results, it seems like a safe assumption.
And, I’m not running benchmarks multiple times or with many iterations because, frankly, I don’t care that much. I just want to get an idea as to how Perl stacks up.
Python Code
def f1(): pass def f2(a, b): pass # return a, b for i in xrange (1, 10000000): pass # f1 # f2(1, 2) # x,y=1,2
Python Results
$ python -V Python 2.6.5 $ time python ./func.py # (pass) real 0m0.722s user 0m0.720s sys 0m0.004s $ time python ./func.py # (x,y = 1,2) real 0m2.030s user 0m2.024s sys 0m0.004s $ time python ./func.py # (f1) real 0m2.265s user 0m2.244s sys 0m0.012s $ time python ./func.py # (f2 - pass) real 0m2.885s user 0m2.880s sys 0m0.004s $ time python ./func.py # (f2 - return a, b) real 0m3.190s user 0m3.144s sys 0m0.012s
Perl Code
sub f1 { } sub f2 { my ($x, $y) = @_; } for ($i = 0; $i < 10_000_000; ++$i) { 1; # f1(); # f2(1, 2); # my ($x, $y) = (1, 2); }
Perl Results
$ time perl ./func.pl # (1) real 0m0.893s user 0m0.888s sys 0m0.004s $ time perl ./func.pl # (f1) real 0m2.932s user 0m2.924s sys 0m0.004s $ time perl ./func.pl # (f2) real 0m5.607s user 0m5.580s sys 0m0.004s $ time perl ./func.pl # (my ($x, $y) ...) real 0m2.687s user 0m2.672s sys 0m0.008s
Conclusions
A few things jump out at me.
1. 10 million subroutine calls take around a couple of seconds. Would reducing call speed actually affect any real program much?
2. Declaring the variables and assigning the values takes almost as long as the empty function call.
3. Python function calls are faster than perl function calls but it’s not by enough to worry about.
My next post should hopefully clarify why I was thinking about this.
Just out of curiosity (and I’m really clueless about this) but what if instead of integers you used strings?
Somehow, I think Perl would be at least a little bit faster in comparison. But then, it’s just a feeling.
Also, keep in mind that Python has things like Psyco while the Perl community only did baby steps so far in this direction (some XS modules doing opcode instrospection/optimization using B::* stuff).
How about method calls?
What version of perl are you running this under?
I believe that list assignment is usually slower than shifting each item off of @_, and there was a regression in 5.10.0 that slowed it down much more. I would be interested in seeing how this version of f2 stacks up:
sub f2
{
my $x = shift;
my $y = shift;
return ($x, $y);
}
@Nilson – I tried strings which just made it slower. Python was able to handle them better. Maybe it is the reassigning of the parameter list from @_ that is slowing Perl down.
@Bartek – Method calls are significantly slower than subroutine calls, in Perl at least.
@James – I’m using version 5.10.1 which should have fixed that regression. However, the subroutine using shift is still slightly quicker.
I’ll post more benchmarks shortly.
For f1, in Perl if a function takes no parameters I believe you can speed it up by telling it as much via prototypes.
sub f1 () {
}
For comparable loop to Python example then its more idiomatic and (pretty sure) faster to use:
for my $i (1 .. 10_000_000 ) { … }
BTW, benchmarks like this have come up on Stackoverflow from time to time. For eg. http://stackoverflow.com/questions/1984871/why-is-my-python-version-slower-than-my-perl-version-closed
/I3az/
As with all benchmarks, domain knowledge is very important.
for ($i = 0; $i < 10_000_000; ++$i) {
1;
}
# time perl tmp.pl
real 0m0.708s
user 0m0.704s
sys 0m0.004s
for my $i (1 .. 10_000_000) {
1;
}
# time perl tmp.pl
real 0m0.523s
user 0m0.520s
sys 0m0.002s
for (1 .. 10_000_000) {
1;
}
# time perl tmp.pl
real 0m0.493s
user 0m0.490s
sys 0m0.002s
Note the last one. The iterator variable is still available through the topic, $_. There's no reason to assign it to a separate variable every iteration, and if you need it, that's what the topic is for.
Just because Perl allows you to write in any style you want, doesn't mean the code will be optimal for execution.
@Adam – Good (and surprising, to me) to know. I didn’t know Perl supported this sort of optimisation.
@draegtun & Kevan – Okay, it is kinda nice to know how to improve the loop slightly. I remember the days that 1 .. 10_000_000 would construct an array in memory
But that isn’t what I’m interested in here. I’m interested in how fast subroutine calls are. That is the reason I measured the loop speed first so I can take it out of the equation. And in my tests, it looks like Python calls are significantly faster than Perl calls (although not sufficiently faster for me to care/switch).
@Jared – Understood. My point wasn’t to show Perl was faster in a specific portion of your tests, but to show that quick benchmarks often yield less than pristine results.
For example, you make the assumption that you need to assign variables for the passed values in f2. While generally not considered best practice, I think few people would complain about accessing them directly as $_[0] and $_[1] if the function itself is only a few lines (I know I can make arguments both ways for a function that small). That brings the results much closer to what you are seeing in Python, but still slower.
The directions the languages have gone are very different, and it might be more fruitful to benchmark common usages in each (although that’s a much larger task). Perl’s arguments as a list with light prototyping sugar yields some interesting and useful practices, but so do python’s stricter prototypes and polymorphism. If each is commonly used in a manner that requires extra processing, that skews the result. Your testing of a function that unpacks some variables shows you at least thought about the base case of this. For example, I’m a fan of using hash semantics to define the list of variables passed in as a list of named parameters, but that’s less than ideal for speed. E.g.
sub myfunc {
my %ops = @_;
}
myfunc(param1 => ‘value1′, param2 => ‘value2′);
An interesting tangent to this is how much boilerplate the language lets you remove from this. Chromatic touched on this when referring to a YAPC talk in his blog at http://www.modernperlbooks.com/mt/2010/06/when-assembly-leaks-through.html. Perl 6 gives you everything I’ve mentioned from both languages plus much more; real named arguments, positional arguments, or slurpy arguments, and you can mix them in a single prototype. Add multiple dispatch and… wow. Yes, this is blatant fanboi advertising. I can’t help it.
Okay, my windbag has sufficiently deflated.
Hi Jared,
Yes I understand what you’re trying to show but its nice to see that the idiomatic empty loop is actually faster in Perl
/I3az/
@Kevan
Interesting – there seems to be a split on this. I don’t like to see direct access of @_ elements in any code. Although admittedly I occasionally do it in one-liners I always feel dirty afterwards. I think Juster agrees and zloyrusskiy disagrees.
I also like the array unpacking idiom (my %args = @_). I flit between using it and not using it. Normally, speed of unpacking doesn’t matter *that* much to me apart from the type of shims I discussed in the Abstraction and Subroutines post.
You’ve piqued my interest in Perl 6 with your mention of the flexible handling of subroutine arguments. Although I’m also a little bit sad that Perl 5 doesn’t have anything as good as lisp macros. Even as weak a variant as emacs lisp has such goodies as keyword parameters easily bolted on by macros in a library.
@draegtun – good point
[...] Benchmarking Perl Subroutine Calls [...]