Sue D. Nymme comments on my blog:
8 – 12 seconds for a program to start!? I don’t remember it being that quite bad when I used it.
moose.pl
use Moose; 1;
nomoose.pl 🙂
1;
test.pl
use Benchmark qw(:hireswallclock); use strict; use warnings; my $perl = 'c:/strawberry/perl/bin/perl'; Benchmark::timethese(100, { 'moose' => sub { system(qq{$perl moose.pl}) }, 'nomoose' => sub { system(qq{$perl nomoose.pl}) }, });
c:\home\jared> perl test.pl Benchmark: timing 100 iterations of moose, nomoose... moose: 33.6734 wallclock secs ( 0.00 usr + 0.14 sys = 0.14 CPU) @ 709.22/s (n=100) (warning: too few iterations for a reliable count) nomoose: 1.25888 wallclock secs ( 0.06 usr + 0.17 sys = 0.23 CPU) @ 427.35/s (n=100) (warning: too few iterations for a reliable count)
Yikes, it’s more than 25 times slower! But is it really that bad? It’s adding between 0.3 and 0.35 seconds to the start time. That actually sounds quite reasonable. And when you consider that a typical large perl program will load a whole bunch of other modules which also add to the start time, maybe it will start to get lost in the noise.
Based on this test, rejecting Moose solely on start-up time overhead seems a little unreasonable. (There could be some more overhead if you actually try to use it mind you.)
test2.pl
Just in case I’d used Benchmark
incorrectly, I tried something a bit more "traditional" (read hacky).
use strict; use warnings; my $script = shift; my $perl = 'c:/strawberry/perl/bin/perl'; for (1..100) { system("$perl $script"); }
$ time perl test2.pl nomoose.pl real 0m5.784s user 0m0.015s sys 0m0.000s $ time perl test2.pl moose.pl real 0m38.146s user 0m0.000s sys 0m0.046s
The startup overhead when using Moose is grows with the number of classes (and their inheritance graph) – so it can easily grow to 8-12 seconds for a real world application.
And I should say that 0.35s is quite noticeable, because a lot of CGI/command line programs finish their main task under 0.1-1s, so the Moose overhead is quite significant. Mouse is at about 0.1s, which is much less worse.
I have a ~9500 sloc application that is Catalyst + KiokuDB (and thus *lots* of Moose).
Running effectively “time perl -Ilib t/*_server.pl” and hitting ctrl-c quickly after startup resulted in:
4.49 real 4.23 user 0.25 sys
running time on the test that makes a single request against the server resulted in
8.79 real 5.10 user 0.32 sys
This is an application that is deployed and has users so is as close to “real world” as you can get. It is however a long-running daemon so I dont’ really care about the startup time since it happens once or worst case twice a release.
I’m trying to get some measure on the much larger application (~50K sloc) but it’s not working at the moment.
@Zbigniew – okay, firstly, have you got any examples? Secondly, do you know how much of that 8-12 seconds is Moose related and how much is loading other modules and connecting to the database (for example). And finally, yikes, 8-12 seconds? What is Moose doing with the other 7.65 – 11.65 seconds? Building an inheritance graph doesn’t sound *that* expensive.
@Steven – You’re right. I’m making the classic blunder of thinking that everyone else is writing similar code to me. Moose probably isn’t the best choice for a CGI script. Your qmail example is similarly reasonable.
For command-line scripts, I’d argue that a 0.35 second overhead is largely irrelevant. I do a lot of commandline work and most of my (even short) scripts spend at least a couple of seconds connecting to various databases and extracting data. Another 0.35 seconds would not bother me.
I’m not convinced by your stability argument either. Most of my stuff is regularly bounced for regular reasons, but if it is running for more than a minute then 0.35 seconds is not much. How frequently do you restart your daemons?
@Chris – excellent, a realworld example. But wait a minute, are you saying that pressing ctrl-c after starting is a reliable test? Even I am not generally quite that slapdash 😉 And for connecting to a remote server, how much of the overhead is down to Moose?
I have to ask a question. I know ‘no Moose’ basically then shuts Moose off so it can’t interfere with functions and such in things using it. But other than that is there any advantage to no Moose? would we gain performance by not using ‘no Moose’? does ‘namespace::clean’ (etc) have the exact same effect? Moose’s negative impact in startup reminds me of Java, and that startup impact is why I don’t use any Java apps.
Hi Caleb,
I have similar reservations about using Java apps, but I probably shouldn’t have. Most significant apps that I use have a start-up time which is significantly greater that starting the JVM, or starting Perl + Moose due to connecting to databases, loading config into memory or that sort of stuff.
I don’t know if it is faster to omit no Moose but it should be fairly easy to benchmark. My guess would be yes, but not much.
Hello Jared,
Thanks for following-up 🙂 I am sorry I didn’t respond sooner; I have been going through the throes of a Windows XP -> 7 upgrade. 😐
Yeah, the problem is that simply including the Moose module without having it do anything doesn’t give you (or, I suspect, the maintainers of Moose) an idea of how it behaves when you actually *use* it.
I whipped up a simple example this morning. You can see a simple business object implemented as a Moose class at http://nopaste.gamedev.pl/?id=7867 (or http://cl.ly/f0d5a82cd42018f0a648). And an equivalent object implemented as an inside-out object at http://nopaste.gamedev.pl/?id=7868 (http://cl.ly/188e46e61f1bc111a673). I didn’t “cheat” — the I-O class implements approximately the same compile-time and run-time error checking; it uses Modern::Perl and Params::Validate and DateTime and stuff. The Moose one uses MooseX::Declare and so on. So I think the two modules should line up as fairly equivalent in terms of the features they provide.
Then I wrote a simple program that just uses one or the other class, creates an object, and exits:
use Modern::Perl;
use MooseObject; # or InsideOutObject
my $thing = MooseObject->new(name => ‘Jim’);
Finally, I ran the program ten times for each object class (on a fairly new, fairly fast workstation with no load). The inside-out object program ran ten times in 11.1 seconds; the Moose object program took 46.2 seconds.
So depending on how you look at it, Moose added 3.5 seconds to each compile, or it tripled the compile time.
And that’s just a toy program. Several weeks ago I started to rewrite a major in-house GUI app in Moose. After I had a couple dozen GUI and database object classes implemented in Moose, the startup time became intolerable. Yes, it literally was 8-12 seconds per startup — that was no exaggeration — and I had only implemented one-half to one-third of the objects that the final app would need.
Moose has the potential to save a lot of programmer time and effort. But honestly, it also has the potential to take away as much as it adds. While I was writing this dopey little test script, I had to fight errors like:
P:\dev\moose-timing>perl moose.pl
Validation failed for ‘MooseX::Types::Structured::Tuple[MooseX::Types::Structured::Tuple[Object,Str,Str],MooseX::Types::Structured::Dict[]]’ with value [ [ MooseObject=HASH(0x17c07c), “Jim” ], { } ], Internal Validation Error is: Validation failed for ‘MooseX::Types::Structured::Tuple[Object,Str,Str]’ with value [ MooseObject{ name: “Jim”, phone: “999-999-9999” }, “Jim” ] at //Summ2/Perl/site/lib/MooseX/Method/Signatures/Meta/Method.pm line 409
MooseX::Method::Signatures::Meta::Method::validate(‘MooseX::Method::Signatures::Meta::Method=HASH(0x4310c04)’, ‘ARRAY(0x47d8a14)’) called at //Summ2/Perl/site/lib/MooseX/Method/Signatures/Meta/Method.pm line 145
MooseObject::_name_changed(‘MooseObject=HASH(0x17c07c)’, ‘Jim’) called at generated method (unknown origin) line 37
MooseObject::new(‘MooseObject’, ‘name’, ‘Jim’) called at moose.pl line 4
Can you figure out what the problem is from that? I had a hell of a time. It turns out that when Moose attribute triggers fire for the *first* time, when the attribute has no value, the triggers are only passed the *new* value, not the new-and-old values. So the method signature barfs because it was expecting two arguments. Making the second argument optional fixed that. Moose::Manual::Attributes does not mention this.
Moose programs are almost impossible to debug, too, because you can’t step through the generated code.
I would love nothing more than for Moose to work. It looks beautiful. Writing Moose classes is a breeze, a joy, compared to the tedium of writing inside-out classes. But until Moose fixes its horrible startup cost, until Moose has decent error messages, until Moose programs are actually debuggable, Moose is simply a pretty experiment, not suitable for the real world.
Thanks for the in-depth comment Pseudo, if that’s your real name (bwahaha, gets me every time!) Sorry, I’ll calm down now.
And apologies your comment got stuck in unapproved purgatory. Out of the box, wordpress does that with comments with more than two links, which catches a lot of spam. I thought it had some learning capability where if I let previous comments from a user run it would let new ones through too.
Although I believe you (why would you make it up?), it does seem surprising. What is it doing with all that time? I’m going to check your working anyway of course just in case. Because it would be silly to say something without checking. And I obviously have to follow-up again.
Cheers for the input!
[…] on from my benchmarking Moose startup overhead post, Sue D. Nymme provided code that actually created an object. Thanks […]
> So the method signature barfs because it was expecting two arguments. Making the second argument optional fixed that. Moose::Manual::Attributes does not mention this.
How would you interpret this then (from Moose::Manual::Attributes under “Triggers”):
> The trigger is called after an attribute’s value is set. It is called as a method on the object, and receives the new and old values as its arguments. **If the attribute had not previously been set at all, then only the new value is passed.** This lets you distinguish between the case where the attribute had no value versus when it was “undef”.
__PACKAGE__->meta->make_immutable;
http://search.cpan.org/~stevan/Moose-0.53/lib/Moose/Cookbook/Recipe7.pod
Ether: Good call. I totally missed that. Boy is there egg on my face.
big: I don’t see how that would help. 1) making the class immutable speeds up runtime creation of objects, not compile-time. 2) MooseX::Declare automatically makes classes immutable by invoking make_immutable behind the scenes.
Try this in test.pl:
'moose' => sub { system(qq{perl moose.pl}) },
'mouse' => sub { system(qq{perl mouse.pl}) },
'anymoose' => sub { system(qq{perl anymoose.pl}) },
'nomoose' => sub { system(qq{perl nomoose.pl}) },
And this in anymoose.pl:
use Any::Moose;
1;