Nested Maps in Perl

I’m not really a fan of nested loops, so when I need to create a list of combinations based on two or three other lists, I really miss list comprehensions¹ such as those in Python.

l = [(x,y) for x in range(5) for y in range(5)]

Very elegant.

If I was using Lisp, I might use a nested mapcar.

(mapcar (lambda (out) (mapcar (lambda (in) (cons in out))
                              '(a b c d e)))
          '(1 2 3 4 5))

Perl map uses $_ so you don’t need to explicitly specify the name of the lambda parameter. How can I differentiate between the outer $_ and the inner $_?

my $outer;
my @x = map { $outer = $_;
              (map { $outer . $_ } qw(a b c d e)) }
            (1..5);

Note: if you are trying to do something as simple as this, take a look at Set::CrossProduct instead.

---

1. It’s available on the CPAN of course

Read Full Post »

That Syncing Feeling – Safety Is Expensive

Recently I’ve been thinking about storing records on disk quickly. For my general use case, an RDBMS isn’t quite fast enough. My first thought, probably like many a NoSQL person before me, is how fast can I go if I give up ACID?

Even if I’m intending to do the final implementation in C++, I’ll often experiment with Perl first. It’s often possible to use the C libraries anyway.

First up, how about serialising records using Storable to an on-disk hash table like Berkeley DB.

(Aside: I’m probably going to appear obsessed with benchmarking now, but really I’m just sticking a finger in the air to get an idea about how various approaches perform. I can estimate 90cm given a metre stick. I don’t need a more precise way to do a rough estimate.)

use Storable;
use Benchmark qw(:hireswallclock);

use BerkeleyDB;

I need to make a random record to store in the DB.

sub make_record
{
    my ($order_id, $fields, $key_len, $val_len) = @_;

    my %record;
    $record{'order_id'} = $order_id;

    for my $field_no (1..$fields-1) {
        my $key = 'key';
        my $val = 'val';
        $key .= chr(65 + rand(26)) for (1..$key_len - length($key));
        $val .= chr(65 + rand(26)) for (1..$val_len - length($val));
        $record{$key} = $val;
        print "$key -> $val\n";
    }
    return \%record;
}

And a wrapper handles the general case I’ll be testing – key and value equal length and order_id starting at 1.

sub rec
{
    my ($fields, $len) = @_;
    return make_record(1, $fields, $len, $len);
}

I’ll compare serialisation only against actually storing the data to disk to see what the upper limit I could achieve is if, for example, I was using an SSD.

sub freeze_only
{
    my ($db, $ref_record, $no_sync) = @_;
    $no_sync //= 0;
    my $key = "/order/$ref_record->{'order_id'}";
    Storable::freeze($ref_record);
}

And I’m curious to know how much overhead syncing to disk adds.

my $ORDER_ID = 0;

sub store_record
{
    my ($db, $ref_record, $no_sync) = @_;
    $no_sync //= 0;
    $ref_record->{'order_id'} = ++$ORDER_ID;
    my $key = "/order/$ref_record->{'order_id'}";
    $db->db_put($key, Storable::freeze($ref_record));
    $db->db_sync() unless $no_sync;
}

The Test Program

A record with 50 fields, each of size 50 seems reasonable.

my $filename = "$ENV{HOME}/test.db";
unlink $filename;

my $db = new BerkeleyDB::Hash
    -Filename => $filename,
    -Flags    => DB_CREATE
    or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;

my $rec_50_50 = rec(50, 50);

Benchmark::cmpthese(-1, {
    'freeze-only-50/50' => sub { freeze_only($db, $rec_50_50) },
    'freeze-sync-50/50' => sub { store_record($db, $rec_50_50) },
    'freeze-no-sync-50/50' => sub { store_record($db, $rec_50_50, 1) },
});

The Results

                        Rate freeze-sync-50/50 freeze-no-sync-50/50 freeze-only-50/50
freeze-sync-50/50     1543/s                --                 -80%              -93%
freeze-no-sync-50/50  7696/s              399%                   --              -63%
freeze-only-50/50    21081/s             1267%                 174%                --

Conclusion

Unsurprisingly syncing is expensive – it adds 400% overhead. However, even with the sync, we’re still able to store 5.5 million records an hour. Is that fast enough for me? (I need some level of reliability) It might well be.

Berkeley DB is fast. It only adds 170% overhead to the serialisation itself. I’m impressed.

In case anyone is interested. I ran a more comprehensive set of benchmarks.

freeze-sync-50/50       1846/s
freeze-sync-50/05       2262/s
freeze-sync-05/50       2546/s
freeze-sync-05/05       2799/s
freeze-no-sync-50/50    7313/s
freeze-no-sync-50/05    9514/s
freeze-no-sync-05/50   11395/s
freeze-no-sync-05/05   12589/s
freeze-only-50/50      20031/s
freeze-only-50/05      21920/s
freeze-only-05/05      26547/s
freeze-only-05/50      26547/s
fcall-only           2975364/s

Read Full Post »

Disappearing CPAN Modules

Somewhere in my Super Programming Languages post, I asserted that in some of my scripts, 95% of the work has already been done by third party libraries. Such is the power of the CPAN! And other folks obviously use Perl for similar reasons – there are libraries available that solve at least part of their problem.

So, I’m surprised when people complain that libraries have been provided that help them out.

Steve obviously had second thoughts over publishing this post (google cache). Is blogpost necrophilia reasonable here? I’m afraid I’ve seen other people make similar arguments, and I can’t remember the location of their posts. Sorry Steve.

Ok, so from that, I can gather this: I’m screwed because I used perl for a script, and now the module author has decided to completely abandon the module, because it is based upon another module that doesn’t install worth anything. Cool!

He used an extremely useful and mature scripting language and a module that solves part of his problem. Because the module doesn’t work with a later version of that excellent scripting language, he’s doomed, doomed, DOOMED I tell you. And of course, it’s Perl’s fault.

The first thing is that he’s not doomed. He is simply going to have to find an alternative module or implement the functionality provided by the module himself. He’d have had to do that if the module wasn’t provided anyway. So in no way was he disadvantaged by having the module (and Perl) available.

The other thing is that this happens in every changing environment. Say your C++ compiler upgrades and you were inadvertently depending on a particular bug. Oops! (Don’t laugh – it happens). If your firm upgrades Excel to version 2000007, maybe your macros aren’t going to run any more. We’re moving to a 64-bit OS and I’m glad there’s a bunch of Perl that I can be confident will more or less work. I’m not looking forward to porting the C++ though.

To try and add a fair and balanced viewpoint, I should point out that part of the argument does make sense. It is disappointing that Daemon::Simpler has gone from Backpan. Having said that, sometimes it is important to take some responsibility. Any module that I bring into our production environment is archived as is the current version of perl we’re using.

Read Full Post »

Perl Compiler Speed – It’s Fast!

As a developer who is not responsible for many infrastructure modules, I shouldn’t really waste so much of time micro-benchmarking. Profiling my code after the fact should be sufficient if even that is necessary.

But it seems I’ve got into a bad habit.

---

I’m not the only person that thinks that MooseX loading all those pre-requisites is what is taking a lot of time. Dave Rolsky says in the latest Moose blog post:

What’s a bit sad, however, is that this only appears to save 6-10% of the compilation time in real usage. I’m not sure why that is, but I think the issue is that the perl core’s compilation time may be a big factor. As I said, loading that one Markdent modules loads a lot of modules (203 individual .pm files), and the raw overhead of simply reading all those files and compiling them may be a significant chunk of the compilation time.

That gets me thinking about the speed of the perl compilation step. Although using eval doesn’t have the same disk IO as use, it still exercises the compiler.

Loop Unrolling with Eval

Does anyone remember back in the day when we used to unroll loops? For example, if we had an instruction sequence that was do_something, add, jne and do_something was relatively short, you could save some time unrolling that to do_something, add, do_something, add, jne – you would only have to execute approximately half as many jne instructions across a loop lifetime.

That is something I never worry about in perl. Unless I have to come up with contrived examples for my blog.

sub eval_loop
{
    my $count = 0;
    my $s = '';
    for (my $i = 0; $i < 100_000; ++$i) {
        $s .= '$count += ' . $i . ";\n";
    }
    eval $s;
    $count;
}

sub vanilla_loop
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
    }
    $count;
}

my $code = '';
for (my $i = 0; $i < 100_000; ++$i) {
    $code .= '$count += ' . $i . ";\n";
}

$code = '
sub totally_unrolled {
    my $count = 0;' . "\n" . $code . '
    $count;
};';

eval $code;

eval_loop compiles the unrolled loop every time the subroutine is called whereas totally_unrolled is compiled prior to benchmarking.

I also wanted some partially unrolled loops for comparison sake. Obviously these examples are not suitable for production use. For example, consider what would happen if unrolled_loop_4 needed to execute a number of times that was not a multiple of 4.

sub unrolled_loop_2
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
    }
    $count;
}

sub unrolled_loop_4
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
        $count += ++$i;
        $count += ++$i;
    }
    $count;
}

Benchmarking

The full benchmarking code is as follows. I test the result of each function to ensure I’m getting the same answer.

use Benchmark qw(:hireswallclock);

use strict;
use warnings;

sub vanilla_loop
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
    }
    $count;
}

sub unrolled_loop_2
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
    }
    $count;
}

sub unrolled_loop_4
{
    my $count = 0;
    for (my $i = 0; $i < 100_000; ++$i) {
        $count += $i;
        $count += ++$i;
        $count += ++$i;
        $count += ++$i;
    }
    $count;
}


sub eval_loop
{
    my $count = 0;
    my $s = '';
    for (my $i = 0; $i < 100_000; ++$i) {
        $s .= '$count += ' . $i . ";\n";
    }
    eval $s;
    $count;
}

my $code = '';
for (my $i = 0; $i < 100_000; ++$i) {
    $code .= '$count += ' . $i . ";\n";
}

$code = '
sub totally_unrolled {
    my $count = 0;' . "\n" . $code . '
    $count;
};';

eval $code;

print vanilla_loop(), "\n";
print unrolled_loop_2(), "\n";
print unrolled_loop_4(), "\n";
print eval_loop(), "\n";
print totally_unrolled(), "\n";

Benchmark::cmpthese(-1, {
    'vanilla' => \&vanilla_loop,
    'unrolled-2' => \&unrolled_loop_2,
    'unrolled-4' => \&unrolled_loop_4,
    'eval' => \&eval_loop,
    'unrolled-max' => \&totally_unrolled,
});

And the results are:

$ perl unrolling.pl
4999950000
4999950000
4999950000
4999950000
4999950000
               Rate        eval  unrolled-2  unrolled-4     vanilla unrolled-max
eval         3.25/s          --        -93%        -95%        -95%         -96%
unrolled-2   48.1/s       1381%          --        -25%        -26%         -45%
unrolled-4   63.9/s       1865%         33%          --         -2%         -27%
vanilla      65.1/s       1902%         35%          2%          --         -26%
unrolled-max 87.7/s       2598%         82%         37%         35%           --

Conclusions

I am actually surprised that eval is as fast as it is. It runs less than 20 times slower than the vanilla loop, yet it has to construct a string by appending to it 100,000 times and run the compiler on the result. Or to put it another way, it compiles and runs 100,000 lines of code in under a third of a second. Good job perl compiler guys! Or did you put something in there for patholgically ridiculous examples like this?

And I was right never to worry about unrolling my loops. unrolled-2 is quite significantly slower than vanilla. As I can’t think of any good reason for that, it makes me worried that I’ve done something wrong.

Even fully unrolling the loop, which is quite ridiculous gives me a relatively tiny 35% speed increase.

---

Related posts:

• Benchmarking Perl Subroutine Calls
• More Subroutine Benchmarking
• Spurious Benchmark.pm Warnings

Read Full Post »

Spurious Benchmark.pm Warnings

My most recent post has a few comments, but because I’ve been on holiday I didn’t reply to them in a timely manner. Sorry about that folks, I did read all of them.

And time has moved on so if I reply to those comments in the comments section, I’m guessing the original commenters won’t read the response. Having said that, they might not read this post either, but I think that is less likely.

Therefore this is a post to respond to the commenters, and to keep up with the IronMan schedule even though I’m not sure I’m being measured.

First of all, Roman, thanks for the comment. The figures you provide are interesting and yours is a second complaint about opaque error messages. That would bother me as well – I get enough of them from emacs macros and I have at least got tools (well, a tool – macroexpand) to debug those.

@Max – Hi. I’m not sure how you got the impression that I was interested in object creation time. We have primarily been talking about Moose start-up overhead here. I think I’m measuring the right thing.

@Anonymous – when you say "Run each benchmark item for 5 seconds or more." I think you missed this bit of the post:

Benchmark: timing 200 iterations of 1just-moose, 2all-includes, 3nocreate, 4create...
  1just-moose: 100.196 wallclock secs ( 0.05 usr +  0.25 sys =  0.30 CPU) @ 673.40/s (n=200)
2all-includes: 279.252 wallclock secs ( 0.08 usr +  0.34 sys =  0.42 CPU) @ 475.06/s (n=200)
    3nocreate: 318.256 wallclock secs ( 0.06 usr +  0.28 sys =  0.34 CPU) @ 581.40/s (n=200)
      4create: 320.979 wallclock secs ( 0.06 usr +  0.27 sys =  0.33 CPU) @ 611.62/s (n=200)

Not one of my benchmarks runs for less than a minute and a half. Most of them are around the five minute mark. And I have deadlines!

So where is this message coming from? I had a look in Benchmark.pm.

    print "            (warning: too few iterations for a reliable count)\n"
        if     $n < $Min_Count # 4
            || ($t->real < 1 && $n < 1000)
            || $t->cpu_a < $Min_CPU; # 0.4

My iterations is 200 so $Min_Count is not the problem. $t->real is at least 100 so that leaves $Min_CPU. Hmmm… yeah, I would have had to double the number of iterations to be sure. Look at 4create – 321 wallclock seconds of which 0.3 is CPU. No worries, I’m good – the warnings were spurious as I originally thought.

Read Full Post »

Benchmarking – Measuring The Right Thing

The debate over whether or not Moose is too slow rumbles on. On one side, are the folks using it in production (and presumably happpy with it). On the other, are folks saying it adds multiple seconds to the start-up time.

Following on from my benchmarking Moose startup overhead post, Sue D. Nymme provided code that actually created an object. Thanks Sue!

I took a look and Sue’s code uses MooseX::Declare that I haven’t installed. One magic CPAN invocation and, oh my gosh, more than 20 minutes later, MooseX::Declare is finally installed. Now I’m thinking that the benchmark time is actually measuring the time for perl to load in and parse MooseX::Declare and tens of thousands of lines of prerequisites rather than how slow Moose itself is.

I’ll compare just loading the modules Moose vs MooseX::Declare.

Then I’ll add the rest of Sue’s code.

And finally I’ll actually create a Moose object.

just-moose.pl

use Modern::Perl;

use Moose;
use Moose::Util::TypeConstraints;
use DateTime;

1;

all-includes.pl

use Modern::Perl;

use MooseX::Declare;
use Moose::Util::TypeConstraints;
use DateTime;

1;

And moose.pl (taken from Sue D. Nymme’s comment here.

bench.pl

use Benchmark qw(:hireswallclock);

use strict;
use warnings;

my $perl = 'c:/strawberry/perl/bin/perl';

Benchmark::timethese(200, {
    '1just-moose' =>
        sub { system(qq{$perl -e "require 'just-moose.pl';"}) },
    '2all-includes' =>
        sub { system(qq{$perl -e "require 'all-includes.pl';"}) },
    '3nocreate' => sub { system(qq{$perl -e "require 'moose.pl';"}) },
    '4create' =>
        sub { system(qq/$perl -e "require 'moose.pl'; MooseObject->new({ name => 'Bob' })"/) },
});

The Results

I’ve elided the too few iterations warnings.

Benchmark: timing 200 iterations of 1just-moose, 2all-includes, 3nocreate, 4create...
  1just-moose: 100.196 wallclock secs ( 0.05 usr +  0.25 sys =  0.30 CPU) @ 673.40/s (n=200)
2all-includes: 279.252 wallclock secs ( 0.08 usr +  0.34 sys =  0.42 CPU) @ 475.06/s (n=200)
    3nocreate: 318.256 wallclock secs ( 0.06 usr +  0.28 sys =  0.34 CPU) @ 581.40/s (n=200)
      4create: 320.979 wallclock secs ( 0.06 usr +  0.27 sys =  0.33 CPU) @ 611.62/s (n=200)

Conclusions

The bulk of the time is in loading the MooseX::Declare library. But on the other hand, declaring a single object did take a significant amount of time (approximately 40 seconds over 200 runs). I can now believe that declaring a lot of objects would be prohibitively expensive.

But that is using MooseX::Declare. I’m sure it is nowhere near as bad if we used Plain Ole Moose.

Read Full Post »

Benchmarking Moose Startup Overhead

Sue D. Nymme comments on my blog:

I think Moose is fantastic, awesome, intuitive, helpful – except for one thing: its awful, unforgivable speed penalty. When writing GUI or CGI programs, I simply cannot wait eight to twelve seconds for a program to start. Ergo, none of my local library modules use Moose.

8 – 12 seconds for a program to start!? I don’t remember it being that quite bad when I used it.

moose.pl

use Moose;

1;

nomoose.pl

1;

test.pl

use Benchmark qw(:hireswallclock);

use strict;
use warnings;

my $perl = 'c:/strawberry/perl/bin/perl';

Benchmark::timethese(100, {
    'moose'   => sub { system(qq{$perl moose.pl}) },
    'nomoose' => sub { system(qq{$perl nomoose.pl}) },
});

c:\home\jared> perl test.pl
Benchmark: timing 100 iterations of moose, nomoose...
     moose: 33.6734 wallclock secs ( 0.00 usr +  0.14 sys =  0.14 CPU) @ 709.22/s (n=100)
            (warning: too few iterations for a reliable count)
   nomoose: 1.25888 wallclock secs ( 0.06 usr +  0.17 sys =  0.23 CPU) @ 427.35/s (n=100)
            (warning: too few iterations for a reliable count)

Yikes, it’s more than 25 times slower! But is it really that bad? It’s adding between 0.3 and 0.35 seconds to the start time. That actually sounds quite reasonable. And when you consider that a typical large perl program will load a whole bunch of other modules which also add to the start time, maybe it will start to get lost in the noise.

Based on this test, rejecting Moose solely on start-up time overhead seems a little unreasonable. (There could be some more overhead if you actually try to use it mind you.)

test2.pl

Just in case I’d used Benchmark incorrectly, I tried something a bit more "traditional" (read hacky).

use strict;
use warnings;

my $script = shift;
my $perl = 'c:/strawberry/perl/bin/perl';

for (1..100) {
    system("$perl $script");
}

$ time perl test2.pl nomoose.pl

real    0m5.784s
user    0m0.015s
sys     0m0.000s

$ time perl test2.pl moose.pl

real    0m38.146s
user    0m0.000s
sys     0m0.046s

Read Full Post »

Does Your Perl Suck if You Don’t Use Moose?

chromatic says that using Moose or a similar abstraction mechanism is an important indicator of Perl ability.

Pshaw.

I’m a 6th level Perlic User. I’m quite capable of constructing my own objects¹ thanks very much.

This is chromatic:

Here’s a quick checklist to help those of you writing Perl to determine if you’re capable of writing Perl well:

• Do you do this?
• Do you do that?
• …
• Do you use Moose or another abstraction mechanism from the CPAN?
• …

Okay you got me. I’m quoting out of context. Here’s chromatic again:

You don’t have to answer all of those questions in the correct way to write good and maintainable Perl, but if you answer most of those questions in the wrong way, of course you’ll write bad code.

I don’t know what these 18 items actually indicate, how well integrated you are with the perlective perhaps or how much you code like chromatic, but for sure only 6 or 7 at best are decent indicators of ability to write Perl well. I answered 9 of them in the wrong way and I’m certainly not rushing to correct the deficiency.

---

1. Having said that, I have used Moose before, and for the right project, I would use it again.

Read Full Post »

Where to Cache Stuff?

A fairly common task I need to do is:

• query a remote service which returns a large amount of data
• extract just the bits I need from that data
• do something with the extracted bits of data

Often the initial query will take a few seconds to run and I’ll be thinking I can’t be bothered to wait for this, why don’t I just cache the data.

If I decide it is worth it, the next question is where and how to cache.

Perl AnyEvent

And what I generally think of first is an AnyEvent-based Proxy Server. Just as quickly, I discard that option as I can’t be bothered to figure out how to ensure the proxy is up when I need it. For example, what happens when the physical host where the proxy is running reboots? What happens if the sysadmin kills my process? Etc.

Storable and Freeze / Thaw

So, next I think: the filesystem is always available (hopefully!), I’ll use that. So then I’m thinking about Cache::File, Storable and Freeze/Thaw.

This often leads to a couple of issues too – where should I store my datafiles. Should I store it somewhere in $HOME and then if multiple people want to cache the data, they are each hitting the remote service, or should I find I have a shared area? Then, should this shared area be on a network drive, or local to the box?

Database

So, finally, I get to thinking about caching the data in the database. I know for sure that will always be available.

Can you guess which option I generally choose? Does anyone else have any thoughts and where and how to cache data?

Read Full Post »

YAML Is Not Dead

Nor is it pining for the fjords.

First sign a technology is dying: supporters start writing articles stating [technology] is not dead.

Ron Savage asserts on my blog:

Whether you like YAML or not, it’s effectively dead and gone. Accept this reality and forget it.

• No supporting argument
• No Link
• No alternative evidence

It’s the first time I’ve heard anyone say this, and normally I’d dismiss a naked assertion like this out of hand. But I’m not too invested in YAML at this point (although getting more invested all the time) and it is probably worth a few minutes digging to avoid several months of time wasted. Whether it is worth the extra minutes to write this blog post is debatable however

So first of all I search at duckduckgo.com for "YAML is dead". None of the first 30 links I glance at are talking about YAML’s moribundity.

Next I look at Stack Overflow. 170 questions tagged YAML vs 3,554 tagged JSON (10,978 tagged xml). But I’m not particularly worried about popularity, otherwise I’d be using Java over Perl right? A lot of the YAML questions are fairly recent and have responses.

That’s enough for me. It has parsers, a perl module and it looks far nicer than JSON. I’m going to stick with it.

Read Full Post »