Feeds:
Posts
Comments

Archive for the ‘Programming’ Category

Even for a curmudgeon like me who doesn’t make much use of object orientation, Moose offers something which supports my programming with types style: a degree of run-time type checking and the ability to create a range of convenience functions with very little code.

Say I have a person who makes a bunch of orders consisting of a bunch of items. My hashref might look something like this:

my $data = {
    fred => {
        orders => [
            {
            order_id => 'fred1',
            items => [
                { description => 'roses' },
            ],
        },
            {
            order_id => 'fred1',
            items => [
                { description => 'one true ring' },
            ],

        },
        ],
    },
};

Then I’ll probably need a bunch of convenience functions to make sure I’m adding items to orders rather than people.

In Moose, including the type checking, that looks like:

Item

package Item;
use Moose;

has description => (
    is => 'ro',
    isa => 'Str',
);

no Moose;
__PACKAGE__->meta->make_immutable;

Order

package Order;
use Moose;

has order_id => (
    is => 'ro',
    isa => 'Str',
    required => 1,
);

has items => (
    is => 'rw',
    isa => 'ArrayRef[Item]',
    default => sub { [] },
    traits  => ['Array'],
    handles => {
        add_item  => 'push',
        get_items => 'elements',
    },
);

no Moose;
__PACKAGE__->meta->make_immutable;

Person

package Person;
use Moose;

has name => (
    is => 'ro',
    isa => 'Str',
    required => 1,
);

has orders =>
    is => 'rw',
    isa => 'ArrayRef[Order]',
    default => sub { [] },
    traits  => ['Array'],
    handles => {
        add_order  => 'push',
        get_orders => 'elements',
    },


no Moose;
__PACKAGE__->meta->make_immutable;

Adding an item as an order gives a nice error message:

my $fred = Person->new(name => 'fred');
my $item = Item->new(description => 'One true ring');
$fred->add_order($item);
$ perl moose-arrays.pl
A new member value for orders does not pass its type constraint because:
Validation failed for 'Order' with value Item=HASH(0x9ad3ec8)
(not isa Order) at moose-arrays.pl line 63

Moose ensures you pass an order to add_order(…).

my $order1 = Order->new(order_id => 'fred1');
my $item = Item->new(description => 'One true ring');
$order1->add_item($item);
$fred->add_order($order1);

my $order2 = Order->new(order_id => 'fred2');
$fred->add_order($order2);

use Data::Dumper;
print Dumper($fred);
$VAR1 = bless( {
                 'orders' => [
                               bless( {
                                        'order_id' => 'fred1',
                                        'items' => [
                                                     bless( {
                                                              'description' => 'One true ring'
                                                            }, 'Item' )
                                                   ]
                                      }, 'Order' ),
                               bless( {
                                        'order_id' => 'fred2',
                                        'items' => []
                                      }, 'Order' )
                             ],
                 'name' => 'fred'
               }, 'Person' );

I like it when someone else has already written the code I would otherwise need to write myself.

Read Full Post »

Plack Vs FastCGI

I’ve been sold on Plack for a while. This clarifies a part of the reason nicely1.

If you design a fibonacci() function, would you make it print the result to STDOUT, or return the result as a return value?

(It’s referring to the PSGI spec rather than Plack, but I figure the latter needs press a bit more)

Quick thought though – if a page takes a long time to generate, the stdout technique can deliver a bit at a time. Is there a plack plugin for that, or is the answer to go for ajax these day and deliver the page outline asap?


1. Namely that printing to stdout always seemed like a horrible and inefficient hack.

Read Full Post »

One thing I like about developing the emacs environment (i.e. writing emacs lisp) is programming in the image. In contrast when I run a perl script, any data loaded into the script vanishes when the script exits.

my $image = {};
populate_particular_set_of_people($image);
populate_particular_set_of_orders($image);
# ... more populate functions ...
produce_report($image);

If the populate functions take a long time to run and there is a syntax error in one of them, I might waste a lot of time waiting for data to be loaded over and over and potentially annoy my DBA.

Hence my thinking about REPLs earlier in the year.

Instead, I can mitigate this problem and support exploratory programming by serialising my data structure to disk after each section has been loaded.

use Storable;

use constant IMAGE_FILE => '/tmp/.jareds-image-RANDOMTEXT';

sub populate_particular_set_of_people
{
    my $image = shift;

    return if (exists $image->{'store'}{'people'});

    # Get the set of people

    store $image->{'store'}, IMAGE_FILE;
}

sub populate_particular_set_of_orders
{
    my $image = shift;

    return if (exists $image->{'store'}{'orders'});

    # Get the set of orders

    store $image->{'store'}, IMAGE_FILE;
}

my $image = {};
$image = retrieve(IMAGE_FILE) if -f IMAGE_FILE;

populate_particular_set_of_people($image);
populate_particular_set_of_orders($image);
# more populate functions
produce_report($image);

Error handling has been elided.

I’m only storing part of the hashref, $image->{'store'} in this example. This is fairly typical of my standard use case for this technique – quick one off reports. Some of the data is slow to load in from the database so I persist it to disk. The rest is calculated based on the loaded data so I don’t persist it. I keep all of the data together so I can pass it to each subroutine as a single parameter.

Of course, I wouldn’t recommend this technique for a production system.

Read Full Post »

Neat solution for deploying/running Perl Apps on Windows:

… [share] your production Perl installation, with all the packages required by all your apps correctly installed. And place the top-level .pl scripts (or some placeholder .cmd files if some local setup is required), within that installation. Then your users can run those applications from that shared installation.

Read Full Post »

Happy Perl Devs

I’m kinda amused to see this old post suggesting that perl developers are happier than other developers. I can well believe it. I’ve always been happy playing with perl than the other languages I know, although I’m not quite convinced by the method.

Just a thought though – personally, I’m happier on smaller code bases than larger code bases. Could it be that perl sees more use for moderately sized systems and other languages are used to create developer depressing byzantine balls of mud?

Read Full Post »

I recommend Dave Rolsky

I can’t believe Dave is still on the job market.

I don’t know him personally, but I know him by his posts and his code and I can recommend him without hesitation. The guy is one of the great thinkers of the Perl Community.

If I had hiring authority, I’d get him for my firm.

Read Full Post »

Forked Processes and Pipes

Last time, I linked to some example code that forks a bunch of processes and communicates with them via pipes. This is the main feature of the code I’m interested in, but the explanation is the article is kinda sparse so you can consider this to be supplemental.

As usual, the perl doco is pretty good for covering this stuff.

Creating a child process (a kid) involves two pipes, one for the parent to send data to the kid, and one for the kid to send data back to the parent.

One probably obvious thing to note, you can’t directly send a reference down a pipe, (well, not in any reasonable way and that’s a feature, not a bug), so you’ll be interested in serialisation modules. I’ve mentioned them in passing before and I generally use JSON::XS these days.

Another hopefully obvious thing is if the writer is buffered and the reader is waiting for something specific, there will probably be some deadlock in your future. Remember to unbuffer the writer.

I made a couple more J – comments inline:

sub create_kid {
    my $to_kid = IO::Pipe->new;
    my $from_kid = IO::Pipe->new;

    # J - Fork returns undef for failure, 0 for the child, and the
    # J - child PID for the parent

    # J - Handle fork error
    defined (my $kid = fork) or return; # if can't fork, try to make do

    unless ($kid) { # I'm the kid
      # J - The kid reads from $to_kid and writes to $from_kid
      $to_kid->reader;
      $from_kid->writer;

      # J - unbuffer writing to the pipes.  Otherwise may deadlock
      $from_kid->autoflush(1);

      # J - Reset all of the signal handling
      $SIG{$_} = 'DEFAULT' for grep !/^--/, keys %SIG; # very important!
      do_kid($to_kid, $from_kid);
      exit 0; # should not be reached
    }

    # J - parent here...
    # J - The parent reads from $from_kid and writes to $to_kid
    $from_kid->reader;
    $to_kid->writer;

    # J - unbuffer writing to the pipes.  Otherwise may deadlock
    $to_kid->autoflush(1);

    $kids{$kid} = [$to_kid, $from_kid];
    $kid;
}

Read Full Post »

Parallel Tasks using Fork

Randal Schwartz wrote an example link checker which used forked processes to run tasks in parallel. Each child process created has a read pipe from and a write pipe to the parent (created with IO::Pipe).

The result is an inverted version of my preferred architecture. I like the parent to dump work on a queue and whichever child is ready to pull it off. This is pretty easy to do with threads.

In Randal’s version, the parent figures out which child is available to do work.

Read Full Post »

My question about how windows works without fork got pointers to two useful Windows perl modules in the comments:

Spawning External Programs

(from dagolden) See Win32::Job or just use system(1, $cmd, @args)

Creating Daemons

(from Christian Walde) Use Win32::Detached


What I was really wondering about was how reliable software was written without fork. What do I mean by that?

What is the difference between?

parent
fork()
do_something_independent()
wait_for_childs_response()

and

parent
thread()
do_something_independent()
wait_for_childs_response()

If the child thread misbehaves – exits, dumps core, leaks memory, or whatever, that’s bad for the parent. Child processes can’t wreak quite so much havoc.

Presumably the solution adopted in Window is don’t do stuff in threads that causes them to accidentally exit [the whole process], dump core or leak memory.

Read Full Post »

How does windows work without fork? So many things start with a call to fork: starting a new process (fork/exec), creating a daemon, creating independent tasks within one program…

The nice thing about all of these things is, if a new process goes bad, it probably won’t take out the parent process. You don’t get the same safety with a thread. Therefore, fork is the basis of a lot of my reliable software.

I’m fairly confident that windows doesn’t have anything similar to fork. Otherwise the perl fork emulation could simply wrap whatever windows has. Sadly it doesn’t quite work that nicely.

While the emulation is designed to be as compatible as possible with the real fork() at the level of the Perl program, there are certain important differences that stem from the fact that all the pseudo child "processes" created this way live in the same real process as far as the operating system is concerned.

If the parent process is killed (either using Perl’s kill() builtin, or using some external means) all the pseudo-processes are killed as well, and the whole process exits.

I have been approaching this problem of reliable windows software from too low a level. Rather than copying what I do on Unix and start with fork (using Perl more or less as my OS wrapper), I now use web servers to manage my processes.

Read Full Post »

« Newer Posts - Older Posts »

Follow

Get every new post delivered to your Inbox.