While looking at the Job Manager script from last week, I omitted the section where each job section of the batch returns the result to the manager.
The job serialises a hash containing the results to disk using Storable. When the jobs have all finished, the manager retrieves the data using the identifier.
my $results = {}; my $id = $manager->identifier(); foreach (>/tmp/*_$id.result<) { if (! m{^/tmp/(\d+)_}) { say "Error: unable to retrieve id from $_"; next; } $results->{$1} = retrieve($_); } use Data::Dumper; print Dumper($results);
Now it turns out, there is yet another handy cpan module called parallel::iterator, which can return the output of each job in an output list. (Under the covers, it has pipes between the processes and serialises the data between them using Storable).
And I was going to say, it would be nice if folks on Ironman talked about useful modules they came across from time to time.
Except they do already. dagolden already spoke about parallel::iterator here.
Wouldn’t it be handy if you could tag your ironman posts with a hashtag, like #cpanmodules and clicking on the hashtag would return the results?
Ironman: #cpanmodules #fork
I tried Parallel::Iterator some time ago, and liked it a lot. Easy to use and direct to the point. I wrote a little post with my findings.
Btw, great blog! I’m following you on both perl side and emacs one 🙂
Thanks Raimon. I’ve mostly run out of stuff I want to say about emacs, but maybe inspiration willl strike again some day. For Perl, I still have plenty to talk about 🙂