Feeds:
Posts
Comments

Remote Killing from Emacs

Previously, I’ve talked about integrating little perl "services" into emacs that read from stdin and write to stdout.

Sometimes the processes that make up your emacs application are running on different servers, picking up various pieces of information and it might be necessary to kill them independently or in groups. By default, comint binds C-c C-c to SIGINT, but if you have a moderately complex application, you don’t want to go through each comint buffer and kill the processes individually.

Each of your perl scripts needs to output the PID and the server it is running on.

use Sys::Hostname;
my $hostname = hostname();

print "Started PID $$ on $hostname\n";

Then the emacs application code needs to use a filter to read that information into a couple of variables.

(defvar proc-pid nil)
(defvar proc-host nil)

(defconst *re-match-pid-host*
  "Started PID \\([0-9]+\\) on \\([a-z0-9]+\\)")

(defun proc-filter (output)
  (when (string-match *re-match-pid-host* output)
    (setq proc-pid (match-string 1 output))
    (setq proc-host (match-string 2 output)))
  output)

(add-hook 'comint-preoutput-filter-functions 'proc-filter)

Running commands remotely is easy with ssh. (Incidentally, how does this work in windows-land?)

(defconst *ssh* "ssh -o StrictHostKeyChecking=no")

(defun kill-proc ()
  (interactive)
  (let ((cmd (format "%s %s kill %s" *ssh* proc-host proc-pid)))
    (message cmd)
    (message (shell-command-to-string cmd))))

I’ve demonstrated the make-comint code previously.

(defun start-proc ()
  (let ((buffer (get-buffer "*stdin-proc*")))
    (when buffer (kill-buffer "*stdin-proc*")))
  (apply 'make-comint "stdin-proc" "perl" nil
         (list "./test-stdin.pl")))

And the result is as expected.

Started PID 5008 on saturn

Process stdin-proc terminated

Relative Popularity

It’s hard to objectively measure Perl’s popularity compared to Python or Ruby. Even before nerds-central used job availability as a measurement, the ease of finding a job was a key requirement for me.

Who cares if I love Ruby if I can’t use it for the 10 hours a day I’m working?

Anecdotal evidence

In the last twelve years I’ve had no difficulty finding plenty of Perl work. In the past couple of years, I’ve had a few calls about Python jobs, but probably ten times fewer than the number of Perl calls I get. Hold the front page – Perl is ten times more popular than Python.

Well, no. Of course I get a lot more Perl calls. I’m a Perl programmer, and my résumé mentions Perl all over the place. Anyone calling about Python must really be desperate!

But I notice that a lot of Python programmers working at Python-centric firms apply similar logic. All the folks they work with do Python, clearly no-one does Perl anymore. Thank goodness – Python is finally the winner it always deserved to be.

Why do people care?

Network effects are important. All other things being equal, a language with more users will get more libraries, will have more companies using it, will get more users, will get more libraries, etc. vs a language with fewer users. And, popularity aside, none of the big three scripting languages has a huge advantage over the others.

Except of course that Perl was the first mover.

So, what can you do if you don’t have a real advantage you can point to? That’s right – you trash-talk the leader, hope you can convince enough people there is a real problem and when they jump ship hopefully you’ll be the one left with the positive feedback loop.

Michael Murdock doesn’t believe in anti-Perlism. Of course he has to indulge in some of his own.

“Every developer with whom I have discussed this issue says roughly the same thing: It’s very difficult to pick up a piece of Perl code, even when it’s your own, and be able to quickly understand the gist of it.”

I find it hard to understand French but I don’t go round saying it is a speak-only language.

You couldn’t make it up. Or was the irony intentional?

Replacing sed

I’ve spoken about processing logfiles with perl previously. Occasionally though, I still reach for sed.

Say I have a logfile that looks like this:

[ <timestamp> ] : somefunc()
[ <timestamp> ] : interesting line 1
[ <timestamp> ] : interesting line 2
... 1000s of lines
[ <timestamp> ] : somefunc()
[ <timestamp> ] : interesting line 1
[ <timestamp> ] : interesting line 2
... 1000s of lines

Picking out lines following a pattern is easy with sed – p prints the current match and n takes the next line.

$ < log.txt sed -n '/somefunc()/ {p;n;p;n;p}'
[ <timestamp> ] : somefunc()
[ <timestamp> ] : interesting line 1
[ <timestamp> ] : interesting line 2
[ <timestamp> ] : somefunc()
[ <timestamp> ] : interesting line 1
[ <timestamp> ] : interesting line 2

My first attempt to replace that with perl looks a bit ugly

< log.txt \
perl -ne 'if (/somefunc\(\)/) {print; print scalar(<>) for (1..2)}'

I’m not that happy with the module I came up with to hide the messiness either.

package Logfiles;

require Exporter;
our @ISA = qw(Exporter);
our @EXPORT_OK = qw(process);

use Carp;

sub process (&$;$)
{
    my ($sub, $regex, $lines) = @_;
    $lines ||= 0;
    return unless /$regex/;
    if (! $lines) {
        $sub->($_);
    } else {
        croak "process() arg 3 not ref ARRAY" unless ref($lines) eq 'ARRAY';
        my $line = 0;
        my @lines = @$lines;
        while (1) {
            if ($lines[0] == $line) {
                $sub->($_);
                shift @lines;
            }
            ++$line;
            last if ($line > $lines[0] or (not @lines));
            $_ = <>;
        }
    }
}

1;

But at least my typical spelunking looks a little cleaner now.

< log.txt perl -MLogfiles=process -ne \
    'process { print } qr/somefunc\(\)/, [0..2]'

Any suggestions on how to improve this (without reverting to sed)?

Moose Types

Rants about generalisations notwithstanding, I’m a fan of typeful programming (I’m sure I’d love Ada). For a script that will be moderately complex, I like to sit down and think about the types I’m going to use before I start.

Any library that will enable me to specify my types more precisely and concisely is obviously a win.

And speaking of Moose…

Moose has a bunch of methods to specify your types and a built-in type hierarchy available that you can build off.

Abusing an example I’ve used before:

use Moose;
use Moose::Util::TypeConstraints;
use MooseX::Params::Validate;

subtype 'LegalDrinkingAge'
    => as 'Int'
    => where { $_ >= 18 }
;

coerce 'LegalDrinkingAge'
    => from 'Int'
    => via { 1 }
;

sub can_legally_drink
{
    my $age = pos_validated_list(
        \@_,
        { isa => 'LegalDrinkingAge' },
    );

    return 1;
}

print can_legally_drink(18), "\n";
print can_legally_drink(17), "\n";

Checking for a LegalDrinkingAge type here is obviously the wrong thing to do, but for the purposes of the example it will do.

The resulting error is fine, if a little ugly.

$ perl5.10.1 moose-types.pl
1
Parameter #1 ("17") to main::can_legally_drink did not pass the 'checking type constraint for LegalDrinkingAge' callback
 at /u/packages/lib/perl5/site_perl/5.10.1/MooseX/Params/Validate.pm line 168
        MooseX::Params::Validate::pos_validated_list('ARRAY(0x976ce40)', 'HASH(0x911b6f8)') called at moose-types.pl line 24
        main::can_legally_drink(17) called at moose-types.pl line 33

Zawinski not withstanding, regular expressions are a hugely useful addition to a programmer’s toolbox. I learnt about them from Jeffrey Friedl’s excellent Mastering Regular Expressions1.

Why am I thinking about this now?

So, I was asked to review a perl script and it was pretty good, apart from the fact that the main regex2 didn’t quite match all the expected patterns.

The line to match was:

<some_tag>quickbrownfox</some_tag>

Other valid values within the tags included quick,brown,fox or quick brown fox.

And the regex in the script was:

m{\<some_tag\>(\w+)\<\/some_tag\>}

As you can see, there were also a few unnecessary backslashes.

I suggested [^<]+ instead of \w+ with my reasons.

The developer changed the regex to:

m{\<some_tag\>\s*(\w+)\s*\<\/some_tag\>}

Okay slightly better, but it still didn’t match all the possibilities.

I pointed out a valid pattern it didn’t match and asked again for [^<]+.

This was the result:

m{\<some_tag\>(\s*[^<]+\s*)\<\/some_tag\>}

Okay, fine. The \s space matchers are redundant, but at least it covers everything.

So basically, if you find yourself writing a bunch of regex regularly, and you don’t really understand it, you could do worse than read Friedl’s book.


1. Not an affiliate link

2. I know… "parsing" xml with regex will unleash cuthulu or something. It’s not my script.

Living Without Emacs

I have integrated emacs into most parts of my daily workflow. In a typical day I might need to:

Some of these I have aliases for in my shell already.

A number of database GUIs have template functionality built-in or I could probably substitute the sql templates with scripts if I was sufficiently creative with the command line args.

And if I kept my TODO/Notes list in a wiki, it might be generally useful to the rest of my team.

Yes, thinking about it, I could live without Emacs. But life would be much less pleasant.

Follow

Get every new post delivered to your Inbox.