Feeds:
Posts
Comments

Archive for March, 2010

Following my earlier tests, I’ve got good hopes for Plack on Windows. To ramp it up from the hello world example I thought I’d try a simple network cache. HTTP isn’t obviously the best protocol for this purpose, but it’s widely used, presumably due to the existence of many robust servers (also maybe because people [including me of course!] can’t be bothered to check for short tcp reads/writes).

To make the example simple, I have (a) elided the actual call to retrieve the data and (b) put all of the code in one file. Obviously I wouldn’t recommend doing this in production but hopefully it will convey the intent.

# —

Most of my perl starts with a preamble that looks somewhat like this.

use 5.010;

use strict;
use warnings;

use POSIX;

get_data represents a call (to a database for example) that will take a while to complete.

sub get_data
{
    my $key = shift;
    sleep 5;
    return "[$key]";
}

get_cached_data is also very much simplified. Normally I’d expect to use something like the Cache::XXX modules.

sub get_cached_data
{
    my $key = shift;
    state %cache;

    if (! exists($cache{$key})) {
        $cache{$key} = get_data($key);
    }

    return $cache{$key};
}

response wraps the PSGI response to make the example a bit cleaner. It obviously wouldn’t be appropriate if other http response types were returned.

sub response
{
    my $data = shift;

    return [
        200,
        ['Content-Type' => 'text/plain'],
        [$data],
    ];
}

Twiggy doesn’t log the information about the request by default in the same way as plackup does. There may be a flag to enable that (should look at the code) but for now I synthesise it from $env.

sub request_info
{
    my $env = shift;
    my $ts = POSIX::strftime('%d/%b/%Y %H:%M:%S', localtime(time()));

    return qq{$env->{REMOTE_ADDR} - - [$ts] }
         . qq{"$env->{REQUEST_METHOD} $env->{PATH_INFO} $env->{SERVER_PROTOCOL}" }
         . qq{"-" "$env->{HTTP_USER_AGENT}"};
}

And there should be nothing surprising in the main.

# -- main

my $app = sub {
    my $env = shift;

    # foreach my $key (keys %$env) {
    #     print "$key $env->{$key}\n";
    # }

    # Twiggy doesn't give the same request info as vanilla plack
    say request_info($env);

    # Could map METHOD/PATH to subroutine calls using a hash
    my $method = $env->{REQUEST_METHOD};
    my $path_info = $env->{PATH_INFO};

    my $start = time();
    my $response = response(get_cached_data($path_info));
    my $time = time() - $start;
    my $s = ($time == 1) ? '' : 's';
    say "Generating response took $time second$s";

    return $response;
};

Testing with firefox is a bit crufty, but I haven’t got around to adapting my test client yet.

$ twiggy --listen :8080 obj-cache.psgi
127.0.0.1 - - [30/Mar/2010 21:24:47] "GET /someurl/ HTTP/1.1" "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 (.NET CLR 3.5.30729)"
Generating response took 5 seconds
127.0.0.1 - - [30/Mar/2010 21:25:02] "GET /someurl/ HTTP/1.1" "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 (.NET CLR 3.5.30729)"
Generating response took 0 seconds # Yes, looks good!

Read Full Post »

Having had a couple of surprising experiences with third-party perl modules using signals on windows, I decided to test them to see how extensive support is. As mst points out, it is good to have confidence in your chosen platform.

Here is my test program.

#!/usr/bin/env perl

use 5.010;

use strict;
use warnings;

say '#', scalar(keys %SIG), ' signals';

my %skip = map { $_, 1 } qw(STOP TSTP TTIN TTOU);

foreach my $sig (sort keys %SIG) {
    next if (exists $skip{$sig});

    my $pid = fork();
    if (! $pid) {
        sleep 1;
        say "$sig failed!";
        exit 0;
    }

    say "Testing $sig (on $pid)";
    kill $sig, $pid;
    my $catch = wait();
    say "($sig) $catch exited...\n";
}

I fork a new process (presumably this is a pseudo-fork piggy-backing on a thread instead of a process in windows, but it is something people might write if they weren’t aiming at portability to windows), and then I try and send it a signal using kill. If the kill fails, then we get a message.

On Linux, this works pretty well as you would expect, signalling both from the parent and from within the child (suiciding?). A few signals unsurprisingly didn’t cause the child to abort: CHLD, CLD, CONT, FPE, NUM32, URG, WINCH and a few needed to be skipped. However, it was consistent from run to run.

The tests on Windows varied from run to run. The first run aborted on the first signal (SIGABRT appropriately enough).

jared@win32 $ ./signal-test.pl
#26 signals
Testing ABRT (on -3628)
Terminating on signal SIGABRT(22)

The next one ran through to SIGFPE.

jared@win32 $ ./tsig2.pl
#26 signals
Testing ABRT (on -268)
ABRT failed!
(ABRT) -268 exited...

Testing ALRM (on -1440)
ALRM failed!
(ALRM) -1440 exited...

Testing BREAK (on -2596)
BREAK failed!
(BREAK) -2596 exited...

Testing CLD (on -3464)
CLD failed!
(CLD) -3464 exited...

Testing CONT (on -2360)
CONT failed!
(CONT) -2360 exited...

Testing FPE (on -3508)
Terminating on signal SIGFPE(8)

And the third ran through to SIGBREAK.

jared@win32 $ ./tsig2.pl
#26 signals
Testing ABRT
ABRT failed!
(ABRT) -3548 exited...

Testing ALRM
ALRM failed!
(ALRM) -2800 exited...

Testing BREAK
Terminating on signal SIGBREAK(21)

*sigh*. Well, that is pretty disappointing. I had a bunch of other signal related tests I was thinking about doing but there seems little point. I always think of Perl as a fairly platform-agnostic wrapper around the underlying OS, but it looks like I was seriously mistaken.

Read Full Post »

I’m a huge fan of code reuse, and I tend to trust other people’s public code more than my own private code. After all, if they put effort into making it public, they must have put a lot of thought into it. And more than likely it is probably their speciality. That’s why I don’t (for example) implement my own webservers :) Having said that, I’ve reimplemented more wheels than I care to admit. Here is me, providing my own version of identity – a built-in emacs function. How embarrassing. Almost put me off blogging that did.

So, I try to do a bit of due dilligence when I’m thinking about writing a simple thing that someone surely has already done. What I want, is an interprocess mutex. There must be a million ways of doing these things, so I have a quick look on CPAN. Searching for mutex brings up LockFile::NetLock which looks interesting (although slightly paranoid, and I don’t have my own ftp server).

IPC::Semaphore looks like it only supports SysV (i.e. not Windows) and the API is somewhat baroque. Win32::Semaphore has the opposite problem (I’m guessing). I did like the look of POSIX::RT::Semaphore but suspected it wouldn’t install on Windows.

dmake.EXE:  Error code 129, while making 'Semaphore.o'
  MJP/POSIX-RT-Semaphore-0.05.tar.gz
  C:\strawberry\c\bin\dmake.EXE -- NOT OK
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible

Yep, I was right. I’ll spare you the results from the other hour I spent googling various things like semaphore and process mutex.

Okay, let’s think about this. Maybe I could write my own simple little thing with a nice API, and if I find a decent module later I can delegate to it, or maybe delegate to Win32::Semaphore on Windows and the SysV version everywhere else. The ACE guys call that a wrapper facade. Does flock work on Windows?

So the plan is to call flock LOCK_EX on a file in the constructor, and then flock LOCK_UN in the destructor in a kinda RAII way. Famous last words, but what else could I possibly need?

package IPC::ProcessMutex;

use Carp;
use Fcntl;

sub import { }

sub new
{
    my $class = shift;
    my $file = shift;

    my $fh;
    if (! sysopen($fh, $file, O_CREAT)) {
        croak "Unable to sysopen $file";
    }

    my $self = { handle => $fh };
    bless $self, $class;

    flock $fh, Fcntl::LOCK_EX;

    return $self;
}

sub DESTROY
{
    my $self = shift;

    if (exists $self->{handle}) {
        flock $self->{handle}, Fcntl::LOCK_UN;
        close $self->{handle};
        delete $self->{handle};
    } else {
        carp 'DESTROY - handle was not defined';
    }
}

1;

And to avoid having to think to hard about whether the code actually works or not, I’ll check it with a little test ;)

use POSIX;
use FindBin;

use lib "$FindBin::Bin/lib/perl5";

use IPC::ProcessMutex;

sub my_log
{
    my $ts = POSIX::strftime('%H:%M:%S', localtime(time()));
    print "[ $ts ] : ", @_, "\n";
}

{
    my_log 'Getting mutex ...';
    my $mutex = IPC::ProcessMutex->new('.filename.lock');
    my_log 'Got mutex.  Sleeping 10 ...';
    sleep 10;
}

my_log 'Released mutex.';

Process 1

jared@win32 $ perl get-mutex.pl
[ 21:37:57 ] : Getting mutex ...
[ 21:37:57 ] : Got mutex.  Sleeping 10 ...
[ 21:38:07 ] : Released mutex.
jared@win32 $ perl get-mutex.pl
[ 21:38:15 ] : Getting mutex ...
[ 21:38:17 ] : Got mutex.  Sleeping 10 ...
[ 21:38:27 ] : Released mutex.
jared@win32 $

Process 2

jared@win32 $ perl get-mutex.pl
[ 21:38:02 ] : Getting mutex ...
[ 21:38:07 ] : Got mutex.  Sleeping 10 ...
[ 21:38:17 ] : Released mutex.
jared@win32 $ perl get-mutex.pl
[ 21:38:24 ] : Getting mutex ...
[ 21:38:27 ] : Got mutex.  Sleeping 10 ...
[ 21:38:37 ] : Released mutex.
jared@win32 $

Yes! Mission accomplished as they say. I’m tentatively going to go with that…

Read Full Post »

Okay, so down to business. One thing I like about straight cgi vs mod_perl is that any changes are immediately reflected on page refresh. And plackup (and twiggy) both offer an option to auto-restart when files change. Sounds good.

jared@win32 $ plackup -h
...
-r, --reload
        Make plackup to watch updates from your development directory and
        restarts the server whenever a file is updated. This option by
        default watches the "lib" directory and the base directory where
        *.psgi* file is located. Use "-R" if you want to watch other
        directories.
...
jared@win32 $ twiggy -r --listen :8080 hello.psgi
Watching ./lib hello.psgi for file updates.
./lib: No such file or directory at c:/strawberry/perl/site/lib/Filesys/Notify/Simple.pm line 156
Terminating on signal SIGINT(2)

Hmmm… I didn’t ask it to watch ./lib (and it doesn’t acknowledge changes to hello.psgi). Let me change the watched files with -R.

jared@win32 $ plackup -r -R hello.psgi --listen :8080 hello.psgi
HTTP::Server::PSGI: Accepting connections at http://0:8080/
Watching hello.psgi ./lib hello.psgi for file updates.
./lib: No such file or directory at c:/strawberry/perl/site/lib/Filesys/Notify/Simple.pm line 156
Terminating on signal SIGINT(2)

Er, okay, -R only allows you to add paths. So there is not an obvious way of removing the paths that already exist. *sigh*. I submit to the inevitable.

jared@win32 $ mkdir lib
jared@win32 $ twiggy -r --listen :8080 hello.psgi
Watching ./lib hello.psgi for file updates.

So I make a change to hello.psgi and get the following:

-- C:\home\jared\plack-tests\hello.psgi updated.
Killing the existing server (pid:-2872)

Almost! But I’m missing the message that says it was able to restart the server.

waitpid($pid, 0);
warn "Successfully killed! Restarting the new server process.\n";

In actual fact, the process it claims it has killed is still running and I’m still able to connect to it. So I hack Restarter.pm a bit (more on that at the end).

jared@win32 $ twiggy -r --listen :8080 hello.psgi
Watching ./lib hello.psgi for file updates.
-- C:\home\jared\plack-tests\hello.psgi updated.
Killing the existing server (pid:-2156)
Successfully killed! Restarting the new server process.
bind: Unknown error at c:/strawberry/perl/site/lib/Twiggy/Server.pm line 71
    -L, --loader

Using plackup instead of twiggy works.

jared@win32 $ plackup -r --listen :8080 hello.psgi
HTTP::Server::PSGI: Accepting connections at http://0:8080/
Watching ./lib hello.psgi for file updates.
-- C:\home\jared\plack-tests\hello.psgi updated.
Killing the existing server (pid:-3544)
Successfully killed! Restarting the new server process.
HTTP::Server::PSGI: Accepting connections at http://0:8080/

Although having thought about it, maybe the Shotgun loader is what I really want.

jared@win32 $ twiggy --listen :8080 hello.psgi -L Shotgun
Attempt to free unreferenced scalar: SV 0x2c78b04,
Perl interpreter: 0x2400054 at
c:/strawberry/perl/site/lib/Plack/Loader/Shotgun.pm line 48.

Again, plackup works here where twiggy does not.

jared@win32 $ plackup --listen :8080 hello.psgi -L Shotgun
HTTP::Server::PSGI: Accepting connections at http://0:8080/
127.0.0.1 - - [22/Mar/2010 21:21:31] "GET / HTTP/1.1" 200 42 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)"
127.0.0.1 - - [22/Mar/2010 21:21:35] "GET /favicon.ico HTTP/1.1" 200 42 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)"
127.0.0.1 - - [22/Mar/2010 21:22:07] "GET / HTTP/1.1" 200 42 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)"

# –

And at the risk of getting pointed comments for using the KILL signal, here is the hack I made to Restarter.pm. (Hey, what can I do? My OS blows.)

--- Plack.orig/Loader/Restarter.pm      2010-03-22 20:50:16 +0000
+++ Plack/Loader/Restarter.pm   2010-03-22 21:31:06 +0000
@@ -37,6 +37,12 @@
     my $pid = $self->{pid} or return;
     warn "Killing the existing server (pid:$pid)\n";
     kill 'TERM' => $pid;
+
+    if (lc($^O) =~ /mswin32/) {
+        sleep 1;
+        kill 'KILL' => $pid;
+    }
+
     waitpid($pid, 0);
     warn "Successfully killed! Restarting the new server process.\n";
 }

Read Full Post »

Now I have my command prompt set up the way I like it, installing modules into strawberry perl from from cpan is easy. Okay, fine, it was easy before, but now I just need to run my batch file and type cpan.

Okay, so one of the things I’m interested in is running Plack on Windows. And Twiggy looks like the obvious choice for HTTP servers supporting PSGI – the benchmarks I looked at state it is the second most performant pure(ish)-perl option after Starman and Starman clearly states it isn’t supported on Windows. Does the fact that Twiggy doesn’t say that mean that it works?

c:\home\jared>cpan

cpan shell -- CPAN exploration and modules installation (v1.9452)
Enter 'h' for help.


cpan> install Plack::Handler::Twiggy

...

t/02_signals.t .......... skipped: Broken perl detected,
skipping tests.
t/03_child.t ............ skipped: Your perl interpreter
is badly BROKEN. Child watchers will not work, ever. Try
upgrading to a newer perl or a working perl (cygwin's perl
is known to work). If that is not an option, you should be
able to use the remaining functionality of AnyEvent, but
child watchers WILL NOT WORK.

...

Yikes! This was emitted when building AnyEvent (a pretty key part of Twiggy). It doesn’t sound good does it? It comes from the following test:

BEGIN {
   # check for broken perls
   if ($^O =~ /mswin32/i) {
      my $ok;
      local $SIG{CHLD} = sub { $ok = 1 };
      kill 'CHLD', 0;

      unless ($ok) {
         print <<EOF;
1..0 # SKIP Your perl interpreter is badly BROKEN. Child watchers will not work, ever. Try upgrading to a newer perl or a working perl (cygwin's perl is known to work). If that is not an option, you should be able to use the remaining functionality of AnyEvent, but child watchers WILL NOT WORK.
EOF
         exit 0;
      }
   }
}

Okay, so child watchers don’t work (whatever they are). Hopefully Twiggy doesn’t need ‘em. Let’s plough on regardless. This is the standard Hello World app.

my $app = sub {
    my $env = shift;
    return [
        200,
        ['Content-Type' => 'text/plain'],
        [ "Hello stranger from $env->{REMOTE_ADDR}!"],
    ];
};

And firing up twiggy works, at least for this simple example.

jared@win32 $ twiggy --listen :8080 hello.psgi

Read Full Post »

I’ve talked a bit previously about alternative windows shells. However, sometimes they are not available and I have no wish to learn powershell so I use the old dos command prompt.

I set up a batch file to configure the path which will be called from Console2. And I use my usual trick of removing unwanted paths and duplicates by calling perl. As dos doesn’t have the nice backtick syntax from the unix shells how do you capture the output from an external command? It turns out you need the dos for command.

c:\home\jared>for /?
Runs a specified command for each file in a set of files.

FOR %variable IN (set) DO command [command-parameters]

...

FOR /F ["options"] %variable IN ('command') DO command [command-parameters]

...

As the perl command uses quotes itself, you need to pass the usebackq option which allows you to use backticks to quote the command and has the added advantage of looking slightly more like unix shell :)

The way that for works is it splits the output based on a delimiter and you have to specify a variable for each result. The default delimiter is space and as many windows paths contain spaces we would have to provide a large number of variables to retrieve all the results. The alternative is to set delims to be blank which allows us to capture all the results in one variable.

@echo off

set PERL=c:\strawberry\perl\bin\perl

for /f "usebackq delims=" %%x in (`%PERL% -e "print join(q{;}, grep { lc($_) !~ /strawberry|msys|mingw|cygwin/ } split /;/, $ENV{PATH})"`) do set PATH=%%x

set STRAWBERRY=c:\strawberry\perl\bin;c:\strawberry\c\bin
set PATH=%STRAWBERRY%;c:\msys\1.0\bin;%PATH%
cd %HOME%

@echo on
%COMSPEC% /k

Read Full Post »

Enablers and Obstructors

A tail about the Windows Unix shells and MSYS path mangling

I am aware of three libre "Unix" shells on Windows, Cygwin (bash), MSYS (bash) and the shell that comes with UnixUtils (zsh). Out of all of these, only the MSYS shell mangles the paths. And sometimes, even though it can be surprising, path mangling is a good thing.

$ perl -e 'print $ARGV[0], "\n"' '/c/xxx:/c/yyy'
c:\xxx;c:\yyy
$ echo '/c/xxx:/c/yyy'
/c/xxx:/c/yyy

Okay, that was a bit unrealistic (although something similar bit me for a couple of hours and indirectly led to this post). Let’s have a look at $PATH.

$ perl -e 'print $ARGV[0], "\n"' "$PATH"
c:\MinGW\bin;c:\strawberry\perl\bin;c:\msys\1.0\bin;c:\Windows\system32;c:\Windows;c:\Windows\System32\Wbem;c:\Program Files\Java\jdk1.6.0_15\bin;c:\Program Files\TortoiseHg;c:\packages\cygwin\bin;c:\packages\lib
$ echo $PATH
/c/MinGW/bin:/c/strawberry/perl/bin:/c/msys/1.0/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Program Files/Java/jdk1.6.0_15/bin:/c/Program Files/TortoiseHg:/c/packages/cygwin/bin:/c/packages/lib

Fun stuff eh?

I believe the heuristic that is used is that native MSYS apps use unix style paths and other apps use windows style paths.

Larry Wall – An Enabler

I really like Larry Wall. He didn’t decide ahead of time how people were going to use Perl. He provided the various features, observed what people were doing with them and then made it easier to do typical tasks. He enabled useage that he hadn’t necessarily foreseen.

http://www.techgnosis.com/wall1.html

[With Perl], I was trying to encourage what I call diagonal thinking. Traditionally computer languages try to be as orthogonal as possible, meaning their features are at all at right angles to each other, metaphorically speaking. The problem with that is that people don’t solve problems that way. If I’m in one corner of a park and the restrooms are in the opposite corner of the park, I don’t walk due east and then due north. I go northeast — unless there’s a pond in the way or something.

I am told that when they built the University of California at Irvine, they did not put in any sidewalks the first year. Next year they came back and looked at where all the cow trails were in the grass and put the sidewalks there. Perl is designed the same way. It’s not just a random collection of features. It’s a bunch of features that look like a decent way to get from here to there. If you look at the diagram of an airline, it’s a network. Perl is a network of features… It’s more like glue than it is like Legos.

Counter example – An Obstructor

Contrast that with Keith talking about a proposed feature for MSYS 1.0.12 in this post. (Emphasis mine)

I agree that there is a need to do something, but if I’m understanding you correctly, you propose implementing some sort of environment variable hook, which will switch path name translation ON or OFF globally? IMO, that simply will not cut the mustard. If path name translation is globally OFF, then MSYS will be emasculated, becoming nothing more than a limited clone of an obsolete version of Cygwin; might just as well ditch it, and use current Cygwin instead.

For me, the principal reason to choose MSYS over Cygwin is the automatic path name translation feature. Take that away, and MSYS is dead; I will use Cygwin instead. (Except, of course, that you make it optional, so I keep the path translation turned ON, and entirely ignore whatever effort you invest in providing the option).

No, we do not need a global method of disabling path name translation

Who is we referring to here? I would find a global method of disabling path name translation useful for two out of the three uses I have for a Unix shell on Windows and I don’t imagine I’m unusual:

  1. An tool for building libraries or executables from a tarball
  2. A superior interactive alternative to cmd.exe.
  3. A superior alternative to cmd.exe for writing batch scripts.

And I’m not thrilled about needing to install cygwin just to get a usable shell. Fortunately, there is an alternative. I can set up zsh from UnxUtils to use the MSYS executables and at a pinch it can provide a reasonable facimile of bash.

This is my .zshrc.

MSYS='c:/msys/1.0'
EMACS='c:/packages/emacs-23'

STRAWBERRY=c:/strawberry/perl/bin
PERL=$STRAWBERRY/perl

PATH="$HOME/bin;$STRAWBERRY;$EMACS/bin;$MSYS/bin;$PATH"

# -- Remove the paths to various perl versions

PATH=`$PERL -e 'print join(";", grep { lc($_) !~ /perl/ }
                      split /;/, $ENV{PATH})'`

# -- Remove the duplicate paths (and ".")

PATH=`$PERL -e '%h = ("" => 1, "." => 1 );
                print join(";", grep { ! $h{$_}++ }
                split /;/, $ENV{PATH})'`

export PATH

The default $TERM is dumb which isn’t very helpful.

TERM=rxvt ; export TERM
EDITOR='emacsclientw' ; export EDITOR

Set tab completion to work in the way I’m used to.

set -o AUTO_LIST

I haven’t managed to get history between sessions working yet. Not sure what the problem with this is …

set -o APPEND_HISTORY
set -o HIST_IGNORE_SPACE
set -o HIST_IGNORE_DUPS

HISTFILE=$HOME/.zsh_history ; export HISTFILE

And this is pretty close to my standard postamble.

CUSTOM=$HOME/.custom
ALIASES=$CUSTOM/.aliases
. $ALIASES

cd $HOME

USER=jared
HOST=`echo $HOST | tr '[:upper:]' '[:lower:]'`

PS1="$USER@$HOST \$ "

One word of warning: although this works fine on Windows XP, on Vista, even with XP compatibility enabled it sometimes crashes with a c:/home/jared/.zshrc: fork failed: no error [48] error. When this happens, sourcing zshrc manually with . ~/.zshrc works.

Read Full Post »

Follow

Get every new post delivered to your Inbox.