Archive for May, 2010

I finally got around to trying out the Perl ArcV2 library that
I mentioned earlier. It didn’t go well.

There were some issues early on that were due to me not having the sasl libraries installed on my Ubuntu. After rectifying that it made it past the pre-requisites.

However, the Arc tests did not work.

Running make test
PERL_DL_NONLAZY=1 /u/jared/packages/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/arc1.t .. 1/17 shutdown() on closed socket GEN0 at /u/packages/lib/perl5/5.10.1/i686-linux-thread-multi/IO/Socket.pm line 295.
t/arc1.t .. 2/17
#   Failed test at t/arc1.t line 61.
#   Failed test at t/arc1.t line 63.

Test Summary Report
t/arc1.t (Wstat: 3328 Tests: 17 Failed: 13)
  Failed tests:  3-15
  Non-zero exit status: 13

I glanced at the tests to see what had managed to succeed. It doesn’t look good. Pretty much only ok(1) commands with no predicate passed.

Having said that, the test script looks interesting. It forks a process to fire up the server before running the tests. I’m reminded again how much I miss stuff like fork() when doing Windows development.

use Test::More tests => 17;
use strict;

# To work as a client/server we have to fork

my $pid = fork();

my $user = "mannfred";
my $pass = "klaus";

if ($pid == 0) { # Child
    use Arc::Server;
    use Arc::Connection::Server;
    my $server = new Arc::Server(...);
    exit 0;
} elsif ($pid) { # Parent
    use Arc::Connection::Client;
    ok(1); #1
    sleep(3); # Wait for the server to get ready
    my $client = new Arc::Connection::Client(...) or ok(0);
    ok(1); #2
    my $s;
    if ($client->StartSession()) { ok(1); } else { ok(0); }
    # ...

I then split out the client and server so I could run them individually.

$ perl -Mblib=../blib test-client.pl
[err]: (client) Evaluation of command _RAUTHTYPE failed
(Can't locate auto/Authen/SASL/Cyrus/need_step.al in @INC (@INC contains: ../blib/arch ../blib/lib
/u/packages/lib/perl5/5.10.1/i686-linux-thread-multi /u/packages/lib/perl5/5.10.1
/u/packages/lib/perl5/site_perl/5.10.1/i686-linux-thread-multi /u/packages/lib/perl5/site_perl/5.10.1 .)
at ../blib/lib/Arc/Connection/Client.pm line 157).

The server output looked like this.

$ perl -Mblib=../blib test-server.pl
[info]: (server) Arc v2.1 Session recognized.
[err]: (server) Connection closed by foreign host.

Okay, so it looks like the code has bit-rotted away (the last release was in 2005). This is the line in Arc::Connection::Client that is failing. need_step() is no longer mentioned in the SASL documentation.

if ($sasl->need_step || $sasl->code == 0) {

The question is, how much I would have to change to get this to work. Or to put another way, is what I have already with my AnyEvent based code closer to what I need than the currently broken Arc.

And if it doesn’t even work on Linux, what chance do I have on Windows.


Read Full Post »

or How do you make perl read input that is unbuffered?

Take a look at this. I’m simulating a process streams unbuffered output output irregularly. First it sends "hello ", then waits 5 seconds and sends "world".

I run the producer on its own, just in case it is buffering the output despite the fact I told it not to by setting $|. Nope, it isn’t. On the command line, I can see the hello output and then a few seconds later the world.

The consumer outputs whatever it receives. I get ctime() from Time::localtime for brevity.

$ date ; perl -e '$|=1; for (1..2) { print "hello "; sleep 5; print
"world\n" }' | perl -MTime::localtime -ne 'print ctime(), " : [$_]\n"'
Thu May 13 22:21:23 GMTDT 2010
Thu May 13 22:21:27 2010 : [hello world
Thu May 13 22:21:32 2010 : [hello world

No dice. The input is buffered so the consumer waits until it has received a whole line before it processes it.

Now, setting $| only affects the output, it doesn’t unbuffer the input. How about fiddling with $/ – the record separator?

$ date ; perl -e '$|=1; for (1..2) { print "hello "; sleep 5; print
"world\n" }' | perl -MTime::localtime -e '$/ = " "; while (<>) { print ctime(), " : [$_]\n" }'
Thu May 13 22:35:04 GMTDT 2010
Thu May 13 22:35:04 2010 : [hello ]
Thu May 13 22:35:09 2010 : [world
hello ]
Thu May 13 22:35:14 2010 : [world

Okay, quite cool, but there is a better solution. sysread is unbuffered. You will probably have to buffer the input yourself to ensure you have enough to process, but at least you get it as soon as it arrives. (Mental note: I should check if AnyEvent has something cool for this for anything more than a quick one-off script).

use Time::localtime;

use strict;
use warnings;

my ($input, $buffer);

while (sysread(STDIN, $input, 1024) > 0) {
    print ctime(), " : Read [$input]\n";
    $buffer .= $input;
    # Process $buffer here ...

And the test:

$ date ; perl -e '$|=1; for (1..2) { print "hello "; sleep 5; print
"world\n" }' | perl unbuffered.pl
Thu May 13 22:46:37 GMTDT 2010
Thu May 13 22:46:36 2010 : Read [hello ]
Thu May 13 22:46:41 2010 : Read [world
Thu May 13 22:46:41 2010 : Read [hello ]
Thu May 13 22:46:46 2010 : Read [world

Read Full Post »

Removed from Planet Emacsen

Now that I write mainly about p.erl I had considered not publishing this post, but I’m curious to know the reason if any and email to the planet emacsen maintainer, Edward O’Connor, has gone unanswered.

A couple of years ago when I started blogging about emacs I asked to be included in planet emacsen. And now I have been removed. What can be the reason for that I wonder.

Well, I have just come back from a blogging break. Between my post on sorting records with emacs in August 2009 and my post on Enablers and Obstructors in March 2010, there were almost 7 months. But the latest post in Amortized Analysis (a blog that hasn’t been removed from planet emacsen) was 4th January 2008. That is more than 2 years ago.

Maybe I’m not that relevant anymore. I have posted mainly P.erl posts since my break. But I do still do emacs stuff, e.g. Directory Aliases Revisited and Programming with Types. And Alexandre Vassalotti’s most recent Emacs post was September 2007.

Or was it the back and forth with Ian Eure (and to a lesser extent, Aaron Hawley) about my Emacs Database Mode. Did someone ask for me to be removed?

Anyway, whatever the reason, it has been done. What solution do I have? I have created a yahoo pipe called Better Planet Emacs which you can subscribe to. It filters a lot of the non-emacs related posts out so for me at least it is better than stock planet emacs. If you want to copy it and customise it for your own use you should go here.

Read Full Post »

Emacs Has QWAN

The Preamble

I enjoy programming in Perl. There, I said it. And, although a lot of the stuff I write could easily be written in Emacs Lisp, I generally reach for Perl to solve most of my programming problems. However, if I had to choose between Perl and Emacs disappearing, I’d choose Perl any day of the week and twice on Sunday.

The reason is that there isn’t any competition in the emacs space1. Vim, Visual Studio and Eclipse are not acceptable substitutes. But if I want a dynamically typed langugage with good syntactical support for arrays, hashes and regular expressions and loads of libraries I know where to find Ruby or Python.

Q. Why not switch to Ruby today?
A. I enjoy programming in Perl

What makes Emacs so great?

(or what would I need to switch from Emacs to another editor)

Many people put it as Emacs has QWAN (the quality without a name), but that doesn’t really help to quantify things. Saying that all of the features work very nicely together is true, but also not that helpful. Let me try and point out a few things.

(If the c2 link above isn’t working, try the Bing cache).

Keystrokes are bound to functions

…and it is all very discoverable. If I want to know what a given keystroke does, I just press C-h k <key> and it tells me the function that calls. For example, arrow key calls forward-char. Now, say I want to make a function that needs to move forward a number of characters, I know what the function is called.

The utility of this discoverability in making composite functions cannot be overstated. You might think that macros are equally good, but what if I want a function that persists between sessions. For my line duplication function I glued together functions found with C-h k.

Okay, new utilities, great but not that impressive.

What about fixing stuff with the editor that you just don’t like. For example, recently I had a problem where <page down> went to the beginning of the line rather than remembering the current column position. You might be thinking a proper editor wouldn’t do that, or it isn’t really a problem, but you can’t imagine how annoying that was after the first ten times.

Anyway, what I did was… I fixed it. With C-h k I found that the command was scroll-up and I wrote a wrapper function that remembered the current column (alternatively I could have used advice). Saved my blood pressure and took around 5 minutes.

To summarise:

  • Key strokes / key chords call functions (which can be user-defined)
  • New functions can be created within the editor at any time
  • any key / key chord can call any function
  • information about the function can be viewed

Okay, you’re not convinced. Vim can do that, albeit with a much less pleasant language (or with perl, ruby, python, etc. if you’ve compiled a fat vim)

Editor Tabs

Tabs are not the right mechanism to deal with hundreds of open files. Emacs (together with ido) works beautifully. Generally I’ll have a couple of hundred files open with different file types and I’ll use progressive filtering to first find the perl files and then narrow down the file I wanted. Very quick and very nice. I’ve spent hours scrolling through tabs in Visual Studio and it was not fun.


So all your keys are bound to various useful commands. M-e steps forward to the next sentence nicely. But then you’re editing lisp (or perl) and sentence doesn’t mean much. What does emacs do?

In this case it has a keymap hierarchy. The global keymap defines a bunch of keys which can be overridden by keys defined in the major mode keymap. These can be added to with keys defined in one or more minor mode keymaps.


Comint is an awesome bit of kit. Out of the box, Emacs provides fantastic support for interacting with asynchronous processes. Comint codifies a lot of the functionality you would want to make a decent REPL for an external interpreter. Because this is one of the more tricky interactions to manage, it is an excellent API for dealing with any external process.

A while ago I wrote an article about a comint based stock price subscriber which demonstrates a few comint features.

Take a look at the interaction with gdb following M-x gdb-many-windows. It looks like an IDE no?

Emacs Applications

Some people have developed some impressive applications in emacs.

  1. Org Mode
  2. Emacs Muse
  3. SLIME (A full-featured Lisp IDE)

And my own Emacs Database Mode

Which in turn is based on the excellent sql-mode.

1. Apart from XEmacs of course.

Read Full Post »

Ask and you shall receive it seems. The commenters have arrived with some great comments. And (possibly this is connected), yesterday was my first 200+ day since my 7 month hiatus. I even got some referrals from planet perl due to Gabor mentioning my post on his [aggregated blog].

Foo pointed out that I haven’t got my tone quite right for irony. Any suggestions how to do that better without being too obvious?

And in another comment, Gabor has given me an idea for my next post – which Emacs features I’d need to switch to Padre.

Overall, I’m very pleased.

Read Full Post »

Hang on, that should be Vim vs Emacs right? No.

I have been blogging for the Perl community I have been writing Perl blog posts for around 6 weeks now. Prior to that, I wrote mostly Emacs related blog posts for around 18 months. My vast experience puts me in a great position to compare them.

Emacs blogging

Back when I wrote about Emacs I averaged a couple of posts a week and around 200 visitors a day. Better still, I got several comments for each post, pointing out things I had got wrong, things I could improve or alternative techniques.

200 visitors isn’t many, but it was plenty for me. I was paid for my blogging effort in comment currency.

I suspect the reason I got the comments was that I never had much competition in the emacs hints and tips space. The main ones were:

Of these, only emacs-fu kept grinding out the handy tips, week in week out.

Interestingly, emacsblog reports 3379 readers by feedburner which indicates a decent level of interest in emacs hints and tips.

There was only one real place to pick up emacs news – Planet Emacsen. My posts would hang around for a week or more and I would pick up pretty much all the readers who were interested as well as probably quite a few who weren’t.

Perl blogging

With my Perl blog posts, I struggle to keep 100 visitors a day and I need to post every other day to get that many. Why the difference? I suspect it is a combination of things.

  • If you want to read about perl, you can read the gurus in the community – the Miyagawas, the Tim Bunces, the Curtis Jewels. And there’s hundreds more perl gurus blogging. Okay, maybe 10 more.
  • There are multiple perl news sources. I listed a few here.
  • If you want to improve your perl-fu there is IRC or Perl Monger groups. Emacs in contrast had fewer options.
  • There are more perl bloggers, so your post disappears off the Ironman aggregation pretty quickly.
  • Six weeks may not be enough to build up a following.
  • There are more areas of perl to be interested in. Maybe no-one else wants to know about writing AnyEvent TCP servers.

It could also be that the quality of my writing is poor. But one of you guys would tell me, right?

Why does this matter?

Blogging has to have some value to me, otherwise I might as well watch TV. I do get something out of it even if there are no readers – I’m able to find my fantastic code snippets as long as I have access to the internet. But the lack of input means this doesn’t really offset the effort to write a story around each post. And these days with github et al there are easier and better ways to get your code out there.

Okay, no worries, you’re thinking, if Jared stops blogging about perl (not that I’m thinking of doing so) no-one loses anything. But maybe my experiences is why the level of blogging activity is low compared to the relative size of the community.

And maybe it just doesn’t matter.

Read Full Post »


So I’m perhaps 1% of the way to a poorly thought out middleware component like CORBA1. No, it’s more light-weight, maybe a messaging layer, sorry I mean wire-level protocol specification implementation such as AMQP.

And then I think (like hundreds have probably thought before me), you know, this would be more useful if it had authentication. After all, I don’t want just anyone to be able to send kill signals to any processes. That would be like everyone being root. Which terrifies me.

Don’t invent your own authentication mechanism

And the golden rule about authentication, as far as I can work out, is don’t invent your own authentication mechanism. You’ll get it wrong and leave gaping vulnerabilities for the bad guys to have their wicked way with you. That is, if anyone besides you ever uses your code. And besides, I don’t want to waste any of my 1500 lines on coming up with Yet Another Broken Authentication System.

A quick trip to CPAN

Then I’m looking through the Authen::XXX modules on CPAN and none of them behave in exactly the way I want. But somehow I find a perl server that includes authentication and perhaps does everything I want and I should definitely put it on my list of things to look into even though I’m having a lot of fun with AnyEvent right now.

But by the time I come to look again, I can’t find it. And I’ve complained about documentation before, but Emacs really does deserve it, and I know of no system or language that is better documented than Perl. But I guess the classification problem is a bit tricky to overcome.


Anyway, long story short, I found it.

authenticated perl server -http

An Authenticating Perl Server

The first link (warning PDF) is a paper about using Authen::SASL in client/servers and it mentions ARCv2 which sounds like what I’m looking for.

The first thing to do is find out if it does what I want. The second is to check if it works on Windows.

1. Ambiguity left in deliberately

Read Full Post »

« Newer Posts - Older Posts »