Archive for May, 2010

Super Programming Languages

Jonathan Edwards, in The Myth of the Super Programming Language makes the extraordinary claim that language choice makes little difference.

There is no reason to believe that language choice makes a big difference. People have been doing studies for decades, and there are no reproducible strongly positive results. The burden of proof is on the poster, who is making a surprising claim.

To be fair, I believe he actually intended to single out Haskell and Lisp and state that it is impossible to get an order of magnitude increase in effectiveness. I’m not sure he is right there either, but I’ll dispute the actual claim he made.

Proof By Obviousness

First of all, there are studies that show that different languages required a different amount of code to implement a certain amount of functionality. Take a look at the language / function point here. Smalltalk (28 lines/fp) is more than 5 times better than C (148 lines/fp) in this metric. And comments about APL not-withstanding, code that you don’t write takes the least time.

Second of all, it is hardly a surprising claim. In the leap from Assembler to C, I’m obviously going to save programming time by not having to write code to push my parameters on to the stack every time I want a function.

Moving from C to Java I don’t have to write [so much] memory management code. Java to Perl, I don’t have to specify my data structures ahead of time.

Each language change, there is code that I just don’t have to write any more1. How can that not make a difference?

I’m not smart enough to get Haskell. If I had to guess, I’d say that Jonathan isn’t either, and is mildly upset that folks can out-perform him by choosing these languages.

Perl is the most super programming language I know.

And just to demonstrate the amount of code I need to specify a simple data structure that I get for free in perl, here is some C++.

struct MyData {
    vector<string> list;
    map<string, string> dict;

MyData data;
if (data.dict.find("a") == data.dict.end()) {
    data.dict.insert(make_pair("a", "b"));
// data.list.push_back(1.5);

The perl is clearly terser. Is it less understandable (assuming you know both languages)? No way – it’s clearer.

And I know I’ve been kinda mean with the heterogenous list.

my %data;
$data{dict}->{a} = 'b' unless exists $data{a};
push @{$data{list}}, 'text';
push @{$data{list}}, 1.5; #  :)

Choice Of Language Isn’t The Most Important Factor

As long as you don’t choose something crazy, like Intercal, or Assembler.

The important thing is that there are libraries that solve the majority of your problem. As many people have pointed out, a lot of development these days is gluing together libraries. For me, as much as 95% of the systems I write are third-party libraries.

And my favourite example of something I wrote that would have been much longer in either Perl or C++ is my Emacs Database Mode. Emacs Lisp isn’t a great language, although it is good enough, but for certain problems it has some amazing libraries.

1. Of course, there are trade-offs.

Read Full Post »

$ date
Sat May 29 18:16:56 BST 2010
$ perl -MPOSIX -le 'print "Week ",
>                         POSIX::strftime("%V", gmtime(time))'
Week 21

My Perl Posts

I have covered a number of topics in the four weeks since my last summary.

Hopefully there was something for everybody :)

No Perl5 on the JVM

An interesting one for me: I’ve often wondered why there was only a single implementation of perl5. Reddit linked to a post from August 2009 that more or less answered that question – porting the parser is too much effort for most.

Because of the misleading reddit headline, chromatic followed up and linked to an earlier post he made discussing methods for parsing non-pathological perl examples. He pointed out that string eval and source filters further complicate things.


Is Perl really available on Android?

And, I’m not the only person looking for a perl alternative to Capistrano

Read Full Post »

Until recently, Python and Ruby were ahead of Perl when it came to developing web applications. Now, thanks to the work of the plackists, we’re right back up there again. By the time I could have become proficient Python + WSGI, Plack and PSGI will be equally good, or better.

So, yeah, when it comes to libraries, we are the champions. Okay, there might be a couple of utilities like Rake and Capistrano that haven’t been duplicated yet, but I’m sure someone is working on it1.

However, there is a reason why you might want to use Python or Ruby ahead of Perl. And that is Google App Engine and heroku. If you have an idea for a web app, you can get started for $0. I know about perl-appengine but it seems to be moribund.

With Perl, if shared hosting is enough for you, a half-decent plan costs $6/month which is $72/year. More likely, you would want a VPS so you can install Plack and Starman which will cost closer to $20 – $25 / month or $240 – $300 / year.

Okay, so it isn’t a reason why I personally would use Python or Ruby over Perl – my free time is somewhat valuable to me. But if you’re getting started and you’re not sure which of the big three scripting languages to choose, it is a strike against Perl.

1. And if they’re not, I don’t need ‘em anyway

Read Full Post »

Just as Steve Knight, I don’t find much use for object oriented programming in my own code. But there is a name for methods (or functions or subroutines) that drag a bit of state around and that is Object Based Programming. And it is really handy.

What does it get you?

If you have a few variables that need to be operated on together in multiple methods, you can stuff them in the same object.

sub new
    my ($class, $dbh, $person) = @_;
    my $self = { dbh => $dbh, person => $person, }
    bless $self, $class;
    return $class;

You can use accessor methods to do lazy initialisation.

sub person
    my $self = shift;
    if (! exists $self->{person}) {
        $self->{person} = Person->new();
    return $self->{person};

And given a half decent language you can perform tidy-up in the destructor (which, as a C++ fiddler, I kinda like).

Read Full Post »

Caleb Cushing has spilled a lot of virtual ink writing about the obligations that come with releasing open source software.

Yes, I think I’ve proven that volunteering to do open source comes with a responsibility or obligation.

(emphasis mine)

I kid you not. That is what he thinks.

And people have responded reasonably. In my opinion.

What is Caleb’s End Goal?

HUH? you people need to READ what I SAID and STOP reading INTO it.

I can think of three possibilities.

  • He likes getting comments on his blog (I can’t blame him)
  • He want open source authors to do more than they are already doing
  • If something is released [on CPAN?] that isn’t up to a certain standard, he would prefer that it wasn’t released.

More Comments

He has started closing commenting on his posts so this can’t be the reason. He could still be deriving satisfaction from high numbers of visitors though.

Open Source Authors do More

This is unrealistic. If we1 wanted to do more, we would be doing it already. If we had the time and the inclination to improve the quality, improve the documentation, fix the bugs…

Basically, someone whining, isn’t going to make people increase the amount of effort they put in which is probably already at the limit of what they want to do.

Don’t Release Imperfect Code

Or when is nothing better than something?

Chromatic mentioned one case in the opposite of modern. When an example is actively bad it reflects badly on Perl. But I think that a programmer who is motivated enough to release code to CPAN will be good enough to not write bad perl.

When a novice writes poor perl examples and others learn from those examples, it reflects poorly on perl. With CPAN, the natural barrier means that a novice is unlikely to be able to upload a module before they become more adept.

Of course, as Dave points out, a lot of what is on CPAN is not exemplary despite the fact that the authors are not novices. However, a certain amount of bad perl is certain to make it onto the internet. Should we exclude all imperfect code and lose perfectly adequate modules?

edit: edited for clarification.

I consider the cases of Authen::SASL, POE and AnyEvent that I’ve relied on recently. Now, in all those cases, the documentation has varied from pretty good to excellent (and they also have handy test cases). However, even if they had no documentation at all, it is surely better to have a working example that you can trace through with the Perl debugger than it is to have to write it yourself from scratch.

In the worst case, if the code is completely unusable I have to write to myself from scratch, but I would have had to do that anyway if the code had not been released. And if I’m not a developer I still wouldn’t have been better off without the code.

So Caleb, consider what you’re asking for. You might think that it would be great if you don’t get perl modules released to CPAN that are a bit rough around the edges, but I assure you that you’re wrong.

1. Yeah, I said we rather than they, even though my own contributions are limited.

Read Full Post »

Interesting. Strawberry Perl installed Authen::SASL::Perl without any problem. You’re right. That is not unexpected. Once you get away from POSIX signals and processes, most pure perl is pretty portable.

cd c:/strawberry/cpan/build/Authen-SASL-2.1401-V48Oe3/t
perl digest_md5.t
ok 1 - new
ok 2 - sasl mechanism
ok 3 - conn mechanism
ok 4 - client_start
ok 5 - we need extra steps

Which line is saying we need extra steps?

is($sasl->mechanism, 'DIGEST-MD5', 'sasl mechanism');

my $conn = $sasl->client_new("ldap","localhost", "noplaintext noanonymous");

is($conn->mechanism, 'DIGEST-MD5', 'conn mechanism');

is($conn->client_start, '', 'client_start');
ok  $conn->need_step, "we need extra steps";

Hang on a minute, wasn’t it need_step() that was failing in Arc? Maybe it is fixable after all if I just move from the XS authentication to the pure perl implementation.

Read Full Post »

Every so often I export the contents of my blog as a chunk of xml as backing your stuff up is always a good idea. I recently had the idea that it would be nice to load it into a locally running version of wordpress so I can do things when I’m offline like figuring out links to older articles whilst writing new posts.

I found a fairly comprehensive set of instructions of how to install wordpress on ubuntu which mentions version 2.1. Current is 2.9.2 but everything still seems to work.

I did deviate from the script slightly for the apache configuration. Ubuntu goes way over the top in my opinion with Including everything so I commented out ports.conf and embedded the code directly in apache2.conf. I also got rid of the NameVirtualHost directive which would have meant that I needed to change the port in two different files in sync.


# Include /etc/apache2/ports.conf

#NameVirtualHost *:8080
Listen 8080

<IfModule mod_ssl.c>
    # SSL name based virtual hosts are not yet supported, therefore no
    # NameVirtualHost statement here
    Listen 443


ServerName localhost

<VirtualHost *>
    Alias /images "/var/www/images/"
    <Directory "/var/www/images/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride all

    Alias /blog "/var/www/blog/"
    <Directory "/var/www/blog/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride all

Read Full Post »

I finally got around to trying out the Perl ArcV2 library that
I mentioned earlier. It didn’t go well.

There were some issues early on that were due to me not having the sasl libraries installed on my Ubuntu. After rectifying that it made it past the pre-requisites.

However, the Arc tests did not work.

Running make test
PERL_DL_NONLAZY=1 /u/jared/packages/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/arc1.t .. 1/17 shutdown() on closed socket GEN0 at /u/packages/lib/perl5/5.10.1/i686-linux-thread-multi/IO/Socket.pm line 295.
t/arc1.t .. 2/17
#   Failed test at t/arc1.t line 61.
#   Failed test at t/arc1.t line 63.

Test Summary Report
t/arc1.t (Wstat: 3328 Tests: 17 Failed: 13)
  Failed tests:  3-15
  Non-zero exit status: 13

I glanced at the tests to see what had managed to succeed. It doesn’t look good. Pretty much only ok(1) commands with no predicate passed.

Having said that, the test script looks interesting. It forks a process to fire up the server before running the tests. I’m reminded again how much I miss stuff like fork() when doing Windows development.

use Test::More tests => 17;
use strict;

# To work as a client/server we have to fork

my $pid = fork();

my $user = "mannfred";
my $pass = "klaus";

if ($pid == 0) { # Child
    use Arc::Server;
    use Arc::Connection::Server;
    my $server = new Arc::Server(...);
    exit 0;
} elsif ($pid) { # Parent
    use Arc::Connection::Client;
    ok(1); #1
    sleep(3); # Wait for the server to get ready
    my $client = new Arc::Connection::Client(...) or ok(0);
    ok(1); #2
    my $s;
    if ($client->StartSession()) { ok(1); } else { ok(0); }
    # ...

I then split out the client and server so I could run them individually.

$ perl -Mblib=../blib test-client.pl
[err]: (client) Evaluation of command _RAUTHTYPE failed
(Can't locate auto/Authen/SASL/Cyrus/need_step.al in @INC (@INC contains: ../blib/arch ../blib/lib
/u/packages/lib/perl5/5.10.1/i686-linux-thread-multi /u/packages/lib/perl5/5.10.1
/u/packages/lib/perl5/site_perl/5.10.1/i686-linux-thread-multi /u/packages/lib/perl5/site_perl/5.10.1 .)
at ../blib/lib/Arc/Connection/Client.pm line 157).

The server output looked like this.

$ perl -Mblib=../blib test-server.pl
[info]: (server) Arc v2.1 Session recognized.
[err]: (server) Connection closed by foreign host.

Okay, so it looks like the code has bit-rotted away (the last release was in 2005). This is the line in Arc::Connection::Client that is failing. need_step() is no longer mentioned in the SASL documentation.

if ($sasl->need_step || $sasl->code == 0) {

The question is, how much I would have to change to get this to work. Or to put another way, is what I have already with my AnyEvent based code closer to what I need than the currently broken Arc.

And if it doesn’t even work on Linux, what chance do I have on Windows.

Read Full Post »

or How do you make perl read input that is unbuffered?

Take a look at this. I’m simulating a process streams unbuffered output output irregularly. First it sends "hello ", then waits 5 seconds and sends "world".

I run the producer on its own, just in case it is buffering the output despite the fact I told it not to by setting $|. Nope, it isn’t. On the command line, I can see the hello output and then a few seconds later the world.

The consumer outputs whatever it receives. I get ctime() from Time::localtime for brevity.

$ date ; perl -e '$|=1; for (1..2) { print "hello "; sleep 5; print
"world\n" }' | perl -MTime::localtime -ne 'print ctime(), " : [$_]\n"'
Thu May 13 22:21:23 GMTDT 2010
Thu May 13 22:21:27 2010 : [hello world
Thu May 13 22:21:32 2010 : [hello world

No dice. The input is buffered so the consumer waits until it has received a whole line before it processes it.

Now, setting $| only affects the output, it doesn’t unbuffer the input. How about fiddling with $/ – the record separator?

$ date ; perl -e '$|=1; for (1..2) { print "hello "; sleep 5; print
"world\n" }' | perl -MTime::localtime -e '$/ = " "; while (<>) { print ctime(), " : [$_]\n" }'
Thu May 13 22:35:04 GMTDT 2010
Thu May 13 22:35:04 2010 : [hello ]
Thu May 13 22:35:09 2010 : [world
hello ]
Thu May 13 22:35:14 2010 : [world

Okay, quite cool, but there is a better solution. sysread is unbuffered. You will probably have to buffer the input yourself to ensure you have enough to process, but at least you get it as soon as it arrives. (Mental note: I should check if AnyEvent has something cool for this for anything more than a quick one-off script).

use Time::localtime;

use strict;
use warnings;

my ($input, $buffer);

while (sysread(STDIN, $input, 1024) > 0) {
    print ctime(), " : Read [$input]\n";
    $buffer .= $input;
    # Process $buffer here ...

And the test:

$ date ; perl -e '$|=1; for (1..2) { print "hello "; sleep 5; print
"world\n" }' | perl unbuffered.pl
Thu May 13 22:46:37 GMTDT 2010
Thu May 13 22:46:36 2010 : Read [hello ]
Thu May 13 22:46:41 2010 : Read [world
Thu May 13 22:46:41 2010 : Read [hello ]
Thu May 13 22:46:46 2010 : Read [world

Read Full Post »

Removed from Planet Emacsen

Now that I write mainly about p.erl I had considered not publishing this post, but I’m curious to know the reason if any and email to the planet emacsen maintainer, Edward O’Connor, has gone unanswered.

A couple of years ago when I started blogging about emacs I asked to be included in planet emacsen. And now I have been removed. What can be the reason for that I wonder.

Well, I have just come back from a blogging break. Between my post on sorting records with emacs in August 2009 and my post on Enablers and Obstructors in March 2010, there were almost 7 months. But the latest post in Amortized Analysis (a blog that hasn’t been removed from planet emacsen) was 4th January 2008. That is more than 2 years ago.

Maybe I’m not that relevant anymore. I have posted mainly P.erl posts since my break. But I do still do emacs stuff, e.g. Directory Aliases Revisited and Programming with Types. And Alexandre Vassalotti’s most recent Emacs post was September 2007.

Or was it the back and forth with Ian Eure (and to a lesser extent, Aaron Hawley) about my Emacs Database Mode. Did someone ask for me to be removed?

Anyway, whatever the reason, it has been done. What solution do I have? I have created a yahoo pipe called Better Planet Emacs which you can subscribe to. It filters a lot of the non-emacs related posts out so for me at least it is better than stock planet emacs. If you want to copy it and customise it for your own use you should go here.

Read Full Post »

Older Posts »


Get every new post delivered to your Inbox.