Posts Tagged ‘emacs lisp’

I am a student of the design your data structures first school of programming1. My first proper language was C and in that language pretty much all you had was typedef struct and a handful of primitive types.

Despite the fact that I now spend most of my time programming with dynamic languages (emacs lisp and perl), I still believe it is important to think about your types before you start programming. Eh, but what’s that? Dynamic languages don’t have types? Au contraire mon ami. Not having static type-checking is not the same as not having types.

So, enough handwaving, what about a real example?

Extensible Vectors

In C++, one of the data structures I use most is the STL vector. But in emacs lisp, the vector is a very different animal as it doesn’t automatically extend. If I want an extensible vector type (an evector), I have to implement it myself (or find a library containing one, but that wouldn’t make a good example).

So, first of all, I need to know what pieces of information each evector will need. There will be the vector itself, plus the current end of the vector and the current size before we need to resize it. If we want to use fibonacci to get the next size of vector we need another slot for the fibonacci counter. So each evector will look like this:

['evector [...] <end position> <fib counter>]

The constructor is generally called make-<typename> so the implementation (assuming we want a vector that starts with 8 slots) is:

(defun make-evector ()
  (vector 'evector (make-vector 8 nil) 0 5))

We also need a predicate for checking if a variable is of type evector.

(defun evectorp (object)
  (eq (aref object 0) 'evector))

What does the push-back operation look like?

(defun evector-push-back (object elem)
  (let* ((vec (aref object 1))
         (pos (aref object 2))
         (len (length vec)))
    ;; increase the size of the vector if necessary
    (when (<= len pos)
      (let* ((new-size (+ len (aref object 3)))
             (new-vec (make-vector new-size nil))
             (i 0))
        ;; copy the original vector into the new one
        (while (< i len)
          (aset new-vec i (aref vec i))
          (incf i))
        (setq vec new-vec)
        (setf (aref object 1) vec)
        (setf (aref object 3) len)))
    ;; set the vector element and update end position
    (aset vec pos elem)
    (incf (aref object 2))))

Hmmm… referring to the member variables by position looks error prone and inflexible. What can we do about that?

If you’re thinking accessors, you’re on the right lines. Fortunately, because of setf, you only need a getter.

(edit: fixed, thanks Jisang)

(defsubst evector-data (object) (aref object 1))
(defsubst evector-end-position (object aref object 2))
(defsubst evector-fib-var (object aref object 3))


But even better, there is a macro that does defines your constructor, predicate and accessors for you, defstruct.

(defstruct evector
  (data (make-vector 8 nil))
  (end-pos 0)
  (fib-var 5))

Yet another advantage of using defstruct, is that the error messages are clearer than with my basic accessors.

(evector-fib-var 'x)

[basic accessor]
Debugger entered--Lisp error: (wrong-number-of-arguments (lambda (object aref object 3)) 1)

[defstruct accessor]
Debugger entered--Lisp error: (error "evector-fib-var accessing a non-evector")
  signal(error ("evector-fib-var accessing a non-evector"))
  error("evector-fib-var accessing a non-evector")

edit: apologies to those who arrived from ironman perl. This isn’t a perl related post so I hadn’t tagged it as such. I didn’t realise that posts that mention perl anywhere get picked up.

1. Although it might not come across in my examples 🙂


Read Full Post »

Sorting Records With Emacs

I use sort-lines fairly frequently to keep my #includes, uses or variable names in order. Occasionally though, I need to sort something slightly more complicated and I end up writing a little perl script. I found myself thinking it must be equally easy in emacs.

For example, I say I have the following data which I want to sort by the second field which is in position 12 and 13.

Edward Wood 9   01/07/2005
Ed Smith    25  06/12/2004
James Brown 18  01/07/2005
Jon James   13  05/15/2007

Now, emacs does have a sort-columns function which does all sorts of fancy things including checking if we are using Unix and shelling out to sort if possible, but it doesn’t seem to be able to do a numerical comparison. However, in sort.el it does have a general purpose sorting function called sort-subr.

(defun sort-subr (reverse nextrecfun endrecfun
                          &optional startkeyfun endkeyfun predicate)

sort-lines itself is a good start for my custom function as it has all the code for handling regions, reversal and skipping between records (lines) that we need.

(defun sort-lines (reverse beg end)
  (interactive "P\nr")
      (narrow-to-region beg end)
      (goto-char (point-min))
      (let ((inhibit-field-text-motion t))
        (sort-subr reverse 'forward-line 'end-of-line)))))

Getting the startkeyfun and endkeyfun correct was fairly straight forward but I wasn’t entirely sure what information was passed to the predicate.

(defun sort-on-field (reverse beg end)
  (interactive "P\nr")
      (narrow-to-region beg end)
      (goto-char (point-min))
      (let ((inhibit-field-text-motion t))
        (sort-subr reverse 'forward-line 'end-of-line
                   (lambda () (forward-char 12))
                   (lambda () (forward-char 2))
                   (lambda (a b)
                     (message (format "[%s] [%s]" a b))))))))

[(703 . 705)] [(676 . 678)]
[(757 . 759)] [(730 . 732)]
[(757 . 759)] [(703 . 705)]
[(730 . 732)] [(703 . 705)]

The final working version:

(defun sort-on-field (reverse beg end)
  (interactive "P\nr")
      (narrow-to-region beg end)
      (goto-char (point-min))
      (let ((inhibit-field-text-motion t))
        (sort-subr reverse 'forward-line 'end-of-line
                   (lambda () (forward-char 12))
                   (lambda () (forward-char 2))
                   (lambda (a b)
                     (let ((s1 (buffer-substring (car a) (cdr a)))
                           (s2 (buffer-substring (car b) (cdr b))))
                       (< (string-to-number s1) (string-to-number s2)))))))))

Providing a nice way of asking for the start and end location of the sorting field is left as an exercise for the reader.

Read Full Post »

I don’t create many lisp macros but it is still a huge advantage having them. I’m a big user of macros that other people have created. For example, by default emacs lisp does not support keyword parameters. Thanks to the magic of macros they have been added in the cl.el common lisp package.

(require 'cl)

(defun* example (&rest args
                 &key (a 1) (b 2))
  (format "%s %s %s" args a b))

(example) works as expected and returns "nil 1 2"

(example :a 10) is pretty good too although you may be surprised that keywords are not removed from the &rest args "(:a 10) 10 2"

However, (example 1) returns the following surprise (error "Keyword argument 1 not one of (:a :b)").

If you are using &rest and &key together, you can fix this with &allow-other-keys.

(defun* example (&rest args
                 &key (a 1) (b 2)
  (format "%s %s %s" args a b))

Okay, so much for pithy examples. How about something practical?

(defun* remote-path (&rest path
                     &key (user "jared") (server "someserver")
  (concat "ftp:" user "@" server ":"
          (mapconcat #'identity path "/")))

(remote-path :user "uat-user" "/tmp" "test-dir") throws this error (wrong-type-argument sequencep :user). This is because if you use &rest arguments and &key arguments together, the keyword arguments are also passed through into &rest.

I couldn’t find an obvious function to remove keyword parameters so I’ve written my own remove-keyword-params.

(defun remove-keyword-params (seq)
  (if (null seq) nil
    (let ((head (car seq))
          (tail (cdr seq)))
      (if (keywordp head) (remove-keyword-params (cdr tail))
        (cons head (remove-keyword-params tail))))))

(defun* remote-path (&rest path
                     &key (user "jared") (server "someserver")
  (concat "ftp:" user "@" server ":"
          (mapconcat #'identity (remove-keyword-params path) "/")))

Now it works correctly.

(remote-path :user "uat-user" "/tmp" "test-dir")
;; "ftp:uat-user@someserver:/tmp/test-dir"

So what do I want this for? For my directory aliases of course.

(defconst *dired-dirs*
  (list (cons "dev" (remote-path "~/dev"))
        (cons "dev:system"
              (remote-path :user "dev-user" "~/system"))
        (cons "uat:system"
              (remote-path :user "dev-user" "~/system"))))

And surprise surprise, because emacs is architected so well, it all works remotely through tramp completely transparently.

Read Full Post »

I have a number of utility functions in a file called my-utils.el. The first one I wrote was add-path.


(defun add-path (&rest path)
  (let ((full-path ""))
    (dolist (var path)
      (setq full-path (concat full-path var)))
    (add-to-list 'load-path (expand-file-name full-path))))

This means that when I add a new third-party library (in its own directory), I can just write:

(add-path *elisp-dir* "muse-3.12")

I have a function called identity that returns whatever was passed to it.

(defun identity (e) e)

This is particularly useful in functions like mapconcat.

(defun add-path (&rest path)
  (add-to-list 'load-path (mapconcat #'identity path "")))

I like this function that I found on the internet a while ago (can’t find it again for attribution unfortunately).


(defun insert-YYYMMDD ()
  (insert (format-time-string "%Y%m%d")))


remove-dupes removes duplicate items that are next to each other in a list in the same way that uniq does. (I mentioned this previously in one of the enabling your users posts.)

(defun remove-dupes (list)
  (let (tmp-list head)
    (while list
      (setq head (pop list))
      (unless (equal head (car list))
        (push head tmp-list)))
    (reverse tmp-list)))


Sometimes you will have an assocation list where you just want to replace one of the values.

(defun assoc-replace (seq values)
  "Replace an element within an association list where the cars match."
  (mapcar (lambda (elem)
            (let* ((key (car elem))
                   (val (assoc key values)))
              (if val (cadr val) elem))) seq))

And often, only the cdr needs to change so kv is a shortcut for that case. (kv 'a 'b) returns (a (a . b)).

(defsubst kv (k v)
  `(,k (,k . ,v)))

(These functions were mentioned in emacs association lists)


ext-mode-map allows me to easily setup a major mode for extensions that are inconsistently capitalized.

(defun file-extensions (l)
  (concat "\\.\\("
           (lambda (s)
             (mapconcat (lambda (c)
                          (let ((c (upcase (char-to-string c))))
                            (concat "[" c (downcase c) "]")))
                        (symbol-name s) ""))
           l "\\|")

(defun ext-mode-map (extensions mode)
  (cons (file-extensions extensions) mode))

Then, for example, I can setup my perl extensions like this:

(add-to-list 'auto-mode-alist (ext-mode-map '(pl perl pm) 'cperl-mode))

(These functions were previously mentioned in autoloading an emacs major mode)


I’m still using this function to duplicate the current line. It works unless you are on the final line which is very rare and you can work around that by opening a line below.

(defun duplicate-current-line ()
  (beginning-of-line nil)
  (let ((b (point)))
    (end-of-line nil)
    (copy-region-as-kill b (point)))
  (beginning-of-line 2)
  (open-line 1)

(This function was mentioned previously in A Simple Emacs Shortcut – Duplicate Line)


I often copy text into buffer and want to read it nicely wrapped around. I’ll change this function to use visual-line-mode when all my emacs versions are upgraded to 23.

(defun set-longlines-mode ()
  (longlines-mode 1))


Other people have talked about their own count-words functions in the past. I have my own ideas about what constitutes a word.

(defun count-words ()
  (let ((words (count-matches "[-A-Za-z0-9][-A-Za-z0-9.]*"
                              (point-min) (point-max))))
    (message (format "There are %d words" words))))

regex-replace and string-repeat

I sometimes have a text transform that I want to apply globally to a buffer. Exercise for the reader, fix regex-replace to only apply to a region if one is selected.

(defun regex-replace (regex string)
  (goto-char (point-min))
  (while (re-search-forward regex nil t)
    (replace-match string)))

(defun string-repeat (str n)
  (let ((retval ""))
    (dotimes (i n)
      (setq retval (concat retval str)))

Read Full Post »

Some time ago, I wrote about doing batch processing of text with an external process running, e.g. Perl. Similarly, emacs-lisp has a lot of functionality for manipulating text.

The Problem

I have a file like this:

John James,Admin,other data,...
Dave Jones,Sales,...
Lisa Sims,IT,...

I want to convert it into the following1:

AND name IN ("Dave Jones", "John James", "Lisa Sims")
AND dept IN ("Admin", "IT", "Sales")

The Solution

First of all I need a helper function that converts lisp lists into a quoted comma-separated list.

(defun make-csv (seq)
  (mapconcat (lambda (e) (format "\"%s\"" e)) seq ", "))

And then I can iterate over the text with re-search-forward, collecting the matched strings. At the end, I’ll output the collected strings. in a sql clause fragment.

(defun process-lines (&optional begin end)
  (interactive "r")
  (goto-char begin)
  (let (names depts)
    (while (re-search-forward "\\([^,]+\\),\\([^\n,]+\\)" end t)
      (push (match-string 1) names)
      (push (match-string 2) depts)
    (insert (format (concat "\n"
                            "AND name IN (%s)\n"
                            "AND dept IN (%s)\n")
                    (make-csv (sort names #'string-lessp))
                    (make-csv (sort depts #'string-lessp))))))

If you liked this post, why not subscribe to my RSS feed.

1. Okay, you got me, I don’t really want to convert it into this. But for the purpose of the example, this will do. Exercise for the reader – how can I convert it into sql that will efficiently extract just the lines I want?

Read Full Post »

or is the documentation to blame?

From time to time I have a little niggle with emacs or need a new feature. As I know a bit of elisp it is all too tempting to jump in and code up a solution rather than investigate whether someone else has already solved it.

A case in point would be my database mode. Ian Eure pointed out that there is a way to get sql-mode (a core module) to do something similar to what I wanted.

(defun enter-db ()
  (let ((sql-sybase-program "/usr/bin/isql")
        (sql-server "DB_SERVER")
        (sql-database "DB_DATABASE")
        (sql-user "USER")
        (sql-password "PASSWORD")
        (sql-sybase-options '()))
    (other-window -1)
    (switch-to-buffer "*sql*" t)

And there are modules that provide similar functionality to my directory aliases although I am extremely pleased I wrote it. I use it many times every day and I love it.

In fact, often when I publish a piece of elisp I have written, people point out that it has already been done. Another example would be when Peter Jones pointed out a much easier way of implementing emacs muse aliases.

So what is the answer? Hang out on #emacs? Improve my google-fu so I can find pre-existing solutions? Or some kind of central documentation / module repository. Yes, the wiki is great but it isn’t comprehensive. It didn’t mention the alternative to database mode until I added it.

Central Documentation / Module Repository

I often think that emacs would make a nice alternative to Visual Basic for creating quick interactive applications if only there was a decent API reference.

I’m not saying CPAN (if you listen carefully, you can hear me very clearly not saying CPAN) but some sort of centralised repository would be nice. However, I suspect the emacs lisp writing part of the community is [far] to small to generate or support the required infrastructure. Yes, I know about ELPA, but when I checked it didn’t have more than half my third-party packages (including org-mode, htmlize and tuareg)

My Solution

No, I think that the best solution is to keep publishing my code and wait for my readers to tell me about the right way to do things. What do you think?

Read Full Post »

Customizing Emacs Muse – Part 2

Here are part 1 (Creating Squidoo Lenses With Emacs Muse) and part 3 (Emacs Muse Aliases).

We all know what an association list is right? If you want to map keys to values, you’re going to use an assocation list.

(defvar v '((GB "Great Britain")
            (UK "United Kingdom")
            (FR "France")))

(assoc 'GB v)

will return the pair (GB "Great Britain").

They are used in a similar way to a hash in various scripting languages. For example, muse-html uses one, muse-html-markup-strings to determine which text to use for each type of markup.

You might remember that previously I advocated the use of regex transforms to correct the html after generation is complete. However, that can cause problems. We therefore need to fix the association list to have the correct mapping.

Assocation lists have a bunch of functions which operate on them, e.g. assoc and rassoc for looking things up and assq-delete-all for removing elements. However, I couldn’t find one which changes elements.

(defun assoc-replace (seq values)
  "Replace an element within an association list where the cars match."
  (mapcar (lambda (elem)
            (let* ((key (car elem))
                   (val (assoc key values)))
              (if val (cadr val) elem))) seq))

This will allow me to replace both key and value within the association list at the same time. Normally, I’ll just need to replace the value so I made a helper which saves me from having to repeat the key.

(defsubst kv (k v)
  `(,k (,k . ,v)))

And now I can fix the assocation list

(defconst sq-code (concat "<b style=\"font-weight: normal; "
                          "font-size: 115%; background: #eee "
                          "padding: 3px;\">"))

(setq muse-html-markup-strings
      (assoc-replace muse-html-markup-strings
                     (list (kv 'begin-literal sq-code)
                           (kv 'end-literal "</b>"))))

Alternatively, I could delete the existing elements and add the new ones on the start of the list, maybe with something like the following

 (cons 'begin-literal sq-code)
 (cons '(end-literal "</b>")
       (assq-delete-all 'begin-literal
                        (assq-delete-all 'end-literal

But I think my way is neater.

Read Full Post »

Older Posts »