Enabling Your Users Part 2

Storing Session History

A little while ago we were talking about writing a little emacs-based application to enable the users to help themselves. The beginning of this tool needs a light-weight dired using emacs buttons to use for navigating around the filesystem. Today we will look at adding functionality to remember which files and directories have been accessed previously.

First of all we need some variables to store the directories and files in.

(defvar file-editor-current-dir nil)

(defvar file-editor-save-dirs nil)
(defvar file-editor-save-dirs '(a b c))
(defvar file-editor-save-files nil)

Then we provide a customizable variable where the history will be saved between emacs sessions.

(defcustom file-editor-history-file "~/.file-editor-history"
  "File in which the file-editor history is saved between invocations.
Variables stored are: `file-editor-save-dirs', `file-editor-save-files'."
  :type 'string
  :group 'file-editor)

We will frequently be adding the same file and directory into the lists and we don’t want to get dupes. I could use a data structure that helps avoid dupes or I could just sort the lists and remove adjacent dupes. Guess which option I chose.

(defun remove-dupes (list)
  (let (tmp-list head)
    (while list
      (setq head (pop list))
      (unless (equal head (car list))
        (push head tmp-list)))
    (reverse tmp-list)))

(defun file-editor-sort-history ()
  (setq file-editor-save-dirs
        (remove-dupes (sort file-editor-save-dirs #'string<)))
  (setq file-editor-save-files
        (remove-dupes (sort file-editor-save-files #'string<))))

ido has code that stores history between sessions. I’ve stolen most of it to save the file editor history. (ido-pp ...) pretty prints the variable contents into the buffer, e.g. something like this.

;; ----- file-editor-save-dirs -----

( "dir1" "dir2" "dir3" )

(require 'ido)

(defun file-editor-save-history ()
  "Save file-editor history between sessions."
  (let ((buf (get-buffer-create " *file-editor data*"))
        (version-control 'never))
    (unwind-protect
        (with-current-buffer buf
          (erase-buffer)
          (file-editor-sort-history)
          (ido-pp 'file-editor-save-dirs)
          (ido-pp 'file-editor-save-files)
          (write-file file-editor-history-file nil))
      (kill-buffer buf))))

When it comes time to load the history back, (read (current-buffer)) loads it back into the variables. You can see the use of unwind-protect and condtion-case in the code below as I talked about in my emacs lisp error handling post.

(defun file-editor-load-history ()
  (let ((file (expand-file-name file-editor-history-file)) buf)
    (when (file-readable-p file)
      (let ((buf (get-buffer-create " *file-editor data*")))
        (unwind-protect
            (with-current-buffer buf
              (erase-buffer)
              (insert-file-contents file)
              (condition-case nil
                  (setq file-editor-save-dirs (read (current-buffer))
                        file-editor-save-files (read (current-buffer)))
                (error nil)))
          (kill-buffer buf))))))

The obvious time to save the history is when we exit emacs.

(defun file-editor-kill-emacs-hook ()
  (file-editor-save-history))

(add-hook 'kill-emacs-hook 'file-editor-kill-emacs-hook)

Modifications To The Original Code

The way we choose which files and directories will be remembered is each time a file is opened, the parent directory and the file including full path are added to the appropriate variable.

(defun file-editor-open-file-editor-file (button)
  (let ((parent (button-get button 'parent))
        (file (button-get button 'file))
        (file-complete (concat parent "/" file)))
    (push parent file-editor-save-dirs)        
    (push file-complete file-editor-save-files)
    (find-file file-complete)
    (file-editor-mode)
    (longlines-mode 1)))

We need to extend the file-editor-default-dirs function to display the previously stored directories and files.

(defun file-editor-default-dirs ()
  (let ((buffer (get-buffer-create "*file-editor-dir-list*")))
    (with-current-buffer buffer
      (let ((inhibit-read-only t))
        (erase-buffer)
        (file-editor-sort-history)

        (insert "*** Default File List ***\n\n")

        (dolist (dir file-editor-default-dirs)
          (file-editor-insert-opendir-button "" dir))

        (when file-editor-save-dirs
          (insert "\n")
          (dolist (dir file-editor-save-dirs)
            (file-editor-insert-opendir-button "" dir)))

        (when file-editor-save-files
          (insert "\n")
          (dolist (file file-editor-save-files)
            (insert-text-button file 'parent "" 'file file
                                :type 'open-file-editor)))))))

And for some future functionality I am thinking about we also store the current directory that is being visited.

(defun file-editor-dir-list (parent)
  (let ((buffer (get-buffer-create "*file-editor-dir-list*")))
    (with-current-buffer buffer
      (let ((inhibit-read-only t) files dirs)
        (setq parent (expand-file-name parent))
        (setq file-editor-current-dir parent)
        (erase-buffer)
        ;; ...
))))

Interesting Emacs Links – 2009 Week 25

Org Mode

The wonderful Org mode has deservedly been getting a lot of [word] press recently. This is a really great tutorial. There is a nice customization guide at orgmode.org. endperform talks about using it for time tracking and remembering useful tricks. Emacs-fu has an article on generating html with org-mode. ByteBaker talks about using it to organise papers he downloaded and to make a wiki.

My Emacs Posts

I’ve started a series about a light-weight alternative to dired mode. Part two, which will remember locations you have visited previously is on the way.

A quick mention of longlines-mode got a comment about visual-line-mode which is the replacement in Emacs 23 onwards. I’ve switched over and it does seem better. longlines-mode was fairly reliable, but occasionally it would forget that it was supposed to be wrapping words and I would need to disable it and enable it.

Other Emacs Posts

alieniloquent talks about using advice to disable other window is you use the universal prefix (C-u). Nice trick.

Aneesh Kumar has post on switching from vim to emacs, or actually viper. As I use vim a lot, I’ve tried viper in the past but I always found that it made accessing various emacs commands harder (or maybe just different) than it is in vanilla emacs so I always switched back.

Armin Sadeghi says that his two favourite editors are SlickEdit and Emacs.

The big difference between SlickEdit and Emacs is that SlickEdit is commercial software and Emacs is open source.

If that is the big difference, why not just use Emacs?

The Real World versus BlogWorld

In the real world

• we don’t get to choose Ruby just because we want to
• we don’t have time to implement perfect solutions
• we don’t go back and fix ugly but working code
• the second law of thermodynamics always holds
• copy/paste and singleton are not the root of all evil
• Java/C++/Perl are not dying
• we still need to write code that works in IE
• for most programmers IDEs are better than emacs or vim
• two journeyman programmers are better than one rockstar

And in blogworld bloggers often forget that the plural of anecdote is not data.

Enabling Your Users

For many of the tasks I have to do day to day, I have a default tool. If I want to edit some text I’ll use Emacs; if I want to write an application slowly that runs quickly I’ll use C++; if I want to write an application quickly that runs slowly I’ll use Perl. And if I have to look after a hundred different configuration files for a hundred different users I’ll write a web interface so they can maintain the files themselves.

Ultimately, with any web app I write, anything complex is entered through a <textarea>. However, I often think that a sufficiently restricted emacs would give the users a nicer experience. If it was for me, I would just add the necessary functionality to dired. However, for a non-IT person, the dired interface is not reasonable.

So, a little experiment – can I make something emacs-based that a non-IT person would be happy to use?

File Selector Design Outline

What I envisage is a file selector that remembers the files that a user has opened before (fairly standard selector functionality right?). When a file is selected it provides a simplified view of the configuration file to the user. When they select save, it will save the full complex configuration file in all its glory.

I can also enforce some policies such as every save will check-in to source code control. Then if my editor doesn’t work correctly or the user does something they didn’t want to do, I can retrieve an earlier, working version.

File Selector Implementation

I had better give the user some defaults to click on to start with. Fortunately the configuration files are stored in two main areas: /data/sales and /data/admin.

(require 'button)
(require 'derived)

(defconst file-editor-default-dirs
  (mapcar (lambda (e) (concat "/data" (symbol-name e)))
          '(sales admin)))

I don’t want to display any files or directories that the users should not be looking in so I exclude them with a regex.

(defvar file-editor-exclude-file-regex "^RCS\\|^#\\|\\.back$\\|~$")

Each line in the file selector will be a button which enters the directory or opens the file respectively. I’ve mentioned emacs buttons previously).

(define-button-type 'open-dir
  'action 'file-editor-open-dir
  'follow-link t
  'help-echo "Open Directory")

(define-button-type 'open-file
  'action 'file-editor-open-file
  'follow-link t
  'help-echo "Open Configuration File")

The configuration files are based on xml. I want to redefine some of the keys, such as C-s for save and C-f for search so I derive a major mode from xml-mode.

(define-derived-mode file-editor-mode xml-mode "File Editor"
  "Major mode for editing configuration files.
Special commands:
\\{file-editor-mode-map}")

(if file-editor-mode-map
    nil
  (setq file-editor-mode-map (make-sparse-keymap))
  (define-key file-editor-mode-map (kbd "C-f") 'isearch-forward)
  (define-key file-editor-mode-map (kbd "C-o") 'file-editor-file-selector)
  (define-key file-editor-mode-map (kbd "C-s") 'file-editor-save-file))

The directory buttons and file buttons call file-editor-open-dir and file-editor-open-file respectively.

(defun file-editor-open-dir (button)
  (let ((parent (button-get button 'parent))
        (dir (button-get button 'dir)))
    (file-editor-dir-list
     (format "%s%s" (if parent (concat parent "/") "") dir))))

(defun file-editor-open-file (button)
  (let ((parent (button-get button 'parent))
        (file (button-get button 'file)))
    (find-file (concat parent "/" file))
    (file-editor-mode)
    (longlines-mode 1)))

When the file selector is first opened, it displays some default directories. Later on I’ll extend this to display files that have been opened recently.

(defun file-editor-insert-opendir-button (parent dir)
  (insert-text-button (format "[%s]" dir)
                      :type 'open-dir 'parent parent 'dir dir)
  (insert "\n"))

(defun file-editor-default-dirs ()
  (let ((buffer (get-buffer-create "*file-editor-dir-list*")))
    (with-current-buffer buffer
      (progn
        (erase-buffer)
        (insert "*** Default File List ***\n\n")
        (dolist (dir file-editor-default-dirs)
          (file-editor-insert-opendir-button nil dir))))))

I keep the file selector buffer read-only and therefore need to set inhibit-read-only to t whenever I write to it. I can then get all the files and directories within the current directory using directory-files-and-attributes.

I skip all the files beginning with a period. A string is a type of array so I can just compare against the first character using aref. I suspect it is more efficient than using a regex (surely it must be?) but I haven’t measured.

I list all of the directories prior to the files.

(defun file-editor-dir-list (parent)
  (let ((buffer (get-buffer-create "*file-editor-dir-list*")))
    (with-current-buffer buffer
      (let ((inhibit-read-only t) files dirs)
        (setq parent (expand-file-name parent))
        (erase-buffer)

        (dolist (vec (directory-files-and-attributes parent))
          (let ((filename (car vec))
                (is-directory (cadr vec)))
            (unless (or (eq (aref filename 0) ?.)
                        (string-match file-editor-exclude-file-regex filename))
              (if is-directory
                  (push filename dirs)
                (push filename files)))))

        (insert (format "Current Directory: %s\n\n" parent))
        (file-editor-insert-opendir-button parent "..")

        (dolist (dir (reverse dirs))
          (file-editor-insert-opendir-button parent dir))

        (dolist (file (reverse files))
          (insert-text-button file 'parent parent 'file file
                              :type 'open-file)
          (insert "\n"))

        (toggle-read-only 1)))))

file-editor-file-selector opens either the default files/directories or the buffer which should contain the last visited location.

(defun file-editor-file-selector ()
  (interactive)
  (let ((buffer (get-buffer "*file-editor-dir-list*")))
    (if buffer (switch-to-buffer buffer) (file-editor-default-dirs))))

(file-editor-file-selector)

Okay that is probably enough for one post. Obviously I have a fair amount of functionality left to implement. What do you guys think? Am I crazy to even consider using emacs over <textarea>? Let me know in the comments.

longlines-mode

Alex Bennee mentions the excellent longlines-mode in a recent post. I learnt something new – that it can highlight hard returns using (longlines-show-hard-newlines). Thanks Alex.

What is so good about it when you can wrap a paragraph at any time using M-q (fill-paragraph)? It wraps a paragraph in realtime as you are editing it, just like a word processor. And not only that, it uses soft newlines. If you cut and paste it, the result is not wrapped which is generally what you want.

I use longlines-mode a lot when I’m not editing source code. One thing I do is dump text in a scratch buffer (created with C-x b *random-name* <RET>) and then I press C-c C-l. Now I can look at two or more parts of the text at the same time using C-x 2 and related functions.

(defun set-longlines-mode ()
  (interactive)
  (text-mode)
  (longlines-mode 1))

(global-set-key (kbd "C-c C-l") 'set-longlines-mode)

Just as Alex does, I use longlines-mode for writing my blog posts. However, I write most of my blog posts using muse so I add it to the muse-mode-hook.

(defun muse-minor-modes ()
  (longlines-mode 1)
  (font-lock-mode 0))

(add-hook 'muse-mode-hook 'muse-minor-modes)

Interesting Emacs Links – 2009 Week 23

HowTo Disable indent-tabs-mode

It sounds like this guy is having problems with emacs automatically converting spaces to tabs. I can sympathise – it took me a while to figure out how to disable indent-tabs-mode correctly.

My post also received a comment asking how to add a tabs if you have disabled indent-tabs-mode. If you need a single tab, use C-q <TAB> (quoted-insert). Otherwise, if you need to enter many tabs you can enable M-x eval-expression <RET>(setq indent-tabs-mode t)<RET> or bind tab to (self-insert-command).

Working With Multiple Databases

To much wailing and nashing of teeth, several weeks ago I introduced my emacs db mode. I have made an extension for quickly and easily switching between multiple databases using ido. It also serves as a nice tutorial for adding custom sections to the mode-line.

Always Prompting Before Exiting Emacs

Nilesh has a post about prompting before exiting to avoid accidents. Now, I occasionally want a prompt – I often get a prompt anyway as I have either processes or unsaved buffers. Occasionally I am caught out when I am sketching things out in scratch buffers (with, e.g. C-x b *stuff* <RET>).

Most functions that are called when emacs is exiting call (save-buffers-kill-emacs) which has a customizable variable you can set to control if and how emacs prompts when exiting. I like to timeout these type of prompts, just in case I was attempting to exit while the screen was off.

(setq confirm-kill-emacs
      (lambda (e)
        (y-or-n-p-with-timeout
         "Really exit Emacs (automatically exits in 5 secs)? " 5 t)))

Emacs As A Windows Editor?

There was a strange thread in the Joel On Software forums. Someone wanted an alternative to Emacs on Windows with the following features:

• macros
• good windows integration
• syntax highlighting for multiple file types
• free as in beer

Why strange? Well, emacs has all of this and more. Keyboard macros are available with C-x ( and C-x ) or <f3> <f4> in more modern versions. In addition, Emacs 23 is extremely good-looking.

Setting cua-mode and recentf will get you most of the way to that authentic windows app experience.

(cua-mode t)
(recentf-mode t)

Remember to use the Emacs Win32 build.

Miscellaneous Tips

Here is a mention of the fantastic occur command.

Someone made a brief post (http://rydow.wordpress.com/2009/06/11/emacs-history-between-sessions) about savehist-mode although they couldn’t remember where they had seen the original. Along with many great Emacs hints, it was from Emacs-fu (it was in a comment by valvo).

A Modern Style of Programming

Something a casual user of a language will miss out on, is using the latest and greatest libraries of that language and generally programming in a modern style. For example, I¹ still naturally use open(FH, 'filename') || die; and have to force myself to use the more modern open(my $fh, '<', 'filename') with its lexically scoped filehandle.

I have been programming Perl for 12 years or so, but aside from one conference (YAPC Muenchen 2002) I’ve never really immersed myself in the community. For this reason, I think I have missed out on quite a few niceties. Moose, DBIx and other modules bring Perl up to the level of its contemporaries if you don’t need to work with people who are not using them. I only came across POE recently (which I keep mentioning because it is so awesome).

Heck, even C++ has boost.

Modifiable Syntax

or DSLs I think they call it.

Ray Dillinger once pointed out that people write scheme in a variety of incompatible styles because the substrate isn’t pleasant for programming on directly. But it is possible to layer any sugar you like on it. This leads to a bunch of different and practically incompatible styles.

Anyway, what I see is that scheme programmers are capable of
doing a heck of a lot as individuals, and are very happy with
the personally-customized language they each work with. But
they tend not to work together on large projects because of
the cognitive overhead of learning each other’s personally-
customized languages, which may have different or conflicting
definitions.

Common Lisp programmers, by contrast, have a lot of standard
libraries and tend to forgive or ignore some small things that
may not fit perfectly with their personal style. But they do
work together on large projects, because they all have the same
set of language extensions and they can read each other’s code.

I still find eager comprehensions about the nicest way of specifying nested loops that I’ve seen. Perl syntax tweaking dudes: if you add these, I’ll never switch! What’s that? Fix it myself? It is easier to move to python or ruby I think.

Is it a coincidence that languages with fixable syntax (Lisp, Perl, Tcl, Ocaml) have ‘lost’ to those with a fixed syntax (Java, Python)? Ruby dudes beware.

Supporting Your Language

There have been a few posts floating around the blogosphere talking about writing posts supporting perl. I put my own effort into doing something similar for Emacs. However, in my opinion, Emacs needs the help and Perl does not.

Emacs could be greatly improved if there were many more Emacs Lisp hackers creating libraries and writing examples and documentation. Perl already has all of those things. <strike>As</strike> If its popularity wanes, what is lost? I guess people are thinking about job opportunities and stuff like that, but I suspect that the outflow of former Perl programmers will outpace the loss of Perl jobs.

Er, So what is my point?

Oh yes, Emacs-using Perl dudes, please add eager comprehensions to Perl and write Emacs blog posts rather than Perl ones. Thank you.

---

1. Even though I’m not really a casual Perl user. I do this stuff professionally don’t you know

Error Handling in Emacs Lisp

One thing that very few code samples (for any language) touch upon is error handling. There are a number of reasons for this.

• Error handling code is ugly
• It can take up a significant amount of the code
• It distracts from what the sample is trying to demonstrate

Robust code almost always needs error handling. How is it done in Emacs Lisp?

Try / Catch / Finally – The Error Handling Trinity

In Java, code that may throw an exception looks like this:

try {
    // Code that may throw
} catch (SomeType exception) {
    // Handler code for SomeType
} catch (Exception exception) {
    // Handler code for Exception
} finally {
    // Code to execute after // Code + the handler if any
}

The finally clause handles the clean-up and is always executed whether or not an exception is thrown.

How Emacs Lisp Does It

Emacs Lisp has an equivalent for each part – from throwing the exception, to catching it and handling clean-up. (error ...) throws the most basic exception (in emacs lisp, they are called signals). If you execute an error call without protecting it, emacs will enter the debugger.

unwind-protect

unwind-protect is the emacs way of implementing a finally clause. In the following code, when you exit from the debugger, hi will be inserted in the buffer.

(unwind-protect
    (error "hello")
  (insert "hi"))

condition-case

Try/Catch is spelled condition-case. There is one handler per signal we want to handle. All signals are derived from ‘error so catching error catches any remaining signals.

(condition-case var bodyform &rest handlers)

(unwind-protect
    (let (retval)
      (condition-case ex
          (setq retval (error "Hello"))
        ('error (message (format "Caught exception: [%s]" ex))))
        retval)
  (message "Cleaning up..."))

We can wrap this all in a macro. Adapt the following for your own use.

(defmacro safe-wrap (fn &rest clean-up)
  `(unwind-protect
       (let (retval)
         (condition-case ex
             (setq retval (progn ,fn))
           ('error
            (message (format "Caught exception: [%s]" ex))
            (setq retval (cons 'exception (list ex)))))
         retval)
     ,@clean-up))

Some Examples

(safe-wrap (error "Hello") (message "Unwinding..."))
Caught exception: [(error Hello)]
Unwinding...
(exception (error "Hello")) ;; return value

;; (safe-wrap (message "Hello") (message "Unwinding..."))
Hello
Unwinding...
"Hello" ;; return value

(safe-wrap (/ 1 0))
Caught exception: [(arith-error)]
(exception (arith-error)) ;; return value

If you liked this post, why not subscribe to my RSS feed.

Using sql-mode or db-mode with multiple databases

I’ve written a new article about how to query multiple databases using db-mode. It demonstrates quickly selecting between multiple databases using ido and you may be surprised to learn that it using a mechanism very similar to my directory aliases. This is a follow-up to my page on emacs db mode.

Why Should You Be Interested?

If you need to work with multiple databases and you like using emacs then the page is especially for you. Go take a look! It also has a good example of using the emacs mode line, in this case to display which database you are currently connected to. And finally it is another example of using ido which if you haven’t tried yet, you really should.

About The Code

The code is designed to work with my own db-mode as that is what I use but it could be easily adapted for use with sql-mode using the enter-db function from one of my earlier posts.

The preamble would be something like this:

;; ...
(let ((sql-sybase-program "/usr/bin/isql")
      (sql-server db-server)
      (sql-database db-database)
      (sql-user db-user)
      (sql-password db-password))
;; ...

Ian Eure also has a post on working with multiple databases here. It uses a new buffer for each database which is probably preferable to killing the connection.

Loops in Ocaml

Following on from my look at basic looping syntax in various languages I thought I’d take a look at various methods of looping in Ocaml to make amends for the Ocaml solution.

gcd is the same as in the previous post.

let rec gcd a b =
  if b = 0 then a
  else gcd b (a mod b);;

Using one function for one loop is nicer than cramming both into one which felt too much like coding using if and goto.

Most of the solutions use the following recursive function.

let count_for_denom denom =
  let rec loop num count =
    if num >= (denom/2+1) then count
    else loop (num+1) (count + if gcd num denom = 1 then 1 else 0)
  in
    loop (denom/3+1) 0;;

The first solution is a pure recursive function. This is really just mental masturbation – needlessly exchanging loops for more verbose recursion.

let solve_it max_denom =
  let rec loop denom count =
    if denom > max_denom then count
    else loop (denom+1) count+count_for_denom denom
  in
    loop 5 0;;

Printf.printf "%d solutions\n" (solve_it 10000);;

$ time ./a.out
XXX solutions

real    0m1.138s
user    0m1.108s
sys     0m0.000s

Functional programming includes a bunch of higher level functions such as map and fold. In a strict language it would be wasteful to construct a list before iterating over it with one of these higher-order functions.

How about a fold-like function that folds over a range of numbers?

let collector s e acc f =
  let rec loop s acc =
    if s > e then acc
    else loop (s+1) (f acc s)
  in loop s acc;;

Printf.printf "%d solutions\n"
  (collector 5 1000 0 (fun acc denom -> acc+count_for_denom denom));;

Here is one with a reference and a for next loop. I think this is quite nice – I like that Ocaml is a pragmatic language and includes this kind of stuff.

let count = ref 0 in
  for i = 5 to 1000 do
    count := !count + count_for_denom i
  done;
  Printf.printf "%d solutions\n" !count;;

And finally the pure nested for loop solution which is about 20% faster than the recursive versions.

let count = ref 0 in
  for i = 5 to 10000 do
    for j = (i/3+1) to (i/2) do
      if gcd i j == 1 then count := !count + 1
    done
  done;
  Printf.printf "%d solutions\n" !count;;

$ time ./a.out
XXX solutions

real    0m0.934s
user    0m0.928s
sys     0m0.004s

So, is there anything I missed? What other types of loops are there in Ocaml?