Getting into a rabbit's hole and - maybe - getting out: Emacs Make Compile (EMC)

In the past years I fell form one rabbit's hole into another. For the first time in a rather long time, I feel I am getting out of one.

First of all let me tell you what I produced. I built the "Emacs Make Compile", or "Emacs Master of Ceremonies", package EMC. Soon it will be available in melpa. The EMC package is a wrapper around compile.el that allows you to semi-transparently invoke make or cmake from Emacs. Either on UN*X, MacOS or Windows.

Once you have loaded the emc library in the usual ways, you can just issue the Emacs command emc:run (yes: Common Lisp naming conventions). The command is just the most general one available in EMC; other ones are the more specialized emc:make and emc:cmake. Emacs will then ask you for the necessary bits and pieces to ensure that you can run, say, make. The README file included in the distribution explains what is available in more details.

Where did it all begin?

Why not stick with compile.el? Because it does not have "out-of-the-box" decent defaults under Windows. At least, that was my original excuse.

I fell into this rabbit's hole coming from another one of course.

Some time ago, I started fiddling around with Emacs Dynamic Modules. I wanted to compile them directly from Emacs in order to "simplify" their deployment. Therefore, I set out to write a make function that would hide the compile setup.

Alas, I found out that, because of the necessary setup, invoking the Microsoft Visual Studio toolchain is not easy before you can get to cl and nmake. That was not all that difficult as a problem to solve, but then I made the mistake of learning to cmake. You know; to ensure that the building process was "more portable". The basic machinery for make and nmake worked to also get cmake up and running. But then I made another mistake: I started to want to get EMC to be usable in the "Emacs" way: at a minimum getting interactive commands working. That got me deeper and deeper in the rabbit's hole.

At the bottom of the hole (yep: I got there!)

I found out many things on my way to the bottom. That is, I learned many things about the Emacs Lisp ecosystem and wasted a lot of time in the process. I never was a fast learner. All in all, I think I can now say two things.

• Making a command, i.e., an interactive function is not trivial, especially if your function has many arguments. Bottom line: your Emacs commands should have *few* arguments. I should have known better.
• The Emacs widget library is woefully underdocumented (which, of course, brings up the question: why did you want to use it?)

In any case, what I was able to concot is that hitting M-x emc:make does what you expect, assuming you have a Makefile in the directory; if not you will be asked for a "makefile", say stuff.mk to be used as in

make -f stuff.mk

or

nmake /F stuff.mk

Issuing C-u M-x emc:make will ask you for the "makefile", the "source directory", the "build directory", "macros", and "targets".

In what other ways could I have wasted some time? By coming up with a widget-based UI! (See my previous post about DeepSeek and the widget library). The result can be invoked by using the command emc:emc, which pops up the window below.

Getting out of the rabbit hole by popping the stack

I kind of consider EMC finished. I am pleased by the result; it was fun to solve all the problems I encountered, although the code is not exaclty nice or nicely organized. Is EMC useful? Probabiy not so much, but I have the luxury of wasting hacking time. I just hope somebody will like it: please try it out and report bugs and suggestions (the minor mode and associated menu need work for sure, as well as emc:emc).

Having said so, I can now go back to play with Emacs Dynamic Modules, which is where I was coming from. After being satisfied with that, I will be able to climb back up a bit more from the rabbit's hole; that is, I will be able to go back to the magiciel library (which kind of works already). You may ask why I am writing magiciel, but you will have to reach down several levels in the rabbit's hole.

In any case, I finished one thing. It's progress.

---

'(cheers)

Emacs Lisp Programming with DeepSeek: A New Widget

The Emacs widget library is useful; alas its guts are ... semi-documented and most of its inner working a bit mysterious. I wanted a column widget where I could insert and remove a few "line-like" widgets. The editable-list widget does not cut it (too many extra things: the INS and DEL buttons) and the group widget is too inflexible.

After too much time trying to understand all the intricacies of the widget library (see my rant in my previous blog post, which perfectly applies in this case) I asked DeepSeek to help me out. The result, the dynamic-group widget (after several iterations and mistakes on part of DeepSeek) is below. It works satisfactorlly, although it could be improved by anybody with a better understanding of the widget library. What is does is to manage a colimn of line-like widgets adding and removing from the end of the :children list. Check the demo-dynamic-group for a test run.

It has been fun. Although I still want a better widget! That's why I am posting this for anybody to pitch in. Any help is welcome.

BTW. There still are some warts in the code. Can you spot them?

;;; -*- Mode: ELisp; lexical-binding: t -*-
;;; emc-dynamic-group.el
;;;
;;; `flycheck-mode' does not like the above.  `flycheck-mode' is wrong.

;;; Code:

(require 'widget)
(require 'wid-edit)


(define-widget 'dynamic-group 'default
  "A container widget that dynamically manages child widgets in a column."
  :format "%v"
  :value ()
  :tag "Dynamic Group"
  :args nil
  
  ;; Core widget methods
  :create (lambda (widget)
            (let ((inhibit-read-only t))
              (widget-put widget :from (point))
              (dolist (child (reverse (widget-get widget :children)))
                (widget-create child))
              (widget-put widget :to (point))))

  :value-get (lambda (widget)
               (mapcar (lambda (child)
                         (widget-apply child :value-get))
                       (widget-get widget :children)))
  
  :value-set (lambda (widget value)
               (widget-put widget :value value))
  
  :value-delete (lambda (widget)
                  (dolist (child (widget-get widget :children))
                    (widget-apply child :value-delete)))
  
  :validate (lambda (widget)
              (let ((children (widget-get widget :children)))
                (catch :invalid
                  (dolist (child children)
                    (when (widget-apply child :validate)
                      (throw :invalid child)))
                  nil)))
  )


(defun dynamic-group-add (widget type &rest args)
  "Add a new widget (of TYPE and ARGS to the WIDGET group."
  (let ((inhibit-read-only t))
    (save-excursion
      (goto-char (widget-get widget :to))
      (let ((child (apply 'widget-create (append (list type) args))))
        (widget-put widget
		    :children (cons child (widget-get widget :children)))
        (widget-put widget
		    :to (point))
        (widget-value-set widget
          (cons (widget-value child) (widget-value widget)))))
    (widget-setup)))

  
(defun dynamic-group-remove (widget)
  "Remove the last widget from the WIDGET group."
  (when-let ((children (widget-get widget :children)))
    (let ((inhibit-read-only t)
          ;; (child (car children))
	  )
      (save-excursion
        (goto-char (widget-get widget :from))
        (delete-region (point) (widget-get widget :to))
        (widget-put widget :children (cdr children))
        (dolist (c (reverse (widget-get widget :children)))
          (widget-create c))
        (widget-put widget :to (point))
        (widget-value-set widget
			  (mapcar 'widget-value
				  (widget-get widget :children)))
        (widget-setup)))))

  
(defun demo-dynamic-group ()
  "Test the dynamic-group widget."
  (interactive)
  (switch-to-buffer "*Dynamic Group Demo*")
  (kill-all-local-variables)
  (let ((inhibit-read-only t))
    (erase-buffer)

    (widget-insert "* Dynamic Group Demo\n\n")

    ;; Now I create the `dynamic-group'.
    
    (let ((group (widget-create 'dynamic-group)))
      (widget-insert "\n")

      ;; The rest are just two buttons testing the widget's behavior,
      ;; invoking`dynamic-group-add' and `dynamic-group-remove'.
      
      (widget-create
       'push-button
       :notify
       (lambda (&rest _)
         (dynamic-group-add group 'string
			    :format "Text: %v\n"
			    :value (format "Item %d"
					   (1+ (length (widget-get group :children))))))
       "(+) Add Field (Click Anywhere)")
      
      (widget-insert " ")
      
      (widget-create
       'push-button
       :notify (lambda (&rest _)
		 (dynamic-group-remove group))
       "(-) Remove Last")
      
      (widget-insert "\n"))

    ;; Wrap everything up using the `widget-keymap' and `widget-setup'
    ;; functions.
    
    (use-local-map widget-keymap)
    (widget-setup)))


(provide 'emc-dynamic-group)

---

'(cheers)

Helping HEΛP Again! ... and Again!

In the heat of the summer (the coolest summer of the next ones), it is never a good thing to get an email from Xach telling you that "something does not compile on SBCL". In this case the issue was the usual, fascist STYLE-WARNING, that prevented a clean build on Quicklisp.

The fix was relatively easy, but it lead to a number of extra changes to properly test the fix itself.

Bottom line, a new version of HEΛP is available at helambdap.sf.net. Soon in Quicklisp as well.

Stay cool, hydrated and enjoy.

---

(cheers)

Helping HEΛP Again!

In a flurry of ... free time, I also went back to HEΛP and fixed a few bugs that were exposed by some of the things I did with CLAST. Recording the documentation strings from the pesky

  (setf (documentation 'foo 'function) "Foo Fun!")
  

are now all working as expected, at least at top-level and within PROGN-like constructs, e.g., EVAL-WHEN.

Meanwhile, I also updated the documentation and the web page adding a few caveats about how to run the DOCUMENT function, and how to work around issues I have seen in my (not so) extensive tests.

Of course, I put my money where my mouth is: the HEΛP documentation web pages are built with HEΛP.

---

(cheers)

CLAST reworked

Prompted by a post on one of the various Common Lisp fora, I finally got my act together and went back to CLAST, i.e., the Common Lisp Abstract Syntax Tree library that I had in the works for ... some time.

The library has an interesting origin, which I will recount in a different post. Suffice to say that eventually I needed a code walker which did a few complicated things. NIH sydrome immediately kicked in.

The main think I needed were functions inspecting code, as in the example below.

cl-prompt> (clast:find-free-variables '(let ((x 42)) (+ x y)))
(Y)

To achieve this (apparently) simple goal, a (mostly) portable environment library had to be developed and a full set of AST node structures had to be provided.

The result is now finally ready for prime time. In Lispworks you can also see how things actually get parsed. As an example, the picture below shows the result of the following command.

cl-prompt> (clast:parse '(loop for i in '(1 2 3)
                               count (oddp (+ qd i)) into odds))
#<LOOP-FORM  24ECE77F>
NIL

Please try it, report bugs, blast my design choices and suggest improvements.

Thank you

---

(cheers)

ELS 2024 in Vienna

I just got back from the 2024 European Lisp Symposium in Vienna. After many years it was good to meet again in person with so many talented and interesting people (all interested in parentheses).

Many, many thanks to Beppe Attardi, Philipp Marek, Georgiy Tugai, Yukari Hafner, and, especially, Didier Verna, for pulling this together.

A Rant about R and Python (and anything with a-lists, p-lists, tuples and "dictionaries").

R and Python come from Lisp (R for sure, Python will deny it). Early Lisp. Even before the first edition of "AI Programming" by Charniak, Riesbek and McDermott.

At that time, there were a-lists and p-lists. Then Charniak, Riesbeck and McDermott taught us (not only them of course) how to create records in Lisp. You know... those things like:

    DECLARE
      1 STUDENT,
        2 NAME     CHAR (30),
        2 SURNAME  CHAR (50),
        2 ID FIXED DECIMAL (8),
        2 ADDRESS,
          3 STREET  CHAR (80),
          3 NUMBER  FIXED DECIMAL (5),
          3 CITY    CHAR (80),
          3 PRST    CHAR (20),
          3 ZIP     CHAR (10),
          3 COUNTRY CHAR (80);

Using a-lists the above may become:

    (defvar student '((name . "John")
                      (surname . "Blutarski")
                      (id . 42)
                      (address . ((street . "United State Senate")
                                  (number . 0)
                                  (city . "Washington")
                                  (prst . "DC")
                                  (zip . "20510")
                                  (country . "U.S.A.")))
                      ))

In Python you can do the following.

    student = {}    # a 'Dict'; i.e., a hash table.
    student['name'] = "John"
    student['surname'] = "Blutarski"
    student['id'] = 42
    student['address'] = {}
    student['address']['street'] = "United State Senate"
    student['address']['number'] = 0
    student['address']['city'] = "Washington"
    student['address']['prst'] = "DC"
    student['address']['zip'] = "20510"
    student['address']['country'] = "U.S.A."

Not that you must, but surely you can; and this, as in the case of R below, is the root of my rant.

In R you use lists; a misnomer for something that is essentially a dictionary like in Python, patterned, ça va sans dire, after a-lists.

    student = list()
    student$name = "John"
    student$surname = "Blutarsky"
    student$id = 42
    student$address = list()
    student$address$street = "United State Senate"
    student$address$number = 0
    student$address$city = "Washington"
    student$address$prst = "DC"
    student$address$zip = "20150"
    student$address$country = "U.S.A."

This of course gives you a lot of flexibility; e.g., if - in the middle of your code - you need to deal with the student's nickname, you just write the following.

In (Common) Lisp:

    (defun tracking-student ()
        ...
        ...
        (setf student (acons 'nickname "Bluto" student))
        ...
        )

In Python:

    def tracking_student():
        ...
        ...
        student['nickname'] = "Bluto"
        ...

In R:

    tracking_student <- function() {
        ...
        ...
        student$nickname = "Bluto"
        student <<- student     # Yes, R scoping rules are ... interesting.
        ...
    }

The example is relatively simple and innocuous, but it has some consequences when you have to actually read some code.

The problem I have (it may be just me) is that this programming style does not give me an overview of what a data structure (a record) actually is. The flexibility that this style allows for is usually not accompanied by the necessary documentation effort telling the reader what is going into an "object". The reader is therefore left wondering, while taking copious notes about what is what and where.

Bottom line: don't do that. Use defstruct and defclass in CL, classes in Python, struct in Julia, etc. etc. etc. In R, please document your stuff.

You may feel that your code is less malleable, but, in the end it becomes easier to read. At least for me. Sorry.

---

(cheers)