just a quick summer update about some documentation cleanup and some checks on debugging HEΛP.
Have a look at the (small) changes and keep sending feedback.
It has been a long time since I posted something here. I have been busy with my day job and bogged down in a major rewrite of something (more on this hopefully very soon now (tm)) that is full of rabbit's holes.
I was able to get out of one of these rabbit's holes with this little hack I cooked up that allows you to, possibly, write more concise code.
This little hack introduces two handy Common Lisp
LETV* that allow you to mix
MULTIPLE-VALUE-BIND forms in
a less verbose way. The amount of indentation needed is also reduced.
The syntax of
LETV*) is very
"loopy", with a nod to SML/OCaml/F#/Haskell/Julia. The current syntax
is the following:
letv ::= 'LETV' [vars] [IN [body]] letvstar ::= 'LETV*' [vars] [IN [body]] vars ::= var | var vars var ::= ids '=' <form> [decls] decls ::= decl | decl decls decl ::= OF-TYPE idtypes ids ::= <symbol> | '(' <symbol> + ')' idtypes ::= <type designator> | '(' <type designator> + ')' body ::= [<declarations>] <form> *
(I know: the grammar is not completely kosher, but I trust you will understand it.)
The two macros expand in forms that mimic the semantics of
MULTIPLE-VALUE-BIND. All type declarations, if present,
are properly handled via
LETV expands into a
LET, with variable
LETV* expands into a form that is an interleaving of
The library exports only the two symbols
LETV*. The other "symbols" mentioned (
OF-TYPE) are checked as in
LOOP; therefore you can use different styles to write
your code, as you would when writing
The library is available here.
(letv x = 42 in (format t "The answer is ~D.~%" x))
(letv x = 42 of-type fixnum in (format t "The answer is ~D." x))
(letv (v found) = (gethash 'the-key *my-hash-table*) in (if found (format t "Found THE-KEY, doing stuff.~%") (error "THE-KEY not found.")))
(letv (v found) = (gethash 'the-key *my-hash-table*) of-type (fixnum boolean) x = 42 in (if found (format t "Found THE-KEY, adding to the answer ~D.~%" (+ v x)) (error "THE-KEY not found.")))
(letv* (v found) = (gethash 'the-key *my-hash-table*) of-type (fixnum boolean) x = (when found (+ v 42)) in (if found (format t "Found THE-KEY, adding to the answer ~D.~%" x) (error "THE-KEY not found.")))
(letv x = 42 of-type integer (v found) = (gethash 'the-key *my-hash-table*) of-type (fixnum boolean) in (if found (format t "Found THE-KEY, adding to the answer ~D.~%" (+ v x)) (error "THE-KEY not found.")))
(letv* (p found) = (gethash 'the-key *points-table*) of-type (point) (x y) = (if found (unpack-point p) (values :missing :missing)) in (declare (type point p) (type real x y)) ; Adding declarations here also works. (do-stuff-with x y) (do-things-with p))
All the examples are meant to illustrate the use of
LETV* are obviously not the first
macros of this kind floating around: others are available and all have
their niceties. But I never claimed not to suffer from NIH syndrome.
LETV* do not do "destructuring" or
"pattern matching". That is a different can of worms; but you can
check cl-unification for a
library (always by yours truly) that provides facilities in that
as many may know, I have been nursing (almost to death!) the Common Lisp Document Repository (CDR). Pascal Costanza et al., started the project many years ago and then I sat on top of it for many more.
I finally found some time to work on it and the result is a revamped site (cdr.common-lisp.dev) with the addition of stashing documents in a Zenodo Community (CDR), which has the benefit of producing a DOI for each write-up.
Moreover, Pascal Bourguignon, Michał "phoe" Herda and Didier Verna have agreed to become CDR editors. Many thanks to them.
So, if anyone wants to submit a "specification" for something of interests to the CL community, she/he is most welcome to do so. Just remember that good specifications are not so easy to write.
I have been bumping on a few "lists of Common Lisp libraries and tools" written by many people. I feel like rantin... pardon, blogging about this state of affairs.
First of all we have CLiki, which is a rather comprehensive list of CL libraries and whatnot, and then we have a few, unnamed, "state of the CL ecosystem", "preferred list of Common Lisp libraries", etc, etc.
I have nothing against people blogging or making lists of course, but I tend not to make generalized statements. Especially in order to avoid disrespecting some people's work, just by not knowing of its existence. This is a hint.
So, in order to proceed with my ran... blog post, here is the list of CL libraries I use. Turns out there is quite a bit of NIH syndrome here, but I never claimed not to suffer from it. Also remember that I have a Mac, a Windows and a Linux system at hand. I always try to have stuff that works on all of them.
... more fixing and, ça va sans dire, more creeping features.
I got prodded to integrate HEΛP with other tools; mostly, of course, ASDF. A simple solution was to define a document-op for a system. After jumping through a few hoops, the solution was to use the :properties of a system to pile up arguments for the main HEΛP document function (well, only one for the time being). Bottom line, suppose you have:
(asdf:defsystem "foosys" :pathname #P"D:/Common Lisp/Systems/foosys/")
now you just issue
(asdf:operate 'hlp:document-op "foosys")
and the documentation for the system "foosys" will appear in the "docs/html/" subfolder.
If you want to pass a title to the document function, you set up your system as:
(asdf:defsystem "foosys" :properties (:documentation-title "The FOO Omnipotent Tool") :pathname #P"D:/Common Lisp/Systems/foosys/")
and the parameter will be used (instead of the bare system name).
It works! 😁
Some more fixing and more extensions may be needed (hlp:document takes a lot of parameters) but it is already usable.
All the necessary bits and pieces are in the HEΛP repository, and they should get into Quicklisp in the next release.
Just a quick note for people following these... parentheses.
I have carved out some time to do some more Lisp hacking and this lead me to look at the very nice usocket library (I want to do some network programming). The usocket library documentation page has a bit of an "old" and "handcrafted" look and feel to it, so I tried to produce a version of the documentation with help from my HEΛP library.
As an example, usocket uses the following idiom to set some of the documentation strings.
(setf (documentation 'fun 'function) "Ain't this fun?")
This is perfectly fine, but it needed some extra twist to get HEΛP do what is, IMHO, the right thing: in this case it meant ensuring that the lambda list of the function was properly rendered in the final documentation.
Apart from that, a few not so nice buglets were exposed in the code parsing lambda lists. The result is that now the logic of that piece of code is simpler and somewhat cleaner.
So, if you want to get HEΛP to document your Common Lisp code, give it a spin.
programmers may write
with-something overt their careers; the
language specification itself is ripe with such constructs: witness
with-open-file. Many other libraries also
introduce a slew of with- macros dealing with this or that case.
So, if this is the case, what prevents Common
Lisp programmers from coming up with a
It appears that the question has been answered, rather satisfactorily, in Python and Julia (at least). Python offers the with statement, alongside a library of "contexts" (Python introduced the with statement in 2005 with PEP 343) and Julia offers its do blocks.
In the following I will present WITH-CONTEXTS, a Common Lisp answer to the question. The library is patterned after the ideas embodied in the Python solution, but with several (common) "lispy" twists.
Here is the standard - underwhelming - example:
(with f = (open "foo.bar") do (do-something-with f))
That's it as far as syntax is concerned
var =' being optional, obviously
not in this example; the syntax was chosen to
be loop-like, instead of using
Python's as keyword). Things become more
interesting when you look under the hood.
Traditional Common Lisp with- macros expand in variations of unwind-protect or handle-case (and friends). The example above, if written with with-open-file would probably expand into something like the following:
(let ((f nil)) (unwind-protect (progn (setq f (open "foo.bar")) (do-something-with f)) (when f (close f))))
Python generalizes this scheme by introducing a enter/exit protocol that is invoked by the with statement. Please refer to the Python documentation on contexts and their __enter__ and __exit__ methods.
In order to introduce a with macro in Common Lisp that mimicked what Python programmers expect and what Common Lisp programmers are used to some twists are necessary. To achieve this goal, a protocol of three generic functions is provided alongside a library of contexts.
The WITH-CONTEXTS library provides three generic functions that are called at different times within the code resulting from the expansion of the onvocation of the with macro.
Given the protocol (from now on referred to as the "EHE-C protocol"), the (undewhelming) "open file" example expands in the following:
(let ((f nil)) (unwind-protect (progn (setq f (enter (open "contexts.lisp"))) (handler-case (open-stream-p f) (error (#:ctcx-err-e-41883) (handle f #:ctcx-err-e-41883)))) (exit f)))
Apart from the
gensymmed variable the expansion is
pretty straightforward. The function enter is
called on the newly opened stream (and is essentially an identity
function) and sets the variable. If some error happens while the
body of the macro is executing then control is passed to
the handle function (which, in its most basic form
just re-signals the condition). Finally, the unwind-protect has a
chance to clean up by calling exit (which, when
passed an open stream, just closes it).
One unexpected behavior for Common Lisp programmers is that the variable (f in the case above) escapes the with constructs. This is in line with what Python does, and it may have its uses. The file opening example thus has the following behavior:
CL-prompt > (with f = (open "contexts.lisp") do (open-stream-p f)) T CL-prompt > (open-stream-p f) NIL
To ensure that this behavior is reflected in the implementation, the actual macroexpansion of the with call becomes the following.
(let ((#:ctxt-esc-val-41882 nil)) (multiple-value-prog1 (let ((f nil)) (unwind-protect (progn (setq f (enter (open "contexts.lisp"))) (handler-case (open-stream-p f) (error (#:ctcx-err-e-41883) (handle f #:ctcx-err-e-41883)))) (multiple-value-prog1 (exit f) (setf #:ctxt-esc-val-41882 f)))) (setf f #:ctxt-esc-val-41882)))
This "feature" will help in - possibly - porting some Python code to Common Lisp.
Python attaches to the with statement the notion of contexts. In Common Lisp, far as the with macro is concerned, anything that is passed as the expression to it, must respect the enter/handle/exit. protocol. The three generic functions enter, handle, exit, have simple defaults that essentially let everything "pass through", but specialized context classes have been defined that parallel the Python context library classes.
First of all, the current library defines the EHE-C protocol for streams. This is the strightforward way to obtain the desired behavior for opening and closing files as with with-open-file.
Next, the library defines the following "contexts" (as Python does).
This should be a good enough base to start working with contexts in Common Lisp. It is unclear whether the Python decorator interface would provide some extra functionality in this Common Lisp implementation of contexts and the with macro.
The current implementation has a semantics that is obviously not the same as the corresponding Python one, but it is hoped that it still provided useful functionality. There are some obvious limitations that should be taken into account.
The current implementation of the library does not take into consideration threading issues. It could, by providing a locking-context based on a portable multiprocessing API (e.g., bordeaux-threads).
The Python implementation of contexts relies heavily on the yield statement. Again, the current implementation does not provide similar functionality, although it could possibly be implemented using a delimited continuation library (e.g., cl-cont).
The code associated to these documents is not completely tested and it is bound to contain errors and omissions. This documentation may contain errors and omissions as well. Moreover, some design choices are recognized as sub-optimal and may change in the future.
The file COPYING that accompanies the library contains a Berkeley-style license. You are advised to use the code at your own risk. No warranty whatsoever is provided, the author will not be held responsible for any effect generated by your use of the library, and you can put here the scariest extra disclaimer you can think of.
The with-contexts library is available on Quicklisp (not yet).
The with-contexts library. is hosted at common-lisp.net.