20220121

CDR is Next...

 Hi

as many may know, I have been nursing (almost to death!) the Common Lisp Document Repository (CDR).  Pascal Costanza et al., started the project many years ago and then I sat on top of it for many more.

I finally found some time to work on it and the result is a revamped site (cdr.common-lisp.dev) with the addition of stashing documents in a Zenodo Community (CDR), which has the benefit of producing a DOI for each write-up.

Moreover, Pascal Bourguignon, Michał "phoe" Herda and Didier Verna have agreed to become CDR editors.  Many thanks to them.

So, if anyone wants to submit a "specification" for something of interests to the CL community, she/he is most welcome to do so.  Just remember that good specifications are not so easy to write.

20220105

My List of Common Lisp Libraries and SW 2022 (Plus a Couple of Other Things)

Hello

I have been bumping on a few "lists of Common Lisp libraries and tools" written by many people.  I feel like rantin... pardon, blogging about this state of affairs.

First of all we have CLiki, which is a rather comprehensive list of CL libraries and whatnot, and then we have a few, unnamed, "state of the CL ecosystem", "preferred list of Common Lisp libraries", etc, etc.

I have nothing against people blogging or making lists of course, but I tend not to make generalized statements.  Especially in order to avoid disrespecting some people's work, just by not knowing of its existence.  This is a hint.

So, in order to proceed with my ran... blog post, here is the list of CL libraries I use.  Turns out there is quite a bit of NIH syndrome here, but I never claimed not to suffer from it.  Also remember that I have a Mac, a Windows and a Linux system at hand.  I always try to have stuff that works on all of them.

Distribution Systems

I use, obviously, Quicklisp.  It works.  It can be improved, but hey!  Let's just give Zach our love! 💓 (And money!)

Implementations

  • I work mostly on Lispworks.  I have the luck to be able to afford the Enterprise edition (or better, my funding does) and I am very happy with it.  The folks at Lispworks know that I can be a pest (environments? rounding modes?) but they are just doing a fantastic job.
  • I use, of course, SBCL.  The implementation is rock solid and it has all the bells and whistles you need for a world class system.  Recompiling and deploying for SBCL usually uncovers bugs and potential pitfalls in your code.  I have a small rant about it though.  Guys, do not write code that "works on SBCL" and just assume all is fine. It is not.
  • I have also CMUCL installed and I use it as well.  It still has a few good things to it.
  • Next I use Armed Bear Common Lisp, an excellent testbed for checking portability.
  • Of course, I have the free edition of Allegro CL: the other excellent commercial alternative.
  • Last but not least (with memories going back decades) I have CCL installed and always ready to use.
  • I am very intrigued by CLASP, but I admit, I have not had the strength to install it (too much of a production; remember that I try to get things working on three platforms); any help will be appreciated.
  • Same for the latest incarnation of ECL.  Sorry guys, I will get back to installing it soon.
  • Finally, we always have CLisp: old but good.
I know I forgot many.  Apologies for my senescence.

System Building

If you checked any of my libraries, you will have noticed that I still have .system files everywhere.  I use ASDF of course, but I have been nursing (with Madhu) a version of MK:DEFSYSTEM; it is clunkier (and old) internally, but I feel it has a simpler and more straightforward interface, especially when it comes to its footprint on your system (i.e., somewhat simpler working of the registry).

Concurrency

I use Bordeaux Threads for portability.  There are a few open issues about it that are of very difficult solution but it is Good Enough (it has a couple of warts: a bad .asd file for example - but this is a separate rant, pardon, blog post).  Of course I also like very much Lispworks multiprocessing library, especially mailboxes, but I understand that not everybody has access to it.

Testing

I must say that none of the libraries I checked does ONE thing I really need: running something and stopping after a timeout (see the open issue I mentioned just before for Bordeaux Threads).
Having said that, I use FiveAM; I have to muck a bit around it to get the timeout I need, and the documentation, especially about macr... pardon, fixtures is lacking, but it does the job.  I am experimenting with Parachute, but have not made the switch yet.

Portability

Most of my libraries try to be portable.  One major library that helps you a lot with this thankless job ("let's have filename case insensitive anyone?") is of course UIOP.  You have it pretty much by default, but it would be nice if it were eventually decoupled from ASDF.
Another little library I advocate (yes, Virginia, here they come!) is CLAD: it does not do much, but it gives you a bedrock upon which to build your work.

Editing

I use the Lispworks IDE or the Editor (as in "thou shalt not have other Editor than...") with SLIME.  End of story.

HTML Generation

Here I start with the rantin..., thoughtful exposition of various libraries.
To generate (X)HTML (or HTML5) I use (X)HTMΛ.  It is a pretty solid library with a couple of bells and whistles: mainly, it has an object model and it leverages the sorrily underleveraged Common Lisp pretty printer to produce readable HTML.  The web page for (X)HTMΛ is produced using itself as a building block; see below.

Web Server and Client

In this case I stick with the tried and true Hunchentoot and its counterpart Drakma, although for simpler, one-shot things I do use Lispworks facilities when needed.

Documentation Generation

As I said, all my libraries suffer from a severe case of NIH, which is reflected in my use of the hacked up HEΛP documentation system.  It is still not perfect, but, again, by using (X)HTMΛ and, again, leveraging the pretty printer, I find that the results are quite pleasant (for the neapolitans: ogni scarrafone...).  Of course, if you agree to format your doc string in a Hyperspec way.  How can it be improved?  Four things.
  1. Incorporating Eclector as a "reader" (as of now the tool uses more than a kludge to get past the non standardized way of dealing with READER- and PACKAGE- errors).
  2. Finishing the HTML5 generation scheme.
  3. Adding Texinfo generation (as per Didier Verna's declt library).
  4. Fixing cross-referencing, especially with standard CL (cfr., Hyperspec).

Mathematics

Why did I start HEΛP?  Many years ago (or many Kgs ago) a post on comp.lang.lisp stated that you could shadow +, -, *, and / in a package and make them generic functions (as an aside, I recently found out that the idea of generics dates back to PL/I - but this is a different rabbit's hole).
The result was what is now a forever embryonic Common Math, which led to my forays in "let's do what R can do" -- Ρ (greek Rho) -- and to the stuff I am talking about in "Why you cannot write an Interval Arithmetic Library...".  All of this another rabbit hole: bottom line, this is what I am working on and off right now, apart from my day job.

Other Libraries

Laziness as a Way of Life

Of course, rabbits' holes are never linear.  They branch in several directions at once.  E.g., once you look at other "modern" languages (I am partial to Haskell) you want to have some of these features in Common Lisp as well); hence the CLAZY library (laziness as a way of life).

Code Handling

Needless to say, CLAZY eventually needed some proper code walking, or better it needed some proper Abstract Syntax Tree (AST) handling, which led to the CLAST library: this last library allows you to code walk and to properly inspect a piece of Common Lisp code, striving very hard to portably manage the environments that were present in CLtL2 but did not make it in the ANSI spec.

Programming Common Lisp

As we all know, Lisp is like a ball of mud.  Over the years I threw a lot of mud at it.
Some of the mud I liked the most is embedded in the following libraries
  • definer is a small hack to have Pythonesque def available in Common Lisp, of course, in an extensible way (documentation being updated soon).
  • cl-unification is a full blown unification machine for Common Lisp objects and "templates"; not as fast as a simpler pattern matcher (like several listed in CLiki Pattern Matching), but very general (it also relies on CL-PPCRE).
  • cl-enumeration was the first full blown Java-like enumeration/iterator library for Common Lisp; it works.
  • defenum is another Java-ism ported to Common Lisp; it essentially creates enums that work like Java, i.e., with quite a bit of underlying machinery.  Of course you do not have Haskell data definitions available, but it was fun to write.
  • with-contexts was a bout of Python-envy, but then again, I think I was able to show that you really can program Common Lisp in a fun way.

Literary Programming in Common Lisp

One final library I am quite fond of expresses my love of (certain) books.  And to deal with books, you need a Library and a Librarian.  Hence the OOK compiler in Common Lisp.  Don't say Monkey!

Not Done Yet...

Although I am dabbling with Julia (a Common Lisp in a drag) and Rust (I have a day job which also involves programming languages and not only Cancer Research) I still have a lot of "not-quite-there" stuff I wrote in Common Lisp.
All I can say about it is... stay tuned,  I do put out stuff and I do pester people about Common Lisp.  You may find some of the stuff interesting.



(cheers)