Getting started with Common Lisp — Part 1: Introduction

This is the first in a series of posts that attempt to walk through a bunch of stuff I didn’t know when starting out learning Common Lisp (CL). Although code examples abound in the literature and on the web, not many sources explain how to compile, debug and turn them into programs. This series intends to collect that information from my experiences so far and to help others get up and running. It is very much a “work in progress” so I’d appreciate feedback.

Part 1 (this part) gives background information on how and why developing in Lisp differs from other programming languages.

Part 2 gives tips on choosing a Common Lisp implementation, a (very quick) look at installation, and how to use the REPL (Read-Eval-Print Loop: Lisp’s interactive command line).

Part 3 explains some useful functions for program development and a little bit about Lisp quoting rules.

Part 4 walks through writing a simple program (a program that prints a Mandelbrot set in asterisks) from first principles.

Part 5 describes the process of loading Common Lisp source files and compiling them for greater execution speed.

Part 6 introduces the Quicklisp package manager and describes how to install it.

In Part 7, we modify the program to use a library to save the Mandelbrot set as a PNG image file, and show how to run the program from the command line.

In Part 8, we show how to increase the program’s performance by adding type annotations.

Finally, Part 9 gives further reading material and on-line reference information.

In this part

Why the weird syntax?

The obvious distinctive feature of a Lisp family language is the way it looks; all those parentheses. Larry Wall has commented that Lisp has the visual appeal of oatmeal with finger nail clippings mixed in (although Larry created Perl so some may say it’s a case of the pot calling the kettle black!). Those unfamiliar with Lisp invariably ask why? There have been efforts to create Lisp-like languages with more familiar notation (such as Dylan) but none have really caught on.

It turns out that the weird-looking syntax (a nested list representation called s-expressions) is key to much of Lisp’s power and flexibility. Most programming languages make a clear distinction between built-in facilities and routines created by the programmer, and this is ingrained in the language’s syntax. With Lisp, however, the syntax is minimal (basically, the code is written directly as a syntax tree). Functions created by the programmer have the same syntax as built-in functions, and apart from a few core functions with special behaviours (so-called special forms), are treated in the same way. This blurs the distinction between the program and the language itself; writing a program in Lisp can be viewed in a sense as extending the language. What’s more, data are also represented as s-expressions. This allows Lisp programs to generate lists of data and turn them into code (the mechanism behind Lisp macros); moreover, this can be done at runtime! Lisp is therefore sometimes called a “programmable programming language”.

Slava Achmechet has written very well on this topic in his defmacro blog post The Nature of Lisp.

Unfortunately, s-expressions are often a major stumbling block to learning Lisp which puts many people off. All I can say is that with experience (plus careful source code layout and a bracket-matching editor) one gets used to it, and you read the indentation and patterns rather than count brackets; but I realise that some may never do. (Personally, I find C++ using templates highly unreadable, and dislike ALGOL-style syntax as verbose, but I’m a Perl hacker. Horses for courses.)

Developing in CL

Most compiled languages consist of a compiler, a few “standard” libraries and a small runtime support system. Programs are prepared as source files using a text editor. These are fed one by one to the compiler to produce object files which are linked together with libraries and the runtime support system to create an standalone executable file. When the program is run, the executable is loaded into memory by the operating system, the runtime sets up the environment (initialising memory and data structures and setting up I/O), starts the application by calling the ‘main’ subroutine and then gets out of the way.

Common Lisp, on the other hand, is an environment that includes a huge number of built-in and library functions, a garbage collector, an interpreter, a compiler, a debugger, and an interactive comnmand line known as the REPL (Read-Eval-Print Loop), all rolled into an image. Writing an application basically extends the core image by adding new functions to it. Functions may be entered directly at the REPL, or prepared as source files or precompiled object files (or libraries) and loaded into the image when required. The extended image may also be saved as a new image that is loaded when you next start CL.

The Smalltalk language also uses this concept of image-based persistence.

An characteristic of this approach is that the entire environment is available at runtime. Rather than a failing program unceremoniously dumping core and crashing, CL programs cause an exception which allows control to return to the REPL. The user or programmer then has an opportunity to inspect the program, alter its state, and if the program uses a feature called restarts, even resume execution once it has been fixed.

The REPL allowed the Remote Agent spacecraft software developed by the NASA Jet Propulsion Laboratory to be debugged and fixed in flight. Unfortunately, the future for LISP at JPL did not have a happy ending, but their efforts led to what is now ClozureCL.

With Common Lisp, the interactive enviroment encourages a different style of development to the traditional edit-compile-debug loop. A new function is created in a source file using a text editor, then copy-and-pasted into the REPL. If the function has a syntax error it is rejected and can be fixed up immediately. Once the function is entered it becomes part of the Lisp environment and can then often be tested and debugged interactively in situ. Once the function is working, the programmer moves on to the next function. In this way, development proceeds bottom-up building on a tested foundation. Once a complete source file is developed and debugged (as a module for example), it can be precompiled for faster loading and execution.

An advantage of using the functional programming methodology and creating pure functions as far as possible is that it makes the functions much easier to test — since there are no side effects, the behaviour of a function depends only on its inputs, so it can often be tested with just a few general and corner cases. This means that programs can be developed bottom-up on a tested, stable foundation.

The next part of this series will take you through installing a Common Lisp and introduce you to using the REPL and some of its facilities.

One thought on “Getting started with Common Lisp — Part 1: Introduction

  1. Pingback: Some Common Lisp Resources | Irreal

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s