LISP -- Berry Web Blog -- page 0

Remotely Modifying a Running Program Using Swank

Posted: 2017-10-28. Modified: 2017-10-28. Tags: howto, emacs, LISP.

One of the strengths of Common Lisp is that it includes support for dynamically redefining classes and functions at run-time, while keeping data intact. Theoretically at least, this should enable support for keeping programs continually running in production while changing pieces of them – "Common Lisp Recipes" by Edi Weitz includes the following disclaimer about this functionality

If you'e ever talked to experienced Lispers, you've probably heard "war stories" of huge and complex systems to which substantial modifications were applied while they kept running and without interrupting the services they provided. Although this sometimes has to be taken with a grain of salt, it is in fact true that many COMMON LISP features were designed from the ground up to be dynamic in the sense that they can be changed at run time. This includes CLOS, where an object can change from one class to another, and where classes can be modified, although they already have objects "hanging off" of them.

– "Common Lisp Recipes" by Edi Weitz, section 13-8

My understanding is that some of this functionality at least is difficult/nonstandard to replicate in other languages such as Python and Java (please feel free let me know that I am wrong if that is the case!).

Anyway, I would like to remotely interact in lisp with remote instances of my current project I am working on – dumb-mq – so I figured it would be helpful to start with a small example of remote connection/redefinition.

This example uses sbcl and SLIME-mode in emacs, but it should work for other lisp implementations and other tools that support the swank protocol as well. The easiest way to get SBCL and emacs set up that I am aware currently of is to download the excellent Portacle IDE by Nicolas Hafner. Alternatively just figure out how to install them yourself.

Write the following lisp file: swankdemo.lisp

;; a little common lisp swank demo
;; while this program is running, you can connect to it from another terminal or machine
;; and change the definition of doprint to print something else out!
;; (ql:quickload :swank)
;; (ql:quickload :bordeaux-threads)

(require :swank)
(require :bordeaux-threads)

(defparameter *counter* 0)

(defun dostuff ()
  (format t "hello world ~a!~%" *counter*))

(defun runner ()
  (bt:make-thread (lambda ()
                    (swank:create-server :port 4006)))
  (format t "we are past go!~%")
  (loop while t do
       (sleep 5)
       (dostuff)
       (incf *counter*)
       ))

(runner)

You can run this program as follows:

sbcl --load swankdemo.lisp

The program will run indefinitely, printing out a message every five seconds and incrementing its counter. Imagine instead that this was program was accepting connections indefinitely and was providing an important service.

By default swank will accept connections only from localhost – if you would like to connect from a different computer you can use ssh tunneling to forward the port on the remote machine to a port on your local computer. For example

ssh -L4006:127.0.0.1:4006 username@example.com

will securely forward port 4006 on the server at example.com to your local computer's port 4006.

Let's connect to the program. Fire up emacs, type M-x slime-connect, at the prompts select 127.0.0.1 (the default) and port 4006 (type this in). If all went well, you are now connected to the remotely running lisp program! Just to check, see if you can retrieve the current value of the counter:

CL-USER> *counter*

Now let's say you want to change the definition of dostuff and reset the counter while you are at it. Type in the following either in an emacs scratch buffer, select it, and send it to the remote lisp program using M-x slime-eval-region (or an alternate method).

(defun dostuff ()
  (format t "goodbye world ~a!~%" *counter*))
(setf *counter* 0)

Observe swankdemo's output in the console – you will see the output change and the counter be reset. Success!

You can do more complicated redefinitions and changes – refer to the Common Lisp Standard (draft) section 7.2 and 7.3 for some information on modifying objects at run-time.


How to Designate Single Window for Popup Buffers in Emacs

Posted: 2017-08-20. Modified: 2017-08-20. Tags: howto, emacs, emacstips, LISP.

This blogpost is inspired by the approach found here.

One of the things that used to annoy me about programming in emacs with SLIME mode (common lisp), is that SLIME would frequently choose to open up a popup buffer in one of the windows I was trying to use for some other task. For instance, various actions in SLIME will open up a Completion Buffer, Debugger Pane or an Inspection Buffer. I eventually realized that what I really wanted was to designate a given window where all emacs popups would open by default, so my train of thought in the other windows can remain undisturbed. Below is some Emacs Lisp code that enables this functionality:

(defun berry-choose-window-for-popups ()
  "run with your cursor in the window which you want to use to open up 
   all popups from now on!"
  (interactive)
  (set-window-parameter (selected-window) 'berrydesignated t)
  (berry-setup-popup-display-handler))

(defun berry-setup-popup-display-handler ()
  "adds an entry to display-buffer-alist which allows you to designate a window 
   for emacs popups. If the buffer is currently being displayed in a given 
   window, it will continue to use that window. Otherwise, it will choose your 
   designated window which should have been already set."
  (interactive)
  (add-to-list 'display-buffer-alist
	       `(".*" .
		 ((display-buffer-reuse-window
		   berry-select-window-for-popup
		   display-buffer-in-side-window
		   )
		  .
		  ((reusable-frames     . visible)
		   (side                . bottom)
		   (window-height       . 0.50)))
		 )))

(defun berry-select-window-for-popup (buffer &optional alist)
  "Searches for the a window which the 'berrydesignated parameter set.
    Returns the first such window found. If none is found, returns nil."
  (cl-block berry-select-window-for-popup
    (let* ((winlist (window-list-1 nil nil t))
	   (outindex 0))
      (while (< outindex (length winlist))
	(let ((candidate (elt winlist outindex)))
	  (if (eql t (window-parameter candidate 'berrydesignated))
	      (progn
		(set-window-buffer candidate buffer)
		(cl-return-from berry-select-window-for-popup candidate)))
	  (cl-incf outindex)
	  ))
      nil)))

(defun berry-clear-popup-setting ()
  "clears the 'berrydesignated flag on all windows, thus removing the designation 
   of any given window to host popups. also removes the popup handler registration"
  (interactive)
  (cl-loop for window in (window-list-1 nil nil t) do
	   (set-window-parameter window 'berrydesignated nil))
  (pop display-buffer-alist)
  )

My usual window layout when programming in Emacs looks like the following (note that in emacs a window is more like a frame in most other environments):

+-----------------+
|        | second |
|        | code   |
|primary | window |
|code    |--------|
|window  | REPL & |
|        | POPUPS |
|        |        |
+-----------------+

So what I do after opening all the windows I want is I put my cursor in the "REPL & POPUPS" window and run berry-choose-window-for-popups. The content of my other windows remains undisturbed by IDE functions unless I tell the computer to change buffers in one of those windows.


Why I prefer emacs+evil to vim

Posted: 2015-10-22. Modified: 2015-12-21. Tags: LISP, Opinion, emacs.
  1. Variable width font support.

    I studied my own reading speed on a variety of documents, and found that I read variable width text about 15% faster than fixed width text. I don't mind writing code in fixed-width fonts, but if I am going to use one text editor for everything, then I very much appreciate having variable-width support.

  2. org-mode > markdown

    Org-mode allows you to write structured content within Emacs, and supports the writer with a variety of useful features and tools. Besides the rich editing, export, and extension possibilities offered by emacs org-mode itself, I find that the org format is superior to markdown for my purposes. Two primary reasons for this are that org provides syntax (such as drawers) for defining all sorts of metadata about your text, and also that org is designed in such a way that it is basically equally usable as variable-width text and fixed-width text. In particular I dislike the extent to which markdown relies on fixed-width text for its display features.

  3. evil >= vim.

    Emacs "Evil" mode pretty much provides a superset of commonly used vim functionality. Evil supports all the commonly used vim editing commands, which allows you to take advantage of vim's ergonomic design as you edit text. Evil actually improves on some vim features – for example, search and replace shows replacements being entered as you type them. Evil also provides access to the full power of Emacs just one M-x away – you get the ergonomics of vim with the power of emacs when you want it.

  4. superior extensibility (> emacs-lisp vimscript)

    Especially for a lisp fan such as myself, Emacs Lisp seems a superior language to Vimscript. Emacs Lisp is kind of like a baby version of common lisp, and supports a rich number of features on its own and with the addition of third-party libraries.

    However the real advantage of Emacs in extensibility is the fact that the majority of emacs is actually written in Emacs Lisp. Emacs' Github mirror indicates that the ratio of elisp to C in the project is ~4:1. I believe most of the C stuff is quite low-level, and is related to multiplatform support, core rendering, and the like. On the other hand Vim's Github repo indicates that Vim's vimscript C ratio is ~0.8:1. Since the vast majority of emacs is written in emacs-lisp, in the emacs environment you can very easily dive in to functionality from within emacs to understand and/or modify how things work.

  5. "self documenting" w/ C-h f, C-h k

    In the Emacs Manual it is stated that Emacs is "extensible, customizable, self-documenting real-time display editor". One feature I really like about emacs is the "self-documenting" component of that description. Emacs makes it very easy to look up the docstring of a given function or command, easy to determine what a keyboard shortcut does, easy to determine what shortcuts are available, easy to determine what functionality is present from various modes, and more. In short, emacs makes it possible to spend a great deal of time within emacs without having to go online to look up how to use a given function or tool.


Common Lisp Standard Draft

Posted: 2015-10-11. Modified: 2017-04-26. Tags: LISP, programming.

UPDATE 2017-04-26: Updated this page to link to my new and improved version of the ANSI CL standard draft which now includes a pdf sidebar outline.

Below is a link to a build of the publicly available ANSI CL standard draft, which has been somewhat modified to include a pdf sidebar outline. The sidebar makes it much easier to navigate this 1200 page plus document!

ANSI CL standard draft.

If you are curious to see what modifications I had to make to the tex sources to obtain the above pdf, please refer to the gitlab repository hosted here.

1 Backstory

The Official Common Lisp standard ANSI X3.226-1994 (now referred to as ANSI INCITS 226-1994) is available from ANSI for a cost of $60. However, besides the fact that this document is expensive, it is also known that this document is actually a low-quality scan of the original.

Many people apparently use the Common Lisp Hyperspec, but I personally find this document highly confusing and difficult to learn from in any meaningful way. It is from the early days of "hypertext" and employs so many links as to be basically unreadable in my opinion.

An alternative to the above two choices for documentation on the Common Lisp language is the final draft to the ANSI standard. According to Franz Inc the final draft of the ANSI standard differs from the official standard only in "formatting and boilerplate," and the final modifications are said to have "no technical implications". The final draft is licensed for "free use, copying, distribution". Tex sources of the individual chapters of the Common Lisp standard are freely available from CMU. Postscript copies of the individual chapters of the Common Lisp standard are freely available from the late Erik Naggum's website.

2 Archived Downloads and Notes

Note: as of 2017-04-26, I highly recommend the version above. These old revisions are here for historical purposes.

I have prepared two PDFs of the final Common Lisp draft standard for download.

I believe they can be treated as an authoritative resource on Common Lisp for the general user, and as a good alternative to the Common Lisp Hyperspec. Below I explain some of the alternative documentation sources for Common Lisp, and how it came to be that I am hosting a link to this document.

Draft Version A was created by concatenating Erik Naggum's ps files into a single pdf. It has two pages side-by-side at a time, and rather poor fonts in my opinion.

Draft Version B was created by recompiling the original Tex sources. The font quality is much higher, and the resulting document scrolls properly on-screen. Steps to reproduce my work are listed here, so that you can verify that I have not changed anything in the standard.

To access the sources I used to build the draft version B or to see the script I used to do the pdf generation, please see my repository clstandard-build.


The Case for LISP

Posted: 2015-09-15. Modified: 2016-01-25. Tags: LISP, Opinion, programming.
  1. There are multiple ways to think about problems. *
  2. Given a real-world or programming problem, often if you find the right abstraction to model it you can make the problem vastly simpler and/or make your software vastly higher quality.
  3. Therefore, solving a problem well should involve trying to find the best abstraction possible.
  4. Programming languages influence your thought processes to match what they offer.
    1. "if you see enough nails everything starts to look like a hammer".
    2. See: Sapir Whorf hypothesis and Paul Graham: Succinctness is Power.
  5. Lisp offers the flexibility to adapt to ANY paradigm, imposes the least constraints on your thinking, and therefore is likely often the best language to use to approach a problem. **

\* I hold the idea that there are multiple ways to think about problems to be fairly obviously true. But in case you don't agree yet, here is some justification. An example of a problem which can be profitably approached from multiple perspectives is "dynamic programming". I can either approach a dynamic programming problem using a top-down recursive viewpoint, or look at it from the bottom up as filling out values in a table. I can reason about DP as either constructing a DAG of subproblems or as filling out a table of precomputed values. There are MULTIPLE WAYS to think about the problem

\** One useful corollary to the idea that different paradigms are good for different applications is the common wisdom about optimal applications for functional and object-oriented programming. Functional languages are often said to be good for problems which resemble computation, with a defined input and output (like compilers). OOP languages are said to be good for problems where you model the natural world as objects. Lisp, of course, lets you choose between object-oriented and functional programming, and lets you implement entire new language structures within the language itself to extend the language to support most any paradigm you can think of.


Java vs Lisp

Posted: 2015-09-15. Modified: 2015-12-21. Tags: LISP, Java, Opinion, programming.

Java is the best for other people to understand your abstractions…

  • with the exception of poor data-structure initializers.
    • but you can get around this pretty easily by writing fclever initializers
  • Excessive OOP considered hazardous
  • Java's static typing provides an excellent source of computer-checked, always up-to-date documentation.

Lisp is the best for creating new abstractions quickly…

  • but lack of static typing means its sometimes difficult to figure out how to properly call code, etc…

Is python pseudocode?

Posted: 2015-09-15. Modified: 2015-12-21. Tags: LISP, Opinion, programming.

People often state that python is the most similar programming language to pseudocode (what you write if you are trying to convey a piece of software to other humans rather than a computer). I've translated numerous pseudocode algorithms from books into various languages, and I agree that python often makes this process natural.

However I think an argument can also be made for LISP as the most "pseudo-code" like language. Perhaps while Python has the greatest surface similarity to pseudocode, LISP has the greatest actual similarity to pseudocode.

Why do I claim that LISP is similar to pseudocode? It's appearance is often rather different, after all.

I make this claim because of the intrinsic linguistic flexibility of a language supporting structural macros. The power of pseudocode is not just that it doesn't conform to a particular bracketing convention or function naming convention – the power of pseudocode is that you can introduce arbitrary code constructs, and explain them to your readers later on.

For example, consider the "with-redeclarations" construct, which rebinds all the functions executing within it. Lisp is MUCH closer to the ideal of the programmer being able to create arbitrary constructs than python is, and there are many instances of where you could imagine a way of thinking about a problem in pseudocode and translate the idea approximately into lisp, but have to rework it completely for python.

Therefore I conclude that while python does often look very similar to pseudocode written in a book, lisp is in a way also very similar to pseudocode because you can dream up very flexible code constructs just as you would if you were explaining a problem to someone with no fixed programming language.


The three Es of Text-editing

Posted: 2015-09-15. Modified: 2016-12-21. Tags: LISP, Opinion.

1 Efficiency

You want to be able to perform editing tasks quickly and use an optimized workflow.

2 Ergonomics

You want to minimize strain on your wrists and hands, primarily by conserving movement.

3 Extensibility

Since you are a programmer, you want to be able to program and customize your environment to meet needs as they arise. Add and remove features to optimize your workflow.

Each of these features is in some way linked to the other two, but at the same time each of these features is an independent goal to achieve.

[CB 8/3/2015] I plan to post some articles about how I think these goals can be achieved quite well with Emacs+Evil soon. Note that emacs additionally supports variable-width text, which appears to increase reading efficiency by ~15%.

Atom Feeds:
Tags: