Predicate Dispatch

A maybe-interesting paper by Aaron Mark Ucko, “Predicate Dispatching in the Common Lisp Object System, argues that predicate dispatching gives the best of the usual CLOS-style type dispatching and the ML/Haskell style pattern matching worlds.

(defpdmethod hailstone (n)
  ((evenp n))
  (/ n 2))

(defpdmethod hailstone (n)
  ((oddp n))
  (+ (* n 3) 1))

(defun hsseq (n)
  (if (= n 1)
    ’(1)
    (cons n (hsseq (hailstone n)))))

The Best Lisp

In a comp.lang.lisp thread about “the best product (or freeware) for developing Lisp applications”, Marco Baringer gave a pretty concise reply [seen at Bill Clementson's weblog], one that put a bit more of a stake in the ground and is therefore somewhat more useful than the true, but unwelcome “it depends”.

I took Marco's answer and shortened it even further, extracting what I felt was the most important advice, and then adding a little bit of my own. This, then, is my advice, based on Marco's but changed enough that he should not be blamed, to people interested in trying Lisp for the first time:

Compilers

As far as the commercial implementations go, I've never heard anyone complain about Allegro, LispWorks, or MCL. As far as the open source stuff goes, if you're on x86 *nix, pick one of CLISP, CMUCL or SBCL. If you're on PPC, try OpenMCL.

If you're just starting out, I'd suggest grabbing one of the free commercial trial versions as they tend to Just Work.

IDEs

The commercial Lisp products tend to have either pretty good IDEs or good emacs interfaces. If you're using an open source Lisp, you can pick between Emacs+SLIME, Emacs+SLIME and, should you feel daring, Emacs+SLIME. Seriously, there's really no other option when it comes to open source Lisp development.

Libraries

Have a look around cliki.net and common-lisp.net. Especially convenient are packages that are ASDF-installable.

Highly Practical Lisp Chapters

Peter Seibel has put several more chapters of his upcoming book, Practical Common Lisp, online. And even given the title of Peter's book, these chapters are highly practical, with explanations and examples of Lisp code for filtering spam, parsing binary files, using pathnames, parsing MP3 ID3 tags, web programming, creating an mp3 database, writing a ShoutCast server, and creating an mp3 browser.

Peter welcomes feedback, so do him a favor and let him know what you think after looking over the new material.

ACL 7.0 Released

Franz has released ACL 7.0

A few of the many dozens of significant new features, minor enhancements and fixes include CLTL2-style environments support, a new regex module, Allegro prolog, a new Oracle interface, XML DOM support, HMAC, RC4, and SHA1, an ACL-specific module for SWIG (to generate FFIs), dbm support, gzip streams, a new OpenGL interface, #!-style fasl scripts and WSDL generation.

For a more complete overview of the changes, see the Franz site or Bill Clementson's post. For a detailed discussion, see Franz's release notes.

It's apparent a lot of work went into version 7.0 of ACL. Oracle, XML, SOAP--these things are modern, problem solving facilities that make it easier to plug Lisp into the rest of the world and make it more likely that Lisp will remain of this world. ACL can be expensive, but you can't deny that those Franz developers are busy.

P.S. Bill Clementson still gets my vote for overall best Lisp weblogger.

NorthStar

PC Magazine has an article on Evolution's new navigation & localization technology, NorthStar [also mentioned on engadget].

NorthStar allows a robot to determine its location within a few inches inside a room. It can also help a robot navigate from room to room, which is the scenario posited in the PC Mag article.

The company that provides the recognition software that helps AIBO see and find its charging station, Evolution Robotics, has a new hardware/software solution called the NorthStar technology that addresses this long-standing problem. It uses a small, inexpensive sensor (usually placed inside the commercial robot) and an infrared, encrypted light device that plugs into a wall outlet to help a robot not only navigate from room to room, but actually know which room it's in.

Announced two weeks ago and demonstrated for the first time at the RoboNexus conference in Satan Clara, CA, the NorthStar infrared light projects two beams onto a ceiling up to 20 feet high. The sensor (which is built into the top of the robot) sees the lights and is able to figure out position information with regard to location inside the room as well as directional information, so it knows where it's going. Sensors can recognize more than one pair of lights. Each infrared light-emitting device includes a knob to change its ID. The sensor uses the ID to identify which room it's in. It's essentially a global positioning system for indoors (although GPS cannot tell you the direction you're heading in).

Tripping

Jeff Francis' GPS graphs reduce his daily driving patterns to the state machines found in the brains of SimCity characters.

MIT CADR Emulator

Brad Parker has a booting emulation of an MIT CADR Lisp machine (a commercialized CADR was Symbolics' first product, which they called the LM-2)

I've been working on a emulator of the MIT CADR. Basically a program to run its microcode. It is now running to the point where it runs a load band. There is basic display, mouse and keyboard support. There is some code for the chaosnet interface but it's not (yet) wired into the real network

The README claims the emulator is 90% complete. Source code is available.

Common LISP Hints

“Common LISP Hints” is over 10 years old, but except for the section on using Lisp with emacs it looks like a concise, useful Lisp tutorial.

BitOcean

This is a couple weeks old, but: Brewster Kahle at Web 2.0: “Universal access to all knowledge is possible, and it's not a non-profit goa.”. This big thinking excites me, in spite of myself.

Brewster is crazy. He wants to create a digital archive of every book, every audio recording, every film, and all television. Actually, I don't think is crazy at all; It's so obviously right that 50 years from now it will be a fact of life.

Cory Doctorow was there and took notes.

Magic

Edward Tufte redesigns the plaque on the Pioneer spacecraft:

Since the principles of physics hold everywhere, magic is conceivably a cosmological entertainment, with the wonder induced by theatrical illusions appreciated by all, regardless of planetary system. Accordingly the plaque aboard the Pioneer spacecraft for extraterrestrial scrutiny billions of years from now might have escaped from its conspicuously anthropocentric gestures by showing instead the universally familiar Amazing Levitation Trick.

Speaking of magic, I think I'll probably go to San Francisco in January to see the City Arts and Lectures interview with Ricky Jay. And that's just a couple days before his show at the Yerba Buena Center for Art opens. “Included are broadsides advertising: an armless dulcimer player, a ghost showman, a singing mouse, a chess playing automaton, a cannon ball juggler, an African hermaphrodite, a chicken incubating machine, a rabbi with prodigious memory, a ventriloquist, a spirit medium, a glass blower, a woman magician, a speaking machine, a mermaid, a bullet catcher, a flea circus, & an equestrian bee keeper.”

CL-EMB 0.2.0

Version 0.2.0 of Stefan Scholl's CL-EMB library is available.

CL-EMB is a library to embed Common Lisp and special template tags into normal text files. Can be used for dynamically generated HTML pages.

You can download it from http://common-lisp.net/project/cl-emb/ or install with ASDF-Install.

Blogging.LA

My city councilman, Eric Garcetti, made Boing Boing because he's active on blogging.la.

I can't hear or read “Eric Garcetti” without thinking of Vice City's Tommy Vercetti. “Tommy Vercetti is an innocent man!”

Booble and Ralph

Rainer Joswig's got some items that you shouldn't miss.

First is that Booble, the porn search engine that drew attention from Google's lawyers in January, uses CMUCL and cl-html-template. From the source:

<meta name="generator" content="cmucl-19a cl-html-template EMs Pg module and of course.. emacs" />

It seems not to work all that well, either as a search engine or as parody. Some queries give lots of results, but then a search for superstar Jenna Jameson gives seven results (and no images), two of which aren't her. No images for supernerd Asia Carrera, either.

No lisp programmers have yet come forward to take responsibility.

Next from Rainer is a fascinating interview with Mikel Evins about the early development of Apple's Newton, using a hacked MCL:

Q: So, Bauhaus was written in Dylan?

Mikel: When I was working on Bauhaus [a Newton OS] (and before, on the first iteration of Newton OS) Dylan was even more a Lisp than it is now. It was called “Ralph”, and was basically Scheme+CLOS+a couple of ideas from other functional languages+a few things to please Smalltalk hackers at Apple. The development environment was a hacked version of MCL that supported two compilers and runtimes: (1) the Common Lisp Compiler and runtime, which ran on 68K hardware and targeted 68K hardware; and (2) the Ralph compiler and runtime, which ran on 68K hardware, but targeted ARM hardware (first on ribbon-cabled Newton prototype boards, and later on development prototypes of actual Newton hardware connected by means of Nubus inserts). There were lengthy meetings with the Apple Cambridge team that was designing Ralph and various interested fractions at Apple, arguing for their favorite language features. The strongest lobbyists were Lisp and Smalltalk programmers.

...

Q: What would have been the main differences of Bauhaus to the released Newton OS?

Mikel: Everything above the kernel was written in Dylan, except for 7 very low-level QuickDraw routines. Size in RAM was about twice the size of the C++ version. The UI was a littler richer and a little faster, and built in Dylan using a graphic system made by wrapping those 7 QuickDraw primitives in Dylan and building a graphical language on it.

It supported mobile software agents (but there was debate about how this could be made both robust and safe).

It had a means of automatically generated UI elements from READ-like forms with class specifications for the arguments; for example, there was a macro get-input-from-user that was like a binding form (like LET*) with class annotations; it dynamically constructed the UI to get the values for the bindings. It had nestable stylesheets for describing the appearance of UI elements (sort of like CSS for UI, but back when the Web was brand new and the W3C What's New page still listed all the websites in the world :-)).

It had a novel event-handling system capable of supporting arbitrary user-defined events. The event system identified events by pattern-matching.

It had a frame system used to implement a knowledge base that stored things like prototypical Person and Place and Date objects, and supported fuzzy comparisons. The so-called “IA” (“Intelligent Assistant”) subsytem used these frames so that it could guess things like the fact that when you write “Joe” you probably mean “Joseph Smith” or “Josephine Baker” from your address book. It was also used for the help system, so that it could guess that, for example, a sequence of actions was probably intended to establish a machine-to-machine connection and pop up information to help you do that, and for the event-handling system so that you could declaratively define new classes of high-level events and handlers for them.

Damn. And that's in 50-100 KLOC, too.

Screen Scraping

Andreas Fuchs has released screen-scraper, “an extensible telnet/vt100 screen-scraper for lisp. By rendering unreadable vt100 control sequences into an array of strings (lines), It allows you to interact with a character-based menu interface and keep your sanity.”

Successful Lisp Starter Kit?

According to Markus Fix, David Lamkins' Successful Lisp will be available by the end of the month. He says they're thinking of selling a “starter kit” containing the book, a CD with some lisp implementation and a t-shirt.

I love the t-shirt--John McCarthy on the front and a cons cell on the back, and you can buy them right now at CafePress.

Brainstabbers

A few people have pointed out that the manly Bollywood lion slayer in the image I posted a few days ago bears a striking resemblance to one of the most distinguished computer scientists alive.

On the left, a man who can look you in the eye while his hands are busy stabbing two lions in the brains with daggers. On the right, one of the designers of Smalltalk, a founding father of object oriented programming, the guy who said “The best way to predict the future is to invent it” and did just that, a man whose ideas may cause a stabbing sensation in your brains: Alan Kay.

XML-RPC Begins In The Home

Back when I moved lemondor from an old G4 sitting in my closet to the server generously provided by Andreas Fuchs, which for all I know is located in someone's closet in Austria, I had some database transfer troubles. One bit of trouble that wasn't fixed until tonight was that the first hundred or so lemonodor posts had their formatting mangled, due to somehow having their “convert linebreaks” option set. I was too lazy to search through every post to find the ones with the incorrect setting and fix it using a not-very-speedy web interface that only allowed me to edit one post at a time.

Tonight I finally got fed up and wrote some code that uses Movable Type's XML-RPC support to automate the repair.

First thing, I needed to be able to do XML-RPC from lisp. Thank you, S-XML-RPC.

(require :s-xml-rpc)

(use-package :s-xml-rpc)

Then I wrote a function to fetch posts by ID.

(defparameter *username* "")
(defparameter *password* "")

(defun get-post (id)
  (xml-rpc-call
   (encode-xml-rpc-call "metaWeblog.getPost"
			id
			*username*
			*password*)
   :host "lemonodor.com"
   :url "http://lemonodor.com/mt/mt-xmlrpc.cgi"))

Posts are represented by s-xml-rpc-structs containing association lists of properties.

Fixing a post is a matter of checking its mt_convert_breaks property and setting it to false if it's true. I return nil from fix-post if the post is OK, otherwise it returns the new post structure.

(defparameter *convert-key* :|mt_convert_breaks|)

(defun fix-post (post)
  (if (equal (get-xml-rpc-struct-member post *convert-key*) "1")
    (set-convert-breaks post NIL)
    nil))

Changing the value of a property is done by creating a new alist containing the new, desired value.

(defun set-convert-breaks (post breaks-p)
  (let ((new-alist (mapcar #'(lambda (pair)
			       (if (eq (car pair) *convert-key*)
				 (cons (car pair)
				       (if breaks-p "1" "0"))
				 pair)))
		   (xml-rpc-struct-alist post)))
    (s-xml-rpc::make-xml-rpc-struct :alist new-alist)))

Once we've switched the flag, we need to save the post back to the weblog. I pass NIL for the “publish” argument, which tells MT not to rebuild all the static pages that contain the post (rebuilding the whole site once I've finished is much more efficient).

(defun save-post (post)
  (let ((id (get-xml-rpc-struct-member post :|postid|)))
    (assert (and (stringp id) (> (length id) 0)))
    (xml-rpc-call
     (encode-xml-rpc-call "metaWeblog.editPost"
			  id
			  *username*
			  *password*
			  post
			  NIL)
     :host "lemonodor.com"
     :url "http://lemonodor.com/mt/mt-xmlrpc.cgi")))

Now it's time to loop over all the posts and fix them. I know that the highest post ID in my database is about 950-something, so I'll just loop over the first 1000 and stop when we get an exception or I see that we're past the block of badly formatted posts.

(defun fix-all-posts ()
  (dotimes (i 1000)
    (let ((id (1+ i)))  ;; post IDs are 1-based.
      (format T "~&~5S" id)
      (let ((post (get-post id)))
	(format T " ~40S"
		(get-xml-rpc-struct-member post :|title|))
	(let ((new-post (fix-post post)))
	  (when new-post
	    (format T " ...fixing")
	    (save-post new-post)))))))

And now it's time to run the code.

CL-USER> (fix-all-posts)
1     "3D Development"                         ...fixing
2     "The Tendency of Unresolved Bugs"        ...fixing
3     "Inside Orbitz"                         
4     "On Hacker"                             
5     "Alex Moffat and Ehud Lamm"              ...fixing

;; Etc.

And thus ends my dabblings in the MetaWeblog API from Lisp. It seems to have worked, but let me know if you see anything that looks like it might be wrong.

XML Hack

Here's an interesting hack, embedding XML inline with Lisp code:

(defun process (command)
  (incf *num-requests*)
  (with-ml-page (:precompute f)
    (with-xml-output (*modlisp-socket*)
      <html>
        <head>
	  <title>"Some Title"</title>
	</head>
	<body>
	  <table>
	    <tr>
	      <td>"Number of requests"</td>
	      <td>*num-requests*</td>
	    </tr>
	    !(dolist (c command)
	      <tr>
	        <td>(car c)</td>
	        <td>(cdr c)</td>
	      </tr>)
	  </table>
	</body>
      </html>)))

Parse-Evaluate-Present

The CLIM breakdown, Arthur Lemmens style:

CLIM generalizes Lisp's textual read-eval-print loop to a read-command evaluate-command present-result loop. Commands can be read from a character-based input-stream but can also be “read” by “parsing” events. The same principle goes for presenting results: you can present a result by printing strings to a standard output-stream, but you can also present results in a graphical way.

Before Need

oakland columbarium, photo by lori

Lori and Dan and Carol and I recently visited the Oakland Columbarium, which was a beautiful and peaceful (unsurprisingly) place. I totally wanted to get comfortable in a big old chair and read a book there. Speaking of books, it was curious to see so many book-shaped urns. I can see how it would be nice to slide the urn containing your spouse's ashes into a bookshelf alongside their favorite books, but it becomes a weird, macabre library when arrayed en masse on the scale we saw. And then you wonder what indexing scheme one would use for such a library.

Ricky Jay Rarities

The Museum of Jurassic Technology has just put some rare Ricky Jay-related items up for sale, including posters, a broadside and even a winkie toy.

TerraMax

The September/October 2004 issue of IEEE Intelligent Systems has a short article on the TerraMax from the 2004 Grand Challenge. “Ohio State University at the 2004 DARPA Grand Challenge: Developing a Completely Autonomous Vehicle,"” by Qi Chen, Ümit Özgüner, and Keith Redmill, describes at a relatively high level the architecture and subsystems of the vehicle [via robots.net.

Following the route, TerraMax covered 1.2 miles within the required corridors and got stuck before a bush. The race log indicated that the communication net-work had malfunctioned and that the envi-ronment-sensing-and-fusion process didn t update the fusion map. So, the map indicated that there was always a bush before the truck, which prevented TerraMax from moving on. However, none of the other vehicles finished the course, either.

Another Shootout

Brent Fulgham has revived Doug Bagley's original Great Computer Language Shootout, “with new languages and revised to work with modern compilers.”

The versions of SBCL and CMUCL they've used are at most a few months old, and the shootout does include tests of newLisp.

Araneida 0.9

Version 0.9 of the Araneida web server has been released.

From the NEWS file:

* Portability to multiple lisps:
- SBCL, as before
- OpenMCL, threaded handlers only
- Allegro, threaded handlers only
- Armed Bear Common Lisp, threaded handlers only
- CMUCL, serve-event only
- CLISP, serve-event only
- LispWorks, threaded (from Jochen Schmidt, based on Portable AServe)

PC Magazine On The New Aibo

PC Magazine has an article that describes some of the new features of the most recent Aibo model, the one I mentioned yesterday.

Chief among the software enhancements are improved voice recognition, navigation, and autonomy in the second version of the AIBO Mind software. As before, the AIBO's image-recognition technology comes by way of Evolutions Robotics software. But now, in addition to being able to recognize and recall objects, AIBO will be able to remember objects and their location. This is especially useful for navigation. In video demonstrations, we saw the AIBO recognize a wall obstacle from roughly twice as far away as did an ESR-7 running last year's software. Additionally, after seeing a wall, turning to the left, and seeing another wall, the AIBO didn't turn back to the first wall. Instead, it appeared to remember the wall's existence and turned left again to find an open pathway.

Peter Norvig at Web 2.0

Jason Calacanis has an mp3 from Web 2.0 of Peter Norvig's presentation on search clustering.

Paul Graham's ILC 2003 Talk

Paul Graham has put more or less the text of his ILC 2003 talk on Arc online.

Aibo ERS-7M2

Because all the relevant web pages are in Japanese I can't find out what's new about the very newest version of the Sony Aibo, the ERS-7M2, other than that some of its brain is controlled by Evolution software.

Update: Oops, actually Engadget has some info.

Shocking

Sometimes things go on and on and you have some idea that they can't really continue forever but it's an idea that remains intellectuallized and in the background of your mind until it's forced into your guts by, say, a sudden, deep round of layoffs. It must be really strange to work someplace for a few years, coming in day after day, then come in to work one morning and, completely unexpectedly, someone tells you that you aren't going to be coming in to work there anymore and that you can go home right now, actually.

It's like, shit. Which isn't necessarily a bad thing. Anway, I've still got my job, it's just some friends who lost theirs.

Lisp Weblog Tutorial

The Elephant (which provides berkeley db access from Lisp) project has a tutorial on building a weblog engine with Elephant and the Araneida web server [via Bill Clementson].

Platforms vs. Implementations

Gordon Weakliem's “Implementations Matter” post seems relevant to my last post, on CLRFI.

The lesson of Perl, Python, Ruby, and PHP is that platform-specific libraries don't hurt the community much. The lesson of Lisp is that implementation-specific libraries have a divisive effect.

CLRFI

CLRFI is in the news.

Bill Clementson quoting Dirk Gerrits:

Anyway, I personally think the info in Nick Levine's short talk about the CLRFI is very much worth being more widely known in the Lisp community. (Has anyone announced it on comp.lang.lisp yet?)

Basically, the idea is to advance Common Lisp without actually having to develop a new ANSI standard. Everyone can write a CLRFI (Common Lisp Request For Implementation) and send it to the CLRFI editors. When it is approved, the CLRFI achieves draft status. It is then put on the website, announced on the clrfi-announce@alu.org mailing list, and it gets its own mailing list for discussion. After a discussion period, the CLRFI will either be withdrawn or become accepted. Accepted CLRFIs become a sort of 'de facto standard' additions to the HyperSpec, and will (hopefully) be widely implemented by Common Lisp vendors. The standard *features* mechanism can be used to check whether your implementation supports a certain CLRFI. All this info can be found in more detail at http://clrfi.alu.org.

Bill's comments:

This approach is similar to the (relatively successful) Scheme SRFI approach. It allows for common extensions to be proposed, debated and approved/declined in the community without going through a formal standards process. Another advantage of this process is that vendors can explicitly support a particular RFI if there is a demand for it and not support other RFI's if there is no demand for the functionality. However, as Christophe Rhodes points out, in order for this to work, people need to get involved in creating high-quality RFI's. It remains to be seen whether this will happen.

I wish I had a better idea of exactly how successful SRFIs have been. Christophe Rhodes claims that most implemented SRFIs merely try to bring Scheme up to the level of Common Lisp. “Scheme needed the SRFIs to allow people to decide on a shared vocabulary for completely ordinary programs, for the utility functions that previously everyone was having to implement for themselves: it's not hard to write remove-if-not, but if everyone has to do it then that's so much less time to spend on solving the actual problem.”

I would argue that at this point basic networking, regular expressions and threading are capabilities that are almost as basic as remove-if-not. One out of twenty programmers will need to do something unusual that won't be covered by the standard library (ICMP, weird threading, whatever), but not all duplicate removal needs are served by remove-duplicates, either. Yesterday I overheard someone say that email should be a type in scripting languages. Why not?

Chris Double:

The problem I generally have isn't taking my Common Lisp programs and running them on other platforms. I rarely have to do this. It's being able to take other peoples library source and use it on whatever Common Lisp system I'm currently using. In practice though it's not that big a problem. There are defacto standards around for a lot of things and some very portable libraries. But if the CLRFI implementations could be shipped with the Common Lisp implementations themselves then there would be no porting necessary and that would be a great thing.

Agreed. Portability means taking someone else's code and being able to use it, absolutely unchanged, on any machine, with any Lisp. It often matters less whether that portability is achieved through international standards (C, ANSI Lisp, POSIX), de facto standard language implementations (Python, Perl, Java) or, if possible, a CLRFI/SRFI style process.

Dave Roberts:

No, there's nothing to force anybody to comply with a memo posted on a certain web site, but you'd be amazed how just writing something down and putting some letters on it like “CLRFI” has the power to change things. It's funny that the IETF originally used to be very lightweight, just an organization for assigning numbers and publishing Request For Comments (RFCs) in an orderly way. As the Internet grew and the infrastructure became more important, the IETF started that gradual slide toward bureaucracy.

We can hope.

P.S. I'm back.