February 10, 2005

I've finally cleaned up the current incarnation of my Lisp Zeroconf service discovery code, CL-ZEROCONF. Apple's implementation of Zeroconf is called Rendezvous, and they probably have the best explanation of its value.

Rendezvous lets you create an instant network of computers and smart devices just by getting them connected to each other. The computers and devices take over from there, automatically broadcasting and discovering what services each is offering for the use of others. The network could be as simple as two AirPort Extreme-equipped PowerBook users sitting in a hotel meeting room miles from the nearest AirPort Extreme Base Station with some large files they need to share. Before Rendezvous, frustration. With Rendezvous, your computer will discover others, making file sharing completely simple.

Here's an example of advertising a service with CL-ZEROCONF:

(dns-sd:publish (make-instance 'dns-sd:service
                   :name "My CLiki"
                   :type "_http._tcp"
                   :port 80)

(dns-sd:process-dns-sd-events 3.0)

Here's an example of browsing for services on the local network:

(defclass my-observer ()

(defmethod dns-sd:browse-add-service ((self my-observer) service &key more-coming-p)
  (declare (ignore more-coming-p))
  (format T "~&Found service ~S." service))

;; Look for iTunes servers.
(let ((browser (dns-sd:browse "_daap._tcp" nil (make-instance 'my-observer))))
  (dotimes (i 10)
    (dns-sd:process-dns-sd-events 3.0))
  (dns-sd:cancel browser))

Even though it's named CL-ZEROCONF, this library only deals with one part of Zeroconf: service discovery. Link-local addressing (allocation of IP addresses without a DHCP server) is outside its scope, and probably doesn't make much sense for a Lisp library.

This is definitely version 0.1 quality code. The common use cases work nicely enough, but I'm sure there are situations it won't handle well. There's no documentation (though Apple's DNSServiceDiscovery API docs are relevant, as are its Rendezvous Network Services pages). There is some code to look at in examples.lisp, which is a collection of short examples of using the library, and test-browser.lisp, which generates an Araneida-based web page that displays services on the local network.

CL-ZEROCONF has been tested under OS X 10.3 with OpenMCL 0.14.2-p1, ACL 7.0, SBCL 0.8.16, and LispWorks Personal 4.3.0. Under Linux, it has been tested with SBCL 0.8.17 and ACL 7.0. But see the notes below for SBCL and ACL.

CL-ZEROCONF is really just a UFFI-based wrapper around Apple's multicast DNS “Responder”, which is a daemon that handles most of the work in doing service discovery for you. mDNSResponder is built in to OS X, and is offered as open source for other operating systems.

It took me a while to come to mDNSResponder. I tried Howl, but it was buggy, the API was incomplete and the event handling model was different in OS X than in the other OSes it supports. I tried the Cocoa NSNetService API and the Core Foundation CFNetServices API, but they weren't portable. I considered writing a complete Lisp implementation of multicast DNS that didn't rely on any external libraries other than for basic networking, but that has some disadvantages too*. mDNSResponder is free and quite portable, and saved me a lot of work.

In order to use CL-ZEROCONF on Linux, you will need to download and install mDNSResponder. It can be downloaded from Apple as a tarball, or via CVS. Make sure to read the instructions on using Apple's Darwin CVS repository--you will need to get a (free) Apple ID first. The mDNSResponder code base is under active development, so I recommend getting the code from CVS. If you run into any trouble, the tagged version I used was mDNSResponder-86. Once you've compiled the code, install it and start the mdnsd daemon.

If you want to use CL-ZEROCONF with SBCL, you will need to install Thomas Burdick's Alien Function package to enable callbacks from foreign code (integration of this functionality into the base SBCL is probably imminent; you may want to check to see if your version of SBCL already includes it):

$ tar -zxf sbcl-af-2004-10-22.tgz
$ cd sbcl-af
$ sbcl --load "system"
* (sb-ext:save-lisp-and-die "/tmp/sbcl.core")
$ cd /usr/local/lib/sbcl
$ mv sbcl.core sbcl.core.orig
$ mv /tmp/sbcl.core .

Coincidentally, both SBCL and ACL seem to have had an identical bug in their handling of foreign shorts on PowerPC. Franz recently fixed the bug in patch “update/paa005.001”, which you can automatically download and install (along with many other patches) by doing (sys:update-allegro). As far as I know, SBCL still has this bug. The bug exhibits itself in CL-ZEROCONF by causing all service port numbers to be reported as 0.

Once you've patched up your compilers and compiled new daemons, you can install CL-ZEROCONF with ASDF-INSTALL (of course).

I hope you find this useful.

* This is the approach that is usually taken in Python and Ruby, for example, but I think it's clear that Zeroconf should be, and soon will be supported at the system level, and I don't think it makes sense to spend the effort on another implementation, which will be at least several hundred lines of code, will not itself be any more portable, will certainly have bugs that will not be immediately apparent and will take months to be shaken out, and will possibly even be in contention with system-level services.

Posted by jjwiseman at February 10, 2005 03:21 PM

Boy, was that easy! Great lib, thanks. I figured out how to use the txt-record to set the path, too. News items merge into one :)


Posted by: Rich on February 11, 2005 01:24 PM

> * This is the approach that is usually taken in Python and
> Ruby, for example,

It is? I delight in your rant re not re-implementing things, but if Pythoneers tend to be bad in that respect, who is good? My experience is that random C libraries are more likely to have Python bindings than bindings for other languages. Could you have done this with an Objective-C bridge? The number one Google hit for [objective-c-bridge] is pyobjc.

Posted by: est on February 11, 2005 03:17 PM

Re est's comment:

""I tried the Cocoa NSNetService API and the Core Foundation CFNetServices API, but they weren't portable.""

I think that covers the Obj-C bridge comment; PyObjC is only useful on Mac OS X (and possibly GNUstep). It would achieve nothing more than the current implementation, and would be less portable.

At least, that's what I assume you're saying I may be misunderstanding your point.

Posted by: Rich on February 11, 2005 03:41 PM

That definitely addresses my second question. :)

Posted by: est on February 11, 2005 03:54 PM

Rich, it makes me very happy that someone found the code useful. What's even better is that it was used in your own very practical project.

Posted by: John Wiseman on February 11, 2005 10:26 PM

A nice library ruined by dismal packaging.

There should be one file that loads the utility. Fullstop.

Classic open source Lisp: advanced language + primitive packaging and configuration.

* There is no system definition file.
* N layers of component packages must be located and loaded.
* No logical pathnames.

Posted by: Abstraction on February 18, 2005 04:58 PM

Abstraction, I can't tell if you're serious.

The Cl-ZEROCONF tarball includes a cl-zeroconf.asd file, which is an ASDF system definition. ASDF is arguably the most popular system definition facility around these days. The tarball has been gpg signed and indexed on CLiki so that in any lisp with ASDF-INSTALL and ASDF, the following line will download the library:

(asdf-install:install :cl-zeroconf)

From there, this line will load the library:

(asdf:oos 'asdf:load-op :cl-zeroconf)

The only dependency CL-ZEROCONF has is Kevin Rosenberg's UFFI package. This doesn't seem unreasonable for a library that's trying to provide a portable lispy interface to a foreign library.

Is there more I could do to better package up this code?

I can think of a one: Provide system definition files for MK-DEFSYSTEM, etc. Maybe provide a plain .lisp file which will load the system.

Let me know if you have other suggestions that are more... concrete.

Posted by: John Wiseman on February 18, 2005 05:18 PM

How about an INSTALL file that describes the installation, what libraries are needed (ASDF, UFFI) and where one gets them.

For example for somebody wanting to take the code on his laptop and wanting to try it out while he is in a train with no Internet access....... or at home with no Internet access... or...

Posted by: RJ on February 19, 2005 01:28 PM

OK, that's a good suggestion.

Posted by: John Wiseman on February 20, 2005 11:18 PM
Post a comment

Email Address:


Unless you answer this question, your comment will be classified as spam and will not be posted.
(I'll give you a hint: the answer is “lisp”.)


Remember info?