Just for fun, I tried to ASDF-INSTALL all the ASDF-INSTALLable libraries listed on CLiki. I used OpenMCL 1.0, so obviously not every library was going to work. And I didn't start a new lisp for every installation attempt, so there could have been some interaction issues (and in fact, it looks like maybe some problems might have been caused by some library installing a reader macro). But the results were still interesting.
Of 202 ASDF-INSTALLable libraries on CLiki, 104 could not be installed.
19 failed due to HTTP 403 responses. This is because of a bug in ASDF-INSTALL whereby it sends an incorrect HTTP request (it does a GET http://domain/path instead of a GET /path).
26 failed due to HTTP 404s. 10 of these are due to Brian Mastenbrook's server, unmutual.info, being down. A few seem to be due to the module listing as a dependency some other module that is not listed anywhere on CLiki.
7 wouldn't install because of “missing components”. Most of these are modules that list another module that wouldn't install as a prerequisite.
13 were “no such package” errors. Most of these are packages that are non-portable and won't work in OpenMCL. Some are just simple mistakes (one module tries to use the "ASFD-INSTALL" package, for example).
One was an “unsupported lisp” error. One module (ZIP) explicitly detected that it wouldn't work in OpenMCL.
And finally, there were 38 other errors. These are primarily compilation errors, like “Unbound variable: CDR”. 9 might have been due to a freaky reader macro.
I was disappointed that over half the libraries wouldn't install. Those just aren't good odds. And in terms of perception, as well as actually getting work done, it doesn't matter what the reasons for the failures are.
Though, on the positive side, it did turn out that many of the problems are easily fixed—within a few minutes of my posting my findings to #lisp, Zach Beane had an ASDF-INSTALL patch that fixed all the HTTP 403 errors, and a couple other people had fixed minor bugs in their libraries.
The next step would be to automate this sort of testing for multiple Lisp implementations. This might be something the CL-janitors/CL-gardeners would be interested in. I can imagine a library validator that would check ASDF system definitions for all the helpful fields like :description and :maintainer, list the Lisp implementations in which the library cleanly installs, check for unit tests, etc. It might be a first cut at estimating the overall quality of a library. Ideally, eventually, such a tool might become less useful as every library is brought into compliance.
Posted by jjwiseman at December 14, 2005 12:38 PM> Of 202 ASDF-INSTALLable libraries on CLiki, 104
> could not be installed.
That's my experience with lisp in general.
The other distros aren't all great either. Getting libraries to build and work under python can be a complete pain as well - I've had huge troubles trying to get psycopg to build on mac os x and linux. Java is usually good, but starts to suffer where projects sit on top of apache foundation tools, particularly where there are cross-dependencies.
But ruby has gem. Perl has cpan. Both are simple and effective.
Maybe one of the lisp distributions could get ahead by creating something similar.
The next step would be to automate this sort of testing for multiple Lisp implementations.
Franz is interested in donating a Lisp for this testing, if the Trial is not sufficient.
common-lisp.net would be happy to host this automatic testing process. I'd be happy to write the code as well but I can't until February, probably.
Erik.
Posted by: Erik Enge on December 15, 2005 12:39 PMIt's great that there's an ASDF-INSTALL patch issued quickly. But the obvious question from there is "how do people become aware that there is an ASDF-INSTALL patch, get it, and be happier?"
Expecting people to pull all the patches is just not a winning strategy; we need a way to at least push out the information that there's a patch that needs to be pulled....
Note that the above, despite the tone, is not meant to be purely negative. I'm happy to help make the situation better. It just requires someone have good idea about what a better solution would look like; then I'm happy to help make it happen!
Posted by: Robert Goldman on December 15, 2005 12:59 PMRobert,
http://groups.google.com/group/comp.lang.lisp/msg/c2db0d58bc7be26a
That thread has some information which relates to what you're talking about. I personally would love it if a directory could serve as a single discovery point for maintainers of packages, including asdf-install, and a single (or at least very popular) distribution point for the same software.
Thanks,
Erik.
Did these ASDF-INSTALL fixes get put into the ASDF-INSTALL that one gets from www.cliki.net?
It might be nice to put a latest asdf-install version number on that page, so that people can know whether they need to pull a new one, absent some update-push mechanism....
Posted by: Robert Goldman on December 15, 2005 01:37 PMRPG: Yes. See http://www.weitz.de/asdf-install/#changelog too.
Although it is disappointing that most of the packages can't be installed properly, I think you need to weight them a bit differently. Araneida is probably a more popular install than KMRCL. You need to figure out which packages don't work (as you've done). Then you need to figure out how many ASDF-INSTALL requests each one gets. If Araneida gets 99 requests and KMRCL only gets one, then that means that it's not a 50% failure, but a 1% failure. This also gives you a good way to prioritize fixing things. I'm glad to see your test has already caused some good changes.
Craig: In many ways, I think ASDF is a lot nicer than CPAN. The one place where CPAN wins is the distributed server-ness. I can't comment on gems as I haven't used Ruby enough to know.
Posted by: Gregory Martin Pfeil on December 15, 2005 06:12 PMHave you considered putting your results page on Cliki? Might be a handy way of people finding out what needs to be fixed and having a whack at fixing some of it....
Cheers