Maximum Satisfaction

Ralph Richard Cook's comment on the issue of writing highly optimized Lisp made me feel all happy and smug.

I look at it this way. A lot of the new, interesting software is being written in a combination of two languages; a high-level, dynamically typed language with good built-in data structures and an environment for rapid software development, and a statically typed, heavily optimized language for performance. For a Python programmer, these two languages are Python and C. For the Common Lisp programmer, the two languages are Common Lisp and Common Lisp.

But then Pinku Surana's take made me worry a little.

The real problem is that normal programmers can not predict the performance of programs written in high-level languages, thus only wizards can optimize them because they have an intimate understanding of the compiler and runtime. A more satisfying response to a troller's assertion that “your language is slow” is to make integration with C drop-dead easy.

It would be nice if Lisp could be integrated with C as easily as Python can (it is kind of neat seeing the progress made in OpenMCL with Objective C integration). And maybe it's true that only wizards can achieve the kind of performance in Lisp that non-magical folk can get from C. Or maybe not, maybe just a couple years spent programming in Lisp, a degree of experience you'd expect from any serious programmer, would be good enough.

Lisp and Java

“Lisp and Java” by Dan Milstein is one of those articles which describes how useful first class functions are in a language which has them (Lisp), then attempts to implement an approximation to them in a language that does not have them (Java, in this case).

The one odd thing here is that Java already has first class functions through its reflection capabilities. But, as the article states, “The complexity of using reflection here is a clear demonstration of why it's too tricky to use in day-to-day programming.”

And check out the code sample that uses Java's reflection and true first class methods:

public class RFMaker implements RowFunc {
  private Constructor _cons;

  public RFMaker(Class c) {
    try {
      _cons =
        c.getConstructor(new Class [] { ResultSet.class });
    }
    catch(NoSuchMethodException e) {
      throw new IllegalArgumentException(
        "RFMaker must be called with a class " +
        "which has a ResultSet constructor");
    }
  }

  public Object of(ResultSet rs) throws SQLException {
    try {
      return _cons.newInstance(new Object[] { rs });
    }
    catch(InstantiationException ie) {
      throw new RuntimeException(
        "RFMaker failed due to: " + ie);
    }
    catch(IllegalAccessException iae) {
      throw new RuntimeException(
        "RFMaker failed due to: " + iae);
    }
    catch(InvocationTargetException ite) {
      try {
        SQLException e =
         (SQLException) ite.getTargetException();     
        throw e;
      }
      catch(ClassCastException cce) {
        throw new RuntimeException(
          "RFMaker failed due to: " + ite);
      }
    }
  }
}

Yikes. Compare to the conceptually equivalent lisp code:

(defun rf-maker (class)
  #'(lambda (rs)
      (make-instance class :result-set rs)))

(That may not quite be fair, since most of the difference in length between the two code samples is due to exception handling. But Java made its bed, and now it is lying in it--right here on lemonodor!)

As Gavin said, “This article is like getting over tickled: it starts to hurt, you panic, can't breath, etc.--started out fun but ended in misery. Java just can't help it.”

March in Chicago

I just got back from 5 days in Chicago, I'm still catching up.

How to Forget

I've seen two movies in the past few days.

Bill Morrison's Decasia is a collage of archival footage, the celluloid base of which is in various stages of deterioration. Images darken, posterize, bubble and melt. A boxer jabs at a roiling ghost, a woman's face blisters and melts as she testifies in court, carnival rocket ships swoop out of what looks like the boundary between our universe and pure unformed chaos.

Very trippy.

Much of the action in Eternal Sunshine of the Spotless Mind takes place in the mind of the main character as his memories of his ex-girlfriend are slowly being erased. There's a visual rendering of the deletion with objects and environments blinking out of existence or receding into shadow (and a lot more, the visual effects are much more impressive than this simple description probably implies).

These movies seem not entirely unrelated.

“We, amnesiacs all, condemned to live in an eternally fleeting present, have created the most elaborate of human constructions, memory, to buffer ourselves against the intolerable knowledge of the irreversible passage of time and the irretrieveability of its moments and events” — Geoffrey Sonnabend

Robolympics

Robolympics. If I think that the term “robot” should be used for machines that have at least as much autonomy as a thermostat, does that make me a kook?

Where Did Our Lisp Go

The last two weeks of lemonodor have been overflowing with robots, pleasantly architecture/planning/design-ariffic, and entirely lisp-less. I think it might be dangerous for everyone if I were to dive right back into hardcore original lisp content, so today I'll just mention a post of Bill Clementson's that I meant to link to weeks ago but never got around to.

The best known non-stupid (real problem, any algorithm) benchmark is probably the Coyote Gulch test. There are many languages that it has been translated into. If you can produce (write or find) and post a Lisp version that is within 10% of C performance, I will admit that #1 [lisp is slow] is incorrect.

Within a few days a collaborative programming and optimization effort paid off with a lisp version of the benchmark that was about 2% faster than the C version.

All About Timing

Last night at the Center for Land Use Interpretation, Steve Rostam from the Automated Traffic Surveillance and Control Center (ATSAC) gave a talk on traffic management technology used in the LA area.

ATSAC is located four levels below city hall in a cold-war era complex complete with heavy swinging steel doors (“Our biggest worry is getting trapped down there if something goes wrong”). It monitors 3200 of the 4500 intersections that have traffic signals, using inductive loops and video cameras, and tries to keep traffic moving smoothly. Software written by ATSAC programmers, including an expert system, processes the data and responds by modifying signal cycle times and offsets, which they call “adaptive traffic control”.

A few points I thought were interesting:

• Despite the huge interdependence between freeway traffic and surface street traffic, ATSAC's system is not inegrated with the California Dept. of Transportation's traffic control. If turning on metering at a freeway on-ramp causes traffic to backup on nearby streets, engineers from both agencies negotiate on the phone.
• ATSAC is experimenting with processing images from video cameras to get information on congestion and vehicle speeds, but the cameras don't give any data beyond what can be had from inductive loops.
• They're experimenting with bike-specific inductive loops, and with automatic pedestrian detection using video cameras.
• Almost the only inputs to the system are from the inductive loops, which just tell you whether a vehicle happens to be directly overhead at any moment in time. The only actions taken are to alter the timing of traffic signals.

Mr. Rostam was obviously an expert and very engaged in his work, repeatedly slipping into acronymic jargon as he answered audience questions. And the audience did ask a lot of questions. “Why did you guys remove a lane at Motor and Pico? It's awful there now!” “Some people in that neighborhood really wanted to get rid of that lane, and that's their right. We ran simulations and we knew it was going to be a bad situation, but they wanted to do it anyway. Now they can't even get into their own neighborhood.”

I asked if there were any data he wished they could get and whether there were any additional actions he wished they could take to affect traffic patterns. I was a little disappointed; it seemed like something he hadn't thought about. When I asked how much room for improvement there was in the system, like, if we had God's traffic control algorithm how much faster could traffic be moving, he didn't really have an answer for that either. Oh well.

After the talk we went outside where Mr. Rostam opened up a traffic control box and showed us its guts, which was kind of neat. Precise, clean electronics inside a banged up, graffiti'd and many-times-overpainted metal box.

Design Conference

I stopped by the Art Center Design Conference last night. They have all sorts of big names: the Segway guy, the vacuum cleaner guy, the swimming and galloping robot guy, the crazy Walt Disney Concert Hall guy and the Computers as Theatre lady.

Saw David Baltimore's keynote, which wasn't bad.

One thing in his speech confused me a little bit, though; He said that just two things enabled the evolution of self-organizing bits of stuff into the complex living organisms we have today: Random variation, and environmental selection. I wondered why he didn't mention the exchange and combination of genetic material through sex, i.e. sexual crossover. But I guess even though it's an important means of speeding up evolution, it's not strictly necessary if you have mutation, and it's true that sex itself hasn't been around as long as life. I should have known that the guy with the Nobel prize probably knows what he's talking about.

Afterwards WIRED threw a reception. Free drinks and coconut shrimp is a good Thursday night for me.

ERSP 3.0

Yesterday at Evolution we shipped ERSP 3.0. And a new robot. Phew.

It Already Is Something

The 2004 DARPA Grand Challenge is over. The final standings are in. Every media report so far has focused on the fact that nobody made it to Las Vegas, and missed the fact that this was actually a historic, impressive display of robotics.

Lori and I went, and we took some photos.

(For more photos, see Trackside Photo.)

Grand Challenge Update In Progress

I have some photos and videos of Saturday's Grand Challenge. They should appear here soon.

Barstow Bound

It's time to head out to Barstow in the desert cruiser.

The Final Fifteen

Alan Boyle says that the 15 teams that will compete in the Grand Challenge are

Red Team, SciAutonics II, Team Caltech, Digital Auto Drive, Virginia Tech, Axion Racing, Team CajunBot, Team ENSCO, Team CIMAR, Palos Verdes High School Road Warriors, SciAutonics I, Team TerraMax, Team Terrahawk, the Golem Group and the Blue Team (with the Ghostrider).

Still nothing on the DARPA site about it, though there are some new photos of yesterday's qualifying runs.

15 Vehicles Selected

The CajunBot team has a weblog. Supposedly 15 vehicles have now been selected for the Grand Challenge. Nothing on the DARPA news page yet, though the status board lists 11 vehicles as having finished more than 0 miles of the qualifying course.

14 Ton Lost Puppies

I'm surprised at how hard it is to find regular, up-to-date news on the Grand Challenge. What happened to this whole “blogging will route around traditional, moribund journalism/media” (hee) thing? Is everyone in Barstow too busy, or is the problem that there are no cafes with wireless out in the desert?

Alan Boyle at MSNBC has somewhat regular coverage.

There are some good quotes in this Atlanta Journal-Constitution article from yesterday:

“Watching a robot is like watching a clock,” shrugged Carnegie Mellon professor William "Red" Whittaker, dripping from the chest full of ice cubes dumped on him by his jubilant Red Team colleagues. “Robots do what they are supposed to do.”

[...]

Most spectators thought that was funny. But no one laughed when “TerraMax,” a hulking, six-wheeled military truck developed by OshKosh Truck Corp. and Ohio State University, tore away chunks of a concrete barricade the first time it lumbered out of the gate.

[Terramax is based on the Oshkosh Truck Company's Medium Tactical Vehicle Replacement (MTVR), the smallest of which has a curb weight of 27,800 lbs.]

“That thing's a monster,” said a worried Sal Fish, president of SCORE, the off-road racing organization coordinating the race events. “We've got to make sure our emergency stops work for all these vehicles. That could have been somebody's house.”

After some initial difficulty, the University of Florida's “naviGATOR” robot seemed to be negotiating the course flawlessly — until it passed beneath a pedestrian bridge over the track. The modified Isuzu Trooper promptly executed a hard right turn and slammed head-on into a chain-link fence. The overpass had blocked satellite signals and left the robot without its bearings.

Losing their satellite “lock” left other vehicles circling aimlessly or wandering down the track like lost puppies.

By the end of the qualifying, judges and race officials were cheering for the underdogs as loudly as were the “geeks and greaseballs” who built them. Cal Tech's “Bob” drew whoops and screams from the crowd. “Go, Bob, go!” they screamed. “Awesome Bob!”

“Bob” edged forward 20 feet and stopped.

“He's thinking,” race commentator Charlie Engelbart announced.

“Stay on the track, Bob,” Engelbart cautioned as the vehicle began straying to one side of the course.

“I can't believe I'm talking to a car,” he said into the open microphone. “This is like teaching your kid how to drive . . . only nobody's driving.”

Qualifying Video

On grandchallenge.org there are some videos from today's qualifying runs. They're simultaneously impressive and pathetic.

DARPA Rules Relaxed?

terramax ramming a minivan, from overbot.com.

Wired News reports that after only one vehicle (Sandstorm, the Red Team entry) finished the qualifying course for the Grand Challenge, DARPA has relaxed the rules.

The DARPA status report says that out of 16 teams attempting to finish the course today, three (SciAutonics II, Team Caltech and Virginia Tech) succeeded.

The results of the attempts of today s group break down as follows: SciAutonics II, Team Cal Tech and Virginia Tech completed the course. Team CIMAR , Team ENSCO, TerraMax nearly completed the course. Axion Racing, Digital Auto Drive, The Golem Group, Palos Verdes High School, Team CajunBot, TerraHawk partially completed the course. The Blue Team, Rover Systems, SciAutonics I and Team Phantasm terminated their attempts.

A comment from the relevant slashdot thread:

The QID was pathetic. We spent two days watching vehicles move around at 1MPH and hit big, obvious obstacles. No way can most of those vehicles operate effectively offroad.

The big design mistakes seem to be these:

• Using a laser rangefinder aimed horizontally forward as primary obstacle detection. That doesn't work reliably on either dark or smooth objects. The black mini-van was both.
• Using fixed line scanners. If you miss a data point, you're stuck. There's no way to take a second look.
• Overreliance on vision. Computer vision in unstructured situations has a very poor track record.

Scraaaape

This guy has some mobile phone photos of Grand Challenge qualifications.

Looks like the Palos Verdes high school team's vehicle ran a little off course there.

By the way, this is now an official race event, not just a bunch of robotics nerds. The Axion Racing team has Axion Racing Twin spokespersons. “They understand how GPS, FLIR, LADAR, RADAR, stereo vision cameras, computers, and the patent pending Axion Arbitrator work together”

I just ordered a copy of the calendar.

Not A Good Sign

There's a new Grand Challenge website up, which includes image galleries that are updated daily, and what will be a live tracking app.

(The DARPA site uses some ridiculous cookie scheme; do they not want people to link to their images?)

Today's LA Times article, “Pentagon's Robot Race Stalls in Gate” (registration required) makes it sound like maybe that fancy 3D tracking app won't be necessary.

“Oh no. That's not a good sign,” Todd Mendenhall said as his six-wheel creation started to bellow and spew smoke.

...

But now, after three days of trial runs, only four vehicles have been able to complete the 1.25-mile qualification course, which includes hairpin curves, 10-foot cattle gates and a minivan blocking the course. They have to do it twice to get into the race.

...

The TerraMax truck from Oshkosh, Wis., pushed a highway concrete barrier out of the way as it left the starting chute and slammed into a minivan, pushing it 5 feet before event organizers pressed the remote kill button — a required feature on these machines. Another vehicle lost its global positioning signal and looked for it for 10 minutes, turning in circles, before it was shut down.

...

Ghost Rider, based on a motorcycle, is one of the more novel entries. ... The motorcycle fell over more than 150 times during early test runs, Levandowski said. In its first trial run Tuesday, it fell after traveling about 15 feet.

This is gonna be so cool.

Vapid Vulgarity

I just started reading Jane Jacobs' The Death and Life of Great American Cities (Joey deVilla recommended it), and already in the introduction there's this great paragraph where she's talking about the misguided idea that all we need to get rid of slums and decaying urban centers is more money, and she lists what we've gotten for the money we've already spent:

But look what we have built with the first several billions: Low-income projects that become worse centers of delinquency, vandalism and general social hopelessness than the slums they were supposed to replace. Middle-income housing projects which are truly marvels of dullness and regimentation, sealed against any buoyancy or vitality of city life. Luxury housing projects that mitigate their inanity, or try to, with a vapid vulgarity. Cultural centers that are unable to support a good bookstore. Civic centers that are avoided by everyone but bums, who have fewer choices of loitering place than others. Commercial centers that are lack-luster imitations of standardized suburban chain-store shopping. Promenades that go from no place to nowhere and have no promenaders. Expressways that eviscerate great cities. This is not the rebuilding of cities. This is the sacking of cities.

“Luxury housing projects that mitigate their inanity, or try to, with a vapid vulgarity.” Just awesome.

I'm reminded of two things:

First, Howard Kunstler's Eyesore of the Month, full of “fried food ghettos”, “post-cheap-oil clusterfucks” and “innumerable” “monkeyshit brown” “places not worth caring about which someday will add up to a nation not worth defending.” (Excuse me, I got all Zagat there for a second.)

Second, a little more tangentially, the CLUI's beautiful exhibit on Los Angeles' ground traffic control systems.

The Real Challenge is the 6 AM Start Time

Last week the Red Team's vehicle suffered severe damage after flipping over when taking a turn too fast, and none of the vehicles attempting to qualify on Monday made it through the obstacle course. The Blue Team's autonomous motorcycle can barely go 500 feet before falling over.

It doesn't look like this is going to be a cakewalk.

The plan is to head out to Barstow Friday night, drink heavily, then drag myself to the starting line early Saturday morning.

There will be live video of the start available via satellite.

She Likes Robots, Too

me, sicker than most dogs

Not only did Lori make me go to the hospital in the first place when it seemed like my flu might have turned into something a little more serious, she read articles from The New Yorker to me while I was there, and when I was at home she brought me food and cake and generally took great care of me. And then took me to the emergency room again late one night when I had an allergic reaction to the antibiotics.

lori, cuter than most anything

This has really endeared her to my family. And while I had thought she was pretty much already maximally endeared to me, as it happened I learned a little more endearment could be eked out.

Movable Type Bugs

I've been really happy with Movable Type overall. It's free, I like the administration interface and the templating system, and it has the (minimal, admittedly) features I need. Last night while trying to move lemonodor between servers, however, I ran into two serious obstacles, both of which led me to think a little less of Movable Type.

The first problem I ran into was that the process of exporting my existing database and importing it into the new Movable Type installation did not preserve the IDs of entries. Unfortunately, entry IDs are used to construct the URLs of individual posts, which means that all URLs anywhere on the web pointing at specific entries would point to the wrong entry. For about 5 minutes I waffled on whether or not this was acceptable, but then I found these instructions on how to work around the problem.

Modifying the Movable Type source, exporting, putting out-of-order entries back into order, and inserting dummy entries to preserve IDs took about 45 minutes of tedious emacs work. Export/import that preserves IDs/URLs seems like a pretty basic capability.

The next obstacle was that even after all that work, about 300 entries into the import process the IDs got out of sync anyway. Movable Type seemed to think I had an extra 5 phantom posts in my database.

This problem was due to a totally amateurish bug in Movable Type's export/import.

MT's export format uses lines of dashes to delimit fields and records. E.g.,

----------
AUTHOR: jjwiseman
ID: 453
-----
BODY: ...
-----
COMMENT:
AUTHOR: jjwiseman
EMAIL: jjwiseman@yahoo.com
IP:
URL:
DATE: 01/23/2002 10:52:18 PM
Well, I installed OS X.  But it feels too clunky to use an
MCL that only works in Classic.
-----

Unfortunately there were some comments posted to lemonodor that included long lines of dashes.

--------------------
(setf foo bar)
--------------------
(print :woo-woo)
--------------------

MT's importer was confused by this and treated each line of dashes as a delimiter for a new post.

Oops.

The very first thing I ask myself when designing a data format or protocol is “How do I ensure that I can save/transmit any data payload at all without getting it confused with the metadata that's not actually part of the content?” Not properly escaping dashes that appear in the body of posts or in comments is a real novice error.

Uncommon Web

Marco Baringer announced Uncommon Web 0.20, “learning to walk”. (version 0.10 was the “learning to crawl” release).

Uncommon Web is a web application framework (with continuations).

Portable AllegroServe

On Feb. 28, Portable AllegroServe 1.2.35 was released. The last release before this was almost two years ago. This release is a “snapshot of current CVS. Not perfect, but then, what ever is?”

1st European Lisp and Scheme Workshop

The 1st European Lisp and Scheme Workshop takes place June 13, in Oslo Norway.

This one-day workshop will address the question what role Lisp-based languages can play in the near future to further the state of art in those and related areas. We want to solicit papers that discuss the opportunities Lisp provides to capture and enhance the possibilities in software engineering. Instead of using Lisp for Artificial Intelligence, as has successfully been done in the past, we rather want to explore the usefulness of Lisp with regard to software technology. We also want to promote lively discussion between researchers proposing new approaches and practitioners reporting on their experience with the strengths and limitations of current Lisp technologies.

The workshop is being organized by Pascal Costanza and Richard Gabriel, among others.

Decaf

David Lichteblau's Decaf is a Java debugger with GUI that uses CLIM and CL-JDI.

“Ultimately it should become a full-featured Java debugger. Currently it is usable as an 'on-line' class browser, can display stack traces, inspect objects, and set breakpoints on methods.”

Roblog

Phillip Torrone's “roblog” is a moblog with posts by his Sony Aibos, hacked Roomba and Evolution ER1.

SF/APE

The weekend before I got sick, Lori and I went to San Francisco for APE. I spent time in the company of lots of talented people (including Scott McCloud, Patrick Farley and Lore Sjöberg).

CodeCon was the same weekend, which meant that I got to hang out at least briefly with some of the codeconners (including Eric Tiedemann, Wes Felter and Kevin Burton).

(Jillzilla bridged the divide between these two cultures and therefore gets her own paragraph.)

The one thing I missed out on was having one of those SF burritos people are always going on about. But I did have a pear-ginger muffin at the Cheese Board, and that's something I might have a hard time finding in LA, too.

Triumphant Return

Lemonodor is back.

We're now coming to you from Austria, thanks to Andreas Fuchs, who got fed up with my ghetto hosting situation enough that he offered to host lemonodor on one of his machines.

I think that the move is complete in every way, except for maybe some of the RSS export options. I'll get to that.

My pneumatica is pretty much over with. Today's the last day I'm on antibiotics (oral), and I think I'm at about 90% strength-wise. Now to gain back the 8 lbs. I lost.

Broken Down

Lemonodor has a corrupt database, and I have pretty bad pneumonia (but I still had a pretty decent weekend). Symptoms of the former include the error message “Can't use an undefined value as a SCALAR reference at /Users/lemon/mt/lib/MT/ObjectDriver/DBM.pm line 173” when trying to post; the latter was confirmed by the big splotches on my chest x-ray. Right now I have a needle in a vein in my hand for antibiotics, which seems to be working, but I'm not sure anything can be done about lemonodor's database short of reinstalling.

So it might be a little bit before I start posting regularly again.

In other news, Lori and I sat next to at breakfast yesterday. He seemed really nice. He looked over at my and then ordered some for himself, to go along with his . Later, walked by and they waved and said hello to each other.