I love this picture!

I think a lot of usage of Common Lisp is probably in the secret & strategic advantage category. I experienced something similar when using Smalltalk at American Express. ParcPlace Systems really wanted to publicize our usage because it would help advertise to other companies how useful it was. Senior management was adamant about seeing it as a competitive advantage and they didn't want to clue in other companies. It's kinda funny they took that attitude, since a few years earlier in the AI boom, AmEx was pumping up Authorizer Assistant at every opportunity. Maybe that's why they clammed up.

Anyway, it's gotta take some serious cash to keep Franz and Xanalys going. Somebody's buying licenses and support and those folks are *not* yapping about it on comp.lang.lisp.

The real question is, how can I get into those companies?

Yahoo stores has been cited many times as a Lisp success story. Admittedly, PG and friends have become rich off Lisp, but I have heard elsewhere that it has been rewritten in Perl and C++, because nobody at Yahoo could maintain the site. So, from the software engineering point of view, Lisp might have been a terrible choice. What is the value of an application hacked by a Lisp-macro-genius if it cannot be maintained by someone else?

I'm sorry, but Klaus's comment just doesn't make sense.

If you believe that software engineering is a way to support your business goals, then Paul was very successful. They were able to develop and deploy a major application quickly with very limited resources. They were able to enhance and maintain it (even while the customer was on the phone) and most importantly, stay way ahead of the competition. So, I think you can make a great argument that it was the right choice, from a software engineering standpoint. It supported their business better than anything else. Remember, folks are in business to make money, and the best way of doing that is providing great value to your customers at low cost, so you can keep more of the profit. A CMM level 5 certification doesn't come with a wad of cash attached.

Who knows why Yahoo bought Viaweb? It could have been because they liked the features, or for a hundred other reasons, none of them technical. Maybe they didn't want a competitor to have it. Maybe they didn't want Paul Graham to be the next Google or Yahoo. Who knows? The point is, they obviously made a well-considered decision to buy Viaweb to support their business goals. At that level of decision making, whether it's in Lisp or anything else is irrelevant.

Anyway, if the mid-level technical management and programmer grunts want to expensively prove Greenspun's Tenth Rule, that's Yahoo's problem, not Paul Graham's.

I understand that some features had to be removed from the system in order to port it to C++/Perl. So even though they had the Common Lisp implementation to work from, they still couldn't produce the same results. They just wanted to save some cash on programmers (there are millions of C++/Perl programmers out there, thus they are cheaper).

Klaus, while you describe well the technical weakness of large companies, you seem to use an interesting tactic: marginalizing successful lispers as "Lisp-macro-geniuses." Since the rest of us programmers are idiots who can't be trusted to tie our own shoes, lessons of the smarties don't apply to us, and should be ignored.

This shifts towards 100% nontechnical conversation, where one can easily claim, "Lisp has no current nontechnical advantages, so we can safely ignore it!" Ahh.. but in technology, technological considerations at least need to be paid lispservice.

As an example, notice Yahoo isn't known for wise engineering. Google's cofounder revealed here:
http://murl.microsoft.com/LectureDetails.asp?895
that other search engine companies refused to buy/license Google's tech! Because "search quality wasn't competitively important." Google's only choice was to compete and win. So if the holy grail, a goose who lays golden eggs falls into Yahoo's lap again, will they know what to do with it? How many large companies would?

Incidentally, I hope that didn't carry an attacking tone. Very easy to sound like this on the anonymous internet, and I haven't slept for a long while.

I merely want to point out that Yahoo turned down Google's tech, giving them a track record of mistaken engineering/business decisions.

Further, programmers are currently learning a "pre-cambrian explosion" of languages: Java, SQL, XML, XSLT, a little language like AWK, and a dynamic language like Ruby or Python. I'm sure I read way too much into your use of the term "Lisp-macro-genius" and I retract it.

(I've requested my posts to be deleted. I've just taken a break and it is clear I'm in a very strange mental state due to lack of sleep. I really do not wish you to feel your point has been ignored, because you do make points which deserve not to be simply rebuffed. However, I am just in the wrong mental state. Sorry to you and everyone for writing these offtopic posts.)

Hi Tassyr, actually I liked your response. I don't even know what point *I* was trying to make .... but isn't it true that overuse of macros makes code hard to understand?

Klaus, I made the point on Ward's Wiki, that if the macros are well-designed and appropriate for the problem at hand, the worst they can do is change how one goes about reading a program. And for sure, reading a Lisp program is much different from reading a program in any mainstream language. Macros can make that effect even stronger. But I don't know that a person experienced in lisp would have trouble coming to a completely new body of lisp code and understanding it just because it used a lot of macros.

Paul Graham said about Viaweb that much of the program was made up of templates written in a DSL I think they called RTML or something. If you were just opening random source files in the Viaweb source tree, and you saw one written in RTML, you would probably find it hard to read if you weren't familiar with the language. But if you were, you would probably find that the code communicated more in less space because RTML was designed to be a more efficient way to express HTML templates than plain lisp code would be. And, as Paul says elsewhere on his site, succinctness is power.

Can you think of an example that would make your point better?

I don't think it takes a genius to write lisp macros. Technically, it's not too complicated once you understand how code is just nested lists (which couldn't be more obvious from the syntax), and how backquote syntax works. The hard part, which /can/ exercise genius if genius is available, though it certainly doesn't require it, is deciding which abstractions to express with macros. However, once these abstractions are made, it doesn't take a similar genius to understand them. Just because it takes a genius to make an application framework as orthogonal and conceptually consistent as the .NET framework (not without its weaknesses, however) doesn't mean it takes a genius to understand the patterns that make the framework good and useful, or to actually use the framework or understand code written in it.

I think the only merit to your suspicions might involve the misuse of macros. Macros are powerful enough that they can certainly obfuscate the meaning instead of clarifying it when the abstractions are bad, or even when the naming is bad. I think the question is whether someone with little skill with macros would tend to write bad macros instead of no macros. <tao te ching>Perhaps knowing the macro means understanding the macro.</tao te ching> Anyone experience this in practice?

As I'm not terribly experienced in lisp, all I can say from experience is that I've found the lisp code I've read to be the most readable code, by far, that I have ever read. The code I've read doesn't make heavy use of macros, though.

Tassir, I found your response to be neither overbearing or offending. In fact, I think it was actually pretty entertaining. It's true that you poked a little fun at Klaus's post, but not at Klaus himself. I think any person that posts something to usenet or a blog comment or a mailing list is inviting the very kind of criticism you gave, and it's usually expected by the poster. When it's not, it ought to be.

Good morning all. I think the loaded word there is "overuse," and in that case definitely.

I have a writeup of a lecture that asked, "Macros and Code Walkers: 'How Useful!' or 'How Useful?'"
http://alu.cliki.net/lisp-user-meeting-amsterdam-april-2004
and it argued that macros indeed have a certain smell at times. And the argument is that in common lisp, you have an ensemble of techniques -- if one of them is too prominent, like a stage-hogging actor, then you'll likely run into problems.

But I think in most non-lisp code, you'll find an underuse of macros. A nice, abstract argument is that programmers like to automate things for other people. Should they also not be given the keys to automate tedious parts of their own code?

In many of those languages, you'll actually find an overuse of something else, like OOP in Java or imperative in C. Others overuse functional programming, in my opinion.

These overused concepts do not always map onto your domain: other disciplines in science or business have invented their own languages to describe rules important to practitioners. There is wisdom in these rules, based on decades of experience, and macros help integrate them into your development environment. It bridges the gap between you and your clients. And I think some benefits are not technical but psychological -- your language is not so alien to them after all.

I'm not a terribly experienced Lisper either, but I think macros are overrated as compared to CLOS. Then again, you might respond that CLOS is implemented in macros. Oh well ...