Sunday, December 7, 2008

A Beginners Guide to ASDF (Ha!)

OK, there is at least one thing about which Lispers are not so smug: ASDF (Another System Definition Facility), the de factor standard for the Lisp library equivalent of Makefiles, and my post today was prompted by ANFOBA (Another Noob Effed Over By ASDF) washing ashore on comp.lang.lisp, and how I almost responded there:

Francogrex wrote:
Hi, I am struggling to understand how to work with asdf.

I have been using it for ten years and I am about 50-50 now on getting it to work. I am assured it does a great job in working with a multitude of Lisps/OSes, and I can assure you it is a disaster at working with programmers.

Come back here and post console output freely, you will need help, especially because one of its great triumphs is useless error messages.

Hints:

(1) It will work. You will have to promise it your first-born, but it will work.

(2) No, PB was not joking: you have to separately push locations onto some magical special variable. Leave off a slash (or add a slash, I forget) and ASDF will take your head off, so don't do that.

(3) No, PB was not joking, the syntax is:

(asdf:operate 'asdf:load-op :cl-pdf)

Why is it not (asdf:load-system :cl-pdf)? In my experience, extremely smart programmers think it astoundingly brilliant when "one function does everything!!!!", remembering the ridiculous op-code first parameter left as an exercise.

Since you asked, this is because extremely smart programmers are tone deaf to programmer productivity; they are so smart they do not even notice when they have created something unuseable, they just brain through it.

(4) ASDF has a featurebug: it loads in the wrong order in order to make McCLIM work. Don't ask. If the author of the .ASD file retreated to the desert for forty days to slave over the dependencies this will not be a problem. Otherwise do what I do: take a sledgehammer to the beast:

(asdf:operate 'asdf:load-op :cl-pdf :force t)

That abandons the whole point of having a build system, but it gets ASDF to work. When for the love of god I am just trying to build this frickin thing!!! that is a nice tradeoff of compile-time vs tracking down and short-sheeting the author of ASDF.

(5) On success ASDF emits more messages than an IBM OS/360 core dump. This is so any errors will scroll well out of view and anyway stand out like a healthy thumb from the surrounding...um, other healthy thumbs. Anyway, do what I do: ignore all messages other than a backtrace and Just Try the Hello World(tm).

If you get a backtrace, Just Post It to c.l.lisp.

(6) If hello world fails, f*ck! But at least now you are a Real Lisp Programmer(tm), this is what we live with.

hth, kenny

12 comments:

Eric TF Bat said...

Thank you, Kenny. Point 3 alone made my day. Why is the syntax so inane? I always heard "it's OK, you can write functions to call it however you like, that's just the baseline to build on" and interpreted it, as I'm sure everyone else does, as "blah blah gibber gibber I am a drooling Lisp wanker so worship my macros wibble wobble woo". I'm glad to see someone else confirmed my interpretation.

Anonymous said...

I suspect that in this case it has more to do with CLOS than macros.

SBCL comes with a trick to wire ASDF into the require mechanism. I've hacked that into my CLISP and my ClozureCL installs too, because I prefer (require 'foo) to (asdf:oos 'asdf:load-op 'foo). :)

Zach Beane had a lot of great material on his blog for setting up ASDF properly. But I think everyone is kind of hoping the new Mudballs project will take off and replace it, because it might actually work on platforms that don't have symlinks.

Anonymous said...

The syntax is "insane" because it shows the entrails of ASDF. ASDF is like C6PO when he met R2D2 the first time.

The problem is not with ASDF, it's with the users. There are so many of them, it's impossible to make friend with all of them. So every user has to define his own userfriendly interface to ASDF (and ASDF-INSTALL).

For me, it's two functions, ASDF-INSTALL and ASDF-LOAD. For somebody else, it could be some button in a slime buffer, or a CAPI window. Who knows?

Robert said...

I have to respectfully disagree with Pascal here. I have never understood why it has been such an issue of faith that it is up to the users to define their own friendly interface on top of ASDF:OOS. Who would it hurt if we were to simply add ASDF:LOAD-SYSTEM, ASDF:COMPILE-SYSTEM, and ASDF:TEST-SYSTEM?

How would the existence of ASDF:LOAD-SYSTEM interfere with your use of, or preference for ASDF-LOAD? How would it keep the SLIME folks from introducing a button? Offering a facility doesn't seem to me to preclude someone else offering the same facility?

It's all about lowering the barrier to the first use of ASDF.

Luke Crook said...

On Pascal's comment; There are 'users' and then there are 'users'.

I do not consider the user that hacks slime or wraps a GUI around ASDF a typical 'user'. The typical user (I'll include myself here) has no interest in extending a package management system. And even if I did, the only way to load this user-friendly (ASDF:LOAD-SYSTEM :*blah*) into a Lisp image is to (asdf:oos 'asdf:load-op 'foo). Catch-22.

So the impression that I am left with then is that ASDF is *not* for the typical user. So what package system are normal users left with?

Ben said...

Thank you.

Unknown said...

Almost everyone's using SLIME anyway, so why don't we just recommend ,l instead of the long syntax?

Robert said...

What does your point #4 mean, Kenny? Can you amplify? What's the wrong order that ASDF uses?

A comment, too --- I don't think that YA exercise in ripping everything up and throwing it away to start over (mudballs) is a great idea. I don't think that ASDF is that busted, and I have lived through The Great MK-DEFSYSTEM to ASDF changeover. I'd rather the next one of those was sometime in the 22nd century.

Kenny Tilton said...

Robert: ASDF will try to compile everything before loading anything, unless explicit dependencies (or the :serial option) are seen. I saw this was necessary for McCLIM, which makes me wonder what is wrong with McCLIM. If ASDF loaded as it went a natural ordering and organization of the files would eliminate the need for any dependencies to be declared.

Robert said...

OK, we are really passing out of the realm of my competence here, but Kenny, if ASDF were always to load everything after compiling, wouldn't that make it impossible to use it for compile-only purposes? I.e., I wouldn't be able to compile a system X without polluting my running lisp with the contents of system X?

The need to declare dependencies or use serial declarations has always been present in lisp defsystems, and it doesn't seem to me to be malign.

Actually, when you said "the wrong order," I was assuming that you meant something which I think is a much worse problem --- the PERFORM operation on a system is not wrapped around the PERFORMs on the system components. This makes system loading very difficult to customize.

Kenny Tilton said...

Robert, I am not saying a system cannot have a compile-only option, I am saying that if one is asking a system to load and compile it should load as it goes. ":serial t" is no option because it forces everything after a file changed to be recompiled. As for other defsystems, the AllegoCL project manager loads as it goes and the little system-builder I wrote to use with MCL loaded as it went. Believe me, complete specification of dependencies is a needless drag.

Anonymous said...

hahahaha wow i be so bored i thought asdf was what evryone typed when they be bored.

asdf
asdf
ssdf
asdf
asdf
asdf
asdf
asdf

its so fun 2 type asdf

haha security code was foack...lolzors