Thursday, February 28, 2008

Macro-ro-ro Your Code Gently Down the Screen...

Sorry about the title but I just feel a little silly having to explain why macros rock so hard when the C preprocessor has been doing them almost as long as Lisp. It just ain't that hard an idea to get. It seems to me anyone writing a sufficient amount of code will notice recurring patterns not susceptible to attack by encapsulation in a function. Consider OpenGL.

With OpenGL, certain things have to happen between a glBegin and a glEnd, and certaing things are not allowed between a glBegin/End pair. One of them being a second glBegin. The poor OpenGL programmer now has one more thing to worry about, and tedious boilerplate to write like a robot (in this case just a one-liner glEnd() but try not to lose the big picture principle, OK Sparky? This is a deliberately simple example). Is there any way to alleviate this drudgery?

Well, quite a few statements will be squeezed of between glBegin and glEnd, cuz OpenGL is no joy to code. We can write a function called do-between-begin-end and pass it a first class function (unless your BDFL does not see the need), but then (in CL) we end up with:
(do-between-begin-end
(lambda ()
...your dozen lines of code here...
))

Not the end of the world, but remember, I am just warming up. These macro posts will go on forever. Anyway, with a macro we lose the lambda speed bump:
(with-gl-begun ()
...your dozen lines here...)

What's with the ()? Well, truth is glBegin takes a parameter that says what we are beginning (line strip? triangles? quads?) so I kinda lied, the contrast should be...
(do-between-begin-end gl_quads
(lambda ()
...your dozen lines of code here...))

and:
(with-gl-begun (gl_quads)
...your dozen lines here...)

So, yeah, I can write do-between-gl-begin-end and never worry about begin and end again, but I am stuck with a little line noise there.

If we look at a classic example of how Lisp macros hide boilerplate, we see this more egregious after/before pair:
(with-open-file (f "xxx")
(print "line 1 of xxx:")
(print (read-line f)))

Versus what I have to write without with-open-file:
(let ((f (open "xxx"))
(success nil))
(unwind-protect
(multiple-value-prog1
(progn
(print "line 1 of xxx follows:")
(print (read-line f))
(close f))
(setf success t))
(unless success
(close f :abort t))))

The odd thing is that Pythonistas reject macros while fervently embracing tidy code -- and Python code does look very clean -- but all we are doing with macros (so far) is making code neater. Well, no, we are also releaving the programmer of drudgery and making the code more reliable by automating the insertion of necessary boilerplate such as error checking.

Let's turn now to a dead simple hiding of boilerplate, Paul Graham's AIF swiped from the inestimable On Lisp:
(defmacro aif (test-form then-form &optional else-form)
‘(let ((it ,test-form))
(if it ,then-form ,else-form)))

The alternative is:
(let ((temp (big-long-fn a b c)))
(if temp
(handle-it temp x y)
(do-something-else a x))

By the way, if you do not like Graham's anaphoric thing (using "it" as the implicit temporary variable), use my BIF which takes a mnemonic for the intermediate result:
(defmacro bif (bindvar boundform yup &optional nope)
`(let ((,bindvar ,boundform))
(if ,bindvar ,yup ,nope)))

And then a more realistic use:
(bif sol (math-problem-solution prb mx)
(display-solution sol)
(tell-user "No can do!"))

In this just my first offering on macros I have discussed only the advantages of automatically generating boilerplate. There is much more to come, but I would like to close by reiterating my befuddlement that this even needs explaining, and in the context of the C preprocessor precedent mentioned above:

I just dug into the old C version of my Algebra software for just one out of hundreds of macros, one which iterates over the children of a node (a custom tree struct in my application) in something akin to the C "for" statement:
#define fork(PAR,TMP_KID,NXT_KID)      \
for ( NXT_KID = (TMP_KID = firstk( PAR))? nexts(TMP_KID):NIL \
; TMP_KID \
; NXT_KID = (TMP_KID = NXT_KID)? nexts(TMP_KID):NIL)

The crucial thing to note is that this macro is meant to allow children to be unlinked from the parent during the traversal, so it is necessary to capture the next child after the current child before processing the current child, because parents only see their first and last child as I had things arranged.

So now I could write code like this (instead of something like the macro definition above):
fork( usg, tusg, nusg) {
kidDetach( tusg); ;; losing the "next child" link
xInsKid( tpar, pusg, tusg);
}

Next time we will do something even jazzier.

14 comments:

DavidM said...

"Well, quite a few statements will be squeezed of between glBegin and glEnd, cuz OpenGL is no joy to code. We can write a function called do-between-begin-end and pass it a first class function (unless your BDFL does not see the need)"

Python has first class functions, it simply doesn't have multi-line lambda type functions.

def draw_stuff():
def draw_it():
...several gl commands here...
call_opengl( 'lines', draw_it );

Kenny Tilton said...

Python has first class functions, it simply doesn't have multi-line lambda type functions.

Right, and as I said it is not the end of the world to have to go through that coding, but look at how much textual noise is needed without macros.

Macros are not about having different functionality, they are about having code written for us (but hidden in the macroexpansion seen only by the compiler).

The extra code I must write without macros is tedious (albeit mechanical), creates more opportunities for me to screw up (I need no more!), and because it is boilerplate it is likely to change when I refactor, in which case I very much want to have it being generated in one place.

Anonymous said...

Neat. To play devil's advocate, though:

If you're using macros primarily to eliminate unsightly lambdas, then I content that you would be better off with a language which supports lazy evaluation.

Macros are not elegant to my mind, they're a piece of hackery that should be a last resort.

Lisp makes them all seem very innocent what with its syntactic conflation of code and data - but really. What you want, a lot of the time, is selective lazy evaluation, and a language with a compiler that supports this natively can play a lot nicely with static type systems, debugging tools and so forth.

Kenny Tilton said...

If you're using macros primarily to eliminate unsightly lambdas,...

No, sometimes it is just a big form, other times it is a whole DSL.

then I content that you would be better off with a language which supports lazy evaluation.

(scratches head) I do not see the connection. Please elucidate.

Macros are not elegant to my mind, they're a piece of hackery that should be a last resort.

Don't let Paul Graham hear you saying that. :) I don't know, macros work at the textual level to automate repetitive text -- sounds Deeply Correct to me pending edification on the lazy alternative.

What you want, a lot of the time, is selective lazy evaluation, and a language...

Like this?: Qi

Anonymous said...

Macros are not elegant to my mind, they're a piece of hackery that should be a last resort.

I challenge you to (1) give an example of a standard macro that is part of Common Lisp-the-language, and (2) explain why it must be considered a hack.

I think that would help the discussion a lot. Actually I don't believe anyone with the above opinion can mention just a single macro in CL. Prove me wrong?!

Anonymous said...

cl macros, lazy evaluation, and call-by-name all associate a name with a computation rather than just its result. lazy eval lets you define a function "lazyif" with exactly the same semantics as if:

(deflazyfun lazyif (a b c) (if (force a) (force b) (force c))

same thing for macros.

it's similar to static v. dynamic typing. at some time scale, even statically typed programs are dealing with dynamic types (i/o) and dynamically typed languages deal with static typing (cpu instructions).

so macros are that language for laziness that poster wanted, just with different terminology. and using macros, you can change the terminology...

Kenny Tilton said...

"cl macros, lazy evaluation, and call-by-name all associate a name with a computation"

Ohhhhhh, that aspect of macros. Yes, absolutely, sorry, I just never use macros for that. But it is one thing they can do.

Another thing they can do is the exact opposite and compute at macroexpansion time:

(defmacro avg (&rest nums)
`(/ (+ ,@nums) ,(length nums)))

I never use that either. :)

Hmmm, I should do a post on all the different things macros can do: lazy eval, pre-compute, hide boilerplate, DSLs, ....?

I should point out that all this depends on the macro function being able to operate on the code (the macro body) as if it were data, and I think that is where I left off in the "My Turn! Why Lisp?" post.

Steve Knight said...

Hmmm, I should do a post on all the different things macros can do: lazy eval, pre-compute, hide boilerplate, DSLs, ....?

I'm sort of getting stuck into CL and I'm trying to figure out when to apply macros and when not. A post like this would be very helpful ... to me at least :-)

Kenny Tilton said...

"I'm sort of getting stuck into CL and I'm trying to figure out when to apply macros and when not."

Of course I know what you meant, but that is too good a pun to pass up since one of the only downsides of macros is that they cannot be used with APPLY. :) Anyway...

The Golden Rule of when to use macros is a negative but very helpful as a guide: "Never where a function can be used." (And I do not mean by putting what would be the macro body into a lambda form and then passing that to a function, as discussed above in these comments.)

Also, forget me, go straight to the Master, specifically Chapter 8 of On Lisp, aka "When to Use Macros". Start here: On Lisp And then read a few of the other chapters, Paul writes very well and just looking at the table of contents makes me think I should save my breath.

What I did was (a) read a few chapters of On Lisp and (b) not force the issue until a voice spoke to me pretty clearly that I had a repeating pattern of code that was getting irritating to write but again could not just be bundled up into a new function.

Steve Knight said...

Of course I know what you meant, but that is too good a pun to pass up since one of the only downsides of macros is that they cannot be used with APPLY. :) Anyway...

I set 'em up you knock 'em down! Oops, I really walked into that one ...

I read PCL and am working through SICP but I didn't get to OnLisp 'cos I thought I could substitute it for PCL. How wrong I was. How wrong. I will remedy that situation shortly.

I think part of my personal confusion about macros (and perhaps I'm alone here) is that one reason for using macros in C (not C++!) is to remove the function call overhead. The more I think about it the more I reckon this must be a really bad reason for using macros in CL. So it's back to the books for me ...

Anonymous said...

an online version: http://www.bookshelf.jp//texi/onlisp/onlisp_9.html

re: macros being haxery: yes, i think in other languages they may have an aura of hax about them, but i don't see that in lisp because the language is so syntactically simple. with the sorts of things you can do with macros in such a language, it would be stupid not to take advantage of them

Kenny Tilton said...

...one reason for using macros in C (not C++!) is to remove the function call overhead.

Ah, yes, add that one to the list, and no, I have never used macros for that either. :)

In CL you can (as with C, IIRC) have functions inlined. Never use that either. :) My performance bottlenecks are never over function call overhead.

Steve Knight said...

Ha! Mind reader! I was just wondering whether I should be using 'inline' functions instead. Thanks for the tip! -- S

Anonymous said...

There is a far more elegant way to express this using Python > 2.5 though -- 'with' blocks.
You could say something like

with glBegin(GL_LINES):
...
...


Naturally you can also use a C macro which expands to a for loop with a little boilerplate code to achieve something similar.