The Road to qooxdoo Part III: Why It Rocks

In the episodes one and two we learned all about how a Lisp God and Interwebby know-nothing found a way to support a client needing an enterprise Ajax Web application, said way turning out to be qooxdoo after solid efforts to make jQuery, Dojo, and YUI fill the same bill. But we did not learn very much detail about qooxdoo, we just learned that it comes with lots of doc, lots of examples, a solid community, and that the engineering is first rate. I may not have mentioned this, but it also was dramatically faster/smoother than the others on my acid test: scrolling a datagrid. 

Okay, that is a lot as a general recommendation. I mean we did not learn much detail. Until now. Here are four Big Things that really stand out:

1. HTML? CSS? Never heard of 'em.
First comes a shocker. The qooxdoo user codes no HTML or CSS and needs know nothing about those two disappointments. Just as my Lisp compiler reads my Lisp code and spits out optimized native machine code, qooxdoo takes your qooxdoo/Javascript and produces the HTML and CSS to drive the browser. Upside: qooxdoo developers write less code and never have to worry about browser variability again.

To a Web noob like me there is a second big win here: I do not have to learn HTML, CSS, or the vagaries of the different Web browsers my client might want supported. I am a bit of a rarity in my ignorance of those things but soon I should have a lot of company now that qooxdoo has set the bar as high as it has on insulating the web app developer from lower level technology.

By the way, in case you are wondering: yes, there is a widget (two, actually) that will accept classic HTML.

2. I do declare. Not!
The second thing that hits us is a consequence of the first: we are going to be writing a lot more Javascript than we would be with libraries such as Dojo where authors still write HTML/CSS. With declarative tools we sit in Lisp or Ruby or PHP on the server generating HTML. Not qooxdoo (but see next)  and not YUI.

I am not at all happy about this but I will live and if it ever bothers me enough I will spend a weekend exploring the qooxdoo markup contrib called QxTransformer. Right now I need to catch up on a backlog of functionality the client has had to do without because of the six weeks it took me to identify qooxdoo as The One.

3. OO is not the Grail, but OO rocks
Stendhal said it best in "On Love" in the chapter on Infatuation: (paraphrasing poorly) we do the object of our desire two injustices, first setting them on a pedestal unreasonably high and then, when they inevitably disappoint, setting them too low. OO is not the Grail, it is just a profound advance in serious application development. All by way of introduction of another stunner: qooxdoo's slightly more substantial object model layered atop the core Javascript OO capabilities. 

To be honest this struck me as a bit of a yawn until I decided to start carving up my existing code into distinct qooxdoo classes. This is a very big application and I needed code re-use almost immediately and doing it in HTML with Javascript glue was...well, before qooxdoo I was pretty much starting to build my own abstraction layer atop the core tools--there was no way around it.

But now that I have used qooxdoo's OO to partition my rapidly growing code base I have more than once had that wonderful sense of getting something for nothing after doing a major refactoring (you know, taking all the pieces and throwing them up in the air) and having it Just Work after two or three tweaks. Furthermore, I will not bore you with the details but I have not even started leveraging the full capabilities of qooxdoo's object layer: even more benefit will be reaped moving forward because of some nifty reactive capabilities.

This came as a pleasant surprise. I do more OO in more ways than you can imagine in part because I normally program with Lisp's CLOS which is to normal OO what normal OO is to binary arithmetic but by the time I stumbled onto qooxdoo I had been doing Javascript, HTML, and CSS for a couple of months under a lot of pressure so I never had the time to build a sensible OO abstraction atop Javascripts primitive little OO pretensions. Thus Javascript had endowed me with a bit of child mind and I was able to experience the mundane code structuring win of OO as if for the first time. It rocks.

3. Layout the Wazoo
I may have lied. This may have been the first stunner I felt in my gut. The others I knew would be huge if they panned out, but this was the first one I saw in action working as beautifully as had the other frameworks left me hanging. With other frameworks I could find no way to get built-in layout schemes to use the full window real estate available. That cuts two ways: I either ended up with unused whitespace (and with our app we need all we can get) or my application panels grew so large that the browser itself put up scroll bars, meaning I had to window-scroll my scroll panes around. No, that was not a candidate for production release. Note that to a large degree this is the same as point #1: a framework is not hiding HTML/CSS from me if I am still victim to surprising browser decisions, such as oh gosh that is big let's use a scroll bar here.

How good is qooxdoo at layout? When I popped open the Firebug debugger in FireFox, where by default it begins by seizing application window real estate, the qooxdoo layout engine handled the downsizing impeccably, right down to adjusting scroll bars to accurately reflect the new dimensions.

4. Message In A Bottle

This feature is huge in a small way, by which I mean I could probably roll my own version of qooxdoo's Message mechanism in a an hour but--wait for it--I did not have to. They did. That is huge, the huge thing being that the qooxdoo team really has set out to create a compleat Web application development environment and with such a beast comes pleasant surprises such as their Message mechanism which allows a nice decoupling of widgets: one widget can dispatch "Make it so" and anyone responsible for "it" knows to roll up their sleeves simply by having subscribed to "Make it so".

Beyond the Message mechanism we have the Event mechanism. The documentation of every widget lists the events it might generate and what data the event will carry, and in toto we have an engine with an event model dozens of times richer than pure DOM.

5. I know, I said "4"

Number five is an umbrella win, too detailed to list: as I whined above, this Lisp God is now writing an awful lot of Javascript leveraging more and more qooxdoo all the time. Again and again your highly critical correspondent is finding nothing but good news in qooxdoo. 

Yes, I could make a bug report every other day if I had the time, but a long time ago I figured out that I am not looking for a perfect library, I am looking for a powerful library that is fundamentally sound (so anything broken can be readily fixed) and actively maintained (so things actually get fixed).

Summary

qooxdoo: fast, powerful, complete, rich, well-engineered, and it hides browsers, HTML, and CSS. Great doc, loads of examples, fine community. It's all good.

The Road to qooxdoo - Part II

Last time, we learned how one self-appointed Lisp God with no knowledge of web programming ended up programming the web and then soon enough embarked on a six week grand tour of Javascript frameworks in search of a platform suitable for an enterprise web application and where his tour ended: qooxdoo. That episode ended with a promise of more information on qooxdoo, but first a note on the methodology: there was none.

My first experiment with web programming had thanks to the marvels of Lisp in general and AllegroCL's web tools in particular mushroomed into a full-blown web application aided here and there by jQuery widgets before we slowed down enough to think, Damn, we could benefit from a full-blown Web application framework.

 So I eyeballed the usual suspects and ran them through googlefight and picked the most promising one and used that to redo what I had accomplished to date. Which was: tab controls, select boxes, data grids as views into server-side data stores, tree views, text input, tool tips, pushbuttons, pop-up windows, and probably one or two things I am forgetting.

How did I assess the libraries? We needed a professional look, good performance, and maximum productivity. Sorry if that seems blindingly obvious. The productivity assessment has some beef: I looked for lots of documentation and tons of examples describing a rich set of widgets at once high-level, feature-rich, and yet highly authorable which may not be a word but it means I get to override stuff without resorting to back doors. 

I considered also two metaqualities: on-line support and internals code quality. Having the Lisp HyperSpec an F1 away is just a little faster than asking for the same information on comp.lang.lisp. ie, Community matters. As for code quality, the good thing about open source is not that it is free, it is that I can use the source to understand things at 3am when the community might be quiet; I can add print statements to debug my code; and maybe I can find and fix a bug in the library. But if the code is of low quality good luck with anything more than the print statement insertion.

It was a rough six weeks. Three times in six weeks I logged onto support groups and said, "Hi, this is my first day..." as I rebuilt a relatively feature-rich web app three times while staggering up to speed three times on three environments. All while the client is getting absolutely no new functionality. I feel bad all the while because they need this stuff now. I am letting them down and there is the strong possibility they will be letting me go: I had been brought on to do the Web front-end. I like learning, but not so much with a gun to my head and a time bomb ticking.

Are we having fun yet? Well, yes. The last framework we tried was qooxdoo and it has worked out great. I can now give the client pretty much what they want fast and looking good once they pony up for a graphic artist because I just saw this impressive example of what can be done with their "appearance" framework. But I ain't no designer.

Funny thing, though: qooxdoo  like YUI and Dojo came up short during the evaluation. It had no example of the widget I needed most, a fairly standard widget at that: a datagrid backed by a remote store. Bad sign. Second, qooxdoo did no better on googlefight than does Lisp, and that is pretty bad. Finally, qooxdoo like YUI lacks a declarative programming model [though the declarative QxTransformer has been revived since my original investigation] which means I cannot just sit in my Lisp IDE catching XHRs and tossing back HTML/CSS. This would put a non-trivial dent in my productivity.

So how did qooxdoo prevail? What can I tell you? Bullet charts are useless, expert systems work great as long as expert humans interpret their output, and resumes tell us nothing about the work we will get out of people. 

qooxdoo did have a contributed example of a remote data store which pretty much worked and when I looked at the qooxdoo internals to learn how to make it work better I found very good code and rather quickly had the datagrid I needed. Otherwise qooxdoo did have loads of examples and documentation and decent community (I say "decent", you'll say "great"--my standard is comp.lang.lisp) and then we get to the bottom line.

While resurrecting my app for the third time with qooxdoo, yes, I worked just as hard as I did using Dojo and YUI. But the effort was different. With qooxdoo I was struggling to learn a very nice framework that needs an index: I was slaving for six hours to find the simple six lines of code I had to write. With YUI I was struggling with a framework that did not work all that well and scared me more and more the more I looked inside it. With Dojo the struggle was with a good framework but the struggle was such that I suspected the struggle would never diminish and probably increase exponentially as the application grew in complexity.

As for the declarative thing, hey, I am a programmer, I can write code. And Javascript has a lot of Lisp soul, I am not stuck with something daft like PHP or Python. Sure, having to spend more time in a JS IDE means a ten percent productivity hit but ten percent we can do. The terror we developers dread is the tool nightmare death by a thousand cuts two steps forward two steps back rope drag death. 

I will blog on rope drag sometime soon, but first let's get specific about qooxdoo. In Part III of my Road to qooxdoo.

The Road to qooxdoo

[The title of this rather boring entry derives from a fun little survey I kicked off years ago.]

OK, why should you listen to a Lisper on Javascript frameworks? Because we use Lisp so obviously we have better instincts when it comes to developing applications. In fact, this road would not have ended so well had not Lars, a fellow Lisper developing a killer new Lisp Web programming framework called Symbolic Web (SW) brushed aside my enthusiasm for Dojo and mentioned he found qooxdoo interesting, although for now SW uses a little jQuery. "A little" because the whole idea of Symbolic Web is to do most of your programming in Lisp which of course is the only way to go but SW is still emerging from the sea and I had a requirement now (well, six weeks ago, but there is major new release of SW just out I need to look into but as this Road will reveal I am not likely to change horses now because of what I found in qooxdoo (not that SW will not eventually catch up)). Gasp.

The second question might be how a Lisp god and Interwebby know-nothing ended up doing heads-down intense Interweb development. Well, it was the Lisp credentials. A former client using mostly Lisp was having great luck with a Ruby/Rails front end but was looking for more speed and hoped having a Lisp image serve the pages directly would make for better response.

What happened next was pretty neat, and again Lisp conquers all. I told the client I would look at what they had and what they needed for a few days to decide if even I could help them, because I can do a lot of things but one thing I cannot do is exaggerate how little I knew about Web programming and even static HTML eight weeks ago.

A glance at the existing screens and Rails code had me thinking, "No way", but I was curious anyway about even getting an application to drive a web site so I fired up a tutorial on Web app development using tools from my Lisp vendor of choice, Franz.

How good are Lisp, Franz, the tutorial, and the WebActions tool? In the three days I was supposed to be deciding whether I could help the client I managed to execute a respectable fraction of the first component they needed. Sorry, no screenshots, NDA and all that.

So now there I am with a contract and a browser toolbar sagging from the weight of bookmarks to sites on HTML, CSS, and for the hell of it jQuery though I was not sure why but everyone was jumping up and down about it. And then I needed it.

Well, not jQuery. I needed a razzle-dazzle spreadsheet-like grid presenting data from an arbitrarily large table of data held on the server. Now at this point I had already rolled my own Ajax paging mechanism, but something superficial but nonetheless important for that this being an important application was holding me back: my HTML looked like crap. Meta-holding me back was my sense from surfing the Web that getting good results out of HTML/CSS was a black art requiring years of practice which certainly leaves me out. Enter FlexiGrid. A URL which as I write responds "Please contact the billing/support department as soon as possible."

Which brings me to why I dumped jQuery. What I found was small, yes, and add-ons, yes, but a hodge-podge of add-ons does not a professional development environment make. By the time I punted I had contribs from three people and while each was lovely alone together they looked like, well, a great decorator on a multiple-personality binge. And beyond appearance there came a surprising result: FlexiGrid did not offer a "row selected" event. Meta-worse was the author's response. "Right, no row-selected event." Ooops. Next up was jqGrid and that was fine but I still had Crazed Decorator Syndrome going and now I am looking for the next widget I need and a light went on: we are building an application here, not a Web page/site. It is wonderful that jQuery is small, but small is exactly the wrong thing for developing a sophisticated application with a coherent look and feel never mind coherent engineering underneath so widgets can play well together. Enter Dojo. But first...

In case you are wondering where the client went, they are encouraging my search for a pre-fab solution. Probably seeing the gorgeous FlexiGrid widget compared to my ugly efforts did not hurt, but the client has good instincts anyway (they use Lisp, right?) so they are leaning towards my original assessment that I am overmatched if I am going to try to turn out a hot application on short experience with HTML and CSS even though at this point I have wondered aloud to them how much HTML/CSS I might have mastered in the same time I was spending beating my head against pre-fab, this being the classic failed deal we make with the devil when we reach for 4GL: yeah, wow, a complete browser screen in five minutes. Gee, could we get the first sub-item appearing on the same row with the item? No. The client wants it. It cannot be done. That was Datatrieve on VAX/VMS twenty-five years ago, but nothing has changed. 4GLs get their apparent power by making decisions for me which works only if I (and my client) are not making any decisions.

But no, the client says "find a framework" and I decide to try again with Dojo. Again?

I had looked at Dojo before. The word on Dojo was incredibly good. Of course that word was from their Web site. When I tried to confirm by eyeballing the doc I could not find it. I mean, there were some great links, but all the pages they led to were blank. Next! But now I was back, encouraged by the news that OpenLaszlo had just placed a bet on Dojo. IBM as well, but OpenLaszlo is more important because I know they are geniuses, they have a dataflow hack like mine.

Oooh-ooh! Another reason Dojo looked good was a benchmark I had found showing the thing was fast. I have no idea if the benchmark is valid, but the pictures are gorgeous so I trust it. So why did I dump Dojo?

Well, OK, I did not really. It came in second. The question is why did I keep looking such that I found the eventual winner. Easy: too frickin hard to get to work. The documentation is awful. They know this and are addressing it but did I mention we need this now? Overall Dojo fit the bill of being a Compleat Solution with solid backing and a great future, but after banging my head against the wall over something (ah, it was the six hours I spent trying to figure out the "build" mechanism for a compressed production release of my JS culminating in watching a blurry video that finally showed how to do it) I went for a better way surf (as in there's gotta be) and saw YUI touted as a serious solution.

A quick glance at the enormous volume of well-organized documentation and exhaustive examples and the client and I agreed at once to give it a try. And I will tell you right now YUI has the best graphic designers, because their stuff looks hot. Dojo is a close second there, too. YUI is a complete solution with a big backer, gorgeous style, and more doc than you can imagine. How did it get dumped?

Stuff was not working well. Even with all the doc and examples it was hard to get working. When I dug into the source to see why things were not working, I did not like what I saw. And the death knell: YUI abandons the declarative model. Why is this so bad? Hellooooo, Lisp? Remember?

Up thru Dojo I was programming in Lisp 90% of the time, doing just enough JS to get information back to the Lisp server application where HTML could be generated and sent back, even that Lispily thanks to HTML generating macros. And YUI was going to make me give that up and be as hard to figure out as Dojo and I suspect worse under the hood. So regretfully I informed the client my battles with Dojo would be resuming.

So how does this road end up at qooxdoo?

The weekend was coming up. Weekends are mine, so I decide to follow-up on Lars's hint about qooxdoo. qooxdoo is the ultimate threat to YUI because qooxdoo also says Just Code Javascript, and if I am forced to do that by YUI I may as well do it in qooxdoo where the code is as good as YUI's is bad. (I looked at about fifty lines of it for five minutes, I should know.) Anyway....

qooxdoo is amazing. They need help on the graphic design (never look directly at their tab control, use a mirror) but other than that the engineering is terrific. [My bad: they simply leave anything more than the utilitarian look up to the developer, witness this before/after case study.] The documentation and examples are also abundant here, and the support is great. They even do dataflow! Yes, I wish I could spend all my time in Lisp sending over HTML, but two things on that.

One, there is a side-project that brings a declarative model back into qooxdoo. QxTransformer. I have no idea where that stands, but I did see a note on-line where a developer spoke of it in the past tense. [Now resurrected.]

Two, hey, I can write some Javascript. I did ten years of C before I did fifteen years of Lisp, Javascript is not a problem. And it even has lambda, something this guy still does not understand. Lisp/markup would be better, but when dealing with mission-critical stuff (the client is doing valuable data crunching but does not win until they make it accessible) any team can roll up its sleeves and crank out the work even if they have to do without the brilliance of Lisp.

This entry is getting long and I am getting tired and my bartenders are starting to worry about me so I will leave a detailed praise singing of qooxdoo to my next entry, but allow me a bit of a rant. What a mission-critical effort cannot stand is fundamentally flawed solutions that seem to offer a fast track but in fact bleed developers to death. This can be jQuery with an incomplete solution forcing me to try to build an application out of mom and pop contribs or YUI with fundamentally bad code or some other framework that might be just plain slow.

As a developer I am not looking not to work, I am looking to have my work yield quality results predictably. I might spend four hours trying to figure out how to do something in qooxdoo but once I do it turns out to be insanely simple and work beautifully. Moral: their documentation needs an index! But I am keeping my big yap shut because documentation is a bitch and they have done a ton of it and I do not document my stuff so...I keep my big yap shut. But the point is that my work now yields results reliably and predictably and this will accelerate as I learn the framework and even learn better how to navigate the documentation.

qooxdoo not only fits the bill of being a complete solution well documented with many examples but it also has high-quality code under the hood, which both gives me confidence that I have found the right tool and not incidentally serves as useful further documentation. More next time.

Why Lisp Packages are so Easy (Hard)

Slobodan Blazeski wrote:
> May somebody please explain me with simple words why cl packages are
> hard to use, 'couse I really don't get it?
> The best is probably to hear from somebody who just learned lisp or
> somebody who teaches lisp (do such people exist anymore?) and his/her
> students has problems with packages.

...or a good teacher, defined as someone who remembers what it was like before they knew what they were teaching.

The first problem is one of those immune system deals. Packages have a lot of the same proteins as, say, C dot aitch definition files so the recognition systems of programmers coming to Lisp from other languages (you know, everyone) does its damndest to parse them as dot aitches.

In fact, packages have nothing to with anything but symbols, neither the functions nor global values they might name. More expansively: packages are about mapping string names to first class Lisp symbol objects and dividing this up into discrete mappings called packages so the same string "Hi, Mom!" can map to multiple |Hi, Mom!| symbols in multiple packages.

The second problem is that symbols take us straight to black-belt Lisp in which one must understand the Lisp reader and fancy things like read-time and compile-time and all that. Example:

I remember getting messed over because some symbol was mysteriously appearing the base CL-USER package. I could not for the life of me see where it was being introduced. I turned the problem over to a few trees worth of monkeys and one of them came up with this rule:

"Never use a normal symbol like |Hi, Mom!| in a defpackage form coded in a source file with (in-package :cl-user) at the top which would likely be the default even without that."

Instead, use a string like "Hi, Mom!" or an uninterned symbol like #:|Hi, Mom!|.

Strings are dicey because you might want to export 'banana and be silly enough to code "banana". Lisp will take you seriously and create |banana| instead of BANANA, case sensitivity being what it is except when it is not when Lisp folds 'banana to 'BANANA which is what it does unless you are in Modern (case sensitive) mode.

Did I mention "black-belt"? The fundamental idea is simple, but noobs get thrown off mostly by the non-simular simularity with C and then in the debugging stage by case sensitivity seeming to come and go at random and the whole read-time compile-time thang.

hth, kxo

A Beginners Guide to ASDF (Ha!)

OK, there is at least one thing about which Lispers are not so smug: ASDF (Another System Definition Facility), the de factor standard for the Lisp library equivalent of Makefiles, and my post today was prompted by ANFOBA (Another Noob Effed Over By ASDF) washing ashore on comp.lang.lisp, and how I almost responded there:

Francogrex wrote:

Hi, I am struggling to understand how to work with asdf.

I have been using it for ten years and I am about 50-50 now on getting it to work. I am assured it does a great job in working with a multitude of Lisps/OSes, and I can assure you it is a disaster at working with programmers.

Come back here and post console output freely, you will need help, especially because one of its great triumphs is useless error messages.

Hints:

(1) It will work. You will have to promise it your first-born, but it will work.

(2) No, PB was not joking: you have to separately push locations onto some magical special variable. Leave off a slash (or add a slash, I forget) and ASDF will take your head off, so don't do that.

(3) No, PB was not joking, the syntax is:

(asdf:operate 'asdf:load-op :cl-pdf)

Why is it not (asdf:load-system :cl-pdf)? In my experience, extremely smart programmers think it astoundingly brilliant when "one function does everything!!!!", remembering the ridiculous op-code first parameter left as an exercise.

Since you asked, this is because extremely smart programmers are tone deaf to programmer productivity; they are so smart they do not even notice when they have created something unuseable, they just brain through it.

(4) ASDF has a featurebug: it loads in the wrong order in order to make McCLIM work. Don't ask. If the author of the .ASD file retreated to the desert for forty days to slave over the dependencies this will not be a problem. Otherwise do what I do: take a sledgehammer to the beast:

(asdf:operate 'asdf:load-op :cl-pdf :force t)

That abandons the whole point of having a build system, but it gets ASDF to work. When for the love of god I am just trying to build this frickin thing!!! that is a nice tradeoff of compile-time vs tracking down and short-sheeting the author of ASDF.

(5) On success ASDF emits more messages than an IBM OS/360 core dump. This is so any errors will scroll well out of view and anyway stand out like a healthy thumb from the surrounding...um, other healthy thumbs. Anyway, do what I do: ignore all messages other than a backtrace and Just Try the Hello World(tm).

If you get a backtrace, Just Post It to c.l.lisp.

(6) If hello world fails, f*ck! But at least now you are a Real Lisp Programmer(tm), this is what we live with.

hth, kenny

The Foisting of An Infinite State Machine

"I will not allow you to foist this convoluted scheme on the bank!", Sam yelled.

This was fun in a lot of ways. First of all come on this is a bank, three or four full floors of programmers all enthralled by nothing more technical than getting through the day unfired and back on the train home to their families and here is this aging cardpunch nutjob throwback not only noticing Someone Else's Code but caring?!

Second, I did not make that up. Sam actually used the word "foist" without trying. I use SAT words all the time but I am usually trying, you can see my eyebrows arch a little as my language organ reaches for the thesaurus. Nothing shabby about "convoluted", but "to foist"? And Sam tossed that off in earnest in anger in the heat of diatribe sans affectation and it is not clear to me I have ever before or since heard the word used in vitro. Which brings me to the third fun bit.

Sam was Jewish. Classic Orthodox home-by-sundown-on-Friday Brooklyn Jewish and of course he would be good with words. Right, get all PC on me and denounce me but what can you say?, he used "foist" in a sentence on the fly and I think I win on this. So there he is with this classic Billy Crystal Princess Bride accent yelling at me for foisting convoluted code on the bank.

"I will not allow this!" he cried. "I will stop you!"

Oh, please, tho Sam was a sweetheart. Yeah, "was". I heard he passed, Lord keep him, a beautiful soul. I could have choked him in that moment but at least he was attacking me for Code Quality. What was my sin?

I had been tossed a throw-away job of persuading two applications to exchange information reliably when either app was free to collapse in a heap at any time and I was not to lose any information. Nowadays we just pull packages off the shelf for that but this was a while ago. Oddly enough in my first IT job ever I had had to do the same and it was fun because there was nothing theoretical about it, the two applications did even during development reliably disappear left and right allllll the time. Decnet, PDP-11, RSTS/E, 1981... you understand.

What was different was that somewhere in the intervening ten years I had gotten this crazy idea of creating a domain-specific language to make it easier to develop a software Algebra tutor so I had gotten a book on compiler design because I did not realize I could Just Use Lisp. I do not like reading so I did not get very far but fortunately one of the earliest chapters was on parsing so I ended up learning about finite state machines. Oh, sorry.

This post came up because I am doing some Web2.0 programming these days from Lisp and I wanted to send over Lisp sexprs and since Javascript for some reason lacks a Lisp reader I was going to have to parse it myself and after ten minutes of trying to do it without a finite state machine I realize, damn, I should code up a finite state machine.

I will not in this space try to explain what is a finte state machine but it is really simple and powerful and no matter what level programmer you are you need to have it in your, well, toolbox. I think the example I learned on was a pocket calculater when folks still had those. Your initial state is "initial". Duh. Then what happens? Oh, I got "/". Bzzt! Or just ignore it and my next state is still "initial". Now I get a "1". Great! Save that, my new state is...I dunno..."got a digit". We're on a roll. But wait, what other inputs could I get? A "+". Ummm...well, it seems no different, but we'll say "got a plus sign", and indeed when we are done one of the cool things that can happen is that as we fill in the table we may well discover that the row for the state (did I mention that rows were for states and columns were for inputs? Sorry) ..the row for the state "got a plus sign" might be no different from the row "initial" and we first go "whoa, enlightenment" and then collapse the two rows into one. I digress.

But the idea is that we just break it down into cases. A fun example is that we now know whether a "+" is addition or a plus sign (or an error) without really trying, because we have aggrssively exploded the analysis into all these different states like "just got a digit" or "just got a left parens" or...yeah, it goes on, but guess what? Here comes the name! "Finite State Machine"! There are only so many!!! You may think there are a kabillion but after ten you are done, maybe after twenty. And you can collapse inputs (1-9 become "positive digit") and so you have a finite number there. Do you now have 150 cases to handle? Yeah, and each one you can code in your sleep. The alternative is one hyperglutionous mass of conditionals, tests, and branches that will fail on the one millionth execution because of a first time path execution.

Back to Sam.

I have realized that the reliable exchange between two autonomous systems permitted to fail at any point will best be solved using my old friend the finite state machine and I have enthusiastically shared this with Sam. Somehow I am now being denounced for using an algorithm learned from the top book in compiler design. Specifically, I am being denounced for foisting. I will be stopped. The bank will be saved from my convoluted scheme.

Ooooookayyyyyyy.

Well, what can one anachronistic lost in the Torah yamaha wearing card punching goto coding nut job do? Would you believe he could force a code review? Nay, a defense. Sam forced a frickin defense of about thirty lines of the best simplest most powerful correct code that bank had ever seen. Mind you Sam is also the man who produced my favorite line of code ever, using the Vax Basic statement modifier mechanism:

return if flag

That's right, an action at a distance code teleportation straight out of the middle of seventy line for loop using a flag he had named.... flag. You can imagine my reaction. And this guy is challenging my code?

If you are not yet suitably horrified, understand that novice programmers were regularly crashing production runs because no one anywhere was looking at their code and they were going to call a meeting of like twelve programmers to review my foistation of convolution upon the bank? Sorry, my ire is showing. Chase cut to.

Code review/defense: twenty minutes, here it is, any fucking questions?

Yes that was pretty much the tone of my presentation -- I like to consider people skills one of my strengths. And then Sam spoke and you will see why I loved him. At pain of repetition, probably no one else on those four floors would have even listened to me as I bragged on my FSM, but go to the mattresses to stop what he thought was bad code? One in a million. And one in ten million in my experience would have said what he said.

"OK," Sam he said. "That looks fine."

Of course now I am pissed off because I am just getting my turbines spun up for a full bore antler to antler knock down drag out kickin and a gougin in the mud and the blood and the beer round fifteen thrilla in manila finish and Sam is saying...fine?

"That is simple," Sam continued, gesturing towards my whiteboard diagram. "This infinite state machine idea is complicated, but the way it works is simple."

The beauty of that being......no. I would only ruin it.

RIP, Sam. RIP.

URBBR #2: How Bad Is This Book?

[The following has been... well, only the names have been changed to protect the endangered toads.]

We have a One-Paragraph Test for fiction. Not a complicated test, it involves reading the first paragraph and then deciding. If We are really undecided We can read a second paragraph, but that is a Bad Sign.

For non-fiction We look at the blurbs the author managed to suck out of unwilling blurbers. Well, OK, We do not really want the blurbs to be from blurbers, We want them to be reviews and We know they have cherry-picked the review sentences so everything gets discounted about an order of magnitude. 

And there are two such reductions applied, the second for the source of the review. Kirkus Review of Books... well, just quoting them gets a book back onto the shelf. In the wrong section, upside-down, binder in.

Here is the Gold Standard for non-fiction back covers (hmm, the URL goes to the whole book, be clever and click "Back Cover"): The Glory of Their Times. Yes, you need to read this book, and no I do not give a rat's ass if you like baseball just read it.

We turn now to a book I have been asked to review, which will go unmentioned for the same reason I restacked it binder-in:

"This work strikes a balance between the pure functional aspects of Blub and the object-oriented and imperative features that make it so useful in practice, enable .NET integration, and make large-scale data processing possible."

The clever unwilling blurber always manages to say something nice by talking about something else, in this case the language .... instead of the book for 90% of the blurb. What does he say about the book? 

It strikes a balance. Wow. How big a put-down is that? Note that the reviewer did not say "a perfect balance" or even "a good balance". And what the Hell does balance buy me anyway?

The good news is that We cannot discount that 90%, Our micrometer does not go that low. But We better check the author of the blurb.

—John Doe, PhD, Researcher, Blubber Ltd.

Oh. The people selling Blub.

The neat thing is that We now know the book is awful and that John was slowest to hide under his desk when the marketing people came around looking for a recommendation.

Unread Book Review: Zen Style Programming, Nils Holm

On comp.lang.lisp, Nils M Holm wrote:

> I am happy to announce that free pre-release copies of my latest book

> 

>           "zen style programming"

> 

> are now available.

> 

> Blurb

> 

> The primary purpose of programs is to be understood by fellow human 

> beings,

Nonsense! Nils has been reading Knuth or something. The purpose of programs is to model other processes so the most important virtue of a language is the ease of building programs with it. Yes, those programs should tend to be clear but that just requires powerful constructs, long names, and support for the functional paradigm. 

Besides, "revise"  means "rewrite from scratch anyway because the way this was done is all cocked up" so we do not generally have to understand programs to work on them.

> hence programming languages should have certain properties:

Bzzt!  The source of obfuscation is the programmer, not the programming language.

>     * Small size and uniformity

Hand-waving! Nils has been reading Graham or something. Small size as a requirement does not follow even if one were to exalt human readability to Prime Directiveness. You could just have good names for functions, like pathname-sans-file. And an editor that knows how to get to a hyperspec.

     * Unambiguousness

     * A high degree of abstraction

     * Achitecture neutrality

 

 The first part of this book introduces the concept of functional  programming and describes a purely symbolic language that fulfills these requirements. This language is a minimalisti variant of Scheme whose only data types are symbols and ordered pairs. Nothing else is required to describe algorithms for solving a variety of different problems.

The second part of the book shows how to apply the techniques of  symbolic programming to some problems of varying complexity. Topics discussed in this part range from simple functions for sorting or permuting lists to ``lazy'' data structures, regular expression matching, formal language translation, and declarative programming.

The third part, finally, shows how to implement the abstraction layer that is necessary for solving problems in an abstract way on a concrete computer. It reproduces the complete and heavily annotated source code for an interpreter of symbolic LISP and provides an example of clear and readable C code.

This book contains the full source code to a source-to-source compiler, a meta-circular interpreter, a logic programming system, and to the language that is used to implement all of these. 

Damn, that was a lot of work. I should be nicer to Nils. Here's the book, anyway. Let us see what it says:

"The first points are no-brainers."

Come on, Nils, we talked about the hand-waving. 

"If a language is too complex..."

Well, by the meaning of the word "too" I think you are on safe ground for the rest of that sentence, but it begs the question of how too is too? Nils needs to make his case or judge someone else's case, not both.

"...programmers will have to look up things in the manual perpetually instead of concentrating on the actual problem."

So now we need programs to be understandable by people who do not know the programming language? I think that was a design goal for COBOL. Anyway, that does not give you a language with a small instruction set, it gives you a language with long, well-chosen names. Like COBOL.

As for looking things up, I want to meet the guy who looks up delete-duplicates when the status bar is saying:

    delete-duplicates sequence &key from-end test test-not start end key

Let's see if there are any more self-serving rationalizations....

"The class of problems that can be solved by programs is much smaller. It typically involves very clearly defined tasks like

-- find permutations of a set;

-- find factors of an integer;

-- represent an infinite sequence;

-- translate formal language A to language B;

-- find a pattern in a sequence of characters;

-- solve a system of assertions.

Of course these tasks have to be defined"

Nils must be a professor of computer science. Programs exist to model real-world entities of arbitrary complexity, tho good luck with the human mind. OTOH, simulations of car crashes are admitted into evidence by finicky courts and some people even model human private Algebra tutors. Unfortunately those things are a ton of work which is why you will not see academics doing that.

Now about the title. Zen zen zen, what an affectation! Zen this, zen that. I am going to write a book on The Zen of Ordering Take-Out Pizza.

What is zen about zenlisp? Never justified, never even addressed. Does the author think zen means small because it is a small word? I like that, actually. I tried googling for synonyms: oops. I think they would be pleased down at the monastery. 

So much for the name. How about the language itself? Raison d'etre? 

Zenlisp is similar to Scheme, but simpler.

That sounds like being rescued from a desert island and then dropped off in the middle of the Sahara. No, the only reason ZenLisp exists is because Lisp is a venus flytrap of a blackhole delivered by John McCarthy an alien from outer space sent to stop good programmers from Actually Programming. Apparently Grace Hopper was just a few lines of code away from anti-gravity and time travel so They sent John down with Lisp so any good programmer would get so excited about the language itself they would stop programming time-travel applications, take out their CodeWarrior CDs, and start implementing a Lisp using (wait for it) C.

The ones who really want to avoid programming will then write a book about their language. 

So I wonder what Nils will do now.

AA, BB, CC, and DD

I am not making it up. Those were the datanames in the code I had inherited from the Wunderkind. So this will be the most boring stating the obvious war story I ever write but that is not why I am writing it. I am writing it because the Lisp subset of the human race is at it again running factual red lights with their prior conviction feet to the pedal.

Yes, kiddies, we revisit today The Unbearable Impenetrability of the Lisper. The cool thing being that Arc is again involved albeit peripherally in this latest train wreck of human comprehension.

Back then the Maddening Crowd was utterly fascinated that I liked Arc which I did not but they were not to be denied, the best part being those people who responded to my objection to the crowd's misperception by saying yeah I saw you were flabbergasted by everyone thinking you liked Arc so tell me, you like Arc? 

No I am not making that up. Twice. 

This time it was my scrimshaw-ready datanames, sample below. Some months after my exploration of Arc I had a laugh as I whipped up a DSL for my Algebra software and found myself approaching Arcitude in the brevity of my names.  I posted something to comp.lang.lisp inadvertently loosing The Hounds of Lisp Density. A sample:

(hard

    (dsb (b x) (rp 2 (rv))

      (m/ (m^ b (m* (r2 7) x))(m^ (xqv b) (xqv x))))

    (dsb (b x) (rp 2 (rv))

      (m* (m^ b (ms* (r+ 7) x))(m^ (xqv b) (ms* (r+ 3) (xqv x)))))

    (dsb (b x) (rp 2 (rv))

      (m*eo (ms^ b (r+ 7))

        (m^ (xqv b) (ms* (r+ 3) (xqv x)))))

    (dsb (b x) (rp 2 (rv))

      (m/eo (ms^ b (r+ 7))

        (m^ (xqv b) (ms* (r+ 3) (xqv x)))))

    (dsb (n d) (rv 2)

      (w (k (r+ 12))

        (m/ (m^ k n) (m^ k d))))

• dsb is short for the Lisp destructuring-bind.
  
• m/ is short for make-fraction.
  
• m*eo is short for make-product reordering the factors randomly (either-order)
• etc etc

I gave a tip of the hat to Arc and explained that I had done this before (in C) and that this code (to generate randomly many varieties of Algebra problems) was a known PITA and even in C I had used the C preprocessor to likewise make the coding manageable.

Enter the Savages of Comp.lang.lisp.To a geek they lectured me on the importance of nice long meaningful names, or as His Sulzberbergerness edified me a ways back what Confucius called The Rectification of Names which is not quite the same thing but I can never resist name-dropping either Jay or Confucius. 

I responded to the jackals nipping at my heels that yeah I know but in this case with vast incessant repetition of a small collection of opcodes that recognition would not be an issue and that it was much better to diminish the low information content (cue Shannon) of long names (what exactly does make-product add to m* in a context where a leading m is used only for makers?)  and learn a dozen opcodes which were mnemonically and predictably built anyway from atoms such as M and * and EO.

The universal response was that longer names were better. My universal response was that these special circumstances flipped the arrow on that otherwise sage rule.

The universal response was that longer names were better.

Trying again, my universal response was that I agreed, but would anyone like to address the salience of the special circumstances I had suggested were germaine, perhaps explaining how they were not special enough or too special or the wrong damn color?

They all responded that longer names were better and I really started to enjoy things at that point. I was reminded of this desert spider that had a routine for burying any captured wasp and it involved positoning the wasp up just so and then going to dig a hole and then dragging in the dead wasp and these researchers would move the wasp a little while the spider was digging so the spider would be thrown off and start again by repositioning the wasp and no matter how many times they moved the wasp while the spider was digging the spider would just start right over readjusting the wasp and then digging the hole. 

The topper was I myself was a staunch proponent of good long names. Tilton's Law of Programming:

 Spend more time on the names you choose than on the algorithm.

 As for the specific quality of length, we have Tilton's Rule of Abbreviation:

Abbreviate no name less than seven characters long and then only if a good abbreviation is no more than half as long. Rounding down.

Which brings us to AA, BB, CC, and DD. I was working as a body shop consultant in easily my most Tall Building job ever. One day this new guy came on board, totally not GQ, scrawny, smart, energetic. I had no idea he was a first round draft pick, destined for greatness. An employee, by the way. Tom. He dives in and starts churning out a front-end application, learning the HLL and OS and tools all at once, a man after my heart. 

At one point he mentions to me a problem. He wants users to be able to make discontinuous menu jumps, say, go sideways without backing up to the menu above (yeah, this was the good old days of modal interfaces) and the programming language would not go sideways. ie, He was using HLL recursion to handle nested menus.

I suggested the obvious: no, you cannot call sideways in a structured language, you have to return from the called function with a "message" always checked by the caller to see if the user should be taken somewhere else. Tom yelled Great! and tore off to code it up.

A couple of weeks later I have inherited the system. Tom was smarter than I: as soon as he had chalked up the win he told his boss to "get some consultant" to maintain it. Moi. So there I am working on the first RFE and I am perusing the code trying to figure out how the hell it works and I find myself slowed a bit by the data names that pretty much controlled everything: AA, BB, CC, and DD.  Come on, you think I could make up names that bad?

So over I wander to Tom's desk, clear my throat, Tom looks up.

"Tom," I said. "I was looking at the code. AA? BB? CC? DD?"

Tom burst out laughing.

"Those are just temporary variables!" he protested, laughing even more.

"Un-hunh," I replied. "And they control the entire program flow."

"Well, change them if you like."

As I said, Tom was a smart cookie, he was wiping his hands of the whole deal.

Now it turns out that one of the things I like to do when working on OPC (Other People's Code) is to stare at it and stare at it and when I see some crucial variable playing a big part in things and its name is getting in my way I pick a better name and do a global change, rinse, repeat until the damn code makes sense. The only reason I had gone to talk to Tom was the same reason we pay to see a two-headed sheep at the carnival.

Two hours into the renaming I had come up with decent names for AA, BB, CC, and DD (that last  one turned out to be three different variables) and I was making the global changes eyeballing each as I stepped through the source and after one such change forgive me I will never in my life remember the specifics but imagine you have just made the substitution and are now looking at a line of code that says:

  total-weight = total-weight + this-length

I did not feel completely comfortable with that edit so I called Tom over. Did I mention he was a smart guy? Two seconds into explaining how I had gotten to that point he yells out, "Great! I gave up on finding that bug!"

Smart guy. Tough bug? He just moved on and chalked up the win. Let some consultant take over the code and run into the bug and think they introduced it.

But there you have it. A bug so hard to find that a very smart programmer gave up on finding it and someone who did not even know the bug existed changed a few datanames and the bug positively jumped off the page. Explaining the corollary to Tilton's Law: 

If the names are right the algorithm will write itself.

Now if only some walking fencepost from c.l.l will post a comment saying... well, that would ruin it, would it not?

Now Batting, Dave. Why is Mariano Rivera not worried?

The venerable Mr. Roberts draws on his profound mastery of things algorithmic as he intones:

I'm actually quite conscious of cons-ing, but I also believe that obsessing about cons-ing leads to premature optimization and far greater numbers of bugs. 

We are fortunate to have the benefit of his wisdom and it occurs to me that we can keep talking past each other enjoyably for weeks like this so i will not suggest we try to establish a concrete example where he would code append and I would code nconc and ruin all the fun. But it is a good question especially if we agree (we do) that as a rule the functional paradigm utterly Rocks the Casbah so how on earth am I going to get a list back from some function I call and not be free to nconc it? The moral being I am right (will it never end?): if we try to find a point of Actual Disagreement(tm) we will fail so let us not.

Getting back to the fun abstract theoretical conflict, the problem is a policy of copying which leads straight to the solar power objection aka death by a thousand cuts which in turn leads us back to the simple question of why we do not just code the right operator at every turn? Genius is in the details which means we are always on our game bringing me to Mariano Rivera.

For those who do not know Mariano plays baseball and is starting to be called the best ever at his position, namely "closer" which means we won the first eight innings and do not want to lose the game please do the rest and he does. Even in the seventh game of the World Series that he lost every hitter broke their bat but the three balls involved known in the game rather morbidly as "dying quails" found their way to the grass. Hard to feel bad about shattering three bats. I digress, Mo is good.

Genius. Details. The great Nconc vs Append War, Roberts-Tilton III, nothing like this since Ali-Frazier, Affirmed and Alydar.

It was Just Another Game. The Yanks had won, Mo had saved. They always collar some poor sod and keep him from his shower and car service home to make those of us who pay to watch baseball on cable TV feel better and tonight poor Mariano's number (42!) came up and by the grace of God I pulled my face out of a carton of cold noodles with seame sauce just in time to witness a perfect moment in sports broadcasting and as well a master class in excellence any time any where.

The announcer on the field for the interview went for a home run question, something way more astute than the usual "How did it feel getting the big out?" and he came up with a beaut: 

"Billy Bob is a rookie and you have never faced him before, there must not be much of a book on him," the announcer observed. "Was it tougher facing an unknown quantity?". 

"No," Mariano replied. "I went out during batting practice to watch him hit."

Mariano's disclosure that as a Hall of Fame shoe-in and multi-millionaire perennial All-Star participant he had looked down at the lineup and seen an unfamiliar name and broken off his clubhouse routine to go watch some ridiculous kid take batting practice had no small effect on our seasoned, professional, expert baseball announcer:

"You did that?!!!!!!". He blurted out, hopping a little in the air, lurching towards Mariano. "You went out to watch him take batting practice?!!"

Now it was Mariano's turn to be amazed, leaning back a little to create a margin of safety.

"Of course," he replied. "I get paid a lot of money."

Doodleoodleoodleoodleoodleoooo. Game over. The announcer (and I, I confess) had been seriously schooled. 

We sit at home and watch these athletes precisely because what they do is so wonderful to us and we endow them in our minds with magical and wonderful qualities because at an early age we were robbed of angels and dragons and Santa Claus and even monsters in the closet and trolls beneath the bridge and then every once in a while we learn that the miracle strikeout we saw with the bases loaded happened because a multi-millionaire punched the time clock, checked his glove for broken laces, sharpened his spikes, and checked the opposing team's roster for any names he did not know.

I say learn the damn language.

Fear No Evil II, or Why Solar Power Sucks

Just a quick note cuz I remembered something that answers Dave's well-reasoned comment:

My own personal rule on this is to use functional programming whenever possible and let the GC deal with it. 

Not much of an excerpt, but it says it all: modern Lisp GCs truly Rock the Casbah, and I myself am on record as denouncing the flipside of this blogcoin, consophobia. Actually, there may be a problem, Dave says "whenever" possible and perhaps then we agree! 

But "whenever possible" has a ring to it like "oh, you know, all the time" and I actually had a good reason for not doing that, I just forgot it. Has to do with solar power. 

The reason solar power sucks even though there is an overwhelming amount of it is that it is diffuse. Yes, enough sunlight falls on Kansas in a month (I am making this up) to power Earth for a year, but just try to find a solar panel as big as Kansas, never mind deal with the Kansans or the inefficiency of the transmission to Australia. I digress.

The problem with a "Damn the consing! Full molasses ahead!" policy is that when ones application drags to a halt there will not be a bottleneck to fix. There will just be hundreds of places one was too lazy to think about whether one could use the right construct. And correcting them all will be as painful, tedious, dreary, mind-numbing (sound like fun yet?) and thus as bug prone as it would be easy and fun whenever reaching for a list manipulation tool to challenge oneself with, gee, can I cut in the afterburner? Stand back, GC! Captain Destructo is in da house!

That is a practical objection. My ethical objection (viz, hey, learn the language) is not all that far behind: jeez, I am building this list right here in front of my eyes, why am I not using a destructive operation for the next step? Do these same folks all have a copy-n-sort function in their personal toolkits that begins with a copy-list on the sequence to be sorted? 

No, they do not. Why not? Hello.

My apps periodically get bogged down. I wait until they do and then dive in for what I like to call Speed Week after a regular TV extravaganza on vehicular racing. Normally this turns up interesting algorithmic gaffes I do not feel too bad about but one time the smoking gun was the most innocuous little needlessly copying statement you can imagine write at the center of the heaviest hitter bit of code you can imagine. This is pretty much the opposite of the solar power argument -- there was just one little guy to track down -- but without the code profiler offered by AllegroC I am not sure how long it would have taken to find that. 

And now I come back to the main point: er, exactly how hard is it to know when one can use destructive list operations?  So why not use them? Only one excuse comes to mind (fear) but always copying is not even a clear win (ie, it can totally suck at times) so.... learn the language?

If I was going to cut the Copying Cowards some slack I would say, shucks, not everybody writes insanely intense tight loops. But then I remember a point I made in FNO/1: we get noobs all the time struggling with slow code because of excess copying. Perhaps complex applications and tight loops are orthogonal?

I say learn the damn language.

Chomsky! Pinker! Get In Here!

[OK, OK, it is a Lisp blog but if I can refute Chomsky's entire body of work on language acquisition in one post, why not?]

These academic PhD geniuses have a problem. They are geniuses. They think they can figure everything out by thinking, like Feynman fixing the neighbor's radio at the age of three or something close to that by thinking. Great story, buy the book. Forget which.

Chomsky and Pinker merely repeat the error of the AI crowd who sniffed at NI (I just made that up, it means "natural intelligence") and said, "NI?! NI?! We don't need no stinkin NI!". They said that because they were geniuses they knew that what NI did with pretty much organic chemistry they could stomp with (drum roll, please) reason.  Ooooohhhh, excuuuuse me!

I guess I can leave Pinker out of this, he did not add anything to Chomsky, he got drugged in only because a correspondent mentioned his "work" on blank slates. I love that "work": Oh, let me lean back and draw on my pipe and I have a PhD so if I think babies learn something too fast they learn them too fast, dammit! As if. And believe it or not I am coming to my point. But first a science trick, a prelude to my witnessing of the refutation of Chomsky. It has no bearing on anything else but it involves a cute girl and something I could not believe when I saw.

There I am in a sports bar/shore bar celebrating a great week almost delivering my world-changing Algebra software when I see a lovely lass drop her drink. Three or four feet, to the floor. It was a plastic cup, which helped a lot come to think of it, because a glass glass being more rigid would not have absorbed the energy and there would have been a wicked rebound preventing what happened from happening: the drink landed upside down and just stood there, trapping half the drink inside. The bad news being there was no elegant way of finishing said drink, but that is not my point. Did I have a point?

Well, if I had to have a point, it would be that no one else was all that impressed. Clearly I need to spend more time in bars late at night watching drinks being dropped and I do not think it is possible to spend more time in bars late at night than do I so I guess I need to find bars with clumsier people, good luck on that.

Oh, and the other useless and irrelevant to the demolition of Chomsky/Pinker thing I learned is that it takes just one unauthorized speech on world affairs (well, he was mumbling pretty badly, I am not sure it was on world affairs) over the unattended band microphone to get escorted unceremoniously out the front door at the sports bar in question. Right, Chomsky.

So I look up and I see a skateboard competition, lucky enough to be watching the winner's winning run, and lucky enough for my brain to clear long enough to understand what I am seeing, a Johnsonian kicking of the rock I refute you thus Dr Berkeley of Chomsky/Pinker by some sixteen year-old if that. What did he do, and why did my brain have to clear? Two good questions.

What he did was give himself a push across a flat floor in a reasonably small room (no gigantic ramps and concomitant acceleration/stored momentum) and upon reaching a ramp at the same time push off in such a way that the skateboard (unattached to his feet for those who do not know) elevated with him to a height just above a rail (er, just an elevated horizontal pipe mebbe a foot off the floor) such that he could land on the skateboard in midair just as the tip of the skateboard landed on the rail and using his/their forward momentum proceed to slide along the rail for a couple of yards before again levitating off to land somewhere else to commence another trick. 

And before the anti-blank-slate morons can scream "Oh that is easy, I have the algorithm right here" allow me to break their hearts and add that as he did the initial elevation springing the board into the air with him (a yawner in these competitons) he took the trouble to pop his deck into a barrel roll (or two, who knows at that speed?) before regaining it to land the rail toe-slide. If the MIT geniuses think that is easy lemme just say that I have spent years in Central Park watching the same people practice on skateboards for years and they still cannot jump over a small backback and land on the skateboard on the other side.

So... these people have skateboard organs? Or do you think learning to barrel-roll a jump into a rail toe-slide is more important to them than is mastering language to an infant so their motivation is stronger? This would be a reasonable belief if one has never been in the presence of an infant who will join in any conversation it hears when it neither understands nor can generate one intelligible syllable.

Oh, and why did my brain have to clear? Because what I saw was a commonplace: inconceivable performance easily explained by sufficient practice. If a recent blog had not been on my mind I would not have given the skateboard performance a second thought, nor the motorcycle-flipping organ manifested in the next story on ESPN.

Come to think of it, this is about why Lisp wins: we simply are not as smart as we think we are. I like to say that I cannot write good code but I do know when I have written crappy code. Recognizing is easier than generating. Hell, crappy is just one bit, right? But how do we produce a huge stream of uncrappy bits? Good luck.

What we want then is an agile language that lets us create awful code really fast and fix it as fast as we identify the problems. Pretty soon our software organs cut in and we are creating not-so-awful code up front. Even better.

Now I just have to figure out an elegant way to finish a drink upside down on a barroom floor.

Buzz off, Charlotte

Omigod. I just made up that title because of the subject of this post and not five minutes ago I ended up daydreaming about a girlfriend I had back in college named Charlotte and as you will see there is no connection. Scary.

Anyway, it is 2:38 am and I am in the middle of a trivial but unwelcome re-de-refactoring with a scant few days to go before a self-imposed but jeez I gotta get this frickin thing out the door it is the middle of July deadline and it is an educational app  and how far is it from the middle of July to frickin September?!!!!!!! I digress. 

It is 2:38 am and a spider I swear to god no more than 2mm across is dangling in front of my flat planel trying to figure out this code (the last bit of the last line, to be precise):

(defun alert (s oks)

      (tk-format-now (conc$ "tk_dialog .oops {} {"

                   (format nil "I thought you were solving for \"~c\"? I do not see it."

                            (key-char sfvar))

                   (format nil "} {} {~s} {~s}" oks oks))))

Charlotte (look it up furriners, E B White) is freaking out over the double occurrence of the variable OKS. I know because the little hussy has lowered herself I swear to God ten feet from the A-frame ceiling of my ranch rental all the way down to within a foot of my desktop and stopped. How still is the air in here? She was swaying no more than another 2mm.  I digress. But how does a 2mm spider pack ten feet of gossamer into its, well, gossamer pack?! I don't care how thin it is, it is stll ten feet!

Charlotte was both deeply concerned and a sufferer of tunnel vision. She ascended to the top of the flat panel (I have it angled down to my slouched position a few inches above desktop), gained a foothold or five, and then inched...no, mmed over and redescended to get a better look at the first occurrence of OKS, just to be sure. It is possible she then pirouetted on her thread to ask why I did not check the CLHS on FORMAT to find out how to back up one argument and re-use it, but I cannot be sure. If my eyes were better, I could tell you. 

Well, I am in a hurry dammit and you know what I think I have the strength to type oks a second time and no I do not think the redundancy will lead to the loss of a satellite but if a damn spider complains I know enough to listen... ah, there it is ~:*, back up one arg.

Now buzz off, Ch... hey, where'd she go?

It's the shoes, Jeff! It's the shoes!

My buddy Jeff wondered aloud on comp.lang.lisp why We Lisp Gods were so much better than the rest of you at adapting to alien environments, aka other languages. Without coming down in favor of any, he offered several possible explanations:

1. We are just smarter than you un-Lisping dopes
   
2. We are accustomed to learning other languages because the food is OK in soup kitchens but the decor? Omigod!
   
3. We are older and you can't teach an old dog new... hang on.

Based on recent copious experience, you can safely forget #1. Lispers are precisely as dense as other programmers, and if anything more so: as the title suggests, we are being carried by the shoes. Anyone both smart and using Lisp is named Graham or Morris and either has or split over forty big ones, and no, a thousand smackolas is not "big".

#2 is a fine answer but is boring and knocks the legs out the upcoming argument so let us declare it false. That was easy

#3 is part #2 and part what I joked: old may be wise but it sure ain't adaptable, it already knows The Right Way, it aint listenin.

That leaves the title of this entry, a reference to one (or more?) humorous Spike Lee ads for Nike sneakers in which he insists the sneakers explain Michael Jordan's Unbearable Lightness, to which we see his Airness reacting with feet-on-ground dubiosity. IIRC, but I think I do.

But in this case, I think for once Chomsky might find a home. Chomsky decided that because he Chomsky could not understand how infants picked up language that there must be a magical device at play, a language organ in the brain dedicated to language acquisiton just as the spleen is dedicated to damn I wish I had not slept through biology.

Chomsky had a tiny little problem with his nonsense in that when one looks at human languages -- any of which can be picked up by any infant raised in its midst, whatever that infant's genetic tree -- well, the structural variation is infinite. I remember one universality they found: adjectives tend to appear very close to the nouns modified. Well, shazaam!

So the Chomsker has a problem: this super-duper language organ has to be hardwired not to learn any particular syntax, it has to be hardwired to learn every syntax. Ok, that makes no sense, shall we try again?........... um............. er....................... I got it! It is hard wired to learn an abstract syntax! If only we could find one!!! Chomsky failed except for that bit about modifiers miraculously always appearing close in sentences to the thing being modified.  Yeah, how come no language ever came up with "The little giant slowly balloon over drifted the mansion red white."? Because the language organ rejected all those! Where was I?

Ah, flexible language acquisition, computer or otherwise. No, it is not the brain, it is the shoes. Basketball no, programming yes. No, Michael did not reach new heights because his sneakers opened new doors for him, but yes: expose someone to Lisp and (as many before me have observed) they become a better programmer.

Chomsky saw the infinite variability of actual syntaxes as a severe objection to his hypothesis, and he was right to do so. Indeed, look at it from the other direction and we see the light: forget infants' ease of language acquisition, look at the difference between those who know one language and those who know five. Something happens after one has learned a bunch of languages -- acquisition of the next becomes a snap. No, I do not have that backwards. There is a small effect of "hey, anyone who has learned five just digs languages and does because they are easy for them", but I learn natural languages insanely easy and get a huge kick out of stumbling thru a conversation in French or Mandarin and learning a new language is still hard for me because circumstances never led me to learn more than what I needed to get out of high school. 

No, what is going on is the same as any learning process: the more we do the more the brain wires up metaskill at that activity. Babies are not born with a language organ, they just happen to have nice blank slates and speech is to them obviously something utterly compelling to master, it seems to control these big creatures moving about before them. Nothing like insane motivation for apt learning, eh Dr. Chomsky? But later on in life when a second language is an amusement? Acquisition is brutal. Where's the language organ now? Withered? Why does it come back after I have learned four the hard way? Ooops.

Was I supposed to be answering Jeff's question? My bad. Chomsky was wrong, McCarthy was right: a language in which code can be data and verse visa is one in which the programmer is stripped of structure and stricture and as Sartre warned us is not free to be not free. In mastering any art true freedom comes only within the constraints of the form, but Lisp offers us no such handholding. Some brilliant wag once observed that Lisp gives us a million ways to do anything but (1- million) of them are wrong -- a working Lisper like one who has learned the hard way five natural languages learns a metalanguage of programming that would delight and disappoint Chomsky, because the learned metalanguage is just one more demonstration of the brain's ability to find patterns in whatever it is immersed in long enough. ie, Organ, schmorgan, neural nets simply rock.

Lispers are not smarter, they just fell into a pool and learned to swim -- ah, good analogy, nothing to hold them up, had to make do in a medium that would not support them, they had to add energy to the system to stay afloat, something like that. Lisp not only supports every paradigm, worse, it even allows any paradigm you can invent one as I thought I did with Cells until I looked up the prior art, but that does not change the fact that while working on a little GUI geometry problem a new paradigm jumped into my code... deal with this kind of language and... well, imagine sneakers that really did have wings. Wear those for a year and then climb back into ordinary sneakers...I'll give you a minute to think. What happens?

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Disappointment? Some. Depression? No. End of flying? Never. You have tasted flight, you know what is possible, and you want to be there again. Now the analogy breaks down, because guess what? Lisp is just another language. When you do not have it, you can Greenspun it. When subjected to a toad of a language you grouse for fun but you also have code to write and you shrug off the downgrade and fly with the sneakers you have. 

In my first programming job I astounded the team so badly they kinda fired me, gave me only the maximum raise and blew me off when I asked for more because in nine months I had finished every system they had in development. At my going away they said they did not go out on a limb for me to exceed the maximum because they knew I would be leaving anyway. They did not know how much I loved the company bowling league or softball beer league. What was my secret in this Cobol shop? I had been programming in Basic on an Apple II. 

One day I came upon a senior guy (OK,they were all senior, I had been there three months at the time) staring at my code as he tried cloning it to his assigned, well, clone. Concerned, I paused and asked if there was a problem. He waved me away, saying, No, I just have never seen Cobol like this. I understood. I knew even as I did it that I had been programming Cobol in Basic. I think I have that backwards. I remember asking them -- I was just learning Cobol at the time -- if a subscripted variable could be a subscript and I remember a very nice twenty-year Cobol veteran asking me very gently if I might not be making things a little too complicated. Nah, just trying to do Basic in these Cobol sneakers.

Lisp then is a substitute in itself for learning five natural languages, because it itself is not Chomsky's language organ, it is the universal syntax Chomsky had to identify for his imagined language organ to have been hardwired to learn. Lisp is the programming language embodiment of Sartre's curse: we are not free to be not free. Program this language and one will first use the wrong language feature at the wrong time (hang out on comp.lang.lisp to witness the hijinx of Lisp noobs trying to do things "The Lisp Way") and eventually get to the point where one can look at problems and see the paradigms they bring with them aided always by your language which is being utterly useless at telling you what paradigm to use, it could care less. 

ie, It's the shoes.