“…you can count the number of games written in a purely functional style on one hand. Is it that language tinkerers are less concerned about writing real applications? That they know you can solve any problem with focused grunt work, but it’s not interesting to them? That the spark and newness of a different language is its own reward? Either way, the BASIC programmers win when it comes down to getting projects finished.”

—James Hague, Slumming with BASIC Programmers

The gist of the post is that uber-languages like Lisp, Erlang, Ruby, and even C are worlds ahead of BASIC, but somehow a group of Neanderthals put down their clubs and wiped away their drool long enough to write a slew of various and impressive computer games using BASIC. OK, he didn’t say that, his exact words were “…largely written by people with minimal programming background.” I was just translating for the language snobs out there. But then, mid-gloat, he hits them, you (even me) with that damning quote above about getting projects finished.

Ouch. I was looking back at what I’ve produced over the last few years (both personally and professionally) and the list is pretty short. Maybe it’s a byproduct of too much school, or not being able to take things apart as a kid, but I tend to default to waterfall mode. Research, plan, gather as much knowledge as possible to build a mental model so complete that the perfect solution will fly from my fingertips like a glorious winged Viking ship sailing off to enjoy another toasty Greenland winter. Alas, I usually just end up learning enough useful knowledge about a subject to write and converse about it but not enough to work with it. Hence my dusty portfolio.

Take my involvement with Lisp, for instance. I’ve become a slightly respected name in the (admittedly small) Lisp community because of what I’ve written about history, Emacs and SLIME, how to learn Lisp, how to setup an environment, access documentation, and the right mindset for developing in Lisp. Heck, I even organized a workshop to help other people learn Lisp! Notice all the working software on that list? [Cricket, cricket, cricket...]. What does all that have in common? It’s all preparation for the day when I’m going to write all this great software in Lisp. Oops.

So why, when I’ve been using C# and .Net at work for 5 years now, am I so worried about having the perfect Lisp environment and knowledge of how to use it? So that when I finally get around to writing something, I will be able to do it quickly because of all the power at my disposal. That’s right, I don’t have a lot of time to spend, so I’m very, very, very gradually building up my ability to write software quickly. That certainly makes more sense than writing a little software at a time, right?

Where was I? Oh yeah, OOPSLA.

OOPSLA

The ACM SIGPLAN Conference on Object Oriented Programming, Systems, Languages, and Applications (OOPSLA) is one of the premier conferences on software development, and despite the title, doesn’t have much to do with object-oriented programming anymore. Twenty years or so when it started, OO was hot, cutting edge stuff, and as time marched on, the conference people decided that rather than stick with objects, they would stick with the hot, cutting edge. For the highly regarded 2007 conference, they recorded their keynote speeches and made a podcast out of them. What outstanding marketing! (The Singularity Summit did this too.) I was interested in OOPSLA because of what Dan W. and others had written about it, but getting to hear the actual speeches put this year’s conference in Nashville at the top of my wishlist.

[Note: if anyone is looking for a way to feel really good about themselves, why not sponsor my trip to OOPSLA 2008? No, really, please! I promise to take pretty pictures and write lots about it.]

Two of those keynotes made a huge impression on me, to the point where I’m scared what will happen if I listen to the 5 remaining talks. First, in Second Life: The World’s Biggest Programming Environment (download 40.5 MB mp3), two Second Life employees gave a similar argument to Hague’s above. They started out by trashing the Second Life scripting language (LSL) and sharing details of its sordid past. For instance, all scripts have to run in a 16K virtual machine. This virtual machine was written in one week, replacing the first version that was written in one night. There is no IDE, no debugging, no tracing – all editing happens in-world. Most appealing language features are missing, and the language isn’t even pretty. Bleak, huh? Well, there are hundreds of thousands (millions?) of people writing scripts in LSL, with over 2 million simultaneous scripted objects and over 2.5 billion lines of code in this terrible language. Why do people put up with it?

Because the payoff is so rewarding – there is a whole world full of animated objects scripted by participants. People love to write in it since there’s immediate, tangible feedback for doing so – you literally change the world your character lives in!

The second talk was by Kathy Sierra, called (no surprise) Creating Passionate Users (download 38 MB mp3). I’m a confessed Kathy Sierra fan, so I looked forward to getting reacquainted with old Kathy. Her remarks enlightened and expanded the points made by the Second Life people – people want high bandwidth experiences, they want feedback, they want their effort to produce results! Why was that such a revelation to me? I mean, I got it, but I didn’t get it until I heard her say it. There are so many examples of this principle:

• people use PHP because it’s installed on every web host in the multiverse, so they don’t have to muck around much with deployment
• people use Ruby on Rails because they can make a blog in 15 minutes
• people use Agile because they always have working software and can see changes immediately
• people don’t use Lisp because there’s no easy download
• and people use BASIC and LSL because they want to get stuff done

That brings me to the two parts of my inflammatory title:

Language Snobs

Language Snobs stress the superiority of one programming language over others. Some languages are extremely guilty of this (Lisp, SmallTalk, Python, Ruby, Scala, Haskell, etc) while others aren’t (Java, C#, VB, BASIC, PHP, Perl, C, etc). The snob languages talk about why they’re so great, and the other languages just point to what has been written in them as proof that they’re great (or at least good enough). Granted, this definition isn’t perfect; Lisp has everything on my Lisp Companies page to brag about, but it seems small in comparison to the hot air that gets circulated (including by me) in its favor. I think Ruby is crossing over into the non-snob category, based on the massive popularity of Rails and all of the sites written in it.

The important point is that the language snobs aren’t wrong! There is no denying that abstractions are useful, that more powerful tools help, and that a sharp, well-trained mind can accomplish things that a simple one can’t. The dearth of machine language and assembly written nowadays is proof of that. But the language snobs are working in the realm of what’s theoretically possible, what can be accomplished in the best possible conditions. This could be in small startups founded on the best principles, or on large companies like Google and ITA that have kept that environment as they’ve grown. But most people and most business don’t operate anywhere near that theoretical maximum, so the bulk of the software gets written by the competent mean, by the bulge in the middle of the standard distribution. Language snobs are also working in the future, since advanced topics tend to trickle down over time (hence the rebranding of OOPSLA).

Penny Pinchers

A penny pincher is someone who is frugal, who “knows the cost of everything and the value of nothing”. Language snobs know what languages are most powerful, so they are loath to use anything less than the best. They don’t want to endure the cost of code duplication, cut-and-paste, reading un-refactored code, leaky abstractions, etc. Everything has to be perfect, and when you add up those little bits of perfection, you get something great.

But life, business, and software are full of trade-offs. If you can accept a little imperfection here, some cruft there, and sloppy duplication there, they you can hire from a bigger pool of employees, deploy to a wider range of platforms, hit more markets more frequently. The key point that the penny pinchers miss is that while using inferior tools has a cost associated with it, so does not using those tools. The important thing is to Always Be Shipping.

Wasabi

It’s not only language snobs that are guilty of penny pinching. A classic example of the non-snob language fans pinching was in reaction to Fog Creek’s Wasabi programming language. For anyone who missed it, here’s the first mention:

FogBugz is written in Wasabi, a very advanced, functional-programming dialect of Basic with closures and lambdas and Rails-like active records that can be compiled down to VBScript, JavaScript, PHP4 or PHP5. Wasabi is a private, in-house language written by one of our best developers that is optimized specifically for developing FogBugz; the Wasabi compiler itself is written in C#.

This simple comment set off a crazed irrational firestorm of criticism, cataloged here. People even thought it was a joke! These people were pinching pennies because they didn’t realize that writing a compiler where you control both the input and the output isn’t that hard if you have smart people working for you. Joel wrote a response to the Wasabi firestorm explaining all this:

Since we are not blub programmers, we like closures, active records, lambdas, embedded SQL a la LINQ, etc. etc. and so those are the kinds of features we put into Wasabi…

Most people don’t realize that writing a compiler like this is only about 2 months work for one talented person who read the Dragon book. Since the compiler only has one body of code to compile, it is much easier to write. It doesn’t have to be a general-purpose compiler. It doesn’t have a math library, for example…

And we have the ability to add any feature to the language that we want easily… this is the same power Paul Graham talks about in On Lisp, the power to invent new language features that suit your exact application domain. Lisp does this through a mechanism called macros…

He also explains the very sound business reason for it, balancing the cost of developing the Wasabi language and compiler (smaller than people think) against the support cost of setting client servers up with .Net, Mono, or PHP on Windows (larger than people think) and the lost opportunity cost of people that won’t even consider buying your product because they don’t support your language (MUCH larger than people think). 37signals at least acknowledges this foregone opportunity, but 37signals has WAY more business savvy than people give them credit for. And despite the bashing Joel gets for writing “a crappy bug-tracking program for stupid windows programmers” or something like that, he has managed to increase his revenue by 17x since September 2003. There’s a very important lesson to be learned from two very successful companies – make balanced strategic decisions, but make sure you understand the consequences of doing so.

Conclusion

It turns out that this wasn’t much about languages at all; languages was just the topic that springboarded the idea. The real message is about people, passion, results, business, and most importantly, balance. If you find yourself focusing too much on one single aspect of your product or your business, there had better be a good reason. If you love your language because you can mold it (Lisp), or because strict functionality and a strong typing system prevent errors (Haskell, OCaml), those features had better be extremely important to the problem you’re trying to solve. If you’re running a software business and all you do is code, you’d better be getting your first version ready to release as soon as possible. All successful companies are balanced companies. If you’re too focused on one area, you’re guilty of premature optimization and unless you are ready to make some painful adjustments, you should get comfy on your local maximum.

———————————-

UPDATE: Joel and Jeff at the stackoverflow podcast had a strikingly similar discussion in Episode #8.  Joel said that for the niche tools (that’s how he characterized what I called “snob languages”), there are two reactions to the pain that the limitations (documentation, fewer programmers, less mature tools, etc) cause: admitting that your platform just isn’t that good yet, or convincing yourself that your platform is so good that it’s worth the pain because it makes you so much more productive.  Choosing the first option causes jarring cognitive dissonance whereas the second choice feels good, so these niche languages and platforms have vocal, fanatic followers who have convinced themselves that they are using the Good Lord’s Own tools.  If you’re using a snob language but still think you’re rational and open-minded, listen to it or read the transcript (not finished yet but will be soon).  It’s between 6:57 and 16:56.

The nice thing is that swapping out Common Lisp implementations within an Emacs/SLIME setup is easy, so maybe 3/4 of this guide is identical to the CLISP guide. I’ll put a disclaimer at the beginning of each section saying whether there’s anything new or whether you can skip it.

0. Files to Download [DIFF: The only difference is the link to the SBCL installer instead of the CLisp installer]

If you want to get started on downloading the setup files, here are the links:

• 7-zip (download v4.57 – 840K) – for unpacking .tar and .tar.gz files commonly used in *nix.
• Emacs (download v22.2 – 37MB)
• SBCL (download v1.0.13 – 9.8 MB)
• SLIME (download CVS snapshot – 700K) – apparently the SLIME 2.0 release is ancient and not worth using

1. Setup Directories [NO DIFF: skip if you've completed the CLisp guide]

Windows XP loves extremely long pathnames like C:\Documents and Settings\Username\Application Data. This is okay if for GUI-driven apps, but in Emacs, you’re not clicking, you’re typing, and you want short path names. Also, much (most?) of the open source Common Lisp tools are developed on *nix, so there’s less impedance mismatch when using stuff off the net if you have a *nix-ish folder structure.

Start with a root folder. In the pathnames in the rest of this guide, I’ll refer to it as [HOME]. I used C:\home\ to match the *nix convention of having user files be in /home/username/. You also need a username directory – I called mine peter and I’ll refer to it as [USR]. So in my case, [HOME]/[USR] would be C:\home\peter\.

Emacs NOTE #1: One important point about filenames in Emacs is that they CAN be entered with ‘/’, but if you prefer to use ‘\’, you have to escape it with another back slash, so in Emacs buffers and files, you would enter “C:/home/” or “C:\\home\\” but never “C:\home\”. The trailing slash is also important because it indicates that it’s a directory. Now that you have chosen your [ROOT] and [USR] values, these are the folders you should setup:

• [HOME]\[USR]\
• [HOME]\[USR]\site\
• [HOME]\[USR]\docs\
• [HOME]\[USR]\lisp\
• [HOME]\[USR]\info\
• [HOME]\bin\

2. Setup Emacs [NO DIFF: skip if you've completed the CLisp guide]

Emacs does not need to be “installed” like most Windows apps – it just runs from its directory. Extract it to [HOME]\bin\. I changed the folder name from emacs-22.2 to just emacs, so my path was c:\home\bin\emacs\. You run emacs by running the file [HOME]\bin\emacs\bin\runemacs.exe.

Emacs settings are stored in a configuration file called “.emacs”. On Windows, the default working folder and .emacs location is C:\Documents and Settings\Username\Application Data, but that’s too painful to type and it’s a hidden folder so it’s not even easy to get through with the GUI. To have Emacs work in a different folder, create a file called site-start.el in the emacs\site-lisp folder. In that folder, put the following line (but change the path to match yours):

(setenv “HOME” “[HOME]/[USR]/”)

(Thanks Nathaniel! This was probably the biggest annoyance of mine that made me put off finishing setup for so long.)

Emacs NOTE #2: Emacs has lots (and lots and lots) of keyboard shortcuts, many more than there are letters in the alphabet. Many of them require multiple keys or a sequence of keys called a “chord”. The two modifier keys used are Control and Alt (also called Meta). The common way to write chords is to use C- and M- for Control and Meta, so to run the command to close Emacs, you hold down Control and X, then let go, then hold down Control and C. This is written as C-x C-c. Sometimes a chord is followed by another letter or the name of a command. For instance, to undo, you hold Control and x, then let go and hit u. This is written as C-x u. Dashes mean press at the same time, space means let go. Easy! I made a cheat sheet for common commands in Emacs and SLIME.

Now it’s time to create your .emacs file. Run emacs and hit C-x C-f. This is the find command, which is used to create or open a file. At the prompt “Find file: “, type “~/.emacs” (without quotes). Right now just put a ; (that’s the comment marker in Emacs Lisp files) and save it by hitting C-x C-s. There’s a world of stuff you can put in a .emacs file, but that’s too much to bite off now.

3. Install SBCL [DIFF: This section is completely different]

Although the Windows port of SBCL is still experimental, there is a Windows installer (.msi) file for v1.0.13 (highest version as of now is v1.0.17). Just run the installer, and install it to the directory:

[HOME]\bin\sbcl-1013\

Then just finish the install wizard. But you’re not finished yet.

Robert Zubek wrote a post about 2 months ago about how to setup SBCL/Emacs/SLIME on Vista, but it doesn’t work exactly right on Windows XP. If you use the .emacs settings from his post, you get this error when you start SLIME (instead of using the .emacs settings in step 4):

fatal error encountered in SBCL pid XXXX:

can’t find core file at /user/local/lib/sbcl//sbcl.core

Process inferior-lisp exited abnormally with code 1

If you use the settings in step 4, you’ll get a different error message:

Searching for program: no such file or directory, sbcl

If you SBCL looks in the environment variable PATH to find the executable and core file. The default directories are *nixy, so it won’t find your sbcl directory on accident. But why should it look for the directory if you entered it into the install wizard?

Apparently Windows XP doesn’t update system environment variables until the system reboots. So right after you install SBCL, you have the user environment variables PATH and SBCL_Home but SBCL looks in the system variables. I couldn’t find out a way to fix that without rebooting, but the good news is that a reboot solves the problem.

So reboot your computer now.

4. Setup SLIME [DIFF: This section has some differences - links and .emacs setup]

Apparently the SLIME 2.0 is very old; several people recommend ignoring it and going straight to the CVS snapshot, so that’s what I linked to above.

Extract the SLIME folder to [HOME]\bin\emacs\site-lisp. (Here’s a tip about putting emacs code in site-lisp).

Now in Emacs, open your .emacs file (C-x C-f, ~/.emacs) and enter the following (general SLIME instructions, SBCL specific setup), changing your pathnames as usual. The good news is that because SBCL looks in the environment variables, you don’t need any of the command line parameters in the inferior-lisp-program variable:

(setq inferior-lisp-program “sbcl”)
(add-to-list ‘load-path “[HOME]/bin/emacs/site-lisp/slime/”)
(require ’slime)
(slime-setup)

Save the .emacs file (C-x C-s), then run the eval-buffer command (M-x eval-buffer). This will run the code that you just entered and tell Emacs how to run SLIME.

Now, you start slime with M-x slime and you’re ready to go!

NOTE: This is a basic SBCL setup – I’ve found some more advanced setup links that I’ll write guides for later:

• Setup ASDF-Install/SBCL on Windows
• Running SBCL on Windows – this is again for Vista, with the newest (v1.0.16 and v1.0.17) versions of SBCL. It also includes configuring shell commands, etc.

Additional comments at comp.lang.lisp.

Info is an output format for the Texinfo typesetting syntax. Texinfo is used for writing documentation that can be output in multiple print and digital formats and is the official documentation format of the GNU project. Info files are strongly associated with Emacs – Emacs is the primary reader for Info files, but there is also a standalone reader. Manuals are organized into a tree of nodes, with a variety of ways to navigate between those nodes.

Info files can technically be written by hand, but it is much more common for them to be compiled from Texinfo. Many Linux/Unix distros come bundled with the TeX toolchain necessary to compile Texinfo, but Windows does not. The TeX/LaTex/Texinfo on Windows post is coming in the near future. For now, stick to precompiled .info files or go dive into TeX yourself.

Emacs ships with several Info manuals – they’re in the emacs/info folder. You can also create your own Info directory, add new manuals to the directory, and create your own manuals. This guide will walk you through the steps needed to get Emacs to find your info directory, add manuals in your directory to the main directory list, and use Info from within Emacs.

[NOTE: This guide assumes you're following the directory conventions I laid out in my previous post Installing CLISP, Emacs, and SLIME on Windows XP. It also gives you the baseline level of familiarity with Emacs to follow along here.]

1) Setup Emacs to recognize your info directory

You should already have an info directory. Mine is C:\home\peter\info, and from emacs the path is ~/info/. To make this the first directory Emacs looks in when compiling the directory, add this to your .emacs file (feel free to indent better than WordPress allows):

(require ‘info)
(setq Info-directory-list
(cons (expand-file-name “[HOME]/info”)
Info-default-directory-list))

2) Create a dir file in [HOME]/info/

When Emacs is looking in directories for info files, it only looks for a file called “dir” in each directory. This dir file is a special kind of Info file that points to the other Info files in the directory and lists and orders them. In order to make your own dir file, you first need a .info file to link to. I recommend the Structure and Interpretation of Computer Programs in Texinfo manual (download precompiled 400K sicp.info.gz file here). There is a whole syntax for defining nodes, but this is the bare minimum you need to get your manual into the directory.

First, start with the characters “^_”. That doesn’t actually work when you type it in directly, so you need to copy it from Emacs’ info/dir. I looked at that file in a hex editor and it had the value 1F (US – unit separator) – the ASCII values for ^_ are 5E 5F. I had to open the emacs/info/dir file and kill/yank it into my dir file. Kill/yank is Emacs’ version of cut/paste. Like all things Emacs, it’s similar, not exactly the same, and much more powerful. Here are the steps (free Emacs tutorial included):

1. Open the Emacs info dir file with: C-x C-f [HOME]/bin/emacs/info/dir
2. Select the ^_ characters
   1. Put the cursor at the beginning and set the beginning of the region to cut (C-SPC)
   2. Move down to the beginning of the next line and cut to the end of the region (C-W)
   3. Yank it back into the file with C-y
   4. Save the file with C-x C-s
3. Create your dir file with find file (C-x C-f) at ~/info/dir
4. Yank (paste) the text into the file with C-y

That should work, but there’s still more to add to the file. Next is the Tree info – at least your file and node names. This works for your personal dir file:

File: dir Node: Top This is the top of the INFO tree

You can put a header if you like (“My Info Manuals”), then on a new line, put:

* Menu:

After that, you can add links to your manuals. The format is:

* TITLE: (filename w/o .info). Summary

For the SICP manual, it would look like this:

* SICP: (sicp). Structure and Interpretation of Computer Programs

You can put a footer after that to separate the manuals in your directory from those in others. The final dir file ends up looking something like this:

^_
File: dir Node: Top This is the top of the INFO tree

Peter’s Info Directory
**********************

* Menu:

* SICP: (sicp). Structure and Interpretation of Computer Programs

—- User’s ‘dir’ ends here —-

[Now that you did all that, feel free to download my dir file and change the labels as you see fit. ]

3) Using Info

Info has a similar structure to HTML, where there can be links within and between nodes, but there’s also an explicit tree structure with its own structured navigation. Here’s a pretty complete set of commands, with shorter descriptions and better grouping than the Info help shows.

Basics
C-h i – start Info
q – quit info
? – info commands
SPC – returns from commands screen

Node Navigation
p, n – previous/next node in subtree
[, ] – previous/next node in tree traversal
l, r – back/forward in node nagivation history
u – Move up
space – pagedown or next node
del – pageup or previous node
b – go to beginning of node

Links
M-TAB/TAB – previous/next cross-reference or menu item
m – type name of menu item and go to that node

Jump
d – return to Info directory
t – return to file root
L – go to menu of visited nodes
T – go to table of contents of the current Info file

Search
s – regex search, move to node where next match is found
S – regex search, case sensitive
M-x Info-search-next – repeat previous regex search
i – search for and jump to index topic
, – repeat previous i search
g – move to node specified by name – to filename by searching for (FILENAME)NODENAME
1..9 – jump to 1st-9th item in node’s menu
w – put name of current node into kill ring
M-n – clone Info buffer to new window

——————————-

That’s all for Info – it only takes 10 minutes to go through the Info on Info tutorial and it’s easy to use. This is just one of the many things that takes a couple hours to muscle through the setup and online docs, then it makes things so much easier EVERY SINGLE TIME you use it afterwards. My goal is to take these 2-3 hour research tasks and turn them into a 10 minutes guide. Hopefully by lowering the barriers to competence, more people will seek out and invest in learning the great tools out there that Windows developers tend to miss out on.

For instance, the Common Lisp Cookbook has a very thorough page on Setting Up an IDE with Emacs on Windows or Max OSX, but as far as I can tell, it’s from 2004 or earlier because it doesn’t mention SBCL or SLIME, two of the most popular pieces of CL development software today. Or when you run into a specific problem, it’s usually not too hard to find an answer, but it might not match the setup you’re using. That’s why I had to quit using Lispbox, because no one had any clue how it was configured, especially not on Windows. n00bs didn’t know the answers and the wizards had a traditional setup.

Here are some of the variables that contribute to the combinatorial explosion of configuration, in order or pain caused:

• Operating system: Linux (many flavors), Mac, Windows, Cygwin
• CL implementation: most common are SBCL, CMUCL, Allegro, LispWorks, CLISP
• CL implementation version (I ran into a problem here)
• SLIME version (stable 2.0 release or CVS snapshot)
• Emacs version (I didn’t run across any problems with this)
• I almost forgot, now there’s Cusp/Eclipse if you don’t want to use Emacs/SLIME, but right now Cusp is only tested on SBCL.

Maybe someday someone will try out all of those combinations and report, but for today, I did exactly one: Windows XP, CLISP 2.44, SLIME CVS, and Emacs 22.2 Why CLISP? After reading lots of internet, I came to the conclusion that (aside from the commercial CLs), CLISP is the best implementation for Windows users. The knocks against it are that it’s slow because it’s byte-code compiled, it doesn’t have threads, and on some points, it’s ideologically different from CL and other implementations.

Well, the performance isn’t a big concern for me (plus as I went to press, 2.45 was released with experimental JIT compilation), the threads also aren’t a big deal right now, and I think the differences will help me be aware of the considerations for writing portable code. Also regarding the SBCL windows port, it’s still considered experimental (currently v1.013 for Windows) but people are saying it’s stable and everything works except the threads. There’s even an msi installer for the Windows version. One of my next projects will be to dive into the SBCL documentation and mailing lists to get a better feel for it.

As far as the philosophy behind this (and future) setup guides, I want them to be a) thorough and b) instructive. Instructive? If you just want to take my word for everything, that’s fine, but I want you to know where I got my info from, so I’ve tried to include links to references for each step. On to the setup guide!

0. Files to Download

If you want to get started on downloading the setup files, here are the links:

• 7-zip (download v4.57 – 840K) – for unpacking .tar and .tar.gz files commonly used in *nix.
• Emacs (download v22.2 – 37MB)
• CLISP (download v2.44 – 4MB)
• SLIME (download CVS snapshot – 700K) – CLISP recently removed a function used in the 2.0 version of SLIME, but the SLIME CVS snapshot has that fixed

1. Setup Directories

Windows XP loves extremely long pathnames like C:\Documents and Settings\Username\Application Data. This is okay if for GUI-driven apps, but in Emacs, you’re not clicking, you’re typing, and you want short path names. Also, much (most?) of the open source Common Lisp tools are developed on *nix, so there’s less impedance mismatch when using stuff off the net if you have a *nix-ish folder structure.

Start with a root folder. In the pathnames in the rest of this guide, I’ll refer to it as [HOME]. I used C:\home\ to match the *nix convention of having user files be in /home/username/. You also need a username directory – I called mine peter and I’ll refer to it as [USR]. So in my case, [HOME]/[USR] would be C:\home\peter\.

Emacs NOTE #1: One important point about filenames in Emacs is that they CAN be entered with ‘/’, but if you prefer to use ‘\’, you have to escape it with another back slash, so in Emacs buffers and files, you would enter “C:/home/” or “C:\\home\\” but never “C:\home\”. The trailing slash is also important because it indicates that it’s a directory. Now that you have chosen your [ROOT] and [USR] values, these are the folders you should setup:

• [HOME]\[USR]\
• [HOME]\[USR]\site\
• [HOME]\[USR]\docs\
• [HOME]\[USR]\lisp\
• [HOME]\[USR]\info\
• [HOME]\bin\

2. Setup Emacs

Emacs does not need to be “installed” like most Windows apps – it just runs from its directory. Extract it to [HOME]\bin\. I changed the folder name from emacs-22.2 to just emacs, so my path was c:\home\bin\emacs\. You run emacs by running the file [HOME]\bin\emacs\bin\runemacs.exe.

Emacs settings are stored in a configuration file called “.emacs”. On Windows, the default working folder and .emacs location is C:\Documents and Settings\Username\Application Data, but that’s too painful to type and it’s a hidden folder so it’s not even easy to get through with the GUI. To have Emacs work in a different folder, create a file called site-start.el in the emacs\site-lisp folder. In that folder, put the following line (but change the path to match yours):

(setenv "HOME" "[HOME]/[USR]/")

(Thanks Nathaniel! This was probably the biggest annoyance of mine that made me put off finishing setup for so long.)

Emacs NOTE #2: Emacs has lots (and lots and lots) of keyboard shortcuts, many more than there are letters in the alphabet. Many of them require multiple keys or a sequence of keys called a “chord”. The two modifier keys used are Control and Alt (also called Meta). The common way to write chords is to use C- and M- for Control and Meta, so to run the command to close Emacs, you hold down Control and X, then let go, then hold down Control and C. This is written as C-x C-c. Sometimes a chord is followed by another letter or the name of a command. For instance, to undo, you hold Control and x, then let go and hit u. This is written as C-x u. Dashes mean press at the same time, space means let go. Easy! I made a cheat sheet for common commands in Emacs and SLIME.

Now it’s time to create your .emacs file. Run emacs and hit C-x C-f. This is the find command, which is used to create or open a file. At the prompt “Find file: “, type “~/.emacs” (without quotes). Right now just put a ; (that’s the comment marker in Emacs Lisp files) and save it by hitting C-x C-s. There’s a world of stuff you can put in a .emacs file, but that’s too much to bite off now.

3. Install CLISP

This is easy if you download the install package (that’s the one I linked to). Run the installer and choose [HOME]\bin\clisp\ as the install directory. That’s it!

4. Setup SLIME

CLISP changed recently (around version 2.44 – not sure exactly) and broke compatibility with the SLIME 2.0 release. This was fixed in development and the CVS snapshot works. If you try to use the 2.0 release, you’ll get an error that looks like this:

INTERN(“FRAME-UP-1″): #<PACKAGE SYSTEM> is locked

So use the CVS snapshot (which I linked to) and you should be fine.

Extract the SLIME folder to [HOME]\bin\emacs\site-lisp. (Here’s a tip about putting emacs code in site-lisp).

Now in Emacs, open your .emacs file (C-x C-f, ~/.emacs) and enter the following (general SLIME instructions, CLISP specific parameters), changing your pathnames as usual:

(setq inferior-lisp-program “[HOME]/bin/clisp/full/lisp.exe
-B [HOME]/bin/clisp/full
-M [HOME]/bin/clisp/full/lispinit.mem
-ansi -q”)
(add-to-list ‘load-path “[HOME]/bin/emacs/site-lisp/slime/”)
(require ’slime)
(slime-setup)

Save the .emacs file (C-x C-s), then run the eval-buffer command (M-x eval-buffer). This will run the code that you just entered and tell Emacs how to run SLIME.

Now, you start slime with M-x slime and you’re ready to go!

————————————–

I hope this works for you – I ran through it on another computer and it worked the first time, so if you use exactly these parts you should be OK. Please let me know if you run into problems so I can incorporate the fix into my doc. Many thanks to everyone who I linked to, since they did the coding, testing, and writeups that I used to get here. More resources to come – tell me what you’d like to see. If it’s something I was planning on doing anyway, I’ll move it up my queue.

The next guide will probably be setting up your built-in documentation (Hyperspec, etc).

I’ve always enjoyed music – from playing the trumpet in elementary school, owning a $230 portable CD player in 1993, to being an iPod man today. Practically every spare dollar I earned between the ages of 13 and 17 was spent on CDs. I love and appreciate all kinds of music – classical, rap, rock, alternative, opera, techno, Hawai’ian (especially Hawai’ian), even country (Claire from 9th grade, if you’re reading this (which I’m pretty darn sure you’re not), it took over 15 years and I’m still not a fan, but I can say now that I appreciate country music). But now I don’t listen to music very often. In fact hardly at all. I have my iPod running most of the day, but I listen to podcasts, sweet delicious podcasts of all stripes that keep me informed and help me learn new things. What’s the difference between me now and when I was in high school (besides the notably reduced mental redardation and 70 extra pounds)?

Music and the Brain

My answer has come from my interest in neuroscience and artificial intelligence. Jeff Hawkins’ book On Intelligence (God bless him for writing it) stresses the temporal nature of the brain – how learning, memory, even sensory perceptions like vision and hearing only work on data that changes through time. Our brain works by processing related events connected by the sequence they occur in – kind of like musical notes! Music speaks our brain’s native language! No compiling, no byte code, etc – it operates directly on our abstract syntax tree (that’s my analogy, don’t hold it against Jeff). That is why it’s easier to remember song lyrics than the words of a speech – the lyrics are attached to a musical sequence that burns into your mind.

[Someday I'm going to give On Intelligence a proper writeup, but I don't have it in me right now. It is definitely the most exciting book I've read in the last couple years. Whoops, gotta stop talking about it - my pulse is starting to race.]

Dan Weinreb recently wrote a review of a book called “This is Your Brain on Music” that I haven’t read but am dying to based on his review. Please read his entire review (and the book too!) but check out these nuggets on how entwined music is in your brain:

-When you hear a musical tone, there are neurons in your brain that send electrical pulses at exactly the same frequency as the note

-most people can recreate the exact tempo of a song they know, to within a few percent, even if they’re not musically trained

-your brain can distinguish between musical consonance and dissonance before your neocortex (higher, human level thinking) gets involved

I find all of that ridiculously fantastic, and if you’re at all interested in thought and music enough to keep listening (er, reading) to me, then you should check out those two books.

Example time! Look at the following sentence:

Janet owned a baby sheep.

Just a normal sentence that’s super easy to forget. Now look at this slightly different wording, set to music:

The words form a sequence (same as in the first sentence), the musical notes form a sequence, and each note links to its word, so even if you couldn’t say the sentence exactly by itself, you can fill in missing words if you sing it. It’s a heavily connected graph.

Emotional Amplifier

That’s the mechanics of why music works on our brains, but most people listening to music aren’t trying to accurately remember sentences. What’s more important is that since music talks directly to our brains, it can cause our brains to affect our emotions, hormones, and all of the other chemical goodness that our brain does without us asking it to. So music can affect your mood, and usually, no matter what mood you’re in, you want more of it. When I’m discouraged or wistful, Fade to Black by Metallica captures my mood perfectly. When I’m exceptionally grateful and loving, Con Te Partiro by Andrea Bocelli amplifies it (I almost cried with joy and love for my family when I heard him sing a lullaby version of it to Elmo on Sesame Street). When I’m frustrated and stubborn and need to unlock the power of my brain to triumph over a knotty problem, Beethoven’s Egmont Overture is the key – rarely can I get through the whole 9 minutes without figuring out how to triumph over what was impeding me, or at least get in the frame of mind to solve the problem shortly thereafter. Regardless of what you’re feeling, if you find a song that matches that feeling, it’s like having your best friend there to pour your heart out to. It listens to your heart, understands you, encourages you, and celebrates with you. Because it is directly in your brain, it is also directly in your heart in a way that few people on earth can be.

That’s the difference between the high school me and the modern me. My teenage hormones gave my mood and feelings much more control over me than my rational, hard working life as an employee, father, and husband currently allows. I also feel that I have much more control over my life at this point so I don’t succumb to the powerlessness that is such a big part of being a teenager. I know my purpose now, so music isn’t as necessary as an outlet for unchecked emotions.

Music is a emotional amplifier and an emotional determinant. Since the human experience is largely an effort to either make ourselves feel a certain way or wallow in the fact that circumstance has made us feel a certain way, then it’s no wonder that music is so powerful that poor people will pay for it and honest people will steal it.

Startup: This is when your revenues can’t support the company and its employees at a standard they would expect. This could mean a lot of things, for instance

• You’re self funding but doing the ramen/living on a friend’s couch/wifi-freeloading super cheapo route. Maybe you’re making a profit but you don’t want to live like this forever.
• You’re living off of investment money unrelated to what your startup is producing: angel/VC money, cash from a prior successful exit, student loans, day job money, etc.

The key thing is that your idea and execution are currently unproven. You should be getting noticeably better each day as you improve your technology, your product, your marketing, your brand, your sales cycle, etc, (all these things create wealth but not necessarily money), trying to find the right combination of those things that lead to market acceptance and self-sustaining revenues. Or your goal might not be self-sustaining revenues but to create a product or technology or business that will be bought by another, larger company.

Business: A business is funded by its customers and revenues, and it has found a successful niche in the market. Once you get to this point, some of the pressure is off, although your niche and success is a moving target and you need to continue working and executing in order to get knocked out of it. There are completely different skills and personalities needed to nurture and grow a successful business/product as opposed to developing a new one and finding a market for it.

One edge case is a successful, profitable business that has taken VC or other aggressive investment. The profits might be good enough for the business size, but the expectation by investors of a big exit is like a huge debt obligation that must be met. $10 million in annual revenue won’t cut it if you’re valued at $1B. In that case, you’re still a startup.

So by this definition, pretty much all YCombinator companies are startups (I don’t know of any that have real revenue – Octopart maybe). Facebook is a startup – a huge and wildly successful one, but until they find their killer revenue model, they’re living off of investment money. 37signals is not a startup – they were profitable from web design consulting while they developed Basecamp, Rails, etc, and now their products fund their business without outside investment. Fog Creek was a startup (living off of Joel’s MSFT cash) until FogBugz took off, and now revenue from that product runs their business. Google is a business, but until they perfected AdWords and Adsense (2002? 2003?), they were like Facebook is now, a really big successful startup. YouTube went from startup to acquisition and never was a business. You’re welcome (nay, encouraged!) to adopt these definitions and spread them, since I think it would clear up a lot of things.

A similar misconception comes from the titles that people who program computers are given (or give themselves): hacker, programmer, software engineer, solution developer, web programmer, code monkey, software architect, etc (I’ve had several of those at times). I’m going to look at two of them – hacker and (software) engineer – since they correspond pretty well to the definitions of startup and business above.

Hacker: Always trying new things, trying to stretch the limits of what’s possible or what exists. This could be from the perspective of science (Einstein, Tesla, Michio Kaku, etc), the environment (Amory Lovins, Paul Hawken), in addition to the more common usage in computers. Linus Torvalds is a great example of a hacker – he had two giant ideas that few thought were possible – writing a free operating system and doing it with a worldwide collection of volunteers (GNU/FSF fans, I’m not forgetting you, I’m just citing the most prominent example). Once something is functional and successful, the hacker moves on to another hard problem, like a serial entrepreneur.

Engineer: All of the definitions of engineering that I’ve found were too complicated and missed the point, so I propose this: An engineer is someone who takes something that is known to be possible, and makes it fit within a given, limiting set of criteria. Some common criteria are:

• within a fixed budget
• meet a performance benchmark
• exceed certain reliability requirements
• make it aesthetically pleasing
• must include a certain capacity
• meet arbitrary regulations (accessibility, environmental, paying living wages, etc)

Note that this doesn’t mean that an engineer is any less creative than a hacker. On the contrary, the constraints provided by the engineering problem can provoke more creative and original thinking than that of the original invention. For example, Ikea has furniture that looks like designer furniture that’s many times more expensive, but it has to 1) be inexpensive, 2) be made of parts that fit in a flat box, 3) be easily assembled by non-technical people, 4) use the common bin of Ikea parts, and 5) be durable enough that people will continue to buy Ikea furniture. So the hacker is the one figuring out how pretty furniture should look, and the engineer is the one doing trial and error with the flat box and the alan wrench. Who’s got more problems to solve? Hard to say. They’re different kinds of problems.

I think a great example of the hacker vs engineer mentality happened when Paul Graham released the first version of Arc with no Unicode support. There was a big brouhaha on the internet with people basically that Paul eats babies and respects Satan’s no-nonsense approach to discipline. Well, pg is also on record in saying that the initial users of a language shape its future and Arc is intended to be a language for hackers. While I believe his reason for not including Unicode (he didn’t want to spend time thinking about character sets), I think that he deliberately mentioned it in his release notes (and in a defiant way) so that engineers wouldn’t want to use Arc. Someone who can’t live without Unicode support is clearly not doing the explorative, rapid development that pg is trying to promote. So with one paragraph (and by ignoring a zillion trolling flame posts), he scared off the audience he didn’t want. He understands the difference between a hacker and an engineer and used it to his advantage.

So what’s better? As with Ikea, there’s no answer to that question. Hackers and startups/entrepreneurs get more favorable ink, and have more outstanding success stories, but engineers make good money, get challenged in their jobs, are generally respected, and their solutions get used by happy clients. Indeed, if you need an engineer, you have clients! Right now I’m a software engineer (really, it says it on my business card!) and it has been good to me. I get unusual technical challenges that  I wouldn’t have come across in my own projects, and unlike the “friend’s couch and ramen” startup life, I can feeds myself and my family and live comfortably. But that doesn’t mean the sparkle of hackerdom and entrepreneurship isn’t gleaming in my eye. That’s where my passion goes and I think the time I spend hacking at night on my own projects helps me engineer better solutions in my day job.

The important thing is to understand the difference and know when you need to hack, and when you need to engineer. Unlike the previous examples where it was all one or all the other, most development work requires you to assume both of these roles at different times. When debugging or scaling, you’ll probably need to put on your engineer hat, while if you’re designing a new feature, rewriting, or doing a greenfield project, you’ll hack more. There can’t be a divide between the two, whether on a team or different roles for an individual. A scaling problem might require big-picture experimentation to overcome a serious limitation. While hacking a new site, you might need to iterate over a small, important area of a page in order to hit your sweet spot in the market. Do both, and know when to do which.

What do you think of these definitions?

UPDATE: Apparently I didn’t explain well that I see this as a continuum. So lets say that pure, exploratory hacking is at x=0, and extremely demanding hard engineering (building bridges, medical devices, rocket boosters, etc) is at x=100, then software engineering might be in the 20s – still imprecise and flexible (by comparison), but constrained by cost, time, and flexibility. Also, just because some types of engineering have severe safety constraints, others may need long term reliability (car parts) or low cost (manufacturing a toothbrush). Those are engineering too, so maybe it makes sense to compare software engineering to business-related engineering.

UPDATE #2: Found a great new definition on HN:

• Technicians implement others’ designs using skills
• Engineers create new designs using knowledge
• Scientists create new knowledge

“Pure” programmers are technicians. Computer scientists are engineers or scientists.

Thanks wwalker3!

Follow the comments on Hacker News, reddit

So with 5 fewer hours a day to work, what did I cut out and what did I keep? Well, I haven’t a) written quality blog material, b) done housework chores, c) programmed as much, or d) made all the great improvements people suggested to my SLIME/Emacs cheat sheet. I decided that if I only had a little time, I should spend it on what’s most important, which right now is sharpening my Lisp skills. I’m trying to get to a point where I can work on programming for my startup without referring to documentation for every single thing I try to do. That consists of 3 things:

1. get better at Lisp (currently working through Practical Common Lisp)
2. learn to use ASDF (so I don’t have to only use one file)
3. customize Emacs and learn more commands

#1 is going nicely. Unfortunately, #2 and #3 pose a bit of a problem. I’m currently using the Allegro-bundled LispBox for Windows from the PCL website. It’s been great for getting started, getting comfortable with basic commands, and getting familiar with SLIME and Emacs. However, it’s not conducive at all to customization. I’m not saying that it can’t be customized, just that it is a minor distribution for new users, so there’s not a lot of documentation out there for working with it. Three big strikes: (Windows, LispBox, Allegro) means that there are very few users, very few comments, and very little help.

For instance, ASDF is included and configured correctly, with the code from PCL already setup and easy to load. However, I tried for (too long) to get ASDF to load my code. I wrote my own packages.lisp and .asd files, I tried pushing values for my directory to the asdf:*central-registry* so I didn’t have to make a zillion shortcut files, etc. Long story short, it didn’t work and nowhere on the Google did anyone have specific advice for my setup. Ditto with .emacs customization with Lispbox (for anyone who’s wondering, I think it’s at C:\Program Files\acl81-express\misc\dot-emacs.el, but I couldn’t get the settings to load. YMMV). It’s just too obscure and I don’t have the time to figure out everything on my own, so I’m moving to a more mainstream setup.

So I put development and practice mostly on hold so I could get a more common setup that will make it easier to get help with when (not if) I have issues. I’m still working on the setup, becuase I spent a lot of time researching recommendations and best practices. I want to get it right (or at least mostly right) the first time so I don’t have to go back and re-do everything again. Tweaking is fine and I plan to keep doing that forever, but no more redos. I’ll put out another n00b type giude (along with rationales) when I get my own setup working, but here’s what I’ve come up with:

• Emacs 22.1
• SLIME 2.0
• CLisp
• Cygwin(?) – I’m trying to limit the number of tools I have to deal with so I wanted to leave out Cygwin. Can anyone give advice on whether it will make my life easier or harder in the short term?

Thanks to everyone who has expressed interest in the meantime. Keep your eyes out for more guides and an update to the cheat sheet. That’s my motto, helping the n00b in all of us!

UPDATE: I wrote this on Thursday but haven’t been able to post until Friday night. I’ve taken two naps, planning on staying up at night, and I feel great! I’m already happy to be back on a polyphasic schedule!

In other news, I’ve got fans!

Well, at least one anyway. A lot of the feedback I’ve received on this site so far has been from experienced Lispers who appreciate my enthusiasm and contributions for new users. Yesterday, I got the first (I think) comment and reply from another new Lisper who’s actually getting some use out of my stuff. Aaron Feng wrote up my SLIME movie reference, and he even caught my later reference to my upcoming “Ultimate N00b SLIME/Emacs Cheat Sheet”. Well Aaron, wait no more!

I’ve been using it for a week or so, and I’m sort of pleased with it. First the positives:

• It includes all of the SLIME commands from the SLIME manual
• It includes all of the Emacs commands from the built in tutorial
• It includes the main Emacs help keys
• The commands are sorted and grouped by functional area
• It’s all fits on one page

Here are some of the things I’m not so pleased with:

• It has very few Emacs commands, relative to what’s available
• It doesn’t include any advanced text manipulation options
• It doesn’t have the sexp-based manipulation commands. I couldn’t find standard bindings for these and I’m not up to Emacs tweaking yet. Adding a key for the slime-selector was enough for me at this point.
• It’s pretty crowded (~100 commands and descriptions on one page)

The Emacs section is weak mainly because I haven’t dived into Emacs the way I have into Lisp/SLIME. I figure for now I can use it as a standard, powerful IDE for writing, testing, and debugging Lisp, and I can add Emacs wizardry later. I think I hit the point of diminishing returns where focusing on my tools will help me learn to code in Lisp. Now I need to get my coding up to par with my tools! I need to improve to the point where it’s worth getting a copy of Learning GNU Emacs.

So without further ado, here is my unfinished yet eminently usable cheat sheet:

Ultimate N00b SLIME/Emacs Cheat Sheet (pdf)

or customize your own!

Ultimate N00b SLIME/Emacs Cheat Sheet (xls)

Enjoy! As usual, feedback is appreciated. If you’ve got an Emacs trick that you use a jillion times a day that you think a new user could use, this is a great way to share!

(PS I hope it goes without saying but there isn’t any kind of beta process. Just that I consider it a work in progress and that it will definitely change in the future, hopefully in response to useful criticism.)

Follow the discussion at: Hacker News, programming.reddit.com, comp.lang.lisp, comp.emacs