Common Lisp on the spotlight

If you’re an OS X user you already now about Spotlight, the file metadata indexer that comes with Tiger. I’m not a heavy user of Spotlight, but today i’ve made a discovery that may change this a bit: John “Lemonodor” Wiseman’s Lisp Metadata Importer, a plug-in that makes Spotlight aware of Common Lisp constructs in your files.

John wrote a nice howto in the above page, complete with screenshots like this one showing a search for Lisp definitions:

Searching Lisp contents in Finder

and here you can see Pathfinder showing the Lisp metadata associated to one of my files:

Lisp and Spotlight

Nifty. Although i think that the possibility of using the command line interface for querying Spotlight metadata is even more interesting… i’m already thinking of writing some Elisp to perform queries within Emacs with Spotlight’s mdfind as a backend (albeit admittedly Slime makes those scripts less useful than one would think!).

One can also use John’s importer’s source code as the basis for new plug-ins supporting other languages (Scheme comes to (my) mind, stay tuned!). There’re already importers for Python and Ruby, but it’s quite easy to write your own if those are not your cup of tea (and learn a bit of Objective-C in the process).

Happy searching!

The object oriented cake

Aboutnext SplashWhile writing my latest post on NeXT, i remembered one of my favorite videos on programming ever: the 1992 demo presentation of NeXTSTEP 3 by, whom else, Steve Jobs. If you haven’t seen it yet, please take a look below, and keep repeating yourself that this was 1992, and that that machine had a clock under 50 MHz! One wonders what on Earth have we been doing during the last fourteen years.

I was about to write a brief overview of the amazing things demoed by Jobs, but the fine guys and gals of Rixstep have already done it for me: The object oriented cake is a guided tour for those of you lacking 35 minutes to see the movie. But i bet you’ll see it anyway after reading that article. More than once!

All these wonders where made using Objective-C. One would say that there’s something to OOP. Definitely, but there’s more than objects at play: Objective-C is a dynamical language, with a living runtime and powerful reflection capabilities. Despite its C-syntax (almost any C snippet is also an Objective-C snippet), its spirit is far closer to Smalltalk, and the right way to use Objective-C is to use it as a dynamically typed language. I’m convinced that these language traits were key to make the marvellous NeXSTEP system a thriving reality.

For the really curious among you, here‘s a bit more about the platform; and one can also take a look at the NeXTstep 3.3 developer documentation.

The Beginning of NeXT

I’ve got a soft spot for the software engineers at NeXT Computer and their OpenSTEP frameworks. Via, i’ve discovered this couple of short videos on how it all started, featuring a pretty young looking Steve Jobs.

Those were times!

Full of stars

Last january, Gna! Hotspot ran a very interesting interview with Étoilé’s main developers (David Chisnall, Jesse Ross, Nicolas Roard, Quentin Mathé and Yen-Ju Chen). These guys are not happy with current desktop environments, and have decided to build their own on top of GNUstep, which is in turn inspired by NeXT/OpenStep, the origin of Mac OS X. In fact, their peeves go far beyond the UI layer, and cover the whole Unix architecture (see, for instance, David’s Ten things I hate about Unix).

Instead of just complaining and telling everybody how his or her environment sucks, they have rolled up their sleeves and produced actual code: the Étoilé project is the umbrella for a series of libraries and frameworks aimed at fulfilling their vision.

The nice thing about Cocoa/GNUstep is that they are probably the only production-ready libraries written in a dynamic language, namely, Objective-C. Although it is not as good as Smalltalk (on which it is inspired) or any Lisp, Objective-C is a far, far better option than the usual C or C++ used by other mainstream toolkits. Of course, one could try to improve instead current support for really nice languages (see my Objective-C related posts), but an interesting way of doing that could be building on top of these projects. As mentioned in the interview, the Étoilé developers are all for dynamic languages, and, if you look carefully at their Making Computers Suck Less presentation, you’ll notice that they’re already planning support for Smalltalk or Ruby. Their proposed architecture is also interesting in other ways reminiscent of dynamic environments like Smalltalk’s or prototype based systems, including the everything (as in up to the user level) is a manipulable object idea, or orthogonal persistence.

The later release includes, among others, the following goodies:

  • Camaelon is a theme engine that goes a long way into disproving the GNUstep default ugliness.
  • LuceneKit is a class-to-class port of Lucene (a high-performance, full-featured text search engine library in Java) in GNUstep.
  • OgreKit is a port of the Cocoa regular expression engine of the same name.
  • SQLiteClient is a SQLClient/SQLite port from SQLClient library in GNUstep/devl-libs. It uses sqlite3 library and is supposed to work only on Mac OS X for portability of Etoile.
  • UnitKit is a unit test framework for Objective-C, again based on a Cocoa project.

All in all, a pretty interesting project. And, if you like reading about your fellow programmers toilings, don’t miss the Étoilé blog, where their developers give details about their daily efforts.

Tags: , , ,

Driving iTunes with Scheme, with a fold tutorial and a Haskell book

In this pretty detailed tutorial, Jim Ursetto puts his Cocoa/Chicken bridge to work. Jim noticed that some of his music files were forgotten by iTunes, and wanted to reimport them without losing any metadata. His article

documents my attempt to do just that by building an application in Chicken Scheme, using the bindings to Objective-C and Cocoa provided by the objc egg. It’s targeted at the intermediate Scheme programmer, who may have some experience with Cocoa. It may also be useful to a beginner looking for examples of an interactive development process, and to a non-Scheme user for the same reason.

Jim has taken the time to document also the program’s design and coding processes, so you’ll not only learn about Cocoa or how to use the Scheme bridge, but also about property lists parsing, symlink creation or exception handling. In addtion, you will have the chance of looking over a fellow programmer shoulder having fun for a while.

Programming in HaskellIt’s a long read, but a very good one. And just in case you don’t get to the end, don’t miss Jim’s last recommended reading: a tutorial on the expressiveness and universality of fold, by Graham Hutton. Graham, by the way, happens to be a Haskell and functional programming erudite and professor, so you may be interested in his publications, including the forthcoming book Programming in Haskell, whose first five chapters are available in PDF here.

Tags: , , , , ,

Programmers love writing (and mocking) tests

Aggressive unit testing (also known, by those willing to earn some easy money, by the buzzword test-driven development) is about the only practice from XP that i embraced happily in my days as a dot com employee (it was around 2000, and i was working for one of those companies that got lots of funding from avid but hopelessly candid investors to do… well, to do basically nothing).

Those were my long gone Java days, and i got test infected all the way down. That writing tests is a good practice is hardly news for any decent programmer, but what i specially liked about putting the stress on writing lots of tests was discovering how conductive the practice is for continuous code rewriting (count that as the second, and last, extreme practice i like). I had yet to discover the real pleasures of bottom-up development in REPL-enabled languages, but even in Java my modus operandi consisted basically in starting with concrete code (sometimes even a mere implementation detail) and make the application grow up from there. Of course, some brainstorming and off-the-envelop diagramming was involved, but the real design, in my experience, only appears after fighting for a while with real code. The first shot is never the right one, nor the second, for that matter. The correct design surfaces gradually, and i know i’ve got it right when unexpected extensions to the initially planned functionality just fit in smoothly, as if they had been foresighted (as an aside, i feel the same about good maths: everything finds its place in a natural way). When you work like that, you’re always rewriting code, and having unit tests at hand provides a reassuring feeling of not being throwing the baby with the bath during your rewriting frenzies.

Of course, there’s also a buzzword-compliant name for such rewritings (refactoring), and you can expend a few bucks to read some trivialities about all that. (Mind you, the two books i just despised have been widely acclaimed, even by people whose opinion i respect, so maybe i’m being unfair here—in my defense, i must say i’ve read (and paid) both of them, so, at least, my opinion has cost me money.)

Sunit in SqueakAnyway, books or not, the idea behind the JUnit movement is quite simple: write tests for every bit of working code you have, or, if you’re to stand by the TDD tenets (which i tend not to do), for every bit of code you plan to write. As is often the case, the Java guys where not inventing something really new: their libraries are a rip-off of the framework proposed by Kent Beck for Smalltalk. Under the SUnit moniker, you’ll find it in every single Smalltalk implementation these days. A key ingredient to these frameworks’ success is, from my point of view, their simplicity: you have a base test class from which to inherit basic functionality, and extend it to provide testing methods. Languages with a minimum of reflection power will discover and invoke those methods for you. Add some form of test runner, and childish talk about an always green bar, and you’ve got it. The screenshot on the left shows the new SUnit Test Runner in one of my Squeak 3.9 images, but you’ll get a better glimpse of how writing unit tests in Squeak feels like by seeing this, or this, or even this video from Stéphane Ducasse‘s collection.

Of course, you don’t need to use an object-oriented language to have a unit testing framework. Functional languages like Lisp provide even simpler alternatives: you get rid of base classes, exchanging them by a set of testing procedures. The key feature is not a graphical test runner (which, as any graphical tool, gets in the way of unattended execution: think of running your tests suites as part of your daily build), but a simple, as in as minimal as possible, library providing the excuse to write your tests. Test frameworks are not rocket science, and you can buy one as cheap as it gets: for instance, in C, i’m fond of MinUnit, a mere three-liner:

/* file: minunit.h */
#define mu_assert(message, test) do { if (!(test)) return message;  \
                                    } while (0)

#define mu_run_test(test) do { char *message = test(); tests_run++; \
                               if (message) return message; } while (0)

extern int tests_run;

(Let me mention in passing, for all of you non-minimalistic C aficionados, the latest and greatest (?) in C unit testing: libtap) Add to this a couple of Emacs skeletons and an appropriate script and you’re well in your way towards automated unit testing. From here, you can get fancier and add support for test suites, reporting in a variety of formats, and so on; but, in my experience, these facilities are, at best, trivial to implement and, at worst, of dubious utility. It’s the quality and exhaustiveness of the tests you write what matters.

Lisp languages have many frameworks available. The nice guys of the CL Gardeners project have compiled a commented list of unit testing libraries for Common Lisp. In Scheme you get (of course) as many testing frameworks as implementations. Peter Keller has written an R5RS compliant library that you can steal from Chicken. Noel Welsh’s SchemeUnit comes embedded into PLT, and the Emacs templates are already written for you (or, if you mileage varies and you’re fond of DrScheme, you can have a stylized version of the green bar too). Personally, i don’t use PLT, and find Peter’s implementation a bit cumbersome (meaning: too many features that i don’t use and clutter the interface). Thus, my recommendation goes to Neil van Dyke of quack‘s fame’s Testeez. Testeez is an R5RS (i.e., portable), lightweight framework that is as simple as possible. Actually, it’s simpler than possible, at least in my book. In my opinion, when a test succeeds it should write nothing to the standard (or error) output. Just like the old good unix tools do. I only want verbosity when things go awry; otherwise, i have better things to read (this kind of behaviour also makes writing automation and reporting scripts easier). So, as a matter of fact, I use a hacked version of Testeez which has customizable verbosity levels. It’s the same version that we use in Spells, and you can get it here. Also of interest are Per Bothner’s SRFI-64, A Scheme API for test suites and Sebastian Egner’s SRFI-78, Lightweight testing (both including reference implementations).

Lisp testing frameworks abound for a reason: they’re extremely useful, yet easy to implement. As a consequence, they’re good candidates as non-trivial learning projects. A nice example can be found in Peter Seibel’s Practical Common Lisp (your next book if you’re interested in Common Lisp), which introduces macro programming by implementing a decent testing library. In the Scheme camp, Noel discusses the ups and downs of DSL creation in an article describing, among other things, the SchemeUnit implementation. A worth reading, even for non-beginners.

Once you settle on a test framework and start writing unit tests, it’s only a question of (short) time before you’re faced with an interesting problem, namely, to really write unit tests. That is, you’re interested in testing your functions or classes in isolation, without relying on the correctness of other modules you’ve written. But of course, your code under test will use other modules, and you’ll have to write stubs: fake implementations of those external procedures that return pre-cooked results. In Lisp languages, which allow easy procedure re-definition, it’s usually easy to be done with that. People get fancier, though, specially in object-oriented, dynamic languages, by using mock objects. The subject has spawn its own literature and, although i tend to think they’re unduly complicating a simple problem, reading a bit about mockology may serve you to discover the kind of things that can be done when one has a reflective run-time available. Smalltalk is, again, a case in point, as Sean Mallory shows in his stunningly simple implementation of Mock Objects. Tim Mackinnon gets fancier with his SMock library, and has coauthored a very interesting article entitled Mock Roles, Not Objects, where mock objects are defined and refined:

a technique for identifying types in a system based on the roles that objects play. In [9] we introduced the concept of Mock Objects as a technique to support Test-Driven Development. We stated that it encouraged better structured tests and, more importantly, improved domain code by preserving encapsulation, reducing dependencies and clarifying the interactions between classes. […] we have refined and adjusted the technique based on our experience since then. In particular, we now understand that the most important benefit of Mock Objects is what we originally called interface discovery […]

An accompanying flash demo shows SMock in action inside Dolphin Smalltalk. The demo is very well done and i really recommend taking a look at it, not only to learn to use Mock Objects, but also as a new example of the kind of magic tricks allowed by Smalltalk environments. Albeit not as fanciful, Objective-C offers good reflective features, which are nicely exploited in OCMock, a library which, besides taking advantage of Objective-C’s dynamic nature, makes use of the trampoline pattern (close to the heart of every compiler implementer) “so that you can define expectations and stubs using the same syntax that you use to call methods”. Again, a good chance to learn new, powerful dynamic programming techniques.

As you can see, writing tests can be, a bit unexpectedly, actually fun.

Tags: , , , , , ,

Dynamic OS X Programming

I am very fond of Mac OS X’s graphical interface. As a GNU/Linux or FreeBSD user, i despise GUIs and usually work in a mostly text-based ambient powered by Emacs running inside a urxvt terminal. Things change dramatically when i enter OS X. There i discovered that it is not that i dislike GUIs, but that i dislike ugly GUIs (although the jury is still out on where i’m more productive).

Thus, OS X is the only environment where i feel like programming a GUI. From a programmer’s viewpoint things also start out better: one can go native and avoid C and C++. Cocoa’s native language is Objective-C, which happens to be a dynamic language. That is, one has a runtime where possibly dynamic objects live, good (but not great) reflective capabilities and a meta layer that let’s you, for instance, access class and metaclass objects. And all that at runtime. That allows (besides the fun of programming) funny tricks, like the ability to interact with running applications from the outside, in what constitutes an incredibly flexible plug-in system.

Probably the description above rang a bell: Smalltalk. Objective-C is a scaled down Smalltalk in many ways. The good news is that you can have the real thing: F-script is a Smalltalk dialect specifically designed to script Cocoa applications. It let’s you inspect any running Cocoa application, and manipulate the objects it contains in real time, so to speak. F-Script Object browser As any decent Smalltalk, it comes with a powerful object browser, which you can admire in all its beauty below. Moreover, F-script can be embedded in your Cocoa application, acting as an excellent extension language, way more cool than Apple’s default scripting language. A beautiful application taking full advantage of these abilities is ObjectiveCLIPS, a powerful rule-based development environment for Cocoa.

But still, to my knowledge, you cannot program a whole application in F-Script and, besides, Smalltalk is all very well, but it’s not Lisp. Support for Lisp in the Cocoa world is not as good, but shows promise. First, there’s OpenMCL, an excellent Common Lisp implementation for the Mac and Unix systems that comes with a Cocoa bridge. It includes a demo Cocoa IDE to get you started, and people have gone as far as creating games using it.

Until recently, poor Schemers were left in the cold. Enter Chicken and its Objective-C egg. Support is still rudimentary, but there’s already a nice tutorial on writing a currency converter, the default Cocoa Hello World app.

Probably, it’s still too hard to write a well-integrated Cocoa application using Lisp, but i think it’s just a matter of time. Happy hacking!

Tags: , , , ,