Monday, April 30, 2007

A Python Plugin System

A couple of months ago André mentioned that having a plugin system in Crunchy would be "nice thing". I was feeling bored when I read his email, and so decided to go away and write one.

This post documents my attempt, which I like to think was actually rather successful.

Basically, Crunchy plugins are .py files that reside in a specific directory. They are automatically imported and initialised by the Crunchy core system on startup.

So the first thing we need to do is figure out what that special path is and enumerate all the .py files in it:

import os
import os.path
import imp

pluginpath = os.path.join(os.path.dirname(imp.find_module("pluginloader")[1]), "plugins/")
pluginfiles = [fname[:-3] for fname in os.listdir(pluginpath) if fname.endswith(".py")]

I'm afraid both lines are rather complicated, but they certainly demonstrate the power of Python. Because this code is run from within the module pluginloader we look up its (absolute) path and join that to the (relative) subpath plugins/. This gives us the absolute path to the plugin directory in a system-independent way.

The second line is (as you probably realised) a list comprehension, it first generates a list of the files in the plugin directory, then adds filters that list for files ending in .py, and finally removes the trailing .pys from the filenames to give a list of module names. All in one line!

Now that we have a list of modules, all we have to do is import them, which we can do with another list comprehension:

import sys

if not pluginpath in sys.path:
imported_modules = [__import__(fname) for fname in pluginfiles]

Once again, the Python code is disarmingly simple: all it does is add the plugin directory to the module import path and import all the modules, generating a list of the imported modulle objects along the way.

In the Crunchy Plugin API we ask that all initialisation code be placed in a custom register() function inside each plugin module. We can now call these functions with just one more line:

[mod.register() for mod in imported_modules]

And there you have it, a simple but powerful plugin system written in just 10 lines of Python code. Enjoy!

Edit: setuptools does the same job, with many more features and more flexibility.

...and back again...

Hi, again.

I've been lazy, and let this lapse (I know, no posts since July last year!). Now I want to resurrect my blog - in order to post some interesting things that I've been working on.

Firstly some news about Crunchy: Since I last posted it has been renamed from Crunchy Frog to just plain Crunchy. This was because of another project called CrunchyFrog. The AJAX based IO system that I mentioned last July has finally come together in the last couple of months as part of a complete rewrite of Crunchy - with a plugin-based architecture (more on that soon), a very neat HTTP Server (with COMET capabilities) &c &c.

And finally some personal news: I successfully completed the Summer of Code last year as a student, and am acting as a mentor this year (for Bryan Psimas - a very promising student from the states). I'm also working in the City of London this year: as a techie at an investment bank.