Deck the Halls with Raspberry Pi

I'm continually on the lookout for ways to connect with my children, especially my eldest whose natural interests seem to be rather different from mine. Going into parenthood I expected to engage my children easily over interesting science and technology subjects. It's certainly in their blood: their mother is a scientist and I am a software developer and general tinkerer. Reality has, as it so often does, proven less straightforward, more ambiguous. I'm sure there are many causes and influences, but one obvious theme concerns the sheer scope and pace of technological advancement since my own childhood. My first computer — a Commodore VIC-20 — seems hopelessly primitive next to the bewildering assortment of hypernetworked computing devices my children see their grandparents carry in a pocket.

While I struggle to identify the essential difference in our experiences, I suspect that part of what has changed is that my children aren't interested in computers as such the way I was. They beg to get tablet time to watch videos or play an online game, but everything they experience is professionally produced and logically ordered, with all the rough edges and sharp corners neatly rounded off into beautiful, pleasing curves.

My formative experiences were quite different. To them, using a VIC-20 would seem like a pathetically empty experience by contrast with the devices they're used to. No slick fade ins to carefully crafted notifications blending seamlessly with expensive, highly tuned interfaces to precisely-defined destinations. Even turning the VIC on felt like a primordial act, the massive, honest-to-goodness SPST coursing with power that flowed past your finger and off toward unknown and terrifying adventures. Yet even after that nothing much happened, nothing but a blinking blue expectancy, a half imagined whisper of potential. But of course that was the magic of it: you didn't turn on a VIC so much for what it could already do; you turned it on because of what you could make it do, for all of the wonderful creative possibilities its existence suggested. While the intervening decades have brought transformative advancements in the capabilities of our information technologies, the exploratory capacity, this frontier-like horizon that was the VIC's foremost asset, can become lost in the well-worn ruts of design.

This helps explain why, on a whim shortly before Christmas, I packed my boys into the truck and headed over to Micro Center to pick up a Raspberry Pi 2. The actual purchase jarred me a little by its contrast with my childhood memories, the agonized handwringing and haggling over the purchase of some part or accessory, which cost much more and did much less than the tiny little box the three of us carried out of the store. But there's something compelling about these little devices that manages to recapture some of the joy of possibility that hooked me on computing in the first place. Perhaps it's the physical board itself, so unlike a phone or tablet, with its rough circuit board edges crammed to bursting with connectors, sharp solder points and interface pins sticking weirdly out of every surface. Or maybe it's the surprising scale that delights, the wonder that something so small can do so much. It could be all those I/O pins practically begging to be hooked up to lights, sensors, and other gizmos. Whatever the reason, there is something about this device that manages to break through the conventional abstractions to reawaken wonder, the possibility of possibility.

Our first Pi project was a humble one: we hooked up a software-controlled LED, which is pretty much the equivalent of “hello, world!” for the Raspberry Pi.

As with most tech projects, even something as seemingly simple as a light on a switch requires daunting amounts of paraphernalia, patience, and time. For us, though, that was part of the fun: trucking around town on some fool idea of dad's, rummaging in obscure corners of the basement closet for resistors, LEDs, and bits of wire, the improbability of seeing the Pi driving an HDTV, fidgeting while the adults scratched their heads over some technical snag.

To get started on your own Pi adventure, you'll need a Raspberry Pi board [1], a micro SD card with the Raspbian OS installed [2], a micro USB power source (such as a phone charger), a screen with HDMI or composite video inputs and appropriate cabling, and a USB keyboard and mouse. (Whew!) Test booting the Pi up to make sure you have a working setup; there is nothing quite like the frustration of getting stuck for an hour and then discovering a bad cable.

For the LED mini project you'll need a few more parts: an LED, a current limiting resistor, some wire, and jumpers to attach to the Pi's GPIO pins. If you've got an old desktop computer lying around, you can scrounge the LED, wire, and jumpers from its indicator lights. And you should probably get yourself a breadboard and jumper wires for it, but for this simple circuit I was content to solder the components directly.

There are many excellent showcase and tutorial sites out there to spur project ideas and help you get going. We headed over to Gordon's Projects Single LED tutorial, which takes you through the steps of getting the LED connected; he does a nice job of breaking the narrative into digestible chunks and interspersing concept explanations to help orient beginners. (I still find the multiplicity of GPIO pin numbering and addressing schemes confusing, but I don't at all fault Gordon for that). Following his instructions we quickly had the LED wired up and connected to our Pi, which we then tested with the gpio utility, prepackaged with Raspbian Jessie, that can be used to communicate with the Pi's GPIO bus. From the console we ran these commands:

The pattern of these commands is gpio <subcommand> <pin> <value>, so the first command translates roughly to “set GPIO user pin 0 to output mode.” GPIO pins can be configured for input or output, depending on whether you are reading from a sensor or driving a motor or LED. Once the pin was configured, we used the second command to turn the LED on and the third to turn it off.

With that working, the obvious next step was to control the LED using Python. There are several Python libraries for interacting with GPIO; we used GPIO Zero, as documented in this tutorial by the Raspberry Pi Foundation. Here's the same sequence of commands as above, but this time using Python:

Instead of interacting directly with the bus, GPIO Zero provides a number of utility classes for dealing with commonly used devices; unsurprisingly the LED class is used to control an LED. The class encapsulates the fact that the LED is an output device and configures the pin to output mode for us when we instantiate it.

Why 17? That's the pin number according to the GPIO numbering scheme, which is equivalent to wiringPi user pin 0 (hence the 0 in the gpio commands above); both refer to physical pin 11 on the board. Ahem.

Pin numbers aside, our next challenge was to make the LED blink, which we accomplished with this simple Python script:

By this point I'd pretty much exhausted the boys' attention capacity, so we got out coloring supplies and drew some Christmas trees. Taking the LED we'd wired up and pushing it through the cardboard yielded a simple Raspberry Pi-powered Christmas display! Over the next few days I added three additional LEDs and some programming to expand the display:

So was this project successful in establishing a deeper interest in the workings of computing technology with my children? Time will tell, but one gain afforded by our little hack is that it has given us a platform to talk about how other things work. They have many small toys that use microcontrollers to control lights, drive motors, react to button presses, and play sounds and music; seeing the construction of this simple project has let me draw their attention to the basic workings of such devices via analogy to our Pi tree. At least in a small way I think they are beginning to understand that electronic devices are not magic and to see how we might build simple devices like the toys they're used to.

My next project? I'm working on building a Raspberry Pi-based geotagger for my Nikon DSLR; I'll be documenting the steps in a series of posts over the coming weeks, so stay tuned!

* * *

Resources

• source code for our Christmas tree display
• Raspberry Pi Learning Resources from the Raspberry Pi Foundation
• GPIO Zero library documentation
• Gordon's Projects GPIO Examples
• Gadgetoid has a nice interactive pinout guide

---

[1]	I recommend the Raspberry Pi 2 Model B, but the older models should work as well. Avoid the Raspberry Pi Zero when you're starting out: the Zero is incredibly cool and tiny, but it is less user-friendly than the other models as it lacks some of their connectivity conveniences.

[2]	There are serveral other OS options for the Pi, and you should obviously go play around with all of them. The examples in this post assume the presence of several supporting tools and libraries that are preloaded on Raspbian Jessie.

A Native Win32 Application in Python

I'm currently working on a small personal project, one component of which needs to use a native Windows edit control. Without getting too bogged down in the why for the moment, I have started exploring the possibility of creating a native implementation in Python using pywin32, rather than relying on a bridge layer such as IronPython/.NET or wx.

What is pywin32? pywin32 is a set of Python bindings over a large chunk of the Windows API. That API presents a formidable visage to a non-Windows developer like me, with its bewildering naming conventions, indistinguishable groupings and boundaries, and sheer age and scale. pywin32 lets you work with much of this API within the comfort of Python, handling many of the tricky details like type marshaling and function/pointer conversions for you. However, it does assume a high degree of familiarity with the Windows API, and the documentation, while a useful reference, doesn't do much to mitigate the difficulties of approaching that API for a beginner.

My goal in this post is to demonstrate how to create a native dialog-based window with Python and pywin32. Our little app will contain a RichEdit text editing control and demonstrate two mechanisms for receiving messages: the parent window uses a message map to route messages to Python methods, but we'll also see how to override the edit control's WndProc function, which offers absolute control over its behavior.

The core of the app is derived from the win32gui_dialog sample in pywin32. If you'd like to try it out yourself, the entire source is here. I'm running Python 2.7.5 32-bit with pywin32 build 218 on Windows XP SP3.

Let's get started!

import win32¶

As usual, we begin with some essential imports.

In [16]:

import struct

import win32gui_struct
import win32api
import win32con
import winerror
import winxpgui as win32gui

win32con is a sort of win32.h header file for Python, defining many of the constants and flags needed when making any meaningful Windows calls. win32api provides access to the core Windows API, including facilities for loading and accessing dynamic libraries.

But it is win32gui that provides all the critical calls needed to create a native Windows application. pywin32 supplies two versions: winxpgui is identical to win32gui except for being built with manifest settings for Windows XP, allowing it to take advantage of certain window system features like the updated theme support built into that version. As I was working, I found the C source for win32gui to be a helpful guide out of a few sticky spots.

win32con defines many, but not all, of the constants needed by our little app. I'm going to define a few of the missing ones I know we'll need now but defer the explanation of what they are for until we need them.

In [17]:

EM_GETEVENTMASK = win32con.WM_USER + 59
EM_SETEVENTMASK = win32con.WM_USER + 69
EM_SETTEXTEX = win32con.WM_USER + 97
EM_EXSETSEL = 1079

As usual when I'm doing any sort of exploratory work in Python, I rely on IPython's object querying facilities to interrogate module attributes and interfaces. For example, this search locates all the attributes in win32con that start with EM and contain the string TEXT; the results confirm the need for the definition of EM_SETTEXTEX above:

In [43]:

win32con.EM*TEXT*?

       win32con.EMR_EXTTEXTOUTA
       win32con.EMR_EXTTEXTOUTW
       win32con.EMR_POLYTEXTOUTA
       win32con.EMR_POLYTEXTOUTW
       win32con.EMR_SCALEVIEWPORTEXTEX
       win32con.EMR_SETTEXTALIGN
       win32con.EMR_SETTEXTCOLOR
       win32con.EMR_SETVIEWPORTEXTEX
       win32con.EM_GETLIMITTEXT
       win32con.EM_LIMITTEXT
       win32con.EM_SETLIMITTEXT

Having established the constants we'll need, we next define a local identifier for our edit control:

In []:

IDC_EDIT = 1024

The IDC prefix seems to be a Windows convention that I guess stands for IDentifier for Control; this constant can be used to retrieve the control's handle and will allow us to filter messages in the parent window later.

The last of the initialization steps get the Windows controls ready for use. In addition to initializing the standard controls, I'm also going to load the RichEdit 4.1 control since that's the version I'd like to use. Even my plain vanilla install of XP probably has at least 3 versions of RichEdit available, so you need to be specific if you want to take advantage of features in the later versions. Be warned: the versioning scheme for this control is completely inscrutable.

In [18]:

win32gui.InitCommonControls()
hInstance = win32gui.dllhandle

# Load the RichEdit 4.1 control
win32api.LoadLibrary('MSFTEDIT.dll')

Out[18]:

1262485504

Scaffolding¶

With the initialization out of the way, we can turn to the task of defining the Python class to encapsulate our window. A typical first chore in any Windows application is to register a window class (WNDCLASS) for your application's main window which defines the options the window system will use when drawing the window. The meat of this registration code is drawn directly from the win32gui_dialog sample; I have simplified it slightly and added the guard to ensure that the registration happens only once.

In [31]:

class DemoWindow(object):
    class_name = "Pywin32DialogDemo"
    class_atom = None

    @classmethod
    def _register_wnd_class(cls):
        if cls.class_atom:
            return

        message_map = {}
        wc = win32gui.WNDCLASS()
        wc.SetDialogProc()  # Make it a dialog class.
        wc.hInstance = hInstance
        wc.lpszClassName = cls.class_name
        wc.style = win32con.CS_VREDRAW | win32con.CS_HREDRAW
        wc.hCursor = win32gui.LoadCursor(0, win32con.IDC_ARROW)
        wc.hbrBackground = win32con.COLOR_WINDOW + 1
        wc.lpfnWndProc = message_map  # could also specify a wndproc.
        # C code: wc.cbWndExtra = DLGWINDOWEXTRA + sizeof(HBRUSH)
        #                         + (sizeof(COLORREF));
        wc.cbWndExtra = win32con.DLGWINDOWEXTRA + struct.calcsize("Pi")

        # load icon from python executable
        this_app = win32api.GetModuleHandle(None)
        wc.hIcon = win32gui.LoadIcon(this_app, 1)

        try:
            cls.class_atom = win32gui.RegisterClass(wc)
        except win32gui.error, err_info:
            if err_info.winerror != winerror.ERROR_CLASS_ALREADY_EXISTS:
                raise

Next we define the configuration for the specific window instance we want to create in the form of a dialog template. The Windows API provides some helper functions that allow dialogs to be created via a list of templates: the template for the window itself followed by any number of templates for items to go into the window. As with the window class registration, the options I'm using here are straight out of the sample; MSDN has more information about the available window style options and the dialog specific options.

In []:

    @classmethod
    def _get_dialog_template(cls):
        title = "pywin32 Dialog Demo"
        style = (win32con.WS_THICKFRAME | win32con.WS_POPUP |
                 win32con.WS_VISIBLE | win32con.WS_CAPTION |
                 win32con.WS_SYSMENU | win32con.DS_SETFONT |
                 win32con.WS_MINIMIZEBOX)

        # These are "dialog coordinates," not pixels. Sigh.
        bounds = 0, 0, 250, 210

        # Window frame and title, a PyDLGTEMPLATE object
        dialog = [(title, bounds, style, None, None, None, cls.class_name), ]

        return dialog

At last we get to the constructor, which registers the window class (if necessary), gets the dialog template, and asks the windowing system to create the dialog window:

In []:

    def __init__(self):
        message_map = {
            win32con.WM_SIZE: self.on_size,
            win32con.WM_COMMAND: self.on_command,
            win32con.WM_NOTIFY: self.on_notify,
            win32con.WM_INITDIALOG: self.on_init_dialog,
            win32con.WM_CLOSE: self.on_close,
            win32con.WM_DESTROY: self.on_destroy,
        }

        self._register_wnd_class()
        template = self._get_dialog_template()

        # Create the window via CreateDialogBoxIndirect - it can then
        # work as a "normal" window, once a message loop is established.
        win32gui.CreateDialogIndirect(hInstance, template, 0, message_map)

Up to this point, the code we've seen is, syntactic differences aside, more or less what you would find in a C or C++ program. Here we come to the first significant difference between a Python and C/C++ implementation: message dispatching. Traditional Windows applications use a "window procedure" (WndProc) to handle the messages that a control receives throughout its existence. These procedures are the brain of the control, defining how it is painted, how it responds to user input, and what control-specific capabilities it offers. They are typically implemented as a switch statement that dispatches messages received from the window system to appropriate handlers.

As we will see shortly, the same capability is available in Python, but pywin32 offers an alternative in the form of a message_map, a dict mapping message identifiers to Python callables. When defined this way, pywin32 builds a small wrapper procedure that uses the message_map to dispatch incoming messages to the appropriate handler in Python. If no handler is defined in the map, the default handler for the control is called. This mechanism makes it easy to subscribe to only messages of interest using whatever Python callable type happens to be convenient. Here we're using methods, but a function or other callable would work just as well.

The main window of our application is listening for two basic kinds of messages: notifications from the windowing system (WM_SIZE, WM_INITDIALOG, WM_CLOSE, and WM_DESTROY) and notifications from child controls (WM_COMMAND and WM_NOTIFY). What child controls? Erm… Right! Child controls, I guess we'll need some of those!

Control Your Edits¶

If you've made it this far and are working on your own application (rather than using notepad), chances are very high that you want to do something other than the default behavior for certain events. In my case, I am primarily interested in three kinds of changes: changes to the text, changes to the selection state, and keystrokes that did not alter the text.

We'll examine the last two cases later, but to detect text changes, I'm going to use a custom WndProc for our (yet to be created) RichEdit control:

In []:

    def edit_proc(self, hwnd, msg, wparam, lparam):
        print 'edit_proc', hwnd, msg, wparam, lparam

        if msg == EM_SETTEXTEX:
            print 'EM_SETTEXTEX'
        elif msg == win32con.WM_SETTEXT:
            print 'WM_SETTEXT'
        elif msg == win32con.EM_REPLACESEL:
            print 'EM_REPLACESEL'

        # I'm not sure why this is needed (i.e. not handled by the default
        # handler), but without it the process hangs on window close.
        elif msg == win32con.WM_DESTROY:
            win32gui.DestroyWindow(hwnd)
            return

        # "super" call to the overridden WndProc
        return win32gui.CallWindowProc(self.old_edit_proc, hwnd, msg, wparam,
                                       lparam)

This method redefines how our edit control will process messages it receives. The msg parameter contains the message identifier that indicates what the message is and determines the meaning of the wparam and lparam arguments. Many of these messages are control-specific, and RichEdit boasts a rich set of messages it understands.

I'm primarily interested in text changes, which are one type of message that can be sent to the control. For this example, when the control receives a message to change all or part of the text (EM_SETTEXTEX, WM_SETTEXT, or EM_REPLACESEL), we simply print out the receipt of the message and continue; an actual application could take any desired action here.

A WndProc can preserve the inherited behavior of the control by calling the default procedure for the control using CallWindowProc; of course nothing requires us to send the same message we received up the chain…

So where did this mysterious old_edit_proc come from? I'm glad you asked!

In []:

    def _setup_edit(self):
        class_name = 'RICHEDIT50W'
        initial_text = ''
        child_style = (win32con.WS_CHILD | win32con.WS_VISIBLE |
                       win32con.WS_HSCROLL | win32con.WS_VSCROLL |
                       win32con.WS_TABSTOP | win32con.WS_VSCROLL |
                       win32con.ES_MULTILINE | win32con.ES_WANTRETURN)
        parent = self.hwnd

Not there!

Following the lead of the win32gui_dialog example, I have deferred the creation of the edit control to the handler for WM_INITDIALOG, which calls _setup_edit to actually create the edit control and install it in our window. If there is a non-whimsical reason for doing it this way, as opposed to installing the control via the dialog template mechanism above, I don't know what it is. I'll make something up if you like.

Regardless, we first establish the basic parameters we want for the control, then create it:

In []:

        self.edit_hwnd = win32gui.CreateWindow(
            class_name, initial_text, child_style,
            0, 0, 0, 0,
            parent, IDC_EDIT, hInstance, None)

Surprise! Yup, you read that correctly: the command used to create a RichEdit control, CreateWindow, is the same command that is used in Windows to create… (drumroll)… windows! I guess it's windows all the way down…

A few notes about this call: the class_name indicates that we want a RichEdit 4.1 control. Yes it does. I warned you that the versioning is weird… We also set the parent of our edit control to be our dialog window and set the local identifier for the control to our fortuitously predefined constant IDC_EDIT. The zeros specify the bounds of the control; we have to pass something, but I want the control to fill the window, which we will handle later.

Still looking for old_edit_proc are you?

The next thing we do is to tell the edit control to notify its parent (our dialog window) of keystroke events, selection changes, and text updates. To do this, we send an EM_SETEVENTMASK message to the RichEdit control, indicating in the message payload which messages should be sent:

In []:

        message_mask = (win32con.ENM_KEYEVENTS | win32con.ENM_SELCHANGE |
                        win32con.ENM_CHANGE | win32con.ENM_UPDATE)
        win32gui.SendMessage(self.edit_hwnd, EM_SETEVENTMASK, 0, message_mask)

Finally, we install the custom edit WndProc we defined above, and, in a fit of nostalgia, save a reference to the old one. (Actually, we save it because we need it…)

In []:

        self.old_edit_proc = win32gui.SetWindowLong(self.edit_hwnd,
                                                    win32con.GWL_WNDPROC,
                                                    self.edit_proc)

There's that rascal old_edit_proc! When changing a control's WndProc, SetWindowLong returns a reference to the old procedure, which we can call by passing the reference to CallWindowProc as above.

So far, we have registered a window class for our main window, created an instance of that window class using a dialog template, added a RichEdit control as a child control of that window, and installed a custom WndProc for the edit control to customize the way the control processes messages. In the next section, we'll see how to configure the notification handlers to allow the main window to respond to notifications it receives from the window system and its contained controls.

Sizing things up¶

The next major section of the application defines the notification handlers referenced earlier in our dialog window's message_map. On initialization, we set up the edit control, then resize it to fill the window:

In []:

    def on_init_dialog(self, hwnd, msg, wparam, lparam):
        self.hwnd = hwnd
        self._setup_edit()
        l, t, r, b = win32gui.GetWindowRect(self.hwnd)
        self._do_size(r, b, 1)

    def _do_size(self, cx, cy, repaint=1):
        # expand the textbox to fill the window
        win32gui.MoveWindow(self.edit_hwnd, 0, 0, cx, cy, repaint)

The resize handler is straightforward: when the user resizes the window, the window system notifies the window with a WM_SIZE message, passing the new size as the lparam payload. Our handler simply extracts the new width and height of the window and passes them to _do_size:

In []:

    def on_size(self, hwnd, msg, wparam, lparam):
        x = win32api.LOWORD(lparam)
        y = win32api.HIWORD(lparam)
        self._do_size(x, y)
        return 1

What's happening?¶

Earlier, we asked the edit control to send certain messages to its parent window. Messages from child controls to their parent use one of two message envelopes, WM_COMMAND or WM_NOTIFY. Which envelope gets used is message-specific and largely a matter of historical accident; RichEdit, being an old but frequently updated control, uses both.

When listening for these messages, the handler will need to examine the envelope to determine the origin control and the type of message being sent. For WM_COMMAND encapsulated messages, lparam is the window handle of the origin control and the upper bits of wparam contain the message identifier:

In []:

    def on_command(self, hwnd, msg, wparam, lparam):
        print 'on_command', hwnd, msg, wparam, lparam

        msg_id = win32api.HIWORD(wparam)
        print msg_id

        if lparam != self.edit_hwnd:
            print 'origin not edit, skipping'
            return 1

        if msg_id == win32con.EN_CHANGE:
            print 'EN_CHANGE'
        elif msg_id == win32con.EN_UPDATE:
            print 'EN_UPDATE'

        return 1

WM_NOTIFY messages are essentially the same but use a different packaging structure: wparam contains the local identifier for the control (e.g. IDC_EDIT) and the message identifier is part of the NMITEMACTIVATE struct passed as lparam. Fortunately for us, win32gui_struct already knows how to parse these structures, so we get the convenience of working with Python objects:

In []:

    def on_notify(self, hwnd, msg, wparam, lparam):
        print 'on_notify', hwnd, msg, wparam, lparam

        info = win32gui_struct.UnpackNMITEMACTIVATE(lparam)

        if wparam != IDC_EDIT:
            print 'origin not edit, skipping'
            return 1

        if info.code == win32con.EN_MSGFILTER:
            print 'EN_MSGFILTER'
        elif info.code == win32con.EN_SELCHANGE:
            print 'EN_SELCHANGE'

        print info.code

        return 1

Wrapping up¶

The last two handlers define our application's shutdown sequence: on_close is triggered when the user closes the window, which gives the application a chance to not close if that is appropriate. For example, this is where a real application would prompt the user if the edit buffer was dirty.

In []:

    def on_close(self, hwnd, msg, wparam, lparam):
        win32gui.DestroyWindow(hwnd)

    def on_destroy(self, hwnd, msg, wparam, lparam):
        # Terminate the app
        win32gui.PostQuitMessage(0)

Our main function is very simple: we create an instance of our window and then start the Windows message pump, which will run until the PostQuitMessage call in on_destroy.

In []:

def main():
    DemoWindow()
    # PumpMessages runs until PostQuitMessage() is called by someone.
    win32gui.PumpMessages()


if __name__ == '__main__':
    main()

Conclusions¶

We have now built a complete, if small, Windows application, including the infrastructure needed to receive and act on messages in both the main window and the edit control. The application is composed of two stages: initialization and message processing. The initialization stage includes registering the window class, setting up configuration parameters for the controls, and wiring up message handling. Message processing involves listening for and reacting to the various forms of messages our application can receive. Our application demonstrates two forms of message processing: the dict-based message_map approach and a control with an overridden WndProc.

I hope that this little tour will help you better navigate the vast and varied terrain in the Windows API if you find yourself needing to write a native application. And what of my motivation, the coy why above? That, dear reader, is a story for another day.

Resources

• the complete source as a gist
• pywin32 download and source root
• Windows and Messages Home on MSDN
• A simpler application in C

Kajiki, Blocks, and Nesting

Last week I switched one of our newer projects to the Kajiki template engine from Genshi. Genshi's flawed mental model and py:match "misfeature" (as Kajiki's documentation rather gently puts it) had become too much of a roadblock to creating dynamic web applications built of small, composeable, reusable fragments.  While there are many worthy contenders in the Python template engine space, Kajiki's Genshi-like syntax made it somewhat less foreign and therefore a bit of an easier sell to others on the team.  Thus far I am pleased with the switch; Kajiki's py:import directive alone is nearly enough to justify changing.

The biggest difference between the two systems is their handling of inheritance: Kajiki jettisons Genshi's problematic include/match template model for true template inheritance with Jinja-inspired blocks.  As a quick example, a minimal parent template such as

<!DOCTYPE html>
<html>
<body>
    <div py:block="content">Eat your veggies!</div>
</body>
</html>

defines a block whose content can optionally be replaced by a child template:

<py:extends href="parent.html">            
    <py:block name="content">
        Candy and TV! (parent said: ${parent_block()})
    </py:block>                                 
</py:extends>

This simple example nevertheless illustrates the main features of Kajiki's inheritance scheme:

• the parent template defines a block that provides an extension point that descendant templates can manipulate
• the child template extends the parent template the same way a child class extends a parent class
• the child template can override the parent's block by defining its own block with the same name; the special parent_block() function provides the child block access to the parent's content

One of the bumps we ran into related to nested blocks: is it possible to nest a block inside another block? In our application, the use case for this is navigation: our site uses two-level navigation, so a page will generally define the elements of the secondary navigation itself while inheriting its main navigation from the master template for that section of the site.

While struggling to fit the pieces together in the midst of a template system conversion, I came up with the following example to help me understand how Kajiki's block system handles nesting.  Here is the slightly modified parent template:

<!DOCTYPE html>
<html>
<body>
    <div py:block="content">
        Eat your veggies!
        
        <span py:block="nest">nested block</span>
    </div>
</body>
</html>

Notice we have the same content block but with another block nested inside of it.  Let's leave the child template exactly as before and create a third template extending the child:

<py:extends href="child.html">
    <py:block name="nest">
        Nested content (nested parent said: ${parent_block()})
    </py:block>
</py:extends>

Rendering this template produces this output from the content block:

Candy and TV! (parent said: <div>
    Eat your veggies!
 
    Nested content (nested parent said: <span>nested block</span>)
</div>)

Which is pretty much exactly how I would have expected this to work.

Download the full examples and play around with them to solidify the interaction between the three templates; both are self-contained Python scripts that depend only on Kajiki to work.  In particular, notice how the organizational hierarchy of the nested blocks is preserved, even after the child template wraps the parent block with additional content.

As a challenge, what do you think happens if nested inherits directly from parent (instead of child)?  How about if you remove the child's call to parent_block()?

Design Properties

I recently encountered an object design rule of thumb to "make any expensive operation a method." Setting aside the rather elastic notion of what "expensive" means, let's examine the implications of the rule itself. Hiding beneath the surface of this seemingly sensible rubric lies a subtle permutation of that pernicious development pitfall, premature optimization. To see why, we'll need to examine some of the metaphors and assumptions used by object based languages and designs.

Object Assumptions

Informally, objects in an object based language provide mechanisms for modeling concepts within the system in terms of data with attached operations or actions, known as methods. Objects are said to be encapsulated, the idea being that the object is organized conceptually into an external interface that wraps an object's implementation details. Think of a cell: the cell membrane forms the external interface that exposes external interaction points but also encapsulates the internal machinery that makes the cell function.

Diagrammatic overview of a typical animal cell. The cell membrane forms the boundary between the cell and the external world and houses the cell's signaling interface.

Returning to a software environment, consider a generic hypothetical list object:

fib = [1, 1, 2, 3, 5, 8, 13]

The object's data consists of the elements it contains, while its methods will provide useful operations over that data like sorting, searching, and counting.

A key assumption in object systems is that the internal details of the implementation of the object are abstracted away from its interface; client code that uses an object is concerned with what an object does and not with how it does it. Returning to the list example, we might implement a list using one of any number of techniques—a memory array, linked list data structure, or something more exotic—but the client code using a list object can query and manipulate a list through its exposed data and methods in blissful ignorance of these details.

Properties

What are properties, and how do they fit into this? Suppose we wanted to know the length of a list object:

> log(fib.length)
7

How would we model and implement this length attribute? Clearly length is a bit of data, namely a number representing the count of the items in the list. On the other hand, length is derived: it is a secondary characteristic of the primary data in a list object, its elements. Should we represent length as a data field or as a method?

fib.length or fib.count()???

Languages like Python, C#, and some dialects of JavaScript offer an alternative modeling technique: properties. A property presents an attribute externally as a data member while providing internal mechanisms (in method form) for customizing the implementation of attribute access and update. Properties serve two main object design needs:

• exposing a data member to the outside world while retaining control within the object over how it is used (e.g. by denying write access)
• presenting as data derived attributes whose value is computed from the object's primary data or some other source

Properties provide a way for object designers to communicate the data nature of an attribute while retaining control within the object over how that data is used.

Conclusion

So what then of the opening rubric? If properties are first and foremost a design tool, then categorically eschewing them for "expensive" operations prematurely subordinates design considerations to performance ones. Yes, performance considerations in a real-world system will shape and reshape object and system design, but such reshaping must be recognized as compromise, a secondary consideration to the prime goal of designing good systems.