:: eat the dots ::

The Power to do it Wrong

2011-10-15T22:26:00.000-07:00

I believe I am not alone in finding myself, more often then I might like, having to make less than ideal decisions in the world of software development. I find myself lamenting inconsistencies and warts, in tools, runtimes, and applications themselves. It is a universe of our own making, sometimes my own personal making, and often it seems to tend toward a proverbial hell rather than heaven.

The process of realizing something, manifesting it from the ideal abstract notion to concrete reality, is a process of compromise. The ideal world cares not of constraints, contradictions, complications, differing opinions, and arbitrary decisions of the past. The ideal world does not care that the human mind and body have physical limitations, and that we cannot sustain long between breaths, meals, and checking twitter. We may assume so, but the ideal world is not lenient, it knows no deviance from ideal elegance. It tolerates nothing that is not perfect, in fact it flatly refuses the existence of imperfection.

There are software tools, and systems, and quite popular ones at that, that seem bent on imposing an ideal model, a "one true way", if you will. I am tolerant of many things, I am not a man of overbearing opinion. I like to think that everyone is largely always wrong, myself included, and I find some pleasure in life watching that prove to be true. But, there is one philosophy that I simply cannot abide as a result, the one true way. This is not something I learned, as Lady Gaga would say, I was just born this way. But my personal inability to deal with dogma is not the point here.

Perfection is assumed to be a great and wonderful thing, how could it be not? What about all those quaint stories of falling from grace, the eternal struggle to regain that which we lost through our own greed and arrogance? The great temptation, the human condition, transcending evil, and all that? Wasn't software supposed to be "easy?" Some sort of mathematical paradise devoid of human blight? Yeah well, only the kind of software to which we deridingly refer, you know the kind with "no users."

Ok, so I'm saying perfection is bad? No, I say perfection is tyrannical, it is the ultimate one true way. However, I needn't let that throw me into an existential funk. Because, if there is one truth that I can hold dear, it is that perfection is impossible. And even if it were possible, I could not exist there, thus any resulting self-loathing is self-prevented.

But, crucially, the notion of perfection, of the ideal, of the simple, of the absence of compromise is not bad. In fact it is an essential human quality. It's a strange notion that chasing something impossible, something dubious, is noble, but that's what we do. Every developer that I know that's worth their salt wants to find the elusive ideal, or at minimum the least worst solution for the problem at hand. The mind of the master knows that perfection of the whole is impossible, yet will still strive to simplify, to idealize, to remove compromise where it can. Mastery itself is largely the ability to direct that effort most effectively, and to know when it is futile.

So, we must embrace that the universe, the software, the hardware, and the wetware is imperfect, and deal with it. And those tools, and technologies that best deal with that reality are the ones we should embrace. Often, it is not even a matter of embracing, but more of natural selection. Those tools that over-idealize, that preach the one true way, will eventually lose. Yes, worse is better.

So the next time you are working on a web page, and you find that no amount of dogmatic adherence to CSS best practices will fix your layout, be glad that the tools are bigger, dare I say better, than the ideal. Sure, you may say a few choice words when you have to use a <br/> or *gasp* a <table> tag. But think about what you would say if there were no workaround. If the tools assumed perfection, we'd all be screwed.

Great technologies are imperfect, they are flawed, they break their own rules. Great people are no different, though often their faces are more symmetrical than yours, but I digress. The point is that there will never be perfect software tools, and you should be wary of those purporting the one true way. Besides, perfection is boring, and imposed perfection is stifling. We need tools with abstractions that leak, with models that don't always map, and with escape valves that let us do it wrong. Because sometimes doing it wrong is the best way, the only way it can be done.

Running Tests for Multiple Pythons

2010-08-17T21:31:00.000-07:00

For my Planar project, I'm committed to being compatible for Python 2.6-3.1 and beyond. Since this project contains C extensions, that means that I need to be able to build and test for each Python conveniently. And yes, different Pythons do behave differently, have different compilation gripes, and sometimes even expose different bugs due to internal differences in GC, etc.

This shell script I wrote is a simple, but effective solution. It requires that all of the versions of Python you are testing are installed (of course), and also have nose installed. It is very easy to extend to more Python versions as well by modifying the for loop at the top. You run it from the top-level source directory, alongside your setup.py. I imagine other folks have come up with more elegant solutions than this, but I figured it might be useful to someone, so here it is:

#!/bin/bash
# Build from scratch and run unit tests 
# from python 2.6, 2.7 and 3.1
# Then run doctests to verify doc examples
# Note: requires nose installed in each python instance

error=0

rm -rf build

for ver in 2.6 2.7 3.1; do
 echo "************"
 echo " Python $ver"
 echo "************"
 echo
 if which python${ver}; then
  python${ver} setup.py build && \
  python${ver} -m nose.core \
   -d -w build/lib.*${ver}/ --with-coverage $@ || error=1
 else
  echo >&2 "!!! Python ${ver} not found !!!"
  error=1
 fi
done

echo
echo -n "Doctests... "
srcdir=`pwd`
cd build/lib.*3.?/ && \
  python3 -m doctest ${srcdir}/doc/source/*.rst && \
  echo "OK" || error=1

exit $error

The bottom part runs doctests you might have in doc/source, which is where I keep the sphinx doc sources.

Planar 0.1

2010-04-26T21:26:00.000-07:00

Planar is a 2D geometry library I'm working on for use by my Grease game engine. For maximum usefulness, however, I'm distributing planar as a separate library.

Since it doesn't have any external dependancies of its own that dictate otherwise, I took the liberty of making it compatible with both Python 2.6+ and 3.1+. Planar implements everything in both Python and C, so it was a good opportunity to finally jump into 3.x with both feet. I was pleased with how easily 2to3 worked to seamlessly take care of the necessary Python code mungification. I was also pleased with the C side of the world as well. Conceptually things haven't changed too much in 3.x, and a few #ifdefs here and there and some macros took care of the API differences there.

Once I got it initially working in Python 3, I found myself coding against it and fixing 2.x incompatibilities afterwards. I think that's a good sign.

The functionality of 0.1 is very basic, but it's fully documented and tested, so I'm satisfied that I'm ready to move on and implement more. Now that I've learned a bunch of the Python 3 ropes, I'm hopeful that I can make fast progress toward 0.2.

Head over the the planar docs site to learn more. Feedback is encouraged and appreciated.

Arg Parsing Micro-Benchmark

2010-04-14T23:38:00.000-07:00

I'm working on a vector class for my shiny new Planar geometry Python library. When I was working on Lepton (which is mostly in C), I was keenly aware of the overhead of the generic arg parsing C-APIs (PyArg_ParseTupleAndKeywords and friends). Since this vector class largely consists of methods that do very little work, but may be called fairly often, I figured this overhead should be considered.

I implemented a class method to instantiate a vector with polar coordinates, which are converted to cartesian. Here it is in Python:

class Vec2(tuple):

    def __new__(self, x, y):
        return tuple.__new__(Vec2, 
            ((x * 1.0, y * 1.0)))

    @classmethod
    def polar(cls, angle, length=1.0):
        """Create a vector from polar coordinates"""
        radians = math.radians(angle)
        vec = tuple.__new__(cls, 
            (math.cos(radians) * length, 
             math.sin(radians) * length))
        return vec

Note, using tuple.__new__ in polar is an optimization that saves a layer of method calls and skips converting the x and y values to floats, since I know they are already. Running this through timeit with Python 3.1.2, I get:

>>> timeit.timeit('Vec2.polar(20, 10)', 
... 'from planar.vector import Vec2')
2.3068268299102783
>>> timeit.timeit('Vec2.polar(angle=20, length=10)', 
... 'from planar.vector import Vec2')
2.3426671028137207

Notice there's not much difference in timing between positional and keyword arguments.

Now let's implement the polar class method in C using generic arg parsing. Here's the method's code:

static PyObject *
Vec2_polar(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
    PlanarVec2Object *v;
    double angle;
    double length = 1.0;

    static char *kwlist[] = {"angle", "length", NULL};

    assert(PyType_IsSubtype(type, &PlanarVec2Type));
    if (!PyArg_ParseTupleAndKeywords(
        args, kwargs, "f|f:Vec2.polar()", 
        kwlist, &angle, &length)) {
        return NULL;
    }

    v = (PlanarVec2Object *)type->tp_alloc(type, 0);
    if (v != NULL) {
        angle = radians(angle);
        v->x = cos(angle) * length;
        v->y = sin(angle) * length;
    }
    return (PyObject *)v;
}

Here's the timing for the above:

>>> timeit.timeit('Vec2.polar(20, 10)', 
... 'from planar.cvector import Vec2')
1.045346975326538
>>> timeit.timeit('Vec2.polar(angle=20, length=10)', 
... 'from planar.cvector import Vec2')
1.578913927078247

This is certainly faster than the Python code, but not tremendously so, though that's not too surprising given the simplicity of this method. However, what is surprising is the performance difference between positional and keyword arguments. But, there it is.

Since this method takes two arguments, and they are both floats, maybe we can speed up the positional case, which should be common, by doing the arg parsing ourselves. We know that PyObject *args is a tuple, so let's take it apart manually and extract the angle and length. A slight complication is that the second length argument is optional, but we can handle it. Here's the beautiful result:

static PyObject *
Vec2_new_polar(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
    PyObject *angle_arg;
    PyObject *length_arg;
    PlanarVec2Object *v;
    int arg_count;
    double angle;
    double length = 1.0;

    static char *kwlist[] = {"angle", "length", NULL};

    assert(PyType_IsSubtype(type, &PlanarVec2Type));
    if (kwargs == NULL) {
        /* No kwargs, do fast manual arg handling */
        arg_count = PyTuple_GET_SIZE(args);
        if (arg_count != 1 && arg_count != 2) {
            PyErr_SetString(PyExc_TypeError, 
                "Vec2.polar(): wrong number of arguments");
            return NULL;
        }
        angle_arg = PyTuple_GET_ITEM(args, 0);
        if (!PyNumber_Check(angle_arg)) {
            PyErr_SetString(PyExc_TypeError, 
                "Vec2.polar(): expected number for argument angle");
            return NULL;
        }
        angle_arg = PyNumber_Float(angle_arg);
        if (angle_arg == NULL) {
            return NULL;
        }
        angle = PyFloat_AS_DOUBLE(angle_arg);
        Py_CLEAR(angle_arg);
        if (arg_count == 2) {
            length_arg = PyTuple_GET_ITEM(args, 1);
            if (!PyNumber_Check(length_arg)) {
                PyErr_SetString(PyExc_TypeError, 
                    "Vec2.polar(): expected number for argument length");
                return NULL;
            }
            length_arg = PyNumber_Float(length_arg);
            if (length_arg == NULL) {
                Py_DECREF(angle_arg);
                return NULL;
            }
            length = PyFloat_AS_DOUBLE(length_arg);
            Py_CLEAR(length_arg);
        }
    } else if (!PyArg_ParseTupleAndKeywords(
        args, kwargs, "f|f:Vec2.polar()", 
        kwlist, &angle, &length)) {
        return NULL;
    }

    v = (PlanarVec2Object *)type->tp_alloc(type, 0);
    if (v != NULL) {
        angle = radians(angle);
        v->x = cos(angle) * length;
        v->y = sin(angle) * length;
    }
    return (PyObject *)v;
}

So, if kwargs is NULL, then we extract the angle, and the length if present from the args tuple. We check if each are numbers, convert them to float objects (so you can pass in ints without complaint), then we extract the C double from each to store in the vector object's struct. Sure looks like a lot of code, let's see if it buys us anything:


>>> timeit.timeit('Vec2.polar(20, 10)', 
... 'from planar.cvector import Vec2')
0.46842408180236816

Bam! Not bad at all! Of course if you do pass keyword args, we fall back to generic parsing, so that should perform the same as the last one:


>>> timeit.timeit('Vec2.polar(angle=20, length=10)', 
... 'from planar.cvector import Vec2')
1.5973320007324219

If I were feeling ambitious, I could extract the args from the PyObject *kwargs dict manually as well, and see some speedup. But at some point the added code and maintenance cost isn't worth it. I know the opportunity is there if needed, but also that passing args positionally is already a lot faster (a fact that is likely worth documenting).

I'm writing this planar library such that every class is coded in Python and C. Also the library is compatible with both Python 2.6+ and Python 3.x. If you're interested in an example of how to write a package that's compatible with Python 2 and 3 with C-extensions, check it out.

Frameworks, Libraries, Namespaces and Distributions

2010-03-22T19:13:00.000-07:00

I've been busy not working on the code for my game engine framework Grease recently. I've actually been working on the documentation, but since I can't help myself, I've been thinking a good bit about the code, and specifically what I want to work on next.

The functionality that I plan to work on next is rather general 2D geometry and vector graphics. The important thing is that although Grease will depend on these things, they are not part of the framework per se. They should be useful for folks who don't buy into Grease's framework dogma, or don't need its features. Simply put they will be libraries that could be perfectly happy outside of Grease or any framework.

So, humble denizens of the Python planet, what is the best approach to naming and packaging these libraries for distribution? I see two possible options, both with pros and cons:

Put them under the grease "brand" packaging them as something like: grease.geometry and grease.vg. They would be distributed separately and probably together with grease the framework as well.
Give them their own independent name (I'm leaning toward flatly) and package them separately from grease, though grease would depend on them.

#1 has some advantages:

Less "brand" complexity. It's obvious that grease.vg is part of the grease project, and when used inside of grease there's one less arbitrary brand-name to remember.
It's easy to envision these libraries fitting snugly with the rest of the framework in terms of documentation and what not.
It's pretty easy to justify folding the release cycle of the libraries in that of the framework.

And some disadvantages:

The independence of these subpackages is unclear, folks may be reluctant to use them if they feel that they carry too much framework baggage or cognitive complexity.
I'm not sure the best way to actually make these separate from a distutils perspective, or whatever distribution flavor of the minute is in fashion. I suspect they should just live in their own directory trees and get stitched into the grease top-level package at installation-time. I'm wary of drawbacks and unintended side-affects of that, however.
They're nested rather than flat (sorry Tim).

#2 also has some advantages:

The independence of the library package(s) is obvious.
It's rather obvious how to organize them from a distribution perspective.
They aren't weighed down by any framework connotations.
The namespace is flatter.

And of course disadvantages:

They don't promote the framework "brand" in any way and may be perceived as less well integrated with the framework.
They would feel like more independent creatures to me, thus I think I would tend to fuss about them individually more. That's a subjective thing, of course, and not necessarily bad for users.
It's not as clear to me how to deal with them as grease dependencies. Bundle them or fetch them at installation time?

Honestly I'm able to justify both ways to myself, and of course it's not a matter of life-and-death. But it does feel like an important api decision and I value any opinion or ideas you may have on the matter. So please comment if you feel inclined.

Also I'm interested in people's opinions on bundling vs. fetching dependencies at installation. In the bad old days, bundling was basically a no-brainer with Python, but given things like setuptools/distribute and buildout, the latter has gained appeal. Which do you prefer?

Grease 0.1 Released

2010-02-28T12:24:00.000-08:00

I released the inaugural 0.1 release of my game engine Grease today. This is basically a throw-it-over-the-wall release so that I can use the code and push it further during the upcoming pyweek compo at the end of March. That said it is complete enough to implement an entire game, an example of which is included in the distribution. Obligatory screenshots below:

The game uses Grease features such as its awesomely-retro vector polygon renderer, collision detection, decorator-based key binding and mode management for hot-seat multiplayer. The entire game is a single script that weighs in about 550 lines, which includes whitespace and docstrings. Even so no real effort is made to make the code short, it was written with clarity in mind, not brevity. Most of the code savings comes from the abstractions available in Grease.

At the moment, Grease is implemented entirely in Python and sits on top of Pyglet. My upcoming efforts on this project will be to make the blasteroids example into a full tutorial and get everything fully documented. After that I will be creating more complex example games and adding native-code parts to Grease where needed. The intention is to always have pure-Python versions of everything available though, and the Python versions will be developed first.

Of course you are welcome and encouraged to try it for yourself. You can download it from pypi here:

http://pypi.python.org/pypi/grease

The code of the example blasteroids game above can be scrutinized here:

blasteroids.py

Enjoy.

Decorate Thy Keyboard Controls

2010-01-16T23:00:00.000-08:00

As part of a game engine called Grease I'm working on (more on that sometime soon) I was thinking about ways to create a clean and efficient api for handling keyboard events. Too often this ubiquitous section of game code consists of a tangle of if/elif statements, a construct that is a personal pet peeve of mine. So to avoid that mess, a typical strategy I've employed is to map keys to methods using a dictionary for easy dispatch. Doing this for one-off applications is simple enough, if not particularly clean and tidy.

Anyway taking a step back for a sec, let's examine some goals. Basically what I'm after is a way to define some methods (or even functions) that get executed in response to key events. Specifically there are three types of key events that I'm interested in: key press, key release and key hold. The first two get dispatched once per key "stroke" as you'd expect. The key hold event fires every game "tick" that a key remains down; useful for continuous functions like thrust, etc.

So I need a clean way to map methods to specific keys and key event types. Sounds like a job for: decorators! Truth be told I haven't felt the need to write many decorators, but after prototyping a non-decorator version that came out less than clean even though it only supported one type of key event, I thought I'd give decorators a go. Below is an example of how to implement key controls using what I've cooked up:

class PlayerControls(KeyControls):

    @KeyControls.key_press(key.LEFT)
    def start_turn_left(self):
        ship.rotation = -ship.turn
    
    @KeyControls.key_release(key.LEFT)
    def stop_turn_left(self):
        if ship.rotation < 0:
            ship.rotation = 0

    @KeyControls.key_press(key.RIGHT)
    def start_turn_right(self):
        ship.rotation = ship.turn
    
    @KeyControls.key_release(key.RIGHT)
    def stop_turn_right(self):
        if ship.rotation > 0:
            ship.rotation = 0

    @KeyControls.key_hold(key.UP)
    def thrust(self, dt):
        ship.body.apply_local_force(
            ship.player.thrust * dt)
    
    @KeyControls.key_press(key.P)
    def pause(self, dt):
        global paused
        paused = not paused

Here's the code needed to wire this into pyglet:


window = pyglet.window.Window()
controls = KeyControls(window)
pyglet.clock.schedule_interval(controls.run, 1.0/60.0)
pyglet.app.run()

Though using the decorators binds the keys to specific methods of the class at compile-time, KeyControls also contains additional methods for changing the key bindings at run-time. I may also add support to load and store key bindings from a configuration file if that feature is needed.

This implementation is designed to work with pyglet, though I think the same general approach could be used with pygame. The module for this, although part of a larger project I am working on, doesn't depend on anything else. Feel free to give it a try and see what you think.

[Edit a new version of this module is now available]

Get the module here.

_ is Better Than _

2010-01-11T16:12:00.000-08:00

If explicit truly was better than implicit, Python code would look like this:

StringBuilder s = new StringBuilder()

And let's face it, that's not enlightening to anyone.

Lepton featured in Python Magazine

2009-07-14T10:20:00.000-07:00

In a nice little article by yours truly:

http://pymag.phparch.com/c/issue/view/100

Lepton Particle Targeting

2009-03-18T21:37:00.000-07:00

The other feature I alluded to in my last post was targeting, which allows you to specify specific starting and ending values for particle attributes and interpolate between them over time. This is used often in spontaneous arrangement effects, such as particles "imploding" from random directions into an orderly pattern, such as an image or logo. But many other effects are possible, since it gives you close control over arbitrary attributes and how they change over time without having to manage all of that complexity directly.

Here is how I envision particle targeting to work in Lepton:

Create a particle group to contain the "target" particles. Populate this group with particles with the desired final state for your interpolated particles.
Create a particle group to contain the interpolated particles with a target controller pointing at the target particle group.
Create or emit particles into the interpolated group however you see fit. Each new particle is matched with a particle in the target group. This matching can be done automatically or manually.
Each time the interpolated group is updated, particle attributes are interpolated between the initial particle state and the target particle state using a tweening function.

The magic happens in the target controller itself. When you instantiate it, you specify the target particle group, the attributes to interpolate, each attribute's tweening function and the particle age where it reaches its target. Since each interpolated particle can have a different initial age value, this means particles can reach their final state in different times. Typically the target particles would not be drawn or even change, but there would be no restriction on that and that could be used to create even more interesting effects.

Here's some theoretical sample code for targeting:

target_particles = ParticleGroup()
# ... populate target_particles ...
terps = ParticleGroup(
    controllers=[
        TargetRandom(target_particles, 
            position=EaseInOutQuad(10), 
            up=EaseInCirc(8)),
    ])

There will be several types of target controllers: Target will require interpolated particles to be matched to their targets manually, TargetRandom will randomly select targets, and TargetSequential will select targets in the order they occur in the target group. The interpolation behavior will be the same for these controller classes.

Interpolation of each attribute will be controlled by tweening functions. The keyword arguments passed to the target controller specify which particle attributes to interpolate and which tweening function to use. The simplest tween is just a linear interpolator, but more complex tweens, such as easings will create more natural looking movement and morphing. You can find a great discussion of tweening and easings in this document.

Better texture support in Lepton

2009-03-17T21:57:00.000-07:00

Now that Lepton 0.9a is out the door, it's time to think about the next steps in the particle engine's evolution. In particular, there are two main features I'd like to work on for the next release:

Greatly improved texture support
Interpolating particle attributes from a specific starting state to an ending state.

The former will be addressed with "texturizers". These will have the following functionality:

Handle the requisite loading of textures into the graphics card
Perform necessary texture-related state changes before rendering particles in a group
Computing texture coordinates for each active particle in the group
Restore the texture state to what is was prior to rendering the group

Here's a tentative interface for a texturizer:

class Texturizer:

 def set_state(self):
   """This method sets the appropriate 
   OpenGL state for rendering. It is called 
   immediately before rendering a group of particles.
   """
 
 def restore_state(self):
   """Restores the OpenGL state to what it was 
   before the call to set_state().
   It is called after rendering a group of particles.
   """

 @property
 def tex_dimension(self):
   """The number of texture coordinates per vertex, 
   either 2 or 3
   """

 def generate_tex_coords(self, group, coord_array):
   """Generate texture coordinates for all active 
   particles in the group, placing them in the 
   cooresponding places in coord_array, an
   array of floats. 
  
   Each particle gets four texture positions 
   cooresponding to its lower left, lower right, upper 
   right and upper left corners respectively.
   The number of coordinates per texture position is 
   determined by the value of tex_dimension.
   """

The texturizer will be instantiated by the user for use with a renderer. The arguments needed to instantiate the texturizer will vary depending on the type of texturizer. Most will accept one or more texture objects (probably borrow some pyglet code for those) and other ancillary arguments to control specific features. Here's an example of how you might use a simple texturizer:

my_texture = Texture('amazing_image.png')
particles = ParticleGroup(
   controllers=[...controllers go here..],
   renderer=BillboardRenderer(SpriteTexturizer(my_texture)))

This would simply apply the same texture derived from the image file amazing_image.png to each particle in the group. By default it would perform bilinear filtering to scale the texture and would clamp at the edges (i.e., it would not tile).

Here are some ideas for different texturizers:

SpriteTexturizer

Could apply one or more texture images to a group of particles. The images would be packed into a single texture atlas and the texturizer would select them by varying the texture coordinates per particle. The images could be applied sequentially (i.e., round-robin) to each particle in a group, or randomly. It may also be possible to apply weights to each sprite image to have some applied more often than others.

AnimatedTexturizer

Could apply a sequence of images to all particles in a group, like above the image frames would be packed into a single texture object in OpenGL. The animation frame assigned to each particle would be based on that particle's age. It would have an option for packing the frames into a 3D texture, so you could smoothly interpolate between the frames on the hardware. This would allow smooth animation with fewer frames.

MultiTexturizer

Could apply multiple textures to all particles in a group simultaneously. This would allow various effects by blending multiple textures together. One usage would be to have a large base texture that may also be tiled. The base texture could stay stationary relative to the particles. A second "splat" texture would be applied to each particle to show the base texture at that particle's position. As particles moved they would reveal different parts of the base texture. The splat texture could have sharp or fuzzy edges depending on the desired effect. This could be used for various simple animation techniques, or different stylized rendering.

On bogus threat levels...

2009-01-23T15:51:00.000-08:00

I fly every few weeks now and for the last who knows how long the threat level has been orange which supposedly represents a "high" threat of a terrorist attack.

to which Steve Mirsky in his regular column in Scientific American has this great retort:

Here’s how you know that the terrorist threat isn’t really high: the airport is still open, and your flight hasn’t been canceled.

Read the whole column here

Multi-texture support for pyglet vertex lists

2009-01-08T16:12:00.000-08:00

I wrote this little hack to allow easy expression of multiple sets of texture coordinates in pyglet vertex lists. I wrote it so I could use the texture splatting technique to draw a terrain map for a game. For that purpose it works very well. It does a small bit of monkey-patching to pyglet itself, so it may break if things are refactored, but it works good for 1.1.x.

To use it, you instantiate your vertex list the normal way except you can specify multiple sets of texture coordinates. Then you call multitexture.enable() passing it your new vertex list. Here's a simple example:


    vl = pyglet.graphics.vertex_list(4,
        ('v2f', (100, 100, 200, 100, 200, 200, 100, 200)),
        ('t2f', (0, 0, 1, 0, 1, 1, 0, 1)),
        ('t2f', (0, 0, 0.25, 0, 0.25, 0.25, 0, 0.25)),
        )
    multitexture.enable(vl)

You can get the module here.

Faking read() support for psycopg's copy_from()

2008-08-28T16:31:00.000-07:00

This has been a burr in my side for a long time. If you've ever tried to use copy_from() in psycopg2 from a source other than a real file, you've probably run into this issue: copy_from() requires a file-like object that supports both readline() and read() (!) methods! This is not an issue if the source of the data is already coming from a file or pipe, but what if you are synthesizing the data line-by-line? Like, I don't know, parsing a huge honking log file and trying to insert it en masse into Postgres? Then what?

Well, I've implemented a semi-ridiculous workaround class that implements read() by buffering up a bunch of lines until it's got enough to shovel off to psycopg2, but of course, lines never exactly fill the requested number of bytes, so you've got to deal with that and you're left with a pretty damn messy, and stupid read() method. Ugh.

Reading the psycopg2 source (ain't open source grand? ;^) made me realize a couple of things. One: readline() is only called with older protocols (i.e., older postgres versions) and would never be used in my case. So really only read() was even used at all. Ok, that's interesting, but read() is the problem-child here, so that doesn't soothe the irritation.

The second thing I saw from the code was that psycopg doesn't care how many bytes read() returns at all. As long as it returns some bytes it's happy. So that made me realize that read() could just return the same thing as readline() and it should still work. I fired up vim and punched in some test code to verify:


import psycopg2

class ReadFaker:

    def __init__(self, data):
        self.iter = iter(data)

    def readline(self, size=None):
        try:
            line = self.iter.next()
        except StopIteration:
            return ''
        else:
            return "%s\t%s\n" % line

    read = readline

fakefile = ReadFaker([
    ('foo', 'bar'),
    ('baz', 'spam'),
    ('bar', 'foo'),
    ('foo', 'foo'),
    ])

db = psycopg2.connect(database='test')
c = db.cursor()
c.copy_from(fakefile, 'foo')
db.commit()

Then I created a test table with two text columns in a local database named test. Voila! It works!

Of course to be proper, my readline() method shouldn't ignore its input size argument, but that's probably more trouble then its worth practically.

Someday I think this use-case should be handled better by psycopg2, i.e., you should just be able to pass it an iterable object of equal-length sequences for each record. Then it can compose the textual representation to throw off to Postgres. There are complexities there around complex types like dates and times, but even if it only handled textual and numeric types and made you deal with the others it would be an improvement. Maybe a "spare time" project for me some day 8^)

Cloudy with a Chance of...

2008-08-19T15:05:00.000-07:00

I got inspired to add another example to my Python noise library. This one creates a real-time animated atmosphere around planet Earth. It uses the GLSL fast fake noise implementation in the library.

(download a higher quality video)

The example uses pyglet. I cooked up a nice little class for capturing frames to png files that can be encoded into a video. It was used to make the above:

class MovieCapture:

    def __init__(self, dirname):
        assert os.path.isdir(dirname), (
            "%s is not a valid directory" % dirname)
        self.dirname = dirname
        self.frame_no = 0

    def capture(self):
        """Capture the current window a file, 
        return the file name"""
        filename = 'frame-%05d.png'
        while os.path.exists(os.path.join(
            self.dirname, filename % self.frame_no)):
            self.frame_no += 1
        mgr = pyglet.image.get_buffer_manager()
        mgr.get_color_buffer().save(
            os.path.join(self.dirname, 
            filename % self.frame_no))
        return filename % self.frame_no

When you instantiate it, you provide a path to a directory to write the captured frames into. Each is written to a sequentially named png file. After drawing each frame, you simply call the capture() method to write it to a new file. Once you've got the frames you need, you can encode them into a movie using mencoder or whatever tool you choose. To make this video I used mencoder like so:

mencoder "mf://atmosphere/*.png" -mf fps=20 -o atmosphere.avi -ovc lavc -lavcopts vcodec=msmpeg4v2:vbitrate=800

Things that will not change in Python 3000

2008-07-25T13:45:00.000-07:00

A useful PEP for you to read:

http://www.python.org/dev/peps/pep-3099/

Why is C faster than Python?

2008-07-17T10:16:00.000-07:00

Questioning the obvious is something we do a lot accidently (and get chided for it), but occasionally it yields some useful insights. A while back, there was a discussion on the pygame mailing list about languages and speed. The poster was insistent that there was no good reason why C was faster than Python. Below is my explanation, which in writing I realized I had never voiced this way before:

When there is no better algorithm, then using a higher performing language is about the only option you have for better performance. There is an even better option, dedicated hardware, but that is far beyond the reach of most mere mortals. But that's the reason why we have things like SIMD instruction sets, GPUs and physics co-processors.

All this conjecturing about Python performance in relation to compiled languages makes me wonder. Why is the performance of C code not as good as that of well-written hand-coded assembler? Surely if the machine can do it, C can?

The problem boils down to one of specificity. Any language that tells the computer more specifically what to do can be faster than one that doesn't. Unfortunately, telling the computer what to do specifically is not a productive way to solve most problems. I'd rather not have to tell the computer which machine codes to execute, but if I did, and I was clever, I could make my program the fastest possible -- for a given specific architecture at least.

Python is more abstract and general than C, C is more abstract and general than machine code. Therefore machine code is potentially faster than C, and C is potentially faster than Python. And unless you reduce the generality of Python, it will always be possible to write faster programs in C, given a competent compiler.

JITs (e.g., psyco) can help mitigating this, but they too are ultimately constrained by Python's expressive power.

So unless you are proposing to reduce Python's expressive power, you are waging a losing battle. Until which time machine intelligence exceeds our own, it will always be possible for a human programmer to get higher performance from the lower-level language. And I further propose that when the time arrives that machine intelligence exceeds our own, this will not be the problem foremost on my mind ;^)

Shader Noise

2008-05-26T00:08:00.000-07:00

After some gnashing of teeth -- mostly involving "stupid" stuff like creating a proper 2-channel 3D texture and computing lighting vectors -- I have working shader noise functions in the noise library. The raw noise functions do 3D Perlin, fbm, and turbulent noise. To demo them I created a script that bump-maps three rotating spheres with some simple real-time procedural textures. The first is just a bumpy texture using straight Perlin noise, the second is a crinkly texture using 4 octaves of fbm noise. The third (bottom) is an embossed turbulent noise texture.

Note that all three spheres use exactly the same vertex geometry, the apparent differences in the shapes are entirely due to the shaders. The effect is even better with a moving image, as is usual with these types of things.

Get the code

Giant Planet Noise

2008-05-20T12:58:00.000-07:00

Among other things, I've been recently working on a Perlin noise library for use in Python games. It does all the heavy lifting in native code, for speed. I'm also working on a shader-based noise function so you can do texture generation on the GPU. I have a few examples of how to use the noise functions, and the other day I added one to generate textures for planets (gas giants to start with). Below are three screen shots illustrating how the texture is composed. The first shot is just several octaves of noise stretched parallel to the equator of the planet mixing two colors. The next adds turbulence by warping the first texture using input from a second noise call. The third uses the turbulence value to alter the luminance of the colors, simulating up and down-drafts in the atmosphere. Given the simplicity of the code and short dev time, I was quite happy with the results.

If I were to use these in a game, I would add some additional features: For one I would use per-pixel lighting instead of the default per-vertex in the fixed functionality to get rid of some ugly faceting artifacts on the shadow border and specular highlight. Also, some sort of planetary glow shading would be nice. I'll probably add the latter to the example sometime.

I haven't released the noise library yet, but it does work and you can grab it from my Google code repository if you want to play with it.

Technique Treasure Trove

2008-04-02T23:10:00.000-07:00

I've recently been on a quest to find the best technique for real-time noise generation in OpenGL. Being able to generate textures is not only good for realism, but a crucial labor-saving device for indie developers with limited resources. While digging around, I came upon a very interesting technical memo on wavelet noise, which greatly reduces certain aliasing artifacts that can occur when Perlin noise and it's many variants are scaled. As cool as this info is, what's even better is the fact that it's part of the larger Pixar online library. I have a distinct feeling that this will be consuming many hours of my time...

SimCity 4's Graphics Engine

2008-03-09T11:43:00.000-07:00

I found this discussion of SimCity 4's graphics engine very interesting. Apparently SC4 is the first version to go with a truly 3D polygonal geometry, though for a variety of reasons discussed in the article they limit the view angles for highest quality and speed. I also found their analysis on the last page that current graphics technology is far better suited to first-person games than god-games like SimCity pretty interesting. Makes sense especially since an FP perspective naturally limits the amount of geometry that must be rendered in high detail.

Taking Flight

2008-03-08T00:05:00.000-08:00

Here is a screenshot from my latest pet-project, a helicopter flight sim. It's written using Python (of course) and Pyglet . This is my first foray into 3D graphics, and I'm having a lot of fun learning OpenGL and training my brain to think in 4x4 transformation matrices ;^)

Writing a semi-realistic simulation (I don't think I can do better having never flown a real helicopter) is a bit humbling and gives me a a lot respect for aerospace engineers who have to make real heavier-than-air objects fly in a controllable fashion. The latter part is the crucial detail and I have spent many an hour trying to figure out why my poor virtual bird (note real pilots don't call them "choppers") would starting flipping out of control seemingly of its own volition.

The terrain is randomly generated by a little python script I wrote last weekend. If you want to play with that part, you can get it from a page I made for it in the Pyglet google group. It's kinda fun and can generate a variety of different terrain types.

Both the terrain script and the prototype heli.py script shown above are available via svn from my code repository. Note the word prototype above. The script is very basic, slightly better than a proof-of-concept and not well factored yet. That said I find it fun to fly around. You'll need Python, Pyglet (1.1) and PyODE to run it.

SQLAlchemy and SqlSoup Rule!

2007-06-19T13:13:00.000-07:00

SQLAlchemy is a python ORM mapper along the lines of Hibernate.

This morning I was trying to figure out how to map all the tables into model objects so I could expose them in this system inventory app I'm working on. Also, I wanted similar functionality to the hibernate session manager. After writing a bunch of code, I discovered that this does everything I need:

db = SqlSoup('postgres://%s:%s/%s' % (dbhost, dbport, dbname))

(where the db* vars are either passed in or derived from a config). I have a table in my database called pc_hardware, after executing the above, with no further configuration whatsoever in the program, I can do the following:

newpc = db.pc_hardware.insert(mac_address='00:16:cb:8c:b5:86', model='macbook pro')
db.flush()

(Confusingly flush() actually means commit)
Then in a later session I can do this:

print db.pc_hardware.select()
[MappedPc_hardware(hardware_id=3L, mac_address='00:16:cb8c:b5:86', cpu_info=None, cpu_cores=None, gb_ram=None, power_supplies=None, console=None, model='macbook pro', form_factor=None, purchase_date=None, receive_date=None, vendor=None, location=None, status='new', last_modified=datetime.datetime(2007, 6,  19, 12, 45, 6, 659993),notes=None)]

pc = db.pc_hardware.get(3)
pc.gb_ram = 16
db.flush()

I can even get at the column definitions if need be:

pprint(list(db.pc_hardware.c))
[Column(u'hardware_id', PGInteger(), primary_key=True, nullable=False, 
default=PassiveDefault()), 
Column(u'mac_address', PGMacAddr(), nullable=False), 
Column(u'cpu_info', PGText(length=None, convert_unicode=False)), 
Column(u'cpu_cores', PGInteger()), 
Column(u'gb_ram', PGInteger()), 
Column(u'power_supplies', PGInteger()), 
Column(u'console', PGBoolean()), 
Column(u'model', PGText(length=None, convert_unicode=False)), 
Column(u'form_factor', PGText(length=None, convert_unicode=False)), 
Column(u'purchase_date', PG2Date()), 
Column(u'receive_date', PG2Date()), 
Column(u'vendor', PGText(length=None, convert_unicode=False)), 
Column(u'location', PGText(length=None, convert_unicode=False)), 
Column(u'status', PGText(length=None, convert_unicode=False), ForeignKey 
(u'hardware_status.status'), nullable=False, default=PassiveDefault()), Column(u'last_modified', PG2DateTime(timezone=False), nullable=False, default=PassiveDefault 
()), 
Column(u'notes', PGText(length=None, convert_unicode=False))]

this is just one of the tables in the db, it automagically maps all the others as well if I access them. SQLAlchemy also creates
associations based on foreign key contraints, but I haven't played with that yet. Here are some more docs on the SqlSoup extension:

http://www.sqlalchemy.org/trac/wiki/SqlSoup

Vim tip of the moment -- Recalling visual selection

2007-04-16T16:43:00.000-07:00

I use visual selection quite a bit in vim, it's quite handy for constraining find/replace operations and the usual select+action type of operations. One annoying thing though is that I often want to perform multiple actions to a given text block I have selected, but vim deselects the block after performing an action. This is ok if you just want to repeat the same action, you can just use . (period), but if you want to do something else, that's too bad for you.

Of course there is an easy way to get the previous selection back, I just didn't know about it until now: gv

Just use that to swap the current selection with the last one. Now it's easy to perform a series of actions on a given selection, yay!

Determining who's blocking who in Postgres

2007-02-13T16:51:00.000-08:00

If you have databases like I do with lots of concurrent queries, you can sometime run into situations where you issue a query and it just hangs there blocked. Or, more likely somebody or something issues a query and then comes calling when it doesn't seem to be doing anything.

Of course, you have the handy pg_stat_activity and pg_locks views at your disposal, but when it comes to determining exactly which queries are blocking which others and on what table, querying those alone is a tedious way to get the answer. What you really need is a query that sums it all up, in one neat and tidy bundle. Well, my friends here is such a query:


SELECT 
  bl.procpid as blocked_pid, 
  bl.usename as user, 
  bl.current_query as blocked_query, 
  bl.query_start, 
  relname as blocked_on , 
  lq.procpid as blocking_pid,
  lq.usename as user, 
  lq.current_query as blocking_query, 
  lq.query_start, 
  pgl2.mode as lock_type
FROM pg_stat_activity bl, pg_locks pgl1, 
pg_stat_activity lq, pg_locks pgl2, pg_class
WHERE bl.procpid = pgl1.pid 
AND not pgl1.granted 
AND pg_class.oid = pgl1.relation
AND pgl2.relation = pgl1.relation 
AND pgl2.granted 
AND lq.procpid = pgl2.pid;

In extended mode (\x) psql returns something along these lines for this query:

blocked_pid    | 21418
user           | sueuser
blocked_query  | insert values ('foo', 'bar', 'baz') 
  into extremely_large_table;
query_start    | 2007-02-13 15:14:06.77606-08
blocked_on     | extremely_large_table
blocking_pid   | 21417
user           | joeuser
blocking_query | delete from extremely_large_table;
query_start    | 2007-02-13 14:45:34.637675-08
lock_type      | AccessExclusiveLock

codecs.StreamRecoder example

2006-11-02T10:11:00.000-08:00

The documentation for the (very useful) Python codecs module is quite terse in places. In particular I didn't find the StreamRecoder docs especially helpful, and there are no examples. Basically what I needed was an on-the-fly conversion from a file encoded in Latin-1 (which makes a good catch-all when you aren't sure of the encoding) into UTF-8. I was using this file as input to the database, via psycopg2's copy_from() function. copy_from() requires a file-like object. To further complicate matters, the incoming file was gzip compressed, but that's not too important for this example.

Given an input file-like object (infile), here is the incantation to recode it from Latin-1 to UTF-8.

    recoder = codecs.StreamRecoder(infile, 
        codecs.getencoder('utf-8'), 
        codecs.getdecoder('utf-8'), 
        codecs.getreader('latin-1'), 
        codecs.getwriter('latin-1'))

The result recoder is a file-like object which when read returns UTF-8 encoded strings. When written it accepts UTF-8, but the data is written to the file in Latin-1--though in my case I was only reading from the file.

vimtabulous

2006-08-15T22:29:00.000-07:00

One thing I've really missed since the days of having fluxbox as my window manager was tabbed windows. It was really useful for chaps like me who have tons of windows open and poor innate visual organization skills. Well vim 7 doesn't quite bring back those glory days (not that I miss XF86Config even a trifle), but it certainly greatly improves the experience with many files open.

Tabs augment (but do not replace) the existing vim buffer model. Each tab is a separate window—actually a separate page of windows if you must know—thus each tab can, if desired, contain several windows and buffers. I'm not sure how useful that is, but it could be interesting if you like to split your windows to view multiple buffers simultaneously.

Normally, if you launch vim and specify multiple files to edit on the command line, it puts all of the files into separate buffers. Now, you can tell vim, using the -p option to put each file in it's own tab instead. The result is easier navigation between files. I've got vim aliased to vim -p now so it uses tabs by default for multiple files.

Once in vim, when you use the :e command, it still opens a new buffer in the current window. To instead open the file in a new tab, use :tabe. To navigate between tabs, you can use the mouse, or the :tabp and :tabn commands. I find those commands pretty tedious, so I've added the following key mappings to my .vimrc file:

"tab mappings 
nmap <D-1> 1gt 
nmap <D-2> 2gt 
nmap <D-3> 3gt 
nmap <D-4> 4gt 
nmap <D-5> 5gt 
nmap <D-6> 6gt 
nmap <D-7> 7gt 
nmap <D-8> 8gt 
nmap <D-9> 9gt 
nmap <D-t> :tabnew<CR> 
nmap <D-w> :tabclose<CR>
nmap <C-D-\> :tabp<CR>
nmap <C-D-]> :tabn<CR>

This gives me [cmd]+1 through [cmd]+9 to jump directly to a tab, [cmd]+t to open a fresh tab, [cmd]+w to close one and [cmd]+left and [cmd]+right to go to the previous and next tabs respectively. This makes navigating vim tabs more-or-less like iterm, consistency, it is good for ui!

Here's a link with more ideas for key mappings. Also check out the full vim documentation on tabs.

Work at home luxury

2006-07-23T07:31:00.000-07:00

Ever since we moved into our new house, working from home -- which I do about one a week -- has been less than ideal. We had this big klunky desk which we got rid of before the move. For now, we have our iMac setup on a makeshift desk, but my laptop was, well, relegated to my lap. Anyone who has a MacBook knows firsthand how impractical that is. Let's just say aluminum is an excellent conductor of heat ;^).

I looked at dozens of different desks and laptop stands online and locally. It's amazing the amount crappy, yet expensive, furniture that's out there. After far too much researching, I found this outfit CSN Office Funiture which has an excellent select of good quality, inexpensive and simple desks; three qualities that seem to occur together infrequently (I mean, check out the stock photos on the home page!). As will all "site unseen" purchases, it was a bit of a leap of faith, but I ordered what looked to be an ideal laptop desk for me (I mean, check out the cup holder! ;^).

It arrived on Friday and I put it together in about 15 minutes yesterday. It is sweet. The height adjusts pneumatically like a swivel chair via that round wheel petal at the bottom. You can adjust the angle of the laptop platform (which could double as a writing tablet or mini drafting table even). So it's basically exactly what I wanted, and I'm very happy with it. I also ordered a desk and chair from them as well, hopefully the experience will be just as good.

Delete in Screen

2006-07-20T22:45:00.000-07:00

screen is one of those essential utilities everyone (at least everyone that touches *nix) should have in their toolbelt. In true Unix tradition, the default key mapping for the [delete] key is not what desktop users would normally expect. At least on Mac OS X, it acts as a "forward delete" when this key is typically the opposite.

Anyhow after scouring around a bit I found the following incantation to add to my .screenrc that makes the world safe for [delete] in screen:

bindkey -k kD stuff \177

And there was much rejoicing.

Darwinports from source

2006-05-15T13:12:00.000-07:00

I've used DarwinPorts for some time now (admittedly as a crutch from when I used FreeBSD as my desktop), but I hadn't installed anything new since I upgraded to my MacBook. It certainly has some rough edges, but it beats the heck out of tracking down the thirty dependencies for the latest software you want to install (in my case subversion).

I prefer to put all my darwinports stuff in a separate directory tree from the rest of the system (~/opt), that way I don't have to build anything as root and I can avoid the unpleasantness of darwinports trashing system software and vice versa. Basically avoiding running anything as root is a good thing[TM] in my book. The old version of dp I had installed was not a universal binary, so I needed to upgrade it. dp can be downloaded as a disk image, but I couldn't find any way to install it in a different location. So, I just downloaded the source and tried to build it myself.

Here's where the rough edges come in. dp configured just fine, but when I ran make I got the inexplicable error:

===> making install in config
/bin/sh: line 1: cd: config: No such file or directory
make: *** [install] Error 1

Now there was indeed a config directory sitting right there, so I rooted through the makefiles to find where it was trying to change to it, thinking it had changed directories somewhere else first or something. I found what I was looking for in the Mk/dports.subdir.mk. I does your typical recursive make dance:

for subdir in config doc src portmgr; do        echo ===\> making all in $subdir;         ( cd $subdir && make DIRPRFX=$subdir/ all) || exit 1; done

I added some code above to echo the cwd and it showed it was in the right dir, but when I added cd config it blew up. I tried a few different things to no avail, then I remembered seeing something similar on our make system here at work when trying to build things on the Mac. It had something to do with the bourne shell the Mac uses, though I didn't remember what. So I figured, "let's try bash":

make SHELL=/bin/bash && sudo make install SHELL=/bin/bash

And voila! It works. I still don't know what gives with /bin/sh, but I'm off to the races...

Slony 1.1.5 subscribe set + autovacuum == deadlock

2006-04-27T01:26:00.000-07:00

Slony has been pretty helpful in our migration to postgresql 8.1. Since you can run different versions of postgres on different nodes of the same cluster, we were able to bring up 8.1 on the primary and vet it for a few days while leaving the secondary on 7.4. That way if we ran into any showstoppers (hey, even with lots of QA, production can often throw you a curve), we could do a switchover and be back on 7.4 in a few minutes.

Well, that scenario never came to pass, and now I am upgrading the secondary to 8.1 as well. After a rather slow experience reloading the primary after its upgrade I opted to rebuild the secondary rather than dumping it, reloading it and letting it sync.

Everything was going swimmingly and the largest table was copied over when suddenly it caught a deadlock and aborted the load. Bummer. Troubleshooting deadlock errors are a bit like coming on an accident scene and trying to figure out what the drivers were thinking before the crash. Only in this case the crash has been rolled back, at least one of the vehicles has disappeared and all you have is a couple of terse notes to go on.

Looking at the logs, I did see connections coming in from some other servers that I didn't expect. I decided to block all connections except local ones and those from the primary (via pg_hba.conf). Things were still trying to connect at regular intervals -- exactly what is a bit of a mystery to me, the secondary should only receive periodic connections at predictable times -- but now failing. This is when a bit of good luck came along: ironically in the form of another deadlock, this time much faster than before. I say this was good luck because I might well have waited another hour (yeah there's a lot of data) before it blew up.

I was scratching my head trying to figure out what was connecting. The pg logs were not particularly helpful even though I log a good bit of detail about the clients for this purpose. It dawned on me that it was probably the autovacuum daemon making the rounds. Now, I had just recreated this same secondary a few days ago under 7.4 with autovacuum (same version of slony) and there was no deadlock. However I know for a fact that the old autovac config was much less aggressive than I have it configured in 8.1 (especially given the key vacuum throttling available now), so I imagine it just never found anything to do, or there was some other behavioral difference in the new version that bit me.

So, long story short, I turned off autovac during the the secondary subscribe and so far so good, it hasn't fallen over. It's still chugging away on one of the big-ass tables, but here's hoping it comes around.

Moral: Make sure nothing else can mess with the secondary while it subscribes. Disable outside connections and turn off autovacuum.

In truth this whole problem is really a known bug in slony 1.1.5. I'm told that slony 1.2 grabs its locks early to block out everyone else at the get-go. Here's hoping that 1.2 hits prime-time soon, though I suspect being a feature release might make our integration longer to sort out. Nevertheless I'm happy to see so much effort going into Slony, it's a good reliable system, not without a gotcha or three, but well suited to its purpose.

vim 7.0 spellcheck

2006-04-12T21:35:00.000-07:00

Here's a primer on using the new integrated spellcheck feature in vim 7.0:

To turn on spell checking add this to your .vimrc:

setlocal spell spelllang=en_us

or simply:

set spell

which seems sufficient for me. I don't want it for all files, so I added it conditionally for text, html and README files:

autocmd BufNewFile,BufRead *.txt,*.html,README set spell

Then create a directory ~/.vim/spell for vim to hold personal word lists.

Once turned on here's a quick rundown of the basics (of course :help spell is your friend):

use ]s and [s to navigate to the next and previous misspelled words.

use z= to suggest spelling for a misspelled word

use zg to add words permanently to your personal word list. This marks it as a "good" word.

use zG to temporarily add words for just this editing session.

use zw to mark a word as misspelled.

There are other shortcuts as well:

While in insert mode, use CTRL-x s to suggest spelling for a word to the left of the cursor. You can then select a suggestion from the popup menu.

If you enter a common misspelling or mis-capitalization, try the 1z= command which replaces the word with the first spelling suggestion straight away. You can always undo it if it's not the right one.

MacBook Pro, Yo

2006-04-11T22:33:00.000-07:00

I've pretty much gotten the new MacBook setup to my liking. I must say, Apple really has it down in transferring from old to new machine. All of my stuff was seemlessly moved over from my "old" G4, barring some intel-incompatible apps. I had to install a new version of vim, but I found the 6.4.000 version would run but crash on save (ouch). So I installed the 7.0c2 beta. It doesn't crash, but there are a few bumps, specifically:

It doesn't seem to draw text quite right. Sometimes the text gets shifted over, when you cursor over it, the character it looks like you are on isn't the right one. The right one appears for a second, but then it shifts back again. Even refreshing the screen doesn't help. I guess I'll try another font, but if I have to give up ProFont I'm gonna be pissed ;^)

It doesn't seem to draw text quite the same way. I was using an anti-aliased ProFont before, but it draws too small now (though the line spacing looks right). In fairness, this may be an issue with Tiger and not vim (I was running 10.3 before)

When you use "change" commands (cw, ct, cf, etc), and you use backspace to delete from the right, the characters don't go away until you switch back to command mode. I've seen vim act like this in the past, so I suspect a change in a config option's default value

On the bright side, 7.0 adds built-in spellcheck, and the new virtualedit=onemore option removes an annoying vi artifact. onemore allows you to place the cursor one character beyond the end of the line without entering insert mode using A for example. The undo branch functionality sounds amazing, if I understand it correctly, it will mean everything you enter will stick around somewhere, regardless of intervening redos/undos. I just installed it today, so there are bound to be other handy options to figure out tomorrow. If I can get the text to draw correctly, I'll be in business.

There is still one thing that annoys me to no end with vim. Try the following: In command mode, place the cursor in the middle of a line of text. Then alternately enter insert and command mode using "i" and [esc]/[CTRL+C]. Notice how the cursor moves to the left each time. Now this doesn't happen if you enter insert mode using "a". Perhaps I'm just assbackward, but what I find is that I'm always one character off to the left from where I want to be when I leave insert mode. The onemore option does help since it allows you to place the cursor "past" the end of the line. I'm sure there's a hack to take care of this and someday I'll find it (and post it here for posterity).

update: I figured out how to get characters to go away right away when using change commands and backspace. I added this to my .vimrc and all was well again:


" Redraw immediately when replacing text within a line
set cpoptions-=$
" Erase backspaced chars immediately
set cpoptions-=v

hacman

2006-04-09T13:44:00.000-07:00

A tribute to this blog's namesake:

http://users.erols.com/mowerman/pacfile.htm

I especially dig the part about "Crazy Otto", the original (and bootleg) game that became Ms. Pacman

what's new in python 2.5

2006-04-09T13:11:00.000-07:00

Andrew Kuchling has provided us with an overview of what to expect in python 2.5. It's not done yet, but there's some good stuff there anyhow:

http://docs.python.org/dev/whatsnew/