Tuesday, August 17

Running Tests for Multiple Pythons

For my Planar project, I'm committed to being compatible for Python 2.6-3.1 and beyond. Since this project contains C extensions, that means that I need to be able to build and test for each Python conveniently. And yes, different Pythons do behave differently, have different compilation gripes, and sometimes even expose different bugs due to internal differences in GC, etc.

This shell script I wrote is a simple, but effective solution. It requires that all of the versions of Python you are testing are installed (of course), and also have nose installed. It is very easy to extend to more Python versions as well by modifying the for loop at the top. You run it from the top-level source directory, alongside your setup.py. I imagine other folks have come up with more elegant solutions than this, but I figured it might be useful to someone, so here it is:
# Build from scratch and run unit tests 
# from python 2.6, 2.7 and 3.1
# Then run doctests to verify doc examples
# Note: requires nose installed in each python instance


rm -rf build

for ver in 2.6 2.7 3.1; do
 echo "************"
 echo " Python $ver"
 echo "************"
 if which python${ver}; then
  python${ver} setup.py build && \
  python${ver} -m nose.core \
   -d -w build/lib.*${ver}/ --with-coverage $@ || error=1
  echo >&2 "!!! Python ${ver} not found !!!"

echo -n "Doctests... "
cd build/lib.*3.?/ && \
  python3 -m doctest ${srcdir}/doc/source/*.rst && \
  echo "OK" || error=1

exit $error

The bottom part runs doctests you might have in doc/source, which is where I keep the sphinx doc sources.


Blogger Michael Foord said...

You should look into tox.


It makes running tests with multiple versions of Python a snip.

12:26 AM  
Anonymous Anonymous said...

This is brilliant. I was recently reading about a project called 'tox' that does the same thing. I don't know what the trade-offs are though, I haven't used it.

1:53 AM  
Blogger illume said...

Hi ya,

good idea, and good practice.

I use a make file for this.


Running the tests after install is useful too. That way you catch problems with the installed version as well - which can be different based on eg, pip, easy_install, buildout, windows installers (exe, msi,zip), mac installer, fink installer, rpm, py2exe install, py2app install, the list goes on.

5:03 AM  
Blogger casey said...

I'll definitely consider tox once Python3 support for virtualenv grows up. As it is, I already have a hacked version of nose to work with py3.1.

One thing I like about my approach is that is doesn't actually install anything, although Rene's suggestion to test with all the various installers is a good one. Given infinite time and patience, I would definitely do that ;^)

10:24 AM  

Post a Comment

<< Home