2011/10/09

Wrestling the Python on Snow Leopard

And it’s all because of Django & Unicode

So I have this error in my Django page when trying to get some values via a many-to-many relathionship:

filter() keywords must be strings

in

/[...]/VIRTUAL_ENV/lib/python2.6/site-packages/django/db/models/fields/related.py in get_query_set line 497



def get_query_set(self):
db = self._db or router.db_for_read(self.instance.class,
instance=self.instance)
return superclass.get_query_set(self)
.using(db)
._next_is_sticky()
.filter(**(self.core_filters)) #<< Here

Which took me while to figure as being a Unicode handling problem between Django and Python (2.6).

Here are the initial conditions:

  • Snow Leopard shipped with python 2.6.1.
  • I use homebrew for installing extra software
  • The web app I am working on depends on (amongst many other):
  • Django 1.3
  • Cairo

Some other facts I have to work with:

  1. Django 1.3.1 does not fix the problem.
  2. Django 1.4 is still pre-alpha.
  3. We use a library which has issues with python 2.7 so it’s out.
  4. The official python 2.6.6 OSX binary is i386 only.
  5. There is no homebrew package for python2.6.* anymore.
  6. There is no pip package for Cairo (py2cairo).

Django 1.3.1: no luck

So I tried to use Django 1.3.1 but it does not include the fix for this particular problem.

Django 1.4 pre-alpha : it says it all on the tin

No more luck with a checkout of Django 1.4 sources. Is still too much on the bleeding part of the bleeding edge (renamed packages, disappearing classes … )

As newer versions of Django don’t fix the problem let’s try newer versions of python.

Python 2.6.6: too much pain

I installed Python 2.6.6 from the official OSX binary distribution. I Set up a virtual environement, but there were some problem with compiling libraries and re-using previously compiled library. It mostly revolved around architecture mismatch between i386 and x86_64:

Error was: 


dlopen([…]/ENV_py266/lib/python2.6/site-packages/cairo/_cairo.so,2):
 no suitable image found.

Did find:
 […]ENV_py266/lib/python2.6/site-packages/cairo/_cairo.so:
 mach-o, but wrong architecture

I would have to recompile pycairo with a different architecture and possibly other libraries. So I decided to skip it and got to the next option.

Python 2.6.7:

So I then had to try and install a newer version of python, from source. I might as well get the last: Python 2.6.7. Googling around I found an article about recovering the old Homebrew formula for python from github when the tracked python was 2.6.

The formula/repository seems to have moved from when the article was written (curl just gets a 301 when fetching the article’s URL), so I went to the page to copy the final url from my browser. I modified the python source url and md5 to match that of the 2.6.7 version. I renamed the Formula and class to python2.6.7 and Python2.6.7 respectively before moving them to their destination. Building and installing was as easy as:

brew install python2.6.7 --universal
sudo brew link python2.6.7

I then created a new virtualenv using that new python. I copied my python 2.6.1 site-packages from my previous virtualenv over as well as the django project itself. To cleanup afterwards I ran the command bellow in the virtual env’s root.

find . -name '*.pyc' -exec rm '{}' ';'

Finally I could start the django project !

Alas, as soon as I tried to get a page I got

Fatal Python error: Interpreter not initialized (version mismatch?)`

I narrowed it down to using a py2cairo compiled for the wrong python.

otool -L _cairo.so
...
=> /System/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)
...

To compile py2cairo with the new python, this stackoverflow question about installing py2cairo on mac os x helped me get started but a bit more research into waf and actually reading the ld man was needed until everything could work together.

After downloading the [py2cairo library source][py2cairo] from its homepage I evolved the following script:

########################
# my messy part: Created by adding and tweaking until it works. 
# Some of it might be unncesary.


python waf clean



export PATH=/usr/local/Cellar/python2.6.7/2.6.7/bin:$PATH
export PYTHONPATH=/usr/local/Cellar/python2.6.7/2.6.7
export LD_LIBRARY_PATH=/usr/local/Cellar/python2.6.7/2.6.7:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/Cellar/python2.6.7/2.6.7/lib:$LD_LIBRARY_PATH
export LINKFLAGS=‘-search_dylibs_first -L /usr/local/Cellar/python2.6.7/2.6.7/lib/’



from stackoverflow



http://stackoverflow.com/



questions/6886578/how-to-install-pycairo-1–10-on-mac-osx-with-default-python



export ARCHFLAGS=‘-arch x86_64’
export CC=/usr/bin/gcc
export PKG_CONFIG_PATH=/usr/local/Cellar/cairo/1.10.2/lib/pkgconfig/



python waf configure
python waf build



#To check against which python the lib was linked
otool -L build_directory/src/_cairo.so
########################

(gist on gihtub)

Once that worked, the output became:

otool -L build_directory/src/_cairo.so
...
=> /usr/local/Cellar/python2.6.7/2.6.7/lib/libpython2.6.dylib (compatibility version 2.6.0, current version 2.6.0)
...

I installed it in the virtal environment:

cp build_directory/src/_cairo.so $PY267/lib/python2.6/cairo/
echo "from _cairo import *" >> $PY267/lib/python2.6/cairo/__init__.py

And then the Django site finally worked :-)

 
Archives / RSS
— ~ —