And it's all because of Django & Unicode
So I have this error in my Django page when trying to get some values via a many-to-many relathionship:
filter() keywords must be strings
in
/[...]/VIRTUAL_ENV/lib/python2.6/site-packages/django/db/models/fields/related.py
in get_query_set line 497
def get_query_set(self):
db = self._db or router.db_for_read(self.instance.__class__,
instance=self.instance)
return superclass.get_query_set(self)
.using(db)
._next_is_sticky()
.filter(**(self.core_filters)) #<< Here
Which took me while to figure as being a Unicode handling problem between Django and Python (2.6).
Here are the initial conditions:
- Snow Leopard shipped with python 2.6.1.
- I use homebrew for installing extra software
- The web app I am working on depends on (amongst many other):
- Django 1.3
- Cairo
Some other facts I have to work with:
- Django 1.3.1 does not fix the problem.
- Django 1.4 is still pre-alpha.
- We use a library which has issues with python 2.7 so it's out.
- The official python 2.6.6 OSX binary is i386 only.
- There is no homebrew package for python2.6.* anymore.
- There is no pip package for Cairo (py2cairo).
Django 1.3.1: no luck
So I tried to use Django 1.3.1 but it does not include the fix for this particular problem.
Django 1.4 pre-alpha : it says it all on the tin
No more luck with a checkout of Django 1.4 sources. Is still too much on the bleeding part of the bleeding edge (renamed packages, disappearing classes ... )
As newer versions of Django don't fix the problem let's try newer versions of python.
Python 2.6.6: too much pain
I installed Python 2.6.6 from the official OSX binary distribution. I Set
up a virtual environement, but there were some problem with compiling
libraries and re-using previously compiled library. It mostly revolved
around architecture mismatch between i386 and x86_64:
Error was:
dlopen([...]/ENV_py266/lib/python2.6/site-packages/cairo/_cairo.so,2):
no suitable image found.
Did find:
[...]ENV_py266/lib/python2.6/site-packages/cairo/_cairo.so:
mach-o, but wrong architecture
I would have to recompile pycairo with a different architecture and possibly other libraries. So I decided to skip it and got to the next option.
Python 2.6.7:
So I then had to try and install a newer version of python, from source. I might as well get the last: Python 2.6.7. Googling around I found an article about recovering the old Homebrew formula for python from github when the tracked python was 2.6.
The formula/repository seems to have moved from when the article was written (curl just gets a 301 when fetching the article's URL), so I went to the page to copy the final url from my browser. I modified the python source url and md5 to match that of the 2.6.7 version. I renamed the Formula and class to python2.6.7 and Python2.6.7 respectively before moving them to their destination. Building and installing was as easy as:
brew install python2.6.7 --universal
sudo brew link python2.6.7
I then created a new virtualenv using that new python. I copied my python 2.6.1 site-packages from my previous virtualenv over as well as the django project itself. To cleanup afterwards I ran the command bellow in the virtual env's root.
find . -name '*.pyc' -exec rm '{}' ';'
Finally I could start the django project !
Alas, as soon as I tried to get a page I got
Fatal Python error: Interpreter not initialized (version mismatch?)`
I narrowed it down to using a py2cairo compiled for the wrong python.
otool -L _cairo.so
...
=> /System/Library/Frameworks/Python.framework/Versions/2.6/Python
(compatibility version 2.6.0, current version 2.6.1)
...
To compile py2cairo with the new python, this stackoverflow question
about installing py2cairo on mac os x helped me get started but a bit
more research into waf and actually reading the ld man was
needed until everything could work together.
After downloading the [py2cairo library source][py2cairo] from its homepage I evolved the following script:
######################## # my messy part: Created by adding and tweaking until it works. # Some of it might be unncesary. python waf clean export PATH=/usr/local/Cellar/python2.6.7/2.6.7/bin:$PATH export PYTHONPATH=/usr/local/Cellar/python2.6.7/2.6.7 export LD_LIBRARY_PATH=/usr/local/Cellar/python2.6.7/2.6.7:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/usr/local/Cellar/python2.6.7/2.6.7/lib:$LD_LIBRARY_PATH export LINKFLAGS='-search_dylibs_first -L /usr/local/Cellar/python2.6.7/2.6.7/lib/' # from stackoverflow # http://stackoverflow.com/ # questions/6886578/how-to-install-pycairo-1-10-on-mac-osx-with-default-python export ARCHFLAGS='-arch x86_64' export CC=/usr/bin/gcc export PKG_CONFIG_PATH=/usr/local/Cellar/cairo/1.10.2/lib/pkgconfig/ python waf configure python waf build #To check against which python the lib was linked otool -L build_directory/src/_cairo.so ########################
Once that worked, the output became:
otool -L build_directory/src/_cairo.so
...
=> /usr/local/Cellar/python2.6.7/2.6.7/lib/libpython2.6.dylib
(compatibility version 2.6.0, current version 2.6.0)
...
I installed it in the virtal environment:
cp build_directory/src/_cairo.so $PY267/lib/python2.6/cairo/
echo "from _cairo import *" >> $PY267/lib/python2.6/cairo/__init__.py
And then the Django site finally worked :-)
