Archive for the 'python' Category

Tornado with VirtualEnv and Pip Quickstart

Friday, October 9th, 2009

Friendfeed’s open source Tornado web server is great, and is incredibly easy to get up-and-running. Just install tornado, write your app and run it.

At some point, however, you’d want more structure in your project and manage dependencies to ease deployment. This is where virtualenv and pip shines. For a few more steps, you can bootstrap your project and have the warm fuzzy feeling that you can easily deploy the stuff when the code is ready.

Installing virtualenv and pip

If you haven’t set up virtualenv, do so (as root):

# easy_install virtualenv

Decide where you’d put your project directory. I’ll use /path/to/myapp for now. The next step is to create a virtualenv where all your Python packages are stored. I like to use the convention of a directory called root where all dependencies are installed. I’d generally also use it as the prefix for any cmmi packages that I’d like to contain within the project.

$ cd /path/to/myapp
$ virtualenv --no-site-packages root

Activate the environment that we just created:

$ . root/bin/activate
(root)[wil@wasabi /path/to/myapp]$ 

From now on, all packages installed with easy_install will be placed in this virtualenv.

Next, we will install pip into this virtualenv:

(root)[wil@wasabi /path/to/myapp]$ easy_install pip

Once pip is installed, as long as you’ve got your virtualenv activated, anything installed with pip will also go into the right place (without your having to remember to use the -E command line argument.)

Installing Tornado

Tornado (as of the current version) needs two mandatory dependencies, i.e. pycURL and simplejson. Make sure you have the right libcURL version installed on your system (using apt-get or other mechanism) and pick the compatible pyCURL version.

(root)[wil@wasabi /path/to/myapp]$ pip install pycurl==7.16.4
(root)[wil@wasabi /path/to/myapp]$ pip install simplejson

Now we’ll install tornado proper. I chose to go with the bleeding edge and ask pip to install from the git trunk.

(root)[wil@wasabi /path/to/myapp]$ pip install -e \

Should you not want that, you can tell pip to install from the tarball URL instead (at least until tornado gets added to PyPI.)

(root)[wil@wasabi /path/to/myapp]$ pip install \

Tornado is installed!

Every now and then, it’s a good idea to save your pip dependencies by running

(root)[wil@wasabi /path/to/myapp]$ pip freeze > pip-req.txt

Start your project

What I like about this is that the project directory has all the dependencies contained within a single directory (root). This is really just my convention; I’d create a src directory where my application code lives.

(root)[wil@wasabi /path/to/myapp]$ mkdir src
(root)[wil@wasabi /path/to/myapp]$ cd src
(root)[wil@wasabi /path/to/myapp/src]$ 

Let’s test drive Tornado:

(root)[wil@wasabi /path/to/myapp/src]$ cp ../root/src/tornado/demos/helloworld/ .
(root)[wil@wasabi /path/to/myapp/src]$ python

From browser, visit your host at port 8888 to verify.

That’s it! with mod_python on Apache

Thursday, January 29th, 2009

Normally, I’m a Django-head, but today I wanted a quick way to write a small web app and have heard good things about so I decided to give it a try. I also wanted minimum fuss to get something up-and-running quickly, and decided to have it served by my existing Apache installation with mod_python.

In the process of setting it up, I ran into a few bumps, and would like to share them:

  • My setup is: Python 2.5.2, 0.31, Apache 2.2.8 with mod_python 3.3.1
  • The section on configuring mod_python on the installation page doesn’t work for this version. In particular, it tells you to do this:
    main = web.wsgifunc(web.webpyfunc(urls, globals()))

    which seems to be the API in 0.2. Using it on 0.3 will land you a nice AttributeError: 'module' object has no attribute 'wsgifunc'. Instead, do this:

    app = web.application(urls, globals(), autoreload=False)
    main = app.wsgifunc()

    It’s also worth noting that the version of I’m using is revision 106. With this version, there is no need to place it in the wsgiref package directory since it no longer depend on wsgiref.

  • My Apache config is:
      Alias /path /vhosts/
      <Directory /vhosts/>
        Allow from all
        <IfModule python_module>
          SetHandler python-program
          PythonHandler modpython_gateway::handler
          PythonOption wsgi.application codep::main
          PythonOption SCRIPT_NAME /path
  • Notice that I specified the autoreload argument, because the other problem I ran into was that complained the following:
    Traceback (most recent call last):
      File "/usr/local/lib/python2.5/site-packages/", line 209, in process
        return p(lambda: process(processors))
      File "/usr/local/lib/python2.5/site-packages/", line 536, in processor
      File "/usr/local/lib/python2.5/site-packages/", line 73, in reload_mapping
        mod = __import__(module_name)
    ImportError: No module named _mp_4ba3724007eff4c08a72054c4da2c222

    I’m not sure the reason for this, but definitely has something to do with—s auto-reload feature, and the fact that it is running under mod_python so the module name seems to be mangled.

  • The other thing is that if you ever run into the problem of getting a file save dialog when requesting the root URL of your app, with the content type of httpd/unix-directory, but not other URLs, this could be due to either Apache or mod_python passing the hint on the request, so if you don’t respond with a Content-Type header (using web.header('Content-Type', 'text/html'), that hint gets copied over to the response.

Python 3.0: Text vs. Data Instead Of Unicode vs. 8-bit

Monday, December 15th, 2008

Python 3.0 (Py3K) is out. I’m with Sam Ruby — this seemingly simple change of paradigm from “Unicode vs. 8-bit” to “Text vs. Data” is a breath of fresh air.

What’s inconsistent in this new version though is that the new bytes type still contains many of the methods with text semantics that should only make sense as string methods: e.g. capitalize() and islower(). I suspect these are provided as convenience methods, which is fine. But one would imagine that these byte methods will work by decoding the bytes using the default encoding of your locale, then performing the operations on the resulting string. As it turns out from my trials, it seems to assume that your bytes are encoded in Latin1:

>>> greek_beta = "Β" # This is the uppercase greek letter "beta", not regular B.
>>> greek_beta.isupper()
>>> greek_beta.islower()
>>> greek_beta.lower()
>>> greek_beta_bytes = greek_beta.encode('iso8859-7')
>>> greek_beta_bytes
>>> greek_beta_bytes.isupper()
>>> greek_beta_bytes.islower()
>>> greek_beta_bytes.lower()
>>> greek_beta_bytes.upper()

This is definitely a gotcha that may lead to hard to find bugs. So, it is best to avoid using those methods on bytes objects.

Otherwise, this change is definitely more “correct” in that Py3K forces you to know the type of your variables earlier or at the interfaces (to the outside world) so errors like these are less likely to sneak up from your back. For example, you can no longer use the + (sequence concatenation operator) to mix text and data. Whereas in Python 2.x, you can do:

>>> name = 'Wil'
>>> greet = lambda n: u'Hello ' + n
>>> greet(name)
u'Hello Wil'
>>> name = u'François'.encode('utf-8')
>>> name
>>> greet(name)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <lambda>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)

What went wrong was that you’re relying on Python’s automatic Unicode conversion, which uses the standard ASCII codec (which is a good thing) to promote your str to unicode.

In Python 3.x, you will be greeted by the TypeError: Can't convert 'bytes' object to str implicitly message if you tried to pass a bytes object to the function. This will happen on any bytes object, so the error is easier to catch.

In this new version, the unicodedata module is upgraded to Unicode version 5.1.0.

Now, I’m not ready to run production code on Py3K yet but it would be nice if Django (my favourite Python-based web framework) can run on it. It looks like Martin von Löwis has started the porting.