Tornado with VirtualEnv and Pip Quickstart

Friendfeed’s open source Tornado web server is great, and is incredibly easy to get up-and-running. Just install tornado, write your app and run it.

At some point, however, you’d want more structure in your project and manage dependencies to ease deployment. This is where virtualenv and pip shines. For a few more steps, you can bootstrap your project and have the warm fuzzy feeling that you can easily deploy the stuff when the code is ready.

Installing virtualenv and pip

If you haven’t set up virtualenv, do so (as root):

# easy_install virtualenv

Decide where you’d put your project directory. I’ll use /path/to/myapp for now. The next step is to create a virtualenv where all your Python packages are stored. I like to use the convention of a directory called root where all dependencies are installed. I’d generally also use it as the prefix for any cmmi packages that I’d like to contain within the project.

$ cd /path/to/myapp
$ virtualenv --no-site-packages root

Activate the environment that we just created:

$ . root/bin/activate
(root)[wil@wasabi /path/to/myapp]$ 

From now on, all packages installed with easy_install will be placed in this virtualenv.

Next, we will install pip into this virtualenv:

(root)[wil@wasabi /path/to/myapp]$ easy_install pip

Once pip is installed, as long as you’ve got your virtualenv activated, anything installed with pip will also go into the right place (without your having to remember to use the -E command line argument.)

Installing Tornado

Tornado (as of the current version) needs two mandatory dependencies, i.e. pycURL and simplejson. Make sure you have the right libcURL version installed on your system (using apt-get or other mechanism) and pick the compatible pyCURL version.

(root)[wil@wasabi /path/to/myapp]$ pip install pycurl==7.16.4
(root)[wil@wasabi /path/to/myapp]$ pip install simplejson

Now we’ll install tornado proper. I chose to go with the bleeding edge and ask pip to install from the git trunk.

(root)[wil@wasabi /path/to/myapp]$ pip install -e \

Should you not want that, you can tell pip to install from the tarball URL instead (at least until tornado gets added to PyPI.)

(root)[wil@wasabi /path/to/myapp]$ pip install \

Tornado is installed!

Every now and then, it’s a good idea to save your pip dependencies by running

(root)[wil@wasabi /path/to/myapp]$ pip freeze > pip-req.txt

Start your project

What I like about this is that the project directory has all the dependencies contained within a single directory (root). This is really just my convention; I’d create a src directory where my application code lives.

(root)[wil@wasabi /path/to/myapp]$ mkdir src
(root)[wil@wasabi /path/to/myapp]$ cd src
(root)[wil@wasabi /path/to/myapp/src]$ 

Let’s test drive Tornado:

(root)[wil@wasabi /path/to/myapp/src]$ cp ../root/src/tornado/demos/helloworld/ .
(root)[wil@wasabi /path/to/myapp/src]$ python

From browser, visit your host at port 8888 to verify.

That’s it!

7 Responses to “Tornado with VirtualEnv and Pip Quickstart”

  1. Nick Says:

    Thanks for posting this! I just have one correction: it should be #egg=tornado, not #egg=Tornado. (The capitalization matters.)

  2. wil Says:

    I didn't know that, thanks for the tip! Text has been fixed.

  3. snakejuice Says:

    What's the best way (or any way) to package the resulting virtual environment and deploy it to another machine (staging, production, etc.) ?

  4. wil Says:

    The cleanest way is to recreate the virtual environment in other machines. To do that, you would want to produce a requirements file with pip, which details the exact version of all the packages you have installed in that virtual env.

    $ pip freeze > requirements.txt

    On the new machine, after you have created the empty virtualenv, and installed pip, you would just do:

    $ pip install -r requirements.txt

    There are other ways of doing it (like making your virtual env relocatable, etc.) but if you depend on modules that link to C libraries, and the machines are different, it won't work. There are also other things are likely to break.

  5. @dpn Says:

    Hey wil, thanks for the post.

    have you figured out how to get the epoll.c module to build using pip?

    when you add a requirement from git it seems to just add the source code to the virtualenv and not compile the module. this of course is not acceptable for a production deployment!

    Any advice you may have would be very helpful!

  6. wil Says:

    Hey David,

    I haven't used epoll because using the mac for dev, and freebsd for prod.

    However, pip *should* do the normal stuff even when installing from git. Maybe you're using Python 2.6 and from what I can see tornado's refuses to build epoll.c unless you're on Linux and *not* using 2.6.

    I've also found the <code>–install-option</code> argument to pip install command to be invaluable when you need to pass additional args to

  7. Matthew Says:

    When are you in South Wales re: Llanelli area? Would love to come and see you perform. Thanks.