I decided to use the /usr/local directory as destination and compile everything from source. Some people favor binaries. However, binary distributions are often pre-packaged and end up in some sort of installer – they could contain certain things that we dislike and so on. This is also the case with Python on Mac OS X. You can download binary distribution from the official Python website (and you are in fact encouraged to do so, if using OS X), which suffers exactly from these kind of problems. It comes with an installer and installs some stuff out of the /usr/local directory which we don’t need. It maybe useful to some of the Cocoa developers who deal with the Python code also, as it eases the installation of the PyObjC (a bridge between the Python and Objective-C) later on. But we don’t need that either. We will end up with the pure, lean and mean installation of Python and some supportive applications.
Setting Up Python Development Environment on
Mac OS X
Background
First, a bit of a background.
You run Mac OS X, one of the finest operating systems available to date. It’s not just the looks, there is a fine machinery under the hood that makes it so good. OS X is essentially a UNIX system, more specifically BSD UNIX. It contains large chunks of FreeBSD code which is a good thing, because FreeBSD is a very robust and quality product. On top of that, there is a beautiful Aqua interface for you to enjoy. For some people, this combination is the best of both worlds.
Because of the UNIX background, you get many benefits as well. For instance, you have quite a collection of useful UNIX tools available for you to use. In fact, Apple even ships Python out of the box! You may as well ask yourself why do I then need to install and set it up? – Good question. It depends. OS X is a commercial product and it takes some time from release to release and because of that Python version that is shipped with the current OS X (10.4) is a bit old. Maybe that is not an issue for you, but for some people it is. We will focus on getting and installing the newest Python version as that brings some other benefits we will mention later.
Remember that /usr/local thing from the beginning? And what’s up with that “safe heaven” talk? Well, there is a thing that is called Filesystem Hierarchy Standard, or FHS. The FHS sets up some basic conventions for use by UNIX and UNIX-like systems when mapping out a filesystem. Mac OS X breaks it at some places (as do many other UNIX variants) but most systems respect it.
The FHS defines the /usr/local as the “tertiary hierarchy for local data installed by the system administrator”, which basically means that it is the safe, standard place for you to put your own custom compiled programs there.
Using the /usr/local directory for this purpose is important for many reasons but there is one that is most critical: System Updates.
System Updates are automatic methods used by operating systems to deliver newer versions of the software to their users. This new software pieces are then installed at their usual location often with brute force, regardless of what was there before. So, for instance if you had modified or installed some newer version of some important system software, the Software Update process will overwrite it, thus rendering your changes lost. To overcome this problem, we will install all of our custom software in this safe place, the /usr/local directory.
Getting to Work
At last, the fun part (or so I hope).
Requirements
First, some prerequisites. You will need the following to get going:
- Mac OS X 10.4
- XCode 2.4 or newer (this contains the necessary compilers)
XCode is not installed by default on new Macs, but it can be obtained from the Mac OS X install DVD or from the Apple Developer Connection for free.
Strategy
As you might have guessed from the previous discussion, I decided to use the /usr/local directory as destination and compile everything from source. Some people favor binaries. However, binary distributions are often pre-packaged and end up in some sort of installer – they could contain certain things that we dislike and so on. This is also the case with Python on Mac OS X. You can download binary distribution from the official Python website (and you are in fact encouraged to do so, if using OS X), which suffers exactly from these kind of problems. It comes with an installer and installs some stuff out of the /usr/local directory which we don’t need. It maybe useful to some of the Cocoa developers who deal with the Python code also, as it eases the installation of the PyObjC (a bridge between the Python and Objective-C) later on. But we don’t need that either. We will end up with the pure, lean and mean installation of Python and some supportive applications.
Additional benefit of compiling from source is that we can look through the actual source code and audit or modify it before we actually install it.
I will focus on Python installation that is web development oriented. You will end up with a basic set of tools which you can use to build database-driven web sites powered by Python scripting language.
Let’s begin, shall we?
Using /usr/local
In order for all this to work, we will have to make some slight adjustments. For a system to see our custom Python installation, we will have to set the path to include /usr/local first. Mac OS X, like other UNIX systems, uses a “path” to determine where it should look for UNIX applications. The path is just an environment variable that is executed (if it’s set) each time you open a new Terminal window.
To set up a path, either create or edit a file .bash_login (notice the dot, it’s hidden file) in your home directory using a text editor. I recommend the following native OS X text editors: TextMate, BBEdit or TextWrangler, and the following UNIX editors: Emacs or vi(m).
To edit the file with TextMate for example, fire up the Terminal and type:
mate ~/.bash_loginThis will open the file with TextMate. Now, add the following line at the end of the file:
export PATH="/usr/local/bin:$PATH"After you save and close the file, apply the changes (from terminal) with the following command:
. ~/.bash_loginWhile we’re at it, we could just as well (using the previous method) enter the following line to make the Terminal UTF-8 aware:
export LC_CTYPE=en_US.UTF-8In general, you should be using UTF-8 anyway, so this is just a bonus. And it is even required to do for some things to work, Subversion for example has some problems if this isn’t set.
Setting Up the Working Directory
It’s nice to have a working directory where you will download all the source files and possibly revert to it later. We’ll create a directory called src in the /usr/local directory:
sudo mkdir -p /usr/local/src
sudo chgrp admin /usr/local/src
sudo chmod -R 775 /usr/local/src
cd /usr/local/srcNotice the sudo command. It means “superuser do” or “substitute user and do”. It will ask you for your password. Just enter it when asked.
We are now in this new working directory and will download and compile everything here.
Python
Finally we are all set up for the actual work. Just enter all the following commands correctly and you should be good to go.
We are starting off with Python, but to compile Python properly we will first install some prerequisites, like readline and sqlite. Technically, SQLite isn’t required, but it is necessary to compile it first, so that later on Python picks up its libraries and makes use of them. One of the new things in the newest Python 2.5 is native SQLite database driver. So, we will kill two birds with one stone ;-).
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2.tar.gz
tar -xzvf readline-5.2.tar.gz
cd readline-5.2
./configure --prefix=/usr/local
make
sudo make install
cd ..If you get an error about no acceptable C compiler, then you haven’t installed XCode.
We can now proceed with SQLite installation.
curl -O http://www.sqlite.org/sqlite-3.3.13.tar.gz
tar -xzvf sqlite-3.3.13.tar.gz
cd sqlite-3.3.13
./configure --prefix=/usr/local --with-readline-dir=/usr/local
make
sudo make install
cd ..Finally, we can download and install Python itself.
curl -O http://www.python.org/ftp/python/2.5/Python-2.5.tgz
tar -xzvf Python-2.5.tgz
cd Python-2.5
./configure --prefix=/usr/local --with-readline-dir=/usr/local --with-sqlite3=/usr/local
make
sudo make install
cd ..This should leave us with the core Python and SQLite installation. We can verify this by issuing the following commands:
python -V
sqlite3 -versionThose commands should report new version numbers we just compiled (2.5 for Python and 3.3.13 for SQLite).
Do the happy dance now!
Before we get to excited, we should also verify are they properly linked together by entering interactive Python interpreter and issuing few commands (don’t type “>>>”, these are here for illustrative purposes, because you also get them in the interpreter):
python
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.3.13'Press C-D (that’s CTRL + D) to exit the interactive Python interpreter. If your session looks like the one above, we’re all set. If you get some error about missing modules, that means something is not right. Did you follow all the steps as mentioned above?
We now have the Python and SQLite installed. The rest is up to you. Do you want to program sites in Django, CherryPy, Pylons, TurboGears, web.py etc.? Just install the web framework you are interested with. Need any additional modules, like Beautiful Soup for parsing HTML? Just go ahead and install it…
For development needs, all frameworks I tried come with a suitable development server, so you don’t need to install any web server to get started. CherryPy in addition even comes with a great production-ready WSGI web server. Also, for all your database needs, I find SQLite more then adequate while in development mode. I even find it more then enough for some live sites also. It’s great little zero-configuration database. If you have bigger needs, it’s easy to switch to some other database on the production server (you are planning to use some database abstraction layer, do you?).
For completeness sake, let’s pretend you’re going to develop sites with CherryPy as web framework, SQLite as database, SQLAlchemy as database abstraction layer (toolkit, ORM) and Mako for templates. So, we are missing CherryPy, SQLAlchemy and Mako. Let’s get them while they’re hot:
cd /usr/local/src
curl -O http://download.cherrypy.org/cherrypy/3.0.1/CherryPy-3.0.1.tar.gz
tar -xzvf CherryPy-3.0.1.tar.gz
cd CherryPy-3.0.1
sudo python setup.py install
cd ..
curl -O http://cheeseshop.python.org/packages/source/S/SQLAlchemy/SQLAlchemy-0.3.5.tar.gz
tar -xzvf SQLAlchemy-0.3.5.tar.gz
cd SQLAlchemy-0.3.5
sudo python setup.py install
cd ..
curl -O http://www.makotemplates.org/downloads/Mako-0.1.4.tar.gz
tar -xzvf Mako-0.1.4.tar.gz
cd Mako-0.1.4
sudo python setup.py install
cd ..Do the happy dance again! This same pattern applies for many other Python web frameworks and modules.
What have we just achieved? Well, we now have “invisible” Python web development environment which is clean, fast, self-contained and in the safe place to rest on. Combine it with TextMate (or any other text editor you like) and you will have some serious good times.
Again, for even more completeness sake, we will cover Subversion. Subversion is a version control system. Sounds exciting, eh? Actually, it’s a very powerful and sane thing to learn and use. But, I’m not covering it because of actual version control, but because many software projects use it, so you will sometimes have the need to checkout (download your own local copy) some projects code. For example, Django project uses it, and their development version is often better than the actual released “stable” version. So, the only way of having (and keeping up with) the development version is to use Subversion to obtain it and keep it updated. All you usually need to do in order to obtain the latest revision of some software is to issue the following command (example for Django):
svn co http://code.djangoproject.com/svn/django/trunk/ django_srcHere are the steps to download and compile Subversion:
curl -O http://subversion.tigris.org/downloads/subversion-1.4.3.tar.gz
curl -O http://subversion.tigris.org/downloads/subversion-deps-1.4.3.tar.gz
tar -xzvf subversion-1.4.3.tar.gz
tar -xzvf subversion-deps-1.4.3.tar.gz
cd subversion-1.4.3
./configure --prefix=/usr/local --with-openssl --with-ssl --with-zlib
make
sudo make install
cd ..However, even on some moderate recent computer hardware, Subversion can take a long time to compile. If that’s the case you don’t want to compile it, or you simply just use it for time to time to do some checkouts, you may prefer to download some pre-compiled binary. I know what I said about binaries before, but there is a very fine one over at Martin Ott. It’s packaged as a standard Mac OS X installer, and it installs just where it should, in /usr/local directory.
When speaking about version control, I’m more a decentralized version control person. I really like Mercurial — it’s fast, small, lightweight, but it also scales fairly well for more demanding scenarios. And guess what, it’s also written in Python. So, go ahead, install it too, and start writing those nice Python powered web sites!
That would be all from me today. While I provided the exact steps for you to follow, that doesn’t mean that you should pick the same components.
These days (coming from the Django background), I’m learning Pylons, Mako, SQLAlchemy, Elixir and a couple of other components. It makes sense currently, as Pylons is strongly built around WSGI compliance and philosophy which makes the components more reusable and should make it easier to switch to or from any other Python WSGI-centric framework in the future.
Good luck!
