Using Amazon S3 with Django
Entry updated May 15, 2008 at 10:01 a.m.
UPDATE: A commenter from this entry's submission on Reddit suggested setting up the CNAME to the S3 servers based on the bucket for your unique domain. By doing this, it will appear that the static files are hosted on your own servers instead of Amazon.
Changing Web hosts from Dreamhost for this and my other Django apps was a must, but using Amazon's Web Service was more of an experiment really. Fortunately it's a cheap one: I owe Amazon eight cents for June.
Follow the directions
I'll paraphrase for readers of this blog:
First, sign up for an Amazon S3 account. There is no charge until you actually use the service. Bonus: You can use your existing Amazon username and password like other purchases.
Next you'll need to create a "bucket" for your files. For instance, I called my bucket for this site "patrickbeeson" and the one for Knox'd "knoxd." You'll need to download the free S3 Python bindings, and put s3.py on your PYTHONPATH. Fire up your Python interpretor and enter the following:
import S3 conn = S3.AWSAuthConnection('your access key', 'your secret key') conn.create_bucket('mybucketname')
Download Adrian's Python script that uploads your media and upload it to your server. I put a copy in each Django project. Shell into your server, and use the following commands:
$ cd /directory/with/media/files/ find | python /path/to/update_s3.py
Change your media paths in your project settings and templates to your new S3 account (http://bucket_name.s3.amazonaws.com), and you're done.
Managing your files
Updating your files is as easy as rerunning the Python script. Each time you do this it will skip over files that already exist to avoid duplication.
I wanted a little more graphical interface for managing the files however. So I downloaded an application for Apple called S3 Browser. It works a lot like an FTP client and allows you to manage files for multiple buckets (or add new buckets).
There are a number of excellent open-source applications for this, including a nifty plugin for FireFox.
Cost and savings
My use of Amazon S3 isn't a great example of the service's cost-savings. I just wanted to experiment with the service out of pure curiosity.
Folks with more robust or demanding Web apps -- apps that ingest photos, video and other large files -- would do well to use the service as it scales to meet most any demand. And at 15 cents a gig per month it's practically free.