How Yipit Deploys Django

This post is republished from the Yipit Django Blog

If you’re managing your own servers, and you don’t use a tool like Chef, you’re crazy. It’s just that simple.

We’ve been using Chef here at Yipit for about 6 months, and when I think about provisioning a new server with our old load book, I cringe.

There were some pretty high upfront costs to learning Chef, especially since no one here had any real Ruby experience (Chef is written in Ruby), but the time we invested into getting set up with Chef has been 100% worth it.

I’m hoping this post will help people get up and running with Django and Chef as quickly as possible. Opscode has their own django-quick-start repository, but I think it’s too complex if you’re not familiar with Chef. This tutorial will cover:

  • Getting set up with Opscode, a hosted chef-server
  • Installing Ruby, Chef, and some knife plugins
  • Setting up your chef-repo and installing a Django/github quickstart cookbook
  • Using a Python script to deploy to ec2

If you don’t understand what any of those things mean, Opscode maintains a pretty good wiki.

Before We Start

This tutorial makes the following assumptions about your systems:

  • Locally, you run a Mac.
  • Remotely, you want to use Ubuntu on AWS
  • Your AWS account has security groups named “web” and “linux” that allow access on ports 22 and 80
  • The private key for the AWS ssh key you define in add_server.py (we’ll get to that later) is located in ~/.ssh/ on your local machine
  • Your Django app contains a pip requirements file at conf/external_apps.txt (and that file contains gunicorn)
  • Your Django app contains a gunicorn configuration file at conf/gunicorn/gunicorn.conf
  • You’re interested in learning about Chef and managing your own servers. Seriously. If you’re just looking for a simple deployment solution, use Heroku. If you want to manage and automate your own infrastructure, use Chef

If you’re not familiar with AWS security groups and key pairs, this tutorial should help. I’ve also provided a simple “Hello World” Django application that I use as an example.

Get the Gems

I like using homebrew for everything, so even though your mac comes with ruby…

brew install ruby
gem install -n /usr/local/bin/ chef
gem install knife-ec2
gem install knife-github-cookbooks

Set up a chef-repo

We’re going to be adding some python scripts to help us manage our servers, so I like to set up my chef-repo in a virtualenv. From wherever you keep your virtualenvs…

virtualenv chef-env
cd chef-env
git clone https://github.com/opscode/chef-repo.git
cd chef-repo

Now you have the barebones Chef repository distributed by opscode.

Get started with opscode and your knife.rb file

Next you’ll want to get setup with Opscode to manage your chef server. You can create an account at opscode.com.

After you’ve signed up, download your private key. You can download it by clicking on your username on the top right of the console, and then following the link to “get private key”.

Next you’ll need to set up an organization and download the organization validator key. You’ll also want to to download the knife.rb file that opscode will generate for you.

Lastly, throw it all in a .chef directory in your chef-reop.

You should also append this code to your knife.rb:

This changes the amount of time knife will wait for ec2 to spawn a new server for you. I’ve found that sometimes the default wait time isn’t long enough for ebs-backed instances.

Setting up your project

Next we need to set up chef to actually do something. Let’s start by installing a Django quick start cookbook I put together. This cookbook will install a django application behind nginx and gunicorn, which will be managed by supervisor. You should read through the commented code for an explanation of how it works.

knife cookbook github install Yipit/djangoquickstart-cookbook

This cookbook has a dependency on the python cookbook, so we’ll want to install that too.

knife cookbook site install python

Note that “site” version of the install command will handle dependencies automatically, while the github command will not. When you use “site”, however, you’re limited to cookbooks distributed by opscode.

The next thing we’ll want to do is define a role. In your chef-repo, put something like this in roles/web.rb:

Get Some Python In there

Knife has a pretty good plugin system, but it requires you to write Ruby. Writing Ruby is worth it in the cookbooks because it’s the only option, but for helper scripts, I just use python. Aside from being more comfortable with the language, familiarity with the libraries is a huge plus. Here’s a script similar to the one we use to add servers at Yipit:

Fill in your AWS keys and put this code into a file called add_server.py. This script requires boto and pychef, so from within your virtualenv:

pip install boto pychef

I keep this file in right in the chef repo.

The last thing we need to do before we get started is upload our roles and cookbooks to the chef server.

knife cookbook upload --all
knife role from file roles/web.rb

Now, if you run the add_server.py script, you should be able to bootstrap your own web server running your django application through gunicorn behind nginx. The add server script does restart the machine, so you may need to wait a minute or two.

Hopefully you can visit your new web server and see your application.

New and Improved Two Column Layout

Two column layouts are all the rage on the internet right now. Twitter did it. Hashable did it.  Even facebook thought about doing it. It’s a great way to consume content, but most current implementations have some serious limitations:

  1. There is a clear and predefined relationship between the two columns - One columns is the parent, and the other is the child. This relationship limits the amount of content available in the lesser of the two panels. Most implementations limit the height of the child column to the height of the browser window and use an overflow for additional content.
  2. Navigation is inconsistant – The page will react differently depending on which browser element is in focus. Because the child column usually uses overflow for extra content, scrolling up or down does not guarantee getting any additional content located in that direction in both columns.

I’ve built a proof of concept demo of a two column layout with no hierarchy between the columns.

You can check it out here.

Easy Ajax Forms with Fancybox

Note: The fancybox jQuery plugin can be found here.

Last week we rolled out a new design for yipit, which I’m really excited about. It looks awesome. Sam Brown did an amazing job. In a few different places we decided to use modal forms that would submit via Ajax. I think technique provides great UX, but  if the forms require server-side validation,  displaying the appropriate HTML can be a bit of a headache.

Optimized Django M2M “in” Queries

ManyToManyFields in Django have a big inefficiency when running a query to get objects that share a ManyToMany relation with another object. Consider these three models:

Does Social Media Make Us Stupider?

There is a lot of debate as to whether the internet makes us smarter or dumber. I’ve always argued that the internet increases our intelligence. It certainly makes us more knowledgeable, and while there is a certain level of intelligence necessary to grasp knowledge, it is our knowledge-base that allows us to act intelligently. It’s a classic Chicken-Egg Situation, but if I had to choose, I’d say knowledge is the more significant propellent of human achievement. The real difficulty of this debate lies in the fact that words like intelligence and knowledge can be hard to define, let alone measure.

Email Isn’t Going Anywhere

Yesterday Sheryl Sandberg proclaimed that “email is probably going away” because teenagers don’t it as their primary mode of social communication. It’s not. Here’s why: