Scaling Python for High-Load Web Sites David Shoemaker & Jamie Turner Polimetrix, Inc. At Polimetrix we conduct surveys and political polls on the web at pollingpoint.com. As such, our traffic increases significantly at election time. There are a lot of freakin' web frameworks... It's an interesting--and great--time to be a python web developer.. ... and these are just the ones with cool logos. ... in fairness, we were going to just list all the ones w/out cool logos, but this is only a 25' screen. Let's focus on methods, not web frameworks How fast is fast enough? It bears repeating: don't prematurely optimizeDon't guess, knowAlways focus on one "pinch point" at a time Premature optimization is the root of all evil. Static: 4000, dynamic (python): 400, db: 40Sustained 40 req/s is 3.4m pages/dayHundreds to low thousands of dynamic page views is usually Good Enough™1000+ pages/s (86.4M pages/day) is a "caviar problem" Scaling isn't about languages, it's about: Don't Repeat YourselfShared Nothing Flickr Killr Version 0.1 Architecture
Python Project Howto — Python Project Howto Version 0.7.1 October 05, 2009 The Goal You have a pile of Python code. You think, “this could be useful to someone else.” This guide will help you release a high-quality Python project. The Anatomy of a Python Project The goal is to address each of these parts in your project before finally releasing it. Project Hosting Choose a Host Your code needs a home on the Internet: a website where people can download your software, learn how to use it, and provide feedback. Choose a Project Name You’ll need a project name in order to register. NoteYour project name or “short name” may be taken on your host; this may affect the URL of your project’s homepage and detract from its “findability.” Sign up for an account on your chosen host. Version Control Create a Repository You will need to set up a version control system (VCS; also called a revision control or source code mangagement system) to hold your code on the Internet. Back Up Your Code $ cd /path/to/$ tar czvf googlemaps.tgz googlemaps/ That’s it.
Twisted Introduction Hey there, so I’ve been thinking about your second question. It’s a big question It basically turns into the question “how should you write software”, since most really useful programs end up getting fairly complicated. I don’t have a real answer to your question, even though I’ve written a lot of software, some of it ending up kind of big. To me there is a legitimate viewpoint that interprets most of the major innovations in software (functions, modules, classes, types, etc.) as different ways to answer this question, basically different strategies for dealing with complexity. But here are some general rules of thumb that I think most programmers would agree with: + Build your software out of small components and make the interfaces to those components as small as possible. + Try to keep the dependencies of any one component small in number. Following this strategy will make it easier to test your components and to change them over time.