gpars (Groovy Parallel Systems)
In my Finding Things chapter of Beautiful Code , the first complete program is a little Ruby script that reads the ongoing Apache logfile and figures out which articles have been fetched the most. It’s a classic example of the culture, born in Awk, perfected in Perl, of getting useful work done by combining regular expressions and hash tables. I want to figure out how to write an equivalent program that runs fast on modern CPUs with low clock rates but many cores; this is the Wide Finder project. ongoing · The Wide Finder Project
Parallel Processing and Multiprocessing in Python A number of Python-related libraries exist for the programming of solutions either employing multiple CPUs or multicore CPUs in a symmetric multiprocessing (SMP) or shared memory environment, or potentially huge numbers of computers in a cluster or grid environment. This page seeks to provide references to the different libraries and solutions available. Symmetric Multiprocessing Some libraries, often to preserve some similarity with more familiar concurrency models (such as Python's threading API), employ parallel processing techniques which limit their relevance to SMP-based hardware, mostly due to the usage of process creation functions such as the UNIX fork system call.
A Curious Course on Coroutines and Concurrency Copyright (C) 2009, All Rights Reserved David Beazley http://www.dabeaz.com Presented at PyCon 2009, March 25, 2009. (7/19/2009) The Python Essential Reference, 4th Edition is now available. If you like this tutorial, you'll like this edition--there is extended coverage of generators, coroutines, and other advanced Python features. Introduction This tutorial is a practical exploration of using Python coroutines (extended generators) for solving problems in data processing, event handling, and concurrent programming.