A number of Python-related libraries exist for the programming of solutions either employing multiple CPUs or multicore CPUs in a symmetric multiprocessing (SMP) or shared memory environment, or potentially huge numbers of computers in a cluster or grid environment. This page seeks to provide references to the different libraries and solutions available. Symmetric Multiprocessing Some libraries, often to preserve some similarity with more familiar concurrency models (such as Python's threading API), employ parallel processing techniques which limit their relevance to SMP-based hardware, mostly due to the usage of process creation functions such as the UNIX fork system call. dispy - Python module for distributing computations (functions or programs) computation processors (SMP or even distributed over network) for parallel execution. Advantages of such approaches include convenient process creation and the ability to share resources. Cluster Computing Cloud Computing Grid Computing
Java 7 concurrencyHardware trends drive programming idioms Languages, libraries, and frameworks shape the way we write programs. Even though Alonzo Church showed in 1934 that all the known computational frameworks were equivalent in the set of programs they could represent, the set of programs that real programmers actually write is shaped by the idioms that the programming model — driven by languages, libraries, and frameworks — makes easy to express. In turn, the dominant hardware platforms of the day shape the way we create languages, libraries, and frameworks. The Java language has had support for threads and concurrency from day 1; the language includes synchronization primitives such as synchronized and volatile, and the class library includes classes such as Thread. Going forward, the hardware trend is clear; Moore's Law will not be delivering higher clock rates, but instead delivering more cores per chip. Exposing finer-grained parallelism Divide and conquer Listing 1. Fork-join Listing 2. Table 1.
Patterns for Concurrent, Parallel, and Distributed SystemsDouglas C. Schmidt, ``Wrapper Facade: A Structural Pattern for Encapsulating Functions within Classes,'' C++ Report, SIGS, Vol. 11, No 2, February, 1999. This paper describes the Wrapper Facade pattern. The intent of this pattern is to encapsulate low-level, stand-alone functions with object-oriented (OO) class interfaces. Common examples of the Wrapper Facade pattern are C++ wrappers for native OS C APIs, such as sockets or pthreads. Programming directly to these native OS C APIs makes networking applications verbose, non-robust, non-portable, and hard to maintain.10 Ideaspublished in 1991 Prelude "Show me." John held up the chalk, holding it by the top, the bottom pointed at his feet. Three months earlier, in my first quarter as graduate student at Stanford, in the Lab at the top of the hill, just before a volleyball game, I asked John McCarthy - the John McCarthy - whether I could have an office at the Lab and be supported by it. "Sure," if I TAed 206, the Lisp course. Some call his classes "Uncle John’s Mystery Hour," in which John McCarthy can and will lecture on the last thing he thought of before rushing late through the door and down the stairs to the front of the lecture hall. But this class started out like anything but a mystery hour: John was reviewing the answers to the midterm. I can hear the worried thoughts behind me: "Er, but multiplying two polynomials seems so easy. John: "It turns out you need to pass global information, and the control structure is not regular. So now you’re thinking that the problem was stated funny. "Nah. Introduction
GHC/Data Parallel Haskell1 Data Parallel Haskell Searching for Parallel Haskell? DPH is a fantastic effort, but it's not the only way to do parallelism in Haskell. Try the Parallel Haskell portal for a more general view. Data Parallel Haskell is the codename for an extension to the Glasgow Haskell Compiler and its libraries to support nested data parallelism with a focus to utilise multicore CPUs. This is the performance of a dot product of two vectors of 10 million doubles each using Data Parallel Haskell. 1.1 Project status Data Parallel Haskell (DPH) is available as an add-on for GHC 7.4 in the form of a few separate cabal package. The current implementation should work well for code with nested parallelism, where the depth of nesting is statically fixed or no user-defined nested-parallel datatypes are used. DPH focuses on irregular data parallelism. Note: This page describes version 0.6.* of the DPH libraries. Disclaimer: Data Parallel Haskell is very much work in progress. 1.2 Where to get it 1.3 Overview