Python-jrpc by alex-sherman A Python remote procedure call framework that uses JSON RPC v2.0 Project maintained by alex-shermanHosted on GitHub Pages — Theme by mattgraham pip install python-jrpc Amazon S3 — Boto 3 Docs 1.3.1 documentation Boto 2.x contains a number of customizations to make working with Amazon S3 buckets and keys easy. Boto 3 exposes these same objects through its resources interface in a unified and consistent way. Creating the Connection Boto 3 has both low-level clients and higher-level resources. For Amazon S3, the higher-level resources are the most similar to Boto 2.x's s3 module: # Boto 2.ximport botos3_connection = boto.connect_s3() # Boto 3import boto3s3 = boto3.resource('s3')
Multiprocessing and Memory Introduction This is a quick and dirty tutorial about a hack that we used to fit our data into the constraints of our memory. When working with some clients, you might find that their database is simply a repository of csv or excel files and you'll simply have to make do; often having to complete your work without updating their data-warehouse. Most of the time these files would be better served being stored in some simple DB framework, but time might not allow for that. This method came about from having contraints set on our time, our machine, and our scope.
Your Development Environment Text Editors Just about anything that can edit plain text will work for writing Python code, however, using a more powerful editor may make your life a bit easier. Vim Vim is a text editor which uses keyboard shortcuts for editing instead of menus or icons. There are a couple of plugins and settings for the Vim editor to aid Python development. Popular Python recipes Welcome, guest | Sign In | My Account | Store | Cart ActiveState Code » Recipes Add a Recipe Languages Tags Authors Sets Learn Python in 10 minutes NOTE: If you would like some Python development done, my company, Stochastic Technologies, is available for consulting. This tutorial is available as a short ebook. The e-book features extra content from follow-up posts on various Python best practices, all in a convenient, self-contained format.
Elastic MapReduce Quickstart — mrjob v0.5.2 documentation Running an EMR Job Running a job on EMR is just like running it locally or on your own Hadoop cluster, with the following changes: The job and related files are uploaded to S3 before being runThe job is run on EMR (of course)Output is written to S3 before mrjob streams it to stdout locallyThe Hadoop version is specified by the EMR AMI version The Python Standard Library — Python v2.7.1 documentation While The Python Language Reference describes the exact syntax and semantics of the Python language, this library reference manual describes the standard library that is distributed with Python. It also describes some of the optional components that are commonly included in Python distributions. Python’s standard library is very extensive, offering a wide range of facilities as indicated by the long table of contents listed below. The library contains built-in modules (written in C) that provide access to system functionality such as file I/O that would otherwise be inaccessible to Python programmers, as well as modules written in Python that provide standardized solutions for many problems that occur in everyday programming.