background preloader


Facebook Twitter

A Beginner’s Guide to Optimizing Pandas Code for Speed. 28 Jupyter Notebook tips, tricks and shortcuts. Jupyter Notebook Jupyter notebook, formerly known as the IPython notebook, is a flexible tool that helps you create readable analyses, as you can keep code, images, comments, formulae and plots together. In this post, we’ve collected some of the top Jupyter notebook tips to quickly turn you into a Jupyter power user! (This post is based on a post that originally appeared on Alex Rogozhnikov’s blog, ‘Brilliantly Wrong’. We have expanded the post and will continue to do so over time — if you have a suggestion please let us know. Thanks to Alex for graciously letting us republish his work here.) Jupyter is quite extensible, supports many programming languages and is easily hosted on your computer or on almost any server — you only need to have ssh or http access.

The Jupyter interface. Project Jupyter was born out of the IPython project as the project evolved to become a notebook that could support multiple languages – hence its historical name as the IPython notebook. Level up your skills! ? ! Welcome to PyGMO — PyGMO 1.1.7dev documentation. PyGMO (the Python Parallel Global Multiobjective Optimizer) is a scientific library providing a large number of optimisation problems and algorithms under the same powerful parallelization abstraction built around the generalized island-model paradigm.

What this means to the user is that the available algorithms are all automatically parallelized (asynchronously, coarse-grained approach) thus making efficient use of the underlying multicore architecture. The user can also program his own solvers ... they also will be parallelized by PyGMO!! PyGMO’s implementation of the generalized migration operator allows the user to easily define “migration paths” (topologies) between a large number of “islands” (CPU cores). Many complex-networks topologies (Hypercube, Ring, Barabasi-Albert, Watts-Strogatz, Erdos-Renyi, etc.) are built-in and may be used to define the migration pathways of good solutions among islands. Custom topologies are also possible. Using C++ in Cython — Cython 0.25.2 documentation. A simple Tutorial An example C++ API Here is a tiny C++ API which we will use as an example throughout this document.

Let’s assume it will be in a header file called Rectangle.h: and the implementation in the file called Rectangle.cpp: This is pretty dumb, but should suffice to demonstrate the steps involved. Specify C++ language in The best way to build Cython code from scripts is the cythonize() function. From distutils.core import setupfrom Cython.Build import cythonize setup(ext_modules = cythonize( "rect.pyx", # our Cython source sources=["Rectangle.cpp"], # additional source file(s) language="c++", # generate C++ code )) Note that the language option has no effect on user provided Extension objects that are passed into cythonize().

The cythonize() function in Cython versions up to 0.21 does not recognize the language option and it needs to be specified as an option to an Extension that describes your extension and that is then handled by cythonize() as follows: Cython/Python/C++ - Inheritance: Passing Derived Class as Argument to Function expecting base class. WrappingSetOfCppClasses · cython/cython Wiki. Wrapping a set of C++ classes Overview This page demonstrates how to wrap a more complicated pair of C++ classes than the basic example. We have two classes, one of which inherits from the other, and both constructors can take a pointer to an instance of the base class to act as a parent to the new instance.

As with any C++ wrapping, Cython 0.13 (when C++ support was added) or later is required. The C++ code The basic C++ API we want to wrap is given below. Note that the constructors all take a pointer to an instance of the base class. For the purposes of the example, we will use the following implementation which prints out various bits of information. Cython definition The first step is to create a Cython definitions file, classes.pxd, to tell Cython what the C++ API looks like: cdef extern from "classes.h": cdef cppclass BaseClass: BaseClass(BaseClass *parent) cdef cppclass MainClass: MainClass(BaseClass *parent) void accumulate(int new_number) int get_total() Wrapping the base class.

A Dramatic Tour through Python’s Data Visualization Landscape (including ggplot and Altair) – Regress to Impress. Why Even Try, Man? I recently came upon Brian Granger and Jake VanderPlas’s Altair, a promising young visualization library. Altair seems well-suited to addressing Python’s ggplot envy, and its tie-in with JavaScript’s Vega-Lite grammar means that as the latter develops new functionality (e.g., tooltips and zooming), Altair benefits — seemingly for free! Indeed, I was so impressed by Altair that the original thesis of my post was going to be: “Yo, use Altair.” But then I began ruminating on my own Pythonic visualization habits, and — in a painful moment of self-reflection — realized I’m all over the place: I use a hodgepodge of tools and disjointed techniques depending on the task at hand (usually whichever library I first used to accomplish that task1).

This is no good. Thus, I’m using my discovery of Altair as an opportunity to step back — to investigate how Python’s statistical visualization options hang together. How’s This Gonna Go? First, let’s welcome our friends2: matplotlib pandas. Piping results with IPython parallel. Pipe and follow are stored on edges. If there is an edge connecting two nodes, if nothing else is specified, it will be a time dependency (after), so the downstream task will run after the upstream task, on any engine. If follow is True, the downstream task will run on the same engine as the upstream task. If pipe is True, the result of the upstream task will be passed as an argument to the downstream task.

If it is a follow dependency, the result will be passed through memory without any serialization. If follow is False and pipe is True, the result of the upstream task will be fetched from the Hub's database. Unit Testing with Python. Mpi4py parallel IO example | Chris Butler's blog. For about 9 months I have been running python jobs in parallel using mpi4py and NumPy. I had to write a new algorithm with MPI so I decided to do the IO in parallel. Below is a small example of reading data in parallel. Mpi4py is lacking examples. It is not pretty, however, it does work. import mpi4py.MPI as MPIimport numpy as npclass Particle_parallel(): """ Particle_parallel - distributed reading of x-y-z coordinates.

Like this: Like Loading...