background preloader

Development

Facebook Twitter

Other

BEM. Node.js. Stack Exchange. Programmers. Programming Texts/Tutorials. Biggus Biggus is a pure-Python library for handling very large (i.e. too large for system memory) n-dimensional arrays. It has two main components: Representation, lazy indexing, and conversion to persistent files and NumPy arrays; andLazy calculation. At the core of Biggus is the Array which provides a simple, consistent, NumPy-esque interface to n-dimensional data which avoids reading data until explicitly requested by user code.

Commonly these Array objects are created by wrapping "concrete" data sources such as HDF5 variables, netCDF4 variables, or even just NumPy arrays. Once created, Array objects can be concatenated and stacked to form new Array objects, which can themselves be concatenated and stacked as required. In addition, all Array objects can be indexed to extract subsets. User code may request any Array object be saved to a "concrete" data form (e.g.