Data Recipes pregel_paper PEGASUS: Peta-Scale Graph Mining System Pegasus An award-winning, open-source, graph-mining system with massive scalability. Analyze petabytes of graph data with ease. English, all platforms We won the Open Source Software World Challenge, Silver Award mikeaddison93/hoya Presto | Distributed SQL Query Engine for Big Data Pregel Many practical computing problems concern large graphs. Standard examples include the Web graph and various social networks. The scale of these graphs - in some cases billions of vertices, trillions of edges - poses challenges to their efficient processing. In this paper we present a computational model suitable for this task. Programs are expressed as a sequence of iterations, in each of which a vertex can receive messages sent in the previous iteration, send messages to other vertices, and modify its own state and that of its outgoing edges or mutate graph topology. This vertex-centric approach is flexible enough to express a broad set of algorithms.
twitter/cassovary TinkerPop mikeaddison93/hadoop-20 Manhattan, our real-time, multi-tenant distributed database for Twitter scale As Twitter has grown into a global platform for public self-expression and conversation, our storage requirements have grown too. Over the last few years, we found ourselves in need of a storage system that could serve millions of queries per second, with extremely low latency in a real-time environment. Availability and speed of the system became the utmost important factor. Not only did it need to be fast; it needed to be scalable across several regions around the world. Over the years, we have used and made significant contributions to many open source databases. Our holistic view into storage systems at TwitterDifferent databases today have many capabilities, but through our experience we identified a few requirements that would enable us to grow the way we wanted while covering the majority of use cases and addressing our real-world concerns, such as correctness, operability, visibility, performance and customer support. We designed with the following goals in mind:
SNAP: Stanford Network Analysis Project COMBINATORIAL_BLAS: Combinatorial BLAS Library (MPI reference implementation) Authors: Aydın Buluç , John R. Gilbert , Adam Lugowski This material is based upon work supported by the National Science Foundation under Grant No. 0709385. The Combinatorial BLAS is an extensible distributed-memory parallel graph library offering a small but powerful set of linear algebra primitives specifically targeting graph analytics. The Combinatorial BLAS is also the backend of the Python Knowledge Discovery Toolbox (KDT) . Download Read release notes . Requirements : You need a recent C++ compiler (gcc version 4.4+, Intel version 11.0+ and compatible), a compliant MPI implementation, and C++11 Standard library (libstdc++ that comes with g++ has them). Documentation : This is a reference implementation of the Combinatorial BLAS Library in C++/MPI. The implementation supports both formatted and binary I/O. SpParMat <int, float, SpDCCols<int,float> > A; Sparse and dense vectors can be distributed either along the diagonal processor or to all processor. New in version 1.3 :
Social Network Analysis JetS3t