background preloader

Apache Giraph

Facebook Twitter

Opensource, Java

Facebook’s trillion-edge, Hadoop-based and open source graph-processing engine. People following the open source Giraph project likely know that Facebook was experimenting with it, and on Wednesday the company detailed just how heavily it’s leaning on Giraph . Facebook scaled it to handle trillions of connections among users and their behavior, as the core of its Open Graph tool. Oh, and now anyone can download Giraph, which is an Apache Software Foundation project, with Facebook’s improvements baked in. Graphs, you might recall from our earlier coverage, are the new hotness in the big data world . Graph-processing engines and graph databases use a system of nodes (e.g., Facebook users, their Likes and their interests) and edges (e.g., the connections between all of them) in order to analyze the relationships among groups of people, places and things. Giraph is an open source take on Pregel , the graph-processing platform that powers Google PageRank, among other things.

Source: Facebook But Hadoop compatibility only takes you so far: Scaling Apache Giraph to a trillion edges. Facebook's Graph Search puts Apache Giraph on the map. Move over Hadoop, there is another highly scalable data-processing powerhouse in town: Apache Giraph. Facebook is using the technology to bring a new style of search to its billion users. When Facebook built its Graph Search service, the social-networking company picked Giraph over other social graphing technologies—such as the Hadoop-based Apache Hive and GraphLab—because of Giraph’s speed and immense scalability. “Analyzing these real-world graphs at our scale ... with available software was impossible last year. We needed a programming framework to express a wide range of graph algorithms in a simple way and scale them to massive datasets,” wrote Facebook software engineer Avery Ching, in a blog post that discussed Facebook’s use of the technology.

With a little modification, Facebook has used Giraph to analyze a trillion edges, or connections between different entities, in under four minutes. Facebook's Graph Search Facebook Giraph's background. Apache Giraph. References[edit] External links[edit] Official website. Giraph - Welcome To Apache Giraph! Giraph System Properties.