An analysis of Facebook photo caching. Posted about 2 months ago Every day people upload more than 350 million photos to Facebook as of December 2012 and view many more in their News Feeds and on their friends’ Timelines.
Facebook stores these photos on Haystack machines that are optimized to store photos. But there is also a deep and distributed photo-serving stack with many layers of caches that delivers photos to people so they can view them. We recently published an academic study of this stack in SOSP. In this post, we describe the stack and then cover four of our many findings: 1. Facebook's photo-serving stack The photo-serving stack has four layers: browser caches, edge caches, origin cache, and Haystack backend. That request will be routed either to our CDN partners or to one of Facebook's many edge caches, which are located at Internet points of presence (PoPs).
The origin cache is a single cache that is distributed across multiple data centers (like Prineville, Forest City, and Lulea). Trace collection. Python Best Practice Patterns by Vladimir Keleshev (Notes) DRAKON Editor.
Manual on development of Visual Studio 2005-2012 plugins in C# Abstract About a year ago we published in our blog a series of articles on development of Visual Studio plugins in C#.
We have recently revised those materials and added new sections and now invite you to have a look at the updated version of the manual. Creating extension packages (plug-ins) for Microsoft Visual Studio IDE appears as quite an easy task at the first sight. Java. Realestate.com.au Tech Blog. We’ve started some new work in Scala!
Most of the back-end developers in the Residential team have a Java background though, so we put together this cheatsheet to help get the team started. Scala does almost everything Java does, plus a whole lot of useful functional stuff. There’s a direct analog in Scala for almost everything in Java. Keep in mind though, real idiomatic Scala needs a bit more than just Java semantics — there’s lots of really powerful and useful functional features and idioms that you can learn as you go. Beautiful Native Libraries. Written on Sunday, August 18, 2013 I'm obsessed with nice APIs.
Not just APIs however, also in making the overall experience of using a library as good as possible. All Machine Learning Platforms are Terrible (but some less so) « Inviting Epiphany. I recently took a medium sized feature set with labels at work and ran it through some of the most popular machine learning platforms.
The goal was to get a feel for each of them via the standard battery of regressions and evaluate each for use in further experimentation. This is a review of my journey. Experimental Setup: Features and Labels in a ~500mb CSV file Labeled Records: ~140,000 Features: ~3500 binary, labels [0-100] Hardware: 4 x 8 = 32 cores, 256gb of ram OS: Windows Server 2008r2 - F# with Math.NET - I used F# to build the features for the rest of these processes. It was quite nice using the SQL Type Provider to grab the data out of the database and then process it into binary features, even though it consisted of fourteen unoptimized tables across two SQL Server databases with rather odd relationships. How to Migrate from Ant to Maven: Project Structure. I’ve seen my fair share of projects migrating from Ant to Maven, and, for a complex project, this migration path can take some time.
You have to worry about dependency management, project structure, and retraining an existing team to use Maven and understand the core concepts behind the tool. PowerShell Command Line Introduction. Java 2 Java developers resource center - Java, Scala, Groovy, Android. JAX-WS SOAP over JMS. Memory – Part 2: Understanding Process memory.
In the previous article, we introduced a way to classify the memory a process reclaimed.
We used 4 quadrants using two axis: private/shared and anonymous/file-backed. We also evoked the complexity of the sharing mechanism and the fact that all memory is basically reclaimed to the kernel. Everything we talked about was virtual. Google Ventures — Startup Lab — Test-driven design. Testing is not about writing tests.
It’s about writing testable code. Google software engineer Vojta Jína recently came to the Startup Lab to show us what makes code testable and why. Vojta’s goal is to make the development of web apps and testing more fun. Www.planetgeek.ch/wp-content/uploads/2013/06/Clean-Code-V2.1.pdf. Mathematics self-proves its own Consistency (contra Gödel et. al.) The issue of consistency has come up in many LtU posts.
Some readers might be interested in the following abstract from Mathematics self-proves its own Consistency: Mathematics self-proves its own Consistency (contra Gödel et. al.) Eigenfaces, for Facial Recognition. This post assumes familiarity with the terminology and notation of linear algebra, particularly inner product spaces.
Seven Myths of Formal Methods Revisited. Mrjob — mrjob 0.4 documentation. The Art of Lisp & Writing. Ignorance more frequently begets confidence than does knowledge. –Charles Darwin Lisp is the language of loveliness. With it a great programmer can make a beautiful, operating thing, a thing organically created and formed through the interaction of a programmer/artist and a medium of expression that happens to execute on a computer. Taught that programming—or the worse "developing software"—is like a routine engineering activity, many find difficulty seeing writing as a model or even a metaphor for programming. Writing is creative, it is self-expression, it is art, which is to say it isn't a science and unlike science and engineering, it isn't a serious activity.
Artists make things and have always done so, gathering knowledge. Bayesian Methods for Hackers. An intro to Bayesian methods and probabilistic programming from a computation/understanding-first, mathematics-second point of view. Prologue The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chapters of slow, mathematical analysis. The typical text on Bayesian inference involves two to three chapters on probability theory, then enters what Bayesian inference is. Unfortunately, due to mathematical intractability of most Bayesian models, the reader is only shown simple, artificial examples.