Scalability

Facebook Twitter

Amazon's Dynamo. In two weeks we’ll present a paper on the Dynamo technology at SOSP, the prestigious biannual Operating Systems conference.

Amazon's Dynamo

Dynamo is internal technology developed at Amazon to address the need for an incrementally scalable, highly-available key-value storage system. The technology is designed to give its users the ability to trade-off cost, consistency, durability and performance, while maintaining high-availability. Let me emphasize the internal technology part before it gets misunderstood: Dynamo is not directly exposed externally as a web service; however, Dynamo and similar Amazon technologies are used to power parts of our Amazon Web Services, such as S3.

We submitted the technology for publication in SOSP because many of the techniques used in Dynamo originate in the operating systems and distributed systems research of the past years; DHTs, consistent hashing, versioning, vector clocks, quorum, anti-entropy based recovery, etc. Java NIO. Building Scalable Systems: an Asynchronous Approach. Bio Theo Schlossnagle is the author of Scalable Internet Architectures (SAMS) and a frequent speaker at worldwide IT conferences.

Building Scalable Systems: an Asynchronous Approach

He was the Principal Architect of the Momentum MTA, a new email system. Theo is a member of the IEEE and a senior member of the ACM. He serves on the editorial board of the ACM’s Queue Magazine. You can follow Theo on Twitter as @postwait. The LMAX Architecture. LMAX is a new retail financial trading platform.

The LMAX Architecture

As a result it has to process many trades with low latency. The system is built on the JVM platform and centers on a Business Logic Processor that can handle 6 million orders per second on a single thread. The Business Logic Processor runs entirely in-memory using event sourcing. Concurrent Programming Using The Disruptor. Bio Trisha Gee is a developer at LMAX, the London Multi Asset eXchange, trying to get her head around low-latency, high performance coding whilst also keeping her fingers in the other pies LMAX has to offer, such as continuous delivery and agile.

Concurrent Programming Using The Disruptor

Trisha is heavily involved in the London Java Community and the Graduate Development Community. Software is changing the world; QCon aims to empower software development by facilitating the spread of knowledge and innovation in the enterprise software development community; to achieve this, QCon is organized as a practitioner-driven conference designed for people influencing innovation in their teams: team leads, architects, project managers, engineering directors. Facebook's Realtime Analytics System. Recently, I was reading Todd Hoff's write-up on FaceBook real time analytics system.

Facebook's Realtime Analytics System

As usual, Todd did an excellent job in summarizing this video from Engineering Manager at Facebook Alex Himel, Engineering Manager at Facebook. In this first post, I’d like to summarize the case study, and consider some things that weren't mentioned in the summaries. High Scalability.