Scalability

FacebookTwitter

Amazon's Dynamo

http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html In two weeks we’ll present a paper on the Dynamo technology at SOSP , the prestigious biannual Operating Systems conference. Dynamo is internal technology developed at Amazon to address the need for an incrementally scalable, highly-available key-value storage system. The technology is designed to give its users the ability to trade-off cost, consistency, durability and performance, while maintaining high-availability. Let me emphasize the internal technology part before it gets misunderstood: Dynamo is not directly exposed externally as a web service; however, Dynamo and similar Amazon technologies are used to power parts of our Amazon Web Services, such as S3. We submitted the technology for publication in SOSP because many of the techniques used in Dynamo originate in the operating systems and distributed systems research of the past years; DHTs, consistent hashing, versioning, vector clocks, quorum, anti-entropy based recovery, etc.

Java NIO

Comme nous l’avons déjà évoqué sur le blog, à l’occasion du challenge USI 2011, nous nous sommes intéressés à différents serveurs et framework web NIO en Java. Le principe était simple en mettant à plat la spécification du challenge, nous avons identifié quelques besoins techniques : http://blog.xebia.fr/2011/11/09/java-nio-et-framework-web-haute-performance/

Building Scalable Systems: an Asynchronous Approach

http://www.infoq.com/presentations/Building-Scalable-Systems-Asynchronous-Approach Catherine & Raj have been working in Enterprise Agile transitions in large hardware manufacturers, they share their experiences and advice on leadership and bringing Scrum to hardware teams. Resistance from management is recognized as a bottleneck in agile adoption.

The LMAX Architecture

LMAX is a new retail financial trading platform. http://martinfowler.com/articles/lmax.html
http://www.infoq.com/presentations/Concurrent-Programming-Using-The-Disruptor This talk explores when to start performance testing, how to avoid the common pitfalls, how to profile when the results cause your team to pull a funny face, and what you can do about that funny face.

Concurrent Programming Using The Disruptor

http://natishalom.typepad.com/nati_shaloms_blog/2011/07/real-time-analytics-for-big-data-an-alternative-approach-to-facebooks-new-realtime-analytics-system.html Recently, I was reading Todd Hoff's write-up on FaceBook real time analytics system . As usual, Todd did an excellent job in summarizing this video from Engineering Manager at Facebook Alex Himel , Engineering Manager at Facebook. In this first post , I’d like to summarize the case study, and consider some things that weren't mentioned in the summaries.

Facebook's Realtime Analytics System