Huang Fuguo thesis WAMP. WAMP - Web Application Messaging Protocol. Celery - Distributed Task Queue — Celery 3.1.7 documentation. RabbitMQ - Messaging that just works. ØMQ - The Guide - ØMQ - The Guide. Distributed Messaging System. Log: Real Time Log Processing. I joined LinkedIn about six years ago at a particularly interesting time. We were just beginning to run up against the limits of our monolithic, centralized database and needed to start the transition to a portfolio of specialized distributed systems. This has been an interesting experience: we built, deployed, and run to this day a distributed graph database, a distributed search backend, a Hadoop installation, and a first and second generation key-value store. One of the most useful things I learned in all this was that many of the things we were building had a very simple concept at their heart: the log. Sometimes called write-ahead logs or commit logs or transaction logs, logs have been around almost as long as computers and are at the heart of many distributed data systems and real-time application architectures.
Part One: What Is a Log? A log is perhaps the simplest possible storage abstraction. Records are appended to the end of the log, and reads proceed left-to-right. What's next. Readings in distributed systems. This post is a work in progress. Inspired by a recent purchase of the Red Book, which provides a curated list of important papers around database systems, I’ve decided to begin assembling a list of important papers in distributed systems. Similar to the Red Book, I’ve broken each group of papers out into a series of categories, each highlighting a progression of related ideas over time focused in a specific area of research within the field. Keeping the tradition of the Red Book, I’ve included both papers which resulted in very successful systems and/or techniques, as well as papers which introduced a concept which was either immediately dismissed or proven incorrect.
This emphasizes the progression of ideas which lead to the development of these systems. Consensus The problems of establishing consensus in a distributed system. Consistency Types of consistency, and practical solutions to solving ensuring atomic operations across a set of replicas. Conflict-free data structures Systems Books.