background preloader

Distributed systems

Facebook Twitter


Api gateway. Key value stores. Sharding. "Building Scalable Stateful Services" by Caitie McCaffrey. Storm, distributed and fault-tolerant realtime computation. Druid | Interactive Analytics at Scale. Apache Spark™ - Lightning-Fast Cluster Computing. Elastic · Revealing Insights from Data (Formerly Elasticsearch) Apache Kafka. Fallacies.pdf. A plain English introduction to CAP theorem « Kaushik Sathupadi. You’ll often hear about the CAP theorem which specifies some kind of an upper limit when designing distributed systems. As with most of my other introduction tutorials, lets try understanding CAP by comparing it with a real world situation.

Chapter 1: “Remembrance Inc” Your new venture : Last night when your spouse appreciated you on remembering her birthday and bringing her a gift, a strange Idea strikes you. People are so bad in remembering things. And you’re sooo good at it. So why not start a venture that will put your talent to use? The more you think about it, the more you like it. Remembrance Inc! So, your typical phone conversation will look like this: Customer : Hey, Can you store my neighbor’s birthday? Chapter 2 : You scale up: Your venture gets funded by YCombinator. And there starts the problem. Your start with a simple plan: Chapter 3 : You have your first “Bad Service” : Jhon: Hey You: Glad you called “Remembrance Inc!”. How did that happen? ” look” , you tell her.. Genius! The Modulith | The ryg blog.

Much has been written about all the myriad ways to go wrong when writing software. Poor management; scope creep; too little structure, not modular enough, and it’s a “big ball of mud”. Too much (or too rigid) and it’s a “software crystal”, impossible to alter. And so on. Suppose you get all that right, and actually ship a useful system to users, it solves their problems well enough, and the code is reasonably clean, has a sound design and a modular structure with interface that, while not perfect, work okay. That’s about as good as it gets! Well done. Alas, you’re not out of the woods. The arguments for modularity are well known: separating concerns breaks large systems down into smaller constituent parts that can be understood individually, with clearly-defined interfaces between them. And there really are systems like that, where the interfaces are rigid and well-specified, components come only in a handful of “shapes”, and everything cleanly fits together, like Lego bricks.

Like this: Bloom Programming Language | Disorderly programming for the cloud and other distributed systems. Cidr11-bloom.pdf.