Fanout architecture design, Apache spark, algorithms beats Moore's law and Linux's SO_REUSEPORT | Sigmoid Analytics. Fanout architecture design guides from Google (pdf): If a server has an average 1-ms latency but has a 1% chance of taking longer than 1 second to finish a request, spreading a request across 100 such servers would mean that 63% requests will take more than 1 second. This paper introduces some good engineering practices for designing a fanout architecture. Apache Spark project: When memory is so cheap, why don’t run mapreduce in memory? Spark was initially developed for two applications where placing data in memory helps: iterative algorithms, which are common in machine learning, and interactive data mining. In both cases, Spark can run up to 100x faster than Hadoop MapReduce.
However, you can use Spark for general data processing too. Linux kernel 3.9 introduced SO_REUSEPORT: This patch allows multiple processes to bind to the same IP. By: Alex Dong |Aug 29th, 2013 Reference: Enterprise Architecture is Dead. The theory of Enterprise Architecture goes something like this: There are massive gaps between business and IT — fueled by a legacy of tactical solutions, disconnects and politics. Enterprise Architecture is supposed to solve these gaps with a unified approach to business, application, data and technology architecture. Quite a challenge — don't you think? Managing Complexity If you're a Enterprise Architect tasked with spanning the gap between business and IT — what do you do? The answer is simple — you stay very high level.
This is exactly what most Enterprise Architects do — draw boxes and clouds around things and leave it at that. The problem with this approach is that Enterprise Architecture gets a reputation for being academic — completely detached from the realities of business, IT execution and political challenges. The Real Problem The real problem facing Enterprise Architects is that the gap between business and IT is too great. The Result The Key to the Future 1. 2.