Introducing the 2.6 Kernel. The kernel has come a long way since Linus branched off 2.4.15 to create 2.5.0 back on November 22, 2001.
Since then, rapid development has ensued, resulting in a drastically different and much-improved kernel. This article discusses the more interesting, important features and their impact on the performance and reliability of Linux. In Linux kernel parlance, the minor version number of a kernel denotes whether that kernel belongs to a stable series or a development series. Even minor version numbers denote stable kernels, and odd minor version numbers denote development kernels. When a development kernel is fully mature and deemed stable, its minor version number is incremented to an even value.
The current development kernel is 2.5. When the development series is complete, Linus releases the kernel as stable. In March 2001 and again in June 2002, the core kernel developers met at Kernel Summits to discuss the kernel. The scheduler should provide full O(1) scheduling. Table 1. KS2009: How Google uses Linux. There may be no single organization which runs more Linux systems than Google.
But the kernel development community knows little about how Google uses Linux and what sort of problems are encountered there. Google's Mike Waychison traveled to Tokyo to help shed some light on this situation; the result was an interesting view on what it takes to run Linux in this extremely demanding setting. Mike started the talk by giving the developers a good laugh: it seems that Google manages its kernel code with Perforce. He apologized for that. There is a single tree that all developers commit to. This way of doing things is far from ideal; it means that Google lags far behind the mainline and has a hard time talking with the kernel development community about its problems. There are about 30 engineers working on Google's kernel. And there's a lot in that tree. Linus asked: why aren't these patches upstream?
Google has a number of "pain points" which make working with the community harder. Hadoop & Big Data Blog » Blog Archive » Job Scheduling in Ha. (guest blog post by Matei Zaharia) When Apache Hadoop started out, it was designed mainly for running large batch jobs such as web indexing and log mining.
Users submitted jobs to a queue, and the cluster ran them in order. However, as organizations placed more data in their Hadoop clusters and developed more computations they wanted to run, another use case became attractive: sharing a MapReduce cluster between multiple users. The benefits of sharing are tremendous: with all the data in one place, users can run queries that they may never have been able to execute otherwise, and costs go down because system utilization is higher than building a separate Hadoop cluster for each group.
However, sharing requires support from the Hadoop job scheduler to provide guaranteed capacity to production jobs and good response time to interactive jobs while allocating resources fairly between users. The Fair Scheduler arose out of Facebook’s need to share its data warehouse between multiple users. Linux scalability: Accept() scalability on Linux. Steve Molloy, CITI - University of Michigan Introduction Network servers that use TCP/IP to communicate with their clients are rapidly increasing their offered loads.
A service may elect to create multiple threads or processes to wait for increasing numbers of concurrent incoming connections. By pre-creating these multiple threads, a network server can handle connections and requests at a faster rate than with a single thread. In Linux, when multiple threads call accept() on the same TCP socket, they get put on the same wait queue, waiting for an incoming connection to wake them up. This report explores the effects of the "thundering herd" problem associated with the accept() system call as implemented in the Linux kernel.
Investigation While researching the TCP/IP accept code, we found a few interesting points. Sock->state_change.................... The code for each one of these methods invokes the wake_up_interruptible() function. Guidelines Solutions Benchmark Description Micro-Benchmark.