Architecture des systèmes informatiques
Get flash to fully experience Pearltrees
Le but de ce cours est de donner une idée de la programmation, algorithmique et sémantique de systèmes parallèles et distribués. Pour y arriver, on utilise un biais dans un premier temps, qui nous permettra d'aborder les principaux thèmes du domaine. On va utiliser le système de ``threads'' de JAVA pour simuler des processus parallèles et distribués. Plus tard, on utilisera d'autres fonctionnalités de JAVA, comme les RMI, pour effectuer véritablement des calculs en parallèle sur plusieurs machines. Les ``threads'' en tant que tels sont une première approche du parallélisme qui peut même aller jusqu'au ``vrai parallélisme'' sur une machine multi-processeurs. Pour avoir un panorama un peu plus complet du parallélisme et de la distribution, il faut être capable de faire travailler plusieurs machines (éventuellement très éloignées l'une de l'autre; penser à internet) en même temps.
Avant-Propos
Hypercube
Architectural, organisational, and engineering aspects of distributed computing . Computer architectures , software architectures , software frameworks , and network architectures related to distributed computing and distributed systems. Different “styles” of distributed computing.
Category:Distributed computing architecture - Wikipedia, the free encyclopedia
In computing , symmetric multiprocessing (SMP) involves a multiprocessor computer hardware architecture where two or more identical processors are connected to a single shared main memory and are controlled by a single OS instance. Most common multiprocessor systems today use an SMP architecture. In the case of multi-core processors, the SMP architecture applies to the cores, treating them as separate processors. Processors may be interconnected using buses, crossbar switches or on-chip mesh networks. The bottleneck in the scalability of SMP using buses or crossbar switches is the bandwidth and power consumption of the interconnect among the various processors, the memory, and the disk arrays. Mesh architectures avoid these bottlenecks, and provide nearly linear scalability to much higher processor counts at the sacrifice of programmability:
Symmetric multiprocessing
Asymmetric multiprocessing , or AMP, was a software stopgap for handling multiple CPUs before symmetric multiprocessing , or SMP, was available. Asymmetric multiprocessing Multiprocessing is the use of more than one CPU in a computer system. The CPU is the arithmetic and logic engine that executes user applications; an I/O interface such as a GPU , even if it is implemented using an embedded processor, does not constitute a CPU because it does not run the user's application program. With multiple CPUs, more than one set of program instructions can be executed at the same time. All of the CPUs have the same user-mode instruction set, so a running job can be rescheduled from one CPU to another. [ 1 ]
Asymmetric multiprocessing
The Blue Gene /P supercomputer at Argonne National Lab runs over 250,000 processors using normal data center air conditioning, grouped in 72 racks/cabinets connected by a high-speed, optical network [ 1 ] A supercomputer is a computer at the frontline of current processing capacity, particularly speed of calculation. Supercomputers were introduced in the 1960s and were designed primarily by Seymour Cray at Control Data Corporation (CDC), and later at Cray Research . While the supercomputers of the 1970s used only a few processors , in the 1990s, machines with thousands of processors began to appear and by the end of the 20th century, massively parallel supercomputers with tens of thousands of "off-the-shelf" processors were the norm. [ 2 ] [ 3 ]
Supercomputer
Simultaneous multithreading , often abbreviated as SMT , is a technique for improving the overall efficiency of superscalar CPUs with hardware multithreading . SMT permits multiple independent threads of execution to better utilize the resources provided by modern processor architectures . [ edit ] Details Multithreading is similar in concept to preemptive multitasking but is implemented at the thread level of execution in modern superscalar processors.
Simultaneous multithreading
Superscalar
Simple superscalar pipeline. By fetching and dispatching two instructions at a time, a maximum of two instructions per cycle can be completed. (IF = Instruction Fetch, ID = Instruction Decode, EX = Execute, MEM = Memory access, WB = Register write back, i = Instruction number, t = Clock cycle [i.e., time]) A superscalar CPU architecture implements a form of parallelism called instruction level parallelism within a single processor. It therefore allows faster CPU throughput than would otherwise be possible at a given clock rate .
Vector processor
A vector processor , or array processor , is a central processing unit (CPU) that implements an instruction set containing instructions that operate on one-dimensional arrays of data called vectors . This is in contrast to a scalar processor , whose instructions operate on single data items. Although Intel processors and their clones were designed initially as scalar, new models contain an increasing number of specialized vector instructions such as those supplied by the Advanced Vector Extensions set. Vector processors first appeared in the 1970s, and formed the basis of most supercomputers through the 1980s and into the 1990s. Improvements in scalar processors, particularly microprocessors , resulted in the decline of traditional vector processors in supercomputers, and the appearance of vector processing techniques in mass market CPUs around the early 1990s.
The Borg, a 52-node Beowulf cluster used by the McGill University pulsar group to search for pulsations from binary pulsars A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them. The result is a high-performance parallel computing cluster from inexpensive personal computer hardware.
Beowulf (computing)
Massively parallel
Massively parallel is a description that appears in computer science, life sciences , medical diagnostics , and other fields. A massively parallel computer is a distributed memory computer system that consists of many individual nodes, each of which is essentially an independent computer in itself, and in turn consists of at least one processor, its own memory, and a link to the network that connects all the nodes together. Such computer systems have many independent arithmetic units or entire microprocessors , that run in parallel. The term massive connotes hundreds if not thousands of such units.
The speedup of a program using multiple processors in parallel computing is limited by the sequential fraction of the program. For example, if 95% of the program can be parallelized, the theoretical maximum speedup using parallel computing would be 20× as shown in the diagram, no matter how many processors are used. Amdahl's law , also known as Amdahl's argument , [ 1 ] is named after computer architect Gene Amdahl , and is used to find the maximum expected improvement to an overall system when only part of the system is improved. It is often used in parallel computing to predict the theoretical maximum speedup using multiple processors.
Amdahl's law
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture: Single Instruction, Single Data stream (SISD) A sequential computer which exploits no parallelism in either the instruction or data streams. Single control unit (CU) fetches single Instruction Stream (IS) from memory. The CU then generates appropriate control signals to direct single processing element (PE) to operate on single Data Stream (DS) i.e. one operation at a time
Flynn's taxonomy
Speedup
Linear speedup or ideal speedup is obtained when . When running an algorithm with linear speedup, doubling the number of processors doubles the speed. As this is ideal, it is considered very good scalability .Cost efficiency
Cost efficiency (or cost optimality ), in the context of parallel computer algorithms , refers to a measure of how effectively parallel computing can be used to solve a particular problem. A parallel algorithm is considered cost efficient if its asymptotic running time multiplied by the number of processing units involved in the computation is comparable to the running time of the best sequential algorithm. For example, an algorithm that can be solved in time using the best known sequential algorithm and in a parallel computer with processors will be considered cost efficient.Gustafson's Law (also known as Gustafson-Barsis' law ) is a law in computer science which says that computations involving arbitrarily large data sets can be efficiently parallelized . Gustafson's Law provides a counterpoint to Amdahl's law , which describes a limit on the speed-up that parallelization can provide, given a fixed data set size. Gustafson's law was first described [ 1 ] by John L. Gustafson and his colleague Edwin H. Barsis : where P is the number of processors, S is the speedup , and