Introduction to Advanced Computer Architecture and Parallel Processing - El‐Rewini, Hesham; Abd‐El‐Barr, Mostafa (Advanced Computer Architecture and Parallel Processing, 2004)
Single processor supercomputers have achieved unheard of speeds and have been pushing hardware technology to the physical limit of chip manufacturing. But there is limit to the computational power that can be achieved with a single processor system. In this chapter, we provide an introduction to the advances made in computer architectures that utilize parallelism via multiple processing units. Parallel processors are computer systems consisting of multiple processing units connected via some interconnection network plus the software needed to make the processing units work together. There are two major factors used to categorize such systems: the processing units, and the interconnection network that ties them together. The processing units can communicate and interact with each other using either shared memory or message passing methods. The interconnection network for shared memory systems can be classified as bus‐based versus switch‐based. In message passing systems, the interconnection network is divided into static and dynamic. Static connections have fixed topology that does not change while programs are running. Dynamic connections create links on the fly as the program executes. Our coverage in this chapter starts by reviewing the advances made in computer architectures in the last four decades. These include the batch era, the time‐sharing era, the desktop era, the network era and beyond. We then provide a quick review of the Flynn's Taxonomy of Computer Architecture. This is followed by a brief introduction to the SIMD and the MIMD model of parallel computing. We then shed some light on the two MIMD architectures, i.e., shared‐memory and message‐passing models. Our coverage in this chapter concludes with a detailed discussion on the interconnection networks used for parallel architectures. This includes interconnection networks classification and some aspects of their performance characteristics.