On the Utility of Communication–Computation Overlap in Data-Parallel Programs

The computational speed of individual processors in distributed memory computers is increasing faster than the communication speed of the interconnection networks. This has led to the general perception among developers of compilers for data-parallel languages that overlapping communications with co...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of parallel and distributed computing Jg. 33; H. 2; S. 197 - 204
Hauptverfasser: Quinn, Michael J., Hatcher, Philip J.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: San Diego, CA Elsevier Inc 15.03.1996
Elsevier
Schlagworte:
Applied sciences
Computer science; control theory; systems
Computer systems and distributed systems. User interface
Exact sciences and technology
Software
Software engineering
Telecommunications
Telecommunications and information theory
Teleprocessing networks. Isdn
Parallel algorithm
Distributed memory multiprocessor system
Parallel program
Interconnected power system
Compiler optimization
Parallel computation
Overlap
Distributed system
Systolic network
Message transmission
Communication
Pipeline
ISSN:0743-7315, 1096-0848
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The computational speed of individual processors in distributed memory computers is increasing faster than the communication speed of the interconnection networks. This has led to the general perception among developers of compilers for data-parallel languages that overlapping communications with computations is an important optimization. We demonstrate that communication–computation overlap has limited utility. Overlapping communications with computations can never more than double the speed of a parallel application, and in practice the relative improvement in speed is usually far less than that. Most parallel algorithms have computational requirements that grow faster than their communication requirements. When this is the case, the gain from communication–computation overlap asymptotically approaches zero as the problem size increases.
ISSN:0743-7315
1096-0848
DOI:10.1006/jpdc.1996.0038