On the Utility of Communication–Computation Overlap in Data-Parallel Programs

The computational speed of individual processors in distributed memory computers is increasing faster than the communication speed of the interconnection networks. This has led to the general perception among developers of compilers for data-parallel languages that overlapping communications with co...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of parallel and distributed computing Ročník 33; číslo 2; s. 197 - 204
Hlavní autoři: Quinn, Michael J., Hatcher, Philip J.
Médium: Journal Article
Jazyk:angličtina
Vydáno: San Diego, CA Elsevier Inc 15.03.1996
Elsevier
Témata:
Applied sciences
Computer science; control theory; systems
Computer systems and distributed systems. User interface
Exact sciences and technology
Software
Software engineering
Telecommunications
Telecommunications and information theory
Teleprocessing networks. Isdn
Parallel algorithm
Distributed memory multiprocessor system
Parallel program
Interconnected power system
Compiler optimization
Parallel computation
Overlap
Distributed system
Systolic network
Message transmission
Communication
Pipeline
ISSN:0743-7315, 1096-0848
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The computational speed of individual processors in distributed memory computers is increasing faster than the communication speed of the interconnection networks. This has led to the general perception among developers of compilers for data-parallel languages that overlapping communications with computations is an important optimization. We demonstrate that communication–computation overlap has limited utility. Overlapping communications with computations can never more than double the speed of a parallel application, and in practice the relative improvement in speed is usually far less than that. Most parallel algorithms have computational requirements that grow faster than their communication requirements. When this is the case, the gain from communication–computation overlap asymptotically approaches zero as the problem size increases.
ISSN:0743-7315
1096-0848
DOI:10.1006/jpdc.1996.0038