Teaching distributed memory programming from mental models

Distributed memory programming, typically through the MPI library, is the de facto standard for programming large scale parallelism, with up to millions of individual processes. Its dominant paradigm of Single Program Multiple Data (SPMD) programming is different from threaded and multicore parallel...

Full description

Saved in:
Bibliographic Details
Published in:Journal of parallel and distributed computing Vol. 118; pp. 120 - 127
Main Author: Eijkhout, Victor
Format: Journal Article
Language:English
Published: Elsevier Inc 01.08.2018
Subjects:
Distributed memory
Mental models
Parallel programming
Teaching parallel programming
Distributed memory
Mental models
Parallel programming
Teaching parallel programming
ISSN:0743-7315, 1096-0848
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Distributed memory programming, typically through the MPI library, is the de facto standard for programming large scale parallelism, with up to millions of individual processes. Its dominant paradigm of Single Program Multiple Data (SPMD) programming is different from threaded and multicore parallelism, to an extent that students have a hard time switching models. In contrast to threaded programming, which allows for a view of the execution with central control and a central repository of data, SPMD programming has a symmetric model where all processes are active all the time, with none privileged, and where data is distributed. This model is counterintuitive to the novice parallel programmer, so care needs to be taken how to instill the proper ‘mental model’. Adoption of an incorrect mental model leads to broken or inefficient code. We identify problems with the currently common way of teaching MPI, and propose a structuring of MPI courses that is geared to explicit reinforcing the symmetric model. Additionally, we advocate starting from realistic scenarios, rather than writing artificial code just to exercise newly-learned routines. •Distributed memory programming requires a different mental conception of parallel than other parallelism types.•We propose explicitly targeting the right mental model.•We argue for a different motivation and sequence of topics than is typically used.
ISSN:0743-7315
1096-0848
DOI:10.1016/j.jpdc.2018.02.029