Iterators, Schedulers, and Distributed-memory Parallelism - GRAEFE, GOETZ (Software, practice & experience, 1996)
In previous work, we demonstrated the advantages of encapsulating query evaluation algorithms as ‘iterators’ for sequential and parallel query evaluation. Unfortunately, those earlier models have a severe drawback with respect to resource allocation in distributed‐memory systems. Since threads may be initiated long before they actually perform useful work, thread placement decisions may be suboptimal. In this paper, we briefly review the iterator model and then extend it to support bottom‐up, just‐in‐time activation of appropriate query plan fragments as well as local and global synchronization and communication among sibling threads. Some of the algorithms described here may seem intricate; however, the intricacy is encapsulated entirely in the parallelism or ‘exchange’ iterator, thus freeing developers of data manipulation iterators to focus on the specific algorithms at hand, instead of on mechanisms for parallelism. Moreoever, code fragments extracted from an operational prototype demonstrate the ease with which our suggestions can be implemented. Run‐time overhead and performance of our techniques are identical to those of earlier, less extensible techniques for bottom‐up activation; the benefits of the presented techniques are their clean design, ion, extensibility, and the resulting ease of maintenance and enhancement.