Search Results - Computing methodologies Distributed computing methodologies Distributed programming languages

MAD-Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems by Hsia, Samuel, Golden, Alicia, Acun, Bilge, Ardalani, Newsha, DeVito, Zachary, Wei, Gu-Yeon, Brooks, David, Wu, Carole-Jean

“…Training and deploying large-scale machine learning models is time-consuming, requires significant distributed computing infrastructures, and incurs high operational costs…”
Get full text

Skywalker: Efficient Alias-Method-Based Graph Sampling and Random Walk on GPUs by Wang, Pengyu, Li, Chao, Wang, Jing, Wang, Taolei, Zhang, Lu, Leng, Jingwen, Chen, Quan, Guo, Minyi

“…Graph sampling and random walk operations, capturing the structural properties of graphs, are playing an important role today as we cannot directly adopt computing-intensive algorithms on large-scale graphs…”
Get full text

Gluon-Async: A Bulk-Asynchronous System for Distributed and Heterogeneous Graph Analytics by Dathathri, Roshan, Gill, Gurbinder, Hoang, Loc, Jatala, Vishwesh, Pingali, Keshav, Nandivada, V. Krishna, Dang, Hoang-Vu, Snir, Marc

“…Distributed graph analytics systems for CPUs, like D-Galois and Gemini, and for GPUs, like D-IrGL and Lux, use a bulk-synchronous parallel (BSP…”
Get full text

AdaGL: Adaptive Learning for Agile Distributed Training of Gigantic GNNs by Zhang, Ruisi, Javaheripi, Mojan, Ghodsi, Zahra, Bleiweiss, Amit, Koushanfar, Farinaz

“…Distributed GNN training on contemporary massive and densely connected graphs requires information aggregation from all neighboring nodes, which leads to an explosion of inter-server communications…”
Get full text

Auto-parallelizing stateful distributed streaming applications by Schneider, Scott, Hirzel, Martin, Gedik, Bugra, Wu, Kun-Lung

“… They are comprised of operator graphs that produce and consume data tuples. The streaming programming model naturally exposes task and pipeline parallelism, enabling it to exploit parallel systems of all kinds, including large clusters…”
Get full text

Optimizing Distributed ML Communication with Fused Computation-Collective Operations by Punniyamurthy, Kishore, Hamidouche, Khaled, Beckmann, Bradford M.

“…Machine learning models are distributed across multiple nodes using numerous parallelism strategies…”
Get full text

Legate Sparse: Distributed Sparse Computing in Python by Yadav, Rohan, Lee, Wonchan, Elibol, Melih, Patti, Taylor Lee, Papadakis, Manolis, Garland, Michael, Aiken, Alex, Kjolstad, Fredrik, Bauer, Michael

“… The standard implementation of SciPy is restricted to a single CPU and cannot take advantage of modern distributed and accelerated computing resources…”
Get full text

Introduction to Parallel and Distributed Programming using N-Body Simulations by Van Craen, Alexander, Breyer, Marcel, Pfluger, Dirk

“…This paper describes how we use n-body simulations as an interesting and visually compelling way to teach efficient, parallel, and distributed programming…”
Get full text

Legate NumPy: Accelerated and Distributed Array Computing by Bauer, Michael, Garland, Michael

“… Legate works by translating NumPy programs to the Legion programming model and then leverages the scalability of the Legion runtime system to distribute data and computations across an arbitrary sized machine. Compared to similar programs written in the distributed Dask array library in Python, Legate achieves speed-ups of up to 10X on 1280 CPUs and 100X on 256 GPUs…”
Get full text

Managing Workflow Malleability in Urgent Computing for Earthquake Alerts by Ejarque, Jorge, Monterrubio-Velasco, Marisol, Bhihe, Cedric, Pienkowska, Marta, De La Puente, Josep, Badia, Rosa M.

“… UCIS4EQ is an urgent computing platform that estimates ground shaking based on high-performance parallel 3D simulations…”
Get full text

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel by Maheshwari, Ketan, Arndt, William, Karimi, Ahmad Maroof, Yin, Junqi, Suter, Frederic, Johnson, Seth, Da Silva, Rafael Ferreira

“…GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing…”
Get full text

A Sparsity-Aware Distributed-Memory Algorithm for Sparse-Sparse Matrix Multiplication by Hong, Yuxi, Buluc, Aydin

“… Distributed-memory parallel algorithms for SpGEMM have mainly focused on sparsity-oblivious approaches that use 2D and 3D partitioning…”
Get full text

Towards an Optimized Heterogeneous Distributed Task Scheduler in OpenMP Cluster by Neveu, Remy, Ceccato, Rodrigo, Leite, Gustavo, Araujo, Guido, Diaz, Jose M. Monsalve, Yviquel, Herve

“…This paper addresses the challenges of optimizing task scheduling for a distributed, task-based execution model in OpenMP for cluster computing environments…”
Get full text

CUDASTF: Bridging the Gap Between CUDA and Task Parallelism by Augonnet, Cedric, Alexandrescu, Andrei, Sidelnik, Albert, Garland, Michael

“…Organizing computation as asynchronous tasks with data-driven dependencies is a simple and efficient model for single- and multi-GPU programs. Sequential Task…”
Get full text

Scalable and Consistent Graph Neural Networks for Distributed Mesh-based Data-driven Modeling by Barwey, Shivam, Balin, Riccardo, Lusch, Bethany, Patel, Saumil, Balakrishnan, Ramesh, Pal, Pinaki, Maulik, Romit, Vishwanath, Venkatram

“…This work develops a distributed graph neural network (GNN) methodology for mesh-based modeling applications using a consistent neural message passing layer…”
Get full text

Accelerating Communications in Federated Applications with Transparent Object Proxies by Pauloski, J. Gregory, Hayot-Sasson, Valerie, Ward, Logan, Hudson, Nathaniel, Sabino, Charlie, Baughman, Matt, Chard, Kyle, Foster, Ian

“… Here, we overcome this obstacle with a new programming paradigm that decouples control flow from data flow by extending the pass-by-reference model to distributed applications…”
Get full text

Distributed-Memory Parallel Algorithms for Sparse Matrix and Sparse Tall-and-Skinny Matrix Multiplication by Ranawaka, Isuru, Hussain, Md Taufique, Block, Charles, Gerogiannis, Gerasimos, Torrellas, Josep, Azad, Ariful

“… Unfortunately, popular distributed algorithms like sparse SUMMA deliver suboptimal performance for TS-SpGEMM…”
Get full text

Reshaping High Energy Physics Applications for Near-Interactive Execution Using TaskVine by Sly-Delgado, Barry, Tovar, Ben, Zhou, Jin, Thain, Douglas

“…High energy physics experiments produce petabytes of data annually that must be reduced to gain insight into the laws of nature. Early-stage reduction executes…”
Get full text

Enhance the Strong Scaling of LAMMPS on Fugaku by Li, Jianxiong, Zhao, Tong, Guo, Zhuoqiang, Shi, Shunchen, Liu, Lijun, Tan, Guangming, Jia, Weile, Yuan, Guojun, Wang, Zhan

“…Physical phenomenon such as protein folding requires simulation up to microseconds of physical time, which directly corresponds to the strong scaling of…”
Get full text

Parallelized Multi-Agent Bayesian Optimization in Lava by Snyder, Shay, Gobin, Derek, Clerico, Victoria, Risbud, Sumedh R., Parsa, Maryam

“…In parallel with the continuously increasing parameter space dimensionality, search and optimization algorithms should support distributed parameter evaluations to reduce cumulative runtime…”
Get full text