A scalable multi-signal approach for the parallelization of self-organizing neural networks
Self-Organizing Neural Networks (SONNs) have a wide range of applications with massive computational requirements that often need to be satisfied with optimized parallel algorithms and implementations. In literature, SONN have been generally parallelized with GPU computing according to a single-sign...
Saved in:
| Published in: | Neural networks Vol. 123; pp. 108 - 117 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
United States
Elsevier Ltd
01.03.2020
|
| Subjects: | |
| ISSN: | 0893-6080, 1879-2782, 1879-2782 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Self-Organizing Neural Networks (SONNs) have a wide range of applications with massive computational requirements that often need to be satisfied with optimized parallel algorithms and implementations. In literature, SONN have been generally parallelized with GPU computing according to a single-signal paradigm: each GPU thread manages one or more nodes of the network and works concurrently on one input signal at the time. This paper presents two contributions. The first one is the experimental proof that the single-signal approach for SONNs is not optimal for the task, as it is intrinsically sequential at its core and thus inherently limited in its performance. The non-optimality of the single-signal paradigm is illustrated via a specific and simplified benchmark. The second contribution is the introduction of a new multi-signal paradigm for the parallelization of SONNs, whereby multiple signals are processed at once in each iteration hence allowing different GPU threads to work on different signals. The advantages of the multi-signal approach are shown through several benchmarks involving the Self-Organizing Adaptive Map (SOAM) algorithm as a basis for evaluation. Having a graph-based termination condition that depends on the features of the network being grown, the SOAM algorithm allows assessing both functional equivalence and performances of the paradigm proposed without relying on arbitrary thresholds. Nonetheless, the evaluation proposed has a broader scope since it refers to a unified framework for the GPU parallelization of a generic SONN.
•The topic is the parallelization of self-organizing neural networks (SONNs).•A single-signal approach is common for GPU implementations.•The paper shows that single-signal is inherently sequential and has limited scalability.•The paper thus proposes an effective, scalable, multi-signal approach for the parallelization of SONNs.•The multi-signal performance is validated with a carefully-constructed benchmark on SOAM. |
|---|---|
| Bibliography: | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ISSN: | 0893-6080 1879-2782 1879-2782 |
| DOI: | 10.1016/j.neunet.2019.11.016 |