The Cascaded Forward algorithm for neural network training

Backpropagation (BP) algorithm has played a significant role in the development of deep learning. However, there exist some limitations associated with this algorithm, such as getting stuck in local minima and experiencing vanishing/exploding gradients, which have led to questions about its biologic...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Pattern recognition Ročník 161; s. 111292
Hlavní autori: Zhao, Gongpei, Wang, Tao, Jin, Yi, Lang, Congyan, Li, Yidong, Ling, Haibin
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier Ltd 01.05.2025
Predmet:
Backpropagation algorithm
Deep neural network
Forward–Forward algorithm
Image classification
Deep neural network
Backpropagation algorithm
Forward–Forward algorithm
Image classification
ISSN:0031-3203
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Backpropagation (BP) algorithm has played a significant role in the development of deep learning. However, there exist some limitations associated with this algorithm, such as getting stuck in local minima and experiencing vanishing/exploding gradients, which have led to questions about its biological plausibility. To address these limitations, alternative algorithms to backpropagation have been preliminarily explored, with the Forward–Forward (FF) algorithm being one of the most well-known. In this paper we propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, CaFo directly outputs label distributions at each cascaded block and waives the requirement of generating additional negative samples. Consequently, CaFo leads to a more efficient process at both training and testing stages. Moreover, in our CaFo framework each block can be trained in parallel, allowing easy deployment to parallel acceleration systems. The proposed method is evaluated on four public image classification benchmarks, and the experimental results illustrate significant improvement in prediction accuracy in comparison with recently proposed baselines. The code is available at: https://github.com/Graph-ZKY/CaFo.
ISSN:0031-3203
DOI:10.1016/j.patcog.2024.111292