Deterministic convergence of complex mini-batch gradient learning algorithm for fully complex-valued neural networks

This paper investigates the fully complex mini-batch gradient algorithm for training complex-valued neural networks. Mini-batch gradient method has been widely used in neural network training, however, its convergence analysis is usually restricted to real-valued neural networks and of probability n...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Neurocomputing (Amsterdam) Jg. 407; S. 185 - 193
Hauptverfasser: Zhang, Huisheng, Zhang, Ying, Zhu, Shuai, Xu, Dongpo
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier B.V 24.09.2020
Schlagworte:
Convergence
Fully complex-valued neural networks
Mini-batch gradient algorithm
Wirtinger calculus
Fully complex-valued neural networks
Mini-batch gradient algorithm
Wirtinger calculus
Convergence
ISSN:0925-2312, 1872-8286
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper investigates the fully complex mini-batch gradient algorithm for training complex-valued neural networks. Mini-batch gradient method has been widely used in neural network training, however, its convergence analysis is usually restricted to real-valued neural networks and of probability nature. By introducing a new Taylor mean value theorem for analytic functions, in this paper we establish deterministic convergence results for the fully complex mini-batch gradient algorithm under mild conditions. The deterministic convergence here means that the algorithm will deterministically converge, and both the weak convergence and strong convergence will be proved. Benefited from the newly introduced mean value theorem, our results are of global nature in that they are valid for arbitrarily given initial values of the weights. The theoretical findings are validated with a simulation example.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2020.04.114