Exploiting layerwise convexity of rectifier networks with sign constrained weights

By introducing sign constraints on the weights, this paper proposes sign constrained rectifier networks (SCRNs), whose training can be solved efficiently by the well known majorization–minimization (MM) algorithms. We prove that the proposed two-hidden-layer SCRNs, which exhibit negative weights in...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Neural networks Ročník 105; s. 419 - 430
Hlavní autoři:	An, Senjian, Boussaid, Farid, Bennamoun, Mohammed, Sohel, Ferdous
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	United States Elsevier Ltd 01.09.2018
Témata:	Geometrically interpretable neural network Rectifier neural network The majorization–minimization algorithm Geometrically interpretable neural network The majorization–minimization algorithm Rectifier neural network
ISSN:	0893-6080, 1879-2782, 1879-2782
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	By introducing sign constraints on the weights, this paper proposes sign constrained rectifier networks (SCRNs), whose training can be solved efficiently by the well known majorization–minimization (MM) algorithms. We prove that the proposed two-hidden-layer SCRNs, which exhibit negative weights in the second hidden layer and negative weights in the output layer, are capable of separating any number of disjoint pattern sets. Furthermore, the proposed two-hidden-layer SCRNs can decompose the patterns of each class into several clusters so that each cluster is convexly separable from all the patterns from the other classes. This provides a means to learn the pattern structures and analyse the discriminant factors between different classes of patterns. Experimental results are provided to show the benefits of sign constraints in improving classification performance and the efficiency of the proposed MM algorithm.
Bibliografie:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
ISSN:	0893-6080 1879-2782 1879-2782
DOI:	10.1016/j.neunet.2018.06.005