Doubly random parallel stochastic methods for large scale learning

We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings of the American Control Conference s. 4847 - 4852
Hlavní autori: Mokhtari, Aryan, Koppel, Alec, Ribeiro, Alejandro
Médium: Konferenčný príspevok.. Journal Article
Jazyk:English
Vydavateľské údaje: American Automatic Control Council (AACC) 01.07.2016
Predmet:
Algorithms
Convergence
Learning
Linear programming
Mathematical analysis
Optimization
Parallel processing
Processors
Program processors
Radio frequency
Stochasticity
Training
Vectors (mathematics)
ISSN:2378-5861
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple processors to operate in a randomly chosen subset of blocks of the feature vector. We call the algorithm parallel stochastic because processors choose elements of the training set randomly and independently. Algorithms that are parallel in either of these dimensions exist, but RAPSA is the first attempt at a methodology that is parallel in both, the selection of blocks and the selection of elements of the training set. In RAPSA, processors utilize the randomly chosen functions to compute the stochastic gradient component associated with a randomly chosen block. The technical contribution of this paper is to show that this minimally coordinated algorithm converges to the optimal classifier when the training objective is convex. In particular, we show that: (i) When using decreasing stepsizes, RAPSA converges almost surely over the random choice of blocks and functions. (ii) When using constant stepsizes, convergence is to a neighborhood of optimality with a rate that is linear in expectation. RAPSA is numerically evaluated on the MNIST digit recognition problem.
Bibliografia:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Conference-1
ObjectType-Feature-3
content type line 23
SourceType-Conference Papers & Proceedings-2
ISSN:2378-5861
DOI:10.1109/ACC.2016.7526120