Doubly random parallel stochastic methods for large scale learning

We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the American Control Conference s. 4847 - 4852
Hlavní autoři: Mokhtari, Aryan, Koppel, Alec, Ribeiro, Alejandro
Médium: Konferenční příspěvek Journal Article
Jazyk:angličtina
Vydáno: American Automatic Control Council (AACC) 01.07.2016
Témata:
Algorithms
Convergence
Learning
Linear programming
Mathematical analysis
Optimization
Parallel processing
Processors
Program processors
Radio frequency
Stochasticity
Training
Vectors (mathematics)
ISSN:2378-5861
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple processors to operate in a randomly chosen subset of blocks of the feature vector. We call the algorithm parallel stochastic because processors choose elements of the training set randomly and independently. Algorithms that are parallel in either of these dimensions exist, but RAPSA is the first attempt at a methodology that is parallel in both, the selection of blocks and the selection of elements of the training set. In RAPSA, processors utilize the randomly chosen functions to compute the stochastic gradient component associated with a randomly chosen block. The technical contribution of this paper is to show that this minimally coordinated algorithm converges to the optimal classifier when the training objective is convex. In particular, we show that: (i) When using decreasing stepsizes, RAPSA converges almost surely over the random choice of blocks and functions. (ii) When using constant stepsizes, convergence is to a neighborhood of optimality with a rate that is linear in expectation. RAPSA is numerically evaluated on the MNIST digit recognition problem.
Bibliografie:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Conference-1
ObjectType-Feature-3
content type line 23
SourceType-Conference Papers & Proceedings-2
ISSN:2378-5861
DOI:10.1109/ACC.2016.7526120