Doubly random parallel stochastic methods for large scale learning
We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple...
Gespeichert in:
| Veröffentlicht in: | Proceedings of the American Control Conference S. 4847 - 4852 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Tagungsbericht Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
American Automatic Control Council (AACC)
01.07.2016
|
| Schlagworte: | |
| ISSN: | 2378-5861 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple processors to operate in a randomly chosen subset of blocks of the feature vector. We call the algorithm parallel stochastic because processors choose elements of the training set randomly and independently. Algorithms that are parallel in either of these dimensions exist, but RAPSA is the first attempt at a methodology that is parallel in both, the selection of blocks and the selection of elements of the training set. In RAPSA, processors utilize the randomly chosen functions to compute the stochastic gradient component associated with a randomly chosen block. The technical contribution of this paper is to show that this minimally coordinated algorithm converges to the optimal classifier when the training objective is convex. In particular, we show that: (i) When using decreasing stepsizes, RAPSA converges almost surely over the random choice of blocks and functions. (ii) When using constant stepsizes, convergence is to a neighborhood of optimality with a rate that is linear in expectation. RAPSA is numerically evaluated on the MNIST digit recognition problem. |
|---|---|
| AbstractList | We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple processors to operate in a randomly chosen subset of blocks of the feature vector. We call the algorithm parallel stochastic because processors choose elements of the training set randomly and independently. Algorithms that are parallel in either of these dimensions exist, but RAPSA is the first attempt at a methodology that is parallel in both, the selection of blocks and the selection of elements of the training set. In RAPSA, processors utilize the randomly chosen functions to compute the stochastic gradient component associated with a randomly chosen block. The technical contribution of this paper is to show that this minimally coordinated algorithm converges to the optimal classifier when the training objective is convex. In particular, we show that: (i) When using decreasing stepsizes, RAPSA converges almost surely over the random choice of blocks and functions. (ii) When using constant stepsizes, convergence is to a neighborhood of optimality with a rate that is linear in expectation. RAPSA is numerically evaluated on the MNIST digit recognition problem. |
| Author | Mokhtari, Aryan Koppel, Alec Ribeiro, Alejandro |
| Author_xml | – sequence: 1 givenname: Aryan surname: Mokhtari fullname: Mokhtari, Aryan email: aryanm@seas.upenn.edu organization: Dept. of Electr. & Syst. Eng., Univ. of Pennsylvania, Philadelphia, PA, USA – sequence: 2 givenname: Alec surname: Koppel fullname: Koppel, Alec email: akoppel@seas.upenn.edu organization: Dept. of Electr. & Syst. Eng., Univ. of Pennsylvania, Philadelphia, PA, USA – sequence: 3 givenname: Alejandro surname: Ribeiro fullname: Ribeiro, Alejandro email: aribeiro@seas.upenn.edu organization: Dept. of Electr. & Syst. Eng., Univ. of Pennsylvania, Philadelphia, PA, USA |
| BookMark | eNotkD1PwzAURQ0CiaawI7F4ZEnxs-PEHkv4lCqxwBy9pM9tkBMXOx3676nUTvcMR2e4Gbsaw0iM3YNYAAj7tKzrhRRQLiotS5DigmVQlJUypZHiks2kqkyuTQk3LEvpVwiwthQz9vwS9q0_8IjjOgx8hxG9J8_TFLotpqnv-EDTNqwTdyFyj3FDPHXoiXvCOPbj5pZdO_SJ7s47Zz9vr9_1R776ev-sl6u8l8JMudSu6Cy6CjRiYQl1oZxCMIhHPHKlsOxQAICzrZRFZwrSqkWxdta1oObs8dTdxfC3pzQ1Q5868h5HCvvUgFFa28ro6qg-nNSeiJpd7AeMh-Z8jfoHt7BZdw |
| ContentType | Conference Proceeding Journal Article |
| DBID | 6IE 6IH CBEJK RIE RIO 7SP 8FD L7M |
| DOI | 10.1109/ACC.2016.7526120 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Xplore IEEE Proceedings Order Plans (POP) 1998-present Electronics & Communications Abstracts Technology Research Database Advanced Technologies Database with Aerospace |
| DatabaseTitle | Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 1467386820 9781467386807 9781467386821 1467386804 |
| EISSN | 2378-5861 |
| EndPage | 4852 |
| ExternalDocumentID | 7526120 |
| Genre | orig-research |
| GroupedDBID | -~X 23M 29O 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ AFFNX ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO RNS 7SP 8FD L7M |
| ID | FETCH-LOGICAL-i208t-25f4c9af715aa49ea543f3a18aaa5443f73a6ca0111f9b224c84e53ba0df9fb13 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 9 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000388376104152&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Thu Jul 10 23:39:14 EDT 2025 Wed Aug 27 02:11:24 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i208t-25f4c9af715aa49ea543f3a18aaa5443f73a6ca0111f9b224c84e53ba0df9fb13 |
| Notes | ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Conference-1 ObjectType-Feature-3 content type line 23 SourceType-Conference Papers & Proceedings-2 |
| PQID | 1835597857 |
| PQPubID | 23500 |
| PageCount | 6 |
| ParticipantIDs | proquest_miscellaneous_1835597857 ieee_primary_7526120 |
| PublicationCentury | 2000 |
| PublicationDate | 20160701 |
| PublicationDateYYYYMMDD | 2016-07-01 |
| PublicationDate_xml | – month: 07 year: 2016 text: 20160701 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | Proceedings of the American Control Conference |
| PublicationTitleAbbrev | ACC |
| PublicationYear | 2016 |
| Publisher | American Automatic Control Council (AACC) |
| Publisher_xml | – name: American Automatic Control Council (AACC) |
| SSID | ssj0019960 |
| Score | 1.6796154 |
| Snippet | We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve... |
| SourceID | proquest ieee |
| SourceType | Aggregation Database Publisher |
| StartPage | 4847 |
| SubjectTerms | Algorithms Convergence Learning Linear programming Mathematical analysis Optimization Parallel processing Processors Program processors Radio frequency Stochasticity Training Vectors (mathematics) |
| Title | Doubly random parallel stochastic methods for large scale learning |
| URI | https://ieeexplore.ieee.org/document/7526120 https://www.proquest.com/docview/1835597857 |
| WOSCitedRecordID | wos000388376104152&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB1q8aAXP1qxfrGCR9M22d1sctRi8SClB5XewiaZ1UJMpGkF_707SVoFvXjLZSHMhH1vMjPvAVx5tqAWWklH-pw7Io1dJ0YpHSPRApKwpU8lu_j8oCaTYDYLpy243uzCIGI1fIZ9eqx6-WmRrOhX2UBJEryyBfqWUn69q7XpGJDKyLoNOQwHN6MRzW35_eZMY57y68atYGS8978X2Ifu9z4em26Q5gBamB_C7g8pwQ7cWiYcZ5_MQk9avDFS9M4yzJjldsmrJjFmVptFl8zSVJbRADgrbYKQNcYRL114Gt89ju6dxh_BmXvDYOl40ogk1Ea5UmsRopaCG67dQGtNsnZGce0nmtzkTRhbrE4CgZLHepia0MQuP4J2XuR4DEyFZoiWKnHuGRKYsazEj3WgBBEyF70edCga0XstgRE1gejB5Tqckf0sqdegcyxWZWRvCqpVAqlO_j56CjuUn3ry9Qzay8UKz2E7-VjOy8VFldsv_mykcg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEF1KFdSLH61YP1fwaNoku5tNjlosFWvpoUpvYZPOaiEm0rSC_96dJK2CXrzlshBmwr43mZn3CLlyTUHNlRSW8Biz-DRyrAiEsLQAA0jclD6F7OLzQA6H_mQSjGrker0LAwDF8Bm08bHo5U-zeIm_yjpSoOCVKdA3BOeuXW5rrXsGqDOyakTaQeem28XJLa9dnarsU37duQWQ9Hb_9wp7pPm9kUdHa6zZJzVID8jODzHBBrk1XDhKPqkBn2n2RlHTO0kgoYbdxa8K5ZhpaRedU0NUaYIj4DQ3KQJaWUe8NMlT727c7VuVQ4I1c21_YblC8zhQWjpCKR6AEpxpphxfKYXCdloy5cUK_eR1EBm0jn0OgkXKnupARw47JPU0S-GIUBloGwxZYszVKDFjeIkXKV9ypGQOuC3SwGiE76UIRlgFokUuV-EMzYeJ3QaVQrbMQ3NXYLXiC3n899ELstUfPw7Cwf3w4YRsY67KOdhTUl_Ml3BGNuOPxSyfnxd5_gK5Sae5 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+of+the+American+Control+Conference&rft.atitle=Doubly+random+parallel+stochastic+methods+for+large+scale+learning&rft.au=Mokhtari%2C+Aryan&rft.au=Koppel%2C+Alec&rft.au=Ribeiro%2C+Alejandro&rft.date=2016-07-01&rft.pub=American+Automatic+Control+Council+%28AACC%29&rft.eissn=2378-5861&rft.spage=4847&rft.epage=4852&rft_id=info:doi/10.1109%2FACC.2016.7526120&rft.externalDocID=7526120 |