PARS-Push: Personalized, Asynchronous and Robust Decentralized Optimization

We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula> agents seeks to find a common point that enables "few-shot" learning (personalization) via loc...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE control systems letters Ročník 7; s. 361 - 366
Hlavní autoři: Toghani, Mohammad Taha, Lee, Soomin, Uribe, Cesar A.
Médium: Journal Article
Jazyk:angličtina
Vydáno: IEEE 2023
Témata:
Asynchronous communication
asynchronous communications
Convergence
Costs
Decentralized optimization
Delays
few-shot learning
model-agnostic meta-learning
Optimization
personalization
Stochastic processes
Task analysis
ISSN:2475-1456, 2475-1456
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula> agents seeks to find a common point that enables "few-shot" learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push under arbitrary multi-step personalization for smooth strongly convex, and smooth non-convex functions. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.
ISSN:2475-1456
2475-1456
DOI:10.1109/LCSYS.2022.3189317