PARS-Push: Personalized, Asynchronous and Robust Decentralized Optimization

We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula> agents seeks to find a common point that enables "few-shot" learning (personalization) via loc...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE control systems letters Jg. 7; S. 361 - 366
Hauptverfasser: Toghani, Mohammad Taha, Lee, Soomin, Uribe, Cesar A.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: IEEE 2023
Schlagworte:
Asynchronous communication
asynchronous communications
Convergence
Costs
Decentralized optimization
Delays
few-shot learning
model-agnostic meta-learning
Optimization
personalization
Stochastic processes
Task analysis
ISSN:2475-1456, 2475-1456
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula> agents seeks to find a common point that enables "few-shot" learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push under arbitrary multi-step personalization for smooth strongly convex, and smooth non-convex functions. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.
ISSN:2475-1456
2475-1456
DOI:10.1109/LCSYS.2022.3189317