PARS-Push: Personalized, Asynchronous and Robust Decentralized Optimization

We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula> agents seeks to find a common point that enables "few-shot" learning (personalization) via loc...

Full description

Saved in:
Bibliographic Details
Published in:IEEE control systems letters Vol. 7; pp. 361 - 366
Main Authors: Toghani, Mohammad Taha, Lee, Soomin, Uribe, Cesar A.
Format: Journal Article
Language:English
Published: IEEE 2023
Subjects:
Asynchronous communication
asynchronous communications
Convergence
Costs
Decentralized optimization
Delays
few-shot learning
model-agnostic meta-learning
Optimization
personalization
Stochastic processes
Task analysis
ISSN:2475-1456, 2475-1456
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of <inline-formula> <tex-math notation="LaTeX">n </tex-math></inline-formula> agents seeks to find a common point that enables "few-shot" learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push under arbitrary multi-step personalization for smooth strongly convex, and smooth non-convex functions. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.
ISSN:2475-1456
2475-1456
DOI:10.1109/LCSYS.2022.3189317