Modified Gauss-Newton Algorithms under Noise

Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing. Their nonsmooth counterparts, modified Gauss-Newton or prox-linear algorithms, can lead to contrasting outcomes when compared to gradient descent in large-scale statistical settings. We...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE Statistical Signal Processing Workshop s. 51 - 55
Hlavní autoři: Pillutla, Krishna, Roulet, Vincent, Kakade, Sham M., Harchaoui, Zaid
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 02.07.2023
Témata:
Composite problems
Conferences
Convergence
Gauss-Newton
Machine learning
Machine learning algorithms
Nonsmooth
Prediction algorithms
Signal processing
Signal processing algorithms
ISSN:2693-3551
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing. Their nonsmooth counterparts, modified Gauss-Newton or prox-linear algorithms, can lead to contrasting outcomes when compared to gradient descent in large-scale statistical settings. We explore the contrasting performance of these two classes of algorithms in theory on a stylized statistical example, and experimentally on learning problems including structured prediction. In theory, we delineate the regime where the quadratic convergence of the modified Gauss-Newton method is active under statistical noise. In the experiments, we underline the versatility of stochastic (sub)-gradient descent to minimize nonsmooth composite objectives.
ISSN:2693-3551
DOI:10.1109/SSP53291.2023.10207977