Multiple-source adaptation theory and algorithms

We present a general theoretical and algorithmic analysis of the problem of multiple-source adaptation, a key learning problem in applications. We derive new normalized solutions with strong theoretical guarantees for the cross-entropy loss and other similar losses. We also provide new guarantees th...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Annals of mathematics and artificial intelligence Jg. 89; H. 3-4; S. 237 - 270
Hauptverfasser: Zhang, Ningshan, Mohri, Mehryar, Hoffman, Judy
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Cham Springer International Publishing 01.03.2021
Springer
Springer Nature B.V
Schlagworte:
Adaptation
Algorithms
Analysis
Artificial Intelligence
Complex Systems
Computational linguistics
Computer Science
Empirical analysis
Entropy (Information theory)
Language processing
Mathematics
Natural language interfaces
Domain adaptation
DC programming
Rényi divergence
68T05
Multiple-source adaptation
Transfer learning
ISSN:1012-2443, 1573-7470
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We present a general theoretical and algorithmic analysis of the problem of multiple-source adaptation, a key learning problem in applications. We derive new normalized solutions with strong theoretical guarantees for the cross-entropy loss and other similar losses. We also provide new guarantees that hold in the case where the conditional probabilities for the source domains are distinct. We further present a novel analysis of the convergence properties of density estimation used in distribution-weighted combinations, and study their effects on the learning guarantees. Moreover, we give new algorithms for determining the distribution-weighted combination solution for the cross-entropy loss and other losses. We report the results of a series of experiments with real-world datasets. We find that our algorithm outperforms competing approaches by producing a single robust predictor that performs well on any target mixture distribution. Altogether, our theory, algorithms, and empirical results provide a full solution for the multiple-source adaptation problem with very practical benefits.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1012-2443
1573-7470
DOI:10.1007/s10472-020-09716-0