Modality Weighting in Multimodal Variational Autoencoders

Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighti...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Innovations in Intelligent Systems and Applications Conference (Online) S. 1 - 6
Hauptverfasser: Yegin, Melike Nur, Amasyali, Mehmet Fatih
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 07.09.2022
Schlagworte:
Intelligent systems
Jacobian matrices
Lenses
Machine learning
Mixture of experts
Multimodal learning
Multimodal variational autoencoders
Optimization
Product of experts
Reliability
Technological innovation
Variational autoencoders
ISSN:2770-7946
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighting the reliable modalities in multimodal learning process. We introduce both weighting with fixed coefficients and several methods for weighting automatically. We compare the proposed methods with classical methods on artificial Gaussian data sets and MNIST/FashionMNIST data sets and we have obtained significiant results.
ISSN:2770-7946
DOI:10.1109/ASYU56188.2022.9925305