Modality Weighting in Multimodal Variational Autoencoders

Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighti...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Innovations in Intelligent Systems and Applications Conference (Online) s. 1 - 6
Hlavní autori: Yegin, Melike Nur, Amasyali, Mehmet Fatih
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 07.09.2022
Predmet:
Intelligent systems
Jacobian matrices
Lenses
Machine learning
Mixture of experts
Multimodal learning
Multimodal variational autoencoders
Optimization
Product of experts
Reliability
Technological innovation
Variational autoencoders
ISSN:2770-7946
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighting the reliable modalities in multimodal learning process. We introduce both weighting with fixed coefficients and several methods for weighting automatically. We compare the proposed methods with classical methods on artificial Gaussian data sets and MNIST/FashionMNIST data sets and we have obtained significiant results.
ISSN:2770-7946
DOI:10.1109/ASYU56188.2022.9925305