Modality Weighting in Multimodal Variational Autoencoders

Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighti...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Innovations in Intelligent Systems and Applications Conference (Online) s. 1 - 6
Hlavní autoři: Yegin, Melike Nur, Amasyali, Mehmet Fatih
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 07.09.2022
Témata:
Intelligent systems
Jacobian matrices
Lenses
Machine learning
Mixture of experts
Multimodal learning
Multimodal variational autoencoders
Optimization
Product of experts
Reliability
Technological innovation
Variational autoencoders
ISSN:2770-7946
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighting the reliable modalities in multimodal learning process. We introduce both weighting with fixed coefficients and several methods for weighting automatically. We compare the proposed methods with classical methods on artificial Gaussian data sets and MNIST/FashionMNIST data sets and we have obtained significiant results.
ISSN:2770-7946
DOI:10.1109/ASYU56188.2022.9925305