Modality Weighting in Multimodal Variational Autoencoders

Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighti...

Full description

Saved in:
Bibliographic Details
Published in:Innovations in Intelligent Systems and Applications Conference (Online) pp. 1 - 6
Main Authors: Yegin, Melike Nur, Amasyali, Mehmet Fatih
Format: Conference Proceeding
Language:English
Published: IEEE 07.09.2022
Subjects:
Intelligent systems
Jacobian matrices
Lenses
Machine learning
Mixture of experts
Multimodal learning
Multimodal variational autoencoders
Optimization
Product of experts
Reliability
Technological innovation
Variational autoencoders
ISSN:2770-7946
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Learning the same subject from different sources is known as multimodal learning in the machine learning literature. In this approach some of the modalities may be more reliable than others and we may want to learn them more to a better understanding of the subject. In this study, we propose weighting the reliable modalities in multimodal learning process. We introduce both weighting with fixed coefficients and several methods for weighting automatically. We compare the proposed methods with classical methods on artificial Gaussian data sets and MNIST/FashionMNIST data sets and we have obtained significiant results.
ISSN:2770-7946
DOI:10.1109/ASYU56188.2022.9925305