Multimodal information fusion using the iterative decoding algorithm and its application to audio-visual speech recognition

The fusion of information from heterogenous sensors is crucial to the effectiveness of a multimodal system. Noise affect the sensors of different modalities independently. A good fusion scheme should be able to use local estimates of the reliability of each modality to weight the decisions. This pap...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2008 IEEE International Conference on Acoustics, Speech and Signal Processing S. 2241 - 2244
Hauptverfasser: Shivappa, S.T., Rao, B.D., Trivedi, M.M.
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.03.2008
Schlagworte:
Acoustic noise
Application software
Drives
Feature extraction
Hidden Markov models
Iterative algorithms
Iterative decoding
Multimedia systems
Multimodal sensors
Robustness
Sensor fusion
Speech recognition
ISBN:9781424414833, 1424414830
ISSN:1520-6149
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The fusion of information from heterogenous sensors is crucial to the effectiveness of a multimodal system. Noise affect the sensors of different modalities independently. A good fusion scheme should be able to use local estimates of the reliability of each modality to weight the decisions. This paper presents an iterative decoding based information fusion scheme motivated by the theory of turbo codes. This fusion framework is developed in the context of hidden Markov models. We present the mathematical framework of the fusion scheme. We then apply this algorithm to an audio-visual speech recognition task on the GRID audio-visual speech corpus and present the results.
ISBN:9781424414833
1424414830
ISSN:1520-6149
DOI:10.1109/ICASSP.2008.4518091