Improvement of Lip Reading Performance in Real Environments Using Speaker and Environmental Adaptation

Lip reading technologies play a great role not only in image pattern recognition e.g. computer vision, but also in audio-visual pattern recognition e.g. bimodal speech recognition. However, it is a problem that the recognition accuracy is still significantly low, compared to that of speech recogniti...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings - IEEE Computer Society Conference on Pattern Recognition and Image Processing s. 346 - 350
Hlavní autoři: Kawasaki, Takuya, Ukai, Naoya, Takumi, Seko, Tamura, Satoshi, Hayamizu, Satoru
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.01.2013
Témata:
Accuracy
Adaptation models
environmental adaptation
Hidden Markov models
lip reading
real environment
speaker adaptation
Speech recognition
Training
Vectors
Visualization
ISSN:0730-6512
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Lip reading technologies play a great role not only in image pattern recognition e.g. computer vision, but also in audio-visual pattern recognition e.g. bimodal speech recognition. However, it is a problem that the recognition accuracy is still significantly low, compared to that of speech recognition. Another problem lies which the performance degradation occurs in real environments. To improve the performance, in this paper we employ two adaptation schemes: speaker adaptation and environmental adaptation. The speaker adaptation is performed to recognition models so as to prevent the degradation caused by the difference of speakers. The environmental adaptation is also conducted to deal with environmental differences. We tested these adaptation schemes using a real-world audio-visual corpus CENSREC-2-AV, we have built this corpus containing real-world data (speech signals and lip images) recorded in a driving car, in which subjects uttered Japanese connected digits. Experimental results show that the lip reading recognition performance was largely improved by the speaker adaptation, and further recovered by the environmental adaptation.
ISSN:0730-6512
DOI:10.1109/ACPR.2013.73