Brain Prediction of Auditory Emphasis by Facial Expressions During Audiovisual Continuous Speech

The visual cues involved in auditory speech processing are not restricted to information from lip movements but also include head or chin gestures and facial expressions such as eyebrow movements. The fact that visual gestures precede the auditory signal implicates that visual information may influe...

Full description

Saved in:
Bibliographic Details
Published in:Brain topography Vol. 28; no. 3; pp. 494 - 505
Main Authors: Strelnikov, Kuzma, Foxton, Jessica, Marx, Mathieu, Barone, Pascal
Format: Journal Article
Language:English
Published: New York Springer US 01.05.2015
Springer Nature B.V
Springer Verlag
Subjects:
Adult
Anticipation, Psychological - physiology
Biomedical and Life Sciences
Biomedicine
Brain - physiology
Cognitive science
Electroencephalography
Facial Expression
Facial Recognition - physiology
Female
Humans
Male
Neurology
Neuroscience
Neurosciences
Original Paper
Psychiatry
Speech Perception - physiology
Young Adult
Predictive coding
Prosody
Audio-visual speech
Mismatch
ISSN:0896-0267, 1573-6792, 1573-6792
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The visual cues involved in auditory speech processing are not restricted to information from lip movements but also include head or chin gestures and facial expressions such as eyebrow movements. The fact that visual gestures precede the auditory signal implicates that visual information may influence the auditory activity. As visual stimuli are very close in time to the auditory information for audiovisual syllables, the cortical response to them usually overlaps with that for the auditory stimulation; the neural dynamics underlying the visual facilitation for continuous speech therefore remain unclear. In this study, we used a three-word phrase to study continuous speech processing. We presented video clips with even ( without emphasis ) phrases as the frequent stimuli and with one word visually emphasized by the speaker as the non-frequent stimuli. Negativity in the resulting ERPs was detected after the start of the emphasizing articulatory movements but before the auditory stimulus, a finding that was confirmed by the statistical comparisons of the audiovisual and visual stimulation. No such negativity was present in the control visual-only condition. The propagation of this negativity was observed between the visual and fronto-temporal electrodes. Thus, in continuous speech, the visual modality evokes predictive coding for the auditory speech, which is analysed by the cerebral cortex in the context of the phrase even before the arrival of the corresponding auditory signal.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0896-0267
1573-6792
1573-6792
DOI:10.1007/s10548-013-0338-2