View in EDS

Semantic-based audiovisual integration enhances active storage in visual working memory.

Saved in:
Bibliographic Details
Title: Semantic-based audiovisual integration enhances active storage in visual working memory.
Authors: Zhao S; Department of Psychology, School of Education, Soochow University, Suzhou, Jiangsu 215123, China., Hui S; Department of Psychology, School of Education, Soochow University, Suzhou, Jiangsu 215123, China., Chen M; Department of Psychology, School of Education, Soochow University, Suzhou, Jiangsu 215123, China., Feng C; Department of Psychology, School of Education, Soochow University, Suzhou, Jiangsu 215123, China., Zhai M; Department of Psychology, School of Education, Soochow University, Suzhou, Jiangsu 215123, China. Electronic address: zhaimengdie0312@163.com., Feng W; Department of Psychology, School of Education, Soochow University, Suzhou, Jiangsu 215123, China; Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou, Jiangsu 215123, China. Electronic address: fengwfly@gmail.com.
Source: NeuroImage [Neuroimage] 2025 Dec 01; Vol. 323, pp. 121577. Date of Electronic Publication: 2025 Nov 08.
Publication Type: Journal Article
Language: English
Journal Info: Publisher: Academic Press Country of Publication: United States NLM ID: 9215515 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1095-9572 (Electronic) Linking ISSN: 10538119 NLM ISO Abbreviation: Neuroimage Subsets: MEDLINE
Imprint Name(s): Original Publication: Orlando, FL : Academic Press, c1992-
MeSH Terms: Memory, Short-Term*/physiology , Semantics* , Auditory Perception*/physiology , Evoked Potentials*/physiology , Visual Perception*/physiology, Humans ; Male ; Female ; Adult ; Young Adult ; Electroencephalography
Abstract: Presenting a semantically congruent sound during visual target encoding can facilitate subsequent short-term memory retrieval, which is generally considered as a beneficial effect of multisensory integration on working memory. However, there remains a fundamental doubt about whether this benefit in performance is truly due to semantic-based multisensory integration rendering information about the target object stored more precisely in working memory, or simply due to better activation of the object's archetypal template preexisting in long-term memory. The present event-related potential (ERP) study investigated this issue in humans with a fine-grained visual object recognition task to prevent any stronger activation of the object's internal template from affecting performance. Meanwhile, a retro-cue was introduced to enable the dissociation of the ERPs reflecting audiovisual interactions and the contralateral delay activity (CDA) indexing active visual working memory maintenance. Our Experiment 1 unveiled that the congruent-sound-induced benefit to recognition performance was accompanied by an increased CDA, both of which could be predicted by an early ERP correlate of semantic-specific audiovisual interaction. Experiment 2 further revealed that these behavioral and electrophysiological effects were present only when the congruent sound was delivered synchronously with, but not prior to, the memory items, invalidating cross-modal semantic priming as an alternative account. These findings offer the first direct evidence that multisensory semantic integration enables details about the target object itself to be stored more precisely in working memory, hence the improved short-term memory performance, which establishes a theoretical foundation for improving working memory with multisensory integration.
(Copyright © 2025. Published by Elsevier Inc.)
Contributed Indexing: Keywords: Active storage; CDA; ERPs; Multisensory integration; Semantic congruency; Working memory
Entry Date(s): Date Created: 20251110 Date Completed: 20251205 Latest Revision: 20251205
Update Code: 20251206
DOI: 10.1016/j.neuroimage.2025.121577
PMID: 41213442
Database: MEDLINE
Description
Abstract:Presenting a semantically congruent sound during visual target encoding can facilitate subsequent short-term memory retrieval, which is generally considered as a beneficial effect of multisensory integration on working memory. However, there remains a fundamental doubt about whether this benefit in performance is truly due to semantic-based multisensory integration rendering information about the target object stored more precisely in working memory, or simply due to better activation of the object's archetypal template preexisting in long-term memory. The present event-related potential (ERP) study investigated this issue in humans with a fine-grained visual object recognition task to prevent any stronger activation of the object's internal template from affecting performance. Meanwhile, a retro-cue was introduced to enable the dissociation of the ERPs reflecting audiovisual interactions and the contralateral delay activity (CDA) indexing active visual working memory maintenance. Our Experiment 1 unveiled that the congruent-sound-induced benefit to recognition performance was accompanied by an increased CDA, both of which could be predicted by an early ERP correlate of semantic-specific audiovisual interaction. Experiment 2 further revealed that these behavioral and electrophysiological effects were present only when the congruent sound was delivered synchronously with, but not prior to, the memory items, invalidating cross-modal semantic priming as an alternative account. These findings offer the first direct evidence that multisensory semantic integration enables details about the target object itself to be stored more precisely in working memory, hence the improved short-term memory performance, which establishes a theoretical foundation for improving working memory with multisensory integration.<br /> (Copyright © 2025. Published by Elsevier Inc.)
ISSN:1095-9572
DOI:10.1016/j.neuroimage.2025.121577