Zobrazit v EDS

Speaker-story mapping as a method to evaluate audiovisual scene analysis in a virtual classroom scenario.

Uloženo v:
Podrobná bibliografie
Název: Speaker-story mapping as a method to evaluate audiovisual scene analysis in a virtual classroom scenario.
Autoři: Fremerey, Stephan, Breuer, Carolin, Leist, Larissa, Klatte, Maria, Fels, Janina, Raake, Alexander
Zdroj: Frontiers in Psychology; 2025, p1-24, 24p
Témata: SHARED virtual environments, VIRTUAL classrooms, SIMULATOR sickness, COMPUTER-generated imagery, VIRTUAL reality
Abstrakt: This study explores how audiovisual immersive virtual environments (IVEs) can assess cognitive performance in classroom-like settings, addressing limitations in simpler acoustic and visual representations. This study examines the potential of a test paradigm using speaker-story mapping, called "audiovisual scene analysis (AV-SA)," originally developed for virtual reality (VR) hearing research, as a method to evaluate audiovisual scene analysis in a virtual classroom scenario. Factors affecting acoustic and visual scene representation were varied to investigate their impact on audiovisual scene analysis. Two acoustic representations were used: a simple "diotic" presentation where the same signal is presented to both ears, as well as a dynamically live-rendered binaural synthesis ("binaural"). Two visual representations were used: 360°/omnidirectional video with intrinsic lip-sync and computer-generated imagery (CGI) without lip-sync. Three subjective experiments were conducted with different combinations of the two acoustic and visual conditions: The first experiment, involving 36 participants, used 360° video with "binaural" audio. The second experiment, with 24 participants, combined 360° video with "diotic" audio. The third experiment, with 34 participants, used the CGI environment with "binaural" audio. Each environment presented 20 different speakers in a classroom-like circle of 20 chairs, with the number of simultaneously active speakers ranging from 2 to 10, while the remaining speakers kept silent and were always shown. During the experiments, the subjects' task was to correctly map the stories' topics to the corresponding speakers. The primary dependent variable was the number of correct assignments during a fixed period of 2 min, followed by two questionnaires on mental load after each trial. In addition, before and/or after the experiments, subjects needed to complete questionnaires about simulator sickness, noise sensitivity, and presence. Results indicate that the experimental condition significantly influenced task performance, mental load, and user behavior but did not affect perceived simulator sickness and presence. Performance decreased when comparing the 360° video and "binaural" audio experiment with either the experiment using "diotic" audio and 360°, or using "binaural" audio with CGI-based, showing the usefulness of the test method in investigating influences on cognitive audiovisual scene analysis performance. [ABSTRACT FROM AUTHOR]
Copyright of Frontiers in Psychology is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Databáze: Complementary Index
FullText Text:
  Availability: 0
CustomLinks:
  – Url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&db=pmc&term=1664-1078[TA]+AND+1[PG]+AND+2025[PDAT]
    Name: FREE - PubMed Central (ISSN based link)
    Category: fullText
    Text: Full Text
    Icon: https://imageserver.ebscohost.com/NetImages/iconPdf.gif
    MouseOverText: Check this PubMed for the article full text.
  – Url: https://resolver.ebscohost.com/openurl?sid=EBSCO:edb&genre=article&issn=16641078&ISBN=&volume=&issue=&date=20250624&spage=1&pages=1-24&title=Frontiers in Psychology&atitle=Speaker-story%20mapping%20as%20a%20method%20to%20evaluate%20audiovisual%20scene%20analysis%20in%20a%20virtual%20classroom%20scenario.&aulast=Fremerey%2C%20Stephan&id=DOI:10.3389/fpsyg.2025.1520630
    Name: Full Text Finder
    Category: fullText
    Text: Full Text Finder
    Icon: https://imageserver.ebscohost.com/branding/images/FTF.gif
    MouseOverText: Full Text Finder
  – Url: https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=EBSCO&SrcAuth=EBSCO&DestApp=WOS&ServiceName=TransferToWoS&DestLinkType=GeneralSearchSummary&Func=Links&author=Fremerey%20S
    Name: ISI
    Category: fullText
    Text: Nájsť tento článok vo Web of Science
    Icon: https://imagesrvr.epnet.com/ls/20docs.gif
    MouseOverText: Nájsť tento článok vo Web of Science
Header DbId: edb
DbLabel: Complementary Index
An: 186133382
RelevancyScore: 1041
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1040.79614257813
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Speaker-story mapping as a method to evaluate audiovisual scene analysis in a virtual classroom scenario.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Fremerey%2C+Stephan%22">Fremerey, Stephan</searchLink><br /><searchLink fieldCode="AR" term="%22Breuer%2C+Carolin%22">Breuer, Carolin</searchLink><br /><searchLink fieldCode="AR" term="%22Leist%2C+Larissa%22">Leist, Larissa</searchLink><br /><searchLink fieldCode="AR" term="%22Klatte%2C+Maria%22">Klatte, Maria</searchLink><br /><searchLink fieldCode="AR" term="%22Fels%2C+Janina%22">Fels, Janina</searchLink><br /><searchLink fieldCode="AR" term="%22Raake%2C+Alexander%22">Raake, Alexander</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: Frontiers in Psychology; 2025, p1-24, 24p
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22SHARED+virtual+environments%22">SHARED virtual environments</searchLink><br /><searchLink fieldCode="DE" term="%22VIRTUAL+classrooms%22">VIRTUAL classrooms</searchLink><br /><searchLink fieldCode="DE" term="%22SIMULATOR+sickness%22">SIMULATOR sickness</searchLink><br /><searchLink fieldCode="DE" term="%22COMPUTER-generated+imagery%22">COMPUTER-generated imagery</searchLink><br /><searchLink fieldCode="DE" term="%22VIRTUAL+reality%22">VIRTUAL reality</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study explores how audiovisual immersive virtual environments (IVEs) can assess cognitive performance in classroom-like settings, addressing limitations in simpler acoustic and visual representations. This study examines the potential of a test paradigm using speaker-story mapping, called "audiovisual scene analysis (AV-SA)," originally developed for virtual reality (VR) hearing research, as a method to evaluate audiovisual scene analysis in a virtual classroom scenario. Factors affecting acoustic and visual scene representation were varied to investigate their impact on audiovisual scene analysis. Two acoustic representations were used: a simple "diotic" presentation where the same signal is presented to both ears, as well as a dynamically live-rendered binaural synthesis ("binaural"). Two visual representations were used: 360°/omnidirectional video with intrinsic lip-sync and computer-generated imagery (CGI) without lip-sync. Three subjective experiments were conducted with different combinations of the two acoustic and visual conditions: The first experiment, involving 36 participants, used 360° video with "binaural" audio. The second experiment, with 24 participants, combined 360° video with "diotic" audio. The third experiment, with 34 participants, used the CGI environment with "binaural" audio. Each environment presented 20 different speakers in a classroom-like circle of 20 chairs, with the number of simultaneously active speakers ranging from 2 to 10, while the remaining speakers kept silent and were always shown. During the experiments, the subjects' task was to correctly map the stories' topics to the corresponding speakers. The primary dependent variable was the number of correct assignments during a fixed period of 2 min, followed by two questionnaires on mental load after each trial. In addition, before and/or after the experiments, subjects needed to complete questionnaires about simulator sickness, noise sensitivity, and presence. Results indicate that the experimental condition significantly influenced task performance, mental load, and user behavior but did not affect perceived simulator sickness and presence. Performance decreased when comparing the 360° video and "binaural" audio experiment with either the experiment using "diotic" audio and 360°, or using "binaural" audio with CGI-based, showing the usefulness of the test method in investigating influences on cognitive audiovisual scene analysis performance. [ABSTRACT FROM AUTHOR]
– Name: Abstract
  Label:
  Group: Ab
  Data: <i>Copyright of Frontiers in Psychology is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edb&AN=186133382
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.3389/fpsyg.2025.1520630
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 1
    Subjects:
      – SubjectFull: SHARED virtual environments
        Type: general
      – SubjectFull: VIRTUAL classrooms
        Type: general
      – SubjectFull: SIMULATOR sickness
        Type: general
      – SubjectFull: COMPUTER-generated imagery
        Type: general
      – SubjectFull: VIRTUAL reality
        Type: general
    Titles:
      – TitleFull: Speaker-story mapping as a method to evaluate audiovisual scene analysis in a virtual classroom scenario.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Fremerey, Stephan
      – PersonEntity:
          Name:
            NameFull: Breuer, Carolin
      – PersonEntity:
          Name:
            NameFull: Leist, Larissa
      – PersonEntity:
          Name:
            NameFull: Klatte, Maria
      – PersonEntity:
          Name:
            NameFull: Fels, Janina
      – PersonEntity:
          Name:
            NameFull: Raake, Alexander
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 24
              M: 06
              Text: 2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 16641078
          Titles:
            – TitleFull: Frontiers in Psychology
              Type: main
ResultId 1