Conversational dynamics in social virtual reality: A large-scale, longitudinal study of speech acts and nonverbal behavior - Santoso, Monique; Wang, Portia; Han, Eugy; Bailenson, Jeremy N. (Computers in human behavior, 2025)
This study explores verbal interactions in social virtual reality (VR). We developed the Virtual Reality Interaction Dynamics Scheme (VRIDS), which includes 10 speech acts (e.g., questioning, opinions, disagreements) by integrating existing speech frameworks with new constructs from our data. Analyzing speech from 109 participants in a metaverse classroom over four weeks, we coded 9738 discourse units. VRIDS introduced new constructs like context-dependent commentary on virtual objects, which increased as users became more familiar with the technology over time. Idea sharing also increased over time, indicating enhanced collaboration. We identified attractor and repeller states through speech act sequences and an “echoing” strategy. Additionally, we examined the link between nonverbal behavior and speech acts; head and hand movements increased during questioning and context-dependent commentary but decreased during disagreements. Our database includes transcripts, speech acts, and nonverbal data. Future work will train large language models to recognize speech acts in VR. •We coded over 9000 speech acts into 10 categories to study interaction patterns in social VR.•Speech act sequences showed attractor-repellor states and revealed an "echoing" strategy in social VR interactions.•Head and hand movements increased with questions and comments but decreased during disagreements in social VR.