Multimodal adaptive social interaction in virtual environment (MASI-VR) for children with Autism spectrum disorders (ASD)

Difficulties in social interaction, verbal and non-verbal communications as well as repetitive and atypical patterns of behavior, characterizes Autism spectrum disorders (ASD). A number of studies indicated that many children with ASD prefer technology and this preference can be explored to develop...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the Workshop on Future Trends of Distributed Computing Systems pp. 121 - 130
Main Authors: Bekele, E., Wade, J., Bian, D., Fan, J., Swanson, Amy, Warren, Z., Sarkar, N.
Format: Conference Proceeding
Language:English
Published: IEEE 01.03.2016
Subjects:
3D social Interaction
Autism
Biomedical monitoring
Context
Electroencephalography
Emotion recognition
Engines
Face
multimodal interaction
psychology
usability
VR-based adaptive systems
ISSN:2375-5334
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Difficulties in social interaction, verbal and non-verbal communications as well as repetitive and atypical patterns of behavior, characterizes Autism spectrum disorders (ASD). A number of studies indicated that many children with ASD prefer technology and this preference can be explored to develop systems that may alleviate several challenges of traditional treatment and intervention. As a result, recent advances in computer and robotic technology are ushering in innovative assistive technologies for ASD intervention. The current work presents design, development and a usability study of an adaptive multimodal virtual reality-based social interaction platform for children with ASD. It is hypothesized that endowing a technological system that can detect the processing pattern and mental state of the child using implicit cues from eye tracking and electrophysiological, including peripheral physiological and electroencephalography (EEG), signals and adapt its interaction accordingly is of great importance in assisting and individualizing traditional intervention approaches. The presented VR system is based on a virtual reality based social environment, a school cafeteria, where an individual with ASD interacts with virtual characters. An eye tracker, an EEG monitor and biosensors to measure peripheral electrophysiological signals are integrated with the VR task environment to obtain gaze, EEG signals and several peripheral physiological signals in real-time. In the current work, we show how eye gaze and task performance can be used in real-time to adapt intervention in VR. The other signals are collected for offline analysis. The results from a usability study with 12 subjects with ASD are presented to demonstrate the viability of the proposed concepts within the VR system.
AbstractList Difficulties in social interaction, verbal and non-verbal communications as well as repetitive and atypical patterns of behavior, characterizes Autism spectrum disorders (ASD). A number of studies indicated that many children with ASD prefer technology and this preference can be explored to develop systems that may alleviate several challenges of traditional treatment and intervention. As a result, recent advances in computer and robotic technology are ushering in innovative assistive technologies for ASD intervention. The current work presents design, development and a usability study of an adaptive multimodal virtual reality-based social interaction platform for children with ASD. It is hypothesized that endowing a technological system that can detect the processing pattern and mental state of the child using implicit cues from eye tracking and electrophysiological, including peripheral physiological and electroencephalography (EEG), signals and adapt its interaction accordingly is of great importance in assisting and individualizing traditional intervention approaches. The presented VR system is based on a virtual reality based social environment, a school cafeteria, where an individual with ASD interacts with virtual characters. An eye tracker, an EEG monitor and biosensors to measure peripheral electrophysiological signals are integrated with the VR task environment to obtain gaze, EEG signals and several peripheral physiological signals in real-time. In the current work, we show how eye gaze and task performance can be used in real-time to adapt intervention in VR. The other signals are collected for offline analysis. The results from a usability study with 12 subjects with ASD are presented to demonstrate the viability of the proposed concepts within the VR system.
Author Warren, Z.
Bekele, E.
Sarkar, N.
Bian, D.
Swanson, Amy
Fan, J.
Wade, J.
Author_xml – sequence: 1
  givenname: E.
  surname: Bekele
  fullname: Bekele, E.
  organization: Electr. Eng. & Comput. Sci. Dept., Vanderbilt Univ., Nashville, TN, USA
– sequence: 2
  givenname: J.
  surname: Wade
  fullname: Wade, J.
  organization: Electr. Eng. & Comput. Sci. Dept., Vanderbilt Univ., Nashville, TN, USA
– sequence: 3
  givenname: D.
  surname: Bian
  fullname: Bian, D.
  organization: Electr. Eng. & Comput. Sci. Dept., Vanderbilt Univ., Nashville, TN, USA
– sequence: 4
  givenname: J.
  surname: Fan
  fullname: Fan, J.
  organization: Electr. Eng. & Comput. Sci. Dept., Vanderbilt Univ., Nashville, TN, USA
– sequence: 5
  givenname: Amy
  surname: Swanson
  fullname: Swanson, Amy
  organization: Treatment & Res. in Autism Spectrum Disorder, Vanderbilt Univ., Nashville, TN, USA
– sequence: 6
  givenname: Z.
  surname: Warren
  fullname: Warren, Z.
  organization: Treatment & Res. in Autism Spectrum Disorder, Vanderbilt Univ., Nashville, TN, USA
– sequence: 7
  givenname: N.
  surname: Sarkar
  fullname: Sarkar, N.
  organization: Electr. Eng. & Comput. Sci. Dept., Vanderbilt Univ., Nashville, TN, USA
BookMark eNotkD1vwjAYhN2qlVooc4cuHmEIfW3HcTxG9AsJVAlaVmQcW7giDrIdKv59I5XpnrvhhmeAbnzrDUKPBKaEgHzerKYUSDEVHPJC8is0kqIkHCRAyQq4RveUCZ5xxvI7NIjxB4ACZ_IenZfdIbmmrdUBq1odkzsZHFvt-u58MkHp5FrfMz65kLp-Nr6n1jfGJzxeVut5tllNsG0D1nt3qIPx-NelPa665GKD49HoFLoG1y62oTYh4nG1fpk8oFurDtGMLjlE32-vX7OPbPH5Pp9Vi8xRKFPGLAgrZSFVDbQQ3ApGBZTcqpppK4mmtjdQcuCa7HQutBJ8l2tLlJBEKcqG6On_1xljtsfgGhXO24sp9gfY9V8I
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/VR.2016.7504695
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE/IET Electronic Library
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Psychology
Computer Science
EISBN 9781509008360
1509008365
EISSN 2375-5334
EndPage 130
ExternalDocumentID 7504695
Genre orig-research
GroupedDBID 29P
6IE
6IH
ALMA_UNASSIGNED_HOLDINGS
CBEJK
RIE
RIO
RNS
ID FETCH-LOGICAL-i208t-3f07f9969ad02675f7327085fad3cf91c2f1098505c1bc47ca75b4cf1a791aa23
IEDL.DBID RIE
ISICitedReferencesCount 58
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000386307300014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:09:32 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i208t-3f07f9969ad02675f7327085fad3cf91c2f1098505c1bc47ca75b4cf1a791aa23
PageCount 10
ParticipantIDs ieee_primary_7504695
PublicationCentury 2000
PublicationDate 20160301
PublicationDateYYYYMMDD 2016-03-01
PublicationDate_xml – month: 03
  year: 2016
  text: 20160301
  day: 01
PublicationDecade 2010
PublicationTitle Proceedings of the Workshop on Future Trends of Distributed Computing Systems
PublicationTitleAbbrev VR
PublicationYear 2016
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0020539
ssib026765277
Score 2.2256925
Snippet Difficulties in social interaction, verbal and non-verbal communications as well as repetitive and atypical patterns of behavior, characterizes Autism spectrum...
SourceID ieee
SourceType Publisher
StartPage 121
SubjectTerms 3D social Interaction
Autism
Biomedical monitoring
Context
Electroencephalography
Emotion recognition
Engines
Face
multimodal interaction
psychology
usability
VR-based adaptive systems
Title Multimodal adaptive social interaction in virtual environment (MASI-VR) for children with Autism spectrum disorders (ASD)
URI https://ieeexplore.ieee.org/document/7504695
WOSCitedRecordID wos000386307300014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LawIxEB5UevBkq5a-mUMPCl3dZB8xR2kr7aEi2oo3ySYbEOqDtQr--ya7q7bQS28hhLCZzCYzk_m-AbhnnhRhGAmHKeo7PpG-E3maO8rYDop5HWNF6LTYBOv3O5MJHxTg4YCFieM4TT6LW7aZvuWrpdzYUFnbUpGHPChCkTGWYbX2ukNDFgaUHZ0to1w8p_IhLm-PhzaNK2zlU_yqpZJeJb3K_z7iFOpHTB4ODrfNGRTiRRUq-6IMmP-jVSgfjrRdDXYpvna-VOIThRIre7ZhFiZHSxSRZLAG08btLLFQEvyBfMPGW3f06oyHTTSmLe5x32hjt9g1GrueY4rUTDZzVDmN5xob3dFTsw4fvef3xxcnL7bgzKjb-XI87TJtnB8ulBVmoJlHmbHHtFCe1JxIqo0cO8ZgkiSSPpOCBZEvNRGMEyGodw6lxXIRXwBSSYgZHGjNIvuuy90ocKnxC1XgSjPtJdSsWKerjE9jmkv06u_uayjbncvyvm6gZNYU38KJ3JpVJnepEnwDu62ygQ
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LawIxEB6sLdSTrVr6bg49KHTtJvvI5ihtRamKqBVvkk02INQHWgX_fZPdVVvopbcQQthMZpOZyXzfADxSR3DfD7lFJXEtFwvXCh3FLKltB0mdQFsRKi42QTudYDRi3Qw87bEwURTFyWdR1TTjt3w5F2sTKns2VOQ-847g2HNdghO01k57iE99j9CDu6XVi6VkPthmz8OeSeTyq-kkv6qpxJdJPf-_zziD0gGVh7r7--YcMtGsAPldWQaU_qUFyO0PtW0RtjHCdjqX_BNxyRfmdENJoBwZqohlAmzQbbSZLA2YBP3AvqFyu9ZvWsNeBWnjFu2Q38hEb1FN6-xqimKs5nI9RTIl8lyhcq3_WinBR_1t8NKw0nIL1oTYwZflKJsq7f4wLo0wPUUdQrVFprh0hGJYEKXlGGiTSeBQuFRw6oWuUJhThjknzgVkZ_NZdAmICIz1YE8pGpqXXWaHnk20Zyg9W-hpr6BoxDpeJIwa41Si1393P8BpY9BujVvNzvsN5MwuJllgt5DV64vu4ERs9IqX97FCfAMwFrXI
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+of+the+Workshop+on+Future+Trends+of+Distributed+Computing+Systems&rft.atitle=Multimodal+adaptive+social+interaction+in+virtual+environment+%28MASI-VR%29+for+children+with+Autism+spectrum+disorders+%28ASD%29&rft.au=Bekele%2C+E.&rft.au=Wade%2C+J.&rft.au=Bian%2C+D.&rft.au=Fan%2C+J.&rft.date=2016-03-01&rft.pub=IEEE&rft.eissn=2375-5334&rft.spage=121&rft.epage=130&rft_id=info:doi/10.1109%2FVR.2016.7504695&rft.externalDocID=7504695