research-article

vrSocial: Toward Immersive Therapeutic VR Systems for Children with Autism

Authors:

LouAnne E. Boyd,

Sagar B. Vikmani,

Carlos M. Gutierrez,

Erik Linstead, and

Gillian R. HayesAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

Paper No.: 204, Pages 1 - 12

https://doi.org/10.1145/3173574.3173778

Published: 21 April 2018 Publication History

Abstract

Social communication frequently includes nuanced nonverbal communication cues, including eye contact, gestures, facial expressions, body language, and tone of voice. This type of communication is central to face-to-face interaction, but can be challenging for children and adults with autism. Innovative technologies can provide support by augmenting human-delivered cuing and automated prompting. Specifically, immersive virtual reality (VR) offers an option to generalize social skill interventions by concretizing nonverbal information in real-time social interactions. In this work, we explore the design and evaluation of three nonverbal communication applications in immersive VR. The results of this work indicate that delivering real-time visualizations of proximity, speaker volume, and duration of one's speech is feasible in immersive VR and effective for real-time support for proximity regulation for children with autism. We conclude with design considerations for therapeutic VR systems.

References

[1]

Alyssa Alcorn, Helen Pain, Gnanathusharan Rajendran, Tim Smith, Oliver Lemon, Kaska Porayska-Pomsta, Mary Ellen Foster, Katerina Avramides, Christopher Frauenberger, and Sara Bernardini. 2011. Social Communication between Virtual Characters and Children with Autism. In Artificial Intelligence in Education, Gautam Biswas, Susan Bull, Judy Kay and Antonija Mitrovic (eds.). Springer Berlin Heidelberg, 7--14.

Digital Library

[2]

Yoselie Alvarado, Claudia Ruth Gatica, Graciela Verónica Gil Costa, and Roberto A. Guerrero. 2016. Evaluating the spoken dialogue system of a conversational character: A Simulation Study.

[3]

David H. Barlow and Steven C. Hayes. 1979. Alternating Treatments Design: One Strategy for Comparing the Effects of Two Treatments in a Single Subject. Journal of Applied Behavior Analysis 12, 2: 199--210.

[4]

Kathy Baxter, Catherine Courage, and Kelly Caine. 2015. Understanding Your Users: A Practical Guide to User Research Methods. Morgan Kaufmann.

Digital Library

[5]

E. Bekele, Z. Zheng, A. Swanson, J. Crittendon, Z. Warren, and N. Sarkar. 2013. Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments. IEEE Transactions on Visualization and Computer Graphics 19, 4: 711--720.

Digital Library

[6]

Tony Bergstrom and Karrie Karahalios. 2007. Seeing More: Visualizing Audio Cues. In Human-Computer Interaction - INTERACT 2007 (Lecture Notes in Computer Science), 29--42.

Digital Library

[7]

Olga Bogdashina. 2016. Sensory Perceptual Issues in Autism and Asperger Syndrome, Second Edition: Different Sensory Experiences - Different Perceptual Worlds. Jessica Kingsley Publishers.

[8]

LouAnne E. Boyd, Xinlong Jiang, and Gillian R. Hayes. 2017. ProCom: Designing and Evaluating a Mobile and Wearable System to Support Proximity Awareness for People with Autism. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI'17), 2865--2877.

Digital Library

[9]

LouAnne E. Boyd, Alejandro Rangel, Helen Tomimbang, Andrea Conejo-Toledo, Kanika Patel, Monica Tentori, and Gillian R. Hayes. 2016. SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4872--4883.

Digital Library

[10]

LouAnne Boyd, Christina McReynolds, and Karen Chanin. 2013. The Social Compass Curriculum: a story-based intervention package for students with autism spectrum disorders. Brookes Publishing Co.

[11]

Moira Burke, Robert Kraut, and Diane Williams. 2010. Social use of computer-mediated communication by adults on the autism spectrum. In Proceedings of the 2010 ACM conference on Computer supported cooperative work, 425--434.

Digital Library

[12]

Enrico Costanza, Samual Inverso, Elan Pavlov, Rebecca Allen, and Pattie Maes. eye-q: Eyeglass Peripheral Display for Subtle Intimate Notifications. In Proceedings of the 2006 ACM conference on Human-Computer Interaction with Mobile Devices and Services, 211--218.

Digital Library

[13]

Nyaz Didehbani, Tandra Allen, Michelle Kandalaft, Daniel Krawczyk, and Sandra Chapman. 2016. Virtual Reality Social Cognition Training for children with high functioning autism. Computers in Human Behavior 62: 703--711.

Digital Library

[14]

Judith Donath, Karrie Karahalios, and Fernanda Viégas. 1999. Visualizing Conversation. Journal of Computer-Mediated Communication 4, 4: 0-0.

[15]

A. Field. 2008. Research methods in psychology: Repeated measures ANOVA using SPSS. East Sussex, UK.

[16]

Paul A. G. Forbes, Xueni Pan, and Antonia F. de C. Hamilton. 2016. Reduced Mimicry to Virtual Reality Avatars in Autism Spectrum Disorder. Journal of Autism and Developmental Disorders 46, 12: 3788--3797.

[17]

Peter F. Gerhardt and Ilene Lainer. 2011. Addressing the Needs of Adolescents and Adults with Autism: A Crisis on the Horizon. Journal of Contemporary Psychotherapy 41, 1: 37--45.

[18]

N. Haber, C. Voss, A. Fazel, T. Winograd, and D. P. Wall. 2016. A practical approach to real-time neutral feature subtraction for facial expression recognition. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 1--9.

[19]

Edward Twitchell Hall. 1966. The hidden dimension (1st ed.). Doubleday & Co, New York, NY, US.

[20]

Alan E. Kazdin. 2011. Single-case research designs: Methods for clinical and applied settings (2nd ed.). Oxford University Press, New York, NY, US.

[21]

Julie A. Kientz, Matthew S. Goodwin, Gillian R. Hayes, and Gregory D. Abowd. 2014. Interactive Technologies for Autism. Morgan & Claypool Publishers.

Digital Library

[22]

Mark L. Knapp, Judith A. Hall, and Terrence G. Horgan. 2013. Nonverbal Communication in Human Interaction. Cengage Learning.

[23]

Jennifer Mankoff, Gillian R. Hayes, and Devva Kasnitz. 2010. Disability studies as a source of critical inquiry for the field of assistive technology. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility, 3--10.

Digital Library

[24]

Esma Mansouri Benssassi, Juan-Carlos Gomez, LouAnne E. Boyd, Gillian R. Hayes, and Juan Ye. 2017. Wearable assistive technologies for autism: opportunities and challenges. IEEE Pervasive Computing.

[25]

Susan Dickerson Mayes, Angela A. Gorman, Jolene Hillwig-Garcia, and Ehsan Syed. 2013. Suicide ideation and attempts in children with autism. Research in Autism Spectrum Disorders 7, 1: 109--119.

[26]

Peter Mundy, Marian Sigman, Judy Ungerer, and Tracy Sherman. 1986. Defining the Social Deficits of Autism: The Contribution of Non-Verbal Communication Measures. Journal of Child Psychology and Psychiatry 27, 5: 657--669.

[27]

Brenda Smith Myles, Melissa Trautman, and Ronda L Schelvan. 2004. The hidden curriculum: practical solutions for understanding unstated rules in social situations. Autism Asperger Pub. Co., Shawnee Mission, KS.

[28]

Sarah Parsons and Sue Cobb. 2011. State-of-the-art of virtual reality technologies for children on the autism spectrum. European Journal of Special Needs Education 26, 3: 355--366.

[29]

Jenny Preece, Helen Sharp, and Yvonne Rogers. 2015. Interaction Design: Beyond Human-Computer Interaction. Forth Edition. John Wiley & Sons, West Sussex, United Kingdom.

[30]

Kathryn E. Ringland, Christine T. Wolf, LouAnne E. Boyd, Mark S. Baldwin, and Gillian R. Hayes. 2016. Would You Be Mine: Appropriating Minecraft As an Assistive Technology for Youth with Autism. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16), 33--41.

Digital Library

[31]

Kathryn E. Ringland, Christine T. Wolf, Lynn Dombrowski, and Gillian R. Hayes. 2015. Making "Safe": Community-Centered Practices in a Virtual World Dedicated to Children with Autism. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW'15), 1788--1800.

Digital Library

[32]

Kathryn E. Ringland, Christine T. Wolf, Heather Faucett, Lynn Dombrowski, and Gillian R. Hayes. 2016. "Will I Always Be Not Social?": Re-Conceptualizing Sociality in the Context of a Minecraft Community for Autism. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 1256--1269.

Digital Library

[33]

A. A. Rizzo and S. Koenig. 2017. Is clinical virtual reality ready for primetime. Neuropsychology.

[34]

Hanna Bertilsdotter Rosqvist, Charlotte Brownlow, and Lindsay O'Dell. 2013. Mapping the social geographies of autism - online and off-line narratives of neuro-shared and separate spaces. Disability & Society 28, 3: 367--379.

[35]

Dan Saffer. 2013. Microinteractions: Designing with Details. O'Reilly Media Inc.

Digital Library

[36]

M. Iftekhar Tanveer, Emy Lin, and Mohammed (Ehsan) Hoque. 2015. Rhema: A Real-Time In-Situ Intelligent Interface to Help People with Public Speaking. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI '15), 286--295.

Digital Library

[37]

Andrea Tartaro and Justine Cassell. 2006. Authorable virtual peers for autism spectrum disorders. In Proceedings of the Combined workshop on Language-Enabled Educational Technology and Development and Evaluation for Robust Spoken Dialogue Systems at the 17th European Conference on Artificial Intellegence.

[38]

Monica Tentori and Gillian R. Hayes. 2010. Designing for interaction immediacy to enhance social skills of children with autism. 51.

[39]

Anthony C. Thiselton. 1996. The two horizons: New Testament hermeneutics and philosophical description with special reference to Heidegger, Bultmann, Gadamer, and Wittgenstein. Wm. B. Eerdmans Publishing.

[40]

Catalin Voss, Peter Washington, Nick Haber, Aaron Kline, Jena Daniels, Azar Fazel, Titas De, Beth McCarthy, Carl Feinstein, Terry Winograd, and Dennis Wall. 2016. Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp '16), 1218--1226.

Digital Library

Cited By

Dumaru PHackler BFlood AAl-Ameen M(2024)“I feel like he’s looking in the computer world to be social, but I can’t trust his judgement”: Reimagining Parental Control for Children with ASDProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642696(1-25)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642696
Alabood LDow TFeeley KJaswal VKrishnamurthy D(2024)From Letterboards to Holograms: Advancing Assistive Technology for Nonspeaking Autistic Individuals with the HoloBoardProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642626(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642626
Kim WSeong MKim KKim S(2024)Engagnition: A multi-dimensional dataset for engagement recognition of children with autism spectrum disorderScientific Data10.1038/s41597-024-03132-311:1Online publication date: 15-Mar-2024
https://doi.org/10.1038/s41597-024-03132-3
Show More Cited By

Index Terms

vrSocial: Toward Immersive Therapeutic VR Systems for Children with Autism
1. Social and professional topics

Recommendations

Talking points: the differential impact of real-time computer generated audio/visual feedback on speech-like & non-speech-like vocalizations in low functioning children with ASD
Assets '09: Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility

Real-time computer feedback systems (CFS) have been shown to impact the communication of neurologically typical individuals. Promising new research appears to suggest the same for the vocalization of low functioning children with Autistic Spectrum ...
Read More
Autistic Children's Use of Technology and Media: A Fieldwork Study
IDC '17: Proceedings of the 2017 Conference on Interaction Design and Children

This qualitative field study conducted at a school for autistic children aimed to explain how low and medium functioning autistic children use technology at school. Additionally, it explored education professionals and parents' attitudes and concerns ...
Read More
vrSensory: Designing Inclusive Virtual Games with Neurodiverse Children
CHI PLAY '19 Extended Abstracts: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts

We explore virtual environments and accompanying interaction styles to enable inclusive play. In designing games for three neurodiverse children, we explore how designing for sensory diversity can be understood through a formal game design framework. Our ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
1,266
Total Downloads

Downloads (Last 12 months)135
Downloads (Last 6 weeks)9

Other Metrics

View Author Metrics

Citations

Cited By

Dumaru PHackler BFlood AAl-Ameen M(2024)“I feel like he’s looking in the computer world to be social, but I can’t trust his judgement”: Reimagining Parental Control for Children with ASDProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642696(1-25)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642696
Alabood LDow TFeeley KJaswal VKrishnamurthy D(2024)From Letterboards to Holograms: Advancing Assistive Technology for Nonspeaking Autistic Individuals with the HoloBoardProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642626(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642626
Kim WSeong MKim KKim S(2024)Engagnition: A multi-dimensional dataset for engagement recognition of children with autism spectrum disorderScientific Data10.1038/s41597-024-03132-311:1Online publication date: 15-Mar-2024
https://doi.org/10.1038/s41597-024-03132-3
Acharya MMohbey K(2024)Virtual Environment Role in Higher Education Students Learning Enhancement with Intellectual DisabilitiesApplied Assistive Technologies and Informatics for Students with Disabilities10.1007/978-981-97-0914-4_3(35-57)Online publication date: 29-May-2024
https://doi.org/10.1007/978-981-97-0914-4_3
Façanha AJúnior JAraújo MSantos JSánchez JViana W(2024)Empowering Orientation and Mobility Instructors: Digital Tools for Enhancing Navigation Skills in People Who Are BlindComputers Helping People with Special Needs10.1007/978-3-031-62846-7_52(436-443)Online publication date: 5-Jul-2024
https://doi.org/10.1007/978-3-031-62846-7_52
Boyd LBoyd L(2024)Timing is Everything: Temporal Processing and MultiSensory IntegrationThe Sensory Accommodation Framework for Technology10.1007/978-3-031-48843-6_7(101-111)Online publication date: 2-Jan-2024
https://doi.org/10.1007/978-3-031-48843-6_7
Boyd LBoyd L(2024)From Sensory Perception to Realtime NonVerbal CommunicationThe Sensory Accommodation Framework for Technology10.1007/978-3-031-48843-6_6(85-99)Online publication date: 2-Jan-2024
https://doi.org/10.1007/978-3-031-48843-6_6
Boyd LBoyd L(2024)Technologies Across the Disciplines for Autistic UsersThe Sensory Accommodation Framework for Technology10.1007/978-3-031-48843-6_2(13-26)Online publication date: 2-Jan-2024
https://doi.org/10.1007/978-3-031-48843-6_2
Lowy RLee CAbowd GKim J(2023)Building Causal Agency in Autistic Students through Iterative Reflection in Collaborative Transition PlanningProceedings of the ACM on Human-Computer Interaction10.1145/36100377:CSCW2(1-26)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610037
Williams RBoyd LGilbert J(2023)Counterventions: a reparative reflection on interventionist HCIProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581480(1-11)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581480
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents