skip to main content
10.1145/3608298.3608309acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmhiConference Proceedingsconference-collections
research-article

Predicting Autism from Head Movement Patterns during Naturalistic Social Interactions

Published: 18 October 2023 Publication History
  • Get Citation Alerts
  • Abstract

    Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized in part by difficulties in verbal and nonverbal social communication. Evidence indicates that autistic people, compared to neurotypical peers, exhibit differences in head movements, a key form of nonverbal communication. Despite the crucial role of head movements in social communication, research on this nonverbal cue is relatively scarce compared to other forms of nonverbal communication, such as facial expressions and gestures. There is a need for scalable, reliable, and accurate instruments for measuring head movements directly within the context of social interactions. In this study, we used computer vision and machine learning to examine the head movement patterns of neurotypical and autistic individuals during naturalistic, face–to–face conversations, at both the individual (monadic) and interpersonal (dyadic) levels. Our model predicts diagnostic status using dyadic head movement data with an accuracy of, highlighting the value of head movement as a marker of social communication. The monadic data pipeline had lower accuracy () compared to the dyadic approach, emphasizing the importance of studying back-and-forth social communication cues within a true social context. The proposed classifier is not intended for diagnostic purposes, and future research should replicate our findings in larger, more representative samples.

    References

    [1]
    Shamma Alokla. 2018. Non-verbal communication skills of children with autism spectrum disorder. (2018).
    [2]
    Wael Alshurman and Ihsani Alsreaa. 2015. The Efficiency of Peer Teaching of Developing Non Verbal Communication to Children with Autism Spectrum Disorder (ASD).Journal of Education and Practice 6, 29 (2015), 33–38.
    [3]
    American Psychiatric Association 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-V)(5th ed.).
    [4]
    Joseph N Cappella. 1981. Mutual influence in expressive behavior: Adult–adult and infant–adult dyadic interaction.Psychological bulletin 89, 1 (1981), 101.
    [5]
    David Curto, Albert Clapés, Javier Selva, Sorina Smeureanu, Julio Junior, CS Jacques, David Gallardo-Pujol, Georgina Guilera, David Leiva, Thomas B Moeslund, 2021. Dyadformer: A multi-modal transformer for long-range modeling of dyadic interactions. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2177–2188.
    [6]
    Starkey Duncan. 1972. Some signals and rules for taking speaking turns in conversations.Journal of personality and social psychology 23, 2 (1972), 283.
    [7]
    Joanne E Flanagan, Rebecca Landa, Anjana Bhat, and Margaret Bauman. 2012. Head lag in infants at risk for autism: a preliminary study. The American Journal of Occupational Therapy 66, 5 (2012), 577–585.
    [8]
    Alexandra Livia Georgescu, Jana Christina Koehler, Johanna Weiske, Kai Vogeley, Nikolaos Koutsouleris, and Christine Falter-Wagner. 2019. Machine learning to study social interaction difficulties in ASD. Frontiers in Robotics and AI 6 (2019), 132.
    [9]
    Alexandra Livia Georgescu, Bojana Kuzmanovic, Daniel Roth, Gary Bente, and Kai Vogeley. 2014. The use of virtual characters to assess and train non-verbal communication in high-functioning autism. Frontiers in human neuroscience 8 (2014), 807.
    [10]
    Hirotaka Gima, Hideki Kihara, Hama Watanabe, Hisako Nakano, Junji Nakano, Yukuo Konishi, Tomohiko Nakamura, and Gentaro Taga. 2018. Early motor signs of autism spectrum disorder in spontaneous position and movement of the head. Experimental brain research 236, 4 (2018), 1139–1148.
    [11]
    Karl Grammer, Kirsten B Kruck, and Magnus S Magnusson. 1998. The courtship dance: Patterns of nonverbal synchronization in opposite-sex encounters. Journal of Nonverbal behavior 22, 1 (1998), 3–29.
    [12]
    Kylie Megan Gray and Bruce John Tonge. 2001. Are there early features of autism in infants and preschool children?Journal of paediatrics and child health 37, 3 (2001), 221–226.
    [13]
    Joanna Hale, Jamie A Ward, Francesco Buccheri, Dominic Oliver, and Antonia F de C Hamilton. 2020. Are you on my wavelength? Interpersonal coordination in dyadic conversations. Journal of nonverbal behavior 44, 1 (2020), 63–83.
    [14]
    Judith A Hall, Terrence G Horgan, and Nora A Murphy. 2019. Nonverbal communication. Annual review of psychology 70, 1 (2019), 271–294.
    [15]
    Mohammed Akram Hamdan. 2018. Developing a Proposed Training Program Based on Discrete Trial Training (DTT) to Improve the Non-Verbal Communication Skills in Children with Autism Spectrum Disorder (ASD).International Journal of Special Education 33, 3 (2018), 579–591.
    [16]
    Zakia Hammal, Jeffrey F Cohn, and David T George. 2014. Interpersonal coordination of headmotion in distressed couples. IEEE transactions on affective computing 5, 2 (2014), 155–167.
    [17]
    Zakia Hammal, Jeffrey F Cohn, and Daniel S Messinger. 2015. Head movement dynamics during play and perturbed mother-infant interaction. IEEE transactions on affective computing 6, 4 (2015), 361–370.
    [18]
    Hifza Javed, WonHyong Lee, and Chung Hyuk Park. 2020. Toward an automated measure of social engagement for children with autism spectrum disorder—a personalized computational modeling approach. Frontiers in Robotics and AI (2020), 43.
    [19]
    Rana el Kaliouby and Peter Robinson. 2005. Real-time inference of complex mental states from facial expressions and head gestures. In Real-time vision for human-computer interaction. Springer, 181–200.
    [20]
    Pradeep Raj Krishnappa Babu, J Matias Di Martino, Zhuoqing Chang, Sam Perochon, Rachel Aiello, Kimberly LH Carpenter, Scott Compton, Naomi Davis, Lauren Franz, Steven Espinosa, 2022. Complexity analysis of head movements in autistic toddlers. Journal of Child Psychology and Psychiatry (2022).
    [21]
    Steven R Livingstone and Caroline Palmer. 2016. Head movements encode emotions during speech and song.Emotion 16, 3 (2016), 365.
    [22]
    Katherine B Martin, Zakia Hammal, Gang Ren, Jeffrey F Cohn, Justine Cassell, Mitsunori Ogihara, Jennifer C Britton, Anibal Gutierrez, and Daniel S Messinger. 2018. Objective measurement of head movement differences in children with and without autism spectrum disorder. Molecular autism 9, 1 (2018), 1–10.
    [23]
    Evelyn Z McClave. 2000. Linguistic functions of head movements in the context of speech. Journal of pragmatics 32, 7 (2000), 855–878.
    [24]
    Denisa Qori McDonald, Casey J Zampella, Evangelos Sariyanidi, Aashvi Manakiwala, Ellis DeJardin, John D Herrington, Robert T Schultz, and Birkan Tunc. 2022. Head Movement Patterns during Face-to-Face Conversations Vary with Age. In Companion Publication of the 2022 International Conference on Multimodal Interaction. 185–195.
    [25]
    Peter Mundy, Marian Sigman, Judy Ungerer, and Tracy Sherman. 1986. Defining the social deficits of autism: The contribution of non-verbal communication measures. Journal of child psychology and psychiatry 27, 5 (1986), 657–669.
    [26]
    Ruth CM Philip, Heather C Whalley, Andrew C Stanfield, R Sprengelmeyer, Isabel M Santos, Andrew W Young, Anthony P Atkinson, AJ Calder, EC Johnstone, SM Lawrie, 2010. Deficits in facial, body movement and vocal emotional processing in autism spectrum disorders. Psychological medicine 40, 11 (2010), 1919–1929.
    [27]
    Sara Ramos-Cabo, Valentin Vulchanov, and Mila Vulchanova. 2019. Gesture and language trajectories in early development: An overview from the autism spectrum disorder perspective. Frontiers in Psychology 10 (2019), 1211.
    [28]
    Allison B Ratto, Lauren Turner-Brown, Betty M Rupp, Gary B Mesibov, and David L Penn. 2011. Development of the contextual assessment of social skills (CASS): A role play measure of social skill for individuals with high-functioning autism. Journal of autism and developmental disorders 41, 9 (2011), 1277–1286.
    [29]
    Evangelos Sariyanidi, Casey J Zampella, Robert T Schultz, and Birkan Tunc. 2020. Inequality-constrained and robust 3D face model fitting. In European Conference on Computer Vision. Springer, 433–449.
    [30]
    Robert T Schultz, Isabel Gauthier, Ami Klin, Robert K Fulbright, Adam W Anderson, Fred Volkmar, Pawel Skudlarski, Cheryl Lacadie, Donald J Cohen, and John C Gore. 2000. Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and Asperger syndrome. Archives of general Psychiatry 57, 4 (2000), 331–340.
    [31]
    Atsushi Senju and Mark H Johnson. 2009. Atypical eye contact in autism: models, mechanisms and development. Neuroscience & Biobehavioral Reviews 33, 8 (2009), 1204–1214.
    [32]
    Marian Sigman, Peter Mundy, Tracy Sherman, and Judy Ungerer. 1986. Social interactions of autistic, mentally retarded and normal children and their caregivers. Journal of child psychology and psychiatry 27, 5 (1986), 647–656.
    [33]
    Josef Sivic and Andrew Zisserman. 2003. Video Google: A text retrieval approach to object matching in videos. In Computer Vision, IEEE International Conference on, Vol. 3. IEEE Computer Society, 1470–1470.
    [34]
    Josef Sivic and Andrew Zisserman. 2008. Efficient visual search of videos cast as text retrieval. IEEE transactions on pattern analysis and machine intelligence 31, 4 (2008), 591–606.
    [35]
    Leslie L Speer, Anne E Cook, William M McMahon, and Elaine Clark. 2007. Face processing in children with autism: Effects of stimulus contents and type. Autism 11, 3 (2007), 265–277.
    [36]
    Judith S Verhoeven, Paul De Cock, Lieven Lagae, and Stefan Sunaert. 2010. Neuroimaging of autism. Neuroradiology 52, 1 (2010), 3–14.
    [37]
    Sarah Weigelt, Kami Koldewyn, and Nancy Kanwisher. 2012. Face identity recognition in autism spectrum disorders: A review of behavioral studies. Neuroscience & Biobehavioral Reviews 36, 3 (2012), 1060–1084.
    [38]
    Zhong Zhao, Zhipeng Zhu, Xiaobin Zhang, Haiming Tang, Jiayi Xing, Xinyao Hu, Jianping Lu, Qiongling Peng, and Xingda Qu. 2021. Atypical Head Movement during Face-to-Face Interaction in Children with Autism Spectrum Disorder. Autism Research 14, 6 (2021), 1197–1208.
    [39]
    Zhong Zhao, Zhipeng Zhu, Xiaobin Zhang, Haiming Tang, Jiayi Xing, Xinyao Hu, Jianping Lu, and Xingda Qu. 2022. Identifying autism with head movement features by implementing machine learning algorithms. Journal of Autism and Developmental Disorders 52, 7 (2022), 3038–3049.

    Cited By

    View all
    • (2024)Ethical considerations for integrating multimodal computer perception and neurotechnologyFrontiers in Human Neuroscience10.3389/fnhum.2024.133245118Online publication date: 16-Feb-2024

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICMHI '23: Proceedings of the 2023 7th International Conference on Medical and Health Informatics
    May 2023
    386 pages
    ISBN:9798400700712
    DOI:10.1145/3608298
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 October 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. bag-of-words approach
    2. behavioral analysis
    3. conversation analysis
    4. dyadic features
    5. head movement patterns
    6. monadic features
    7. non-verbal communication
    8. video analysis

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    Conference

    ICMHI 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)63
    • Downloads (Last 6 weeks)2

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Ethical considerations for integrating multimodal computer perception and neurotechnologyFrontiers in Human Neuroscience10.3389/fnhum.2024.133245118Online publication date: 16-Feb-2024

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media