Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 May 23:9:855819.
doi: 10.3389/frobt.2022.855819. eCollection 2022.

A Music-Therapy Robotic Platform for Children With Autism: A Pilot Study

Huanghao Feng  1 , Mohammad H Mahoor  2 , Francesca Dino  2
Affiliations

A Music-Therapy Robotic Platform for Children With Autism: A Pilot Study

Huanghao Feng et al. Front Robot AI. .

Erratum in

Abstract

Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills including motor control, turn-taking, and emotion recognition. Innovative technology, such as socially assistive robots, has shown to be a viable method for Autism therapy. This paper presents a novel robot-based music-therapy platform for modeling and improving the social responses and behaviors of children with ASD. Our autonomous social interactive system consists of three modules. Module one provides an autonomous initiative positioning system for the robot, NAO, to properly localize and play the instrument (Xylophone) using the robot's arms. Module two allows NAO to play customized songs composed by individuals. Module three provides a real-life music therapy experience to the users. We adopted Short-time Fourier Transform and Levenshtein distance to fulfill the design requirements: 1) "music detection" and 2) "smart scoring and feedback", which allows NAO to understand music and provide additional practice and oral feedback to the users as applicable. We designed and implemented six Human-Robot-Interaction (HRI) sessions including four intervention sessions. Nine children with ASD and seven Typically Developing participated in a total of fifty HRI experimental sessions. Using our platform, we collected and analyzed data on social behavioral changes and emotion recognition using Electrodermal Activity (EDA) signals. The results of our experiments demonstrate most of the participants were able to complete motor control tasks with 70% accuracy. Six out of the nine ASD participants showed stable turn-taking behavior when playing music. The results of automated emotion classification using Support Vector Machines illustrates that emotional arousal in the ASD group can be detected and well recognized via EDA bio-signals. In summary, the results of our data analyses, including emotion classification using EDA signals, indicate that the proposed robot-music based therapy platform is an attractive and promising assistive tool to facilitate the improvement of fine motor control and turn-taking skills in children with ASD.

Keywords: autism; emotion classification; motor control; music therapy; social robotics; turn-taking.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

FIGURE 1
FIGURE 1
Mallet griper.
FIGURE 2
FIGURE 2
Instrument stand front view.
FIGURE 3
FIGURE 3
Experiment room.
FIGURE 4
FIGURE 4
Experiment session illustration.
FIGURE 5
FIGURE 5
The distribution of the targeted emotions across all subjects and events.
FIGURE 6
FIGURE 6
Block diagram of Module-based acoustic music interactive system.
FIGURE 7
FIGURE 7
Color detection from NAO’s bottom camera: (A) single blue color detection (B) full instrument color detection (C) color based edge detection.
FIGURE 8
FIGURE 8
Melody detection with Short Time Fourier Transform.
FIGURE 9
FIGURE 9
Motor control accuracy result.
FIGURE 10
FIGURE 10
Main music therapy performance accuracy.
FIGURE 11
FIGURE 11
Normalized turn-taking behavior result for all subjects during intervention sessions.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Anzalone S. M., Boucenna S., Ivaldi S., Chetouani M. (2015). Evaluating the Engagement with Social Robots. Int J Soc Robotics 7, 465–478. 10.1007/s12369-015-0298-7 - DOI
    1. Anzalone S. M., Tilmont E., Boucenna S., Xavier J., Jouen A.-L., Bodeau N. (2014). How Children with Autism Spectrum Disorder Behave and Explore the 4-dimensional (Spatial 3d+ Time) Environment during a Joint Attention Induction Task with a Robot. Res. Autism Spectr. Disord. 8, 814–826. 10.1016/j.rasd.2014.03.002 - DOI
    1. Askari F., Feng H., Sweeny T. D., Mahoor M. H. (2018). “A Pilot Study on Facial Expression Recognition Ability of Autistic Children Using Ryan, a Rear-Projected Humanoid Robot,” in 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN IEEE; ), 790–795. 10.1109/roman.2018.8525825 - DOI
    1. Association A. P. (2000). Diagnostic and Statistical Manual of Mental Disorders. Dsm-iv 272, 828–829. 10.1001/jama.1994.03520100096046 - DOI
    1. Beer J. M., Boren M., Liles K. R. (2016). “Robot Assisted Music Therapy a Case Study with Children Diagnosed with Autism,” in 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (IEEE; ), 419–420. 10.1109/hri.2016.7451785 - DOI

LinkOut - more resources