Mobile detection of autism through machine learning on home video: A development and prospective validation study
- PMID: 30481180
- PMCID: PMC6258501
- DOI: 10.1371/journal.pmed.1002705
Mobile detection of autism through machine learning on home video: A development and prospective validation study
Abstract
Background: The standard approaches to diagnosing autism spectrum disorder (ASD) evaluate between 20 and 100 behaviors and take several hours to complete. This has in part contributed to long wait times for a diagnosis and subsequent delays in access to therapy. We hypothesize that the use of machine learning analysis on home video can speed the diagnosis without compromising accuracy. We have analyzed item-level records from 2 standard diagnostic instruments to construct machine learning classifiers optimized for sparsity, interpretability, and accuracy. In the present study, we prospectively test whether the features from these optimized models can be extracted by blinded nonexpert raters from 3-minute home videos of children with and without ASD to arrive at a rapid and accurate machine learning autism classification.
Methods and findings: We created a mobile web portal for video raters to assess 30 behavioral features (e.g., eye contact, social smile) that are used by 8 independent machine learning models for identifying ASD, each with >94% accuracy in cross-validation testing and subsequent independent validation from previous work. We then collected 116 short home videos of children with autism (mean age = 4 years 10 months, SD = 2 years 3 months) and 46 videos of typically developing children (mean age = 2 years 11 months, SD = 1 year 2 months). Three raters blind to the diagnosis independently measured each of the 30 features from the 8 models, with a median time to completion of 4 minutes. Although several models (consisting of alternating decision trees, support vector machine [SVM], logistic regression (LR), radial kernel, and linear SVM) performed well, a sparse 5-feature LR classifier (LR5) yielded the highest accuracy (area under the curve [AUC]: 92% [95% CI 88%-97%]) across all ages tested. We used a prospectively collected independent validation set of 66 videos (33 ASD and 33 non-ASD) and 3 independent rater measurements to validate the outcome, achieving lower but comparable accuracy (AUC: 89% [95% CI 81%-95%]). Finally, we applied LR to the 162-video-feature matrix to construct an 8-feature model, which achieved 0.93 AUC (95% CI 0.90-0.97) on the held-out test set and 0.86 on the validation set of 66 videos. Validation on children with an existing diagnosis limited the ability to generalize the performance to undiagnosed populations.
Conclusions: These results support the hypothesis that feature tagging of home videos for machine learning classification of autism can yield accurate outcomes in short time frames, using mobile devices. Further work will be needed to confirm that this approach can accelerate autism diagnosis at scale.
Conflict of interest statement
I have read the journal's policy and the authors of this manuscript have the following competing interests: DPW is the scientific founder of Cognoa, a company focused on digital pediatric healthcare; the approach and findings presented in this paper are independent from/not related to Cognoa. All other authors have declared no competing interests exist.
Figures
Similar articles
-
Detecting Developmental Delay and Autism Through Machine Learning Models Using Home Videos of Bangladeshi Children: Development and Validation Study.J Med Internet Res. 2019 Apr 24;21(4):e13822. doi: 10.2196/13822. J Med Internet Res. 2019. PMID: 31017583 Free PMC article.
-
Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection.Sci Rep. 2021 Apr 7;11(1):7620. doi: 10.1038/s41598-021-87059-4. Sci Rep. 2021. PMID: 33828118 Free PMC article.
-
Sparsifying machine learning models identify stable subsets of predictive features for behavioral detection of autism.Mol Autism. 2017 Dec 19;8:65. doi: 10.1186/s13229-017-0180-6. eCollection 2017. Mol Autism. 2017. PMID: 29270283 Free PMC article.
-
Machine learning based on eye-tracking data to identify Autism Spectrum Disorder: A systematic review and meta-analysis.J Biomed Inform. 2023 Jan;137:104254. doi: 10.1016/j.jbi.2022.104254. Epub 2022 Dec 9. J Biomed Inform. 2023. PMID: 36509416 Review.
-
A Review of Machine Learning Methods of Feature Selection and Classification for Autism Spectrum Disorder.Brain Sci. 2020 Dec 7;10(12):949. doi: 10.3390/brainsci10120949. Brain Sci. 2020. PMID: 33297436 Free PMC article. Review.
Cited by
-
From Clinic to Computer and Back Again: Practical Considerations When Designing and Implementing Machine Learning Solutions for Pediatrics.Curr Treat Options Pediatr. 2020;6(4):336-349. doi: 10.1007/s40746-020-00205-4. Epub 2020 Sep 15. Curr Treat Options Pediatr. 2020. PMID: 38624409 Free PMC article. Review.
-
A Perspective on Crowdsourcing and Human-in-the-Loop Workflows in Precision Health.J Med Internet Res. 2024 Apr 11;26:e51138. doi: 10.2196/51138. J Med Internet Res. 2024. PMID: 38602750 Free PMC article.
-
Harnessing the Power of Mobile Phone Technology: Screening and Identifying Autism Spectrum Disorder With Smartphone Apps.Cureus. 2024 Feb 26;16(2):e55004. doi: 10.7759/cureus.55004. eCollection 2024 Feb. Cureus. 2024. PMID: 38550425 Free PMC article. Review.
-
Using #ActuallyAutistic on Twitter for Precision Diagnosis of Autism Spectrum Disorder: Machine Learning Study.JMIR Form Res. 2024 Feb 14;8:e52660. doi: 10.2196/52660. JMIR Form Res. 2024. PMID: 38354045 Free PMC article.
-
Longitudinal study of stool-associated microbial taxa in sibling pairs with and without autism spectrum disorder.ISME Commun. 2021 Dec 18;1(1):80. doi: 10.1038/s43705-021-00080-6. ISME Commun. 2021. PMID: 37938270 Free PMC article.
References
-
- Baio J, Wiggins L, Christensen DL, Maenner MJ, Daniels J, Warren Z, et al. Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years—Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2014. MMWR Surveillance Summaries. 2018;67(6):1 10.15585/mmwr.ss6706a1 . PMCID: PMC5919599. - DOI - PMC - PubMed
-
- Christensen DL, Baio J, Van Naarden Braun K, Bilder D, Charles J, Constantino JN, et al. Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years--Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2012. MMWR Surveill Summ. 2016;65(3):1–23. 10.15585/mmwr.ss6503a1 . - DOI - PMC - PubMed
-
- Christensen DL, Bilder DA, Zahorodny W, Pettygrove S, Durkin MS, Fitzgerald RT, et al. Prevalence and characteristics of autism spectrum disorder among 4-year-old children in the autism and developmental disabilities monitoring network. Journal of Developmental & Behavioral Pediatrics. 2016;37(1):1–8. 10.1097/DBP.0000000000000235 . - DOI - PubMed
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Medical