Detecting Developmental Delay and Autism Through Machine Learning Models Using Home Videos of Bangladeshi Children: Development and Validation Study
- PMID: 31017583
- PMCID: PMC6505375
- DOI: 10.2196/13822
Detecting Developmental Delay and Autism Through Machine Learning Models Using Home Videos of Bangladeshi Children: Development and Validation Study
Abstract
Background: Autism spectrum disorder (ASD) is currently diagnosed using qualitative methods that measure between 20-100 behaviors, can span multiple appointments with trained clinicians, and take several hours to complete. In our previous work, we demonstrated the efficacy of machine learning classifiers to accelerate the process by collecting home videos of US-based children, identifying a reduced subset of behavioral features that are scored by untrained raters using a machine learning classifier to determine children's "risk scores" for autism. We achieved an accuracy of 92% (95% CI 88%-97%) on US videos using a classifier built on five features.
Objective: Using videos of Bangladeshi children collected from Dhaka Shishu Children's Hospital, we aim to scale our pipeline to another culture and other developmental delays, including speech and language conditions.
Methods: Although our previously published and validated pipeline and set of classifiers perform reasonably well on Bangladeshi videos (75% accuracy, 95% CI 71%-78%), this work improves on that accuracy through the development and application of a powerful new technique for adaptive aggregation of crowdsourced labels. We enhance both the utility and performance of our model by building two classification layers: The first layer distinguishes between typical and atypical behavior, and the second layer distinguishes between ASD and non-ASD. In each of the layers, we use a unique rater weighting scheme to aggregate classification scores from different raters based on their expertise. We also determine Shapley values for the most important features in the classifier to understand how the classifiers' process aligns with clinical intuition.
Results: Using these techniques, we achieved an accuracy (area under the curve [AUC]) of 76% (SD 3%) and sensitivity of 76% (SD 4%) for identifying atypical children from among developmentally delayed children, and an accuracy (AUC) of 85% (SD 5%) and sensitivity of 76% (SD 6%) for identifying children with ASD from those predicted to have other developmental delays.
Conclusions: These results show promise for using a mobile video-based and machine learning-directed approach for early and remote detection of autism in Bangladeshi children. This strategy could provide important resources for developmental health in developing countries with few clinical resources for diagnosis, helping children get access to care at an early age. Future research aimed at extending the application of this approach to identify a range of other conditions and determine the population-level burden of developmental disabilities and impairments will be of high value.
Keywords: Bangladesh; Biomedical Data Science; autism; autism spectrum disorder; clinical resources; developmental delays; machine learning.
©Qandeel Tariq, Scott Lanyon Fleming, Jessey Nicole Schwartz, Kaitlyn Dunlap, Conor Corbin, Peter Washington, Haik Kalantarian, Naila Z Khan, Gary L Darmstadt, Dennis Paul Wall. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.04.2019.
Conflict of interest statement
Conflicts of Interest: DPW is the founder of Cognoa.com. This company is developing digital health solutions for pediatric care. All other authors declare no competing interests.
Figures
Similar articles
-
Mobile detection of autism through machine learning on home video: A development and prospective validation study.PLoS Med. 2018 Nov 27;15(11):e1002705. doi: 10.1371/journal.pmed.1002705. eCollection 2018 Nov. PLoS Med. 2018. PMID: 30481180 Free PMC article.
-
Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection.Sci Rep. 2021 Apr 7;11(1):7620. doi: 10.1038/s41598-021-87059-4. Sci Rep. 2021. PMID: 33828118 Free PMC article.
-
Classification of Children With Autism and Typical Development Using Eye-Tracking Data From Face-to-Face Conversations: Machine Learning Model Development and Performance Evaluation.J Med Internet Res. 2021 Aug 26;23(8):e29328. doi: 10.2196/29328. J Med Internet Res. 2021. PMID: 34435957 Free PMC article.
-
Machine learning based on eye-tracking data to identify Autism Spectrum Disorder: A systematic review and meta-analysis.J Biomed Inform. 2023 Jan;137:104254. doi: 10.1016/j.jbi.2022.104254. Epub 2022 Dec 9. J Biomed Inform. 2023. PMID: 36509416 Review.
-
A review of screening tools for the identification of autism spectrum disorders and developmental delay in infants and young children: recommendations for use in low- and middle-income countries.Autism Res. 2019 Feb;12(2):176-199. doi: 10.1002/aur.2033. Epub 2019 Feb 1. Autism Res. 2019. PMID: 30707000 Review.
Cited by
-
A Study on the Screening of Children at Risk for Developmental Disabilities Using Facial Landmarks Derived From a Mobile-Based Application.Psychiatry Investig. 2024 May;21(5):496-505. doi: 10.30773/pi.2023.0315. Epub 2024 May 23. Psychiatry Investig. 2024. PMID: 38810998 Free PMC article.
-
A Perspective on Crowdsourcing and Human-in-the-Loop Workflows in Precision Health.J Med Internet Res. 2024 Apr 11;26:e51138. doi: 10.2196/51138. J Med Internet Res. 2024. PMID: 38602750 Free PMC article.
-
A Local Community-Based Social Network for Mental Health and Well-being (Quokka): Exploratory Feasibility Study.JMIRx Med. 2021 Oct 27;2(4):e24972. doi: 10.2196/24972. JMIRx Med. 2021. PMID: 37725541 Free PMC article.
-
A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autism.Annu Rev Biomed Data Sci. 2023 Aug 10;6:211-228. doi: 10.1146/annurev-biodatasci-020722-125454. Epub 2023 May 3. Annu Rev Biomed Data Sci. 2023. PMID: 37137169 Free PMC article. Review.
-
A data driven machine learning approach to differentiate between autism spectrum disorder and attention-deficit/hyperactivity disorder based on the best-practice diagnostic instruments for autism.Sci Rep. 2022 Nov 5;12(1):18744. doi: 10.1038/s41598-022-21719-x. Sci Rep. 2022. PMID: 36335178 Free PMC article.
References
-
- Cromer J. U.S. Army. 2018. Apr 05, [2019-04-17]. Autism: fastest-growing developmental disability https://www.army.mil/article/203386/autism_fastest_growing_developmental... .
-
- Baio J, Wiggins J, Christensen DL, Maenner MJ, Daniels J, Warren Z, Kurzius-Spencer M, Zahorodny W, Robinson Rosenberg C, White T, Durkin MS, Imm P, Nikolaou L, Yeargin-Allsopp M, Lee LC, Harrington R, Lopez M, Fitzgerald RT, Hewitt A, Pettygrove S, Constantino JN, Vehorn A, Shenouda J, Hall-Lande J, Van Naarden Braun K, Dowling NF. Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2014. MMWR Surveill Summ. 2018 Dec 27;67(6):1–23. doi: 10.15585/mmwr.ss6706a1. http://europepmc.org/abstract/MED/29701730 - DOI - PMC - PubMed
-
- Hossain M, Ahmed HU, Jalal Uddin MM, Chowdhury WA, Iqbal MS, Kabir RI, Chowdhury IA, Aftab A, Datta PG, Rabbani G, Hossain SW, Sarker M. Autism Spectrum disorders (ASD) in South Asia: a systematic review. BMC Psychiatry. 2017 Dec 01;17(1):281. doi: 10.1186/s12888-017-1440-x. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-017-1440-x 10.1186/s12888-017-1440-x - DOI - DOI - PMC - PubMed
-
- Durkin M, Elsabbagh M, Barbaro J, Gladstone M, Happe F, Hoekstra RA, Lee LC, Rattazzi A, Stapel-Wax J, Stone WL, Tager-Flusberg H, Thurm A, Tomlinson M, Shih A. Autism screening and diagnosis in low resource settings: Challenges and opportunities to enhance research and services worldwide. Autism Res. 2015 Oct;8(5):473–6. doi: 10.1002/aur.1575. http://europepmc.org/abstract/MED/26437907 - DOI - PMC - PubMed