Skip to main content
SpringerLink
Log in

Examining the Role of Race, Ethnicity, and Gender on Social and Behavioral Ratings Within the Autism Diagnostic Observation Schedule

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

The Autism Diagnostic Observation Schedule (ADOS) is widely used to assess symptoms of autism spectrum disorder (ASD). Given well-documented differences in social behaviors across cultures, this study examined whether item-level biases exist in ADOS scores across sociodemographic groups (race, ethnicity, and gender). We examined a subset of ten ADOS items among participants (N = 2458). Holding level of overall ADOS behavioral symptoms constant, we found significant item level bias (measurement noninvariance) for race and ethnicity on three ADOS items. Item-level bias was not apparent across gender. Although the magnitude of bias was small, our findings highlight the need to reevaluate norms and operational definitions used in assessments to increase ASD diagnostic accuracy among culturally-diverse groups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Al-Salehi, S. M., & Al-Hifthy, E. H. (2009). Autism in Saudi Arabia: Presentation, clinical correlates and comorbidity. Transcultural Psychiatry, 46(2), 340–347.

    Article  PubMed  Google Scholar 

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washington, DC: American Psychiatric Association.

    Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental health disorders (5th ed.). Washington, DC: American Psychiatric Association.

    Book  Google Scholar 

  • Begeer, S., El Bouk, S., Boussaid, W., Terwogt, M. M., & Koot, H. M. (2009). Underdiagnosis and referral bias of autism in ethnic minorities. Journal of Autism and Developmental Disorders, 39(1), 142–148. doi:10.1007/s10803-008-0611-5.

    Article  PubMed  Google Scholar 

  • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238.

    Article  PubMed  Google Scholar 

  • Bracken, B. A., & Barona, A. (1991). State of the art procedures for translating, validating and using psychoeducational tests in cross-cultural assessment. School Psychology International, 12, 119–132. doi:10.1177/0143034391121010.

    Article  Google Scholar 

  • Browne, M. W., Cudeck, R., Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit. Sage Focus Editions, 154, 136–136.

    Google Scholar 

  • Caron, K. G., Schaaf, R. C., Benevides, T. W., & Gal, E. (2012). Cross-cultural comparison of sensory behaviors in children with autism. American Journal of Occupational Therapy, 66(5), e77–e80.

    Article  PubMed  Google Scholar 

  • Carter, J. A., Lees, J. A., Murira, G. M., Gona, J., Neville, B. G., & Newton, C. R. (2005). Issues in the development of cross-cultural assessments of speech and language for children. International Journal of Language & Communication Disorders, 40(4), 385–401. doi:10.1080/13682820500057301.

    Article  Google Scholar 

  • Chaidez, V., Hansen, R. L., & Hertz-Picciotto, I. (2012). Autism spectrum disorders in Hispanics and non-Hispanics. Autism: The International Journal of Research and Practice, 16(4), 381–397. doi:10.1177/1362361311434787.

    Article  Google Scholar 

  • Chua, H. F., Boland, J. E., & Nisbett, R. E. (2005). Cultural variation in eye movements during scene perception. Proceedings of the National Academy of Sciences of the United States of America, 102(35), 12629–12633. doi:10.1073/pnas.0506162102.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chung, K.-M., Jung, W., Yang, J.-W., Ben-Itzchak, E., Zachor, D. A., Furniss, F., … Barker, A. A. (2012). Cross cultural differences in challenging behaviors of children with autism spectrum disorders: An international examination between Israel, South Korea, the United Kingdom, and the United States of America. Research in Autism Spectrum Disorders, 6(2), 881–889.

    Article  Google Scholar 

  • Collett, P. (1971). Training Englishmen in the non-verbal behaviour of Arabs. International Journal of Psychology, 6(3), 209–215. doi:10.1080/00207597108246684.

    Article  Google Scholar 

  • Constantino, J. N., & Gruber, C. P. (2012). Social responsiveness scale, second edition: SRS-2. Los Angeles: Western Psychological Services.

    Google Scholar 

  • Daley, T. C. (2004). From symptom recognition to diagnosis: children with autism in urban India. Social Science & Medicine, 58(7), 1323–1335.

    Article  Google Scholar 

  • El Bouk, S., Boussaid, W., Meerum Terwogt, M., & Koot, H. (2009). Underdiagnosis and referral bias of autism in ethnic minorities. Journal of Autism and Developmental Disorders, 39(1), 142–148.

    Article  Google Scholar 

  • Elfenbein, H. A. (2013). Nonverbal dialects and accents in facial expressions of emotion. Emotion Review, 5(1), 90–96. doi:10.1177/175407391245133.

    Article  Google Scholar 

  • Elfenbein, H. A., Beaupré, M., Lévesque, M., & Hess, U. (2007). Toward a dialect theory: Cultural differences in the expression and recognition of posed facial expressions. Emotion, 7(1), 131. doi:10.1037/1528-3542.7.1.131.

    Article  PubMed  Google Scholar 

  • Ember, C. R., & Cunnar, C. M. (2015). Children’s play and work: The relevance of cross-cultural ethnographic research for archaeologists. Childhood in the Past, 8(2), 87–103. doi:10.1179/1758571615Z.00000000031.

    Article  Google Scholar 

  • Farver, J. M., & Howes, C. (1993). Cultural differences in American and Mexican mother-child pretend play. Merrill-Palmer Quarterly, 39, 344–358.

    Google Scholar 

  • Fischbach, G. D., & Lord, C. (2010). The Simons Simplex Collection: a resource for identification of autism genetic risk factors. Neuron, 68(2), 192–195.

    Article  PubMed  Google Scholar 

  • Freeth, M., Milne, E., Sheppard, E., & Ramachandran, R. (2014). Autism across cultures: Perspectives from non-western cultures and implications for research. In F. R. Volkmar, S. J. Rogers, R. Paul, & K. A. Pelphrey (Eds.), Handbook of autism and pervasive developmental disorders, (4th ed.,Vol. 2, pp. 997–1013). Hoboken, NJ: Wiley.

    Google Scholar 

  • Freeth, M., Sheppard, E., Ramachandran, R., & Milne, E. (2013). A cross-cultural comparison of autistic traits in the UK, India and Malaysia. Journal of Autism and Developmental Disorders, 43(11), 2569–2583.

    Article  PubMed  Google Scholar 

  • Fugita, S. S., Wexley, K. N., & Hillery, J. M. (1974). Black-White differences in nonverbal behavior in an interview setting. Journal of Applied Social Psychology, 4(4), 343–350. doi:10.1111/j.1559-1816.1974.tb02606.x.

    Article  Google Scholar 

  • Gotham, K., Risi, S., Dawson, G., Tager-Flusberg, H., Joseph, R., Carter, A., … Hyman, S. L. (2008). A replication of the Autism Diagnostic Observation Schedule (ADOS) revised algorithms. Journal of the American Academy of Child & Adolescent Psychiatry, 47(6), 642–651. doi:10.1097/CHI.0b013e31816bffb7.

    Article  Google Scholar 

  • Hall, W. S., & Freedle, R. O. (1975). Culture and language: The black American experience. Washington D.C.: Hemisphere Publishing.

    Google Scholar 

  • Hall, W. S., Reder, S., & Cole, M. (1975). Story recall in young Black and White children: Effects of racial group membership, race of experimenter, and dialect. Developmental Psychology, 11(5), 628. doi:10.1037/0012-1649.11.5.628.

    Article  Google Scholar 

  • Hess, U., Adams, R., & Kleck, R. (2005). Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition & Emotion, 19(4), 515–536. doi:10.1080/02699930441000364.

    Article  Google Scholar 

  • Hiller, R. M., Young, R. L., & Weber, N. (2015). Sex differences in pre-diagnosis concerns for children later diagnosed with autism spectrum disorder. Autism: The International Journal of Research and Practice, 20(1), 75–84. doi:10.1177/1362361314568899.

    Article  Google Scholar 

  • Hilton, C. L., Fitzgerald, R. T., Jackson, K. M., Maxim, R. A., Bosworth, C. C., Shattuck, P. T., … Constantino, J. N. (2010). Brief report: Under-representation of African Americans in autism genetic research: A rationale for inclusion of subjects representing diverse family structures. Journal of Autism and Developmental Disorders, 40(5), 633–639. doi:10.1007/s10803-009-0905-2.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hu, L.-T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424.

    Article  Google Scholar 

  • Jack, R. E., Garrod, O. G., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences, 109(19), 7241–7244. doi:10.1097/01.mlr.0000245250.50114.0f.

    Article  Google Scholar 

  • Jones, R. N. (2006). Identification of measurement differences between English and Spanish language versions of the Mini-Mental State Examination: detecting differential item functioning using MIMIC modeling. Medical Care, 44(11), S124–S133.

    Article  PubMed  Google Scholar 

  • Kirkovski, M., Enticott, P. G., & Fitzgerald, P. B. (2013). A review of the role of female gender in autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(11), 2584–2603.

    Article  PubMed  Google Scholar 

  • Knapp, M. L., Hall, J. A., & Horgan, T. G. (2013). Nonverbal communication in human interaction. Boston: Wadsworth Cengage Learning.

    Google Scholar 

  • Knickmeyer, R. C., Wheelwright, S., & Baron-Cohen, S. B. (2008). Sex-typical play: Masculinization/defeminization in girls with an autism spectrum condition. Journal of Autism and Developmental Disorders, 38(6), 1028–1035. doi:10.1007/s10803-007-0475-0.

    Article  PubMed  Google Scholar 

  • Kochman, T. (1981). Black and white styles in conflict: Chicago: University of Chicago Press.

    Google Scholar 

  • LaFrance, M., & Hecht, M. A. (2000). Gender and smiling: A meta-analysis. In A. Fischer (Ed.), Gender and emotion: Social psychological perspectives (pp. 118). Cambridge, UK: Cambridge University Press.

  • LaFrance, M., & Mayo, C. (1976). Racial differences in gaze behavior during conversations: Two systematic observational studies. Journal of Personality and Social Psychology, 33(5), 547.

    Article  Google Scholar 

  • Lai, M.-C., Lombardo, M., Auyeung, B., Chakrabarti, B., & Baron-Cohen, S. (2015). Sex/gender differences and autism. Journal of the American Academy of Child & Adolescent Psychiatry, 54(1), 11–24.

    Article  Google Scholar 

  • Lancy, D. F. (2007). Accounting for variability in mother–child play. American Anthropologist, 109(2), 273–284. doi:10.1525/aa.2007.109.2.273.

    Article  Google Scholar 

  • Landa, R., & Garrett-Mayer, E. (2006). Development in infants with autism spectrum disorders: A prospective study. Journal of Child Psychology & Psychiatry, 47(6), 629–638. doi:10.1111/j.1469-7610.2006.01531.x.

    Article  Google Scholar 

  • Liptak, G. S., Benzoni, L. B., Mruzek, D. W., Nolan, K. W., Thingvoll, M. A., Wade, C. M., & Fryer, G. E. (2008). Disparities in diagnosis and access to health services for children with autism: Data from the National Survey of Children’s Health. Journal of Developmental & Behavioral Pediatrics, 29(3), 152–160.

    Article  Google Scholar 

  • Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., ... Rutter, M. (2000). The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.

    Article  PubMed  Google Scholar 

  • Lord, C., Rutter, M., DiLavore, P., & Risi, S. (2002). Autism Diagnostic Observation Schedule: ADOS. Los Angeles: Western Psychological Services.

    Google Scholar 

  • Lord, C., Rutter, M., DiLavore, P., Risi, S., Gotham, K., & Bishop, S. L. (2012). Autism Diagnsotic Observation Schedule, second edition (ADOS-2) manual (Part 1): Modules 1–4. Torrance, CA: Western Psychological Services.

    Google Scholar 

  • Magaña, S., Lopez, K., Aguinaga, A., & Morton, H. (2013). Access to diagnosis and treatment services among Latino children with autism spectrum disorders. Intellectual and Developmental Disabilities, 51(3), 141–153. doi:10.1352/1934-9556-51.3.141.

    Article  PubMed  Google Scholar 

  • Magiati, I., Goh, D. A., Lim, S. J., Gan, D. Z. Q., Leong, J., Allison, C., … Saw, S. M. (2015). The psychometric properties of the Quantitative-Checklist for Autism in Toddlers (Q-CHAT) as a measure of autistic traits in a community sample of Singaporean infants and toddlers. Molecular Autism, 6(1), 40.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mandell, D. S., Listerud, J., Levy, S. E., & Pinto-Martin, J. A. (2002). Race differences in the age at diagnosis among Medicaid-eligible children with autism. Journal of the American Academy of Child & Adolescent Psychiatry, 41(12), 1447–1453. doi:10.1097/01.CHI.0000024863.60748.53.

    Article  Google Scholar 

  • Mandell, D. S., Wiggins, L. D., Carpenter, L. A., Daniels, J., DiGuiseppi, C., Durkin, M. S., Giarelli, E., … Pinto-Martin, J. A. (2009). Racial/ethnic disparities in the identification of children with autism spectrum disorders. American Journal of Public Health, 99(3), 493.

    Article  PubMed  PubMed Central  Google Scholar 

  • Marsh, A. A., Elfenbein, H. A., & Ambady, N. (2003). Nonverbal “accents” cultural differences in facial expressions of emotion. Psychological Science, 14(4), 373–376.

    Article  PubMed  Google Scholar 

  • Matson, J. L., Matheis, M., Burns, C., Esposito, G., Venuti, P., Pisula, E., … Kamio, Y. (2017). Examining cross-cultural differences in autism spectrum disorder: A multinational comparison from Greece, Italy, Japan, Poland, and the United States. European Psychiatry, 42, 70–76.

    Article  PubMed  Google Scholar 

  • Matson, J. L., Nebel-Schwalm, M., & Matson, M. L. (2007). A review of methodological issues in the differential diagnosis of autism spectrum disorders in children. Research in Autism Spectrum Disorders, 1(1), 38–54. doi:10.1016/j.rasd.2006.07.004.

    Article  Google Scholar 

  • Matson, J. L., Worley, J. A., Fodstad, J. C., Chung, K.-M., Suh, D., Jhin, H. K., … Furniss, F. (2011). A multinational study examining the cross cultural differences in reported symptoms of autism spectrum disorders: Israel, South Korea, the United Kingdom, and the United States of America. Research in Autism Spectrum Disorders, 5(4), 1598–1604.

    Article  Google Scholar 

  • Mazefsky, C. A., & Oswald, D. P. (2006). The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting. Autism: The International Journal of Research and Practice, 10(6), 533–549.

    Article  Google Scholar 

  • McCarthy, A., Lee, K., Itakura, S., & Muir, D. W. (2006). Cultural display rules drive eye gaze during thinking. Journal of Cross-Cultural Psychology, 37(6), 717–722. doi:10.1177/0022022106292079.

    Article  PubMed  PubMed Central  Google Scholar 

  • McClure, E. B. (2000). A meta-analytic review of sex differences in facial expression processing and their development in infants, children, and adolescents. Psychological Bulletin, 126(3), 424. doi:10.1037/0033-2909.126.3.424.

    Article  PubMed  Google Scholar 

  • Mundy, P. (1995). Joint attention and social-emotional approach behavior in children with autism. Development and Psychopathology, 7(01), 63–82. doi:10.1017/S0954579400006349.

    Article  Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (2005). Mplus: Statistical analysis with latent variables: User’s guide. Los Angeles: Muthén & Muthén.

    Google Scholar 

  • Norbury, C. F., & Sparks, A. (2013). Difference or disorder? Cultural issues in understanding neurodevelopmental disorders. Developmental Psychology, 49(1), 45. doi:10.1037/a0027446.

    Article  PubMed  Google Scholar 

  • Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2005). Evidence-based assessment of autism spectrum disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34(3), 523–540. doi:10.1207/s15374424jccp3403_8.

    Article  PubMed  Google Scholar 

  • Pae, T.-I., & Park, G.-P. (2006). Examining the relationship between differential item functioning and differential test functioning. Language Testing, 23(4), 475–496.

    Article  Google Scholar 

  • Richesson, R., Anderson, M., & Smerek, M. (2014). Race/ethnicity data standards. Retrieved from https://www.nihcollaboratory.org/Products/RaceEthnicity_standard.pdf.

  • Runnels, J. (2013). Measuring differential item and test functioning across academic disciplines. Language Testing in Asia, 3(1), 9. doi:10.1186/2229-0443-3-9.

    Article  Google Scholar 

  • Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In Innovations in Multivariate Statistical Analysis, pp. 233–247. New York: Springer.

    Chapter  Google Scholar 

  • Schofield, T. J., Parke, R. D., Kim, Y., & Coltrane, S. (2008). Bridging the acculturation gap: Parent-child relationship quality as a moderator in Mexican American families. Developmental Psychology, 44(4), 1190. doi:10.1037/a0012529.

    Article  PubMed  Google Scholar 

  • Sell, N. K., Giarelli, E., Blum, N., Hanlon, A. L., & Levy, S. E. (2012). A comparison of autism spectrum disorder DSM-IV criteria and associated features among African American and white children in Philadelphia County. Disability and Health Journal, 5(1), 9–17. doi:10.1016/j.dhjo.2011.08.002.

    Article  PubMed  Google Scholar 

  • Simons. (2010). SFARI base/SSC researcher welcome packet. Retrieved April 19, 2010, from http://simonsfoundation.s3.amazonaws.com/share/SFARI_Researcher_Welcome.pdf.

  • Sipes, M., Furniss, F., Matson, J. L., & Hattier, M. (2012). A multinational study examining the cross cultural differences in social skills of children with autism spectrum disorders: A comparison between the United Kingdom and the United States of America. Journal of Developmental and Physical Disabilities, 24(2), 145–154.

    Article  Google Scholar 

  • Soto, S., Linas, K., Jacobstein, D., Biel, M., Migdal, T., & Anthony, B. J. (2014). A review of cultural adaptations of screening tools for autism spectrum disorders. Autism, 19(6), 646–661.

    Article  PubMed  Google Scholar 

  • United States Census Bureau. (2017). Race. Retrieved from https://www.census.gov/topics/population/race/about.html.

  • Vrana, S. R., & Rollock, D. (2002). The role of ethnicity, gender, emotional content, and contextual differences in physiological, expressive, and self-reported emotional responses to imagery. Cognition & Emotion, 16(1), 165–192. doi:10.1080/0269993014300018.

    Article  Google Scholar 

  • Western Psychological Services. (2016). Published translations. Retrieved from http://www.wpspublish.com/app/OtherServices/PublishedTranslations.aspx.

  • Yuki, M., Maddux, W. W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43(2), 303–311. doi:10.1016/j.jesp.2006.02.004.

    Article  Google Scholar 

  • Zebrowitz, L. A., Kikuchi, M., & Fellous, J.-M. (2010). Facial resemblance to emotions: Group differences, impression effects, and race stereotypes. Journal of Personality and Social Psychology, 98(2), 175. doi:10.1037/a0017990.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

Ashley J. Harrison’s work on this project was supported by a NIMH T32 fellowship to (5T32MH019927_20). The authors would like to express their appreciation to the following students for their help with this project and manuscript, Margaret Naughton, Rebecca Miller, and Alejandra Perez-Ramirez and the families that participated in the Simons Simplex Collection data set.

Author Contributions

AJH and RNJ conceived of the study and designed the study. AJH requested SSC data access. RNJ and DCT performed the statistical analysis and the provided the written interpretation of the data analyses. AJH and KAL helped interpret the data and aided in contextualizing the findings in the cross-cultural literature. All authors drafted and revised the manuscript.

Author information

Authors and Affiliations

  1. Department of Educational Psychology, University of Georgia, Aderhold Hall 110 Carlton Street, Athens, GA, 30602, USA

    Ashley J. Harrison

  2. Boston University, Boston, MA, USA

    Kristin A. Long

  3. Alpert Medical School of Brown University, Providence, RI, USA

    Douglas C. Tommet & Richard N. Jones

Authors
  1. Ashley J. Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristin A. Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Douglas C. Tommet
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Richard N. Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashley J. Harrison.

Ethics declarations

Conflict of interests

All the authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Harrison, A.J., Long, K.A., Tommet, D.C. et al. Examining the Role of Race, Ethnicity, and Gender on Social and Behavioral Ratings Within the Autism Diagnostic Observation Schedule. J Autism Dev Disord 47, 2770–2782 (2017). https://doi.org/10.1007/s10803-017-3176-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-017-3176-3

Keywords

Search

Navigation