Systematic Review of Attention Testing in Allegedly “Untestable” Populations

Systematic Review of Attention Testing in Allegedly “Untestable” Populations

Verônica M. Rosário, Cristiano Mauro A. Gomes and Cybelle Maria V. Loureiro

Universidade Federal de Minas Gerais, Belo Horizonte, Brasil

Psychiatric Research and Reviews1

Test use is extremely important, not only for clinical practice, but also for scientific research. Nonetheless, some populations have been considered “untestable”. Among the different cognitive abilities assessed using tests, attention is a fundamental one. The present study presents a systematic review of the literature on attention testing in people with Intellectual Disability or Autism Spectrum Disorder, in order to identify: (1) if there are any tests that are fit to assess these populations; (2) which adaptations would be necessary for such tests to become fit; and (3) what limits and needs are involved. Our literature review identified 39 studies (review papers and empirical studies), all of which concern the administration of attention tests for people with Intellectual Disability or Autism Spectrum Disorder. The selected papers are presented and discussed from two analysis categories: (a) tests, attention abilities, and populations under study; (b) procedures and adaptations made to the testing settings. We identified 72 attention tests, where the majority of the groups of participants in the studies that were analyzed presented mild symptoms. The main adaptations done to the tests refer to strategies used to assist the comprehension of tasks, to communicate instructions, to assure engagement during the procedure, and ways to emit answers. The implications of our results are discussed.

Verônica Magalhães Rosário, Departamento de Instrumentos e Canto, Universidade Federal de Minas Gerais, Brasil; Cristiano Mauro Assis Gomes, Departamento de Psicologia, Universidade Federal de Minas Gerais, Brasil; Cybelle Maria Veiga Loureiro, Departamento de Instrumentos e Canto, Universidade Federal de Minas Gerais, Brasil. The Second author earns a Research Productivity Fellowship granted by Brazilian National Council for Scientific and Technological Development – CNPq.

Keywords: Test; Attention; Untestable; Intellectual disability; Autism spectrum disorder.

Free Full-text PDF

How to cite this article:
Verônica M. Rosário, Cristiano Mauro A. Gomes and Cybelle Maria V. Loureiro. Systematic Review of Attention Testing in Allegedly “Untestable” Populations. International Journal of Psychological Research and Reviews, 2019, 2:19. DOI: 10.28933/ijprr-2019-07-1905


1. Alpern, G. D. (1967). Measurement of “untestable” autistic children. Journal of Abnormal Psychology, 72(6), 478–86.
2. Aman, M. G., Kern, R. a, McGhee, D. E., & Arnold, L. E. (1993). Fenfluramine and methylphenidate in children with mental retardation and ADHD: clinical and side effects. Journal of the American Academy of Child and Adolescent Psychiatry, 32(4), 851–859.
3. American Psychiatric Association. (2014). Manual Diagnóstico e Estatístico de Transtornos Mentais – DSM-V. Porto Alegre: Artmed.
4. Bathurst, K., & Gottfried, A. W. (1987). Untestable Subjects in Child Development Research: Developmental Implications. Child Development, 58(4), 1135–1144.
5. Bogte, H., Flamma, B., Van Der Meere, J., & Van Engeland, H. (2009). Divided attention capacity in adults with autism spectrum disorders and without intellectual disability. Autism: The International Journal of Research and Practice, 13(3), 229–243.
6. Borgwardt, L., Thuesen, A. M., Olsen, K. J., Fogh, J., Dali, C. I., & Lund, A. M. (2015). Cognitive profile and activities of daily living: 35 patients with alpha-mannosidosis. Journal of Inherited Metabolic Disease, 38(6), 1119–1127.
7. Capesgovbr. (2017). Capesgovbr. Retrieved from
8. Cohen, R. A. (2014). The neuropsychology of attention. 2 ed. Nova York: Springer.
9. Conforto, E.C., Amaral, D. C. & Silva, S. L. (2011). Roteiro para revisão bibliográfica sistemática: aplicação no desenvolvimento de produtos e gerenciamento de projetos. Anais do Congresso Brasileiro de Gestão de Desenvolvimento de Produtos, Porto Alegre, RS, Brasil, Retrieved from
10. Corbett, B. A., & Constantine, L. J. (2006). Autism and attention deficit hyperactivity disorder: assessing attention and response control with the integrated visual and auditory continuous performance test. Child Neuropsychology, 12(4–5), 335–348.
11. Courchesne, V., Meilleur, A.-A. S., Poulin-Lord, M.-P., Dawson, M., &Soulières, I. (2015). Autistic children at risk of being underestimated: school-based pilot study of a strength-Informed assessment. Molecular Autism, 6(1), 12.
12. Coutinho, G., Mattos, P. & Abreu, N. (2010). Atenção. In L. Malloy-Diniz, D. Fuentes, P. Mattos & N. Abreu (Eds), Avaliação Neuropsicológica. (pp. 86- 93). Porto Alegre: Artmed.
13. Crutcher, E., Ali, M., Harrison, J., Sovago, J., Gomez-Mancilla, B., & Schaaf, C. P. (2016). Assessment of Cognitive Outcome Measures in Teenagers with 15q13.3 Microdeletion Syndrome. Journal of Autism and Developmental Disorders, 46(4), 1455–1463.
14. De Vries, P. J., & Watson, P. (2008). Attention deficits in tuberous sclerosis complex (TSC): Rethinking the pathways to the endstate. Journal of Intellectual Disability Research, 52(4), 348–357.
15. Deutsch, C. K., Dube, W. V., & McIlvane, W. J. (2008). Attention deficits, attention-deficit hyperactivity disorder, and intellectual disabilities. Developmental Disabilities Research Reviews, 14(4), 285–292.
16. Diamond, A. (2013). Executive functions. Annual review of psychology, 64, pp.135-168.
17. Fan, J., Bernardi, S., Dam, N. T., Anagnostou, E., Gu, X., Martin, L., … Hof, P. R.(2012). Functional deficits of the attentional networks in autism. Brain and Behavior, 2(5), 647–660.
18. Foster-owens, M. (2016). Sleep & Attention Profiles in Children with Autism Spectrum Disorder (Doctoral Dissertation). Monasg University, Melbourne, Australia.
19. Freeman, N. C., Gray, K. M., Taffe, J. R., & Cornish, K. M. (2016). A cross-syndrome evaluation of a new attention rating scale: The Scale of Attention in Intellectual Disability. Research in Developmental Disabilities, 57, 18–28.
20. Garretson, H.B., Fein, D. & Waterhouse, L. (1990). Journal of Autism and Developmental Disorders, 20(1), 101-114.
21. Gillberg, C. (2007). The autism spectrum. In J. W. Jacobson, J.A. Mulick, J. Rojahn (Eds.) Handbook of intellectual and developmental disabilities (pp. 41-59). New York: Springer.
22. Giuliani, F., Favrod, J., Grasset, F., & Schenk, F. (2011). Accurate memory for object location by individuals with intellectual disability: Absolute spatial tagging instead of configural processing? Research in Developmental Disabilities, 32(3), 986–994.
23. Haigh, S. M., Heeger, D. J., Heller, L. M., Gupta, A., Dinstein, I., Minshew, N. J., & Behrmann, M. (2016). No difference in cross-modal attention or sensory discrimination thresholds in autism and matched controls. Vision Research, 121, 85–94.
24. Hall, S. S., Hammond, J. L., Hirt, M., & Reiss, A. L. (2012). A “learning platform” approach to outcome measurement in fragile X syndrome: A preliminary psychometric study. Journal of Intellectual Disability Research, 56(10), 947–960.
25. Honora M. & Frizanco, M. L. (2008). Esclarecendo as deficiências: Aspectos teóricos e práticos para contribuir com uma sociedade inclusiva. São Paulo: Ciranda Cultural.
26. Huguenin, N. H. (1997). Employing computer technology to assess visual attention in young children and adolescents with severe mental retardation. Journal of Experimental Child Psychology, 65(2), 141–170.
27. Huguenin, N. H. (2004). Assessing visual attention in young children and adolescents with severe mental retardation utilizing conditional-discrimination tasks and multiple testing procedures. Research in Developmental Disabilities, 25(2), 155–181. 40
28. Jauregi, J., Arias, C., Vegas, O., Alén, F., Martinez, S., Copet, P., &Thuilleaux, D. (2007). A neuropsychological assessment of frontal cognitive functions in Prader-Willi syndrome. Journal of Intellectual Disability Research, 51(5), 350–365.
29. Jiang, Y. V, Capistrano, C. G., & Palm, B. E. (2014). Spatial working memory in children with high-functioning autism: Intact configural processing but impaired capacity. Journal of Abnormal Psychology., 123(1), 248–257.
30. Joseph, R. M. (2011). The Significance of IQ and Differential Cognitive Abilities for Understanding ASD. In D. A. Fein (Ed.), The Neuropsychology of Autism (pp. 281-294). New York: Oxford University Press.
31. Keehn, B., Lincoln, A. J., Müller, R. A., & Townsend, J. (2010). Attentional networks in children and adolescents with autism spectrum disorder. The Journal of Child Psychology and Psychiatry, 51(11), 1251–1259.
33. Klin, A. (2009). Subtyping the Autism Spectrum Disorders: Theoretical, Research and Clinical Considerations. In S. Goldstein., J. A. Naglieri., & S. Ozonoff. Assessment of Autism Spectrum Disorders. New York, NY: Guilford Press.
34. Klinger, G. & Renner, P. (2000). Performance-based measures in autism: implications for diagnosis, early detection, and identification of cognitive profiles. Journal of Clinical Child Psychology, 29 (4), 479-492.
35. Knox, A., Schneider, A., Abucayan, F., Hervey, C., Tran, C., Hessl, D., & Berry-Kravis, E. (2012). Feasibility, reliability, and clinical validity of the Test of Attentional Performance for Children (KiTAP) in Fragile X syndrome (FXS). Journal of Neurodevelopmental Disorders, 4(1), 2.
36. Lamb, T. (2012). Assessment of autonomy: should we put our ability to the test? In.: A. Paran., &L. Sercu. (Eds) Testing the Untestable in Language Education. Toronto, Canadá: Multilingual Matters.
37. Lerner, M. D., Potthoff, L. M., & Hunter, S. J. (2015). Optimizing cross-sectional prediction of social functioning in youth referred for neuropsychological testing. PLoS ONE, 10(5), 1–17.
38. Lezak, M.D. (1995). Neuropsychological assessment (3th ed.) New York: Oxford University Press.
39. Loureiro, D., Pio-Abreu, J. L., Machado, A., Gonçalves, R., & Cerejeira, J. (2015). Instruments for the Assessment of Autism Spectrum Disorders in Adults Without Intellectual Disabilities: a Systematic Review. European Psychiatry, 30, 1857.
40. Lundervold, A. J., Stickert, M., Hysing, M., Sørensen, L., Gillberg, C., & Posserud, M.-B. (2016). Attention Deficits in Children With Combined Autism and ADHD: A CPT Study. Journal of Attention Disorders, 20(7), 599–609.
41. Matson, J. L., &Laus, R. B. (2007). Assessment and Treatment Psychopathology Among People with Developmental Delays. In J. W. Jacobson& J.A. Mulick, J. Rojahn (Eds.), Handbook of intellectual and developmental disabilities (pp. 41-59). New York: Springer.
42. Matson, J. L., Rieske, R. D., & Williams, L. W. (2013). The relationship between autism spectrum disorders and attention-deficit/hyperactivity disorder: An overview. Research in Developmental Disabilities, 34(9), 2475–2484.
43. Matsuura, N., Ishitobi, M., Arai, S., Kawamura, K., Asano, M., Inohara, K.,…Kosaka, H. (2014). Distinguishing between autism spectrum disorder and attention deficit hyperactivity disorder by using behavioral checklists, cognitive assessments, and neuropsychological test battery. Asian Journal of Psychiatry, 12, 50-57.
44. McCallum, R. (Ed.). (2003). Handbook of Nonverbal Assessment. New York: Klumer Academic/ Plenum Publishers.
45. Mecca, T. (2013). Verificação das Propriedade Psicométricas da Bateria de Visualização e Raciocínio da Leiter-R em Crianças dos 2 aos 8 anos e nos Distúrbios do Desenvolvimento. (Unpublished doctoral dissertation).Universidade Presbiteriana Mackenzie, São Paulo, Brasil.
46. Megargee, E. I. (2000). Encyclopedia of Psychology. In A. E. Kazdin (Ed.) (Vol. 8, pp. 3611–3616). American Psychological Association & Oxford University Press.
47. Merrill, E. C., Conners, F. A., Yang, Y., &Weathington, D. (2014). The acquisition of contextual cueing effects by persons with and without intellectual disability. Research in Developmental Disabilities, 35(10), 2341–2351.
48. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group (2009). Preferred reporting items for systematic reviews and metanalyses: The PRISMA statement. PLoS Med, 6, e1000097. doi:10.1371/journal.pmed.1000097
49. Montiel, J. & Capovilla, A. (2008). Teste de Atenção por Cancelamento: análise de critérios de correção. Integração, 54, 288-296.
50. Montiel – Nava, C. & Peña, J. (2011). Déficit de atención e hiperactividad em los trastornos del espectro autista. Investigación Clínica, 52(2), 195-204. Retrieved from:
51. Mungkhetklang, C., Crewther, S. G., Bavin, E. L., Goharpey, N., & Parsons, C. (2016). Comparison of measures of ability in adolescents with intellectual disability. Frontiers in Psychology, 7(MAY), 1–11.
52. Noterdaeme, M., Amorosa, H., Mildenberger, K., Sitter, S., &Minow, F. (2001). Evaluation of attention problems in children with autism and children with a specific language disorder. European Child & Adolescent Psychiatry, 10(1), 58–66.
53. Oka, K., & Miura, T. (2008). Allocation of attention and effect of practice on persons with and without mental retardation. Research in Developmental Disabilities, 29(2), 165–175.
54. Palmer, G. A. (2006). Neuropsychological profiles of persons with mental retardation and dementia. Research in Developmental Disabilities, 27(3), 299–308.
55. Pascualvaca, D. M., Fantie, B. D., Papageorgiou, M., & Mirsky, A. F. (1998). Attentional capacities in children with autism: Is there a general deficit in shifting focus? Journal of Autism and Developmental Disorders, 28(6), 467–478.
56. Plesa-Skwerer, D., Jordan, S. E., Brukilacchio, B. H., &TagerFlusberg, H. (2016). Comparing methods for assessing receptive language skills in minimally verbal children and adolescents with ASD. Autism: The International Journal of Research and Practice, 20, 591–604.https://doi: 10.1177/1362361315600146.
57. Posner, M.I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25. doi: 10.1080/00335558008248231
58. Posner, M.I., & Fan, J. (2004). Attention as an organsystem. In J.R. Pomerantz, & M.C. Crair (Eds.), Topics in integrative neuroscience: From cells to cognition. Cambridge: Cambridge University Press.
59. Ravizza, S. & Carter, c. (2008). Shifting set about task switching: Behavioral and neural evidences for distinct forms of cognitive flexibility. Neuropsychologia, 46(12), 2924–2935.
60. Remington, A., Campbell, R., & Swettenham, J. (2012). Attentional status of faces for people with autism spectrum disorder. Autism: The International Journal of Research and Practice, 16(1), 59–73.
61. Roid, G., Nellis, L. & McLellan, M. (2003). Assessment with the Leiter International Performance Scale – Revised and the S-BIT. In S., McCallum (Ed.) Handbook of Nonverbal Assessment, Springer: New York.
62. Rose, E., Bramham, J., Young, S., Paliokostas, E., & Xenitidis, K. (2009). Neuropsychological characteristics of adults with comorbid ADHD and borderline/mild intellectual disability. Research in Developmental Disabilities, 30(3), 496–502.
63. Rose, S. A., Djukic, A., Jankowski, J. J., Feldman, J. F., Fishman, I., &Valicenti-Mcdermott, M. (2013). Rett syndrome: An eye-tracking study of attention and recognition memory. Developmental Medicine and Child Neurology, 55(4), 364–371.
64. Rosvold, H., Mirsky, A., Sarason, I., Bransome, E., & Beck, L. (1956). A continuous performance test of brain damage. Journal of Consulting and Clinical Psychology, 20, 343-352.
65. Ryan, A. M., Sackett, P. R. (2013). Stereotype threat in workplace assessments. In K.F. Geisinger (Ed.). APA Handbook of testing and assessment in psychology (pp. 643-659). Washington, DC, US: American Psychological Association.
66. Schatz, A., Weimer, K. &Trauner, D. (2002). Brief report: Attention differences in Asperger Syndrome. Journal of Autism and Developmental Disorders, 32 (4), 333-336.
67. Schott, N., & Holfelder, B. (2015). Relationship between motor skill competency and executive function in children with Down’s syndrome. Journal of Intellectual Disability Research, 59, n/a-n/a.
68. Schum, R. (2004). Psychological Assessment of Children with Multiple Handicaps Who Have Hearing Loss. The Volta Review. 104 (4), 237-255. Retrieved from
69. Siegel, B. V., Nuechterlein, K. H., Abel, L., Wu, J. C., & Buchsbaum, M. S. (1995). Glucose metabolic correlates of continuous performance test performance in adults with a history of infantile autism, schizophrenics, and controls. Schizophrenia Research, 17(1), 85–94.
70. Silva, M., & Mulick, J. A. (2009). Diagnosticando o transtorno autista: aspectos fundamentais e considerações práticas. Psicologia: Ciência E Profissão, 29(1), 116–131.
71. Sinzig, J., Morsch, D. &Lehmkuhl, G. (2008). Do hyperactivity, impulsivity and inattention have an impact on the ability of facial affect recognition in children with autism and ADHD? European Child & Adolescent Psychiatry, 17, 63–72.
72. Sternberg, R. J. & Sternberg, K. (2012). Cognitive Psychology (6th ed.). Belmont, CA: Wadsworth Cengage.
73. Strauss,E., Sherman,E., &Spreen, O. (2006).A compendium of neuropsychological tests: Administration, norms, and commentary(3th ed). New York: Oxford University Press.
74. Taddei, S., &Contena, B. (2013). Brief report: Cognitive performance in autism and Asperger’s Syndrome: What are the differences? Journal of Autism and Developmental Disorders, 43(12), 2977–2983.
75. Tenenbaum, E. J., Amso, D., Abar, B., & Sheinkopf, S. J. (2014). Attention and word learning in autistic, language delayed, and typically developing children. Frontiers in Psychology, 5(MAY), 1–9.
76. Thompson, L., Thompson, M., & Reid, A. (2010). Neurofeedback outcomes in clients with Asperger’s Syndrome. Applied Psychophysiology Biofeedback, 35(1), 63–81.
77. Trezise, K. L., Gray, K. M., & Sheppard, D. M. (2008). Attention and vigilance in children With down syndrome: Journal of Applied Research in Intellectual Disabilities, 21(6), 502–508.
78. Tylenda, B. Beckett, J., &Barett, R. (2007). Assessing mental retardation using standardized intelligence tests. In J. L. Matson (Ed.), Handbook of assessment in persons with intellectual disability (pp. 27-97). Amsterdam: Elsevier.
79. Urbina, S. (2014). Essentials of Psychological Testing (Second). New Jersey: Wiley.
80. Vinck, A., Verhagen, M. M. M., Gerven, M. Van, de Groot, I. J. M., Weemaes, C. M. R., Maassen, B. a M., & Willemsen, M. a P. (2011). Cognitive and speech-language performance in children with ataxia telangiectasia. Developmental Neurorehabilitation, 14(5), 315–22.
81. Walton, K. M., & Ingersoll, B. R. (2013). Improving social skills in adolescents and adults with autism and severe to profound intellectual disability: A review of the literature. Journal of Autism and Developmental Disorders, 43(3), 594–615.
82. Wolf‐Schein, E. G. (1998). Considerations in Assessment of Children with Severe Disabilities including Deaf‐Blindness and Autism. International Journal of Disability, Development and Education, 45(1), 35–55.
83. Zeaman, D., & House, B. J. (1963). The role of attention in retardate discrimination learning. In N.Ellis (Ed.), Handbook of mental deficiency (pp. 159–220). New York: McGraw-Hill.
84. Zeilinger, E, Stiehl, K., & Weber, G. (2013). A systematic review on assessment instruments for dementia in persons with intellectual disabilities. Research in Developmental Disabilities, 34, 3962–3977. doi:10.1016/j. ridd.2013.08.013