Neurodevelopmental assessments used to measure preschoolers’ cognitive development in Latin America: a systematic review

Abstract

Objective

The objective of this study was to systematically review the standardized neurodevelopmental assessments used to study preschool-aged children’s cognitive development in Spanish-speaking Latin America.

Methods

The authors systematically searched PubMed, PsycINFO, and ERIC databases for peer-reviewed articles from Spanish-speaking Latin American countries. Articles were included if they measured cognitive development among children aged 2–6 years using at least one standardized assessment tool; 97 articles were included and reviewed in accordance with PRISMA guidelines to assess their use of these tools.

Results

Ninety-seven studies across 13 countries used a total of 41 assessments to measure cognitive development; most widely used were the Wechsler intelligence scales (n = 46/97), particularly the Wechsler Preschool and Primary Scale of Intelligence and Wechsler Intelligence Scale for Children (n = 23 and 29, respectively). Other common assessments included the McCarthy Scales of Children’s Abilities (n = 9), Raven’s Progressive Matrices (n = 9), Child Neuropsychological Assessment (n = 8), and Peabody Picture Vocabulary Test (n = 7). In regions where normative data for a given assessment were unpublished, authors commonly used norms from the United States, Mexico, or Spain or did not report standard scores in their analyses.

Conclusions

The wide range of tools used in these studies presents a challenge for generalizing results when measuring the neurodevelopment of Latin American preschool-aged children. The low availability of normative data for specific regions reveals concerns if some tools are culturally and linguistically appropriate even when Spanish is a common language, particularly in low-resource settings. Future work to forge greater consistency in the use of validated measures, clarity in reporting research methods, and publication of regional normative data would benefit the field.

The use of standardized cognitive assessments to quantify and compare child developmental outcomes is a vital component of conducting rigorous child neurodevelopmental outcome research. To assess neurodevelopment and cognitive function within a study cohort and effectively determine whether children are developing on track with their peers, researchers need to consider both the cultural suitability of standardized assessments that were developed and normed in different areas of the world as well as the prior application of these assessments in similar samples (Gladstone et al., 2017). However, validated measures or information about commonly used measures may be limited or unavailable depending upon the region in which a study is conducted. Despite the high importance of assessing child development globally, conducting research that includes participants from different countries and with different primary languages can enhance the complexity of finding cognitive measures that can be used and compared between these cohorts. While it has been acknowledged that “remarkable progress” has been made regarding child neurodevelopmental testing and research in low- and middle-income countries (LMICs) in recent years (Semrud-Clikeman et al., 2017), the need to accurately measure and report on child cognitive development within the context of factors specific to lower-resource settings is ongoing.

Like those in other LMICs, families in Spanish-speaking Latin America face a high burden of social inequities including infectious illness, inadequate access to proper nutrition, toxic environmental exposures, and violence, all of which place children at risk of abnormal neurodevelopment (Bitta et al., 2017; Dix-Cooper et al., 2012; Dórea, 2020; Flores-Mendoza et al., 2021; Manrique Millones et al., 2015; Schady, 2006; Velez van Meerbeke et al., 2007). Latin America has been identified by organizations such as the World Bank (see Schady, 2006), United Nations Educational, Scientific and Cultural Organization (UNESCO, 2010), and UNICEF (Hartwig, 2020; Lumpkin, 2023) as a region requiring unique and intentional investment in early childhood care and development. As advocates, policymakers, and other stakeholders in these regions invest in high-quality programs to reduce inequity and promote nurturing care for young children, it is important to monitor cognitive outcomes to understand children’s shifting trajectories, identify unique needs of certain subgroups of children, and find opportunities for continued improvement (Hartwig, 2020; Lumpkin, 2023; UNESCO, 2010). Identifying standardized cognitive tests that are used to monitor neurodevelopment in Latin America is an important step to help researchers maximize the relevance, validity, and impact of their child outcome research in these regions (Semrud-Clikeman et al., 2017).

We conducted a systematic review to determine the standardized neurodevelopmental assessments used to study child cognitive development in Spanish-speaking Latin America. We defined cognitive development broadly to include a broad range of skills that the authors of included studies defined as their measure of a neurodevelopmental or cognitive outcome. As such, these included measures of intelligence, language, nonverbal reasoning, learning, and other skills. The objective of the systematic review was to describe the most frequently used assessments in Spanish-speaking Latin America and to present an overview of how these cognitive assessments have been used in the literature to date. We hope that the systematic review will inform multinational and multiregional child cognitive outcome studies now and in the future.

Methods

A systematic review was conducted in accordance with PRISMA Guidelines for Systematic Reviews (Page et al., 2021); the final PRISMA checklist is available in Supplementary File S1. Beginning in July 2022, the authors systematically searched PubMed, PsycINFO, and ERIC databases for relevant articles. An initial search was conducted in October 2022 using database-specific search strategies and initial search terms, identified following a standard protocol as suggested by a research librarian at Children’s National Hospital, Washington, DC. Additional search terms (e.g., adding “-ing” or “-s” to terms when applicable, and listing all Spanish-speaking countries in Latin America) were later added to capture as many articles as possible, and the searches were re-run in July and August 2023. The search terms are outlined below, and the final reproducible searches for each database are available in the Supplementary Material:

• (Cognition OR neurodevelopment OR cognitive OR neurodevelopmental OR intelligence OR comprehension OR IQ OR FSIQ) AND (test OR tests OR assessment OR assessments OR evaluation OR evaluations OR delay OR delays OR score OR scores) AND (“South America” OR “South American” OR “Latin America” OR “Latin American” OR “Central America” OR “Central American” OR Mexico OR Mexican OR Colombia OR Colombian OR Argentina OR Argentinian OR Bolivia OR Bolivian OR Chile OR Chilean OR Ecuador OR Ecuadorian OR Paraguay OR Paraguayan OR Peru OR Peruvian OR Uruguay OR Uruguayan OR Venezuela OR Venezuelan OR Panama OR Panamanian OR Nicaragua OR Nicaraguan OR Honduras OR Honduran OR Costa Rica OR Costa Rican OR Cuba OR Cuban OR Dominican OR El Salvador OR El Salvadorian OR Guatemala OR Guatemalan)

Inclusion and exclusion criteria were identical in all searches. Results were filtered to include only peer-reviewed articles published in English or Spanish from Spanish-speaking countries in Latin America. We included studies assessing children between 2 and 6 years of age to reflect the diversity in ages and experiences of children in preschool programs in Latin America (UNESCO, 2010; World Education Network, n.d.); additional database-specific filters were applied to limit results to articles assessing children within this age group.

The searches yielded 4,949 articles for further review (Figure 1). All articles were uploaded into Covidence, a systematic review management software (Covidence, n.d.). Once duplicate records were removed, all titles and abstracts were screened using Covidence (by M.E.W. and S.M.R.); another author (E.A.C.) independently resolved any conflict between the ratings. Articles were selected for inclusion if the studies directly assessed the cognitive development of children aged 2–6 years using a standardized assessment tool, or select subtest(s) of a standardized tool, in Spanish-speaking Latin American countries. We did not exclude articles based on year of publication. Systematic reviews, meta-analyses, and articles reporting only on psychometric properties or validation of assessment tools were excluded. Articles were also excluded if they only reported data on children older or younger than 2–6 years of age or utilized proxy standardized assessments only (e.g., parent questionnaires), assessed children from geographic regions outside of Spanish-speaking Latin America, or were published in a language besides English or Spanish. After the initial title and abstract screening, the authors (M.E.W., E.C., S.M.R., and R.A.-S.) retrieved the remaining articles and assessed the full text of each to determine whether it met the above-mentioned inclusion criteria. Similarly, each full text article required at least two investigators to agree on its inclusion. If multiple articles reported on data from the same cohort, only the article that was published most recently was included. Once all included articles were selected, M.E.W., E.C., S.M.R., and R.A.S. extracted relevant data on the methods of each study (i.e., study design, location, number of participants, participant age ranges, participant health conditions, and inclusion criteria; and outcome measures used to assess cognitive development) and grouped studies based on the assessments used to measure cognitive development. Initial and full-text screening and data extraction were performed in Covidence. Because the goal of this systematic review was to describe studies’ utilization of standardized assessment tools and not to harmonize their results or appraise instruments based on psychometric properties, a meta-analysis and assessment of each study’s methodological quality and risk of bias were not appropriate for this review. Study investigators were not contacted to obtain or confirm unclear or missing data due to limited resources; the authors of this review did not make assumptions about unclear or missing information and only reported data available in the full texts. This review was not registered and the protocol is not published.

Results

Figure 1 shows the study selection and process of the systematic review. Our search identified 97 studies for inclusion (Table 1). Across the 97 studies, we identified 41 standardized assessments used to measure cognitive development; information about these standardized assessments is outlined in Table 2. The most widely used assessments, defined as those utilized by greater than 5 studies, are described in further detail below.

Wechsler Intelligence Scales

The Wechsler intelligence scales, including the Wechsler Preschool and Primary Scale of Intelligence (WPPSI), the Wechsler Intelligence Scale for Children (WISC), and the Wechsler Abbreviated Scale of Intelligence (WASI), are some of the most well-known standard measures of intelligence. Each scale assesses different cognitive indices, such as verbal and nonverbal reasoning skills, working memory, and/or processing speed, which render index scores including a Full-Scale Intelligence Quotient (FSIQ) as well as verbal IQ (VIQ) and nonverbal IQ (NVIQ). Other indices and supplementary scores are also options depending on how many and which subtests are administered. Forty-six studies used at least one Wechsler scale; 23 used the WPPSI, 29 used the WISC, and 1 used the WASI. Some authors that studied broad age ranges of participants, such as González-Cortés (2019) and Herrera-Chinchay et al. (2021), used more than one Wechsler scale to assess intelligence using the age-appropriate Wechsler test (e.g., WPPSI or WISC). For example, Herrera-Chinchay et al. (2021) studied children aged 2–16 years; investigators administered the WPPSI to children aged 2–7 years, and the WISC to other children from 6 up to 16 years of age. Studies that used the Wechsler scales were conducted in 11 countries: Mexico (n = 13), Argentina (n = 6), Colombia (n = 6), Chile (n = 5), Peru (n = 5), Costa Rica (n = 4), Ecuador (n = 2), Bolivia (n = 2), Panama (n = 1), Cuba (n = 1), and Guatemala (n = 1).

The WPPSI was originally developed in 1967 and has had three new editions since: the WPPSI-R in 1989 (Wechsler, 1989), the WPPSI-III in 2002 (Wechsler, 2002), and the WPPSI-IV in 2012 (Wechsler, 2012). Each new edition of the WPPSI has encompassed a broader age range, the most recent version intended for use in children between 2 years 6 months of age and 7 years 7 months of age (Wechsler, 2012). The Spanish language version of the WPPSI (Escala de Inteligencia de Wechsler para preescolar y primaria [Pearson]) has been validated for use in Mexico and Spain with large normative samples, translation of language from the English version, and adaptation of language to cultural differences (Sánchez-Escobedo et al., 2016). Some studies in this review specifically reported use of the Mexican edition of the test (e.g., Cando et al., 2019) or the Spain-normed version of the test (e.g., Caulfield, 2011). Chasiotis et al. (2006) report use of a “Costa Rican version of the verbal IQ scale,” and McKay (1978) used a local adaptation of the original WPPSI, although it is unclear how these versions were modified from the original. Others (e.g., Warthon-Medina, 2015; Zambrano-Sánchez et al., 2010) report using the “Spanish version” of the test; however, this way of reporting is unclear whether this refers to the version normed in Spain or the Spanish-language edition with standardization in a Mexican population (e.g., Blouin et al., 2018 mention using “the Spanish version” in their methods, but cite a version of the WPPSI-III standardized in Mexico; Caulfield et al. [2011] also report using “the Spanish adaptation”, but cite a 1993 version from Argentina). Eleven studies (48%) that used the WPPSI (Álvarez González et al., 2004; Baroni et al., 2020; Clark, 2006; Galván et al., 2014; Hermida et al., 2019; Herrera-Chinchay et al. 2021; Lozoff et al., 2006; McInnes, 2005; Moguel-Ancheita et al., 2008; Morales-Buenrostro et al., 2019; Salinas-Piélago et al., 1998) did not specifically mention or cite a Spanish language edition, and Caraballo et al. (2004) did not specify in their methods which of the “Wechsler Intelligence” tests were used. While the assumption is that all tests were administered in the Spanish language, seven studies (30%) were published in English and did not mention the language of administration of the WPPSI for their participants (Baroni et al., 2020; Clark et al., 2006; Galván et al., 2014; Hermida et al., 2019; Lozoff et al., 2006; McInnes et al., 2005; McKay, 1978; Morales-Buenrostro et al., 2019).

Some studies, such as McKay et al. (1978) and Chasiotis et al. (2006), administered select subtests of the WPPSI thus were not able to render any composite index scores. Table 3 summarizes the WPPSI results across studies that assessed intelligence of typically developing children; of these 15 studies, nine reported mean IQ scores including FSIQ as well as VIQ and NVIQ. Most did not report their range of scores, only standard deviation. Mean FSIQ scores ranged from 86.61 (Warthon-Medina, 2015) to 108.56 (Morales-Buenrostro et al., 2019), VIQ scores from 83.31 (Warthon-Medina, 2015) to 104.33 (Morales-Buenrostro et al., 2019), and NVIQ scores from 96.39 (Cando et al., 2019) to 112.22 (Morales-Buenrostro et al., 2019). Six of nine (67.7%) of the studies with reported FSIQ scores had mean FSIQ under 95 which is below the measurement error of the WPPSI and falls in the lower end of the average range. Six studies did not report index scores, which was largely because the study only administered select subtests of the WPPSI (e.g., Chasiotis et al., 2006; McKay et al., 1978).

The WISC was originally developed in 1949 and, like the WPPSI, has been updated with the revised edition in 1974 (WISC-R), the third edition in 1991 (WISC-III), the fourth edition in 2003 (WISC-IV), and the fifth edition in 2014 (WISC-V). The most recent version of the test is intended to evaluate children from 6 years through 16 years 11 months of age (Kezer & Arik, 2012). Five studies in our analysis specifically report using the Mexican version of the WISC (Aguilar, 1993; Morales, 2013; Porras-Kattz, 2011; Rico, 2006; Van Wendel de Joode, 2016). Johnston et al. (1987) used a version of the WISC that was modified for a Guatemalan population but did not report standard scores due to lack of local norming. Many other remaining studies did not mention the edition and/or norms that were used.

McCarthy Scales of Children’s Abilities

The McCarthy Scales of Children’s Abilities (MSCA) were developed in 1972 to measure cognitive development among children aged 2 years 6 months to 8 years 6 months by assessing their perceptual performance; memory; and motor, quantitative, verbal, and general cognitive skills (Sands & D’Amato, 2017). Each domain is evaluated using between three and seven subtests that include both cognitive and motor activities. Nine studies in this review used the MSCA; they all took place in Mexico except one in Costa Rica (McInnes et al., 2005). Four studies (Gonzalez-Casanova et al., 2021; Grandjean, 2022; Osorio-Valencia, 2018; Schnaas et al., 2000) used a validated Spanish version of the MSCA (McCarthy, 2004) which was administered by trained psychologists at their respective institutions. Three studies (Braun, 2012; Knauer et al., 2018; Kupsco, 2020) used a translated and adapted version of the test from researchers at the Instituto Nacional de Perinatología in Mexico City. Osorio-Valencia et al. (2018) and Braun et al. (2012) reported use of the U.S. norms given the lack of official Mexican normative data for the test.

Raven’s Progressive Matrices (RPM, RCPM, RSPM)

The Raven’s Progressive Matrices (RPM) are a series of tests originally developed in the 1930s that measure cognitive ability through a nonverbal reasoning task. RPM was originally designed to limit the influence of cultural bias by employing a nonverbal measure of intelligence (Counter et al., 1998). The basic version of the test, Raven’s Standard Progressive Matrices (RSPM), relies on problem-solving using a series of black-and-white matrices that increase in difficulty and is validated for use in individuals aged 6 years through adulthood (Raven, 1936). Raven’s Colored Progressive Matrices (RCPM) is a shorter version of the test and is validated for use in children aged 5–11, the elderly, and adults with intellectual disabilities. Nine studies used the Raven’s Matrices tests in the countries of Argentina (n = 2), Chile (n = 2), Ecuador (n = 2), Bolivia (n = 1), Costa Rica (n = 1), and Mexico (n = 1). Population norms for the RPM tests have been established in Latin American regions such as Peru, Puerto Rico, Argentina, Guatemala, and Uruguay (Counter et al., 2019; Kihn, 2005). Rindermann et al. (2015) described the specific Austria and U.K.-based norms utilized for analysis of RCPM data in their cohort of Costa Rican children. Hill et al. (2014) acknowledged the lack of local Bolivian norms for their study population and reported using z-score analysis based on the mean scores of their own local patient samples in place of standard scores.

Child Psychological Assessment/Evaluación Neuropsicológica Infantil

The Evaluación Neuropsicológica Infantil (ENI) was published in 2007 and is the first neuropsychological battery designed specifically for use with Latin American children (Matute et al., 2007; Matute et al., 2022). The ENI was developed by the Laboratorio de Neuropsicología y Neurolingüística (Neuropsychology and Neurolinguistics Laboratory) at the University of Guadalajara, Mexico and normed using children from Colombia and Mexico between the ages of 5 and 16 (Matute et al., 2022). The ENI evaluates numerous cognitive and academic domains such as memory, perception, language, attention, reading, writing, arithmetic, visuospatial abilities, and executive function. In total, there are 13 domains, 36 subdomains, and 79 tasks in the battery (Matute et al., 2022).

Eight studies in this review assessed child cognitive development using the ENI. Two of these studies took place exclusively in Mexico (Bauer, 2014; Morales, 2013), two in Colombia (Pinto-Dussán et al., 2010; Urazán-Torres et al., 2013), and one each in Argentina (Juric, 2013) and Chile (Varela et al., 2020). The other two studies recruited participants from multiple countries; Ardila et al. (2011) assessed children in Colombia and Mexico, while Treviño et al. (2021) conducted assessments in Mexico, Colombia, and Guatemala. A second edition of the ENI (ENI-2) was published in 2013 (Matute et al., 2013) and a version of the ENI adapted for preschool-aged children aged 2–5 (ENI-P) was released in 2021 (Matute et al., 2021). Five of the studies in this review cite use of the first edition of the ENI published in 2007 (Ardila et al., 2011; Bauer, 2014; Juric, 2013; Morales, 2013; Urazán-Torres et al., 2013). Pinto-Dussán et al. (2010) do not explicitly cite an ENI manual, but based on the date of study publication the first edition must have been used, as no other edition had yet been published. Treviño et al. (2021) cite use of both the 2007 edition of the ENI as well as the ENI-P. Varela et al. (2020) was the only study to use the ENI-2.

Peabody Picture Vocabulary Test/Test de Vocabulario en Imágenes Peabody

The Peabody Picture Vocabulary Test (PPVT) was published in 1959 as a test of vocabulary developed for individuals between the ages of 2 years 6 months and 90 years of age. The PPVT was revised as the PPVT-R in 1981 (Dunn & Dunn, 1981), the PPVT-III in 1997 (Dunn & Dunn, 1997), the PPVT-IV in 2007 (Dunn & Dunn, 2007), and the PPVT-V in 2018 (Dunn, 2018). The Test de Vocabulario en Imágenes Peabody (TVIP) is the Spanish version of the test that was adapted from the PPVT-R in 1986 and has been used extensively in Spanish-speaking populations in the United States and Latin America (Bernal & Fernández, 2013). The Latin American version of the TVIP was standardized in both Mexican and Puerto Rican populations and includes a composite Hispanic set of norms that was developed from both samples of participants (Dunn & Dunn, 2007). A Spain edition based on the PPVT-III was published in 2010 (Dunn et al., 2010).

Of the seven studies that used any version of the PPVT/TVIP, three of them used the Latin American adaptation of the TVIP published in 1986 (Bernal & Fernández, 2013; Lohndorf, 2019; Rico, 2006). Two describe using a Spanish-language versions of the 1997 PPVT-III (Reynolds, 2022; Rosales-Rueda, 2018). Rosales-Rueda (2018) specifies that norms from Mexican and Puerto Rican children were used for scoring, and Reynolds (2022) notes that their study team also adapted some words to the Chilean context. Diuk et al. (2019) cite a 1986 version of the PPVT but do not make note of which Spanish adaptation was used. Aizawa (2021) uses versions of the PPVT that were adapted by local members of the study team as part of the Young Lives Study of poverty and inequality that has been tracking the development of 12,000 children across Ethiopia, India, Peru, and Vietnam since 2001.

Woodcock–Johnson, Woodcock Muñoz Battery, Woodcock Muñoz Language Survey

The Woodcock–Johnson (WJ) Tests of Cognitive Abilities assess general cognition, language, and academic achievement (Grizzle & Davis, 2011). The WJ was developed initially in 1977 and has been revised in 1989 (WJ-R), 2001 (WJ-III), and 2014 (WJ-IV). The Batería Woodcock–Muñoz (WMB) is the Spanish edition of the Woodcock–Johnson Tests of Cognitive Abilities. The test is validated in ages 5–95, but some subtests are appropriate for children as young as 2 years of age. The normative data for the WMB were acquired in Spanish-speaking individuals of Hispanic origin living in the United States (LaForte et al., 2019). Eight studies in this analysis utilized the WMB tests or its subtests (Bernal & Fernández, 2013; Clark, 2006; Gomez & Yoshikawa, 2017; Korzeniowski et al., 2017; Lamminmäki et al., 1997; Leyva et al., 2015; Lozoff et al., 2006; Rosselli et al., 2001); one study cites WJ (Sánchez-Escobedo et al., 2019). Leyva et al. (2015) rely on the WMB along with the Woodcock–Muñoz Language Survey (WMLS) to assess language skills and literacy in Chilean kindergarteners. The WMLS is a test that focuses specifically on language proficiency (Muñoz-Sandoval et al., 2015); the Spanish version of the WMLS was also used by Castro et al. (2002).

Discussion

This systematic review illustrates the breadth of standardized neurodevelopmental assessments of cognition used in preschool-aged children in Latin America. We found large variability of measures across publications, even in a relatively narrow age range of preschool children. We identified 41 neurodevelopmental tests used to assess children’s abilities across 97 studies from 13 countries. The Wechsler intelligence scales were the most widely and consistently used assessments, appearing in studies from every decade of the results, in 11 countries, and half of the studies. Other common measures included the WJ tests, MSCA, RPM test, Child Neuropsychological Assessment (Evaluación Neuropsicológica Infantil), and the PPVT, which are all measures of cognitive development but have distinct strengths and weaknesses that differentiate them from one another. Twenty-eight additional measures were used in between 1 and 5 studies in this review. Researchers and clinicians working with Spanish-speaking preschoolers may use the results of this review as a starting point as they select assessments that are most appropriate for their particular samples, regions, and goals.

Our review supports a need for continued research to validate, standardize, and ensure cultural relevance of cognitive assessments for this population. International studies on child neurodevelopment are complicated by a lack of normative data for all regions; findings from this review echo existing concerns in the field of neuropsychology about the appropriateness of internationally developed standardized assessments available for use in multicultural settings without additional research (Shuttleworth-Edwards, 2016; Statucka et al., 2021). For instance, the Wechsler scales have norms available for children from Mexico and Spain, but not other Spanish-speaking areas of the world. Many researchers in Latin America have utilized the Spain edition of the test; however, it remains unclear whether these norms and even the items are culturally and linguistically appropriate for populations of children in Latin America, particularly in low-income and low-resource areas. For example, the Similarities subtest of the WPPSI requires respondents to understand the connection between two seasons (winter and spring), which relies on an understanding of four seasons in the way that those in Spain or the United States do; the common understanding in equatorial countries in Latin America may be that there is only a dry season and a rainy season (Louison, 2016). Additionally, the Spanish language has numerous regional dialects; any testing of specific knowledge on standardized assessments would require that vocabulary words be specific to the regions in which testing occurs. Critiques that these norms are not appropriate were supported by our descriptive analysis of studies assessing typically developing children using the WPPSI. Of the three studies assessed with a reported mean FSIQ scores above 95, two were from Mexico using normative data from Mexico. Of the remaining six studies that reported IQ scores, the mean FSIQ was below 95. A cutoff of 95 is significant because the measurement error for the WPPSI is ∼5 IQ points. Five of six of the studies that used normative data from a country outside their sample reported a downward shift in IQ that falls in the lower end of the average range. This has implications for how an investigator interprets their results if they are assuming that the average IQ score is supposed to be 100. Many other neurodevelopmental assessments used by studies in this review, even those with Spanish-language norms such as the MSCA, have only been officially normed within the United States. While other assessments such as the PPVT have been normed in Spanish-speaking populations of Latin America, even these tests are only generalizable to a limited number of countries in the region (e.g., Mexico). Normative samples should reflect the vast socioeconomic and cultural diversity of Spanish-speaking Latin America to best characterize the neurodevelopmental performance of children.

Most studies in our review did not report why they selected certain assessments, which editions or adaptations of assessments were used, or which normative samples were used to develop scores for their selected assessments, which limits generalizability of their results. It is possible that researchers faced barriers to accessing and utilizing certain assessments, such as high costs of obtaining a license for use. For tests in which normative versions exist in both Spain and Latin America, it was often unclear whether using the “Spanish” version of a test (a description often used in study methods) implies the use of Spain norms, or simply a translated version of the test into the Spanish language with Latin American norms. MSCA has been appraised as an outdated test given its lack of re-norming since the early 1970s (Levin, 2011; Semrud-Clikeman et al., 2017). Braun et al. (2012) described the MSCA as a “valid” and “reliable” test, yet others who used the MSCA did not provide insight into why they chose the MSCA over other modern and more recently normed scales of child abilities and intelligence. Obtaining accurate norms is an expensive endeavor by publishing companies, and test developers often have limitations in what population is available to establish their normative data. As such, researchers often utilize normative data cited in research articles (Neils-Strunjaš, 1998) for updated and demographically relevant norms as a next best option. To improve international child development research moving forward, it would be beneficial for authors to be very specific in describing research methods about what normative datasets were used and the reasons for such comparisons. Future researchers should use caution when selecting tests of cognitive functioning to ensure that rigorous testing has allowed them to maintain validity and reliability in the modern age, clearly state if they made any adaptations to their chosen assessment, and discuss ways in which the assessment they chose may be biased or may not be fully applicable to their specific study sample.

Strengths and limitations

This systematic review focuses on performance-based child assessments rather than proxy questionnaires by parents, teachers, or caregivers. A systematic review by Semrud-Clikeman et al. (2017) examined the use of neurodevelopmental assessments in LMICs; however, the authors considered both parent-rating forms and direct child assessments. While proxy questionnaires are useful tools for measuring child development and may be used in conjunction with direct child assessments, the latter often provide different information regarding a child’s functioning and are sometimes preferred to proxy questionnaires (Munoz-Chereau et al., 2021). Future studies should review the proxy questionnaire measures of child development used in Latin America so that clinicians and researchers can select the type of tool or combination of tools that is best for the necessary evaluation. Furthermore, we excluded any assessment or task that was developed specifically for a single study or study sample, as these assessments are hard to replicate and typically are not scored in a standardized manner (e.g., see assessment tools utilized in Puche-Navarro and Millán [2007] or Zimerman et al. [1982]). Our focus on standardized performance-based assessments limited the number of eligible child outcome studies to review, but enhanced the implication of our findings by providing a narrower scope of child assessment tools used. Since only standardized assessments were addressed in our review, our results can inform future research on child development in Spanish-speaking Latin America with tools that are well-established and used in research across various disciplines, which will improve data harmonization for multisite studies and future meta-analyses.

While much research on child cognitive development has occurred in high-resource settings, outcomes in LMICs are particularly important to study due to the differing threats to the health of developing children, such as adverse environmental, pathogenic, and socioeconomic exposures. Like children in other regions burdened by poverty, discrimination, gender inequity, and violence, Latin American children exposed to high rates of systemic inequality are at risk of adverse early childhood developmental outcomes (Hartwig, 2020). The early preschool years represent a particularly important period of neurodevelopment, wherein the brain undergoes a high proportion of its morphologic and physiological changes relative to other periods in the life course (Brown & Jernigan, 2012). Further, diagnosis of neurodevelopmental and cognitive delays during the preschool years allows for the introduction of crucial interventions while the brain remains plastic enough to respond to them.

The ability to harmonize data between diverse, multinational cohorts using standardized assessment tools can enhance the applicability and validity of study findings. For example, the Zika Virus Individual Participant Data Consortium and meta-analysis initiative aims to harmonize data from 28 countries and territories and 52 cohort studies to understand the neurodevelopmental outcomes of children with in utero Zika-virus exposure; however, heterogeneity of methods and outcome measures across studies presents a significant challenge to harmonization (Zika Virus Individual Participant Data Consortium, 2020). The worldwide coronavirus disease 2019 (COVID-19) pandemic had even more extensive geographic involvement than the Zika virus, amplifying the need for consistency in data reporting to mitigate the effects of COVID-19 on child development as efficiently and comprehensively as possible. Beyond infectious disease threats, relatively common conditions experienced in childhood such as congenital heart disease and epilepsy, and rare diseases such as hypothalamic hamartoma, urea cycle disorders, and white matter disease, would also benefit from data sharing and combined cohorts to strengthen research findings (e.g., Berl et al., 2017; Sanz et al., 2021).

There are several inherent limitations to our systematic review. First, this study was not developed as an appraisal of neurodevelopmental instruments, but rather as a description of the tests that are commonly utilized in LMIC populations of Spanish-speaking Latin America. Therefore, descriptions of the strengths of the individual assessments for a given study sample are outside the scope of this review; however, future research may seek to explore the utility of the most commonly used assessment tools identified in this study. Second, this study only evaluated the neurodevelopmental tools that have been utilized for research and not those used for clinical assessment; thus, this systematic review may not capture the full spectrum of neurodevelopmental evaluation tools used in Latin America for diagnostic purposes in this young child age range. We also were unable to ascertain whether there were variables such as funding limitations or inaccessibility of assessment tools which led researchers to choose one assessment over another or which potentially biased their methods. Third, although articles written in Spanish were included in the review and many were indexed in English, we recognize that some studies conducted in Latin America are written, published, and indexed in Spanish. Other internationally published research articles, even those written in English, may not be indexed on the search engines utilized by this study. Therefore, future work utilizing different databases or Spanish translations of words like “cognition” and “neurodevelopment” as search terms may yield additional results. Some articles that were indexed in our included databases were unavailable for download and thus not able be considered for inclusion in this analysis (e.g., Ayala et al., 1990, Mora et al., 1974). An additional limitation of this study is that we did not define time-based inclusion criteria and therefore did not preclude articles that may be using now-outdated neurodevelopmental assessment tools. Thus, this analysis is not necessarily a reflection of the most current tools, but rather a survey of child neurodevelopmental assessment tools that have been used across Spanish-speaking Latin America over many decades. As we were not able to include studies that were not indexed in the search engines used by this study, it is likely that studies published less recently (prior to the early 2000s) are more scarcely indexed as compared to recent studies. Thus, the results may not accurately reflect the distribution of the uses of neurodevelopmental assessments over time. While this review illustrates the range of assessments used across Spanish-speaking Latin America, researchers assessing child neurodevelopment in this region may also consider reviewing prior research conducted in Latin American regions with a primary language other than Spanish, such as Brazil. Regardless of the region in which their work is conducted, researchers committed to effectively measuring and interpreting child neurodevelopmental outcome data should consider whether an assessment tool and/or its available norms have been adapted to fit the needs and demographics of modern-day society for their samples and populations of interest.

Supplementary material

Supplementary material is available online at Journal of Pediatric Psychology (http://jpepsy.oxfordjournals.org/).

Data availability

The data underlying this article are available from the corresponding author upon reasonable request.

Author contributions

Meagan E. Williams (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Investigation [equal], Methodology [equal], Project administration [equal], Supervision [equal], Writing—original draft [equal], Writing—review & editing [equal]), Elizabeth A. Corn (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Investigation [equal], Methodology [equal], Project administration [equal], Writing—original draft [equal], Writing—review & editing [equal]), Santiago Martinez Ransanz (Conceptualization [equal], Data curation [equal], Investigation [equal], Methodology [equal], Writing—original draft [supporting]), Madison Berl (Conceptualization [equal], Funding acquisition [equal], Methodology [equal], Supervision [equal], Writing—review & editing [equal]), Regan Andringa-Seed (Data curation [supporting], Formal analysis [supporting], Writing—review & editing [supporting]), and Sarah Mulkey (Conceptualization [equal], Funding acquisition [equal], Methodology [equal], Supervision [lead], Writing—review & editing [equal])

Funding

This work was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health (grant number R01HD102445) (S.B.M.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Authors also received support from the DC Intellectual and Developmental Disabilities Research Center (NICHD grant number P50HD105328-01) (M.M.B.) and Thrasher Research Fund (S.B.M.).

Conflicts of interest

The authors have no conflicts of interest to disclose.

Acknowledgments

The authors would like to recognize and thank Children’s National research librarian Susan Keller, MLS, MS-HIT, and the Global Health Initiative program at Children’s National Hospital and George Washington University, for their support of this study.

References

Author notes