https://orcid.org/0000-0003-3649-9555
-
PDF
- Split View
-
Views
-
Cite
Cite
Yohannes Gebreegziabhere, Kassahun Habatmu, Matteo Cella, Atalay Alem, Development and Evaluation of a Cognitive Battery for People With Schizophrenia in Ethiopia, Schizophrenia Bulletin, Volume 50, Issue 4, July 2024, Pages 931–943, https://doi.org/10.1093/schbul/sbad178
Close - Share Icon Share
Abstract
Cognitive difficulties significantly burdened people with schizophrenia (PWS). However, cognitive assessment is often unavailable in low- and middle-income counties (LMICs) due to a lack of validated and culturally adapted cognitive assessment tools. In this study, we developed and evaluated a culturally sensitive cognitive battery for PWS in Ethiopia.
This study was conducted in three phases. First, we selected appropriate tests through an instrument selection procedure and created a new battery. Then, we rigorously adapted the tests using culturally competent procedures, including cognitive interviewing and expert meetings. Finally, we tested the new battery in 208 PWS and 208 controls. We evaluated its psychometric properties using advanced statistical techniques, including Item Response Theory (IRT).
The Ethiopian Cognitive Assessment battery for Schizophrenia (ECAS) was developed from three different batteries. Participants reported tests were easy to complete, and the raters found them easy to administer. All tests had good inter-rater reliability, and the composite score had very high test-retest reliability (ICC = 0.91). One-factor structure better represented the data with excellent internal consistency (α = .81). ECAS significantly differentiated PWS from controls with 77% sensitivity and 62% specificity at a Z-score ≤0.12 cut-off value. IRT analysis suggested that the battery functions best among moderately impaired participants (difficulty between −0.06 and 0.66).
ECAS is a practical, tolerable, reliable, and valid assessment of cognition. ECAS can supplement current assessment tools in LAMICs for PWS and can be used to measure cognitive intervention outcomes.
Introduction
Cognitive impairment is one of the key symptoms of schizophrenia. The Measurement And Treatment Research to Improve Cognition in Schizophrenia (MATRICS) group identified seven domains of cognition impaired in people with schizophrenia (PWS).1 These are associated with worse functional and clinical outcomes.2–5
Even if there is no conclusive evidence about the effect of antipsychotic medication on cognition, there is promising evidence that adjunctive pharmacological6–10 and psychosocial11–13 interventions can mitigate the impact of cognitive difficulties in PWS. However, these interventions are not offered routinely. One barrier to implementing and using interventions for cognition, particularly in low- and middle-income countries (LAMICs), is the limited availability of validated and culturally sensitive cognitive assessment tools and staff training. To scale up promising interventions, assessing the level of impairment with a contextually suitable and psychometrically sound measure is essential.
There are cognitive assessment tools for PWS which have been used extensively and internationally. One extensively translated battery is the Brief Assessment of Cognition in Schizophrenia (BACS).14 This measure was initially developed in the United States to assess five cognitive domains more impaired in PWS. It has been used both in high-income countries such as Japan,15 Germany,16 France,17 Spain,18 and China19 and LMICs such as Brazil,20,21 Lebanon,22 Serbia,23 Indonesia,24 Iran,25 and Malaysia.26 However, some of its tests might be difficult to administer in low-income settings due to the low education level and cultural differences. The cost of the tool also limits its use in LAMICs.
The MATRICS Consensus Cognitive Battery (MCCB) is also widely used internationally27 (http://matricsinc.org), with several translations. However, license cost and longer administration time make it less accessible to LAMICs settings.
To date, no published study has reported a complete adaptation and validation of cognitive tests for PWS from low-income settings.28 A previous review stressed that the adapted tests should consider challenges of cognitive assessment, such as the influence of education, culture-specific values, and population-specific norms.29
Hence, it is helpful to consider what existing batteries may be a good template for adaptation. Batteries with short administration time, limited resource use (which does not require a license and extensive training), and paper and pencil-based administration are better suited. Measure selection should also consider aspects such as familiarity with the material presented (eg, pictures, names, and activities) and using locally available materials (such as pens or objects [eg, cubes]). Therefore, the selection and adaptation of cognitive tests need strong input from local service users and clinicians.
In Ethiopia, Africa’s second most populous country, there is no culturally appropriate cognitive battery to assess cognition in PWS. The only adapted cognitive test available is for dementia screening using the mini-mental state examination (MMSE).30 Clinicians in Ethiopia occasionally use this test in PWS, but this is problematic as the test is not sensitive to mild impairments and does not evaluate the relevant cognitive difficulties in PWS.
Given the lack of specific and sensitive assessment tools for PWS in Ethiopia, this study aims to select a battery, adapt the tests, and evaluate a novel culturally apt performance-based cognitive battery for PWS in Ethiopia.
Methods
The study was conducted in three interrelated and consecutive phases after obtaining ethical approval from the Institutional Review Board of the College of Health Sciences at Addis Ababa University (Protocol number: 042/19/Psy).
Phase I: Measure Selection
We followed Prinsen et al’s31 four-step measure selection. In the first step, we conducted an umbrella review of the cognitive domains most affected in PWS and identified five domains.32 In the second step, after conducting a systematic review,28 we identified 12 performance-based cognitive measures validated in PWS in LMICs.28 Using pre-determined five criteria, BACS, MCCB, and CogState Battery (CSB) were ranked most suitable for adaptation.
In the third step, we evaluated the quality of psychometric properties reported for each battery using criteria for good measurement properties.33 We used the COnsensus based selection of health Measurement INstrument (COSMIN) criteria to assess the quality of each study.34 We synthesized evidence for each battery using the COSMIN best evidence synthesis criteria (Supplementary material 1). Finally, we ranked the batteries regarding feasibility using the adapted COSMIN feasibility checklist (Supplementary material 2) and conducted two expert meetings with two groups. The meetings included junior and senior experts and aimed to decide on the appropriate battery for adaptation. In the meeting, first, the group agreed on the different characteristics of a cognitive measure in Ethiopia and recommended appropriate measures for adaptation.
Phase II: The Adaptation Process
We adapted the selected tests for the battery following standardized instrument adaption guidelines.35–37
Translation.
Each of the tests was independently translated from English into Amharic by seven native Amharic speakers. Five of them were familiar with the concept, and the rest were language experts. The forward translators came together and produced one reconciled version, which was then back-translated by another group of four translators. Two were familiar with the concept but blind to the tests, and the rest were linguistic experts. Again, the back-translators came together and created one backward-translated version. Finally, a multidisciplinary group of experts compared the reconciled forward-translated version with the reconciled backward-translated version and the original version.
Cognitive Interviewing (Pre-testing).
The battery was pre-tested among purposively selected 15 PWS receiving care at Amanueal Mental Specialized Hospital (AMSH), Addis Ababa, Ethiopia. Participants had a diagnosis of schizophrenia and fulfilled the following additional list of inclusion criteria. (1) Age between 18 and 65. (2) In remission and able to communicate. (3) Fluent in Amharic and can read numbers. (4) Absence of diagnosed substance use disorder, neurological disorders, organic brain disease, and recent history of head injury with loss of conciseness. Characteristics of the study participants are presented in Supplementary material 3.
The first author (YG) conducted all the interviews in Amharic using a semi-structured interview guide. The topic guide was developed before the interview and was refined through the process. The main aim of the interview was to assess the acceptability of the tests. Participants were asked to rate each task on accessibility and acceptability—tasks with poor accessibility and acceptability rating modified.38
The interviews lasted an average of 40 min. All the interviews were transcribed verbatim, translated into English, and then coded and analyzed using Microsoft Excel. We employed thematic analysis to identify major challenges and difficulties.
Expert Consensus Meeting.
We further used an expert panel to improve the battery’s conceptual and content equivalence. The panel included ten experts, in line with Lynn’s39 recommendations. The mother tongue of the experts was Amharic, except one. A description of the characteristics of the participants is presented in Supplementary material 3.
Experts were asked to reflect on the test’s clarity and relevance and comment on the content equivalence of the battery. The experts also advanced possible issues to consider during the cognitive interviews and suggested solutions.
Phase III: Psychometric Evaluation (Pilot Testing)
Study Area, Population, Period, and Eligibility Criteria.
We conducted the study at AMSH and Zewditu Memorial Hospital (ZMH) in Addis Ababa. We sub-sampled participants from an ongoing study.40 PWS in the study aged between 18 and 65, who are stable, able to communicate and fluent in Amharic, and can read numbers, were included in the study. We excluded PWS in the study with a confirmed diagnosis of substance use disorder, neurological disorders, organic brain diseases, and a recent history of head injury with loss of conciseness. This study was conducted from March 29 to August 5, 2022.
We considered a sample of 208 PWS would give sufficient power to conduct exploratory factor analysis (EFA); of these, 48 were randomly selected for test-retest reliability at 4 to 8 weeks (average of 6 weeks). We also included 208 controls to determine known group validity. We maintained the groups to have similar mean regarding age, sex, and educational status. Another independent 20 PWS were included for inter-rater reliably (IRR) assessment.
Measures.
We collected data on the socio-demographic and clinical characteristics of the participants using a self-developed structured questionnaire. We used an interview-based cognitive tool, ie, the Cognitive Assessment Interview (CAI), for concurrent validity assessment. This has been reported elsewhere.41 We also used a performance-based battery. The adapted tests in the battery are listed below, and the details are in Supplementary material 4. ECAS measures six domains of cognition with the following seven tests.
Verbal Memory: Word List Learning Test (WLLT) from Prince et al.42
Working memory: Digit Sequencing Task (DST)14 and Corsi Block Tapping Test (CBTT).43 Digit Sequencing Task is from BACS, and it is the property of WCG.
Verbal fluency: Category Instances (Semantic Fluency): Animal Naming Test (ANT) from Prince et al.42
Attention and speed of information processing: Digit Symbol Substitution Tests (DSST)44 and Trail Making Test: Part A (TMT Part A).45
Executive function: Trail Making Test: Part B (TMT Part B).45
We recorded administration time for each test and the battery. Practicality and tolerability were assessed using a 5-point Likert scale item for each test and the entire battery. For three of the tests, support was needed during the pre-test; as a result, we recorded the level of support provided using a 4-point Likert scale.
Data Management and Analysis.
Field supervisors checked the completeness and appropriateness of each questionnaire daily. Data were double-entered in EpiData version 4.6.0.6 software and analyzed using Stata 17. Medcalc software46 was used for the receiver operating characteristic (ROC)-curve analysis.
We evaluated differences between PWS and controls using an independent sample t-test and a chi-square test. Since the tests in the battery were scored using different scales, we converted the raw scores of PWS into z-score using the control’s mean and standard deviation (SD) for each test. Afterward, we created an average composite score based on the sum of each test divided by seven. A higher standardized score reflects better cognitive function. Based on the recommendation of Woods47 and Davis,48 the antipsychotic doses were converted into chlorpromazine equivalent Defined Daily Dose (DDD).
A chi-square test was conducted to evaluate the difference in the time taken to administer and score the tests between PWS and controls.
We calculated the proportion of PWS and controls who rated the easiness of the tests to respond, for tolerability. We calculated the proportion of administrators who rated the easiness of the tests to administer, for practicality. Similarly, the proportion of participants who received support was also calculated.
We used a two-way mixed effect Intra-Class Correlation Coefficient (ICC 3, k) to examine IRR and test-retest reliability.49 We used a paired sample t-test to examine practice effect. We computed Cohen’s d to quantify the change from time one to time two administration.50
We considered significant floor or ceiling effects when over 15% of the participants achieved the lower or maximum score. We used Cronbach’s alpha (α) to determine the interrelatedness of the tests.51 We used Pearson product-moment correlation coefficient (r) to examine the correlation between each test in the battery with the composite score and each other.51
We conducted a common factor EFA to examine the structural validity of the battery in the clinical sample. We evaluated the normality of the data through a visual examination of histograms and analyzing kurtosis and skewness.52 Then, we confirmed that the correlation matrix was factorable with Bartlett’s test of sphericity and the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy (MSA).53 The number of factors to be extracted was decided using a combination of latent root criterion, parallel analysis, and percentage of variance explained. We used oblique rotation with the PROMAX approach. Adequacy of factor loadings was established a priori (ie, a practical significance level of 0.3 and a 0.4 statistical significance level).53 Cross-loading at those cut-offs was considered problematic. For factors to be considered separate, they are expected to have at least a minimum of three tests with significant loadings, internal consistency reliability ≥0.70, commonality >0.5 for each test, and theoretical meaningfulness.
Since there is no gold standard cognitive measure for the study population and setting, we used an interview-based tool, ie, the Amharic version of Cognitive Assessment Interview (CAI-A), to examine concurrent validity. We used the Spearman correlation coefficient rho (ρ) to determine the correlation between each CAI-A item and the total score with each test and the composite score of the battery.54,55
We used an independent sample t-test to determine the difference between PWS and controls in terms of cognitive performance. We computed Cohen’s d to quantify the difference.50
We used ROC-curve analysis56 to examine the performance of each test and the composite score in discriminating PWS from controls. We used the Youden index (J) to set the cut-offs.57,58 To determine the accuracy of the ROC-curve, we reported the Area Under the Curve (AUC).57
To determine where the tests give more information and the difficulty and discrimination ability of the tests, we conducted Item Response Theory (IRT) in the total sample.59–63 We checked the three assumptions of IRT analysis (i.e., unidimensionality, local independence, and monotonicity).64 We used the unidimensional two-parameters logistic (2pl) IRT model as appropriate for this study since we categorized the tests into two based on cut-offs from the ROC-curve analysis. We checked if the chosen model (i.e., 2pl) fits the data better than a more restrictive model (i.e., one-parameter logistic [1pl] IRT model) using a log-likelihood ratio test and Akaike’s information criterion (AIC).
We conducted a differential item functioning (DIF) analysis to determine whether or not participants with the same ability perform differently because of years of education.65 We conducted a Mantel-Haenszel Odds Ratio (MH OR) to quantify the direction and amount of bias for those tests with uniform DIF.
Results
Phase I: Measure Selection
The expert’s agreement on test characteristics suitable for Ethiopia is presented in Supplementary material 5. Experts scrutinized 23 tests from three batteries and selected seven tests to address six cognitive domains, which were collected under one battery called the Ethiopian Cognitive Assessment battery for Schizophrenia (ECAS) (see table 1 for details).
Decisions Made by the Research Team on Tests to be Included in the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS)
| Domain Following BACS Structure . | Experts Meeting . | Research Team . | |
|---|---|---|---|
| Selected Tests from the Presented Batteries . | Suggested Alternative Tests . | Decision Made . | |
| Verbal memory | List Learning Test from BACS | WLLT42,a | WLLT42,a |
| Working memory | DST form BACS (verbal test) WMS-III: Spatial Span (Non-verbal test) from MCCB | DST14,b (Verbal) CBTT43,c (Non-verbal) | |
| Motor speed | Token motor task from BACS | Drop this domain | |
| Verbal fluency | Category Instances (Semantic Fluency) and Phonetic or Letter Fluency from BACS | Category Instances (Semantic Fluency): ANT42,a | Category Instances (Semantic Fluency): ANT42,a |
| Attention and speed of information processing | Symbol Coding from BACS, Category Fluency Test: ANT and TMT: Part A from MCCB | DSST44,d and TMT Part A45,e | |
| Reasoning and problem-solving | No test was recommended | Stroop Test Color Trail Test | Decided to look more at Executive function with TMT Part B45,e |
| Domain Following BACS Structure . | Experts Meeting . | Research Team . | |
|---|---|---|---|
| Selected Tests from the Presented Batteries . | Suggested Alternative Tests . | Decision Made . | |
| Verbal memory | List Learning Test from BACS | WLLT42,a | WLLT42,a |
| Working memory | DST form BACS (verbal test) WMS-III: Spatial Span (Non-verbal test) from MCCB | DST14,b (Verbal) CBTT43,c (Non-verbal) | |
| Motor speed | Token motor task from BACS | Drop this domain | |
| Verbal fluency | Category Instances (Semantic Fluency) and Phonetic or Letter Fluency from BACS | Category Instances (Semantic Fluency): ANT42,a | Category Instances (Semantic Fluency): ANT42,a |
| Attention and speed of information processing | Symbol Coding from BACS, Category Fluency Test: ANT and TMT: Part A from MCCB | DSST44,d and TMT Part A45,e | |
| Reasoning and problem-solving | No test was recommended | Stroop Test Color Trail Test | Decided to look more at Executive function with TMT Part B45,e |
Note: ANT, Animal Naming Test; BACS, Brief Assessment of Cognition in Schizophrenia; CBTT, Corsi Block Tapping Task; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; MCCB, MATRICS Consensus Cognitive Battery; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; WLLT, Word List Learning Test; WMS-III, Wechsler Memory Scale-Third Edition.
afrom previous work of Prince et al.
bfrom previous work of Kessels et al.
cfrom BACS, and it is the property of WCG.
dfrom previous work of Jaeger J.
efrom previous work of Bowie CR and Harvey PD.
Decisions Made by the Research Team on Tests to be Included in the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS)
| Domain Following BACS Structure . | Experts Meeting . | Research Team . | |
|---|---|---|---|
| Selected Tests from the Presented Batteries . | Suggested Alternative Tests . | Decision Made . | |
| Verbal memory | List Learning Test from BACS | WLLT42,a | WLLT42,a |
| Working memory | DST form BACS (verbal test) WMS-III: Spatial Span (Non-verbal test) from MCCB | DST14,b (Verbal) CBTT43,c (Non-verbal) | |
| Motor speed | Token motor task from BACS | Drop this domain | |
| Verbal fluency | Category Instances (Semantic Fluency) and Phonetic or Letter Fluency from BACS | Category Instances (Semantic Fluency): ANT42,a | Category Instances (Semantic Fluency): ANT42,a |
| Attention and speed of information processing | Symbol Coding from BACS, Category Fluency Test: ANT and TMT: Part A from MCCB | DSST44,d and TMT Part A45,e | |
| Reasoning and problem-solving | No test was recommended | Stroop Test Color Trail Test | Decided to look more at Executive function with TMT Part B45,e |
| Domain Following BACS Structure . | Experts Meeting . | Research Team . | |
|---|---|---|---|
| Selected Tests from the Presented Batteries . | Suggested Alternative Tests . | Decision Made . | |
| Verbal memory | List Learning Test from BACS | WLLT42,a | WLLT42,a |
| Working memory | DST form BACS (verbal test) WMS-III: Spatial Span (Non-verbal test) from MCCB | DST14,b (Verbal) CBTT43,c (Non-verbal) | |
| Motor speed | Token motor task from BACS | Drop this domain | |
| Verbal fluency | Category Instances (Semantic Fluency) and Phonetic or Letter Fluency from BACS | Category Instances (Semantic Fluency): ANT42,a | Category Instances (Semantic Fluency): ANT42,a |
| Attention and speed of information processing | Symbol Coding from BACS, Category Fluency Test: ANT and TMT: Part A from MCCB | DSST44,d and TMT Part A45,e | |
| Reasoning and problem-solving | No test was recommended | Stroop Test Color Trail Test | Decided to look more at Executive function with TMT Part B45,e |
Note: ANT, Animal Naming Test; BACS, Brief Assessment of Cognition in Schizophrenia; CBTT, Corsi Block Tapping Task; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; MCCB, MATRICS Consensus Cognitive Battery; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; WLLT, Word List Learning Test; WMS-III, Wechsler Memory Scale-Third Edition.
afrom previous work of Prince et al.
bfrom previous work of Kessels et al.
cfrom BACS, and it is the property of WCG.
dfrom previous work of Jaeger J.
efrom previous work of Bowie CR and Harvey PD.
Phase II: The Adaptation Process
The forward and backward translation produced draft I of ECAS, and the harmonization meeting yielded a second draft of the battery. We identified two major themes based on 15 complete interviews, ie, inclusion criteria-related issues and difficulties at the task level, discussed during the expert’s meeting. Based on the cognitive interview findings and experts’ recommendations. Some of the changes include cutting steps of CBTT, removing the upper limits of all time-based tests until we have our own normative reference, and changing the scoring of DSST from the number of boxes completed in 120 seconds to the time to complete the task. Supplementary material 6 summarizes the cognitive interview results, the expert’s suggestions, and draft II of ECAS changes. These procedures yielded draft III of ECAS.
Phase III: Psychometric Evaluation (Pilot Testing)
Socio-demographic and Clinical Characteristics of the Study Participants.
Table 2 shows details of the socio-demographic and clinical characteristics of the participants. There was no statistically significant difference between PWS and controls regarding most variables, including sex, age, and years of education (P > .05).
| Socio-demographic Characteristics . | PWS (n = 208) . | Control (n = 208) . | P-value . | Test-retest (n = 48) . | P-value . |
|---|---|---|---|---|---|
| Sex, % male | 67 | 58 | .068 | 62 | .468 |
| Age in years, mean (SD) | 37.1 (9.3) | 35.4 (8.8) | .052 | 35.5 (8.1) | .189 |
| Education in years, mean (SD) | 10.8 (3.2) | 11.1 (2.9) | .371 | 11.6 (2.8) | .049 |
| Marital status, % | |||||
| Single | 65 | 38 | <.001 | 56 | .073 |
| Married | 24 | 55 | 27 | ||
| Separated | 1 | 1 | 4 | ||
| Widowed | 1 | 2 | 2 | ||
| Divorced | 10 | 3 | 10 | ||
| Occupational status, % | |||||
| Unable to work | 7 | — | <.001 | — | .009 |
| Unemployed | 25 | 3 | 38 | ||
| Farmer | 4 | 1 | — | ||
| Daily laborer | 7 | 2 | 4 | ||
| Student | 4 | 3 | 8 | ||
| Government employee | 14 | 56 | 8 | ||
| Private business | 39 | 23 | 30 | ||
| NGO employee | 2 | 8 | 4 | ||
| Housewife | 3 | 3 | 6 | ||
| Other* | — | 0.5 | — | ||
| Monthly income in USD†, median | 58.0 | 58.0 | .010 | 58.0 | .446 |
| Relative wealth, % | |||||
| Low | 58 | 64 | .391 | 54 | .681 |
| Medium | 36 | 30 | 42 | ||
| High | 5 | 5 | 4 | ||
| Religion, % | |||||
| Orthodox | 57 | 73 | <.001 | 58 | .595 |
| Muslim | 28 | 8 | 25 | ||
| Protestant | 14 | 19 | 12 | ||
| Other** | 2 | 1 | 4 | ||
| Residence, % urban | 92 | 97 | .054 | 98 | .096 |
| Clinical characteristics | |||||
| Illness course††, % | |||||
| Episodic | 24 | — | — | 35 | .055 |
| Continues | 6 | — | — | 8 | |
| Remission (partial/complete) | 70 | — | — | 56 | |
| Age of onset in years, mean (SD) | 25.0 (8.0) | — | — | 24.7 (8.2) | .792 |
| DOI in years, mean (SD) | 12.1 (8.7) | — | — | 10.8 (7.3) | .239 |
| Years in treatment, mean (SD) | 10.8 (8.2) | — | — | 9.71 (7.3) | .308 |
| Number of admissions†, % | |||||
| 0 | 91 | — | — | 94 | .57 |
| 1 | 8 | — | — | 6 | |
| 2 | 1 | — | — | — | |
| Types of antipsychotics, % | |||||
| Typical | 64 | — | — | 15 | .12 |
| Atypical | 29 | — | — | 77 | |
| Mix | 8 | — | — | 8 | |
| Chlorpromazine equivalent DDD in mg/day, mean (SD) median | 326.8 (480.3) 200 | — | — | 346.4 (630.5) 200 | .425 |
| Socio-demographic Characteristics . | PWS (n = 208) . | Control (n = 208) . | P-value . | Test-retest (n = 48) . | P-value . |
|---|---|---|---|---|---|
| Sex, % male | 67 | 58 | .068 | 62 | .468 |
| Age in years, mean (SD) | 37.1 (9.3) | 35.4 (8.8) | .052 | 35.5 (8.1) | .189 |
| Education in years, mean (SD) | 10.8 (3.2) | 11.1 (2.9) | .371 | 11.6 (2.8) | .049 |
| Marital status, % | |||||
| Single | 65 | 38 | <.001 | 56 | .073 |
| Married | 24 | 55 | 27 | ||
| Separated | 1 | 1 | 4 | ||
| Widowed | 1 | 2 | 2 | ||
| Divorced | 10 | 3 | 10 | ||
| Occupational status, % | |||||
| Unable to work | 7 | — | <.001 | — | .009 |
| Unemployed | 25 | 3 | 38 | ||
| Farmer | 4 | 1 | — | ||
| Daily laborer | 7 | 2 | 4 | ||
| Student | 4 | 3 | 8 | ||
| Government employee | 14 | 56 | 8 | ||
| Private business | 39 | 23 | 30 | ||
| NGO employee | 2 | 8 | 4 | ||
| Housewife | 3 | 3 | 6 | ||
| Other* | — | 0.5 | — | ||
| Monthly income in USD†, median | 58.0 | 58.0 | .010 | 58.0 | .446 |
| Relative wealth, % | |||||
| Low | 58 | 64 | .391 | 54 | .681 |
| Medium | 36 | 30 | 42 | ||
| High | 5 | 5 | 4 | ||
| Religion, % | |||||
| Orthodox | 57 | 73 | <.001 | 58 | .595 |
| Muslim | 28 | 8 | 25 | ||
| Protestant | 14 | 19 | 12 | ||
| Other** | 2 | 1 | 4 | ||
| Residence, % urban | 92 | 97 | .054 | 98 | .096 |
| Clinical characteristics | |||||
| Illness course††, % | |||||
| Episodic | 24 | — | — | 35 | .055 |
| Continues | 6 | — | — | 8 | |
| Remission (partial/complete) | 70 | — | — | 56 | |
| Age of onset in years, mean (SD) | 25.0 (8.0) | — | — | 24.7 (8.2) | .792 |
| DOI in years, mean (SD) | 12.1 (8.7) | — | — | 10.8 (7.3) | .239 |
| Years in treatment, mean (SD) | 10.8 (8.2) | — | — | 9.71 (7.3) | .308 |
| Number of admissions†, % | |||||
| 0 | 91 | — | — | 94 | .57 |
| 1 | 8 | — | — | 6 | |
| 2 | 1 | — | — | — | |
| Types of antipsychotics, % | |||||
| Typical | 64 | — | — | 15 | .12 |
| Atypical | 29 | — | — | 77 | |
| Mix | 8 | — | — | 8 | |
| Chlorpromazine equivalent DDD in mg/day, mean (SD) median | 326.8 (480.3) 200 | — | — | 346.4 (630.5) 200 | .425 |
Note: Bold is for a significantly associated variable at a P-value of less than .05 between PWS and controls or between PWS from the total sample and those involved in test-retest. We converted all the atypical antipsychotics dose into chlorpromazine equivalent defined daily dose (DDD) based on the recommendation of Woods.47 For the typical antipsychotics, we used the study by Davis.48
DDD, defined daily doze; DOI, duration of illness; NGO, None Government Organization.
*Pension; **Catholic, Jehovah (n = 3), Adventist, Wake feta.
†1 USD = 51.76 ETB during the study period.
††Assessed for the last 2 years; bold is for P-value < .05.
| Socio-demographic Characteristics . | PWS (n = 208) . | Control (n = 208) . | P-value . | Test-retest (n = 48) . | P-value . |
|---|---|---|---|---|---|
| Sex, % male | 67 | 58 | .068 | 62 | .468 |
| Age in years, mean (SD) | 37.1 (9.3) | 35.4 (8.8) | .052 | 35.5 (8.1) | .189 |
| Education in years, mean (SD) | 10.8 (3.2) | 11.1 (2.9) | .371 | 11.6 (2.8) | .049 |
| Marital status, % | |||||
| Single | 65 | 38 | <.001 | 56 | .073 |
| Married | 24 | 55 | 27 | ||
| Separated | 1 | 1 | 4 | ||
| Widowed | 1 | 2 | 2 | ||
| Divorced | 10 | 3 | 10 | ||
| Occupational status, % | |||||
| Unable to work | 7 | — | <.001 | — | .009 |
| Unemployed | 25 | 3 | 38 | ||
| Farmer | 4 | 1 | — | ||
| Daily laborer | 7 | 2 | 4 | ||
| Student | 4 | 3 | 8 | ||
| Government employee | 14 | 56 | 8 | ||
| Private business | 39 | 23 | 30 | ||
| NGO employee | 2 | 8 | 4 | ||
| Housewife | 3 | 3 | 6 | ||
| Other* | — | 0.5 | — | ||
| Monthly income in USD†, median | 58.0 | 58.0 | .010 | 58.0 | .446 |
| Relative wealth, % | |||||
| Low | 58 | 64 | .391 | 54 | .681 |
| Medium | 36 | 30 | 42 | ||
| High | 5 | 5 | 4 | ||
| Religion, % | |||||
| Orthodox | 57 | 73 | <.001 | 58 | .595 |
| Muslim | 28 | 8 | 25 | ||
| Protestant | 14 | 19 | 12 | ||
| Other** | 2 | 1 | 4 | ||
| Residence, % urban | 92 | 97 | .054 | 98 | .096 |
| Clinical characteristics | |||||
| Illness course††, % | |||||
| Episodic | 24 | — | — | 35 | .055 |
| Continues | 6 | — | — | 8 | |
| Remission (partial/complete) | 70 | — | — | 56 | |
| Age of onset in years, mean (SD) | 25.0 (8.0) | — | — | 24.7 (8.2) | .792 |
| DOI in years, mean (SD) | 12.1 (8.7) | — | — | 10.8 (7.3) | .239 |
| Years in treatment, mean (SD) | 10.8 (8.2) | — | — | 9.71 (7.3) | .308 |
| Number of admissions†, % | |||||
| 0 | 91 | — | — | 94 | .57 |
| 1 | 8 | — | — | 6 | |
| 2 | 1 | — | — | — | |
| Types of antipsychotics, % | |||||
| Typical | 64 | — | — | 15 | .12 |
| Atypical | 29 | — | — | 77 | |
| Mix | 8 | — | — | 8 | |
| Chlorpromazine equivalent DDD in mg/day, mean (SD) median | 326.8 (480.3) 200 | — | — | 346.4 (630.5) 200 | .425 |
| Socio-demographic Characteristics . | PWS (n = 208) . | Control (n = 208) . | P-value . | Test-retest (n = 48) . | P-value . |
|---|---|---|---|---|---|
| Sex, % male | 67 | 58 | .068 | 62 | .468 |
| Age in years, mean (SD) | 37.1 (9.3) | 35.4 (8.8) | .052 | 35.5 (8.1) | .189 |
| Education in years, mean (SD) | 10.8 (3.2) | 11.1 (2.9) | .371 | 11.6 (2.8) | .049 |
| Marital status, % | |||||
| Single | 65 | 38 | <.001 | 56 | .073 |
| Married | 24 | 55 | 27 | ||
| Separated | 1 | 1 | 4 | ||
| Widowed | 1 | 2 | 2 | ||
| Divorced | 10 | 3 | 10 | ||
| Occupational status, % | |||||
| Unable to work | 7 | — | <.001 | — | .009 |
| Unemployed | 25 | 3 | 38 | ||
| Farmer | 4 | 1 | — | ||
| Daily laborer | 7 | 2 | 4 | ||
| Student | 4 | 3 | 8 | ||
| Government employee | 14 | 56 | 8 | ||
| Private business | 39 | 23 | 30 | ||
| NGO employee | 2 | 8 | 4 | ||
| Housewife | 3 | 3 | 6 | ||
| Other* | — | 0.5 | — | ||
| Monthly income in USD†, median | 58.0 | 58.0 | .010 | 58.0 | .446 |
| Relative wealth, % | |||||
| Low | 58 | 64 | .391 | 54 | .681 |
| Medium | 36 | 30 | 42 | ||
| High | 5 | 5 | 4 | ||
| Religion, % | |||||
| Orthodox | 57 | 73 | <.001 | 58 | .595 |
| Muslim | 28 | 8 | 25 | ||
| Protestant | 14 | 19 | 12 | ||
| Other** | 2 | 1 | 4 | ||
| Residence, % urban | 92 | 97 | .054 | 98 | .096 |
| Clinical characteristics | |||||
| Illness course††, % | |||||
| Episodic | 24 | — | — | 35 | .055 |
| Continues | 6 | — | — | 8 | |
| Remission (partial/complete) | 70 | — | — | 56 | |
| Age of onset in years, mean (SD) | 25.0 (8.0) | — | — | 24.7 (8.2) | .792 |
| DOI in years, mean (SD) | 12.1 (8.7) | — | — | 10.8 (7.3) | .239 |
| Years in treatment, mean (SD) | 10.8 (8.2) | — | — | 9.71 (7.3) | .308 |
| Number of admissions†, % | |||||
| 0 | 91 | — | — | 94 | .57 |
| 1 | 8 | — | — | 6 | |
| 2 | 1 | — | — | — | |
| Types of antipsychotics, % | |||||
| Typical | 64 | — | — | 15 | .12 |
| Atypical | 29 | — | — | 77 | |
| Mix | 8 | — | — | 8 | |
| Chlorpromazine equivalent DDD in mg/day, mean (SD) median | 326.8 (480.3) 200 | — | — | 346.4 (630.5) 200 | .425 |
Note: Bold is for a significantly associated variable at a P-value of less than .05 between PWS and controls or between PWS from the total sample and those involved in test-retest. We converted all the atypical antipsychotics dose into chlorpromazine equivalent defined daily dose (DDD) based on the recommendation of Woods.47 For the typical antipsychotics, we used the study by Davis.48
DDD, defined daily doze; DOI, duration of illness; NGO, None Government Organization.
*Pension; **Catholic, Jehovah (n = 3), Adventist, Wake feta.
†1 USD = 51.76 ETB during the study period.
††Assessed for the last 2 years; bold is for P-value < .05.
Feasibility.
The participants rated the tests high for engagement, with mean scores ranging from 3.83 for DSST to 4.14 for the ANT. Only very few participants required support to perform the tasks in the tests. However, PWS needed significantly more support than controls for DSST and TMT Part B (Supplementary material 7; table 1).
The overall practicality score for the ECAS fell above the scale midpoint (3.75), and scores for individual tests ranged from 3.38 to 4.38. The most challenging test for administration and scoring was TMT Part B, where 50% (n = 4) of the administrators rated this test as neutral or difficult to administer (Supplementary material 7: figures 1 and 2). In response to the open-ended questions, the test administrators raised points for improvement, which we considered in improving draft III of ECSA, which includes adding a trial before starting the actual test.
The tests took an average of 35.45 ± 3.85 min in PWS. This is significantly longer than the mean administration time in controls (27.95 ± 4.52 min, P = .005). The mean time taken to administer and score the individual tests was also significantly longer in PWS than in controls (P < .05), except for WLLT. Individual test administration time is reported in Table 3.
Duration of Administration, Item-Level, and Factor Analysis of the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS) Among People With Schizophrenia
| Name of the Test . | Duration of Administration . | Item-Level Analysis . | Factor Analysis . | |||||
|---|---|---|---|---|---|---|---|---|
| PWS (n = 208) Mean ± SD (in Minutes) . | Controls (n = 208) Mean ± SD (in Minutes . | P-value . | Test-Total Correlation . | Minimum Score (n, % Endorsed) . | Maximum Score (n, % Endorsed) . | Factor 1 Factor Loading . | Communalities . | |
| WLLT | 3.11 (± 0.93) | 3.20 (± 0.93) | .317 | 0.50 | 3 (1.44) | 1 (0.48) | 0.37 | 0.14 |
| DST | 3.85 (± 1.78) | 3.43 (± 1.29) | .006 | 0.63 | 2 (0.96) | 52 (25.00) | 0.49 | 0.24 |
| CBTT | 2.62 (± 0.90) | 2.32 (± 0.96) | .001 | 0.62 | 2 (0.96) | 35 (16.83) | 0.53 | 0.28 |
| ANT | 2.38 (± 0.92) | 2.17 (± 0.56) | .005 | 0.64 | 3 (1.44) | 2 (0.96) | 0.53 | 0.28 |
| DSST | 12.32 (± 5.09) | 8.48 (± 3.78) | <.001 | 0.81 | 1 (0.48) | 1 (0.48) | 0.81 | 0.66 |
| TMT Part A | 2.81 (± 1.26) | 2.19 (± 0.97) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.79 | 0.63 |
| TMT Part B | 5.47 (± 2.47) | 3.95 (± 2.11) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.75 | 0.56 |
| Total | 35.45 (±9.22) | 27.95 (±7.25) | <.001 | 1 (0.48) | 1 (0.48) | |||
| Name of the Test . | Duration of Administration . | Item-Level Analysis . | Factor Analysis . | |||||
|---|---|---|---|---|---|---|---|---|
| PWS (n = 208) Mean ± SD (in Minutes) . | Controls (n = 208) Mean ± SD (in Minutes . | P-value . | Test-Total Correlation . | Minimum Score (n, % Endorsed) . | Maximum Score (n, % Endorsed) . | Factor 1 Factor Loading . | Communalities . | |
| WLLT | 3.11 (± 0.93) | 3.20 (± 0.93) | .317 | 0.50 | 3 (1.44) | 1 (0.48) | 0.37 | 0.14 |
| DST | 3.85 (± 1.78) | 3.43 (± 1.29) | .006 | 0.63 | 2 (0.96) | 52 (25.00) | 0.49 | 0.24 |
| CBTT | 2.62 (± 0.90) | 2.32 (± 0.96) | .001 | 0.62 | 2 (0.96) | 35 (16.83) | 0.53 | 0.28 |
| ANT | 2.38 (± 0.92) | 2.17 (± 0.56) | .005 | 0.64 | 3 (1.44) | 2 (0.96) | 0.53 | 0.28 |
| DSST | 12.32 (± 5.09) | 8.48 (± 3.78) | <.001 | 0.81 | 1 (0.48) | 1 (0.48) | 0.81 | 0.66 |
| TMT Part A | 2.81 (± 1.26) | 2.19 (± 0.97) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.79 | 0.63 |
| TMT Part B | 5.47 (± 2.47) | 3.95 (± 2.11) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.75 | 0.56 |
| Total | 35.45 (±9.22) | 27.95 (±7.25) | <.001 | 1 (0.48) | 1 (0.48) | |||
Note: ANT, Animal Naming Test; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; PWS, People With Schizophrenia; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; WLLT, World List Learning Test.
Duration of Administration, Item-Level, and Factor Analysis of the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS) Among People With Schizophrenia
| Name of the Test . | Duration of Administration . | Item-Level Analysis . | Factor Analysis . | |||||
|---|---|---|---|---|---|---|---|---|
| PWS (n = 208) Mean ± SD (in Minutes) . | Controls (n = 208) Mean ± SD (in Minutes . | P-value . | Test-Total Correlation . | Minimum Score (n, % Endorsed) . | Maximum Score (n, % Endorsed) . | Factor 1 Factor Loading . | Communalities . | |
| WLLT | 3.11 (± 0.93) | 3.20 (± 0.93) | .317 | 0.50 | 3 (1.44) | 1 (0.48) | 0.37 | 0.14 |
| DST | 3.85 (± 1.78) | 3.43 (± 1.29) | .006 | 0.63 | 2 (0.96) | 52 (25.00) | 0.49 | 0.24 |
| CBTT | 2.62 (± 0.90) | 2.32 (± 0.96) | .001 | 0.62 | 2 (0.96) | 35 (16.83) | 0.53 | 0.28 |
| ANT | 2.38 (± 0.92) | 2.17 (± 0.56) | .005 | 0.64 | 3 (1.44) | 2 (0.96) | 0.53 | 0.28 |
| DSST | 12.32 (± 5.09) | 8.48 (± 3.78) | <.001 | 0.81 | 1 (0.48) | 1 (0.48) | 0.81 | 0.66 |
| TMT Part A | 2.81 (± 1.26) | 2.19 (± 0.97) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.79 | 0.63 |
| TMT Part B | 5.47 (± 2.47) | 3.95 (± 2.11) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.75 | 0.56 |
| Total | 35.45 (±9.22) | 27.95 (±7.25) | <.001 | 1 (0.48) | 1 (0.48) | |||
| Name of the Test . | Duration of Administration . | Item-Level Analysis . | Factor Analysis . | |||||
|---|---|---|---|---|---|---|---|---|
| PWS (n = 208) Mean ± SD (in Minutes) . | Controls (n = 208) Mean ± SD (in Minutes . | P-value . | Test-Total Correlation . | Minimum Score (n, % Endorsed) . | Maximum Score (n, % Endorsed) . | Factor 1 Factor Loading . | Communalities . | |
| WLLT | 3.11 (± 0.93) | 3.20 (± 0.93) | .317 | 0.50 | 3 (1.44) | 1 (0.48) | 0.37 | 0.14 |
| DST | 3.85 (± 1.78) | 3.43 (± 1.29) | .006 | 0.63 | 2 (0.96) | 52 (25.00) | 0.49 | 0.24 |
| CBTT | 2.62 (± 0.90) | 2.32 (± 0.96) | .001 | 0.62 | 2 (0.96) | 35 (16.83) | 0.53 | 0.28 |
| ANT | 2.38 (± 0.92) | 2.17 (± 0.56) | .005 | 0.64 | 3 (1.44) | 2 (0.96) | 0.53 | 0.28 |
| DSST | 12.32 (± 5.09) | 8.48 (± 3.78) | <.001 | 0.81 | 1 (0.48) | 1 (0.48) | 0.81 | 0.66 |
| TMT Part A | 2.81 (± 1.26) | 2.19 (± 0.97) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.79 | 0.63 |
| TMT Part B | 5.47 (± 2.47) | 3.95 (± 2.11) | <.001 | 0.79 | 1 (0.48) | 1 (0.48) | 0.75 | 0.56 |
| Total | 35.45 (±9.22) | 27.95 (±7.25) | <.001 | 1 (0.48) | 1 (0.48) | |||
Note: ANT, Animal Naming Test; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; PWS, People With Schizophrenia; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; WLLT, World List Learning Test.
Reliability and Item-Level Analysis
Inter-rater Reliability
Inter-rater reliability (IRR) was good to excellent for most tests. In group one, the ICC values ranged from 0.62 to 0.99, except DST, whereas in group two, ICC values ranged from 0.85 to 0.98, except TMT Part B. Since the IRR for these two tests was poor, we gave additional training to all raters on scoring these tests before the actual data collection for the main study (Table 4).
Inter-rater Reliability, Test-Retest Reliability, and Practice Effect of the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS)
| Test . | Inter-rater Reliability (n = 20) . | Tests-Retest Reliability (n = 48) . | Practice Effect (n = 48) . | |||
|---|---|---|---|---|---|---|
| Group 1 (n = 11 PWS, 4 Raters) ICC (95% CI) . | Group 2 (n = 9 PWS, 3 Raters) ICC (95% CI) . | ICC . | First Assessment Mean (± SD) . | Second Assessment Mean (± SD) . | Cohen’s d (95% CI) . | |
| WLLT | 0.96 (0.90, 0.99)*** | 0.98 (0.95, 1.00)*** | 0.64 (0.33–0.80)*** | 15.94 (± 3.47) | 17.54 (± 3.80) | −0.44 (−0.84, −0.04)** |
| DST | 0.35 (0.06, 0.70)** | 0.98 (0.96, 0.99)*** | 0.79 (0.62–0.88)*** | 5.92 (± 1.57) | 5.81 (± 1.76) | 0.06 (−0.34, 0.46) |
| CBTT | 0.62 (0.34, 0.86)*** | 0.90 (0.74, 0.98)*** | 0.71 (0.48–0.84)*** | 32.56 (± 12.14) | 32.12 (± 12.18) | 0.04 (−0.36, 0.44) |
| ANT | 0.97 (0.92, 0.99)*** | 0.90 (0.73, 0.97)*** | 0.81 (0.65–0.89)*** | 15.17 (± 3.77) | 16.04 (± 3.74) | −0.23 (−0.63, 0.17) |
| DSST | 0.99 (0.97, 1.00)*** | 0.96 (0.87, 0.99)*** | 0.94 (0.82–0.98)*** | 574.27 (± 277.35) | 507.94 (± 220.97) | 0.26 (−0.14, 0.67)*** |
| TMT Part A | 0.93 (0.84, 0.98)** | 0.85 (0.62, 0.96)*** | 0.76 (0.57–0.87)*** | 104.10 (± 57.11) | 109.35 (± 85.39) | −0.07 (−0.47, 0.33) |
| TMT Part B | 0.97 (0.93, 0.99)*** | −0.00 (−0.29, 0.50) | 0.81 (0.56–0.91)*** | 267.75 (± 148.17) | 211.25 (± 16.88) | 0.42 (0.02, 0.83) *** |
| Composite score+ | 0.91 (0.83–0.95)*** | −0.81 (± 0.88) | −0.61 (± 0.96) | −0.22 (−0.62, 0.18)** | ||
| Test . | Inter-rater Reliability (n = 20) . | Tests-Retest Reliability (n = 48) . | Practice Effect (n = 48) . | |||
|---|---|---|---|---|---|---|
| Group 1 (n = 11 PWS, 4 Raters) ICC (95% CI) . | Group 2 (n = 9 PWS, 3 Raters) ICC (95% CI) . | ICC . | First Assessment Mean (± SD) . | Second Assessment Mean (± SD) . | Cohen’s d (95% CI) . | |
| WLLT | 0.96 (0.90, 0.99)*** | 0.98 (0.95, 1.00)*** | 0.64 (0.33–0.80)*** | 15.94 (± 3.47) | 17.54 (± 3.80) | −0.44 (−0.84, −0.04)** |
| DST | 0.35 (0.06, 0.70)** | 0.98 (0.96, 0.99)*** | 0.79 (0.62–0.88)*** | 5.92 (± 1.57) | 5.81 (± 1.76) | 0.06 (−0.34, 0.46) |
| CBTT | 0.62 (0.34, 0.86)*** | 0.90 (0.74, 0.98)*** | 0.71 (0.48–0.84)*** | 32.56 (± 12.14) | 32.12 (± 12.18) | 0.04 (−0.36, 0.44) |
| ANT | 0.97 (0.92, 0.99)*** | 0.90 (0.73, 0.97)*** | 0.81 (0.65–0.89)*** | 15.17 (± 3.77) | 16.04 (± 3.74) | −0.23 (−0.63, 0.17) |
| DSST | 0.99 (0.97, 1.00)*** | 0.96 (0.87, 0.99)*** | 0.94 (0.82–0.98)*** | 574.27 (± 277.35) | 507.94 (± 220.97) | 0.26 (−0.14, 0.67)*** |
| TMT Part A | 0.93 (0.84, 0.98)** | 0.85 (0.62, 0.96)*** | 0.76 (0.57–0.87)*** | 104.10 (± 57.11) | 109.35 (± 85.39) | −0.07 (−0.47, 0.33) |
| TMT Part B | 0.97 (0.93, 0.99)*** | −0.00 (−0.29, 0.50) | 0.81 (0.56–0.91)*** | 267.75 (± 148.17) | 211.25 (± 16.88) | 0.42 (0.02, 0.83) *** |
| Composite score+ | 0.91 (0.83–0.95)*** | −0.81 (± 0.88) | −0.61 (± 0.96) | −0.22 (−0.62, 0.18)** | ||
Note: ANT, Animal Naming Test; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; ICC, Intra-Class Coefficient; SD, Standard deviation; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; WLLT, World List Learning Test.
*P < .05; **P < .01, ***P < .001.
+The composite score is a standardized value, but the others are raw values.
Inter-rater Reliability, Test-Retest Reliability, and Practice Effect of the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS)
| Test . | Inter-rater Reliability (n = 20) . | Tests-Retest Reliability (n = 48) . | Practice Effect (n = 48) . | |||
|---|---|---|---|---|---|---|
| Group 1 (n = 11 PWS, 4 Raters) ICC (95% CI) . | Group 2 (n = 9 PWS, 3 Raters) ICC (95% CI) . | ICC . | First Assessment Mean (± SD) . | Second Assessment Mean (± SD) . | Cohen’s d (95% CI) . | |
| WLLT | 0.96 (0.90, 0.99)*** | 0.98 (0.95, 1.00)*** | 0.64 (0.33–0.80)*** | 15.94 (± 3.47) | 17.54 (± 3.80) | −0.44 (−0.84, −0.04)** |
| DST | 0.35 (0.06, 0.70)** | 0.98 (0.96, 0.99)*** | 0.79 (0.62–0.88)*** | 5.92 (± 1.57) | 5.81 (± 1.76) | 0.06 (−0.34, 0.46) |
| CBTT | 0.62 (0.34, 0.86)*** | 0.90 (0.74, 0.98)*** | 0.71 (0.48–0.84)*** | 32.56 (± 12.14) | 32.12 (± 12.18) | 0.04 (−0.36, 0.44) |
| ANT | 0.97 (0.92, 0.99)*** | 0.90 (0.73, 0.97)*** | 0.81 (0.65–0.89)*** | 15.17 (± 3.77) | 16.04 (± 3.74) | −0.23 (−0.63, 0.17) |
| DSST | 0.99 (0.97, 1.00)*** | 0.96 (0.87, 0.99)*** | 0.94 (0.82–0.98)*** | 574.27 (± 277.35) | 507.94 (± 220.97) | 0.26 (−0.14, 0.67)*** |
| TMT Part A | 0.93 (0.84, 0.98)** | 0.85 (0.62, 0.96)*** | 0.76 (0.57–0.87)*** | 104.10 (± 57.11) | 109.35 (± 85.39) | −0.07 (−0.47, 0.33) |
| TMT Part B | 0.97 (0.93, 0.99)*** | −0.00 (−0.29, 0.50) | 0.81 (0.56–0.91)*** | 267.75 (± 148.17) | 211.25 (± 16.88) | 0.42 (0.02, 0.83) *** |
| Composite score+ | 0.91 (0.83–0.95)*** | −0.81 (± 0.88) | −0.61 (± 0.96) | −0.22 (−0.62, 0.18)** | ||
| Test . | Inter-rater Reliability (n = 20) . | Tests-Retest Reliability (n = 48) . | Practice Effect (n = 48) . | |||
|---|---|---|---|---|---|---|
| Group 1 (n = 11 PWS, 4 Raters) ICC (95% CI) . | Group 2 (n = 9 PWS, 3 Raters) ICC (95% CI) . | ICC . | First Assessment Mean (± SD) . | Second Assessment Mean (± SD) . | Cohen’s d (95% CI) . | |
| WLLT | 0.96 (0.90, 0.99)*** | 0.98 (0.95, 1.00)*** | 0.64 (0.33–0.80)*** | 15.94 (± 3.47) | 17.54 (± 3.80) | −0.44 (−0.84, −0.04)** |
| DST | 0.35 (0.06, 0.70)** | 0.98 (0.96, 0.99)*** | 0.79 (0.62–0.88)*** | 5.92 (± 1.57) | 5.81 (± 1.76) | 0.06 (−0.34, 0.46) |
| CBTT | 0.62 (0.34, 0.86)*** | 0.90 (0.74, 0.98)*** | 0.71 (0.48–0.84)*** | 32.56 (± 12.14) | 32.12 (± 12.18) | 0.04 (−0.36, 0.44) |
| ANT | 0.97 (0.92, 0.99)*** | 0.90 (0.73, 0.97)*** | 0.81 (0.65–0.89)*** | 15.17 (± 3.77) | 16.04 (± 3.74) | −0.23 (−0.63, 0.17) |
| DSST | 0.99 (0.97, 1.00)*** | 0.96 (0.87, 0.99)*** | 0.94 (0.82–0.98)*** | 574.27 (± 277.35) | 507.94 (± 220.97) | 0.26 (−0.14, 0.67)*** |
| TMT Part A | 0.93 (0.84, 0.98)** | 0.85 (0.62, 0.96)*** | 0.76 (0.57–0.87)*** | 104.10 (± 57.11) | 109.35 (± 85.39) | −0.07 (−0.47, 0.33) |
| TMT Part B | 0.97 (0.93, 0.99)*** | −0.00 (−0.29, 0.50) | 0.81 (0.56–0.91)*** | 267.75 (± 148.17) | 211.25 (± 16.88) | 0.42 (0.02, 0.83) *** |
| Composite score+ | 0.91 (0.83–0.95)*** | −0.81 (± 0.88) | −0.61 (± 0.96) | −0.22 (−0.62, 0.18)** | ||
Note: ANT, Animal Naming Test; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; ICC, Intra-Class Coefficient; SD, Standard deviation; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; WLLT, World List Learning Test.
*P < .05; **P < .01, ***P < .001.
+The composite score is a standardized value, but the others are raw values.
Test-Retest Reliability and Practice Effect
Scores in each of the tests in the battery and the composite score in the first assessment significantly correlated with the scores in the second assessment conducted 4 to 8 weeks apart. Correlations ranged from moderate to very high (ICC = 0.64–0.94). The composite score of the battery was found to have very high test-retest reliability (ICC = 0.91, 95% CI [0.83–0.95]). We noted practice effect with small to medium effect sizes for WLLT, DSST, TMT Part B, and composite score (Cohen’s d values of −0.44, 0.26, 0.42, and −0.22, respectively) (Table 4).
Item-Level Analysis
Among PWS, all the tests had weak to moderate as well as significant correlations with one another (r = .25–.66, P < .001) except for WLLT and CBTT (P > .05). Moderate correlations were found in tests that measure similar domains (r = .38 for working memory domains and r = .66 for attention and speed of information processing domains) (Supplementary material 7; table 2). In the other direction, the test-total correlation ranged from moderate to high (r = .50–.81) (Table 3).
Among PWS, no floor effect was observed in any of the tests as well as in the composite score. Except for DST (25.0% endorsed the maximum score) and CBTT (16.8% endorsed the maximum score), there were no other tests with the maximum score endorsed by over 15% of the participants. The composite score had no ceiling effect (Table 3).
Validity
Structural Validity
Kurtosis and skewness tests showed normal distribution in most of the tests. Visualization of the histograms suggested normal distribution for WLLT, DST, CBTT, and ANT. Bartlett’s test of sphericity and KMO MSA confirmed the appropriateness of the correlation matrix for factor analysis (ie, Bartlett’s test of sphericity P < .001 and KMO MSA = 0.826). The analysis provided three factors, two of which had eigenvalue greater than one and explained 89.66% of the total variance. Horn’s parallel analysis also suggested two factors. However, theoretically, we expected one latent factor of cognition in PWS. Therefore, we examined the two- and one-factor solutions sequentially.
The two-factor solution was inappropriate because WLLT and DST did not load onto any of the factors at a level of statistical significance (0.4). At the practical significance level (0.3), DST loaded to factor one, but still, WLL did not load to any of the factors. Moreover, only one test (ANT) was loaded on factor two (with a factor loading considered abnormal [1.05]). As a result, we decided to continue with a one-factor structure and performed the analysis again.
In the one-factor solution, WLLT did not load at the statistical significance level (factor loading of 0.4); however, it is significantly loaded at the theoretical significance level (factor loading of 0.3). The one-factor solution explained 39.95% of the variance (suggesting a dominant factor). Except for DSST, TMT Part A, and TMT Part B, all the other tests had a commonality of less than 0.5. Changing the rotation as well as the extraction method did not increase the commonality. Therefore, we decided to include all the tests (for theoretical reasons). The coefficient alpha for the whole test was .81. Given these results, we concluded the one-factor solution to be an adequate structure for this battery (Table 3).
Concurrent Validity
We found a weak correlation between each of the ECAS tests and the global impression and the total score of CAI-A, except ANT and CBTT. We also found a weak correlation between ANT and CBTT and each item of the CAI-A. A weak correlation was also reported between the total score of CAI-A and the composite score of ECAS (Supplementary material 7; table 3).
Known Group and Criterion Validity
Compared with PWS, controls performed better on the composite score and each test, with a medium to large effect size (d between 0.4 and 1.04, P < .001) (table 5). On average, PWS scored 0.76 SD lower than controls in the composite score (Supplementary material 7; figure 3).
Known Group and Criterion Validity Analysis of the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS)
| Tests of the ECAS (Domain) (Scoring) . | Known Group Validity . | Criterion Validity . | ||||||
|---|---|---|---|---|---|---|---|---|
| Sample Type . | P-value . | Cohen’s d (Effect Size) (95% CI) . | Cut-off . | AUC (95% CI) . | Sensitivity . | Specificity . | ||
| PWS (n = 208) Mean ± SD . | Controls (n = 208) Mean ± SD . | |||||||
| WLLT (VL) (# of Words recalled) | 15.95 (± 3.29) | 19.42 (± 3.37) | <.001 | −1.04 (−1.25, −0.84) | ≤16 | 0.76 (0.72–0.80) *** | 0.58 (0.51–0.64) | 0.82 (0.76–0.87) |
| DST (verbal WM) (# of longest sequences) | 5.76 (±1.62) | 6.86 (±1.45) | <.001 | −0.72 (−0.92, −0.51) | ≤7 | 0.69 (0.64–0.73)*** | 0.75 (0.68–0.81) | 0.57 (0.50–0.64) |
| CBTT (visual WM) (# of Digit span * # of trail) | 34.79 (±12.62) | 39.83 (±12.24) | <.001 | −0.40 (−0.60, −0.21) | ≤30 | 0.61 (0.56–0.66)*** | 0.42 (0.36–0.49) | 0.77 (0.71–0.82) |
| ANT (VF) (# of Animals mentioned) | 14.85 (± 4.03) | 17.80 (± 4.69) | <.001 | −0.68 (−0.87, −0.48) | ≤16 | 0.68 (0.63–0.72)*** | 0.66 (0.60–0.73) | 0.60 (0.53–0.66) |
| DSST (At and SP) (Compilation seconds) | 600.33 (± 266.73) | 400.60 (±206.82) | <.001 | 0.84 (0.63, 1.04) | >539 | 0.78 (0.74–0.82) *** | 0.51 (0.44–0.58) | 0.90 (0.85–0.94) |
| TMT Part A (At & SP) (Compilation seconds) | 100.55 (±51.86) | 73.80 (±41.38) | <.001 | 0.57 (0.37, 0.77) | >65 | 0.70 (0.65–0.74)*** | 0.76 (0.70–0.82) | 0.55 (0.48–0.62) |
| TMT Part B (EF) (Compilation seconds) | 236.30 (±117.25) | 158.81 (± 84.72) | <.001 | 0.75 (0.56, 0.96) | >127 | 0.73 (0.69–0.76)*** | 0.88 (0.82–0.92) | 0.48 (0.41–0.55) |
| Average composite score (Standardized) | ≤0.12 | 0.78 (0.74–0.82)*** | 0.77 (0.71–0.82) | 0.62 (0.55–0.69) | ||||
| Tests of the ECAS (Domain) (Scoring) . | Known Group Validity . | Criterion Validity . | ||||||
|---|---|---|---|---|---|---|---|---|
| Sample Type . | P-value . | Cohen’s d (Effect Size) (95% CI) . | Cut-off . | AUC (95% CI) . | Sensitivity . | Specificity . | ||
| PWS (n = 208) Mean ± SD . | Controls (n = 208) Mean ± SD . | |||||||
| WLLT (VL) (# of Words recalled) | 15.95 (± 3.29) | 19.42 (± 3.37) | <.001 | −1.04 (−1.25, −0.84) | ≤16 | 0.76 (0.72–0.80) *** | 0.58 (0.51–0.64) | 0.82 (0.76–0.87) |
| DST (verbal WM) (# of longest sequences) | 5.76 (±1.62) | 6.86 (±1.45) | <.001 | −0.72 (−0.92, −0.51) | ≤7 | 0.69 (0.64–0.73)*** | 0.75 (0.68–0.81) | 0.57 (0.50–0.64) |
| CBTT (visual WM) (# of Digit span * # of trail) | 34.79 (±12.62) | 39.83 (±12.24) | <.001 | −0.40 (−0.60, −0.21) | ≤30 | 0.61 (0.56–0.66)*** | 0.42 (0.36–0.49) | 0.77 (0.71–0.82) |
| ANT (VF) (# of Animals mentioned) | 14.85 (± 4.03) | 17.80 (± 4.69) | <.001 | −0.68 (−0.87, −0.48) | ≤16 | 0.68 (0.63–0.72)*** | 0.66 (0.60–0.73) | 0.60 (0.53–0.66) |
| DSST (At and SP) (Compilation seconds) | 600.33 (± 266.73) | 400.60 (±206.82) | <.001 | 0.84 (0.63, 1.04) | >539 | 0.78 (0.74–0.82) *** | 0.51 (0.44–0.58) | 0.90 (0.85–0.94) |
| TMT Part A (At & SP) (Compilation seconds) | 100.55 (±51.86) | 73.80 (±41.38) | <.001 | 0.57 (0.37, 0.77) | >65 | 0.70 (0.65–0.74)*** | 0.76 (0.70–0.82) | 0.55 (0.48–0.62) |
| TMT Part B (EF) (Compilation seconds) | 236.30 (±117.25) | 158.81 (± 84.72) | <.001 | 0.75 (0.56, 0.96) | >127 | 0.73 (0.69–0.76)*** | 0.88 (0.82–0.92) | 0.48 (0.41–0.55) |
| Average composite score (Standardized) | ≤0.12 | 0.78 (0.74–0.82)*** | 0.77 (0.71–0.82) | 0.62 (0.55–0.69) | ||||
Note: ANT, Animal Naming Test; At, Attention; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; ECAS, Ethiopian version of Cognitive Assessment battery in Schizophrenia; EF, Executive Function; PWS, People with Schizophrenia; SP, Speed of Processing; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; VF, Verbal Fluency; VL, Verbal Learning; WLLT, World List Learning Test; WM, Working Memory.
***Is for P < .001.
Known Group and Criterion Validity Analysis of the Ethiopian Cognitive Assessment Battery for Schizophrenia (ECAS)
| Tests of the ECAS (Domain) (Scoring) . | Known Group Validity . | Criterion Validity . | ||||||
|---|---|---|---|---|---|---|---|---|
| Sample Type . | P-value . | Cohen’s d (Effect Size) (95% CI) . | Cut-off . | AUC (95% CI) . | Sensitivity . | Specificity . | ||
| PWS (n = 208) Mean ± SD . | Controls (n = 208) Mean ± SD . | |||||||
| WLLT (VL) (# of Words recalled) | 15.95 (± 3.29) | 19.42 (± 3.37) | <.001 | −1.04 (−1.25, −0.84) | ≤16 | 0.76 (0.72–0.80) *** | 0.58 (0.51–0.64) | 0.82 (0.76–0.87) |
| DST (verbal WM) (# of longest sequences) | 5.76 (±1.62) | 6.86 (±1.45) | <.001 | −0.72 (−0.92, −0.51) | ≤7 | 0.69 (0.64–0.73)*** | 0.75 (0.68–0.81) | 0.57 (0.50–0.64) |
| CBTT (visual WM) (# of Digit span * # of trail) | 34.79 (±12.62) | 39.83 (±12.24) | <.001 | −0.40 (−0.60, −0.21) | ≤30 | 0.61 (0.56–0.66)*** | 0.42 (0.36–0.49) | 0.77 (0.71–0.82) |
| ANT (VF) (# of Animals mentioned) | 14.85 (± 4.03) | 17.80 (± 4.69) | <.001 | −0.68 (−0.87, −0.48) | ≤16 | 0.68 (0.63–0.72)*** | 0.66 (0.60–0.73) | 0.60 (0.53–0.66) |
| DSST (At and SP) (Compilation seconds) | 600.33 (± 266.73) | 400.60 (±206.82) | <.001 | 0.84 (0.63, 1.04) | >539 | 0.78 (0.74–0.82) *** | 0.51 (0.44–0.58) | 0.90 (0.85–0.94) |
| TMT Part A (At & SP) (Compilation seconds) | 100.55 (±51.86) | 73.80 (±41.38) | <.001 | 0.57 (0.37, 0.77) | >65 | 0.70 (0.65–0.74)*** | 0.76 (0.70–0.82) | 0.55 (0.48–0.62) |
| TMT Part B (EF) (Compilation seconds) | 236.30 (±117.25) | 158.81 (± 84.72) | <.001 | 0.75 (0.56, 0.96) | >127 | 0.73 (0.69–0.76)*** | 0.88 (0.82–0.92) | 0.48 (0.41–0.55) |
| Average composite score (Standardized) | ≤0.12 | 0.78 (0.74–0.82)*** | 0.77 (0.71–0.82) | 0.62 (0.55–0.69) | ||||
| Tests of the ECAS (Domain) (Scoring) . | Known Group Validity . | Criterion Validity . | ||||||
|---|---|---|---|---|---|---|---|---|
| Sample Type . | P-value . | Cohen’s d (Effect Size) (95% CI) . | Cut-off . | AUC (95% CI) . | Sensitivity . | Specificity . | ||
| PWS (n = 208) Mean ± SD . | Controls (n = 208) Mean ± SD . | |||||||
| WLLT (VL) (# of Words recalled) | 15.95 (± 3.29) | 19.42 (± 3.37) | <.001 | −1.04 (−1.25, −0.84) | ≤16 | 0.76 (0.72–0.80) *** | 0.58 (0.51–0.64) | 0.82 (0.76–0.87) |
| DST (verbal WM) (# of longest sequences) | 5.76 (±1.62) | 6.86 (±1.45) | <.001 | −0.72 (−0.92, −0.51) | ≤7 | 0.69 (0.64–0.73)*** | 0.75 (0.68–0.81) | 0.57 (0.50–0.64) |
| CBTT (visual WM) (# of Digit span * # of trail) | 34.79 (±12.62) | 39.83 (±12.24) | <.001 | −0.40 (−0.60, −0.21) | ≤30 | 0.61 (0.56–0.66)*** | 0.42 (0.36–0.49) | 0.77 (0.71–0.82) |
| ANT (VF) (# of Animals mentioned) | 14.85 (± 4.03) | 17.80 (± 4.69) | <.001 | −0.68 (−0.87, −0.48) | ≤16 | 0.68 (0.63–0.72)*** | 0.66 (0.60–0.73) | 0.60 (0.53–0.66) |
| DSST (At and SP) (Compilation seconds) | 600.33 (± 266.73) | 400.60 (±206.82) | <.001 | 0.84 (0.63, 1.04) | >539 | 0.78 (0.74–0.82) *** | 0.51 (0.44–0.58) | 0.90 (0.85–0.94) |
| TMT Part A (At & SP) (Compilation seconds) | 100.55 (±51.86) | 73.80 (±41.38) | <.001 | 0.57 (0.37, 0.77) | >65 | 0.70 (0.65–0.74)*** | 0.76 (0.70–0.82) | 0.55 (0.48–0.62) |
| TMT Part B (EF) (Compilation seconds) | 236.30 (±117.25) | 158.81 (± 84.72) | <.001 | 0.75 (0.56, 0.96) | >127 | 0.73 (0.69–0.76)*** | 0.88 (0.82–0.92) | 0.48 (0.41–0.55) |
| Average composite score (Standardized) | ≤0.12 | 0.78 (0.74–0.82)*** | 0.77 (0.71–0.82) | 0.62 (0.55–0.69) | ||||
Note: ANT, Animal Naming Test; At, Attention; DSST, Digit Symbol Substitution Test; DST, Digit Sequencing Task; ECAS, Ethiopian version of Cognitive Assessment battery in Schizophrenia; EF, Executive Function; PWS, People with Schizophrenia; SP, Speed of Processing; TMT Part A, Trail Making Test: Part A; TMT Part B, Trail Making Test: Part B; VF, Verbal Fluency; VL, Verbal Learning; WLLT, World List Learning Test; WM, Working Memory.
***Is for P < .001.
ROC analysis revealed that ECAS significantly differentiates PWS from controls. At the cut-off point of z-score ≤0.12, the AUC was 0.78 (95% CI = 0.74–0.82, P < .001). Sensitivity and specificity were 0.77 and 0.62, respectively. The cut-off and the AUC for the rest of the tests were also significant and ranged from 0.61 to 0.73 (table 5).
Item Response Theory.
We found none of the tests to have a discrimination coefficient (a) >4, and the test characteristic curve is “S” shaped, suggesting the fulfillment of the local independence and monotonicity assumptions, respectively. The difficulty coefficient for each test ranged from −0.60 to 0.66. TMT Part B, TMT Part A, DST, and ANT were endorsed at lower ability levels. The discrimination coefficient for each test ranged from 1.33 to 3.22. DSST, TMT Part B, TMT Part A, and DST had higher discrimination abilities. The detailed IRT analysis is in Supplementary material 8.
We found that the chi-square test for the log-likelihood difference between the 2pl and 1pl models is significant (P-value < .001), and the AIC is lower for the 2pl model (i.e., 3192.48 for 1pl vs 3175.50 for 2pl). Therefore, we reject the null hypothesis and can conclude that the 2pl better fits the data.
Differential Item Functioning.
ANT and DSST showed a uniform DIF (P = .041 and .013, respectively) in terms of educational status. We found that the odds of those who have an educational level ≤8 years scoring lower results for ANT were 0.52 times less likely compared to those who have >8 years of education (MH OR = 0.52, 95%; CI = 0.28, 0.99, P = .066). The odds of those who have ≤8 years of education scoring lower results in DSST were 2.81 times higher compared to those who have >8 years of education (MH OR = 2.81, 95%; CI = 1.33, 5.95, P = .009) (Supplementary material 8; table 2). A summary of the findings of this study and the changes we made following the pilot study are presented in Supplementary material 9.
Discussion
In this study, we developed and evaluated ECAS (a novel and culturally competent cognitive assessment for PWS in Ethiopia). ECAS takes an average of 35 min and has minimal practice effect for most tests. Our results indicate that ECAS has excellent internal consistency and very high test-retest reliability, high IRR, and no floor and ceiling effects. This makes the tool ideal for studies measuring changes in cognition using different raters and in different periods. The study also suggests that ECAS is valid. We found a one-factor structure explaining 40% of the variance and an overall ability to discriminate PWS from controls. IRT-based analysis showed that the tool best functions among moderately impaired participants. ANT and DSST showed DIF regarding educational status. This suggested a different cut-off for those groups; however, we recommend future studies on this matter.
The findings of this study showed that ECAS psychometric properties align with previous validation studies of Western-developed batteries, including BACS.14,15,17,18,22,23,25
A significant proportion of participants and administrators rated the battery as engaging, practical, and tolerable. Administrators rated TMT Part B and the DSST as the most challenging tests to administer. This aligns with a previous study that reported TMT Part B as the most complex test.66 We recommend that when training administrators, special attention should be given to these tests.
IRR of the ECAS is high for most of the tests, and it would be important to provide adequate theoretical and practical training to achieve even higher IRR. With adequate training, we ascertain that multiple assessors can quickly administer and score the battery without significant variation.
ECAS has high test-retest reliability, making it suitable for studies with repeated assessment.14,67,68 We noticed some practice effects in WLLT, DSST, TMT Part B, and the composite score. Similarly, previous studies reported practice effects for similar tests.14–16,67 Still, the effect sizes are small and it is possible to account for this effect in the analyses, with the current study indicating the likely size. However, a relatively larger practice effect was seen for WLLT, which needs special attention. We recommend future updates of this battery to have alternate version for this test.
We found that a one-factor structure better represents the data. Previous studies also demonstrated similar findings for other cognitive batteries in PWS.18–20,22,25,69 In line with this, ECAS was also found to have excellent internal consistency, further demonstrating the interrelatedness of the tests. This finding suggested that a dominant factor explains the different domains of cognitive impairment in PWS, and it is possible to sum the scores of each domain to create a composite score.
An interview-based cognitive assessment tool, CAI-A, was used to examine the concurrent validity of ECAS and found a weak correlation between the two. A previous study comparing MCCB and CAI also reported a weak correlation.67 Another study reported a similar finding using the schizophrenia cognition rating scale (an interview-based measure) and BACS.70 These findings suggest differences in performance-based and interview-based measures. This aligns with previous research and underscores the different contributions these assessment tools may bring to evaluating cognition in PWS.71
This study found that ECAS can differentiate PWS from controls (with similar mean regarding age, sex, and educational status) regarding cognitive impairment, where it showed 77% sensitivity and 62% specificity with an AUC of .78 at a cut-off value of ≤0.12. PWS performed 0.76 SD lower than controls in the composite score. Previous studies also demonstrated that PWS performed 1 to 2.5 SD below the normative mean in the composite score of other batteries.14,67,72,73 Overall, previous studies reported higher effect size differences than ours. The reason might be related to the characteristics of the study participants. Some of the above studies reported a significant difference between PWS and controls concerning years of education, likely influencing the findings. In addition, in our study, two-thirds of the sample was in remission, while most of the above studies included chronic and/or inpatients with active symptoms.
The IRT analysis showed that the tool best functions among participants with moderate impairment with a difficulty coefficient between −0.60 and 0.66. Another study using the Cognitive Screening Scale for Schizophrenia (CSSS) demonstrated a higher difficulty coefficient (0.55–1.63).69 However, the two batteries measure different domains (i.e., CSSS gives more attention to domains not given much attention in the ECAS, such as attention, and reasoning and problem-solving).
Finally, ANT and DSST showed DIF regarding years of education. Previous studies also reported that the scores of some cognitive tests depend on the test taker’s level of education. For example, the MCCB composite score was influenced by the participant’s level of education.74–77 The same result was reported for BACS.78–80 This finding alerts us to consider the education status of participants while interpreting the findings.
We involved experts from different disciplines in the test selection and adaptation and followed strict measure selection and adaption guidelines. Unlike most previous studies on a similar issue, this study used a large sample size (208 PWS and 208 controls). Controls were selected to give similar means with cases regarding sex, age, and educational status. We evaluated over 10 psychometric properties. We used advanced statistical techniques involving both classical test theory and IRT-based analyses. However, some methodological limitations should be considered.
(1) The controls used in this study cannot be considered normative since they do not represent the demography of the general population in Ethiopia; however, we maintained the means of important factors affecting cognition to be similar for cases and controls. (2) For performance-based tests, practice effect is recommended to be determined over a short period (eg, 2 weeks); however, we used a relatively extended reassessment period, which may be more in line with clinical practice. (3) Since we conducted the study in an urban outpatient setting, participants in this study might not be representative of PWS in rural Ethiopia.
This study has implications for clinicians and researchers. Clinicians in Ethiopia can use ECAS to evaluate cognition as part of their initial assessment. This may be easily included in existing assessment procedures due to its quick and easy administration, suitable for busy clinics.
Researchers who would like to conduct a similar study in a comparable setting might take this study as a baseline to adapt/develop a contextually appropriate cognitive measure. Since ECAS is found to be stable, it may be helpful to evaluate changes in cognitive status over time. Other researchers can learn from our methodology in developing novel cognitive batteries in similar settings. This study further emphasizes the need for clinical measures to be culturally and context-sensitive and relay on data collected and evaluated entirely locally, in line with local clinical practice guidelines, and can be responsive to local insight.
Supplementary Material
Supplementary material is available at https://dbpia.nl.go.kr/schizophreniabulletin/.
Acknowledgments
We would like to thank the staff of the Department of Psychiatry, Addis Ababa University, who provided valuable comments at various steps of the study. We thank the data collectors and participants (people with schizophrenia and experts); this study would have been nothing without their unconditional support. We are grateful to Debre Berhan University for sponsoring the primary investigator to conduct this study. Last but not least, we would like to thank the African Mental Health Research Initiative (AMARI) because this work was supported by the DELTAS Africa Initiative [DEL-15-01] through the first author (YG). The DELTAS Africa Initiative is an independent funding scheme of the African Academy of Sciences (AAS) Alliance for Accelerating Excellence in Science in Africa (AESA) and supported by the New Partnership for Africa’s Development Planning and Coordinating Agency (NEPAD Agency) with funding from the Wellcome Trust [DEL-15-01] and the UK government. Our special thanks go to Prof. Richard Keefe and his team for allowing us to use the Digit Sequencing Task (DST). DST is part of the BACS, and the copyright belongs to WCG. We also thank Prof Martine Prince and his team for letting us use the Word Learning Test (WLLT) and the Animal Naming Test (ANT), part of a validation study of a cross-cultural measure of dementia in developing countries. Finally, we thank Prof. Robert Bilder, Prof. Joseph Ventura, and their team for allowing us to adapt CAI into Amharic. The CAI was developed by Robert Bilder and Joseph Ventura from UCLA and used for this study in line with the author’s recommendation.
Conflict of interest
All the authors declare that they have no conflict of interest.
Funding
This work was supported by the DELTAS Africa Initiative [DEL-15-01] through the first author (YG). The DELTAS Africa Initiative is an independent funding scheme of the African Academy of Sciences (AAS) Alliance for Accelerating Excellence in Science in Africa (AESA) and supported by the New Partnership for Africa’s Development Planning and Coordinating Agency (NEPAD Agency) with funding from the Wellcome Trust [DEL-15-01] and the UK government. The views expressed in this publication are those of the author(s) and not necessarily those of AAS, NEPAD Agency, Wellcome Trust, or the UK government. The funder has no role in the interpretation of findings and publication decisions.
Ethics Approval and Consent to Participate
After obtaining ethical approval from the Institutional Review Board of the College of Health Sciences, Addis Ababa University (Protocol No: 042/19/PSY), we secured permission from the hospitals where we conducted the study. We also took written consent from each participant.
Author Contributions
YG, AA, KH, and MC conceived and designed the study. YG coordinated the collection and analyzed the data. YG drafted the manuscript. All the authors read the manuscript several times and have given their final approval for publication.
Availability of Data and Materials
All the data used is made available in the manuscript and supplementary materials. Additional information can be available from the corresponding author upon reasonable request.
