Evaluation of In-person versus Remote Cognitive Assessment in Cognitively Unimpaired Older Adults via Regression-Based Change Analysis Open Access

Abstract

Objective

Compare the changes in neuropsychological test scores between remote and in-person follow-up assessment over a 1-year period using standardized regression–based (SRB) change indices.

Method

Participants were from the Wake Forest Alzheimer’s Disease Research Center (ADRC; N = 230) [mean age: 68.6 (7.8) years; education: 16.3 (2.3) years; 71% female; 86% White] and cognitively normal (as defined by a CDR of 0) at baseline and follow-up [mean days: 420.03 (48.53)]. Follow-up testing with the Uniform Data Set v3 Cognitive Battery was completed in person (n = 121) or remotely (n = 109) via phone (n = 61) or video (n = 48). SRB change scores were calculated using published formulas. Chi-square analysis compared the frequency of scores falling outside of an SRB cut-point +/−1.645 for follow-up assessments and mean SRB change scores were compared.

Results

There were no significant differences in the frequency of SRB change scores for in-person versus remote follow-up assessments at the SRB cut-point. Similarly, one-way ANOVAs comparing mean SRB change scores revealed no significant differences between in-person, telephone, and video follow-up means for any of the tests.

Conclusions

Telephone and video cognitive assessments performed similarly to in-person assessment and offer a valuable tool for research and clinical applications.

INTRODUCTION

In-person administration of neuropsychological tests has long been the gold standard for the evaluation of cognitive functioning for both clinical and research purposes. Remote administration (i.e., via telephone or video) has become increasingly common in recent years, due in part to the COVID-19 pandemic (Sperling et al., 2024). Particularly for older adults, for whom COVID-19-related health concerns were often greatest, modification of neuropsychological tests for remote administration became an immediate and urgent need to allow for continuity of clinical care and research evaluations.

There is evidence that telephone (Rapp et al., 2012) and video (Alegret et al., 2021; Barton et al., 2011; Loh et al., 2007; Wadsworth et al., 2016) administered cognitive assessments are able to identify cognitive impairment and are well received by older adults. Further, a pilot study found that a remotely administered Uniform Data Set Version 3 (UDSv3) battery was well tolerated by older adults and resulted in similar adjudication outcomes compared to the in-person version (Sachs et al., 2024). A review of tele-neuropsychology literature published by Sperling and colleagues (2024) demonstrated that at the outset of the COVID-19 pandemic, there was a sharp increase in publications on tele-neuropsychology. Though this increased use of remote assessment largely arose out of an immediate need, remote assessments also have the potential to increase the accessibility of neuropsychological testing for individuals for whom in-person clinic is burdensome or unfeasible (e.g., older adults with mobility issues, individuals in rural areas, and individuals from underserved communities). However, there is a need for further evaluation of the validity of remotely administered neuropsychological tests (Sperling et al., 2024).

An important question is whether remote assessment can be used with in-person assessments to measure change in cognitive functions. Standardized regression–based (SRB) change indices quantify a difference between an observed follow-up score and an expected follow-up score that considers baseline scores, practice effects, test–retest reliability, and other desired variables (e.g., demographic characteristics, days between assessments). SRB estimates are thought to estimate change that may occur because of normal fluctuations and measurement error, with change outside of these estimates representing “true” change, though this change is agnostic to cause. These estimates can be reported as z-scores to allow for comparisons of “true” change across individuals and groups. To our knowledge, there are no existing studies that utilized SRB change indices to compare patterns of longitudinal change between in-person and remotely administered cognitive assessments. A benefit of using SRB change indices is that they can be used to develop more precise change estimates in test performance over time that account for baseline scores, practice effects, reliability, regression to the mean, and other desired variables (age, education, test–retest interval; McSweeny et al., 1993). These indices have already been developed for existing UDSv3 tests for individuals with normal cognition (as defined by a Clinical Dementia Rating [CDR] of 0 and amyloid negatively) at baseline and first follow-up (Kiselica et al., 2025) and therefore can be used to identify increases or decreases in test scores that fall outside of the predicted ranges.

We use the SRB indices to identify and compare the frequency of score changes that exceed SRB cut-points (above and below) in participants with stable normal cognition at baseline and 1-year follow-up tested via either remotely (via telephone or video) or in-person assessment with the UDSv3. We also compare mean SRB scores between in person, telephone, and video scores at follow-up. The aim of this analysis is to determine whether in-person, telephone, and video follow-up assessments yield similar SRB scores, which would be informative about whether remote follow-up testing, and what type of remote follow-up testing (in person or video), may be acceptable alternative to in-person follow-up.

METHODS

Participants

All participants (N = 230) were recruited from the Wake Forest Alzheimer’s Disease Research Center Clinical Core (ADRC). All participants completed informed consent for study procedures as part of their participation. Additionally, all study procedures for the clinical core study are approved by the university’s institutional review board. In order to be considered for the current study, participants had to have complete UDSv3 neuropsychological testing data from their initial study visit and their first follow-up visit. Follow-up visits had to occur at least 6 months after baseline visits and no more than 2 years afterward. Additionally, participants needed to have stable, normal cognition as defined by a CDR score of 0.0 at both baseline and first follow-up.

Cognitive Testing

Detailed descriptions of the cognitive tests within the UDSv3 neuropsychological battery are described in detail by Weintraub and coworkers (2018). Baseline cognitive testing was completed in-person for all participants. Follow-up cognitive testing was completed either in person (N = 121) or remotely (N = 109) via telephone or video. For this study, data from the following UDSv3 tests were used: semantic fluency (animals and vegetables), letter fluency (combined F and L), Craft Story (immediate and delayed verbatim recall), and number span forward and backward (number of correct trials). Other UDSv3 measures (e.g., Multilingual Naming Test; Montreal Cognitive Assessment) were not included because at the time these data were collected, they were not part of our site’s remotely administered cognitive battery.

SRB Change Index

An SRB change index score was computed for each participant for each of the UDSv3 tests listed earlier using SRB equations published and described in detail by Kiselica and coworkers (2025). Briefly, expected follow-up scores were calculated from each individual’s baseline score, years of education, age, race, and days between evaluations. These predicted scores were subtracted from the actual follow-up scores resulting in a difference score that is divided by the published standard error of the estimate to produce an SRB change index. SRB change index scores that fall outside of designated z-score cut points may be indicative of “true change” not attributable to baseline scores, practice effects, test–retest reliability, or demographic characteristics.

Statistical Analysis

Mean test scores at baseline and follow-up for participants who completed telephone, video, and in-person follow-up were compared to determine whether the participants who opted for different follow-up modalities were not different from one another. Demographic variables (age, education, race, gender) and number of days between assessments were also compared between groups. Chi-square analyses were used to compare the number of participants in each group whose scores fell outside of a Z-score range of +/−1.645 at the follow-up visit. This particular cutoff was selected because it is the cutoff most often used by neuropsychologists (Duff, 2012) resulting in only the top and bottom 5% of scores being identified as abnormal (McSweeny et al., 1993). Though +/−1.645 is the most common cutoff used by neuropsychologists in dementia research, it is somewhat arbitrary and scores falling just below the cutoff may also be indicative of change. Because independent-samples t-test analyses were conducted to evaluate mean differences in SRB scores across follow-up modalities, potentially meaningful group differences could still be detected even if individual test scores did not fall outside of the +/−1.645 cutoff.

RESULTS

Sample Characteristics

Data from 230 individuals (mean age: 68.64 years; education 16.25 years; 70.9% female; 86.1% White) at the Wake Forest ADRC were used for analysis (Table 1). Of the 230 individuals included, 121 completed their follow-up visits in-person, and 109 completed their follow-up visits remotely via telephone (n = 61) or video (n = 48). Demographic differences between the three follow-up type groups (in-person, telephone, and video) were analyzed via one-way ANOVAs (age and education) or chi-square analysis (gender, race). There were not significant differences for age, race, or gender, but there was a significant difference for education, F(2, 227) = 4.14, p = .02. Post hoc analyses revealed that this effect was due to differences between telephone and video participants with video follow-up participants being more highly educated (mean = 17.00 years) than telephone follow-up participants (mean = 15.72 years). One-way ANOVAs were also run to evaluate potential differences in days between baseline and follow-up assessment [F(2, 227) = 5.46, p = .05] and baseline Montreal Cognitive Assessment (MoCA) scores [F(2, 227) = 3.90, p = .02]. Post hoc analyses revealed that there were significantly greater days between assessments for the in-person follow-up group (429.82 days) compared to the telephone follow-up group (407.74 days), and that baseline MoCA scores were higher for video follow-up participants (26.94) when compared to telephone follow-up participants (25.57). Of note, SRBCI scores account for demographic variables, baseline scores, and days between assessments, so these group differences should not affect the primary analysis. SRB change scores were not significantly different between groups on any of the cognitive tests used in the primary analysis (all ps > .10). There were slight differences in the raw scores but not SRB change scores. These differences are summarized in Tables 2 and 3. However, because only SRBCI scores (which account for demographic variables), not raw scores, were used for the primary analyses these differences in age, years of education, and fluency scores did not affect primary group comparisons.

The chi-square analysis revealed that there were not significant differences in the frequency of scores exceeding a Z-score cutoff of +/−1.645 between in-person and remote follow-up visits for any of the UDSv3 tests (Table 4). The specific frequencies of score elevations versus drops can also be found in Table 4, though, due to the small cell size and insignificant findings for the total frequences, a separate analysis of elevations versus drops was not conducted.

One-way ANOVAs comparing mean SRB change scores for in-person, telephone, and video follow-up visits indicated that there were no significant differences for any of the cognitive tests (Table 5). For number span backwards, there was a trend toward significance [F(2,227), p = .08] that appears to be driven by slightly higher SRB change scores for telephone follow-up assessments, but the effect size is very small (partial η² = 0.02).

DISCUSSION

This study examined differences between remote versus in-person follow-up testing performance in cognitively normal older adults over a 1-year follow-up period using SRB change indices. We found no significant differences in the frequency of score elevations or drops across follow-up modalities for tests that could be administered by audio only, among cognitively normal older adults. Additionally, there were not significant differences in mean change scores between any of the modalities for any of the tests. Strengths of this work include the novel application of SRB change indices to the growing area of teleneuropsychology, use of a relatively large cohort that completed all baseline assessments in-person and follow-up assessments approximately 1 year later via different modalities, and the ability to compare not only remote versus in-person testing but also specific remote modalities (e.g., telephone and video). In an increasingly remote world, a thorough understanding of remotely administered neuropsychological assessment is more needed than ever. The ability to assess neuropsychological functioning without the individual needing to leave their home has clear implications for increasing research and clinical access for some of the hardest to reach populations. This is particularly important in this current era of an ever-growing aging population and high demand for dementia-related assessments due to novel therapeutics.

Our findings of generally comparable performance for remote versus in-person testing are consistent with prior work demonstrating that remotely administered cognitive tests produce comparable scores and adjudication outcomes compared to cognitive tests that are administered in person (Alegret et al., 2021; Barton et al., 2011; Loh et al., 2007; Sachs et al., 2024; Wadsworth et al., 2016). Comparing change scores with a SRB index is another, and to our knowledge, more precise metric to test the validity of remote cognitive assessment. The use of regression-based change data addresses some of the statistical limitations of prior work on remote assessment validity like more tightly controlling for practice effects, base rates of change, and demographic characteristics. Additional work utilizing a randomized, counterbalanced study design and larger samples is still needed, but, in the interim, these findings provide preliminary support for the use of remote assessment.

Limitations, Challenges, and Future Directions

One limitation of this work is that these analyses were constrained to cognitive tests for which regression-based change formulas had already been published. Though not part of the UDSv3, list learning tests (e.g., the Rey Auditory Verbal Learning Test; Consortium to Establish a Registry for Alzheimer’s Disease list learning test) are often administered by ADRCs to assess memory. The development of SRB change formulas for list learning tests would be helpful in future work exploring the validity of remotely administered list learning tests in older adults with concern for memory decline. In a similar vein, the restriction to UDSv3 tests in an ADRC population in this work limits the generalizability of these other testing batteries and other non-ADRC populations. Another restriction on the generalizability is the use of a clinical cohort that is predominately White and relatively highly educated. Remote assessments may have different implications in more racially and educationally diverse populations.

CONCLUSION

This work provides evidence that on most UDSv3 measures, remote and in-person testing over a 1-year follow-up period produce comparable scores. Ultimately, this suggests that remote research assessments may be a viable alternative to in-person testing. The ability to use remotely administered testing in place of in-person testing has the potential to ease participant/patient burden and increase access to clinical care and participation in research.

FUNDING

This work was supported by the National Institute of Health P30 AG049638 and R01AG075959-01.

CONFLICT OF INTEREST

None declared.

AUTHOR CONTRIBUTIONS

Lauren Latham (Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing—original draft), Suzanne Craft (Conceptualization, Data curation, Funding acquisition, Investigation, Resources, Supervision, Writing—review & editing), Stephen Rapp (Conceptualization, Methodology, Supervision, Writing—review & editing), James Bateman (Funding acquisition, Investigation, Writing—review & editing), Maryjo Cleveland (Funding acquisition, Investigation, Writing—review & editing), Samantha Rogers (Funding acquisition, Investigation, Writing—review & editing), Benjamin Williams (Funding acquisition, Investigation, Writing—review & editing), Mia Yang (Funding acquisition, Investigation, Writing—review & editing), and Bonnie Sachs (Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing—review & editing)

References