Search
Close
Search
Advanced Search
Search Menu
AI Discovery Assistant

Abstract

Objective

The objective of this study was to investigate the validity, reliability, and measurement error of the Fugl-Meyer Assessment (FMA) when it was remotely administered by videoconferencing (Tele-FMA) and to describe barriers to remote administration of the FMA.

Methods

Forty-five participants who had strokes and had a smartphone or laptop computer with a camera and internet access were included. An in-person assessment was compared with a remote assessment in 11 participants, and 34 participants completed only the remote assessment. Rater 1 (R1) remotely administered, recorded, and scored the items of the FMA, after which the recording was forwarded to be scored by Rater 2. At least 7 days later, R1 rated the videorecording of the remote assessment a second time for the evaluation of intrarater reliability. In-person assessment was completed by R1 at the participant’s home. Criterion validity was analyzed using the Bland–Altman limits of agreement, and convergent validity was analyzed using Spearman correlation coefficient. The intrarater and interrater reliability was analyzed using the intraclass correlation coefficient, and individual items were analyzed using the weighted kappa. The standard error of measurement and minimal detectable change were calculated to evaluate the measurement error.

Results

Bland–Altman plots showed adequate agreement of in-person FMA and tele-FMA. A moderate positive correlation was found between Tele-FMA lower extremity (LE) scores and step test results, and a strong positive correlation was found between Tele-FMA-upper extremity (UE) and Stroke Impact Scale hand function domain. Significant and excellent (0.96 ≤ ICC ≤ 0.99) interrater and intrarater reliabilities of the Tele-FMA, Tele-FMA-UE, and tele-FMA-LE were found. Regarding the individual items, most showed excellent reliability (weighted kappa > 0.70). The standard error of measurement for both reliabilities was small (≤3.1 points). The minimal detectable change with 95% CI for both the Tele-FMA and Tele-FMA-UE was 2.5 points, whereas it was 1.3 points for the Tele-FMA-LE.

Conclusion

Tele-FMA has excellent intrarater and interrater reliability and should be considered as a valid measurement.

Impact

The FMA is widely used in clinical practice. However, the measurement properties of the remote version applied by videoconferencing were unknown. This study’s results demonstrate the validity and reliability of the Tele-FMA for assessing poststroke motor impairment remotely via videoconferencing. The Tele-FMA may be used to implement telerehabilitation in clinical practice.

Motor Assessment, Reliability, Stroke, Telerehabilitation, Validity

Introduction

Stroke is the second-leading cause of death in the world according to the World Health Organization. Besides its high lethality rate, stroke produces a significant number of dependent survivors, 70% of whom experience 1 or more sequelae that often restrict daily, social, occupational, and leisure activities.1–3

The provision of poststroke telerehabilitation may be especially important for geographical regions with few socioeconomic resources or rural areas, especially where neuro-rehabilitation specialists are scarce. Moreover, the introduction of social restrictions to combat the COVID-19 pandemic has increased the importance of telerehabilitation.4–6 Tyagi et al,7 in their study that assessed the barriers to and facilitators of telerehabilitation as perceived by patients post stroke and therapists, found ease of accessibility was the main facilitator reported by patients and caregivers because there is no need to leave the home. Among the main barriers perceived by therapists were the difficulties in assessing the patient and obtaining sufficiently reliable measurements to plan an intervention.7

Studies have shown there is low-quality evidence of no significant differences between telerehabilitation and in-person rehabilitation for individuals post stroke regarding independence in activities of daily living, balance, and upper limb function. Also, there is moderate-quality evidence that there were no significant differences between telerehabilitation and usual care on self-reported health-related quality of life.6 Even though the quality of evidence is too low to draw definitive conclusions on the relative effectiveness of telerehabilitation, this field is still emerging, and reliable outcome measurements can be valuable to future research.8 Nevertheless, to prescribe the ideal physical therapy treatment, professionals need to know which movements are altered after a stroke and how this influences task performance. The Fugl-Meyer Assessment (FMA) is the instrument most widely used in clinical settings and research to assess motor impairment and recovery in individuals post stroke.9

The FMA scale is recognized as being highly reliable when applied in person,10,11 when an in-person application is recorded, and when that recording is later assessed.12,13 The FMA, besides being a reliable and valid measurement tool widely used by physical therapists, is supported by standard instructions for administration and scoring, important characteristics when selecting a tool for telerehabilitation.14 However, the validity of remotely administering the FMA and the reliability of the score obtained remain unknown. Therefore, the present study investigated the validity, reliability, and measurement error of the FMA scale (without the reflex items)15,16 when applied and remotely assessed. It is important to emphasize that in the studies that examined the results obtained from telerehabilitation, the patient assessment procedures were usually carried out in person and only the rehabilitation interventions were remotely delivered.4–8

Although the FMA has excellent reliability when administered in person,10,11 no studies, to our knowledge, have attempted to evaluate the validity and reliability of the FMA to assess motor impairment when remotely administered. Therefore, the aims of this study were to investigate the measurement properties, namely criterion and construct-convergent validity, intrarater and interrater reliability, and measurement error of the FMA when remotely administered and to describe barriers to remotely administering the FMA.

Methods

Study Design

This study qualifies as a methodological study of measurement properties following the Consensus-based Standards Measurements Instruments checklist recommendations, the subject being the validity and reliability of the motor domains in the FMA administered and assessed remotely. Research protocols were approved by the institutional research ethics board (4.339.940 and 5.377.738, respectively, for reliability and validity studies). All participants provided written or audio consent prior to participation.

Participants and Recruitment

The inclusion criteria were a diagnosis of stroke, no cognitive impairment (using ≥13 or 15 from 22 points as a cut-off on the Brazilian telephone Mini-Mental State Examination (Braztel-MMSE)17 according to educational level), older than 18 years, and access to a mobile device with internet access (smartphone or computer). The exclusion criteria were a history of hearing disorders that could compromise communication with the raters, a diagnosis of another neurological condition (eg, Parkinson disease); a non–stroke-related orthopedic impairment, and the unavailability of a family member or caregiver to follow the assessment.

People listed on a register of individuals with stroke who had previously participated in a university care project were telephoned to screen for all the eligibility criteria except for cognition and hearing. If the relevant eligibility criteria were met and the person was interested in participating, a video call was scheduled during which hearing capacity was assessed and the Braztel-MMSE administrated to screen for cognition. Accordingly, individuals who failed to meet the cognition and hearing criteria were excluded and referred to appropriate university care projects. The informed consent was explained during a phone call and/or audio/text message via WhatsApp. At this stage, the evaluator requested the participant’s permission to record the videoconferences. Once the informed consent and recording permission had been obtained, the data collection sessions were scheduled.

For the validity study, people currently participating in a Health Care Program for People with Stroke at the university were invited to participate in 2 sessions, 1 remote and the other in person. The same procedure was followed except that the order of sessions was randomly assigned prior to scheduling.

Measurements

The Braztel-MMSE was applied for inclusion criteria purposes. For those participating in only the remote session, first the clinical and demographic data were collected using an information form, then the FMA was administered. For those participating in both the remote and in-person sessions, the FMA was administered first, followed by the Stroke Impact Scale (SIS) hand function domain and then the Step Test (ST). The clinical and demographic information form was applied during the first assessment session (remote or in-person) according to the random assignment. The in-person sessions took place in the participants’ homes.

Braztel-MMSE

The telephone-administered Mini-Mental State Examination (MMSE),18 consisting of 22 items compared with the 30-item MMSE original version, has been translated and culturally adapted for the Brazilian population in the Braztel-MMSE.17 The Braztel-MMSE has been used for the purpose of cognitive screening in individuals post stroke.19

Participant’s Clinical and Demographic Characterization Measurements

The researcher interviewed each participant to complete a form with their clinical (eg, stroke history, date, and type) and personal (eg, name, age, address) information.

Remotely Administered Tele-FMA

For this study, the FMA manual version translated to Brazilian Portuguese language was used.10 Items are scored on a 3-point ordinal assessment, where 0 = cannot be performed and 2 = performs completely with a total 100 points for the motor domain. This version presents adequate reliability and was adapted for remote use by videoconferencing as follows:

  • 1) Reflex items were not assessed, as proposed by 2 studies by Woodbury et al (2008)15 and Woodbury et al (2013)16 in which a Rasch analysis showed reflex testing did not contribute to the FMA-upper extremity (UE) scores. This version with 30-FMA-UE voluntary movement items has a maximum score of 60 points instead of 66 points. Also, in the present study, this adapted version of the FMA-lower extremity (LE) does not assess the reflex items, thus the maximum score is 28 points, instead of 34 points and the total FMA score is 88 points.

  • 2) For the items that needed resistance (wrist stable with elbow at 90° and extended and grasp items), the caregiver was guided on how to provide the resistance (Fig. 1).

  • 3) For the item measuring extensor synergy on the FMA-LE, no manual resistance was provided; the item was scored as 0 or 2 only, where 0 = did not perform the movement and 2 = perform the movement.

Items modified for remote application of the Fugl-Meyer Assessment via videoconferencing (Tele-FMA). A–C, change in the form of application; D–F, change in camera positioning and image angle.
Figure 1

Items modified for remote application of the Fugl-Meyer Assessment via videoconferencing (Tele-FMA). A–C, change in the form of application; D–F, change in camera positioning and image angle.

Open in new tabDownload slide

For this remotely administered FMA, the manual cited above was adapted (see Suppl. Appendix 1). Frontal or lateral views were as follows:

  • 4) The following items of the FMA-UE were evaluated in the sagittal plane: shoulder retraction and shoulder external rotation in the flexor synergy field, hand to lumbar spine, wrist stable, and wrist flexion/extension for both elbow positions.

  • 5) For the FMA-LE, items were evaluated in the lateral plane.

Barriers of the Remotely Administered FMA From the Perspective of the Patients and Caregivers

Immediately following the administration of the Tele-FMA, a questionnaire in the form of a semi-structured interview was applied via telephone to assess the barriers the participants and caregivers perceived during the application. The semi-structured interview has 5 objective items and 2 contrasting open questions, giving the patient the opportunity to express their personal experiences (Suppl. Appendix 2).

In-Person Assessment

In-person assessments were performed at the participant’s home. As for the Tele-FMA, reflex items were not assessed, and the total score was 88 points.

Assessments Applied to Analyze Convergent Validity

The SIS is based on a self-report questionnaire composed of 59 specific questions to evaluate 8 domains relevant to this population.20 In this research, only hand function domain (5 items) was assessed. Each item was scored between 1 and 5 points, with the rating categories ranging from “cannot do it at all” to “not difficult at all.” Final SIS hand function scores are generated with an algorithm with a range of 0 to 100. SIS hand function domain was chosen because it presents adequate measurement properties, and their convergent validity with FMA-UE applied in person has been evaluated21 and validated for telephone application.22 The ST measures the number of times a participant is able to place 1 foot on a step and back to the ground, as fast as possible, within 15 seconds. The step test was chosen because it is a valid and reliable test in the stroke population23 and has been validated for remote application.24

Data Collection

Before the remote assessment by videoconferencing, a document was sent to the participant and caregiver with an explanation of how to arrange the assessment environment, position the camera, and prepare the necessary materials (Suppl. Appendix 3). For both the assessments made in a sitting or lying position, the device’s (smartphone or computer) camera had to permit a full head-to-foot view of the participant. The required materials were a pen or pencil, a sheet of paper folded into 4, a diary (or similar) of the correct thickness, a 500-mL bottle, either a tennis ball or an orange or apple for the spherical grip item, and an armless chair without wheels. Finally, the document reemphasized the need for a caregiver to be present during the assessment.

For those participating in only the remote session, the assessor first collected the clinical and demographic data using an information form and then administered the FMA.

Both raters were trained by a researcher with more than 20 years of experience in administering the FMA in person. Before data collection, specifically for training purposes, the raters received a video recording of a patient completing the FMA, which they scored independently. The scores they gave were compared, and the differences were discussed with the more experienced researcher.

The Tele-FMA was remotely administered by Rater 1 (R1) via WhatsApp videoconferencing, with each participant and recorded using Open Broadcaster Software. The patient was instructed to position themselves in the environment previously organized for the assessment, while the rater was also similarly positioned. In the UE assessment, shoulder retraction and external rotation were assessed in the flexor synergy field in both the frontal and lateral planes. After conducting all items in the frontal plane, the camera position was changed to assess the lateral view items. Also, to better visualize the prehensions, the caregiver placed the camera close to the participant’s hand. Furthermore, in the LE assessment, to enable patients to clearly see the rater demonstrating how to correctly perform distal movement, when the participant laid face up, the caregiver moved the camera closer to their field of vision and then returned the camera to its original position. For all items, the rater gave instructions and demonstrated the movement, then the participant performed the item. Each item was performed first using the nonparetic limb and then the paretic limb. All the assessments were recorded with the participant’s consent.

After time 1 (T1), when R1 administered, recorded and scored the Tele-FMA, the video was sent to Rater 2 (R2) to give an independent score based on observation of the videorecording. At least 7 days after applying and scoring the Tele-FMA, R1 scored the assessment again based on observation of the video recording (T2).

For participants completing both the remote and in-person sessions, the order of sessions (remote or in-person) was randomly selected for each participant. The in-person assessment was also conducted by R1 at the participant’s home. The clinical and demographic information were collected during the first session, whether remote or in-person, according to the randomization.

The FMA was administered first, followed by the SIS hand function domain, and then the ST in both the remote and in-person sessions. For FMA-UE wrist extension and prehension items in the remote assessment, the caregiver applied the resistance, whereas at the in-person assessment, the resistance was applied by R1. For the ST, the caregiver was asked if they had a step at home, and if so, to then show it to the evaluator to validate the height for use in the test in question. In the cases where there was no step, a shoe box was used with an object inside to stabilize it. In the remote evaluation, the camera was positioned in front of the individual, while the caregiver positioned themself behind the participant to monitor for safety in the test.

Data Analysis

The sample characterization data were analyzed using descriptive statistics through mean and SD for continuous data and number and percentage (%) for categorical data. This study tested the agreement between in-person and remotely administered by videoconferencing assessments (criterion validity),25 construct–convergent validity, intrarater and interrater reliability, and measurement error domains for remote assessments.

For criterion validity analysis, the instruments applied in-person (FMA, SIS hand function, and ST) were considered the “gold-standard,” and the instrument remotely applied was the comparator. The agreement between in-person and remote assessments was analyzed using the Bland–Altman plot mean difference and limits of agreement. The y-axis is the difference in scores (Tele-FMA – in-person FMA) for each participant. Values at zero indicate no difference between assessments. Values on the x-axis represent the mean of the 2 measures for each participant.25 Construct–convergent validity was analyzed using Spearman correlation coefficient (ρ) between the Tele-FMA-UE with the SIS hand function domain and Tele-FMA-LE with the ST. Correlation magnitudes were classified as moderate to good (ρ = 0.50–0.75) or strong (ρ ≥ 0.75)25 and a hypothesis of moderate to strong positive correlations between those assessments. Additionally, we analyzed the reliability of in-person and Tele-FMA UE wrist extension and prehension items with linear-weighted κ (κw).

The intrarater and interrater reliability of the upper limb scores (FMA-UE), the lower limb score (FMA-LE), and the total Tele-FMA score (FMA-UE + LE) of the FMA was analyzed using the intraclass correlation coefficient (ICC) with 95% CI. In the test, a mixed model and a random model were used for the intrarater ICC and interrater ICC, respectively, with the single measurement score for each one. A value of ICC > 0.90 was classified as excellent reliability.25

The intrarater and interrater reliability of the scoring of the individual items on the FMA were analyzed using a κw. A value ≥0.70 is recommended, as suggested by Terwee et al.26 Interrater reliability was analyzed using T2 of R1 with R2, so that both scores were based on the observation of the video recordings. Bland–Altman plotting was used to assess the magnitude and direction of bias of the agreement between the sets of scores. On the y-axis are plotted values for the difference in scores between 2 times (T1-T2) or between 2 raters (R1-R2) for each participant. The x-axis values represent the mean of the 2 measures (T1 and T2 or R1 and R2) for each participant.25 This allowed us to assess the dispersion between the individual means of the 2 sets of scores and the individual differences between the sets of scores from the 2 raters and/or between the 2 sets of scores from R1 performed at 2 different times.

To test the intrarater and interrater measurement error, standard error of measurement (SEM) was used, which determines the divergence between the results of repeated measures. The SEM was determined using the following formula: SEM = |$\frac{\sqrt{\left({S}_1^2+{S}_2^2\right)}}{2}$| x |$\sqrt{1- CCI}$|⁠, where S corresponds to the SD of the measure and the ICC. The minimal detectable change (MDC) with 95% CI (MDC95) was determined by the following formula: MDC = z* SEM* |$\sqrt{2}$|⁠, where z = 1.96. This is used to set the amount of change in a variable so we can be sure that the error is not responsible for all the measured difference and that some true change has occurred. The SEM used to compute the MDC was the SEM obtained on intrarater reliability. The SEM was also expressed as a percentage of the total possible range of the instrument and stated that percentages lower than 10 are adequate. Data were analyzed with MedCalc Software.

The evaluation of barriers was analyzed based on the percentage of answers, which were “yes” or “no” for the objective questions and percentages of the most frequent answers for the subjective questions.

Role of the Funding Source

The funders played no role in the design, conduct, or reporting of this study.

Results

Participants

A total of 45 stroke survivors participated in this study. The 34 individuals who participated in only the remote assessment session were recruited from a register of people with stroke who had previously participated in a university care project. Of the 196 people on this list, 127 were immediately excluded due to the presence of another neurological condition (5), wrong telephone number (44), not answering the telephone (56), or were deceased (22). Sixty-nine were interviewed over the telephone. For those who accepted (n = 47), a video call was scheduled during which the researcher applied the Braztel-MMSE. Of those who participated in the video call, 13 were excluded due to cognitive impairment (n = 10) or hearing disorders (n = 3).

Additionally, individuals currently participating in a Health Care Program for People with Stroke at the University (n = 16) were telephoned and invited to participate in the validity study. Five did not answer the telephone, and 11 were assessed who were found eligible for inclusion and agreed to participate in both the remote and in-person sessions.

Considering the participants who completed both assessment sessions (in-person and remote) and those who completed only the remote assessment session, respectively, the mean ages were 66 and 61 years, predominantly men (91% and 73%), and had an ischemic stroke (64% and 73%). In both samples, the majority (64%) had hemiparesis on the left side (Tab. 1).

Table 1
Open in new tab

Demographic and Clinical Characteristics of the Study Samplesa

IndividualsIn-Person and Tele-FMA (n = 11)Tele-FMA (n = 34)
Age, mean (SD), y66 (12)61 (14)
Sex
 Male, no.10 (91%)25 (73%)
 Female, no.1 (9%)9 (26%)
Left hemiparesis, no.7 (64%)21 (64%)
Right dominance, no.11 (100%)28 (85%)
Phase after stroke (no. subacute/no. chronic)2/917/17
Diagnosis of ischemic, no.7 (64%)25 (73%)
Motor impairment (Tele-FMA 0–88 points), mean (SD)57 (27)50 (31)
Education, no.
 Incomplete elementary school6 (55%)15 (44%)
 Elementary school3 (27%)3 (9%)
 High school2 (18%)10 (29%)
 Graduate incomplete02 (6%)
 Graduate complete02 (6%)
Braztel-MEEM (0–22)18 (2)17 (3)
IndividualsIn-Person and Tele-FMA (n = 11)Tele-FMA (n = 34)
Age, mean (SD), y66 (12)61 (14)
Sex
 Male, no.10 (91%)25 (73%)
 Female, no.1 (9%)9 (26%)
Left hemiparesis, no.7 (64%)21 (64%)
Right dominance, no.11 (100%)28 (85%)
Phase after stroke (no. subacute/no. chronic)2/917/17
Diagnosis of ischemic, no.7 (64%)25 (73%)
Motor impairment (Tele-FMA 0–88 points), mean (SD)57 (27)50 (31)
Education, no.
 Incomplete elementary school6 (55%)15 (44%)
 Elementary school3 (27%)3 (9%)
 High school2 (18%)10 (29%)
 Graduate incomplete02 (6%)
 Graduate complete02 (6%)
Braztel-MEEM (0–22)18 (2)17 (3)
a

Braztel-MEEM = Brazilian telephone Mini-Mental State Examination; Tele-FMA = Fugl-Meyer Assessment applied by videoconferencing.

Table 1
Open in new tab

Demographic and Clinical Characteristics of the Study Samplesa

IndividualsIn-Person and Tele-FMA (n = 11)Tele-FMA (n = 34)
Age, mean (SD), y66 (12)61 (14)
Sex
 Male, no.10 (91%)25 (73%)
 Female, no.1 (9%)9 (26%)
Left hemiparesis, no.7 (64%)21 (64%)
Right dominance, no.11 (100%)28 (85%)
Phase after stroke (no. subacute/no. chronic)2/917/17
Diagnosis of ischemic, no.7 (64%)25 (73%)
Motor impairment (Tele-FMA 0–88 points), mean (SD)57 (27)50 (31)
Education, no.
 Incomplete elementary school6 (55%)15 (44%)
 Elementary school3 (27%)3 (9%)
 High school2 (18%)10 (29%)
 Graduate incomplete02 (6%)
 Graduate complete02 (6%)
Braztel-MEEM (0–22)18 (2)17 (3)
IndividualsIn-Person and Tele-FMA (n = 11)Tele-FMA (n = 34)
Age, mean (SD), y66 (12)61 (14)
Sex
 Male, no.10 (91%)25 (73%)
 Female, no.1 (9%)9 (26%)
Left hemiparesis, no.7 (64%)21 (64%)
Right dominance, no.11 (100%)28 (85%)
Phase after stroke (no. subacute/no. chronic)2/917/17
Diagnosis of ischemic, no.7 (64%)25 (73%)
Motor impairment (Tele-FMA 0–88 points), mean (SD)57 (27)50 (31)
Education, no.
 Incomplete elementary school6 (55%)15 (44%)
 Elementary school3 (27%)3 (9%)
 High school2 (18%)10 (29%)
 Graduate incomplete02 (6%)
 Graduate complete02 (6%)
Braztel-MEEM (0–22)18 (2)17 (3)
a

Braztel-MEEM = Brazilian telephone Mini-Mental State Examination; Tele-FMA = Fugl-Meyer Assessment applied by videoconferencing.

Criterion and Construct–Convergent Validity of Tele-FMA

The data obtained with the 2 assessment methods (in-person and remote) for FMA-UE, FMA-LE, SIS hand function, and ST are displayed in Table 2 along with the mean differences and limits of agreement obtained with Bland–Altman analysis. The limits of agreement for the total scores of in-person FMA and Tele-FMA were 3.1 and 3.8 points, respectively (Fig. 2A).

Table 2
Open in new tab

Reliability and Agreement Between In-Person and Remote Scores for FMA, SIS Hand Function, and STa

In-Person Mean (SD)Tele-FMA Mean (SD)In-Person vs Tele-FMA
FMA-UE items requiring resistancebκw (95% CI)
Wrist stable; elbow at 90° (0–2)1.2 (1.0)1.2 (1.0)1.00 (1.00–1.00)
Wrist stable; elbow extended (0–2)0.9 (1.0)1.0 (1.0)0.91 (0.74–1.00)
Hook grasp (0–2)1.3 (1.0)1.3 (1.0)1.00 (1.00–1.00)
Lateral prehension (0–2)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)
Opposition thumb and index (0–2)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)
Cylindrical grasp (0–2)1.4 (0.9)1.4 (0.9)1.00 (1.00–1.00)
Spherical grasp (0–2)0.9 (1.0)0.9 (1.0)1.00 (1.00–1.00)
Reliability ICC (3,1) (CI)Bland–Altman Plot MD (LoA); MAD
FMA-UE (0–60 points)37(21)36 (22)0.99 (0.97 to 1.00)−0.7 (−3.9 to 2.4); 1.3
FMA-LE (0–28 points)21 (8)20 (7)0.99 (0.86 to 1.00)−0.4 (−2.6 to 2.8); 0.7
SIS hand function (0–100)48 (34)47 (29)0.93 (0.77 to 0.98)−0.8 (−24 to 23); 7
ST paretic + nonparetic LE (no. of steps)12 (9)12 (9)0.99 (0.86 to 1.00)−0.5 (−8 to 7); 2
In-Person Mean (SD)Tele-FMA Mean (SD)In-Person vs Tele-FMA
FMA-UE items requiring resistancebκw (95% CI)
Wrist stable; elbow at 90° (0–2)1.2 (1.0)1.2 (1.0)1.00 (1.00–1.00)
Wrist stable; elbow extended (0–2)0.9 (1.0)1.0 (1.0)0.91 (0.74–1.00)
Hook grasp (0–2)1.3 (1.0)1.3 (1.0)1.00 (1.00–1.00)
Lateral prehension (0–2)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)
Opposition thumb and index (0–2)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)
Cylindrical grasp (0–2)1.4 (0.9)1.4 (0.9)1.00 (1.00–1.00)
Spherical grasp (0–2)0.9 (1.0)0.9 (1.0)1.00 (1.00–1.00)
Reliability ICC (3,1) (CI)Bland–Altman Plot MD (LoA); MAD
FMA-UE (0–60 points)37(21)36 (22)0.99 (0.97 to 1.00)−0.7 (−3.9 to 2.4); 1.3
FMA-LE (0–28 points)21 (8)20 (7)0.99 (0.86 to 1.00)−0.4 (−2.6 to 2.8); 0.7
SIS hand function (0–100)48 (34)47 (29)0.93 (0.77 to 0.98)−0.8 (−24 to 23); 7
ST paretic + nonparetic LE (no. of steps)12 (9)12 (9)0.99 (0.86 to 1.00)−0.5 (−8 to 7); 2
a

FMA = Fugl-Meyer Assessment; ICC = intraclass correlation coefficient; κw = κ weighted; LE = lower extremity; LoA = limits of agreement; MAD = mean absolute difference; MD = mean difference; Tele = remote; UE = upper extremity; SIS = Stroke Impact Scale; ST = Step Test.

b

Items from FMA-UE requiring resistance of caregiver or examiner.

Table 2
Open in new tab

Reliability and Agreement Between In-Person and Remote Scores for FMA, SIS Hand Function, and STa

In-Person Mean (SD)Tele-FMA Mean (SD)In-Person vs Tele-FMA
FMA-UE items requiring resistancebκw (95% CI)
Wrist stable; elbow at 90° (0–2)1.2 (1.0)1.2 (1.0)1.00 (1.00–1.00)
Wrist stable; elbow extended (0–2)0.9 (1.0)1.0 (1.0)0.91 (0.74–1.00)
Hook grasp (0–2)1.3 (1.0)1.3 (1.0)1.00 (1.00–1.00)
Lateral prehension (0–2)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)
Opposition thumb and index (0–2)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)
Cylindrical grasp (0–2)1.4 (0.9)1.4 (0.9)1.00 (1.00–1.00)
Spherical grasp (0–2)0.9 (1.0)0.9 (1.0)1.00 (1.00–1.00)
Reliability ICC (3,1) (CI)Bland–Altman Plot MD (LoA); MAD
FMA-UE (0–60 points)37(21)36 (22)0.99 (0.97 to 1.00)−0.7 (−3.9 to 2.4); 1.3
FMA-LE (0–28 points)21 (8)20 (7)0.99 (0.86 to 1.00)−0.4 (−2.6 to 2.8); 0.7
SIS hand function (0–100)48 (34)47 (29)0.93 (0.77 to 0.98)−0.8 (−24 to 23); 7
ST paretic + nonparetic LE (no. of steps)12 (9)12 (9)0.99 (0.86 to 1.00)−0.5 (−8 to 7); 2
In-Person Mean (SD)Tele-FMA Mean (SD)In-Person vs Tele-FMA
FMA-UE items requiring resistancebκw (95% CI)
Wrist stable; elbow at 90° (0–2)1.2 (1.0)1.2 (1.0)1.00 (1.00–1.00)
Wrist stable; elbow extended (0–2)0.9 (1.0)1.0 (1.0)0.91 (0.74–1.00)
Hook grasp (0–2)1.3 (1.0)1.3 (1.0)1.00 (1.00–1.00)
Lateral prehension (0–2)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)
Opposition thumb and index (0–2)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)
Cylindrical grasp (0–2)1.4 (0.9)1.4 (0.9)1.00 (1.00–1.00)
Spherical grasp (0–2)0.9 (1.0)0.9 (1.0)1.00 (1.00–1.00)
Reliability ICC (3,1) (CI)Bland–Altman Plot MD (LoA); MAD
FMA-UE (0–60 points)37(21)36 (22)0.99 (0.97 to 1.00)−0.7 (−3.9 to 2.4); 1.3
FMA-LE (0–28 points)21 (8)20 (7)0.99 (0.86 to 1.00)−0.4 (−2.6 to 2.8); 0.7
SIS hand function (0–100)48 (34)47 (29)0.93 (0.77 to 0.98)−0.8 (−24 to 23); 7
ST paretic + nonparetic LE (no. of steps)12 (9)12 (9)0.99 (0.86 to 1.00)−0.5 (−8 to 7); 2
a

FMA = Fugl-Meyer Assessment; ICC = intraclass correlation coefficient; κw = κ weighted; LE = lower extremity; LoA = limits of agreement; MAD = mean absolute difference; MD = mean difference; Tele = remote; UE = upper extremity; SIS = Stroke Impact Scale; ST = Step Test.

b

Items from FMA-UE requiring resistance of caregiver or examiner.

(A) Bland–Altman plot, limits of agreement for in-person Fugl-Meyer Assessment (FMA), and remote Fugl-Meyer Assessment (Tele-FMA). (B) Scatter plot demonstrating concurrent validity between in-person FMA (as gold standard) and Tele-FMA total scores. (C) Scatter plot demonstrating convergent validity of Tele-FMA-UE (upper extremity) scores with Stroke Impact Scale (SIS) hand function domain. (D) Scatter plot demonstrating convergent validity of Tele-FMA-LE (lower extremity) scores with Step Test (ST) sum of steps by paretic and nonparetic lower extremities. Correlation coefficient values and (95% CI) for B, C and D are given on the text.
Figure 2

(A) Bland–Altman plot, limits of agreement for in-person Fugl-Meyer Assessment (FMA), and remote Fugl-Meyer Assessment (Tele-FMA). (B) Scatter plot demonstrating concurrent validity between in-person FMA (as gold standard) and Tele-FMA total scores. (C) Scatter plot demonstrating convergent validity of Tele-FMA-UE (upper extremity) scores with Stroke Impact Scale (SIS) hand function domain. (D) Scatter plot demonstrating convergent validity of Tele-FMA-LE (lower extremity) scores with Step Test (ST) sum of steps by paretic and nonparetic lower extremities. Correlation coefficient values and (95% CI) for B, C and D are given on the text.

Open in new tabDownload slide

Excellent reliability (ICCs ≥ 0.93) was found between in-person and remotely administered FMA-UE, FMA-LE, SIS hand function, and ST, as well as for wrist extension and prehension items of the FMA-UE (κ ≥ 0.91; Tab. 2). Correlations between in-person and remote assessments were high for the FMA-UE (ρ = 0.99, 95% CI = 0.96 to 0.99; P < .0001), FMA-LE (ρ = 0.94, 95% CI 0.78 to 0.98; P < .0001), SIS hand function (ρ = 0.93, 95% CI = 0.73 to 0.98; P < .0001), and the ST (ρ = 0.97, 95% CI = 0.88 to 0.99; P < .0001). A positive and high correlation coefficient was found between the total scores of the in-person FMA and the total scores for the Tele-FMA (ρ = 0.97, 95% CI = 0.88 to 0.99; P < .0001) (Fig. 2B).

As for convergent validity, the total score for the Tele-FMA UE showed a high positive correlation (ρ = 0.82, 95% CI = 0.43 to 0.95; P = .002) with the SIS hand function score (Fig. 2C) and a moderate positive correlation with the ST values for the sum of steps for paretic and nonparetic LE (ρ = 0.77, 95% CI = 0.33 to 0.94; P = .005) (Fig. 2D).

Reliability

The intrarater reliability for the total Tele-FMA was excellent and similar for the UE (ICC = 0.99; 95% CI = 0.99 to 1.00) and LE (ICC = 0.99; 95% CI = 0.99 to 1.00). The scores for the individual items also presented excellent intrarater reliability, with all κw values ≥0.80.

The interrater reliability of the Tele-FMA-UE score was excellent (ICC = 0.99; 95% CI = 0.98 to 1.00; P < .001). All the individual items presented excellent intrarater reliability (κw ≥ 0.70), except for 2 items (κw < 0.70): item 1.1, flexor synergy, shoulder retraction (κw = 0.63; 95% CI = 0.42 to 0.83) and item 1.4, flexor synergy, external shoulder rotation (κw = 0.66; 95% CI = 0.47 to 0.85).

The interrater reliability of the Tele-FMA-LE score was also excellent (ICC = 0.96; 95% CI = 0.93 to 0.98). The interrater reliability for the related individual items was excellent for all items (κw ≥ 0.70), with the exception of 4 (κw < 0.70); item 1.2, flexor synergy, flexion (knee) (κw = 0.69; 95% CI = 0.47 to 0.92); item 1.3, flexor synergy, dorsi-flexion (κw = 0.67; 95% CI = 0.48 to 0.87); item 7, heel-shin tremor item (κw = 0.52; 95% CI = 0.24 to 0.81); and item 8, heel-shin dysmetria (κw = 0.59; 95% CI = 0.39 to 0.80) (Tab. 3).

Table 3
Open in new tab

Mean, SD, Intrarater and Interrater Reliability, and Measurement Error for Tele-FMAa

Rater A Time 1  
Mean (SD)		Rater A Time 2  
Mean (SD)		Rater B  
Mean (SD)		Intrarater Reliability  
(CI)		Interrater Reliability  
(CI)
Individual items UE(0–2)(0–2)(0–2)κ wκ w
1.1. Flexor synergy – shoulder retraction1.2 (0.8)1.3 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.63 (0.42–0.83)
1.2. Flexor synergy – shoulder elevation1.4 (0.8)1.5 (0.7)1.3 (0.8)0.80 (0.64–0.95)0.77 (0.61–0.94)
1.3. Flexor synergy – shoulder abduction (90°)1.4 (0.8)1.4 (0.8)1.2 (0.8)1.00 (1.00–1.00)0.79 (0.62–0.95)
1.4. Flexor synergy – shoulder external rotation1.1 (0.9)1.0 (1.0)1.1 (0.9)0.84 (0.70–0.99)0.66 (0.47–0.85)
1.5. Flexor synergy – elbow flexion1.4 (0.8)1.4 (0.8)1.5 (0.8)0.89 (0.77–1.00)0.80 (0.64–0.97)
1.6. Flexor synergy – forearm supination0.8 (0.9)0.8 (0.9)0.9 (0.9)1.00 (1.00–1.00)0.81 (0.67–0.95)
2.1. Extensor synergy – shoulder adduction with internal rotation1.2 (0.9)1.3 (0.9)1.3 (0.9)0.90 (0.79–1.00)0.83 (0.69–0.97)
2.2. Extensor synergy – extension1.2 (0.9)1.2 (0.9)1.2 (0.9)1.00 (1.00–1.00)0.96 (0.90–1.00)
2.3. Extensor synergy – pronation1.1 (1.0)1.2 (1.0)1.1 (0.9)0.87 (0.76–0.99)0.87 (0.76–0.99)
3. Hand to lumbar spine1.3 (0.9)1.3 (0.9)1.4 (0.8)0.90 (0.79–1.00)0.86 (0.74–0.98)
4. Shoulder flexion – 0°- 90°1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
5. Elbow 90° – pronation/supination1.1 (0.9)1.1 (0.9)1.1 (0.9)0.96 (0.91–1.00)1.00 (1.00–1.00)
6. Shoulder abduction 0°- 90°1.0 (1.0)1.0 (1.0)1.1 (0.9)0.97 (0.91–1.00)0.85 (0.73–0.97)
7. Shoulder flexion 90°-180°0.7 (0.8)0.8 (0.9)0.8 (0.9)0.90 (0.79–1.00)0.87 (0.76–0.98)
8. Elbow 0° – pronation and supination0.9 (1.0)0.8 (0.9)1.0 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
9. Wrist stable, elbow at 90°0.9 (1.0)1.0 (1.0)1.0 (0.9)0.91 (0.81–1.00)0.76 (0.61–0.90)
10. Wrist flexion/extension. Elbow at 90°0.9 (0.9)0.9 (0.9)1.1 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
11. Wrist stable, elbow extended1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
12. Wrist flexion/extension, elbow extended0.9 (1.0)0.9 (1.0)0.9 (0.9)0.96 (0.91–1.00)0.90 (0.81–1.00)
13. Wrist circumduction0.9 (0.9)0.9 (0.9)0.8 (0.8)1.00 (1.00–1.00)0.83 (0.70–0.96)
14. Finger mass flexion1.2 (0.9)1.2 (0.9)1.3 (0.8)0.96 (0.90–1.00)0.83 (0.69–0.97)
15. Finger mass extension (relaxation of flexion)1.1 (0.9)1.1 (0.9)1.2 (0.8)1.00 (1.00–1.00)0.90 (0.80–1.00)
16. Hook grasp1.1 (1.0)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
17. Lateral prehension1.1 (1.0)1.1 (1.0)1.2 (1.0)1.00 (1.00–1.00)0.82 (0.63–1.00)
18. Opposition thumb and index0.9 (1.0)0.9 (1.0)0.9 (1.0)0.97 (0.91–1.00)0.88 (0.77–0.99)
19. Cylindrical grasp1.3 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.90–1.00)0.77 (0.59–0.94)
20. Spherical grasp0.9 (1.0)0.9 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.83 (0.69–1.00)
21. Movement without tremor0.8 (0.9)1.0 (0.9)1.0 (1.0)0.84 (0.69–0.99)0.73 (0.57–0.88)
22. Movement without dysmetria0.9 (0.9)0.9 (0.9)1.0 (1.0)0.87 (0.76–0.99)0.75 (0.61–0.90)
23. Movement with normal speed0.7 (0.9)0.7 (0.9)0.7 (0.9)1.00 (1.00–1.00)0.80 (0.66–0.94)
Total score Tele-FMA-UE(0–60 points)(0–60 points)(0–60 points)ICC(3,1) (CI)ICC (2,1) (CI)
31(24)32(24)32 (24)0.99 (0.99–1.00)0.99 (0.98–1.00)
 Tele-FMA-UE SEM points (SEM%)0.9 (1.5%)2.2 (3.7%)
 Tele-FMA-UE MDC95 points (MDC95%)2.5 (2.8%)
Individual Items LE(0–2)(0–2)(0–2)
1.1. Flexor synergy - flexion (hip)1.6 (0.7)1.6 (0.7)1.7 (0.6)1.00 (1.00–1.00)0.83 (0.66–1.00)
1.2. Flexor synergy - flexion (knee)1.5 (0.8)1.6 (0.8)1.7 (0.6)0.95 (0.87–1.00)0.69 (0.47–0.92)
1.3. Flexor synergy – dorsi-flexion1.1 (0.9)1.1 (0.9)1.3 (0.9)0.93 (0.85–1.00)0.67 (0.48–0.87)
2.1. Extension synergy – extension (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.90 (0.78–1.00)
2.2. Extension synergy – adduction (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.86 (0.71–1.00)
2.3. Extension synergy – extension (knee)1.6 (0.8)1.6 (0.8)1.5 (0.7)0.95 (0.86–1.00)0.70 (0.50–0.90)
2.4. Extension synergy – plantar flexion1.1 (0.8)1.1 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.73 (0.55–0.90)
3. Knee flexion (sitting)1.6 (0.7)1.6 (0.7)1.6 (0.7)1.00 (1.00–1.00)0.95 (0.86–1.00)
4. Ankle dorsiflexion (sitting)1.1 (0.8)1.1 (0.9)1.3 (0.9)0.93 (0.84–1.00)0.77 (0.61–0.93)
5. Knee flexion (standing)1.1 (0.8)1.1 (0.8)1.1 (0.8)0.92 (0.83–1.00)0.79 (0.63–0.95)
6. Ankle dorsiflexion (standing)1.0 (0.8)1.0 (0.9)1.0 (0.8)0.90 (0.79–1.00)0.83 (0.69–0.97)
7. Heel-shin tremor1.4 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.89–1.00)0.52 (0.24–0.81)
8. Heel-shin dysmetria1.4 (0.8)1.4 (0.8)1.1 (0.8)0.92 (0.82–1.00)0.59 (0.39–0.80)
9. Heel-shin speed1.0 (0.9)1.0 (1.0)0.9 (0.9)0.93 (0.85–1.00)0.72 (0.57–0.88)
Total score Tele-FMA-LE(0–28 points)(0–28 points)(0–28 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19 (09)0.99 (0.99–1.00)0.98 (0.96–0.99)
 Tele-FMA-LE SEM points (SEM%)0.5 (1.8%)1.8 (6.4%)
 Tele-FMA-LE MDC95 points (MDC95%)1.3 (4.6%)
Total score Tele-FMA(0–88 points)(0–88 points)(0–88 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19(09)0.99 (0.99–1.00)0.98 (0.96–0.99)
50(31)51(31)52(30)0.99 (0.99–1.00)0.98 (0.98–0.99)
 Tele-FMA SEM points (SEM%)0.9 (1.0%)3.1 (3.5%)
 Tele-FMA MDC95 points (MDC95%)2.5 (2.8%)
Rater A Time 1  
Mean (SD)		Rater A Time 2  
Mean (SD)		Rater B  
Mean (SD)		Intrarater Reliability  
(CI)		Interrater Reliability  
(CI)
Individual items UE(0–2)(0–2)(0–2)κ wκ w
1.1. Flexor synergy – shoulder retraction1.2 (0.8)1.3 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.63 (0.42–0.83)
1.2. Flexor synergy – shoulder elevation1.4 (0.8)1.5 (0.7)1.3 (0.8)0.80 (0.64–0.95)0.77 (0.61–0.94)
1.3. Flexor synergy – shoulder abduction (90°)1.4 (0.8)1.4 (0.8)1.2 (0.8)1.00 (1.00–1.00)0.79 (0.62–0.95)
1.4. Flexor synergy – shoulder external rotation1.1 (0.9)1.0 (1.0)1.1 (0.9)0.84 (0.70–0.99)0.66 (0.47–0.85)
1.5. Flexor synergy – elbow flexion1.4 (0.8)1.4 (0.8)1.5 (0.8)0.89 (0.77–1.00)0.80 (0.64–0.97)
1.6. Flexor synergy – forearm supination0.8 (0.9)0.8 (0.9)0.9 (0.9)1.00 (1.00–1.00)0.81 (0.67–0.95)
2.1. Extensor synergy – shoulder adduction with internal rotation1.2 (0.9)1.3 (0.9)1.3 (0.9)0.90 (0.79–1.00)0.83 (0.69–0.97)
2.2. Extensor synergy – extension1.2 (0.9)1.2 (0.9)1.2 (0.9)1.00 (1.00–1.00)0.96 (0.90–1.00)
2.3. Extensor synergy – pronation1.1 (1.0)1.2 (1.0)1.1 (0.9)0.87 (0.76–0.99)0.87 (0.76–0.99)
3. Hand to lumbar spine1.3 (0.9)1.3 (0.9)1.4 (0.8)0.90 (0.79–1.00)0.86 (0.74–0.98)
4. Shoulder flexion – 0°- 90°1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
5. Elbow 90° – pronation/supination1.1 (0.9)1.1 (0.9)1.1 (0.9)0.96 (0.91–1.00)1.00 (1.00–1.00)
6. Shoulder abduction 0°- 90°1.0 (1.0)1.0 (1.0)1.1 (0.9)0.97 (0.91–1.00)0.85 (0.73–0.97)
7. Shoulder flexion 90°-180°0.7 (0.8)0.8 (0.9)0.8 (0.9)0.90 (0.79–1.00)0.87 (0.76–0.98)
8. Elbow 0° – pronation and supination0.9 (1.0)0.8 (0.9)1.0 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
9. Wrist stable, elbow at 90°0.9 (1.0)1.0 (1.0)1.0 (0.9)0.91 (0.81–1.00)0.76 (0.61–0.90)
10. Wrist flexion/extension. Elbow at 90°0.9 (0.9)0.9 (0.9)1.1 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
11. Wrist stable, elbow extended1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
12. Wrist flexion/extension, elbow extended0.9 (1.0)0.9 (1.0)0.9 (0.9)0.96 (0.91–1.00)0.90 (0.81–1.00)
13. Wrist circumduction0.9 (0.9)0.9 (0.9)0.8 (0.8)1.00 (1.00–1.00)0.83 (0.70–0.96)
14. Finger mass flexion1.2 (0.9)1.2 (0.9)1.3 (0.8)0.96 (0.90–1.00)0.83 (0.69–0.97)
15. Finger mass extension (relaxation of flexion)1.1 (0.9)1.1 (0.9)1.2 (0.8)1.00 (1.00–1.00)0.90 (0.80–1.00)
16. Hook grasp1.1 (1.0)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
17. Lateral prehension1.1 (1.0)1.1 (1.0)1.2 (1.0)1.00 (1.00–1.00)0.82 (0.63–1.00)
18. Opposition thumb and index0.9 (1.0)0.9 (1.0)0.9 (1.0)0.97 (0.91–1.00)0.88 (0.77–0.99)
19. Cylindrical grasp1.3 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.90–1.00)0.77 (0.59–0.94)
20. Spherical grasp0.9 (1.0)0.9 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.83 (0.69–1.00)
21. Movement without tremor0.8 (0.9)1.0 (0.9)1.0 (1.0)0.84 (0.69–0.99)0.73 (0.57–0.88)
22. Movement without dysmetria0.9 (0.9)0.9 (0.9)1.0 (1.0)0.87 (0.76–0.99)0.75 (0.61–0.90)
23. Movement with normal speed0.7 (0.9)0.7 (0.9)0.7 (0.9)1.00 (1.00–1.00)0.80 (0.66–0.94)
Total score Tele-FMA-UE(0–60 points)(0–60 points)(0–60 points)ICC(3,1) (CI)ICC (2,1) (CI)
31(24)32(24)32 (24)0.99 (0.99–1.00)0.99 (0.98–1.00)
 Tele-FMA-UE SEM points (SEM%)0.9 (1.5%)2.2 (3.7%)
 Tele-FMA-UE MDC95 points (MDC95%)2.5 (2.8%)
Individual Items LE(0–2)(0–2)(0–2)
1.1. Flexor synergy - flexion (hip)1.6 (0.7)1.6 (0.7)1.7 (0.6)1.00 (1.00–1.00)0.83 (0.66–1.00)
1.2. Flexor synergy - flexion (knee)1.5 (0.8)1.6 (0.8)1.7 (0.6)0.95 (0.87–1.00)0.69 (0.47–0.92)
1.3. Flexor synergy – dorsi-flexion1.1 (0.9)1.1 (0.9)1.3 (0.9)0.93 (0.85–1.00)0.67 (0.48–0.87)
2.1. Extension synergy – extension (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.90 (0.78–1.00)
2.2. Extension synergy – adduction (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.86 (0.71–1.00)
2.3. Extension synergy – extension (knee)1.6 (0.8)1.6 (0.8)1.5 (0.7)0.95 (0.86–1.00)0.70 (0.50–0.90)
2.4. Extension synergy – plantar flexion1.1 (0.8)1.1 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.73 (0.55–0.90)
3. Knee flexion (sitting)1.6 (0.7)1.6 (0.7)1.6 (0.7)1.00 (1.00–1.00)0.95 (0.86–1.00)
4. Ankle dorsiflexion (sitting)1.1 (0.8)1.1 (0.9)1.3 (0.9)0.93 (0.84–1.00)0.77 (0.61–0.93)
5. Knee flexion (standing)1.1 (0.8)1.1 (0.8)1.1 (0.8)0.92 (0.83–1.00)0.79 (0.63–0.95)
6. Ankle dorsiflexion (standing)1.0 (0.8)1.0 (0.9)1.0 (0.8)0.90 (0.79–1.00)0.83 (0.69–0.97)
7. Heel-shin tremor1.4 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.89–1.00)0.52 (0.24–0.81)
8. Heel-shin dysmetria1.4 (0.8)1.4 (0.8)1.1 (0.8)0.92 (0.82–1.00)0.59 (0.39–0.80)
9. Heel-shin speed1.0 (0.9)1.0 (1.0)0.9 (0.9)0.93 (0.85–1.00)0.72 (0.57–0.88)
Total score Tele-FMA-LE(0–28 points)(0–28 points)(0–28 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19 (09)0.99 (0.99–1.00)0.98 (0.96–0.99)
 Tele-FMA-LE SEM points (SEM%)0.5 (1.8%)1.8 (6.4%)
 Tele-FMA-LE MDC95 points (MDC95%)1.3 (4.6%)
Total score Tele-FMA(0–88 points)(0–88 points)(0–88 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19(09)0.99 (0.99–1.00)0.98 (0.96–0.99)
50(31)51(31)52(30)0.99 (0.99–1.00)0.98 (0.98–0.99)
 Tele-FMA SEM points (SEM%)0.9 (1.0%)3.1 (3.5%)
 Tele-FMA MDC95 points (MDC95%)2.5 (2.8%)
a

FMA = Fugl-Meyer Assessment; ICC = intraclass correlation coefficient; ICC(3,1) = mixed model, single measurement; ICC(2,1) = random model, single measurement; κw = κ weighted; LE = lower extremity; LoA = limits of agreement; MDC95 = minimal detectable change with 95% CI; SEM = standard error of measurement; Tele = remote; UE = upper extremity. SEM and MDC95 are presented in points and percent (%) of total.

Table 3
Open in new tab

Mean, SD, Intrarater and Interrater Reliability, and Measurement Error for Tele-FMAa

Rater A Time 1  
Mean (SD)		Rater A Time 2  
Mean (SD)		Rater B  
Mean (SD)		Intrarater Reliability  
(CI)		Interrater Reliability  
(CI)
Individual items UE(0–2)(0–2)(0–2)κ wκ w
1.1. Flexor synergy – shoulder retraction1.2 (0.8)1.3 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.63 (0.42–0.83)
1.2. Flexor synergy – shoulder elevation1.4 (0.8)1.5 (0.7)1.3 (0.8)0.80 (0.64–0.95)0.77 (0.61–0.94)
1.3. Flexor synergy – shoulder abduction (90°)1.4 (0.8)1.4 (0.8)1.2 (0.8)1.00 (1.00–1.00)0.79 (0.62–0.95)
1.4. Flexor synergy – shoulder external rotation1.1 (0.9)1.0 (1.0)1.1 (0.9)0.84 (0.70–0.99)0.66 (0.47–0.85)
1.5. Flexor synergy – elbow flexion1.4 (0.8)1.4 (0.8)1.5 (0.8)0.89 (0.77–1.00)0.80 (0.64–0.97)
1.6. Flexor synergy – forearm supination0.8 (0.9)0.8 (0.9)0.9 (0.9)1.00 (1.00–1.00)0.81 (0.67–0.95)
2.1. Extensor synergy – shoulder adduction with internal rotation1.2 (0.9)1.3 (0.9)1.3 (0.9)0.90 (0.79–1.00)0.83 (0.69–0.97)
2.2. Extensor synergy – extension1.2 (0.9)1.2 (0.9)1.2 (0.9)1.00 (1.00–1.00)0.96 (0.90–1.00)
2.3. Extensor synergy – pronation1.1 (1.0)1.2 (1.0)1.1 (0.9)0.87 (0.76–0.99)0.87 (0.76–0.99)
3. Hand to lumbar spine1.3 (0.9)1.3 (0.9)1.4 (0.8)0.90 (0.79–1.00)0.86 (0.74–0.98)
4. Shoulder flexion – 0°- 90°1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
5. Elbow 90° – pronation/supination1.1 (0.9)1.1 (0.9)1.1 (0.9)0.96 (0.91–1.00)1.00 (1.00–1.00)
6. Shoulder abduction 0°- 90°1.0 (1.0)1.0 (1.0)1.1 (0.9)0.97 (0.91–1.00)0.85 (0.73–0.97)
7. Shoulder flexion 90°-180°0.7 (0.8)0.8 (0.9)0.8 (0.9)0.90 (0.79–1.00)0.87 (0.76–0.98)
8. Elbow 0° – pronation and supination0.9 (1.0)0.8 (0.9)1.0 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
9. Wrist stable, elbow at 90°0.9 (1.0)1.0 (1.0)1.0 (0.9)0.91 (0.81–1.00)0.76 (0.61–0.90)
10. Wrist flexion/extension. Elbow at 90°0.9 (0.9)0.9 (0.9)1.1 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
11. Wrist stable, elbow extended1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
12. Wrist flexion/extension, elbow extended0.9 (1.0)0.9 (1.0)0.9 (0.9)0.96 (0.91–1.00)0.90 (0.81–1.00)
13. Wrist circumduction0.9 (0.9)0.9 (0.9)0.8 (0.8)1.00 (1.00–1.00)0.83 (0.70–0.96)
14. Finger mass flexion1.2 (0.9)1.2 (0.9)1.3 (0.8)0.96 (0.90–1.00)0.83 (0.69–0.97)
15. Finger mass extension (relaxation of flexion)1.1 (0.9)1.1 (0.9)1.2 (0.8)1.00 (1.00–1.00)0.90 (0.80–1.00)
16. Hook grasp1.1 (1.0)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
17. Lateral prehension1.1 (1.0)1.1 (1.0)1.2 (1.0)1.00 (1.00–1.00)0.82 (0.63–1.00)
18. Opposition thumb and index0.9 (1.0)0.9 (1.0)0.9 (1.0)0.97 (0.91–1.00)0.88 (0.77–0.99)
19. Cylindrical grasp1.3 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.90–1.00)0.77 (0.59–0.94)
20. Spherical grasp0.9 (1.0)0.9 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.83 (0.69–1.00)
21. Movement without tremor0.8 (0.9)1.0 (0.9)1.0 (1.0)0.84 (0.69–0.99)0.73 (0.57–0.88)
22. Movement without dysmetria0.9 (0.9)0.9 (0.9)1.0 (1.0)0.87 (0.76–0.99)0.75 (0.61–0.90)
23. Movement with normal speed0.7 (0.9)0.7 (0.9)0.7 (0.9)1.00 (1.00–1.00)0.80 (0.66–0.94)
Total score Tele-FMA-UE(0–60 points)(0–60 points)(0–60 points)ICC(3,1) (CI)ICC (2,1) (CI)
31(24)32(24)32 (24)0.99 (0.99–1.00)0.99 (0.98–1.00)
 Tele-FMA-UE SEM points (SEM%)0.9 (1.5%)2.2 (3.7%)
 Tele-FMA-UE MDC95 points (MDC95%)2.5 (2.8%)
Individual Items LE(0–2)(0–2)(0–2)
1.1. Flexor synergy - flexion (hip)1.6 (0.7)1.6 (0.7)1.7 (0.6)1.00 (1.00–1.00)0.83 (0.66–1.00)
1.2. Flexor synergy - flexion (knee)1.5 (0.8)1.6 (0.8)1.7 (0.6)0.95 (0.87–1.00)0.69 (0.47–0.92)
1.3. Flexor synergy – dorsi-flexion1.1 (0.9)1.1 (0.9)1.3 (0.9)0.93 (0.85–1.00)0.67 (0.48–0.87)
2.1. Extension synergy – extension (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.90 (0.78–1.00)
2.2. Extension synergy – adduction (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.86 (0.71–1.00)
2.3. Extension synergy – extension (knee)1.6 (0.8)1.6 (0.8)1.5 (0.7)0.95 (0.86–1.00)0.70 (0.50–0.90)
2.4. Extension synergy – plantar flexion1.1 (0.8)1.1 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.73 (0.55–0.90)
3. Knee flexion (sitting)1.6 (0.7)1.6 (0.7)1.6 (0.7)1.00 (1.00–1.00)0.95 (0.86–1.00)
4. Ankle dorsiflexion (sitting)1.1 (0.8)1.1 (0.9)1.3 (0.9)0.93 (0.84–1.00)0.77 (0.61–0.93)
5. Knee flexion (standing)1.1 (0.8)1.1 (0.8)1.1 (0.8)0.92 (0.83–1.00)0.79 (0.63–0.95)
6. Ankle dorsiflexion (standing)1.0 (0.8)1.0 (0.9)1.0 (0.8)0.90 (0.79–1.00)0.83 (0.69–0.97)
7. Heel-shin tremor1.4 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.89–1.00)0.52 (0.24–0.81)
8. Heel-shin dysmetria1.4 (0.8)1.4 (0.8)1.1 (0.8)0.92 (0.82–1.00)0.59 (0.39–0.80)
9. Heel-shin speed1.0 (0.9)1.0 (1.0)0.9 (0.9)0.93 (0.85–1.00)0.72 (0.57–0.88)
Total score Tele-FMA-LE(0–28 points)(0–28 points)(0–28 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19 (09)0.99 (0.99–1.00)0.98 (0.96–0.99)
 Tele-FMA-LE SEM points (SEM%)0.5 (1.8%)1.8 (6.4%)
 Tele-FMA-LE MDC95 points (MDC95%)1.3 (4.6%)
Total score Tele-FMA(0–88 points)(0–88 points)(0–88 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19(09)0.99 (0.99–1.00)0.98 (0.96–0.99)
50(31)51(31)52(30)0.99 (0.99–1.00)0.98 (0.98–0.99)
 Tele-FMA SEM points (SEM%)0.9 (1.0%)3.1 (3.5%)
 Tele-FMA MDC95 points (MDC95%)2.5 (2.8%)
Rater A Time 1  
Mean (SD)		Rater A Time 2  
Mean (SD)		Rater B  
Mean (SD)		Intrarater Reliability  
(CI)		Interrater Reliability  
(CI)
Individual items UE(0–2)(0–2)(0–2)κ wκ w
1.1. Flexor synergy – shoulder retraction1.2 (0.8)1.3 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.63 (0.42–0.83)
1.2. Flexor synergy – shoulder elevation1.4 (0.8)1.5 (0.7)1.3 (0.8)0.80 (0.64–0.95)0.77 (0.61–0.94)
1.3. Flexor synergy – shoulder abduction (90°)1.4 (0.8)1.4 (0.8)1.2 (0.8)1.00 (1.00–1.00)0.79 (0.62–0.95)
1.4. Flexor synergy – shoulder external rotation1.1 (0.9)1.0 (1.0)1.1 (0.9)0.84 (0.70–0.99)0.66 (0.47–0.85)
1.5. Flexor synergy – elbow flexion1.4 (0.8)1.4 (0.8)1.5 (0.8)0.89 (0.77–1.00)0.80 (0.64–0.97)
1.6. Flexor synergy – forearm supination0.8 (0.9)0.8 (0.9)0.9 (0.9)1.00 (1.00–1.00)0.81 (0.67–0.95)
2.1. Extensor synergy – shoulder adduction with internal rotation1.2 (0.9)1.3 (0.9)1.3 (0.9)0.90 (0.79–1.00)0.83 (0.69–0.97)
2.2. Extensor synergy – extension1.2 (0.9)1.2 (0.9)1.2 (0.9)1.00 (1.00–1.00)0.96 (0.90–1.00)
2.3. Extensor synergy – pronation1.1 (1.0)1.2 (1.0)1.1 (0.9)0.87 (0.76–0.99)0.87 (0.76–0.99)
3. Hand to lumbar spine1.3 (0.9)1.3 (0.9)1.4 (0.8)0.90 (0.79–1.00)0.86 (0.74–0.98)
4. Shoulder flexion – 0°- 90°1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
5. Elbow 90° – pronation/supination1.1 (0.9)1.1 (0.9)1.1 (0.9)0.96 (0.91–1.00)1.00 (1.00–1.00)
6. Shoulder abduction 0°- 90°1.0 (1.0)1.0 (1.0)1.1 (0.9)0.97 (0.91–1.00)0.85 (0.73–0.97)
7. Shoulder flexion 90°-180°0.7 (0.8)0.8 (0.9)0.8 (0.9)0.90 (0.79–1.00)0.87 (0.76–0.98)
8. Elbow 0° – pronation and supination0.9 (1.0)0.8 (0.9)1.0 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
9. Wrist stable, elbow at 90°0.9 (1.0)1.0 (1.0)1.0 (0.9)0.91 (0.81–1.00)0.76 (0.61–0.90)
10. Wrist flexion/extension. Elbow at 90°0.9 (0.9)0.9 (0.9)1.1 (0.9)0.93 (0.85–1.00)0.81 (0.68–0.95)
11. Wrist stable, elbow extended1.0 (1.0)1.0 (1.0)1.0 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
12. Wrist flexion/extension, elbow extended0.9 (1.0)0.9 (1.0)0.9 (0.9)0.96 (0.91–1.00)0.90 (0.81–1.00)
13. Wrist circumduction0.9 (0.9)0.9 (0.9)0.8 (0.8)1.00 (1.00–1.00)0.83 (0.70–0.96)
14. Finger mass flexion1.2 (0.9)1.2 (0.9)1.3 (0.8)0.96 (0.90–1.00)0.83 (0.69–0.97)
15. Finger mass extension (relaxation of flexion)1.1 (0.9)1.1 (0.9)1.2 (0.8)1.00 (1.00–1.00)0.90 (0.80–1.00)
16. Hook grasp1.1 (1.0)1.1 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.94 (0.86–1.00)
17. Lateral prehension1.1 (1.0)1.1 (1.0)1.2 (1.0)1.00 (1.00–1.00)0.82 (0.63–1.00)
18. Opposition thumb and index0.9 (1.0)0.9 (1.0)0.9 (1.0)0.97 (0.91–1.00)0.88 (0.77–0.99)
19. Cylindrical grasp1.3 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.90–1.00)0.77 (0.59–0.94)
20. Spherical grasp0.9 (1.0)0.9 (1.0)1.1 (1.0)1.00 (1.00–1.00)0.83 (0.69–1.00)
21. Movement without tremor0.8 (0.9)1.0 (0.9)1.0 (1.0)0.84 (0.69–0.99)0.73 (0.57–0.88)
22. Movement without dysmetria0.9 (0.9)0.9 (0.9)1.0 (1.0)0.87 (0.76–0.99)0.75 (0.61–0.90)
23. Movement with normal speed0.7 (0.9)0.7 (0.9)0.7 (0.9)1.00 (1.00–1.00)0.80 (0.66–0.94)
Total score Tele-FMA-UE(0–60 points)(0–60 points)(0–60 points)ICC(3,1) (CI)ICC (2,1) (CI)
31(24)32(24)32 (24)0.99 (0.99–1.00)0.99 (0.98–1.00)
 Tele-FMA-UE SEM points (SEM%)0.9 (1.5%)2.2 (3.7%)
 Tele-FMA-UE MDC95 points (MDC95%)2.5 (2.8%)
Individual Items LE(0–2)(0–2)(0–2)
1.1. Flexor synergy - flexion (hip)1.6 (0.7)1.6 (0.7)1.7 (0.6)1.00 (1.00–1.00)0.83 (0.66–1.00)
1.2. Flexor synergy - flexion (knee)1.5 (0.8)1.6 (0.8)1.7 (0.6)0.95 (0.87–1.00)0.69 (0.47–0.92)
1.3. Flexor synergy – dorsi-flexion1.1 (0.9)1.1 (0.9)1.3 (0.9)0.93 (0.85–1.00)0.67 (0.48–0.87)
2.1. Extension synergy – extension (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.90 (0.78–1.00)
2.2. Extension synergy – adduction (hip)1.6 (0.8)1.6 (0.8)1.6 (0.7)1.00 (1.00–1.00)0.86 (0.71–1.00)
2.3. Extension synergy – extension (knee)1.6 (0.8)1.6 (0.8)1.5 (0.7)0.95 (0.86–1.00)0.70 (0.50–0.90)
2.4. Extension synergy – plantar flexion1.1 (0.8)1.1 (0.8)1.1 (0.8)0.89 (0.78–1.00)0.73 (0.55–0.90)
3. Knee flexion (sitting)1.6 (0.7)1.6 (0.7)1.6 (0.7)1.00 (1.00–1.00)0.95 (0.86–1.00)
4. Ankle dorsiflexion (sitting)1.1 (0.8)1.1 (0.9)1.3 (0.9)0.93 (0.84–1.00)0.77 (0.61–0.93)
5. Knee flexion (standing)1.1 (0.8)1.1 (0.8)1.1 (0.8)0.92 (0.83–1.00)0.79 (0.63–0.95)
6. Ankle dorsiflexion (standing)1.0 (0.8)1.0 (0.9)1.0 (0.8)0.90 (0.79–1.00)0.83 (0.69–0.97)
7. Heel-shin tremor1.4 (0.9)1.4 (0.9)1.3 (0.9)0.96 (0.89–1.00)0.52 (0.24–0.81)
8. Heel-shin dysmetria1.4 (0.8)1.4 (0.8)1.1 (0.8)0.92 (0.82–1.00)0.59 (0.39–0.80)
9. Heel-shin speed1.0 (0.9)1.0 (1.0)0.9 (0.9)0.93 (0.85–1.00)0.72 (0.57–0.88)
Total score Tele-FMA-LE(0–28 points)(0–28 points)(0–28 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19 (09)0.99 (0.99–1.00)0.98 (0.96–0.99)
 Tele-FMA-LE SEM points (SEM%)0.5 (1.8%)1.8 (6.4%)
 Tele-FMA-LE MDC95 points (MDC95%)1.3 (4.6%)
Total score Tele-FMA(0–88 points)(0–88 points)(0–88 points)ICC (3,1) (CI)ICC (2,1) (CI)
18 (10)18 (10)19(09)0.99 (0.99–1.00)0.98 (0.96–0.99)
50(31)51(31)52(30)0.99 (0.99–1.00)0.98 (0.98–0.99)
 Tele-FMA SEM points (SEM%)0.9 (1.0%)3.1 (3.5%)
 Tele-FMA MDC95 points (MDC95%)2.5 (2.8%)
a

FMA = Fugl-Meyer Assessment; ICC = intraclass correlation coefficient; ICC(3,1) = mixed model, single measurement; ICC(2,1) = random model, single measurement; κw = κ weighted; LE = lower extremity; LoA = limits of agreement; MDC95 = minimal detectable change with 95% CI; SEM = standard error of measurement; Tele = remote; UE = upper extremity. SEM and MDC95 are presented in points and percent (%) of total.

Minimal Detectable Change

Values for the SEM are given on Table 3. SEM calculated from the intrarater variability presented values below 1 point, whereas the SEM calculated from the variability between raters presented values between 1.8 (Tele-FMA-LE) and 2.2 (Tele-FMA total score). The SEM for all was <10% of the score. The MDC95 calculated were ≤2.5 points and <5% of the score.

The Bland–Altman plot analysis indicated that the mean difference between the 2 raters did not significantly differ from zero. For the total Tele-FMA scores, the mean difference between R1 and R2 scores was −0.5 points. The 95% limits of agreement were +8.1 and − 9.1 points. Mean difference between T1 and T2 scores was −0.7 points. The 95% limits of agreement were 1.8 and − 3.3 points. For the R1 and R2 differences, all but 2 points, and for the T1 and T2 differences all but 1 point fall within the 95% limits, with low and high scores evenly distributed across the total Tele-FMA (Fig. 3). The Bland–Altman graphs show that most of the differences between the scores were within the limits of agreement.

Bland–Altman plot, limits according to evaluators, considering total remote Fugl-Meyer Assessment (Tele-FMA), Tele-FMA-LE (lower extremity), and Tele-FMA-UE (upper extremity). (A) Intrarater total Tele-FMA score. (B) Interrater total Tele-FMA score. (C) Intrarater of Tele-FMA-UE. (D) Interrater of Tele-FMA-UE. (E) Intrarater of Tele-FMA-LE. (F) Interrater of Tele-FMA-LE.
Figure 3

Bland–Altman plot, limits according to evaluators, considering total remote Fugl-Meyer Assessment (Tele-FMA), Tele-FMA-LE (lower extremity), and Tele-FMA-UE (upper extremity). (A) Intrarater total Tele-FMA score. (B) Interrater total Tele-FMA score. (C) Intrarater of Tele-FMA-UE. (D) Interrater of Tele-FMA-UE. (E) Intrarater of Tele-FMA-LE. (F) Interrater of Tele-FMA-LE.

Open in new tabDownload slide

Patients’ Perceived Barriers to Tele-FMA

After completing the Tele-FMA, the participants were asked to engage in a semi-structured interview designed to identify the main barriers found with the use of the Tele-FMA. Problems with the internet signal during the assessment were reported by 9 participants, and 8 participants reported having difficulty understanding the correct way to execute the movements. Three participants had to reschedule their assessments due to problems with the internet signal, but for all the others, the internet problems were solved in the same session.

More than one-half (59%) of the individuals assessed needed the presence of a caregiver most of the time because they felt their limitations prevented or hindered them from using the electronic device. With the 2 open questions, we obtained answers regarding the greatest advantages and greatest disadvantages of this type of assessment. For about 60% of the participants, the greatest advantage was not having to leave the house to perform the assessment. Regarding the greatest disadvantage, about 32% mentioned the absence of the therapist during the assessment and the consequent lack of physical touch and guidance when attempting to execute the movements. However, a considerable percentage (41%) of the participants reported no disadvantage with the Tele-FMA (Suppl. Appendix 4).

The duration of the total Tele-FMA was on average 34.12 (SD 11.46) minutes, for the Tele-FMA-UE it was 21.07 (SD 08.51) minutes, and for the Tele-FMA-LE, 13.05 minutes (SD 05.21).

Discussion

The present study evaluates the validity, reliability, and measurement error of the remotely administered Tele-FMA.

Using the in-person FMA as a gold standard, our results showed that Tele-FMA presents good criterion validity. The construct–convergent validity was established by the correlation between Tele-FMA-UE and SIS hand function domain and between Tele-FMA-LE and the ST.

A few previous studies have reported, in the neurological population, an agreement between in-person and remote assessments. Russel et al24 showed that the Functional Reach Test, the Timed-Up-and Go, and the ST, performed remotely via telerehabilitation system in people with Parkinson disease, were of sufficient accuracy to be used in the clinical setting. In our study, in-person assessment sessions were conducted at the participant’s home; in the case of remotely applied, the assessment was conducted via videoconferencing over the internet (in the remote assessment, the participant was assisted by a caregiver). Results have shown that items during which a relative assists the administration, that is, items where resistance is needed, are not an issue and present excellent reliability. These results are related to the instructions and demonstration given by the rater to the caregiver and also the fact that assessment was synchronous, creating an environment for an immediate and “online” correction in the eventual case of misunderstanding. Mean absolute differences between in-person and remote assessments scores were low. As established for the original FMA-UE,21 the version for remote application, Tele-FMA-UE, presents an adequate convergent validity with SIS hand function domain. Furthermore, and as we hypothesized, the Tele-FMA-LE showed moderate magnitude correlation with ST.

Total Tele-FMA, Tele-FMA-UE, and Tele-FMA-LE present excellent intrarater and interrater reliability. The estimates of measurement error (%SEM) were low (<7%), and the minimal difference needed to conclude the presence of a real change in the Tele-FMA (MDC95%) varied from 2.8% for the total Tele-FMA to 4.6% for the Tele-FMA-LE. Most individual items indicated excellent agreement in this study. Only 2 items from the Tele-FMA-UE items and 4 from the Tele-FMA-LE had κw values <0.70. Those items were shoulder retraction and external rotation for UE flexor synergy and knee flexion and ankle dorsiflexion for LE flexor synergy. Recording the assessment and later viewing the movements at a slower speed27 might facilitate more accurate scoring. Also, heel-shin tremor and heel-shin dysmetria presented κw values <0.70. Difficulties arising from poor internet stability might have compromised the assessment of movement coordination because frames were cut, and it was not possible to assess fine movement. To minimize those difficulties, during remote assessment it may be necessary to request additional footage and/or to ensure there is adequate internet quality to increase scoring accuracy. Although the literature refers to space limitations as a barrier to telerehabilitation.4,6 In this study, no participant was excluded due to a lack of sufficient space in the home to perform the assessment.

In the included studies on systematic reviews about telerehabilitation,4,6 the initial assessment tests were administered in-person, thus masking the real difficulties involved with telerehabilitation, which include remote assessment. Because reliable remote physical assessment is relatively rare, it is of considerable importance to have a widely employed instrument available, such as the FMA, that is reliable for remote administration. The results of this study will allow clinicians to not only employ remote treatment but also use a reliable remotely administered instrument to evaluate patients in relation to their motor impairment post stroke.

The main barrier in the application of the Tele-FMA identified in this study is poor internet quality. A poor or unstable internet connection can lead to interruptions that hinder the understanding of commands and extend the assessment time and may make the assessment unfeasible. However, of the individuals recruited in this study, only 3 had unstable connections that required pausing the assessment, and for 3 others, the assessment had to be rescheduled.

In agreement with Tyagi et al,7 in the present study the principal facilitator reported by the patients and caregivers is accessibility (to be able to stay at home), and the main disadvantage was the absence of physical manual guidance from the clinicians.

It may be difficult for patients with severe impairment who show alterations in trunk control to undertake the Tele-FMA because it is challenging for the caregiver to both position the device and patient in the best position to enable visualization of the items. However, only 3% of the caregivers reported great difficulty in assisting with the Tele-FMA. Only 17% of the participants felt unsafe when performing some movements, and 14% perceived some difficulty in understanding the correct position necessary to perform the proposed items.

The length of time required to apply the Tele-FMA did not differ from that required to apply the in-person FMA: 45 minutes,9 40 minutes,13 and 30 minutes.28 However, for the remote assessment, the instructions must be sent at least 24 hours before the appointment.

When analyzing the scientific and clinical implications of this study, we can affirm that the Tele-FMA, in which the analysis and scoring is based on a video recording, does not require the patient to repeat the task several times, thus allowing this form of assessment to be used in blinded studies. Regarding the clinical implications, having access to Tele-FMA as a reliable remotely administered instrument may facilitate the assessment and reassessment process, enabling this outcome to be used to improve the telerehabilitation modality.

This study has limitations, one being the small sample size for assessing validation. However, as convergent validity is in agreement with the results found in studies with an in-person application21,29 and also looking at the plots, it is reasonable to expect that results regarding the validity will be retained with a bigger sample size. Also, MDC95 has been calculated using SEM obtained from intrarater reliability, and this can represent a limitation of the generalizability of the findings.

The results of the study demonstrate the validity and excellent levels of reliability for the Tele-FMA. Both the measurement error calculated by the SEM and the MDC95 for Tele-FMA were presented. The administration of the Tele-FMA, therefore, is reliable and can contribute to the arsenal of instruments for use in telerehabilitation, helping professionals to obtain reliable data to assess the effectiveness of a proposed intervention.

Author Contributions

Larissa Liz (Conceptualization-Equal, Formal analysis-Equal, Investigation-Equal, Methodology-Equal, Writing – original draft-Equal), Tayara da Silva (Conceptualization-Equal, Investigation-Equal, Methodology-Equal, Writing – review & editing-Equal), Stella Michaelsen (Conceptualization-Equal, Formal analysis-Equal, Funding acquisition-Lead, Investigation-Equal, Methodology-Equal, Project administration-Lead, Resources-Lead, Supervision-Equal, Writing – original draft-Equal, Writing – review & editing-Equal).

Funding

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) – Brazil (Finance Code 001) and by the Fundação de Amparo à Pesquisa e Inovação do Estado de Santa Catarina (FAPESC) (TO 2019TR767).

Ethics Approval

Research protocols were approved by the Institutional Research Ethics Board (4.339.940 and 5.377.738, respectively, for reliability and validity studies).

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Disclosures

The authors completed the ICMJE Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.

References

1.

Virani
 
SS
,
Alonso
 
A
,
Benjamin
 
EJ
.
Heart disease and stroke statistics—2020 update: a report from the American Heart Association
.
Circulation
.
2020
;
141
:
e139
–
e596
.

Google Scholar

PubMed
OpenURL Placeholder Text

 
2.

Mckenna
 
S
,
Jones
 
F
,
Glenfield
 
P
,
Lennon
 
S
.
Bridges self-management program for people with stroke in the community: a feasibility randomized controlled trial
.
Int J Stroke
.
2015
;
10
:
697
–
704
. https://doi.org/10.1111/ijs.12195.

Google Scholar

Crossref
Search ADS
PubMed

 
3.

Morris
 
JH
,
Oliver
 
T
,
Kroll
 
T
,
Joice
 
S
,
Williams
 
B
.
From physical and functional to continuity with pre-stroke self and participation in valued activities: a qualitative exploration of stroke survivors’, carers’ and physiotherapists’ perceptions of physical activity after stroke
.
Disabil Rehabil
.
2015
;
37
:
64
–
77
. https://doi.org/10.3109/09638288.2014.907828.

Google Scholar

Crossref
Search ADS
PubMed

 
4.

Tchero
 
H
,
Teguo
 
MT
,
Lannuzel
 
A
,
Rusch
 
E
.
Telerehabilitation for stroke survivors: systematic review and meta-analysis
.
J Med Internet Res
.
2018
;
20
:
e10867
. https://doi.org/10.2196/10867.

Google Scholar

Crossref
Search ADS
PubMed

 
5.

Chen
 
J
,
Jin
 
W
,
Zhang
 
XX
,
Xu
 
W
,
Liu
 
XN
,
Ren
 
CC
.
Telerehabilitation approaches for stroke patients: systematic review and meta-analysis of randomized controlled trials
.
J Stroke Cerebrovasc Dis
.
2015
;
24
:
2660
–
2668
. https://doi.org/10.1016/j.jstrokecerebrovasdis.2015.09.014.

Google Scholar

Crossref
Search ADS
PubMed

 
6.

Laver
 
KE
,
Adey-Wakeling
 
Z
,
Crotty
 
M
, et al.  
Telerehabilitation services for stroke (review)
.
Cochrane Libr
.
2020
;
1
:
CD010255
. https://doi.org/10.1002/14651858.CD010255.pub3.

Google Scholar

OpenURL Placeholder Text

 
7.

Tyagi
 
S
,
Lim
 
DSY
,
Ho
 
WHH
, et al.  
Acceptance of tele-rehabilitation by stroke patients: perceived barriers and facilitators
.
Arch Phys Med Rehabil
.
2018
;
99
:
2472
–
2477.e2
. https://doi.org/10.1016/j.apmr.2018.04.033.

Google Scholar

Crossref
Search ADS
PubMed

 
8.

Appleby
 
E
,
Gill
 
ST
,
Hayes
 
LK
,
Walker
 
TL
,
Walsh
 
M
,
Kumar
 
S
.
Effectiveness of telerehabilitation in the management of adults with stroke: a systematic review
.
PLoS One
.
2019
;
14
:
1
–
18
. https://doi.org/10.1371/journal.pone.0225150.

Google Scholar

OpenURL Placeholder Text

 
9.

Sullivan
 
KJ
,
Tilson
 
JK
,
Cen
 
SY
, et al.  
Fugl-Meyer assessment of sensorimotor function after stroke: standardized training procedure for clinical practice and clinical trials
.
Stroke
.
2011
;
42
:
427
–
432
. https://doi.org/10.1161/STROKEAHA.110.592766.

Google Scholar

Crossref
Search ADS
PubMed

 
10.

Michaelsen
 
SM
,
Rocha
 
AS
,
Knabben
 
RJ
,
Rodrigues
 
LP
,
Fernandes
 
CGC
.
Translation, adaptation and inter-rater reliability of the administration manual for the Fugl-Meyer assessment
.
Braz J Phys Ther
.
2011
;
15
:
80
–
88
. https://doi.org/10.1590/S1413-35552011000100013.

Google Scholar

Crossref
Search ADS

 
11.

Hernández
 
ED
,
Galeano
 
CP
,
Barbosa
 
NE
, et al.  
Intra and interrater reliability of the upper extremity Fugl-Meyer evaluation in stroke
.
J Rehabil Med
.
2019
;
51
:
652
–
659
. https://doi.org/10.2340/16501977-2590.

Google Scholar

Crossref
Search ADS
PubMed

 
12.

Amano
 
S
,
Umeji
 
A
,
Uchita
 
A
, et al.  
Reliability of remote evaluation for the Fugl–Meyer assessment and the action research arm test in hemiparetic patients after stroke
.
Top Stroke Rehabil
.
2018
;
25
:
432
–
437
. https://doi.org/10.1080/10749357.2018.1481569.

Google Scholar

Crossref
Search ADS
PubMed

 
13.

Cacho
 
E
,
Melo
 
F
,
Oliveira
 
R
.
Assessment of motor recovery of hemiplegic patients through the Fugl-Meyer physical performance protocol
.
Revista Neurociências
.
2004
;
12
:
94
–
102
. https://doi.org/10.34024/rnc.2004.v12.8877.

Google Scholar

Crossref
Search ADS

 
14.

Durfee
 
K
,
Savard
 
L
,
Weinstein
 
S
.
Technical feasibility of teleassessments for rehabilitation
.
IEEE Trans Neural Syst Rehabil Eng
.
2007
;
15
:
23
–
29
. https://doi.org/10.1109/TNSRE.2007.891400.

Google Scholar

Crossref
Search ADS
PubMed

 
15.

Woodbury
 
M
,
Velozo
 
CA
,
Richards
 
LG
,
Duncan
 
PW
.
Rasch analysis staging methodology to classify upper extremity movement impairment after stroke
.
Arch Phys Med Rehabil
.
2013
;
94
:
1527
–
1533
. https://doi.org/10.1016/j.apmr.2013.03.007.

Google Scholar

Crossref
Search ADS
PubMed

 
16.

Woodbury
 
M
,
Velozo
 
CA
,
Richards
 
LG
,
Duncan
 
PW
,
Studenski
 
S
,
Lai
 
S
.
Longitudinal stability of the Fugl-Meyer assessment of the upper extremity
.
Arch Phys Med Rehabil
.
2008
;
89
:
1563
–
1569
. https://doi.org/10.1016/j.apmr.2007.12.041.

Google Scholar

Crossref
Search ADS
PubMed

 
17.

Camozzato
 
AL
,
Kochhann
 
R
,
Godinho
 
C
,
Costa
 
A
,
Chaves
 
ML
.
Validation of a telephone screening test for Alzheimer's disease. Neuropsychol dev Cogn B
.
Aging Neuropsychol Cogn
.
2011
;
18
:
180
–
194
. https://doi.org/10.1080/13825585.2010.521814.

Google Scholar

Crossref
Search ADS

 
18.

Roccaforte
 
WH
,
Burke
 
WJ
,
Bayer
 
BL
,
Wengel
 
SP
.
Validation of a telephone version of the mini-mental state examination
.
J Am Geriatr Soc
.
1992
;
40
:
697
–
702
. https://doi.org/10.1111/j.1532-5415.1992.tb01962.x.

Google Scholar

Crossref
Search ADS
PubMed

 
19.

Fantauzzi
 
M.
 
Profile of individuals returning to work after stroke
 
[master's thesis]
.
Minas Gerais
:
Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
,
2020
.
100
.

20.

Carod-Artal
 
FJ
,
Coral
 
LF
,
Trizotto
 
DS
,
Moreira
 
CM
.
The stroke impact scale 3.0: evaluation of acceptability, reliability, and validity of the Brazilian version
.
Stroke
.
2008
;
39
:
2477
–
2484
. https://doi.org/10.1161/STROKEAHA.107.513671.

Google Scholar

Crossref
Search ADS
PubMed

 
21.

Duncan
 
PW
,
Wallace
 
D
,
Lai
 
SM
,
Johnson
 
D
,
Embretson
 
S
,
Laster
 
LJ
.
The stroke impact scale version 2.0. Evaluation of reliability, validity, and sensitivity to change
.
Stroke
.
1999
;
30
:
2131
–
2140
. https://doi.org/10.1161/01.STR.30.10.2131.

Google Scholar

Crossref
Search ADS
PubMed

 
22.

Kwon
 
S
,
Duncan
 
P
,
Studenski
 
S
,
Perera
 
S
,
Lai
 
SM
,
Reker
 
D
.
Measuring stroke impact with SIS: construct validity of SIS telephone administration
.
Qual Life Res
.
2006
;
15
:
367
–
376
. https://doi.org/10.1007/s11136-005-2292-2.

Google Scholar

Crossref
Search ADS
PubMed

 
23.

Hong
 
SJ
,
Goh
 
EY
,
Chua
 
SY
,
Ng
 
SS
.
Reliability and validity of step test scores in subjects with chronic stroke
.
Arch Phys Med Rehabil
.
2012
;
93
:
1065
–
1071
. https://doi.org/10.1016/j.apmr.2011.12.022.

Google Scholar

Crossref
Search ADS
PubMed

 
24.

Russell
 
TG
,
Hoffmann
 
TC
,
Nelson
 
M
,
Thompson
 
L
,
Vincent
 
A
.
Internet-based physical assessment of people with Parkinson disease is accurate and reliable: a pilot study
.
J Rehabil Res Dev
.
2013
;
50
:
643
–
650
. https://doi.org/10.1682/JRRD.2012.08.0148.

Google Scholar

Crossref
Search ADS
PubMed

 
25.

Portney
 
LG
.
Foundations of clinical research: applications to evidence-based practice
.
FA Davis
;
2020
.

Google Scholar

OpenURL Placeholder Text

 
26.

Terwee
 
CB
,
Bot
 
SD
,
de
 
Boer
 
MR
, et al.  
Quality criteria were proposed for measurement properties of health status questionnaires
.
J Clin Epidemiol
.
2007
;
60
:
34
–
42
. https://doi.org/10.1016/j.jclinepi.2006.03.012.

Google Scholar

Crossref
Search ADS
PubMed

 
27.

Venkataraman
 
K
,
Morgan
 
M
,
Amis
 
KA
, et al.  
Tele-assessment of the berg balance scale: effects of transmission characteristics
.
Arch Phys Med Rehabil
.
2017
;
98
:
659
–
664.e1
. https://doi.org/10.1016/j.apmr.2016.10.019.

Google Scholar

Crossref
Search ADS
PubMed

 
28.

See
 
J
,
Dodakian
 
L
,
Chou
 
C
, et al.  
A standardized approach to the Fugl-Meyer assessment and its implications for clinical trials
.
Neurorehabil Neural Repair
.
2013
;
27
:
732
–
741
. https://doi.org/10.1177/1545968313491000.

Google Scholar

Crossref
Search ADS
PubMed

 
29.

Gladstone
 
DJ
,
Danells
 
CJ
,
Black
 
SE
.
The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties
.
Neurorehabil Neural Repair
.
2002
;
16
:
232
–
240
. https://doi.org/10.1177/154596802401105171.

Google Scholar

Crossref
Search ADS
PubMed

 
© The Author(s) 2023. Published by Oxford University Press on behalf of the American Physical Therapy Association. All rights reserved. For permissions, please e-mail: [email protected]
This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://dbpia.nl.go.kr/pages/standard-publication-reuse-rights)
Issue Section:
ORIGINAL RESEARCH
Download all slides
  • Supplementary data

    • Supplementary data

    Suppl_Appendix_1_r3_pzad054 - pdf file
    Suppl_Appendix_2_r3_pzad054 - pdf file
    Suppl_Appendix_3_r3_pzad054 - pdf file
    Suppl_Appendix_4_r3_pzad054 - pdf file

    Comments

    0 Comments
    Comments (0)
    Submit a comment
    You have entered an invalid code
    Thank you for submitting a comment on this article. Your comment will be reviewed and published at the journal's discretion. Please check for further notifications by email.