Quantifying Turning Tasks With Wearable Sensors: A Reliability Assessment

Abstract

Objective

The aim of this study was to establish the test–retest reliability of metrics obtained from wearable inertial sensors that reflect turning performance during tasks designed to imitate various turns in daily activity.

Methods

Seventy-one adults who were healthy completed 3 turning tasks: a 1-minute walk along a 6-m walkway, a modified Illinois Agility Test (mIAT), and a complex turning course (CTC). Peak axial turning and rotational velocity (yaw angular velocity) were extracted from wearable inertial sensors on the head, trunk, and lumbar spine. Intraclass correlation coefficients (ICCs) were established to assess the test–retest reliability of average peak turning speed for each task. Lap time was collected for reliability analysis as well.

Results

Turning speed across all tasks demonstrated good to excellent reliability, with the highest reliability noted for the CTC (45-degree turns: ICC = 0.73–0.81; 90-degree turns: ICC = 0.71–0.83; and 135-degree turns: ICC = 0.72–0.80). The reliability of turning speed during 180-degree turns from the 1-minute walk was consistent across all body segments (ICC = 0.74–0.76). mIAT reliability ranged from fair to excellent (end turns: ICC = 0.52–0.72; mid turns: ICC = 0.50–0.56; and slalom turns: ICC = 0.66–0.84). The CTC average lap time demonstrated good test–retest reliability (ICC = 0.69), and the mIAT average lap time test–retest reliability was excellent (ICC = 0.91).

Conclusion

Turning speed measured by inertial sensors is a reliable outcome across a variety of ecologically valid turning tasks that can be easily tested in a clinical environment.

Impact

Turning performance is a reliable and important measure that should be included in clinical assessments and clinical trials.

Introduction

Turning is common during walking and accounts for over 40% of steps taken during daily life.¹ Compared to linear gait (ie, straight walking), turning is a more complex movement, as it requires greater spatial coordination between limbs, more coupling between the balance and gait control systems, and precise timing between body segments.^2–7 Turning during daily life is particularly important to consider because turning has been identified as a primary cause of falls in older adults.⁸ Consequently, assessing turning performance is a critical component of comprehensive mobility assessments.

The relevance of turning extends across many different clinical populations.^9–16 Older adults turn less efficiently, as evidenced by altered kinematics and neuromuscular modulation.¹⁷ Specifically, older adults take longer to turn, take more steps per turn, and have higher trunk peak roll (side-to-side) velocities than younger adults, and these deficits are associated with diminished balance confidence.¹⁸ Similarly, persons with Parkinson disease (PD) take more steps per turn, have slower turn times, and demonstrate poor intersegmental coordination.¹⁹ In individuals with PD, freezing of gait is often elicited during a turn.²⁰ These impairments in turning may cause an increased risk of falls. Importantly, people with PD fall 5 times more often than elderly people matched for age.²¹ In older adults, falls during turns are 8 times more likely to result in hip fracture.²² Similarly, high rates of falls occur in people recovering after a stroke, with most falls in patients after stroke occurring when they turn toward their paretic side.²³ Specific turning deficits in people with stroke may relate to a significant delay in initiating body segment movement compared with individuals who are healthy.²⁴

The importance of turning across populations encourages the use of quantitative assessments of turning, such as turning speed, for discriminating populations that are healthy and populations with clinical conditions, even in the early or acute stages of a disease or condition. Instrumented measures of turning during the Timed “Up & Go” Test (TUG) can differentiate people in the early stages of PD, while stopwatch measures of the total TUG time did not²⁵; similar findings have been reported in people with mild multiple sclerosis.¹⁶ Additionally, subtle motor deficits after concussion have been detected during turning tasks both in the clinic and during home monitoring.^15,26 Several studies have reported that the quality of turns can be impaired, while regular activity metrics may be normal.^26,27 Early identification of individuals at risk for falls using turning measures as outcomes would allow for early interventions and possibly the avoidance of falls and injury.

Unfortunately, there are limited clinical scales that can directly and objectively measure turning in clinical practice. The TUG, a clinical test of mobility and fall risk, is one of the most commonly used tests that includes 2 180-degree turns as part of the assessment.²⁸ The TUG evaluates performance based on total time to complete the task of standing, walking, turning, and returning to a sitting position. However, using total time as the outcome variable for the TUG can collapse the overall performance during the trial into 1 measure. Thus, total time lacks the resolution to determine the performance of the individual components such as turning during the TUG. Another widely used clinical test is the Berg Balance Scale, which consists of 14 tasks that assess static and dynamic balance abilities, including a 360-degree turn.²⁹ Although useful in detecting moderate to severe balance deficits in patients, the Berg Balance Scale has a reported ceiling effect and does not detect more subtle deficits in people with known turning impairments.⁶

Objectively quantifying turning is readily available with the use of wearable inertial measurement units (IMUs), which allow for precise quantification of turning during both prescribed tasks and free-living mobility. Inertial measurements of turning are reliable and accurate when compared to motion capture systems.^30,31 Recently, there has been an increase in the number of studies using IMU-based turning metrics as outcome measures including the duration, speed, and number of steps used to turn^32–34 both in the laboratory and in natural, daily-life environments.^10,33,34

Standard clinical measures, such as the TUG, can easily be instrumented with IMUs to differentiate the turn from the overall task of standing, walking, and turning. An instrumented version of the TUG is reliable, valid, and sensitive.²⁵ However, since the TUG only evaluates a 180-degree turn, it does not capture the range of turns relevant to participants’ daily life, such as various turn angles, speeds, and directions. Our group recently published a study comparing turning speed using several instrumented turning tasks that attempt to more closely mimic daily life including the complex turning course (CTC), the modified Illinois Agility Test (mIAT), and a 1-minute walk (1 MW).³⁵ We compared the performance of participants who were healthy across different research sites and found between-site equivalencies, lending more evidence to the usefulness of instrumented measures of turning as an outcome.³⁶

There are a limited number of assessments analyzing turning outcomes and fewer with established clinometric properties. To enhance the utility of turning as a meaningful clinical measure, clinometric properties including test–retest reliability of turning across different turning tasks need to be assessed. Examining the reliability of turning metrics during different turning tasks will yield foundational work necessary for interpreting meaningful changes in longitudinal study results. Therefore, the aim of this study was to establish, in adults who were healthy, the test–retest reliability of turning tasks reflective of those performed during routine daily activities using metrics obtained from IMUs.

Methods

Participants

This investigation was part of a larger study aimed to assess the diagnostic accuracy, predictive capacity, and responsiveness to rehabilitation of objective, dual-task turning measures in people with mild traumatic brain injury.³⁴ Our data were collected from 71 participants at 3 participating sites: University of Utah, Salt Lake City, Utah (n = 34; 8 men, 26 women; mean age = 29.5 [SD = 7.8] years; mass = 69.5 [SD = 15.12] kg; height = 1.7 [SD = 0.1] m); Courage Kenny Research Center, Minneapolis, Minnesota (n = 11; 1 man, 10 women; age = 40.4 [SD = 8.6] years; mass = 73.4 [SD = 13.7] kg; height = 1.6 [SD = 0.1] m); and Fort Sam Houston, San Antonio, Texas (n = 26; 16 men, 10 women; age = 27.2 [SD = 4.1] years; mass = 78.4 [SD = 14.9] kg; height = 1.7 [SD = 0.1] m). The institutional review board at each site approved this study. Participants were included if they were between the ages of 18 and 50 years. Exclusion criteria included having any medical condition that could affect balance (such as a concussion within 7 years, stroke, multiple sclerosis, lower limb injury, or pregnancy), psychological disorder (such as schizophrenia or psychosis), substance use disorder within the prior month (based on criteria for moderate to severe substance use disorder, as defined by the Diagnostic and Statistical Manual of Mental Disorders, 5th edition),³⁷ significant pain (>7/10 by patient subjective report), or inability to communicate in English. All participants provided informed consent prior to data collection.

Procedures

Participants performed the following turning tasks in a randomized order (Figure): 1 MW, mIAT, and CTC. For the 1 MW, participants were instructed to walk back and forth at a self-selected pace between 2 lines 6 m apart for 1 minute (see Suppl. Material for full instructions on each task), the direction of their turn at the ends of the walkway was not designated. The mIAT is a test of speed and agility in which participants were instructed to run at their maximum safe speed while performing U-turns (end and mid turns) and weaving through cones (slalom turns)^38,39 (Suppl. Material). The CTC is a walking course designed to mimic turn angles performed in daily life.³⁵ For the CTC, participants were instructed to walk at their self-selected speed for 2 minutes. Scripted instructions for each task were read to the participant, and practice laps were performed before testing (Suppl. Material). The participants repeated the same turning tasks in a new randomized order, approximately 1 hour later (ie, test–retest reliability). During the time interval between testing, participants remained in the testing facility. We assessed 180-degree turns during the 1 MW, 180-degree, and slalom turns during running in the mIAT, and 45-, 90-, and 135-degree turns during the CTC, consistent with previous studies on intersite differences in these tasks.³⁶ This study was compliant with the Consensus-Based Standards for the Selection of Health Measurement Instruments.⁴⁰

Data were collected during turning tasks using synchronized IMUs (128 Hz; APDM Inc, Portland, OR, USA) attached to the forehead (head), sternum (trunk), and pelvis (lumbar spine) via elastic straps over the clothes and underneath any outerwear (ie, coats, jackets, hats). IMUs contain onboard accelerometers, gyroscopes, and a magnetometer used to capture head and trunk movement.

Data Analysis

Data were batch processed and turn outcomes were calculated after data collection. This process removes bias that may occur as the administrator is unaware of other participants’ or previous trial outcomes. The data analysis process that we used is detailed in our previous work.³⁶ To establish a consistent reference point for each trial, a method called functional alignment was employed.⁴¹ This method aligns the vertical axis with gravity and creates a reference frame fixed to the body. A 1.5-Hz low-pass phaseless Butterworth filter was used to determine the peak turning speed of each turn performed in the turning tasks.³⁶ The maximum filtered angular velocity about the vertical axis, the peak turning speed, was our primary outcome. Peak turning speeds were extracted from the 1 MW, CTC, and mIAT assessments and processed using custom MATLAB (The MathWorks, Natick, MA, USA) algorithms. Peak turning speed has demonstrated sensitivity in discerning between populations with postural instability^15,16,42,43 and populations that are healthy (controls) and between fallers and nonfallers.⁴⁴ The turning phases were identified using the same turn identification algorithm for all tasks.³⁶ The final outcome for the 1 MW included the average peak turning speed at the head, trunk, and lumbar spine. For the mIAT, the peak turning speeds were determined throughout the trial and were matched to each turn type (slalom, mid, and end turns). The final outcome for mIAT included the average peak turning speeds at the head, trunk, and lumbar spine for each turn type, as well as the average lap time measured from the IMUs. Similarly, peak turning speeds were identified and matched to each turn angle of the CTC: 45, 90, and 135 degrees. The final outcome for CTC included the average peak turning speed at the head, trunk, and lumbar spine for each turn angle and the average lap time.

Statistical Analysis

Descriptive statistics of peak turning speed in each trial and the difference between trials were calculated. The ICC(2,1) was used to assess the test–retest reliability of the average peak turning speed for 1 MW, mIAT, and CTC and the lap/total time of the CTC and mIAT.⁴⁵ ICC values were classified using the following scale⁴⁶: poor (0–0.39), fair (0.40–0.59), good (0.60–0.74), and excellent (0.75–1.0). Limits of agreement (LoA) were used to quantify the spread among the paired differences and were chosen as the measurement error parameter. The 95% LoA were calculated with the following formula: d ± 1.96 × SD_difference,⁴⁷ where d represents the mean difference in turning speed between trial 1 and trial 2, and SD_difference represents the SD of the mean difference.

Role of the Funding Source

The funders played no role in the design, conduct, or reporting of this study.

Results

Descriptive data of the test and retest trials and reliability statistics by task, turn type, and body segment are provided in the Table.

1-Minute Walk

Excellent test–retest reliability was found for peak turning speed of the 180-degree turn at the trunk (ICC = 0.76) and lumbar segments (ICC = 0.75) during the 1 MW. The peak turning speed at the head displayed good test–retest reliability during the 1 MW (ICC = 0.74).

Modified Illinois Agility Test

We assessed the peak turning by turn type across the segments. End-turn reliability ranged from fair to good (ICC = 0.52–0.72), mid-turn reliability was fair (ICC = 0.5–0.56), and slalom-turn reliability ranged from good to excellent (ICC = 0.66–0.84). mIAT average lap time test–retest reliability was excellent (ICC = 0.91; 95% CI = 0.851–0.945).

Complex Turning Course

Across all body segments, good to excellent test–retest reliability was found for peak turning speed during the 45-degree turns (ICC = 0.73–0.81), 90-degree turns (ICC = 0.71–0.83), and 135-degree turns (ICC = 0.72–0.80). Average lap time of CTC demonstrated good test–retest reliability (ICC = 0.69; 95% CI = 0.22–0.85).

Discussion

This study established the reliability of various turning assessment tasks using IMUs to quantify turning performance. We found that the peak turning speed of most turning tasks had good to excellent reliability with the best reliability noted for the 3 different angle turns of the CTC. Noninstrumented measures of lap time had reliability ranging from good to excellent. Overall, our findings suggest that turning speed and timing measures are reliable and may be an important measure to include in clinical assessments and clinical trials.

A strength of this study design was capturing reliability of turning speed during several turn angles during running and walking tasks, making these reliability metrics applicable to a variety of turning assessments. Our battery of turning tasks included angles that ranged from 45 to 180 degrees as well as a slalom turn and were tested during both comfortable speed walking and running at the fastest safe speed. When people were tested at their natural speed across turning angles, although not statistically assessed, it did not appear to be an obvious trend of 1 turn angle being more reliable than any other. This finding is important since there is evidence that some patient populations have difficulty with specific turning angles. People in the chronic stage of mild traumatic brain injury tend to perform larger turn angles than people who are healthy (controls),²⁶ while people with PD and freezing of gait perform smaller turn angles than people with PD but without freezing of gait.²⁰ However, in our study, when participants were tested using the mIAT, which involves running at their maximum speeds, we observed greater variability in our reliability results across the tasks that we used in our study. Specifically, the ICC values in the running task (mIAT) were lower than the ICC values for the walking tasks (1 MW and CTC). This observation should be explored further as IMUs are emerging in sports and highly demand environments.^48–50

Although the use of IMUs in research has increased, it is still uncommon for clinicians to have access and expertise to use IMUs to quantify turning in the clinic.^32,34,44 Thus, clinical assessments of basic turning performance, such as lap time, are also important. Lap times were included in our study because they could be captured without IMUs. Interestingly, the reliability of these simple clinical outcomes of lap time or completion time, when compared to instrumented outcomes of turning speed, varied between the mIAT and the CTC. Instrumented measures of turning from IMUs had higher reliability than lap time during the CTC but lower reliability than lap time during the mIAT. These differences are likely due to differences in task instruction (Suppl. Material); the mIAT instructions are “maximum safe speed,” while the CTC instructions are “at a comfortable pace” that may fluctuate more freely than a maximum speed. Future work may wish to investigate whether the CTC could be completed at faster speeds, and whether such instructions improve the reliability of noninstrumented measures of lap time. In settings where an IMU is not available (such as most clinics), lap time can be measured using a simple stopwatch. It is important to note that measuring lap time does not allow for differentiation between individual turns. Therefore, adopting a fully integrated IMU-based turn analysis would also include automatic start and end time identification that could yield detailed information for clinicians.

IMU placement is a practical consideration when measuring performance. Most commonly, a lumbar or a pelvis location for IMUs is used to evaluate turning outcomes.^{15,26,51–53} The sacrum, lower anterior thigh, middle lateral shank, and heel have been shown to produce relatively low error for the segments during the TUG.⁵⁴ Optimizing sensor placement is important for complex tasks that involve different speed changes (rapid turns) or visual search strategies involving both eye and head motion to identify orientation landmarks.^55,56 In our study, we found that the turning tasks in which participants walked at their self-selected walking speed (1 MW and CTC) had relatively similar ICC values across all 3 IMU locations compared to the running task (mIAT), which showed a higher variability in ICC values. Specifically, for the walking tasks (1 MW and CTC), the turning ICC values ranged from 0.71 to 0.83 across all 3 IMU locations, while for the running task (mIAT), the turning ICC values ranged from 0.50 to 0.84 across all 3 IMU locations.

Assessment and treatment of turning-related deficits have been gaining interest within the rehabilitation community in recent years. In addition to turns being impaired in the early stages of PD, IMU-based measures of turning can detect fluctuations in turning throughout the day.^34,57 These measures can provide important information on both fatigue and optimization of medications. Moreover, fatigue causes alteration in gait,⁵⁸ but there is no information on the effects of fatigue on turning. Optimizing medication regimen has enormous benefits in people with PD, including increased activity and reduction of fall risk.^59,60 Similarly, quantification of turning has implications for rehabilitation. Clinical practice guidelines from the American Physical Therapy Association recommend task-specific training for turning in people with PD.⁶¹ However, only 2 studies were able to successfully incorporate a turning intervention in an aquatic setting⁶² or using a rotational treadmill and turn-specific training program⁶³ that led to improvements in turning. Although there is limited research on the effectiveness of a turning intervention,⁶⁴ future rehabilitation research focused on improving turning can be confident in test–retest reliability of turning assessments.

Presenting a measurement error parameter enables users to determine whether changes in measurement should be attributed to actual change or simply reflect measurement error. The 95% LoA are expected to encompass 95% of the differences between paired measurements. In our study, we observed that the LoA were wider for larger turn angles (Table), indicating a greater variability in measurements for larger turn angles than for smaller ones. We chose to utilize LoA because our sample consisted of participants who were healthy, and this method does not rely on ICC values, which are highly influenced by the heterogeneity of the population under study. It is worth noting that a positive difference between the first and second trials indicated a faster turning speed during the second trial, possibly indicating a learning effect.

The results of our reliability analysis may also help interpret previously published data in clinical populations. In addition to providing reliability metrics for turning angle speed, the SD between trials can help with comparisons between populations that are healthy and populations with various clinical conditions. These data inform us about how people differ from each other (interpersonal SD), the SD of sample in trial 1 and trial 2, as well as how they differ within themselves from trial to trial (intrapersonal SD), SD of the difference between trials. For instance, turning speed was measured during 180-degree turns in participants with an acute vestibular schwannoma resection and people who were healthy (controls).⁴³ As expected, the participants with an acute vestibular schwannoma resection turned significantly slower than their counterparts who were healthy. Their average turning speed also fell outside of the variance expected in a population that is healthy. The SD of the 180-degree turn calculated from our sample (interpersonal SD = 41.83 degrees per second) was smaller than the mean difference in turn speed between the participants with an acute vestibular schwannoma resection and people who were healthy (controls) (51.7 degrees per second). An earlier version of the CTC assessed turning velocities in people with chronic mild traumatic brain injury and people who were healthy (controls).¹⁵ A notable distinction in turning speed was found between the 2 groups when measured using trunk and lumbar sensors.¹⁵ However, the difference in the study of Fino et al¹⁵ did not exceed the interpersonal SD calculated from our sample. This is not surprising as differences between group means can be detected more easily than the SD for individual observations.⁴⁷

Limitations

This study is the first to demonstrate the reliability of turning in different turning tasks. However, this study is not without limitations. One potential limitation of our study is the time interval (1-hour difference) between the test–retest measurements of turning tasks, which might have induced a practice or retest effect.⁶⁵ It is likely that a longer duration between measurements (eg, several days) may yield even better agreement between trials. Another possible limitation is that our study assessed turning at 3 different data collection sites, each with potentially different parameters such as testing environment and experimenters. However, we previously found equivalency in turning parameters at 3 different sites.³⁶ Although these turning parameters were established by previous literature,^32,36,66 it should be noted that these results may be specific to the turning phase thresholds and filters used. One possible factor that we did not consider is directional preference, which could potentially influence the patterns of right versus left directional turns. For instance, since no specific instructions were given regarding the direction participants should turn during the 1 MW task, their preference might have an impact on the observed turning patterns. Future studies should consider collecting data on directional preference to further examine the influence of this factor on turning patterns. Finally, turning during activities of daily living becomes more difficult with age due to increasing sensorimotor impairments.^67,68 Our study used a relatively young adult sample and future research should evaluate turns in older and clinical populations.

Our study provides a reliability analysis of using wearables to quantify turning behaviors of young adults who are healthy—a potential step toward increasing the use of clinical turning assessments. Establishing turning reliability in adults who are healthy provides a reference group when examining participants with postural instabilities in cross-sectional research as well as a comparison for assessments at multiple time points (eg, acute and postrecovery assessments in mild traumatic brain injury). The 1 MW demonstrated good test–retest reliability across all body segments. Although turning speed proved to be reliable during the mIAT, it was observed to vary across turn angles and body segments. The highest reliability of turning speed was noted for the CTC, a clinical assessment designed to mimic turning angles found during daily life. Future work should validate the CTC by comparing turns during this test to turns during daily life, outside of the clinic or laboratory. Turning is a novel and emerging area of both research and potentially clinical care.

Author Contributions

Angela R. Weston (Data curation, Formal analysis, Investigation, Writing—original draft-Lead, Writing—review & editing), Prokopios Antonellis (Formal analysis, Investigation, Writing—original draft, Writing—review & editing), Peter C. Fino (Conceptualization, Data curation, Funding acquisition, Methodology, Resources, Writing—review & editing), Carrie W. Hoppes (Conceptualization, Data curation, Funding acquisition, Methodology, Resources, Writing—review & editing), Mark E. Lester (Conceptualization, Data curation, Methodology, Writing—review & editing), Margaret M. Weightman (Conceptualization, Data curation, Methodology, Resources, Writing—review & editing), Leland E. Dibble (Conceptualization, Data curation , Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing–review & editing), and Laurie A. King (Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing–review & editing)

Ethics Approval

This study protocol was approved by the Courage Kenny Research Center, Minneapolis, Fort Sam Houston, San Antonio, and the University of Utah, Salt Lake City Institutional Review Boards, as well as the overseeing IRB; Oregon Health & Science University and Human Research Protection Office (HRPO).

Funding

This study was supported by the Assistant Secretary of Defense for Health Affairs endorsed by the Department of Defense, through the Congressionally Directed Medical Research Program under Award No. W81XWH1820049 and by the Telemedicine and Advanced Technology Research Center (TATRC) through the Army Medical Department Advanced Medical Technology Initiative (AAMTI), 2018.

Data Availability

The data presented in this study will be available upon request.

Disclosures and Presentations

The authors completed the ICMJE Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.

A poster with the findings of this study was presented at the 2023 American Physical Therapy Association Combined Sections Meeting; February 23–25; San Diego, California.

References

Author notes