Validation of the Musculoskeletal Health Questionnaire in inflammatory arthritis: a psychometric evaluation Free

Abstract

Objectives

The Musculoskeletal Health Questionnaire (MSK-HQ) is a recently developed Patient Reported Outcome Measure for use across patients with musculoskeletal conditions. This study provides a validation of the MSK-HQ examining construct validity and reliability in inflammatory arthritis.

Methods

287 adults with inflammatory arthritis completed the MSK-HQ and other Patient Reported Outcome Measures at baseline and after 3 months. Construct validity was assessed using item response theory methods. Concurrent validity was considered in relation to the HAQ and EuroQol-5D in all patients, as well as the RA Impact of Disease (RAID), PsA Impact of Disease scales, and AS Quality of Life Questionnaire (ASQoL) in disease subtypes.

Results

The MSK-HQ was approximately normally distributed with no evidence of floor or ceiling effects. A unidimensional structure was confirmed. Two items were less weakly related to the latent musculoskeletal health variable, providing some evidence of multidimensionality. Reliability was high (α = 0.93). The total score correlated highly with the HAQ (r = −0.81) and EuroQol-5D index (r = 0.80) in all patients, RAID in RA patients (r = −0.88), PsA Impact of Disease in PsA patients (r = −0.88), and ASQoL in AS patients (r = −0.86). Test-retest reliability was high (r_ICC = 0.73).

Conclusion

This study provides evidence for the validity and reliability of the MSK-HQ in people with inflammatory arthritis. The major advantage of the MSK-HQ is that it is not disease specific and has high content validity in rheumatological conditions. The MSK-HQ score will be of value in clinical rheumatology practice, providing a measure that can be used across disease areas.

Introduction

Collecting patient views about their own health using Patient Reported Outcome Measures (PROMs) is central to evaluating clinical care in routine practice and research [1, 2]. For instance, PROMS can act as a catalyst for organisational change by improving standards, as evidenced by the NHS England National PROMs Programme [3]. A large number of PROMs currently exist, many addressing aspects of musculoskeletal health. Generic PROMs assessing health related quality of life (e.g. 36-Item Short Form Health Survey, EuroQol-5D) allow for broad comparisons to be made across health conditions. However, generic instruments may have limited sensitivity to compare between musculoskeletal conditions, as they do not necessarily capture constructs pertinent to musculoskeletal health [4]. Within rheumatology, PROMs assessing musculoskeletal health related quality of life have, in general, been developed specific to individual diseases (e.g. RA Impact of Disease (RAID), PsA Impact of Disease (PsAID)); until recently there was no musculoskeletal health PROM validated across the spectrum of musculoskeletal conditions.

The Arthritis Research UK Musculoskeletal Health Questionnaire (MSK-HQ) is a recently developed PROM assessing musculoskeletal health related quality of life for use by patients with different musculoskeletal conditions, across a range of settings [5]. This scale incorporates the assessment of physical symptoms such as pain and fatigue, plus the impact of disease on psychological well-being.

The initial development of the MSK-HQ included a detailed scoping exercise and qualitative development phase involving both patients and clinicians [5]. In a sample of 570 people with osteoarthritis, the MSK-HQ was demonstrated to have good internal consistency between items, as well as high test-retest reliability and convergent validity compared with other measures of musculoskeletal health used in osteoarthritis populations. While initial findings are promising, additional validation studies are needed to confirm the utility of the MSK-HQ as an outcome measure for more widespread use [6]. Specifically, though content (e.g. face) validity for the MSK-HQ is high, further research is needed to examine the construct validity of the scale, by confirming the assumed unidimensional structure, and its reliability in other musculoskeletal conditions.

The aim of the present study was to examine the construct validity and test-retest reliability of the MSK-HQ in people with inflammatory arthritis.

Methods

Sample

Patients were recruited from six centres between September 2015 and July 2016. Eligibility criteria were adults with an established diagnosis of inflammatory arthritis (RA, undifferentiated inflammatory arthritis (UIA), PsA, AS) who were initiating a new synthetic or biologic anti-rheumatic medication as part of routine care. Follow up data were collected by post at 3 months, following baseline visit.

Outcomes measures

MSK-HQ

The MSK-HQ consists of 14 items relating to facets of musculoskeletal health including pain, fatigue, physical function, symptom interference, sleep, self-efficacy and psychological well-being [5]. Respondents rate how much their musculoskeletal condition in the previous 2 weeks has affected each facet using a five-point ordinal scale from not at all to extremely, respectively scored from four to zero. Items 12, relating to understanding, and 13, relating to self-efficacy, have their response scale reversed where not at all is scored zero. Items are summed to generate a total score with a range from 0 to 56, where higher scores indicate better musculoskeletal health status. A fifteenth item assessing physical activity in the past week is not included in the total score and was therefore not considered in the current analysis. Previous analysis in non-inflammatory musculoskeletal disease has demonstrated good reliability (Cronbach's α = 0.88, test-retest = 0.84) [5]. Sample copies of the MSK-HQ and a license for use can be obtained at https://innovation.ox.ac.uk/clinical-outcomes/patient-reported-outcome-measures. Ethical approval was granted (15/WM/0189) and all patients gave informed consent to participate in this study.

Other PROMs

All participants completed the HAQ [7], which assesses functioning across eight domains. This is the most widely used measure of functional ability in musculoskeletal conditions and contains 20 items assessing the difficulty of completing specific daily activities across eight domains, as well as a further 18 items with a binary response format relating to the use of assistive devices or help from others. Scale scores range between 0 and 3, increasing in increments of 0.125, where higher scores indicate worse functional limitation.

All participants also completed the EQ5D-5L, a generic measure of health related quality of life. The version used includes response categories with five levels of severity for each of five items assessing mobility, self-care, usual activities, pain/discomfort and anxiety/depression. The responses to these items are combined against value sets to generate an index value (utility) that reflects the preferences compared with other responses. An additional item asks respondents to rate the health on a 0–100 visual analogue scale.

Patients with RA and UIA completed the RAID questionnaire [8]. This scale consists of seven items scored between 0 and 10. The total score is calculated as a weighted mean across all items, where weights are based on patient rating of importance. The scale score ranges between 0 and 10, where higher scores indicate greater impact of disease. A minimum clinically important difference (MCID) of an absolute change of three has been recommended [9].

Patients with PsA completed the nine item version of the PsAID [10]. This scale consists of nine items scored between 0 and 10. As with the RAID, the total score is calculated as a weighted mean across all items providing a scale score ranging between 0 and 10, where higher scores indicate greater impact of disease. An MCID of three has also been proposed.

Patients with AS completed the AS Quality of Life questionnaire (ASQoL) [11] and BASDAI [12].

Data analysis

Construct validity was assessed using item response theory (IRT) methods [13]. These approaches assume responses to the set of items forming the scale are due to a single underlying latent variable (e.g. musculoskeletal health). Therefore, unidimensionality was initially examined using principal components analysis with parallel analysis to determine the number of underlying latent variables explaining more than chance variance in item response correlations [14]. Because the response scale is ordinal, the polychoric method was used to estimate the correlation between items [15]. After examining unidimensionality, IRT methods explored the functioning of individual items. A graded response model for ordinal items was estimated [16]. Preliminary analyses confirmed a graded response model provided a better fit to the data than a partial credit or generalised partial credit model. The graded response model estimates a discrimination parameter for each item and a difficulty parameter per response category for each item. Respectively, these parameters represent the strength of the association (correlation) between the item and the level of the latent musculoskeletal health variable being assessed, and the level of the latent variable where an individual is estimated to have 50% probability of responding positively to the response category for the item. Items with low discrimination that do not correlate highly with the latent musculoskeletal health variable are considered poor indicators that should not be included in the scale [14]. Here we use the rule of thumb from the factor analytic literature that items that correlate <0.6 are weak.

To test the assumption that scores on the MSK-HQ are comparable across disease groups, differential item functioning was assessed using ordinal logistic regression models. This approach, which is a reparameterisation of the graded response model [17], sequentially compares models for differences in both difficulty and discrimination parameters (uniform and non-uniform differential item functioning). A false discovery rate correction was applied to control for multiple testing.

Reliability, representing the precision with which the scale score assesses an individual’s true level of musculoskeletal health, was assessed using four methods. Using the baseline assessment, the association between items was summarized using Cronbach’s estimate of internal consistency α. Measurement precision across the range of musculoskeletal health was further assessed by examining the test information function from the graded response model. A change greater than measurement error was computed based on the reliable change index [18], which uses alpha and the baseline standard deviation to determine the 95% interval where changes that may be due to chance alone would lie. Test-retest reliability for the scale score was assessed using the intra-class correlation coefficient and for individual items using Kendall’s concordance coefficient, a descriptive measure indicating agreement between ordinal rankings. All analyses were conducted in Stata 14.2.

Results

The sample included people with RA (208, 72%), PsA (45, 16%), AS (22, 8%) and UIA (12, 4%). All patients were included in the main analyses; disease specific analyses were performed for RA and PsA only. Of the 287 providing data at baseline, 178 (62%) provided data again after 3 weeks to allow test-retest reliability to be assessed. Baseline information is provided in Table 1.

Information about the functioning of the 14 items is given in the supplementary data, available at Rheumatology online. High acceptability was indicated by low levels of missing data per item (range 0.0% to 1.7%). All items showed responses across the range of categories with positive inter-item correlations ranging from weak to strong (range 0.07–0.82, mean inter-item correlation 0.57), though the majority of correlations were strong (i.e. r > 0.5).

The distribution of the MSK-HQ total score was approximately normally distributed with participants scoring across the entire range from 0 to 56. No floor or ceiling effect was observed. RA patients had the most favourable MSK-HQ total scores, with a mean score of 26.8 (s.d. = 11.9) compared with 24.6 (13.1) for PsA patients and 23.2 (8.6) for those with other inflammatory arthritis conditions.

Structural validity

The first principal component of the item correlation matrix explained 62.8% of the total variance, indicating that a single underlying latent musculoskeletal health variable explained the majority of the variability in item responses. However, parallel analysis suggested the pattern of correlations between item responses was likely to result from two underlying latent variables. Specifically, an additional minor latent variable explained a further 9.5% of the item variance, which was greater than expected by chance. Examining inter-item and item-total correlations (supplementary materials, available at Rheumatology online) revealed that items 12 understanding and 13 self-efficacy were less related to overall musculoskeletal quality of life compared with other items and potentially reflect a separate construct.

The IRT graded response model confirmed this observation. The item characteristic curves for each response category for each item are shown in the supplementary material, available at Rheumatology online. These plots show the probability that each response category is endorsed given an individual’s level of the latent musculoskeletal health variable. The gradient of the item characteristic curve is defined by the item’s discrimination parameter, and the point at which it crosses 0.5 by its difficulty parameter. Transforming the discrimination parameters to the more familiar correlation metric indicates that the correlation between the response to each item and the latent musculoskeletal health variable was between r = 0.75 and 0.91 for all variables, except for understanding (r = 0.19) and self-efficacy (r = 0.52). Further analyses consider the total scale score across all items and a reduced scale score excluding these two items.

Additional analysis considered differential item functioning by inflammatory arthritis subtype for those with RA and PsA. Due to low numbers those with AS and UIA were not included in this analysis. For the total score across all items, item 9 sleep (χ²(2) = 11.37, P = 0.003) and item 14 overall impact (χ²(2) = 7.79, P = 0.020) demonstrated non-uniform differential item functioning between those with RA and PsA. This was also observed for the reduced scale score. In both instances, responses to the items were better at discriminating between levels of musculoskeletal health in the PsA group than the RA group. The magnitude of bias for the overall impact item was small but more considerable for the sleep item.

Concurrent validity

The MSK-HQ was compared against five instruments assessing similar constructs as a test of construct validity (Fig. 1). A strong positive correlation was observed with the EQ5D Index (n = 285, r = 0.80) and visual analogue scale (n = 285, r = 0.60) for all patients. Strong negative correlations were observed with the HAQ for all participants (n = 284, r = −0.81), the RAID for those with RA or UIA (n = 218, r = −0.88), the PsAID for those with PsA (n = 43, r = −0.88), and the ASQoL for those with AS (n = 22, r = −0.86). This supports the assumption that the MSK-HQ total score measures a highly related latent variable to these measures.

There was some indication of non-linearity in the relationship between the MSK-HQ with the HAQ and EQ5D Index, which is suggestive of the HAQ and EQ5D Index being more sensitive at differentiating between those with extremely poor musculoskeletal health. However, the MSK-HQ performs better in differentiating between those with better function as there is no floor effect observed, as is present for HAQ.

Reliability

Reliability was high for both the total scale score including all items and the reduced scale score excluding items 12 and 13: α = 0.93 and α = 0.95, respectively. The IRT construct of information is related to reliability and has the advantage that measurement precision can be assessed at different points across the range of the latent variable. The test information curve (Fig. 2) demonstrates that reliability was high (>0.8) between around three standard deviations below and above the mean of the latent musculoskeletal health variable. Converted to the metric of the MSK-HQ, reliability is above 0.8 for people scoring between ∼3 and 54, where the range 0–56. A reliable change above measurement error was estimated to be a change of nine points or more for the full scale, and seven points or more for the reduced scale, given its higher reliability and reduced range.

Test-retest reliability

Data at follow up were provided by 178 (67.9%) participants with RA, PsA or UIA. Of these, 143 (80.3%) had stable symptoms as indicated by differences in either RAID or PsAID scores less than the MCID. Within those with stable symptoms, the MSK-HQ total score test-retest reliability had an intraclass correlation coefficient r_ICC = 0.73 (95% CI: 0.67, 0.81) and for the reduced scale score r_ICC = 0.75 (95% CI: 0.65, 0.80). Examining test-retest agreement at the item level, Kendall’s concordance coefficient ranged from 0.54–0.81 across all items and was >0.7 for all items except 12 understanding and 13 self-efficacy)

Discussion

This study has confirmed that the MSK-HQ performs well as a PROM for assessing musculoskeletal health related quality of life within an inflammatory arthritis cohort, with acceptable psychometric properties. The MSK-HQ is simple to administer and has good acceptability with patients. The tool includes seven domains including pain, fatigue, physical function, symptom interference, sleep, self-efficacy and psychological well-being. These domains were selected through a process of extensive consultation with patients to ensure the measure captured items that matter most [5]. The total score is reached by summing up each individual item, making the tool also simple for clinicians to use. The result ranges from 0 (worst musculoskeletal health) to 56 (best). We estimate a statistically reliable change, where measurement error can be ruled out for 95% of cases, to be a change in the full scale score of nine points or higher.

The MSK-HQ was highly related to other measures of musculoskeletal health in the sample, with correlations with the HAQ, EQ5D index, RAID PsAID, ASQoL all >0.8. Compared with the HAQ, the MSK-HQ will perform better at differentiating between those where the impact of disease is relatively mild due to the lack of floor effect. Furthermore, while there have been attempts to improve the validity of the HAQ as a tool for capturing musculoskeletal health [19, 20], the HAQ predominantly considers functional limitation related to the upper limbs and as such assesses only one important facet of musculoskeletal health.

Given the high degree of content overlap relating to both the RAID and PsAID, the high correlations with the MSK-HQ and these instruments are not surprising. The RAID and PsAID perform well psychometrically [8, 10], and there is little reason to recommend the use of the MSK-HQ over these two recently and well developed PROMs in disease specific studies. In our view, each will adequately assess health related quality of life; defined by the impact of inflammatory arthritis on a range of lower level constructs including physical function, pain, fatigue, sleep, and psychological well-being. The advantage of the MSK-HQ, over the RAID and PsAID, is that it can potentially be used as a universal questionnaire across musculoskeletal conditions. This not only allows for the comparison of disease burden across conditions but reduces the logistical burden on clinicians and researchers having to manage multiple PROMs; reducing the risk that incorrect PROMs are completed. Although the MSK-HQ, RAID and PsAID assess a broad health related quality of life construct, they are still specific to populations with a musculoskeletal disorder. In situations where health economic analysis requires the calculation of utilities (e.g. for comparison across conditions) generic health related quality of life measures, such as the EQ5D, will also need to be administered.

The psychometric properties of the MSK-HQ are important to consider. While the overall analysis supports the pragmatic use of the total score as a single measure of musculoskeletal health, interrogation of the measure reveals some interesting, if not unexpected, results. Two of the 14 items, relating to understanding and self-efficacy (illness perception items), appear to behave differently to the other items: the relationship between overall musculoskeletal health and individual items is weaker for the illness perception items. In particular, the understanding item was only weakly related to musculoskeletal health, indicating that an individual’s reported level of understanding is not particularly related to the severity of their condition. Understanding, as a construct, reflects something distinct from, though related to, musculoskeletal health. It is likely that the understanding item assesses the concept of illness coherence from the self-regulation model of illness [21]. Similarly, the self-efficacy item is related to the concept of personal control within this theoretical framework. The self-regulation model of illness, which includes other important illness perceptions of consequences, chronicity, stability, and treatment control, describes the process whereby beliefs influence the individual’s emotional response to their illness and their coping behaviours, such as adherence to treatment [22, 23]. In inflammatory arthritis, patients’ perceptions of their illness have been shown to be associated with treatment adherence, function, pain and mood even after controlling for underlying disease [22, 24–26].

The relevance of these psychometric characteristics depends upon how the tool is intended to be used. As a measure for a clinical trial or assessing the standard of care in routine practice where a high degree of reliability is desired, it might be prudent to analyse the 12 items that performed best independently (items 1–11, & 14). When using the tool in the clinic, the impact of the two items is unlikely to make meaningful difference. In contrast, having insight into an individual’s illness perceptions has substantial clinical utility. Supporting people to develop understanding and self-efficacy is an important goal for health services to achieve given their relationship to self-management and outcomes. Furthermore, given the method of development of the MSK-HQ, the inclusion of illness perceptions highlights their importance to patients. The use of validated PROMs for assessing the broader range of illness perceptions alongside the MSK-HQ may provide additional information about the mechanisms by which the disease impacts musculoskeletal health [27, 28].

The study must be considered alongside its limitations. The sample was predominantly RA and PsA, and so the conclusions across the other disease areas need further study. We found some indication of differential item functioning across inflammatory arthritis subtypes. While the level observed is unlikely to bias comparisons between different inflammatory arthritis subtypes, it is difficult to draw robust inferences given the size of the sub groups. There is a clear need for further study with considerably larger sample sizes. In addition, while we provide some information potentially relevant to sensitivity to change, in terms of a change that is greater than the expected natural variation in scores, more robust assessment is required to determine whether such a change is clinically meaningful to patients. Calculation of the MCID may reveal a change that is smaller than the statistically reliable change is meaningful.

In this study, we have provided further support for the validity and reliability of a new musculoskeletal health PROM in people with inflammatory arthritis. The tool compares well to existing PROMs, with the added advantage of lacking the floor effect of HAQ. The major advantage of the MSK-HQ is that it is not disease specific and has high content validity in rheumatological conditions. We believe that the MSK-HQ score will be of value in clinical rheumatology practice, providing a measure that can be used across disease areas.

Acknowledgements

The authors thank the patients participating in this study and their clinicians. S.N. was supported by an MQ/Arthritis Research UK fellowship.

Funding: This project was funded by Arthritis Research UK (Ref. 20518).

Disclosure statement: The authors have declared no conflicts of interest.

References