Small differences in patient-reported outcomes before and after treatment might be statistically significant but clinically irrelevant to patients.¹ The minimal clinically important or meaningful difference (MCID) was devised to address this issue.² The MCID is the smallest difference in a score that a patient would perceive as important. This measure is helpful for sample size calculations. Trials should aim to detect the smallest difference that a patient would find meaningful.

The MCID is calculated as the difference between people who don’t improve and those who improve minimally. It’s not possible to measure a MCID when all patients are expected to improve, as with low-risk arm fractures (e.g. a nondisplaced distal radius or radial head fracture).³ For this setting the appropriate measure of a meaningful change to patients is termed “substantial clinical benefit” (SCB). The SCB is the difference in clinical outcome scores between patients with marginal improvement as compared to those with significant improvement.⁴

The SCB for pain intensity and QuickDASH (Disabilities of the Arm, Shoulder and Hand) during recovery after a low-risk arm fracture is unknown. Also, while patient-reported outcomes after musculoskeletal trauma are associated with psychological factors,⁵ it is unknown what factors are associated with experiencing a SCB. This study aims to assess which changes in QuickDASH score and pain intensity score indicate a SCB during recovery from an adequately-aligned metacarpal, distal radius, or radial head fracture. This study tests the hypotheses that there are no factors independently associated with experiencing a SCB in (1) QuickDASH and (2) pain score 2 to 6 months after such fractures.

After institutional review board approval was obtained for secondary use of previously collected data, we reviewed 120 adult (18 years or older) patients between May 2012 and September 2014. Patients were enrolled if they had an adequately aligned metacarpal fracture (n=63), non or minimally displaced distal radius fracture (n=39) and isolated non or minimally-displaced radial head fracture (n=18). Patients who were non-English speaking or unable to complete enrollment forms were excluded. Patients were enrolled at the end of their clinical visit with a hand surgeon after diagnosis was determined. Informed consent for the original study was obtained prior to enrollment. During the study period, four invited patients declined participation.

Measured Variables

At the end of their clinical visit, patients chose whether or not to schedule a follow-up appointment. Subsequently, a research assistant recorded (1) patients’ demographics: age, sex, marital status, employment, education, and tobacco use; (2) fracture type: metacarpal, distal radius, radial head; and (3) QuickDASH, Patient Health Questionnaire-2 (PHQ-2), Pain Self-Efficacy Questionnaires (PSEQ), pain score, overall satisfaction with their hand/arm, satisfaction with care delivered, satisfaction with the physician, overall evaluation of the visit, and if the patient felt that the initial visit could have been performed by phone or email without the need for physical examination.

Patients were contacted between two and six months after enrollment (mean 93 days ±46 days) by phone or email. At this time we recorded QuickDASH, pain score, satisfaction with upper extremity function, satisfaction with care, satisfaction with the physician, and current employment status.

We used the anchor-based approach to define the SCB.4 The anchor-based approach uses an external criterion (the anchor) to identify substantial benefit. We used change in satisfaction with upper extremity function to identify patients who sustained a SCB. The biggest decrease in satisfaction was eight points, while the largest increase was nine points. We selected the middle value (median) of 1.4 points as our cut-off to define clinical benefit (i.e., a median split). If satisfaction changed less than or equal to 1.4 points, we regarded this as no or limited clinical benefit. If satisfaction increased more than 1.4 points this indicated SCB.4 Because the anchor-based method does not take into account an instrument’s measurement error, we calculated the minimum detectable change in QuickDASH and pain intensity. The minimum detectable change establishes what change in outcome can reliably be measured by the instrument. If the SCB is below the minimum detectable change, it cannot be reliably measured.

Questionnaires

Disability was measured using the, QuickDASH.^6,7 This 11-question survey is scored from 0-100 with a higher score indicating greater disability.

The Patient Health Questionnaire-2 (PHQ-2)⁸ measures symptoms of depression and consists of 2 questions. Scores range from 0 to 6, with higher scores indicating more depressive symptoms.

The Pain Self-Efficacy Questionnaire (PSEQ)⁹ is a 10-measure questionnaire, scored from 0-60 with higher scores indicative of having more confidence performing activities while having pain.

Subjects rated pain intensity, satisfaction with upper extremity function, and overall evaluation of their visit on 11-point ordinal scales ranging from 0 (no pain, dissatisfaction, and useless) to 10 (worst pain, satisfaction, and worthwhile). Care delivered at the office and satisfaction with the physician was rated on a 5-point ordinal scales ranging from 1 (poor) to 5 (excellent).

Statistical Analysis

Missing variables were estimated by median imputation for questionnaire values as described below: 13 items for PSEQ at enrollment and 5 items for QuickDASH at follow-up; and mean imputation for 1 complete QuickDASH at enrollment, 1 complete PHQ-2 at enrollment, 1 complete PSEQ at enrollment, 5 education, 4 hand and arm satisfaction, 1 satisfaction with personal manner of the physician, 2 evaluation of the visit, and 2 pain at enrollment.

We used frequencies to describe discrete variables; continuous variables are reported as means and standard deviations.

We calculated the minimum detectable change using a one-sided confidence level of 90%, similar to previous literature:

Minimum detectable change = 1.65 x √(2) x standard error of measurement.¹⁰

The standard error of measurement is the error estimate for a single measurement. It is calculated using the following formula:

Standard error of measurement = standard deviation x √(1-alpha).¹⁰

We used the standard deviation of the variable at enrollment and alpha is the test-retest reliability coefficient. Enrollment QuickDASH standard deviation was 18.5 and 2.3 for pain score. The intraclass correlation coefficient for test-retest reliability of the QuickDASH is 0.9611 and 0.88 for pain score.¹² This results in a minimum detectable change of 8.6 for QuickDASH and 1.8 for pain score (Table 2).

TABLE 2

Change in patient reported outcomes

All Rights Reserved. Permission For Use Required.

-View Large-

We determined the SCB by Receiver Operator Characteristics (ROC) curve. We considered changes in QuickDASH and pain score as diagnostic and changes in satisfaction with upper extremity function (the anchor) the gold standard. The ROC curve plots sensitivity against 1-specificity for all possible cutoff points of the change in QuickDASH and pain score. The C statistic (area under the ROC curve) ranges from 0.5 to 1, with a higher score indicating better discrimination. The optimal ROC cutoff point, representing the SCB, is the value for which the sum of percentages of false positives and false negatives is smallest. The C statistic can be used to assess how accurately the model identifies patients achieving a SCB. A C statistic of 0.5 indicates that the measure is no better than chance at predicting a result. A C statistic of greater that 0.7 is considered a reasonable predictive model and if greater than 0.8, the model is considered excellent.

Fischer’s Exact test was conducted to determine the differences between categorical variables. Unpaired Student’s t-test was used to determine the differences between continuous and dichotomous variables. All variables with p < 0.10 on explanatory bivariate analysis were entered into a multivariable logistic regression model (Table 3). We regarded a two-tailed p value < 0.05 to be significant.

TABLE 3

Factors associated with achieving the substantial clinical benefit (SCB) in QuickDASH and pain score

All Rights Reserved. Permission For Use Required.

-View Large-

As there is no consensus regarding the optimal sample size to determine SCB a power analysis was not performed.

Patient Characteristics

A total of 120 patients were enrolled of whom 82 (68%) were available for followup two to six months after enrollment. Mean age was 45 (±20) years and 48 (59%) were women (Table 1). QuickDASH scores were 46 (±19) at enrollment and 12 (±15) after two to six months. Pain score was 3.6 (±2.3) and changed to 1.6 (±2.4). Satisfaction with upper extremity function was 6.3 (±2.8) after fracture and 8.2 (±1.8) two to six months later (Table 2). Men (responder 41% [34/82] male vs. non-responder 76% [29/38] male, p< 0.001) and patients with greater symptoms of depression (mean responders PHQ-2 0.46 ± 1.0 vs. non-responders 1.2 ± 1.7, p = 0.0033) were less likely to be available for follow-up (Appendix 1).

TABLE 1

Patient demographics

All Rights Reserved. Permission For Use Required.

-View Large-

Thresholds for Substantial Clinical Benefit

The SCB threshold was 28 points for QuickDASH and 2 points for pain intensity. Both are higher than the minimum detectable change. The C statistic of 0.59 indicated that our model for QuickDASH differentiated poorly between patients achieving a SCB and those who did not. Our model for pain score performed somewhat better, C statistic 0.65 (Table 2).

Accounting for potential interaction of variables using multivariable analysis, there were no variables associated with achieving a SCB in QuickDASH (Table 4).

TABLE 4

Multivariable analysis of independent variables

All Rights Reserved. Permission For Use Required.

-View Large-

Greater self-efficacy at enrollment was independently associated with a lower chance of experiencing a SCB in pain intensity (odds ratio 0.93, 95% confidence interval 0.88 to 0.99, standard error 0.029, P = 0.029, C statistic 0.74) during recovery. In other words, each point increase in self-efficacy decreased the odds of achieving a SCB in pain intensity by 0.07 (Table 4). Additional analysis showed that less self-efficacy was associated with more pain at enrollment (r = -0.43, P < 0.001), but not 2 to 6 months after fracture (r = -0.16, P = 0.15).

Small statistically significant differences in patient-reported outcomes before and after treatment may not be clinically relevant. We calculated the amount of improvement in disability and pain intensity after low-risk arm fractures that patients would perceive as a substantial clinical benefit (SCB). The ability of the SCB to differentiate between people satisfied and unsatisfied with their arm function is limited. Less adaptive people (i.e. with lower self-efficacy) were more likely to achieve a SCB in pain intensity. This complicates the interpretation of minimum clinical improvement measures.

This study has some limitations. First, only 62% (82 of 120) of patients were available for a follow-up via phone or email 2 to 6 months after enrollment; however, this is not unusual for this type of prospective research studying a trauma cohort.¹³ Secondly, median and mean imputations were used to calculate missing questionnaire values. While different values may have effected our results missing data was minimal and use of median and mean imputations was utilized only for a small amount of the total study data.

To our knowledge, no studies to date have assessed the SCB for QuickDASH and only one study has assessed the MCID for QuickDASH. Our SCB score was much higher (28 points on QuickDASH) than the MCID in patients with shoulder pain completing physical therapy (8 points).¹⁴ This study found the MCID for numerical pain score to be 1.1 and found a lower C statistic for pain intensity (0.67) than QuickDASH (0.82). The SCB in back and leg pain after surgery for lumbar spinal stenosis was 2.5 points.⁴ This study found a C statistic of 0.84 for back pain and 0.82 for leg pain. The difference in SCB and C statistic for QuickDASH and numerical pain score show that clinically relevant improvement measures are likely affected by the condition being evaluated, the type of treatment, and the timing of evaluation. In other words, there may not be one clinical improvement threshold for a measurement instrument that applies to all contexts. Also the low C statistic in our study demonstrates that for the specific conditions evaluated calculation of a clinical benefit score by ROC curves cannot reliably differentiate between satisfied and unsatisfied patients.

We found no factors to be independently associated with experiencing a SCB in QuickDASH. Lower self efficacy – less belief in one’s ability to successfully achieve a desired outcome – was associated with a higher change of achieving a SCB in pain intensity. We know of no study that previously assessed factors associated with achieving a clinical relevant difference in QuickDASH and pain intensity in this context. However, previous studies have found that functional outcome and pain intensity after musculoskeletal trauma is associated with stress, distress, and self-efficacy.⁵

Our results suggest that, in minor upper extremity fractures with uniformly good recovery, the biggest changes occur in the least adaptive people. This is may be due to the fact that less adaptive people have lower baseline scores. The fact that patients with more baseline limitations show greater improvement is consistent with prior research.¹⁵ Potential reasons for this are regression to the mean, floor and ceiling effects of a questionnaire, and non-interval scales (e.g. on a 0 to 10 scale the experienced difference between 1 and 2 is potentially greater than the difference between 9 and 10). If payers, policy makers, doctors and patients utilize measures or thresholds such as clinically-relevant differences to determine effective treatments (i.e. treatments that exceed the MCID or SCB), we might select treatments that help patients with less self-efficacy, whether or not those treatments effectively treat the underlying pathophysiology. Future studies should assess the SCB for other conditions in which the majority of patients are expected to improve, and assess if psychological factors influence experiencing a SCB.