Advertisement

Assessing the clinical significance of change scores recorded on subjective outcome measures

      Abstract

      Background

      To date, clinical trials have relied almost exclusively on the statistical significance of changes in scores from outcome measures in interpreting the effectiveness of treatment interventions. It is becoming increasingly important, however, to determine the clinical rather than statistical significance of these change scores.

      Objective

      To determine cutoff values for change scores that distinguish patients who have clinically improved from those who have not.

      Method

      Data were obtained from 165 back and 100 neck patients undergoing chiropractic treatment. Patients completed the Bournemouth Questionnaire (BQ) before treatment and the BQ and Patient's Global Impression of Change (PGIC) scale after treatment. Three statistical methods were applied to individual change scores on the BQ. These were (1) the Reliable Change Index (RCI); (2) the effect size (ES); and (3) the raw and percentage change scores. The PGIC scale was used as the “gold standard” of clinically significant change.

      Results

      The RCI, using the cutoff value of >1.96, appropriately identified clinical improvement in back patients but not in neck patients. An individual ES of approximately 0.5 had the highest sensitivity and specificity in distinguishing back and neck patients who had undergone clinically significant improvement from those who had not. In terms of raw score changes, percentage BQ change scores [(raw change score/baseline score) x 100] of 47% and 34% were identified as having the highest sensitivity and specificity in distinguishing clinically significant improvement from nonimprovement in back and neck patients, respectively.

      Conclusion

      This study provides a methodological framework for identifying clinically significant change in patients. This approach has important implications in providing clinically relevant information about the effect of a treatment intervention in an individual patient.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Manipulative & Physiological Therapeutics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Guyatt G.H
        • Juniper E.F
        • Walter S.D
        • Griffith L.E
        • Goldstein R.S
        Interpreting treatment effects in randomized trials.
        Br Med J. 1998; 316: 690-693
      1. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine. London: Churchill Livingstone; 2000. p. 105-53

        • Turk D.C
        Statistical significance and clinical significance are not synonyms!.
        Clin J Pain. 2000; 16: 185-187
        • Wyrwich K.W
        • Wolinsky F.D
        Identifying meaningful intra-individual change standards for health-related quality of life measures.
        J Eval Clin Pract. 2000; 6: 39-49
        • Middel B
        • Stewart R
        • Bouma J
        • van Sonderen E
        • van den Heuvel W
        How to validate clinically important change in health-related functional status. Is the magnitude of the effect size consistently related to magnitude of change as indicated by a global question rating?.
        J Eval Clin Pract. 2001; 7: 399-410
      2. Cohen J. Statistical power analysis for the behavioural sciences. New York: Academic Press; 1977

        • Testa M
        Interpreting quality of life clinical trial data for use in the clinical practice of antihypertensive therapy.
        J Hypertens Suppl. 1987; 5: S9-S13
        • Jacobson N.S
        • Follette W.G
        • Revenstorf D
        Psychotherapy outcome research.
        Behav Ther. 1984; 15: 336-352
        • Christensen L
        • Mendoza J
        A method of assessing change in a single subject.
        Behav Ther. 1986; 17: 305-308
        • Wyrwich K
        • Nienaber N
        • Tierney W
        • Wolinsky F
        Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life.
        Med Care. 1999; 37: 469-478
        • Farrar J.T
        • Young J.P
        • LaMoreaux L
        • Werth J.L
        • Poole M
        Clinical importance of changes in chronic pain intensity measured on an 11-point numerical rating scale.
        Pain. 2001; 94: 149-158
        • Rowbotham M.C
        What is a ‘clinically meaningful’ reduction in pain?.
        Pain. 2001; 94: 131-132
        • Deyo R.A
        • Centor R.M
        Assessing the responsiveness of functional scales to clinical change.
        J Chronic Dis. 1986; 39: 897-906
        • Bolton J.E
        • Breen A.C
        The Bournemouth Questionnaire.
        J Manipulative Physiol Ther. 1999; 22: 503-510
        • Bolton J.E
        • Humphreys B.K
        The Bournemouth Questionnaire.
        J Manipulative Physiol Ther. 2001; 25: 141-148
        • Turk D.C
        • Okifuji A
        • Sinclair J.D
        • Starz T.W
        Interdisciplinary treatment for fibromyalgia syndrome.
        Arthritis Care Res. 1998; 11: 186-195
        • Kazis L.E
        • Anderson J.J
        • Meenan R.F
        Effect sizes for interpreting changes in health status.
        Med Care. 1989; 27: S178-189
        • Little D.G
        • MacDonald D
        The use of the percentage change in Oswestry Disability Index score as an outcome measure in lumbar spinal surgery.
        Spine. 1994; 19: 2139-2143
        • Farrar J.T
        • Portenoy R.K
        • Berlin J.A
        • Kinman J.L
        • Strom B.L
        Defining the clinically important difference in pain outcome measures.
        Pain. 2000; 88: 287-294
      3. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine. London: Churchill Livingstone; 2000. p. 67-93

        • Jaeschke R
        • Singer J
        • Guyatt G.H
        Measurement of health status.
        Control Clin Trials. 1989; 10: 407-415
        • Juniper E.F
        • Guyatt G.H
        • Willan A
        • Griffith L.E
        Determining a minimal important change in a disease-specific quality of life questionnaire.
        J Clin Epidemiol. 1994; 47: 81-87
        • Beurskens A.J.H.M
        • de Vet H.C.W
        • Koke A.J.A
        Responsiveness of functional status in low back pain.
        Pain. 1996; 65: 71-76
        • Lijmer J.G
        • Mol B.W
        • Heisterkamp S
        • Bonsel G.J
        • Prins M.H
        • van der Meulen J.H.P
        • et al.
        Empirical evidence of design-related bias in studies of diagnostic tests.
        JAMA. 1999; 282: 1061-1066