Using Evidence-Based Practice to Decide on an Outcome Measure

Original Editor - Wanda van Niekerk based on the course by Benita Olivier

Top Contributors - Wanda van Niekerk and Jess Bell  

Introduction

An outcome measure is a tool used to assess a patient's current status. It can provide a score, an interpretation of results and / or a risk categorisation of the patient. It can provide baseline data prior to an intervention, and the initial results may help determine and direct the course of a treatment intervention. The outcome measure can also be used during the course of treatment to assess any "change in the construct of interest over time, during and after rehabilitation."

Outcome measures that are used in clinical practice include:

• Patient-reported outcome measures (PROM)
• Therapist-administered outcome measures
• Clinical tests
• Observed outcome measures

Psychometric Properties of Outcome Measures

Reliability

• Refers to the extent to which a measurement is consistent and free from error
• Does the outcome measure produce consistent results?
• Will the results be the same when the test is done by different clinicians or by the same clinician at different times?

Types of reliability:

• Inter-rater reliability
  • different evaluators, usually within the same time period
  • describes the stability of the scores obtained when two different raters carry out the same test
  • "the degree to which independent observers (raters) agree in their assessment"
• Intra-rater reliability
  • same evaluator over time
  • relates to the stability of the scores obtained by a rater when they carry out the test on two separate occasions
• Test-retest reliability
  • describes the stability of scores obtained by a patient when they are evaluated on two separate occasions
  • "the extent to which the outcome measure produces consistent results over time"

You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability here.

• Internal consistency
  • "the extent to which features within the outcome measure produce consistent results"
• Measurement error
  • "the difference between what is measured and its 'true' value"

Validity

The degree to which the test measures what it claims to measure.

Types of validity:

• Face validity
  • judging on face value, does the content of the outcome measure, adequately reflect that which it is designed to measure?
  • for example, does an outcome measure about balance ask questions about this?
• Construct validity
  • the extent to which an outcome measure really represents and explores the construct (such as quality of life) that it intends to measure
• Criterion validity - consists of:
  • Concurrent validity
    • "the extent to which the scores/results are consistent with those produced by a known outcome measure with good validity"
  • Predictive validity
    • "the extent to which the outcome measure can accurately predict an outcome, result or relationship"
• Content validity
  • "the extent to which the content measures what it is intended to measure, has sufficient breadth / depth" and is free from bias
  • "the degree in which a test includes all the necessary items to represent the concept to be measured"

You can read more on validity here.

Responsiveness

Responsiveness is:

• The ability of an outcome measure to detect a true change in a patient's status over time
• Also known as sensitivity to change
• You can read more about responsiveness here

If you would like to read more, please see Psychometric Properties.

Applying Evidence-Based Practice when Deciding on an Outcome Measure

Step 1: Determining the patient's needs to formulate a clinical question.

• Deciding on an outcome measure:
  • consider what you want to measure (function, perception, experience?)
  • how do you want to measure it (self-reported or therapist-administered)?
• Examples of a clinical question: "What are the best outcome measures to determine function and well-being in a patient with ...?" OR "Is outcome measure (x) or outcome measure (y) a better outcome measure to determine function and well-being in a patient with...?"

Step 2: Locating the knowledge sources.

• Use a modified PICOT mnemonic and focus on the POT part
  • P - patient/population
  • O - outcome - specific outcome
  • T - type - type of outcome measure (e.g. patient-reported outcome)
• Decide on a database to use

Step 3: Appraising the quality of the outcome measure.

• For this step, please see Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals

Step 4: Discussing the options with the patient.

• Combine the discussion on which outcome measures to use with the discussion on the best treatment options
• The outcome you use should align with the needs of the condition, patient context, goals, values and preferences and patient environment
• Discuss cost implications, time taken to administer the outcome measure and frequency

Step 5: Include the outcome measure in the treatment plan (formulating the plan).

• The chosen outcome measure should fit the goal of the intervention

Step 6: Implement the treatment plan with the outcome measure.

• Apply the treatment plan and outcome measure

Step 7: Evaluate the effectiveness of the treatment plan, outcome measure and the evidence-based practice process.

• Does the outcome measure do what it needs to do?
• Is it as responsive as stated in the literature?
• Did you find a freely available outcome measure?
• Where can you make changes in the process?

Resources

• Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals
• COSMIN: A global initiative for the selection and development of outcome measure instruments.
• Orthopaedic Scores: a free outcome measure calculation service for certain orthopaedic outcome measures.
• Shirley Ryan, Ability Lab. Rehabilitation Measures Database: developed to help clinicians and researchers identify reliable and valid instruments used to assess patient outcomes during all phases of rehabilitation.
• The Neurology Section: recommendations for outcome measures used in neurological clinical practice, research and education.
• Outcome Measures