Using Evidence-Based Practice to Decide on an Outcome Measure: Difference between revisions
No edit summary |
(added citation to direct quotes) |
||
Line 30: | Line 30: | ||
* Test-retest reliability | * Test-retest reliability | ||
** describes the stability of scores obtained by a patient when they are evaluated on two separate occasions | ** describes the stability of scores obtained by a patient when they are evaluated on two separate occasions | ||
** "the extent to which the outcome measure produces consistent results over time" | ** "the extent to which the outcome measure produces consistent results over time"<ref name=":0">Allied Health Professions (AHP) Outcome Measures UK Working Group: [https://www.rcslt.org/wp-content/uploads/media/docs/selecting-outcome-measures.pdf Key questions to ask when selecting outcome measures: a checklist for allied health professionals]. November 2019</ref> | ||
You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability [[Psychometric Properties#Reliability|here]]. | You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability [[Psychometric Properties#Reliability|here]]. | ||
* Internal consistency | * Internal consistency | ||
** "the extent to which features within the outcome measure produce consistent results" | ** "the extent to which features within the outcome measure produce consistent results"<ref name=":0" /> | ||
* Measurement error | * Measurement error | ||
** "the difference between what is measured and its " | ** "the difference between what is measured and its 'true' value"<ref name=":0" /> | ||
=== '''Validity''' === | === '''Validity''' === |
Revision as of 14:12, 2 January 2024
Original Editor - Wanda van Niekerk based on the course by Benita Olivier
Top Contributors - Wanda van Niekerk and Jess Bell
Introduction[edit | edit source]
An outcome measure is a tool used to assess a patient's current status. It can provide a score, an interpretation of results and / or a risk categorisation of the patient. It can provide baseline data prior to an intervention, and the initial results may help determine and direct the course of a treatment intervention. The outcome measure can also be used during the course of treatment to assess any "change in the construct of interest over time, during and after rehabilitation."[1]
Outcome measures that are used in clinical practice include:
- Patient-reported outcome measures (PROM)
- Therapist-administered outcome measures
- Clinical tests
- Observed outcome measures
Psychometric Properties of Outcome Measures[edit | edit source]
Reliability[edit | edit source]
- Refers to the extent to which a measurement is consistent and free from error[2]
- Does the outcome measure produce consistent results?
- Will the results be the same when the test is done by different clinicians or by the same clinician at different times?
Types of reliability:
- Inter-rater reliability
- different evaluators, usually within the same time period
- describes the stability of the scores obtained when two different raters carry out the same test
- "the degree to which independent observers (raters) agree in their assessment"[3]
- Intra-rater reliability
- same evaluator over time
- relates to the stability of the scores obtained by a rater when they carry out the test on two separate occasions
- Test-retest reliability
- describes the stability of scores obtained by a patient when they are evaluated on two separate occasions
- "the extent to which the outcome measure produces consistent results over time"[4]
You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability here.
- Internal consistency
- "the extent to which features within the outcome measure produce consistent results"[4]
- Measurement error
- "the difference between what is measured and its 'true' value"[4]
Validity[edit | edit source]
The degree to which the test measures what it claims to measure.[5][6]
- Face validity
- judging on face value, does the test measure what it says it will measure?
- Construct validity
- the extent to which an outcome measures what it intends to measure
- Criterion validity - consists of:
- Concurrent validity
- the extent to which the scores/results are consistent with those produced by a known outcome measure with good validity
- Predictive validity
- the extent to which the outcome measure can accurately predict an outcome, result or relationship
- Concurrent validity
- Content validity
- the extent to which the content measures what it is intended to measure, and is free from bias
You can read more on validity here.
Responsiveness[9][edit | edit source]
- The ability of an outcome measure to detect a true change in a patient's status over time
- Also known as sensitivity to change
- You can read more about responsiveness here
If you would like to read more, please see Psychometric Properties.
Applying Evidence-Based Practice when Deciding on an Outcome Measure[edit | edit source]
Step 1: Determining the patient's needs to formulate a clinical question.
- Deciding on an outcome measure:
- consider what you want to measure (function, perception, experience?)
- how do you want to measure it (self-reported or therapist-administered)?
- Examples of a clinical question: "What are the best outcome measures to determine function and well-being in a patient with ...?" OR "Is outcome measure (x) or outcome measure (y) a better outcome measure to determine function and well-being in a patient with...?"
Step 2: Locating the knowledge sources.
- Use a modified PICOT mnemonic and focus on the POT part
- P - patient/population
- O - outcome - specific outcome
- T - type - type of outcome measure (e.g. patient-reported outcome)
- Decide on a database to use
Step 3: Appraising the quality of the outcome measure.
- For this step, please see Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals
Step 4: Discussing the options with the patient.
- Combine the discussion on which outcome measures to use with the discussion on the best treatment options
- The outcome you use should align with the needs of the condition, patient context, goals, values and preferences and patient environment
- Discuss cost implications, time taken to administer the outcome measure and frequency
Step 5: Include the outcome measure in the treatment plan (formulating the plan).
- The chosen outcome measure should fit the goal of the intervention
Step 6: Implement the treatment plan with the outcome measure.
- Apply the treatment plan and outcome measure
Step 7: Evaluate the effectiveness of the treatment plan, outcome measure and the evidence-based practice process.
- Does the outcome measure do what it needs to do?
- Is it as responsive as stated in the literature?
- Did you find a freely available outcome measure?
- Where can you make changes in the process?
Resources[edit | edit source]
- Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals
- COSMIN: A global initiative for the selection and development of outcome measure instruments.
- Orthopaedic Scores: a free outcome measure calculation service for certain orthopaedic outcome measures.
- Shirley Ryan, Ability Lab. Rehabilitation Measures Database: developed to help clinicians and researchers identify reliable and valid instruments used to assess patient outcomes during all phases of rehabilitation.
- The Neurology Section: recommendations for outcome measures used in neurological clinical practice, research and education.
- Outcome Measures
References[edit | edit source]
- ↑ Alreni AS, McRobert C, McLean SM. Utilisation of outcome measures in the management of non-specific neck pain: a national survey of current physiotherapy practice in the UK. Musculoskeletal Science and Practice. 2021 Apr 1;52:102347.
- ↑ Portney L., Watkins M. Chapter 4: Principles of Measurement, within Foundations of Clinical Research : Applications to Practice, 3rd Edition. F.A. Davis Company, Pennsylvania, United States 2015. ISBN10 0803646577.
- ↑ D’lima J, Taylor SE, Mitri E, Harding A, Lai J, Manias E. Assessment of inter-rater reliability of screening tools to identify patients at risk of medication-related problems across the emergency department continuum of care. Australasian Emergency Care. 2023 Nov 14.
- ↑ 4.0 4.1 4.2 Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals. November 2019
- ↑ Haff GG, Dumke C. Laboratory manual for exercise physiology. Human Kinetics; 2022 Dec 2.
- ↑ Dunn WW. Validity. InDeveloping norm-referenced standardized tests 2020 Feb 13 (pp. 149-168). Routledge.
- ↑ Erlinawati E, Muslimah M. Test Validity and Reliability in Learning Evaluation. Bulletin of Community Engagement. 2021 Jan 6;1(1):26-31.
- ↑ Chetwynd E. Critical analysis of reliability and validity in literature reviews. Journal of Human Lactation. 2022 Aug;38(3):392-6.
- ↑ Mokkink L, Terwee C, de Vet H. Key concepts in clinical epidemiology: Responsiveness, the longitudinal aspect of validity. Journal of clinical epidemiology. 2021 Dec 1;140:159-62.