Using Evidence-Based Practice to Decide on an Outcome Measure: Difference between revisions
No edit summary |
(fixed broken link) |
||
(9 intermediate revisions by 2 users not shown) | |||
Line 5: | Line 5: | ||
</div> | </div> | ||
== Introduction == | == Introduction == | ||
An [[Outcome Measures|outcome measure]] is a tool used to assess a patient's current status. It can provide a score, an interpretation of results and/or a risk categorisation of the patient. It can provide baseline data prior to an intervention and the initial results may help determine and direct the course of | An [[Outcome Measures|outcome measure]] is a tool used to assess a patient's current status. It can provide a score, an interpretation of results and / or a risk categorisation of the patient. It can provide baseline data prior to an intervention, and the initial results may help determine and direct the course of a treatment intervention. The outcome measure can also be used during the course of treatment to assess any "change in the construct of interest over time, during and after rehabilitation."<ref>Alreni AS, McRobert C, McLean SM. Utilisation of outcome measures in the management of non-specific neck pain: a national survey of current physiotherapy practice in the UK. Musculoskeletal Science and Practice. 2021 Apr 1;52:102347.</ref> | ||
Outcome measures that are used in clinical practice include: | Outcome measures that are used in clinical practice include: | ||
Line 30: | Line 30: | ||
* Test-retest reliability | * Test-retest reliability | ||
** describes the stability of scores obtained by a patient when they are evaluated on two separate occasions | ** describes the stability of scores obtained by a patient when they are evaluated on two separate occasions | ||
** "the extent to which the outcome measure produces consistent results over time" | ** "the extent to which the outcome measure produces consistent results over time"<ref name=":0">Allied Health Professions (AHP) Outcome Measures UK Working Group: [https://www.rcslt.org/wp-content/uploads/media/docs/selecting-outcome-measures.pdf Key questions to ask when selecting outcome measures: a checklist for allied health professionals]. November 2019</ref> | ||
You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability [[Psychometric Properties#Reliability|here]]. | You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability [[Psychometric Properties#Reliability|here]]. | ||
* Internal consistency | * Internal consistency | ||
** "the extent to which features within the outcome measure produce consistent results" | ** "the extent to which features within the outcome measure produce consistent results"<ref name=":0" /> | ||
* Measurement error | * Measurement error | ||
** "the difference between what is measured and its " | ** "the difference between what is measured and its 'true' value"<ref name=":0" /> | ||
=== '''Validity''' === | === '''Validity''' === | ||
Line 42: | Line 42: | ||
Types of validity<ref>Erlinawati E, Muslimah M. [https://attractivejournal.com/index.php/bce/article/view/96 Test Validity and Reliability in Learning Evaluation.] Bulletin of Community Engagement. 2021 Jan 6;1(1):26-31.</ref><ref>Chetwynd E. Critical analysis of reliability and validity in literature reviews. Journal of Human Lactation. 2022 Aug;38(3):392-6.</ref>: | Types of validity<ref>Erlinawati E, Muslimah M. [https://attractivejournal.com/index.php/bce/article/view/96 Test Validity and Reliability in Learning Evaluation.] Bulletin of Community Engagement. 2021 Jan 6;1(1):26-31.</ref><ref>Chetwynd E. Critical analysis of reliability and validity in literature reviews. Journal of Human Lactation. 2022 Aug;38(3):392-6.</ref>: | ||
* Face validity | * Face validity | ||
** | ** judging on face value, does the content of the outcome measure, adequately reflect that which it is designed to measure?<ref name=":0" /> | ||
** for example, does an outcome measure about balance ask questions about this? | |||
* Construct validity | * Construct validity | ||
** the extent to which an outcome | ** the extent to which an outcome measure really represents and explores the construct (such as quality of life) that it intends to measure<ref name=":0" /><ref name=":1">Souza A., Alexandre N., Guirardello E.. [https://pubmed.ncbi.nlm.nih.gov/28977189/ Psychometric properties in instruments evaluation of reliability and validity]. Epidemiologia e Serviços de Saúde. 2017 Jul;26:649-59.</ref> | ||
* Criterion validity | * Criterion validity - consists of: | ||
** Concurrent validity | ** Concurrent validity | ||
*** the extent to which the scores/results are consistent with those produced by a known outcome measure with good validity | *** "the extent to which the scores/results are consistent with those produced by a known outcome measure with good validity"<ref name=":0" /> | ||
** Predictive validity | ** Predictive validity | ||
*** the extent to which the outcome measure can accurately predict an outcome, result or relationship | *** "the extent to which the outcome measure can accurately predict an outcome, result or relationship"<ref name=":0" /> | ||
* Content validity | * Content validity | ||
** the extent to which the content measures what it is intended to measure, and is free from bias | ** "the extent to which the content measures what it is intended to measure, has sufficient breadth / depth" and is free from bias<ref name=":0" /> | ||
** "the degree in which a test includes all the necessary items to represent the concept to be measured"<ref name=":1" /> | |||
You can read more on validity [[Psychometric Properties#Validity|here]]. | You can read more on validity [[Psychometric Properties#Validity|here]]. | ||
{{#ev:youtube|Q7jYDVGmlro|300}}<ref>Dr J.5 Types of Construct Validity (Face, Content, Criterion, Convergent, Discriminant)- Research Methods Available from: https://www.youtube.com/watch?v=Q7jYDVGmlro [last accessed 02/01/2024]</ref> | |||
=== '''Responsiveness'''<ref>Mokkink L, Terwee C, de Vet H. Key concepts in clinical epidemiology: [https://linkinghub.elsevier.com/retrieve/pii/S0895-4356(21)00180-3 Responsiveness, the longitudinal aspect of validity.] Journal of clinical epidemiology. 2021 Dec 1;140:159-62.</ref> | === '''Responsiveness''' === | ||
Responsiveness is<ref>Mokkink L, Terwee C, de Vet H. Key concepts in clinical epidemiology: [https://linkinghub.elsevier.com/retrieve/pii/S0895-4356(21)00180-3 Responsiveness, the longitudinal aspect of validity.] Journal of clinical epidemiology. 2021 Dec 1;140:159-62.</ref>: | |||
* The ability of an outcome measure to detect a true change in a patient's status over time | * The ability of an outcome measure to detect a true change in a patient's status over time | ||
* Also known as ''sensitivity to change'' | * Also known as ''sensitivity to change'' | ||
Line 65: | Line 68: | ||
** consider what you want to measure (function, perception, experience?) | ** consider what you want to measure (function, perception, experience?) | ||
** how do you want to measure it (self-reported or therapist-administered)? | ** how do you want to measure it (self-reported or therapist-administered)? | ||
* Examples of a clinical question: "What are the best outcome measures to determine function and well-being in a patient with ...? OR Is outcome measure (x) or outcome measure (y) a better outcome measure to determine function and well-being in a patient with...? | * Examples of a clinical question: "What are the best outcome measures to determine function and well-being in a patient with ...?" OR "Is outcome measure (x) or outcome measure (y) a better outcome measure to determine function and well-being in a patient with...?" | ||
Step 2: '''Locating''' the knowledge sources. | Step 2: '''Locating''' the knowledge sources. | ||
* Use a modified PICOT mnemonic and focus on the POT part | * Use a modified PICOT mnemonic and focus on the POT part | ||
Line 73: | Line 76: | ||
* Decide on a database to use | * Decide on a database to use | ||
Step 3: '''Appraising''' the quality of the outcome measure. | Step 3: '''Appraising''' the quality of the outcome measure. | ||
* | * For this step, please see Allied Health Professions (AHP) Outcome Measures UK Working Group: [https://www.rcslt.org/wp-content/uploads/media/docs/selecting-outcome-measures.pdf Key questions to ask when selecting outcome measures: a checklist for allied health professionals] | ||
Step 4: '''Discussing''' the options with the patient. | Step 4: '''Discussing''' the options with the patient. | ||
* Combine the discussion on which outcome measures to use with the discussion on best treatment options | * Combine the discussion on which outcome measures to use with the discussion on the best treatment options | ||
* The outcome you use should align with the needs of the condition, patient context, goals, values and preferences and patient environment | * The outcome you use should align with the needs of the condition, patient context, goals, values and preferences and patient environment | ||
* Discuss cost implications, time taken to administer the outcome measure and frequency | * Discuss cost implications, time taken to administer the outcome measure and frequency | ||
Step 5: Include the outcome measure | Step 5: Include the outcome measure in the treatment plan ('''formulating''' the plan). | ||
* The chosen outcome measure should fit the goal of the intervention | * The chosen outcome measure should fit the goal of the intervention | ||
Step 6: '''Implement''' the treatment plan with the outcome measure. | Step 6: '''Implement''' the treatment plan with the outcome measure. | ||
* Apply the treatment plan and outcome measure | * Apply the treatment plan and outcome measure | ||
Step 7: '''Evaluate''' the effectiveness of treatment plan, outcome measure and the evidence-based practice process. | Step 7: '''Evaluate''' the effectiveness of the treatment plan, outcome measure and the evidence-based practice process. | ||
* Does the outcome measure do what it needs to do? | * Does the outcome measure do what it needs to do? | ||
* Is it as responsive as stated in the literature? | * Is it as responsive as stated in the literature? |
Latest revision as of 21:54, 9 January 2024
Original Editor - Wanda van Niekerk based on the course by Benita Olivier
Top Contributors - Wanda van Niekerk and Jess Bell
Introduction[edit | edit source]
An outcome measure is a tool used to assess a patient's current status. It can provide a score, an interpretation of results and / or a risk categorisation of the patient. It can provide baseline data prior to an intervention, and the initial results may help determine and direct the course of a treatment intervention. The outcome measure can also be used during the course of treatment to assess any "change in the construct of interest over time, during and after rehabilitation."[1]
Outcome measures that are used in clinical practice include:
- Patient-reported outcome measures (PROM)
- Therapist-administered outcome measures
- Clinical tests
- Observed outcome measures
Psychometric Properties of Outcome Measures[edit | edit source]
Reliability[edit | edit source]
- Refers to the extent to which a measurement is consistent and free from error[2]
- Does the outcome measure produce consistent results?
- Will the results be the same when the test is done by different clinicians or by the same clinician at different times?
Types of reliability:
- Inter-rater reliability
- different evaluators, usually within the same time period
- describes the stability of the scores obtained when two different raters carry out the same test
- "the degree to which independent observers (raters) agree in their assessment"[3]
- Intra-rater reliability
- same evaluator over time
- relates to the stability of the scores obtained by a rater when they carry out the test on two separate occasions
- Test-retest reliability
- describes the stability of scores obtained by a patient when they are evaluated on two separate occasions
- "the extent to which the outcome measure produces consistent results over time"[4]
You can read more on reliability, the different types of reliability, and quantitative and qualitative measures of reliability here.
- Internal consistency
- "the extent to which features within the outcome measure produce consistent results"[4]
- Measurement error
- "the difference between what is measured and its 'true' value"[4]
Validity[edit | edit source]
The degree to which the test measures what it claims to measure.[5][6]
- Face validity
- judging on face value, does the content of the outcome measure, adequately reflect that which it is designed to measure?[4]
- for example, does an outcome measure about balance ask questions about this?
- Construct validity
- Criterion validity - consists of:
- Content validity
You can read more on validity here.
Responsiveness[edit | edit source]
Responsiveness is[11]:
- The ability of an outcome measure to detect a true change in a patient's status over time
- Also known as sensitivity to change
- You can read more about responsiveness here
If you would like to read more, please see Psychometric Properties.
Applying Evidence-Based Practice when Deciding on an Outcome Measure[edit | edit source]
Step 1: Determining the patient's needs to formulate a clinical question.
- Deciding on an outcome measure:
- consider what you want to measure (function, perception, experience?)
- how do you want to measure it (self-reported or therapist-administered)?
- Examples of a clinical question: "What are the best outcome measures to determine function and well-being in a patient with ...?" OR "Is outcome measure (x) or outcome measure (y) a better outcome measure to determine function and well-being in a patient with...?"
Step 2: Locating the knowledge sources.
- Use a modified PICOT mnemonic and focus on the POT part
- P - patient/population
- O - outcome - specific outcome
- T - type - type of outcome measure (e.g. patient-reported outcome)
- Decide on a database to use
Step 3: Appraising the quality of the outcome measure.
- For this step, please see Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals
Step 4: Discussing the options with the patient.
- Combine the discussion on which outcome measures to use with the discussion on the best treatment options
- The outcome you use should align with the needs of the condition, patient context, goals, values and preferences and patient environment
- Discuss cost implications, time taken to administer the outcome measure and frequency
Step 5: Include the outcome measure in the treatment plan (formulating the plan).
- The chosen outcome measure should fit the goal of the intervention
Step 6: Implement the treatment plan with the outcome measure.
- Apply the treatment plan and outcome measure
Step 7: Evaluate the effectiveness of the treatment plan, outcome measure and the evidence-based practice process.
- Does the outcome measure do what it needs to do?
- Is it as responsive as stated in the literature?
- Did you find a freely available outcome measure?
- Where can you make changes in the process?
Resources[edit | edit source]
- Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals
- COSMIN: A global initiative for the selection and development of outcome measure instruments.
- Orthopaedic Scores: a free outcome measure calculation service for certain orthopaedic outcome measures.
- Shirley Ryan, Ability Lab. Rehabilitation Measures Database: developed to help clinicians and researchers identify reliable and valid instruments used to assess patient outcomes during all phases of rehabilitation.
- The Neurology Section: recommendations for outcome measures used in neurological clinical practice, research and education.
- Outcome Measures
References[edit | edit source]
- ↑ Alreni AS, McRobert C, McLean SM. Utilisation of outcome measures in the management of non-specific neck pain: a national survey of current physiotherapy practice in the UK. Musculoskeletal Science and Practice. 2021 Apr 1;52:102347.
- ↑ Portney L., Watkins M. Chapter 4: Principles of Measurement, within Foundations of Clinical Research : Applications to Practice, 3rd Edition. F.A. Davis Company, Pennsylvania, United States 2015. ISBN10 0803646577.
- ↑ D’lima J, Taylor SE, Mitri E, Harding A, Lai J, Manias E. Assessment of inter-rater reliability of screening tools to identify patients at risk of medication-related problems across the emergency department continuum of care. Australasian Emergency Care. 2023 Nov 14.
- ↑ 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 Allied Health Professions (AHP) Outcome Measures UK Working Group: Key questions to ask when selecting outcome measures: a checklist for allied health professionals. November 2019
- ↑ Haff GG, Dumke C. Laboratory manual for exercise physiology. Human Kinetics; 2022 Dec 2.
- ↑ Dunn WW. Validity. InDeveloping norm-referenced standardized tests 2020 Feb 13 (pp. 149-168). Routledge.
- ↑ Erlinawati E, Muslimah M. Test Validity and Reliability in Learning Evaluation. Bulletin of Community Engagement. 2021 Jan 6;1(1):26-31.
- ↑ Chetwynd E. Critical analysis of reliability and validity in literature reviews. Journal of Human Lactation. 2022 Aug;38(3):392-6.
- ↑ 9.0 9.1 Souza A., Alexandre N., Guirardello E.. Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia e Serviços de Saúde. 2017 Jul;26:649-59.
- ↑ Dr J.5 Types of Construct Validity (Face, Content, Criterion, Convergent, Discriminant)- Research Methods Available from: https://www.youtube.com/watch?v=Q7jYDVGmlro [last accessed 02/01/2024]
- ↑ Mokkink L, Terwee C, de Vet H. Key concepts in clinical epidemiology: Responsiveness, the longitudinal aspect of validity. Journal of clinical epidemiology. 2021 Dec 1;140:159-62.