Appraise the evidence

Original Editor - Your name will be added here if you created the original content for this page.

Top Contributors - Admin, Rachael Lowe, 127.0.0.1, Alex Benham, Grace Barla, Angeliki Chorti, Tony Lowe, Evan Thomas, George Prudden and WikiSysop  

The Appraisal

After we have searched for the evidence, we need to decide if it is both valid and important before deciding if we can apply the evidence to our individual patients. The order in which we consider validity and importance depends on individual preference.

The third step of evidence based practice, critical appraisal is the systematic evaluation of clinical research papers in order to establish:

1. Does this study address a clearly focused question?
2. Did the study use valid methods to address this question?
3. Are the valid results of this study important?
4. Are these valid, important results applicable to my patient or population?

If the answer to any of these questions is “no”, you can save yourself the trouble of reading the rest of it.

The following questions are appropriate to appraise the literature:

1. Are the results of the study valid?
2. What were the results?
3. Will the results help me in caring for my patients?

Are the results of the study valid?

1. Hierarchy of evidence
2. Internal and external validity
3. Randomisation and baseline homogenity of groups - was the assignment of patients to treatments randomised? Were the groups similar at the start of the trial?
4. Concealment of allocation to groups - were patients analysed in the groups to which they were randomised?
5. Blinding - were patients, health workers, and study personnel "blind" to treatment?
6. Completeness of follow up (Intention to treat principle) - was follow up complete? Were all patients who entered the trial properly accounted for and attributed at its conclusion?
7. Equivalent experience of groups apart from treatemnt of interest - aside from the experimental intervention, were the groups treated equally?
   

Hierarchy of evidence

When evaluating evidence for effectiveness of an intervention, clinicians often find it helpful to use a system to determine the level of evidence and/or grade of recommendation for a particular study.

Internal and external validity

In addition to identifying the level of evidence on the hierarchy, therapists must also consider critically appraising the study’s overall quality and the study’s internal and external validity.  Internal validity relates to elements of research design intended to exert control over extraneous variables that could potentially impact the outcomes of the study, including interactions between patient assignment, competing interventions, history, maturation, and instrumentation.  External validity refers to the generalizability of the study’s results to actual clinical practice.

Randomisation and baseline homogenity of groups 

Randomization should theoretically ensure that each group of subjects is similar at baseline so that no extraneous variables (such as known and unknown prognostic factors) compete with the intervention to explain observed outcomes. Extraneous variables that could potentially affect outcomes in studies of treatment effectiveness include patient, age, race, gender, symptom duration, condition severity, comorbidities, intellectual status, motivation, and treatment expectations. Although randomization should ideally produce observed homogenous groups at baseline, there is always a chance, particularly with small samples, that groups may be dissimilar in important known and unknown prognostic factors, which may affect group homogeneity. For this reason a reader performing a critical appraisal must independently judge the extent to which groups are similar in key prognostic factors.

Concealment of allocation to groups 

Blinding 

Completeness of follow up (Intention to treat principle)

Equivalent experience of groups apart from treatemnt of interest

What were the results?

• How large was the treatment effect?
• How precise was the estimate of the treatment effect?
  

Will the results help me in caring for my patients?

• Can the results be applied to my patient care?
• Were all clinically important outcomes considered?
• Are the likely treatment benefits worth the potential harms and costs?
  
  

Critical Appraisal Worksheets

These critical appraisal worksheets from the centre for evidence based medicine are very useful:

• Systematic Review Critical Appraisal Sheet
• Diagnosis Critical Appraisal Sheet
• Prognosis Critical Appraisal Sheet
• Therapy / RCT Critical Appraisal Sheet

Resources

The Pedro Tutorial is designed to help readers of clinical trials differentiate those trials which are likely to be valid from those that might not be. It also looks briefly at how therapists might use the findings of properly performed studies to make clinical decisions. The PEDro scale is a valid measure of the methodological quality of clinical trials. DeMorton (2009) suggests it is valid to sum PEDro scale item scores to obtain a total score that can be treated as interval level measurement and subjected to parametric statistical analysis.

Understand Levels and Grades of evidence with our related page

Understand simple statistics with our Test Diagnostics page

Recent Related Research (from Pubmed)

References

References will automatically be added here, see adding references tutorial.