Systematic Reviews
Original Editor - Stefano Berrone
Top Contributors - Stefano Berrone and Angeliki Chorti
Introduction[edit | edit source]
A Systematic Review (SR) is a tool of quantitative research methodology that, through a systematic search and analysis of the literature, aims to answer to a research question. In fact, after identyfing all the literature relevant for contents and study design, it collects and analyses data from the studies included in the review. [1]
A SR can include a meta-analysis, that allows to synthesize the quantitative data from the studies included into a single statistical analysis. The presence or not of a metanalysis depends mostly on criteria such as the homogeneity and quantity of the studies.
Systematic review methodology[edit | edit source]
The following information reports the main steps of the process of carrying out a systematic review. [1]
Research question formulation[edit | edit source]
As for every work of research, the research question is the crucial step that guides the choice of the paradigm and of the methodology of research. The research question can come from a dilemma inherent to the clinical practice, a gap in the practical or theorethical knowledge, or a mix of these elements.[3] The systematic review is an approach that can be included in quantitative methodologies, because it shares the main features of this methodology, such as hypothesis testing, numerical data collection and analysis and the role of the researcher detached from the research.
Search strategy and location of studies[edit | edit source]
The search strategy is the way in which the researcher runs the elctronic search to select the articles to include in the review. There is not a list of search strategies among which we can choose one. In fact, a systematic review is supposed to describe in details the search strategy that was used, as it influences the quality of the study. The most important elements of a search strategy are:
- balancing sensitivity and specifity. Where sensitivity means finding a high proportion of relevant studies, and specificity means finding a low proportion of irrelevant studies. [1]
- selecting databases (the presence of a thesaurus improves the coverage) [4]
- identifying MeSH terms to search, related to the research question or research framework (for example PICO).
- using database-appropriate syntax (for example parentheses, Boolean operators, field codes, variations in search terms)
Moreover, a search can also include other actions rather than electronic database search, such as reference lists checking and direct searching on key journals.
Inclusion and exclusion criteria[edit | edit source]
Inclusion and exclusion criteria interest the contents of the studies selected and their design. For the contents the PICO (Population, Intervention, Comparison, Outcomes) framework can be used. It summarizes the main components to consider when selecting studies. On the base of the objectives of the research, the researcher puts some inclusion and exclusion criteria for each of these components. The PICO framework is a useful tool for the inclusion/exclusion startegy, but it is not mandatory, as there are other strategies that are effective (for example the SPIDER framework for qualitative evidence synthesis). It is up to the researcher to adopt the most appropriate strategy for the research question. About the study design, the researcher decides a priori what types of studies to include and exclude, for example RCT, quasi experimental trials, cohort studies or combinations of them. More criteria can be added, for example language restrictions, free-acces studies, or published versus unpublished studies.[1]
After having obtained a list of titles and abstracts, the studies that meet inclusion criteria are then reviewed in full. To ensure inter-rater reliability the review process has to be done by two or more researchers. All reviewed studies should be recorded with thier own reasons dor inclusion and exclusion. [1]
Study selection[edit | edit source]
Study selection starts with the review of the abstracts that have been retrieved by the search strategy. The full-text of the studies that seem to meet inclusion criteria are obtained and reviewed. This process is preferrably done by at least two reviewers to establish inter-rater reliability.
A useful strategy is for authors to keep a log of all reviewed studies with reasons for inclusion or exclusion. In case more information about the study is required study authors may be contacted for information needed for data pooling (e.g., means, standard deviations). Translations may also be required.
Data extraction[edit | edit source]
The data extraction follows the identification of eligible studies. The aim of this step is to gain information about the included studies, and, in the case of a metanalysis, to obtain the quantitative data to carry out the statistical analysis. The information to be extracted from the studies are choosen by the researcher. The goal is to provide all the information necessary to generalise the results.[5]
Quality assessment[edit | edit source]
A systematic reviews evaluates the quality of the studies included through a risk of bias assessment. Different study designs have different risk of bias assessment tools. Here some examples of tools that can be used for three study designs:
- RCT: Cochrane Risk of Bias tool (ROB)
- case control studies: JBI Checklist for Case-Control Studies
- cohort studies: CASP- Cohort Studies
Data analysis and synthesis[edit | edit source]
For meta-analyses, statistical programmes are commonly used to calculate effects sizes.
An example of a statistical programme for data review analyses is the Review Manager (RevMan) endorsed by the Cochrane Collaboration.
Reporting of effect sizes is combined with a 95 % confidence interval (CI) range, and presented in both quantitative format and graphical representation (e.g., forest plots).
Forest plots visually depict each trial as a horizontal diamond shape with the middle representing the effect size (e.g., SMD) and the end points representing both ends of the CI. These diamonds are presented on a graph with a centre line representing the zero mark. Often the left side of the graph (< zero) represents the side favoring treatment, while the right side (> zero) represents the side favouring the control condition. At the bottom of the graph is a summary effect size or diamond representing all of the individual studies pooled together. Ideally, we would like to see this entire diamond (effect size and both anchors of the CI) falling below zero, indicating that the intervention is favoured over the control. In addition, most programs also calculate a heterogeneity value to indicate whether the individual studies are similar enough to compare. In this case, it is preferable to have non-significant findings for heterogeneity. It is still possible to pool studies when significant heterogeneity exists, although these results should be interpreted with caution or reasons for the heterogeneity should be explored.
Interpretation of the synthesis of study results is the last step in the writing process. This process usually involves summarising the findings, and providing clinical and future research recommendations.
Dissemmination of findings[edit | edit source]
Some SRs (e.g. from the Cochrane Collaboration) are automatically published full text in the online database. Abbreviated versions of the review may also be published in relevant academic journals, as long as they are clearly indicated as such. In some cases, plain language summaries are also prepared for families and patients.
Reporting[edit | edit source]
Reporting of SRs and meta-analyses should be conducted in a clear, accurate and transparent manner. [6] The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement and its update in 2020 provide guidance and a checklist of items that details reporting recommendations for SRs and meta-analyses of health, social or educational interventions interventions, irrespective of the design of the included studies. [6]
Review updates[edit | edit source]
Reviews should be regularly updated to ensure they are always up-to-date and relevant.
Systematic Reviews in Healthcare[edit | edit source]
SRs have a relevant role in Evidence Based Practice (EBP) because they are on the top of the pyramid in the hierarchy of evidence. As a result, conclusions are often used to guide assessment and treatment choices in routine clinical practice. SRs are also a key component of the content of Clinical Practice Guidelines (CPGs), which provide guidance and recommendations at a higher level of decision-making.
As the results and conclusion of systematic reviews in healthcare are often not net and exclusive mostly due to the heterogeneity of studies, as a user of research it is important to analyse and interpret the all the information given by the systematic review. In fact, we can gain useful information for evidence based practice from the whole process of research of a systematic review, for example from the research question that identifies a gap in the knowledge and from the presence or not of studies on a topic and from their quality.
Resources[edit | edit source]
Cochrane - an international network providing and supporting best evidence for healthcare decisions.
COSMIN - an international multidisciplinary initiative aiming to improve the selection of outcome measurement instruments both in research and in clinical practice by developing methodology and practical tools for selecting the most suitable outcome measurement instrument.
PEDro - a free Physiotherapy Evidence Database including among others systematic reviews and meta-analyses on physiotherapy practice.
PRISMA - Transparent Reporting of Systematic Reviews and Meta-analyses
References[edit | edit source]
- ↑ 1.0 1.1 1.2 1.3 1.4 Uman L. Systematic reviews and meta-analyses. J Can Acad Child Adolesc Psychiatry. 2011 Feb;20(1):57-9.
- ↑ Research Shorts. Conducting a Systematic Literature Review. Available from: https://www.youtube.com/watch?v=WUErib-fXV0 [Accessed: 12/03/2023]
- ↑ Fandino W. Formulating a good research question: Pearls and pitfalls. Indian J Anaesth. 2019 Aug;63(8):611-616.
- ↑ Bramer W., de Jonge G., Rethlefsen M., Mast F., Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. J Med Libr Assoc. 2018 Oct;106(4):531-541.
- ↑ Taylor K., Mahtani K., Aronson J. Summarising good practice guidelines for data extraction for systematic reviews and meta-analysis BMJ Evidence-Based Medicine 2021;26:88-90.
- ↑ 6.0 6.1 Page M, Bossuyt P, Mulrow C, Tetzlaff J, Brennan S, Grimshaw J, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
