What is a systematic review?
The Community Preventive Services Task Force (CPSTF) defines a systematic review as a review of scientific studies on a specific topic. It uses a formal process to:
- Identify all relevant studies
- Assess their quality
- Summarize the evidence
The methods used in a systematic review must be reproducible and transparent.
References
- Community Preventive Services Task Force (CPSTF) https://www.thecommunityguide.org/about/our-methodology
- Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., The PRISMA Group. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine 6(7) e1000097
-
Are there different types of reviews?
Yes, there are multiple types of reviews that follow a specific methodology.
Type of Review
Definition
Critical review
Aims to demonstrate that the writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include a degree of analysis and conceptual innovation. Typically results in hypothesis or model
Literature review
Generic term: published materials that provide examination of recent or current literature. Can cover a wide range of subjects at various levels of completeness and comprehensiveness. May include research findings
Mapping review/ systematic map
Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature
Meta‐analysis
Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results
Mixed studies review/mixed methods review
Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies
Overview
Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics
Qualitative systematic review/qualitative evidence synthesis
Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies
Rapid review
Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research
Scoping review
Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research)
State‐of‐the‐art review
Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research
Systematic review
Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review
Systematic search and review
Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’
Systematized review
Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment
Umbrella review
Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results
Reproduced and adapted from: Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x. Review. PubMed PMID: 19490148.
-
What is a knowledge synthesis?
A knowledge synthesis is the over-arching term used for systematic and comprehensive research projects such as systematic reviews, meta-analyses, and scoping reviews.
-
Are there standards or methodologies to follow when completing a systematic review?
Yes, there are several organizations that have published standards and methdologies for systetamtic reviews and scoping review. The standards can vary based on the type of systematic review you are completing. For example, there are more specific standards for systematic reviews of randomised trials, observational studies, practice guidelines. etc. Some generic standards include:
-
Is there software available through UPEI to help conduct my systematic review?
There is a variety of software that is available to assist systematic reviews with the screening process.
The Library currently has access to the online, subscription-based software Covidence. If you would like access to a Covidence account, please contact publications@upei.ca.
There are also several free options available:
- Rayyan: A free online designed to manage the stages of systematic reviews and other knowledge synthesis projects.
- Abstrackr: free online
- https://sysrev.com/: an online system for Literature Reviews, data Extraction, and Systematic Review
- Unlimited public projects
Unlimited project reviewers
Project management
Free lifetime storage for public projects
- Unlimited public projects
-
Can I get access to Covidence for my project?
The Library currently has access to Covidence.
If you are interested in getting Covidence for a knowledge synthesis project please contact publications@upei.ca.
-
Why work with a librarian?
The Systematic Review Team: Contributions of the Health Sciences Librarian Dudden & Protzko (2011)
Engaging Medical Librarians to Improve the Quality of Review Articles Rethlefsen, Murad, & Livingston (2014)
Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews Rethlefsen, Farrell, Osterhaus Trzasko, & Brigham (2015)
Adapted from: Dalhousie Libraries Systematic Reviews: A How-To
-
What is a systematic review protocol?
According to PRISMA:
"A systematic review protocol describes the rationale, hypothesis, and planned methods of the review. It should be prepared before a review is started and used as a guide to carry out the review."
Protocols are important because they help prevent duplication in knowledge syntheses and allow for transparency in the research process. Specifically, the methods, reasoning for inclusion/exclusion criteria and other important decisions are explained in the protocol. Additionally, the protocol provides a good map for the systematic review because it poses important questions about the research topic and is then available for the team to refer back to.
-
How long does it take to complete a systematic review?
A proper systematic review requires a significant amount of time to complete (many months). Several research studies have been completed to help determine this, but there is no definitive answer because each project is different.
Resources:
- Kate Nyhan, a librarian at Yale University, created a spreadsheet that provides time estimates for workflows.
- Cochrane timeline for completing a review
- PredicTER - A tool for predicting how long systematic reviews take.
- Haddaway, N.R. and Westgate, M.J. (2019), Predicting the time needed for environmental systematic reviews and systematic maps. Conservation Biology, 33: 434-443. https://doi.org/10.1111/cobi.13231
- Borah R, Brown AW, Capers PL, et al. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 2017;7:e012545. doi: 10.1136/bmjopen-2016-012545
-
How many articles should be included in my systematic review?
Unfortunately, due to the nature of systematic reviews and knowledge syntheses, there is no number or range of articles that "should be" in a review. The goal is to find all the research that exists on your chosen topic and therefore, the number of articles will depend on your question and your previously chosen inclusion and exclusion criteria.
-
Do you recommend any training resources on systematic reviews?
If you have never completed any form of knowledge synthesis before we recommend that you take the free online Coursera course, created by John Hopkin's University, Introduction to Systematic Reviews and Meta-Analysis. Even if you are planning on completing a different type of review, this course gives a very good overview and understanding of what is expected out of a research project that is to be completed systematically.
Check out our other books and resources.
-
What is grey literature and how do I search for it?
Grey literature describes information or resources that are not published as a journal article or book.
Examples include:
- conference proceedings
- government or private companies documentation/handbooks/guidelines etc.
- these or dissertations
- preprints or other unpublished studies
Adapted from: https://guides.library.queensu.ca/knowledge-syntheses/grey-literature
Your librarian can help you develop a plan for searching effectively for grey literature. Here are a few places to search for grey literature depending on your topic:
- Targeted website searching
- Dissertation and Theses databases
- Government Documents/Resources:
- Canadian Electronic Library
- National Criminal Justice Reference Service - U.S. Dept of Justice
- TRIP -Turning Research Into Practice
An article on how to systematically search for grey literature:
Godin, K., Stapleton, J., Kirkpatrick, S.I. et al. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev 4, 138 (2015). https://doi.org/10.1186/s13643-015-0125-0
-
How many people are on a systematic review team?
The number of people needed to conduct a proper systematic review is guided by the standards you are following to conduct for your review. For example, the Cochrane Handbook for Systematic Reviews of Interventions requires that "reviews should be undertaken by more than one person. The project team should consider the need for clinical and methodological expertise for the review, as well as the perspectives of stakeholders. Cochrane author teams are encouraged to seek and incorporate the views of users, including consumers, clinicians and those from varying regions and settings to develop protocols and reviews. Author teams for reviews relevant to particular settings (e.g. neglected tropical diseases) should involve contributors experienced in those settings."
The Institute of Medicine mandates that "the review team should be capable of defining the clinical question and performing the technical aspects of the review. It should be multidisciplinary, with experts in SR methodology, including risk of bias, study design, and data analysis; librarians or information specialists trained in searching bibliographic databases for SRs; and clinical content experts. Other relevant users and stakeholders should be included as feasible (CRD, 2009; Higgins and Green, 2008; Slutsky et al., 2010). A single member of the review team can have multiple areas of expertise (e.g., SR methodology and quantitative analysis). The size of the team will depend on the number and complexity of the question(s) being addressed. The number of individuals with a particular expertise needs to be carefully balanced so that one group of experts is not overly influential."
The JBI Manual of Evidence Synthesis indicates that "to maintain the required rigorous standards and alleviate risk of bias in the review process, a JBI review requires a minimum of two reviewers to conduct a systematic review to adequately complete the work to the standards."
References
Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-24-01
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5 (updated August 2024). Cochrane, 2024. Available from www.training.cochrane.org/handbook.
Institute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. https://doi.org/10.17226/13059.
-
What standards are available for conducting a rapid review?
A rapid review is "a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner" (Tricco et al., 2015). There are several resources available to guide you in conducting a rapid review, such as:
National Collaborating Centre of Methods and Tools (NCCMT) Rapid Review Guidebook
Lefebvre C, Glanville J, Briscoe S, Featherstone R, Littlewood A, Metzendorf M-I, Noel-Storr A, Paynter R, Rader T, Thomas J, Wieland LS. Chapter 4: Searching for and selecting studies [last updated September 2024]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane, 2024. Available from www.training.cochrane.org/handbook.
Tricco, AC, Antony, J, Zarin, W et al. (2015). A scoping review of rapid review methods. BMC Medicine, 13, Article number: 224. https://doi.org/10.1186/s12916-015-0465-6
Tricco AC, Langlois EV, Straus SE, editors.(2017). Rapid reviews to strengthen health policy and systems: a practical guide. Geneva: World Health Organization; . Licence: CC BY-NC-SA 3.0 IGO.
Garritty C, Gartlehner G, Nussbaumer-Streit B, King VJ, Hamel C, Kamel C, L A, Stevens A. Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol. 2020 Oct 14:S0895-4356(20)31146-X. doi: 10.1016/j.jclinepi.2020.10.007.
Canadian Agency for Drugs and Technologies in Health's (CADTH) Rapid Review Process 2.0
PRISMA-RR: an extension to PRISMA for rapid reviews is currently under development: https://www.equator-network.org/library/reporting-guidelines-under-deve…