How to approach a Systematic Review
When it comes to reviews, it is important to consider the different options available and choose the one that will work best for you.
Consider factors such as the amount of time you have and the type of research question you are trying to answer. By doing so, you can ensure that your review is both effective and efficient.
If you are in doubt about the kind of review that would be most appropriate for your specific goals and constraints, Cornell University Library has developed a helpful Review Methodology Decision Tree that can guide you towards the most suitable option.
The chart above demonstrates the various steps involved in conducting a systematic review in the field of Education.
Plan your strategy
- search strategy (where, keywords, subject headings, references lists) and chosen databases)
- identify the best sources and methods for searching
Search string example:
Search string: ((student-centered learning substring) AND (educational technology substring) AND (literature review substring)) Student-centered learning substring: “student-centered” OR “learner-centered” Educational Technology substring: “educational technology” OR “EdTech” OR “e-learning” OR “instructional technology” OR “technology-enhanced learning” OR “computer-based instruction” OR “computer managed instruction” OR “computer-based training” OR “computer-assisted instruction” OR “computer-aided instruction” OR “internet-based training” OR “web-based training” OR “online education” OR “digital education” OR “computer-mediated communication” OR “cyber-learning” OR “virtual education” OR “networked learning” OR “virtual learning environments” OR “ubiquitous learning”Literature review substring: “review” OR “systematic review” OR “scoping review” OR “rapid review” OR “umbrella” OR “state-of-art” OR “state of the art” OR “meta-analysis” OR “meta analytic study” OR “meta-synthesis” OR “overview”
Adapted from “How are we doing with student-centered learning facilitated by educational technologies? A systematic review of literature reviews,” by B. Shehata, A. Tlili, R. Huang, M. A. Adarkwah, M. Liu, & T. Chang, 2023, Education and Information Technologies. Advance online publication, p.9 (https://doi.org/10.1007/s10639-023-12112-w).
What sources to include?
- Google Scholar, specialist websites, bibliographic databases (both subject-specific (e.g. ERIC) and generic), hand-searching of specialist journals or books, contacts with experts, and citation checking
- consider any other relevant research sources, including those not published yet or grey literature
A protocol is like a blueprint for conducting a systematic review. It is a comprehensive plan that outlines the purpose and methodology of your review, including how you will search for literature and analyse the information you find. To keep things transparent, it is best to create a protocol and share it publicly before starting your review.
Taking the time to define your protocol can help make the systematic review process smoother for you. It also gives readers a chance to see how your review plan differs from the final results, which can help identify any potential biases.
For more information view the chapter “Writing a protocol” by G. M. Cherry & Boland in M. G. Cherry, A. Boland, & R. Dickson, Doing a systematic review: A student’s guide (3rd ed., p. 66) 2023. SAGE Publications. https://app.talis.com/textbooks/9781529615272
Conducting testing searches will help you to:
- define your key concepts
- develop an initial list of search terms
- define the potential number of results
- check if the review has already been done or is registered to be done
- confirm that there is enough evidence available to answer your question
When searching for reviews, try using a variety of relevant keywords and look for fields and limits
specific to the type of review you need. Remember to define your search criteria.
When conducting a research study review, reviewers face the important task of selecting which studies to include in their analysis. To ensure fairness and transparency, they should establish clear and consistent rules for study selection, referred to as selection criteria or inclusion/exclusion criteria. These criteria serve as a helpful guide to limit the scope of the review and ensure that the studies under consideration meet the necessary standards. Remember, all reviews - whether systematic or not - aim to provide constructive feedback to help improve research practices and outcomes (Newman & Gough, 2020).
The PRISMA Flow Diagram is a valuable aid that assists in documenting the different steps involved in the process of literature search.
It provides a clear visual representation of the researcher's progress from the databases searched with specific terms to the ones that will be included in the research project.
With the help of a PRISMA diagram, the reader can quickly and easily determine:
- the number of studies reviewed,
- the number of studies included,
- the exclusion criteria used.
It's worth noting that PRISMA can be adapted to meet the research needs of different individuals and groups.
From: PRISMA Flow Diagram by PRISMA Statement, 2020 (http://www.prisma statement.org/PRISMAStatement/FlowDiagram). Copyright 2024 by PRISMA.
Evaluate the data
There are many critical appraisal tools that are designed to be used when reading and evaluating research and clinical studies. Example: https://jbi.global/critical-appraisal-tools
Extract the data
You should gather data that is relevant to your systematic review question.
Start with watching the video below on how to export references from EndNote.
https://www.youtube.com/watch?v=iXgtrQxUv9A
You need to develop both evidence tables and summary tables for a detailed and high-level overview of your review findings. Proper planning is required for data extraction.
Synthesise the data
The goal is to synthesise the studies to answer your research question. You must organise the collected data in a way that will help you to combine the results and draw conclusions. This can be achieved through quantitative methods such as statistical analysis, like meta-analysis, or qualitative methods.
Write your paper
Summarise and present the findings of your systematic review to answer your research question.
You can find many examples of systematic reviews in Education on https://idesr.org/
Resources:
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. M. K., Hammerstrøm, K., & Sathe, N. (2016). Searching for studies: A guide to information retrieval for Campbell. Campbell Systematic Reviews, 13(1), 1-73. https://doi.org/10.4073/cmg.2016.1
Newman, M. & Gough, D. (2020). Systematic reviews in educational research: Methodology, perspectives and application. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond & K. Buntins (Eds.), Systematic reviews in educational research: Methodology, perspectives and application (pp. 3-22). Springer. https://doi.org/10.1007/978-3-658-27602-7
Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., & Buntins, K. (2020). Systematic reviews in educational research: Methodology, perspectives and application. Springer. https://doi.org/10.1007/978-3-658-27602-7
Preparing Literature Reviews-2nd Ed by Call Number: 808.066 PANISBN: 9781884585562Publication Date: 2004
Systematic Review Library Guide
A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias in order to produce more reliable findings that can be used to inform decision making.
Many systematic reviews contain meta-analyses. Meta-analysis is the statistical combination of results from two or more separate studies. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review. They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.
(Cochrane handbook for systematic reviews of interventions)
Meta-analysis: Chapter 10: Analysing data and undertaking meta-analyses
Handbooks, manuals, guidelines
Conducting research literature reviews by
The Third Edition of Conducting Literature Reviews: From the Internet to Paper provides readers with an accessible but in-depth look at how to synthesize research literature. Bestselling author Arlene Fink shows researchers how to justify the need for and significance of research, and explain a studyâe(tm)s findings. Key Updates to the Third Edition Presents nearly a hundred new online examples and references from the social, behavioral, and health sciences Offers a revised and updated list of online articles databases Includes case studies in the use of major online databases Expands the exercises at the end of the chapter to include more online searching Clarifies some of the basic concepts of research that are essential in making judgments about the quality of research methods Incorporates more examples and clearer explanations of the differences between research design validity (internal and external) and measurement validity Introduces the major formal systems for evaluating the literatureâe(tm)s methodological quality Provides more qualitative research examples and information on how to evaluate their quality Illustrates in greater detail how to write up reviews and how others have done it Contains nearly 100 Power Point Slides linked to each of the five chapters available at www.sagepub.com/fink3einstr qualified instructors who adopt the book The text also discusses the use of Boolean operators for simple and advanced searches, tells readers how to use bibliographic software to organize literature reviews and search "The Virtual File Cabinet.âe Intended Audience This text is for graduate students in the social science, health, educational, and business fields. It is also appropriate for anyone who design and manage public and private agencies, conduct research studies, and/or prepare strategic plans and grant proposals.
Call Number: 001.42 FINISBN: 9781412971898Publication Date: 2009
Doing a systematic review : a student's guide by
Completing a systematic review and unsure where to start or what path to take? Set out on your journey confidently with this practical guide written by a team of experienced advisors. With a friendly, accessible style, the book covers every step of the systematic review process, from planning to dissemination. Whether you want to blaze a direct path from A to B or pause to reflect on the available options, this book will give you the skills to find your own way. This book will help you to: • Work with qualitative, quantitative and mixed methods data, and critically assess any type of research • Understand the how-to of systematic reviews in different contexts, with examples and case studies from a range of disciplines • Learn from students who have been in your shoes with FAQs taken from actual supervision meetings, supporting you to approach the most common challenges.
ISBN: 9781529615265Publication Date: 2023
The literature review : Six steps to success by
The essential guide--updated, expanded, and easier to use than ever. Creating a successful literature review can be a daunting task, which is why so many researchers have relied on previous editions of this book to make the process more manageable. Using the six-step model, you'll work seamlessly to narrow your research topic, focus your literature search, negotiate the myriad of books, periodicals, and reports about your topic--and, of course, write the review. Updated, expanded, and reorganized to improve ease of use, the fourth edition of this bestselling handbook includes: New and improved graphics ideal for visualizing the process More explanations and tips, especially for writing in the early stages An expanded range of learning tools Additional reflection sections to direct metacognitive activities Four new reference supplements This pioneering book has provided countless graduate students and researchers with a road map to success. Its model takes you through the logical progression needed for producing a quality literature review--while taking the mystery out of the process.
ISBN: 9781071858974Publication Date: 2022
Systematic approaches to a successful literature review by
Reviewing the literature is an essential part of every research project. This book takes you step-by-step through the process of approaching your literature review systematically, applying systematic principles to a wide range of literature review types. Through numerous examples, case studies and exercises, the book covers often neglected areas of literature review such as concept analysis, scoping and mapping. The book includes practical tools for supporting the various stages of the review process, including; - managing your literature review - searching the literature - assessing the quality of the literature - synthesising qualitative and/or quantitative data - writing up and presenting data Systematic Approaches to a Successful Literature Review is essential reading for any student or researcher looking to approach their literature review in a systematic way.
Call Number: 001.42 BOOISBN: 9780857021342Publication Date: 2012
Writing literature reviews by
This useful guide educates students in the preparation of literature reviews for term projects, theses, and dissertations. The authors provide numerous examples from published reviews that illustrate the guidelines discussed throughout the book. New to the seventh edition: Each chapter breaks down the larger holistic review of literature exercise into a series of smaller, manageable steps Practical instructions for navigating today's digital libraries Comprehensive discussions about digital tools, including bibliographic and plagiarism detection software Chapter activities that reflect the book's updated content New model literature reviews Online resources designed to help instructors plan and teach their courses (www.routledge.com/9780415315746).
Call Number: 808.066 GALISBN: 9781351858922Publication Date: 2017
- Cochrane handbook for systematic reviews of interventions (version 6, 2019)The Cochrane Handbook for Systematic Reviews of Interventions is the official document that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions. This is Version 6 of the Handbook; updated 2019.
- JBI Manual for Evidence Synthesis.JBI, 2020. https://doi.org/10.46658/JBIMES-20-01
The JBI Manual for Evidence Synthesis is designed to provide authors with a comprehensive guide to conducting JBI systematic reviews. It describes in detail the process of planning, undertaking and writing up a systematic review using JBI methods.
Journals with Review Articles
- Review of Educational ResearchThe Review of Educational Research (RER, quarterly, begun in 1931; approximately 640 pp./volume year) publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research unless it is incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented.
- Review of Research in EducationReview of Research in Education (RRE), published annually, provides a forum for analytic research reviews on selected education topics of significance to the field. Each volume addresses a topic of broad relevance to education and learning, and publishes articles that critically examine diverse literatures and bodies of knowledge across relevant disciplines and fields. RRE volumes advance the state of the knowledge, promote discussion, and shape directions for future research.
- Educational Research ReviewEducational Research Review is an international journal addressed to researchers and various agencies interested in the review of studies and theoretical papers in education at any level. The journal accepts high quality articles that are solving educational research problems by using a review approach. This may include thematic or methodological reviews, or meta-analyses.
- Educational Psychology ReviewEducational Psychology Review is an international forum for the publication of peer-reviewed integrative review articles, special thematic issues, reflections or comments on previous research or new research directions, interviews, and research-based advice for practitioners - all pertaining to the field of educational psychology. The contents provide breadth of coverage appropriate to a wide readership in educational psychology and sufficient depth to inform the most learned specialists in the discipline.