The importance of systematic reviews in psychology
Systematic reviews are considered to be the gold-standard of medical research. The primary goal of a systematic review is to summarise all of the relevant studies in a particular area of interest, producing a convenient resource for readers to stay up-to-date with the latest research. Because systematic reviews aim to prevent biased interpretations of research, psychologists have become interested in utilising them to build trust in psychological findings.
Systematic Review Image 1
Keep reading if you’re interested in exactly how systematic reviews reduce bias and how conducting one may be beneficial to you!
Systematic reviews minimise author bias
Author bias refers to when an author shows favouritism towards certain evidence (whether intentionally or unintentionally) and threatens the trustworthiness of a review. Whilst I’m sure that most researchers set out to be unbiased when selecting research to include in their review, it is almost impossible to be aware of every piece of evidence relevant to a particular research area.
Fortunately, a systematic search will reduce the impact of author bias on your review by attempting to find all of the available evidence on your chosen research question. Tip: for best results, I suggest consulting with an expert to help you produce your search, such as a librarian trained to conduct systematic reviews.
Systematic reviews consider bias in individual studies
Another source of bias originates from your included studies because a review is only as trustworthy as the research included within it. This is why a good systematic review includes a quality assessment that highlights high-quality research and alerts the reader to studies that may be influenced by some bias. For example, sampling bias, whereby the participants recruited for the study do not accurately represent the study’s target population.
Unfortunately, a tool that specifically addresses the quality of non-medical research doesn’t currently exist. This means that researchers are forced to edit existing quality assessment tools to suit their purposes (for example, the Downs and Black checklist).
Luckily, the NIROES collaboration hopes to solve this problem by creating a quality assessment tool specific to non-medical research. Such a tool is incredibly important for building trust in the quality of psychological research.
Systematic reviews minimise publication bias
An additional source of bias that can affect the quality of your review is publication bias. Publication bias refers to the fact that journals are much more likely to publish papers with positive findings compared to research that doesn’t find any statistically significant effects. This can seriously affect how you interpret the findings of your included studies.
Tips: to combat publication bias, researchers are encouraged to search for relevant grey literature (for example, pre-prints, dissertations, and theses) using online platforms like PsyArXiv. Authors should also conduct a funnel plot, which will provide you with a visual representation of how much publication bias has affected your research area of interest.
An asymmetrical funnel plot indicates the presence of publication bias.
Study precision: standard error or sample size.
Funnel Plot
Systematic reviews encourage reproducibility
Besides reducing bias, systematic reviews are beneficial because they encourage open and transparent research practices. For example, many journals will now only publish your systematic review if you have pre-registered it.
Pre-registering involves making your hypotheses, methodology, and statistical analyses publically available on online platforms, such as the Open Science Framework, before you begin conducting your review. Tip: the NIRO-SR Checklist provides clear guidance for pre-registering your review and encourages you to be transparent about your decisions.
Systematic reviews help identify gaps in the research
Another advantage of systematic reviews is that they will help you (and your readers!) identify new research questions that haven’t been addressed by the current literature. Systematic reviews can also help you to identify limitations of the research in your area of interest (for example, methodological issues and common causes of bias) that you could address in a future study.
Concluding remarks
I hope I’ve managed to convince you that systematic reviews are an important research method in psychology (and have maybe encouraged you to have a go at conducting one yourself). Thankfully, with the vast amount of open-access and free online tools available (for example, CADIMA), it is easier than ever to conduct your own systematic review!
This blog post has kindly been contributed by Melissa Gibbs