Behaviour change 101 series: How to do a Rapid Review

Authored by Peter Slattery

In our work at BehaviourWorks Australia (BWA) we are frequently asked, ‘What does the research say about getting audience Y to do behaviour X?’ When our partners need an urgent answer, we can conduct a Rapid Review.

In this article, I explain Rapid Reviews, why you should do them and a process that you can follow to conduct one.

What is a Rapid Review?

Rapid Reviews are “a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner”.

Indeed, with sufficient resources (e.g., multiple staff working simultaneously) you can do a Rapid Review in less than a day. The outputs of these reviews are, of course, brief and descriptive, but they can be very useful where rapid evidence is needed, for example, in addressing COVID-19.

Specifically, while a Rapid Review follows similar protocols to a Systematic Review (e.g., establishing a search strategy and setting inclusion and exclusion criteria) it is differentiated from conventional systematic reviews by virtue of one or more deliberate simplifications, such as:

• focus on a narrow topic that will return few search results
• use of short time frames when searching (e.g., only searching for studies published in the last 2 – 5 years) use of relatively few databases
• exclusion of certain types of literature (e.g., primary studies) use of only one researcher (rather than two) to screen and extract results
• extraction of relatively few fields of information from the paper
• simple rather than complex synthesis of results
• limited or no quality assessment of included reviews creating short or simple outputs.

At BWA we frequently start with ‘a review of reviews’ approach. If no reviews are available, recent high-quality primary studies are sought.

The subsequent report provides a quick way to access and understand the most high-quality findings within a particular area.

Our Rapid Reviews are often accompanied by Practice Reviews – which provide ‘real world’ insights from those involved in developing and/or delivering strategies to address the problem under investigation.

Why do a Rapid Review?

The short answer is that review-level evidence is the most robust form of research to inform decision-making.

Good decision-making requires an understanding of context and causality. This is perhaps especially challenging when making decisions about behaviour change; behaviour occurs across a multitude of different contexts and has complex, varied and interacting causes which differ over time and across actors.

Research is one of many inputs into ‘good decision-making’ and different types of research provide different qualities of input. The best research input provides a summary of the full body of relevant research. This minimises risks that can come from making decisions based on considering a single study in isolation; for instance, choosing an intervention that other research shows to only be effective within a particular lab setting or audience.

For example Figure 1  illustrates how the field of Evidence- Based Medicine ranks the quality of different types of research evidence.

It shows that Evidence-Based Medicine regards Meta-Analytic and Systematic Reviews as the gold standard for research evidence; two systematic review techniques which involve analysing secondary data (e.g., checking and comparing multiple studies). However, the rigour of such research comes at a cost to its rapidness.

Meta-Analytic and Systematic Reviews can require extensive resources and take up to two years to complete.

As A/Prof Peter Bragge, Director of Health Programs at BehaviourWorks and Lead of the Monash-Social Systems Evidence Collaboration points out, “Governments are often unable to commission a Systematic Review due to time constraints”.

As a result, he notes that “relatively few policymakers, decision-makers and media are using Systematic Reviews to respond to complex challenges. Instead, they are searching Google and hoping that something useful will turn up amongst an estimated 6.19 billion web pages”.

If there is an insufficient resource, or impetus, to do a Systematic Review, then a Rapid Review is the next best option and one that is considerably better than searching Google! These reviews can provide a systematic overview of research and are easier to complete within short timeframes.

What are the limitations of Rapid Reviews?

Of course, there are limitations to Rapid Reviews that should always be made explicit. While Systematic Reviews and Rapid Reviews generally result in similar conclusions, the expedited nature of a Rapid Review creates an increased risk of not identifying relevant literature or insights that would be gained from a full review. Thus, one can be more confident in the results of a longer review process.

Additionally, for complex issues that are not easily or immediately solvable, such as obesity and chronic disease management, it can be more efficient in the long term to do one ‘living Systematic Review’ (i.e., a systematic review that is kept up to date) rather than repeated Rapid Reviews over long time periods, or develop a living database such as Social Systems Evidence.

What is needed to do a Rapid Review?

In most cases, doing a Rapid Review will require research training and expertise.

If you do not have this training, or the time to learn, then we recommend consulting an expert, particularly if producing a review with significant implications.

If you proceed with a review, then make sure that you have a university, library or personal subscription, so that you will have access to: i) relevant research databases, ii) the publications you retrieve from them and iii) software and processes needed to complete the review process.

How we conduct Rapid Reviews

The following section briefly summarises six stages that BWA uses to do Rapid Reviews.

Preparation

• Plan the type of Rapid Review to be undertaken (e.g., see the Selecting Approaches for Rapid Reviews (STARR) Decision Tool).
• Identify and refine a specific topic, research question and set of aims (consider using the PICO framework).

Develop and refine the:

• search terms (e.g., terms to search for) strategy (where to search, who to contact, what types of literature to include)
• inclusion and exclusion criteria
• databases that will be searched (e.g., Scopus, PubMed)
• quality appraisal tools to be used (e.g., AMSTAR 2 for systematic reviews)
• extraction plan and template to collect data from the relevant papers.

Pilot test initial ideas, for example, by testing different searches on different databases to get the ideal sensitivity and specificity, then revise plan and timeline as needed.

Finalise a protocol (i.e., write a plan for the search that describes the elements above).

Register the protocol using a service such as PROSPERO or OSF.

Search

Conduct searches on the selected databases and, evaluate search yields and,

• revise due dates if necessary
• download the records returned into research software such as Endnote, Zotero or Mendeley)
• de-duplicate records if needed.

Screening and selection

Conduct title and abstract screening:

• screen paper title and abstracts based on your inclusion and exclusion criteria
• resolve disagreements between those doing the screening (if relevant).

Conduct full-text screening:

• download full-text files for all relevant articles
• screen using screening software
• resolve disagreements.

Quality appraisal

• Conduct quality appraisal on the relevant papers, for example, using something like an excel spreadsheet.

Data extraction

• Extract data from all relevant papers using an extraction template and relevant software.

Synthesis and write up

• Generate a PRISMA diagram to summarise your review.
• Generate readable extraction tables.
• Write a narrative summary of findings using the extraction tables.

Recent Rapid Reviews completed by BWA

In 2019 we completed a rapid overview of reviews of Research co-design in heath.

On OSF, you can see the protocol that we registered and our materials, in addition to the extraction tables and quality assessment.

Extraction and Quality Assessment were conducted in Excel then later converted into tables.

Other recent Rapid Reviews that BWA has undertaken include:

1. Digital Inclusion & Health Communication: A Rapid Review of Literature.
2. Interventions to promote healthy eating choices when dining out: A systematic review of reviews
3. What interventions could reduce diagnostic error in emergency departments? A review of evidence, practice and consumer perspectives.

Summary

Rapid Reviews enable governments and other organisations to gain a rigorous understanding of current knowledge in weeks, rather than waiting months or longer for a full  Systematic Review.

In our training, we teach about Rapid Reviews and other methods of evidence synthesis. We can also help develop evidence synthesis and storage capacity within organisations.

Please get in touch if you think that we can help you. Email: Peter.slattery@monash.edu