When you need to gather the evidence - fast
In this ‘Behaviour change 101’ series, we draw on our experience and educational resources to discuss approaches for planning and executing behaviour change research and projects.
Authored by Peter Slattery
In our work at BehaviourWorks Australia (BWA) we are frequently asked, ‘What does the research say about getting audience Y to do behaviour X?’ When our partners need an urgent answer, we can conduct a Rapid Review.
In this article, I explain Rapid Reviews, why you should do them and a process that you can follow to conduct one.
Rapid Reviews are “a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner”.
Indeed, with sufficient resources (e.g., multiple staff working simultaneously) you can do a Rapid Review in less than a day. The outputs of these reviews are, of course, brief and descriptive, but they can be very useful where rapid evidence is needed, for example, in addressing COVID-19.
Specifically, while a Rapid Review follows similar protocols to a Systematic Review (e.g., establishing a search strategy and setting inclusion and exclusion criteria) it is differentiated from conventional systematic reviews by virtue of one or more deliberate simplifications, such as:
At BWA we frequently start with ‘a review of reviews’ approach. If no reviews are available, recent high-quality primary studies are sought.
The subsequent report provides a quick way to access and understand the most high-quality findings within a particular area.
Our Rapid Reviews are often accompanied by Practice Reviews – which provide ‘real world’ insights from those involved in developing and/or delivering strategies to address the problem under investigation.
The short answer is that review-level evidence is the most robust form of research to inform decision-making.
Good decision-making requires an understanding of context and causality. This is perhaps especially challenging when making decisions about behaviour change; behaviour occurs across a multitude of different contexts and has complex, varied and interacting causes which differ over time and across actors.
Research is one of many inputs into ‘good decision-making’ and different types of research provide different qualities of input. The best research input provides a summary of the full body of relevant research. This minimises risks that can come from making decisions based on considering a single study in isolation; for instance, choosing an intervention that other research shows to only be effective within a particular lab setting or audience.
For example Figure 1 illustrates how the field of Evidence- Based Medicine ranks the quality of different types of research evidence.
It shows that Evidence-Based Medicine regards Meta-Analytic and Systematic Reviews as the gold standard for research evidence; two systematic review techniques which involve analysing secondary data (e.g., checking and comparing multiple studies). However, the rigour of such research comes at a cost to its rapidness.
Meta-Analytic and Systematic Reviews can require extensive resources and take up to two years to complete.
As A/Prof Peter Bragge, Director of Health Programs at BehaviourWorks and Lead of the Monash-Social Systems Evidence Collaboration points out, “Governments are often unable to commission a Systematic Review due to time constraints”.
As a result, he notes that “relatively few policymakers, decision-makers and media are using Systematic Reviews to respond to complex challenges. Instead, they are searching Google and hoping that something useful will turn up amongst an estimated 6.19 billion web pages”.
If there is an insufficient resource, or impetus, to do a Systematic Review, then a Rapid Review is the next best option and one that is considerably better than searching Google! These reviews can provide a systematic overview of research and are easier to complete within short timeframes.
Of course, there are limitations to Rapid Reviews that should always be made explicit. While Systematic Reviews and Rapid Reviews generally result in similar conclusions, the expedited nature of a Rapid Review creates an increased risk of not identifying relevant literature or insights that would be gained from a full review. Thus, one can be more confident in the results of a longer review process.
Additionally, for complex issues that are not easily or immediately solvable, such as obesity and chronic disease management, it can be more efficient in the long term to do one ‘living Systematic Review’ (i.e., a systematic review that is kept up to date) rather than repeated Rapid Reviews over long time periods, or develop a living database such as Social Systems Evidence.
In most cases, doing a Rapid Review will require research training and expertise.
If you do not have this training, or the time to learn, then we recommend consulting an expert, particularly if producing a review with significant implications.
If you proceed with a review, then make sure that you have a university, library or personal subscription, so that you will have access to: i) relevant research databases, ii) the publications you retrieve from them and iii) software and processes needed to complete the review process.
The following section briefly summarises six stages that BWA uses to do Rapid Reviews.
Develop and refine the:
Pilot test initial ideas, for example, by testing different searches on different databases to get the ideal sensitivity and specificity, then revise plan and timeline as needed.
Finalise a protocol (i.e., write a plan for the search that describes the elements above).
Conduct searches on the selected databases and, evaluate search yields and,
Screening and selection
Conduct title and abstract screening:
Conduct full-text screening:
Synthesis and write up
In 2019 we completed a rapid overview of reviews of Research co-design in heath.
On OSF, you can see the protocol that we registered and our materials, in addition to the extraction tables and quality assessment.
Extraction and Quality Assessment were conducted in Excel then later converted into tables.
Other recent Rapid Reviews that BWA has undertaken include:
Rapid Reviews enable governments and other organisations to gain a rigorous understanding of current knowledge in weeks, rather than waiting months or longer for a full Systematic Review.
In our training, we teach about Rapid Reviews and other methods of evidence synthesis. We can also help develop evidence synthesis and storage capacity within organisations.
Please get in touch if you think that we can help you. Email: Peter.firstname.lastname@example.org
Check out our Monash University accredited courses, along with our short and bespoke training programs.
Get monthly behaviour change content and insights