Assessing a Body of Evidence of Complex Interventions

IBTN Core Member Dr. Paul Montgomery (University of Birmingham), in collaboration with the GRADE Working Group, is leading a research project to develop guidance for assessing and rating the quality of a body of evidence in systematic reviews of complex interventions. The guidance will be relevant for researchers in both clinical and social disciplines who conduct reviews on the effectiveness of complex interventions.

It is now standard practice for policy makers to draw on systematic reviews as the superior source of evidence to inform decision-making regarding effective interventions. Many interventions are currently conceptualised as “complex”, as they involve a number of interacting components, multiple outcomes, levels of target (such as communities and entire populations), and their effects are amenable to various contextual factors.

Consideration of aspects of complexity may require using alternative approaches and additional guidance when synthesising and assessing a body of evidence in systematic reviews and guideline development. For example, many complex interventions cannot be evaluated using randomised controlled trials and may need to integrate a large body of non-randomised evidence. In the meantime, there are ambiguities around how best to conceptualise and interpret certainty in the body of evidence on intervention effects, when these are contingent upon intervention programming, implementation and contextual factors. Concerns have been raised that the oversight of the aspects of complexity might result in uninformative synthesis results and often-observed conclusions of weak evidence for many complex interventions.

A research project will develop guidance for assessing and rating the quality of a body of evidence in systematic reviews of complex interventions. The project is led by Prof Paul Montgomery (University of Birmingham) in collaboration with the GRADE Working Group and stakeholders in evidence synthesis and guideline development. The guidance will be relevant for researchers in both clinical and social disciplines who conduct reviews on the effectiveness of complex interventions.

The project team is currently conducting an international online panel involving multidisciplinary expertise to identify aspects that need to be modified in the GRADE guidance for complex interventions. Following the online expert panel, a consensus meeting will be hosted with a select group of participants to build consensus for the new guidance.

Submitted by IBTN core member Dr. Paul Montgomery