Complex Reviews Support Unit: supporting the successful delivery of complex reviews. A workshop, November 2019

Complexity should not be feared, it can be enriching! This was the take home message from the joint National Institute for Health Research (NIHR) Complex Review Support Unit (CRSU) & Cochrane day workshop, attended by Cochrane Infectious Diseases Group (CIDG) Research Associate Dr Rebecca Thomas. The workshop took place in November 2019, around the corner from the bagel shops and curry houses of Brick Lane, at GCU London.

After starting the day with the perfect simplicity of a Brick lane bagel, the workshop began with a fantastic talk by Toby Lasserson, Deputy Editor-in-Chief of Cochrane, describing how Cochrane methodologies have evolved and adapted over time. He explained how moving away from simple comparisons can add value to a review and highlighted the Cochrane resources that can assist in this: for example, Chapter 11 of the latest Cochrane Handbook for Systematic Reviews of Interventions on undertaking network meta-analysis, as well as the new Cochrane Risk of Bias 2.0 tool.

NIHR CRSU team members described the methodologies and support for sources of complexity. First was a talk on Network Meta-Analysis (NMA) by Professor Neil Hawkins. NMA is a means of comparing three or more interventions by synthesising evidence from RCTs. It uses a statistical model to analyse a network of evidence, allowing for the comparison of treatment effects from multiple studies simultaneously, including those without direct evidence. Although it can improve precision of effect estimates, there was a discussion of the need to ensure a comparable distribution of effect modifiers across studies to ensure transitivity and consistency. As NMA becomes a well-recognised methodology in systematic reviews, the CRSU can provide tailored support to authors, as well as providing a wealth of online resources.

The next topic was diagnostic test accuracy reviews (DTA), what makes DTA reviews complex, and what are the key questions to be answered to produce a meaningful review. These include but are not limited to: what is the clinical pathway? How do the tests affect the downstream management of patients in that pathway? What are the health system effects? To which patient groups will the tests apply? What is an acceptable sensitivity and specificity? What is the gold standard? What to do if there is no gold standard? This talk was of particular interest as the CIDG is planning some forthcoming DTA reviews with complex clinical pathways.

Dr Hilary Thomason spoke about synthesis without meta-analysis. The complexity here lies in how to manage and appraise the data presented in a methodologically robust and transparent way. Analysis of published narrative reviews found that only 18% of Cochrane reviews reported methods of narrative synthesis. The interpretation and presentation of the data needs to be carefully considered in advance of the review without synthesis. Chapter 12 of the latest Cochrane Handbook for Systematic Reviews of Interventions discusses this further.

The morning rounded off with examples from the Cochrane Dementia and Cognitive Improvement Group, who have worked closely with the CRSU to produce varied complex reviews that contribute to crucial evidence in dementia diagnosis and care. They are pushing new methodologies in synthesising evidence in prognosis. They described how updating reviews can be an opportunity to introduce complexity to increase review impact, for example by breaking down interventions into its subcomponents and subgroups for NMA. This is something the CIDG can consider in its review update process.

The afternoon session consisted of a tutorial of MetaInsight 1.1, software produced by the CRSU team to perform NMA, as well as software to assist with analysis in DTA reviews. This software will be essential for incorporating NMA and DTAs into future CIDG reviews.

Overall the day demonstrated, with practical examples, how complexity in reviews can improve impact and value. Complexity should be embraced, and there is a range of support available in the form of online resources, new software, personalised support, and collaboration. Participants rounded off the day in a most appropriate way: sampling the complex flavours of the Brick Lane curry houses and contemplating how the CIDG can incorporate complexity into our future.

Links to Apps

• MetaInsight 1.1 (Frequentist analysis)
  • Binary: https://crsu.shinyapps.io/metainsightc/
  • Continuous: https://crsu.shinyapps.io/metainsightb/
• MetaInsight 2.0 Beta (Bayesian analysis)
  • Binary : https://crsu.shinyapps.io/metainsight_binary2/
  • Continuous:https://crsu.shinyapps.io/metainsight_continuous2/
• DTA DTA-MA: https://crsu.shinyapps.io/dta_ma/
• Primer: https://crsu.shinyapps.io/diagprimer/
• Workshop Webpage: http://www.nihrcrsu.org/workshops/joint_crsu__cochrane_workshop_2019/

Rebecca Thomas is supported by the Research, Evidence and Development Initiative (READ-It) project. READ-It (project number 300342-104) is funded by UK aid from the UK government; however, the views expressed do not necessarily reflect the UK government’s official policies.