Cochrane, GRADE, and gelato: assessing the certainty of the evidence

Modena is home to balsamic vinegar, Ferrari, Pavarotti, and Cochrane Italy. So, where better to learn about GRADE – an approach that allows researchers and guideline developers assess the certainty of research evidence. For years we struggled with how to express how accurate an estimate is in a meta-analysis, how much it varies between studies, and whether we are confident this is a true effect and applicable to the population we are interested in. Then GRADE came along and changed everything.

The Cochrane Infectious Diseases Group (CIDG) has used GRADE (which stands for Grading of Recommendations Assessment, Development and Evaluation) for over eight years, yet the methods keep developing. Leslie ChoiRachael Milligan, and Joe Pryce, participated in this workshop to get to the cutting edge of the approaches, with other Cochrane authors and editors from all over the world. The workshop also introduced a new method for assessing certainty of evidence in qualitative research, called GRADE-CERQual.


 The streets of Modena

Leslie, Rachael, and Joe attended the ‘effectiveness of intervention’ stream, led by Holger Schünemann, who is chair of the GRADE working group. This was an excellent opportunity to learn about GRADE and discuss GRADE queries related to CIDG reviews with one of the people instrumental in establishing this tool. Leslie and Joe are currently working with the World Health Organization (WHO) on a collection of Cochrane Reviews on malaria vector control, so the sessions on evidence to decision frameworks were highly relevant to them.  As well as working on her own Cochrane Reviews, Rachael is involved in teaching GRADE to the Diploma in Tropical Medicine and Hygiene (DTM&Hand MSc students at the Liverpool School of Tropical Medicine (LSTM), so she found the training very useful for this and will be adapting the CIDG’s training materials accordingly. 


Participants at the GRADE CERQual training meeting in Modena, June 2017

As you might expect – Cochrane Italy outdid themselves in hospitality and catering – with sessions organized around when the next coffee and pastry break was! Productivity was high, and not just because the attendees were averaging a cappuccino an hour.

And last but not least, all CIDG attendees agreed that the gelato in Italy was second to none (1 trial, 3 participants, high quality evidence*). *Upgraded due to large effect in quality of life improvement.


 (L-R) Joe, Rachael, and Leslie assess the certainty of the evidence

Amendment: the CIDG editorial base amended the original article on 10 July 2017. To clarify, the Italian GRADE Centre is located in Rome at the Department of Epidemiology of the Lazio Region, and not in Modena as stated in the earlier version of the article.