Interpreting results of Cochrane reviews and Summary of Findings Tables: GRADE and SoF Workshop Magnitude of effect and confidence on effect are important parameters in quality of evidence
The Cochrane Colloquium 2014 is keeping up with its trend of providing high quality training for doing systematic reviews. The workshop today focused on the very useful issue of interpreting reviews and summary of findings. The event was done by the McMaster University , Canada faculty.
Here are some useful notes from the workshop .
Important parameters assessed in GRADE are
risk of bias
inconsistency
indirectness
imprecision
publication bias
dose response ,
size of effect ,
confounding
The conclusion about the effect of studies should provide the following information.
1.results of section
2.no of studeies
3.magnitude of effect
4.converting it to numbers which stakeholders understand (example : how many fractures were prevented after intervention : use confidence interval to report )
5. GRADE quality of evidence and brief on rational behind it.
New online version of grade software : GDT www.guidelinedevelopment.org.
When using scales and reporting in meta-analysis figure or GRADE table it is always important to communicate direction of scale and what is it about.
Remember to adjust for the image scale in forest plot .
Reviewers often forgot the red, green and yellow dots they have created in the risk of bias during interpretation of results .
Funnel plot cannot be created when less than 10 studies
Consider search strategy comprehensiveness, foreign language missing . smaller studies , gray literature search if funnel plot appears skewed.
Also Look at funding of study and competing interests when looking at publication bias .
Look at the characteristics of study carefully when doing GRADE
Do not use terms like ” not statistically significant” .
It is Important to calculate the optimal information size when using GRADE for imprecision. quality
Imprecision is done on basis of following for dichotomous outcomes : 1. sample size and number of events 2, confidence intervals
Imprecision is done on basis of following for Continuos outcomes : at least 400 people providing outcome measures : if not GRADE for imprecision.
Rule of thumb is if CI includes 0.75 to 1.25 indicates null effect and appreciable benefit or harm
Distinction between serious and very serious is important : but the balance is to be done by the reviewers: the thinking behind the judgment should be reflected in the footnotes. Let people know the thinking behind the grade done
small sample size but large effect. : could be indicative of the obeservation being just due to chance. Therefore do not depend on confidence interval but on the number of events.
Their are multiple ways of choosing a baseline risk for GRADE process. It can be the average or the extremes or even baseline risk from observational study. However this has to be justified.
All presentations in the colloquium will be available http://cebgrade.mcmaster.ca/hyderabad/by the end of the week.