Abstract
After reading our Focus article herein and the Commentaries one may very well ask: What then do I use as a yardstick by which to judge meta-analysis for clinical decision-making purposes? Is meta-analysis equivalent to “mega-silliness” [6]? Can meta-analysis be used to resolve scientific brouhahas [14]? Or, does it actually create such brouhahas?
Some guidance, then, for caution in the interpretation of meta-analysis for the reader of the pain literature may be of utility. Several clear caveats can be offered. A caution should be stated regarding publication bias—that is, significant results are more likely to be published—an issue that is greater for trials with a smaller sample size. Results that seem “too good to be true” probably are. Always examine the quality of the trials included in the review and evaluate the consistency of the results. A reader should also examine whether both clinical and statistical criteria were offered for the inclusion of trials for analysis from the larger set of trials screened for entry. When the results are on the same side of the “no difference between groups” line, then more confidence may be taken in the reported metaanalysis result [5]; however, although it is more time consuming, we would recommend separate evaluation of each trial by the reader, as does Horwitz [9]. Finally, any subgroup analyses should be examined carefully. Although subgroup analyses can help define a response surface, lack of sufficient data and appropriate analytic techniques can vitiate this approach.
The broader area of meta-analytic review of the analgesic effects of antidepressants for chronic, benign pain syndromes has largely shown significant effects for the antidepressants, though the size of the effects reported has been more modest in recent reviews. Hence, it might be concluded that our interpretation of the results of trials conducted in this area to date are nihilistic in that we have stated that no effect can clearly be demonstrated, with the exception of headache; however, we emphasize that a well-justified conclusion (that we largely do not know the effect of this treatment) is not equivalent to a decrease in knowledge or a denial of knowledge of the experts in the field [1]. In fact, in this or any area of research, such a conclusion may well represent an actual increase in scientific knowledge. That is, we could conclude that trial quality assessment and the extent of heterogeneity in antidepressant medications tested and in pain syndromes investigated may account for our differential results. Stating that there is currently insufficient evidence to conclude that all antidepressants have analgesic effects in all chronic, benign pain syndromes is different from stating that selected antidepressant medications may well demonstrate analgesic effects in specific chronic, benign pain syndromes. It is the latter statement with which we are in accord and believe merits further research.