On his good days, Goldacre would patiently explain the difference between trustworthy evidence and dubious evidence. Published studies are better than unpublished studies, peer review is better than no peer review, randomised control trials are better than observational studies, literature reviews are better than individual studies, and so on. In short, the pyramid of evidence looks like this...
Wrong. In December, the first - and so far only - Cochrane review of e-cigarettes was published. It found 'evidence from two trials that ECs [e-cigarettes] help smokers to stop smoking long-term compared with placebo ECs'.
This week, a conference abstract for another literature review - one that included non-RCTs - was released to the press. It concluded that 'Evidence that electronic cigarettes are effective for smoking cessation long-term is lacking'.
To my knowledge, Goldacre has never mentioned the Cochrane review, nor has he mentioned the systematic review that was published in PLoS One last month which concluded that 'Use of e-cigarettes is associated with smoking cessation'. Nor, indeed, has he mentioned the study in Addiction which found e-cigarettes to be more than twice as effective than nicotine-replacement therapy, or the study published three months ago which found that intensive users of e-cigarettes were six times more likely to quit than non-users. But when presented with an unpublished conference abstract, he wasted no time in spreading the news with this bilious Twitter communiqué...
Systematic review: e-cigs don’t help smokers quit. Tweeting this specifically because e-cig campaigners are so vile. http://t.co/vtOQf1MwlY
— ben goldacre (@bengoldacre) May 19, 2015
What's got into him? He has previously referred to e-cigarettes as the 'tobacco industry's latest scheme', seemingly unaware that until 2012 there was no tobacco industry involvement in the sector whatsoever and even today the combined might of global Big Tobacco owns just seven of the thousands of brands on the market. On the subject of vaping, Goldacre has shown himself to be no better than the two-bit hacks who write "I reckon" articles about e-cigarettes based on whatever stray thought enters their head. Gateway! Formaldehyde! Children!
But this is not a post about Goldacre's confirmation bias. There is another mistake he makes which was also made by the Daily Mail, namely that the study shows that 'e-cigs don't help smokers quit'. Even the press release doesn't claim that. It says only that, based on its findings, the evidence is 'lacking'.
Absence of evidence is not evidence of absence, particularly when you look at the results in the abstract. The review found the odds of e-cig users (versus placebo users) staying abstinent after one month were 1.71 (1.08-2.72). After three months they were 1.95 (0.74-5.13) and after six months they were 1.32 (0.59-2.93). The first of these is statistically significant, the others are not, hence the lack of firm evidence for cessation at six months.
The study also mentions that the 'only study to evaluate continuous abstinence' found that e-cigarette users' odds of abstinence were 1.77 (0.54-5.77) after six months compared to the placebo group. This, again, is not statistically significant, but nor is it close enough to 1.0 for a positive effect to be ruled out. The same is true of the other findings, all of which are well above 1.0. It would be quite wrong to say that these statistically non-significant odds ratios proved that e-cigarettes work better than placebos, but it is equally wrong to say that they show that 'e-cigs don't help people quit'. All they show is that larger trials will be needed unless e-cigarettes more than double the chance of quitting. The Cochrane review identified one randomised controlled trial which did indeed find that e-cigarettes more than doubled the chances of quitting. We don't know which studies the conference review included (or why) because, as I say, it hasn't been published.
I wouldn't claim for a moment that the Cochrane review is the last word on e-cigarettes and nor do its authors. Cochrane reviews only look at RCTs and only two RCTs have so far been conducted, making the review weak by Cochrane's high standards. Nevertheless, it does have the merit of being peer-reviewed and published, which is normally the minimum Goldacre requires before taking research seriously. That is, unless he wants to wind up a section of society - most of whom no longer smoke thanks to e-cigarettes - that he deems 'vile', in which case any old press release will do.
No comments:
Post a Comment