The lead author, Sonal Singh, has been getting rather excited about the results.
"This is just like driving a car without brakes,” he said. “Going forward, I don’t know how we will convince our patients to take Champix for what, to increase your risk for heart attack?... People should be concerned. They don’t need Chantix to quit and this is another reason to avoid Chantix all together."
The days of the sober, disinterested academic may be over, but it would nice if researchers could release their results without sounding quite so much like campaigners. Especially when your results really aren't that impressive.
The meta-analysis took 14 studies and found a relative risk of 1.72. That sounds OK until you look at the results of the studies themselves (click to enlarge).
Of the 14 results, only one of them is statistically significant (just) and the rest of them are miles away from achieving significance. This is because the number of people suffering heart attacks is tiny—about one in a thousand overall. In only one of the studies was there more than seven cardiovascular events between the Champix users and the placebo group combined. In six of the studies, there was only one heart attack in the entire cohort. When all studies are combined you still only have 79 heart attacks from over 8,000 people (and 45 of these come from just one study, which clearly used a very different methodology). This, at the very least, indicates that if there is a risk here—and there may be—it is very small in absolute terms. It is simply impossible to take these studies as proof of anything.
This is not the (original) researchers' fault. The studies are too small for enough cases to emerge. If and when larger studies are conducted, it may become apparent that there is a real risk. The fact that 11 of the 14 studies show an increase in risk—albeit a very flimsy one—is suggestive of an effect, but I would put it no more strongly than that. Comparing it to driving a car without brakes is ludicrous and suggests a certain crusading zeal from the study's authors that may have biased their approach.
This is the problem with meta-analysing statistically nonsignificant relative risks. It's a fallacy to think that by combining different studies with different flaws and biases you are creating one good, robust study. You're not. If the original studies don't tell you anything, combining them usually doesn't make them any more informative. Meta-analyses are pretty good at estimating an accurate relative risk when there is strong evidence that a risk exists. When the risk is far from proven—and most of these studies aren't even suggestive—they generate more heat than light.
I realise that this view is far from universal in the debased world of modern epidemiology (not least by those who gather together a similar grab-bag of weak and conflicting secondhand smoke studies to 'prove' various outcomes), but achieving statistical significance by combining a bunch of near-useless studies is, in my view, tantamount to cheating. It is the equivalent of multiplying zero by ten and coming up with two. Consequently, the researchers have found a relative risk of 1.72 that is much higher or much lower than any of the risks found in the studies themselves. That should be a clue that something is wrong.
Lord knows, I hold no brief for the pharmaceutical industry and there are already good reasons not to take Champix without worrying about heart problems, but if the drug really does cause users to have heart attacks, this meta-analysis falls a long way short of proving it. It would be wrong and inconsistent to expect a lower burden of proof against Champix just because it is made by Big Pharma. This is one of those instances where more research really is needed.
