Tuesday, 5 July 2011

Does Champix cause heart attacks? Hmm, maybe.

The news that Champix/Chantix has been linked to heart disease has been widely reported in the last day or two, after a meta-analysis found that those who use the controversial stop-smoking drug had a 72% increased risk of adverse cardiovascular events.

The lead author, Sonal Singh, has been getting rather excited about the results.

"This is just like driving a car without brakes,” he said. “Going forward, I don’t know how we will convince our patients to take Champix for what, to increase your risk for heart attack?... People should be concerned. They don’t need Chantix to quit and this is another reason to avoid Chantix all together."

The days of the sober, disinterested academic may be over, but it would nice if researchers could release their results without sounding quite so much like campaigners. Especially when your results really aren't that impressive.

The meta-analysis took 14 studies and found a relative risk of 1.72. That sounds OK until you look at the results of the studies themselves (click to enlarge).

Of the 14 results, only one of them is statistically significant (just) and the rest of them are miles away from achieving significance. This is because the number of people suffering heart attacks is tiny—about one in a thousand overall. In only one of the studies was there more than seven cardiovascular events between the Champix users and the placebo group combined. In six of the studies, there was only one heart attack in the entire cohort. When all studies are combined you still only have 79 heart attacks from over 8,000 people (and 45 of these come from just one study, which clearly used a very different methodology). This, at the very least, indicates that if there is a risk here—and there may be—it is very small in absolute terms. It is simply impossible to take these studies as proof of anything.

This is not the (original) researchers' fault. The studies are too small for enough cases to emerge. If and when larger studies are conducted, it may become apparent that there is a real risk. The fact that 11 of the 14 studies show an increase in risk—albeit a very flimsy one—is suggestive of an effect, but I would put it no more strongly than that. Comparing it to driving a car without brakes is ludicrous and suggests a certain crusading zeal from the study's authors that may have biased their approach.

This is the problem with meta-analysing statistically nonsignificant relative risks. It's a fallacy to think that by combining different studies with different flaws and biases you are creating one good, robust study. You're not. If the original studies don't tell you anything, combining them usually doesn't make them any more informative. Meta-analyses are pretty good at estimating an accurate relative risk when there is strong evidence that a risk exists. When the risk is far from proven—and most of these studies aren't even suggestive—they generate more heat than light.

I realise that this view is far from universal in the debased world of modern epidemiology (not least by those who gather together a similar grab-bag of weak and conflicting secondhand smoke studies to 'prove' various outcomes), but achieving statistical significance by combining a bunch of near-useless studies is, in my view, tantamount to cheating. It is the equivalent of multiplying zero by ten and coming up with two. Consequently, the researchers have found a relative risk of 1.72 that is much higher or much lower than any of the risks found in the studies themselves. That should be a clue that something is wrong.

Lord knows, I hold no brief for the pharmaceutical industry and there are already good reasons not to take Champix without worrying about heart problems, but if the drug really does cause users to have heart attacks, this meta-analysis falls a long way short of proving it. It would be wrong and inconsistent to expect a lower burden of proof against Champix just because it is made by Big Pharma. This is one of those instances where more research really is needed.


Anonymous said...

"... this meta-analysis falls a long way short of proving it. It would be wrong and inconsistent to expect a lower burden of proof against [Second Hand Smoke] just because it is made by Big [Tobaccl]. This is one of those instances where more research really is needed."

Dick Puddlecote said...

One day, ASH will report meta-analyses on SHS like this, Chris.

Yes, thought that would make you laugh. :)

Anonymous said...

So, the same bastardisation of statistics used to damn SHS is now used to damn pharmaceuticals? The irony is pungent.

[BTW, I wonder - especially as most drugs are already given to people who are sick - just how many pharmaceutical products could withstand this kind of analysis? Not many, I would suspect.]

BrianB said...

the researchers have found a relative risk of 1.72

Actually, Chris, the Relative Risk on those numbers (52/4908 Champix, 27/3308 Placebo) is approx 1.11 with a lower CI of 0.9. The 'proper' Odds ratio is approx 1.3 with a lower CI of 0.8 - so neither option achieves any recognisable level of statistical significance.

But of course these researchers used the Peto Odds Ratio, (mathematically, it isn't an odds ratio at all - but that's for another debate), which magically transforms meaningless statistics into press-release headlines.

Oh, and the 'Peto' in question is none other than Richard Peto who:

"was made a Fellow of the Royal Society in 1989 for his contributions to the development of meta-analysis. He is a leading expert on deaths related to tobacco use."

So, the anti-tobacco supremo devised his own statistical procedure that coincidentally enables anti-tobacco to 'improve' the results of their home-grown method of numerical cheating (meta-analysis). Well colour me cynical!

I concur with Anon above; the fact that this methodology is now going to be used to damn big pharma is pungently ironic (or is that ironically pungent?).


I wonder when the ASH knuckle-draggers will be wheeled out to complain about the 'inadequacies' of this study. That I will then be forced to agree with them is somewhat frightening!

D'babe said...

It won't happen. They'd be too shit feared that the tripe 'studies' they've been citing might attract some proper scrutiny ...

WestBerkeleyFlats said...

Oh, please, meta-analysis in general is a fairly useful tool. It's useful in cases where you have relatively small studies due to inherent sample size limitations or relatively rare outcomes, and want to demonstrate statistical significance.

The specific problem here is that 55% of the weight is given to one study.

WestBerkeleyFlats said...

And meta-analysis has been consistently been used by people such as Peter Lee to demonstrate that many supposed risks associated with tobacco products are grossly inflated in popular perception.