Hurling brickbats

I'm always slightly taken aback when seemingly intelligent and well-informed people nonchalantly dismiss whole bodies of evidence as if they don't exist. Last year, I had a number of face-to-face discussions with people who had some authority in their field but who cheerfully denied the existence of facts which displeased them. Not denied knowledge of them, mind. They knew that they were generally regarded as facts, they just felt that if they said the facts were wrong then they were wrong. No evidence required.

Without naming names, the head of an addiction charity and a peer of the realm both blithely responded to the well-documented fact that alcohol consumption has, by any measure, fallen in the last decade, with words along the lines of "I don't believe it." On another occasion, a senior figure in public health asked if I really believed the Scottish government's own figures about how many drinks would go up in price under a minimum pricing regime (it was a very large percentage—why would they lie about that?). Similarly, I sat in rooms with a well-known TV doctor and the head of a relatively well-known public health association, both of whom flatly denied that there was any benefit to cardiovascular health from drinking moderate quantities of alcohol.

I hesitate to use the rather silly word 'denier', but these are issues about which the evidence could hardly be stronger. What all these examples have in common is that the facts are inconvenient to the person hearing them and so, rather than acknowledge reality, they simply shake their head and wish them away. In a pub conversation without access to Google, this is an effective way of shutting down debate, but I was surprised to find influential people—all of whom no doubt regard themselves as 'evidence-based'—being so badly informed/self-deluding. I suppose you can afford to be if you're in a position of power. Not a very comforting thought.

A couple of weeks ago, I wrote about the systematic review which found a slight reduction in mortality risk from being overweight. This, too, was dismissed with contemptuous insouciance. You may recall Walter Willett, an eager warrior in the battle against obesity, being so annoyed by the findings that he said: "This study is really a pile of rubbish and no one should waste their time reading it." Fine words from an academic. It reminded me of Stanton Glantz's response when the British Medical Journal published a passive smoking study that displeased him, conducted by two professors from UCLA. He said: "the science that the UCLA study did was crap" but did not feel he needed to explain in what way this was so. Later, the BMJ's editor, Richard Smith, said of the angry reaction to the study that it was "disturbing that so many people and organisations referred to the flaws in the study without specifying what they were. Indeed, this debate was much more remarkable for its passion than its precision."

This is often the way when research is deemed unhelpful to certain interest groups. Mud is slung and voices are raised. For the campaigner who is emotionally involved in the issue and has already made his mind up, substantive points are less important than creating the perception that the study in question has been 'debunked'.

There was a minor example of this in the Guardian this week when an unassuming little study about genetics was savaged by the postgraduate epidemiology student and award winning blogger Suzi Gage. The study found that people with certain genotypes are more likely to cut down or quit their tobacco use in response to higher tobacco taxes than are people with other genotypes. To put it another way, some people's genes make it harder for them to quit.

This is by no means a new discovery. It is very plausible that certain people get more pleasure from smoking than others and are therefore less willing, or less able, to quit, even with the strong disincentive of higher taxes. This may well be a genetic difference, as many other well-publicised studies have suggested.

This, however, was not a well-publicised study, so I was puzzled by why Gage bothered to write about it at all, let alone why she went at it with both barrels...

Poorly conducted research is a gift to the opponents of tobacco control

...this study is a gift to those who would like to prevent attempts to reduce levels of smoking. Tobacco control research is under huge amounts of scrutiny from those trying to undermine it. Poorly conducted experiments where the conclusions overstate the actual findings make public health's battle with vested interests all the more difficult.

Her wrath mainly seems to have been stirred by the study's title: 'Why Have Tobacco Control Policies Stalled? Using Genetic Moderation to Examine Policy Impacts'. You would think that this would be a question that those who want "to reduce levels of smoking" would be interested in. It is inarguable that the decline in smoking rates has slowed, stalled or even gone backwards in some countries in the last decade. It is self-evident that stop-smoking policies have not worked on everyone and it is an observable fact that there is a 'hardcore' of smokers who seem impervious to taxation, advertising bans, display bans, graphic warnings, smoking bans etc. Moreover, this 'hardcore' makes up a significant minority of the population—at least 20 per cent in most countries. It is worth asking why.

Gage thinks it is dangerous to even ask the question. She thinks the mere existence of this kind of research plays into the anti-anti-smokers' hands by casting doubt on existing policy and therefore needs a damned good debunking. This is despite the fact that the author of the study, from the Department of Health Policy at Yale, appears to have no axe to grind and concludes his study with these less-than-inflammatory words...

This is an important first step in future health policy efforts to further reduce adult smoking rates. Additionally, this study has begun a new examination of potential gene X policy (G X P) interactions that may have broad scope in learning why some policies are effective and others are not and also deepen our understanding of the genetic response to broad-based policy interventions.

This, apparently, is a "gift to those who would like to prevent attempts to reduce levels of smoking" and Gage goes on to provide some "alternative explanations" for the study's findings. Actually, she doesn't. She just offers various caveats and suggestions for future research that would normally fall under the heading of 'limitations' in your average epidemiological study. Some of it is relevant, some of it is nitpicking and some of it doesn't have much to do with the study at all. None of it justifies her claim that the research is "poorly conducted." It is rather like saying that an epidemiological study is "poorly conducted" because correlation doesn't equal causation. There are inherent limitations in some fields and there is usually room for more research. That doesn't make the current research worthless.

What surprises me about all this is that Gage thinks that this little study is such a potential hand-grenade that it needs to be eviscerated in the popular press. I freely admit that I am one of the "opponents of tobacco control" insofar as I think the movement is on the wrong track and often does more harm than good. It is not just that it isn't conducive to liberty, but that it doesn't work even on its own terms. If one is serious about reducing the damage to health of cigarette smoking—which can be the movement's only justification for its existence—then the promotion of e-cigarettes, snus and other alternative nicotine delivery devices could make the whole smoking issue yesterday's news within a decade. Unfortunately, the tobacco control movement has become so caught up in vested interests, self-interest and self-righteousness that it pursues increasingly trivial and useless policies ("let's make the warnings bigger!") while blocking the effective remedies that stare them in the face.

But although I am one of those "opponents", it never occurred to me to exploit this particular piece of research, nor has anyone else in the media or blogosphere as far I know. At the most, the author of the study is saying that other anti-smoking policies need to be implemented in addition to Pigovian taxation. Who in tobacco control does not agree with that? The aim of Gage's piece is not to challenge a claim that has been made by the study's author, nor is it to torpedo claims made by someone who has latched onto it. Rather it is to destroy the credibility of a modest piece of research on the off-chance that someone in the future may use it to challenge the orthodoxy of tobacco control.

It takes extreme hyper-sensitivity to throw brickbats at a fellow scientist because his research might inspire someone to question whether every political action you support is 100% effective on 100% of the population. Let's not forget—as Gage apparently did—that there is an academic behind this study who wasn't expecting his work to be maligned just because he dipped his toes into the paranoid world of tobacco control. His name is Jason Fletcher and he is not best pleased, as his comments at the Guardian reveal...

jasonfletcher
17 January 2013 1:32 AM

What about a poorly conducted blog post incorrectly describing the study?

As one of many examples:

"Perhaps even more importantly, there is such a distance between individual genetics and state-wide policy, that to say one affects the other is to ignore a huge amount of influence that happens between the level of the cell and that of the population."

This is not at all what is being examined in the paper (genetics affecting state policy or vice versa). Perhaps before writing this type of critique you might ask the author for clarification or response?

You might also clarify precisely what overstatements were made in the paper when writing: "Poorly conducted experiments where the conclusions overstate the actual findings...'

Jason Fletcher (Study Author)


soozaphone
17 January 2013 11:48 AM

Jason, thanks very much for your comment.

If your paper is not examining the relationship between genotype and a state level policy (in this case, tobacco taxation), I am very sorry that I've misunderstood it. Please could you correct me?

The overstatements I believe the paper makes are mentioned in the article. In particular, the title of your paper is 'why have tobacco control policies stalled?', which suggests you're going to answer this question, yet your study doesn't look at how policy affects quit rate; you don't investigate either the impact of a change in policy, or indeed the number of people successfully quitting.

I am sorry if there is a negative tone in the headline. I think that ascertaining why tobacco control policies do not work for everyone is really important and useful research. I am not sure that genotype is the best method for choosing sub populations for targeted interventions.


jasonfletcher
17 January 2013 1:48 PM

@soozaphone - But your language is very precise in the article:

"Poorly conducted experiments where the conclusions overstate the actual findings...'

What CONCLUSIONS in the article overstate the actual findings?

Secondly, if you are unsure you understood the article, perhaps you would want to retract your blog post?

Sadly, Gage left the discussion at that point so we may never know the answer.


UPDATE

The Guardian has given Fletcher the right to reply. He makes the point I make above very neatly, saying: ""Imperfect" research is not the same as "poorly conducted" research. Most research is imperfect and some research is indeed poorly conducted." He concludes by saying:

First, just because a finding can be used for evil by industry does not mean we should censor or question the finding, as implied by Gage's description of my research as a "gift". Some well done science will be used by "evil corporations".

Second, it is truly frustrating and disappointing that such an inflammatory headline ("poorly conducted research") was used in the Guardian to describe my paper without any discussion with me (or any others with relevant expertise as far as I can tell) to clarify the various mistakes and points of contention that Gage makes in her post. The headline was an affront to my professional reputation as a researcher and the post was an affront to the perceived quality of the Guardian.

Share

Tweet