Wednesday 9 June 2010

Latest smoking ban/heart attack study is pure junk science


Last September, I mentioned an article in The Sunday Times which signalled that a new smoking ban/heart attack study was marching with ill-deserved confidence in our direction.

The ban on public smoking has caused a fall in heart attack rates of about 10%, a study has found.

Researchers commissioned by the Department of Health have found a far sharper fall than they had expected in the number of heart attacks in England in the year after the ban was imposed in July 2007.

This was truly remarkable news because, as regular readers of this blog and Dr Siegel's blog will know, NHS hospital admissions data clearly show that incidence of acute myocardial infarction (AMI) has continued to decline at the same rate as before the smoking ban.




This posed a problem for Dr Anna Gilmore, the ASH board member who was charged with turning this wholly unexceptional data into a new 'heart miracle'. Earlier junk studies from Scotland (Pell et al., 2008) and Helena, Montana (Sargent, Shephard & Glantz et al., 2004) had claimed a fall in heart attacks of 17% and 40% respectively. Pell did it by ignoring the NHS data set, picking an unusual time-frame and using a very idiosyncratic definition of a heart attack. Glantz did it by simply finding an unusual blip in a very small community.

But all the signs were that Gilmore would be using the full hospital admissions data record for England, which was already available online and which showed that heart attack admissions were falling by less than 5% every year before and after the smoking ban. How would she do it?

In January, I showed England's heart attack data for the second year of the smoking ban. This only confirmed the lack of any effect, and I even apologised for telling you about yet another heart miracle no-show.

The story is always the same, and I apologise for boring readers with one null study after another. But spare a thought for Dr Anna Gilmore, whose job it is to turn this mundane data into a newsworthy study showing that the smoking ban has saved thousands of lives. She may be working on it at this very moment.

The one limitation of the data used by myself and Dr Siegel was that they showed AMI admissions in the financial year (April to March). The smoking ban started on July 1 2007. We still had data for 21 months after the smoking ban, which was more than enough to show that there was no nose-dive in admissions. Still, it would have been better to have the data from July to June.

And now we do, because Anna Gilmore has published her long awaited paper in the British Medical Journal. This is what her figures show:



As expected, there are small discrepancies between the two data sets. (There tends to be slightly more admissions in the April-March set because the timeline goes further back, and Gilmore only shows one year after the ban.) But the story is the same in each—the rate of decline was the same after the ban as it was before. 

If you want specifics, here are the figures Gilmore uses (table 1 of the study)...

Emergency AMI admissions in English hospitals

2002/03: 61,498

2003/04: 60,680 (a fall of 1.33%)

2004/05: 58,803 (a fall of 3.1%)

2005/06: 55,752 (a fall of 5.19%)

2006/07: 53,964 (a fall of 3.21%)

2007/08: 51,664 (a fall of 4.26%)


As you can see, the decline in admissions in the year after the smoking ban was larger than the year before but smaller than the year before that. In fact, the average in the previous two years was 4.2%—almost exactly what it was in the year after the ban (4.26%).

Faced with this evidence, from a nation of 49 million people, what else can you do but hold up your hands and admit that smoking bans have no perceivable effect on a nation's heart attack rate?

So what's Gilmore's conclusion?

We therefore conclude that the implementation of smoke-free public places is associated with significant reductions in hospital admissions for myocardial infarction

Huh?

And the accompanying press release reads:

Smokefree legislation linked to drop in admissions for heart attacks

A 2.4 percent drop in the number of emergency admissions to hospital for a heart attack has been observed following the implementation of smokefree legislation in England, researchers from the University of Bath’s Tobacco Control Research Group have found.

The legislation was introduced on 1 July 2007 and this study, funded by the Department of Health and published this week in the British Medical Journal, is the first to evaluate its impact on heart attacks.

The team, led by Dr Anna Gilmore, Director of the Tobacco Control Research Group, part of the UK Centre for Tobacco Control Studies, found there were 1200 fewer emergency hospital admissions for myocardial infarction, commonly known as heart attacks, in the year after the legislation was introduced...

Dr Gilmore said: “Given the large number of heart attacks in this country each year, even a relatively small reduction has important public health benefits. This study provides further evidence of the benefits of smokefree legislation.”

Huh?!?!! 

This is a joke, right?

I'm afraid she's serious. Desperate to spin gold from straw, Gilmore throws everything she can at the data. She makes adjustments for surface air temperature, flu seasons, population size and Christmas holidays but she does not address more significant factors like smoking status, diet, exercise or stress (that's not her fault—the NHS does not have this data—but let's not pretend she is isolating smoking in public as the sole uncontrolled risk factor). 

None of these calculations are shown or can be verified, but these manipulated data are then fed into a series of computer programs to arrive at an adjusted average daily admissions figure. Although Gilmore uses the July-June figures for all pre-ban years, she goes up to September 2008 for her post-ban year, thereby leaving in July and August, which is when the AMI rate is invariably at its lowest. It's baffling and more than a little suspicious. Why not go up to June 2009? Or at least stop at June 2008?

From this almost incomprehensible mass of heavily adjusted data, she arrives at the figure of 2.4% mentioned above. This 2.4% is the supposed decline in AMI admissions that she directly attributes to the smoking ban. Since the total decline was only 4.26%, this means that the smoking ban was responsible for more than half of the drop; hence the newsworthy but entirely spurious 'smoking ban prevents 1,200 heart attacks' claim. 

Although the AMI rate had fallen by 3.21% and 5.19% in 2005/06 and 2006/07, we are expected to believe the decline would only have been 1.86% in 2007/08 if the smoking ban had never happened. 

In short, we are expected to believe that there was going to be a smaller than average decline in AMI in 2007/08, and that the smoking ban saved the day. The fact that the decline in AMI was unexceptional in 2007/08 is therefore used as proof that the smoking ban had an exceptional effect! 

This is fairy-tale science. It is sheer statistical manipulation and it is breath-taking in it scope and ambition. But then, as I have said before, it always had to be.

More on this tomorrow, I'm sure...



29 comments:

Anonymous said...

The British Medical Journal is becoming such an outlandish publication, I am looking forward to seeing peer reviewed studies on the benefits of homeopathy and aromatherapy next.

Seriously, don't the clowns running the BMJ appreciate the damage they are doing, not only to their own publication, but to the long-term reputation of British medicine, by publishing this witchcraft?

Anonymous said...

Forgive me if you've already covered this elsewhere, Chris, but what is (really) causing the downward secular trend?

In relation to that question, what is classified as an "AMI admission"? Being admitted for a suspected heart attack is quite a common occurrence, even if a heart attack never actually happened.

The only way to properly collect the data would be to count up all of the patients who were admitted, and then later confirm that they'd suffered an actual heart attack. I was just reading today the story of a woman who didn't know until weeks after that she'd suffered a heart attack. (Instapundit Glenn Reynold's wife, in fact.)

http://drhelen.blogspot.com/2005/10/more-than-you-wanted-to-know-about-my.html

I'm trying desperately to remember what book it was, but I remember reading about new methods being used in evaluating heart patients. More precisely, the problem emergency rooms face in diagnosing heart attacks at the time patients show up in the ER. When hospital beds and staff are in short supply, there's an issue with being overcautious regarding chest pain. The financial costs for unnecessary admissions are high as well. I remember reading that doctors were trying to put new techniques into practice to quickly and properly assess whether or not a patient was actually having a heart attack. The purpose was to decrease the number of unnecessary admissions.

I remember the book now. It was "Blink" by Malcolm Gladwell.

Anyway, I think you can see where I'm going with this. It would seem that collecting data on how many heart attack admissions are actual heart attacks would be no big deal. And perhaps it isn't. But if we didn't have a warranted degree of skepticism about medical data collection methods, we wouldn't be here, of course.

I'm always a bit suspicious of even the secular trend in lower heart attack admissions. The supposed explanation is an increase in healthy lifestyles, etc. That seems to fit the narrative a bit too well.

So, for what it's worth, I think it's possible that the secular trend in lower heart attack admissions may not even reflect actual heart attack admissions, but possibly lower rates in suspected heart attack admissions due to more efficient diagnostic techniques.

Then again, I may be completely wrong. It occurs to me, though.

-WS

Anonymous said...

It is going to be interesting when heart attack rates go up again isn't it? As the stresses imposed by an economic downturn bite, heart attack rates will rise.

The point made about changes in diagnostic criteria is well taken, you can even further distort the figures by using MI (myocardial infarction) deaths as more people survive their first MI than ever before due to improvements in both active treatments after the event and the use of a wide range of therapies in patients with CVD that both reduce the risk of MI and the severity of the heart attack when it occurs. Interesting times.

Mark Wadsworth said...

I still think it's a massive improvement on the original propaganda that heart attacks were cut by 30% or whatever the made-up figure was a year ago.

Green E-Cigarette said...

There people lying on hospital beds, hooked up to machines, on respirators fighting for their life- and you need proof?!?!?!?

Anonymous said...

Green E-Cigarette says "People lying in hospital beds hooked up to respirators." Yep if someone says to me that they know WHY the people are there they have to have proof.

Anonymous said...

And yet again the BBC (Believe in Bullshit Corporation)give this headline news! It's enough to make one wrap their TV licence round and wad of tobacco and light it. At least the second-hand smoke so generated would be politically correct!

Anonymous said...

I don't know whether anyone else noted the little hostage to fortune in the report: Apparently the 'smoking ban' saved 8.5 million pounds to the NHS. Set this aside against the lost revenue from pubs and the rest of the costs of the smoking ban and.......

oh dear.

Well health concerns have to be more important than money, don't they? That's why we chose the more expensive but safer braking system after the Barnet train crash...

oh we didn't.

Er.......

Anonymous said...

Pell is on the front page of the Mail this morning with a claim that births at 39 weeks rather than 40 carry a 1.09 RR of some mental degradation in the child - can't remember what exactly. Interestingly she said that mothers shouldn't worry as the risk to the individual is small...yet the Public Health implications are substantial. A pretty fair definition of fascism. How does the risk compare with that of walking into a room, furnished with a powerful extraction system, for 10 minutes each night to collect glasses?

Christopher Snowdon said...

WS,

Lifestyle changes have some impact on AMI rates going down, I'm sure, and perhaps a large impact. Falling smoking rates play a part, although the 20th century decline does not closely follow smoking rates. Its notable that the obesity 'epidemic' has not stopped, let alone reversed, the decline. James Le Fanu has suggested that infection has played a key part - the rise and fall of coronary heart disease follows the pattern of a typical infectious epidemic, he says. Then you have statins, diet, stress and-as you say-diagnosis.

All in all, there are more unknowns than knowns, and in places like Australia rates of AMI have been increasing for years. With so many risk factors at work—many of which we don't fully understand—it is absurd for Gilmore to claim to know what the AMI rate would have been in a parallel universe without a smoking ban. Nor does she provide any evidence for her assertion. She gives this figure of 2.4% but never gives us even a glimpse at how this was arrived at. We are supposed to take it on trust, I suppose. I'm sorry, but that's not how it works. Extraordinary claims require extraordinary evidence and since the raw data shows no effect from the ban, the onus is on her to provide some. This she never does, and without it, her figure is just speculation.

Unknown said...

You've been noticed on the Telegraph Chris.

Dr Evil said...

She says 2.4% is a significant drop. It isn't. An order of magnitude would be significant. Since the trend is maintained after the ban then the ban had no effect. Gilmore is talkinmg rubbish and doing real science no favours. Does she not know of impartiality? also the peer reviewers must be either myopic, useless or biased.

Dr Evil said...

Ah, why the downward trend you ask? well, better diagnosis and control of diabetes, statins being used routinely for preventing cardiovascular disease, better diagnosis of risk factors in general and people making useful lifestyle choices. It all adds up. It will flatten out eventually. Another proof if the ban continues that it has no effect.

Michael J. McFadden said...

Well done as always Chris. I loved the comment you wrote a year ago about Anna probably being busily at work to find a way around the numbers. heh... ya called that one right, didn't ya? :>

I have a question for you: Any idea how many of these abominations you've analyzed so far and how many of them were shown to BE abominations upon analysis? Just sticking to the "heart attack reduction" studies that have claimed a decline.

:?
Michael

Anonymous said...

Chris, you ask

"Although Gilmore uses the July-June figures for all pre-ban years, she goes up to September 2008 for her post-ban year, thereby leaving in July and August, which is when the AMI rate is invariably at its lowest. It's baffling and more than a little suspicious. Why not go up to June 2009? Or at least stop at June 2008?"

This does appear at first sight suspicious, although, if the analysis has been done correctly, it shouldn't matter that Aug and Sept have the lowest admissions. What would matter is if Aug and Sept 08 had low admissions compared to the same months in other years. In her paper she gives a reference (ref 44) for the type of model she uses - a segmented regression model. In this paper (Wagner et al), it is suggested that there should be 12 data points before the intervention (ban) and 12 after, so that seasonal variation can be evaluated.
As you might imagine, this can be better done with 24 points rather than 15 after the intervention, as we then have two complete cycles. As long term trend and temperature were included in the model, this seems all the more crucial. Why this was not done if the data was available is a mystery to me.
If the computer code and the exact model was available, this could be investigated, but otherwise we'll never know.

Fredrik Eich said...

"We retained only the first episode (known as the admission
episode) of a patient’s stay in hospital because these
were more likely to reflect myocardial infarction
events that had occurred outside of hospital rather
than those occurring as complications of hospital treatment."
It's an interesting exclusion and for a plausible reason. But I would be interested to see if there was a post smoking ban trend break in the excluded numbers - maybe ones occurring as complications of hospital smoking bans.

Anonymous said...

I believe we know the figures for the year after that mentioned in this "report" though, don't we? I seem to remember that they showed an increase in admissions, the first in 30 years. Will they report that, I wonder?

Then there is the whole causal thing, of course. Cliff Richard didn't have a hit record last decade, for the first time in 50 years. We also saw a massive rise in smoking bans. Therefore, with a handy graph I can prove that the absence of Cliff from the charts causes smoking bans!

Then of course, as you say, the sheer mangling of statistics is amazing. Not mentioning that it is simply the continuation of a 30 year ongoing trend is just fraud, pure and simple. But what really gets me is that we can expect this kind of magic from Glantz and his ilk - this sort of pseudo-junk is what they do. But why are all the reporters so gullible? Do ANY of them even have ANY basic scientific training?

And as for the BMJ? If the IPCC reports using "references" from magazines and Greenpeace press releases didn't put peer review into doubt this sort of rubbish must. Don't they realise people can find the original figures and see what's actually happening? Don't they realise that if they follow this line they run the risk of discrediting the scientific method itself?

It shows how lacking the peer review process is. If we could ever get tobacco control in the dock they'd be demolished in minutes. Yet they can continue to pump this rubbish out in journals year in, year out with all criticism seemingly being ignored. It actually frightens me, to be honest. We are a very short step from where Nazi scientists were when they measured Jews' heads and found them to be more stupid. Obviously claptrap yet undoubtedly reported as fact by Hitler's media of the day, just as this is being reported now.

When will there be an Enlightenment?

Anonymous said...

Anon 07.01:

No doubt any increase will be reported with a conspicuous silence, accompanied only by the gentle sound of the carpet being lifted and the broom being used deftly to good effect ……..

Michael J. McFadden said...

Anonymous wrote, "Why this was not done if the data was available is a mystery to me."

My guess is that it was done to fulfill the promises made in the grant commitment. Specially selected data points are nothing at all new in the world of antismoking research.

- MJM

Iro said...

Yea !!! with all these miracles happening, look at all the money our socialized medicine will save on statins prescribed to prevent heart attacks ! Who needs Lipitor, Crestor, Zocor....when you have smoking bans doing the same job for 0 cost?

Anonymous said...

Below is a comment from below the Times article. It looks interesting.

"keith dutton wrote:
I do not know why my last post did not get past the monitor - no foul language or mistruths! I'll try again.
I agree wholeheartedly with Penny Webster-Brown on cost and statistical trends.
A study some years ago in the States compared the incidence of diagnosed heart disease over a 20 year period split between the sexes. The ratio remained virtuallu unchanged over this period, whilst the number of femal smokers relative to men more than doubled. Taken in isolation these figures would indicate that there is no link whatsoever between smoking and heart disease."

Anonymous said...

The American Medical mafia confirms time and time again that 450,000 die every year by smoking related diseases by inclusion of the numerous heart miracles which must have occurred, the numbers continue to rise obviously. In spite of what you might have read.

In reflection of the 1960 population when 120 million 50% prevalence included 60 million smokers and the all cause mortality rate was 900,000.

Today 300 million and 20% prevalence leaves us with a constant number at 60 million which varied little in the interim. Today there are 2.3 M all cause mortalities so the mortality rate has not changed all that much either.

Would it be credible to state that 50% of all cause mortality in 1960 was caused by smoking?

Looking back; coal was the norm as heating fuel, Lead in gasoline, Asbestos everywhere, Job safety was primarily the word DUCK, no product safety standards, No emission standards no PELS and a host of supposedly progressive changes all aimed at reducing all cause mortality.

450,000/900,000 = 50%

50% was not remotely credible then.

Why is the identical claim so credible today?

I find the one useful device of a non biased observer, in separating the politicians or advocates, from the postings of legitimate ethical voices, is the repetitive talking points we see among the many public discussion forums. The focus group tested chants you hear over and over until your ears bleed.

It makes it easy to identify those with an opinion from those with an agenda.

The ad agencies will tell you as Hitler once did; "If you repeat a lie long enough it becomes the truth" There are limitations to that truth, found with over saturation. The annoyance factor leads to investigation, if for no other reason, but to silence the cult like chanting of people claiming to be professionals, scientists and experts.

The technocrats who consider the rest of us in a global view, as ignorant children, who's personal autonomy is now solely in their self entitled care.

The simplistic view of technocratic statistics and "irrefutable scientific fact" production, entitles a more ignorant view, in an absolute failure to recognize our individuality and disparities.

Michael Siegel said...

Thanks for bringing this article to my attention, Christopher. I agree with your analysis. The most important thing I think people need to realize is that with a decline as small as 2.5% (let's assume there was an actual decline for now), one MUST rule out the possibility that such a small effect is due to some other factor. The most likely factors, in my opinion, are the increasing use of statins to control cholesterol and better treatment for coronary artery disease, including advanced angioplasty techniques which did not exist just years earlier.

It does baffle me how the study can draw such a definitive conclusion when there isn't even a control group. This is almost unheard of in other areas of science.

But the bottom line, as you point out, is that an analysis of the trends simply shows no change. It's a straight line!

Michael J. McFadden said...

In addition to something as technical as statins, don't overlook something as simple as trans-fats. I don't know if it's the same there as here, but I'd say trans-fats have probably been reduced by a good 80% or more in the American diet over the last five to ten years.

- MJM

Anonymous said...

yepper Michael thats so true and the reason I quit eating at most of the major hamburger joints........french fries cooked in plastic oil tastes like %$%$$!!

Mom and pops get my business especially the smoking ones......ya they still exist.Come visit sometime and I will take you to a few. harleyrider1978
repealthebans@yahoo.com

Belinda said...

Don't know if this has come to your attention yet (sorry have not been able to read the thread in detail) http://www.spectator.co.uk/alexmassie/6069805/smoking-bans--fewer-heart-attacks-up-to-a-point-lord-copper.thtml

includes comment from Ben Goldacre

Anonymous said...

Thanks Belinda. Interesting comment by Goldacre. He doesn't appear to have Delphic status with the Spectator regulars. Interesting comment by Snowdrop claiming ASH sacked an agency regarding air pollution experiment. Perhaps Chris, now a.k.a Massie's mate, could subtly invite Snowdrop to email him with this bombshell.

Ondřej Palkovský said...

I find that statistical approach quite interesting - they actually used some 'false date' to simulate the strength of the test.

However.

1) I would like to see exact same procedure to be tested on the Australian data
2) The results are 300-2100 admissions on 95% probability. You get 300 more heart attacks and the hypothesis is statistically insignificanet
3) Remember! The study should have studied PASSIVE smoking. They DID NOT take into account changes in active smoking - yet you get reports that 400.000 people stopped smoking. Assuming 20% of the coronary admissions are strongly connected to active smoking, this factor alone can easily make more than 300 people thus rendering the hypothesis statistically insignificant
4) There are reports that the cigarette sales dropped subtantially; again as there are huge differences among heavy and light smokers, if many people switched to be light smokers, this could make a difference.

The smoking ban could have easily caused many people to quit active smoking and thus attribute to this drop; however the purpose of these studies is to study passive smoking.

Anonymous said...

The simple fact that reporting etc is
false or not properly researched is
that smoking is now regarded as the new evil by the non smoking zelots.
No doubt this distracts from other far more important issues.
Could this be a "smokescreen" I wonder!