Thursday, 16 December 2010

Is this the love child of Glantz and Gilmore?

The other day I was thinking of running a worst-junk-science-of-the-year poll. Thank God I bided my time otherwise I would have missed the chance to nominate this beauty.

Isle of Man smoking ban 'cuts heart attacks'

A ban on smoking in public places has reduced heart attack admissions, according to research commissioned by the Isle of Man's Department of Health.

The department has compared admissions in the two years prior to introduction of the ban on 30 March 2008 and the two years since.

It discovered that the number of men over 55 admitted for heart attacks had dropped since the ban.

But if we take a look at the 'study' (unpublished and not peer-reviewed, not that that makes a lot of difference these days), a very different picture emerges:




Do my eyes deceive me or does this graph show that there significantly more heart attacks after the smoking ban?

They don't and there were. In the 23 months before the smoking ban, there were 109 heart attack admissions, or 4.7 per month. In the 23 months after the smoking ban, there were 153 heart attacks, or 6.65 per month.In what universe does this count as a drop in heart attacks?

In the crazy world of tobacco control, that's where. Note the regression lines, designed to take your eye off what is actually happening. Note how the second half of the graph has a line that is driven down by the lowish figure for the last month shown (since the next month needed to make it a full two years has mysteriously gone missing).

This is a method taken straight out of the Anna Gilmore's box of tricks, with a dash of Glantz's Helena magic thrown in for good measure (small community, inaccessible hospital records, data mining etc.). If there isn't a drop in heart attacks, you simply 'predict' how many would have occurred if the smoking ban hadn't come in and make sure your prediction is higher than the real number. And before you know it the BBC will be falling over itself to report that "a ban on smoking in public places has reduced heart attack admissions" and the New England Journal of Medicine will be beating a path to your door.

And the feeble effort shown above is the best this researcher—a maths student at Rutherford Polytechnic the University of Northumbria—could conjure up. The graph that shows all heart attack admissions, (ie. the relevant, non-cherry-picked data set) is even less compelling.





Notice that before the ban, there were usually fewer than ten heart attack admissions. Notice, too, that after the ban the rate was usually well above ten. And, of course, there were more heart attacks in total after the ban than before it. And, as the flat black line shows, the monthly rate of admissions did not go down one bit in the nigh-on two years after the ban.

But you're not supposed to look at any of that. Instead, you are invited to look at the upwards line in the pre-ban period and assume that the rate would have continued rising, even though that line only goes up because of a big jump (by Isle of Man standards) to 14 cases shortly before the ban. Nor are you supposed to notice that any responsible statistician would identify that unusual leap as a statistical artifact. The fact that more than two-thirds of the data points are below the regression shows that it's being contorted by an outlier.

It's truly unbelievable that this sort of stuff gets taken seriously. Or it would be if it didn't happen every few months. This is a world where a flat line equals a decline, and a 50% increase in heart attacks equals a reduction in heart attacks.

In a year that has seen fierce competition for the title, Ms Howda Jwad of Northumbria University—for it is she—may just have clinched the inaugural World's Worst Junk Science Award in the dying days of the year. Glantz, Pell, Gilmore, Winickoff—it's time to up your game.


Thanks to Brian Bond for the tip

8 comments:

  1. Eh.. Was SG an abbreviation for Surgeon General or for Stanton Glantz?

    Doesn't matter, the result is the same...

    ReplyDelete
  2. Evidently, blindness has increased since implementation of the smoking ban too.

    But our heart and soul can't be decieved and the stress caused by an overregulated society and economic woes will only increase the rate of heart attack.

    It looks too, as if heart attack increases around November and December, which are high-stress times when it would be nice to unwind at a pub with a little cigarette and a pint.

    I don't know if Christmas time is tax time in the U.K., but in the U.S. everyone gets their tax bills about the same time they're trying to figure out how to afford their Christmas shopping.

    It's quite stressful having a tax bill for several thousand dollars which leaves only a couple hundred for gifts for the family and food for the feasts -- while at the same time heating and other extra costs of winter are also in need of payment.

    With all of this stress it would actually lower heart attack if a person were allowed to get away for a couple hours and relax with friends and exhale some of that stress out with a cigarette.

    The best repair for a heart is freedom and time with friends who can help ameliorate the tension -- rather than sitting alone at home in front of the television with a cheap beer.

    Beer and tobacco aren't evil if used respectully and responsibly. They're enablers of communion and thoughtfulness with our fellows, but I guess, some people are jealous of this and want to cut off all access to the things that minister to broken and stressed hearts. It breaks my heart.

    ReplyDelete
  3. Chris, here’s your critical mistake. When looking at the graph your head needs to be tilted 45° to the right, i.e., “statistical correction” (closing your left eye substantially will also help. In fact, closing both eyes should work perfectly).

    Howda was top of the one-student class - Mathematics for Profit: Facts are Overrated. With this gem that clearly demonstrates a mastery of Public Health® “ethics” and an entrepreneurial mindset, there’s no doubt that she’ll soon be invited to some TC conference to collect one of the many awards they bestow on each other. She might even meet the heroes of haughtiness, Glantz and Pell.

    ReplyDelete
  4. There are plenty of flaws in reasoning behind this method of fitting separate regression lines to different sections of the data, but what invalidates it completely is the lack of a control population. From memory I think even Gilmore acknowledged that in the "instant heart attack" paper to which you referred. Why it was then published, I don't know.
    JB

    ReplyDelete
  5. JB "what invalidates it completely is the lack of a control population"

    No, what invalidates it completely is its total failure to demonstrate any mathematical and statistical competence. It is an exercise in deceit, pure and simple.

    From a statistical perspective, it doesn't even get as far as needing a control population - as they are measuring nothing that can sensibly be 'controlled' for.

    As far as comparing two regression lines goes, even if it were a valid technique to use, the conclusion is completely dishonest. This is what is said in the paper:

    "Before the smoke free legislation was introduced, myocardial infarctions episodes were increasing at a rate of 0.23 per month. Which denotes that roughly every 4 months, there would be on average one more patient admitted with myocardial infarction in the Isle of Man than the previous 4 months? In the 2 years after the legislation was put in place, the results show that there was no longer an average increase. This decrease in inclination, 22.5%, is significantly different (p-value 0.04)"

    Which of course will sound impressive to those who are fooled by the "P<0.05" mantra into thinking that it means anything at all. However...

    I repeated the "All patients" linear regression calculations, and the resulting R-squared values (strangely not reported in the paper) for the two lines were 0.15 (pre-ban) and 0.00 (post-ban).

    R-squared is a residual error statistic that indicates how good a fit the line is to the data points. It has a value range between 0 (no fit) and 1 (perfect fit). Some take the R value (square root) to mean the 'probability of fit', which isn't entirely true, but close.

    Now 0.15 shows a pretty pathetic fit, but 0.00 shows no fit at all!

    In other words, it is totally dishonest to claim that there was a statistically signifant difference between the two regression lines, since neither of them was a statistically significant measure of its respective trend in the first place. You can't subtract one wrong from another wrong and claim that the answer is a right!

    It all goes to show that Public Health practitioners are all too often ignorant in the use of statistical methods, but are entirely dependent on them for their existence - especially on the dreaded 'P<0.05' crutch!

    People will choose to believe them - they are 'Doctors' after all!

    ReplyDelete
  6. As a footnote to the above regression calculations.

    I also carried out a regression calculation over the whole data range (4 years), after first calculating 12-point moving averages (to smooth out seasonal variability). This time the R-squared value was a whopping 0.8, which translates (roughly) to a 90% confidence that the line fits the points.

    In other words there is a reliable straight line trend over the whole 4 years, whilst there isn't in either of the two 2-year periods either side of the ban.

    In other words, it is quite obvious that nothing changed after March 2008, so the smoking ban's headline statistical achievement was actually...

    ...nothing!

    Go figure!

    ReplyDelete
  7. BB: “You can't subtract one wrong from another wrong and claim that the answer is a right!”

    If I may clarify a major problem here.

    Most people are familiar with the standard or usual arithmetic functions, e.g., addition, division, etc. Most, however, are unaware that Public Health® Arithmetik includes a number of additional "functions". For example, breaking the data into two lots – as has been done - is called “subdivision”. Subtracting a wrong from a wrong is called “estrelation”. Performing estrelation to produce the right answer is called “strangulation”: “Strangulation” involves the function of suspending or canceling other rules of arithmetic inference. Therefore, having performed estrelation on the data, strangulation defines the result as “right”.

    Hope this helps :)

    ReplyDelete
  8. In other words:

    "Torture the data until it confesses!"

    Yes that explains it!

    ReplyDelete

Comments are only moderated after 14 days.