Thursday 11 November 2010

Pursue the truth wherever it leads

An anonymous commenter has alerted me to an article in the Financial Times calling for an end to "propaganda disguised as science". Most aptly, it cites the heart miracles of England and Scotland as cases in point:

What of the claim that the smoking ban reduced the incidence of what lay people call heart attacks in England by 2.4 per cent? This follows the more extravagant claim that the similar ban in Scotland had an effect of 17 per cent. The evidence for the former proposal is weak, and the latter claim is implausible. The incidence of heart attacks is falling steadily in both countries by around 3 to 4 per cent per year, with fluctuations but no obvious break in this trend. Better health education and reduction in smoking are part of the explanation.

In the year after the English ban, the fall in reported cases was 4.3 per cent. The effect claimed for the ban is the difference between the actual figure and that predicted by a model constructed by the researchers. In Scotland there was an above average fall in heart attacks – though nearer 7 per cent than 17 per cent – immediately after the ban, but this seems to have been a freak, since it was reversed the following year. The researchers derived their conclusions from a specially compiled dataset constructed over a limited period.

All of which will be familiar to regular readers of Spiked and of this blog (eg. here and here).

The studies I have cited are carefully referenced and use advanced statistical techniques. But sophistication of method is used to torture data to reveal conclusions that do not obviously follow from them, but which fit either the researchers’ preconceptions or the sponsor’s policy objectives, or both.

Indeed so. "Advanced statistical techniques" cover a multitude of sins. Far from providing empirical evidence, they offer unlimited scope for manipulation and require blind faith in return—the classic case being Anna Gilmore's risible computer model (which claims to be able to predict the annual number of heart attacks based on nothing but average air temperature and knowing which day of the week Christmas falls on).

Bad arguments do not necessarily invalidate the causes in which they are deployed. People should not drink and drive. Smoking is unpleasant and perhaps harmful to non-smokers. But these observations do not justify blurring the distinction between genuine scientific analysis and propaganda disguised as science. Policy should follow evidence, not evidence policy. It is time to reassert the principle that research must pursue the truth wherever it leads: the principle on which the social and economic progress of the past few centuries has depended.

Well said, that man. That man being John Kay, professor of economics at the London Business School, with whom I am already familiar since he is one of several serious academics to have criticised The Spirit Level—another fine example of propaganda disguised as science. He said of that book:

"The evidence presented in the book is mostly a series of scatter diagrams with a regression line drawn through them. No data is provided on the estimated equations, or on relevant statistical tests. If you remove the bold lines from the diagram, the pattern of points mostly looks random, and the data dominated by a few outliers."

As usual, when presented with substantive criticism, The Spirit Level's authors resorted to feeble put-downs "He didn't read the book thoroughly, obviously," Kate Pickett said of the Financial Times' book reviewer (on the BBC's More or Less programme).

And they're at it again today in the New Statesman, plumbing new depths of ad hominem:

In their book Merchants of Doubt, the American academics Naomi Oreskes and Erik Conway suggest that the defence of a kind of free-market fundamentalism is the most plausible explanation of why the same individuals and institutions are often involved in attacks on research in areas as diverse as tobacco control and the evidence on climate change. As well as defending the free market, they see themselves as countering tendencies to big government and protecting democracy. The same beliefs are likely to guide the attacks on the evidence of inequality's socially damaging effects.

This is an incoherent, self-aggrandising and paranoid argument. Incoherent because The Spirit Level has been criticised from people across the political spectrum, including John Kay who is "sympathetic to its basic stance". It is self-aggrandising because it equates their own social science with the physical sciences, and paranoid because even if such a cadre of powerful 'free-market fundamentalists" (ie. anyone to the right of them) existed, they would scarcely bother to direct their fire at a soft target like The Spirit Level.

To tie this fantasy together with the phrase "the same beliefs are likely to guide attacks [on us]..." is to employ the weasliest of weasel words. But now that John Kay has implicitly criticised tobacco control (albeit in a very roundabout way), Wilkinson and Pickett can rest easy in the renewed belief that they are the subject of an organised rightwing posse, rather than a few individuals who bothered to test their claims.

Speaking of whom, while I was incommunicado I received an e-mail from Dr. Mick O'Connell at University College Dublin, who has also taken an interest in Wilkinson and Pickett's work ("I purchased their book with high hopes of a convincing read, and the disappointment in the content propelled me to write a critique.") His working paper is titled 'Affluence versus Equality? A critique of Wilkinson and Pickett’s book The Spirit Level' and can be downloaded here. This is a sample:

Part of the implicit Spirit Level thesis is an obsession with consumerism – the latter authors pinpoint “our consumerism [and] addiction to shopping and spending” (p. 221). One has to wonder whether these people have tried high-street shopping recently? Am I alone in thinking it borders on the hellish?

Shopping, like air travel, perhaps once conferred a kind of jetset mystique, a bit like cheesy-pineapples on a stick. But now the negotiation of a busy shopping mall, or a thronged airport is about as glamorous as, well, cheesy-pineapple appetizers. Most of us, I contend, dislike shopping, but do it because we have to, because the kids demand Cheerios, the baby needs Pampers and the cat gets pretty mean without her Whiskas.

Of course, there is also a small but determined subset of people who still like air travel or shopping. But these people should be seen for what they are - strange eccentrics, not a substantial social trend. How do we know this? Because shops have to spend a fortune advertising just to try to get us through their door, and change their look and design every year to persuade us to stay.

Our ‘addiction’ to brands is actually so slight that as soon as ‘hard discounters’ like Lidl and Aldi expanded beyond their German base into the European grocery market, they enjoyed spectacular success. ‘Hard discounters’ offer a limited product range and a predominance oflow-price generic brands. “Second-class goods make us look like second-class people” argues The Spirit Level, (p. 222) – this is why we’re supposed to be obsessed with luxury goods. Not a bit of it – decent low-priced goods make us look like smart and astute people. Bye-bye Whiskas, hallo Katzen-Imbiss!


Dick Puddlecote said...

I did kinda mention John Kay myself, but Belinda was first. :)

BenRymer said...

John Kay's FT article is not nearly as critical as you make out. Indeed, his reservations about the wide-ranging use of cross-section analysis do not mean he thinks there is *no* link between the phenomenon of inequality and social ills, only that more work -- particularly time series studies -- are needed. He offers a thoughtful critique while recognising that there is no economic argument for the levels of inequality seen in the Anglosphere.

Christopher Snowdon said...

What did I make out? I just said he criticised it, which he did.