Peer review, The Spirit Level and the appeal to authority

Brendan O'Neil has written a great piece at Spiked on the subject of peer review (with a few quotes from me). The article was inspired by the ongoing debate about The Spirit Level, and its authors attempt to confine it to peer reviewed journals.

The irony is that the authors owe their success in spreading their political message to writing a book for the mass market. This book was not peer-reviewed (obviously - although it refers to peer-reviewed studies, as mine does), and many of the statements they have made in the mainstream media would never get past peer review. Having engaged with the real world, they seem keen to withdraw to the confines of academia at the first sign of trouble.

Since it was published last year, The Spirit Level – Kate Pickett and Richard Wilkinson’s book on why equal societies do better than unequal ones – has become a sparkplug for heated, testy debate. Not one, not two, but three pamphlet-length critiques of it have been published, while others have rushed to man the book’s intellectual barricades (‘This book’s inconvenient truths must be faced’, said a Guardian editorial).

Yet now Pickett and Wilkinson have imposed an extraordinary condition on future debate about their book. Because much of the criticism of The Spirit Level has consisted of ‘unsubstantiated claims made for political purposes’ (in their view), ‘all future debate should take place in peer-reviewed journals’, they decree.

Wow. In one fell swoop they have painted any criticism of their book that appears in non-peer-reviewed journals as somehow illegitimate. They snootily say that ‘none of [the] critiques are peer-reviewed’ and announce that from now on they’ll only engage in discussions that ‘take place in peer-reviewed journals’. So any peep of a critique that appears in a newspaper, a book published by a publishing house that doesn’t do peer review, a non-academic magazine, an online magazine, a blog or a radio show – never mind those criticisms aired in sweaty seminar rooms, bars or on park benches – is unworthy because it hasn’t been stamped with that modern-day mark of decency, that indicator of seriousness, that licence which proves you’re a Person Worth Listening To: the two magic words ‘Peer Reviewed.’

This is not to say that peer review isn't important. It is clearly better than the alternative. The problem, as Brendan says, is more to do with public perception and the inclination of some researchers to appeal to authority.

Being peer-reviewed no longer simply means that you wrote an academic report that was considered by other academics to be serious enough for publication – it means you possess the truth, Pure Knowledge, elevated insights that are not available to mere mortals who have not been PR’d...

To be peer-reviewed is to have the right to speak publicly on important matters – to be non-peer-reviewed makes you immediately untrustworthy, a bit of an intellectual charlatan, possibly even suspect in your motives.

The Spirit Level provides a useful example here. It owes much of its success to the simplicity of its evidence, but this simplicity—and the fact that all the data used are publicly available—makes it easy for anybody to check. It doesn't require a PhD in social epidemiology to take data from the United Nations website and plot it on a graph. And, as I show in my most recent book, the evidence simply doesn't stand up (and, if you want to appeal to authority, several distinguished professors have said the same thing).

Remember, we're not talking nuclear physics here. We're talking about the most basic statistical fact-checking of the most basic type of epidemiological study. If a scientist tells me there are black holes in the Universe I'm happy to take his word for it. If a social epidemiologist tells me that life expectancy is higher in Cuba than it is in America, I'm going to check it. And if she's wrong I'm going to say so.

As Brendan concludes:

It is ironic that Pickett and Wilkinson, so very keen on the idea of equality, don’t like the idea of an equal right to speak and critique. In this area of life, their attitude is: ‘If you’ve been peer-reviewed, let’s talk. If not? Screw you.’

Go read.


UPDATE

More Spirit Level-related discussion at The Taxpayers' Alliance. There is also an exhaustive list of links covering the debate of the last month on the right-hand side of The Spirit Level Delusion website.