Over-reported studies

As reported by the Express, the Guardian and the BBC, a study has been published which looks at the gap between self-reported alcohol consumption and recorded alcohol sales. According to the BBC, which never knowingly under-reports any news about alcohol...

The amount of alcohol consumed in England could be much higher than previously thought, a study suggests.

This is very much in line with the study's press release...

Alcohol consumption could be much higher than previously thought, with more than three quarters of people in England drinking in excess of the recommended daily alcohol limit

Or, as the Express puts it in its irrepressible style...

Booze-fuelled Britain: Now 80% of women are ‘binge drinking’

Anybody who is familiar with these sorts of statistics is well aware that self-reported consumption figures for alcohol (and drugs) significantly under-estimate the true level of consumption. If people only drank what they said they do it would mean that up to half of all the alcohol sold is poured down the drain. This seems rather unlikely. (UPDATE: Did I speak too soon?... Thousands of litres of whisky flushed down drain in Dumbarton.)

The amount that is under-reported can be easily worked out by comparing the total self-reported estimates with the total sales recorded by HMRC. There are a few other factors to confuse things, such as alcohol consumed by those under the age of 15 (a very small amount) and unrecorded booze sold on the black market (not such a small amount), but this basic piece of maths gives us a working estimate.

This particular study assumes that 40 per cent of all alcohol sold goes unreported by drinkers in surveys. This is not a new finding. As the authors of the study note: "International studies have shown that self-reported alcohol consumption only accounts for between 40 and 60 per cent of alcohol sales."

The basic premise of the press release—and the BBC's story—is therefore misleading. It is not true that alcohol consumption "could be much higher than previously thought". The Office for National Statistics, HMRC, the General Lifestyle Survey, the drinks industry and everyone working in the field of alcohol did not 'previously think' that self-reported consumption figures were reliable.

It is precisely because we know they're not reliable that we prefer to look at per capita consumption of 100% alcohol by people aged 15+ based on sales. Regular readers will know that these figures show a large decline in consumption in recent years—from 11.5 litres in 2004 to 10.0 litres in 2011.

So why do we bother asking people about their consumption if they're just going to lie? Partly it's because it gives us a rough idea of drinking habits. If we factor in the under-reporting we can still get an idea of who's a heavy drinker, moderate drinker or light drinking. And partly it helps us to see trends over time. People might lie (or forget) about how much they drink, but the level of inaccuracy should stay fairly constant, and so we can see how things change over the years—as in the graph below.

You'll note the change of methodology that the Office for National Statistics introduced in 2006. This was a further attempt to improve estimates based on self-reported consumption. In this instance, the ONS assumed larger glass sizes and stronger beer and wine. Despite this, the trend is still downward.

In short, there is nothing new or exciting about saying that sales figures are more reliable than self-reported estimates. Nor is there anything insightful about this new study which employs such a crude, back-of-a-fag-packet methodology that the authors should be commended for managing to stretch it out over six pages.

In short, they take the 40 per cent of alcohol that is missing from the self-reported data and spread it evenly across the drinking population. Or, to put it still more simply, they assume that everybody drinks 40 per cent more than they say they do.

Because the infamous 'guidelines' are set so low (3 units a day for a woman and 4 units for a man), adding 40 per cent immediately throws millions of people into the 'unhealthy drinking' category. And so, voila!, Britain's binge-drinking crisis is even worse than was previously thought.

Er, not quite. First of all, drinking more than the 'guideline' in a day is not 'binge-drinking', even by the debased definition of that term. Aside from the fact that the guidelines are notoriously evidence-free, they apply to weekly drinking of 14/21 units. The splitting into daily 'limits' was a later amendment by government and, even then, the advice is to not regularly drink more than 3 or 4 units in a day. The definition of 'binge-drinking', on the other hand, is more than 8 units in a single session for a man or more than 6 units for a woman.

The headline in the Express is therefore simply wrong, but what about the study itself? The authors concede that the UK government advice is "not to regularly exceed daily limits of 3-4 alcohol units" (my emphasis), but they nevertheless focus on the heaviest drinking day as if exceeding these limits on an irregular basis was a health concern.

But the big problem with the study is the assumption that everyone under-estimates their alcohol consumption equally. Based on this assumption, the authors conclude that "the daily limits effectively become 2.4 units for men and 1.8 units for women." This doesn't give an honest women much of a chance or not being a 'hazardous drinker' (if, by that term, we mean someone who drinks more than the risible guidelines dictate). Since there are two units in a pint of lager or a reasonable sized glass of wine, having just one drink once a week puts them into the hazardous category once their supposed lying or forgetfulness is factored in.

It is, however, unrealistic to assume that someone who drinks two or three units a day is as likely to misreport their consumption as someone who drinks twenty units a day. Even in these puritanical times, light drinkers have no reason to be ashamed of their alcohol consumption and therefore have little incentive to lie. Moreover, those kind of quantities are not enough to affect the memory. Someone who gets smashed every night is obviously more likely to forget how much they've consumed than someone who has one pint after work or a glass of sherry before dinner.

These observations fall under the banner of "common sense" and therefore have little appeal to policy-oriented epidemiologists. In fairness, the authors say that they made an alternative calculation in which heavy drinkers under-report more than light drinkers, but they do not bother to show the results of this, nor do they discuss it in any detail. Instead they focus on an unrealistic scenario in which everybody lies and forgets to the same extent.

Another feature of the study is that it looks at data from 2008 when per capita alcohol consumption was 8 per cent higher than it is today. The ironic result is that a study which wants to draw attention to under-reporting of alcohol consumption ends up over-reporting it.

Ultimately, the most reliable way to measure alcohol consumption remains per capita figures based on sales. By this measure, the UK consumed 10.0 litres per adult in 2011. In 1979 it was 9.8 litres. In 1997 it was 10.0 litres. So, please, would everybody just calm down?


UPDATE:

Tim Worstall makes a good point about what the misreporting of alcohol consumption means for assumed rates of harm. He also leads me to the woeful Telegraph report which adds a whole new layer of wrong to the story... 

In addition, the research suggests the ‘average’ drinker is actually knocking back at least the weekly limit, week-in week-out, and probably more.

Really? Is that what it says?

Oh no, it doesn't say that at all...

Sir Richard Thompson, president of the Royal College of Physicians, said the study estimated 44 per cent of men and 31 per cent of women were exceeding weekly alcohol consumption guidelines.

Interesting definition of 'average' they have at the Telegraph.

Share

Tweet