Wednesday, 27 March 2019

Sugar taxes - the state of the evidence


I'm speaking at the Global Sugar Summit today. It's an industry event, the topic is sugar taxes and they've invited arguably the two most fanatical anti-sugar campaigners in the world - Robert Lustig and Graham MacGregor - to argue in favour. Whether they will choose to address the issue or rant at the audience (and me) is an open question, but it's a good opportunity to assess the state of the evidence on sugar taxes in 2019.

Here's the story so far...

Early research came from the USA, where more than 30 states have some form of tax on soda. In 2009, Powell et al. found 'no statistically significant associations between state-level soda taxes and adolescent BMI'.

Similarly, Fletcher et al. (2010) looked at the effect of soda taxes on soda consumption and body weight among minors and found 'a moderate reduction in soft drink consumption' but no impact on weight because 'this reduction in soda consumption is completely offset by increases in consumption of other high-calorie drinks.'

In 2013, Fitts and Vader looked at the impact of soda taxes on adult obesity and, again, found no effect:

Our research does not support the theory that soda taxes have a negative effect on body-mass index.

However, they suggested that this might be because most of the taxes on soda were quite low...

Current soda tax rates range from two percent to 7.25 percent and it’s possible these may not be high enough to affect BMI.

To test this, Fletcher et al. (2015) looked specifically at the largest soda taxes. Alas, the results were much the same. Regardless of whether the taxes were small or large (they ranged up to 12 per cent), there was no impact on body weight or obesity:

Together, our results cast serious doubt on the assumptions that proponents of large soda taxes make on its likely impacts on population weight. Together with evidence of important substitution patterns in response to soda taxes that offset any caloric reductions in soda consumption (), our results suggest that large changes to policy proposals relying on large soda taxes to be a key component in reducing population weight may be required to successfully reduce weight.

In the same year, Colantuoni and Rojas looked at two reasonably substantial soda taxes in Ohio and Maine (5.5 per cent). Their conclusion?

Results suggest that neither sales tax had a statistically significant impact on the consumption of soft drinks.

By 2015, the real world evidence hadn't shaped up too nicely for the soda tax campaigners. Fortunately, several of them had been running their own computer models and the results were published to widespread attention. Researchers like Mike 'God is calling me to work towards the introduction of soft-drink taxes in this country' Rayner and Kelly 'Twinkie Tax' Brownell came up with studies like this and this which promised small but non-trivial health gains if their assumptions were right.

Their main assumption was that soda taxes work, so it was all a bit circular, but from around 2014 a few places experimented with soft drink taxes of around ten per cent. Mexico and Berkeley, California both offered a chance for researchers to study the impact of a substantial soda tax on calorie consumption, weight and health.

In January 2016, a study was published which claimed that Mexico's soda tax had reduced consumption by six per cent in its first year. Newspapers spread the story around the world, but only those who read the study carefully noticed that the authors - among them veteran soda tax campaigner Barry Popkin - were not reporting an actual decline in soda consumption, rather they were comparing the actual rate to a counterfactual that they had created.

Months later, Mexico's National Institute of Public Health admitted that soda consumption had not declined after the tax was implemented (in 2014). In fact, it had gone up slightly.

As keen advocates of the tax, the Institute was anxious to point out that 'it is not possible to make determinations on the usefulness of the tax on sugar sweetened beverages in Mexico during 2015 using raw sales data'. Like Popkin, they claimed that sales would have gone up even more in the absence of the tax. Perhaps they would. It is a hypothesis that is impossible to falsify. But it is clear that consumption did not go down and, therefore, that the tax could not possibly bring about the promised decline in obesity.

Sure enough, the number of obese Mexicans rose by four million between 2012 and 2016.

Popkin returned in 2017 with a study of the Berkeley soda tax which, he claimed, had led to sales falling by 9.6 per cent. This time, the flaw was in plain sight. The 9.6 per cent decline (compared with a counterfactual) related to stores in Berkeley, but the tax encouraged people to do their shopping out of town. Although sales fell by 0.8 fluid ounces per transaction in Berkeley, they rose by 0.7 fluid ounces per transaction in neighbouring areas.

The authors found that self-reported sugary drink consumption' did not change significantly compared to baseline'. Not only was there no statistically significant change in calorie intake from sugary drinks among Berkeley residents, but the authors noted that 'caloric intake of untaxed beverages (milk and other diary-based [sic] beverages) increased.' Energy consumed from sugary drinks fell by a statistically non-significant six calories while energy consumed from dairy drinks rose by a statistically significant 32 calories. In other words, overall calorie intakes from beverages actually increased after the tax was implemented. Great success!

Finally, there is Chile. The coverage of a study of Chile's sugar tax last year followed the by-now familiar pattern of big press release, impressive claim and underwhelming study. This time the claim was that an increase in tax on soft drinks in 2011 had caused sales to 21.6 per cent.

This would have been remarkable if true. The tax on sugary drinks only rose from 13 per cent to 18 per cent while the tax on low/zero-sugar drinks dropped from 13 per cent to 10 per cent.

But it wasn't true, and the researchers were kind enough to show us the sales figures so that we could see it wasn't true.

They also admitted that...

For all soft drinks and for the relevant soft drink subcategories (except the no-tax soft drinks, which show a trend increase), it is hard to detect a clear overall time trend based on pure visual inspection alone. While the peaks in the data certainly become less pronounced over time, so have some of the troughs. It is equally difficult to discern an obvious pre- versus post-tax pattern in any of the categories.

So how did they come up with the idea that there had been a decline of 21.6%? They never explained, other than to say they had used a model. It must have been quite a model.

To my knowledge, these are the only studies of the impact of soda taxes on soda sales and/or calorie consumption in the real world. None of them managed to find a measurable decline in soda or calorie consumption, let alone in obesity. Back to the computer models, lads!

The debate should be live-streamed. It begins at 4.40pm GMT.

No comments: