An interview with Dr Ashok Kaul

Last week I mentioned that the only piece of real world evidence about underage smoking rates in Australia since plain packaging was introduced was not mentioned in the Chantler review. The research was carried out by the statisticians Dr Ashok Kaul and Dr Michael Wolf. It was presented to Chantler's team in a meeting in London last month and clearly indicates no increase in the rate of decline of smoking prevalence amongst 14-17 year olds in the thirteen months between December 2012 (when plain packs came in) and December 2013.

ASH director Deborah Arnott has since co-authored a letter to The Lancet in which she complains that 13 months is not long enough to see an effect (she doesn't explain why) and claims (wrongly - see below) that an implausibly large effect would be needed to show up as statistically significant.

I decided to e-mail one of the authors, Dr Kaul, to ask him what was going on and he generously agreed to answer a few questions on the record. Here they are...

You met with some of Chantler's team in March 2014 and explained your research in great detail. Why do you think it was not mentioned in his final report?

It is difficult to speculate about the reasons why our research was not explicitly referenced. The Chantler review team claims that our work was considered in reaching the conclusions of the review. We find this quite remarkable, though, since Sir Chantler apparently did not feel the need to reference the only paper on plain packaging based on real-world data in his report to support his findings. In particular, a neutral reader of the review would expect to find a complete reference list of original research to be able to draw his own conclusions.

In response to your research, some proponents of plain packaging have implicitly accused you of 'misrepresentation of the evidence'. In particular they criticise the 'small sample size'. How do you respond?

We not appreciate being implicitly accused of `misrepresentation of the evidence'. We have analyzed the data in a completely open and reproducible way. If anything, as detailed in the paper, there is a (slight) bias in our methodology in favor of finding a plain packaging effect. But we have not found any evidence of a plain packaging effect. What exactly does `misrepresentation' mean in this context? We agree that the sample sizes are relatively small, and state this explicitly in the paper. Currently, in our opinion, there is no better data set available.

How much of an immediate effect on smoking prevalence would be needed to pick up a statistically significant decline in this data set?

Despite the relatively small sample sizes, the power of our methodology against a meaningful immediate effect on smoking prevalence is not tiny. For example, the power against an immediate effect of reducing smoking prevalence by 0.5 percentage points (beyond the existing time trend) would have been around 0.65. This is quite large, actually. A power near 1.00 is never realistic; a typically number one aims for in controlled (!) experiments is 0.8. In a nutshell, our approach would detect a non-negigible effect of plain packaging on smoking prevalence of minors in Australia with a pretty high probability - despite the small sample size. Criticizing the 'small sample size' is therefore quite absurd.

Are you aware of any other empirical evidence on youth smoking since plain packaging was introduced in Australia?

We are not aware of any other piece of research based on real-world data on the effects of plain packing on youth smoking behavior in Australia. We are therefore quite disappointed that our work was not explicitly referenced in the Chantler review.

Some campaigners have said that they would not expect any short term effect from plain packaging on underage smoking. Do you have a view on this?

Some people expected a short-term effect, others did not. It is a legitimate and important research question to ask whether there is evidence for a short-term (lasting) effect of plain packaging. The empirical evidence so far does not support the conclusion of a short term effect. Of course, short-term effects are important for policy makers around the world who would like to chose their regulatory policies from a set of alternatives that have been proven to be effective - plain packaging is so far not part of this set.

Together with the fact that Chantler ignored the 0.3 per cent increase in cigarette sales since plain packaging came in - along with other empirical evidence - the decision to overlook the only real world figures on smoking prevalence seems rather strange, to say the least.