Friday 12 February 2021

The COVID modellers have jumped the shark

Academics from Warwick have modelled the effects of vaccinating the population and removing 'non-pharmaceutical interventions' in England over the next eighteen months. The study was produced, I think, on 13 January. The findings are certainly eye-catching.

In their best case scenario, lockdown reduces R to 0.8 and 3 million vaccines are delivered each week from February (with 1 million a week delivered in January). Lockdown ends on 22 February and all NPIs are dropped by July. 
This means that everybody who wants a vaccine has had one (or, indeed, two) by the summer. The authors nevertheless predict that England will see 2,000 deaths per day in August. If the vaccines don't work as well as expected, this rises to nearly 5,000 per day.

I am only an interested amateur and am happy to be put straight, but WTF?!? Last August, when there was no vaccine and minimal NPIs, England had about eight deaths a day. At the height of the winter second wave, it had 1,238 deaths (January 19th). 
5,000 deaths per day is more than 70 per million. Even the worst hit countries such as Belgium and Czechia never got above 30 per million at the height of their epidemics. And yet these guys think that it could far exceed than that in Britain in the summer after the vaccines have been fully rolled out. Even in their best case scenario, there would still be 1,000 deaths a day.

When your model gives you such an implausible result, you have to question your assumptions. So what are they?

According to the brief text, the authors expect all these deaths to come about because some people will refuse to take the vaccine and some people who take the vaccine won't be protected. 

Uptake: Throughout we assume 95% uptake in care homes, 85% in the general population above 50 and 75% in adults below 50 for the first dose. This drops to 75% for the over 50s and 66% for the under 50s for the second dose.

People aged under 50 are almost irrelevant in terms of Covid mortality so the figure to focus on is 85%. 
Here are their assumptions about protection against symptomatic disease...

Efficacy: We sub-divide into the effects of protection against symptoms (disease efficacy) and reduction in transmission – we assume that transmission blocking acts by stopping infection. Disease efficacy is taken as 70% and 88% after dose 1 rising to 88% and 94% after dose 2 for the Oxford and Pfizer vaccine respectively. Transmission efficacy is taken to be 48% rising to 60% for both. Protection is lagged by 14 days after the dose is delivered.

This roughly reflects what the trials have shown us about these vaccines.

The authors don't provide a figure for the total number of deaths in their projected third wave, but it looks around twice as bad as the second wave which has killed about 60,000 people in England and will probably end up killing around 75,000. So, as a very rough estimate, they're suggesting there will be 150,000 deaths after everybody who wants a vaccine has had one. That's more than all the deaths we've had already. Unsurprisingly, the authors call for the government to hold back on relaxing lockdown, although quite what this would achieve is unclear: the lesson of the study is that COVID-19 will get us all eventually.

Imperial have also done some modelling. This study was produced on 14 January, the day after Warwick's. It makes exactly the same assumptions about transmission efficacy and disease efficacy. Unlike Warwick, they assume vaccine uptake is 85% across all age groups. They assume that NPIs are gradually lifted on the first day of each month but do not say which ones. As with Warwick, all NPIs are lifted by 1 July.

The assumptions are therefore very similar and the conclusions are only slightly less gloomy. Like Warwick, they predict a massive summer wave and 130,800 dead even under their most optimistic scenario.

Again, the authors encourage the government to be extremely cautious in relaxing the lockdown.

In both cases, they are predicting a dramatic rise in cases followed by a steep fall. The Warwick model shows a classic epidemiological bell curve. Both models imply that everyone who can get infected will get infected and that the death toll will only be ameliorated by the vaccines providing a measure of protection against symptomatic disease. The lack of a resurgence in winter 2021/22 suggests that the epidemic will have run its course after this final, devastating wave leaves no one left to infect.

If the assumption is that nearly everyone gets infected, the 15% of the population who are not vaccinated will die at the normal rate of about 1 in 100. A smaller proportion of the 85% who are vaccinated will die. That's still a lot of deaths. 

But there are two big problems.

Firstly, both models underestimate how many people will take the vaccine. So far, 91% of the 80+ cohort have taken it and 96% of those aged 75-79 have taken it. That's a lot more than 85% assumed in these models.

Secondly, and more importantly, the estimates of disease efficacy (the reduction in risk of getting symptomatic disease) are roughy correct, but the authors seem to have overlooked the crucial point about the AstraZeneca vaccine which is that there were 'no severe cases and no hospitalisations' in the trials. The vaccine seems to be 100% effective in preventing death from COVID-19. The Pfizer vaccine is 95% effective in preventing symptomatic disease altogether, so the number of deaths that would occur among the cohort who take it would, presumably, be very low (in Israel, there were 4 severe cases and no deaths among 523,000 vaccinated people).

It is incredible that neither study factors in the effect of the vaccines on severe disease and mortality. You can forgive them for not predicting that so many people would take up the vaccines, but we've known about the AZ vaccine's ability to prevent severe disease and death since November.

Their results seem to be based entirely on transmission and the prevalence of symptomatic disease. The mortality figures they come up with are so enormous, I can only assume they jumped to the conclusion that the fatality rate would be the same for someone who was vaccinated as for someone who wasn't. It seems hard to believe they would make such a basic error. Perhaps someone can explain what I'm missing?

Obviously I'm just some chump barstool epidemiologist, but to a simple man like me, it seems that if we have a vaccine that is 100% effective in preventing death, we won't have any COVID-19 deaths except among the minority of people who don't take the vaccine (which, to be blunt, is their problem; there are not enough of them to overwhlem the NHS). Even if it turns out to be a bit less effective than 100%, it's very hard to see how the kind of mortality figures in these studies can be justified.

The rate of transmission implied in these models doesn't seem plausible when 30% have already had the virus (according to Neil Ferguson), 85% have had the vaccine and it's summer, although both sets of modellers do seem to assume that Kent variant is extremely infectious. Maybe it is, but transmission isn't really the issue when people are protected from severe disease and death.

So why isn't this crucial factor included in the models?
UPDATE (23 February)
This pyramid of piffle has been formalised in a SAGE document and used to lobby against ending the NPIs in May. See this thread.

No comments: