Peer review in action

I mentioned in May that the British Medical Journal had issued corrections to two articles, one of which was written by Aseem Malhotra, and had also set up an expert panel to decide whether the articles should be retracted altogether. Malhotra article was about saturated fat, statins and heart disease. Malhotra portrayed the former as being largely irrelevant (indeed, he called sat fats 'protective') while portraying statins are virtually useless and frequently dangerous.

Sat fats have probably been wrongly demonised (sceptics have been saying this for decades despite the medical establishment's consensus) and statins are probably over-prescribed. Malhotra's basic conclusions were therefore fairly sensible by his standards but, as usual, he overcooked his argument by exaggerating and cherry-picking evidence. In particular, he relied on one weak, uncontrolled study to make the claim that 20 per cent of statin users experience 'unacceptable' side effects. This is what led to the BMJ's correction and investigation.

The panel has now decided that the article should not be retracted. This seems to me to be a reasonable decision. It was, after all, only an opinion piece. If journals retracted every article that cited a questionable piece of research there wouldn't be much left.

More interesting than the article itself is the correspondence that has been published in the course of the investigation. Malhotra's initial pitch the BMJ can be read here and the peer review comments can be read here. Both are illuminating. Here is his initial e-mail:

From: aseem malhotra
Date: Friday, September 27, 2013
Subject: Re: Request to peer review a column for the BMJ

To: Rebecca Coombes

Hi Rebecca, I am in the final stages of a potentially game changing and much needed piece busting the myth of saturated fat and heart disease. It is well referenced including a strong evidence base for why all calories are not the same and I also briefly question statins in primary prevention and provide an explanation why the evidence is weak for benefit here. This is something I have conceived and looked into for over a year now. I can and was planning to do it for the Observations column but it's close to 1600 words and thought it may be more suitable as a feature? But I don't know what your requirements/usual process is for feature pieces? I have a heads up that the WHO is going to announce a revision on dietary guidelines on sugar at the end of October and the Scientific advisory committee on nutrition is currently reviewing dietary guidelines on carbs. But I am not confident that they will be totally impartial as one of their lead scientists has recently been exposed as being on the pay roll of Coca Cola! I therefore ideally want to publish this in the next few weeks but if it is not something you see as viable for features then I would shorten it down and do it for Observations but just thought it was worth asking you first!

Best

AseemAseem

Firstly, note the title of the e-mail. It reveals that the BMJ had previously asked this guy to be a peer reviewer. Of what, we don't know, but let's hope it didn't have anything to do with diet, sugar or obesity, as Malhotra has a long track record of getting the facts hopelessly wrong. Malhotra's previous contributions to the BMJ had been generic rants about sugar and 'junk food' so it is odd that the journal should have considered him to have any particular expertise other than as a rank-and-file medic.

Secondly, note the delusions of grandeur ("game changing", "busting the myth"). Malhotra's 'myth-busting' is not news to anyone who is familiar with the literature, including the popular literature. James Lefanu, for example, explained it much better in The Rise and Fall of Modern Medicine (1999).

Thirdly, note the motivation for the article which Malhotra openly gives the BMJ. He's in a rush to get it published because he had a "heads up that the WHO is going to announce a revision on dietary guidelines on sugar at the end of October and the Scientific advisory committee on nutrition is currently reviewing dietary guidelines on carbs." The article is therefore explicitly policy-based, as he confirms a few days later once he has been passed on to the journal's deputy editor, Trevor Jackson.

Hi Trevor,

I also wanted to give you a heads up and check that it's ok that I'll be submitting an Observations piece in the next couple of days that I've been putting together for some time. I am just editing it down from 1600 words.

It essentially busts the myth of saturated fat and heart disease, why all calories are not the same and also seriously questions statins for primary prevention. It's provocative and timely with the SACN currently reviewing their advice on carbs but as Sarah Bosely has recently exposed a leading scientist on the board has been co-opted by Coca Cola. Don't worry there's nothing potentially libelous this time!

Potentially libelous? I wonder what the story is there.

I believe it will definitely trigger a debate.

I know this is very short notice but do you think there's possibly space time for next week's October 12th issue? It's partly because I think we'll be ahead of the game before the WHO make an announcement on sugar at the end of October but also I happen to be off next week in case the press are interested.

What is the BMJ's response to this? Do they tell Malhotra that they are a serious medical journal and will not be exploited by single-issue political campaigners? Do they tell him to quit it with the Coca-Cola conspiracy theory? (I think Malhotra must be referring to Ian MacDonald, a man who has forgotten more than Malhotra will ever know about the scientific evidence on sugar.)

They do not. Instead, they immediately commission the article which Malhotra sends over the following day. Jackson tells Malhotra that he "really likes it" and mentions that the editor-in-chief, Fiona Godlee, is "particularly interested in this subject".

So begins the peer review process, such as it is. The first potential reviewer doesn't want to do it, the second reviewer can't do it and so it ends up being reviewed by just one person, John Abramson. He agrees to do it on 11 October and by 15 October Malhotra is already starting to get impatient...

From: aseem malhotra  
Date: 15 October 2013 16:16 
Subject: Re: Statins and heart disease trends 
To: Trevor Jackson

Hi Trevor, 
Hope you're well! Am I correct in presuming you've not heard back from the reviewers yet? 

Best 
AseemAseem 


From: Trevor Jackson
Date: Wed, 16 Oct 2013 11:18:04

Subject: Re: Statins and heart disease trends

Dear Aseem

Yes, that's correct. One reviewer denied, one said he couldn't go it until November, but I am expecting the third ASAP. Today, I hope. I am keen to include your piece in 26 Sept [sic - he means 26 Oct.] issue, and there is plenty of time for that, but Fiona is keen that all pieces like yours are externally peer reviewed, and I agree with her.

Best wishes
Trevor

 
From: aseem malhotra  
Date: 16 October 2013 13:20 
Subject: Re: Statins and heart disease trends 
To: Trevor Jackson 

Thanks Trevor, I am also happy that the piece is peer reviewed also. Better not to give any ammunition to detractors with vested interests! 

Best Aseem

Despite waiting for the reviewer's comments, Malhotra immediately starts planning his media assault.

From: aseem malhotra  
Date: 17 October 2013 14:10 
Subject: Re: Statins and heart disease trends 
To: Trevor Jackson  

Hi Trevor, 

Just to let you know that BBC Breakfast have pencilled me in to discuss busting the myth of saturated fat/heart disease for next wednesday morning. I have taken the day off as annual leave. I hope that yourself and Emma will be happy to press release (I can chat to her directly as I know you've got plenty on your plate to do!) early next week if we don't get a response till Monday from the peer reviewer. 

I am sorry if I appear over indulgent but for such an interesting and important public health message it would be fantastic opportunity to spread the message and maximise impact.

After a poke from the BMJ, John Abramson sends his comments over and the BMJ forwards them to Malhotra, giving him the name of the reviewer. Abramson also knows the identity of the author (ie. it wasn't a blind, let alone a double-blind, peer review). Abramson has a much better grasp of the science than Malhotra but he was unlikely to be very critical of the article since he also believes that statins are heavily over-prescribed and have serious side effects. Nevertheless, he offers several suggestions and corrections, most of which Malhotra ignores. 

Malhotra gets the comments at 10.11pm and sends the final draft to the BMJ at 5.14am the following day. He does nothing more than change one statistic and a handful of words. When Abramson describes one of Malhotra's assertions as "a stretch... a weak and tangential argument", Malhotra responds to the BMJ saying "Just his opinion - no change". Elsewhere, when Malhotra portrays a controversial theory as a fact, Abramson says "This is one theory about the mechanism of benefit, but certainly not proven" to which Malhotra simply replies "He agrees it's a theory - No change".

Although Abramson makes some substantive criticisms, he does not pick up on the dodgy reference which led to the retraction that got Malhotra in hot water. This is not surprising. Remember that Malhotra's article was one of two papers from the October 26th issue to be flagged up with a correction and be investigated by an expert panel. Both articles claimed that 20 per cent of statin users suffered serious side effects and they both cited the same study. 

And who was the lead author of the other study? Step forward John Abramson.

So here we have two peer-reviewed articles, both of which have caused the BMJ some embarrassment. At least one of the articles was reviewed by the author of the other and by no one else. This reviewer was known by the BMJ to be someone who agreed with Malhotra's argument but even if he hadn't it would scarcely have mattered since the BMJ clearly doesn't expect its authors to take heed of what reviewers say anyway. Both Malhotra and the BMJ were more focused on having the articles rushed out so that they could influence policy and make a splash in the media. Which, of course, they did.

Viva science.

Share

Tweet