* The views in this post are mine alone, and may not reflect the views of my previous or current employers, co-authors, or other colleagues *
As a young(ish) reproductive health epidemiologist whose scientific career will depend in part on publications, it is pretty scary to challenge a Goliath like Lancet journals. I respect these journals deeply. They bring important science to the forefront, they help us to improve health, medicine, and policy. Lancet journals, in particular, also engage admirably on social media and encourage researchers to do the same - better bringing science to the public.
But I experienced a terrible situation when publishing my work at Lancet Infectious Diseases, and it is important to challenge people, even our heroes, to do better. So, it's time to share my story. My intention is that this post contribute to efforts to protect other scientists and journals from experiencing the frustrating situation that I did, perhaps by stimulating more nuanced discussion on how to better prevent and address misconduct by peer reviewers (a topic which receives less attention than misconduct by authors). This issue may also be applicable to discussions on the pros and cons of open peer review. Please do share your thoughts in the comments section.
As a young(ish) reproductive health epidemiologist whose scientific career will depend in part on publications, it is pretty scary to challenge a Goliath like Lancet journals. I respect these journals deeply. They bring important science to the forefront, they help us to improve health, medicine, and policy. Lancet journals, in particular, also engage admirably on social media and encourage researchers to do the same - better bringing science to the public.
But I experienced a terrible situation when publishing my work at Lancet Infectious Diseases, and it is important to challenge people, even our heroes, to do better. So, it's time to share my story. My intention is that this post contribute to efforts to protect other scientists and journals from experiencing the frustrating situation that I did, perhaps by stimulating more nuanced discussion on how to better prevent and address misconduct by peer reviewers (a topic which receives less attention than misconduct by authors). This issue may also be applicable to discussions on the pros and cons of open peer review. Please do share your thoughts in the comments section.
By way of background, I've worked for many years on a complicated, contentious scientific question: whether various hormonal contraceptive methods impact various HIV-related risks. If you are interested in work on this issue, please see this 2012 systematic review (discussed at length below), this 2014 systematic review update, or other resources shared here. Feel free to email any questions about this topic!
And I thought "Reviewer 3's" were usually the cranky ones...
The story begins in May 2012, when we submitted a draft of a systematic review on hormonal contraception and HIV acquisition to Lancet Infectious Diseases. The review had already been presented to more than 90 people at a World Health Organization technical consultation, and we looked forward to publishing our work in a peer-reviewed journal. The Editor-in-Chief of Lancet Infectious Diseases, John McConnell, even originally said he'd fast-track peer review, given the importance of our topic.
When we received peer review comments in July 2012, Reviewers B and C were supportive and provided useful feedback which helped to improve the manuscript. However, Reviewer A made extremely hostile comments -- accusing us of lying to Africans and putting them at risk of HIV. Startlingly, this reviewer provided absolutely no scientific substantiation for these extreme comments. Nonetheless, Sally Hargreaves (the Lancet Infectious Diseases Editor handling our paper) said she'd specifically "be interested to see [y]our response to this reviewer’s comments”.
I was highly distressed and intimidated by receiving such unprofessional peer review feedback from such a highly respected journal. Lancet Infectious Diseases is a member journal of the Committee on Publication Ethics (COPE). COPE has a code of conduct for journal editors, which encourages "sending reviewers' comments to authors in their entirety unless they contain offensive or libelous remarks" and notes that "Editors should strive to ensure that peer review at their journal is fair, unbiased and timely." To me, the unsubstantiated accusations seemed offensive, unscientific, and unfair, but Lancet Infectious Diseases appeared to want us to take them seriously.
When we received peer review comments in July 2012, Reviewers B and C were supportive and provided useful feedback which helped to improve the manuscript. However, Reviewer A made extremely hostile comments -- accusing us of lying to Africans and putting them at risk of HIV. Startlingly, this reviewer provided absolutely no scientific substantiation for these extreme comments. Nonetheless, Sally Hargreaves (the Lancet Infectious Diseases Editor handling our paper) said she'd specifically "be interested to see [y]our response to this reviewer’s comments”.
I was highly distressed and intimidated by receiving such unprofessional peer review feedback from such a highly respected journal. Lancet Infectious Diseases is a member journal of the Committee on Publication Ethics (COPE). COPE has a code of conduct for journal editors, which encourages "sending reviewers' comments to authors in their entirety unless they contain offensive or libelous remarks" and notes that "Editors should strive to ensure that peer review at their journal is fair, unbiased and timely." To me, the unsubstantiated accusations seemed offensive, unscientific, and unfair, but Lancet Infectious Diseases appeared to want us to take them seriously.
Regroup, resubmit...and rejoice!
This shook me, deeply. So deeply that we pulled the paper from consideration and took extra time to go over every sentence and every number with a fine-toothed comb, reconfirming (again, and again) that every word was accurate. At the time, it felt like the most open-minded, scientifically responsible reaction to such an amorphous, blanket condemnation -- one which the highly respected minds at Lancet Infectious Diseases had apparently considered fair game.
After reviewing everything with fresh eyes, I was confident that our review stood on its merits, and that Reviewer A had been extraordinarily unscientific and unprofessional. I felt sheepish for having ever spent time and energy taking his comments so deeply to heart - but had wanted to ensure that this complicated subject had been done justice. Little did I know -- once his identity later became known (and verbally confirmed) to me, how shocked I'd be that they'd ever selected this person as a peer reviewer in the first place...
In December 2012, we asked Lancet Infectious Diseases if we could resubmit the paper, attaching responses to the three original peer review comments. The journal agreed to put the paper back under peer review, and assigned six new peer reviewers to it. We received these new comments in May 2013. We responded to all of them, and the paper was accepted in Lancet Infectious Diseases that month. I should have been popping champagne (my first publication in a Lancet journal!), but a debacle had only just begun...
After reviewing everything with fresh eyes, I was confident that our review stood on its merits, and that Reviewer A had been extraordinarily unscientific and unprofessional. I felt sheepish for having ever spent time and energy taking his comments so deeply to heart - but had wanted to ensure that this complicated subject had been done justice. Little did I know -- once his identity later became known (and verbally confirmed) to me, how shocked I'd be that they'd ever selected this person as a peer reviewer in the first place...
In December 2012, we asked Lancet Infectious Diseases if we could resubmit the paper, attaching responses to the three original peer review comments. The journal agreed to put the paper back under peer review, and assigned six new peer reviewers to it. We received these new comments in May 2013. We responded to all of them, and the paper was accepted in Lancet Infectious Diseases that month. I should have been popping champagne (my first publication in a Lancet journal!), but a debacle had only just begun...
Breaches in peer review confidentiality, violations of copyright, and bears - oh my!
In July 2013, just before our systematic review was scheduled to be published online, I saw that a version of our manuscript which could only have come from one of the original three peer reviewers had been leaked to (and posted publicly online by) the Rebecca Project, an unscientific organization which has spread misinformation about contraception. In their (evidence-free) 'report' they accuse my co-author and I of "bartering women's lives to advance our careers" and publicly post our leaked manuscript.
I wondered how our confidential manuscript (accessible only to the Lancet Infectious Diseases and the three peer reviewers) could have been leaked. I was less concerned about the manuscript being publicly available (since it had already been accepted), but was VERY concerned about the flagrant breach of ethics underlying the breach in peer reviewer confidentiality and the violation of copyright. I immediately brought it to the attention of Lancet Infectious Diseases.
At the same time, I noticed that a member of the (then) Board of Directors of the Rebecca Project for Human Rights (later renamed the Rebecca Project for Justice - due to a change in leadership) had also published a letter on hormonal contraception and HIV acquisition in Lancet Infectious Diseases. This piqued my interest, but I was doubtful - at the time - that a respected journal would have selected this particular individual, given that his theories have been roundly debunked (in Lancet!). According to Dr. Seth Kalichman (and as evidenced by this person's publication record in PubMed; largely of letters to editors), this individual (who has no institutional affiliation) “has not done research of his own, and selectively reviews past research to support his views" and is viewed as an AIDS denialist. But I digress...
I wondered how our confidential manuscript (accessible only to the Lancet Infectious Diseases and the three peer reviewers) could have been leaked. I was less concerned about the manuscript being publicly available (since it had already been accepted), but was VERY concerned about the flagrant breach of ethics underlying the breach in peer reviewer confidentiality and the violation of copyright. I immediately brought it to the attention of Lancet Infectious Diseases.
At the same time, I noticed that a member of the (then) Board of Directors of the Rebecca Project for Human Rights (later renamed the Rebecca Project for Justice - due to a change in leadership) had also published a letter on hormonal contraception and HIV acquisition in Lancet Infectious Diseases. This piqued my interest, but I was doubtful - at the time - that a respected journal would have selected this particular individual, given that his theories have been roundly debunked (in Lancet!). According to Dr. Seth Kalichman (and as evidenced by this person's publication record in PubMed; largely of letters to editors), this individual (who has no institutional affiliation) “has not done research of his own, and selectively reviews past research to support his views" and is viewed as an AIDS denialist. But I digress...
Lancet Infectious Diseases: not COPE-ing well
McConnell noted that he would follow up on the matter, but weeks and months went by with no satisfactory response. I checked in periodically, but mostly heard crickets. When I reiterated that COPE guidelines suggest that in cases of peer review misconduct, the peer reviewer's institution should be contacted for an investigation, he told me that the reviewer had no institutional affiliation, so he considered the avenue closed.
To me, this was a great opportunity to highlight this issue to COPE, so they could consider extending their guidance on misconduct by non-institutionally affiliated peer reviewers. I asked Lancet Infectious Diseases to bring the case formally to COPE, so the wider scientific community could benefit, perhaps stimulating productive discussion on preventing reviewer misconduct (or strengthening procedures for follow up of non-affiliated reviewers). COPE itself recommends "referring troubling cases to COPE, especially when questions arise that are not addressed by the COPE flowcharts, or new types of publication misconduct are suspected."
To my great frustration, McConnell repeatedly ignored my request. Instead, he told me he’d ask a friend who’d been involved with COPE previously her opinion. I felt this would deny the chance for the case to be made useful to the broader scientific community, so I escalated the issue to Richard Horton (Editor-in-Chief of Lancet) and Charles Warlow (Ombudsman of Lancet, at that time).
Nearly five months in, it was finally noted that the case would go to COPE. You can read how McConnell described the case to COPE, and how COPE responded here. I thought the description lacked important details. On December 4, 2013, I tried to listen to an online COPE discussion (for which a weblink had been publicly posted), but was contacted and told the meeting was not intended for my participation. Discussion of my case was postponed to happen outside of the publicly accessible session. They later posted audio of at least part of the discussion - pieces of which are quite infuriating. Ultimately, COPE did recommend (as I had, months earlier) that McConnell write an Editor's Note so the case was part of the public record. So, the (rather toothless, IMO) note below was published.
To me, this was a great opportunity to highlight this issue to COPE, so they could consider extending their guidance on misconduct by non-institutionally affiliated peer reviewers. I asked Lancet Infectious Diseases to bring the case formally to COPE, so the wider scientific community could benefit, perhaps stimulating productive discussion on preventing reviewer misconduct (or strengthening procedures for follow up of non-affiliated reviewers). COPE itself recommends "referring troubling cases to COPE, especially when questions arise that are not addressed by the COPE flowcharts, or new types of publication misconduct are suspected."
To my great frustration, McConnell repeatedly ignored my request. Instead, he told me he’d ask a friend who’d been involved with COPE previously her opinion. I felt this would deny the chance for the case to be made useful to the broader scientific community, so I escalated the issue to Richard Horton (Editor-in-Chief of Lancet) and Charles Warlow (Ombudsman of Lancet, at that time).
Nearly five months in, it was finally noted that the case would go to COPE. You can read how McConnell described the case to COPE, and how COPE responded here. I thought the description lacked important details. On December 4, 2013, I tried to listen to an online COPE discussion (for which a weblink had been publicly posted), but was contacted and told the meeting was not intended for my participation. Discussion of my case was postponed to happen outside of the publicly accessible session. They later posted audio of at least part of the discussion - pieces of which are quite infuriating. Ultimately, COPE did recommend (as I had, months earlier) that McConnell write an Editor's Note so the case was part of the public record. So, the (rather toothless, IMO) note below was published.
Violation of copyright, schmiolation of schmopyright?
With regard to the violation of copyright and illegal posting of our manuscript, I received no clear answers on how the journal would proceed, other than that McConnell sent a few emails/letters to the Rebecca Project, which went unanswered. This is how the Rebecca Project eventually spun those events to LifeSitenews.com.
McConnell later told me verbally that they didn't pursue more serious strategies with the Rebecca Project because of "reputational" concerns. Specifically, Lancet did not want to appear to be a big powerful journal coming down hard on a small advocacy organization. Nevermind that this "small advocacy organization" was catching the ear of certain folks in the US Congress (e.g., here and here) with their highly unscientific "report".
At one point, McConnell suggested that I contact the Rebecca Project to request our draft manuscript be removed from their website. Being aware of some concerning history about certain individuals associated with their group, I declined. I could see that I wasn't getting very far, so held out hope that the Lancet Ombudsman would make some sense out of this mess.
McConnell later told me verbally that they didn't pursue more serious strategies with the Rebecca Project because of "reputational" concerns. Specifically, Lancet did not want to appear to be a big powerful journal coming down hard on a small advocacy organization. Nevermind that this "small advocacy organization" was catching the ear of certain folks in the US Congress (e.g., here and here) with their highly unscientific "report".
At one point, McConnell suggested that I contact the Rebecca Project to request our draft manuscript be removed from their website. Being aware of some concerning history about certain individuals associated with their group, I declined. I could see that I wasn't getting very far, so held out hope that the Lancet Ombudsman would make some sense out of this mess.
A non-responsive ombudsman and a very, very private ombudsman walk into a bar...
For several months, I tried to get the (then) journal ombudsman, Charles Warlow, to respond to my emails, but was ignored. Eventually, I got a note that Warlow was being replaced by a new ombudsman. Several weeks later, Dr. Wisia Wedzicha wrote to say that she would review the case. A few weeks after that, she sent me her report. I'd love to share that report here (she agreed with some concerns I'd raised, which continue to be unacknowledged by McConnell or Lancet Infectious Diseases). However, she said:
"I consider my Ombudsman report to be strictly confidential and it should not be made public in any way. The report contains some opinions on 3rd parties and also some of my own views on publications and processes. I would regard any breach of confidentiality in this respect as a very serious professional matter, and one which undermines the whole process of a confidential independent report by the Ombudsman."
-- Wisia (Jadwiga) Wedzicha, Friday, June 6, 2014
So, I was asked to be quiet about this, because THIS was considered a "very serious professional matter" - apparently more serious than addressing the misconduct I had endured, since no concrete actions came of the report. I've never contacted a journal ombudsman before; so I don't really know - are Ombudsman reports only for authors eyes? What's the point if there is no follow up from the journal (there wasn't) AND I can't discuss the details of it (I haven't...)?
#AskLancet (responses not included)
Recently, the journal did a Tweetchat called #AskLancet - and I had some follow up questions on my situation. How this conversation unfolded is best encapsulated in a Storify; in short, my questions were not responded to, until they were retweeted by a man who shared my concern. Click here to read the Storify (UPDATE: the Storify website is no longer, but you can click below to download a document containing the content that had been in the Storify).
| Ask Lancet Storify content.pdf |
Do better: protect science and scientists
In sum, Lancet Infectious Diseases failed to perform due diligence in selecting an unbiased and professional peer reviewer, allowed unscientific and offensive comments to move forward in the review process, failed to protect my work from being leaked by an unethical reviewer who the journal was unable to follow up on, dragged their feet in taking my case formally to COPE, failed to fully describe the situation to COPE, and shrugged their shoulders at the violation of copyright given "reputational concerns". They've never apologized for this situation, or shown any interest in implementing more meaningful solutions to prevent this from happening in the future (e.g., telling reviewers that if they breach confidentiality that their names will be publicly shared so that other scientists can avoid being reviewed by them, or putting additional measures in place when asking unaffiliated individuals to serve as peer reviewers). Why not?
I continue to be, as I have for over two years now, in disbelief of this situation, particularly at a journal of this caliber. My message for Lancet journals? I respect you enormously and I believe strongly in your mission. But you must be able to do better than this. To protect science and scientists, to promote ethics in publishing, and to discourage future misconduct in publishing - please do better by your contributing authors.
I continue to be, as I have for over two years now, in disbelief of this situation, particularly at a journal of this caliber. My message for Lancet journals? I respect you enormously and I believe strongly in your mission. But you must be able to do better than this. To protect science and scientists, to promote ethics in publishing, and to discourage future misconduct in publishing - please do better by your contributing authors.
RSS Feed