Does the Anonymous Voice Have a Place in Scholarly Publishing?

In November 2015, Jaime A. Teixeira da Silva contacted Editor-in-Chief Mike Blatt with a request to submit a response to his editorial, Vigilante Science, published in the October 2015 issue of Plant Physiology. An offer to respond had been made to Brandon Stell of PubPeer, who ultimately declined. Mike proposed instead that Jaime and he enter into a dialogue, which is published jointly and in full here.

Dear Mike,

Key aspects of the fabric of science integrity are disintegrating and the noble cause for which true scientists have strived is being undermined by a wide array of disruptive factors. Even though traditional peer review (TPR) remains central to the integrity of the scientific literature, multiple weaknesses, including the general lack of a triple-blind peer review, indicate that TPR has in fact become corrupted to varying levels (Teixeira da Silva and Dobránszki, 2015). Given these weaknesses, there is a need for scientists to become proactively vigilant of the literature, peers, editors, and publishers. An expanding “predatory” open access movement undermines the quality control enforced in TPR by truly academic journals, which themselves are facing multiple limiting factors. It is therefore safe to assume that the literature is being increasingly filled with studies that have not been properly vetted and thus carry a wide range of errors. The peer pool is currently under great strain and stretched thin while journals are under constant pressure to be more productive. In such a state, corners in quality control will be cut. How does a journal that is already under great pressure to perform under limiting conditions detect all errors in submitted manuscripts, in the literature that is cited in those manuscripts, and in the previously published literature? It is here that the centrality of postpublication peer review (PPPR) in plant science has emerged as a corrective factor (Teixeira da Silva, 2013).

Not all readers are peers and not all peers, or editors, have a clinical eye. Yet, there are readers, peers and not, who are able to detect errors (sensu lato) in the literature. Reporting an error is generally welcomed by authors and editors. However, a person reporting multiple errors may be viewed as an irritant who overburdens an already stressed TPR system. Thus, there may be a negative association with individuals who complain or report errors in excess. Unlike a discussion in a scholarly meeting, reporting errors is an uncomfortable process and there are clear risks for the person reporting errors, not the least of which may be to their image or job security. Setting aside malicious commentators whose objectives are clearly not to better science or to correct the literature, one viable way of defending the integrity of the literature in PPPR is by allowing anonymous voices to exist, and by giving them the right to express themselves, including in social media.¹ Valid anonymous voices have become essential players in correcting the literature, a clear example of this in plant science being Olivier Voinnet’s literature.² Aspects of the literature in the plant sciences are in crisis, especially quality control, and greater accommodation of the anonymous voice to correct shortcomings is urgently required. If not, efforts in quality control by noble journals such as Plant Physiology that rightfully defend the integrity of their own territory, may be in vain.

Sincerely,

Jaime

Dear Jaime,

I do support postpublication assessment in science, what you call PPPR. I believe it is fundamental to the scientific process, and always has been. Science does not end with publication. Publication is only the beginning of scientific debate, even online.

However, scientific debate should not be conflated with the pursuit of fraud. Confusing these purposes leads to much misunderstanding and, in my opinion, entrenches a logic and behaviors that undermine the fabric of the scientific process.

There is a case for improving the way fraudulent science is handled, and I will argue due process is essential to protect the scientific community and civil society. I noted in my editorial of October 2015 that estimates suggest at most 1% of publications incorporate some element of deliberate misconduct and that this percentage has not changed over the past 20 years (Blatt, 2015).

Of the remaining 99% of scientific publications, the vast majority of any ‘errors’ arise from honest mistakes, oversights, and misunderstandings. In my opinion, such errors are best addressed by open (nonanonymous) debate, communicating with authors directly, and by publication.

As a scientist still active at the bench and training many young researchers, I am deeply concerned that open debate is not lost, both for science today and for the community that this next generation will build in the future. In my opinion, there is a place for PPPR, but it must be tempered or debate will become vindictive and its benefits thereby lost.

For these reasons, I am not a fan of anonymity. I noted (Blatt, 2015) that “anonymity is intimidating in itself, regardless of how polite a commenter may be; psychologically, it wrong-foots the author from the start, and the price often is an absence of worthwhile discussion.” To my mind, anonymity also implies a lack of conviction and a refusal to engage meaningfully.

You (Teixeira da Silva and Dobránszki, 2015) and others^3,4 argue that scientists hesitate to question colleagues for fear of reprisal. I maintain that it is possible for a PhD student or postdoctoral researcher to address errors, even with a senior scientist, in a way that is constructive and nonthreatening. Scientific debate is about questioning data and concepts. It is a warped world view in which scientists are so fearful of engaging that they never challenge others’ research and ideas openly, whether online or in publication.

Anonymity opens the door to abuse. You need only review the comments³ to my editorial to see many examples of such behavior. In the context of ‘scientific discussion,’ anonymous commenting quickly escalates into de facto trial by kangaroo court, if only because small errors, omissions, and misunderstandings are pilloried in equal measure with suspicions of bona fide fraud (Blatt, 2015). Without a distinction between debate and the pursuit of fraud, it is honest debate that is devalued.

Of course, it is important to protect the whistleblower and, you may argue, to redress inequities that supposedly pervade science today. Yet anonymity entrenches inequality in every situation, simply because one side is identified, the other is not. Surely, whatever the perceived inequities, the solution is not to further embed inequality. There is an old saying about two wrongs and what is right.

Yours sincerely,

Mike

Dear Mike,

I am very glad that we have this opportunity of discussing these ideas. I can appreciate your perspectives related to fear and reticence associated with the notion of anonymity. At its base, and most importantly, we share common goals and desires for the integrity of the plant science literature through rigorous and strict peer review, although we seem to differ about who may participate in that process. Postpublication perfection (that is, elimination of all errors) can be achieved through PPPR since the input is endless and comes from known and unknown as well as qualified and unqualified readers. Among all these categories, there will be those who wish to comment anonymously. And if we truly represent a democratic global society that embraces freedom of speech, even of criticisms of minor errors, then we have no option but to embrace the anonymous voice and to protect the fundamental right to anonymity. The underlying argument here is that the identity should not be the focus, but rather the message.

There is absolutely no doubt that commenting anonymously for positive purposes is done to protect, but not necessarily to hide, multiple factors associated with the commentator or reader and to focus simply on the message.

One cannot deny the risks or the blow-back associated with PPPR. Being critical of a paper is, by extension, a criticism of the editorial process that vetted and approved the paper for publication. It may be, by extension, an indirect criticism of the journal and/or publisher for their lack of oversight. Under such circumstances, it is realistic to expect individuals to assume a protective anonymous mask.

In many cases, anonymous comments are valid, and to discount all anonymous comments based on the fear factor will not advance science or correct the literature where it is desperately needed. So, while I respect your opinion, I disagree with much of your position on who may contribute to PPPR and how they may do so. So much more is still to be said and debated beyond our two-way discussion, and I expect there will be a wide range of supporters and detractors of the anonymous voice in coming weeks and months.

For now, the purpose of my opinion is not to suggest a solution about how the anonymous voice could be integrated into current publishing models, as that involves deeper reflection and greater planning. I am simply arguing the need for, and importance of, the anonymous voice.

I am looking forward to the outcome of this discussion.

Sincerely,

Jaime

Dear Jaime,

Thank you for your reply. I applaud your wish to engage the scientific community in PPPR. Nonetheless, you raise points here and in your recent publication (Teixeira da Silva and Dobránszki, 2015) that are inherently contradictory, proposing (1) that review in PPPR is possible by “qualified and unqualified readers” and (2) that science is a democratic process. Both, I submit, also undermine your arguments for anonymity.

Of course, any literate person can read a scientific article, but engaging unqualified reviewers in any circumstance is unhelpful and is more likely to create misunderstandings than to resolve them.

In your article, you quote the survey of Ho et al. (2013) criticizing TPR to note that “only 39.6% of respondents believed that reviewers were competent.” You also criticize TPR for engaging ‘peers’ who may be underqualified, unqualified, or positively biased. Yet you argue that unqualified “non-peer reviewers, members of the public, or nonspecialists can be classified as PPPR reviewers.” Do you really believe it is helpful for PPPR—but not for TPR—that so many, you claim, are unqualified and incompetent? I see fundamental contradictions here.

Your argument for democracy in science is also conflicted. Yes, science is a “massively cooperative undertaking,” to quote one of my PubPeer commenters,³ but that does not mean it is a democratic process. Science requires substantial training; it builds on the merits of expertise and of knowledge; it is not a ‘one man, one vote’ endeavor with universal enfranchisement. To argue otherwise is patently absurd.

I do find it deeply depressing that you consider anonymity an essential crutch for the modern scientist to debate or to challenge errors and misunderstandings. Again, anonymity ensures inequality in every situation, and as I noted before “the price often is an absence of worthwhile discussion” (Blatt, 2015). You suggest the focus should be “on the message,” yet an appropriate response almost always demands some knowledge of the context and the commenter. That sites like PubPeer encourage anonymous commenting only entrenches the practice and thereby degrades communication.

True, there will be circumstances in which questioning a colleague, perhaps a senior scientist, is difficult, and there are individuals who will never be open-minded. Even so, I noted that it is possible to challenge openly (nonanonymously) in ways that are constructive and nonthreatening. Usually the outcome is successful, benefiting both. Again, I agree there must be ways of protecting the whistleblower, but anonymity is not the answer, not if due process is to ensure civil society and protect the innocent from denouncement or worse.

Finally, I disagree with your claim that criticism of an article is “by extension, a criticism of the editorial process … or the journal.” Debate is the cornerstone of any scientific advance and is therefore vital for scientific publishing. By its nature, constructive criticism is welcomed in publication, as in scientific debate, when it advances understanding, including when it uncovers errors (Blatt, 2016).

Yours sincerely,

Mike

Dear Mike,

I must come back to my point that, even though traditional peer review (TPR) remains central to sustaining the integrity of scientific literature, there are weaknesses in this system. For example, it can be difficult to avoid bias, yet few journals have taken up the use of triple-blind peer reviewing. Even when TPR is unbiased, reviewers and editors can miss important errors. The expansion of predatory journals also adds pressure to academic publishing. It is therefore safe to assume that the literature is increasingly filled with studies that have not been fully or properly vetted and thus carry a wide range of errors. Given these weaknesses, there is a need for scientists to become proactively vigilant of the literature.

Postpublication peer review—what you refer to as postpublication assessment—is a powerful means to detecting errors in the literature, including those of fraud. Yet, as I have argued, reporting errors may carry negative associations and risks for the persons who report errors. I agree that anonymity introduces many risks, including those of malicious commenters whose objectives are clearly not to better science or to correct the literature. The question is whether this is an acceptable risk: Is it acceptable for PPPR to include the anonymous voice and to hold authors and journals accountable to anonymous commenters? There is no denying that in recent times the anonymous voice has been instrumental in correcting the literature. I believe that greater accommodation of the anonymous voice as a tool to correcting the literature is required.

Sincerely,

Jaime

Dear Jaime,

Our disagreement is not about PPPR but how it is implemented to facilitate discussion and improve the scientific literature. I maintain that any solution must include checks to protect the innocent and ensure civil debate. Allow me to disentangle the several threads in our letters in order to lay bare the tensions. I see three separate challenges for the future that we do agree must be addressed.

First and foremost, it is clear that academic publishing as a whole must adapt to embrace better the potential of the internet. The ASPB journals lead by example and currently offer a number of routes for scientific commentary. Plant Physiology has long published two standard formats that facilitate literature discussion, Letters to the Editor and Scientific Correspondence. Starting in 2014, the journal began publishing Commentaries on selected articles by research scientists in the field. The Plant Cell, too, offers Commentaries that often question publications which are held by the community to be flawed. Additionally, Plant Physiology has sought to promote discussion of particularly contentious topics by encouraging individual scientists with conflicting views to engage through joint reviews; an excellent example was the article by Robinson et al. (2015) published last year.

Of course, all of these routes are peer reviewed and, in instances of unilateral confrontation, authors of the article in question are rightly given the opportunity to respond. Still more can, and should, be done to encourage discussion, especially online. So, the first challenge for the future is to enhance and speed this process without opening the door to abuse that you agree is a risk with anonymity.

A second task must be to improve the way that the scientific community deals with misconduct. I noted in my October 2015 editorial (Blatt, 2015) that the Rockefeller University Press maintains the longest record of image data analysis, having forensically examined the images of all articles accepted for publication in its flagship journals since 2002. These records show that the percentage of articles for which acceptance was subsequently withdrawn has remained almost unchanged for more than a decade at around 1%.

It is true that the number of articles retracted since the mid-1990s has accelerated disproportionately as a percentage of all scientific articles published, rising above 0.01% within the past decade (Fang et al., 2012; Stern et al., 2014); however, this increase may be explained by changes in journal policies, strengthening national policies against scientific misconduct, as well as greater community awareness (Fanelli et al., 2015). If we allow that the 0.01% figure represents a 1% ‘tip of the iceberg’ of research misconduct that is actually detected, then the numbers tally closely with the results of the Rockefeller University Press and with estimates of the percentage of NIH research costs in grant allocations lost over the same period (Fang et al., 2012; Stern et al., 2014).

Of course, even a rate of 1% in fraudulent publications is not acceptable and demands attention. You are correct in noting that postpublication analysis by the research community plays a part here. I agree, too, that a questioning mind means scientists often approach debate in ways that overlap with their ability to detect misconduct. Once suspect, however, the methods by which potential misconduct is investigated, and ultimately resolved, must be separated from debate lest the quality and content of debate are degraded. For the pursuit of fraud, anonymity is not the answer, not if due process is to ensure civil society and protect the innocent from denouncement or worse.

Plant Physiology and The Plant Cell have always protected whistleblowers, holding their identities confidential; following initial complaints, investigative and remedial actions are undertaken fairly and rigorously and are the responsibility of the ASPB Ethics Review Committee.⁵ Of course, not all publishers follow a similar practice. Furthermore, there is little communication between journals about misconduct. So, the challenge here is to enable an effective strategy for tackling misconduct on a community-wide basis that protects the scientific community and its integrity while also ensuring the rights of the individual scientist, both accuser and accused.

The third concern relates to attitudes toward retractions and, more importantly, corrections in the literature. Within a research scientist’s mind, correcting errors is all too often associated with misconduct (Van Noorden, 2011). Yet a substantial percentage of retractions and the much larger body of corrections arise from honest mistakes, omissions, and the generation of new data (Van Noorden, 2011; Fang et al., 2012; Stern et al., 2014). Correcting the literature in these circumstances is appropriate and must be recognized and valued. The challenge in this case is to encourage a constructive and transparent approach to making such corrections. Again, such an approach must dissociate the process of correction from that of handling fraud in order to avoid the stigma of misconduct.

As you have noted, our discussion here is a step in the right direction, if only a small one. As a final thought, let me add that the science journalist Leonid Schneider, although initially one of my fiercest critics, has also come to question the anonymous voice.⁶ Indeed, it is testimony to the benefits of open debate that he and I have continued to correspond since our first contact through PubPeer and, like you, we are now working together to address the concerns of bona fide fraud.

Yours sincerely,

Mike

LITERATURE CITED

Author notes