'Peer-review:' What does it mean?

Issue: February 2006

ByDouglas W. Jackson, MD

Douglas W. Jackson [photo] --- Douglas W. Jackson, Chief Medical Editor

Medical journals listed in Index Medicus must meet certain standards, including using a peer-review process for article selection. But when you read an article that appears in a "peer-reviewed" journal, what exactly does it mean?

Does it mean the article is scientifically sound and free of author conflicts of interest and of funding-sponsor influence on the published research? Those who read and submit articles under the "peer-reviewed" moniker know that publishers show a wide discrepancy in acceptance guidelines and in the quality of work ultimately published. And recently, unfavorable press reports suggested that nearly one-third of the papers published in peer-reviewed journals from 1990 to 2003 either potentially exaggerated study results or were later seriously questioned.

How authors choose

Authors usually solicit the peer-reviewed journal in their field or a related field, targeting the publications they feel are: the most prestigious; the most widely read or quoted; read by the audience they most want to see their work; or, those most necessary for academic promotions. If their submission represents reasonable work, yet gets rejected, the authors usually turn to a secondary list of publications for resubmission.

How the journals choose

On the other side of the equation, most journal editors and reviewers easily recognize original, quality work. Most submissions, however, fall into the grey area, as do some acceptance criteria. Acceptance rates at some journals can be as low as 10% to 15% - while many are far less selective. Standards tend to become less clear when acceptance rates increase a lot. Sometimes journals publish the numbers of articles they choose because it influences subscription and advertisement income. Elsewhere, reviewer bias and editorial positions can strongly influence which papers get chosen.

Most medical journals, meanwhile, keep the identity of their article reviewers anonymous. Reviewers seldom get paid, and instead volunteer their time for the good of the readership and to help authors improve their articles. Most feel the process helps them stay more current. A few seek out a specific editorial board because they view it as prestigious and/or necessary for their CV. Whatever the motivation, most reviewers improve the final product, in part by asking the authors critical questions the readers will want or should want to know about.

"Would paid, openly declared reviewers, whose comments appear with the article at publication time, improve the quality and openness of the process?"

Do we need blind reviewing?

This whole system has worked reasonably well. But is it time to question the need for blind reviewing and anonymous challenges to the authors? Would readers and authors be better served by seeing, first-hand, reviewers' critiques and suggestions? Reviewer comments could get published alongside the paper with author rights to rebuttal. When an author disagrees with a reviewer(s) and does not want to change his or her opinion, for example, the entire exchange could be discussed in print for the readers' benefit.

Research and policy

The quality of review, analysis and acceptance among all of the leading medical journals must be discussed openly and re-evaluated. Why? Because articles published today are being used more and more as the basis for health policy decisions.

We will be confronted more frequently in the future with evidence-based - or literature-based - denials of requests for medical care, treatment algorithms and guidelines. The public will continue to get much of their medical advice from recently published articles about treatment trends. Is the current system of anonymous, unpaid and at times an unaccountable peer-review process the best system? Would paid, openly declared reviewers, whose comments appear with the article at publication time, improve the quality and openness of the process?