Journal editors, grant administrators, and others rely on peers to
assess the quality of proposed and published research. Some parts of
an application or manuscript can be checked fairly easily. Are the calculations
correct? Is the method that has been used or proposed appropriate? Do
the reported results support the conclusions? Other parts are more difficult
to confirm. Have the data been accurately recorded and reported? Were
the experiments run? Did the subjects give consent? Do the articles
cited in the references and bibliography contain the information they
are said to contain?
Peers who are asked to make judgments about the quality of a proposed
or completed project must do their best to determine whether the work
they have been asked to review is internally consistent and conforms
to the practices of their field of research. This certainly includes:
- assessing whether the research methods are appropriate;
- checking calculations and/or confirming the logic of important
- making sure the conclusions are supported by the evidence presented;
- confirming that the relevant literature has been consulted and
At the very least, peer reviewers should be expected to assess whether
the manuscript or proposal under review makes sense and conforms to
accepted practices, based on the information presented.
Research that conforms to accepted practices can still have problems.
Research quality can be compromised by:
- careless mistakes made in reporting data and/or listing citations;
- the deliberate fabrication and falsification of data;
- improper use of material by others (plagiarism);
- inaccurate reporting of conflicts of interest, contributors/authors;
- the failure to mention important prior work, either by others or
by the researcher submitting a paper for publication.
However, how much peer reviewers can or should do to detect these
and other deceptive or sloppy practices remains subject to debate.
There are limits to the amount of checking that is both reasonable
and practical. Unless given permission to do so, reviewers should not
discuss the work they are reviewing with the authors. In many cases,
reviews are “blind” (no author identification), so reviewers
could not check with authors even if they wanted to. In addition, it
is not reasonable to expect reviewers to check every reference and detail.
The fact remains, however, that peer reviewers frequently miss problems
that might have been detected had the reviewer checked a little more
If you agree to serve as a peer reviewer, remember that you have essentially
been asked to provide your stamp of approval for someone else’s
work. In such circumstances, it is wise to do your homework. Do not
give your stamp of approval too easily.