The NYT ran, a few days ago, this piece, “Scholars test web alternative to peer review,” in which the author notes, among other things, that:
[S]ome humanities scholars have begun to challenge the monopoly that peer review has on admission to career-making journals and, as a consequence, to the charmed circle of tenured academe. They argue that in an era of digital media there is a better way to assess the quality of work. Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience. . .
The traditional method, in which independent experts evaluate a submission, often under a veil of anonymity, can take months, even years.
Clubby exclusiveness, sloppy editing and fraud have all marred peer review on occasion. Anonymity can help prevent personal bias, but it can also make reviewers less accountable; exclusiveness can help ensure quality control but can also narrow the range of feedback and participants. Open review more closely resembles Wikipedia behind the scenes, where anyone with an interest can post a comment. This open-door policy has made Wikipedia, on balance, a crucial reference resource. . . .
Any thoughts? Do such developments represent a welcome “democratization” of the process, or instead a failure to appreciate the fact that (as Michele Lamont is quoted in the piece as saying) “knowledge is not democratic”? Or something else?
Posted by Rick Garnett on September 5, 2010 at 05:57 PM
Comments
Peer-reviewed systems are designed with definite conflicts of interest. The gatekeepers are academics who have definitive opinions and ideas on subjects being reviewed. The ability to challenge conventional wisdom can therefore be stunted if the challenged wisdom is that of a peer reviewer’s. This exists no matter how much anonymity goes into the system.
The student-edited law review system removes some of this bias. Sure, student editors have biases as to what they think the issues are or should be, but these are less well-defined than that of professors. Further, a student-editor’s identity is less wrapped up into their scholastic and academic opinions, perceptions, and views. This, I believe, allows greater inclusivity of ideas and allows for more challenges to conventional wisdom than in a peer-reviewed system.
In the end, a challenge to conventional wisdom deserves to be heard, considered, and either accepted or rejected. If it is a poor challenge, then it will be exposed as such post-publication. There is nothing wrong with this exposure occurring post-publication. After all, there are a number of examples of challenges to conventional wisdom being disregarded at first and only accepted much later on.
I personally believe that the end product is also better. Students are, more or less, the audience to which academic works should be directed. They represent intelligent and reasonably informed individuals who, if a paper is written well, can be convinced of certain ideas or propositions. Alternatively, if a paper is written poorly, then students will be less likely to understand or even read it.
This is more of a personal opinion, however. It is based on my own experiences reading humanities papers by well-known scholars that are poorly written, poorly cited, and scattered. This experience was echoed during the time I spent studying in Europe and reading UK and European legal works published through peer-reviewed legal journals. But, as I said, it is only an opinion.
As for knowledge not being democratic; a noble, if flawed sentiment. Knowledge and fact are as much about popularity and perception as reality, even if that should not be the case.
Posted by: John | Sep 5, 2010 7:30:01 PM
The process shouldn’t become democratic, in that publication shouldn’t be based on a certain number of votes in an Internet poll. But Internet-review can provide many of the services that peer-review does, and perhaps do so more efficiently.
One of the main purposes of peer-review (and why it exists in te first place) is that it helps editors maintain the quality (and thus reputation) of their journals.
For that purpose, Internet-review can help replace peer-review. A monitored Internet-comment process could give editors a clue as to the quality of a piece of writing (supposing, of course, that there are enough quality comments.) While I doubt such a process could ever completely replace peer-review, especially at high-tier publications, it could never hurt as a supplementary or initial tool to aid editors.
Posted by: Andrew MacKie-Mason | Sep 5, 2010 7:25:52 PM