Introduction

The Internet has been described variously as "the grandest revolution in the capture and dissemination of emerging academic and professional knowledge and information since Caxton developed his printing press,"[1] "the most significant advance since the penny stamp,"[2] and (paraphrased) "the end of the need to print to paper....the harbinger of an economic, social, and cultural revolution as significant as industrialization and urbanization in the 18th and 19th Centuries....people will not just work differently; they will think differently."[3]

One of the problems with being at the start of a revolution is that one can hardly reflect on it adequately. It is a little like the schoolboy joke of the letter saying "Dear Mum, All is well over here in the Holy Roman Empire, except that the Hundred Years War started today". The joke being, of course, that you can look back on the Hundred Years War and see what its shapes and patterns were, how long it lasted, and who won; indeed, you can give it a name. But looking forward, you can't make a lot of sense of it.

There is a sense of history in these early explorations of what modem-linked computers can do and a sense of typical early phase revolution behavior taking place too; the zealots, the reactionaries, the contras, the indifferent, the opportunists. Those involved so far in trying to make sense of the capabilities of the Internet, and in playing a part in the crafting of its future shape, are what Everett Rogers would describe as "innovators," or at least "early adopters;"[4] that is to say, they are people inclined towards leading edge exploration. As such, the documented discussions are likely to err on the side of enthusiasm, but in the interests of balance I have tried to draw together dissenting voices too.

The purpose of this article is to discuss electronic peer review of academic papers on the Internet, drawing specifically from a conference of that name which I moderated between April and August 1995, sponsored by MCB University Press Ltd,[5] and specifically by the journal Environmental Management and Health.[6] The reasons for so doing are several. First, it was an interesting conference. Second, the field itself is an interesting one to any of us involved in learned publishing, writing, and editing. There is a debate about whether the form of the carrying medium changes the form and function of that which is carried. And thirdly, this is one of the first attempts (to my knowledge) to construct an article in the academic form from the proceedings of a Net conference. As such, I have tried to settle on a referencing and reporting style that is generalisable for others.

The article covers the following broad areas:

• what peer review means in academic publishing
• what things change, and what stay the same, when the review medium shifts from paper to electronic
• the implication of Internet review to the "blind" (that is, anonymous) assessment of papers
• Internet review and possible plagiarism
• some guidelines for potential authors, editors, and reviewers in using this medium
• some indications of further research directions

We begin with a brief review of what electronics might mean to publishing as an industry.

Electronic publishing—where are we?

When Mark Twain was in London in the early years of the century, he was shown a cable from the New York Journal which asked for circumstances of his death there. Twain cabled back to the editor of the Journal, ending with "Reports of my death are greatly exaggerated."

Simon Jenkins, former editor of the London Times, made a similar observation about the impact of the Internet on the print medium.[7] Television, 24-hour news radio, fax machines, and, more recently, networked information technology have been forecast to kill the printed page among them. But, Jenkins suggests, they have so far failed to make a dent in the printed word. British newspaper sales have been steady since 1945 at about 17 million daily; book sales are increasing at about two per cent per annum over the past ten years, and newspaper sales are rising in the United States, France, Germany, and Japan. Jenkins suggests that the demise of printed paper has been an exaggeration, even in the wake of the Internet, due to the fundamental portability and attractiveness of words on paper.

Chad Rubal, writing in Marketing News,[8] points to a similar picture of news publishing, although describing the two technologies as developing in parallel, rather than one threatening to supplant, and consequently being fought off by the other. Rubal cites U.S. publications such as the Gainesville Sun and the San Jose Mercury News, which have developed Internet pages to complement their paper products.

But if the newspaper publishing prognosis may be favorable, what of that for periodicals? Kimbell's report in Business Marketing [9] paints a picture of "stabs in the dark;" a confused flurry of activity to jump on the electronic bandwagon. Kimbell quotes CMP Publications' editorial director Jerry Colonna predicting that "a lot of people will lose their shirts in the process."

Allyn Jackson, reporting on the 1994 Mathematical Sciences Research Institute meeting, noted "in excess of one hundred peer-reviewed online journals, available for free and produced by scholars."[10] At the same conference, Andrew Odlyzko of AT & T Bell Laboratories predicted "the demise of traditional scholarly publishing ... Publishers today often insist on author-prepared manuscripts...and once you reach that stage when you typeset the papers yourself, the question is, why not eliminate the publisher and distribute the information directly? ... Authors are increasingly viewing the price of journal subscriptions as a barrier to the wide dissemination of their work."[11]

Jackson pointed to the possible vulnerabilities of the scholarly publisher with the rise of the Internet: "Universities pay faculty to produce research, the research is turned over to publishers, and the universities repurchase that research in the form of journal subscriptions. Free electronic journals cut publishers out of the loop, posing the risk of huge revenue losses."[10]

Arnold reaffirms the threat to traditional publishing in his eloquent essay, "The Body in the Virtual Library."[12] Arnold likens academic publishers to "Neanderthals clustered round a fire" and electronic "virtual" librarians, information providers, and accessors, to "Cro-Magnon humans sweeping down from the mountainside." He continues the analogy by suggesting "that the E-people contemplate not conquest but intermarriage with the P-people," by using the capability of the new systems coupled to the virtues and controls of the old. He quotes Day [13] in classifying electronic publishing as a "Type 2 innovation," which bears the seeds of the destruction of the industry that created it, rather than a "Type 1 innovation," which changes the physical form of the product.

The need for publishers to retain their position as leaders of the scholarly information supply chain by finding new ways to consolidate their added value is clear. Some of the traditionally necessary functions of the publisher, such as typesetting and distribution, have become easily accessible to most authors. However, the publisher's role is far more profound than mere typesetting and printing. It is both the organization (administration and management) and authorization (through peer review) of the process. It is the authorization aspect we will address next.

Peer review in academic publishing

Peer review is "aimed at making a publication reflective of the peer community, not the editor's individual preferences and scope of knowledge."[14] Peer review is normally implemented as follows. The editor receives a paper and selects one or two reviewers, or referees, from a pre- selected Review Board. Referees make recommendations about whether the editor should accept the submission, reject it, or ask for revision.[14] Revision is the most common conclusion—if an editor has made an initial "weeding out" decision, an estimated 80 % of papers are returned for revision, with 10% accepted as is, and 10% rejected by reviewers.[15]

The review process is an anonymous one (usually described as "blind"), in that the author's name will be removed from the paper. Thus, reviewers are prejudiced neither for nor against a particular author. Reciprocally, the reviewers of a paper are not known to the author, although he or she may know the composition of the review board as a whole. A reviewer's comments are, therefore, anonymous and never attributed by an editor.

Reviewers are, typically, other scholars—researchers, teachers, and authors in and around the subject areas represented by the journal—although managers, consultants, and other practitioners often appear on editorial Review Boards. A good editor will select reviewers well, and generally will trust their recommendations, although he or she will make a final decision. An editor will often take a first view of a submitted paper to decide whether it should be sent for review at all—whether it is within the editorial scope of the journal, whether it appears obviously and seriously flawed, and so on. Thus, an author submitting a paper to a reviewed (the term is sometimes "refereed") journal may be rejected before, during, and possibly after the review process, but may not be accepted until the review process has been undertaken.

The Electronic Peer Review Conference

The Internet conference described above sought to explore the issues surrounding electronic peer review. It began with a keynote paper, authored by myself,[14] but drawing heavily on ideas expressed by Stevan Harnad at a 1993 conference.[16] Harnad also edits a peer reviewed electronic journal, "Psycoloquy." Harnad argues strongly for the advantages that electronic review brings: "The Net offers the possibility of implementing peer review more efficiently and equitably, and of supplementing it with what is the Net's real revolutionary dimension: interactive publication in the form of open peer commentary on published work"

Although robust and well accepted, the traditional blind review process has its critics. One drawback is speed, or the lack of it. The process of submission, review, recommendation, revision, re-review (if appropriate), acceptance, and publication can be interminable. Four to six months is fast for a scholarly journal; two years not uncommon.

The second criticism is the formulaic approach often adopted by reviewers—articles must have a beginning, middle, and end; anecdote is frowned upon; a formality of tone must be adopted; there is a strong bias towards linearity of approach and analysis over intuition; new ideas or new ways of doing things are (some say) dissuaded. A scholarly journal can be likened to a club where non-members will not be told the house rules, but are expected to know them, and will not be admitted if they transgress.

Harnad suggests a system by which an author posts a "working paper" on the Net, which can be commented on by any and all who are interested, followed by a traditionally rigorous selection process, which does however take into account the comments made in open review. He describes this process as "scholarly skywriting"[16]—real-time interactive and open comment and reply, which is however more slow-paced and reflective than the "undisciplined babble" of a live, face-to-face symposium.

Writing on a different occasion, Harnad differentiated strongly between peer commentary and peer review,[17] citing commentary (his "scholarly skywriting") as a "supplement to, not a substitute for, peer review." Harnad is firm on the necessity for peer review to remain as a quality control mechanism: "if there is no peer reviewed region at all [on the Internet] ...you will have no idea what is worth reading...peer review is an active feedback mechanism for quality control."[17]

These thoughts formed the basis of the discussion proposition for the Electronic Peer Review conference cited above. We did not know how discussions would pan out or where they would branch to. In the end, we gathered a number of themes, which will be discussed below.

"Plus ca change, plus c'est la meme chose"

The more things change, the more they stay the same. The Internet is a dramatic change of medium, from paper exchange in sealed envelopes, to largely open information access from a computer screen via a telephone line, modem, and appropriate software (such as Netscape).

But has the substance of the academic review and publication process changed? The information for authors provided by the Web Journal of Legal Issues [18] cited by Wills [19] looks, well, familiar. In fact, one would hardly differentiate the Notes for Authors from those of any other academic journal. Wills makes the point that "precise standards of quality can be maintained" in electronic publishing, and points out that the Higher Education Funding Council for England and Wales "have announced that articles published in electronic journals will be treated on the same basis as those appearing in printed journals for purposes of the 1996 Research Assessment Exercise.[20]

Peters [21] suggested assessment and review guidelines, and concluded, "Is there a difference between assessment of a paper-based article and of an electronic one? I think not." The aforementioned Simon Jenkins reinforces the point that the editorial judgment process per se must remain unchanged: "of all the boasts of the Internet, the most absurd is that it does not edit...most of its sites are like a newspaper correspondent's office: 95 percent pure rubbish....the last thing a user wants is the undiluted sweepings of a hundred minds. Yet, the more it edits, the more it approximates to what it purports to supplant: the printed book or newspaper."[7]

As a newspaperman, Jenkins is presumably unfamiliar with the sometimes glacial progress of academic ideas. If the Net can bring something of the hustle and bustle of the newspaper world to the development of learned ideas, it will have achieved something that the print medium has not so far achieved.

Howard [22] commented that the assessment process described by Peters "fails to make the distinction between "assessment" as an outcome or as a process in the context of electronic exchange. The outcome, insofar as a paper in its unchanged form is concerned, should be the same whether the review is undertaken along traditional lines or through the electronic medium. The latter may however lead to profoundly different consequences if an "open" review process is adopted.

Open review should lead (eventually!) to a more profound contribution to knowledge, but is the process likely to be acceptable to academics who are unable to "patent" their original thoughts? A practical and beneficial outcome of review through the electronic medium is the continual and visible pressure which can be exerted on reviewers to respond."

Oliver [23] reinforced the idea that the real value of open peer review was in multiple participation: "The most important practical problem surely is how can we get ALL the reviewers to join an electronic system. At present the best use will be for posting Working Papers or Discussion Papers where, on an open 'conferencing' basis, comments can be collected from a disparate group of interested folk."

In summary, then, there may be no appreciable differences in the assessment process per se. There should be no "slackening" of standards in transferring academic publishing from one medium to another. But to gain benefits from the new medium, more than the traditional one or two peer reviewers might be expected or encouraged to join in at the working paper stage. This openness is well summed up by Davies: "Yes, the Internet as a medium will develop its own form of scholarly review, separate but with an integrity of its own. The critical contribution that reviewing via the Internet can make is the speed at which comments and 'comments on comments' are made. The fact that anyone can 'review' avoids the elitism that sometimes occurs in academic journals, and is analogous to defending your thesis in the town square speakers' corner. Within all the 'noise' that is likely to appear in such a forum, the occasional 'jewel' of wisdom may progress scholarly knowledge in ways totally unexpected and counter to the linear logic that the traditional review process encourages."[24]

Two issues lurked at the back of these positive comments, however. The first was the removal of the "blindness"—the anonymity—from the review process. The second was the issue of protection of ideas and openness to plagiarism. We will address these below.

Reviewer anonymity

The anonymity of the review process is one of its cherished bastions. As stated above, authors are not identified to reviewers, which protects them against bias towards (or against) people they know, people they have heard of, and so on. Peer review on the Net may make it more difficult for anonymity to be preserved—although not necessarily. The CompuServe "CB Simulator," for example, is an open forum exchange of news and chat, where identity is shielded if required through caller number or nickname.

The issue of reviewer identity was of concern to Rushton, [25] who stated that reviewers would necessarily "pull their punches" if they were readily identifiable. Davies [26] agreed, comparing open review to the "leniency error" effect experienced in performance appraisal, while making the point that electronic peer review can still preserve anonymity: "We can post through anonymous mailers. Additionally, a paper can be forwarded to a server and reposted anonymously on the Internet."

Barker [27] reinforces this in suggesting that "technological and systematic" solutions can easily guarantee anonymity if desired. Peters [28] suggested that the problem may run deeper—that anonymity of comment allowed reviewers to make destructive rather than constructive criticism, which countermanded the intention of the review process: "It is perfectly possible to make hard criticism in a way which others can consume. Granted, it takes more work. But how, for example, do you tell an employee you like and who is generally doing well and who has a great future that he or she has messed up? With care and empathy I think—because you want them to understand what they have done 'wrong' and improve it, without getting disillusioned or hostile. As reviewers, we don't always take time and care to do that."

The implication is that non-blind review may over time lead to a beneficial reconstruction of the peer review process, more akin to development and education than to judgment.

The monster under the bed

One issue mentioned by several participants in the debate on peer review was that of ideas being stolen or plagiarized. The argument can be summarized as follows: Authors will be nervous about posting their ideas as "working papers" for widespread peer review in case others steal their idea and use it themselves. Therefore we need to seek a technological solution if possible.

Davies [29] suggests that working papers might be electronically marked "may not be cited" on each page of text, and points out that unreviewed information in, say, the medical field may be positively dangerous (although not in social sciences and business). Peters [30] suggested that the theft of ideas may be "an academic equivalent of a monster under a child's bed"—frightening, but with no real substance. He suggests that "cyberspace pirates" waiting to copy and pass off as their own an idea for an academic article may not exist, while acknowledging that a technological solution such as that suggested by Davies [28] may reassure the nervous.

The perception of the reality of the "pirates" threat is an area worthy of investigation for those researching electronic publishing applications. There may be a reluctance for some authors to commit their ideas to the Net for that reason.

Conclusions

Electronic publishing is with us, though at present and in all likelihood until the turn of the century, as a baby cousin of paper-based academic publishing. Undoubtedly, the Internet or something akin to it will become the preferred medium for scholarly publishing within the early years of the new century, if not sooner. Opinion within our conference sample, and within the literature reviewed, tends towards electronic publishing as a "Type 1 innovation"[13]—one that changes form but not, substantially, content—rather than a "Type 2 innovation," which changes the very nature of the industry, and rewrites all the rules in so doing.

A major implication for authors, editors, publishers, and reviewers is the need for familiarity and comfort in using the available technology. Just as in the past ten years the word processor has become a necessary tool for participation in the scholarly publishing game, the Net will be a similar prerequisite. This is not to say in any way that computer literacy will replace the art of crafting, editing, and reviewing an article—simply that it will become the dissemination medium of choice.

Reviewers will, I believe, need to become familiar with a more open, less secretive and anonymous style of review, more akin to development of an idea rather than judgment of it. No slight is intended to the vast majority of academic reviewers who cheerfully and usually without recompense devote time to constructively improving the editorial process—but the required style of so doing is liable to evolve.

Publishing on the Net must be able to leverage its clear strengths—interactivity and speed. We must be able to achieve Harnad's "scholarly skywriting"[16] or we will be failing to utilize the medium to its full advantage. That means that publishers, editors, and review boards must settle on suitable systems solutions that enable widespread participation in some stage of the review process, albeit possibly supported by a second phase review more akin to that currently used. As Barker [27] points out, the editor's control of what actually appears as accepted and "published" material should not be compromised in an academic setting, which means that an editor still must select his or her reviewers in some way, rather than rely on the judgments of self-selected people "sticking their oar in" without appropriate qualification to so do.

The issue of potential piracy of ideas should be investigated further. Scholarly publishing on the Net will be primarily supply driven. Without good authors willing to do it, the thing won't work. If the insecurities of exposing working papers outweigh the benefits of so doing, technological solutions must be found to create security.

This article has attempted to explore some of these issues by reporting on a real-life manifestation of them—an Internet "conference" that explored such issues. In creating it I have attempted to adopt a style of referencing and reporting that makes sense to the academic reader. I will watch and participate with interest as the electronic publishing revolution takes hold in the years to come. Whether we have indeed created a monster which will yet turn on its creator and destroy it, remains to be seen.

Footnote

Dear Mum, the Hundred Years War started today. Reflecting on the past is easy; reflecting on the future rather more difficult. My own feelings on "academic netiquette" are summarized below.

"In the end, style and substance will evolve. Good ideas will emerge, if we let them and encourage them. I would not seek to inhibit them through the laying down of overly rigid procedural guidelines. We have a chance at the outset to encourage the right frameworks for the future, and let us bear in mind that this is, like it or not, an important and revolutionary new information transmission medium, which is going to change all our lives in the years ahead. I would therefore like to think that we can be recording and reflecting on the process of Internet conference editing/moderation, as well as working on the content."[31]

In letting style and substance evolve, we can watch the war develop, or we can take off our jackets, roll up our sleeves, and join in the fight. My publishers at MCB University Press, the sponsors of the electronic peer review conference, have committed to shaping the future, following their invitation to contribute to discussion on its possible shape. During 1996, MCB's editorial support activities will be extended to cover electronic peer review in what I believe will be the world's first systematization of peer review on the Net. This will be branded Better-Ed III; following the earlier release of a unique computerized system of editorial support, reviewer liaison, and manuscript tracking (Better-Ed I) and the Microsoft Windows version of the same (Better-Ed II). Better-Ed III promises to be a unique resource to allow editors to capture some of the knowledge learned from electronic peer review. A follow-up report on its progress should be publishable by mid-1996.

About the Author

References and Notes

1. Wills, G., "Embracing Electronic Publishing," The Learning Organisation, Vol 2, no 4, 1995. [doi: 10.1108/09696479510100364]

2. Keller, A.Z., "Re: Editorial Assessment," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

3. Davidow, W. and M. Malone, The Virtual Corporation, 1993, HarperCollins, New York.

4. Rogers, E.M., Diffusion of Innovations, Macmillan, 1962.

5. Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

6. Environmental Management and Health, available via the World Wide Web at URL [formerly http://www.mcb.co.uk]; ISSN 0956 6163.

7. Jenkins, S., The British Library Chadwyck-Healey Lecture, 1995.

8. Rubal, C., "Don't stop the presses yet," Marketing News (USA), 23 Nov 1994.

9. Kimbell, J., "Mad dash to Online," Business Marketing, March 1994.

10. Jackson, A., Mathematical Sciences Research Institute 1994 Conference Proceedings, MSRI 1995.

11. Odlyzko, A., Mathematical Sciences Research Institute 1994 Conference Proceedings, MSRI 1995.

12. Arnold, K., "The Body in the Virtual Library," contributed to the University of Michigan Journal of Electronic Publishing, 1995. [doi: 10.3998/3336451.0001.104]

13. Day, C., quoted in K. Arnold, "The Body in the Virtual Library," contributed to the University of Michigan Journal of Electronic Publishing, 1995. [doi: 10.3998/3336451.0001.104]

14. Peters, J., "Electronic peer review," keynote paper of the Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

15. Personal experience, based on editing several journals over the past 10 years.

16. Harnad, S., "Implementing Peer Review on the (Inter)Net," International Conference on Refereed Electronic Journals, Winnipeg, Canada, 1993.

17. Harnad, S., "There's Plenty of Room in Cyberspace," Times Higher Educational Supplement, 12/5/95.

18. Web Journal of Legal Issues, published from Newcastle University, England, on http://www.ncl.ac.uk.

19. Wills, M., "Re; Peer Review," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

20. HEFC Circular RAE96, 1/94 1996 Research Assessment Exercise, published by the Higher Education Funding Council for England and Wales.

21. Peters, J., "Assessment of articles; electronic and paper-based", Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

22. Howard, K., "Re; Editorial assessment," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

23. Oliver, C., "Re; Editorial assessment," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

24. Davies, R., "My 2 sens worth", Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

25. Rushton, A., "re: Breaking out of cartels," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

26. Davies, R., "re: Breaking out of cartels," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

27. Barker, J., "re: Breaking out of cartels," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

28. Peters, J., "re: Breaking out of cartels," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

29. Davies, R., "re: Plagiarism," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

30. Peters, J., "re: Plagiarism," Electronic Peer Review Internet Conference, 1995, [formerly www.mcb.co.uk].

31. Peters, J., "MCB University Press Internet Conferencing," guidelines produced for MCB University Press Ltd, May 1995.