Abstract

While book reviews have been a significant and influential form of scholarly communication since the earliest scholarly journals, this form of post-publication peer review remains limited to print books for the most part. Despite the increase in electronic publications—e-books and more experimental forms—book reviews continue to represent paper publications almost exclusively. There has been attention to incorporating pre-publication peer review into the process of publishing digital scholarship, but the post-publication review has received far less consideration. Given its long–standing importance, the absence of the equivalent of the book review for the world of electronic scholarship may impact its academic acceptance, especially for promotion and tenure.

Introduction

Academic scholarship is heavily dependent on the many traditional measures of peer review. It is widely acknowledged that strong and sustainable peer review must play an important role in digital scholarship: “new forms of scholarship must be perceived as having undergone rigorous peer review.”[1] “Pre-publication” (or “publication-based”)[2] peer review for both journal articles and books, the evaluation by which scholars determine the worth of a piece of scholarship and move it toward publication, is the most recognized and important form, both for print and digital scholarship. New forms of pre-publication review, like open peer review, are emerging and fit naturally with digital publications. Post-publication review for monographs in the form of book reviews is also a crucial part of the system of scholarly communications, particularly for the humanities, and one that provides rewards for the reviewer and the reviewed. Yet in the world of digital publications, this particular measure of scholarship is largely missing; it is not yet clear whether equivalents will develop, nor is it clear whether such equivalents are essential to the integration and acceptance of digital scholarship in academia.

Despite predictions for the disruption, innovation, and transformation of scholarly communications from digital publications and projects in the digital humanities, there remain many uncertainties about their role in academia, particularly in promotion and tenure. There are in fact few academic positions in digital humanities per se (while many scholars of long–standing academic departments may claim secondary specialization, in any of the different manifestations of the term); those scholars who do pursue digital work as a primary focus are often situated outside academic faculties.[3]

The way scholars actually choose to publish has changed little, and acceptance of digital publications has been slow, an exception being journals whose primary or only mode of dissemination is now digital (electronic journals are usually edited and reviewed in a similar fashion to their print equivalents and have evolved from or emulate their print ancestors or counterparts; they do not, in many ways, challenge traditional notions of scholarly publication[4]). But other types of scholarly publication: digital monographs without print equivalents, digital scholarship that can exist only online, or digital collections or libraries, have not received the same level of academic acceptance, either in the form of adoption by authors or recognition by peers. Nor have such publications, even those that represent sustained, original scholarship received general recognition within the tenure and promotion process.[5] The preponderance of national and institutional reports and recommendations on innovative digital publications and academic recognition points up this lack of acceptance.[6] Meanwhile, print monographs (increasingly with exact digital surrogates) continue to be published in great numbers even as academic presses struggle. Junior scholars in particular have little incentive to publish in new ways, and few models and tools exist for successful, sustainable, and stable all–digital publications. Importantly, those that do exist are not recognized with reviews.

Book reviews and digital publications

Since book reviews have always played an important role in scholarly discourse, particularly in the humanities, the absence of an equivalent type of post-publication peer review in the world of digital publications is significant. Digital publications are rarely seriously reviewed unless they have a print component; post-publication peer review remains very much within the world of print publications.

Academic scholarship without the equivalent of the book review would mark a major break in the tradition of scholarly communications. Book reviews were a major part of the very first European scholarly journals, the French Journal des Sçavans (1665), which was largely a compilation of extracts of books. A few months later, the Philosophical Transactions of the Royal Society was founded in England; it too contained reviews of books.[7] These early book reviews are completely recognizable: reviews have followed the same form ever since, with the bibliographic information about the book, description, and assessment. Reviewers and review journals through the eighteenth century were dedicated to the presentation and review of all literature; by the nineteenth century, however, both the books reviewed and the reviews were to reach a higher standard of quality, and as more books were published, it was no longer possible to review all books.[8] Thus the book review became a marker of the importance of the book (sometimes even regardless of whether the review itself is good or bad). Book reviews have also changed and evolved, to limit the type of publications reviewed. In recent decades, for example, there has been a diminution in the number of reviews of anything other than the scholarly monograph; there are now few reviews of primary sources, bibliographies, or reference materials, where once this type of review was more prevalent.[9] (This last has particular implications for review of scholarly digital collections or digital libraries.)

The 2007 Report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion includes the book review as a form of scholarly communication that should be encouraged for scholars at all levels: “The book review plays an essential role in humanities scholarship, disseminating information about new works, critically evaluating them, and engaging them in often pointed debate. As such, the best published reviews constitute an important scholarly activity that helps direct, alter, and sustain ongoing conversations in the field.” In recognizing the importance of book reviews for the humanities, the report adds recommendations for increasing the importance of book reviews as academic scholarship, including urging senior scholars to write book reviews in order to “model this critical genre for younger scholars and to help identify significant new work.”[10] Dossiers for tenure and promotion of faculty book authors are expected to include published reviews of the work, and academic recognition for writing book reviews (as scholarship and as service to the profession) acknowledges the importance of the genre to scholarship. In fact, there is little difficulty in finding reviewers for books; writing reviews is both useful and prestigious for the reviewer.[11]

But given the MLA’s own emphasis on the importance of reviews of books, where is scholarship in new media to be reviewed? Is the importance of post-publication review great enough that reviews of digital publications, of whatever sort, will eventually infiltrate journals? Or do new methods of producing scholarship mean a new paradigm for scholarly discourse, one that dispenses with the formalities of post-publication review? Can digital publications that evolve in parallel with, or even supplant, monographs ever receive acceptance in academia, or recognition for tenure and promotion, without such reviews? Can new forms of peer review made possible online, like crowd–sourced peer review, replace the traditional form of book reviews?

Wholly digital publications aside, even digital books that can be represented in print format fail to acquire reviews. A recently published assessment of the Gutenberg-e project provides a specific and detailed account of such issues.[12] Gutenberg-e was an initiative begun in 1999, led by Robert Darnton and funded by the Mellon Foundation, to encourage the production of digital monographs in history. The intent was that the authors would take advantage of the digital medium in presenting their scholarship while not abrogating the prestige of the monograph, a critical credential for scholarly advancement. The books were published by Columbia University Press and made available online as part of a subscription package, not as individual titles. Because of the unusual nature of the publications and because the model for distribution of the titles followed the model for electronic journals, the titles did not circulate widely, were only available at some academic institutions, and were not publicized for review. Very few book reviews appeared, and those mostly in the American Historical Review (the American Historical Association had co–sponsored the project). A commissioned article (also in the AHR) called attention to the lack of reviews, which led Gutenberg-e authors to request that the press produce print versions of most of the Gutenberg-e titles, which could be sent out for review.[13] The print books (which could not fully represent the electronic publication) generated a few more reviews, but the numbers were insufficient. “While the print editions of the Gutenberg-e titles were a pragmatic and necessary response to a serious problem, their very existence showed the degree to which electronic monographs had so far failed to achieve professional legitimacy.”[14]

With such issues in mind, some new digital publishing initiatives firmly resist consideration of a digital product that cannot be represented on paper. A new open-access initiative at the University of California, Berkeley, California Classical Studies, for example, specifies the publications will be printed on demand in order to send copies for review; that the digital will be subsumed into the traditions of the print world.[15]

The experience of BMCR and BMERR

Even electronic journals have not provided a venue for reviews of electronic publications. Bryn Mawr Classical Review (BMCR)[16] has been published electronically since its inception in 1990, but despite being in the forefront of online open-access journals, counts among its several thousand published reviews only a handful of reviews of electronic publications.

In 2005, the editors of BMCR wrote, in a failed attempt to attract reviews of scholarly electronic publications:

In this, BMCR is not alone; another long–lived electronic review site, H-Net Reviews in the Humanities and Social Sciences[18] likewise advertises reviews of non-monographic electronic publications (e.g., websites, software), but in practice has published hardly any reviews of a digital publication for nearly a decade.[19]

Ironically, there was probably more interest in post-publication reviews of electronic publications in their early years than now, when the achievements are so much greater. While BMCR had never shied from publishing reviews of electronic resources, in 1998, Bryn Mawr Electronic Resources Review (BMERR), a sister publication to BMCR, was inaugurated in order to focus exclusively on digital publications.[20] During its brief existence from 1998 to 2000, 33 reviews of websites and CD-ROMs were published. BMERR made an effort to address the permanence and the scholarship of the sites by including in reviews questions such as peer review, availability, permanence, and publication date.[21] Somewhat surprisingly, especially for the 1999 and 2000 reviews, many websites still exist (although few of those reviewed in 1998 do), but the CD-ROMs no longer work.[22] Some websites exist in almost unchanged form, obvious immediately by their late 1990s colors and animated gifs. Some have evolved into sophisticated databases and publications (e.g., the Theban Mapping Project).[23]

The reviews in BMERR, though, are mostly of historical interest as a picture of digital websites and projects in classics in the late 1990s, since apart from the now inoperable CD-ROMs, none of the electronic resources reviewed were intended to remain static. As the resource changed, and because there was no version of record to review, the review lost its worth, and for those resources that remain in unchanged form, that very fact has rendered the sites obsolete, and the review as well. Reviews of electronic resources, and particularly reviews where a significant amount of each review was given over to questions of sustainability and usability are useful to the authors of those projects or resources; they can use those reviews to improve their publications. Once that digital publication has changed or disappeared, however, the review itself has lost much of its value, since the scholarship that was reviewed no longer exists.

Book reviews are more enduring works of scholarship because of the physical endurance of the book and the usually unchanging text (few books achieve second editions). Apart from other reasons that reviewers are reluctant to take on electronic resources (the lack of tangible reward in the form of a free book), reviews of digital non–monographic publications may seem from the outset unlikely to be considered scholarship in the same vein. BMERR is a picture of digital websites and projects in classics in the late 1990s; the reviews have lost scholarly value. The problem continues. The 2005 request in BMCR for reviewers of digital publications solicited reviews for four publications. Two of them, Aphrodisias in Late Antiquity and the Vindolanda Tablets Online, are enhanced (and seemingly fixed and stable) digital editions of works first published in print,[24] but the other two publications are no longer at the advertised website. But in any case, the plea generated no reviews for these works.

Lack of standards for reviewing digital work is also a considerable barrier to achieving the equivalent of the book review. Book reviews follow conventional strategies for interaction between reviewer and author. “Reviewers and their reviews are shaped by the expectations and practices of their disciplines, and part of a reviewer’s competence lies in the appropriate expression of criticism, attending to disciplinary practices that embody values of collegial respect and scholastic fairness. In these ways, the book review not only draws on readers’ familiarity with the research networks and disciplinary knowledge of the field, but also on an interpretive framework which includes an understanding of the appropriate social interactions.” Reviews of digital scholarship often focus on usability and design in a way that is only tangentially covered in book reviews; a book review may include a comment on the attractiveness of the book, quality of illustrations, or errata but usually only in a sentence or two. Reviews of reference works and collections of primary sources are perhaps not that dissimilar, but reviews of such publications—especially as most transition to online publications—are themselves rare. While there has been concern to note that digital scholarship should be evaluated for itself, not as an exact equivalent to print scholarship, it is equally important that reviews of digital projects evaluate the intellectual contribution of the scholarship, and not just the user’s experience. Usability is important to note if it impinges on the comprehension of the work, but review should address the contribution of the scholarship.

Experiments in venues for reviews of digital publications continue. A new journal, DHCommons is envisioned as a platform for review of “mid–stage digital projects” and is now (2014) soliciting suggestions from authors and creators of those digital projects; it “will become the robust and recognizable system of academic credit that its practitioners require.”[25] But the reviews suggested for DHCommons may suffer the same way BMERR did. Proposed reviews for DHCommons should cover contribution, presentation, and preservation of digital humanities projects, but discussion of both presentation and preservation by their very nature will only be valid for a limited time.

Changing notions of post-publication review

Terminology surrounding peer review has itself changed because of the fluid nature of publication of digital scholarship: “post-publication review” has come to be applied to new models of open peer review for journals and books. Kathleen Fitzpatrick has been in the forefront of arguing for new forms of peer review and in her book and online publication Planned Obsolescence: Publishing, Technology, and the Future of the Academy, both enacts and discusses these changes. She argues for substantive reevaluation of the forms of scholarly publishing, and most notably reconsideration of the peer review process.[26] The traditional process of peer review, she argues, is one that should change in order to take advantage of the open discourse possible between authors and reviewers through online tools; she promulgates the benefits of peer review, with open discussion and revision, and a questioning of the meaning of scholarly authority. In using the term “post-publication peer review” for review of scholarship posted online, she explicitly undermines the traditional forms of scholarly (usually blind) peer review conducted by academic presses, but also edges in on the book review, the pre-eminent post-publication review.

Several experiments in open peer review have been conducted in recent years, including Fitzpatrick’s book, which was submitted for conventional pre-publication peer review by the press, but was also posted as a series of open posts on a platform (MediaCommons) that allows comments at the paragraph and section level.[27] Experiments in open peer review have been judged as both failures and successes; best–known, perhaps, is the trial the journal Nature conducted in 2006, whereby authors of articles could choose to make their articles available for open review while the articles were under consideration by the journal (without, however, having the open review count for determination of which papers would be published). In this case, the articles under open peer review were not considered “published”—the papers, and the comments, were taken down once the experiment was over. The editors judged the experiment a failure: that interest in the potential (by authors and by readers of the papers) was not matched by sufficient participation (almost half the 71 papers received no comments). They concluded that “Feedback suggests that there is a marked reluctance among researchers to offer open comments.”[28] Fitzpatrick has frequently criticized the Nature study by objecting to the lack of importance of the open peer review (since the reviews did not count for acceptance of the articles), and pointing to the success of open review in other settings. More recently, crowd-sourcing peer review of two issues of Shakespeare Quarterly (also through MediaCommons) did not generate large numbers of comments, but were viewed successes by the editors; here, the crowd review did help determine the fate of the articles, all of which were published in the journal.[29] The open peer review remains available for Shakespeare Quarterly, as it did not for Nature. Each draft and the review comment themselves represent a sort of publication and post-publication peer review, parallel to the published article, although the comments were intended to guide the authors in revision of papers for publication as a traditional article.

Such open peer review might also mark the worth of a publication in an ongoing way. In a discussion of open review in postmedieval FORUM, whose editors have tried open peer review, Eileen Joy writes that she hopes that “more work is published without too many editorial restraints prior to publication, and post-publication review, where work rises and falls according to its merits (or lack thereof), as determined by a disciplinary community, or according to whoever might need it at any given time for whatever reasons, becomes the measure of ‘achievement’...”[30] One such project under development, Hypothes.is, is intended to allow open (pseudonymous or not) annotation, or peer review, of web documents (particularly scholarly communications). Hypothesis will allow comments on specific parts of an article through software separate from the presentation of the scholarship itself.[31]

The possibilities for naming reviewers are intended to counteract the burden of performing anonymous peer review. Regardless, one of the issues with this type of post-publication peer review, in the form of unedited and informal comments on digital texts, whether integrated with the publication platform, like MediaCommons, or published separately (as planned by Hypothes.is) is that it occurs without academic recognition (although with peer recognition). Fitzpatrick acknowledges this, and argues for the necessity of changing the system to allow for and acknowledge different modes of review.[32]

Another model that tweaks the notion of pre-publication peer review but where reviews can be published along with the final article is the open peer review established by the open access journal in medicine and science, PeerJ. Reviewers can choose provide their names to article authors. Profile pages publicly identify reviewers who provide their names, thus providing a record of their academic service. Authors, moreover, can choose to publish the entire peer review history of the article, and if the reviewer has provided his or her name, then the review itself is open; the editors explicitly hope that reviewers reap academic benefit from such exposure.

Conclusion

These new models, sometimes designated post-publication peer review, each address aspects of the benefits of book reviews, and provide possible hybrid solutions that may help address the lack of reviewers and reviews of digital scholarship. But formalized credit is still lacking in the new environment, regardless of the possibilities of identifying reviewers, and trust in new systems and models has not developed.[33] Meanwhile, book reviews remain an important part of the scholarly communications process. Authors, reviewers, and publishers are invested in the process and all benefit, and given that there are many fewer books than articles published, the system is in any case less burdensome than the anonymous pre-publication peer review process. Book reviews are markers of the legitimacy and importance of a completed scholarly product, the reviews are themselves scholarship and represent the authority of the reviewer, and they promote sales. In the case of electronic publications, especially those that cannot be represented in print form, it can be unclear when the publication is finished (and it may never be finished)—if there is no point at which the publication is complete, a review will never represent the whole and is then bound lose its validity, for it will represent something that no longer exists. Reviewers may also be uncomfortable addressing questions of design, presentation, and usability, and more importantly, such discussions detract from the consideration of the intellectual content of the publication, thus marginalizing the review and the scholarship reviewed even further in a scholarly context. Moreover, and surprisingly importantly, a formal (peer-reviewed, especially) review of a digital publication offers only one reward to the reviewer, instead of two: a scholarly publication may result, but the reviewer does not receive a physical book as a partial payment for the endeavor.[34]

Use of the term “post-publication review” to represent any type of review or comment on something that appears online (whether or not it is a “final” version) muddies the water, perhaps in a beneficial way. Nonetheless, it seems clear that reviewers will not reward potentially mutable digital publications with complete and formal reviews, and the number of people willing to expend effort on unrefereed and informal comments as reviews is limited. BMCR from its inception has allowed formal, peer-reviewed, published responses to reviews. In 2008, the journal expanded to a blog platform to allow more (and faster) responses to reviews, or communication about reviews.[35] The editors envisioned that providing this platform would encourage communication between readers and reviewers, but while the experiment has not been a complete failure, but reviewers and readers of BMCR rarely have any desire to communicate publicly in the informal setting of online commentary. Anecdotally, many reviewers receive private correspondence from readers and from book authors, but such communication remains private. Those who wish to respond to a review usually forego the immediacy of a comment in favor of a slower, formal, peer-reviewed response.

Peer review is often decried as a growing problem in academia, as the need to publish creates an increasing need for time–consuming reviewing.[36] But the formal post-publication review, the book review, does not seem to be one of them. The book review accrues benefit to author, publisher, and reviewer, and one that is important as recognition of the scholarship reviewed (in the tenure and promotion process, for example). New forms of named open review for electronic scholarship may lessen the burden of anonymous review and provide recognition to the reviewer, as PeerJ allows, but this type of gatekeeping peer review is not the equivalent of the post-publication assessment of a completed work of scholarship, itself a work of scholarship. The absence of this form of review for electronic publications may eventually be accepted within the scholarly community, but it is also possible that without this long–established and entrenched form of dialogue and criticism, digital publications will never attain the same status within academia as their print counterparts.

The lack of notice about digital scholarship in a sense signals its producers recognize and are resigned to the role non–traditional digital scholarship will play in academia. Despite the plea from the BMCR editors ten years ago, and despite the interestingly sophisticated nature of much digital scholarship, its publishers and authors rarely attempt to garner reviews. Traditional publishers (with very different economic models than most producers of digital humanities publications) are active in promoting their books and placing books with journals for review. But notice of new electronic scholarship is rarely sent to journals that review books; BMCR almost never receives such notices. Since formal reviews of any digital publications are rare, there may be no expectation of soliciting such reviews. Nonetheless, perhaps producers and publishers of digital scholarship should take a lesson from the print world, and scholars should subject these publications to formal, rigorous, post-publication peer review.

Considerable attention has been devoted to promoting and adapting pre-publication peer review and applying such models to digital publications of various sorts (e.g., the digitally curated and aggregated content about nineteenth–century scholarship in NINES and archaeological data in Open Context),[37] but I would contend that more should be paid to ensuring peer review that results in a scholarly publication. Online review journals like BMCR would be an excellent place to start.

Camilla Mackay is Head of the Carpenter Library and Visual Resources and Scholarly Communications Librarian at Bryn Mawr College while also serving as a Senior Editor for the Bryn Mawr Classical Review.

Notes

1. Diane Harley et al., Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines (Berkeley: Center for Studies in Higher Education, 2010), 9, http://escholarship.org/uc/search?entity=cshe_fsc.

2. Diane Harley and Sophia Krzys Acord, Peer Review in Academic Promotion and Publishing: Its Meaning, Locus, and Future (Berkeley: Center for Studies in Higher Education, 2011): 2, http://escholarship.org/uc/item/1xv148c8.

3. See in particular Jerome McGann, “What do Scholars Want?” chap. 7 in A New Republic of Letters: Memory and Scholarship in the Age of Digital Reproduction (Cambridge, MA: Harvard University Press, 2014). On the many definitions of digital humanities, Matthew K. Gold, Debates in the Digital Humanities (Minneapolis: University Of Minnesota Press, 2012), also at http://dhdebates.gc.cuny.edu/.

4. Open access models do, however, challenge traditional economic models.

5. Christine Borgman distinguishes two types of digital humanities projects, digital libraries, and “assemblages of digitized cultural objects with associated analyses and interpretations...the equivalent of digital books in that they present an integrated research story;” digital open projects can reach large audiences, but the “integrated research story” is the form of scholarship necessary for promotion and tenure. Christine L Borgman, Scholarship in the Digital Age: Information, Infrastructure, and the Internet (Cambridge, Mass: MIT Press, 2007), 220–223.

6. For example, Harley et al., Assessing the Future Landscape of Scholarly Communication; Modern Language Association, “Guidelines for Evaluating Work in Digital Humanities and Digital Media,” revised 2012, http://www.mla.org/resources/documents/rep_it/guidelines_evaluation_digital.

7. Edward A. Bloom, “‘Labors of the Learned’: Neoclassic Book Reviewing Aims and Techniques,” Studies in Philology 54 (1957): 538. Roger Philip McCutcheon, “The ‘Journal Des Scavans’ and the ‘Philosophical Transactions of the Royal Society’,” Studies in Philology 21 (1924): 626–627.

8. “All the researches, speculations, discoveries, and achievements of that age of progress were recorded in these [eighteenth–century] journals by means of a systematic review of as many new publications as possible—ideally, of all.” Derek Roper, Reviewing Before the Edinburgh, 1788–1802 (Newark, DE: University of Delaware Press, 1978): 36–37.

9. Margaret Stieg Dalton, The Origin and Development of Scholarly Historical Periodicals (Tuscaloosa, AL: University of Alabama Press, 1986): 58–59.

10. Modern Language Association, “Report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion” (Modern Language Association of America, 2007), 55–56, http://www.mla.org/pdf/taskforcereport0608.pdf.

11. Rebecca Griffiths, Michael Dawson, and Matthew Rascoff, “Scholarly Communications in the History Discipline: A Report Commissioned by JSTOR,” August 31, 2006, pp. 6–7, http://www.sr.ithaka.org/sites/default/files/reports/4.11.1.pdf.

12. John T. Seaman Jr. and Margaret B. W. Graham, “Sustainability and the Scholarly Enterprise: A History of Gutenberg–e,” Journal of Scholarly Publishing 43 (2012): 257–293.

13. Patrick Manning, “Gutenberg-e: Electronic Entry to the Historical Professoriate,” The American Historical Review 109 (2004): 1505–1526.

14. Seaman and Graham, “Sustainability and the Scholarly Enterprise,” 279.

15. http://calclassicalstudies.org/?page_id=57.

16. http://bmcr.brynmawr.edu.

17. Richard Hamilton and James J. O’Donnell, “From the Editors’ Disk: Reviews Wanted,” BMCR 2005.12.20, http://bmcr.brynmawr.edu/2005/2005-12-20.html.

18. https://www.h-net.org/reviews/home.php.

19. There are, of course, reviews of websites and digital scholarship for the library community, but these often focus on usability and scope, and less on the intellectual content of the resource. See Penelope Kaiserlian, “Rotunda: A University Press Starts a Digital Imprint,” in Online Humanities Scholarship: The Shape of Things to Come, Jerome McGann, ed. (Houston: Rice University Press, 2010), http://cnx.org/content/col11199/1.1/, who reports on the reviews of Rotunda editions (a digital imprint of the University of Virginia Press), which were published in journals for libraries like Library Journal and Choice, but adds “We have found that scholarly journals are very slow to review digital publications, or perhaps pass on them altogether.”

20. BMERR was edited by Harrison N. Eiteljorg, II, and exists at http://bmcr.brynmawr.edu/bmerr/ and http://csanet.org/bmerr/.

21. When the reviewer noted this, it was usually to note that there was no indication of provision for permanent archiving.

22. Of the websites reviewed in 1998, thirteen out of fourteen no longer exist. In 1999, of ten websites, one no longer exists, two exist but have not been updated, and the rest have moved, but are still maintained. Of thirteen in 2000, five no longer exist, two have not been updated, and six are still maintained.

23. http://www.thebanmappingproject.com/.

24. [Formerly http://www.insaph.kcl.ac.uk/]; http://vindolanda.csad.ox.ac.uk/index.shtml.

25. http://dhcommons.org/journal.

26. Kathleen Fitzpatrick, “Peer-to-Peer Review and the Future of Scholarly Authority,” Cinema Journal 48, no. 2 (2009): 124–129; Kathleen Fitzpatrick, “Peer-to-Peer Review and the Future of Scholarly Authority,” Social Epistemology 24 (2010): 161–179; Kathleen Fitzpatrick, Planned Obsolescence: Publishing, Technology, and the Future of the Academy (New York: New York University Press, 2011).

27. http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/.

28. “Overview: Nature’s Trial of Open Peer Review” Nature (2006), doi:10.1038/nature05535.

29. Special issue on Shakespeare and New Media (Shakespeare Quarterly 61:3 [2010]), edited by Katherine Rowe (http://mediacommons.futureofthebook.org/mcpress/ShakespeareQuarterly_NewMedia/) and special issue on Shakespeare and Performance (Shakespeare Quarterly 62:3 [2011]), edited by Sarah Werner. (http://mediacommons.futureofthebook.org/mcpress/shakespearequarterlyperformance/). See Sarah Werner, “Building Community,” postmedieval FORUM, http://postmedieval-forum.com/forums/forum-ii-states-of-review/werner/.

30. Eileen Joy, “Not Far From, but Close to, the Madding Crowd Review,” postmedieval FORUM, http://postmedieval-forum.com/forums/forum-ii-states-of-review/joy/.

31. http://hypothes.is.

32. Fitzpatrick, “Peer Review,” chap. 1 in Planned Obsolescence.

33. “Attempts to extend formal peer-review procedures into open Web environments do not appear to be gaining ground, perhaps because, in our opinion, today’s scholars ultimately trust established publishing outlets, are already overburdened, and appear to avoid informal reader–generated open commentary,” Harley and Acord, “Peer Review in Academic Promotion and Publishing,” 46. Also Amy Friedlander, “The Triple Helix: Cyberinfrastructure, Scholarly Communication, and Trust,” The Journal of Electronic Publishing 11.1 (2008), http://dx.doi.org/10.3998/3336451.0011.109.

34. Reviewers for BMCR are reluctant to review e-books; if given the choice, they almost always opt for a physical copy. The reason is not only the book itself, but also the difficulty of reading a scholarly book in electronic format.

35. http://www.bmcreview.org.

36. Harley and Acord, Peer Review in Academic Promotion and Publishing.

37. http://www.nines.org/about/scholarship/peer-review/; http://opencontext.org/about/publishing.