This paper was refereed by the Journal of Electronic Publishing's peer reviewers.

Abstract

This study reports on five disciplinary case studies that explore academic value systems as they influence publishing behavior and attitudes of University of California, Berkeley faculty. The case studies are based on direct interviews with relevant stakeholders — faculty, advancement reviewers, librarians, and editors — in five fields: chemical engineering, anthropology, law and economics, English-language literature, and biostatistics. The results of the study strongly confirm the vital role of peer review in the choices faculty make regarding their publishing behavior. The perceptions and realities of the reward system keep faculty strongly adhered to conventional, high-stature print publications (and their electronic surrogates) as the means of reporting research and having it institutionally evaluated. Perceptions of electronic-only publications are frequently negative because those venues are considered to lack strong peer review and are, consequently, believed to be of relatively lower quality. There is much more experimentation, however, with regard to means of in-progress communication, where single means of publication and communication are not fixed so deeply in values and tradition as they are for final, archival publication. We conclude that approaches that try to "move" faculty and deeply embedded value systems directly toward new forms of archival, "final" publication are destined largely to failure in the short-term. From our perspective, a more promising route is to (1) examine the needs of scholarly researchers for both final and in-progress communications, and (2) determine how those needs are likely to influence future scenarios in a range of disciplinary areas.

Many opportunities and concerns are at play in scholarly communication and publication. These result from capabilities afforded by new technologies, pressures associated with the purchasing power of library budgets,[1] marginal operations by university presses,[2] and the pricing structures of the publishing industry.[3] Those involved in supporting new publishing and communication ventures often see “the lack of willingness of the faculty to change” as a key barrier to moving to more cost-effective publishing models in an environment of escalating costs and constrained resources.[4]

This article summarizes the results of a planning study carried out during the 2005-6 academic year and funded by the Andrew W. Mellon Foundation. In it we describe our assessment of the criteria by which faculty decide when and in what venues to publish or otherwise communicate the results of scholarly research. We were interested in how faculty values relating to advancement and stature in their fields affect these decisions. This was the first step toward making a nuanced and insightful analysis of the roles that universities and faculty do and can play in the resolution of the perceived “crisis in scholarly communication.” Specifically, we developed five disciplinary case studies that are based on direct interviews with relevant stakeholders—faculty, advancement reviewers, librarians, and editors—in five fields. In doing so, we explored academic value systems as they influence publishing behavior and attitudes of a subset of University of California (UC), Berkeley faculty. The larger report is available online.[5]

Our goal was to provide a preliminary descriptive analysis and understanding of the academic value systems associated with scholarly publication and communication, including means of communication extending beyond archival publication. We examined the value systems from two points of view:

  • Within a discipline, we looked at what scholars perceive as necessary to make a name for themselves. We recognized that there are different needs and value systems for different disciplines, that different disciplines are in different stages of incorporating electronic communication, and that some disciplines, e.g., architecture, have products other than text. In addition, we recognized that final archival publication and scholarly communication undertaken “in-progress” to final publication represent different phases of the scholarly communication life cycle; both can contribute to a scholar’s reputation.
  • Within a university, we looked at the value systems of the academic promotion and advancement processes, as perceived by different actors in those processes.

OVERVIEW OF METHODS

The five disciplinary case studies were based on direct interviews with relevant stakeholders, almost all of whom were associated with the UC Berkeley campus. These case studies describe the state of scholarly communication in each of five fields: chemical engineering, anthropology, law and economics, English-language literature, and biostatistics. In the case of law and economics, it was the intersection of these two broad fields that was examined, not the sum.[6] These particular five fields were selected with the goal of obtaining a diverse array of disciplines and publishing traditions, and taking advantage of the fact that at least one member of our project steering committee had deep knowledge of each of the disciplines selected. We also developed two smaller case studies representing the views of librarians and former members of the Committee on Budget and Interdepartmental Relations (the “Budget Committee,” which oversees academic advancement and promotions at Berkeley) across these five disciplines. These “thickly described” case studies can lead us to a more precise identification of the factors associated with academic and disciplinary value systems. More specifically, such case studies should facilitate the identification of the factors that influence attractiveness, viability, and financial sustainability of different methods of scholarly communication for various participants in the publication/communication system, including authors (producers), researchers (consumers), libraries, and publishers.

We conducted formal interviews during the 2005-06 academic year with 49 individuals, 31 of whom were faculty (comprising regular faculty, former and current faculty administrators, and recent ex-Budget Committee members). Of the faculty, 22 were also editors of scholarly journals or had been so in the recent past. Five librarians were interviewed, as were two campus-level academic administrators. The remaining 11 interviewees were drawn from our steering committee. The basic interview protocol, initially designed for faculty, was modified as required over the course of the project to include questions of particular relevance to each class of stakeholders. Because it was a one-year project, our sample was biased (UC Berkeley only) and relatively small (fewer than 10 informants per discipline), so extensions to other institutions and disciplines should be made with caution. As an external check to the UC Berkeley perspective, in each case we included background research on innovations taking place internationally in the targeted fields.

Our research was considerably facilitated by the highly structured review process for appointment, promotion, and advancement at the University of California. That process involves formal review at regular intervals, both before and after tenure is awarded. Reviews are initiated in comprehensive written form by the department chair, using materials assembled and submitted by the faculty member. External letters of evaluation are solicited and included for appointments, promotions, and certain critical advancements within the rank of professor. The package, or “case,” is then reviewed by the dean, by a specially appointed ad hoc committee for promotions, and by the Committee on Academic Personnel (which for the Berkeley campus is named the Committee on Budget and Interdepartmental Relations, or “Budget Committee”), who then make a recommendation. That recommendation is followed by the campus administration in nearly all cases.[7] (The criteria for advancement are put forward in the “University of California Academic Personnel Manual.[8])

FINDINGS

Peer Review

Conventional peer review is so central to scholars’ perception of quality that its retention is essentially a sine qua non for any method of archival publication, new or old, to be effective and valued. Peer review is the hallmark of quality that results from external and independent valuation. It also functions as an effective means of winnowing the papers that a researcher needs to examine in the course of his or her research.

Peer review was cited as an essential factor when faculty were asked about their perceptions of both standard and newer forms of publication, disadvantages of newer forms of publication, where to publish to make a name for oneself in the field, and, of course, when we asked about peer review specifically.

There is a strong tendency for many members of the research community to equate electronic-only publication with lack of peer review, despite the fact that there are many examples to the contrary. Because of the very nature of peer review, this factor inhibits publication in electronic-only venues even among those who are aware of the existence of fully peer-reviewed e-journals. Simply put, they know that the individuals reviewing their work for advancement may well not have that awareness.

It will be important to try to separate the issue of peer review for newer, electronic journals from those issues associated with the fact that most such journals are simply new and not yet well established. To some degree, however, peer review and the means of publication and dissemination can be separated. For example, there are authors whose work is peer reviewed and published in prestigious print journals, but who also retain rights to place the article on their own Web site. As noted by some interviewees, the result is that the work appears to be accessed far more often on the Web site than in the published print journal.

Although peer review is essential, there was some worry among interviewees that the quality of peer review may be declining. The result is that it may be easier to rely on tried-and-true outlets such as prestigious journals and presses. Some also worry that the locus of peer review has, in some cases, moved almost entirely out of the institution itself. Specifically, there is a growing tendency to rely on secondary measures associated with peer review. These include perceived journal quality, selectivity, and/or stature; whether papers or keynote lectures for conferences are invited; and the growing reliance on editors of university presses and reviewers for journals to evaluate scholarly work. (Even though reviewers for university presses are academic faculty, the editor exerts much more independent judgment than is typical for peer-reviewed journals published by scholarly societies.) In some cases, the impact factor[9] may also serve as a gauge of quality, a development that many view as problematic.

Despite the goals and quality of peer review, interviewees mentioned several times that the proliferation of journals has resulted in the possibility of getting almost anything published somewhere, if the author persists in trying to gain acceptance by different journals.

The peer-review process is more complicated for compound disciplines because many such fields are relatively nascent and therefore result in small, specialized communities of scholarship. Faculty in these interdisciplinary fields often prefer to publish within a single traditional discipline because the most highly respected and recognizable outlets reside there; however, divergent expectations (ranging from quantity to methodology to writing style) and standards (especially with regard to quality) among fields often make it difficult for reviewers in standard fields to judge submissions from compound disciplines. Interdisciplinary publications may address this concern more readily as they become more prestigious. In fields that are joined with law, such as law and economics, the utilization and perception of peer review is particularly complicated, given the prominence of student-edited law reviews.

Online Publishing

Although online-only publication may be less of a concern to senior faculty with regard to advancement, they are often hindered in using it by their lack of ability or time. There is also no perceived reward for changing the status quo. Personal desire and interest, however, are often the drivers for participation in newer modes of communication and publication for senior faculty. Publishing in online-only resources is perceived among junior faculty as a possible threat to achieving tenure because online publication may not be counted as much, or even at all, in review. Even if policy indicates that online publications should not be undervalued in consideration of advancement, actual practice may vary.

Some interviewees observed that new modes of communication and publication contribute to a proliferation of scholarly material. The result is that it is more difficult for time-pressed faculty to sift through all that is available in their fields. There is the perception that it is easier to get published in newer electronic-only journals and that they contain material of lesser and dubious quality. There is also a perception that the number of pages publishable in a journal is not restricted by cost for e-journals in the same way that it is for print journals, and thus editors of e-journals are not pressed to be as selective.

Other publishing venues—Web sites, conference papers, etc.—are more often used for in-progress reporting that, interview subjects said, could help scholars build reputations even if they did not directly affect promotion. In some cases, they said that reputation building had as a byproduct the ability to get published in mainstream traditional journals, both because so many disciplinary communities are small and because interest in research gains them invitations to publish. Some interviewees noted, however, that too much reliance on informal “publishing” such as blogs, at the expense of publishing in more traditional outlets, can have a negative effect on tenure and promotion in some fields.

Perceptions of Crisis, Cost and Open Access Issues

Many faculty interviewees believed that UC Berkeley is insulated to a large degree from any crisis in scholarly publishing. The prestige of the institution and the quality of faculty work often enable faculty to publish with the most prestigious journals or presses. For the most part, faculty do not concern themselves with the burden of cost to the institution resulting from the scholarly publication process.

These scholars had minimal, if any, understanding of open-access models, although they were somewhat familiar with the “open” concept. We found that scholars are generally receptive to the ideal of making knowledge available for the “public good.”

Positive Perceptions

Faculty did have a good understanding that the high cost of journals is problematic and faculty in chemical engineering, in particular, viewed open-access models as a possible alternative to commercial presses. Some faculty refuse to publish in particular journals because of their high cost and pricing mechanisms. Senior faculty appeared to be more comfortable with the idea of sharing material at the early stages of work (e.g., preprint servers), as did faculty in chemical engineering, biostatistics, and law and economics in general. Archaeologists already use some open-access Web sites to share field observations.

Negative Perceptions

The largest concern among scholars was the perception that open-access models had little or no means of quality control, such as peer review. Some faculty in biostatistics, interestingly, equated the high cost of print journals with quality and believed that online open-access models are “cheaper” and therefore might be prone to lower standards.

Others expressed fear that scholarly work placed in open-access media could be “stolen,” although faculty with a better understanding of the online publication process saw licensing bodies, such as Creative Commons, as a potential solution.

There was also some concern about the ownership of open-access and author-pays journals. Should universities act as repositories and implement some sort of selection process, there could be legal liabilities regarding the acceptance and rejection of work submitted by the institution’s own faculty, whom the institution then judges for advancement. Faculty also expressed concern regarding how such repositories would be managed, including how subjects would be organized.

Author-pays Publishing Models

Scholars were generally not aware of author-pays models. Once it was explained, faculty responses were universally negative. Paying to publish one’s work was perceived as self-promotion and fundamentally in conflict with the peer-review process. English-language literature faculty, in particular, equated the author-pays models to vanity presses, while those in the sciences equated it with advertising and therefore believed that any such publication would compromise academic integrity. Many faculty realized that publication costs are an issue and believed that the author-pays model could possibly serve to discriminate against countries, institutions, and faculty with fewer financial resources. In particular, scholars from all fields expressed concern that such a model might exacerbate differences between the sciences and the humanities, since funds to cover any charges would likely come from grants. Faculty who were in fields that lacked a sense of urgency in scholarship, especially in English-language literature (and one interviewee in biostatistics), viewed the author-pays model as particularly irrelevant. It should be noted that page charges or submission fees are a reality for disciplines such as the biological sciences and economics. Page charges have now largely disappeared, however, for some scientific disciplines such as chemical engineering. Page chargescould have a particularly chilling effect on those who rely on expensive graphics in publications, especially in the humanities.

Enhanced Capabilities of Electronic Communication

It was clear from our interviews that a distinction needed to be made between where faculty publish their final work and what scholarly material they consume in their day-to-day scholarly practice. Many faculty utilize scholarly material through new modes of communication and publication, but for the last stage of scholarly practice, archival dissemination of scholarly work, scholars said that, with few exceptions, they relied on traditional publishing formats.

Scholars nevertheless described many of the advantages of newer formats. These advantages include the ability to reach a larger audience, ease of access by readers, more rapid publication even when peer reviewed, the ability to search within and across texts, and the opportunity to make use of hyperlinks. Administrators and faculty both cited the fact that new technologies enable innovation in scholarly work. Anthropologists and chemical engineers agreed that moving images and three-dimensional (3D) models are particularly positive attributes. English-language literature faculty noted that technologies enable new ways of conducting scholarly work, most notably manuscript comparison in which single interpretations are no longer necessary because access to multiple interpretations is possible. Faculty, especially chemical engineers, believed that newer technologies have a democratizing effect on scholarship outside of North America. The ability to have enough information (e.g., software code, back-end data, etc.) to enable the reproduction of statistical analyses was of particular importance to faculty in biostatistics.

Data Storage and Management Needs

Data storage and data management needs vary depending on the discipline and even subspecialty. Data can vary both across and within disciplines, as well as among media and formats; whether the data involve interpretive text, visual or motion images, 3D renderings, computer simulations, or observations in numeric or text form can impact the sort of interpretive possibilities open to researchers. Scholars in some fields also rely on existing data sets rather than new data. In the sciences, grant monies often fund data management and storage. It was noted that funders rarely dictate how data should be preserved.

There is little to no institutional support for data management and preservation, according to our interviewees. As a result, individual scholars are responsible for maintaining data integrity. Overall, faculty were concerned about the rapid evolution of technologies, which often results in archaic storage devices and thereby loss of work. In fields with data-rich scholarship, such as biostatistics, interviewees were concerned that not all data can be stored. Some suggested that their department or the university should have policies in place to address this problem.

The Budget Committee

Interviewees who had served on the Budget Committee with terms ending more than two years ago had not encountered the need to review non-conventional forms of publication and communication, and thus this was not a significant issue during their service. Because of academic specialization, the Budget Committee’s nine members rarely have the disciplinary knowledge necessary to judge the research in cases that they review. Thus there is a heavy reliance on peer review of published articles to aid the Budget Committee in its evaluation of scholarly work. For newer forms of publication, Budget Committee members may assume—correctly or not—that there is no associated peer review.

Former Budget Committee members told us that the advancement process should be supportive of non-traditional publishing models, provided that peer review is strongly embedded in the process, and that advancement should be unprejudiced toward those scholars exploring new modes of publication.

Despite faculty perceptions to the contrary, those with Budget Committee experience indicate that there is some degree of flexibility built into the review process. Former Budget Committee members and administrators who receive Budget Committee recommendations commented that the committee reflects standards in disciplinary fields and does not mandate appropriate methods. This effectively serves to maintain the status quobecause disciplinary fields are slow to recognize new forms of publication. Some members and administrators explained that if the faculty member or department chair could make the case that a particular publication outlet was sufficiently peer reviewed for quality and well known within a particular subfield, then the Budget Committee would give it appropriate weight. Regardless, faculty are often unwilling to take risks by using newer publishing technologies that they presume may not be recognized by the Budget Committee as reputable and/or prestigious venues.

Librarians

Librarians appeared to have a much better understanding of available resources and the politics among publishers than Budget Committee members, administrators, and faculty. They often were more technologically savvy than their faculty counterparts and were well aware of new technologies likely to affect available resources.

Unlike many faculty, librarians who were interviewed strongly perceive a crisis in scholarly communication and see the rise in new forms of communication and publication as a positive step, however slow and evolutionary they may be. Librarians indicated that they try to educate faculty about the scholarly communication crisis and how faculty might play a larger role. Although new modes of communication are not widely used by faculty for presenting their work, librarians believe that open-access and author-pays models are viable alternatives to the problem of unsustainable journal costs. Online resources were also viewed largely as advantageous from a consumer perspective for many of the same reasons that faculty provided: ease of access, speedy dissemination, and so on. Librarians also believed that online technologies enable them to connect faculty and students with better information.

Librarians’ main concerns about new modes of publication were along fiscal and technological lines, namely the economic sustainability of newer models and the role of the library in that financial equation. Librarians also pointed out the “version” problem for placing scholarly material in repositories. As it becomes easier to make changes after initial publication, determining and archiving the “canonical” version becomes more difficult. Most problematic for librarians, however, is the increasing reliance by both students, and to some degree, faculty, on search engines such as Google and Yahoo. There is the perception that vast amounts of important scholarly material –both electronic and analog— will be missed in such searches.

Publishers and Editors  

Publishers with whom we made formal and informal contact included the University of California Press, the Berkeley Electronic Press (bepress), the E-Scholarship project of the California Digital Library (CDL), the Public Library of Science (PLoS), Ithaka, the Electronic Publishing Initiative at Columbia (EPIC), the Stanford Encyclopedia of Philosophy, and Elsevier. Formal interviews of principals were conducted for the first four of these. We recognize that this is not a representative group of publishers. Twenty-two of the faculty members whom we interviewed were also editors of journals or had been in the recent past.

Perceptions among publishers and editors were tied closely to the mode (print or electronic) of publication, their institutional affiliation and philosophy, and often the disciplinary fields in which they specialized; thus, their opinions often reflected those different viewpoints. Some felt that print publications were ineffective compared to electronic venues in disseminating work in a timely matter, although all recognized the challenges associated with newer forms of publication, regardless of format. Many of the publishers and editors we interviewed were aware of the concern about increasing costs.

 Publishers concurred that academia is in a transition period with regard to publishing, and they understood the complex interplay of tenure requirements and distribution of publishing choices among faculty. Although scholarship is inherently innovative in both approach and method, and therefore a natural match for newer forms of publication, change is often hindered by institutional requirements and standard practices, such as the perceived necessity of traditional publication for advancement and achieving tenure. Most agreed that use of newer forms of publication has not yet reached a sufficient saturation point to tip the scale, and opined that the power to change rests with the university world—for both production and consumption. They recommended incentives for faculty, both in terms of policies (e.g., advancement process) and resources, as well as budgetary and technical support for libraries.

Publishers shared with other interviewees the concern about perceptions that equate low cost with low quality for electronic forms of publication. Although all agreed that quality-control systems are not in place in most open-access repositories, publishers in general pointed out that electronic journals can and often do use the same review process as traditional print journals. Several also expressed concern about the peer-review process, and believed that too much emphasis is placed on outside opinion and prestige rather than a review of actual content quality. 

One issue for faculty editors is the difficulty in finding reviewers who are qualified, neutral, and objective scholars in a fairly closed academic community. This is compounded by the fact that the increasing quantity of publications requires more scholarly input for the review process, while already overburdened academics have limited time to participate. Editors, in particular, have a difficult time coordinating reviewers’ schedules and available time.

REFLECTIONS

Lessons Learned and Challenges

While our investigations have yielded rich and descriptive case studies that shed light on the current state of scholarly communication, there are limitations to our study. Our small sample, both in the number of participants and in the range of disciplines, makes generalizations at this stage sketchy at best. Furthermore, we focused specifically on one campus in the University of California public system, UC Berkeley, and our results at this stage are thereby obviously biased both by lack of variety of institution and by being focused upon an elite research university. To develop general conclusions applicable to wider populations, future investigations will need to include other campuses and institutions.

The highly structured advancement system of the University of California has been advantageous to us in many ways in conducting our research. First, we know who the actors are at the several stages of the review process and we were able to talk with them directly. Second, we were able to compare and contrast the views of those in the different steps of the review process—faculty, department chairs, deans, (former) Budget Committee members, and campus-level academic administrators. As well, we can ascertain the views of bodies such as the Budget Committee by those involved in other stages of the review process. Third, at each of the levels of review we had access to persons who have reviewed many advancement cases and who therefore can make informed comparative judgments. This fact gives reviewers the ability to identify the relative importance of different factors, including the medium and nature of the publication vehicle. Fourth, the nature of this review process affords us the wherewithal to assess the degree of importance and roles of peer review, as well as the vehicles for peer review that hold cachet. By talking with reviewers involved with the Budget Committee at different times in recent years, we have been able to make initial inferences of whether and how the values ascribed to new media by that body are changing.

The academic values that we have identified in this project may be specific to the most prestigious of universities, wherein faculty researchers nearly always have their papers accepted for publication and can publish wherever they want. Indeed, the formal and informal feedback we have received on this research from individuals in other institutions suggest this is so. It is also true that the values exercised at these leading universities are likely to be emulated throughout the academic community. Only comparable research studies of a variety of institution types will be able to verify that prediction.

Conclusions

The descriptive case-study approach began to elucidate the ways in which faculty do or do not perceive electronic means and other new capabilities as enhancing the quality, effectiveness, and immediacy of communication of a scholar’s research output; how that research is recognized by peers; and the efficiency and effectiveness of scholarship as a whole.

The disciplinary case studies also enabled a more precise identification of the factors associated with academic and disciplinary value systems that influence viability and financial sustainability of different methods of scholarly communication for various participants in the publication and communication system, including authors (producers), researchers (consumers), libraries, and publishers.

From an examination of the ways in which value systems in five disciplinary areas affect scholarly publication and communication practices, we have reached the following conclusions:

  • Peer review is the coin of the realm. It is the value system supporting assessment and the perceived quality of research. It is commonly viewed as the primary mechanism through which research quality is nurtured, and through which research is made both effective and efficient. There was also a strong perception that peer review provides an excellent quality filter for the proliferating mass of scholarly information available on the Web.
  • Some equate low cost with low quality for electronic-only forms of publication. Many interviewees held that open-access models and electronic-only publications had little or no means of quality control, such as peer review. Additionally, some equated the high cost of print journals with high quality. The issue of peer review for newer, electronic journals needs to be separated from those issues associated with the fact that most such journals are simply new and not yet well established. Practically, peer review and the means of publication and dissemination can be separated.   Electronic-only journals can and often do use the same review process as traditional print journals.
  • There is some concern that the locus of peer review has moved out of the institution. This has particular repercussions for academic advancement as increasing reliance is placed on such things as journal stature, which then becomes a surrogate for selectivity and good review. This is especially a concern for those scholars in compound disciplines, where peer review can be complicated by differing standards and expectations among fields, and where quality assurance depends upon a small group of specialized academics.
  • There is presently a somewhat dichotomous situation in which electronic forms of print publications are used heavily, even nearly exclusively, by researchers in many fields, but perceptions and realities of the reward system have those same researchers turning to conventional, high-stature print publications as the means of record for reporting research and having it evaluated for tenure and promotion purposes. This was true of all of the disciplines we examined. In the sciences, although major journals are maintained in print form, electronic replicates are used increasingly for most access and research.
  • While both are critically important to a scholar’s career, the means of publication and communication for gaining advancement within the institution can differ significantly from those for making a name within a discipline. The former depends almost exclusively upon final, fully peer-reviewed archival publication, whereas the latter is more fluid and oriented toward partial results and meetings and information exchanges with other researchers during the course of the research (“in-progress communication”), as well as final, archival publication.
  • Such “in-progress” communication also fulfills needs such as gaining the critical thoughts of others while one’s research is in progress, “staking claim” to one’s activity and accomplishments in an area, and sparking thoughts and new ideas as a product of the discussion.[10]
  • In-progress communication does not substitute for the need for final, archival presentation and dissemination of research results. They serve different purposes and needs. Both are important.
  • There is much more experimentation with regard to means of in-progress communication, in which single means of publication and communication are not fixed so deeply in values and tradition as they are for final, archival publication.
  • From an institutional standpoint, there are looming questions about how to support faculty in their scholarly practice. Our interviews suggested that (at UC Berkeley, at least) there are currently few, if any, mechanisms or structures that support storing, archiving, and sharing the significant research products of faculty, such as databases, collections of literature, etc., that are created en routeto ultimate archival publication. Based on our preliminary research, this is true of other institutions as well, except in a few fields.
  • Campus-level academic administrators perceive an inevitable but slow evolution toward new forms of publication (particularly in fast-moving scientific disciplines), similar to the shift from print journals to conference proceedings that occurred in computer science in the 1970s. They see this evolution gaining momentum and credibility. Respected scholars, however, will begin using such venues in great numbers only once these venues, and the peer review associated with them, become better established.
  • According to our interviewees, the Budget Committee has so far rarely needed to address the issue of publication venue. This is because so few new forms of publication are represented in the cases that come through the committee.[11] Former Budget Committee members believed that the committee would be open to new forms provided that they meet the same standards for peer review and quality as traditional forms. Until more cases demanding the evaluation of digital scholarship come before tenure and review committees, we foresee that the situation will remain relatively stagnant.
  • Campus-level academic administrators perceived a distinction between peer review in the discipline and peer review for promotion. While the two kinds of peer review are clearly interconnected, administrators maintained that discipline-based peer review cannot stand on its own as a basis for promotion; the input of local colleagues in addition to discipline-based peer review is necessary for promotional consideration. On the other hand, administrators believed that it was possible, in some cases, for local peer review to substitute for discipline-based peer review, for instance in considering the quality of work published in a non-peer-reviewed journal.

Results from the project indicate that the values surrounding final archival publication are likely to be deep and relatively inflexible in many, if not most, disciplines at top-tier research universities. Yet, we predict that what scholars value and want will eventually become accepted practice. It is reasonable to assume that tenure and promotion policies will evolve, perhaps slowly, to reflect those values and needs. This is a much more realistic way of looking at these issues than devising models and modes of communication based on their cost efficiencies or other non-research criteria and then trying to draw scholars to them. Approaches that attempt to “move” faculty and deeply embedded value systems directly toward new forms of archival, “final” publication are destined largely to fail in the short term. Thus, it is our opinion that the development of any new models should focus on the needs of scholarly researchers for both final and in-progress communications in order to determine how those needs are likely to influence future scenarios in a range of disciplinary areas.

In summary, we suggest that more innovation does and will occur first in in-progress communication than in final archival publication. One can foresee a scenario in which useful and effective innovations in in-progress communication will eventually drive improvements in final archival publication. It is therefore worthwhile to gain deeper insights into the needs, motives, and new capabilities within in-progress communication as well as for final, archival publication.

ACKNOWLEDGEMENTS

We would like to thank the Andrew W. Mellon Foundation for generously funding this research. We are also indebted to an unusually active and involved steering committee that has provided invaluable guidance, support, and time. We thank the more than 50 formal and informal interviewees who graciously scheduled time to provide candid opinions and ideas. Irene Perciali participated in preparing background information on publication practices in the five disciplines considered.

Diane Harley is a senior researcher at the Center for Studies in Higher Education (CSHE), University of California, Berkeley (http://cshe.berkeley.edu/people/dharley.htm).   Her work focuses on the policy implications of integrating information and communication technologies into complex academic environments. Areas of investigation include the analysis of digital resource use in humanities and social science education, the economics of technology integration into large lecture courses, the policy implications of cross border e-learning, and faculty attitudes about new models of scholarly communication. Her work at Berkeley's Multimedia Research Center (BMRC) included contributions to the development and deployment of the prototype for Berkeley Webcast (http://webcast.berkeley.edu). Prior to her work at UC, Diane managed multimedia education projects with various universities, publishers, museums, and software developers, as well as taught undergraduate and graduate courses in anthropology and human biology.  She holds M.A. and Ph.D. degrees in Anthropology from UC Berkeley.

C. Judson King is Provost and Senior Vice President – Academic Affairs, Emeritus of the University of California system, having served as Provost from 1995 until 2004. He has also been Provost, Professional Schools and Colleges and Dean of the College of Chemistry at Berkeley, and now directs the Center for Studies in Higher Education (http://cshe.berkeley.edu/). He is also Professor Emeritus of Chemical Engineering, having researched and published extensively in that field during his 45-year Berkeley career, including authoring a textbook, “Separation Processes.” He is a member of the National Academy of Engineering and has received various awards from the American Institute of Chemical Engineers, the American Society for Engineering Education, the American Chemical Society, the Council for Chemical Research and the Yale Science and Engineering Association. In addition to the project reported here, his experience with scholarly communication has included oversight of the University of California Press, the California Digital Library, and the E-Scholarship project while he was UC system-wide provost.

NOTES

    1. Donald W. King, Peter B. Boyce, Carol Hansen Montgomery, and Carol Tenopir. 2003. "Library Economic Metrics: Examples of the Comparison of Electronic and Print Journal Collections and Collection Services." Library Trends 51(3): 276-300. Also see, Roger C. Schonfeld, Donald W. King, Ann Okerson, and Eileen Gifford Fenton. The Nonsubscription Side of Periodicals: Changes in Library Operations and Costs between Print and Electronic Formats, 2004, http://www.clir.org/pubs/reports/pub127/contents.html (06 December 2006). Also see, Donald J. Waters, "Managing Digital Assets in Higher Education: An Overview of Strategic Issues." ARL Bimonthly Report 244, February 2006. http://www.arl.org/newsltr/244/assets.html (06 December 2006).return to text

    2. Judith Ryan, Idelber Avelar, Jennifer Fleissner, David E. Lashmet, J. Hillis Miller, et al. The Future of Scholarly Publishing from the Ad Hoc Committee on the Future of Scholarly Publishing, 2002, http://www.mla.org/resources/documents/issues_scholarly_pub/repview_future_pub (06 December 2006). Also see, Leigh Estabrook, The Book as the Gold Standard for Tenure and Promotion in the Humanistic Disciplines , 2003, http://lrc.lis.uiuc.edu/reports/CICBook.html (06 December 2006).return to text

    3. Theodore C. Bergstrom, "Free Labor for Costly Journals?" Journal of Economic Perspectives 15, no. 3 (2001), http://repositories.cdlib.org/postprints/20 (06 December 2006),183-198. Also see, Aaron S. Edlin and Daniel L. Rubinfeld. 2004. "Exclusion or Efficient Pricing? The 'Big Deal' Bundling of Academic Journals." Antitrust Law Journal 72, no. 1 (2004), http://works.bepress.com/aaron_edlin/37 (06 December 2006), 119-157. Also see, Roger Noll and W. Edward Steinmueller. "An Economic Analysis of Scientific Journal Prices: Preliminary Results." Serials Review 18 (1992):32-37.return to text

    4. Deborah Lines Andersen, ed. 2003. Digital Scholarship in the Tenure, Promotion, and Review Process. Armonk, NY: M.E. Sharpe. Also see Nature's peer review debate at http://www.nature.com/nature/peerreview/debate/index.html (06 December 2006).return to text

    5. C. Judson King, Diane Harley, Sarah Earl-Novell, Jennifer Arter, and Shannon Lawrence. Scholarly Communication: Academic Values and Sustainable Models. 27 July 2006, http://cshe.berkeley.edu/publications/publications.php?id=23 (06 December 2006).return to text

    6. These particular five fields were selected with the goal of obtaining a diverse array of disciplines and publishing traditions, and taking advantage of the fact that at least one member of our project steering committee had deep knowledge of each of the disciplines selected.return to text

    7. Academic Senate Berkeley Division. "Introduction to the Budget Committee." February 2006, http://academicsenate.berkeley.edu/pdf/Intro_to_BC.pdf (06 December 2006).return to text

    8. University of California Office of the President, "Appointment and Promotion, Review and Appraisal Committees," Academic Personnel Manual, 26 July 2002, section 210, http://www.ucop.edu/acadadv/acadpers/apm/apm-210.pdf (06 December 2006).return to text

    9. The so-called Impact Factor is a measure of the citation frequency of papers in journals and is thereby equated by some to the stature and presumably the prestige of the journal. See, e.g., Richard Monastersky, "The Number That's Devouring Science." Chronicle of Higher Education. 14 October 2005 http://chronicle.com/weekly/v52/i08/08a01201.htm (06 December 2006). Ibid., "Impact Factors Run into Competition." http://chronicle.com/weekly/v52/i08/08a01701.htm (06 December 2006).return to text

    10. A classic example of in-progress communication from the pre-electronic era is the Gordon Research Conferences, where current research is presented and discussed at length, but there are no written materials, nor are the presentations final or archival. The stimulating environment of these and similar conferences is particularly valuable for generating one's own research ideas. Another traditional example is faculty invitations to other institutions for visits built around a seminar.return to text

    11. The recently released MLA Taskforce report on evaluating scholarship notes that of the departments it surveyed, 40.8 percent at doctoral institutions, 29.3 at master's institutions, and 39.5 percent at baccalaureate institutions report having "no experience" evaluating digital scholarship. "MLA Task Force on Evaluating Scholarship for Tenure and Promotion." December 2006. http://www.mla.org/tenure_promotion (10 December 2006).return to text