It might well seem that the oft-cited "crisis in scholarly publishing" has been with us long enough to beggar the very term crisis. For decades now, exorbitant increases in the cost of academic journals have been forcing research libraries to reduce their serial holdings. The Association of Research Libraries (ARL) reports that among its members the average drop in serial holdings has been on the order of six percent over the last decade. It is to the universities' credit that the reductions have not been worse, given average subscription increases of more than 200 percent from 1986 to 1999.[1] Still, a leading academic publishing conglomerate like Reed Elsevier continues to expand its hold on the market, adding another 400 journals in 2000, while sustaining profit levels of 20 percent of its share of the scholarly exchange.[2] The Age of Information may not bode well for the public sector of this knowledge economy. And the ARL has decided not to take it anymore. It has launched more than one campaign to help scholars reassert control over academic publishing, including the Scholarly Publishing and Academic Resources Coalition which sponsors such online publishing projects as BioOne.[3]

Scholarly publishing is caught, then, in a struggle between corporate and civic forces, and can be said to be, at best, in a transitional stage. Critical to that transition is, of course, the transformation of publishing by online-publishing technologies. While some researchers are utilizing these technologies to broaden global access to research through open-access (meaning free) e-print services and online journals, corporate publishing interests are using digital services to further consolidate and bundle their offerings, increasing client value and shareholder wealth.[4] It should be clear that these new technologies will not, of themselves, free us of this ongoing crisis.

Any hope for a solution to this crisis will lie in convincing scholars, in their capacity as writers, reviewers, editors, and professional association leaders, that now is a time to move away from the commercialization of academic publishing that has taken place over the last four or five decades. As faculty members have been largely protected from their library's economic troubles, they have given little thought to changing a publishing system they've learned the hard way, from the few predictabilities of peer review to the hierarchy of journal titles. With the notable exception of the more than 28,000 scientists worldwide who have signed the Public Library of Science petition calling for open-access to scientific research, the vast majority of scholars have yet to take a stand on how research sits within the commercial and public sectors of this knowledge economy.[5]

Scholars are unlikely to side with this public-access approach until they see that non-commercial online alternatives can not only reduce the cost and increase access, but actually improve the exchange of knowledge that is so vital to their work.[6] They will need to see how open-access systems could advance their work on a day-to-day basis, even as it promises to advance their careers as researchers.[7] Scholars may also come to appreciate how these gains in universal access would both greatly assist universities in developing countries and raise the potential public contribution of their work.

Now, most of the attention on changes in scholarly publishing has been focused on e-journals. We wish to expand that circle of light so that it takes in the indexing of serials. The index, as every scholar knows, is critical to the quality of the research. The value of a library's serial collection is only as good as its indexing. What scholar has not wondered about the impact of overlapping, inconsistent, and incomplete indexing services on their work? When the weaknesses of the current indexing services are matched against the potential of open-access systems, we may have a tipping point in convincing scholars that the profession would be far better served by open-access publishing systems. We argue that a primary candidate for scholarly publishing's tipping point is the coherence, integration, and precision that these open-access systems can bring to the scholarly exchange and enhancement of knowledge, especially when compared to the current state of the serial index and the hit-and-miss of full-text Web searches.

The Overlap and Oversight of Current Indexes

The first measure of an index is the scale and scope of its coverage. A major index, such as the Web of Science published by the Institute of Scientific Information (ISI), contains bibliographic entries, as well as works cited, drawn from over 16,000 international journals, books, and proceeding in the sciences, social sciences, and the humanities. Although the index's coverage is clearly extensive, it is not comprehensive. James Testa, a senior manager with ISI, reports that "ISI's editorial staff reviews nearly 2,000 new journal titles annually, but only 10-12 percent of the journals evaluated are selected." ISI accepted its first electronic journal in 1994 and is continuing cautiously to "monitor the growing body of journals published in electronic form," according to Testa. This restrictiveness may seem justified by researchers' reliance on a small circle of journals in their work. Testa points out that "as few as 150 journals account for half of what is cited and one quarter of what is published," while "a core of approximately 2,000 journals now account for about 85 percent of published articles and 95 percent of cited articles."[8] The quality of research in these core journals may account for the concentration of citations, but one may still wonder at what researchers overlook with the number of journals missing from the major indexing services. Indexes would seem to provide, at best, a partial view of the available knowledge.

To compensate for this incomplete coverage, librarians tend to rely on a number of overlapping services, though having to look through multiple indexes can further fragment the research process, as noted in a Library of Congress survey.[9] Cavanaugh tested the indexing of a comprehensive sample of biology articles, and found that a quarter of them were not listed in the five relevant Table of Contents (TOC) databases or three specialized biological databases in his study.[10] Compounding the problem, he said, were the different search methods of the various databases.

The comprehensiveness of specialized indexes also has been found wanting. For example, Pillow discovered many inconsistencies in coverage between print and electronic indexes dedicated to African-American Studies.[11] Similar gaps have been found in women's studies by Gerhard, Jacobsen, and Williamson, who examined the Social Sciences Index, the MLA Bibliography, and the Humanities Index.[12] In comparing the Education Index and the Current Index to Journals in Education, Brown, Edwards, and Lasee-Willemssen could not help but wonder whether the research was affected by the gaps in their overlapping coverage:

A quarter century ago, Lee Burchinal, a key figure in the development of the Educational Resources Information Center (ERIC) index for education, was struck by "how chaotic access to journal literature is in education," and one has to wonder how true that still is in many disciplines.[14]

To augment and update the study of serial indexing, we conducted a broad analysis of twelve arts and science indexes at the University of British Columbia Library (UBC) (Table 1).[15] Given our interest in testing the viability of scholarly publishing's public sector, we included four of the major open-access (or free) indexes in our sample: ERIC (Education Resources Information Center) and PubMed (National Library of Medicine), both sponsored by the U.S. government; Stanford University Library's HighWire Press; and the NEC Corporation's ResearchIndex. We then conducted a number of tests to establish the degree of overlap and oversight among the indexes.

Table 1. Indexes Sampled in this Article Academic Search Elite (Chicago, IL: EBSCO Information Services). Alternative Press Index (Baltimore, MD: Alternative Press Index). Canadian Periodicals Index (Toronto, ON: Thomson Corporation). Canadian Business Current Affairs Full-Text (Boston, MA: Silver Platter). Contemporary Women's Issues (Luton, UK: Responsive Database Services). Education Index (Boston, MA: Silver Platter). ScienceDirect (London, UK.: Reed-Elsevier). Education Resources Information Clearinghouse (ERIC) (Rockville, MD: United States Department of Education). Journals@Ovid (Amsterdam, Netherlands: Wolters Kluwer N.V.). HighWire Press (Palo Alto, CA: Stanford University Library). NEC ResearchIndex (Princeton, NJ: NEC Research Institute). Web of Science (Philadelphia, PA: Institute of Scientific Information).

With the science indexes, we took two approaches. First, we worked with a list from the Scientist of the fifty most-cited science journals for 1994.[16] We then established the degree to which they were covered by Academic Search Elite, Journals at OVID, the Web of Science, PubMed, HighWire Press, NEC ResearchIndex,, and ScienceDirect. We found that the indexes varied in their coverage, from 18 percent (Ovid) to 92 percent (PubMed), with an average of 55 percent for the seven sciences indexing services that we analyzed, giving a rough measure of overlap among them (Table 2). That PubMed (92 percent) and the Web of Science index (88 percent) are so closely aligned suggests current redundancy levels between public and private services. We did a further analysis of the health sciences, this time using a random selection of journals among those available at UBC. This naturally reduced the degree of overlap, with an average of 31 percent of the titles selected covered by the seven relevant science indexes. Obviously, no one index is sufficient, even consulting all of the indexes an area provides no guarantee of complete coverage.

Table 2. Cost, size, and coverage of science indexing services (2000-01)
	Academic Search Elite	Science-Direct	HighWire Press	Journals@Ovid	PubMed	NEC Research Index	Web of Science
Annual cost, UBCa	$31,000	N/Aa	0	$28,000	0	0	N/Ab
No. of titles in index	2,920	1,300	313c	30	4,000	N/A	5,700
No. of Top 50 titlesd	18	18	14	9	47	42	44
Percent of top 50	36%	36%	28%	18%	94%	84%	88%
Sampled titlese	4	0	5	9	23	14	18
Percent of sample	9%	0%	15%	27%	69%	42%	55%

We then drew a random sample of humanities and social sciences titles from Academic Search Elite and checked the degree of their coverage in seven other indexes: Alternative Press Index, Canadian Periodicals Index, CBCA Full Text Education, Contemporary Women's Issues, Education Index, ERIC, and Web of Science (Table 3). The overlap for Academic Search Elite with the other indexes ranged from 16 percent (ERIC) to 37 percent (Education Index), with an average of 22 percent among the whole set. In a further comparison, this time of the single-subject indexes, Education Index and ERIC, we found a 53 percent overlap across a random set of 43 journal titles (Table 4). To live by the results of a single index — even one dedicated to the field of study under investigation — is to research incompletely.

Table 3. Cost, size, and overlap among social science and humanities indexes (2000/01)
	Academic Search Elite (ASE)	Alternative Press Index	Canadian Periodicals Index	CBCA Full-text Education	Women's Issues	Education Index	ERIC	Web of Science
Annual cost, UBCa	$31,000	$1,432	$7,466	$2,757	$1,763	$3,482	0b	N/A
Titles indexed	2,920	357	400	670	156	400	750	3,500
ASE overlapc	N/A	18%	20%	17%	24%	37%	16%	28%

Table 4. Cost, size, and open-access e-journal coverage in Education Index and ERIC
	Education Index	ERIC
Annual costa	$3,482	0b
Journal titles indexed	400	750
Overlap for 2 indexes	54%	N/A
E-journals indexedc	10%	13%

A more recent area of concern with indexing is the sheer lack of it among open-access e-journals. This lack of indexing underlines the need to develop a new approach to this scholarly tool. The two education indexes we consulted provided coverage for only thirteen of the ninety-six e-journal titles listed by the Communication of Research group of the American Educational Research Association. As these journals increase public access to knowledge, the quality of that access may be diminished by their failure to be indexed. Open-access e-journals do not reduce the need to index. We need to develop a means of collectively searching these resources through an indexing system that can identify their status as refereed journals in education as well as the details of author, title, and subject, rather than relying on the Web's common search engines.

In investigating this overlap and oversight among indexes, we repeatedly experienced how the differences among indexing systems can hamper the search process. It was not immediately apparent what journals and other resources were indexed by each service or each service's relevance to a topic. The inconsistencies among the databases and an inability to collectively search among them, as well as variations in journal-title abbreviations, search parameters, and user interfaces, all point to the advantages in a universal or integrated system.

The partial overlap of journal titles among indexes adds considerably to the cost of maintaining a research library. A perfect overlap between two indexes, for example, would make it easy for libraries to subscribe to one or the other. Partial overlaps, however, make such decisions difficult, and libraries often end up subscribing to a number of overlapping indexes, ostensibly covering the same areas. This means paying to have some journals indexed a number of times. It also means that scholars must learn which indexes to consult for a given topic. Imagine wanting to locate a book in the library, and rather than turning to a single main catalogue, having to decide where to start the search from among, say, nearly 200 often overlapping catalogues. (The University of British Columbia library makes nearly 200 online indexing services available, which is why we chose that number.) Then there is the question of the journals that are not covered, which is particularly the case with the new electronic journals.

The overlap, inconsistencies, and lack of comprehensiveness among indexes today needs to be seen in relation to the indexes' share of the research library's acquisition's budget. In the academic year 1999-00, the University of British Columbia Library in Vancouver spent over $1M CAD on online indexing services, compared to the $6M CAD it spent on print serials, out of a total of budget of just over $11M CAD for acquisitions in all formats. Although nearly ten percent of the university library's acquisitions budget is spent on online indexing services, the coverage of the serial literature is bound to be partial, with considerable overlap in coverage.

Three things seem apparent, or at least worth testing, given the current state of indexing. Any system that seeks to improve online services needs to create a far more integrated and comprehensive approach to indexing. As more research results are reported electronically, this will most likely be achieved by building an integrated indexing system that would automatically harvest the relevant information from electronic sources. The cost of developing and maintaining such systems could come from the budget that research libraries currently dedicate to indexing services.

A Way Ahead

Scholars have learned to live with the current state of serial indexes, just as they work around the "crisis in scholarly publishing" through interlibrary loan, online pay-per-view services, and other means. However, we have the technology at this point to transform the index into a far superior research tool by making it an integral part of the public (open-access) domain. The indexes could take advantage of emerging metadata standards such as the Dublin Core Metadata Initiative which is employed by the Open Archives Initiative for scholarly resources, for fully identifying a document's origins, subject, and type.[17] Metadata greatly increases the accuracy and precision of online searches (Table 5). As we will go on to discuss, this metadata also can be used to integrate access to an array of research resources beyond the published article, from research instruments to pertinent data, from related online discussions to relevant public documents. Once one accepts the advantages and inevitability of digital journal publishing and the wisdom of moving scholarship more fully into the public domain, it is a small step to building comprehensive, integrated, and automated indexing systems into the very design of e-journal publishing systems.

Table 5. Metadata System Developed for Public Knowledge Project Systems
Dublin Core	PKP Metadata Items
1. Title	Title of document
2. Creator	Author's name, affiliation, e-mail, URL
2. Creator	Biographical statement
3. Subject	Discipline: assigned by conference director
3. Subject	Topic: Author selected, includes writers discussed in detail
3. Subject	Open: Director and/or author's choice
4. Description	Abstract of paper
5. Publisher	Organizing agency [name, city, state/province]
6. Contributor	Funding agency
6. Contributor	Supervision: Conference directors
7. Date	First day of conference
8. Type	Conference Paper
8. Type	Approach or genre: [e.g., review, critique, empirical study, N/A]
8. Type	Method: Research methodology [for empirical studies]
9. Format	HTML or PDF
10. Identifier	Universal Resource Indicator
11. Source	Conference title
11. Source	Journal volume and issue
12. Language	En = English
13. Relation	Status of conference paper [e.g., invited, refereed, non-refereed]
13. Relation	Appendix: Related research materials e.g., data, instruments, etc.
14. Coverage	Geographical, historical coverage
14. Coverage	Research sample [by age, gender, ethnicity, class]
15. Rights	Authors retain copyright, with permission granted for use on conference site.
The metadata are based on Dublin Core standards, which are employed by the Open Archives Initiative to enable research databases to be harvested and collectively searched, with the metadata or index terms further developed by the Public Knowledge Project, in this case, for its conference system.

A few years ago, Robert Cameron proposed a "universal, Internet-based, bibliographic and citation database [that] would link every scholarly work ever written — no matter how published — to every work that it cites and every work that cites it."[18] The system he proposed relies on authors to use a bibliographic software, along the lines of the widely used Endnote or Reference Manager, that would be integrated with this database. The new database would house full bibliographic information, including abstracts and a unique "canonical citation identifier" that would provide the link to the paper and bibliographic entry. Such a system would improve scholarly quality by ensuring access to a far wider range of materials, a more accurate means to trace the dissemination and movement of ideas, and an improved guide to citation rates. Such a system would also take advantage of bibliographic software to make the citation process and citation formatting that much easier. The cost of compliance for authors, then, would not be much more than is required for properly formatting a paper today, while the sponsorship of the universal database itself could be the work of a university consortium intent on investing its indexing dollars in a universal system.

Cameron's dream of a universal citation index has come close to finding its realization with NEC's ResearchIndex, an online open-access citation index, which provides access to four million citations in the sciences, along with 300,000 articles.[19] It was developed by Steve Lawrence, C. Lee Giles, and Kurt Bollacker, all of the NEC Research Institute. The ResearchIndex offers innovative indexing features like providing the actual context in which a paper is cited (with a 100-word excerpt), while automating indexing's most labor-intensive aspects. The ResearchIndex "autonomously" captures citational information from research documents that are located by monitoring mailing lists, newsgroups, and publishers, as well as by crawling the Web, looking for pages that contain the words "publications," "papers," and "postscript." It then parses out the relevant information by using "font and spacing information to identify the title and author of documents being indexed." It also locates papers related to "a given article by using common citation information or word similarity." It graphs citation links, and computes "hubs" (articles that cite many highly cited articles) and "authorities" (highly cited articles). "Ranking by hubs is useful to identify survey, tutorial, or review style articles," the developers point out. ResearchIndex is open source, and we are currently exploring its potential for the social sciences for which there is no comprehensive open-access indexing service comparable to PubMed, HighWire Press, or ResearchIndex.

Such automated processes are not infallible, of course, but the algorithm has been carefully refined and the system "trained" to minimize omissions and errors. Yet it is itself more of a proof of concept then a system for the future. It was built for a pre-metadata era of digital publishing and relies on non-standard, generalized ways to identify and index research. The next generation of automated indexing, based on harvesting highly detailed metadata, is bound to create a far more powerful indexing system that will boost the quality of scholarship, including the peer review process, because it will facilitate a more thorough consultation of the relevant literature and related materials.

Among current systems, the open-access systems provided by ResearchIndex, PubMed, and HighWire Press compare well with the commercial services of Web of Science and Science Direct (see Table 6). While ResearchIndex provides the context in which a work has been cited, for example, PubMed offers multiple search functions for both scholars and clinicians, with its "therapy," "diagnosis," and "prognosis" search categories, a medical glossary and a search-terms thesaurus, and links to related documents including a medical textbook and some full-text articles. Whether sponsored by government (PubMed), industry (NEC ResearchIndex), or university library (HighWire Press), these three public-domain indexes make a good case for both the viability and quality of open-access scholarly publishing. Judging by the Association of Research Libraries' sponsorship of BioOne, university librarians now recognize that open-access publishing is their best response to the scholarly publishing crisis, and we might expect to see research libraries shifting their indexing and serial budgets to supporting such open-access services.[20]

Table 6. Comparison of features among five leading biomedical and science indexes
Index Features	PubMed	NEC ResearchIndex	HighWire Press	Web of Science	ScienceDirect
Alert service	No	Yes	Yes	No	Yes
Automated indexing of sources	No	Yes	Yes	No	Yes
Boolean search	Yes	Yes	Yes	Yes	Yes
Category search (author, etc.)	Yes	No	Yes	Yes	Yes
Citation graphs	No	Yes	No	No	No
Citation indexing	No	Yes	No	Yes	No
Citation statistics	No	Yes	No	Yes	No
Cost (annually)	Free	Free	Free	N/A	N/A
Coverage	4K journals; 11 million records	300K articles; 4 million records	272K articles; 1 million records	8.6K journals; 22 million recordsa	1.3K journals; 30 million records
Display formats	Yes	No	No	No	No
Document comparisonb	No	Yes	No	No	No
Full-text articles	Some	Some	Some	Subscription dependent	Subscription dependent
Matches search terms to database terms	Yes	No	No	No	No
Order articles online	Yes	No	Yes	Yes	Yes
Professional searches	Clinical searches	No	No	No	No
Provides context of citations	No	Yes	No	No	No
Rating or comments on articles	No	Yes	Yes	No	Yes
Related article linking	Yes	Yes	Yes	Yes	Yes
Related resources	Yes	Yes	Yes	No	Yes
Save/edit searches	Yes	No	No	Yes	Yes
Scientific glossary	Yes	No	No	No	No
Sort parameters	No	Yes	No	Yes	Yes
Reference library	Yes	No	No	No	Yes
User customizable	Yes	No	No	No	Yes
User tutorial/help	Yes	No	No	No	Yes

This viability and quality of open-access indexing systems have also not gone unnoticed by commercial interests. As we write, the U.S. House of Representatives has approved an Energy Department's Appropriations Bill for 2002 that would, if signed into law, cut the half-million dollar budget for PubScience, a government-run indexing service, after pressure from the Software and Information Industry Association on behalf of member companies, including Chemical Abstracts Services, Reed Elsevier, and Cambridge Scientific Abstracts. The argument is that as PubScience goes beyond the "appropriate scientific journals" for the sponsoring Department of Energy, it competes "improperly with similar services available from the private sector," according to the bill's accompanying report.[21] Indexing is clearly playing its role in the struggle between the private and public domains of scholarly knowledge.

While combined print and digital publication of journals is still the scholarly norm at this point, it is hard to imagine that this expensive dual mode will continue indefinitely, especially as on-screen services improve, leading to more comprehensive indexing services. One way forward is found with Stanford University Library's HighWire Press, which has developed an e-journal publishing system that includes indexing hyperlinks on each research article. Those links allow searches for other works by the authors and related articles in major indexes such as PubMed. HighWire Press also links together the 282 sites — mostly journals, with many providing open access — using its publishing system, which has allowed it create "one of the two largest archives of free full-text science on Earth!" as the HighWire Web site puts it.[22] This seamless integration of publishing and indexing systems is clearly the way ahead, and the leadership role of the university library, in this case, is one of its most promising aspects.

The next step will be to develop open-access indexing systems that are continually and automatically harvesting the metadata which authors and editors encode in each journal article, conference paper, thesis, and dissertation posted on line. The virtual and universal index constituted by metadata harvesting of scholarly Web sites and databases around the world will allow researchers to search by topic, type, and language of document, as well as to ascertain an article's citation record and its contribution as a hub of ideas as a ready measure of quality.

To further improve the indexing process, we are currently experimenting with easy-to-use online templates that will allow authors to do increasingly fine-grained indexing of their own work on submitting it for publication, specifying, for example, the research methodology and the sample studied, the research instrument employed, and the geographical and historical period of inquiry (Figure 1). Questions and eyebrows have been raised over the idea of authors indexing their own work. We appreciate that they will not be as skilled as professional trained librarians, but the benefit of self-indexing is how it will encourage authors to think more about how they position their work, in terms of audience, and how they integrate their into their field, since that field will be far more present for their work: only a click or two away. Whether this manner of indexing research will reduce the notable fragmentation among studies ever so slightly, whether it will make it easier for authors to break ranks with conventions and create new categories and terms, whether it will level the scholarly playing field by making every work equally accessible, and finally whether greater public access will alter scholarly writing and style are open questions that we look forward to investigating.

Growing out of our work with the Public Knowledge Project on social science publishing and indexing systems, we are working on ways of linking research studies with a wide range of related materials (see Figure 2). This has led to the development of a portable indexing tool for education research, comparable to the one used by HighWire Press, that sits in the margins of digital journal articles, conference papers, and dissertations. This tool will allow readers to not only find related research studies, by utilizing the document's metadata on its topic, but to search The New York Times (last seven days) and other media sources for related stories. It will search federal and state governments for the pertinent policies, federally sponsored teaching sites for relevant curriculum materials, and online forums for ongoing discussions of the topic (Figure 3). The intent is to enable social science research to make a far greater contribution to public policy and individual knowledge by reducing its isolation and fragmentation, while increasing its connections to the larger world.[23] This indexing tool is designed to work with a series of downloadable, open-source publishing systems that we are building for conference directors and journal editors to install on their own servers, based on Open Archives Initiative standards. These standards are also being utilized by the Electronic Dissertations and Theses (EDT) system developed by Virginia Tech and deployed by many of the universities that belong to the Networked Digital Library of Theses and Dissertations.

The current state of serial indexing presents its own set of good reasons for research libraries and professional associations, as well as individual researchers and editors, to work together on compatible systems that greatly improve the comprehensiveness of indexing and the universal access to research. These open-access systems could place scholarship squarely within the public domain of this knowledge economy. They could improve not only scholarly access to research but scholarly quality and public access in the process. More comprehensive and coherent access to research is also bound to change how scholars conduct and write about research.

The current state of scholarly indexing forms a good starting point for rethinking the place of scholarship in publishing's digital revolution. It could well provide a tipping point in the current struggle between open-access efforts and commercial interests to take hold of this new medium, with far too many researchers having yet to see just how the quality and reach of their scholarly contribution hangs in the balance. For scholars, better access to knowledge is the key to this reform, and the index is critical to improving access, that is, to improving the support that libraries provide to scholars and the public at large, while at the same time helping that tired crisis in scholarly publishing finally become a thing of the past.

Acknowledgment

We wish to express our thanks to Janice Kreider and Pia Christensen of the UBC Library whose helpful assistance and thoughtful comments aided the development of this paper, and to Henry Kang for his talented work on the figures, along with Jahan Badar, Kevin Jamieson, Henry Kang, Sam Ladner, Miriam Orkar, Robert Wickert, and Eunice Yung for their contributions to the Public Knowledge Project. This Project is supported by the Social Sciences and Humanities Research Council of Canada and the Max Bell Foundation.

---

John Willinsky is the Pacific Press Professor of Literacy and Technology at the University of British Columbia and a Fellow of the Royal Society of Canada. He is the author of Learning to Divide the World: Education at Empire's End, which won Outstanding Book Awards from the American Educational Research Association and History of Education Society, as well as of the more recent titles, Technologies of Knowing and If Only We Knew: Increasing the Public Value of Social Science Research. He directs the Public Knowledge Project, http://www.pkp.ubc.ca/ which is dedicated to developing online systems that improve the scholarly and public quality of academic research, and has been known to play at academic conferences with an international blues band of scholar-musicians. He can be reached by e-mail at john.willinsky@ubc.ca

Larry Wolfson teaches English and Photography at a Vancouver high school in Canada. Recently he gained his PhD in Educational Studies from the University of British Columbia with a study of service learning among high school students who were engaged in managing the information technology in their schools and communities. He served as a post-doctoral fellow on the Public Knowledge Project. http://www.pkp.ubc.ca/.

Endnotes

1. Martha Kyrillidou, "Spending More for Less" (Washington: Association of Research Libraries, June 1999), http://www.arl.org/newsltr/204/spending.html; ARL Monograph and Serial Costs in ARL Libraries, 1986-1999, http://www.arl.org.

2. Kaylyn Hipps, "Update on Scholarly Publisher Profits" (Washington: Association of Research Libraries, December 1999), http://www.arl.org/newsltr/207/pubprofits.html. See also: Reed Elsevier, http://www.reed-elsevier.com.

3. See the Association of Research Libraries, http://www.arl.org; SPARC, http://www.arl.org/sparc; BioOne, http://www.bioone.org, and Create Change, http://www.createchange.org.

4. For the e-print story, see Paul Ginsparg's Los Alamos Archive, http://xxx.lanl.gov. For a full range of e-journal features, see Gerry McKiernan, "EJI(sm): A Registry of Innovative E-Journal Features, Functionalities, and Content" (Ames, Iowa: Iowa State University Library, January 2001), http://www.public.iastate.edu/~CYBERSTACKS/EJI.htm, as well as the Public Knowledge Project for prototypes, http://pkp.ubc.ca.

5. See the Public Library of Science, http://www.publiclibraryofscience.org.

6. On the economics of e-journals, we have found the following (1) Gene Glass, editor of the on-line Education Review and the Education Policy Analysis Archives, estimates his publishing costs as "zero, nada, no budget, no grad assistant, no secretary" (Tempe, Arizona: Email, February 2001); (2) Ginsparg estimates his arXiv.org's costs at roughly $9 a paper (Glanz, 2001) (3) the Chicago Journal of Theoretical Computer Science budgets $2,070 per article; Janet H. Fisher, "Comparing Electronic Journals to Print Journals: Are there Savings?" in Technology and Scholarly Communication: The Institutional Context, ed. By Richard Ekman and Richard E. Quandt (Berkeley: University of California Press, 1999), pp. 95-101; and (4) King and Tenopir put the cost of electronic publishing is $368 per page or about $175,000 per year (per journal); Donald W. King and Carol Tenopir, "Economic Cost Models of Scientific Scholarly Journals" (Knoxville, Tennessee: University of Tennessee School of Information Sciences Center for Information Studies, September 1998), http://www.bodley.ox.ac.uk/icsu/index.html; and (5) the Electronic Publishing Committee at Cornell University found that it would cost $2,700,000 to establish an electronic publishing program; Knoxville, Electronic Publishing Electronic Steering Committee, "Electronic Publishing Steering Committee Report on Electronic Publishing" (Ithaca, New York: Cornell University, December 2000), http://www.library.cornell.edu/ulib/pubs/EPSCFinalReport1998.htm.

7. A recent study found that among a sample of social sciences and humanities scholars, some 85 percent of survey respondents agreed that "publishing in non-electronic outlets is more credible than publishing in electronic outlets"; see Raymond Siemens et al., "The Credibility of Electronic Publishing: A Report to the Humanities and Social Sciences Federation of Canada" (Nanaimo, BC: Unpublished paper, 2001), http://web.mala.bc.ca/hssfc/Final/Credibility.htm.

8. James Testa, "The ISI Database: The Journal Selection Process" (Philadelphia: Institute for Scientific Information, March 1998), http://cs.nju.edu.cn/~gchen/isi/help/HowToSelectJournals.html.

9. Carolyn Larson and Linda Arret, "Descriptive Resource Needs from the Reference Perspective" (Washington: Library of Congress, January 2001), http://www.unicamp.br/bc/arret_paper.html.

10. Anthony Cavanaugh, "A Comparison of the Retrieval Performance of Multi-disciplinary Table-of-contents Data Bases with Conventional Specialized Data Bases," Australian Academic & Research Libraries 28, (2, 1997). See also Albert J. LaRose, "Inclusiveness of Indexes and Abstracts of Interest to Students of Communication," RQ (Fall 1989): 29-35; J. Holt and KA Schmidt, "Carl-Uncover or Faxon-Finder: A Comparison of Articles and Journals in Carl-Uncover and Faxon-Finder," Library Resources & Technical Services 39,. (3, 1995): 221-228.

11. Lisa Pillow, "Scholarly African American Studies Journals: An Evaluation of Electronic Indexing Service Coverage," Serials Review 25, (4, 1999): 21-28.

12. Kristin H. Gerhard, Trudi E. Jacobsen, and Susan G. Williamson, "Indexing Inadequacy and Interdisciplinary Journals: The Case of Women's Studies," College and Research Libraries 54, (March 1993): 125-135. See also, Linda A. Krikos, "Women's Studies Periodical Indexes: An In-depth Comparison," Serials Review, (Summer 1994): 64-75.

13. M. Suzanne Brown, Jana S. Edwards, and Lasee-Willemssen, "A New Comparison of the Current Index to Journals in Education and the Education Index: A Deep Analysis of Indexing," The Journal of Academic Librarianship 25, (3, 1999): 216.

14. Cited by Brown, Edwards, and Lasee-Willemssen, "A New Comparison": 218.

15. According to ARL 1999 data, the University of British Columbia library ranked 28^th of 112 member libraries with 112,000 volumes. The average holdings for all members was 86,221 volumes; ARL, "Association of Research Librarians Statistics: Data Tables for Academic Institutions" (Washington: Association of Research Libraries, Sept 2000), [formerly http://fisher.lib.Virginia.EDU/newarl/listyear.html].

16. The Scientist, "Most Cited Journals, 1994 and 1989," 10, (17,1996): 13.

17. Dublin Core Metadata Initiative (Dublin: Dublin Core Metadata Initiative, December 2000), http://dublincore.org/; Digital Library Foundation, "Harvesting Research Metadata. Aims, Objectives, Planning Process: Research Metadata Harvest Project. Draft Request to the Mellon Foundation for Funds to Support a Planning Process, April 2000" (Boston, Harvard University and DLF, September 2000), [formerly http://www.clir.org/diglib/architectures/lycospub.htm]; Library of Congress, "Z39.50: Gateway to Library of Catalogs" (Washington: Library of Congress, September 2001), http://lcweb.loc.gov/z3950/gateway.html#about. See Open Archives Initiative, http://www.openarchives.org.

18. Robert D Cameron, "A Universal Citation Database as a Catalyst in Scholarly Communication," First Monday (1997), http://www.firstmonday.org/issues/issue2_4/cameron/.

19. NEC ResarchIndex (Princeton: NEC Research Institute, May 2000), http://researchindex.com/. Steve Lawrence, C. Lee Giles, and Kurt Bollacker, "Digital Libraries and Autonomous Citation Indexing," IEEE Computer 32 (1999): 67-71.

20. For a discussion of the pivotal position of the library in this digital revolution in scholarly publishing, see John Willinsky "Proposing a Knowledge Exchange Model for scholarly publishing," Current Issues in Education (3, No. 6, 2000), http://cie.ed.asu.edu/volume3/number6/.

21. Andrea Foster, "House of Representatives Seeks to Close Free Bibliographic Database," Chronicle of Higher Education, July 2, 2001 http://chronicle.com/free/2001/07/2001070202t.htm.

22. The other large open-access archive, which we do not discuss in this article but is certainly worth noting is the Astrophysics Data System (ADS), which "is a NASA-funded project whose main resource is an Abstract Service, which includes four sets of abstracts: 1) astronomy and astrophysics/planetary sciences/solar physics, containing 640,373 abstracts; 2) instrumentation, containing 598,523 abstracts; 3) physics and geophysics, containing 949,953 abstracts; and 4) Los Alamos preprint server, containing 3,575 abstracts. Each dataset can be searched by author, object name (astronomy only), title, or abstract text words" (http://adswww.harvard.edu/).

23. For a full discussion of these issues, see John Willinsky, If Only We Knew: Increasing the Public Value of Social Science Research (New York: Routledge, 2000).

Links from this article:

Academic Search Elite (Chicago, IL: EBSCO Information Services), http://www-us.ebsco.com/home/default.asp

Alternative Press Index (Baltimore, MD: Alternative Press Index), http://www.altpress.org

Association of Research Libraries, http://www.arl.org/

Canadian Business Current Affairs Full-Text (Boston, MA: Silver Platter), http://library.dialog.com/bluesheets /html/bl0262.html

Canadian Periodicals Index (Toronto, ON: Thomson Corporation), http://www.galegroup.com/servlet/SearchResultServlet?searchText=canadian+periodical+index&imprint=000&region=9&searchType=As&searchBy=subTitle&search_dropdown=0

Contemporary Women's Issues, (Luton, UK: Responsive Database Services), http://www.oclc.org/support/documentation/firstsearch/databases/dbdetails/details/CWI.htm

Dublin Core, http://www.dublincore.org/

Education Index (Boston, MA: Silver Platter), http://www.educationindex.com/

Education Resources Information Clearinghouse (ERIC) (Rockville, MD: United States Department of Education), http://www.eric.ed.gov/

Electronic Journals in the Field of Education, http://aera-cr.ed.asu.edu/links.html

EndNote, http://www.endnote.com/

ERIC (Education Resources Information Clearinghouse), http://www.isinet.com/isi/

Highwire Press (Palo Alto, CA: Stanford University Library), http://highwire.stanford.edu/

Institute of Scientific Information, http://www.isinet.com/isi/

Journals@Ovid (Amsterdam, Netherlands: Wolters Kluwer N.V.), http://www.ovid.com/products/journals/inde x.cfm

NEC ResearchIndex (Princeton, NJ: NEC Research Institute), http://citeseer.nj.nec.com/cs

Networked Digital Library of Theses and Dissertations (NDLTD), http://www.theses.org/

Open Archives Initiative, http://www.openarchives.org/

Public Knowledge Project, http://pkp.ubc.ca/

Public Library of Science petition, http://www.publiclibraryofscience.o rg/plosLetter.shtml

PubMed (National Library of Medicine), http://www.pubmed.gov

Reference Manager, http://www.refman.com/

Scholarly Publishing and Academic Resources Coalition, http://www.arl.org/sparc/

ScienceDirect (London, UK.: Reed-Elsevier), http://www.sciencedirect.com/

University of British Columbia Library, http://www.library.ubc.ca/

Web of Science (Philadelphia, PA: Institute of Scientific Information), http://www.isinet.com/isi/ or [formerly http://www.isinet.com/isi/produ cts/citation/wos/index.html]

See Letter to the Editor about this article.

John Willinsky may be reached by e-mail at john.willinsky@ubc.ca.

Larry Wolfson may be reached by e-mail at lwolfson@iunixg.ubc.ca.

Figures

Top of page