/ Potpourri

The Indexing of Scholarly Journals by John Willinsky and Larry Wolfson

Good Indexing

by Dan Duncan
Executive Director
December 2001

As Executive Director of NFAIS, the premiere trade association representing the interests of information aggregators, I am writing to comment on the December 2001 article by Messrs. Willinsky and Wolfson, entitled "The Indexing of Scholarly Journals: A Tipping Point for Publishing Reform."

At first glance, the authors appear to raise new concerns about the role of aggregators — often referred to as abstract and indexing services or secondary publishers — in the field of scholarly research, particularly research using digital or electronic content. After a closer read, however, one understands that the authors are concerned less with the valuable role that aggregators perform and more with encouraging scholar-authors to begin abstracting and indexing their works as part of a general "move away from the commercialization of academic publishing that has taken place over the last four or five decades." In other words, the message to scholars is this: simply become an indexer or abstractor in addition to a scholar-author, contribute your work freely to an open access system, and something good happens to the world of scholarly publishing.

Whether one agrees or disagrees with the prospect that the rise in the cost of scholarly publishing reflects a comparable rise in the value of the resulting content is a debate that has raged since the beginning of publishing. It is also one likely continue far beyond our lifetimes. Although there are those who may never be comfortable with the concept that information is, to some extent, a commodity subject to market forces — including market pricing — an information marketplace for information services has existed for hundreds of years, including in the field of scholarly publishing. As does any market, it provides choices for consumers, in terms of value and price, and when consumers no longer find value in a product or service they can, and do, find alternatives.

The fact is that there exists today, and has existed for decades, any number of vehicles for scholarly publishing and research. The information-aggregator sector alone is comprised not only of products and services provided by the private sector, but also those produced by nonprofit scholarly societies (many of whose members are the very scholars and researchers of concern to Messrs. Willinsky and Wolfson). So-called "free" services such as those made available by universities or under the auspices of government agencies are also part of the information-aggregator marketplace, and even though they may not charge for access, their construction and maintenance is certainly far from cost-free.

Regardless of which type of organization supplies an information aggregator service, it is precisely this question of value that is of the greatest concern to the NFAIS community, their content partners, and their customers. Messrs. Willinsky and Wolfson make much of the fact that current indexing resources, including both pay services and no-fee access services, vary in their comprehensiveness and their redundancy. These same arguments have been made since the advent of such services. However, I would suggest that some more comparisons are necessary before we condemn information aggregators generally as contributing to the self-proclaimed "crisis in scholarly publishing." For example, it would be helpful to compare the volume and types of literature covered by information aggregators today, as opposed to the time before the internet. It would have also been more useful in assessing the authors' conclusions about the "weaknesses of the current indexing services and the hit-and-miss full-text Web searches," had they compared the number and availability of electronic aggregator services today with similar services provided only in hard-copy print format just a few years ago.

Anecdotal evidence suggests that aggregators today provide information about more content, in more formats, and through a greater variety of services that ever before. Despite the faults of current systems, they do provide much greater convenience to users than has ever been the case before. In assessing the two authors' conclusions and recommendations, it should also be kept in mind that like all things "Internet," we are speaking of markets and products as works in progress. As much as some of us may have assumed that the Web would help make all things perfect, such perfection is an illusive goal. Most users are as interested in value, and I would argue that the improvements in information retrieval since the advent of digital information systems translates as value in anyones objective pocketbook.

Nevertheless, one cannot deny the authors' assessment that cash-strapped budgets for library acquisition departments will eventually force libraries to choose fewer and fewer aggregator services. It is also quite logical that marketplace behavior will result in libraries not renewing subscriptions to services that are not relatively comprehensive of their fields of study or that simply duplicate other indexing and abstracting sources.

It is less certain that scholars in general will want, or be able, to apply effectively the types of automated indexing procedures discussed by the authors in a manner that anticipates and meets the needs of researchers, whether in the scholars' field of study or in another field. Indexing expertise contributed by disinterested third parties ensures relevant and appropriate information retrieval. Different information-seeking behaviors require that there be many ways for various types of users to find and retrieve a single item in a database.

Automated procedures lack any human component, such as the judgments made by aggregators on a daily basis. Good indexing requires that the aggregator be a neutral, uninterested third party with the experience and skills needed to understand and categorize research across a broad range of disciplines and sub-disciplines. The Dublin Core Standard, a project in which many NFAIS members worked closely with the scholarly and user communities, provides only top-level information templates. It does not provide standard templates for metadata beyond the basic information outlined by the articles two authors. One important reason for this lack of standardized, detailed metadata is precisely because it was recognized that the human interface and judgment skills are required to anticipate user needs and determine how best to meet them.

Relating this concept directly to the proposals made by Messrs. Willinsky and Wolfson, scholar-authors within a narrow field may not properly recognize the value of their work to those outside their own immediate niche. Even more to the point, scholars are not always properly equipped to recognize how others outside their immediate environment may be approaching a search or wording a query — an increasingly important consideration in interdisciplinary research.

As an example, if the author of a clinical study of a drug provides only the brand name of a particular pharmaceutical product as a keyword and fails to include the generic name or drug class as a keyword, a user who searches under those terms will not retrieve the document. Hence an individual author's selection of keywords may actually limit or prevent retrieval of a specific document by the widest possible audience for whom that document might prove useful. Aggregators are well aware that an article published in Nature would likely be assigned indexing terms differently by EMBASE, MEDLINE and BIOSIS. Messrs. Willinsky and Wolfson view these differences among indexing services as disadvantageous and lacking standards. In truth, such differences are purposeful, because each indexer serves a different audience and has worked hard to understand that audience's needs.

In conclusion, it is understandable, given the views expressed by Messrs. Willinsky and Wolfson about the commercial publishing sector and scholarly publishing in general, that their primary goal is economic — a redistribution of the wealth of information, as it were, more than improving the quality of scholarship. There is certainly nothing wrong with seeking new economic models and proposing their testing in the marketplace. As stated above, market forces are something that NFAIS members have long experienced, regardless of whether they are private sector, nonprofit or government providers.

However, NFAIS members have a different prime objective in mind: doing the best job possible in aggregating information for a wide variety of users and uses. It has been this selective approach to indexing various journal titles that has proven valuable to end users. To propose that these tasks can be adequately replaced by lay persons using minimally standardized technologies, and without controlled vocabulary or taxonomy, does a disservice to current aggregators. Such a haphazard system of indexing may also doom scholarly researchers to wasteful, illogical hunting and pecking through the enormous fields of information in order to obtain those one or two fruits of knowledge essential to their discoveries and writings.

Thanks in large part to information aggregators in all fields and across all sectors, we have moved a long way from the days before Melvyl Dewey. Let us not allow self-proclaimed crises or larger economic agendas force scholarly research to return to those times of haphazard, random (and often abandoned) attempts to categorize, index, and abstract information. Should that occur, we will inhibit, rather than facilitate, access to and use of the vast wealth of information provided by the world's scholars.