Digital Libraries: A Vision for the 21st Century: A Festschrift in Honor of Wendy Lougee on the Occasion of her Departure from the University of Michigan

Developing a Successful Digital Library Environment

To explore these and other issues, a year-long campuswide Information Symposium was held in 1991, bringing together faculty, librarians, researchers, engineers, and information technologists to chart a course for the campus in the coming years. The symposium yielded a formal partnership between the main information and computing "players" on campus— the University Library, the University's Information Technology Division, and the interdisciplinary School of Information - and in 1993, Wendy Lougee was tapped to lead this partnership to develop Mochigan's digital library environment.While the University of Michigan had (and continues to have) a tradition of innovation in technology, the establishment of this formal digital library effort provided a critical catalyst for change and collaboration. The digital library partnership was given a "simple" mandate: harness the complementary expertise of the technology and library communities; demonstrate the potential of new information technologies through highly visible and useful projects; and, build a sense of a campus "information community" with attention to policy and economic issues in support of networked information resources. The central strategy to realize this mandate was to create and expand partnerships within the campus, with other institutions, and with entities outside academe and to foster a highly collaborative environment that encouraged creativity and accountability.

Collaboration and Partnership: Harnassing Complementary Expertise on Campus

The first challenge to the new digital library partnership was creating and strengthening ties between the various campus organizations that had been responsible for managing the local information environment. It was critical that this partnership offer freedom from organizational boundaries, the resources to foster experimentation (including some tolerance for failure), and enough structure to promise long-term management and accountability. The earliest effort at building this campus collaboration was the development of a program advisory group. The group brought together librarians, technologists, and information researchers to assess the campus environment and recommend strategies for addressing information needs, while providing a forum for these disparate groups to learn to work together. Representatives to the group included high-level managers within the three partner organizations, individuals with the ability to make resource commitments and to influence initiatives within their own organizations. Among the first projects developed through this group were the University's first (albeit primitive) web gateway and the Library's first homepage. While the advisory group was phased out as the digital library evolved, it laid the early groundwork for a strong three-way partnership that continues to flourish. [1]

The partnership between the three founding groups was further cemented through direct involvement of the deans and directors of those units in the oversight of the program. This oversight occurred within larger group meetings that included senior staff from the organizations and in regular meetings with the Program's director to review strategies and progress.

The Library's early efforts were project-based, exploring a variety of information technologies and issues. At the same time, they were production-oriented, aimed at developing user-centered resources that would have immediate impact on the scholarly community. Organizationally, as well, the Michigan's program showed a distinctive preference for models that would ensure immediate impact. As part of her coordinating role in campus digital library initiatives, Lougee convened regular meetings with the directors of the sponsoring organizations (i.e., the Library, the Information Technology Division, and the School of Information), as well as the principal operational heads in each of those three organizations.

Two of the most notable production-oriented initiatives that were shaped significantly by the Michigan digital library program were JSTOR[2] and Making of America.[3] Both are treated extensively in the literature, so no effort will be made here to document their histories, but both have significant roots at Michigan because of the production orientation under Lougee's leadership. As mentioned previously, Michigan undertook these efforts at least in part because earlier work had confirmed that certain big questions had already been answered. Michigan's work with the multi-institutional Elsevier partnership, TULIP (The University Licensing Project), was an unqualified success, with materials being electronically delivered to users campus-wide through a number of access points, including print-only service from the Library's catalog, and search and view access through locally-developed systems.[4] In large part because of Michigan's success with TULIP, The Andrew W. Mellon Foundation could approach the establishment of JSTOR as the use of a production system in service of asking important questions about the storage of library materials and user behaviors when using electronic resources. After initial difficulties in establishing processes and procedures to manage such a large-scale conversion effort, Michigan began shipping a high volume of material to vendors for scanning and OCR and loaded those materials into a working access system, all with relatively few production issues. The lessons learned in the early days of JSTOR, including issues that now seem like minor procedural problems expected in any such effort, laid the ground work for Michigan's future successes in creating and managing large digital collections.

Similarly, in its part of the first Making of America project, Michigan worked first to create a production resource and only secondarily posed questions about the digital materials and their use. Undertaken cooperatively with Cornell, the first Making of America project sought to leverage the collective experience of the two institutions, particularly preservation-related work at Cornell, high-capacity imaging at Michigan (with JSTOR), and Michigan's experience with access systems. Like many other Michigan initiatives, Making of America sought to ask "interesting" questions about social or cultural behavior by using production digital library resources: more than 1,000 volumes were drawn from storage at Michigan and released to an unrestricted and thus world-wide user base. The basic question being raised was: What would the user reaction be to this relatively huge body of unexceptional material, with searchability provided only through the use of uncorrected OCR? By contrast, similar efforts at other institutions left materials in a relatively inert form, browseable though not searchable, and several other preservation-oriented efforts using proprietary Xerox systems simply did not put the volumes online. The answer to the question about use was a resounding vote of approval, with a broad range of users signaling their unqualified support. This principle, of course, has become a given in our community: confirming Ranganathan's third law of library ("Every book his reader") to an extent that we had never imagined.

The PEAK project followed soon after JSTOR and Making of America, again leveraging production technologies as it sought to ask questions about economics and unexplored facets of user behavior. Like those other initiatives, PEAK (i.e., Pricing Electronic Access to Knowledge) has been widely discussed in our literature, and so will not be explored in detail here. Notably, and despite an explicit goal of mounting all 1,200 Elsevier journals, the framers of PEAK did not have in mind questions of production technology. PEAK, like JSTOR, had its roots in the TULIP project, but brought from TULIP not only assumptions about what was possible, technologically, but what was necessary from the user perspective. TULIP, despite its success, had a relatively narrow focus (materials science journals) and was only moderately used; studies with users confirmed that the largest problem was the relatively isolated body of materials. Although used by researchers in many disciplines, the roughly fifty journals did not serve any single discipline well. In discussions with Elsevier, Lougee suggested that only a much broader and chronologically deeper body of materials would have much greater impact, and that in the process, real questions about users and their relationship to the economics of information could be posed. In large part because of Michigan's successful outcome-oriented program, Elsevier permitted Michigan to mount its entire collection of journals and to serve as a "host service," in effect competing with Elsevier, for twelve other organizations or institutions. Like the NSF/DARPA/NASA-funded digital library project at Michigan, a large body of content was put into play to ask big questions. Unlike that grant-funded project, however, the focus would be on questions of immediate relevance to libraries and to publishers. In relatively short order, and long before Elsevier's own delivery of the content, the materials were mounted at Michigan and made available to users at the twelve diverse sites. Not only was the initial hypothesis about the need for a broader body of content confirmed, but other aspects of user behavior that have since become truisms of our profession emerged. For example, each participating institution learned that members of their user communities sampled materials in ways that little resembled the print subscriptions they maintained, and that even the smallest obstacle— particularly authentication— had a significant impact on patterns of use.

Established two years prior to the Making of America and PEAK, the Humanities Text Initiative (HTI) was another example of Michigan efforts directed at real and immediate impacts. The HTI sought to change the tool and resource world of the academic humanist by creating or acquiring and mounting large bodies of textual resources in the humanities. Frequently partnering with faculty in the application of encoding or the basic design of systems, the HTI quickly became the largest resource of its kind on the Internet. Like PEAK and MoA, the HTI built on known technologies, principally those of the earlier (1990-1993) UMLibText.[5] Working with faculty at Michigan, HTI launched such efforts as the Middle English Compendium, applying the TEI encoding guidelines in the conversion of the Middle English Dictionary and the creation of an accompanying reference source, the Hyperbibliography of Middle English Prose and Verse. Later, with a generous equipment grant from Sun and responsibility for ongoing support of Making of America and its successor efforts, the HTI shifted its focus from support for licensed and unlicensed humanities collections, to freely available collections of textual resources, regardless of discipline.

The Museum Educational Site Licensing (MESL) also illustrates the success of this early focus on bringing together the complementary expertise resident on campus. With a proposal developed by technologists and librarians, the University was selected to participate in the Getty Art History Information Project's MESL effort. The focus of the effort was to explore the myriad of issues involved in offering high-quality digital photographs of art and museum objects to an academic community. Art history faculty participated by providing input on collection content, while technologists and librarians provided central distribution services for the digital images. The project was an early success in both exploring technologies to make high quality digital images available and in building partnerships with faculty in a discipline little used to electronic resources at the time.

The MESL project whetted the campus appetite for ready access to high-quality digital images, and with the success of that project, Michigan's program expanded its support for image collections. The Image Services group was formed within the Digital Library Production Service to focus on technologies and services to convert and store continuous tone images, methods to enable access appropriate to the content (e.g., the ability to zoom into a specific point on an image for closer inspection), and the development of system architecture that supports distributed collection management. In the years since its inception, Image Services has partnered with a number of campus museums and archives and numerous commercial partners to build a web-accessible collection of over 100,000 images. In addition, Image Services has shared its expertise with organizations outside the University to develop resources of interest to the campus and the broader scholarly community. For example, staff collaborated with a small company to make its large collection of archaeological and anthropological-related slides available online.

From Project to Production: Institutionalizing Digital Library Services

By 1995, Michigan's program was firmly established on the UM campus with nearly 25 projects and services underway, and many efforts were receiving wide attention. From its organizational "home" in the University Library, the program facilitated campus discussions related to information resources and technology as well as contributing to national and international dialogue related to information architecture, digital preservation and conservation, and standards for electronic content. In 1996, recognizing the pressures created by such a large number of "production" digital library activities, Wendy helped assemble the resources to create Michigan's Digital Library Production Service (DLPS), charging it with responsibility for creating and sustaining the infrastructure for digital library resources, as well as for ensuring that those resources were well-positioned to adapt to future changes in the environment.[6] This move ensured not only a permanently-funded organizational home for efforts like PEAK and Making of America, but also, by the charge given to DLPS, reinforced that focus on impact-oriented activities. Although "research" was not foregrounded in the mission of DLPS, there was of course the expectation to incorporate the ongoing research effort of the larger community in its practices, as well as an expectation to contribute to that research effort through lessons learned. DLPS staff were and continue to be active in a number of community efforts to investigate questions about digital libraries, and initiatives like DLXS [see below], and particularly its "class" structure, grow out of lessons learned in the pursuit of creating production digital libraries.

Ongoing funding for Michigan's digital library program was received in 1999, with additional funds to expand support for system development and maintenance, programming, and program development. The focus was expanded to include more deliberate efforts in creating resources, tools, and services. And, the program's role within the University Library and the campus was expanded through the integration of the Library's systems and desktop support units into the program. This integration offered a more coordinated information technology environment within the Library and extended the program's focus to issues of information technology infrastructure such as library workstation authentication and security. By the turn of the century, Michigan's program offered a mature digital library environment, capable of shaping campus, national and international dialogue on information technology issues, and able to anticipate and meet the needs of a community of users with a growing appetite for digital resources. The program had also firmly established itself within the organizational structure of the University Library, serving as one of the Library's three organizational divisions.

With the local organization soundly established, the program looked to share its expertise worldwide. The Digital Library eXtension Service (DLXS) was established in 2000 to share the tools and methods for creating and building digital library collections more broadly within the library and other non-profit organizations. Also in 2000, the Scholarly Publishing Office was formed to partner with scholars and researchers in exploring online venues for the distribution of scholarship. SPO offers tools and methods for the electronic publication and distribution of scholarly resources. It supports the traditional constructs of journal and monographic publication in a networked environment, as well as publishing scholarly work expressly designed for electronic delivery.

Outcome Orientation: Real Products, Real Services

Certainly one of the most distinctive characteristics of the early University of Michigan digital library effort was its focus on resources that would have an appreciable impact on users and on the information environment. While the Michigan efforts did not eschew research questions and in fact were driven significantly by the outcome of investigations into appropriate formats and technologies, in nearly all cases, the early digital library initiatives were distinguished by their efforts to put in place systems, services, and collections that were widely available and could be used for teaching and research. It was perhaps this emphasis on usable digital libraries that fostered much of the organizational development, and then later won significant financial and moral support from the University's administration.

Michigan initiatives that have had an enduring presence, such as JSTOR and the Making of America, and even some that have not continued such as PEAK, were conceived and implemented at a time when most other peer institutions were focused on other digital library questions or strategies. Harvard, for example, initiated an ambitious and admirable effort that focused on exploration into the technological and organizational questions of digital libraries, including large questions of infrastructure and funding.[7] Most others focused their energies on the creation of smaller bodies of content from which they might draw conclusions about the needs of larger collections and initiatives.[8] Instead, the Michigan leadership put aside many of these questions, preferring to work from relatively well-understood previous successes and postpone questions that might be resolved in the later stages of implementation or release. Of course this strategy was not always without its problems. For example, several early, large content-oriented projects worked with an explicit assumption that Moore's Law would aid them in avoiding high disk costs, when in fact the content grew at a greater rate than the affordability of disk space to support them, forcing teams at Michigan to resort to compression schemes that might have been rejected in earlier stages of the initiatives. Overwhelmingly, however, this strategy worked not only to ensure a faster start to the Michigan digital library projects, but also helped to shape later questions that bubbled up from real successes and failures.

Sustainability: Fuel for the Long Haul

Fairly early in the development of the Michigan program, it became clear that while a project-based development and operational model was an excellent foundation for learning, it was not a model that would ensure long-term viability for the digital library. In order to exist as a continuing and vital part of the Library, the Digital Library program needed to become institutionalized and mainstreamed. To achieve this mainstream role, work was focused in several key areas, including:

• Creation of a flexible infrastructure that can be leveraged across many types of projects;
• Education of and participation by staff throughout the Library in digital efforts;
• Focus on the creation and acquisition of diverse and re-purposable digital content, i.e., content that can be used in many different contexts by many different types of users;
• Adherence to standards and best practices to guarantee long-term preservation of digital materials;
• Attention to the creation of policies to guide the digital library work.

It was a consistent imperative of to build information systems that can handle a variety of content, moving material online quickly, responsively, and cost-effectively. In early thinking about digital libraries, the focus was often on building systems tailored to the needs of particular projects. As the Michigan program developed, it focused on creating a hardware and software infrastructure that allows materials that fall into identifiable classes (such as full text, images, numeric data, finding aids and fielded data) to be put online with established "assembly-line" routines. Since class-based software works across subject-based collections, this also means that improvements can be made across many collections at one time, rather than retrofitting each collection with the improvements. So, for example, when sorting capability was added to the "text class" software, it became available to dozens of full text collections at once. This generic approach to online content has been codified in the DLXS suite of digital library tools. Although developed at the Digital Library Production Service for the needs of digital library collections, the DLXS tools are also deployed in the Scholarly Publishing Office, where they are used to put individual publications online at marginal cost. In addition, the supporting components of the digital library infrastructure, such as authorization and authentication mechanisms and statistics reporting tools are applicable across collections and classes.

Production-minded design is also manifested in the Michigan Library's move to digital reformatting of historical materials in need of preservation treatment, a move that began with the Making of America project. While MoA was a substantial effort in its own right, it was finally a learning project that helped the Library to understand the elements of digital reformatting. The Library then began to move toward digitization techniques, work tools and staffing that allow for the digital reformatting of any of the material in its collections, using permanent funding rather than seeking out grant funding on a per-project basis.

To ensure the viability of the digital library, it was also important to integrate its work into the general work of the Library and to draw upon staff in all areas of the organization, not just the technology experts. This meant a two-way process of education, in which non-digital library staff members needed to learn about the possibilities and constraints of digital library technology, and digital library staff members needed to learn about the ways in which more traditional library services can help to build sustainable systems. Although there is a strong core digital library team, it draws heavily upon the expertise and work of colleagues in Cataloging, Preservation, and Public Services. Thus, for example, in all of the large-volume book conversion projects such as MoA and, more recently, the NSF-funded conversion of historical math materials and the NEH-funded conversion of volumes important to the history of the Philippines, Monographic Cataloging staff members prepared and reviewed metadata, the Preservation Department helped formulate standards and ensure quality in digital reproductions, and subject specialists guided content development and assisted in interface specification. Moreover, by bringing the other library technology units— the Library Systems Office and Desktop Support Services— into the Digital Library Services division, the University Library has an organizational structure that allows for a rich interplay and communication between all aspects of its technology operation.

Although the development of a sound technical and human infrastructure was central to the development of the digital library at Michigan, the most important factor in ensuring sustainability has been the primacy of content. Like any library, the digital library needs to ensure that its content is of both immediate and enduring value to its users. In addition to this baseline commitment, the digital library must also make sure that its content is not tied to proprietary formats or created using non-standard methods that will interfere with migration to new formats and systems. For instance, after considerable investment in reformatting brittle books into bitonal page images, it is important to make sure that those page images will usable in many contexts for a long time into the future. The digital library also needed to find a balancing point between interface development that makes its systems both usable and useful and a willingness to consider those interfaces ephemeral and to always move the content to the interface that serves it best. In short, a long-term digital library must invest heavily in quality content that is independent of any one delivery system.

In service to this content and in the interest of making a durable digital library, it was (and continues to be) important to pay attention to standards and best practices that help to guarantee preservation of digital materials over time and to codify the digital library's use of these standards in policy statements. By participating in and adhering to the standards developed and promulgated by such bodies as the ISO and similar standards bodies, and to "best practices" promulgated by the library community (particularly under the aegis of the Digital Library Federation), the Michigan began working to create and maintain digital materials that are as enduring as their intellectual value warrants. In pursuit of these aims, the digital library led efforts within the library to write and promulgate policies that guide retention (and disposal) of digital materials (and in some cases the print materials from which they originated) and that establish guidelines for the care of digital masters.

All of the work described in the paragraphs above was intended to move the digital library out of the laboratory and, in some sense, into the factory. At times, this move necessitated losing some of the energy and excitement that accompanies innovative exploratory projects. At the same time, it resulted in a sense of stability in the digital library and an ever-increasing confidence in the enduring value of its work and the centrality of its role in the Library writ large.

Building Ties Beyond the University

It was clear from the early days that collaboration beyond the campus would be essential to creating the digital information environment envisioned by the founders of Michigan's program. Obviously, financial resources would be needed to supplement the small budget available to build infrastructure and create content. In addition, relationships with commercial content providers would need to be established to provide content for ongoing and experimental projects. Creating lasting partnerships with other institutions and digital library programs would also provide another layer of complementary expertise to support continued development. It is this area of collaboration that has perhaps been the Program's greatest success.

MoA and PEAK both illustrate the digital library program's unique ability to bring together a number of disparate participants to create resources and services. As mentioned earlier, MoA began as a collaborative endeavor with Cornell University to explore the viability of digitization as a method of preserving rapidly-deteriorating 19th century materials. In this early iteration of MoA, the focus was primarily on creating high-quality digital images for the purpose of creating print reproductions, with a limited number of information access points to allow retrieval of images. Michigan and Cornell staff collaborated in all aspects of the effort including original proposal development, writing the scanning RFP and selecting a vendor, content selection, and interface design. The effort was a marriage of two philosophies of digital content creation. Cornell's focus was on high-quality digital images that would be used to create print facsimiles to replace deteriorating paper copies. Michigan's focus was on creating online access methods and tools that would result in a highly-functional thematic digital library. Both philosophies resulted in a substantial collection that satisfied both goals. In addition, Michigan staff learned a great deal about digital preservation and scanning that would be put to good use in the second phase of the MoA project, funded by the Mellon Foundation in 1998.

PEAK utilized economic models developed at the University to explore and implement infrastructure components for security, e-commerce, and management of subscription models while pursuing ground-breaking pricing research. PEAK represented a coalition of sorts between parties that often have competing interests: libraries of varying sizes and focus, a publisher in control of a major share of the academic marketplace, a university press struggling to expand beyond traditional publishing models, and Library and School of Information researchers seeking to replace outdated modes of information dissemination with new methodologies and technologies. Each participant brought their unique perspective and expectations to the PEAK research environment, and the end result was greater discussion within the academic community of new and better ways to price and disseminate journal content and the availability of concrete data on how digital content is used.