Abstract

In order to establish a baseline understanding of the needs and the perceptions of service quality among graduate business and economics students and faculty, we employed a multi-method assessment approach combining quantitative and qualitative statistics through a survey, exit polls, and direct observations to inform improvement planning of library services and spaces.

While a substantial number of faculty and students were completely satisfied with the services and staff of the Thomas J. Watson Library of Business and Economics, there were some differences among student groups in terms of their satisfaction and interest in new services, technology, furniture, amount of space in the library, and service hours. The Watson Library is both a social and academic space, and students want to complete their academic work efficiently, without wasting time by switching locations, rooms, and/or devices. Our observation revealed that students rarely seem to engage in just one activity and are often studying, texting and talking with friends simultaneously.

Keywords: Assessment; Graduate students; Surveys

Introduction

Libraries have changed rapidly in the last decade as librarians attempt to keep up with the ever-changing needs and demands of users. The diverse service needs of students and faculty in various disciplines makes this challenge even more daunting. In this climate of constant change, understanding how faculty and students become aware of and interact with new services and technologies is critical. A well-planned assessment can provide evidence that extends beyond anecdote, allowing library administrators and other interested parties to make informed decisions that better meet the needs of their academic community.

In the past decade, the interest in library assessment has expanded greatly — particularly as a method to gather evidence and context for strategic planning and decision making. As has been the case with our counterparts across the nation, assessment of the quality and effectiveness of services and collections has grown exponentially at Columbia University Libraries. Over the past several years, the library staff at Columbia University has been collecting and analyzing an array of quantitative data, including card swipe access and interlibrary loan and circulation statistics, in an attempt to understand library visit patterns and collection use. Since 2003, Columbia University has administered campus-wide LibQUAL surveys, web-based instruments created and maintained by the Association of Research Libraries, to measure student and faculty perceptions of library service quality and to solicit feedback. While LibQUAL surveys have been useful in generating broad feedback on a wide range of library activities, they have provided few insights regarding the specific needs of graduate students and faculty in business and economics. Recognizing the differences in research and instruction practices across disciplines, we decided to implement an iterative assessment process to gain insight into the highly specialized needs of students and faculty of the School of Business and the Department of Economics. During the 2012-2013 academic year, Columbia University’s Watson Library initiated the first iteration of its assessment process, targeting its core user population of graduate students and faculty in business and economics in order to gain insight into their needs. The rationale for our assessment is grounded in the Columbia University’s Libraries’ 2010-2013 strategic plan, specifically the plan’s principles of research support and facilities improvement.

An important and useful way to begin the assessment process is by considering the context within which a library system functions and operates. Watson Library primarily supports the School of Business and the Department of Economics and is therefore mostly a graduate student library. However, students from across campus value Watson Library’s central location, group study space, and open food and drink policy. This makes Watson Library a popular spot for those who prefer a lively atmosphere when they study either alone or with others. More than half a million visits to Watson Library were recorded during the academic year 2011-2012, making it the second most frequented library on campus after Butler Library, which is the largest library on campus by both the number of seats and collection size.

In seeking to understand how our target groups view and use the space and services of Watson Library, it should be noted that MBA students in particular have expressed concerns about the library’s facilities. These concerns are based on their perceptions of how and by whom the space is used. More specifically, business school students assert that there is not enough space for them in Watson Library, because they feel that there are too many undergraduates using the space. These concerns, in conjunction with our desire to better understand the needs of our users, led us to undertake these assessment projects, the results of which are discussed in this paper.

Literature Review

Despite the overall growth of academic library assessment (Kyrillidou & Jaggars, 2013), to the best of our knowledge, almost nothing has been published on the use of library spaces and services by business and economics graduate students and faculty. Although one study targeted a population that is relevant to our assessment objectives – the study focused on the information-seeking behavior of management and commerce faculty in Sri Lankan universities (Wickramanayake, 2010) — the paper did not focus on the research behavior of users. However, there have been several studies with a broader focus on the needs of graduate students and faculty in a variety of disciplines.

Berger and Hines (1994) found in their survey that four distinct user populations — undergraduates, graduate students, faculty, and university staff — each used the library differently. Their research showed that graduate students behaved like a hybrid of undergraduates and faculty – like undergraduates they used the libraries for study space and like faculty they used the libraries for the collections and services (Berger & Hines, 1994). More than a decade later, Kayongo and Helm (2009) surveyed graduate students to assess their use of services and collections. They found that graduate students tended not to ask librarians for research help and relied heavily on e-collections.

The library assessment team at the University of Chicago surveyed graduate students about their use of libraries and collections (Tatarka, Larsen, Olson, Schilt & Twiss-Brooks, 2010). The results showed that students placed a greater emphasis on library study spaces, and their improvement, over services. Collections were also important, particularly the ease of access to resources (Tatarka, Larsen, Olson, Schilt & Twiss-Brooks, 2010). It is important to note that the students of the Booth School of Business at the University of Chicago were underrepresented in the survey results due to low response rate.

At Oregon State University, librarians conducted focus groups to assess the needs of graduate students on campus. While students were generally happy with services and spaces, researchers discovered that some graduate students, particularly those in less well-funded departments, needed extra support and spaces for their teaching and research (Rempel, Hussong-Christian & Mellinger, 2011).

In 2011, Georgetown University Libraries’ graduate student assessment needs group conducted a survey and focus groups of graduate students who used a specific graduate library at Georgetown (Gibbs, Boettcher, Hollingsworth & Slania, 2012). The first part of the study involved an online survey in which participants were asked about their perceptions of the library. The second part involved focus groups with subsets of the online survey participants. The researchers learned that students were generally happy with the services provided by the library staff but were dissatisfied with the quality and amount of study space in the library (Gibbs, Boettcher, Hollingsworth & Slania, 2012).

The findings from Chicago, Oregon, and Georgetown were echoed in a joint study by the Cornell and Columbia Libraries (Gessner, Jaggars, Rutner and & Tancheva, 2011), which focused on graduate students in the humanities. As the other studies found, PhD students who were interviewed for the study stated that they were generally happy with library collections but concerned about the type of space provided to them. They expressed a need for more space for individual study and a greater sense of community on campus (Gessner, Jaggars, Rutner and & Tancheva, 2011).

In a seminal study, launched to understand how University of Rochester undergraduate students write research papers, Foster and Gibbons (2007) borrowed observation techniques from anthropologists. Using surveys, focus groups, and interviews, the study led the way for the use of anthropological techniques in studying students’ behavior in libraries.

Methodology

In this study, we used a multi-method assessment approach, which combined quantitative and qualitative statistics and allowed comparison of outcomes across methods to tell the story of Watson Library. The development of the assessment plan began by establishing a clear and shared idea of goals we wanted to achieve. After a brainstorming session, the Watson Library staff defined their assessment objectives in specific and measurable terms:

1. To assess the degree to which students are satisfied with the library space;
2. To assess the degree to which students and faculty are satisfied with library services such as research help, circulation, course reserves, and technology equipment;
3. To assess the need for services not currently offered to faculty and students;
4. To understand how students use library spaces and to gain insight into their study habits.

Once the staff identified the objectives, the appropriate assessment methods became readily apparent. The assessment plan consisted of three parts:

1. Faculty and student survey to establish a baseline and understand perceptions of service quality;
2. Exit polls to learn what users are coming to the library to do;
3. Observation study to gain insight into students’ study habits.

In the first part of the assessment, we used two locally developed online surveys, Faculty Service Quality Survey and Student Service Quality Survey, which were distributed by the Head of Watson Library to all faculty and students in the School of Business and Department of Economics via email.

Data were collected between September 25, 2012 and October 12, 2012. There were 377 participants in the survey: 347 students and 30 faculty members. The overall response rate for the survey was 13% for students and 6% for faculty; the response rate at the item level varied as noted in the findings. The survey respondents were a representative sample of MS, MBA, EMBA, and PhD students, and adjunct and full-time faculty in the School of Business and Department of Economics. Therefore, the findings derived from the survey results are representative of our users.

The second part of the assessment involved polling students as they were leaving Watson Library. Exit polls involved asking library visitors to write down their affiliation and what they came to the library to do on that day. The polls were conducted during a typical week (one that is neither unusually busy nor unusually slow) in the 2013 spring semester. Interns and library staff administered the polls during the week of March 4 on a Wednesday morning, Thursday late afternoon, and Saturday early afternoon, polling a total of 220 people.

In the third part of the assessment, we used direct and structured observations to capture structured data, thoughts, and ideas with minimal intrusiveness and modest resources. Three observers over three different time periods, 10 a.m., 3 p.m., and 8 p.m., conducted observations during a typical week, between Monday, April 1, 2013 and Sunday, April 7, 2013. Through use of a floor plan for each room included in the study, we employed codes (Table 1) to mark the activity of each person occupying a user seat in the space. Overall, we coded a total of 3,290 library visitors during that week.

In addition, all observers were asked to stand or sit in one place to observe the space for five minutes at the end of the coding and make a few notes on their impression of the space.

Results and Discussion

Conclusions

Following are the key findings from the study:

Overall, a substantial number of students are completely satisfied with the services and staff of the Watson Library.

• The students say that librarians are available for consultations when needed and are able to clarify things they do not understand. The library staff is excellent in all aspects of service– they are helpful, available, responsive, and knowledgeable.
• PhD students and EMBA students expressed interest in services regarding statistical support.

Students believe that technology is important to academic success, and they are frustrated with network connectivity and printers.

• Students want a fast and reliable wireless connection at all times. Students report that the Business School’s wireless system has improved in the library this year as compared to the previous year, but it still performs well below expectations.
• Students report that printers often do not work and, when they do, they are very slow.
• When it comes to mobile technology, most of the students report that they use mobile devices in connection with their academic work, though a much smaller percentage of students report that they use mobile devices to access library resources and services.

Students are dissatisfied with the furniture, amount of space in the library, and service hours of the facilities.

• Students want to complete their academic work efficiently, without wasting time by switching locations, rooms, and/or devices.
• Students report that they cannot find a seat in the library on their visit.
• Students expect the library to provide comfortable and readily available group study areas.
• The lack of quiet study space was also a frequent student complaint. While group study space remains important for business school students, quiet study space is also critical to their studies.
• Library hours need to be significantly extended to accommodate students’ academic schedules. Students want the library to open an hour earlier and to remain open after midnight, especially during exam periods, and to extend summer hours to accommodate the EMBA summer schedule.
• Students report that furniture is old, dirty, uncomfortable, and inadequate.

Students rarely seem to engage in just one activity and often study, text, and talk with friends at the same time.

• Users tend to frequently switch between activities. While they may have their laptop and notes open for academic purposes, they were frequently observed switching among various activities including studying, texting and chatting with friends.
• Users are also using multiple devices in their work when they are working on their own or with others.

Overall faculty are very satisfied with the services and staff of Watson Library.

• The faculty appreciates the skills and knowledge of librarians; many stated that the librarians have been very helpful in addressing their research needs.
• Faculty agree with students that librarians are available for consultations and help clarify things they don’t understand.

Adjunct faculty have different needs than tenure track faculty.

• These faculty would like help from the library staff in using and formatting data.
• They would also like assistance using statistical software.
• Adjunct faculty have a greater preference for accessing library resources via mobile devices.

The findings do not surprise us; we have known the shortcomings of library space as both anecdotal and hard evidence have suggested. Analysis of card swipe data shows that due to a very high number of library visitors in October, February, and April, Watson Library reaches maximum seating capacity on Monday, Tuesday, and Wednesday afternoons. The data are consistent with student comments, stating that they very often cannot find a seat in the library to study either individually or as a group. We understand the importance of managing the high expectations of business school and economics students and are aware that more needs to be done despite our serious space constraints.

Actions

Assessment projects should result in data that can be used to justify concrete actions. In this section we review the actions we took based on the assessment results and discuss plans for future projects.

• Replaced old chairs in the reading room; to date we have replaced over 100 and will have replaced all reader chairs and soft seating by the end of summer 2015.
• Renovated a small area of the library to add twenty-four new seats.
• Replaced the library printers and added three scanning stations.
• Worked with University IT and Business School IT to improve the wireless network in the library.
• Replaced the shades in the study rooms and refinished some of the tables in these rooms.
• Added tabletop outlets and USB ports to all tables in main reading room.

As stated earlier, Watson Library is mainly a group study space. The observation study showed that students take full advantage of the space by studying in groups or sitting together while working individually. Based on our observations, the library is both a social and academic space; this culture has been determined and maintained by those who use the space. We plan to continue observation studies systematically to capture data for comparison purposes.

In the future, it would be useful to conduct focus groups with EMBA and adjunct faculty to determine their exact needs with statistical software tools and training support.

The first iteration of our assessment plan was successful. This initial cycle has yielded useful results for improvements to the library services and spaces as well as deeper insights into user needs. The knowledge and information that we have gained through the assessment implementation process will be used to improve the assessment plan prior to the next cycle.

References

• Berger, K. W., & Hines, R. W. (1994). What does the user really want? The library user survey project at Duke University. Journal of Academic Librarianship, 20(5/6), 306.
• Castro Gesner, G., Jaggars, D., Rutner, J. & Tancheva, K. (2011). Supporting humanities doctoral student success: A collaborative project between Cornell University Library and Columbia University Libraries. Retrieved from http://www.clir.org/pubs/ruminations/02cornellcolumbia/report.html/report.pdf
• Gibbons et al. (2007). Studying Students: The undergraduate research project at the University of Rochester. Retrieved from https://bit.ly/2YvQ9oz.
• Gibbs, D., Boettcher, J., Hollingsworth, J. & Slania, H, (2012). Assessing the research needs of graduate students at Georgetown University. Journal of Academic Librarianship, 38(5), 268 – 276.
• Kayongo, J. & Helm, C. (2009). Graduate students and the library: A survey of research practices and library use at the University of Notre Dame. Reference & User Services Quarterly, 49(4), 341-349.
• Kyrillidou, M.,& Jaggars D. (2013) Current themes in academic library assessment: Select papers from the 2010 Watson Library Assessment Conference. Evidence Based Library and Information Practice, 8(2).
• Rempel, H. G., Hussong-Christian, U., & Mellinger, M. (2011). Graduate Student Space and Service Needs: A Recommendation for Cross-campus Solution. The Journal of Academic Librarianship, 37(6), 480–487. doi:10.1016/j.acalib.2011.07.004
• Tatarka,A., Larsen,D., Olson, T., Schilt, M. & Twiss-Brooks, A.(2010). Library survey 2010: Graduate and professional students. Retrieved from http://www.lib.uchicago.edu/e/surveys/2010/Lbrary%20Survey%202010%20Full%20Report.pdf
• Wickramanayake, L. (2010). Information-seeking behavior of management and commerce faculty in Sri Lankan universities: An evaluation. Library Review, 59(8), 624 – 636.

APPENDIX

Kathleen Dreyer is the head of the Watson Library of Business and Economics at Columbia University, [email protected]. Nisa Bakkalbasi is assessment coordinator for Columbia University Libraries, [email protected].