Abstract

Despite user experience (UX) research becoming more commonplace in libraries, library staff don't always see its value or relevance. They may ignore or actively resist UX insights. We interviewed fifteen UX practitioners working in academic libraries in the US and Canada to get their perspectives on the reception of UX research in their libraries. We learned that as UX researchers, none of us is alone in facing moments of skepticism, mistrust, and indifference about UX research. We share some of the practical strategies and advice we learned from our interview respondents.

Introduction

This study was motivated by frustration with our work as user experience (UX) librarians in an academic library. Our job is to discover what our students and faculty need, want, and expect from the library. We explore their research and learning practices, and we examine how library services, resources, and spaces support these practices. This is enjoyable and often fascinating. We love talking to our users and observing their behaviors. But when it comes to applying our findings to improve the user experience, we run into problems. We rely on the skills and expertise of our library colleagues to implement many of our findings; however, they don’t always share our enthusiasm for UX research. Instead, they often challenge, reject, or just plain ignore our work, and as a result we aren’t able to make improvements and address user pain points.

Over the years we have developed strategies to work more effectively with our colleagues. We have invited them to participate as researchers, collaborated with them to develop insights and recommendations, and become more concise and strategic with our reports and communications. Nevertheless, we continue to encounter resistance to our research findings. Years of facing these challenges has left us feeling that our colleagues don’t trust our expertise, our methods, or our analysis and interpretation. We were curious to know whether other UX practitioners in libraries have had similar experiences. And if so, what are they doing about it?

Literature Review

There are no previous studies which focus explicitly on the connection between UX research, libraries, and trust. Here we approach our theme obliquely through a review of the literature on the experiences of UX practitioners in libraries and the literature on skepticism and mistrust of UX and qualitative research in general. We also look at the literature on the use of evidence for decision-making in libraries.

Studies on the work and experience of UX librarians acknowledge many of the challenges involved in assimilating UX practice in the library. Priestner (2019) pointed out, “UX usually sits awkwardly in most organisations—neither owned nor actively approved” (p.41). Lundberg’s (2017) master thesis touched on the importance of organizational buy-in and advocacy for UX methods in libraries (p. 70). Tobias (2015) described the importance of educating library staff about the mission and value of UX research, and of establishing clear roles and expectations in user experience projects involving library colleagues. The title of her article, “From dandelion to seeds to cottage garden: The transformation of user experience in the MSU libraries,” hints that some may view UX as a nuisance weed if not carefully managed.

Two recent studies investigated the experiences of UX library practitioners. The first, by MacDonald (2017), explored—among other things—“the challenges UX librarians face when doing their work, and the degree to which library staff understand UX librarianship” (p.195). These challenges include navigating the library’s culture, overcoming resource limitations, finding ways to scale and scope UX work, gaining trust/support from colleagues, not having enough training or expertise, and troubleshooting technology problems. More recently, MacDonald (2019) proposed a way forward to improve the adoption of UX research in organizations, acknowledging that “overcoming these organizational challenges is a persistent pain point for UX professionals” (p.187).

The second is a survey by Young et al. (2020) on UX maturity in academic libraries. The authors correlated UX maturity with a number of factors, including the UX practitioner’s facility with a variety of UX research methods and organizational leadership and support for UX. Most relevant to our own study, they also identified the importance of collaborating successfully with library colleagues on UX projects and using UX research for decision making. A library with a more mature UX program enjoys “a widespread understanding of UX...with user research data integrated into decision-making” (pp. 17–18). Many respondents to the survey acknowledged the opposite was true in their own library because “other staff or departments within the organization are not ready to embrace the potential of UX data, methods, and insights” (p. 18). One respondent lamented, “We are at the stage where people know they should consult with us, but either they don’t OR they do but don’t really hear the results, they are using us to confirm what they want to hear” (p.22).

Trust and acceptance of UX research in organizations is a common concern among UX professionals in many sectors. There are numerous articles in UX industry publications about advocating for UX research (Nielsen, 2005; Six, 2009), promoting a UX mindset (Caballer, 2016), convincing leadership of the value of UX (Berchild, 2018; Philips, n.d.; Travis, 2012), building trust in the UX process (Ross, 2011), and managing the stress of UX work (Wojnarowska, 2020).

In recognition of the challenges facing UX integration, researchers have proposed several UX maturity models which chart the levels of UX adoption and integration into an organization. Young et al. (2020) proposed an eight-stage model based on an older model by Nielsen Norman Group. At the most mature stage, “UX is practiced throughout the organization; decisions are made and resources are allocated only with UX insights as a guide” (p. 19). Similarly, in Chapman and Plewes’ (2014) five-stage maturity model, an organization reaches the highest level of maturity when leaders and employees value and integrate the contributions of UX research. Carraro (2014) also proposed a model in which UX maturity means everyone at all levels of the organization participates in UX. Priestner (2017) created a UX adoption model consisting of four spectra on which to plot the likelihood of UX success. The organization which values and practices innovation, agility, acceptance of failure, and user focus enjoys the greatest UX success.

UX researchers have learned that no UX department can succeed as an isolated unit. The wider organization must commit to the user perspective, otherwise UX research will have limited impact. Priestner (2017) emphasized that “in order for UX to really work [for an organization]...it needs to be part of everything they do” (para. 1). Godfrey (2015) stressed the importance of organization-wide “join-in” for usability to permeate the culture. Tempelman-Kluit (2017) provided strategies for garnering library buy-in for UX research.

As UX researchers, we want our findings to be used as evidence to help inform decisions and lead to positive changes for our users. But for many reasons, evidence-based decision making is not a given in most libraries. The barriers to using evidence effectively lie in organizational structure and culture. Koufogiannakis (2015) concluded that “[t]o practice in an evidence-based manner, a librarian needs a positive work environment, time to use or create evidence, a positive outlook, ongoing education and training, and access to relevant information” (p. 111). Hiller et al. (2008) cited library staff’s lack of training in research methods, as well as skepticism about user research “preferring instead to rely on their own assumptions and past practices to make decisions” (p. 228). Organizational culture and leadership are other important factors which can contribute to or detract from good practice.

These challenges are not unique to libraries. The literature on evidence-based medicine cites organizational politics and other system-level barriers as obstacles to using research evidence effectively. Bowen et al. (2009) noted political factors, lack of time and resources, and organizational structures among reasons evidence is not integrated into decision-making. When it comes to using qualitative research as evidence, the challenges can be even greater.

In most organizations, qualitative research is misunderstood and mistrusted compared to quantitative research. Morse (2006) argued that qualitative researchers “are in for a long and rough ride” (p. 399) before their work is taken seriously in health science. Ladner, a respected sociologist and UX researcher, frequently writes and speaks on the challenge of conveying the purpose and value of qualitative research in business organizations. In an online course on ethnographic research, she described the general bias toward quantitative, scientific “truth” and a lack of understanding of the value of qualitative insights (2020).

Qualitative research explores, discovers, and generates new ideas while quantitative research measures and validates existing things. Both are important, but qualitative research often takes more time, does not provide easy answers, and tends to be misunderstood and underappreciated. Ladner (2020) ascribed a natural skepticism toward qualitative research to a variety of human cognitive biases:

• We experience cognitive dissonance if the research contradicts our strongly held beliefs.
• We are vulnerable to the just-world fallacy when we believe that there’s nothing wrong with our product or service; it’s the incompetent or lazy user who needs to change.
• We are risk-averse and loss-averse; we don’t want to hear about problems so we can keep doing things the way we’ve always done them.

Research Questions

The literature demonstrates that organizations may block any research, and especially qualitative research, due to cognitive and cultural bias, internal politics, and organizational constraints. This legitimizes our experience and helps us understand some of the reasons it is so hard to develop a successful UX research program. Still, it does not assuage or explain the visceral feeling of not having our colleagues’ trust. We wanted to explore this further with other UX practitioners. Do they feel support from their colleagues for their UX work, or do they also experience trust issues? If they encounter resistance from library staff to their UX research, how do they explain this resistance, and what strategies do they employ to counter it?

Research Method

We obtained research ethics approval to investigate how UX practitioners in academic libraries perceive how their colleagues use and value UX research. We used a criterion-based purposive sampling method to recruit participants. Criterion sampling “involves searching for cases or individuals who meet a certain criterion” (Palys, 2008, p. 698). In our study, eligible participants were currently or recently employed in an academic library in Canada or the US and held positions in which they devoted a significant portion of their work to UX research. They must have worked in user research for at least two years, but did not need to be a professional librarian or have “user experience” in their job title. We scoured our own networks and reached out to colleagues who have presented and published in the field of library UX.

Before conducting the interviews, we individually wrote short reflections as an exercise to understand our positionality as researchers and to surface any biases that might influence the data analysis process. We discussed these reflections with each other. For example, Robin wrote in her reflection:

Juliene’s reflections were a lot less harsh! Robin’s strong view of our colleagues surprised her. By expressing these values and assumptions in advance of the interviews, we were more careful not to let our own biases slip into the conversations with respondents or into our analysis of their responses.

We conducted one-hour semi-structured interviews by phone or Google Hangouts with fifteen UX practitioners, all of whom happened to be professional librarians. During the interviews we explored the context of UX in their workplace and their feelings and experiences of working in library UX. The interview questions are available as an appendix. We did not ask every question to every participant, but let the conversation evolve according to the interests and passions of each participant. Mindful of bringing our own biases to the interview questions, we explicitly asked about trust only once, and we did not directly ask respondents if they feel their colleagues trust them or their research.

We transcribed the interviews and uploaded them to the qualitative data analysis software Nvivo. We each identified key concepts and themes in the first three interviews, then discussed these concepts and themes and agreed upon the strongest and most interesting. This is “open coding,” an inductive process of “bringing order to and making sense of the data” (Benaquisto, 2008, p. 86). We continued to code three interviews at a time, and throughout the process we met to discuss any emerging codes. For example, the concept of “self-care” was a strong theme which became one of our codes. We identified words, phrases, and paragraphs in the transcripts which related to the self-care theme. Through regular discussions, we came to a shared understanding of what “self-care” and the other codes we identified meant, though we did not develop an explicit coding frame with definitions of each code.

In total, we created nineteen codes, known as “nodes” in Nvivo. Nvivo let us sort the contents of the transcripts according to our code categories so we could reevaluate the strength of the patterns we were seeing. Where we saw overlaps and relationships in the themes, we collapsed some nodes, and we removed those we deemed weaker. In the end we had seven strong nodes, or themes, which we describe in detail below.

Limitations

Purposive sampling suited our research question. We were looking for UX library practitioners who had a track record of experience to draw on and the inclination to reflect on their UX practice. Our sampling choice is vulnerable to the critique of researcher bias. Did we deliberately (or unconsciously) select respondents who would reinforce our own beliefs and experiences? Is there an inherent commonality of thought and experience within our professional network of contacts, and among those practitioners who publish and present their work, which limited the potential breadth of responses? It is possible, but the advantage of talking to practitioners whom we knew had reflected on their practice and the exploratory nature of our research justifies the nonprobability sampling technique.

Findings

The participants work in a variety of academic library environments, both public and private, and in large medical/doctoral research institutions as well as smaller comprehensive universities in the US and Canada. Eight participants work at Canadian academic libraries. Three are at medical doctoral institutions, three are at comprehensive institutions, and one participant works at a primarily undergraduate university. This Canadian university classification scheme comes from the annual university rankings report by the Canadian current affairs magazine Maclean’s (Dwyer, 2018).

The remainder of the participants work at US university libraries. According to the Carnegie Classification of Institutions of Higher Education (n.d.), five participants work at public doctoral institutions and two work at private not-for-profit doctoral universities. The UX work experience of participants ranges from three to nine years; the average is just over five years. Most participants held other library positions before coming to UX. Some participants work in libraries with incipient UX programs or no formal UX program at all; others lead a large UX research department in their library. Some focus on UX; others have many other job responsibilities.

Despite these differences, we found many common concerns and perspectives among our participants. We ended up with seven codes that cut across nearly every interview:

1. Experiencing negative feelings
2. Managing trust
3. Handling organizational constraints
4. Building collegial relationships
5. Developing strong communication practices and educating colleagues
6. Working strategically
7. Practicing self-care

We describe each of these themes below.

Discussion

Despite the diversity of libraries and UX programs in which the participants work, we found much in common in their perspectives. Some participants were more successful than others in integrating UX research into their organization, but these successes did not come without long investment in building trusting relationships with colleagues and advocating for the value and legitimacy of UX work. Trust is an important factor, as is leadership support and a conducive organizational structure and culture. UX work can be draining, and it’s important for UX practitioners to recognize they are not alone in needing to protect and take care of themselves.

Most participants acknowledged that they taught themselves how to do UX research and did not have any formal credentials. Avery said many UX practitioners in libraries experience imposter syndrome, which is magnified when colleagues don’t trust our work. One participant who decided to pursue formal education in UX said this lent some credibility to her work. Perhaps pursuing credentialed professional development would raise our value in the eyes of our colleagues.

An interesting, though less prevalent, theme emerged which may be ripe for future research. This is the neoliberal critique of UX work. UX practitioners should be aware of the neoliberal critique and be ready to respond to it. Lee told us “there are people who are suspicious of me because they feel that UX work [like assessment work] is...just part of this big conspiracy to neoliberalize the library.” Some library staff worry that an over-emphasis on a frictionless user experience might be at the expense of student learning. Sam acknowledged that some colleagues flag “tensions between being user-centered and being learner-centered.” To this concern, Taylor responded, “I think about UX as care work. I don’t want things to be harder [for users] than they need to be.”

Recommendations

As UX librarians ourselves, this research helped advance our practice in many useful ways. We experienced the value of connecting with other UX practitioners for ideas, strategies, and support. Other librarians’ experiences validated our frustrations and concerns. Almost all the participants expressed that they were grateful for the opportunity to reflect on and share their experiences. Iman said “It’s actually really reassuring to hear these questions from you and know that we’re not alone.”

There are no easy answers to how to convince our colleagues that UX work is valuable and necessary; however, we can offer some insights from this study:

• Accept that the fruits of UX labor take time. Be patient and embrace small wins and improvements.
• Have empathy for colleagues as well as users. Seek to understand their motivations and pain points.
• Develop relationships with colleagues. They are critical, and this is time well spent.
• Be pragmatic. Understand how decisions get made in your organization and adapt your practices to the culture.
• Practice self-care. Empathy for the user is fundamental to UX research, but empathy can be a burden too, leading to adverse physical and emotional symptoms related to compassion fatigue (Castillo, 2019).

Conclusion

This research explored the experiences of UX practitioners in libraries with respect to feeling trusted and getting buy-in from library colleagues. Our findings showed us that we are not alone in the challenges we face as UX researchers. Respondents expressed relief and gratitude for being able to commiserate and share their frustrations with us. All of us strongly believe that UX research is necessary and important, and we will continue the journey to convince other library staff of the value of our work. It helps to know that we are on this journey together.

References

• Benaquisto, L. (2008). Codes and coding. In L. M. Given (Ed.), The SAGE encyclopedia of qualitative research methods. (pp. 85–87). SAGE Publications.
• Berchild, S. (2018, February 26). It starts with you: Introducing UX design at your organization. Medium.https://medium.springboard.com/it-starts-with-you-ux-and-influence-at-your-organization-a62b1f0b6d44
• Bowen, S., Erickson, T., Martens, P., & Crockett, S. (2009). More than “using research”: The real challenges in promoting evidence-informed decision-making. Health Policy, 4(3), 87–102. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2653695/
• Caballer, J. (2016, August 8). Changing your company culture to a ‘UX mindset?’ Medium.https://blog.prototypr.io/the-ux-mindset-53a343053bef
• Carnegie Classification of Institutions of Higher Education. (n.d.). Basic classification description.https://carnegieclassifications.iu.edu/classification_descriptions/basic.php
• Carraro, J.M. (2014, March 17). How mature is your organization when it comes to UX? UX Magazine.https://uxmag.com/articles/how-mature-is-your-organization-when-it-comes-to-ux
• Castillo, V. (2019, December 3). The emotional toll of working in UX. Medium.https://modus.medium.com/the-emotional-toll-of-being-in-ux-a7cc35bff4b5
• Chapman, L., & Plewes, S. (2014). A UX maturity model: Effective introduction of UX into organizations. In A. Marcus (Ed.) Design, user experience, and usability: User experience design practice (pp. 12–22). Springer International Publishing.
• Dwyer, M. (October 11, 2018). University rankings 2019: Our methodology. Maclean’s. https://www.macleans.ca/education/university-rankings/university-rankings-2019-our-methodology/
• Godfrey, K. (2015). Creating a culture of usability. Weave: Journal of Library User Experience, 1(3). http://dx.doi.org/10.3998/weave.12535642.0001.301
• Hiller, S., Kyrillidou, M., & Self, J. (2008). When the evidence is not enough: Organizational factors that influence effective and successful library assessment. Performance Measurement and Metrics, 9(3), 223–230. https://doi.org/10.1108/14678040810928444
• Koufogiannakis, D. (2015). Determinants of evidence use in academic librarian decision making. College & Research Libraries, 76(1), 100–114. https://doi.org/10.5860/crl.76.1.100
• Ladner, S. (2020). Ethnographic research design and innovation. EPIC. https://www.epicpeople.org/ethnographic-research-design-innovation/
• Lundberg, S. (2017). Change or die: A study on the phenomenon of library UX at two academic libraries in Sweden [Unpublished master’s thesis]. Uppsala University.
• MacDonald, C.M. (2017). “It takes a village”: On UX librarianship and building UX capacity in libraries. Journal of Library Administration, 57(2), 194–214. https://doi.org/10.1080/01930826.2016.1232942
• MacDonald, C.M. (2019). User experience (UX) capacity-building: A conceptual model and research agenda. Proceedings of the 2019 on Designing Interactive Systems Conference, 187–200. https://doi.org/10.1145/3322276.3322346
• Morse, J. M. (2006). The politics of evidence. Qualitative Health Research, 16(3), 395–404. https://doi.org/10.1177/1049732305285482
• Nielsen, N. (2005, March 27). Evangelizing usability: Change your strategy at the halfway point. Nielsen Norman Group. https://www.nngroup.com/articles/evangelizing-usability/
• Palys, T. (2008). Purposive sampling. In L. M. Given (Ed.), The SAGE encyclopedia of qualitative research methods. (pp. 697–698). SAGE Publications.
• Philips. M. (n.d.). The true ROI of UX: Convincing the executive suite. Designers.https://www.toptal.com/designers/ux/roi-of-ux-convince-executives
• Priestner, A. (2017, June 12). A new model of UX adoption. Constructivist.https://libreaction.wordpress.com/2017/06/12/a-new-model-of-ux-adoption/
• Priestner, A. (2019, September). Barriers to UX design: Towards a culture of experimentation. Information Professional, 40–41.
• Ross, J. (2011, February 7). Why don’t usability problems get fixed? UX Matters. https://www.uxmatters.com/mt/archives/2011/02/why-dont-usability-problems-get-fixed.php
• Six, J. M. (2009, March 9). Evangelizing UX across an entire organization. UX Matters.https://www.uxmatters.com/mt/archives/2009/03/evangelizing-ux-across-an-entire-organization.php
• Tempelman-Kluit, N. (2017). Creating UX buy-in. Library Journal, 142(10), 24.
• Tobias, C. (2015). From dandelion seed to cottage garden: The transformation of user experience in the MSU libraries. Re-think It: Libraries for a New Age—Conference Proceedings8, 1–9. https://scholarworks.gvsu.edu/rethinkit_proceedings/8
• Travis, D. (2012, April 2). How to tell managers they’re wrong about UX research and still get hired. User Focus.https://www.userfocus.co.uk/articles/how_to_tell_managers_theyre_wrong.html
• Wojnarowska, O. (2020, January 20). Burnout: The ugly side of UX. UX Collective.https://uxdesign.cc/burnout-the-ugly-side-of-ux-1a6b436a8f46
• Young, S.W.H., Chao, Z., & Chandler, A. (2020). User experience methods and maturity in academic libraries. Information Technology and Libraries, 39(1), 1–31. https://doi.org/10.6017/ital.v39i1.11787

Appendix: Interview Questions