/ Primo New User Interface–Not Just for Undergrads: A Usability Study

Introduction

Prior to the launch of the new Primo user interface in the summer of 2018, the University of Haifa conducted a usability study that aimed to determine whether users' information needs were met in a timely, convenient, and pleasant manner, the problems they encountered, and the changes necessary to meet their needs and improve their overall library experience. In order to remain relevant to its users, the library's discovery tool, as well as its services and physical conditions, must continue to make a significant contribution to the student and faculty learning and research processes.

The University of Haifa, located on top of Mount Carmel in Israel overlooking the Mediterranean Sea, has 18,000 students from a wide variety of ethnic backgrounds, studying in the following faculties: Humanities, Social Sciences, Law, Education, Management, Natural Sciences and Social Welfare & Health. Its library has been using Primo as its main discovery tool since 2009. In the summer of 2016, Ex Libris launched a new Primo user interface. Due to some Hebrew language and right to left script issues, the lack of some functionality, and the wish to examine the new interface more carefully, a decision was made at the University of Haifa to delay the new interface adoption and to launch it as a secondary interface to the Primo Classic user interface.

Since August 2017, the library has been customizing the user interface based on a number of known design principles:

  1. Enhancing accessibility (for example, the background color in most of the webpages was changed to white)
  2. Keeping visual consistency (calm colors, balanced graphics, helpful icons, short captions, bolder and normal-sized fonts, and more)
  3. Organizing the information in a consistent and known manner that takes into consideration the emotional and cognitive needs of users (the order of information in the full display of records, positioning the facets' pane on the opposite side, adding a link to a known Hebrew online dictionary—as it is a multilingual institution)
  4. Facilitating navigation and short paths to known functions
  5. Maintaining quick responses to user actions, and ensuring that the system is available, up-to-date, and functioning

As some of the issues were beyond the responsibility of the Primo Team, customer support cases were opened, questions were sent to the Primo community list, and changes were made after consultation with library colleagues.

Prior to usability testing, a feedback form was published on the website, which provided mostly vague and irrelevant responses. As human-computer interactions dominate patrons' lives, and as many are very familiar with aspects of the user experience, it was hoped that by conducting a systematic usability test they would be able to provide a different perspective to the new Primo user interface design.

Literature Review

Recent technological developments have raised a new type of learner who experiences the learning and research processes in a very different way from the old type of learner. Thus, the user experience is not only about getting the job done, or doing it efficiently, it is also about satisfying users' expectations of receiving a more interactive experience, giving them more control in the discovery process, and offering them as many options as possible to perform their assignments, while taking into consideration their different needs (Dede, 2005; Melcher, 2007). Another characteristic of the new millennial learner is multitasking (Judd & Kennedy, 2011) and the use of a number of media and parallel applications (Rideout, Foehr, & Roberts, 2010, 11–21; Roberts & Foehr, 2008). Some researchers argue that millennials often use fast linear task switching (also known as 'switch of attention') and rarely achieve full multitasking, but this depends on the type of the activity being performed (Wallis, 2006; Wallis, 2010).

Changing the library's main search interface demands consideration of young users as well as older users (lecturers, senior lecturers, and even students) who face technological barriers due to changes and innovations (Wandke, Sengpiel, & Sönksen, 2012). It is important to think about a wide variety of interpretations of terminology, icons, and other elements in the interface. For example, the icon of three dots means "additional options" in a menu, and a white box usually means “type here.” Additionally, smart devices (like smartphones) are useful tools for learners of all ages (Norris, Hossain, & Soloway, 2011), so it is important to have a new user interface that fully supports these devices in order to ensure that the default design, as well as functionality, took into consideration all types of user experience.

According to Krug (2010, pp. 16–17), usability testing is conducted because by observing users, designers will be better equipped to understand their needs. Library discovery tools, like commercial websites, aim for novice and experienced users to navigate them with ease, and to perceive them as an asset to their study or research. Usability tests are usually conducted during the planning stages of a new website, or before customization is done on a purchased product, such as a library discovery tool. It should be an ongoing process and is largely dependent on the ability and willingness of library systems' analysts to implement changes once problems have been identified.

Academic libraries usually conduct usability tests with the aid of an assessment team that decides on the population groups, sample, recruiting methods, and whether to compensate participants for their time (Krug, 2010, p. 39). It is important to adhere to the User Experience Professionals Association’s Code of Conduct (2005). According to Nielsen (2000), it is sufficient to conduct a usability test with five users as almost as many problems are gleaned with five as with double that number. He recommends using the time and resources saved by testing fewer participants for conducting additional rounds of testing. Krug goes one step further by claiming that three users are enough because "you’re not interested in what it takes to uncover most of the problems; you only care about what it takes to uncover as many problems as you can fix" (Krug, 2010, p. 43). He states that the first three users are very likely to encounter the most significant problems, it is less labor-intensive to test three people, and testing more than three often leads to discovering many minor issues, making it harder to see the most serious problems—the ‘can’t see the forest for the trees' effect. Rubin and Chisnell (2008, p. 54) recommend using a Verbal Protocol or a Think Aloud approach as "asking participants to think aloud during their sessions also reveals important clues about how they are thinking about the product or system they are using and whether the way it works matches up with the way it was designed." However, as talking slows down performance, this method should only be used when there is a more flexible time frame for task completion.

Although several usability studies have been conducted on Primo since its inception in 2012 (Brett, Lierman, & Turner, 2016; Kliewer, Monroe-Gulick, Gamble, & Radio, 2016; Nichols, Billey, Spitzform, Stokes, & Tran, 2014; Perrin, Clark, De Leon, & Edgar, 2014; Valentine & West, 2016; Warren, 2017), very few have been carried out on the new user interface. In addition, none have been carried out on a multilingual or bilingual interface (such as English and Hebrew). The current study was performed on the Hebrew version of the interface, which was its default language. Although the mother tongue of several participants—who come from different ethnic backgrounds—was not Hebrew (e.g. English and Russian), English is the language of most of their academic requirements. Furthermore, unlike other libraries, the University of Haifa does not use a blended search for the library catalog and Primo Central Index (Arguello, Wu, Kelly, & Edwards, 2012). This was primarily because a major Israeli database, the Index to Hebrew Periodicals (consisting of mostly Hebrew articles), could not be incorporated into Primo due to its use of a locally-developed thesaurus, rather than Library of Congress Subject Headings. As a result, Haifa's interface has three distinct tabs (in the new user interface they appear in a drop-down menu): (1) Library Catalog; (2) Haifa Index to Hebrew Periodicals; and (3) Primo Central Index—and two interface languages (Hebrew and English), which made the usability test slightly more complicated than with a blended-search, single-language interface.

Methodology

In December 2017, an email invitation and online registration form were sent to a representative sample of randomly selected students and faculty at the University of Haifa. Notwithstanding Nielsen's and Krug's recommendations to test no more than five users, a total of ten users were invited to participate in one of the three sessions, primarily because they were from three different population groups: one was a professor from the Faculty of Management and two were from the Department of Nursing, three were B.A. students from the Departments of Business Administration, Biology, and English Language & Literature, and four were M.A. students from the Departments of Hebrew Language, Hebrew & Comparative Literature, Archaeology, and Philosophy. An additional reason for testing ten users was that it enabled more questions to be asked: five participants were asked to perform one set of tasks for a maximum of one hour, and five others were asked to perform a different set of tasks. Two participants performed both sets of tasks (see appendix).

For internal reasons, the questions consisted of some very specific issues that were known problems, and not just general, easily solvable ones, as recommended by Krug. Specifically, the Media Department requested a question regarding the video recordings collection. The participants' verbalized thoughts and actions were recorded using Camtasia screen recorder, while a member of the assessment team took notes and answered questions (but did not provide solutions to the tasks, unless the participant was at a dead end for more than a minute). This method was more interactive than recommended by Nielsen and Krug, and it allowed participants to state their frustrations, expectations, and recommendations at every stage, which better suits Israeli users. After they had completed each task, the facilitators asked them whether the experience had been positive or negative, and gave them an opportunity to provide general comments on the visual display and layout. After all the sessions were completed, the team members viewed the recordings and their notes and analyzed the findings.

Results

The prevailing view was that the interface was more user-friendly than the classic Primo interface, and was especially suited to undergrads and younger users who are at ease with visual and interactive interfaces. Comments such as "it is intuitive" and "I found it easy to use" were typical of this group. Even when users in this group encountered difficulties, they quickly overcame them. Among older users, comments such as, "It is not simple," "I need to learn," "I need to receive instruction," "Next time, I will perform better," "I don't understand all the symbols," "It is different from the classic Primo," and "It is designed for undergrads" were typical. All participants, however, gave positive feedback on the graphical design of the interface with comments such as, "It has clean lines," "It is simple and modern," and "It is pleasant to the eye."

Most participants easily located specific and general information and performed a variety of tasks. They liked the display of all details on one screen and the save pin and facets, which were considered useful and convenient features. The functions they performed successfully were: finding specific items and accessing their full details, locating physical items and viewing electronic items, navigating within a list of results and sorting them, narrowing results using facets, using the advanced search, saving items and queries, creating alerts, and signing into their account.

The main problems we observed were connected to unclear terminology and layout, however, others were due to lack of familiarity with the interface, lack of technological literacy, and inexperience with library systems in general. The problems are divided into six categories below.

Spatial Organization & Navigation

Several participants mentioned the large number of functions and buttons; that the full display of records covered the entire screen, hiding the main menu and the link to the Library Card; that the chat banner hid the Close button to return to the results list; and that the New Search button was not clearly displayed. One participant opened another version of Primo every time he performed a new search rather than clicking New Search button in the main menu.

Personal Area & My Account (Library Card)

Some participants had difficulty finding where to log in to their Primo account, and once logged in, did not understand the default order of the list of checked out items.

Locating & Requesting Items

Several participants mentioned the amount of text in the results list–especially for items with many authors–and that the list of journal volumes is only displayed after an additional click (under the Get It section in the full display) instead of within the results list. Some could not find the language or subtitles language of videos, and said it was not clear that clicking anywhere in the result such as on the title (and not just on the link to Online Access) led to the full display. Others mentioned that clicking Online Access sometimes led to more than one vendor, that the first did not always provide access to the requested volume, and that there were too many options in the Get It section in general.

When requesting an item, all the participants clicked on the Loan Period drop-down menu even though it was a non-mandatory option, before they clicked on the Request button. Some also mentioned that there was no indication of how they would be informed of a book's arrival, where to collect it, or how they could see the status of all their requests.

Saving, Sending Emails, and Printing

None of the participants saved their items or searches to create a personal library, nor did they use the last searches feature, although one participant said that the My Last Searches icon indicated "history." Others thought it was a refresh button. Some participants had difficulty finding the Save pin to save items (especially in the Full Display screen), but once found, indicated it was clear that these items had been transferred to My Saved Items and could be reached by clicking on the red pin. None of the participants had difficulty sending several articles by email. Some participants noted that the Saved Items pin "moves from side to side" depending on the screen being viewed. Several expected to find My Saved Items under My Account (Library Card). Some noted that it was not clear that selecting all queries in My Saved Searches only selects a maximum of ten queries, and that they must continue to the next page in order to save more; others did not notice. Several mentioned that under My Saved Searches it was not clear that the bell icon indicated defining an alert; in addition, this icon was confused with the RSS link.

Searching for Items

Several participants did not realize that Books & More included other material types such as journals, theses, video recordings and maps, or that the text "What would you like to find?" indicated that they should enter a search term in place of this text. Some participants did not realize that clicking on the magnifying glass performed the search, so they just clicked Enter or chose the relevant tab from the autocomplete options. Others mentioned that it was not clear that the title option in the advanced search of "Articles & More" tab (Primo Central Index) referred to the journal title and not the article title (in the Index to Hebrew Periodicals tab this was not the case). Although the pre-filters of the advanced search were used with ease, some participants found the field "Words from Subject" confusing as it was not consistent with other fields such as Title and Author, which did not have the prefix "Words from." One participant asked how he could find an article in other languages as it was not clear that the term "English Articles" includes non-English languages as well as other material types, such as theses, reviews, and conference proceedings. Finally, none of the participants used the search scope (the drop-down menu within the catalog that one chooses before the execution of a search) “Peer-reviewed journals / articles” as they expected to find it under the facets (the filters that one chooses in order to filter the results received after performing a search).

Narrowing Results

None of the participants realized they had to select a value of a facet and then click Refine, and they all found it difficult to refine the creation date as there was no slider or clearly-defined fields to fill in. Two people stated that it was not clear that M.A. theses in Haifa (a value in the Material Type facet) were included in M.A. theses in general (another value in this facet), and that "Video Cassettes" and "All Video Recordings" were actually different material types (both appear separated in the Material Type facet). Some participants mentioned that they expected the selected pre-filters in the advanced search to be displayed above the facets, like when facets are selected, and not below the search box. Most participants began their searches with the default simple search and narrowed it using facets, and not with the advanced search. The majority of participants liked using the facets, but a local customization—of always showing the check box for each value in the facets, made it hard for them to use them.

Recommendations

The assessment team formulated the following recommendations and presented them to the Alma-Primo team, the library information systems team, and department heads. Once a consensus was reached, the solutions were ranked according to complexity. Some were implemented immediately, but others needed more planning or assistance from Ex Libris.

Table 1. Problems observed and solutions implemented.

Problems observed

Solutions implemented (or soon to be implemented)

Searching

Search box contained placeholder text, so not clear whether to type search term over this text or delete it first

Placeholder text was suppressed in Search box and in Browse Search

Books & More: not clear what "More" refers to

Alternative text could not be added in order to clarify that Books & More includes journals, e-journals, theses, archives, pictures, collections, and more material types

Not clear if the Title pre-filter in the advanced search in Primo Central Search refers to journal title or article title

Although a distinction between article title and journal title exists in the Index to Hebrew Periodicals, it cannot be implemented in Articles & More due to the software design

Inconsistent terminology in pre-filters: Author and Title do not have "Words From" before them, but Subject does

Changed "Words from Subject" to "Subject"

Button in the main menu for A-Z list of electronic journals is confusing as they are searchable in the simple search

Removed A-Z list of Electronic Journals button from main menu

Layout

New Search button is not clearly positioned as it is at the end of the main menu

New Search button moved so it is the first item in the main menu

Chat banner hides parts of the main menu

Live person chat was embedded in Primo front end via customization options. The Chat banner now appears on right side of the screen for the English interface and the left side for Hebrew

Personal Area & My Account (Library Card)

Login to My Account: not clear that must first enter Personal Area

Ex Libris recently redesigned this so there is a menu and a separate button to sign in to the Primo account

No indication under "Sort by" of the default order of List of Active Loans

Ex Libris is fixing the problem see Published (open) SalesForce case #541185

Locating & Requesting Physical / Electronic Items

No indication on results list of physical volumes available

One additional click on Location provides this information; an idea was posted on Idea Exchange

Clicking on Online Access sometimes leads to only one vendor even though there may be other more relevant vendors available

Clicking on Online Access now displays all the vendors

Too many clicks to narrow search results by Creation Date; unclear whether Creation Date is same as Publication Date

Two framed boxes were created around the From and To dates fields. Two sliders (From + To) were also created, but were not added to the production environment; changed to Publication Date

"Check Availability" in the View It section was confused with a system error

For local portfolios changed to "View Fulltext"

Public Notes about holdings were unclear when there were no print holdings

Removed unnecessary texts from Public Notes, displaying more "clean" View It section to the user

Coverage notes were unclear for e-journals with local portfolios

Deactivated most local portfolios and added the details of coverage as Public Notes

Barcode was redundant and overloaded screen

Changed color of barcode to pale gray so it is less prominent

Difficult to find language of subtitles for videos; the value "Video Recordings" in Resource Type facet is not clear

Added field Subtitles Language as a separated field to videos by using 041$$j MARC field and a suitable Mapping Table; changed the values of Resource Type facet to "DVD," "Video Cassettes," and "Video Recording - All" until the video cassettes collection is converted to DVDs

Not clear if facet Resource Type Ph.D. and M.A. Theses included Haifa Ph.D. & M.A. Theses

The facet labels were changed so the hierarchy is clear

Loan Period drop-down displayed when requesting an item even when it was a non-mandatory field

An idea was posted on Idea Exchange to display a default value in this menu when there is only one option

No indication of whether a request was successful or where to collect the requested item

A confirmation text now appears on the request screen, together with details about where to collect the item

My Last Searches button in the top bar was confused with the Refresh button

The My Last Searches button was removed from the top bar

Saving, Sending Emails, and Printing

Save Item pin in Full Display is easily-missed as it is in the top corner of the screen

An idea was posted on Idea Exchange to move the Save Item pin to the Share/Save section instead in the corner of the Full Display NERS 5671 “Currently, when in the full record display, the pin for saving items is displayed in the upper part of the screen. It would be more accessible to the patron if it would be displayed in Share/Save section”

No indication that when selecting "all searches" from My Last Searches or My Saved Searches lists, only the visible queries are saved

After opening a SalesForce case, the answer was that Ex Libris will not fix this issue, so a local customization was added: after selecting "all" queries, a message appears indicating that only visible queries were selected

No indication that searches have been successfully saved

Confirmation message in red text indicates that search query was saved in My Saved Items

RSS not used and overloaded screen

The RSS button was removed

Not clear that under My Saved Searches the Bell icon indicates creating an alert

No change was implemented as this was considered a matter of learning, but the Bell icon for an active alert was colored red

Cannot access My Saved Items from My Account

A link to My Saved Items (and searches) was added to the My Account menu using local customizations

Not clear that it was necessary to click three dots to send an email from My Account

The three dots were colored red and animated when one or more records were selected to highlight that there are more options once it has been clicked. Hopefully, it will be improved by Ex Libris in the near future

Narrowing results

Needed to select a value and click it to narrow search results in facets

This was a result of a local customization that forced check boxes to be shown next to each value in the facets. This was changed back to the default: a click on a value name narrows the search results; selecting value name with check boxes allow multiple narrowing options

Only facets used are displayed at top of the facet column and not the chosen pre-filters

Idea posted on Idea Exchange: to display the pre-filters used and not just facets Error! Hyperlink reference not valid.

Several categories in the Scopes were missed completely

A new static facet was created to include most of the collections that used to be under the Scopes drop-down menu (the historical "bases" in Aleph)

Summary and Conclusions

Although usability testing is a time-consuming pursuit, it proved to be very worthwhile at the University of Haifa as it highlighted various known and new issues, enabling staff to prioritize and take care of them. The usability report was approved by the department heads and solutions were ranked in order of importance. Several issues were implemented immediately by the Primo Team, while others required more planning, or were considered bugs or enhancements and were submitted through various Ex Libris channels. The results and recommendations, as well as the decisions and actions, can be implemented by other institutions based on local and institutional considerations.

Appendix

Questionnaire: Version A

First task

  1. Find the book Fictions by Borges
  2. What details do you need to find it on the shelf in the library?
  3. Find the video recording of the play Fictions by Pasolini
  4. Save it for future use
  5. Change the interface language to English
  6. How can you identify the items you saved in the results list?
  7. Save the Red Notebook listed below in the list of results
  8. Change the interface language to Hebrew. Cancel saving the Red Notebook
  9. Send the item information you saved to your email address

Second task

  1. Find three books in Hebrew related to the topic "the work of the historian"
  2. Save your search
  3. Email yourself the details of the book Apologie pour l'histoire ou metier d'historien
  4. What are the possible loan periods for the book The Archeology of Knowledge?
  5. How many items are on loan to you in total?
  6. What is the earliest date you must return one of the items?

Third task

  1. Find the video recording of the play Hamlet directed by Omri Nitzan, from 2005
  2. How many of these items exist? What is the difference in format between them?
  3. Find a 2014 movie in DVD format that begins with the word winter
  4. What is the source language of the movie you found? Does the film have subtitles?
  5. Save your search
  6. Search the subject Children in the Kibbutz
  7. What are the two most recent items held by the library?

Fourth task

  1. Find M.A. theses on the subject of Russian Jewry
  2. How many of the theses are available online and how many are available for borrowing?
  3. Save the search
  4. How many searches have you saved so far?
  5. Select one of your saved searches and set an alert for them to be emailed when new items arrive
  6. How many searches are saved in your recent searches?
  7. Select one of these searches and save it for future use

Fifth task

  1. Search for Hebrew articles related to the subject Sexual harassment in the army published after 2000
  2. How many articles did you find?
  3. Search for peer-reviewed articles on this subject
  4. Send yourself an email with the articles you found
  5. Find an article on the subject Collective Memory in the journal "2,000"

Questionnaire: Version B

First task

  1. Find the book Homesick by Eshkol Nevo
  2. Print the details (no need to actually print, only to reach the print screen)
  3. How many copies of the book are available for loan?
  4. Request the book
  5. Make sure the book was requested
  6. Cancel the request
  7. Check whether you have fines or fees that you owe the library

Second task

  1. Find the video recording of the play Hamlet directed by Omri Nitzan after 2005.
  2. How many of these items exist?
  3. Find a 2014 movie in DVD format that begins with the word winter
  4. What is the original language of the movie you found?
  5. Save your search
  6. Perform a search on the subject Children in the Kibbutz
  7. What are the two most recent items held by the library?

Third task

  1. Find M.A. theses on Russian Jewry
  2. How many of the theses are available online and how many are available for borrowing?
  3. Save the search
  4. How many searches have you saved so far?
  5. Select one of your saved searches and set an email notification when new items arrive
  6. How many search queries are saved in your recent searches?
  7. Select one of these queries and save it for future use

Fourth task

  1. Search only English language books with words Terror and France
  2. Did you find any items with online access? If so, what types of material formats?
  3. Cancel the language limitation in your list of results
  4. Did you find any items with online access?
  5. Cancel the limitation by material type Books, in the list of results you received
  6. Did you find any items with online access? If so, what types of material?

Fifth task

  1. Find the journal Philosophical Quarterly
  2. Find a print article published in this journal in 2011, issue 60
  3. Find the journal Nature
  4. Is there online access to the issue that was published on 30 November 2017?
  5. Find English articles on USA Policy and the Mexico Border
  6. Limit the results only to articles with online access from peer-reviewed journals
  7. Find English articles on Ideology in the Hebrew Language published during the last year
  8. Select three articles from the list you received and send them to your email

References

  • Arguello, J., Wu, W., Kelly, D., & Edwards, A. (2012). Task complexity, vertical display and user interaction in aggregated search. Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval (pp. 435–444). New York: ACM. Retrieved from https://ils.unc.edu/~jarguell/ArguelloSIGIR12.pdf
  • Brett, K., Lierman, A., & Turner, C. (2016). Lessons learned: A Primo usability study. Information Technology and Libraries, 35(1), 7–25. doi:10.6017/ital.v35i1.8965
  • Dede, C. (2005). Planning for neomillennial learning styles. Educause Quarterly, 28(1), 7–12. Retrieved from http://www.wou.edu/~girodm/670/Dede_on_neo-millenials.pdf
  • Judd, T., & Kennedy, G. (2011). Measurement and evidence of computer-based task switching and multitasking by ‘Net Generation’ students. Computers & Education, 56(3), 625–631. doi:10.1016/j.compedu.2010.10.004
  • Kliewer, G., Monroe-Gulick, A., Gamble, S., & Radio, E. (2016). Using Primo for undergraduate research: A usability study. Library Hi Tech, 34(4), 566–584. doi:10.1108/LHT-05-2016-0052
  • Krug, S. (2010). Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. Berkeley, CA: New Riders. Retrieved from https://books.google.co.il/books?id=9Q3OQVyX_-QC&pg=PA2&source=gbs_toc_r&cad=4#v=onepage&q&f=false
  • Melcher, M. (2007). Usability 2.0 again. Retrieved from https://x28newblog.wordpress.com/2007/05/26/usability-20-again
  • Nichols, A., Billey, A., Spitzform, P., Stokes, A., & Tran, C. (2014). Kicking the tires: A usability study of the Primo discovery tool. Journal of Web Librarianship, 8(2), 172–195. doi:10.1080/19322909.2014.903133
  • Nielsen, J. (2000). Why you only need to test with 5 users. Retrieved from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  • Norris, C., Hossain, A., & Soloway, E. (2011). Using smartphones as essential tools for learning: A call to place schools on the right side of the 21st century. Educational Technology, 51(3), 18–25.
  • Perrin, J. M., Clark, M., De Leon, E., & Edgar, L. (2014). Usability testing for greater impact: A Primo case study. Information Technology and Libraries, 33(4), 57–66. doi:10.6017/ital.v33i4.5174
  • Rideout, V. J., Foehr, U. G., & Roberts, D. F. (2010). Generation M²: Media in the Lives of 8- to 18-Year-Olds. Menlo, CA.: Henry J.Kaiser Family Foundation. Retrieved from http://www.kff.org/entmedia/upload/8010.pdf
  • Roberts, D. F., & Foehr, U. G. (2008). Trends in media use. The Future of Children, 18(1), 11–37. Retrieved from https://www.jstor.org/stable/pdf/20053118.pdf
  • Rubin, J., Chisnell, D. (2008). Handbook of usability testing: How to plan, design, and conduct effective tests (2nd ed.). Indianapolis, IN: Wiley Publishing. Retrieved from http://ccftp.scu.edu.cn:8090/Download/efa2417b-08ba-438a-b814-92db3dde0eb6.pdf
  • User Experience Professionals Association. (2005). UXPA Code of Professional Conduct. Retrieved from https://uxpa.org/resources/uxpa-code-professional-conduct
  • Valentine, B., & West, B. (2016). Improving Primo usability and teachability with help from the users. Journal of Web Librarianship, 10(3), 176–196. doi:10.1080/19322909.2016.1190678
  • Wallis, C. (2006). The multitasking generation. Time Magazine, 167(13), 48–55. Retrieved from http://www.balcells.com/blog/images/articles/entry558_2465_multitasking.pdf
  • Wallis, C. (2010). The impacts of media multitasking on children’s learning and development: Report from a research seminar. New York: The Joan Ganz Cooney Center at Sesame Workshop.
  • Wandke, H., Sengpiel, M., & Sönksen, M. (2012). Myths about older people’s use of information and communication technology. Gerontology, 58(6), 564–570. doi:10.1159/000339104
  • Warren, R. (2017). Usability study identifies vocabulary, facets, and education as primary Primo discovery system interface problems. Evidence Based Library and Information Practice, 12(3), 177–179. doi:10.18438/B89M14