spobooks5621225.0001.001 in

    6.6 Improving library budgeting with usage information

    Librarians are in an unenviable position when they select subscriptions to scholarly journals.[19] They must determine which journals best match the needs and interests of their community subject to two important constraints. The budgetary constraint has become increasingly binding because renewal costs have risen faster than serial budgets Haar (1999). The second constraint is that libraries have incomplete information about community needs. A traditional print subscription forces libraries to purchase publisher-selected bundles of information (the journal), while users are interested primarily in the articles therein. Users only read a small fraction of articles,[20] and the library generally lacks information about which articles the community values. Further compounding the problem, a library makes an ex ante (before publication) decision about the value of a bundle, while the actual value is realized ex post.

    The PEAK electronic access products relaxed these constraints. First, users had low-cost access to articles in journals to which the institution did not subscribe. This appeared to be important: at institutions that purchased traditional subscriptions, 37% of the most accessed articles in 1998 were outside the institution's traditional subscription base. This figure was 50% in 1999. Second, the transaction logs that are feasible for electronic access allowed us to provide libraries with monthly reports not only on which journals their community valued, but also which articles. Detailed usage reporting should enable libraries to provide additional value to their communities. They can better allocate their serials budgets to the most valued journal titles or to other access products.

    In this section we present analyses of the extent to which improved information available from an electronic usage system could lead to reduced expenditures and better service.

    Improved budgeting with improved usage forecasts

    We first estimate an upper bound on how much the libraries could benefit from better usage data. We analyze each institution's accesses to determine what would have been its optimal bundle if it had been able to perfectly forecast which material would be accessed. We then calculate how much this bundle would have cost the institution, and compare this perfect foresight cost with the institution's actual expenditures. Obviously even with extensive historical data, libraries would not be able to perfectly forecast future usage, so the realized efficiencies from better usage data would be less. Below we analyze how the libraries used the information from 1998 to change their purchasing decisions in 1999.

    We present these results by access product in Table 6.10. We found that actual expenditures were markedly higher than optimal purchases in 1998. In particular, institutions in the Red and Blue groups purchased far more traditional subscriptions than would be justified if they had perfect foresight. Most institutions purchased more generalized subscriptions than would have been optimal with perfect foresight. We believe that much of the budgeting "error" can be explained by a few factors:

    • First, institutions overestimated demand for access, particularly for journals for which they purchased traditional subscriptions.[21]

    • Second, institutional practices, such as "use it or lose it" budgeting and a preference for fixed, predictable expenditures, might have affected decisions. A preference for predictable expenditures would induce a library to rely more heavily on traditional and generalized subscriptions, and less on reimbursed individual article purchases or interlibrary loan.[22] However, Kantor et. al. (2001) Kantor et al. (this volume) report the opposite: that libraries dislike bundles because they perceive them as forcing expenditures for low-value items.

    • Third, because demand foresight is necessarily important, libraries might want to "over-purchase" to provide insurance against higher than expected usage demand. Of course, per-article purchases (possibly reimbursed to users) provide insurance (as does an interlibrary loan agreement), but at a higher cost per article than pre-purchased generalized subscription tokens, or than traditional subscriptions.

    Table 6.10: Actual versus optimal expenditures per access product for 1998-1999
    Access Product Totals
    Traditional Generalized Per Article
    Year Instid Actual Optimal Actual Optimal Actual Optimal Actual Optimal $ Savings % Savings
    1998 3 25,000 17,000 2,740 3,836 7 133 27,747 20,969 6,778 24.43%
    5 N/A 0 15,344 6,576 0 169 15,344 6,745 8,599 56.04%
    6 N/A 0 0 548 672 0 672 548 124 18.45%
    7 N/A 0 24,660 12,604 0 0 24,660 12,604 12,056 48.89%
    8 N/A 0 13,700 2,740 0 0 13,700 2,740 10,960 80.00%
    9 0 556 13,700 6,576 0 56 13,700 7,188 6,512 47.53%
    10 4,960 323 8,220 7,672 0 483 13,180 8,478 4,701 35.67%
    11 70,056 5,217 2,192 13,700 0 84 72,248 19,001 53,247 73.70%
    12 2,352 107 2,192 1,096 0 98 4,544 1,301 3,243 71.37%
    13 28,504 139 N/A 0 952 1,120 29,456 1,259 28,197 95.73%
    14 17,671 0 N/A 0 294 504 17,965 504 17,461 97.19%
    15 18,476 0 N/A 0 0 1,176 18,476 1,176 17,300 93.63%
    1999 3 12,500 10,528 2,740 1,096 84 0 15,324 11,624 3,699 24.14%
    5 N/A 0 8,768 2,740 0 399 8,708 3,139 8,708 63.96%
    6 N/A 0 0 548 686 0 686 548 138 20.12%
    7 N/A 0 10,960 9,864 0 511 10,960 10,375 585 5.34%
    8 N/A 0 6,028 5,480 0 462 6,028 5,942 86 1.43%
    9 0 278 7,124 6,576 7 182 7,131 7,036 94 1.33%
    10 2,480 1,401 8,768 6,576 0 210 11,247 8,187 3,060 27.21%
    11 0 576 4,384 2,740 427 532 4,559 3,848 711 15.60%
    12 0 0 1,644 548 0 539 1,644 1,087 557 33.88%
    13 9,635 7,661 N/A 0 19,964 7,175 29,599 14,836 14,763 49.88%
    14 0 0 N/A 0 623 623 623 623 0 0%
    15 8,992 1,058 N/A 0 511 1,694 9,502 2,751 6,751 71.04%
    Table 6.11: Predicted vs. actual direction of expenditure change for traditional and generalized subscriptions (by institution, 1998-99).
    Change in expenditure 1998-99
    Traditional Generalized
    Institution Predicted Actual Predicted Actual
    3 - 0 + +
    5 N/A N/A - -
    6 N/A N/A + 0
    7 N/A N/A - -
    8 N/A N/A - -
    9 + 0 - -
    10 - 0 - +
    11 - - + +
    12 - - - +
    13 - 0 N/A N/A
    14 - 0 N/A N/A
    15 - + N/A N/A
    NOTE: Predicted change direction is based on whether institution over- or under-purchased that product in 1999.
    "0" indicates no change; "N/A" indicates the access product was not available to that institution; "+" and "-" indicate an increase and decrease, respectively.

    We also analyzed changes in purchasing behavior from the first to the second year of the project. The PEAK team provided participating institutions with regular reports detailing usage. We hypothesized that librarian decisions about purchasing access products for the second year (1999) might be consistent with a simple learning dynamic: increase expenditures on products under-purchased in 1998 and decrease expenditures on products they over-purchased in 1998. For each institution we compared the direction of 1998-99 expenditure change for each access product to the change we hypothesized.[23] We present the results in Table 6.11.

    Six of the nine institutions adjusted the number of generalized subscriptions in a manner consistent with our hypothesis.[24] Fewer adjusted traditional subscriptions in the predicted direction. Two of the seven institutions that purchased more traditional subscriptions in 1998 than was ex post optimal then decreased the number purchased in 1999. Indeed, only three of the eight institutions made any changes at all to their traditional subscription lineup. This suggests an inertia that cannot be explained solely by direct costs to the institution. Perhaps libraries see a greater insurance value in having certain titles freely available through traditional subscriptions than from having generalized subscription tokens available that can be used on articles from any title. Generalized subscription tokens are also more expensive per article than traditional subscription prices, so the libraries are purchasing more potential usage with their budgets. Another explanation might be that libraries were more cautious about purchasing generalized subscriptions because it was a less familiar product.

    Table 6.12: Estimation results for forecast error
    Independent variable Coefficient (standard error)
    Year 1999 -35.7*
    (9.3)
    Green 54.6*
    (10.0)
    Red 53.3*
    (8.1)
    Blue 85.8*
    (9.2)
    Sample Size 24
    R2 0.85
    NOTE: Dependent variable is forecast error (in percent).
    No constant term is included in the regressions.
    Standard errors are shown in parentheses.
    * Significant at the 99% level.

    We performed a regression analysis to assess the differences between apparent over-purchasing in 1998 and 1999. Our dependent variable was the difference between the perfect forecast expenditure and actual expenditure, which we call the "forecast error". In Table 6.12 we report the effects of learning (the change in the error for 1999) and the average differences across experimental groups. The perfect foresight overspending over the life of the project averaged between 53% (Red) and 86% (Blue). However, the overspending was on average 36 percentage points lower in 1999. This represents a reduction of about one-half in perfect foresight overspending.[25]

    We also considered other control variables, such as the institution's level of expenditures, fraction of the year participating in the experiment and number of potential users, but their contribution to explaining the forecast error was not statistically significant. The between-group variation and the 1999 improvement account for about 85% of the variation, as measured by the R2 statistic.

    Decisions about specific titles

    In addition to comparing the total number of subscriptions for an institution with the optimal number, we can also identify the optimality for each particular title subscribed. We calculate, based on observed usage and prices, which titles an institution with perfect foresight should have obtained through traditional subscriptions, and call this the optimal set. Then we calculate two measures of actual behavior. First, we determine which titles in the optimal set an institution actually purchased. Second, we determine which traditional subscription titles the institution would have been better off foregoing because actual access would have been less expensive using other available access products.

    In Table 6.13 we present our analysis of the traditional subscription titles selected by institutions. There is wide variation both in the percent of purchased subscriptions that are in the optimal set, and in the percent of journals in the optimal set to which the institution did not subscribe,[26] Overall, there is substantial opportunity for improvement. This is not a criticism of institutional decisions. Rather, it indicates the opportunity for improved purchasing decisions if libraries obtain the type of detailed usage information PEAK provided.

    We do generally see better decisions in 1999. However, in both years a rather large percentage of subscribed journals were not accessed at all.

    Table 6.13: Optimality of subscription choices
    Institution Year Total subscriptions Percent subscribed that are in optimal set Percent of optimal set that were not subscribed Percent of subscriptions accessed at least once
    3 1998 907 53.3% 3.4% 92.5%
    10 1998 23 0.0% 100.0% 65.2%
    11 1998 663 3.6% 0.0% 84.5%
    12 1998 22 0.0% 100.0% 81.8%
    13 1998 205 0.5% 0.0% 12.7%
    14 1998 72 0.0% N/A 36.1%
    15 1998 102 0.0% N/A 48.0%
    3 1999 907 75.0% 7.7% 97.0%
    10 1999 23 13.0% 76.9% 65.2%
    13 1999 205 29.8% 62.6% 86.8%
    14 1999 72 0.0% N/A 20.8%
    15 1998 102 10.8% 8.3% 84.3%
    Dynamic Optimal Choice

    Access product purchasing decisions made by institutions have a profound impact on the costs faced by users, and thus on the realized demand for access. Therefore, in deciding what access products, electronic or otherwise, to purchase, an institution must not only consider the demand realized for a particular level of user cost, but also what would be demanded at differing levels of user costs. Likewise, in our determination of the optimal bundle of access products, we should not take the given set of accesses as fixed and exogenous. As a simple example, let us assume that a subscription to a given journal requires 25 accesses in order to pay for itself. Now assume that the institution in question did not subscribe to that journal, and that 20 tokens were used to access articles in the time period. At first look, it appears as though the institutions did the optimal thing. Let us assume, however, that we know that accesses would increase by 50%, to 30, when no password is required. It now appears as though the institution should have subscribed, since the reduced user costs would stimulate sufficient demand to justify these higher costs.

    Table 6.14: Optimal bundles with barrier-free access: Selected institutions
    Institution Year Trad. Subscriptions Addit. Articles Increase Total
    Actual Optimal Rescaled Optimal Actual Optimal Rescaled Optimal Optimal Cost Access Increase
    3 1998 500 556 1099 1130 9.39% 12.53%
    3 1999 737 805 236 146 4.85% 7.46%
    11 1998 24 31 2532 3019 21.11% 21.09%
    12 1998 1 1 254 287 17.76% 13.67%
    14 1999 0 0 168 249 48.21% 48.21%
    15 1999 12 17 242 366 47.56% 60.36%

    In Table 6.4 we reported results that allow us to estimate how much usage would increase if no passwords or other user costs were incurred. We now calculate the product purchases that would have optimally matched the usage demand that we estimate would have occurred had the library removed or absorbed all user costs. We report the results in Table 6.14.[27] For most institutions, the optimal number of journal subscriptions increases, because greater usage makes the subscription more valuable. In general, the estimated institution cost of the optimal bundle would not increase greatly to accommodate the usage increase that would follow from eliminating user costs. Although we cannot quantify a dollar value for the eliminated user costs (because they include nonpecuniary costs such as those from requiring a password), we show in the last two columns that the modest institutional cost increase would be accompanied by comparable or larger increases in usage. The greatest cost increase (48%) occurs for the institutions (14 and 15) at which generalized subscription tokens were not available and the institution did not directly subsidize the per-article fee, i.e. at those institutions where users faced the highest user costs. Thus, the higher institutional costs should be weighed against high savings in user costs (including money spent on per-article purchases).