ER&L 2014 — Beyond COUNTER: The changing definition of “usage” in an open access economy

Speakers: Kathy Perry (VIVA), Melissa Blaney (American Chemical Society), and Nan Butkovitch (Pennsylvania State University)

In 1998, ICOLC created guidelines for delivering usage information, and they have endorsed COUNTER and SUSHI. COUNTER works because all the players are involved and agree to reasonable timeframes.

COUNTER Code of Practice 4 now recognizes media and tracking of use through mobile devices.

PIRUS (Publisher and Institutional Repository Usage Statistics) is the next step, but they are going to drop the term and incorporate it as an optional report in COUNTER (Article Report 1). There is a code of practice and guidelines on the website.

Usage Factor metric as a tool for assessing journals that aren’t covered by impact factor. It won’t be comparable across subject groups because they are measuring different things.

If your publishers are not COUNTER compliant, ask them to do it.

ACS chose to go to COUNTER 4 in part because it covers all formats. They like being able to highlight usage of gold open access titles and denials due to lack of license. They also appreciated the requirement for the ability to provide JR5, which reports usage by year of publication.

Big increases in search can also mean that people aren’t finding what they want.

ACS notes that users are increasingly coming from Google, Mendeley, and other indexing sources, rather than the publisher’s site itself.

They hear a lot that users want platforms that allow sharing and collaborating across disciplines and institutions. Authors are wanting to measure the impact of their work in traditional and new ways.

Science librarian suggests using citation reports to expand upon the assessment of usage reports. If you have time for that sort of thing and only care about journals that are covered by ISI.

Chemistry authors have been resistant to open access publishing, particularly if they think they can make money off of a patent, etc. She thinks it will be useful to have OA article usage information, but needs to be put in the context of how many OA articles there are available.

What you want to measure in usage can determine your sources. Every measurement method has bias. Multiple usage measurements can have duplication. A new metric is just around the corner.

ER&L 2014 — Diving Into Ebook Usage: Navigating the Swell of Information

“Two boys jumping & diving” by Xosé Castro Roig

Speakers: Michael Levine-Clark (University of Denver) & Kari Paulson (ProQuest eBrary/EBL)

ProQuest is looking at usage data across the eBrary and EBL platforms as they are working to merge them together. To help interpret the data, they asked Levine-Clark to look at it as well. This is more of a proof-of-concept than a final conclusion.

They looked at 750,000 ebooks initially, narrowing it down for some aspects. He asked several questions, from the importance of quality to disciplinary preferences to best practices for measuring use, and various tangential questions related to these.

They looked at eBrary data from 2010-2013Q3 and EBL data from 2011-2013Q3. They used only the titles with an LC call number, and separate analysis of those titles that come from university presses specifically.

Usage was defined in three ways: sessions, views (count of page views), and downloads (entire book). Due to the variations in the data sets (number of years, number of customers, platforms), they could not easily compare the usage information between eBrary and EBL.

Do higher quality ebooks get used more? He used university press books as a measure of quality, though he recognizes this is not the best measure. For titles with at least one session, he found that the rate of use was fairly comparable, but slightly higher for university press books. The session counts and page views in eBrary was significantly higher for UP books, but not as much with EBL. In fact, consistently use was higher for UP books across the categories, but this may be because there are more UP books selected by libraries, thus increasing their availability.

What does usage look like across broad disciplines? Humanities, Social Sciences, and STEM were broken out and grouped by their call number ranges. He excluded A & Z (general) as well as G (too interdisciplinary) out of the equation. The social sciences were the highest in sessions and views on eBrary, but humanities win the downloads. For EBL, the social sciences win all categories. When he looked at actions per session, STEM had higher views, but all downloaded at about the same rate on both platforms.

How do you measure predicted use? He used the percentage of books in an LC class relative to the total books available. If the percentage of a use metric is lower then it is not meeting expected use, and vice versa. H, L, G, N, and D were all better than expected. Q, F, P, K and U were worse than expected.

How about breadth versus depth? This gets complicated. Better to find the slides and look at the graphs. The results map well to the predicted use outcomes.

Can we determine the level of immersion in a book? If more pages are viewed per session in a subject area, does that mean the users spend more time reading or just look at more pages? Medicine (R), History of the Americas (F), and Technology (T) appear to be used at a much higher rate within a session than other areas, despite performing poorly in breadth versus depth assessment. In other words, they may not be used much per title, but each session is longer and involves more actions than others.

How do we use these observations to build better collections and better serve our users?

Books with call numbers tend to be use more than those without. Is it because a call number is indicative of better metadata? Is it because publishers of better quality will provide better metadata? It’s hard to tell at this point, but it’s something he wants to look into.

A white paper is coming soon and will include a combined data set. It will also include the EBL data about how long someone was in a book in a session. Going forward, he will also look into LC subclasses.

ER&L 2014 — Making Usage Data Meaningful

“Big Data” by JD Hancock

Speakers: Jill Morris & Emily Guhde, NC Live

NC Live is a multi-type library consortium that includes public and private universities as well as public libraries. Everything they provide is provided equally to all libraries across the state.

They wanted to figure out how to better manage the resources, both financial and scholarly. They wanted to be able to offer advice to libraries on better accessibility of the resources like authentication and discovery. They wanted to determine what kind of use should be expected for a library or library type, and how to improve it.

They did not want to determine if a library’s use is good or bad, or to compare databases with each other. They also didn’t want to define the value of the provided database or explain why certain factors may impact database use, but they do hope to get to these things in future iterations of the study.

They began by trying to identify peer groupings of NC libraries based on information about the libraries, population served, degrees offered. These peer groups were then created by a working group of librarians from across the state. Some of the libraries were not included because they were incomparable.

The next objective was to determine what data points would be used to measure usage of each of the databases (they were going to study only five that were broadly applicable across all members of the consortium). For academics, they looked at full-text use per full-time enrollment, but for publics they did something different that I didn’t capture before the speaker moved on. See the study for details.

No one library was at the top or bottom of their peer group for usage across all resources studied. The use of the databases varied wildly, even among peers. The feedback from the consortium members indicated that flexible peer groups might be more useful than permanent peer groups based on what they are wanted to analyze at the time.

Finally, they looked at the qualities of the high usage libraries (top third of peer groupings), such as their access & authentication, size of collection, outreach & support, community characteristics, and library characteristics.

Use of Academic Search Complete was higher in community college libraries with these characteristics:

  • librarians attend faculty meetings
  • have an NC Live representative
  • high number of total information services per FTE
  • high number of circ transactions per FTE

Trends in community college libraries for all five databases:

  • embedded librarians in courses
  • librarian-initiated engagement
  • library orientation
  • librarians attending faculty meetings

Use of Academic Search Complete was higher in four-year college and university libraries with these characteristics:

  • authenticate with local proxy
  • direct link to NC Live provided resources
  • high number of librarians per 1000 enrolled student
  • NC Live representative

Trends in four-year college and university libraries for all 5 databases:

  • higher use with local proxy authentication and federated search service
  • lower use with a link to NC Live website database list instead of individual linking; authentication with a password; displaying an NC Live search box; essentially, less customized services which may indicate fewer tech staff to support eresources

Trends for higher use of Academic Search Complete among all schools:

  • authentication with a local proxy
  • total library expenditures per FTE
  • UNC institution
  • NCICU institution

Use of Academic Search Complete was higher in public libraries with these characteristics:

  • direct links to the resources
  • chat reference box
  • high number downloads of stats from the NC LIVE website
  • high number of promotional items requests
  • staff training for NC LIVE provided resources

Trends in public libraries for all 5 databases:

  • percentage of legal service population with a bachelor’s degree
  • number of stats downloads
  • population density
  • and total operating expenditures per legal service population

Next steps: Planning for future consortium services related to usage data. Need to understand more about what libraries need from them. They plan to share their findings and offer best practices for member libraries. Finally, they plan to develop usage reports and other data that are helpful for collection assessment at both the library and consortium levels.

Recommendations for future research: Libraries need to be better informed consumers of database and set goals for use. We need to work with each other and vendors to develop use and/or cost per use profiles. Similar studies should be done elsewhere to allow for comparison of results that might help explain why the variables are impacting use.

ejournal use by subject

A couple of weeks ago I blogged about an idea I had that involved combining subject data from SerialsSolutions with use data for our ejournals to get a broad picture of ejournal use by subject. It took a bit of tooling around with Access tables and queries, including making my first crosstab, but I’ve finally got the data put together in a useful way.

It’s not quite comprehensive, since it only covers ejournals for which SerialsSolutions has assigned a subject, which also have ISSNs, and are available through sources that provide COUNTER or similar use statistics. But, it’s better than nothing.

dreaming about the future of data in libraries

I spent most of the past two months downloading, massaging, and uploading to our ERMS a wide variety of COUNTER and non-COUNTER statistics. At times it is mind-numbing work, but taken in small doses, it’s interesting stuff.

The reference librarians make most of the purchasing decisions and deliver instruction to students and faculty on the library’s resources, but in the end, it’s the research needs of the students and faculty that dictate what they use. Then, every year, I get to look at what little information we have about their research choices.

Sometimes I’ll look at a journal title and wonder who in the world would want to read anything from that, but as it turns out, quite a number of someones (or maybe just one highly literate researcher) have read it in the past year.

Depending on the journal focus, it may be easy to identify where we need to beef up our resources based on high use, but for the more general things, I wish we had more detail about the use. Maybe not article-level, but perhaps a tag cloud — or something in that vein — pulled together from keywords or index headings. There’s so much more data floating around out there that could assist in collection development that we don’t have access to.

And then I think about the time it takes me to gather the data we have, not to mention the time it takes to analyze it, and I’m secretly relieved that’s all there is.

But, maybe someday when our ERMS have CRM-like data analysis tools and I’m not doing it all manually using Excel spreadsheets… Maybe then I’ll be ready to delve deeper into what exactly our students and faculty are using to meet their research needs.