Charleston 2017: COUNTER Release 5 — Consistency, Clarity, Simplification and Continuous Maintenance

Speakers: Lorraine Estelle (Project COUNTER), Anne Osterman (VIVA – The Virtual Library of Virginia), Oliver Pesch (EBSCO Information Services)

COUNTER┬áhas had very minimal updates over the years, and it wasn’t until release 4 that things really exploded with report types and additional useful data. Release 5 attempts to reduce complexity so that all publishers and content providers are able to achieve compliance.

They are seeking consistency in the report layout, between formats, and in vocabulary. Clarity in metric types and qualifying action, processing rules, and formatting expectations.

The standard reports will be fewer, but more flexible. The expanded reports will introduce more data, but with flexibility.

A transaction will have different attributes recorded depending on the item type. They are also trying to get at intent — items investigated (abstract) vs. items requested (full-text). Searches will now distinguish between whether it was on a selected platform, a federated search, a discovery service search, or a search across a single vendor platform. Unfortunately, the latter data point will only be reported on the platform report, and still does not address teasing that out at the database level.

The access type attribute will indicate when the usage is on various Open Access or free content as well as licensed content. There will be a year of publication (YOP) attribution, which was not in any of the book reports and only included in Journal Report 5.

Consistent, standard header for each report, with additional details about the data. Consistent columns for each report. There will be multiple rows per title to cover all the combinations, making it more machine-friendly, but you can create filters in Excel to make it more human-friendly.

They expect to have release 5 published by July 2017 with compliance required by January 2019.

Q&A
Q: Will there eventually be a way to account for anomalies in data (abuse of access, etc.)?
A: They are looking at how to address use triggered by robot activity. Need to also be sensitive of privacy issues.

Q: Current book reports do not include zero use entitlements. Will that change?
A: Encouraged to provide KBART reports to get around that. The challenge is that DDA/PDA collections are huge and cumbersome to deliver reports. Will also be dropping the zero use reporting on journals, too.

Q: Using DOI as a unique identifier, but not consistently provided in reports. Any advocacy to include unique identifiers?
A: There is an initiative associated with KBART to make sure that data is shared so that knowledgbases are updated so that users find the content so that there are fewer zero use titles. Publisher have motivation to do this.

Q: How do you distinguish between unique uses?
A: Session based data. Assign a session ID to activity. If no session tracking, a combination of IP address and user agent. The user agent is helpful when multiple users are coming through one IP via the proxy server.

Slides

ER&L 2016: Access Denied!

outlier
“outliers” by Robert S. Donovan

Speakers: Julie Linden, Angela Sidman, and Sarah Tudesco, Yale University

Vendors often use the data from COUNTER turn away reports as marketing tools to convince a library to purchase new content.

How are users being turned away from the content? How are they finding it in the first place? Google/Google Scholar, PUBMED, and publisher platforms that don’t allow for limiting to your content only are generally the sources.

Look for patterns in the turnaway data. Does it match the patterns in your use data and the academic year? Corroborate with examples from access issue reports. This can lead to a purchase decision. Or not.

Look for outliers in the turnaway data. What could have caused this? Platform changes, site outages (particularly for content you do license but appears on the turnaway report), reported security breaches, etc. You can ask for more granular data from the vendor such as turnaways by day or week, as well as IP address. You can ask the vendor for contextual information such as platform changes/issues, and more pointedly, do they think the turnaways are coming from real users.

Combine the data from the turnaway reports with ILL requests. Do they match up? This might mean that those titles are really in demand. However, bear in mind that many users will just give up and look for something else that’s just as good but available right now.

Analysis checklist:
IF you see a steady pattern:

  • Check holdings for access to the content
  • Consider the access model (MU/SU)

IF you see outliers:

  • Consider outside events

ASK the vendor for more information

  • Can you provide more granular data?
  • Can you provide contextual information?
  • Do you think this represents real users?

Audience Q&A:

Journal turnaways can include archival years for current subscriptions that aren’t included.

One very aggressive vendor used the library’s purchase request form to flood them with requests from users that don’t exist.

How are the outliers documented? Hard to do. Vendors certainly hang on to them, even when they acknowledge they know this isn’t legit.

ER&L 2016: Hard Data for Tough Choices: eBooks and pBooks in Academic Libraries

ebooks
“ebooks” by Randy Rodgers

Speakers: Katherine Leach and Matthew Connor Sullivan, Harvard

eBooks have not supplanted pBooks. Providing access to both formats is not possible…even for Harvard.

Users really do want and use both. There is a need for a better understanding of user behavior for both formats.

In 2014, they purchased the complete Project Muse collection, which included a significant and intentional overlap with their print collection. This allowed for a deep comparison and analysis.

You cannot compare them directly in a meaningful way. There are many ways of counting eBooks and pBooks are notoriously undercounted in their use. They looked at whether or not a book was used, and if it was used in only one format or multiple, and then how that compared to the average use across the collection.

26% of titles were used in both formats over the time period, only .5% on a monthly basis. It’s sometimes suggested that eBooks are used for discovery, but even at the monthly level this is not reflected in the data. The pattern of use of each format is generally about the same over the semester, but eBook use tends to be a little behind the pBook use. But, again, it’s difficult to get precise patterns of eBook use with monthly reports. There was no significant differences in format use by subject classification or imprint year or publisher, particularly when factoring the number of titles in each category.

They looked at the average decrease of a pBook over a four year period. They found a 35% decrease in circulation for each imprint year over that time, and this is without any impact of eBook. This is not always factored into these kinds of studies. They found that the decrease increases to 54% when eBooks are added to the mix. There’s also the issue of print use decreasing generally, with monographs losing out to eresources in student and faculty citation studies.

HSS at Harvard has been very clear that they want to continue the print collection at the level it has been, but they also want electronic access. How do we work with publishers to advocate for electronic access without having to purchase the book twice?

Audience Q&A:
What about providing short term loan access for the first 3-4 years? Harvard doesn’t like to purchase eBooks they don’t have perpetual access to.

P&E has been available for journals, why not books? Some publishers have worked with them to give deep discounts on print with an eBook package.

What has been the impact of electronic reserves on use? Haven’t looked at it.

How do you know if someone looked at the eBook and determined they didn’t want/need and that is why the pBook wasn’t used? Hard to determine. They don’t use eBook usage to drive the print acquisition — usually they already have the pBook.

Considering the lifecycle and the decrease in use over a short period of time from imprint year, does that cause you to question the purchase of eBook backfiles? eBook use over that time didn’t seem to decrease as significantly as the pBook.

ER&L 2016: Collections and Use

Infographics
“Infographics” by AJ Cann

The Bigger Picture: Creating a Statistics Dashboard That Ties Collection Building to Research
Speaker: Shannon Tharp, University of Wyoming

How can they tie the collection building efforts with the university’s research output? Need to articulate value to the stakeholders and advocate for budget increases.

She used Tableau to develop the dashboard and visualizations. Started with a broad overview of collections and then have expanded from there. The visualizations include a narrative and an intuitive interface to access more information.

The dashboard also includes qualitative interviews of faculty and research staff. They are tentatively calling this “faculty talk” and plan to have it up soon, with rotating interviews displaying. They are thinking about including graduate and undergraduate student interviews as well.

 

(e)Book Snapshot: Print and eBook Use in an Academic Library Consortium
Speaker: Joanna Voss, OhioLINK

What can we do to continue to meet the needs of students and faculty through the print to electronic book transition? Are there any patterns or trends in their use that will help? Anecdotally we hear about users preferring print to electronic. How do we find data to support this and to help them?

They cleaned up the data using Excel and OpenRefine, and then used Tableau for the analysis and visualization. OpenRefine is good for really messy data.

 

A Brief History of PaperStats
Speaker: Whitney Bates-Gomez, Memorial Sloan Kettering Cancer Center

Web-based tool for generating cost-per-use reports. It’s currently in beta and only working with JR1 reports. It works most of the time for COUNTER and SUSHI reports, but not always. The costs function requires you to upload the costs in a CSV format, and they were able to get that data from their subscription agent.

But, too bad for you, it’s going away at the end of the spring, but there might be a revised version out there some day. It’s through PubGet and Copyright Clearance Center decided to not renew their support.

ER&L 2016: Trying Something New: Examining Usage on the Macro and Micro Levels in the Sciences

Cheaper by the yard
“Cheaper by the yard” by Bill Smith

Speakers: Krystie (Klahn) Wilfon, Columbia University; Laura Schimming and Elsa Anderson, Icahn School of Medicine at Mount Sinai

Columbia has reduced their print collection in part due to size, but more because their users prefer electronic collections. Wilfon has employed a systematic collection of cost and data over time, a series of analysis templates based on item type and data source, and an organized system of distributing the end product. [She uses similar kinds of metrics I use in my reports, but far more data-driven and detailed. She’s only done this for two years, so I’m not sure how sustainable this is. I know how much time my own reports take each month, and I don’t think I would have the capacity to add more data to them.]

Mount Sinai had a lot of changes in 2013 that changed their collection development practices. They wanted to assess the resources they have, but found that traditional metrics were problematic. Citation counts don’t factor in the resources used but not cited; journal impact factors have their own issues; etc. They wanted to include altmetrics in the assessment, as well. They ended up using Altmetrics Explorer.

Rather than looking at CPU for the journal package as a whole, she broke it up by journal title and also looked at the number of articles published per title as a percentage of the whole. This is only one picture, though. Using Altmetric Explorer, they found that the newsletter in the package, while expensive in the cost per use, had a much higher median Altmetric score than the main peer reviewed journal in the package (score divided by the number of articles published in that year). So, for a traditional journal, citations and impact factor and COUNTER usage are important, but maybe for a newsletter type publication, altmetrics are more important. Also, within a single package of journal titles, there are going to be different types of journals. You need to figure out how to evaluate them without using the same stick.