Tag: usage data

Charleston 2016: COUNTER Release 5 — Consistency, Clarity, Simplification and Continuous Maintenance

Speakers: Lorraine Estelle (Project COUNTER), Anne Osterman (VIVA – The Virtual Library of Virginia), Oliver Pesch (EBSCO Information Services)

COUNTER has had very minimal updates over the years, and it wasn’t until release 4 that things really exploded with report types and additional useful data. Release 5 attempts to reduce complexity so that all publishers and content providers are able to achieve compliance.

They are seeking consistency in the report layout, between formats, and in vocabulary. Clarity in metric types and qualifying action, processing rules, and formatting expectations.

The standard reports will be fewer, but more flexible. The expanded reports will introduce more data, but with flexibility.

A transaction will have different attributes recorded depending on the item type. They are also trying to get at intent — items investigated (abstract) vs. items requested (full-text). Searches will now distinguish between whether it was on a selected platform, a federated search, a discovery service search, or a search across a single vendor platform. Unfortunately, the latter data point will only be reported on the platform report, and still does not address teasing that out at the database level.

The access type attribute will indicate when the usage is on various Open Access or free content as well as licensed content. There will be a year of publication (YOP) attribution, which was not in any of the book reports and only included in Journal Report 5.

Consistent, standard header for each report, with additional details about the data. Consistent columns for each report. There will be multiple rows per title to cover all the combinations, making it more machine-friendly, but you can create filters in Excel to make it more human-friendly.

They expect to have release 5 published by July 2017 with compliance required by January 2019.

Q&A
Q: Will there eventually be a way to account for anomalies in data (abuse of access, etc.)?
A: They are looking at how to address use triggered by robot activity. Need to also be sensitive of privacy issues.

Q: Current book reports do not include zero use entitlements. Will that change?
A: Encouraged to provide KBART reports to get around that. The challenge is that DDA/PDA collections are huge and cumbersome to deliver reports. Will also be dropping the zero use reporting on journals, too.

Q: Using DOI as a unique identifier, but not consistently provided in reports. Any advocacy to include unique identifiers?
A: There is an initiative associated with KBART to make sure that data is shared so that knowledgbases are updated so that users find the content so that there are fewer zero use titles. Publisher have motivation to do this.

Q: How do you distinguish between unique uses?
A: Session based data. Assign a session ID to activity. If no session tracking, a combination of IP address and user agent. The user agent is helpful when multiple users are coming through one IP via the proxy server.

Slides

ER&L 2016: Access Denied!

outlier
“outliers” by Robert S. Donovan

Speakers: Julie Linden, Angela Sidman, and Sarah Tudesco, Yale University

Vendors often use the data from COUNTER turn away reports as marketing tools to convince a library to purchase new content.

How are users being turned away from the content? How are they finding it in the first place? Google/Google Scholar, PUBMED, and publisher platforms that don’t allow for limiting to your content only are generally the sources.

Look for patterns in the turnaway data. Does it match the patterns in your use data and the academic year? Corroborate with examples from access issue reports. This can lead to a purchase decision. Or not.

Look for outliers in the turnaway data. What could have caused this? Platform changes, site outages (particularly for content you do license but appears on the turnaway report), reported security breaches, etc. You can ask for more granular data from the vendor such as turnaways by day or week, as well as IP address. You can ask the vendor for contextual information such as platform changes/issues, and more pointedly, do they think the turnaways are coming from real users.

Combine the data from the turnaway reports with ILL requests. Do they match up? This might mean that those titles are really in demand. However, bear in mind that many users will just give up and look for something else that’s just as good but available right now.

Analysis checklist:
IF you see a steady pattern:

  • Check holdings for access to the content
  • Consider the access model (MU/SU)

IF you see outliers:

  • Consider outside events

ASK the vendor for more information

  • Can you provide more granular data?
  • Can you provide contextual information?
  • Do you think this represents real users?

Audience Q&A:

Journal turnaways can include archival years for current subscriptions that aren’t included.

One very aggressive vendor used the library’s purchase request form to flood them with requests from users that don’t exist.

How are the outliers documented? Hard to do. Vendors certainly hang on to them, even when they acknowledge they know this isn’t legit.

ER&L 2016: Hard Data for Tough Choices: eBooks and pBooks in Academic Libraries

ebooks
“ebooks” by Randy Rodgers

Speakers: Katherine Leach and Matthew Connor Sullivan, Harvard

eBooks have not supplanted pBooks. Providing access to both formats is not possible…even for Harvard.

Users really do want and use both. There is a need for a better understanding of user behavior for both formats.

In 2014, they purchased the complete Project Muse collection, which included a significant and intentional overlap with their print collection. This allowed for a deep comparison and analysis.

You cannot compare them directly in a meaningful way. There are many ways of counting eBooks and pBooks are notoriously undercounted in their use. They looked at whether or not a book was used, and if it was used in only one format or multiple, and then how that compared to the average use across the collection.

26% of titles were used in both formats over the time period, only .5% on a monthly basis. It’s sometimes suggested that eBooks are used for discovery, but even at the monthly level this is not reflected in the data. The pattern of use of each format is generally about the same over the semester, but eBook use tends to be a little behind the pBook use. But, again, it’s difficult to get precise patterns of eBook use with monthly reports. There was no significant differences in format use by subject classification or imprint year or publisher, particularly when factoring the number of titles in each category.

They looked at the average decrease of a pBook over a four year period. They found a 35% decrease in circulation for each imprint year over that time, and this is without any impact of eBook. This is not always factored into these kinds of studies. They found that the decrease increases to 54% when eBooks are added to the mix. There’s also the issue of print use decreasing generally, with monographs losing out to eresources in student and faculty citation studies.

HSS at Harvard has been very clear that they want to continue the print collection at the level it has been, but they also want electronic access. How do we work with publishers to advocate for electronic access without having to purchase the book twice?

Audience Q&A:
What about providing short term loan access for the first 3-4 years? Harvard doesn’t like to purchase eBooks they don’t have perpetual access to.

P&E has been available for journals, why not books? Some publishers have worked with them to give deep discounts on print with an eBook package.

What has been the impact of electronic reserves on use? Haven’t looked at it.

How do you know if someone looked at the eBook and determined they didn’t want/need and that is why the pBook wasn’t used? Hard to determine. They don’t use eBook usage to drive the print acquisition — usually they already have the pBook.

Considering the lifecycle and the decrease in use over a short period of time from imprint year, does that cause you to question the purchase of eBook backfiles? eBook use over that time didn’t seem to decrease as significantly as the pBook.

ER&L 2016: Collections and Use

Infographics
“Infographics” by AJ Cann

The Bigger Picture: Creating a Statistics Dashboard That Ties Collection Building to Research
Speaker: Shannon Tharp, University of Wyoming

How can they tie the collection building efforts with the university’s research output? Need to articulate value to the stakeholders and advocate for budget increases.

She used Tableau to develop the dashboard and visualizations. Started with a broad overview of collections and then have expanded from there. The visualizations include a narrative and an intuitive interface to access more information.

The dashboard also includes qualitative interviews of faculty and research staff. They are tentatively calling this “faculty talk” and plan to have it up soon, with rotating interviews displaying. They are thinking about including graduate and undergraduate student interviews as well.

 

(e)Book Snapshot: Print and eBook Use in an Academic Library Consortium
Speaker: Joanna Voss, OhioLINK

What can we do to continue to meet the needs of students and faculty through the print to electronic book transition? Are there any patterns or trends in their use that will help? Anecdotally we hear about users preferring print to electronic. How do we find data to support this and to help them?

They cleaned up the data using Excel and OpenRefine, and then used Tableau for the analysis and visualization. OpenRefine is good for really messy data.

 

A Brief History of PaperStats
Speaker: Whitney Bates-Gomez, Memorial Sloan Kettering Cancer Center

Web-based tool for generating cost-per-use reports. It’s currently in beta and only working with JR1 reports. It works most of the time for COUNTER and SUSHI reports, but not always. The costs function requires you to upload the costs in a CSV format, and they were able to get that data from their subscription agent.

But, too bad for you, it’s going away at the end of the spring, but there might be a revised version out there some day. It’s through PubGet and Copyright Clearance Center decided to not renew their support.

ER&L 2016: Trying Something New: Examining Usage on the Macro and Micro Levels in the Sciences

Cheaper by the yard
“Cheaper by the yard” by Bill Smith

Speakers: Krystie (Klahn) Wilfon, Columbia University; Laura Schimming and Elsa Anderson, Icahn School of Medicine at Mount Sinai

Columbia has reduced their print collection in part due to size, but more because their users prefer electronic collections. Wilfon has employed a systematic collection of cost and data over time, a series of analysis templates based on item type and data source, and an organized system of distributing the end product. [She uses similar kinds of metrics I use in my reports, but far more data-driven and detailed. She’s only done this for two years, so I’m not sure how sustainable this is. I know how much time my own reports take each month, and I don’t think I would have the capacity to add more data to them.]

Mount Sinai had a lot of changes in 2013 that changed their collection development practices. They wanted to assess the resources they have, but found that traditional metrics were problematic. Citation counts don’t factor in the resources used but not cited; journal impact factors have their own issues; etc. They wanted to include altmetrics in the assessment, as well. They ended up using Altmetrics Explorer.

Rather than looking at CPU for the journal package as a whole, she broke it up by journal title and also looked at the number of articles published per title as a percentage of the whole. This is only one picture, though. Using Altmetric Explorer, they found that the newsletter in the package, while expensive in the cost per use, had a much higher median Altmetric score than the main peer reviewed journal in the package (score divided by the number of articles published in that year). So, for a traditional journal, citations and impact factor and COUNTER usage are important, but maybe for a newsletter type publication, altmetrics are more important. Also, within a single package of journal titles, there are going to be different types of journals. You need to figure out how to evaluate them without using the same stick.

Charleston 2014 – To Go Boldly Beyond Downloads

Speaker: Gabriel Hughes, Elsevier

New to the industry, and didn’t know what usage data was when he started. He’s interested in usage that COUNTER doesn’t count.

Internet based storage and sharing technology results in higher volume of reading of material than is reflected in download statistics due to scholars sharing the content more easily. Elsevier has done surveys on this, and 65% of those researchers surveyed this year agreed that they access articles from a shared folder or platform, which is increasing over time.

For the most part, sharing doesn’t happen because the recipient doesn’t have access. It’s more out of convenience, particularly with annotations or attached notes. Of course, he recommends using Mendeley (or similar tools, whatever they may be) to meet this need.

Elsevier is funding the research that Tenopir is doing on how and why researchers share, and how that compares with measured usage.

 

Speaker: Carol Tenopir, University of Tennessee

There are many tools and platforms designed to share citations and content, and they were designed to fit the research workflow. Informal methods are tools that weren’t designed for sharing citations/documents, but are used widely both personally and professionally to do so (i.e. Twitter, blogs).

They have done interviews and focus groups, and an international survey that went out two days ago. Sharing a citation or link is more common than sharing a document. Those that share their own work say that they mostly share what was uploaded to their institutional repository.

Altruism and the advancement of research trump any concerns about copyright when it comes to sharing content with other scholars.

There are some differences when it comes to books. Articles and research reports are more easily shared, but book royalties are a consideration that causes many to hesitate. They certainly wouldn’t want their own books shared instead of purchased.

Is a COUNTER-like measure/calculation possible? Good question. Any thoughts on that are welcome.

#libday8 day 2 — mushy brain work

Arrived and was greeted with paper renewal notifications covering my keyboard. Set those aside, logged in, and began sorting through the new email that arrived overnight and earlier this morning. Updated my calendar with new meetings/events, as well as the time I’ve blocked out for various tasks for the day.

First thing I tackled was notifications to the subject liasions about upcoming eresource renewals. I’m using the modified annual review checklist and data thinger that I shared about last month, and I’ve received positive responses from the subject liaisons.

At 10, I started my on-call shift for the Main Service Desk. I don’t normally do this, but I’m covering for a reference librarian who has to teach a class this morning. Basically, it means I monitor the IM reference and email, and am available to help at the desk if they need me. It also means I can keep working on whatever I’m working on, unless I get interrupted.

One of the things I’ve been working on lately is retrieving use statistics for calendar year 2011. But, it has been slow going, as I’ve been distracted with other pressing projects that normally would not interrupt this annual Jan/Feb activity. Part of what is taking me longer to prepare the annual review checklist & data for the upcoming renewal is that I have to retrieve the 2011 stats and clean them up, rather than just pulling from the files I have already.

I would like to take a moment here to say that I would almost prefer no use statistics to the ones where they only provide them for a month at a time. This requires running 12 different reports for a year, and 24 if searches and sessions are not in the same report. I say almost, because at least I get something, even if it is a royal pain in the ass to retrieve and exemplifies the short-sightedness of the publisher. I’m looking at trends, not miniscule pieces of data.

My on-call-ness and/or electronic resources librarian-ness kicked in midway through the shift when I was called out to help a student download a book in EBSCOhost. We figured out that he needed an account in EBSCOhost, and also to install Adobe Digital Editions on the lab PC. This worked for now, but I have put in a request that ADE be added to the image for all student lab computers.

Half the Sky: Turning Oppression into Opportunity for Women WorldwideFinally wrapped up the renewal stuff plus the associated use statistics stuff in time for my on-call shift to end and my lunch hour to begin. I took the opportunity to enjoy the spring weather by heading off-campus to run some errands. As it happened, I finished listening to an audiobook just as I returned, so I left a short review on GoodReads. The book is Half the Sky: Turning Oppression into Opportunity for Women Worldwide by Nicholas D. Kristof and Sheryl WuDunn, and it’s the One Book, One Campus selection at my university this year.

The next 20 min or so consisted of responding to email that had come in over lunch and checking Twitter. I didn’t want to get into much, since I was about to start my one hour shift at the Main Service Desk (actually staffing it this time around).

Desk was pretty slow. I had one question about where to find a book, and a few people looking for specific librarians. My co-consipirator at the desk and I spent some time kvetching about why it is that one of the highest ILL net lenders in the state (us) is still using clunky, out-dated software when even the most podunk libraries have ILLiad now. I looked at the stats from 2011, and ILLiad would have cost us less than a penny per transaction, and saved the user and the ILL staff so much time and lost productivity. My coworker thinks we’ll probably get it in the next year, but still… I can’t believe it’s taking so long!

Now back at my desk, I took a moment to follow up with EBSCOhost tech support regarding a problem we’ve encountered with the “Linked Full Text” in Academic Search Complete. I’d called it in last week and was waiting for a response. They still don’t know what’s broken, and it is still broken. Anyone else having problems with this?

Next I spent some time trying to sort out why in one month we received two invoices followed by two credit memos from the same publisher for the same resources. Turns out the invoices had errors and the credit memos were their way of zeroing out our balance. A simple explanation or note would have saved me a phone call. Ah, the joys of automated billing systems! While I was at it, I sent them a note with updated contact info, as one invoice/credit was addressed to a predecessor of more than six or seven years, and the other addressed to the collection development librarian who will be retiring this summer. Figured I’d get it taken care of now so we don’t miss something in the future.

To wrap up the day, I reviewed the responses to an RFI for discovery services that we sent out last month. We’ll be having demos of three different systems tomorrow, and I wanted to prep some follow-up questions in advance. So. Many. Words. I know I’m going to need a drink or two by the end of the day.

dreaming about the future of data in libraries

I spent most of the past two months downloading, massaging, and uploading to our ERMS a wide variety of COUNTER and non-COUNTER statistics. At times it is mind-numbing work, but taken in small doses, it’s interesting stuff.

The reference librarians make most of the purchasing decisions and deliver instruction to students and faculty on the library’s resources, but in the end, it’s the research needs of the students and faculty that dictate what they use. Then, every year, I get to look at what little information we have about their research choices.

Sometimes I’ll look at a journal title and wonder who in the world would want to read anything from that, but as it turns out, quite a number of someones (or maybe just one highly literate researcher) have read it in the past year.

Depending on the journal focus, it may be easy to identify where we need to beef up our resources based on high use, but for the more general things, I wish we had more detail about the use. Maybe not article-level, but perhaps a tag cloud — or something in that vein — pulled together from keywords or index headings. There’s so much more data floating around out there that could assist in collection development that we don’t have access to.

And then I think about the time it takes me to gather the data we have, not to mention the time it takes to analyze it, and I’m secretly relieved that’s all there is.

But, maybe someday when our ERMS have CRM-like data analysis tools and I’m not doing it all manually using Excel spreadsheets… Maybe then I’ll be ready to delve deeper into what exactly our students and faculty are using to meet their research needs.

ER&L: Buzz Session – Usage Data and Assessment

What are the kinds of problems with collecting COUNTER and other reports? What do you do with them when you have them?

What is a good cost per use? Compare it to the alternative like ILL. For databases, trends are important.

Non-COUNTER stats can be useful to see trends, so don’t discount them.

Do you incorporate data about the university in makings decisions? Rankings in value from faculty or students (using star rating in LibGuides or something else)?

When usage is low and cost is high, that may be the best thing to cancel in budget cuts, even if everyone thinks it’s important to have the resource just in case.

How about using stats for low use titles to get out of a big deal package? Comparing the cost per use of core titles versus the rest, then use that to reconfigure the package as needed.

How about calculating the cost per use from month to month?

css.php