ER&L 2012: Trials by Juries — Suggested Practices for Database Trials

IBM System/370 Model 145
photo by John Keogh

Speakers: Annis Lee Adams, Jon Ritterbush, & Christine E. Ryan

The discussion topic began as an innocent question on ERIL-L listserv about tools or techniques used in gathering feedback on database trials, whether from librarians or library users.

Trials can come from many request sources — subject librarians, faculty, students, and the electronic resources or acquisitions librarian. Adams evaluates the source and their access points. Also, they try to trial things they are really interested in early enough in the year to be able to request funding for the next year if they choose to purchase it. She says they don’t include faculty in the evaluation unless they think they can afford the product.

Criteria for evaluation: content, ease of use/functionality, cost, and whether or not a faculty member requested it. One challenge is how to track and keep an institutional memory of the outcome. They use an internal blog on WordPress to house the information (access, cost, description, and evaluation comments) with password protection on each entry. After the trial ends, the blog entry is returned to draft status so it’s not there, and a note is added with the final decision.

The final thing Adams does is create a spreadsheet that tracks every trial over a year, and it includes some renewals of existing subscriptions.

Ritterbush… lots of no-brainer stuff. Is it relevant to your collection development policy? Can you afford it? Who is requesting it? And so on.

Avoid scheduling more than three trials simultaneously to avoid “trial fatigue.” Ritterbush says they only publicize extended trials (>3 months) — the rest are kept internal or only shared with targeted faculty.

For feedback, they found that email is a mediocre solution, in part because the responses weren’t very helpful. The found that short web forms have worked better, incorporating a mix of Likert scale and free-text questions. The tool they use is Qualtrics, but most survey products would be fine.

Ritterbush tries to compose the trial information as a press release, making it easy for librarians and faculty to share with colleagues. A webinar or live demonstration of the product can increase interest and participation in the evaluation.

Ryan says you need to know why you are doing the trial, because that tells you who it will impact and then what approach you’ll need to take. Understand your audience in order to reach them.

Regardless of who is setting up the trials, it would be good to have a set of guidelines for trials that spells out responsibilities.

Kind of tuning out, since it seems like Ryan doesn’t really do anything directly with trials — just gives all that over to the subject liaisons. This would be disastrous at my library. Also, really not happy about her negative attitude towards public trials. If it’s IP-based, then who cares if you post it on your website? I’ve received invaluable feedback from users that would never see the trials if I followed Ryan’s method.

Questions:
What about trials to avoid expensive subscriptions? Some libraries will do it, but some have policies that prohibit it. [We have had sales agents recommend it to us, which I’ve never understood.]

How do you have trials for things when you don’t know if you have funding for them? Manage expectations and keep a healthy wishlist. [We will also use trials to justify funding increases or for replacing existing subscriptions with something new.]

NASIG 2009: ERMS Integration Strategies – Opportunity, Challenge, or Promise?

Speakers: Bob McQuillan (moderator), Karl Maria Fattig, Christine Stamison, and Rebecca Kemp

Many people have an ERM, some are implementing it, but few (in the room) are where they consider to be finished. ERMS present new opportunity and challenges with workflow and staffing, and the presenters intend to provide some insight for those in attendance.

At Fattig’s library, their budget for electronic is increasing as print is decreasing, and they are also running out of space for their physical collections. Their institution’s administration is not supportive of increasing space for materials, so they need to start thinking about how to stall or shrink their physical collection. In addition, they have had reductions in technical services staffing. Sound familiar?

At Kemp’s library, she notes that about 40% of her time is spent on access setup and troubleshooting, which is an indication of how much of their resources is allocated for electronic resources. Is it worth it? They know that many of their online resources are heavily used. Consorital “buying clubs” makes big deals possible, opening up access to more resources than they could afford on their own. Electronic is a good alternative to adding more volumes to already over-loaded shelves.

Stamison (SWETS) notes that they have seen a dramatic shift from print to electronic. At least two-thirds of the subscriptions they handle have an electronic component, and most libraries are going e-only when possible. Libraries tell them that they want their shelf space. Also, many libraries are going direct to publishers for the big deals, with agents getting involved only for EDI invoicing (cutting into the agent’s income). Agents are now investing in new technologies to assist libraries in managing e-collections, including implementing access.

Kemp’s library had a team of three to implement Innovative’s ERM. It took a change in workflow and incorporating additional tasks with existing positions, but everyone pulled through. Like libraries, Stamison notes that agents have had to change their workflow to handle electronic media, including extensive training. And, as libraries have more people working with all formats of serials, agents now have many different contacts within both libraries and publishers.

Fattig’s library also reorganized some positions. The systems librarian, acquisitions librarian, and serials & electronic resources coordinator all work with the ERMS, pulling from the Serials Solutions knowledge base. They have also contracted with someone in Oregon to manage their EZproxy database and WebBridge coverage load. Fattig notes that it takes a village to maintain an ERMS.

Agents with electronic gateway systems are working to become COUNTER compliant, and are heavily involved with developing SUSHI. Some are also providing services to gather those statistics for libraries.

Fattig comments that usage statistics are serials in themselves. At his library, they maintained a homegrown system for collecting usage statistics from 2000-07, then tried Serials Solutions Counter 360 for a year, but now are using an ERM/homegrown hybrid. They created their own script to clean up the files, because as we all know, COUNTER compliance means something different to each publisher. Fattig thinks that database searches are their most important statistics for evaluating platforms. They use their federated search statistics to weigh the statistics from those resources (will be broken out in COUNTER 3 compliance).

Kemp has not been able to import their use stats into ERM. One of their staff members goes in every month to download stats, and the rest come from ScholarlyStats. They are learning to make XML files out of their Excel files and hope to use the cost per use functionality in the future.

Fattig: “We haven’t gotten SUSHI to work in some of the places it’s supposed to.” Todd Carpenter from NISO notes that SUSHI compliance is a requirement of COUNTER 3.

For the next 12-18 months, Fattig expects that they will complete the creation of license and contact records, import all usage data, and implement SUSHI when they can. They will continue to work with their consorital tool, implement a discovery layer, and document everything. Plans to create a “cancellation ray gun and singalong blog” — a tool for taking criteria to generate suggested cancellation reports.

Like Fattig, Kemp plans to finish loading all of the data about license and contacts, also the coverage data. Looking forward to eliminating a legacy spreadsheet. Then, they hope to import COUNTER stats and run cost/use reports.

Agents are working with ONIX-PL to assist libraries in populating their ERMS with license terms. They are also working with CORE to assist libraries with populating acquisitions data. Stamison notes that agents are working to continue to be liaisons between publishers, libraries, and system vendors.

Dan Tonkery notes that he’s been listening to these conversations for years. No one is serving libraries very well. Libraries are working harder to get these things implemented, while also maintaining legacy systems and workarounds. “It’s too much work for something that should be simple.” Char Simser notes that we need to convince our administrations to move more staff into managing eresources as our budgets are shifting more towards them.

Another audience member notes that his main frustration is the lack of cooperation between vendors/products. We need a shared knowledge base like we have a shared repository for our catalog records. This gets tricky with different package holdings and license terms.

Audience question: When will the ERM become integrated into the ILS? Response: System vendors are listening, and the development cycle is dependent on customer input. Every library approaches their record keeping in different ways.

Kai Ryssdal is my best friend

Do you keep an eye on the currency markets? If you’re an acquisitions librarian, you should. In particular, pay attention to the dollar against the pound or the euro. I remember a professors in my graduate program emphasizing that point, but it wasn’t until recently that the practical implications sunk in. If you think 5% annual price increases are bad, factor in the current rate of exchange and be prepared for a shock.

Many major journal publishers, if not most, are headquartered in Europe. Pricing is therefore based on the euro or pound, which are both currently much stronger than the dollar. I’m not suggesting that you watch the market daily or scour each issue of The Wall Street Journal, but be aware of economic trends, and when the bill comes for your annual renewals, you’ll be ready with either additional funding already secured, or a list of titles to cancel.