Tag: data

NASIG 2009: Managing Electronic Resource Statistics

Presenter: Nancy Beals

We have the tools and the data, now we need to use them to the best advantage. Statistics, along with other data, can create a picture of how our online resources are being used.

Traditionally, we have gathered stats by counting when re-shelving, ILL, gate counts, circulation, etc. Do these things really tell us anything? Stats from eresources can tell us much more, in conjunction with information about the paths we create to them.

Even with standards, we can run into issues with collecting data. Data can be “unclean” or incorrectly reported (or late). And, not all publishers are using the standards (i.e. COUNTER).

After looking at existing performance indicators, applying them to electronic resources, then we can look at trends with our electronic resources. This can help us with determining the return on investment in these resources.

Keep a master list of stats in order to plan out how and when to gather them. Keep the data in a shared location. Be prepared to supply data in a timely fashion for collection development decision-making.

When you are comparing resources, it’s up to individual institutions to determine what is considered low or high use. Look at how the resources stack up within the over-all collection.

When assessing the value of a resource, Beals and her colleagues are looking at 2-3 years of use data, 10% cost inflation, and the cost of ILL. In addition, they make use of overlap analysis tools to determine where they have multiple formats or sources that could be eliminated based on which platforms are being used.

Providing readily accessible data in a user-friendly format empowers selectors to do analysis and make decisions.

gathering statistics

For the past couple of weeks, the majority of my work day has been spent on tracking down and massaging usage statistics reports from the publishers of the online products we purchase. I am nearly half-way through the list, and I have a few observations based on this experience:

1. There are more publishers not following the COUNTER code of practice than those who are. Publishers in traditionally library-dominated (and in particular, academic library-dominated) markets are more likely to provide COUNTER-compliant statistics, but that is not a guarantee.

2. Some publishers provide usage statistics, and even COUNTER-compliant usage statistics, but only for the past twelve months or some other short period of time. This would be acceptable only if a library had been saving the reports locally. Otherwise, a twelve month period is not long enough to use the data to make informed decisions.

3. We are not trying to use these statistics to find out which resources to cancel. On the contrary, if I can find data that shows an increase in use over time, then my boss can use it to justify our annual budget request and maybe even ask for more money.

Update: It seems that the conversation regarding my observations is happening over on FriendFeed. Please feel free to join in there or leave your thoughts here.

usage statistics

The following is an email conversation between myself and the representative of a society publisher who is hosting their journals on their own website. Can I access the useage information for my institution? We subscribe to both the print and online [Journal Name]. Anna Creech Dear Ms. Creech, At the most recent meeting of the … Continue reading “usage statistics”

The following is an email conversation between myself and the representative of a society publisher who is hosting their journals on their own website.

Can I access the useage information for my institution? We subscribe to both the print and online [Journal Name].

Anna Creech

Dear Ms. Creech,

At the most recent meeting of the [Society] Board of Directors, the topic of usage statistics was discussed at length. As I am sure you are aware, usage statistics are a very coarse measure of the use of a web resource. As just one example, there is no particular relationship between the number of downloads of an article and the number of times it is read or the number of times it is cited. An article download could represent anything from glancing at the abstract, to careful reading. Once downloaded, articles can be saved locally, re-read and redistributed to others. Given the lack of any evidence that downloads of professional articles have any relationship to their effective audience size or their value to readers, the Board decided that [Society] will not provide potentially misleading usage statistics. We do periodically publish the overall usage of the [Society] website, about 10 million hits per year.

Regards,

[Name Removed]
[Society] Web Editor

Dear Mr. [Name Removed],

Your Board of Directors are certainly a group of mavericks in this case. Whether they think the data is valuable or not, libraries around the world use it to aid in collection development decisions. Without usage data, we have no idea if an online resource is being used by our faculty and students, which makes it an easy target for cancellation in budget crunch times. I suggest they re-think this decision, for their own sakes.

We all know that use statistics do not fully represent the way an online journal is used by researchers, but that does not mean they are without value. No librarian would ever make decisions base on usage data alone, but it does contribute valuable information to the collection development process.

Hits on a website mean even less than article downloads. Our library website gets millions of hits just from being the home page for all of the browsers in the building. I would never use website hits to make any sort of a decision about an online resource.

Provide the statistics using the COUNTER standard and let the professionals (i.e. librarians) decide if they are misleading.

Anna Creech

UPDATE: The conversation continues….

Dear Ms. Creech,

Curiously, the providers of usage statistics are primarily commercial publishing houses. Few science societies that publish research journals are providing download statistics. In part, this is a matter of resources that the publisher can dedicate to providing statistics-on-demand: commercial publishing houses have the advantage of an economy of scale. They are also happy to provide COUNTER-compliant statistics in part because they are relatively immune to journal cancellation, as a result of mandatory journal bundling.

In any event, after careful consideration and lengthy discussion with a librarian-consultant, the Board concluded that usage statistics are easy to acquire and tempting to use, but are in effect “bad data”. I certainly respect your desire to make the most of a tight library budget, but also respectfully disagree that download statistics are an appropriate tool to make critical judgements about journals. Other methods to learn about the use of a particular journal are available- for example, asking faculty and students to rate the importance of journals to their work, or using impact factors. I am sure you take these into account as well.

I will copy this reply to the [Society] Board so that they are aware of your response. No doubt the Board will revisit the topic of usage statistics in future meetings.

Regards,

[Name Removed]

Dear Mr. [Name Removed],

I never ment to imply that we exclusively use statistics for collection development decisions. We also talk with faculty and students about their needs. However, the numbers are often a good place to begin the discussions. As in, “I see that no one has downloaded any articles from this journal in the past year. Are you still finding it relevant to your research?” Even prior to online subscriptions, librarians have looked at re-shelve counts and the layer of dust on the tops of materials as indicators that a conversation is warranted.

I suggest your Board take a look at the American Chemical Society. They provide COUNTER statistics and are doing quite well despite the “bad data.”

Anna Creech

tagging

So, I’m finally hopping on the blog tag bandwagon. I thought my categories were enough, and I didn’t know how to make the keyword field show in the entry creation process. But now that I have a brand new plugin, I’ve started adding keywords to my posts with Technorati links. I tagged the last however … Continue reading “tagging”

So, I’m finally hopping on the blog tag bandwagon. I thought my categories were enough, and I didn’t know how to make the keyword field show in the entry creation process. But now that I have a brand new plugin, I’ve started adding keywords to my posts with Technorati links. I tagged the last however many entries just now and I will tag future posts, but at 457 entries, I don’t plan to do any retrospective tagging. Heck, I think some of my earlier entries aren’t categorized, either. Probably for the best. There are some things I’d like to forget.

Oh! I had a brainstorm yesterday evening for an article topic, so maybe I’ll get cracking on that soon. After all, I just have to have stuff submitted. If it gets published, well, so much the better.

star trek is dead

Orson Scott Card writes, “So they’ve gone and killed “Star Trek.” And it’s about time.” As long as it’s the twisted wreck of Star Trek that Rick Berman is creating, I agree whole-heartedly with him. However, I am still mourning the end of Star Trek: the Next Generation — the only Gene Roddenberry version that … Continue reading “star trek is dead”

Orson Scott Card writes, “So they’ve gone and killed “Star Trek.” And it’s about time.” As long as it’s the twisted wreck of Star Trek that Rick Berman is creating, I agree whole-heartedly with him. However, I am still mourning the end of Star Trek: the Next Generation — the only Gene Roddenberry version that was allowed to grow and develop over time.

As Card points out in his commentary, TOS was hampered by television convention of its time, and the characters were never able to truly develop in the show, although I would argue that the published book series provided ample opportunity for that to happen. I will also concede that TOS was not necessarily good science fiction, particularly compared to what was being written at the time. However, Roddenberry never intended to be a science fiction writer. His experience was with the Western genre. As it has been quoted many times, Star Trek was meant to be a “Wagon Train to the stars.”

Twenty-odd years later, Star Trek fans were given a new generation of characters and plot lines. I grew up watching re-runs of TOS, but I wasn’t a fan until I saw TNG. Back in 1988, I was a geeky junior high kid with few friends and too much time for watching TV, but I found solace in this vision of the future presented to me by Roddenberry and the script writers for TNG. It was my interest in this television show that introduced me to the science fiction genre. I would not have read any of Clark’s books had I not first come to love Picard, Data, Yar, and all of the rest of the ST:TNG characters.

So I say to you, Mr. Clark, do not look down your nose and scoff at the demise of the Star Trek universe. Be grateful to it for making science fiction accessible to the general public, and for paving the path to those series that you deem to be good science fiction.

Thanks to Bookslut for the heads-up.

css.php