Reference Services Evaluation

Reference Services Evaluation

Introduction: In order for libraries to remain relevant in a constantly changing technology age, they need to offer reference services online. These services are known as virtual reference services and take on the forms of instant messaging, email, telephone, text messaging, and videoconferencing. Chow & Croxton (2014) state, “usability metrics are a good predictor of user preferences centered on high return on investment, speed of transaction, convenience, and minimal effort” (p.309). Through the use of online reference services, patrons have a more accessible way to find information and resources, however; these online reference services should be easy to find, navigate, and use.

Reference Service: This evaluation will focus on the usability of online reference services within two academic universities. The online reference services being studied are instant messaging chat, email, telephone, text messaging, and Skype. The evaluation’s goal is to assess which online reference service is preferred by students when rating the ease of usability.

Participants:

• This evaluation will span a five month period during the academic year, so all students enrolled within the two academic universities will be able to participate. The group of students incorporated into the study are: freshmen, sophomores, juniors, seniors, graduate, and doctoral.
• Study participants are selected by random selection from undergraduate, graduate, and doctoral students. A total of 31 students were chosen for this study.
• The results of this study will represent which online reference services are used most often by students in regards to their usability. Chow & Croxton (2014) state, “If user preferences for particular reference services are an indicator of the actual usability of these services, one may expect the usability ratings to mirror that which is preferred” (p. 313). As a result, the study results may not reflect the entire population, but may give a good indication as to which online reference services college and university students choose to use in terms of usability.

Data Collection Methods:

• A weakness to this study is, students will do a usability survey on their own instead of a controlled testing environment. This study is reliant on the fact that students actually research all five online reference services and complete the questions accurately and honestly. Another weakness is the redundancy of questions within the online reference services where library staff begin to recognize the students participating in the study, due to the low participation size.
• The strength of this study allows researchers to see the correlation between usability of these online reference services and satisfaction rate of the students. Chow & Croxton (2014) state, “the higher the usability, the more highly rated the reference service was (p.319).  The usability evaluation is also quicker, cost effective, and practical to conduct.
• A survey will be distributed to all of the participants in the study consisting of ten questions. The participants will rate each of the online reference services for preference and usability. Two questions per service will be supplied, totaling ten, consisting of a quick fact question and research question. Quick fact questions are questions considered to be questions like “What time does the library close”? Research questions are considered as “How do I cite my paper in APA format”?
• In addition to a survey, records can be kept off all online transactions in the reference service such as name, date, and time. All identifying information will be removed before concluding the research.
• The instructions of the study along with the survey can be pilot tested with graduate students to consider the study’s validity, construction, and ways to revise for easy understanding. The graduate students who participate in the pilot study will not be considered for the actual study.

Environment:

• The locations collecting the information in this study will be two academic universities who provide four main online reference services.
• Since the schools are evaluating services provided online, the participants will be notified about the goals and instructions of the evaluation project through email, one of the online reference services provided.
• There are currently no other evaluation projects being conducted that could be consolidated with this reference service evaluation.

Personnel:

• Librarians in the two academic libraries will be trained to provide participants the purpose of the evaluation being conducted. They should have knowledge of all survey questions being asked, so they can clear up any confusion a participant may have.
• Librarians should also be trained in how to in-put information from the surveys into the software being used to analyze the evaluation.
• The librarians should participate in a workshop that provides the proper ethics and procedures when conducting an evaluation.
• Librarians should have this knowledge already, but should be reminded to keep online reference interviews private by removing any identifying information from the interaction.

Equipment and Supplies:

• The specialized equipment and supplies needed to collect, analyze, and communicate the project’s evaluation findings are as follows:

Survey services: Survey Monkey, Type Form, and Google Forms. These are free survey services, and there are others with a limited trial but are ultimately pay per survey.

Software for Managing and Analyzing Data: This could primarily be done through Microsoft Office Suite, but other software could be purchased if needed.

• Funding will be obtained by grants written by library staff to obtain all materials necessary to conduct this evaluation. Even though this evaluation should be relatively cost effective and cheap, funds should be secured for further studies and research.

References:

Chow, A. S. & Croxton, R. A. (2014). A usability evaluation of academic virtual reference services. College & Research Libraries, 75(3): 309-361