Statement of Competency 14

Competency 14: Evaluate programs and services using measurable criteria

Introduction

Library and information programs and services are systematically assessed and evaluated to determine their success in achieving their goals, continued usefulness, and proper functioning. Research-derived or observable evidence, combined with professional knowledge and experience, help professionals deliver content and services that address the needs of their patrons (Todd, 2008, p. 18). Assessments of library activities and services are conducted according to measurable criteria, and results guide professional action. Decisions and planning must be based on clear evidence about what works, what does not, and what requires improvement. Professionals may choose from any number of established sets of criteria, depending on the activity or service to be assessed.

The Reference and User Services Association (RUSA) has prepared a set of Guidelines for Behavioral Performance of Reference and Information Service Providers (2013). This document is meant to “identify and recommend observable behavioral attributes that could be correlated with positive patron perceptions of reference librarian performance” (para. 1). The interaction between the librarian and patron during the reference process can be observed and measured according to visibility/approachability, interest, listening/inquiring, searching, and follow-up. An interpretation of the librarian’s performance is evaluated and recommendations for improvement are made. This process leads to improved information services.

The American Association of School Librarians (AASL) has created Standards for the 21st-Century Learner (2007, section 2), which “shape the library program and serve as a tool for school librarians to use to shape learning of students.” These standards focus on the skills to inquire and think critically; draw conclusions based on knowledge and create new knowledge; produce and share knowledge ethically; and pursue personal growth. School library programs and services can be evaluated against the standards detailed by the AASL. Clear evidence, such as lesson plans, student assessments, reports, images, videos, and teacher assignments, can specify areas that are being addressed effectively, and areas that require improvement. A systematic assessment plan ensures continued improvement of school library services and programs.

User experience refers to designing “a product or service to optimize how people will interact with it” (Schmidt, 2010, para. 4). User experience of library products and services can be measured according to design concepts developed by Norman (1988). Properties that affect design include affordance, constraint, mapping and natural mapping, conceptual and mental models, and the paradox of technology. These concepts provide a framework with which to analyze how patrons interact with various aspects of library systems. Organizations that use measurable criteria to evaluate library systems are able to make specific recommendations for improvements, thus providing a more satisfying experience for users.

Evidences

The following evidence is submitted to satisfy Competency 14:

Analyses of various reference transactions, a compilation of assignments from INFO 210 Reference and Information Services
Assessment Review, an assignment from INFO 233 School Media Centers
Evaluation of RefWorks bibliography function, a group project from INFO 202 Information Retrieval Systems

INFO 210 Analyses of various reference transactions

This paper is a compilation of four assignments from INFO 210, in which I analyzed various types of reference transactions according to guidelines established by RUSA. The purpose of the assignments was to evaluate a library’s reference services using measurable criteria, including visibility/approachability, interest, listening/inquiring, searching, and follow-up. The first reference interview is a fictional dialog between a reference librarian and a patron that addresses an ethical issue. The second transaction analyzed is a face-to-face reference transaction between myself and a local public librarian. Next, a synchronous phone reference interview is described and evaluated. Finally, I assess and compare virtual reference transactions with two different public libraries.

This set of assignments demonstrates my ability to evaluate library services by measurable criteria. Comparing what should happen with what actually took place is a good indication of where services succeed or fall short. Areas of weakness can be brought to the attention of library staff, along with specific recommendations for improvement. I feel that my assessments are thorough and useful, and could be used as a model on which to base future evaluations. This paper demonstrates my mastery of Competency 14.

INFO 233 Assessment Review

The Assessment Review, an assignment from INFO 233, outlines an assessment plan for a school library. The purpose of the assignment was to understand how to develop an assessment plan that forms an integral element of a comprehensive vision for a school library. My assessment strategy is framed by the AASL’s Standards for the 21s-Century Learner (2007). Multiple techniques are specified for answering questions about how the librarian and the library services and programs address the Standards. Various types of evidence to be collected and analyzed for assessment are detailed. A timeline and guidelines for presentation of the data provide a solid foundation for annual assessments.

My assessment review clearly demonstrates that I understand the importance of using measurable criteria to form compelling justification for improvements and modifications in school library programs and services. Linking evidence to standards is essential in creating an optimal environment for library patrons. Since the completion of this assignment, I have incorporated my plan into my professional situation. Continued systematic assessment now forms the basis for my recommendations to administration. This assignment shows that I have mastered the skills required by Competency 14.

INFO 202 Evaluation of RefWorks bibliography function

My final piece of evidence is a group project for INFO 202, the purpose of which was to evaluate an information retrieval system based on measurable criteria. Design concepts developed by Norman (1988) were used as standards. My part of this project was to 1.) define conceptual and mental models and the paradox of technology, 2.) explain why these concepts are helpful in studying information seeking, and 3.) evaluate four features of the RefWorks “create a bibliography” function in terms of conceptual/mental models and the paradox of technology.

This assignment demonstrates my ability to use a set of measurable criteria to evaluate a library system. Aspects of a library system are examined in terms of defined standards to determine the success of its design. Insights gained from these assessments lead to specific recommendations for improvements, which will in turn provide a more satisfying user experience. This paper is evidence of my proficiency in Competency 14.

Conclusion

Library professionals may choose a set of standards that suits their evaluation needs. Measurable criteria produce sound evidence on which to base decisions and plans for improvements. Examples from my course work in the SJSU MLIS program demonstrate that I understand the concept and importance of measurable criteria in terms of professional activities and services. I have shown how evaluation can improve the design or provision of information services and programs. I feel that the knowledge I have acquired in my degree has provided me with a foundation on which to base evaluations of library systems in a professional environment. I am currently evaluating our school library management system by Norman’s design concepts. My evidence proves that I have mastered Competency 14.

References

American Association of School Librarians (2007). Standards for the 21st-Century Learner. Retrieved from http://www.ala.org/aasl/standards

Norman, D. A. (1988). The psychopathology of everyday things. In The Design of Everyday Things (Chapter 1). New York, NY: Doubleday.

Reference and User Services Association (RUSA) (2013). Guidelines for behavioral performance of reference and information service to providers [Web page]. Retrieved from http://www.ala.org/rusa/resources/guidelines/guidelinesbehavioral

Schmidt, A. (2010). New column launch: The user experience [Web log post]. Library Journal. Retrieved from http://lj.libraryjournal.com/2010/01/opinion/aaron-schmidt/new-column-launch-the-user-experience/ – _

Todd, R. J. (2008). A question of evidence. Knowledge Quest 37(2), 16-21. Retrieved from http://web.b.ebscohost.com.libaccess.sjlibrary.org/ehost/pdfviewer/pdfviewer?vid=3&sid =911d808f-0a4f-4706-a6c8-572e67d0eae2%40sessionmgr104