Skip to Main Content

The University of Tennessee

School of Information Sciences

Frequently Used Tools:

INSC 554

Main Navigation:

Developing an Evaluation Action Plan


The following work will propose user-centered feedback regarding the Tennessee Collection for the Art Circle Public Library in Crossville, Tennessee. This feedback will determine if the users of the library find value in the Tennessee Collection that would justify its continued existence as a separate collection rather than reintegrating the materials into the broader collections of the library.

5.1 Develop a rationale to propose user-centered assessment/evaluation of the selected PL service. Assess the selected PL service carefully and explain your proposal in relation to participation of primary, secondary, and tertiary users identified in the earlier assignment.

Due to the current low usage statistics regarding the Tennessee Collection of the Art Circle Public Library, the staff feels that the space could be put to better use by expanding the non-fiction collection and reintegrating the materials currently in the Tennessee Collection (M. Brown, personal communication, March 27, 2015). However, as no user-centered feedback regarding this service has been obtained since the library moved to its current facility, it would perhaps be in the best interests of the library to conduct a survey regarding patron feelings and thoughts about the collection before dismantling it.

The primary patrons that would be asked to participate, and those that would be most affected, are adults aged 45 and over. As discussed in section 2.1, these patrons are the primary user group of the library and they are the ones most likely to be using both the Tennessee collection that would be dismantled and the non-fiction section that could expand into those book ranges. The patrons would be asked to voluntarily and anonymously answer a brief survey regarding the Tennessee collection. This survey would be available at the Adult Services desk and left on the tables that are immediately in front of the Tennessee collection, with instructions to turn them in at the Adult Services desk.

The secondary user group of the library, children from birth to age 10, would not be asked to participate in the survey as the Tennessee Collection contains almost no material that this group would be likely to use; they would be unaffected by any change to the Tennessee collection. Further, the very young would be unable to answer a survey at all. The tertiary user group, the poor of Cumberland County, would also be asked to fill out the survey if they were old enough to participate.

The proposed survey will be brief and printed for the patrons, so as to get immediate feedback and to avoid having to direct people to websites on their own computers or have intrusive pop-ups on the public Internet computers.

5.2 Identify the problem (related to your selected service) in terms of: production bottlenecks; tasks that are performed frequently; activities that require frequent movement; and declining budgets.

Reintegrating the Tennessee Collection into the main collections of the library would not require an ongoing budget. It would require some time on the part of the processing staff to adjust the holdings information to reflect the new location of the materials and time on the part of the Adult Services staff to make room for and then re-shelve the materials. If the Director or Board wanted this project completed in a tight time frame, it would require either putting off other tasks for the processing staff, increasing their hours temporarily, or bringing in temporary help for a few days. Similarly, to complete the task quickly, volunteers or temporary staff would need to be brought in to shift and re-shelve the materials in the regular collections (M. Brown, personal communication, March 27, 2015).

If, however, this project is not on a tight time frame, the materials can be moved to the processing room and their catalog listings adjusted as time permitted, and then reshelved as time permitted with no additional staff hours to budget at all (M. Brown, personal communication, March 27, 2015). Regardless of the timetable, this project would be a one-time effort, with no ongoing movement or tasks.

5.3 Determine the scope of analysis in terms of what specific information/aspects about the selected PL service do you hope to gain by eliciting feedback from the PL users. In other words, what is the purpose for doing the evaluation? Is the library attempting to improve its operational efficiencies (internal focus), or is the study being done to better understand the effectiveness of a library service? Explain in detail.

The purpose of eliciting feedback from the patrons is to see if they find value in having the Tennessee Collection about which that the staff is not currently aware. In the event that the patrons find considerable value in the Tennessee collection, this project would be abandoned and the Tennessee Collection left in place and with potential collection development to enhance it and potential advertising to promote usage. In the event that patrons would find more value in an expanded non-fiction collection, the Tennessee Collection would be reintegrated into the main collections. In either scenario, no materials would be discarded.

5.4 Determine the kinds of data that you hope to gather (be specific). What evaluation methodology and design will be used (quantitative and/or qualitative methods: be very specific in the types of tools that will be employed)? How will data be collected?

The survey will be set up as a set of multiple-choice questions for quantitative data such as questions about the Tennessee collection, the patron's use of it, and demographics. After these questions, the survey will have optional space for qualitative data should the patron wish to voice any opinion regarding the collection not included in the questions. The survey will be developed by the author, then printed and distributed as detailed in section 5.1.

Once the time period for the survey is completed, one week due to time constraints, the sheets will be collected and the quantitative data collated and analyzed, with the qualitative data examined for any information that might have been overlooked and the general emotional reaction of the patrons to the Tennessee Collection.

5.5 Describe the user-centered assessment/evaluation strategies that you are proposing to conduct community analysis and gather feedback from users about the selected service at your PL. Identify and analyze what methods you will use to gather feedback from various kinds of users as a part of your efforts. How do you propose getting access/permissions to select users? Provide a timeline for implementing your proposed methods.

As detailed in sections 5.4 and 5.1, the survey for this project will be developed by the author, then printed and distributed to the patrons of the ACPL who are willing to give feedback on the Tennessee Collection. Participants in the survey will choose their answers and potentially offer additional comments, then turn the forms over to the Adult Services desk. Over the course of one week, the Adult Services desk will accumulate the forms, after which time they will be collected and analyzed for any useful data regarding the Tennessee Collection and potentially reintegrating it into the main ACPL collections.

The selection of users to survey will be random, based on those that are willing to complete it that come into the library and approach the desk or table over the course of the one-week testing period. Permission to distribute the survey would be acquired from the director, Susie Randleman, and the Adult Services Librarian, Margo Brown. Survey distribution could begin as soon as permission is obtained and the survey approved, and the results prepared a few days after the end of the testing time frame.

5.6 Share the data collecting instrument that you propose to use. Identify the qualitative and/or quantitative measures you propose to use (what aspects are you trying to measure). Provide strategies to gather consent (consent forms, assent forms as appropriate) and your data-collecting tool(s) with listed questions/prompts, etc. You can explore multiple research methods to gather user-centered assessment/evaluation of the selected service.

As shown in the Appendix, questions 1-5 are quantitative, asking about the participant's knowledge of and experiences with the Tennessee Collection with some basic demographic information in question 5. Question 3 includes optional further information of a qualitative nature, and question 6 is entirely qualitative. This is a simple project, and the survey is reflective of this simplicity.

5.7 Provide strategies to maintain anonymity and confidentiality of users participating in your study (if appropriate).

As the survey will contain only the barest demographic information and no names, the person analyzing the data will have no way of identifying any individual who participated. Any individual who is concerned with anonymity beyond that would either not answer any qualitative question or simply not take the survey. As it will be done via paper, there will be no digital data trail. The anonymity of the patrons taking the survey will be protected.

5.8 Create a budget in terms of investment of resources (e.g., money, time, efforts, human input, etc.) to orchestrate user-centered assessment/evaluation of various kinds. Identify stages for orchestrating a plan for the PL (e.g., previous efforts made, current efforts, future efforts) and describe what will be feasible as part of your efforts in this course during this semester.

Due to the nature of this particular project, the investment of monetary resources to conduct a user-assessment of the Tennessee Collection will be minimal, as the survey will be a single page per patron willing to participate. The production of the survey will take some time and effort, as would collecting and collating the data, but the collection is something that staff can do around their other tasks: the ACPL would not need to hire someone specifically to come in and collect the data. The production of the survey and analysis of the data will be done by the author, and will require no resources on the part of the ACPL.

As there have been no efforts made to have a user-centered evaluation of this collection since the library moved to the current facility, this would be considered the first and potentially the last if the user-data reflects the current usage statistics (M. Brown, personal communication, March 27, 2015). There are no prior efforts, this would be considered the current effort, and any future efforts would either be non-existent or would need to be determined after the initial survey, should results come back that indicate the collection should be left intact.

5.9 Determine the kinds of analysis you will do with the data collected. How do you plan to report the findings based on the data collected?
NOTE: The research protocols will not actually be submitted to the Office of Research since the assignments are for class activities and participant feedback will not be individually reported. Yet the process for completing the research protocols will create awareness of expectations and requirements that need to be completed for conducting research involving human subjects.

The data will be analyzed for trends in the multiple-choice answers and for any useful information in the open-ended feedback. These findings will be collated into a report and offered to the Director and potentially the Board of Trustees. They will be published online via the University of Tennessee web module for IS 554 Spring 2015, but no other public disclosure will be necessary.


Spring 2015

Contact K.C. Williams

451 Communications Building, 1345 Circle Park Drive, Knoxville, TN 37996-0341