Skip to Main Content

The University of Tennessee

School of Information Sciences

Frequently Used Tools:



INSC 554


Main Navigation:

Developing and Evaluation Action Plan

Abstract

Throughout the spring semester, the author will be doing a case study analysis of the Kingsport Public Library and focusing on their Time for Two’s storytime for two-year-olds. The results will be published via web in seven parts: the public library context, identification of the public library users, service to assess/evaluate, case study analysis, developing an evaluation action plan, collecting and analyzing data, and user centered assessment/evaluation in public library services. For the fifth assignment, the author designed an evaluation action plan, which includes a voluntary survey for the users to asses their satisfaction level with Time for Two’s and to gain their input on aspect to improve to be shared with library staff.

5.1 Develop a rationale to propose user-centered assessment/evaluation of the selected PL service. Assess the selected PL service carefully and explain your proposal in relation to participation of primary, secondary, and tertiary users identified in the earlier assignment.

The public library service to assess is the Time for Two’s storytime. The program is offered weekly on Thursday mornings from 10:30-11:00 AM. The program has previously had no formal evaluation. One staff member, Kyndra Jones, does the storytime each week. Resources beyond her time being spent planning and performing storytime include materials for crafts (typically construction paper, markers, and glue sticks) and money for snacks and punch for the storytime parties, which happen about 4-5 times a year.

The audience for storytime is relatively small as it is designed for children 24-36 months. Occasionally, a child young or older will attend the program, but the audience includes the children and their caregiver(s). Of the primary (Internet Users), secondary (Asking Reference Questions), and tertiary (Program Attendees), the audience for storytime falls into the tertiary group. Within that tertiary group, the Time for Two’s attendees are a small number as the focus is only one program.

5.2 Identify the problem (related to your selected service) in terms of: production bottlenecks; tasks that are performed frequently; activities that require frequent movement; and declining budgets.

Time for Two’s is performed frequently, one a week, almost every month of the year. Time for Two’s takes a break around Christmas and for a week or two in May before the Summer Reading Program starts. As such, and especially since no evaluation has been performed before, Time for Two’s is a prime candidate for evaluation. Though it is performed frequently, the user base changes about every year to year and a half as the children age out of Time for Two’s and attend Preschool Storytime. A yearly voluntary survey would be ideal as it would get the opinion of those who have been coming for weeks and those who only recently started. Currently, there is one staff member who performs both storytimes and one adult volunteer at each storytime. This takes this one staff member and each volunteer away from performing other tasks for the library.

Time for Two’s requires frequent movement. Due to a lack of space in the Youth Services office/desk space, much of the storytime materials are split between several different locations. Many of the puppets are in the auditorium behind a partition. The puppet stage is also here. The storytime bags are now all in the same location on the third floor in the mezzanine office, which is a shared space with the Friends of the Library to price and sort through donated books for the annual book sale. It is only within the past few months that the storytime materials were gathered in one space. Before being in the mezz office, storytime bags were also in the magazine closet on the second floor and some supplies were in a different closet on the second floor next to the staff kitchen. Even with the storytime bags all in one spot and all the puppets now in one spot in the auditorium, there are crafts, craft supplies, books to display for checkout, and the materials in the storytime bags that must be transported to and from the auditorium each week. Ideally, all the storytime supplies would be stored in the Youth Services department, but the department is severely lacking in staff space and this is simply not a reality for the moment. The library hopes to have a new building seven or more years down the line. Currently, these frequent movements, even over the short distance of walking up about four stairs and a few feet into the auditorium, take time away from other tasks.

The Kingsport Public Library is not facing a steeply declining budget, but if storytime was offered more than once a week and/or a new storytime was added, the program budget would need to be increased, which may not be possible. Storytime is a relatively low budget program as the book used are books the library already owns and many of the craft supplies are what Youth Services buys yearly for decorations for the department, summer reading, and other programs throughout the year. Volunteers donate some of the supplies, like construction paper and glue sticks. While the budget isn’t necessarily declining, it isn’t increasing either.

The user-centered evaluation will determine what currently works for the attendees and what they wish to see in the future. The evaluation will help to determine what things needs to be kept, what need to be changed, and if increasing storytime is an effort worth looking into.

5.3 Determine the scope of analysis in terms of what specific information/aspects about the selected PL service do you hope to gain by eliciting feedback from the PL users. In other words, what is the purpose for doing the evaluation? Is the library attempting to improve its operational efficiencies (internal focus), or is the study being done to better understand the effectiveness of a library service? Explain in detail.

A user-centered feedback evaluation has never been performed of Time for Two’s storytime. Staff have questions they would like answered in regards to what patrons want to see more of, especially if offering storytime more than once a week is a welcome idea. The purpose of the survey is to determine the effectiveness of storytime along with seeking ways to improve the service. The survey is essentially a satisfaction survey where users will be asked “to evaluate the effectiveness of the service(s) provided” (Matthews, 2007, p. 88).

The evaluation will assess users’ satisfaction on a scale of 1-10, with 1 being completely unsatisfied and 10 being completely satisfied. The library also wants to know if offering Time for Two’s more than once a week is something they want. It is noted that if Time for Two’s was offered more than once a week, the content would be the same at each program that week. Users will also be asked if they would like take-home sheets, like the lyrics to the songs or the rhymes performed.

Users will be asked if there is a different day in the week that would be more convenient for them to attend storytime. Though attendance is well, there is always the possibility that patrons would be able to come on a different day and are afraid to say so. Library staff also want to know if there is anything patrons want to see more of in storytime, like crafts, songs, flannel boards, puppets, and interactive play. Users can select as many of these options as they like. Additionally, staff want to gauge the interest in a storytime for children younger than two.

The survey will be used for internal focus to determine what patrons enjoy about storytime and what they would like to see improved. Each question will have space for comments to be left and a comment box at the end for any comments users wish to share that were not already covered. The survey will allow “the library to learn what matters to customers and apply that information to improve service delivery” (Matthews, 2007, p. 88).

5.4 Determine the kinds of data that you hope to gather (be specific). What evaluation methodology and design will be used (quantitative and/or qualitative methods: be very specific in the types of tools that will be employed)? How will data be collected?

The goal of the user-centered feedback of the Time for Two’s storytime is to gauge patrons’ satisfaction with the way the program currently is, aspects they want to improve, and if there is any interest in a program for children younger than two. The survey will be created and distributed using an online tool called SurveyMonkey. The first question is actually a comment letting the users know the survey is being done by a student and is anonymous and voluntary. The next six questions focus on storytime. The last two focus on demographics and the last one is open ended for any comments the respondents wish to share. All of the questions will be quantitative with multiple-choice answers with the exception of the last question, which is qualitative, and allows for users simply to provide feedback. The questions will be a combination of past/present/future behavior, factual, and opinion based. Data will be collected via the returned print survey and the online tools available through SurveyMonkey.

5.5 Describe the user-centered assessment/evaluation strategies that you are proposing to conduct community analysis and gather feedback from users about the selected service at your PL. Identify and analyze what methods you will use to gather feedback from various kinds of users as a part of your efforts. How do you propose getting access/permissions to select users? Provide a timeline for implementing your proposed methods.

Getting access to users will not be a problem. The survey will be explained at Time for Two’s storytime and the author will be available that day to give a more in-depth explanation if necessary. Small flyers will be printed with an explanation of the survey, a written link to the website for the survey, and a QR code for easier access for those with smartphones. A link will be posted on the library’s Facebook page, giving further access. All of the responses will be anonymous. Users will know that the survey is being done by the author for a school project. The surveys will be completely voluntary, as each question will be also. If a user does not wish to answer a question on the survey, the user does not have to.

A timeline for the survey is:

April 2: Finalize first draft of questions; disburse to friends and co-workers; tweak questions; send to director for approval

April 6: Print final copy of survey and place in Youth Services; post a link on the library’s Facebook page

April 9: Promote survey and hand out flyers at Time or Two’s storytime

April 14: All surveys received and analysis completed

5.6 Share the data collecting instrument that you propose to use. Identify the qualitative and/or quantitative measures you propose to use (what aspects are you trying to measure). Provide strategies to gather consent (consent forms, assent forms as appropriate) and your data-collecting tool(s) with listed questions/prompts, etc. You can explore multiple research methods to gather user-centered assessment/evaluation of the selected service.

The goal of the user-centered survey for Time for Two’s storytime is to gauge patrons’ satisfaction with the way the program currently is, aspects they want to improve, and if there is any interest in a program for children younger than two. This will be accomplished through a brief survey of nine questions. The questions will be opinion based and include past/present behavior and what they hope to see in the future from the storytime. There are two demographic questions asking about the user’s relation to the child they bring to storytime and to please indicate their age and gender. The opinion questions ask users to rate their current satisfaction with storytime on a scale of 1-10, where 1 is not satisfied at all and 10 is completely satisfied. The rest of the questions ask if they would like take-home sheets, if storytime should be offered more than once a week, what they would like to see more of in storytime, and if there is any interest in a storytime for children younger than two. Each question also has a comment box if the users would like to more detailed feedback.

Most of these questions will result in numerical data, such as the percentage of users who wish to see or improve each thing. However, the open-ended question at the end is qualitative and with each question having a comment box, there is a possibility for a large amount of qualitative data also.

At the beginning of the survey, users will be notified that the survey is being done by a student for the purposes of a class assignment, but the information will be provided to the Kingsport Public Library. Users will be made aware that the survey, and each question in it, is completely voluntary and anonymous. The demographics only ask for age and gender, so it will be impossible to identify the respondent. By participating, users are giving the author and the library permission to use their opinions and responses for evaluative purposes for library staff and for the author’s assignment.

The survey to evaluate Time for Two’s storytime is as follows:

Time for Two’s Survey

1)This survey is being conducted by a student at the University of Tennessee as part of an assignment. However, the information gathered will be provided to the Kingsport Public Library. The survey, and each question in it, is voluntary and anonymous. However, the results will be shared with the Kingsport Public Library to evaluate Time for Two’s storytime.

2)Please rate your satisfaction of the Time for Two’s storytime on a scale of 1-10 where 1 is not satisfied at all and 10 is completely satisfied.

1

2

3

4

5

6

7

8

9

10

3) Would you like for Time for Two’s to be offered more than once a week? Please note that if Time for Two’s was offered more than once a week, the program at both storytimes that week would be the same.

Yes

No

4) Would you like to have take-home sheets of the lyrics to the songs performed in storytime or any of the rhymes?

Yes

No

5) Which would you like to see more of in Storytime? Please check all that apply.

Crafts

Songs

Puppets

Flannel Boards

Interactive Play

Other (please specify)

6) Is there a different day in the week that would be more convenient for you for storytime?

Yes (If yes, please specify which day and time in the field below)

No

7) Would you (or anyone you know) be interested in a storytime for children younger than two?

Yes

No

8) If you already attend storytime, please indicate your relation to the child you bring.

Parent

Grandparent

Great grandparent

Aunt

Uncle

Cousin

Nanny/babysitter

Sibling

Other (please specify)

9) Please indicate your age and gender. Please check all that apply.

Female

Male

Under 20

21-25

26-30

31-35

36-40

41-50

51-60

60+

Prefer not to answer

10)Feel free to leave any comments on any aspect of storytime in the box below. Thank you for your time!

5.7 Provide strategies to maintain anonymity and confidentiality of users participating in your study (if appropriate).

The survey is completely voluntary. Users do not have to take the survey if they do not wish to and they do not have to answer every question if they do not wish to. The survey will be completed online or via the printed copies in Youth Services. The demographic questions are only age and gender and are voluntary. If completing online, respondents will take the survey on whatever Internet enabled device they choose. Individual results will not be shared. Only the overall results will be shared with staff and discussed in the paper.

5.8 Create a budget in terms of investment of resources (e.g., money, time, efforts, human input, etc.) to orchestrate user-centered assessment/evaluation of various kinds. Identify stages for orchestrating a plan for the PL (e.g., previous efforts made, current efforts, future efforts) and describe what will be feasible as part of your efforts in this course during this semester.

The primary resource for the survey is time. The author must devote her time to creating the survey and disbursing it. Her friends and co-workers must give their time to look over the questions before submitting to the director, who then must use her time to read the survey and approve it for distribution. The patrons must also give their time to read and take the survey. Further time must be devoted by the author to analyze the results. This time is voluntary on both the author’s and the users’ time and should not be included in the cost of the assessment. The results will need to be shared with Kyndra Jones, the storytime leader, Casey Applebaum, Youth Services Librarian, and Helen Whittaker, Library Director, which again will require their time and energy. An average of these individual’s hourly pay would be needed times one hour to determine the cost to the library.

The printed flyers will be printed at the Kingsport Public Library with the author paying for the printing. The flyers will be four to a page and there will be 15 pages printed for a total of 60 surveys. Printing is .15 per page for black and white. The author already has to drive to the library for work, therefore her gas does not need to be included as this money would be spent anyway. However, her time spent speaking to the individuals will need to be included, as it will be done on work time. The cost of the evaluation can be calculated three ways: the total cost to the author, the total cost to the library, and the total cost of the library.

xy=Actual Cost to the Library where x=average of hourly pay and y=time spent

15(.15)=$2.25 Actual Cost to the Author for printing

xy+$2.25=Total Cost of the Evaluation

The stages of implementation for a survey are as follows: identify a service for evaluation, identify what aspect(s) of the service need to be evaluated, identify the goal of the evaluation, choose an evaluation method, determine how long the survey will be available, get approval from the library director to implement the survey, analyze survey results, share/publish survey results, and implement changes based on survey results.

In previous assignments the author identified Time for Two’s storytime as the service in need of evaluation. Time for Two’s has never had a formal survey or evaluation. Staff generally base the program’s success by attendance. Workers have had users express a desire to have storytime more than once a week and to offer a program for babies, but have never gathered any sort of formal data to support a plan to implement a new storytime. After being without a department head for almost seven months, the Youth Services department is now fully staffed, and now is an opportune time to complete an evaluation. Not only will the new supervisor be able to gauge user interest, so will the veteran staff. With the responses both the Library Assistants and the Youth Services Librarian can create a plan for more or new storytimes to present to the library director.

The current stage is distributing the survey and future stages will include analyze and implementation. As a Library Assistant at Kingsport Public Library, the author may be involved with helping to present a case for Time for Two’s to be offered more than once a week and/or starting a storytime for children younger than two. As a formal evaluation has never been done before, the author is unsure how the results of the survey will be handled by Youth Services staff and the library administration. As with most libraries, staffing and funding are a concern. Additionally, the biggest concern is space and scheduling for an additional Time for Two’s and/or a storytime for younger children. Currently, all programs share one space, the auditorium on the second floor and scheduling a weekly program can be difficult.

5.9 Determine the kinds of analysis you will do with the data collected. How do you plan to report the findings based on the data collected?

The results from the survey will be analyzed to gauge users’ satisfaction with Time for Two’s, aspects they want to improve, and if there is any interest in a program for children younger than two. The results should reveal what is most sought after by users, if starting a storytime for children two and if offering Time for Two’s more than once a week is desirable.

The results of the survey will lend themselves easily to either simple written communication with percentages for each questions or place in a pie chart, bar graph, or some either form of infographic to share with library staff. The opinion questions, and any comments left on any of the questions, will be grouped together by theme in a table or chart, depending on what is most appropriate. The survey should reveal trends of what users find most important about storytime and what users want to see more of along with their overall satisfaction with the program. The primary method of communication will be email with the library staff and will be shared through the University of Tennessee Information Science course 554 web module.

References

Matthews, J. R. (2004). Measuring for Results: The Dimensions of Public Library Effectiveness.

Westport, CT: Libraries Unlimited.

Spring 2015

Contact K.C. Williams

451 Communications Building, 1345 Circle Park Drive, Knoxville, TN 37996-0341

Phone:865-974-2148
Fax:865-974-4967
Email