Part 5: Developing an Evaluation Action Plan
Abstract: This project will focus on the Nashville Public Library system (NPL) in the Nashville-Davidson County area, serving [around 650,000 residents, of which 330,000 have library cards. The project will analyze and assess the Studio NPL service of the library system, which functions as a teen center. The overall project will focus on the number and quality of services provided by Studio NPL as a part of the NPL system. This fifth section outlines the planned data gathering strategy and what the information will be used for.
Part 5 Developing an Evaluation Action Plan (20 Points)
This assignment will help you prepare an evaluation by discussing various discrete activities in your proposed plan. For this assignment especially study Ch. 1 and Ch. 3-6 EMLS.
1 Develop a rationale to propose user-centered assessment/evaluation of the selected PL service. Assess the selected PL service carefully and explain your proposal in relation to participation of primary, secondary, and tertiary users identified in the earlier assignment.
One of the core services of Nashville Public Libraries is providing services that are beyond books, following into their motto that books being only "half of the story." (Nashville Public Library, 2015) These outreach sources are expanded into providing services that are educational as well as providing new sources of entertainment for the teens to enjoy their experience in the library. They also assist teens by helping them learn off of new technologies such as 3D Printers and Music production software (N. Tognoni, personal communication, January, 30, 2015). The Studio system that has been observed over the last four assignments has potential as it grows to be a significant impact on the teen demographic in the long term.
The complication in assessing the primary, secondary, and tertiary sources is that the teen center is specifically for the tertiary users. The Studio system is specifically developed for the needs of these tertiary users in order to create a space that services their interests, much like the general adult area, which services the primary users of adults, as well as the childrens area. This is also a factor on the secondary users, seniors, who have specialized programs and events based on their interests. This teen area rectifies the gap that comes from providing services to other specific user groups.
Nashville Public Libraries is dependent upon user centered feedback in order to make the determination of the services that are best used in the library system. As described in a previous assignment, teens are a tertiary set of users with regards to the Nashville community. Within the tertiary set of users, there is remarkable potential to get a consistent user base of the Studio services at the branches, which is the primary objective of the Studio center. The focus of this survey will be to cater to their interests and gather feedback on what their ideas on the direction of their teen area should look like in the long term. This could be by determining what types of events teens would like to see in the Studio system.
2 Identify the problem (related to your selected service) in terms of: production bottlenecks; tasks that are performed frequently; activities that require frequent movement; and declining budgets.
The main issue with the Studio in its present state is the relative youth of the program. Only four branches are in full operation at this stage, with the Studio area at the main branch still under construction. From this, it is impossible to see the long term issues of the system. However, conversations with staff have led to a few major issues that are a present concern. The main concerns expressed by Niq Tognoni, Studio Coordinator, stems from staffing, branch distribution, and movement of the staff (personal communication, January, 30, 2015). The Studio system functions under an umbrella organization, of which Studio is the top program and their functions are below it, including the Spoken Word and the Nashville After Zone Alliance (N. Tognoni, personal communication, January, 30, 2015).
One of the main issues, the recruiting of staff for the Studio department, which includes library staff, mentors, and program specialists are being rectified. However, Niq Tognoni clarified the difficulty in finding the most qualified staff. Considering the youth of the program as teen services like this one, it is difficult to find technology minded staff who are in the library field (N. Tognoni, personal communication, January, 30, 2015). However, as the staff is hired on, the issue then becomes how to arrange the staff in order to fill gaps at these Studio centers. At present, the programs are run by a limited number of persons, therefore one person might have to travel to multiple events at two or three branches in a week.
Another issue at hand is the way in which Studio outreaches across social media to the teen demographic. Currently, Studio's website is live, but missing information about the mission, goals, and program descriptions. Furthermore, the social media presence is lacking, considering there are no independent social media accounts for Studio NPL aside from the website (Studio NPL, 2015). This bottleneck prevents information about Studio, the programs at hand, and the online footprint from being more present and available to the teen demographic. This will be one of the focuses of gathering data in order to determine how teens might want to interact with the library on social media.
3 Determine the scope of analysis in terms of what specific information/aspects about the selected PL service do you hope to gain by eliciting feedback from the PL users. In other words, what is the purpose for doing the evaluation? Is the library attempting to improve its operational efficiencies (internal focus), or is the study being done to better understand the effectiveness of a library service? Explain in detail.
The focus of this assessment is to determine the external factors to improve the internal assessment of the Studio system. However, the main consideration at the beginning of this assessment is determining the impact of the outreach programs at present and what the teens know about the facility around them. The goal of this assessment is to outreach directly to teen users, for whom the Studio area is specifically constructed for. The user-centered assessment reaches the tertiary users of the system, the teens, who are the smallest set of in house users in the library system. This is partially due to there not being previously an area specifically for the teens to interact and develop. The author wishes to see how the teens would like to interact with Studio both on and off library facilities.
4 Determine the kinds of data that you hope to gather (be specific). What evaluation methodology and design will be used (quantitative and/or qualitative methods: be very specific in the types of tools that will be employed)? How will data be collected?
The focus of this project is on determining the data of three major ideas:
- What teens already know of the Studio facility
- What teens want to see
- Where/How the library can outreach to teens
This information will then be analyzed and presented to Studio and Teen services for their consumption and determination of web based strategies. These data sets could be conceivably used in order to develop the methods by which teens are engaged in ways similar to the focus groups already in existence. The survey itself is intended to be a way to quantify the information that Studio already collects in an informal way that brings together concepts for the future. The added benefit of this data can help determine the methods by which teens can be reached and engaged outside of the library hours.
5 Describe the user-centered assessment/evaluation strategies that you are proposing to conduct community analysis and gather feedback from users about the selected service at your PL. Identify and analyze what methods you will use to gather feedback from various kinds of users as a part of your efforts. How do you propose getting access/permissions to select users? Provide a timeline for implementing your proposed methods.
The best opportunity to engage with the teen demographic is through a paper survey in the teen area. An alternate consideration would be to create a Survey Monkey or similar web format to obtain responses, but at present the staff of Studio has expressed using the paper format as the simplest way. This opinion stems from the fact that Studio already uses paper formats for their survey, therefore it will follow the format that is already used (M. Ramos, personal communication, March, 15, 2015). The concern for the paper surveys is that without directives to complete the survey, the teens might not complete it or might only complete a part of the survey. This issue is intended to be rectified by having fun size candy near the survey area as an incentive.
The timeline for this survey is intended to run over two days after final approval to obtain as many responses as possible. Currently the plan is slated for:
- 30 March-1 April: Gain final approval for the survey and align format.
- 2-3 April: Run Survey
- ~6-8 April: Present survey results to Studio
6 Share the data collecting instrument that you propose to use. Identify the qualitative and/or quantitative measures you propose to use (what aspects are you trying to measure). Provide strategies to gather consent (consent forms, assent forms as appropriate) and your data-collecting tool(s) with listed questions/prompts, etc. You can explore multiple research methods to gather user-centered assessment/evaluation of the selected service.
The print survey will be used to gather important data on what the teens already know about the Studio format, what they would like to see from Studio, and the best strategies for contact. The survey in itself follows under the twofold intention of determining the teens interests as well as the best ways to reach them for their interests to be expressed. By engaging the users at the branch the author currently works at, it is possible to provide a face to face engagement that can solicit the teens input while observing the trends of what the teens focus on when in the Studio area.
The survey will consist of six questions that are required and two that are optional demographic questions. These questions aim to engage the teen in what they would like to see out of the Studio area, how they wish to see the Studio area grow, and how they want to interact with Studio through social media. The format of the survey on paper is going to mirror what the NPL system currently uses for feedback, which will be obtained upon final approval of the survey.
1) Were you aware that Nashville Public Library has a specialized teen area (called Studio) under further development?
2) If yes, how did you hear about the teen area?
3) What is or can be appealing about an area just for teens?
4) How often would you want to visit a Studio area if it was available every week day?
- Every Day
- A few times a week
- Once a week
- Only for events
5) What type of activities are you most interested in seeing in the Studio? (Choose All That Apply)
- Music Development
- Spoken Word
- 3D Printing
- Video Games
- Not Interested in Any Activities
6) What social media websites would you like to see Nashville Public Libraries post on? (Choose all that apply)
- Other (Write in)
OPTIONAL: The following questions are demographic questions, which will be kept anonymous.
7) Gender. (Choose One)
8) Age Group. (Choose One)
- Grade 7-8
- Grade 9-10
- Grade 11-12
7 Provide strategies to maintain anonymity and confidentiality of users participating in your study (if appropriate).
For this survey, the respondents will only be given two questions related to demographic information, the only identifying factors in the survey. All of the respondents will be teen users and therefore their privacy is paramount in ensuring their identities are protected. The information that will be obtained for this survey will only be used in the context of the project and along with Studio, after which it will not be used.
8 Create a budget in terms of investment of resources (e.g., money, time, efforts, human input, etc.) to orchestrate user-centered assessment/evaluation of various kinds. Identify stages for orchestrating a plan for the PL (e.g., previous efforts made, current efforts, future efforts) and describe what will be feasible as part of your efforts in this course during this semester.
The both previous and current efforts regarding user assessment collection are the paper formats similar to the method that will be used for the survey. This aligns with the program and provides the best way in order to reach the users in the way Nashville Public Library intends to reach their users on a normal basis (N. Tognoni, personal communication, March, 26, 2015) . Based off of the current understanding, there will be a gradual transition to online surveys and further social media feedback. These programs will mirror the growth of the program into a more technology based format.
Time is the largest consideration of the project due to formatting and the actual logistical process of distributing the survey and processing the data obtained in the survey. Effort wise, the program is dependent on further program assessments in order to develop the program as it continues to grow. Monetarily, the program assessments costs are small, with paper surveys on one page costing considering the regular cost of printing at the library being 15 cents per page. This quantified over time can bring even minimal costs lower by transitioning to online web surveys.
9 Determine the kinds of analysis you will do with the data collected. How do you plan to report the findings based on the data collected?
After completion of the survey, this data will be shared with Studio and Teen Library staff to discuss the information obtained. This information will be obtained through paper forms filled out by the teen users of the Green Hills branch Studio center. This will be a voluntary survey so only willing respondents will be involved with the assessment. From this information, it will be possible to determine what the students already know about Studio and what they would like to see out of the Studio system. Further, there will be a way to assess the best ways by which teens can be reached through social media.
As to how the information will be presented to the Studio and Teen staff, a report will be generated as to the choice answers if there are trends that emerge. This report is not intended to cut the outlier responses out but it is intended to highlight any common factors as well as bring in individual ideas that may be of note to the Studio system. Depending on the responses to the demographic information, it is possible to understand with the data as to the age and gender differences in event interest and outreach potential. This information would be valuable to make future determinations as to the direction of both the event planning as well as new social media strategies. With regard to the expected results, the anticipation is finding a clear winner of methods of contact with the teen demographic and their interests in Studio, if there is any clear interest. A curious question is to see if the teens understand the branding of the "Studio" umbrella.
Nashville Public Library. Retrieved March 30, 2015, from http://www.library.nashville.org/
Studio NPL. (n.d.). Retrieved March 30, 2015, from http://nashvillepubliclibrary.org/studionpl/
Contact K.C. Williams451 Communications Building, 1345 Circle Park Drive, Knoxville, TN 37996-0341