Skip to Main Content

The University of Tennessee

School of Information Sciences

Frequently Used Tools:



INSC 554


Main Navigation:

Collecting and Analyzing Data

The focus of this study is on the Bristol Public Library (BPL) which serves approximately 44,000 residents of Bristol, Tennessee and Bristol, Virginia. More specifically this study seeks to assess and evaluate BPL's Adult Learning Lab (A.L.L.) in terms of the library's strategic plan and the needs of its community. This paper summarizes the findings of the user-centered assessment and evaluation performed using a web-based survey of A.L.L. users. The data analysis is discussed and suggestions are offered for improvement based on the findings. This paper critiques the evaluation action plan used to execute this user-centered assessment and offers suggestions for improvement in the plan for future use of the survey.

6.1 Critique and analyze data collected: Discuss and report user feedback about the PL service in terms of their needs and information experiences, availability and access, barriers/challenges faced in using the select service, kinds of benefits in using the service, suggestions for improving the service, etc.

The evaluation action plan consisted of the creation and distribution of a web-based survey. The survey was created and distributed with Google Forms on the A.L.L. Facebook page, Bristol Public Library Facebook page, and through the libraries email drop list. Google Forms was also used to collect and organize the data. The web-based survey was open to A.L.L. users beginning on Thursday April 23rd at 12 p.m. and ending Tuesday April 28th at 12 p.m. A total of 15 surveys were completed.

The first four survey questions asked respondents to rate their experiences with the A.L.L. using a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). These four questions were mandatory. The survey also included three voluntary open-ended questions to allow respondents to fully express their opinions. The last question on the survey was a mandatory factual question asking respondents to report their age. The survey results were gathered using Google Forms. Based on the findings, overall users are satisfied with the service provided by the A.L.L.

For question one (The material covered during the session(s) was relevant), 80% strongly agreed, 6.7% agreed, and 13.3% answered neutrally. No one disagreed or strongly disagreed with the statement. The mean for this question is 4.67, median 5, and mode 5. These findings imply that the material covered during sessions at the A.L.L. are relevant and successfully meeting the needs of the users.

For question two (The instructor clearly presented the information), 80% strongly agreed, 6.7% agreed, 6.7% answered neutrally, and 6.7% disagreed. The mean for this question is 4.6, median 5, and mode 5. These results signify that the A.L.L. instructors are doing an overall good job presenting the material to the users in a way that is easy to understand and utilize. One user disagreed with the statement but did not provide any additional feedback as to why or how the staff could have presented the information differently.

For question three (The session(s) helped me to improve my skill set), 80% strongly agreed and 20% answered neutrally. No one disagreed or strongly disagreed with this statement. The mean for this question is 4.6, median 5, and mode 5. These findings suggest that the A.L.L. is successful in assisting users in the development of the skills needed to meet their personal and professional goals.

For question four (I would recommend this service to a friend or family member), 86.7% strongly agreed, 6.7% answered neutral, and 6.7% disagreed. The mean for this question is 4.4, median 5, and mode 5. These results reveal that the users of the A.L.L. are highly likely to promote the service to their friends and family. This is seen at the A.L.L. by the high number of users they receive from word-of-mouth promotion. Once again, one user disagreed with the statement but provided no feedback as to why.

Following this section were three open-ended questions that allowed respondents to further elaborate their opinion on the service if they desired. The results are included exactly how the respondent stated them. Question five asked, "Is there anything you would change about the Adult Learning Lab?" Out of the 15 respondents, six chose to respond to this question. Overall, most respondents could not think of anything they would change about the A.L.L. The two suggestions for change are both unfortunately not very controllable by the library at this time. One expressed that they would like to see more instructors and funding provided to the A.L.L. An issue the library staff are aware of and are working to improve. Another expressed that they would limit who is able to receive service at the A.L.L., expressing a concern that the "rough group" that currently uses the lab services may be deterring others from taking advantage of the offerings. However, this is a free public service and anyone needing services are allowed to do so. The comments provided are included below.

  • "They need more teachers and more funding."
  • "The individuals that come to the lab are a rough group and I think that that deters a lot of people from coming to get assistance. It can be very uncomfortable some times."
  • "No while I was enrolled there they made me content with what I had to do."
  • "The Literacy Academy helped me pass the GED in 1 week. My tutor was awesome!"
  • "No"
  • "Not a thing"

Question six asked respondents "Are there any additional services that you would like the Adult Learning Lab to offer?" Only three respondents answered this question. The users would like to have more classroom style sessions and though it did not specifically answer the question being asked another respondent suggested that the lab needs more instructors. Once again, the same respondent expressed a concern about the individuals using the lab causing problems. During meetings with A.L.L. and other library staff members' issues with users were not brought up. This may be an area that staff need to pay closer attention to in the future and address as necessary to assure a comfortable learning space for all of its users. The responses to this question are provided below.

  • "More classroom style classes. Such as MS word, excel, PowerPoint, access. I also think if you offered less free time and more classes it would cut down on the individuals that come there and cause problems."
  • "I think if they had a couple more people it would help."
  • "No"

Question seven asked, "What days and times are most convenient for you to visit the Adult Learning Lab?" Based on the responses the A.L.L. is accommodating to the scheduling of its users. The only recommendation that can be taken from the results, is to extend hours to 8 p.m. This question may have received more usable data if it had been a mandatory quantitative question instead of allowing respondents the ability to write in their response. One respondent chose to give a testimony of their success after receiving help at the lab and appreciation for its services in this field. The responses are provided below.

  • "4-8 in the evenings"
  • "between three and five in the afternoon"
  • "Anytime they helped several members of my family and friends get their own GED's"
  • "I came on my day off, to study"
  • "Any"
  • "When I went, I choose to go threw out the day time, often afternoon. I was a mother of 4 and a wife whom worked. I achieved my GED thanks to the kind ladies and gentlemen whom offered their help each day to show me the way. I am eternally grateful to each of them and this program as well. It helped show my children that though I had messed up and made bad choices as a young mother that there is still options to turn my life back around and I did it all right alongside of my babies."

Lastly, out of the 15 respondents to the survey 10 (66.7%) were between the ages of 35-65 and five (33.3%) were between the ages of 18-34. No one completed the survey over the age of 65 or under the age of 18. The majority of the respondents represented the primary and tertiary user groups of the A.L.L. It is a possibility that the secondary user group, elderly adults, did not respond due to a lack of computing skills or internet access.

6.2 Improvement strategies: Present a plan for improvement strategies to implement in relation to the selected service for the PL to follow based on the data that you collected during this study. Develop a categorization scheme for the improvement strategies under different headings as appropriate (e.g., web representation, training, marketing to users, etc.).

The analysis of the data collected for this project suggests that overall users of the A.L.L. are satisfied with the service being provided. Unfortunately, the only respondent to disagree with any of the closed-ended questions did not leave any suggestions for improvement. Nor did any of the individuals who felt neutral regarding the material, presentation, and ability of the A.L.L. to improve user skills. As was mentioned in assignment five, this study only serves as a prototype of the study due to time constraints. The data collected only represents the perspectives and opinions of a small number of users. This study was successful in demonstrating the importance of integrating user-centered evaluation into the ongoing workflow of the A.L.L. Implementing this survey on a larger scale will allow the library to gain more insight. Based on the results of this small sampling, it was possible to identify a few improvement strategies to begin implementing once the A.L.L. moves into its new location in the library.

Lab Instructors

One recommended area of improvement involves the need for more lab instructors. The fulfillment of this improvement will also allow the library to extend the hours of the A.L.L. to accommodate community members who prefer evening sessions. The library is already aware of this issue and is working to improve this. The main obstacle for this improvement is finding sources of additional funds to hire more instructors or connecting with individuals willing to volunteer. One affordable suggestion is to form a relationship with King University and Virginia Intermont College. The A.L.L. would be a great internship opportunity for individuals in various programs at the two universities such as education and communication studies. Another relationship that could prove to be beneficial for the A.L.L. is the Sullivan County Broadband Committee. One of the goals listed in the Sullivan County Technology Action Plan is to further develop the current digital learning efforts of Sullivan County, Tennessee (Sullivan County Broadband Committee, 2013, October). The action items for this goal are to "discuss opportunity with community libraries, identify locations for implementing the programs, and work with local media to bring awareness to the programs" (Sullivan County Broadband Committee, 2013, October).

Classroom Instruction

Another recommendation for improvement identified from the survey data is the addition of structured classes during scheduled time frames. The classes could be offered on a weekly basis and cover topics such as basic computing, specific software programs such as Microsoft Word and Excel. BPL received similar feedback during the focus group performed in January and February of 2015. Kingsport Public Library provides an excellent local example of these classroom style instructions in action. Offering a combination of classroom style classes and walk-in hours will allow the A.L.L. to accommodate different learning styles and schedules.

6.3 Report findings to your PL and provide feedback about their response to your findings.

After the data was analyzed a report of the results (including the bar charts and pie chart created by Google Forms) and suggested improvements based on the findings were provided to Karen Gaylor, A.L.L. Coordinator and Doris Stickley, BPL Director of Marketing. They were both pleased with the results of the user-centered evaluation. The user feedback was in sync with the feedback received during the focus groups. Based on this the survey results provided reinforcement to the focus group data. This is important to the library as it undergoes a renovation as well as begins to transition the A.L.L. to the Adult Creativity Center. The findings in the study reinsured the library staff of the importance of adding structured class sessions to the community. It also helped to establish the importance of periodically surveying the users to assure the service is still satisfying them and to collect improvement suggestions. This will allow the service to stay current on community needs and keep on track with fulfilling those needs.

6.4 Critique your process: Discuss and report your process while conducting user-centered assessment in your evaluation action plan and include your observations, what worked and did not work, obstacles and challenges, etc. Provide a discussion of the most interesting facts that emerged during the process.

The action plan designed and implemented for this user-centered assessment resulted in the compilation of data from 15 user responses. The researcher created the online survey using Google Forms, a free easy to use survey program offered by Google. The survey consisted of nine questions. Four questions used a Likert-scale, four allowed the respondent to provide feedback in their own words, and one factual question was multiple choice. Dividing the survey like this allowed the researcher to fully capture the respondent's opinions and perspectives. The length was kept short to keep respondents engaged and ensure completion. Some of the responses to the qualitative questions did not actually answer the question asked, this should be taken into account for future surveys. More work needs to be done to assure the questions are worded in a way that the target audience understands before another survey is conducted. This demonstrates the importance of testing a prototype on a small number of users before launching a large-scale study.

The web-based survey was a great tool for providing convenience and privacy for the respondent. It was easy to make and distribute as well. However, it may have also been beneficial to offer a print survey format to users after completing sessions at the A.L.L. The print format would have been ideal for users who are still new to computers and did not complete the survey due to a low computer confidence level or skill level. The results to the print surveys could have been manually entered to the Google Forms spreadsheet for calculating and analyzing results. Google Forms collects the survey responses in a spreadsheet on Google Drive. The spreadsheet provides a time stamp and the answers to each question in a separate field. This made the data easily available to monitor, download and analyze. Google Forms also provides what is called a summary of responses, which is the results of the quantitative question displayed in bar charts (except the factual question which was displayed in a pie chart) and the responses to the qualitative questions in list form.

One of the greatest challenges was the limited time frame for the survey. It would be ideal to promote the survey for a longer period of time, based on the response rate during the five-day period, two to four weeks would be a more appropriate period. This survey was promoted on both the A.L.L. Facebook and the library Facebook page. It was also sent out to an email chain of approximately 1,000 library users. Despite this amount of promotion, the survey still received a low response rate. For future surveys, additional promotional methods may be necessary. One suggestion is to put up signage in the library advertising the survey and to provide print copies of the survey at the reference and circulation desks. Another suggestion regarding future surveys would be to include the libraries logo. Not including a logo may have had an impact on the response rate, as the survey may not have appeared official to the audience.

The survey results were compiled and analyzed on Tuesday April 28th. The results of the quantitative questions were analyzed by calculating the mean, median, and mode and using the percentages of responses provided by Google Forms. The researcher then examined the qualitative data for trends. The results from this survey were a great source of encouragement for the A.L.L. and provided evidence of the satisfaction of its users. There were no surprises or new revelations revealed within the data collected. As was mentioned previously, the findings reported in this analysis matched the findings from the focus group. Offering an incentive, such a chance to win gift cards, for taking the survey would have been another great option to increase the response rate. This would have also been a way for the A.L.L. to collect user contact information so that a follow-up survey could be sent after a set period of time to inquire on continued success and development after sessions.

References

Sullivan County Broadband Committee. (2013, October). Sullivan County Technology Action Plan. Retrieved from http://www.connectedtn.org/sites/default/files/connected-nation/Tennessee/files/sullivan_county_tn_technology_action_plan_final.pdf

Spring 2015

Contact K.C. Williams

451 Communications Building, 1345 Circle Park Drive, Knoxville, TN 37996-0341

Phone:865-974-2148
Fax:865-974-4967
Email