Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic
Objective – The purpose of this research was to explore user sentiment on Ask a Librarian, a consortial chat service for university libraries in Ontario, Canada, between 2019 to 2021. We tested how the characteristics of the chat (such as year, semester, user type, operator type, affiliation mismat...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Alberta
2025-06-01
|
| Series: | Evidence Based Library and Information Practice |
| Online Access: | https://journals.library.ualberta.ca/eblip/index.php/EBLIP/article/view/30642 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839640184537219072 |
|---|---|
| author | Kathryn Barrett Ansh Sharma |
| author_facet | Kathryn Barrett Ansh Sharma |
| author_sort | Kathryn Barrett |
| collection | DOAJ |
| description |
Objective – The purpose of this research was to explore user sentiment on Ask a Librarian, a consortial chat service for university libraries in Ontario, Canada, between 2019 to 2021. We tested how the characteristics of the chat (such as year, semester, user type, operator type, affiliation mismatch, and user complaints) and the onset of the COVID-19 pandemic affected sentiment scores.
Methods – The researchers analyzed 3,339 chat transcripts using VADER, a free, open-source Python natural language processing library for sentiment analysis. We tested the significance of relationships between study variables and sentiment score using either a two-samples t-test or ANOVA.
Results – Between 2019 to 2021, overall sentiment on Ask a Librarian was positive and higher among operators than users. There was a significant relationship between sentiment scores and operator type, affiliation mismatch, and complaints respectively. The year, semester, and pandemic status of the chat were also significantly associated with sentiment score. Chats that took place during the COVID-19 pandemic had a significantly higher overall sentiment score than pre-pandemic chats. Average user sentiment score was also higher during the pandemic, but there were no significant differences in average operator sentiment score.
Conclusion – The COVID-19 pandemic had a significant effect on the emotional tone of the overall chat interaction, as well as the sentiment within the user’s messages. Practitioners can replicate our approach to understand user emotions, opinions, attitudes, or appraisals during times of disruption or emergency, as well as for regular service assessment.
|
| format | Article |
| id | doaj-art-f05c8b48c128435e8cad1aab35e9082b |
| institution | Matheson Library |
| issn | 1715-720X |
| language | English |
| publishDate | 2025-06-01 |
| publisher | University of Alberta |
| record_format | Article |
| series | Evidence Based Library and Information Practice |
| spelling | doaj-art-f05c8b48c128435e8cad1aab35e9082b2025-07-03T18:00:02ZengUniversity of AlbertaEvidence Based Library and Information Practice1715-720X2025-06-0120210.18438/eblip30642Library Chat Transcript Evaluation for User Sentiment During the COVID-19 PandemicKathryn Barrett0https://orcid.org/0000-0002-7940-2816Ansh Sharma1https://orcid.org/0000-0001-6899-3270University of Toronto Scarborough, Scarborough, Ontario, CanadaUniversity of Toronto Scarborough, Scarborough, Ontario, Canada Objective – The purpose of this research was to explore user sentiment on Ask a Librarian, a consortial chat service for university libraries in Ontario, Canada, between 2019 to 2021. We tested how the characteristics of the chat (such as year, semester, user type, operator type, affiliation mismatch, and user complaints) and the onset of the COVID-19 pandemic affected sentiment scores. Methods – The researchers analyzed 3,339 chat transcripts using VADER, a free, open-source Python natural language processing library for sentiment analysis. We tested the significance of relationships between study variables and sentiment score using either a two-samples t-test or ANOVA. Results – Between 2019 to 2021, overall sentiment on Ask a Librarian was positive and higher among operators than users. There was a significant relationship between sentiment scores and operator type, affiliation mismatch, and complaints respectively. The year, semester, and pandemic status of the chat were also significantly associated with sentiment score. Chats that took place during the COVID-19 pandemic had a significantly higher overall sentiment score than pre-pandemic chats. Average user sentiment score was also higher during the pandemic, but there were no significant differences in average operator sentiment score. Conclusion – The COVID-19 pandemic had a significant effect on the emotional tone of the overall chat interaction, as well as the sentiment within the user’s messages. Practitioners can replicate our approach to understand user emotions, opinions, attitudes, or appraisals during times of disruption or emergency, as well as for regular service assessment. https://journals.library.ualberta.ca/eblip/index.php/EBLIP/article/view/30642 |
| spellingShingle | Kathryn Barrett Ansh Sharma Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic Evidence Based Library and Information Practice |
| title | Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic |
| title_full | Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic |
| title_fullStr | Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic |
| title_full_unstemmed | Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic |
| title_short | Library Chat Transcript Evaluation for User Sentiment During the COVID-19 Pandemic |
| title_sort | library chat transcript evaluation for user sentiment during the covid 19 pandemic |
| url | https://journals.library.ualberta.ca/eblip/index.php/EBLIP/article/view/30642 |
| work_keys_str_mv | AT kathrynbarrett librarychattranscriptevaluationforusersentimentduringthecovid19pandemic AT anshsharma librarychattranscriptevaluationforusersentimentduringthecovid19pandemic |