Deriving Insights from Open-Ended Learner Feedback: Exploring Natural Language Processing Approaches

Researchers at the Centre for Addiction and Mental Health in Toronto, Canada, have investigated the use of natural language processing (NLP) methods to analyze open-ended feedback from learners participating in continuing health education. The study aimed to evaluate the effectiveness of NLP approaches in providing valuable insights for adapting education to learners' needs. The researchers found that large language model (LLM)-based clustering was the most effective method in generating meaningful clusters characterized by semantically similar words for both 'intent-to-use' and 'open-ended feedback' responses.

Key Takeaways:

• The study used a dataset of 480 survey responses from staff participants, including two text responses on how participants intended to use the training and 291 open-ended feedback responses.
• Topic modeling was not useful in differentiating content between topics for the 'intent-to-use' responses due to their short length and lack of diversity.
• Sentiment analysis did not accurately reflect the valence of responses in 'open-ended feedback' due to the complexity of emotions and nuances in language.
• LLM-based clustering approach generated meaningful clusters characterized by semantically similar words for both 'intent-to-use' and 'open-ended feedback' responses.
• This study provides evidence that LLMs can be a useful approach for deriving insights from learner feedback due to their ability to capture context and distinguish between responses that use similar words to convey different topics.
• Future directions include exploring other methods involving LLMs and examining how these methods fare on other data sets or types of learner feedback.

Statistics:

• 480 survey responses from staff participants included 'intent-to-use' and 'open-ended feedback' responses.
• 291 open-ended feedback responses were analyzed using sentiment analysis and LLM-based clustering.
• 1307 (NewsRx, 2025) reports the discovery of LLM-based clustering for deriving learner feedback insights.

Sources:

• Deriving Insights From Open-ended Learner Feedback: an Exploration of Natural Language Processing Approaches. Journal of Continuing Education In the Health Professions, 2025;45(3):203-209.
• Lippincott Williams & Wilkins. Journal of Continuing Education In the Health Professions. Two Commerce Sq, 2001 Market St, Philadelphia, PA 19103, USA. (Wiley-Blackwell - www.wiley.com/; Journal of Continuing Education In the Health Professions - onlinelibrary.wiley.com/journal/10.1002/(ISSN)1554-558X)