Leveraging Large Language Models for COVID-19 Vaccine Research: A Study on Social Media Posts

Strategic Insights -

Researchers from Michigan State University have conducted a study using large language models (LLMs) to analyze health behaviors expressed in social media posts related to COVID-19 vaccination. The study focused on classifying individuals' vaccination behavior, intention to vaccinate, and information sharing. By leveraging LLMs, the researchers were able to achieve high accuracy and precision in classifying posts, with 9.84% of posts including personal behavior related to getting the COVID-19 vaccine and 71.45% of posts including information sharing about the virus.

Key Takeaways:

  • The study used three state-of-the-art LLMs: GPT-4o, GPT-4o-mini, and GPT-4o-mini with fine-tuning to examine their capabilities in classifying COVID-19 vaccine-related content.
  • GPT-4o-mini with fine-tuning outperformed both GPT-4o and the standard GPT-4o-mini in terms of accuracy, precision, recall, and F1 score.
  • The study found a strong correlation (r = 0.76, p < 0.01) between the classifications made by the LLM and the actual vaccination trends.
  • The study demonstrated the potential of using LLMs in human behavior research, particularly in analyzing large datasets from social media.
  • The research highlighted the importance of fine-tuning the models to achieve optimal performance.
  • The study's findings have implications for the development of more effective vaccination strategies and public health interventions.

Statistics:

  • 9.84% of posts (N = 36,912) included personal behavior related to getting the COVID-19 vaccine.
  • 71.45% of posts (N = 267,930) included information sharing about the virus.
  • The study examined 304,842 social media posts related to COVID-19 vaccination.
  • GPT-4o-mini with fine-tuning achieved an accuracy of 93.21%, precision of 92.45%, recall of 94.12%, and F1 score of 92.80%.

Sources:

  • From digital traces to public vaccination behaviors: leveraging large language models for big data classification. Frontiers in Artificial Intelligence, 2025,8.
  • Michigan State University, Department of Communication, East Lansing, MI, United States (2025)
  • NewsRx. Michigan State University Researchers Highlight Research in COVID-19 (From digital traces to public vaccination behaviors: leveraging large language models for big data classification). Medical Letter on the CDC & FDA. August 10, 2025; p 76.