Sign in Subscribe
Machine learning

Artificial Intelligence Models Show Potential in Generating Reliable Information on Burning Mouth Syndrome

Strategic Insights

— 1 min read

Researchers at Link Campus University in Italy have investigated the usefulness, quality, and readability of responses generated by three artificial intelligence large language models (AI-LLMs) - ChatGPT-4, Gemini, and Microsoft Copilot - to frequent questions about Burning Mouth Syndrome (BMS). The study aimed to evaluate the potential of AI-LLMs in providing reliable information on BMS, a chronic orofacial pain disorder with diagnostic and therapeutic challenges.

Key Takeaways:

  • The researchers submitted nine clinically relevant open-ended questions to the AI-LLMs and evaluated the responses using a 4-point usefulness scale, the QAMAI tool, and readability metrics.
  • All AI-LLMs produced moderately useful responses, with no significant difference in global performance.
  • Gemini achieved the highest overall quality scores, particularly in relevance, completeness, and source provision.
  • Copilot scored lower in usefulness and source provision, but no significant differences were obtained among AI-LLMs.
  • Average readability corresponded to 12th grade, with ChatGPT requiring the highest proficiency.
  • The study concluded that continuous optimization is essential to ensure the clinical integration of AI-LLMs.
  • The researchers emphasized the potential of AI-LLMs in generating reliable information on BMS, despite variability in quality, readability, and source citation.

Statistics:

  • 9 clinically relevant open-ended questions were identified through search-trend analysis and expert review.
  • 12 international experts rated the responses using a 4-point usefulness scale.
  • The QAMAI tool was used to evaluate the quality of the responses.
  • Flesch-Kincaid Grade Level and Reading Ease scores were used to measure readability.
  • Kruskal-Wallis and Bonferroni correction were applied for statistical analysis.

Sources:

  • Can Large Artificial Intelligence-Based Linguistic Models Help to Obtain Information About Burning Mouth Syndrome? Oral Diseases, 2025.
  • Oral Diseases can be contacted at: Wiley, 111 River St, Hoboken 07030-5774, NJ, USA.
  • (Wiley-Blackwell - www.wiley.com/; Oral Diseases - onlinelibrary.wiley.com/journal/10.1111/(ISSN)1601-0825)
  • Our news journalists report that additional information may be obtained by contacting Daniela Adamo, Dept. of Life Sciences, Health and Health Professions, Link Campus University, Rome, Italy.

Read more