The Deluge of AI-Generated 'Science' Threatening Public Health and Corporate Interests

Strategic Insights -

The explosion of research papers generated by artificial intelligence (AI) tools has become a major concern in the field of public health and science. The use of AI to produce papers with convincing but misleading health claims is a growing threat, and experts warn that it could be used to promote corporate interests. A recent study found that the number of single-factor studies published in leading academic databases has increased from four per year between 2014 and 2021 to 190 in the first ten months of 2024. These studies often have misleading correlations that occur by chance, and they can be easily produced with AI tools. The UK government's recent guidance on baby-food marketing has incentivized firms to find results that promote the health benefits of their products, further increasing the demand for AI-assisted "scientific evidence."

Key Takeaways:

  • The use of AI tools to generate papers with convincing but misleading health claims is a growing threat to public health and corporate interests.
  • The number of single-factor studies published in leading academic databases has increased from four per year between 2014 and 2021 to 190 in the first ten months of 2024.
  • These studies often have misleading correlations that occur by chance, and they can be easily produced with AI tools.
  • The UK government's recent guidance on baby-food marketing has incentivized firms to find results that promote the health benefits of their products.
  • The current peer-review system is ill-equipped to cope with the deluge of papers generated by AI tools, and experts warn that it needs to be reformed to ensure the credibility of scientific research.
  • Researchers have proposed several methods to reduce the risk of specious findings in published papers, including preregistration, transparent reporting, and specification curve analysis.
  • Some journals have adopted these proposals and updated their rules to requires authors to publish their data, code, and funding sources, as well as disclose conflicts of interest.
  • Public trust in science remains high, but AI threatens to take us further from the ideal of impartial, truthful research.

Statistics:

  • 4 single-factor studies per year were published between 2014 and 2021.
  • 190 single-factor studies were published in the first 10 months of 2024.
  • The number of papers published per year has increased significantly due to AI tools.
  • 75-100% of researchers have reported using AI to analyze data.
  • 30-50% of researchers have reported using AI to write papers.

Sources:

  • David Comerford, Professor of Economics and Behavioural Science, University of Stirling, "We risk a deluge of AI-written 'science' pushing corporate interests -- here's what to do about it", The Conversation, https://theconversation.com/we-risk-a-deluge-of-ai-written-science-pushing-corporate-interests-heres-what-to-do-about-it-264606
  • The Conversation, "We risk a deluge of AI-written 'science' pushing corporate interests -- here's what to do about it", 2024
  • Gorodenkoff Journal