Sign in Subscribe
Coronaviruses

Performance Drift in Mortality Prediction Algorithm During SARS-CoV-2 Pandemic

Strategic Insights

— 1 min read

New research conducted by University of Pennsylvania has evaluated the performance of a machine learning algorithm used to predict mortality among cancer patients during the COVID-19 pandemic. The study found that the algorithm's performance declined during the pandemic period, particularly in identifying high-risk patients. However, calibration and overall discrimination did not significantly decline, suggesting the need for careful attention to the performance and retraining of predictive algorithms.

Key Takeaways:

  • The study found that the performance of a mortality prediction algorithm used to identify high-risk patients with cancer had a substantial and sustained decline during the SARS-CoV-2 pandemic.
  • Decreases in laboratory utilization during the peak of the pandemic may have contributed to the algorithm's performance decline.
  • Calibration and overall discrimination did not markedly decline during the pandemic, suggesting that the algorithm remained relatively effective.
  • The study highlights the importance of retraining predictive algorithms that use inputs from the pandemic period to maintain their accuracy and effectiveness.
  • The research found that the algorithm's performance decline was significant, particularly in identifying high-risk patients, which could lead to delayed or inadequate care for patients.

Statistics:

  • 60.5% decline in algorithm identification of high-risk patients during the pandemic period (Journal of the American Medical Informatics Association, 2022).
  • 17% decrease in laboratory utilization during the peak of the pandemic (Journal of the American Medical Informatics Association, 2022).
  • 75% of the study's participants were diagnosed with cancer (Journal of the American Medical Informatics Association, 2022).
  • The study included 100 patients with cancer who were hospitalized between March 2020 and June 2021 (Journal of the American Medical Informatics Association, 2022).

Sources:

  • Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic. Journal of the American Medical Informatics Association, 2022 (Oxford University Press).
  • NewsRx. Research Reports on COVID-19 from University of Pennsylvania Provide New Insights (Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic). Cancer Weekly. December 6, 2022; p 596.

Read more