Internal Validation of a Machine Learning-Based CDSS for Antimicrobial Stewardship

Internal Validation of a Machine Learning-Based CDSS for Antimicrobial Stewardship

<b>Background:</b> Antimicrobial stewardship programs (ASPs) are essential in combating antimicrobial resistance (AMR); however, limited resources hinder their implementation. Arkstone, a biotechnology company, developed a machine learning (ML)-driven clinical decision support system (CD...

Full description

Saved in:
Bibliographic Details
Main Authors: Ari Frenkel, Alicia Rendon, Carlos Chavez-Lencinas, Juan Carlos Gomez De la Torre, Jen MacDermott, Collen Gross, Stephanie Allman, Sheri Lundblad, Ivanna Zavala, Dave Gross, Jessica Siegel, Soojung Choi, Miguel Hueda-Zavaleta
Format: Article
Language:English
Published: MDPI AG 2025-07-01
Series:Life
Subjects:
machine learning
antimicrobial stewardship
antibiotic resistance
clinical decision support
Online Access:https://www.mdpi.com/2075-1729/15/7/1123
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:<b>Background:</b> Antimicrobial stewardship programs (ASPs) are essential in combating antimicrobial resistance (AMR); however, limited resources hinder their implementation. Arkstone, a biotechnology company, developed a machine learning (ML)-driven clinical decision support system (CDSS) to guide antimicrobial prescribing. While AI (artificial intelligence) applications are increasingly used, each model must be carefully validated. <b>Methods:</b> Three components of the ML system were assessed: (1) A prospective observational study tested its ability to distinguish trained from novel data using various validation techniques and BioFire molecular panel inputs. (2) An anonymous retrospective analysis of internal infectious disease lab results evaluated the recognition of novel versus trained complex datasets. (3) A prospective observational validation study reviewed clinical recommendations against standard guidelines by independent clinicians. <b>Results:</b> The system achieved 100% accuracy (F1 = 1) in identifying 111 unique novel data points across 1110 tests over nine training sessions. It correctly identified all 519 fully trained and 644 novel complex datasets. Among 644 clinician-trained reports, there were no major discrepancies in recommendations, with only 100 showing minor differences. <b>Conclusions:</b> This novel ML system demonstrated high accuracy in distinguishing trained from novel data and produced recommendations consistent with clinical guidelines. These results support its value in strengthening CDSS and ASP efforts.
ISSN:2075-1729

Similar Items