| Title |
|
Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation. |
|
|
| Publication Date |
|
| Author(s) |
|
| Subject |
|
| Type of document |
|
| OrcId |
|
#PLACEHOLDER_PARENT_METADATA_VALUE# |
|
|
#PLACEHOLDER_PARENT_METADATA_VALUE# |
|
|
#PLACEHOLDER_PARENT_METADATA_VALUE# |
|
|
#PLACEHOLDER_PARENT_METADATA_VALUE# |
|
|
| DOI |
|
| Abstract |
|
Artificial intelligence (AI) has gradually found its way into healthcare, and its future integration into clinical practice is inevitable. In the present study, we evaluate the accuracy of a novel AI algorithm designed to predict admission based on a triage note after clinical implementation. This is the first of such studies to investigate real-time AI performance in the emergency setting. The novel AI algorithm that predicts admission using a triage note was translated into clinical practice and integrated within St Vincent's Hospital Melbourne's electronic emergency patient management system. The data were collected from 1 January 2021 to 17 August 2022 to evaluate the diagnostic accuracy of the AI system after implementation. A total of 77 125 ED presentations were included. The live AI algorithm has a sensitivity of 73.1% (95% confidence interval 72.5-73.8), specificity of 74.3% (73.9-74.7), positive predictive value of 50% (49.6-50.4) and negative predictive value of 88.7% (88.5-89) with a total accuracy of 74% (73.7-74.3). The accuracy of the system was at the lowest for admission to psychiatric units (34%) and at the highest for gastroenterology and medical admission (84% and 80%, respectively). Our study showed the diagnostic evaluation of a real-time AI clinical decision-support tool became less accurate than the original. Although real-time sensitivity and specificity of the AI tool was still acceptable as a decision-support tool in the ED, we propose that continuous training and evaluation of AI-enabled clinical support tools in healthcare are conducted to ensure consistent accuracy and performance to prevent inadvertent consequences. |
|
|
| Link |
|
| Citation |
|
Emergency Medicine Australasia: EMA 2024-02; 36(1)
: |
|
|
| Jornal Title |
|
Emergency Medicine Australasia : EMA |
|
|
| ISSN |
|
