Comparison of an Artificial Intelligence Model and General Practitioners in Detecting Pneumoperitoneum on Chest Radiographs

Purpose To compare the diagnostic performance of an artificial intelligence (AI) model and general practitioners in detecting pneumoperitoneum on chest radiographs.
Materials and Methods In this study, 300 chest x-rays were collected from a tertiary care hospital between February 2020 and January 2024. The patient data used in this study were collected with approval from the Institutional Review Board (IRB) and were fully anonymized prior to analysis. A vision-language alignment AI model was developed to compute similarity scores between chest radiographs and textual descriptions, such as findings and impressions, presented as numeric values. The decision threshold for the AI model was determined based on the optimal performance on a separate validation dataset. Two general practitioners participated in a reader study to assess the presence or absence of pneumoperitoneum. Sensitivity, specificity, and accuracy were evaluated for both the AI model and human readers.
Results A total of 300 patients (mean age 51.0 ± 17.4 years) were included. 150 pneumoperitoneum cases and 150 non-pneumoperitoneum cases were selected based on confirmed reports from the hospital database. Reader 1 demonstrated sensitivity, specificity, and accuracy of 99.3% each, and Reader 2 showed 94.7%, 100%, and 97.3%, respectively. The average sensitivity, specificity, and accuracy of the two readers were 97.0%, 99.6%, and 98.3%, respectively, whereas the AI model reported 96.0%, 95.3%, and 95.7%, respectively. Notably, the AI model correctly identified all 9 cases (3%) missed by general practitioners, including 8 cases of pneumoperitoneum (false negatives) and 1 case of non-pneumoperitoneum (false positive).
Conclusion The AI model demonstrated diagnostic performance closely aligned with that of general practitioners and successfully identifying all cases that were missed by them. These findings indicate that the AI model outperformed general practitioners in critical value reporting, as it detected all life-threatening cases missed by human readers. Further study is needed to validate its clinical utility as an assistive tool in real-world clinical settings.
Clinical Relevance Statement The AI model demonstrated the ability to detect pneumoperitoneum cases that were overlooked by general practitioners, suggesting its potential value in settings where radiologists or specialists are not readily available. This can be especially beneficial in smaller clinics or rural hospitals, where AI can assist general practitioners in identifying acute conditions such as pneumoperitoneum, thus improving early diagnosis and patient care.