The diagnostic performance of an AI-based CADx system for pulmonary nodules with localization and measurement: a multi-center clinical trial and validation study

Purpose We evaluated the diagnostic performance of a CADx system, which provides information on the type (solid, ground-glass opacity and part-solid), size, malignancy classification, lung-RADS category, localization and measurement of pulmonary nodules in LDCT images.
Materials and Methods We conducted two retrospective multi-center studies in South Korea to evaluate the CADx system, involving a total of 804 participants. The clinical trial included 455 participants across three institutions from 2019 to 2023. The validation study included 349 participants across four institutions from 2023 to 2024. In both studies, 8 thoracic radiologists independently annotated pulmonary nodules using medical image annotation software, marking size and type with a freehand tool. The CADx system (DEEP:LUNG, DEEPNOID Inc., Seoul, Korea) then analyzed the LDCT images. Primary endpoints were localization-adjusted sensitivity and specificity for nodule detection and sensitivity and specificity for malignancy classification. Secondary endpoints included performance stratified by lung-RADS category and nodule size comparison between CADx predictions and radiologist annotations. Additionally, we performed an analysis of sensitivity across different Dice Similarity Coefficient thresholds.
Results In the clinical trial, the CADx system achieved a localization-adjusted sensitivity of 91.7% and specificity of 93.1%. Malignancy classification sensitivity and specificity were 85.7% and 87.5%, respectively. In the validation study, sensitivity and specificity were 90.6% and 80.0%, while malignancy classification sensitivity and specificity were 92.2% and 91.0%. The sensitivities for lung-RADS categories 1, 2, 3, 4A and 4B were 86.9%,
68.9%, 17.1%, 50.7% and 84.7%, respectively, across both studies. The intraclass correlation coefficient (ICC) for nodule size between CADx and radiologists was 0.86 (95% CI, 0.84–0.87) and Bland–Altman analysis demonstrated acceptable agreement.
Conclusion The CADx system demonstrated robust and consistent diagnostic performance in both the clinical trial and realworld validation settings. It offers reliable support for pulmonary nodule localization and malignancy risk classification in LDCT screening, although further improvement is warranted for part-solid nodules. However, performance for Lung-RADS category 3 was relatively suboptimal, likely due to the heterogeneous and borderline nature of such nodules.
Clinical Relevance Statement An AI-based CADx system can consistently assist pulmonary nodule detection and malignancy classification across clinical and real-world settings.