Abstract
Objective To determine inter-reader reliability and diagnostic accuracy of MSKI-RADS among early-career radiologists with different experience levels. Materials and methods This is a retrospective multicenter study of MRIs of proven musculoskeletal (MSK) infections. MSKI-RADS categories included: 0-incomplete imaging, I-negative for infection, II-superficial soft tissue infection, III-deeper soft tissue infection, IV-possible osteomyelitis (OM), V-highly suggestive of OM, VI-known OM, and NOS-nonspecific bony lesions unrelated to infection. There were 22 readers from 9 institutions with 1-8 years of radiology experience, including radiology residents, MSK radiology clinical fellows, and junior MSK attendings. After initial training with expert readers, a dataset of 210 cases across a spectrum of extremity infections from a single institution was evaluated. The readers recorded the following for each case: MSKI-RADS score, final qualitative diagnosis, and confidence levels for both. Inter-reader agreements (ICC) and accuracy were obtained. Results Among 210 cases, there were 17 negative, 32 superficial soft tissue infections, 39 deeper soft tissue infections, 24 possible OM, 41 highly suggestive of OM, 18 known OM, and 39 NOS cases. A moderate inter-reader agreement was seen for 22 readers (ICC: 0.57 (CI = 0.52, 0.67)). The average true positive rates for non-infectious cases (I and NOS), soft tissue infection (II and III), and bony infection (IV and V) were 70% (95% CI: 67-73%), 66% (95% CI: 63-68%), and 82% (95% CI: 80-84%), respectively. Average true positive rates for classes V and VI were 80% (95% CI: 77-82%) and 86% (95% CI: 82-89%), respectively. Overall, MSKI-RADS reader accuracy was 66 +/- 10%, higher than the qualitative diagnosis accuracy of 59 +/- 11% (p < 0.05). Average confidence levels for MSKI-RADS score and final qualitative diagnosis were 3.56 +/- 0.91 and 3.52 +/- 0.92, respectively (p > 0.05). Conclusion MSKI-RADS is an accurate and reliable MRI-based classification system for the diagnosis of a spectrum of musculoskeletal infections among early-career radiologists, like expert readers.