Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans: results from a phase 3 trial

Abstract

Training for accurate image interpretation is essential for the clinical use of β-amyloid PET imaging, but the role of interpreter training and the accuracy of the algorithm for routine visual assessment of florbetaben PET scans are unclear. The aim of this study was to test the robustness of the visual assessment method for florbetaben scans, comparing efficacy readouts across different interpreters and training methods and against a histopathology standard of truth (SoT). METHODS: Analysis was based on data from an international open-label, nonrandomized, multicenter phase-3 study in patients with or without dementia (ClinicalTrials.gov: NCT01020838). Florbetaben scans were assessed visually and quantitatively, and results were compared with amyloid plaque scores. For visual assessment, either in-person training (n = 3 expert interpreters) or an electronic training method (n = 5 naïve interpreters) was used. Brain samples from participants who died during the study were used to determine the histopathologic SoT using Bielschowsky silver staining (BSS) and immunohistochemistry for β-amyloid plaques. RESULTS: Data were available from 82 patients who died and underwent postmortem histopathology. When visual assessment results were compared with BSS + immunohistochemistry as SoT, median sensitivity was 98.2% for the in-person-trained interpreters and 96.4% for the e-trained interpreters, and median specificity was 92.3% and 88.5%, respectively. Median accuracy was 95.1% and 91.5%, respectively. On the basis of BSS only as the SoT, median sensitivity was 98.1% and 96.2%, respectively; median specificity was 80.0% and 76.7%, respectively; and median accuracy was 91.5% and 86.6%, respectively. Interinterpreter agreement (Fleiss κ) was excellent (0.89) for in-person-trained interpreters and very good (0.71) for e-trained interpreters. Median intrainterpreter agreement was 0.9 for both in-person-trained and e-trained interpreters. Visual and quantitative assessments were concordant in 88.9% of scans for in-person-trained interpreters and in 87.7% of scans for e-trained interpreters. CONCLUSION: Visual assessment of florbetaben images was robust in challenging scans from elderly end-of-life individuals. Sensitivity, specificity, and interinterpreter agreement were high, independent of expertise and training method. Visual assessment was accurate and reliable for detection of plaques using BSS and immunohistochemistry and well correlated with quantitative assessments.