Study evaluates malignancy prediction models for pulmonary nodules

In a study of models for predicting malignancy in pulmonary nodules, researchers found that those developed using data from the Pan-Canadian Early Detection of Lung Cancer low-dose CT screening trial showed good discrimination and fared better than several other prediction models.

For the study, the researchers estimated malignancy probabilities by applying eight prediction models to data from patients in the intervention arm of the German Lung Cancer Screening Intervention (LUSI) trial. Inclusion criteria included at least one noncalcified pulmonary nodule detected on one of five annual screening visits, a lung cancer diagnosis within the active screening phase of the LUSI trial and an unequivocal identification of the malignant nodule.

The prediction models tested included four developed from the Pan-Canadian Early Detection of Lung Cancer (PanCan) study, including one using a parsimonious approach with nodule spiculation (PanCan-1b), one using a comprehensive approach with nodule spiculation (PanCan-2b), PanCan-2b replacing the nodule diameter variable with mean diameter (PanCan-MD) or PanCan-2b replacing the nodule diameter variable with volume (PanCan-VOL). The researchers also evaluated a model developed by the UK Lung Cancer Screening Trial and three models developed in clinical settings, including the Department of Veterans Affairs, Mayo Clinic and Peking University People’s Hospital.

The final analysis included data from 1,159 patients (median age, 57.63 years; 65.8% men) with 3,903 pulmonary nodules.

‘Excellent’ discrimination

For pulmonary nodules detected during the first round, or prevalence round, of low-dose CT screening in the LUSI trial, discrimination was “excellent” for the PanCan models, with areas under the curve of 0.93 for PanCan-1b, 0.94 for PanCan-2b, 0.94 for PanCan-MD and 0.94 for PanCan-VOL. Among the clinical models, discrimination was “moderately good,” according to the researchers, with AUCs of 0.84 for the VA model, 0.89 for the Mayo Clinic model and 0.87 for the Peking University People’s Hospital model. However, with an AUC of 0.58, discrimination was poor for the UK Lung Cancer Screening Trial model.

When applied to pulmonary nodules first observed on follow-up, or incidence, screening, discrimination was reduced but remained useful for all models, except the UK Lung Cancer Screening Trial, according to the data.

Calibration also appeared to be acceptable for the PanCan-1b, PanCan-2b and UK Lung Cancer Screening Trial models, but not for the PanCan-MD or PanCan-VOL models or any of the prediction models developed in clinical settings.

“Our findings suggest that the PanCan models have good discrimination and confirmatory evidence for calibration accuracy of predicted malignancy risks when applied to nodules observed in an individual’s first (prevalence) screening examination, suggesting that such models may become useful tools for optimizing nodule management in population screening settings,” the researchers wrote.

Potential limitations

In an invited commentary, Marjolein A. Heuvelmans, MD, PhD, from the department of epidemiology at the University of Groningen and University Medical Center and the department of pulmonology at Medisch Spectrum Twente in the Netherlands, and Matthijs Oudkerk, MD, PhD, from the Institute for DiagNostic Accuracy and the faculty of medical sciences at the University of Groningen in the Netherlands, discussed some of the study’s limitations.

“In the LUSI trial, nodule size was determined based on 3D nodule segmentation instead of manual caliper measurements in the axial plane, leading to more precise and reproducible diameter measurements. Performance of the models in a manual nodule diameter-based lung cancer screening trial is expected to be worse. Furthermore, using semi-3D nodule measurements (diameter based on the nodule in three dimensions) might have masked a difference in performance of the diameter and volume versions of the Pan-Canadian model,” they wrote.

The study also emphasized two points, according to Heuvelmans and Oudkerk, including the importance of external testing of prediction models and confirmation that pulmonary nodules detected after the initial screening require separate management. – by Melissa Foster

Disclosures: González Maldonado reports no relevant financial disclosures. Please see the study for all other authors’ relevant financial disclosures. Heuvelmans and Oudkerk report no relevant financial disclosures.