Synthetic PET Generator: A Novel Method to Improve Lung Nodule Detection by Combining Outputs from a Pix2pix Conditional Adversarial Network and a Convolutional Neural Network Based Malignancy Probability Estimator

(Accepted Poster at RSNA 2018, Tue Nov 27 2018 12:15PM – 12:45PM)


Assessment of malignancy of lung nodules on CT scans is a subjective and arduous task for radiologists with low reported accuracy rates, especially for small nodules. We propose a novel method to generate a synthetic PET image of the lung from CT images using Conditional Generative Adversarial Networks (cGAN) that can improve the sensitivity of the radiologist in the detection of malignant lung nodules.


We used a combination of a PET Generator combined with a Malignancy Probability Estimator to generate a synthetic PET image from Lung CT scan. The PET Generator is a conditional adversarial network (pix2pix) trained on slices containing the Lung from 100 PET-CT scans which were acquired on patients suspected or diagnosed with Lung Cancer. The model performed at a mean squared error of 0.08 when compared in SUV units. The malignancy probability estimator is a 20-layer deep residual convolutional neural network trained on a dataset of 1595 scans from the NLST trial. The model performed produced a ROC of 0.89 when tested on 822 patients. The outputs of the PET Generator provides a background for overlaying outputs of the Malignancy Probability Estimator which together produce the synthetic PET image.


When tested on a dataset of 30 images, the synthetic PET model performed at a mean squared error of 0.08 when compared in SUV units. The malignancy model was independently tested on 350 scans and produced an AUC of 0.89.

A dataset of 22 malignant scans is used to benchmark performance of malignancy detection. Using the CT scan alone, three radiologists had sensitivities of 86%, 81% and 72% in detecting malignant studies. Using the synthetic PET as an additional modality, an increased sensitivity of 95% can be obtained. However, it is important to note that the SUV values detected on the nodules were not correlated with the actual SUV values.


Synthetic PET images can potentially increase the sensitivity of malignant nodule detection from Lung CT images. Such a modality can be easier for radiologists to understand than naive probability heatmaps. Further research is required to investigate the effect of potential biases and investigate appropriate clinical application.