Moving from 2D to 3D: volumetric medical image classification for rectal cancer staging

Volumetric images from Magnetic Resonance Imaging (MRI) provide invaluable information in preoperative staging of rectal cancer. Above all, accurate preoperative discrimination between T2 and T3 stages is arguably both the most challenging and clinically significant task for rectal cancer treatment, as chemo-radiotherapy is usually recommended to patients with T3 (or greater) stage cancer. In this study, we present a volumetric convolutional neural network to accurately discriminate T2 from T3 stage rectal cancer with rectal MR volumes. Specifically, we propose 1) a custom ResNet-based volume encoder that models the inter-slice relationship with late fusion (i.e., 3D convolution at the last layer), 2) a bilinear computation that aggregates the resulting features from the encoder to create a volume-wise feature, and 3) a joint minimization of triplet loss and focal loss. With MR volumes of pathologically confirmed T2/T3 rectal cancer, we perform extensive experiments to compare various designs within the framework of residual learning. As a result, our network achieves an AUC of 0.831, which is higher than the reported accuracy of the professional radiologist groups. We believe this method can be extended to other volume analysis tasks

Multi-Task Learning with a Fully Convolutional Network for Rectum and Rectal Cancer Segmentation

In a rectal cancer treatment planning, the location of rectum and rectal cancer plays an important role. The aim of this study is to propose a fully automatic method to segment both rectum and rectal cancer with axial T2-weighted magnetic resonance images. We present a fully convolutional network for multi-task learning to segment both rectum and rectal cancer. Moreover, we propose an assessment method based on bias-variance decomposition to visualize and measure the regional model robustness of a segmentation network. In addition, we suggest a novel augmentation method which can improve the segmentation performance and reduce the training time. Our proposed method not only is computationally efficient due to its fully convolutional nature but also outperforms the current state-of-the-art in rectal cancer segmentation. It also shows high accuracy in rectum segmentation, for which no previous studies exist. We conclude that rectum information benefits the training of rectal cancer segmentation model, especially concerning model variance.

Fully Convolutional Network-based Multi-Task Learning for Rectum and Rectal Cancer Segmentation

In this study, we present a fully automatic method to segment both rectum and rectal cancer based on Deep Neural Networks (DNNs) with axial T2-weighted Magnetic Resonance images. Clinically, the relative location between rectum and rectal cancer plays an important role in cancer treatment planning. Such a need motivates us to propose a fully convolutional architecture for Multi-Task Learning (MTL) to segment both rectum and rectal cancer. Moreover, we propose a bias-variance decomposition-based method which can visualize and assess regional robustness of the segmentation model. In addition, we also suggest a novel augmentation method which can improve the segmentation performance as well as reduce the training time. Overall, our proposed method is not only computationally efficient due to its fully convolutional nature but also outperforms the current state-of-the-art for rectal cancer segmentation. It also scores high accuracy in rectum segmentation without any prior study reported. Moreover, we conclude that supplementing rectum information benefits the rectal cancer segmentation model, especially in model variance.