Combiner and HyperCombiner Networks: Rules to Combine Multimodality MR Images for Prostate Cancer Localisation

One of the distinct characteristics in radiologists' reading of multiparametric prostate MR scans, using reporting systems such as PI-RADS v2.1, is to score individual types of MR modalities, T2-weighted, diffusion-weighted, and dynamic contrast-enhanced, and then combine these image-modality-specific scores using standardised decision rules to predict the likelihood of clinically significant cancer. This work aims to demonstrate that it is feasible for low-dimensional parametric models to model such decision rules in the proposed Combiner networks, without compromising the accuracy of predicting radiologic labels: First, it is shown that either a linear mixture model or a nonlinear stacking model is sufficient to model PI-RADS decision rules for localising prostate cancer. Second, parameters of these (generalised) linear models are proposed as hyperparameters, to weigh multiple networks that independently represent individual image modalities in the Combiner network training, as opposed to end-to-end modality ensemble. A HyperCombiner network is developed to train a single image segmentation network that can be conditioned on these hyperparameters during inference, for much improved efficiency. Experimental results based on data from 850 patients, for the application of automating radiologist labelling multi-parametric MR, compare the proposed combiner networks with other commonly-adopted end-to-end networks. Using the added advantages of obtaining and interpreting the modality combining rules, in terms of the linear weights or odds-ratios on individual image modalities, three clinical applications are presented for prostate cancer segmentation, including modality availability assessment, importance quantification and rule discovery.

The impact of using voxel-level segmentation metrics on evaluating multifocal prostate cancer localisation

Dice similarity coefficient (DSC) and Hausdorff distance (HD) are widely used for evaluating medical image segmentation. They have also been criticised, when reported alone, for their unclear or even misleading clinical interpretation. DSCs may also differ substantially from HDs, due to boundary smoothness or multiple regions of interest (ROIs) within a subject. More importantly, either metric can also have a nonlinear, non-monotonic relationship with outcomes based on Type 1 and 2 errors, designed for specific clinical decisions that use the resulting segmentation. Whilst cases causing disagreement between these metrics are not difficult to postulate. This work first proposes a new asymmetric detection metric, adapting those used in object detection, for planning prostate cancer procedures. The lesion-level metrics is then compared with the voxel-level DSC and HD, whereas a 3D UNet is used for segmenting lesions from multiparametric MR (mpMR) images. Based on experimental results we report pairwise agreement and correlation 1) between DSC and HD, and 2) between voxel-level DSC and recall-controlled precision at lesion-level, with Cohen's [0.49, 0.61] and Pearson's [0.66, 0.76] (p-values}<0.001) at varying cut-offs. However, the differences in false-positives and false-negatives, between the actual errors and the perceived counterparts if DSC is used, can be as high as 152 and 154, respectively, out of the 357 test set lesions. We therefore carefully conclude that, despite of the significant correlations, voxel-level metrics such as DSC can misrepresent lesion-level detection accuracy for evaluating localisation of multifocal prostate cancer and should be interpreted with caution.

Segmentation of common and internal carotid arteries from 3D ultrasound images using adaptive triple U-Net

Objective: Vessel-wall-volume (VWV) and localized vessel-wall-thickness (VWT) measured from 3D ultrasound (US) carotid images are sensitive to anti-atherosclerotic effects of medical/dietary treatments. VWV and VWT measurements require the lumen-intima (LIB) and media-adventitia boundaries (MAB) at the common and internal carotid arteries (CCA and ICA). However, most existing segmentation techniques were capable of automating only CCA segmentation. An approach capable of segmenting the MAB and LIB from the CCA and ICA was required to accelerate VWV and VWT quantification. Methods: Segmentation for CCA and ICA were performed independently using the proposed two-channel U-Net, which was driven by a novel loss function known as the adaptive triple Dice loss (ADTL). A test-time augmentation (TTA) approach is used, in which segmentation was performed three times based on axial images and its flipped versions; the final segmentation was generated by pixel-wise majority voting. Results: Experiments involving 224 3DUS volumes produce a Dice-similarity-coefficient (DSC) of 95.1%$\pm$4.1% and 91.6%$\pm$6.6% for the MAB and LIB, in the CCA, respectively, and 94.2%$\pm$3.3% and 89.0%$\pm$8.1% for the MAB and LIB, in the ICA, respectively. TTA and ATDL independently contributed to a statistically significant improvement to all boundaries except the LIB in ICA. The total time required to segment the entire 3DUS volume (CCA+ICA) is 1.4s. Conclusion: The proposed two-channel U-Net with ADTL and TTA can segment the CCA and ICA accurately and efficiently from the 3DUS volume. Significance: Our approach has the potential to accelerate the transition of 3DUS measurements of carotid atherosclerosis to clinical research.

Area-preserving mapping of 3D ultrasound carotid artery images using density-equalizing reference map

Carotid atherosclerosis is a focal disease at the bifurcations of the carotid artery. To quantitatively monitor the local changes in the vessel-wall-plus-plaque thickness (VWT) and compare the VWT distributions for different patients or for the same patients at different ultrasound scanning sessions, a mapping technique is required to adjust for the geometric variability of different carotid artery models. In this work, we propose a novel method called density-equalizing reference map (DERM) for mapping 3D carotid surfaces to a standardized 2D carotid template, with an emphasis on preserving the local geometry of the carotid surface by minimizing the local area distortion. The initial map was generated by a previously described arc-length scaling (ALS) mapping method, which projects a 3D carotid surface onto a 2D non-convex L-shaped domain. A smooth and area-preserving flattened map was subsequently constructed by deforming the ALS map using the proposed algorithm that combines the density-equalizing map and the reference map techniques. This combination allows, for the first time, one-to-one mapping from a 3D surface to a standardized non-convex planar domain in an area-preserving manner. Evaluations using 20 carotid surface models show that the proposed method reduced the area distortion of the flattening maps by over 80% as compared to the ALS mapping method.