A dual branch deep neural network for classification and detection in mammograms

Objective: This paper suggests a novel deep learning architecture for multi-class classification and localization of abnormalities in mammograms. We first assume a weakly supervised setting and present a new approach with data driven decisions. Next, we extend this method to a semi-supervised setting that engages a small set of local annotations. Methods: This novel network combines two learning branches with region-level classification and region ranking, explicitly handling the common normal images without any finding. Our method enables detection of abnormalities in full mammogram resolution for both weakly and semi-supervised settings. A novel objective function allows engagement of local annotations into the model. Results: We present the impact of our schemes with several performance measures for classification and localization, to evaluate the cost effectiveness of lesion annotation effort. Our evaluation is mainly made over a large multi-center mammography dataset of $\sim$3,000 mammograms with various findings. The results for weakly supervised learning show improvement of 4% in AUC, 10-17% in partial AUC (AUC at high sensitivity) and 8-15% in specificity at 0.85 sensitivity, compared to a previous approach. Local annotation of only 5% of the mammograms further boosts performance by 8.2% in specificity and 30% in detection. Conclusion: Lack of laborious local annotations for supervised learning can be addressed by a weakly supervised method that can leverage a subset of locally annotated data. Significance: Weakly and semi-supervised methods coupled with detection suggest a cost effective and explainable model to be adopted by radiologists in the field.

Classification and Detection in Mammograms with Weak Supervision via Dual Branch Deep Neural Net

The high cost of generating expert annotations, poses a strong limitation for supervised machine learning methods in medical imaging. Weakly supervised methods may provide a solution to this tangle. In this study, we propose a novel deep learning architecture for multi-class classification of mammograms according to the severity of their containing anomalies, having only a global tag over the image. The suggested scheme further allows localization of the different types of findings in full resolution. The new scheme contains a dual branch network that combines region-level classification with region ranking. We evaluate our method on a large multi-center mammography dataset including $\sim$3,000 mammograms with various anomalies and demonstrate the advantages of the proposed method over a previous weakly-supervised strategy.