Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification

The development of deep segmentation models for computational pathology (CPath) can help foster the investigation of interpretable morphological biomarkers. Yet, there is a major bottleneck in the success of such approaches because supervised deep learning models require an abundance of accurately labelled data. This issue is exacerbated in the field of CPath because the generation of detailed annotations usually demands the input of a pathologist to be able to distinguish between different tissue constructs and nuclei. Manually labelling nuclei may not be a feasible approach for collecting large-scale annotated datasets, especially when a single image region can contain thousands of different cells. However, solely relying on automatic generation of annotations will limit the accuracy and reliability of ground truth. Therefore, to help overcome the above challenges, we propose a multi-stage annotation pipeline to enable the collection of large-scale datasets for histology image analysis, with pathologist-in-the-loop refinement steps. Using this pipeline, we generate the largest known nuclear instance segmentation and classification dataset, containing nearly half a million labelled nuclei in H&E stained colon tissue. We have released the dataset and encourage the research community to utilise it to drive forward the development of downstream cell-based models in CPath.

Cells are Actors: Social Network Analysis with Classical ML for SOTA Histology Image Classification

Digitization of histology images and the advent of new computational methods, like deep learning, have helped the automatic grading of colorectal adenocarcinoma cancer (CRA). Present automated CRA grading methods, however, usually use tiny image patches and thus fail to integrate the entire tissue micro-architecture for grading purposes. To tackle these challenges, we propose to use a statistical network analysis method to describe the complex structure of the tissue micro-environment by modelling nuclei and their connections as a network. We show that by analyzing only the interactions between the cells in a network, we can extract highly discriminative statistical features for CRA grading. Unlike other deep learning or convolutional graph-based approaches, our method is highly scalable (can be used for cell networks consist of millions of nodes), completely explainable, and computationally inexpensive. We create cell networks on a broad CRC histology image dataset, experiment with our method, and report state-of-the-art performance for the prediction of three-class CRA grading.

Semantic annotation for computational pathology: Multidisciplinary experience and best practice recommendations

Recent advances in whole slide imaging (WSI) technology have led to the development of a myriad of computer vision and artificial intelligence (AI) based diagnostic, prognostic, and predictive algorithms. Computational Pathology (CPath) offers an integrated solution to utilize information embedded in pathology WSIs beyond what we obtain through visual assessment. For automated analysis of WSIs and validation of machine learning (ML) models, annotations at the slide, tissue and cellular levels are required. The annotation of important visual constructs in pathology images is an important component of CPath projects. Improper annotations can result in algorithms which are hard to interpret and can potentially produce inaccurate and inconsistent results. Despite the crucial role of annotations in CPath projects, there are no well-defined guidelines or best practices on how annotations should be carried out. In this paper, we address this shortcoming by presenting the experience and best practices acquired during the execution of a large-scale annotation exercise involving a multidisciplinary team of pathologists, ML experts and researchers as part of the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a real-world case study along with examples of different types of annotations, diagnostic algorithm, annotation data dictionary and annotation constructs. The analyses reported in this work highlight best practice recommendations that can be used as annotation guidelines over the lifecycle of a CPath project.

Automatic Recognition of the Supraspinatus Tendinopathy from Ultrasound Images using Convolutional Neural Networks

Tendon injuries like tendinopathies, full and partial thickness tears are prevalent, and the supraspinatus tendon (SST) is the most vulnerable ones in the rotator cuff. Early diagnosis of SST tendinopathies is of high importance and hard to achieve using ultrasound imaging. In this paper, an automatic tendinopathy recognition framework based on convolutional neural networks has been proposed to assist the diagnosis. This framework has two essential parts of tendon segmentation and classification. Tendon segmentation is done through a novel network, NASUNet, which follows an encoder-decoder architecture paradigm and utilizes a multi-scale Enlarging cell. Moreover, a general classification pipeline has been proposed for tendinopathy recognition, which supports different base models as the feature extractor engine. Two feature maps comprising positional information of the tendon region have been introduced as the network input to make the classification network spatial-aware. To evaluate the tendinopathy recognition system, a data set consisting of 100 SST ultrasound images have been acquired, in which tendinopathy cases are double-verified by magnetic resonance imaging. In both segmentation and classification tasks, lack of training data has been compensated by incorporating knowledge transferring, transfer learning, and data augmentation techniques. In cross-validation experiments, the proposed tendinopathy recognition model achieves 91% accuracy, 86.67% sensitivity, and 92.86% specificity, showing state-of-the-art performance against other models.

NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images

Object Segmentation is an important step in the work-flow of computational pathology. Deep learning based models as the best forming models require huge amount of labeled data for precise and reliable prediction. However, collecting labeled data is expensive, because it necessarily involves expert knowledge. This is perhaps best illustrated by medical tasks where measurements call for expensive machinery and labels are the fruit of a time-consuming analysis that draws from multiple human experts. As nuclei, cells and glands are fundamental objects for downstream analysis in histology, in this paper we propose a simple CNN-based approach to speed up collecting segmentation annotation for these objects by utilizing minimum input from an annotator. We show for nuclei and cells as small objects, one click inside objects is enough to have precise annotation. For glands as large objects, providing a squiggle to show the extend of gland can guide the model to outline the exact boundaries. This supervisory signals are fed to network as an auxiliary channels along with RGB channels. With detailed experiments, we show that our approach is generalizable, robust against variations in the user input and that it can be used to obtain annotations for completely different domains. Practically, a model trained on the masks generated by NuClick could achieve first rank in LYON19 challenge. Furthermore, as the output of our framework, we release two data-sets: 1) a dataset of lymphocyte annotations within IHC images and 2) a dataset of WBCs annotated in blood sample images.

PanNuke Dataset Extension, Insights and Baselines

The emerging area of computational pathology (CPath) is ripe ground for the application of deep learning (DL) methods to healthcare due to the sheer volume of raw pixel data in whole-slide images (WSIs) of cancerous tissue slides. However, it is imperative for the DL algorithms relying on nuclei-level details to be able to cope with data from `the clinical wild', which tends to be quite challenging. We study, and extend recently released PanNuke dataset consisting of ~200,000 nuclei categorized into 5 clinically important classes for the challenging tasks of segmenting and classifying nuclei in WSIs. Previous pan-cancer datasets consisted of only up to 9 different tissues and up to 21,000 unlabeled nuclei and just over 24,000 labeled nuclei with segmentation masks. PanNuke consists of 19 different tissue types that have been semi-automatically annotated and quality controlled by clinical pathologists, leading to a dataset with statistics similar to the clinical wild and with minimal selection bias. We study the performance of segmentation and classification models when applied to the proposed dataset and demonstrate the application of models trained on PanNuke to whole-slide images. We provide comprehensive statistics about the dataset and outline recommendations and research directions to address the limitations of existing DL tools when applied to real-world CPath applications.