Comprehensive Pathological Image Segmentation via Teacher Aggregation for Tumor Microenvironment Analysis

The tumor microenvironment (TME) plays a crucial role in cancer progression and treatment response, yet current methods for its comprehensive analysis in H&E-stained tissue slides face significant limitations in the diversity of tissue cell types and accuracy. Here, we present PAGET (Pathological image segmentation via AGgrEgated Teachers), a new knowledge distillation approach that integrates multiple segmentation models while considering the hierarchical nature of cell types in the TME. By leveraging a unique dataset created through immunohistochemical restaining techniques and existing segmentation models, PAGET enables simultaneous identification and classification of 14 key TME components. We demonstrate PAGET's ability to perform rapid, comprehensive TME segmentation across various tissue types and medical institutions, advancing the quantitative analysis of tumor microenvironments. This method represents a significant step forward in enhancing our understanding of cancer biology and supporting precise clinical decision-making from large-scale histopathology images.

Multimodal Whole Slide Foundation Model for Pathology

The field of computational pathology has been transformed with recent advances in foundation models that encode histopathology region-of-interests (ROIs) into versatile and transferable feature representations via self-supervised learning (SSL). However, translating these advancements to address complex clinical challenges at the patient and slide level remains constrained by limited clinical data in disease-specific cohorts, especially for rare clinical conditions. We propose TITAN, a multimodal whole slide foundation model pretrained using 335,645 WSIs via visual self-supervised learning and vision-language alignment with corresponding pathology reports and 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology. Without any finetuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis. We evaluate TITAN on diverse clinical tasks and find that TITAN outperforms both ROI and slide foundation models across machine learning settings such as linear probing, few-shot and zero-shot classification, rare cancer retrieval and cross-modal retrieval, and pathology report generation.

Pathology Foundation Models

Pathology has played a crucial role in the diagnosis and evaluation of patient tissue samples obtained from surgeries and biopsies for many years. The advent of Whole Slide Scanners and the development of deep learning technologies have significantly advanced the field, leading to extensive research and development in pathology AI (Artificial Intelligence). These advancements have contributed to reducing the workload of pathologists and supporting decision-making in treatment plans. Recently, large-scale AI models known as Foundation Models (FMs), which are more accurate and applicable to a wide range of tasks compared to traditional AI, have emerged, and expanded their application scope in the healthcare field. Numerous FMs have been developed in pathology, and there are reported cases of their application in various tasks, such as disease diagnosis, rare cancer diagnosis, patient survival prognosis prediction, biomarker expression prediction, and the scoring of immunohistochemical expression intensity. However, several challenges remain for the clinical application of FMs, which healthcare professionals, as users, must be aware of. Research is ongoing to address these challenges. In the future, it is expected that the development of Generalist Medical AI, which integrates pathology FMs with FMs from other medical domains, will progress, leading to the effective utilization of AI in real clinical settings to promote precision and personalized medicine.

Machine learning methods for histopathological image analysis

Abundant accumulation of digital histopathological images has led to the increased demand for their analysis, such as computer-aided diagnosis using machine learning techniques. However, digital pathological images and related tasks have some issues to be considered. In this mini-review, we introduce the application of digital pathological image analysis using machine learning algorithms, address some problems specific to such analysis, and propose possible solutions.