Deep Representation Learning of Tissue Metabolome and Computed Tomography Images Annotates Non-invasive Classification and Prognosis Prediction of NSCLC

The rich chemical information from tissue metabolomics provides a powerful means to elaborate tissue physiology or tumor characteristics at cellular and tumor microenvironment levels. However, the process of obtaining such information requires invasive biopsies, is costly, and can delay clinical patient management. Conversely, computed tomography (CT) is a clinical standard of care but does not intuitively harbor histological or prognostic information. Furthermore, the ability to embed metabolome information into CT to subsequently use the learned representation for classification or prognosis has yet to be described. This study develops a deep learning-based framework -- tissue-metabolomic-radiomic-CT (TMR-CT) by combining 48 paired CT images and tumor/normal tissue metabolite intensities to generate ten image embeddings to infer metabolite-derived representation from CT alone. In clinical NSCLC settings, we ascertain whether TMR-CT achieves state-of-the-art results in solving histology classification/prognosis tasks in an unseen international CT dataset of 742 patients. TMR-CT non-invasively determines histological classes - adenocarcinoma/ squamous cell carcinoma with an F1-score=0.78 and further asserts patients' prognosis with a c-index=0.72, surpassing the performance of radiomics models and clinical features. Additionally, our work shows the potential to generate informative biology-inspired CT-led features to explore connections between hard-to-obtain tissue metabolic profiles and routine lesion-derived image data.

Development of a Multi-Task Learning V-Net for Pulmonary Lobar Segmentation on Computed Tomography and Application to Diseased Lungs

Automated lobar segmentation allows regional evaluation of lung disease and is important for diagnosis and therapy planning. Advanced statistical workflows permitting such evaluation is a needed area within respiratory medicine; their adoption remains slow, with poor workflow accuracy. Diseased lung regions often produce high-density zones on CT images, limiting an algorithm's execution to specify damaged lobes due to oblique or lacking fissures. This impact motivated developing an improved machine learning method to segment lung lobes that utilises tracheobronchial tree information to enhance segmentation accuracy through the algorithm's spatial familiarity to define lobar extent more accurately. The method undertakes parallel segmentation of lobes and auxiliary tissues simultaneously by employing multi-task learning (MTL) in conjunction with V-Net-attention, a popular convolutional neural network in the imaging realm. In keeping with the model's adeptness for better generalisation, high performance was retained in an external dataset of patients with four distinct diseases: severe lung cancer, COVID-19 pneumonitis, collapsed lungs and Chronic Obstructive Pulmonary Disease (COPD), even though the training data included none of these cases. The benefit of our external validation test is specifically relevant since our choice includes those patients who have diagnosed lung disease with associated radiological abnormalities. To ensure equal rank is given to all segmentations in the main task we report the following performance (Dice score) on a per-segment basis: normal lungs 0.97, COPD 0.94, lung cancer 0.94, COVID-19 pneumonitis 0.94 and collapsed lung 0.92, all at p<0.05. Even segmenting lobes with large deformations on CT images, the model maintained high accuracy. The approach can be readily adopted in the clinical setting as a robust tool for radiologists.