Learning Treatment Plan Representations for Content Based Image Retrieval

Objective: Knowledge based planning (KBP) typically involves training an end-to-end deep learning model to predict dose distributions. However, training end-to-end KBP methods may be associated with practical limitations due to the limited size of medical datasets that are often used. To address these limitations, we propose a content based image retrieval (CBIR) method for retrieving dose distributions of previously planned patients based on anatomical similarity. Approach: Our proposed CBIR method trains a representation model that produces latent space embeddings of a patient's anatomical information. The latent space embeddings of new patients are then compared against those of previous patients in a database for image retrieval of dose distributions. Summary metrics (e.g. dose-volume histogram, conformity index, homogeneity index, etc.) are computed and can then be utilized in subsequent automated planning. All source code for this project is available on github. Main Results: The retrieval performance of various CBIR methods is evaluated on a dataset consisting of both publicly available plans and clinical plans from our institution. This study compares various encoding methods, ranging from simple autoencoders to more recent Siamese networks like SimSiam, and the best performance was observed for the multitask Siamese network. Significance: Applying CBIR to inform subsequent treatment planning potentially addresses many limitations associated with end-to-end KBP. Our current results demonstrate that excellent image retrieval performance can be obtained through slight changes to previously developed Siamese networks. We hope to integrate CBIR into automated planning workflow in future works, potentially through methods like the MetaPlanner framework.

Learning the Physics of Particle Transport via Transformers

Particle physics simulations are the cornerstone of nuclear engineering applications. Among them radiotherapy (RT) is crucial for society, with 50% of cancer patients receiving radiation treatments. For the most precise targeting of tumors, next generation RT treatments aim for real-time correction during radiation delivery, necessitating particle transport algorithms that yield precise dose distributions in sub-second times even in highly heterogeneous patient geometries. This is infeasible with currently available, purely physics based simulations. In this study, we present a data-driven dose calculation algorithm predicting the dose deposited by mono-energetic proton beams for arbitrary energies and patient geometries. Our approach frames particle transport as sequence modeling, where convolutional layers extract important spatial features into tokens and the transformer self-attention mechanism routes information between such tokens in the sequence and a beam energy token. We train our network and evaluate prediction accuracy using computationally expensive but accurate Monte Carlo (MC) simulations, considered the gold standard in particle physics. Our proposed model is 33 times faster than current clinical analytic pencil beam algorithms, improving upon their accuracy in the most heterogeneous and challenging geometries. With a relative error of 0.34% and very high gamma pass rate of 99.59% (1%, 3 mm), it also greatly outperforms the only published similar data-driven proton dose algorithm, even at a finer grid resolution. Offering MC precision 400 times faster, our model could overcome a major obstacle that has so far prohibited real-time adaptive proton treatments and significantly increase cancer treatment efficacy. Its potential to model physics interactions of other particles could also boost heavy ion treatment planning procedures limited by the speed of traditional methods.

Modeling biomedical breathing signals with convolutional deep probabilistic autoencoders

One of the main problems with biomedical signals is the limited amount of patient-specific data and the significant amount of time needed to record a sufficient number of samples for diagnostic and treatment purposes. We explore the use of Variational Autoencoder (VAE) and Adversarial Autoencoder (AAE) algorithms based on one-dimensional convolutional neural networks in order to build generative models able to capture and represent the variability of a set of unlabeled quasi-periodic signals using as few as 10 parameters. Furthermore, we introduce a modified AAE architecture that allows simultaneous semi-supervised classification and generation of different types of signals. Our study is based on physical breathing signals, i.e. time series describing the position of chest markers, generally used to describe respiratory motion. The time series are discretized into a vector of periods, with each period containing 6 time and position values. These vectors can be transformed back into time series through an additional reconstruction neural network and allow to generate extended signals while simplifying the modeling task. The obtained models can be used to generate realistic breathing realizations from patient or population data and to classify new recordings. We show that by incorporating the labels from around 10-15\% of the dataset during training, the model can be guided to group data according to the patient it belongs to, or based on the presence of different types of breathing irregularities such as baseline shifts. Our specific motivation is to model breathing motion during radiotherapy lung cancer treatments, for which the developed model serves as an efficient tool to robustify plans against breathing uncertainties. However, the same methodology can in principle be applied to any other kind of quasi-periodic biomedical signal, representing a generically applicable tool.