Helmholtz-Decomposition and Optical Flow: A new method to characterize GCamP recordings

During deep sleep and under anaesthesia spontaneous patterns of cortical activation frequently take the form of slow travelling waves. Slow wave sleep is an important cognitive state especially because of its relevance for memory consolidation. However, despite extensive research the exact mechanisms are still ill-understood. Novel methods such as high speed widefield imaging of GCamP activity offer new potentials. Here we show how data recorded from transgenic mice under anesthesia can be processed to analyze sources, sinks and patterns of flow. To make the best possible use of the data novel means of data processing are necessary. Therefore, we (1) give a an brief account on processes that play a role in generating slow waves and demonstrate (2) a novel approach to characterize its patterns in GCamP recordings. While slow waves are highly variable, it shows that some are surprisingly similar. To enable quantitative means of analysis and examine the structure of such prototypical events we propose a novel approach for the characterization of slow waves: The Helmholtz-Decomposition of gradient-based Dense Optical Flow of the pixeldense GCamP contrast (df/f). It allows to detect the sources and sinks of activation and discern them from global patterns of neural flow. Aggregated features can be analyzed with variational autoencoders. The results unravel regularities between slow waves and shows how they relate to the experimental conditions. The approach reveals a complex topology of different features in latent slow wave space and identifies prototypical examples for each stage.

A differentiable Gaussian Prototype Layer for explainable Segmentation

We introduce a Gaussian Prototype Layer for gradient-based prototype learning and demonstrate two novel network architectures for explainable segmentation one of which relies on region proposals. Both models are evaluated on agricultural datasets. While Gaussian Mixture Models (GMMs) have been used to model latent distributions of neural networks before, they are typically fitted using the EM algorithm. Instead, the proposed prototype layer relies on gradient-based optimization and hence allows for end-to-end training. This facilitates development and allows to use the full potential of a trainable deep feature extractor. We show that it can be used as a novel building block for explainable neural networks. We employ our Gaussian Prototype Layer in (1) a model where prototypes are detected in the latent grid and (2) a model inspired by Fast-RCNN with SLIC superpixels as region proposals. The earlier achieves a similar performance as compared to the state-of-the art while the latter has the benefit of a more precise prototype localization that comes at the cost of slightly lower accuracies. By introducing a gradient-based GMM layer we combine the benefits of end-to-end training with the simplicity and theoretical foundation of GMMs which will allow to adapt existing semi-supervised learning strategies for prototypical part models in future.

But that's not why: Inference adjustment by interactive prototype deselection

Despite significant advances in machine learning, decision-making of artificial agents is still not perfect and often requires post-hoc human interventions. If the prediction of a model relies on unreasonable factors it is desirable to remove their effect. Deep interactive prototype adjustment enables the user to give hints and correct the model's reasoning. In this paper, we demonstrate that prototypical-part models are well suited for this task as their prediction is based on prototypical image patches that can be interpreted semantically by the user. It shows that even correct classifications can rely on unreasonable prototypes that result from confounding variables in a dataset. Hence, we propose simple yet effective interaction schemes for inference adjustment: The user is consulted interactively to identify faulty prototypes. Non-object prototypes can be removed by prototype masking or a custom mode of deselection training. Interactive prototype rejection allows machine learning na\"{i}ve users to adjust the logic of reasoning without compromising the accuracy.