Annolid: Annotate, Segment, and Track Anything You Need

Annolid is a deep learning-based software package designed for the segmentation, labeling, and tracking of research targets within video files, focusing primarily on animal behavior analysis. Based on state-of-the-art instance segmentation methods, Annolid now harnesses the Cutie video object segmentation model to achieve resilient, markerless tracking of multiple animals from single annotated frames, even in environments in which they may be partially or entirely concealed by environmental features or by one another. Our integration of Segment Anything and Grounding-DINO strategies additionally enables the automatic masking and segmentation of recognizable animals and objects by text command, removing the need for manual annotation. Annolid's comprehensive approach to object segmentation flexibly accommodates a broad spectrum of behavior analysis applications, enabling the classification of diverse behavioral states such as freezing, digging, pup huddling, and social interactions in addition to the tracking of animals and their body parts.

Automated Behavioral Analysis Using Instance Segmentation

Animal behavior analysis plays a crucial role in various fields, such as life science and biomedical research. However, the scarcity of available data and the high cost associated with obtaining a large number of labeled datasets pose significant challenges. In this research, we propose a novel approach that leverages instance segmentation-based transfer learning to address these issues. By capitalizing on fine-tuning the classification head of the instance segmentation network, we enable the tracking of multiple animals and facilitate behavior analysis in laboratory-recorded videos. To demonstrate the effectiveness of our method, we conducted a series of experiments, revealing that our approach achieves exceptional performance levels, comparable to human capabilities, across a diverse range of animal behavior analysis tasks. Moreover, we emphasize the practicality of our solution, as it requires only a small number of labeled images for training. To facilitate the adoption and further development of our method, we have developed an open-source implementation named Annolid (An annotation and instance segmentation-based multiple animal tracking and behavior analysis package). The codebase is publicly available on GitHub at https://github.com/cplab/annolid. This resource serves as a valuable asset for researchers and practitioners interested in advancing animal behavior analysis through state-of-the-art techniques.

Signal Conditioning for Learning in the Wild

The mammalian olfactory system learns rapidly from very few examples, presented in unpredictable online sequences, and then recognizes these learned odors under conditions of substantial interference without exhibiting catastrophic forgetting. We have developed a brain-mimetic algorithm that replicates these properties, provided that sensory inputs adhere to a common statistical structure. However, in natural, unregulated environments, this constraint cannot be assured. We here present a series of signal conditioning steps, inspired by the mammalian olfactory system, that transform diverse sensory inputs into a regularized statistical structure to which the learning network can be tuned. This pre-processing enables a single instantiated network to be applied to widely diverse classification tasks and datasets - here including gas sensor data, remote sensing from spectral characteristics, and multi-label hierarchical identification of wild species - without adjusting network hyperparameters.

Rapid online learning and robust recall in a neuromorphic olfactory circuit

The mammalian olfactory system learns new odors rapidly, exhibits negligible interference among odor memories, and identifies known odors under challenging conditions. The mechanisms by which it does so are unknown. We here present a general theory for odor learning and identification under noise in the olfactory system, and demonstrate its efficacy using a neuromorphic model of the olfactory bulb. As with biological olfaction, the spike timing-based algorithm utilizes distributed, event-driven computations and rapid online learning. Localized spike timing-dependent plasticity rules are employed iteratively over sequential gamma-frequency packets to construct odor representations from the activity of chemosensor arrays mounted in a wind tunnel. Learned odors then are reliably identified despite strong destructive interference. Noise resistance is enhanced by neuromodulation and contextual priming. Lifelong learning capabilities are enabled by adult neurogenesis. The algorithm is applicable to any signal identification problem in which high-dimensional signals are embedded in unknown backgrounds.