Assessing thermal imagery integration into object detection methods on ground-based and air-based collection platforms

Object detection models commonly deployed on uncrewed aerial systems (UAS) focus on identifying objects in the visible spectrum using Red-Green-Blue (RGB) imagery. However, there is growing interest in fusing RGB with thermal long wave infrared (LWIR) images to increase the performance of object detection machine learning (ML) models. Currently LWIR ML models have received less research attention, especially for both ground- and air-based platforms, leading to a lack of baseline performance metrics evaluating LWIR, RGB and LWIR-RGB fused object detection models. Therefore, this research contributes such quantitative metrics to the literature. The results found that the ground-based blended RGB-LWIR model exhibited superior performance compared to the RGB or LWIR approaches, achieving a mAP of 98.4%. Additionally, the blended RGB-LWIR model was also the only object detection model to work in both day and night conditions, providing superior operational capabilities. This research additionally contributes a novel labelled training dataset of 12,600 images for RGB, LWIR, and RGB-LWIR fused imagery, collected from ground-based and air-based platforms, enabling further multispectral machine-driven object detection research.

An explainable deep vision system for animal classification and detection in trail-camera images with automatic post-deployment retraining

This paper introduces an automated vision system for animal detection in trail-camera images taken from a field under the administration of the Texas Parks and Wildlife Department. As traditional wildlife counting techniques are intrusive and labor intensive to conduct, trail-camera imaging is a comparatively non-intrusive method for capturing wildlife activity. However, given the large volume of images produced from trail-cameras, manual analysis of the images remains time-consuming and inefficient. We implemented a two-stage deep convolutional neural network pipeline to find animal-containing images in the first stage and then process these images to detect birds in the second stage. The animal classification system classifies animal images with more than 87% sensitivity and 96% specificity. The bird detection system achieves better than 93% sensitivity, 92% specificity, and 68% average Intersection-over-Union rate. The entire pipeline processes an image in less than 0.5 seconds as opposed to an average 30 seconds for a human labeler. We also addressed post-deployment issues related to data drift for the animal classification system as image features vary with seasonal changes. This system utilizes an automatic retraining algorithm to detect data drift and update the system. We introduce a novel technique for detecting drifted images and triggering the retraining procedure. Two statistical experiments are also presented to explain the prediction behavior of the animal classification system. These experiments investigate the cues that steers the system towards a particular decision. Statistical hypothesis testing demonstrates that the presence of an animal in the input image significantly contributes to the system's decisions.