Learning Neural Radiance Fields of Forest Structure for Scalable and Fine Monitoring

This work leverages neural radiance fields and remote sensing for forestry applications. Here, we show neural radiance fields offer a wide range of possibilities to improve upon existing remote sensing methods in forest monitoring. We present experiments that demonstrate their potential to: (1) express fine features of forest 3D structure, (2) fuse available remote sensing modalities and (3), improve upon 3D structure derived forest metrics. Altogether, these properties make neural fields an attractive computational tool with great potential to further advance the scalability and accuracy of forest monitoring programs.

Representation Disentaglement via Regularization by Identification

This work focuses on the problem of learning disentangled representations from observational data. Given observations ${\mathbf{x}^{(i)}}$ for $i=1,...,N $ drawn from $p(\mathbf{x}|\mathbf{y})$ with generative variables $\mathbf{y}$ admitting the distribution factorization $p(\mathbf{y}) = \prod_{c} p(\mathbf{y}_c )$ we ask whether learning disentangled representations matching the space of observations with identification guarantees on the posterior $p(\mathbf{z}| \mathbf{x}, \hat{\mathbf{y}}_c)$ for each $c$, is plausible. We argue modern deep representation learning models are ill-posed with collider bias behaviour; a source of bias producing entanglement between generating variables. Under the rubric of causality, we show this issue can be explained and reconciled under the condition of identifiability; attainable under supervision or a weak-form of it. For this, we propose regularization by identification (ReI), a regularization framework defined by the identification of the causal queries involved in the learning problem. Empirical evidence shows that enforcing ReI in a variational framework results in disentangled representations equipped with generalization capabilities to out-of-distribution examples and that aligns nicely with the true expected effect between generating variables and measurement apparatus.

Automated Structural-level Alignment of Multi-view TLS and ALS Point Clouds in Forestry

Access to highly detailed models of heterogeneous forests from the near surface to above the tree canopy at varying scales is of increasing demand as it enables more advanced computational tools for analysis, planning, and ecosystem management. LiDAR sensors available through different scanning platforms including terrestrial, mobile and aerial have become established as one of the primary technologies for forest mapping due to their inherited capability to collect direct, precise and rapid 3D information of a scene. However, their scalability to large forest areas is highly dependent upon use of effective and efficient methods of co-registration of multiple scan sources. Surprisingly, work in forestry in GPS denied areas has mostly resorted to methods of co-registration that use reference based targets (e.g., reflective, marked trees), a process far from scalable in practice. In this work, we propose an effective, targetless and fully automatic method based on an incremental co-registration strategy matching and grouping points according to levels of structural complexity. Empirical evidence shows the method's effectiveness in aligning both TLS-to-TLS and TLS-to-ALS scans under a variety of ecosystem conditions including pre/post fire treatment effects, of interest to forest inventory surveyors.

Learning to Pre-process Laser Induced Breakdown Spectroscopy Signals Without Clean Data

This work tests whether deep neural networks can clean laser induced breakdown spectroscopy (LIBS) signals by using only uncleaned raw measurements. Our view of this problem considers a disentanglement of the effects of the target of interest from those of the nuisance factors (with non-zero mean) by leveraging the vast amounts of redundancies in LIBS data and our proposed learning formulation. This later aims at promoting consistency between repeated measurement views of a target while simultaneously removing consistencies with all other LIBS measurements taken throughout the history of the instrument. Evaluations on real data from the ChemCam instrument onboard the Martian Curiosity rover show a superior performance in cleaning LIBS signals compared to the standard approaches being used by the ChemCam team.

Deep Spectral CNN for Laser Induced Breakdown Spectroscopy

This work proposes a spectral convolutional neural network (CNN) operating on laser induced breakdown spectroscopy (LIBS) signals to learn to (1) disentangle spectral signals from the sources of sensor uncertainty (i.e., pre-process) and (2) get qualitative and quantitative measures of chemical content of a sample given a spectral signal (i.e., calibrate). Once the spectral CNN is trained, it can accomplish either task through a single feed-forward pass, with real-time benefits and without any additional side information requirements including dark current, system response, temperature and detector-to-target range. Our experiments demonstrate that the proposed method outperforms the existing approaches used by the Mars Science Lab for pre-processing and calibration for remote sensing observations from the Mars rover, 'Curiosity'.

Learning Spatial Relationships between Samples of Patent Image Shapes

Binary image based classification and retrieval of documents of an intellectual nature is a very challenging problem. Variations in the binary image generation mechanisms which are subject to the document artisan designer including drawing style, view-point, inclusion of multiple image components are plausible causes for increasing the complexity of the problem. In this work, we propose a method suitable to binary images which bridges some of the successes of deep learning (DL) to alleviate the problems introduced by the aforementioned variations. The method consists on extracting the shape of interest from the binary image and applying a non-Euclidean geometric neural-net architecture to learn the local and global spatial relationships of the shape. Empirical results show that our method is in some sense invariant to the image generation mechanism variations and achieves results outperforming existing methods in a patent image dataset benchmark.

Diagram Image Retrieval using Sketch-Based Deep Learning and Transfer Learning

Resolution of the complex problem of image retrieval for diagram images has yet to be reached. Deep learning methods continue to excel in the fields of object detection and image classification applied to natural imagery. However, the application of such methodologies applied to binary imagery remains limited due to lack of crucial features such as textures,color and intensity information. This paper presents a deep learning based method for image-based search for binary patent images by taking advantage of existing large natural image repositories for image search and sketch-based methods (Sketches are not identical to diagrams, but they do share some characteristics; for example, both imagery types are gray scale (binary), composed of contours, and are lacking in texture). We begin by using deep learning to generate sketches from natural images for image retrieval and then train a second deep learning model on the sketches. We then use our small set of manually labeled patent diagram images via transfer learning to adapt the image search from sketches of natural images to diagrams. Our experiment results show the effectiveness of deep learning with transfer learning for detecting near-identical copies in patent images and querying similar images based on content.

Learning Spatial Relationships between Samples of Image Shapes

Many applications including image based classification and retrieval of scientific and patent documents involve images in which brightness or color is not representative of content. In such cases, it seems intuitive to perform analysis on image shapes rather than on texture variations (i.e., pixel values ). Here, we propose a method that combines sparsely sampling points from image shapes and learning the spatial relationships between the extracted samples that characterize them. A dynamic graph CNN producing a different graph at each layer is trained and used as the learning engine of node and edge features in a classification/retrieval task. Our set of experiments on multiple datasets demonstrate a variety of point sampling sparsities, training-set size, rigid body transformations and scaling; and show that the accuracy of our approach is less likely to degrade due to small training sets or transformations on the data.

Motion Guided LIDAR-camera Autocalibration and Accelerated Depth Super Resolution

In this work we propose a novel motion guided method for automatic and targetless calibration of a LiDAR and camera and use the LiDAR points projected on the image for real-time super-resolution depth estimation. The calibration parameters are estimated by optimizing a cost function that penalizes the difference in the motion vectors obtained from LiDAR and camera data separately. For super-resolution, we propose a simple, yet effective and time efficient formulation that minimizes depth gradients subject to an equality constraint involving measured LiDAR data. We perform experiments on real data obtained in urban environments and demonstrate that the proposed calibration technique is robust and the real-time depth super-resolution reconstruction outperforms the quality of state of the art approaches.