A Semi-supervised Object Detection Algorithm for Underwater Imagery

Detection of artificial objects from underwater imagery gathered by Autonomous Underwater Vehicles (AUVs) is a key requirement for many subsea applications. Real-world AUV image datasets tend to be very large and unlabelled. Furthermore, such datasets are typically imbalanced, containing few instances of objects of interest, particularly when searching for unusual objects in a scene. It is therefore, difficult to fit models capable of reliably detecting these objects. Given these factors, we propose to treat artificial objects as anomalies and detect them through a semi-supervised framework based on Variational Autoencoders (VAEs). We develop a method which clusters image data in a learned low-dimensional latent space and extracts images that are likely to contain anomalous features. We also devise an anomaly score based on extracting poorly reconstructed regions of an image. We demonstrate that by applying both methods on large image datasets, human operators can be shown candidate anomalous samples with a low false positive rate to identify objects of interest. We apply our approach to real seafloor imagery gathered by an AUV and evaluate its sensitivity to the dimensionality of the latent representation used by the VAE. We evaluate the precision-recall tradeoff and demonstrate that by choosing an appropriate latent dimensionality and threshold, we are able to achieve an average precision of 0.64 on unlabelled datasets.

Improved Benthic Classification using Resolution Scaling and SymmNet Unsupervised Domain Adaptation

Autonomous Underwater Vehicles (AUVs) conduct regular visual surveys of marine environments to characterise and monitor the composition and diversity of the benthos. The use of machine learning classifiers for this task is limited by the low numbers of annotations available and the many fine-grained classes involved. In addition to these challenges, there are domain shifts between image sets acquired during different AUV surveys due to changes in camera systems, imaging altitude, illumination and water column properties leading to a drop in classification performance for images from a different survey where some or all these elements may have changed. This paper proposes a framework to improve the performance of a benthic morphospecies classifier when used to classify images from a different survey compared to the training data. We adapt the SymmNet state-of-the-art Unsupervised Domain Adaptation method with an efficient bilinear pooling layer and image scaling to normalise spatial resolution, and show improved classification accuracy. We test our approach on two datasets with images from AUV surveys with different imaging payloads and locations. The results show that generic domain adaptation can be enhanced to produce a significant increase in accuracy for images from an AUV survey that differs from the training images.

Maximising Wrenches for Kinematically Redundant Systems with Experiments on UVMS

This paper presents methods for finding optimal configurations and actuator forces/torques to maximise contact wrenches in a desired direction for Underwater Vehicles Manipulator Systems (UVMS). The wrench maximisation problem is formulated as a linear programming problem, and the optimal configuration is solved as a bi-level optimisation in the parameterised redundancy space. We additionally consider the cases of one or more manipulators with multiple contact forces, maximising wrench capability while tracking a trajectory, and generating large wrench impulses using dynamic motions. We look at the specific cases of maximising force to lift a heavy load, and maximising torque during a valve turning operation. Extensive experimental results are presented using an underwater robotic platform equipped with a 4DOF manipulator, and show significant increases in wrench capability compared to existing methods for UVMS.

GeoCLR: Georeference Contrastive Learning for Efficient Seafloor Image Interpretation

This paper describes Georeference Contrastive Learning of visual Representation (GeoCLR) for efficient training of deep-learning Convolutional Neural Networks (CNNs). The method leverages georeference information by generating a similar image pair using images taken of nearby locations, and contrasting these with an image pair that is far apart. The underlying assumption is that images gathered within a close distance are more likely to have similar visual appearance, where this can be reasonably satisfied in seafloor robotic imaging applications where image footprints are limited to edge lengths of a few metres and are taken so that they overlap along a vehicle's trajectory, whereas seafloor substrates and habitats have patch sizes that are far larger. A key advantage of this method is that it is self-supervised and does not require any human input for CNN training. The method is computationally efficient, where results can be generated between dives during multi-day AUV missions using computational resources that would be accessible during most oceanic field trials. We apply GeoCLR to habitat classification on a dataset that consists of ~86k images gathered using an Autonomous Underwater Vehicle (AUV). We demonstrate how the latent representations generated by GeoCLR can be used to efficiently guide human annotation efforts, where the semi-supervised framework improves classification accuracy by an average of 11.8 % compared to state-of-the-art transfer learning using the same CNN and equivalent number of human annotations for training.

Feature Space Exploration For Planning Initial Benthic AUV Surveys

Special-purpose Autonomous Underwater Vehicles (AUVs) are utilised for benthic (seafloor) surveys, where the vehicle collects optical imagery of near the seafloor. Due to the small-sensor footprint of the cameras and the vast areas to be surveyed, these AUVs can not feasibly full coverage of areas larger than a few tens of thousands of square meters. Therefore AUV paths which sample sparsely, yet effectively, the survey areas are necessary. Broad scale acoustic bathymetric data is ready available over large areas, and often is a useful prior of seafloor cover. As such, prior bathymetry can be used to guide AUV data collection. This research proposes methods for planning initial AUV surveys that efficiently explore a feature space representation of the bathymetry, in order to sample from a diverse set of bathymetric terrain. This will enable the AUV to visit areas that likely contain unique habitats and are representative of the entire survey site. The suitability of these methods to plan AUV surveys is evaluated based on the coverage of the feature space and also the ability to visit all classes of benthic habitat on the initial dive. This is a valuable tool for AUV surveys as it increases the utility of initial dives. It also delivers a comprehensive training set to learn a relationship between acoustic bathymetry and visually-derived seafloor classifications.

Towards Adaptive Benthic Habitat Mapping

Autonomous Underwater Vehicles (AUVs) are increasingly being used to support scientific research and monitoring studies. One such application is in benthic habitat mapping where these vehicles collect seafloor imagery that complements broadscale bathymetric data collected using sonar. Using these two data sources, the relationship between remotely-sensed acoustic data and the sampled imagery can be learned, creating a habitat model. As the areas to be mapped are often very large and AUV systems collecting seafloor imagery can only sample from a small portion of the survey area, the information gathered should be maximised for each deployment. This paper illustrates how the habitat models themselves can be used to plan more efficient AUV surveys by identifying where to collect further samples in order to most improve the habitat model. A Bayesian neural network is used to predict visually-derived habitat classes when given broad-scale bathymetric data. This network can also estimate the uncertainty associated with a prediction, which can be deconstructed into its aleatoric (data) and epistemic (model) components. We demonstrate how these structured uncertainty estimates can be utilised to improve the model with fewer samples. Such adaptive approaches to benthic surveys have the potential to reduce costs by prioritizing further sampling efforts. We illustrate the effectiveness of the proposed approach using data collected by an AUV on offshore reefs in Tasmania, Australia.

Sound Source Localization in a Multipath Environment Using Convolutional Neural Networks

The propagation of sound in a shallow water environment is characterized by boundary reflections from the sea surface and sea floor. These reflections result in multiple (indirect) sound propagation paths, which can degrade the performance of passive sound source localization methods. This paper proposes the use of convolutional neural networks (CNNs) for the localization of sources of broadband acoustic radiated noise (such as motor vessels) in shallow water multipath environments. It is shown that CNNs operating on cepstrogram and generalized cross-correlogram inputs are able to more reliably estimate the instantaneous range and bearing of transiting motor vessels when the source localization performance of conventional passive ranging methods is degraded. The ensuing improvement in source localization performance is demonstrated using real data collected during an at-sea experiment.

Adaptive Path Planning for Depth Constrained Bathymetric Mapping with an Autonomous Surface Vessel

This paper describes the design, implementation and testing of a suite of algorithms to enable depth constrained autonomous bathymetric (underwater topography) mapping by an Autonomous Surface Vessel (ASV). Given a target depth and a bounding polygon, the ASV will find and follow the intersection of the bounding polygon and the depth contour as modeled online with a Gaussian Process (GP). This intersection, once mapped, will then be used as a boundary within which a path will be planned for coverage to build a map of the Bathymetry. Methods for sequential updates to GP's are described allowing online fitting, prediction and hyper-parameter optimisation on a small embedded PC. New algorithms are introduced for the partitioning of convex polygons to allow efficient path planning for coverage. These algorithms are tested both in simulation and in the field with a small twin hull differential thrust vessel built for the task.