Toward Semantic Scene Understanding for Fine-Grained 3D Modeling of Plants

Agricultural robotics is an active research area due to global population growth and expectations of food and labor shortages. Robots can potentially help with tasks such as pruning, harvesting, phenotyping, and plant modeling. However, agricultural automation is hampered by the difficulty in creating high resolution 3D semantic maps in the field that would allow for safe manipulation and navigation. In this paper, we build toward solutions for this issue and showcase how the use of semantics and environmental priors can help in constructing accurate 3D maps for the target application of sorghum. Specifically, we 1) use sorghum seeds as semantic landmarks to build a visual Simultaneous Localization and Mapping (SLAM) system that enables us to map 78\\% of a sorghum range on average, compared to 38% with ORB-SLAM2; and 2) use seeds as semantic features to improve 3D reconstruction of a full sorghum panicle from images taken by a robotic in-hand camera.

3D Skeletonization of Complex Grapevines for Robotic Pruning

Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabilities necessary for pruning in denser and more complex vine structures by extending plant skeletonization techniques. The proposed pipeline generates skeletal grapevine models that have lower reprojection error and higher connectivity than baseline algorithms. We also show how 3D and skeletal information enables prediction accuracy of pruning weight for dense vines surpassing prior work, where pruning weight is an important vine metric influencing pruning site selection.

3D Reconstruction-Based Seed Counting of Sorghum Panicles for Agricultural Inspection

In this paper, we present a method for creating high-quality 3D models of sorghum panicles for phenotyping in breeding experiments. This is achieved with a novel reconstruction approach that uses seeds as semantic landmarks in both 2D and 3D. To evaluate the performance, we develop a new metric for assessing the quality of reconstructed point clouds without having a ground-truth point cloud. Finally, a counting method is presented where the density of seed centers in the 3D model allows 2D counts from multiple views to be effectively combined into a whole-panicle count. We demonstrate that using this method to estimate seed count and weight for sorghum outperforms count extrapolation from 2D images, an approach used in most state of the art methods for seeds and grains of comparable size.

The Application of Market-based Multi-Robot Task Allocation to Ambulance Dispatch

Multi-Robot Task Allocation (MRTA) is the problem of distributing a set of tasks to a team of robots with the objective of optimising some criteria, such as minimising the amount of time or energy spent to complete all the tasks or maximising the efficiency of the team's joint activity. The exploration of MRTA methods is typically restricted to laboratory and field experimentation. There are few existing real-world models in which teams of autonomous mobile robots are deployed "in the wild", e.g., in industrial settings. In the work presented here, a market-based MRTA approach is applied to the problem of ambulance dispatch, where ambulances are allocated in respond to patients' calls for help. Ambulances and robots are limited (and perhaps scarce), specialised mobile resources; incidents and tasks represent time-sensitive, specific, potentially unlimited, precisely-located demands for the services which the resources provide. Historical data from the London Ambulance Service describing a set of more than 1 million (anonymised) incidents are used as the basis for evaluating the predicted performance of the market-based approach versus the current, largely manual, method of allocating ambulances to incidents. Experimental results show statistically significant improvement in response times when using the market-based approach.