Semantic Radiance Fields as Simulators for Spatial Reasoning in Real-World Scenes

Training and evaluating spatial reasoning in embodied agents requires diverse environments that are both geometrically faithful and semantically queryable. Synthetic simulators offer ground truth semantics but sacrifice realism; simulators based on reconstructions of real-world environments have realistic appearance but lack ground truth semantics by default. We propose using Semantic Radiance Fields (SRF) as simulators for spatial reasoning agents. SRFs are a representation that unifies these requirements by lifting 2D semantic segmentations from pretrained vision models into a 3D radiance field that jointly encodes geometry, appearance, and per-class semantic identity. The resulting fields are reconstructed from posed RGB captures of real scenes and support novel-view synthesis, semantic and free-space queries within a single grounded representation. This enables the efficient generation of diverse real-world environments to train and evaluate spatial reasoning models. As an example application, we outline an SRF-driven simulator for an orchard apple-reaching task, in which the radiance field supplies camera rendering, semantic ground truth, and occupancy queries to a physics engine.

Real-time windrow detection from onboard tractor sensors for automated following

Proprietary design in commercial windrow-detection systems restricts transparency and limits progress in open autonomous forage-harvesting research. We present a multi-modal dataset combining stereo vision and LiDAR from tractor-mounted sensors during real baling operations. The dataset includes synchronized sensor data with GNSS trajectories, partly released as ROS2 Humble bags on Zenodo, with additional data available on request. Using this dataset, we implement a real-time (>20 Hz) centroid-based windrow-following method on an NVIDIA Jetson AGX Orin. Across the critical 4-10 m guidance range, stereo and LiDAR depth measurements show strong agreement (0.965 +/- 0.021), indicating that low-cost stereo sensors can approach LiDAR performance. Our open-source ROS 2 pipeline provides a reproducible benchmark for GPS-free windrow detection and supports development of practical autonomous forage-harvesting systems. Dataset: https://zenodo.org/records/17486318

A survey of datasets for computer vision in agriculture

In agricultural research, there has been a recent surge in the amount of Computer Vision (CV) focused work. But unlike general CV research, large high-quality public datasets are sparsely available. This can be partially attributed to the high variability between different agricultural tasks, crops and environments as well as the complexity of data collection, but it is also influenced by the reticence to publish datasets by many authors. This, as well as the lack of a widely used agricultural data repository, are impactful factors that hinder research in applied CV for agriculture as well as the usage of agricultural data in general-purpose CV research. In this survey, we provide a large number of high-quality datasets of images taken on fields. Overall, we find 45 datasets, which are listed in this paper as well as in an online catalog on the project website: https://smartfarminglab.github.io/field_dataset_survey/.