Render-in-the-loop aerial robotics simulator: Case Study on Yield Estimation in Indoor Agriculture

Inspired by recent promising results in sim-to-real transfer in deep learning we built a realistic simulation environment combining a Robot Operating System (ROS)-compatible physics simulator (Gazebo) with Cycles, the realistic production rendering engine from Blender. The proposed simulator pipeline allows us to simulate near-realistic RGB-D images. To showcase the capabilities of the simulator pipeline we propose a case study that focuses on indoor robotic farming. We developed a solution for sweet pepper yield estimation task. Our approach to yield estimation starts with aerial robotics control and trajectory planning, combined with deep learning-based pepper detection, and a clustering approach for counting fruit. The results of this case study show that we can combine real time dynamic simulation with near realistic rendering capabilities to simulate complex robotic systems.

Robotic Irrigation Water Management: Estimating Soil Moisture Content by Feel and Appearance

In this paper we propose a robotic system for Irrigation Water Management (IWM) in a structured robotic greenhouse environment. A commercially available robotic manipulator is equipped with an RGB-D camera and a soil moisture sensor. The two are used to automate the procedure known as "feel and appearance method", which is a way of monitoring soil moisture to determine when to irrigate and how much water to apply. We develop a compliant force control framework that enables the robot to insert the soil moisture sensor in the sensitive plant root zone of the soil, without harming the plant. RGB-D camera is used to roughly estimate the soil surface, in order to plan the soil sampling approach. Used together with the developed adaptive force control algorithm, the camera enables the robot to sample the soil without knowing the exact soil stiffness a priori. Finally, we postulate a deep learning based approach to utilize the camera to visually assess the soil health and moisture content.