Environment-aware Interactive Movement Primitives for Object Reaching in Clutter

The majority of motion planning strategies developed over the literature for reaching an object in clutter are applied to two dimensional (2-d) space where the state space of the environment is constrained in one direction. Fewer works have been investigated to reach a target in 3-d cluttered space, and when so, they have limited performance when applied to complex cases. In this work, we propose a constrained multi-objective optimization framework (OptI-ProMP) to approach the problem of reaching a target in a compact clutter with a case study on soft fruits grown in clusters, leveraging the local optimisation-based planner CHOMP. OptI-ProMP features costs related to both static, dynamic and pushable objects in the target neighborhood, and it relies on probabilistic primitives for problem initialisation. We tested, in a simulated poly-tunnel, both ProMP-based planners from literature and the OptI-ProMP, on low (3-dofs) and high (7-dofs) dexterity robot body, respectively. Results show collision and pushing costs minimisation with 7-dofs robot kinematics, in addition to successful static obstacles avoidance and systematic drifting from the pushable objects center of mass.

Learning on the Job: Long-Term Behavioural Adaptation in Human-Robot Interactions

In this work, we propose a framework for allowing autonomous robots deployed for extended periods of time in public spaces to adapt their own behaviour online from user interactions. The robot behaviour planning is embedded in a Reinforcement Learning (RL) framework, where the objective is maximising the level of overall user engagement during the interactions. We use the Upper-Confidence-Bound Value-Iteration (UCBVI) algorithm, which gives a helpful way of managing the exploration-exploitation trade-off for real-time interactions. An engagement model trained end-to-end generates the reward function in real-time during policy execution. We test this approach in a public museum in Lincoln (UK), where the robot is deployed as a tour guide for the visitors. Results show that after a couple of months of exploration, the robot policy learned to maintain the engagement of users for longer, with an increase of 22.8% over the initial static policy in the number of items visited during the tour and a 30% increase in the probability of completing the tour. This work is a promising step toward behavioural adaptation in long-term scenarios for robotics applications in social settings.

EDLaaS; Fully Homomorphic Encryption Over Neural Network Graphs

We present automatically parameterised Fully Homomorphic Encryption (FHE), for encrypted neural network inference. We present and exemplify our inference over FHE compatible neural networks with our own open-source framework and reproducible step-by-step examples. We use the 4th generation Cheon, Kim, Kim and Song (CKKS) FHE scheme over fixed points provided by the Microsoft Simple Encrypted Arithmetic Library (MS-SEAL). We significantly enhance the usability and applicability of FHE in deep learning contexts, with a focus on the constituent graphs, traversal, and optimisation. We find that FHE is not a panacea for all privacy preserving machine learning (PPML) problems, and that certain limitations still remain, such as model training. However we also find that in certain contexts FHE is well suited for computing completely private predictions with neural networks. We focus on convolutional neural networks (CNNs), fashion-MNIST, and levelled FHE operations. The ability to privately compute sensitive problems more easily, while lowering the barriers to entry, can allow otherwise too-sensitive fields to begin advantaging themselves of performant third-party neural networks. Lastly we show encrypted deep learning, applied to a sensitive real world problem in agri-food, and how this can have a large positive impact on food-waste and encourage much-needed data sharing.

Benchmark of visual and 3D lidar SLAM systems in simulation environment for vineyards

In this work, we present a comparative analysis of the trajectories estimated from various Simultaneous Localization and Mapping (SLAM) systems in a simulation environment for vineyards. Vineyard environment is challenging for SLAM methods, due to visual appearance changes over time, uneven terrain, and repeated visual patterns. For this reason, we created a simulation environment specifically for vineyards to help studying SLAM systems in such a challenging environment. We evaluated the following SLAM systems: LIO-SAM, StaticMapping, ORB-SLAM2, and RTAB-MAP in four different scenarios. The mobile robot used in this study equipped with 2D and 3D lidars, IMU, and RGB-D camera (Kinect v2). The results show good and encouraging performance of RTAB-MAP in such an environment.

Navigate-and-Seek: a Robotics Framework for People Localization in Agricultural Environments

The agricultural domain offers a working environment where many human laborers are nowadays employed to maintain or harvest crops, with huge potential for productivity gains through the introduction of robotic automation. Detecting and localizing humans reliably and accurately in such an environment, however, is a prerequisite to many services offered by fleets of mobile robots collaborating with human workers. Consequently, in this paper, we expand on the concept of a topological particle filter (TPF) to accurately and individually localize and track workers in a farm environment, integrating information from heterogeneous sensors and combining local active sensing (exploiting a robot's onboard sensing employing a Next-Best-Sense planning approach) and global localization (using affordable IoT GNSS devices). We validate the proposed approach in topologies created for the deployment of robotics fleets to support fruit pickers in a real farm environment. By combining multi-sensor observations on the topological level complemented by active perception through the NBS approach, we show that we can improve the accuracy of picker localization in comparison to prior work.

Action-Conditional Recurrent Kalman Networks For Forward and Inverse Dynamics Learning

Estimating accurate forward and inverse dynamics models is a crucial component of model-based control for sophisticated robots such as robots driven by hydraulics, artificial muscles, or robots dealing with different contact situations. Analytic models to such processes are often unavailable or inaccurate due to complex hysteresis effects, unmodelled friction and stiction phenomena,and unknown effects during contact situations. A promising approach is to obtain spatio-temporal models in a data-driven way using recurrent neural networks, as they can overcome those issues. However, such models often do not meet accuracy demands sufficiently, degenerate in performance for the required high sampling frequencies and cannot provide uncertainty estimates. We adopt a recent probabilistic recurrent neural network architecture, called Re-current Kalman Networks (RKNs), to model learning by conditioning its transition dynamics on the control actions. RKNs outperform standard recurrent networks such as LSTMs on many state estimation tasks. Inspired by Kalman filters, the RKN provides an elegant way to achieve action conditioning within its recurrent cell by leveraging additive interactions between the current latent state and the action variables. We present two architectures, one for forward model learning and one for inverse model learning. Both architectures significantly outperform exist-ing model learning frameworks as well as analytical models in terms of prediction performance on a variety of real robot dynamics models.

Planning Actions by Interactive Movement Primitives: pushing occluding pieces to pick a ripe fruit

While many efforts are currently devoted by research bodies to investigate in robot harvesting, challenges related to picking fruits from clusters are still considered an open issue which can limit the operation success. On the other hand, existing planning frameworks for robotic manipulation in cluttered and uncertain environment are getting more and more attention for their ability to deal with physics-based strategies to free the robot path to a goal object. However, those approaches are either computationally expensive and/or designed for 2-D occlusion scenes. Consequently, they are not readily applicable to the complex 3-D geometry of fruits in clusters. In this work, we present a path planning algorithm for pushing occluding fruits to reach-and-pick a ripe one. Hence, we propose an Interactive Probabilistic Movement Primitives (I-ProMP) which is computationally efficient and is readily used for 3-D problems. We demonstrate the efficiency of our approach with pushing unripe strawberries in a simulated polytunnel. Our experimental results confirm I- ProMP successfully pushes table top grown strawberries and reaches a ripe one.

Are you still with me? Continuous Engagement Assessment from a Robot's Point of View

Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way towards in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behaviour optimisation. However, engagement is often considered very multi-faceted and difficult to capture in a workable and generic computational model that can serve as an overall measure of engagement. Building upon the intuitive ways humans successfully can assess situation for a degree of engagement when they see it, we propose a novel regression model (utilising CNN and LSTM networks) enabling robots to compute a single scalar engagement during interactions with humans from standard video streams, obtained from the point of view of an interacting robot. The model is based on a long-term dataset from an autonomous tour guide robot deployed in a public museum, with continuous annotation of a numeric engagement assessment by three independent coders. We show that this model not only can predict engagement very well in our own application domain but show its successful transfer to an entirely different dataset (with different tasks, environment, camera, robot and people). The trained model and the software is available to the HRI community as a tool to measure engagement in a variety of settings.

Agricultural Robotics: The Future of Robotic Agriculture

Agri-Food is the largest manufacturing sector in the UK. It supports a food chain that generates over {\pounds}108bn p.a., with 3.9m employees in a truly international industry and exports {\pounds}20bn of UK manufactured goods. However, the global food chain is under pressure from population growth, climate change, political pressures affecting migration, population drift from rural to urban regions and the demographics of an aging global population. These challenges are recognised in the UK Industrial Strategy white paper and backed by significant investment via a Wave 2 Industrial Challenge Fund Investment ("Transforming Food Production: from Farm to Fork"). Robotics and Autonomous Systems (RAS) and associated digital technologies are now seen as enablers of this critical food chain transformation. To meet these challenges, this white paper reviews the state of the art in the application of RAS in Agri-Food production and explores research and innovation needs to ensure these technologies reach their full potential and deliver the necessary impacts in the Agri-Food sector.