In-Hand Cube Reconfiguration: Simplified

We present a simple approach to in-hand cube reconfiguration. By simplifying planning, control, and perception as much as possible, while maintaining robust and general performance, we gain insights into the inherent complexity of in-hand cube reconfiguration. We also demonstrate the effectiveness of combining GOFAI-based planning with the exploitation of environmental constraints and inherently compliant end-effectors in the context of dexterous manipulation. The proposed system outperforms a substantially more complex system for cube reconfiguration based on deep learning and accurate physical simulation, contributing arguments to the discussion about what the most promising approach to general manipulation might be. Project website: https://rbo.gitlab-pages.tu-berlin.de/robotics/simpleIHM/

Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation

This paper presents a feedback-control framework for in-hand manipulation (IHM) with dexterous soft hands that enables the acquisition of manipulation skills in the real-world within minutes. We choose the deformation state of the soft hand as the control variable. To control for a desired deformation state, we use coarsely approximated Jacobians of the actuation-deformation dynamics. These Jacobian are obtained via explorative actions. This is enabled by the self-stabilizing properties of compliant hands, which allow us to use linear feedback control in the presence of complex contact dynamics. To evaluate the effectiveness of our approach, we show the generalization capabilities for a learned manipulation skill to variations in object size by 100 %, 360 degree changes in palm inclination and to disabling up to 50 % of the involved actuators. In addition, complex manipulations can be obtained by sequencing such feedback-skills.

Augmentation for Learning From Demonstration with Environmental Constraints

We introduce a Learning from Demonstration (LfD) approach for contact-rich manipulation tasks with articulated mechanisms. The extracted policy from a single human demonstration generalizes to different mechanisms of the same type and is robust against environmental variations. The key to achieving such generalization and robustness from a single human demonstration is to autonomously augment the initial demonstration to gather additional information through purposefully interacting with the environment. Our real-world experiments on complex mechanisms with multi-DOF demonstrate that our approach can reliably accomplish the task in a changing environment. Videos are available at the: https://sites.google.com/view/rbosalfdec/home

Passive and Active Acoustic Sensing for Soft Pneumatic Actuators

We propose a sensorization method for soft pneumatic actuators that uses an embedded microphone and speaker to measure different actuator properties. The physical state of the actuator determines the specific modulation of sound as it travels through the structure. Using simple machine learning, we create a computational sensor that infers the corresponding state from sound recordings. We demonstrate the acoustic sensor on a soft pneumatic continuum actuator and use it to measure contact locations, contact forces, object materials, actuator inflation, and actuator temperature. We show that the sensor is reliable (average classification rate for six contact locations of 93%), precise (mean spatial accuracy of 3.7 mm), and robust against common disturbances like background noise. Finally, we compare different sounds and learning methods and achieve best results with 20 ms of white noise and a support vector classifier as the sensor model.

A Virtual 2D Tactile Array for Soft Actuators Using Acoustic Sensing

We create a virtual 2D tactile array for soft pneumatic actuators using embedded audio components. We detect contact-specific changes in sound modulation to infer tactile information. We evaluate different sound representations and learning methods to detect even small contact variations. We demonstrate the acoustic tactile sensor array by the example of a PneuFlex actuator and use a Braille display to individually control the contact of 29x4 pins with the actuator's 90x10 mm palmar surface. Evaluating the spatial resolution, the acoustic sensor localizes edges in x- and y-direction with a root-mean-square regression error of 1.67 mm and 0.0 mm, respectively. Even light contacts of a single Braille pin with a lifting force of 0.17 N are measured with high accuracy. Finally, we demonstrate the sensor's sensitivity to complex contact shapes by successfully reading the 26 letters of the Braille alphabet from a single display cell with a classification rate of 88%.

One Object at a Time: Accurate and Robust Structure From Motion for Robots

A gaze-fixating robot perceives distance to the fixated object and relative positions of surrounding objects immediately, accurately, and robustly. We show how fixation, which is the act of looking at one object while moving, exploits regularities in the geometry of 3D space to obtain this information. These regularities introduce rotation-translation couplings that are not commonly used in structure from motion. To validate, we use a Franka Emika Robot with an RGB camera. We a) find that error in distance estimate is less than 5 mm at a distance of 15 cm, and b) show how relative position can be used to find obstacles under challenging scenarios. We combine accurate distance estimates and obstacle information into a reactive robot behavior that is able to pick up objects of unknown size, while impeded by unforeseen obstacles.

"The World Is Its Own Best Model": Robust Real-World Manipulation Through Online Behavior Selection

Robotic manipulation behavior should be robust to disturbances that violate high-level task-structure. Such robustness can be achieved by constantly monitoring the environment to observe the discrete high-level state of the task. This is possible because different phases of a task are characterized by different sensor patterns and by monitoring these patterns a robot can decide which controllers to execute in the moment. This relaxes assumptions about the temporal sequence of those controllers and makes behavior robust to unforeseen disturbances. We implement this idea as probabilistic filter over discrete states where each state is direcly associated with a controller. Based on this framework we present a robotic system that is able to open a drawer and grasp tennis balls from it in a surprisingly robust way.

Surprisingly Robust In-Hand Manipulation: An Empirical Study

We present in-hand manipulation skills on a dexterous, compliant, anthropomorphic hand. Even though these skills were derived in a simplistic manner, they exhibit surprising robustness to variations in shape, size, weight, and placement of the manipulated object. They are also very insensitive to variation of execution speeds, ranging from highly dynamic to quasi-static. The robustness of the skills leads to compositional properties that enable extended and robust manipulation programs. To explain the surprising robustness of the in-hand manipulation skills, we performed a detailed, empirical analysis of the skills' performance. From this analysis, we identify three principles for skill design: 1) Exploiting the hardware's innate ability to drive hard-to-model contact dynamics. 2) Taking actions to constrain these interactions, funneling the system into a narrow set of possibilities. 3) Composing such action sequences into complex manipulation programs. We believe that these principles constitute an important foundation for robust robotic in-hand manipulation, and possibly for manipulation in general.

RBO Hand 3 -- A Platform for Soft Dexterous Manipulation

We present the RBO Hand 3, a highly capable and versatile anthropomorphic soft hand based on pneumatic actuation. The RBO Hand 3 is designed to enable dexterous manipulation, to facilitate transfer of insights about human dexterity, and to serve as a robust research platform for extensive real-world experiments. It achieves these design goals by combining many degrees of actuation with intrinsic compliance, replicating relevant functioning of the human hand, and by combining robust components in a modular design. The RBO Hand 3 possesses 16 independent degrees of actuation, implemented in a dexterous opposable thumb, two-chambered fingers, an actuated palm, and the ability to spread the fingers. In this work, we derive the design objectives that are based on experimentation with the hand's predecessors, observations about human grasping, and insights about principles of dexterity. We explain in detail how the design features of the RBO Hand 3 achieve these goals and evaluate the hand by demonstrating its ability to achieve the highest possible score in the Kapandji test for thumb opposition, to realize all 33 grasp types of the comprehensive GRASP taxonomy, to replicate common human grasping strategies, and to perform dexterous in-hand manipulation.

A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition

Soft robotics is an emerging field that yields promising results for tasks that require safe and robust interactions with the environment or with humans, such as grasping, manipulation, and human-robot interaction. Soft robots rely on intrinsically compliant components and are difficult to equip with traditional, rigid sensors which would interfere with their compliance. We propose a highly flexible tactile sensor that is low-cost and easy to manufacture while measuring contact pressures independently from 14 taxels. The sensor is built from piezoresistive fabric for highly sensitive, continuous responses and from a custom-designed flexible printed circuit board which provides a high taxel density. From these taxels, location and intensity of contact with the sensor can be inferred. In this paper, we explain the design and manufacturing of the proposed sensor, characterize its input-output relation, evaluate its effects on compliance when equipped to the silicone-based pneumatic actuators of the soft robotic RBO Hand 2, and demonstrate that the sensor provides rich and useful feedback for learning-based in-hand object recognition.