On Hand-Held Grippers and the Morphological Gap in Human Manipulation Demonstration

Collecting manipulation demonstrations with robotic hardware is tedious - and thus difficult to scale. Recording data on robot hardware ensures that it is in the appropriate format for Learning from Demonstrations (LfD) methods. By contrast, humans are proficient manipulators, and recording their actions would be easy to scale, but it is challenging to use that data format with LfD methods. The question we explore is whether there is a method to collect data in a format that can be used with LfD while retaining some of the attractive features of recording human manipulation. We propose equipping humans with hand-held, hand-actuated parallel grippers and a head-mounted camera to record demonstrations of manipulation tasks. Using customised and reproducible grippers, we collect an initial dataset of common manipulation tasks. We show that there are tasks that, against our initial intuition, can be performed using parallel grippers. Qualitative insights are obtained regarding the impact of the difference in morphology on LfD by comparing the strategies used to complete tasks with human hands and grippers. Our data collection method bridges the gap between robot- and human-native manipulation demonstration. By making the design of our gripper prototype available, we hope to reduce other researchers effort to collect manipulation data.

Cooperative Task and Motion Planning for Multi-Arm Assembly Systems

Multi-robot assembly systems are becoming increasingly appealing in manufacturing due to their ability to automatically, flexibly, and quickly construct desired structural designs. However, effectively planning for these systems in a manner that ensures each robot is simultaneously productive, and not idle, is challenging due to (1) the close proximity that the robots must operate in to manipulate the structure and (2) the inherent structural partial orderings on when each part can be installed. In this paper, we present a task and motion planning framework that jointly plans safe, low-makespan plans for a team of robots to assemble complex spatial structures. Our framework takes a hierarchical approach that, at the high level, uses Mixed-integer Linear Programs to compute an abstract plan comprised of an allocation of robots to tasks subject to precedence constraints and, at the low level, builds on a state-of-the-art algorithm for Multi-Agent Path Finding to plan collision-free robot motions that realize this abstract plan. Critical to our approach is the inclusion of certain collision constraints and movement durations during high-level planning, which better informs the search for abstract plans that are likely to be both feasible and low-makespan while keeping the search tractable. We demonstrate our planning system on several challenging assembly domains with several (sometimes heterogeneous) robots with grippers or suction plates for assembling structures with up to 23 objects involving Lego bricks, bars, plates, or irregularly shaped blocks.

Simplifying deflation for non-convex optimization with applications in Bayesian inference and topology optimization

Non-convex optimization problems have multiple local optimal solutions. Non-convex optimization problems are commonly found in numerous applications. One of the methods recently proposed to efficiently explore multiple local optimal solutions without random re-initialization relies on the concept of deflation. In this paper, different ways to use deflation in non-convex optimization and nonlinear system solving are discussed. A simple, general and novel deflation constraint is proposed to enable the use of deflation together with existing nonlinear programming solvers or nonlinear system solvers. The connection between the proposed deflation constraint and a minimum distance constraint is presented. Additionally, a number of variations of deflation constraints and their limitations are discussed. Finally, a number of applications of the proposed methodology in the fields of approximate Bayesian inference and topology optimization are presented.

Robotic additive construction of bar structures: Unified sequence and motion planning

Additive robotic construction of building-scale discrete bar structures, such as trusses and space frames, is increasingly attractive due to the potential improvements in efficiency, safety, and design possibilities. However, programming complex robots, such as manipulators with seven degrees of freedom, to successfully complete construction tasks can be tedious, challenging, or impossible for a human to do manually. Namely, the structure must be constructed in a sequence that preserves structural properties, such as stiffness, at each step. At the same time, this sequence must allow for the robot to precisely manipulate elements within the in-progress structure while respecting geometric constraints that, for example, ensure the robot does not collide with what it has built. In this work, we present an automated and newly generalized planning approach for jointly finding a construction sequence and robot motion plan for additive construction that satisfies these requirements. Our approach can be applied in a variety of additive construction processes, and we demonstrate it specifically on spatial extrusion and discrete bar assembly in this paper. We demonstrate the effectiveness of our approach on several simulated and real-world extrusion and assembly tasks, including a human-scale physical prototype, for which our algorithm is deployed for the first time to plan the assembly of a complicated double tangent bar system design.

Scalable and Probabilistically Complete Planning for Robotic Spatial Extrusion

There is increasing demand for automated systems that can fabricate 3D structures. Robotic spatial extrusion has become an attractive alternative to traditional layer-based 3D printing due to a manipulator's flexibility to print large, directionally-dependent structures. However, existing extrusion planning algorithms require a substantial amount of human input, do not scale to large instances, and lack theoretical guarantees. In this work, we present a rigorous formalization of robotic spatial extrusion planning and provide several efficient and probabilistically complete planning algorithms. The key planning challenge is, throughout the printing process, satisfying both stiffness constraints that limit the deformation of the structure and geometric constraints that ensure the robot does not collide with the structure. We show that, although these constraints often conflict with each other, a greedy backward state-space search guided by a stiffness-aware heuristic is able to successfully balance both constraints. We empirically compare our methods on a benchmark of over 40 simulated extrusion problems. Finally, we apply our approach to 3 real-world extrusion problems.

Automated sequence and motion planning for robotic spatial extrusion of 3D trusses

While robotic spatial extrusion has demonstrated a new and efficient means to fabricate 3D truss structures in architectural scale, a major challenge remains in automatically planning extrusion sequence and robotic motion for trusses with unconstrained topologies. This paper presents the first attempt in the field to rigorously formulate the extrusion sequence and motion planning (SAMP) problem, using a CSP encoding. Furthermore, this research proposes a new hierarchical planning framework to solve the extrusion SAMP problems that usually have a long planning horizon and 3D configuration complexity. By decoupling sequence and motion planning, the planning framework is able to efficiently solve the extrusion sequence, end-effector poses, joint configurations, and transition trajectories for spatial trusses with nonstandard topologies. This paper also presents the first detailed computation data to reveal the runtime bottleneck on solving SAMP problems, which provides insight and comparing baseline for future algorithmic development. Together with the algorithmic results, this paper also presents an open-source and modularized software implementation called Choreo that is machine-agnostic. To demonstrate the power of this algorithmic framework, three case studies, including real fabrication and simulation results, are presented.