Abstract:Teaching robots dexterous skills from human videos remains challenging due to the reliance on low-level trajectory imitation, which fails to generalize across object types, spatial layouts, and manipulator configurations. We propose Graph-Fused Vision-Language-Action (GF-VLA), a framework that enables dual-arm robotic systems to perform task-level reasoning and execution directly from RGB and Depth human demonstrations. GF-VLA first extracts Shannon-information-based cues to identify hands and objects with the highest task relevance, then encodes these cues into temporally ordered scene graphs that capture both hand-object and object-object interactions. These graphs are fused with a language-conditioned transformer that generates hierarchical behavior trees and interpretable Cartesian motion commands. To improve execution efficiency in bimanual settings, we further introduce a cross-hand selection policy that infers optimal gripper assignment without explicit geometric reasoning. We evaluate GF-VLA on four structured dual-arm block assembly tasks involving symbolic shape construction and spatial generalization. Experimental results show that the information-theoretic scene representation achieves over 95 percent graph accuracy and 93 percent subtask segmentation, supporting the LLM planner in generating reliable and human-readable task policies. When executed by the dual-arm robot, these policies yield 94 percent grasp success, 89 percent placement accuracy, and 90 percent overall task success across stacking, letter-building, and geometric reconfiguration scenarios, demonstrating strong generalization and robustness across diverse spatial and semantic variations.
Abstract:This paper proposes a novel robust adaptive model predictive controller for on-orbit dislodging. We consider the scenario where a servicer, equipped with a robot arm, must dislodge a client, a time-varying system composed of an underpowered jammed solar panel with a hybrid hinge system on a space station. Our approach leverages online set-membership identification to reduce the uncertainty to provide robust safety guarantees during dislodging despite bounded disturbances while balancing exploration and exploitation effectively in the parameter space. The feasibility of the developed robust adaptive MPC method is also examined through dislodging simulations and hardware experiments in zero-gravity and gravity environments, respectively. In addition, the advantages of our method are shown through comparison experiments with several state-of-the-art control schemes for both accuracy of parameter estimation and control performance.
Abstract:The challenge of satellite stabilization, particularly those with uncertain flexible dynamics, has become a pressing concern in control and robotics. These uncertainties, especially the dynamics of a third-party client satellite, significantly complicate the stabilization task. This paper introduces a novel adaptive detumbling method to handle non-rigid satellites with unknown motion dynamics (translation and rotation). The distinctive feature of our approach is that we model the non-rigid tumbling satellite as a two-link serial chain with unknown stiffness and damping in contrast to previous detumbling research works which consider the satellite a rigid body. We develop a novel adaptive robotics approach to detumble the satellite by using two space tugs as servicer despite the uncertain dynamics in the post-capture case. Notably, the stiffness properties and other physical parameters, including the mass and inertia of the two links, remain unknown to the servicer. Our proposed method addresses the challenges in detumbling tasks and paves the way for advanced manipulation of non-rigid satellites with uncertain dynamics.
Abstract:Transportation missions in aerospace are limited to the capability of each aerospace robot and the properties of the target transported object, such as mass, inertia, and grasping locations. We present a novel decentralized adaptive controller design for multiple robots that can be implemented in different kinds of aerospace robots. Our controller adapts to unknown objects in different gravity environments. We validate our method in an aerial scenario using multiple fully actuated hexarotors with grasping capabilities, and a space scenario using a group of space tugs. In both scenarios, the robots transport a payload cooperatively through desired three-dimensional trajectories. We show that our method can adapt to unexpected changes that include the loss of robots during the transportation mission.