AI-Enabled Image-Based Hybrid Vision/Force Control of Tendon-Driven Aerial Continuum Manipulators

This paper presents an AI-enabled cascaded hybrid vision/force control framework for tendon-driven aerial continuum manipulators based on constant-strain modeling in $SE(3)$ as a coupled system. The proposed controller is designed to enable autonomous, physical interaction with a static environment while stabilizing the image feature error. The developed strategy combines the cascaded fast fixed-time sliding mode control and a radial basis function neural network to cope with the uncertainties in the image acquired by the eye-in-hand monocular camera and the measurements from the force sensing apparatus. This ensures rapid, online learning of the vision- and force-related uncertainties without requiring offline training. Furthermore, the features are extracted via a state-of-the-art graph neural network architecture employed by a visual servoing framework using line features, rather than relying on heuristic geometric line extractors, to concurrently contribute to tracking the desired normal interaction force during contact and regulating the image feature error. A comparative study benchmarks the proposed controller against established rigid-arm aerial manipulation methods, evaluating robustness across diverse scenarios and feature extraction strategies. The simulation and experimental results showcase the effectiveness of the proposed methodology under various initial conditions and demonstrate robust performance in executing manipulation tasks.

Real-Time Projected Adaptive Control for Closed-Chain Co-Manipulative Continuum Robots

In co-manipulative continuum robots (CCRs), multiple continuum arms cooperate by grasping a common flexible object, forming a closed-chain deformable mechanical system. The closed-chain coupling induces strong dynamic interactions and internal reaction forces. Moreover, in practical tasks, the flexible object's physical parameters are often unknown and vary between operations, rendering nominal model-based controllers inadequate. This paper presents a projected adaptive control framework for CCRs formulated at the dynamic level. The coupled dynamics are expressed using the Geometric Variable Strain (GVS) representation, yielding a finite-dimensional model that accurately represents the system, preserves the linear-in-parameters structure required for adaptive control, and is suitable for real-time implementation. Closed-chain interactions are enforced through Pfaffian velocity constraints, and an orthogonal projection is used to express the dynamics in the constraint-consistent motion subspace. Based on the projected dynamics, an adaptive control law is developed to compensate online for uncertain dynamic parameters of both the continuum robots and the manipulated flexible object. Lyapunov analysis establishes closed-loop stability and convergence of the task-space tracking errors to zero. Simulation and experiments on a tendon-driven CCR platform validate the proposed framework in task-space regulation and trajectory tracking.

Hybrid Visual Servoing of Tendon-driven Continuum Robots

This paper introduces a novel Hybrid Visual Servoing (HVS) approach for controlling tendon-driven continuum robots (TDCRs). The HVS system combines Image-Based Visual Servoing (IBVS) with Deep Learning-Based Visual Servoing (DLBVS) to overcome the limitations of each method and improve overall performance. IBVS offers higher accuracy and faster convergence in feature-rich environments, while DLBVS enhances robustness against disturbances and offers a larger workspace. By enabling smooth transitions between IBVS and DLBVS, the proposed HVS ensures effective control in dynamic, unstructured environments. The effectiveness of this approach is validated through simulations and real-world experiments, demonstrating that HVS achieves reduced iteration time, faster convergence, lower final error, and smoother performance compared to DLBVS alone, while maintaining DLBVS's robustness in challenging conditions such as occlusions, lighting changes, actuator noise, and physical impacts.

Impedance Control for Manipulators Handling Heavy Payloads

Attaching a heavy payload to the wrist force/moment (F/M) sensor of a manipulator can cause conventional impedance controllers to fail in establishing the desired impedance due to the presence of non-contact forces; namely, the inertial and gravitational forces of the payload. This paper presents an impedance control scheme designed to accurately shape the force-response of such a manipulator without requiring acceleration measurements. As a result, neither wrist accelerometers nor dynamic estimators for compensating inertial load forces are necessary. The proposed controller employs an inner-outer loop feedback structure, which not only addresses uncertainties in the robot's dynamics but also enables the specification of a general target impedance model, including nonlinear models. Stability and convergence of the controller are analytically proven, with results showing that the control input remains bounded as long as the desired inertia differs from the payload inertia. Experimental results confirm that the proposed impedance controller effectively shapes the impedance of a manipulator carrying a heavy load according to the desired impedance model.

Adaptive Visual Servoing for On-Orbit Servicing

This paper presents an adaptive visual servoing framework for robotic on-orbit servicing (OOS), specifically designed for capturing tumbling satellites. The vision-guided robotic system is capable of selecting optimal control actions in the event of partial or complete vision system failure, particularly in the short term. The autonomous system accounts for physical and operational constraints, executing visual servoing tasks to minimize a cost function. A hierarchical control architecture is developed, integrating a variant of the Iterative Closest Point (ICP) algorithm for image registration, a constrained noise-adaptive Kalman filter, fault detection and recovery logic, and a constrained optimal path planner. The dynamic estimator provides real-time estimates of unknown states and uncertain parameters essential for motion prediction, while ensuring consistency through a set of inequality constraints. It also adjusts the Kalman filter parameters adaptively in response to unexpected vision errors. In the event of vision system faults, a recovery strategy is activated, guided by fault detection logic that monitors the visual feedback via the metric fit error of image registration. The estimated/predicted pose and parameters are subsequently fed into an optimal path planner, which directs the robot's end-effector to the target's grasping point. This process is subject to multiple constraints, including acceleration limits, smooth capture, and line-of-sight maintenance with the target. Experimental results demonstrate that the proposed visual servoing system successfully captured a free-floating object, despite complete occlusion of the vision system.

Seamless Capture and Stabilization of Spinning Satellites By Space Robots with Spinning Base

This paper introduces an innovative guidance and control method for simultaneously capturing and stabilizing a fast-spinning target satellite, such as a spin-stabilized satellite, using a spinning-base servicing satellite equipped with a robotic manipulator, joint locks, and reaction wheels (RWs). The method involves controlling the RWs of the servicing satellite to replicate the spinning motion of the target satellite, while locking the manipulator's joints to achieve spin-matching. This maneuver makes the target stationary with respect to the rotating frame of the servicing satellite located at its center-of-mass (CoM), simplifying the robot capture trajectory planning and eliminating post-capture trajectory planning entirely. In the next phase, the joints are unlocked, and a coordination controller drives the robotic manipulator to capture the target satellite while maintaining zero relative rotation between the servicing and target satellites. The spin stabilization phase begins after completing the capture phase, where the joints are locked to form a single tumbling rigid body consisting of the rigidly connected servicing and target satellites. An optimal controller applies negative control torques to the RWs to dampen out the tumbling motion of the interconnected satellites as quickly as possible, subject to the actuation torque limit of the RWs and the maximum torque exerted by the manipulator's end-effector.

Autonomous Robots for Active Removal of Orbital Debris

This paper presents a vision guidance and control method for autonomous robotic capture and stabilization of orbital objects in a time-critical manner. The method takes into account various operational and physical constraints, including ensuring a smooth capture, handling line-of-sight (LOS) obstructions of the target, and staying within the acceleration, force, and torque limits of the robot. Our approach involves the development of an optimal control framework for an eye-to-hand visual servoing method, which integrates two sequential sub-maneuvers: a pre-capturing maneuver and a post-capturing maneuver, aimed at achieving the shortest possible capture time. Integrating both control strategies enables a seamless transition between them, allowing for real-time switching to the appropriate control system. Moreover, both controllers are adaptively tuned through vision feedback to account for the unknown dynamics of the target. The integrated estimation and control architecture also facilitates fault detection and recovery of the visual feedback in situations where the feedback is temporarily obstructed. The experimental results demonstrate the successful execution of pre- and post-capturing operations on a tumbling and drifting target, despite multiple operational constraints.

Coordination Control of Free-Flyer Manipulators

This paper presents a method for guiding a robot manipulator to capture and bring a tumbling satellite to a state of rest. The proposed approach includes developing a coordination control for the combined system of the space robot and the target satellite, where the satellite acts as the manipulator payload. This control ensures that the robot tracks the optimal path while regulating the attitude of the chase vehicle to a desired value. Two optimal trajectories are then designed for the pre- and post-capture phases. In the pre-capturing phase, the manipulator manoeuvres are optimized by minimizing a cost function that includes the time of travel and the weighted norms of the end-effector velocity and acceleration, subject to the constraint that the robot end-effector and a grapple fixture on the satellite arrive at the rendezvous point with the same velocity. In the post-grasping phase, the manipulator dumps the initial velocity of the tumbling satellite in minimum time while ensuring that the magnitude of the torque applied to the satellite remains below a safe value. Overall, this method offers a promising solution for effectively capturing and bringing tumbling satellites to a state of rest.

Automated Rendezvous & Docking Using 3D Vision

The robustness and accuracy of a vision system for motion estimation of a tumbling target satellite are enhanced by an adaptive Kalman filter. This allows a vision-guided robot to complete the grasping of the target even if occlusion occurs during the operation. A complete dynamics model, including aspects of orbital mechanics, is incorporated for accurate estimation. Based on the model, an adaptive Kalman filter is developed that estimates not only the system states but also all the model parameters such as the inertia ratio, center-of-mass, and the rotation of the principal axes of the target satellite. An experiment is conducted by using a robotic arm to move a satellite mockup according to orbital mechanics while the satellite pose is measured by a laser camera system. The measurements are sent to the Kalman filter, which, in turn, drives another robotic arm to grasp the target. The results demonstrate successful grasping even if the vision system is blocked for several seconds.

Hybrid Simulator for Space Docking and Robotic Proximity Operations

In this work, we present a hybrid simulator for space docking and robotic proximity operations methodology. This methodology also allows for the emulation of a target robot operating in a complex environment by using an actual robot. The emulation scheme aims to replicate the dynamic behavior of the target robot interacting with the environment, without dealing with a complex calculation of the contact dynamics. This method forms a basis for the task verification of a flexible space robot. The actual emulating robot is structurally rigid, while the target robot can represent any class of robots, e.g., flexible, redundant, or space robots. Although the emulating robot is not dynamically equivalent to the target robot, the dynamical similarity can be achieved by using a control law developed herein. The effect of disturbances and actuator dynamics on the fidelity and the contact stability of the robot emulation is thoroughly analyzed.