Trimanipulation: Evaluation of human performance in a 3-handed coordination task

Many teleoperation tasks require three or more tools working together, which need the cooperation of multiple operators. The effectiveness of such schemes may be limited by communication. Trimanipulation by a single operator using an artificial third arm controlled together with their natural arms is a promising solution to this issue. Foot-controlled interfaces have previously shown the capability to be used for the continuous control of robot arms. However, the use of such interfaces for controlling a supernumerary robotic limb (SRLs) in coordination with the natural limbs, is not well understood. In this paper, a teleoperation task imitating physically coupled hands in a virtual reality scene was conducted with 14 subjects to evaluate human performance during tri-manipulation. The participants were required to move three limbs together in a coordinated way mimicking three arms holding a shared physical object. It was found that after a short practice session, the three-hand tri-manipulation using a single subject's hands and foot was still slower than dyad operation, however, they displayed similar performance in success rate and higher motion efficiency than two person's cooperation.

A Three-limb Teleoperated Robotic System with Foot Control for Flexible Endoscopic Surgery

Flexible endoscopy requires high skills to manipulate both the endoscope and associated instruments. In most robotic flexible endoscopic systems, the endoscope and instruments are controlled separately by two operators, which may result in communication errors and inefficient operation. We present a novel teleoperation robotic endoscopic system that can be commanded by a surgeon alone. This 13 degrees-of-freedom (DoF) system integrates a foot-controlled robotic flexible endoscope and two hand-controlled robotic endoscopic instruments (a robotic grasper and a robotic cauterizing hook). A foot-controlled human-machine interface maps the natural foot gestures to the 4-DoF movements of the endoscope, and two hand-controlled interfaces map the movements of the two hands to the two instruments individually. The proposed robotic system was validated in an ex-vivo experiment carried out by six subjects, where foot control was also compared with a sequential clutch-based hand control scheme. The participants could successfully teleoperate the endoscope and the two instruments to cut the tissues at scattered target areas in a porcine stomach. Foot control yielded 43.7% faster task completion and required less mental effort as compared to the clutch-based hand control scheme. The system introduced in this paper is intuitive for three-limb manipulation even for operators without experience of handling the endoscope and robotic instruments. This three-limb teleoperated robotic system enables one surgeon to intuitively control three endoscopic tools which normally require two operators, leading to reduced manpower, less communication errors, and improved efficiency.

Performance evaluation of a foot-controlled human-robot interface

Robotic minimally invasive interventions typically require using more than two instruments. We thus developed a foot pedal interface which allows the user to control a robotic arm (simultaneously to working with the hands) with four degrees of freedom in continuous directions and speeds. This paper evaluates and compares the performances of ten naive operators in using this new pedal interface and a traditional button interface in completing tasks. These tasks are geometrically complex path-following tasks similar to those in laparoscopic training, and the traditional button interface allows axis-by-axis control with constant speeds. Precision, time, and smoothness of the subjects' control movements for these tasks are analysed. The results demonstrate that the pedal interface can be used to control a robot for complex motion tasks. The subjects kept the average error rate at a low level of around 2.6% with both interfaces, but the pedal interface resulted in about 30% faster operation speed and 60% smoother movement, which indicates improved efficiency and user experience as compared with the button interface. The results of a questionnaire show that the operators found that controlling the robot with the pedal interface was more intuitive, comfortable, and less tiring than using the button interface.

A Subject-Specific Four-Degree-of-Freedom Foot Interface to Control a Robot Arm

In robotic surgery, the surgeon controls robotic instruments using dedicated interfaces. One critical limitation of current interfaces is that they are designed to be operated by only the hands. This means that the surgeon can only control at most two robotic instruments at one time while many interventions require three instruments. This paper introduces a novel four-degree-of-freedom foot-machine interface which allows the surgeon to control a third robotic instrument using the foot, giving the surgeon a "third hand". This interface is essentially a parallel-serial hybrid mechanism with springs and force sensors. Unlike existing switch-based interfaces that can only un-intuitively generate motion in discrete directions, this interface allows intuitive control of a slave robotic arm in continuous directions and speeds, naturally matching the foot movements with dynamic force & position feedbacks. An experiment with ten naive subjects was conducted to test the system. In view of the significant variance of motion patterns between subjects, a subject-specific mapping from foot movements to command outputs was developed using Independent Component Analysis (ICA). Results showed that the ICA method could accurately identify subjects' foot motion patterns and significantly improve the prediction accuracy of motion directions from 68% to 88% as compared with the forward kinematics-based approach. This foot-machine interface can be applied for the teleoperation of industrial/surgical robots independently or in coordination with hands in the future.

Future of Flexible Robotic Endoscopy Systems

Robotics enables a variety of unconventional actuation strategies to be used for endoscopes, resulting in reduced trauma to the GI tract. For transmission of force to distally mounted endoscopic instruments, robotically actuated tendon sheath mechanisms are the current state of the art. Robotics in surgical endoscopy enables an ergonomic mapping of the surgeon movements to remotely controlled slave arms, facilitating tissue manipulation. The learning curve for difficult procedures such as endoscopic submucosal dissection and full-thickness resection can be significantly reduced. Improved surgical outcomes are also observed from clinical and pre-clinical trials. The technology behind master-slave surgical robotics will continue to mature, with the addition of position and force sensors enabling better control and tactile feedback. More robotic assisted GI luminal and NOTES surgeries are expected to be conducted in future, and gastroenterologists will have a key collaborative role to play.

Performance Control of Tendon-Driven Endoscopic Surgical Robots With Friction and Hysteresis

In this study, a new position control scheme for the tendon-sheath mechanism (TSM) which is used in flexible medical devices is presented. TSM is widely used in dexterous robotic applications because it can flexibly work in limited space, in constrained environments, and provides efficient power transmission from the external actuator to the distal joint. However, nonlinearities from friction and backlash hysteresis between the tendon and the sheath pose challenges in achieving precise position controls of the end effector. Previous studies on the TSM only address the control problem under the assumptions of known tendon-sheath configuration and known model parameters of the backlash hysteresis nonlinearity. These approaches can have adverse impact and limitations on the overall system performances and practical implementation. This paper presents a new approach to model and control the TSM-driven flexible robotic systems. The designed controller does not require exact knowledge of nonlinear friction and backlash hysteresis parameters, only their bounds are online estimated. Simulation and experimental validation results show that the proposed control scheme can significantly improve the tracking performances without the presence of the exact knowledge of the model parameters and the sheath configuration.