Cooperative Path-following Control of Remotely Operated Underwater Robots for Human Visual Inspection Task

Remotely operated vehicles (ROVs) have drawn much attention to underwater tasks, such as the inspection and maintenance of infrastructure. The workload of ROV operators tends to be high, even for the skilled ones. Therefore, assistance methods for the operators are desired. This study focuses on a task in which a human operator controls an underwater robot to follow a certain path while visually inspecting objects in the vicinity of the path. In such a task, it is desirable to realize the speed of trajectory control manually because the visual inspection is performed by a human operator. However, to allocate resources to visual inspection, it is desirable to minimize the workload on the path-following by assisting with the automatic control. Therefore, the objective of this study was to develop a cooperative path-following control method that achieves the above-mentioned task by expanding a robust path-following control law of nonholonomic wheeled vehicles. To simplify this problem, we considered a path-following and visual objects recognition task in a two-dimensional plane. We conducted an experiment with participants (n=16) who completed the task using the proposed method and manual control. The results showed that both the path-following errors and the workload of the participants were significantly smaller with the proposed method than with manual control. In addition, subjective responses demonstrated that operator attention tended to be allocated to objects recognition rather than robot operation tasks with the proposed method. These results indicate the effectiveness of the proposed cooperative path-following control method.

Motion Sickness Modeling with Visual Vertical Estimation and Its Application to Autonomous Personal Mobility Vehicles

Passengers (drivers) of level 3-5 autonomous personal mobility vehicles (APMV) and cars can perform non-driving tasks, such as reading books and smartphones, while driving. It has been pointed out that such activities may increase motion sickness. Many studies have been conducted to build countermeasures, of which various computational motion sickness models have been developed. Many of these are based on subjective vertical conflict (SVC) theory, which describes vertical changes in direction sensed by human sensory organs vs. those expected by the central nervous system. Such models are expected to be applied to autonomous driving scenarios. However, no current computational model can integrate visual vertical information with vestibular sensations. We proposed a 6 DoF SVC-VV model which add a visually perceived vertical block into a conventional six-degrees-of-freedom SVC model to predict VV directions from image data simulating the visual input of a human. Hence, a simple image-based VV estimation method is proposed. As the validation of the proposed model, this paper focuses on describing the fact that the motion sickness increases as a passenger reads a book while using an AMPV, assuming that visual vertical (VV) plays an important role. In the static experiment, it is demonstrated that the estimated VV by the proposed method accurately described the gravitational acceleration direction with a low mean absolute deviation. In addition, the results of the driving experiment using an APMV demonstrated that the proposed 6 DoF SVC-VV model could describe that the increased motion sickness experienced when the VV and gravitational acceleration directions were different.

Soft-Jig: A Flexible Sensing Jig for Simultaneously Fixing and Estimating Orientation of Assembly Parts

For assembly tasks, it is essential to firmly fix target parts and to accurately estimate their poses. Several rigid jigs for individual parts are frequently used in assembly factories to achieve precise and time-efficient product assembly. However, providing customized jigs is time-consuming. In this study, to address the lack of versatility in the shapes the jigs can be used for, we developed a flexible jig with a soft membrane including transparent beads and oil with a tuned refractive index. The bead-based jamming transition was accomplished by discharging only oil enabling a part to be firmly fixed. Because the two cameras under the jig are able to capture membrane shape changes, we proposed a sensing method to estimate the orientation of the part based on the behaviors of markers created on the jig's inner surface. Through estimation experiments, the proposed system could estimate the orientation of a cylindrical object with a diameter larger than 50 mm and an RMSE of less than 3 degrees.

Simultaneous Achievement of Driver Assistance and Skill Development in Shared and Cooperative Controls

Advanced driver assistance systems have successfully reduced drivers' workloads and increased safety. On the other hand, the excessive use of such systems can impede the development of driving skills. However, there exist collaborative driver assistance systems, including shared and cooperative controls, which can promote effective collaboration between an assistance system and a human operator under appropriate system settings. Given an effective collaboration setup, we address the goal of simultaneously developing or maintaining driving skills while reducing workload. As there has been a paucity of research on such systems and their methodologies, we discuss a methodology applying shared and cooperative controls by considering related concepts in the skill training field. Reverse parking assisted by haptic shared control is presented as a means of increasing performance during assistance, while skill improvement following assistance is used to demonstrate the possibility of simultaneous achievement of driver assistance through the reduction of workload and skill improvement.

Knee Motion Generation Method for Transfemoral Prosthesis based on Kinematic Synergy and Inertial Motion

Previous research has shown that the effective use of inertial motion (i.e., less or no torque input at the knee joint) plays an important role in achieving a smooth gait of transfemoral prostheses in the swing phase. In our previous research, a method for generating a timed knee trajectory close to able-bodied individuals, which leads to sufficient clearance between the foot and the floor and the knee extension, was proposed using the inertial motion. Limb motions are known to correlate with each other during walking. This phenomenon is called kinematic synergy. In the present study, we measure gaits in level walking of able-bodied individuals with a wide range of walking velocities. We show that this kinematic synergy also exists between the motions of the intact limbs and those of the knee as determined by the inertial motion technique. We then propose a new method for generating the motion of the knee joint using its inertial motion close to the able-bodied individuals in mid-swing based on its kinematic synergy, such that the method can adapt to the changes in the motion velocity. The numerical simulation results show that the proposed method achieves prosthetic walking similar to that of able-bodied individuals with a wide range of constant walking velocities and termination of walking from steady-state walking. Further investigations have found that a kinematic synergy also exists at the start of walking. Overall, our method successfully achieves knee motion generation from the initiation of walking through steady-state walking with different velocities until termination of walking.

Haptic Shared Control in Steering Operation Based on Cooperative Status Between a Driver and a Driver Assistance System

Haptic shared control is expected to achieve a smooth collaboration between humans and automated systems, because haptics facilitate mutual communication. A methodology for sharing a given task is important to achieve effective shared control. Therefore, the appropriate cooperative relationship between a human operator and automated system should be considered. This paper proposes a methodology to evaluate the cooperative status between the operator and the automated system in the haptic shared control of a steering operation using a pseudo-power pair of torque from each agent and the vehicle lateral velocity as each agent's contribution to vehicle motion. This method allows us to estimate cooperative status based on two axes: the initiative holder and the intent consistency between the two agents. A control method for a lane-keeping assist system (LKAS) that enables drivers to change lanes smoothly is proposed based on the estimated cooperative status. A gain-tuning control method based on the estimated cooperative status is proposed to decrease the assistance system's pseudo-power when intent inconsistency occurs. A method for switching the followed lane to match the driver's and assistance system's intentions is also proposed. A user study using a driving simulator is conducted to demonstrate the effectiveness of the proposed methods. The results demonstrate that the proposed methods facilitate smooth driver-initiated lane changes without significantly affecting the driver's torque or steering wheel angle while significantly improve lane-keeping performance.