On Multi-Human Multi-Robot Remote Interaction: A Study of Transparency, Inter-Human Communication, and Information Loss in Remote Interaction

In this paper, we investigate how to design an effective interface for remote multi-human multi-robot interaction. While significant research exists on interfaces for individual human operators, little research exists for the multi-human case. Yet, this is a critical problem to solve to make complex, large-scale missions achievable in which direct human involvement is impossible or undesirable, and robot swarms act as a semi-autonomous agents. This paper's contribution is twofold. The first contribution is an exploration of the design space of computer-based interfaces for multi-human multi-robot operations. In particular, we focus on information transparency and on the factors that affect inter-human communication in ideal conditions, i.e., without communication issues. Our second contribution concerns the same problem, but considering increasing degrees of information loss, defined as intermittent reception of data with noticeable gaps between individual receipts. We derived a set of design recommendations based on two user studies involving 48 participants.

Direct and Indirect Communication in Multi-Human Multi-Robot Interaction

How can multiple humans interact with multiple robots? The goal of our research is to create an effective interface that allows multiple operators to collaboratively control teams of robots in complex tasks. In this paper, we focus on a key aspect that affects our exploration of the design space of human-robot interfaces -- inter-human communication. More specifically, we study the impact of direct and indirect communication on several metrics, such as awareness, workload, trust, and interface usability. In our experiments, the participants can engage directly through verbal communication, or indirectly by representing their actions and intentions through our interface. We report the results of a user study based on a collective transport task involving 18 human subjects and 9 robots. Our study suggests that combining both direct and indirect communication is the best approach for effective multi-human / multi-robot interaction.

Transparency in Multi-Human Multi-Robot Interaction

Transparency is a key factor in improving the performance of human-robot interaction. A transparent interface allows humans to be aware of the state of a robot and to assess the progress of the tasks at hand. When multi-robot systems are involved, transparency is an even greater challenge, due to the larger number of variables affecting the behavior of the robots as a whole. Significant effort has been devoted to studying transparency when single operators interact with multiple robots. However, studies on transparency that focus on multiple human operators interacting with a multi-robot systems are limited. This paper aims to fill this gap by presenting a human-swarm interaction interface with graphical elements that can be enabled and disabled. Through this interface, we study which graphical elements are contribute to transparency by comparing four "transparency modes": (i) no transparency (no operator receives information from the robots), (ii) central transparency (the operators receive information only relevant to their personal task), (iii) peripheral transparency (the operators share information on each others' tasks), and (iv) mixed transparency (both central and peripheral). We report the results in terms of awareness, trust, and workload of a user study involving 18 participants engaged in a complex multi-robot task.

Improving Human Performance Using Mixed Granularity of Control in Multi-Human Multi-Robot Interaction

Due to the potentially large number of units involved, the interaction with a multi-robot system is likely to exceed the limits of the span of apprehension of any individual human operator. In previous work, we studied how this issue can be tackled by interacting with the robots in two modalities --- environment-oriented and robot-oriented. In this paper, we study how this concept can be applied to the case in which multiple human operators perform supervisory control on a multi-robot system. While the presence of extra operators suggests that more complex tasks could be accomplished, little research exists on how this could be achieved efficiently. In particular, one challenge arises --- the out-of-the-loop performance problem caused by a lack of engagement in the task, awareness of its state, and trust in the system and in the other operators. Through a user study involving 28 human operators and 8 real robots, we study how the concept of mixed granularity in multi-human multi-robot interaction affects user engagement, awareness, and trust while balancing the workload between multiple operators.

Mixed-Granularity Human-Swarm Interaction

We present an augmented reality human-swarm interface that combines two modalities of interaction: environment-oriented and robot-oriented. The environment-oriented modality allows the user to modify the environment (either virtual or physical) to indicate a goal to attain for the robot swarm. The robot-oriented modality makes it possible to select individual robots to reassign them to other tasks to increase performance or remedy failures. Previous research has concluded that environment-oriented interaction might prove more difficult to grasp for untrained users. In this paper, we report a user study which indicates that, at least in collective transport, environment-oriented interaction is more effective than purely robot-oriented interaction, and that the two combined achieve remarkable efficacy.