Project blog authored by: Mohammad Divband Soorati, Project Lead Contact: Trustworthy human-swarm partnerships in extreme environments
One of the key challenges in human-swarm interaction is designing an interface with a usable visualization for human operators of the swarm. This is the first step for creating a human-swarm partnership. The goal is to allow the human operator of the swarm to easily understand the state of the swarm.
Title: Usable and Interpretable Human-Swarm Visualisations: A User Evaluation Study
Authors: Clark, J.R., Divband Soorati, M. & Ramchurn, S.D. (2022). Proceedings of the CIEHF Annual Conference 2022. Accepted 04 January 2022.
Abstract: In the field of robotic swarms, human factors research is deficient. Human-swarm interaction, the process in which humans work with collections of 50 or more unmanned and autonomous vehicles (UVs) to conduct activities, require an operator to track and manage vast numbers of robotic agents. Due to an increase in complexity, visualising a swarm is challenging and requires usable visualisation methods that do not detract from user interpretability and swarm transparency. This study explores two initial swarm displays for swarm geographical coverage – an individual point display, and a heatmap display. Users viewed a simulation of each display, provided subjective usability and acceptance ratings, and provided their display preferences related to eight contextual factors related to time constraints, swarm size, communication-constrained environments, displaying coverage and motion, time-criticality, error detection and transparency. It was found that heatmaps improved usability and acceptance, and were preferred for displaying coverage and motion, and when UV numbers are high or when time is limited.
Individual point displays, although being overall less usable, acceptable, and preferable, were still deemed as being a useful tool for detecting errors in swarm operation. It is therefore concluded that aggregated data displays may be a promising avenue of research for visualising swarm coverage, however, how this directly affects trust and situation awareness requires further research as this work domain continues to advance.
Figure 1: coverage display methods. Left = individual drone point display. Right = heat-map representing coverage
We now have a visualization method, a way to show the state of the swarm to the operator. The follow-up question is how the human operator can send the information or commands to the swarm. Our second work focuses on designing the entire interaction for a small number of human operators to control a large robot swarm.
Title: Designing a User-Centered Interaction Interface for Human–Swarm Teaming
Authors: Soorati, M.D., Clark, J.R., Ghofrani, J., Tarapore, D., Ramchurn, S.D. (2021). Featured article in MDPI Drones Journal. Accepted 02 November 2021.
Link: https://doi.org/10.3390/drones5040131
Abstract: A key challenge in human–swarm interaction is to design a usable interface that allows the human operators to monitor and control a scalable swarm. In our study, we restrict the interactions to only one-to-one communications in local neighbourhoods between UAV-UAV and operator-UAV. This type of proximal interactions will decrease the cognitive complexity of the human–swarm interaction to O(1). In this paper, a user study with 100 participants provides evidence that visualizing a swarm as a heat map is more effective in addressing usability and acceptance in human–swarm interaction.
We designed an interactive interface based on the users’ preference and proposed a controlling mechanism that allows a human operator to control a large swarm of UAVs. We evaluated the proposed interaction interface with a complementary user study. Our testbed and results establish a benchmark to study human–swarm interaction where a scalable swarm can be managed by a single operator.
Figure 2: Human operator receives updates about the change in the position of the disasters using a scalable interaction interface
Part II: Industry-driven Use-cases for Human-swarm Interaction