Telemanipulation of Autonomous Drones Using Digital Twins of Aerial Traffic

0 ratings - Please login to submit your rating.


``By touching an instrument placed in the southern gallery, a miniature Spanish cruiser anchored in the fountain lake on the lower floor, 90 feet away, was blown into the air. There was no connection between the transmitter and the vessel in the lake'' reported the New York Times in 1898. Although it was believed as magic by many, Nikola Tesla, by pushing the boundaries once again, became the ancestor of modern-day remotely controlled wireless drones -- ``The first of a race of robots, mechanical men which will do the laborious work of the human race'' -- by inventing the first radio-controlled vessel using radio waves -- an invention beyond its time. Human-robot dialogue as well as the manipulation of drones has evolved substantially since then in line with the development of autonomy. In this direction, a scalable agent-based platform -- the so-called AITL-WING-HITL -- was designed for the telemanipulation of autonomous unmanned aerial vehicles (A-UAVs) from the perspective of human-multirobot architecture using Digital Twins (DTs) of aerial traffic when the new driver -- AI agent -- runs into an unorthodox situation where autonomy is insufficient to cope with. The platform, equipped with an advanced precision control, force-sensitive patented mouse, the so-called WING, as an immersive device, was tested with participants using measurable parameters to verify the viability of the platform and to find out the further improvement points in the developed architecture. The results suggest that AITL (Autonomy-In-The-Loop) A-UAV agents and HITL (Human-In-The-Loop) Human Telemanipulator (HTM) agents can co-work to achieve a synergistic task performance using a socio-cognitive interaction model within the platform. Autonomous intervention mechanisms and human teleoperation scenarious are placed in this reposotory.


TravelTubes, i.e., safety travel tube (STT), are produced based on the most recent consecutive waypoints of autonomous drones to visualise if the path is in safe with no other aerial vehicles. Autonomous UAVs are said to be within Flight of Safety Route where there are no overlapping STTs.

``What happens if no intervention is performed either by the human or the system, resulting in collision'', and ``How the AITL-WING-HITL platform responds to the impending, probable and imminent collision risks autonomously with minimum deviation strategy'' were demonstrated to make the participants understand the objectives well in the episodes. Readers are referred to the 7-minute recorded videos, namely, ``01_No-intervention-no-CA.wmv'', and ``02_No-intervention-with-CA.wmv'' in the supplements to observe these two cases within the AITL-WING-HITL platform.

``FirstScenario.wmv'' is for an example of manoeuvres that are supposed to be executed by the participants in the first two episodes.



Funding Agency: 
The European Regional Development Fund’s (ERDF) business support programme
Grant Number: