Context
Multi-Modal Teleoperation
Teleoperation is the remote control of a robot or other machine from a distance. This can be as simple as using a remote desktop environment to control a computer somewhere else. Within robotics, we are more interested in operating robots over a distance. This introduces numerous complications, such as time delays between command and execution. For this assignment, we are specifically interested in systems with multiple modalities, i.e., different senses which are used to control a remote robot as naturally as possible. An example of such a multi-modal tele-operated setup is the i-Botics XPRIZE submission, which can be seen here and here.
Latency and Temporal Congruency
Within Teleoperation, a network often exists between the controller and the remote environment. This will inevitably lead to delays between input and output. This is often called Latency. In multi-modal systems, the issue is even worse, as multiple information streams are now sent around. Preferably, all streams arrive simultaneously, but network routing often causes this not to be the case. Furthermore, it is preferred that everything is received as fast as possible. However, for multiple modalities, a point of importance is temporal congruency, sometimes also called synchronicity. This describes the need for information to arrive within a certain temporal window to be interpreted as one congruous signal, rather than multiple different responses. Finally, for some modalities, it is found that the stability of the signal is also of great importance. Especially for Audio and Video, this jitter seems to have major implications. This also leads to a clear discrepancy between signal stability (jitter), temporal congruency and latency requirements.
Goal
The goal of this assignment is to explore the trade-off between the factors named above and on how to balance latency, jitter and temporal congruency better in tele-operation systems. Towards this goal, you will be doing the following:
- Design and implement a system allowing for experimentation on the trade-off between jitter, latency and temporal synchronicity.
- Design and execute a user experiment which evaluates the trade-off of these components.
Your profile
For this assignment, the best-equipped students are those who:
- Are students MSc. I-Tech or MSc. Robotics specialisation Human-Robot Interaction.
- Have some programming skills, most notably C++ and ROS.