Background: Human motion prediction is important for safe and proactive physical human–robot interaction. Existing AI-based methods can learn complex relationships between IMU, vision and future movement, but may generalize poorly and produce physically implausible predictions. Traditional approaches, such as Dynamic Movement Primitives, inter-joint synergies and musculoskeletal models, provide interpretable knowledge about movement phase, coordination and physical dynamics. This project aims to combine the flexibility of AI with these structured representations to achieve more general and reliable upper-limb motion prediction.
Task: The student will develop a closed-loop framework that predicts future upper-limb movement from partial visual and IMU observations. A neural network will estimate the current joint state, movement phase, intended target and possible action primitive. DMPs will describe how the movement develops over time, while a low-dimensional synergy representation will describe coordination among the shoulder, elbow and wrist. A simplified physical model will then generate feasible future trajectories. As new sensor observations arrive, the system will compare them with its previous short-term predictions and continuously update the predicted future.
Main Goal: The main goal is to generate an accurate and calibrated probability distribution of possible future upper-limb trajectories rather than a single deterministic prediction. The predicted trajectories will be converted into a time-dependent spatial probability cloud describing the likely future positions of the hand, elbow and forearm. The project will investigate whether combining neural observation models, probabilistic movement primitives, inter-joint synergies and physical constraints improves prediction accuracy, physical plausibility and generalization across participants, targets, movement speeds and observation conditions.
Side Goal: The project will evaluate the contribution of each component by comparing the complete framework with purely data-driven models, standard DMP or ProMP predictors, and models without synergy, physical constraints or online feedback. It will also study robustness to visual occlusion, missing IMU measurements and unexpected changes in movement direction. If time and suitable measurements are available, a simplified differentiable musculoskeletal model may be explored as an extension. Finally, the predicted probability cloud may be demonstrated in a simulated collaborative-robot scenario for human-aware path planning and collision-risk estimation.