Background: Predicting human upper-limb motion is important for robots that must work proactively and safely with people. In collaborative and object-manipulation tasks, the future movement of one hand is not determined only by its previous trajectory. It also depends on the state of the other hand, the pose and geometry of the manipulated object, the available contact regions, and the action that the object affords. This project therefore combines AI-based visual–inertial perception with movement primitives, inter-joint synergies and physical constraints to predict human motion in an object-centred collaborative context.
Task: The student will develop a closed-loop model that predicts the future state of the left arm and hand from partial observations of the left arm, the current and recent motion of the right hand, and information about the relevant object. A neural observation model will estimate the current body state, movement phase, possible action and object affordance, including likely contact regions or functional object parts. Conditional probabilistic movement primitives will describe possible left-hand actions, while synergy and simplified upper-limb dynamics will generate coordinated and physically feasible shoulder–elbow–wrist trajectories. New visual and IMU observations will continuously update the predicted action, target and trajectory distribution.
Main Goal: The main goal is to determine whether information from the other hand and the object can improve early prediction of the left hand’s future trajectory, contact location and final functional state. The system will generate a time-dependent probability cloud for the left hand, elbow and forearm, conditioned on the observed collaborative context. It should preserve multiple possible futures when the object affords several actions, while progressively reducing uncertainty as the right-hand motion, object state and early left-hand movement reveal the intended collaboration.
Side Goal: The project will compare individual-motion prediction with increasingly contextual models using left-hand history alone, both hands, both hands plus object pose, and the complete model including affordance, synergy and physical constraints. It will examine unseen objects, altered object orientations, visual occlusion and changes in collaborative roles. If time permits, the predicted probability cloud can be connected to a simulated robot planner, allowing the robot to select a safe assisting motion, prepare an object handover or avoid regions likely to be occupied by the human arm.