Introduction:
Imagine controlling a prosthetic arm not just with muscle signals but also through your gaze—creating a seamless, intuitive user experience. This MS thesis project explores the latest deep-learning techniques to predict forearm movements by integrating surface electromyography (sEMG) signals and eye-tracking data. Traditional methods, which rely on static upper arm positions, limit the potential for natural movement. By combining muscle signals from the residual limb with gaze data, this research aims to improve prosthetic robotics, offering enhanced functionality, improved movement accuracy, and an overall more responsive user experience.
Why it matters? Prosthetic robotics becomes more intuitive and dynamic, allowing greater freedom in real-world conditions where the upper arm is in motion.
Project Objectives:
- Design the Experiment: Develop an experiment that collects synchronized data from surface EMG signals, eye-tracking, and forearm movements. This will form the backbone of the system.
- Create a Deep-Learning Model: Use advanced AI structures (e.g., sequence decoders, GANs, etc., anything is possible) to predict dynamic forearm states from the combined data. You'll design the model and explain the rationale behind your decision.
- Analyze and Evaluate: Measure the system’s accuracy in real-world conditions—when the upper arm is in motion—and visualize the predicted movements.
Key Research Questions:
- How can the experiment be designed to gather synchronized datasets of sEMG, eye-tracking, and forearm movements? What should be prioritized during data collection?
- How do deep learning techniques enhance the prediction of forearm movements in real time, compared to existing static-arm methods? What makes this approach better?
- What post-processing steps can further refine predictions, making the system more reliable in dynamic, everyday settings?
- How do we evaluate the model's performance in terms of prediction accuracy? What metrics will provide the best insights into the system's efficiency?
Expected Outcomes:
- A full understanding of how to synchronize and leverage sEMG and eye-tracking data for forearm movement predictions.
- A novel deep-learning model tailored to this complex task, capable of predicting movements with high accuracy during upper-arm motion.
- Real-time improvements through post-processing, along with a complete visualization of the forearm’s predicted movements.
- An evaluation framework to assess and continually improve the system's performance in real-world scenarios.