Reinforcement Learning for Robotic-assisted Percutaneous Coronary Intervention

MSc assignment

Project description 

Robotic-assisted percutaneous coronary intervention (R-PCI) aims to automate endovascular procedures to reduce clinicians’ exposure to X-ray radiation and improve the precision and stability of interventions. During PCI procedures, catheters and guidewires are navigated through coronary arteries to deliver balloons and stents to target lesions. However, navigation remains highly challenging due to the limited visual information provided by 2D X-ray fluoroscopy. 
 
Recently, reinforcement learning (RL) has shown great potential for autonomous navigation and decision-making in robotic endovascular interventions. However, training RL agents requires a realistic and efficient simulation environment. Within our research group, we have already developed a simulation framework for X-ray fluoroscopy and are currently developing corresponding hardware prototypes for navigation. 
 
In this master’s thesis project, the student will further develop the simulation environment by incorporating catheter and guidewire navigation functionalities. Subsequently, reinforcement learning methods will be explored to enable autonomous or semi-autonomous navigation based on fluoroscopic images and/or reconstructed 3D models of coronary arteries. Depending on the project progress, the developed algorithms may also be validated on a hardware setup. 

Possible research directions 

  • Development of catheter and guidewire simulation models 
  • Reinforcement learning for autonomous navigation 
  • Vision-based navigation using fluoroscopic images 
  • Integration of 3D coronary artery models into the navigation framework 
  • Sim-to-real transfer and domain randomization 
  • Experimental validation on robotic hardware 

Requirements 

  • A background in Electrical Engineering, Computer Science, Biomedical Engineering, Robotics, or related fields 
  • Experience with Python programming and deep learning frameworks such as PyTorch 
  • Familiarity with reinforcement learning, machine learning, or computer vision methods (or strong motivation to learn) 
  • Interest in robotics, medical imaging, and image-guided interventions 
  • Motivation to validate algorithms on hardware systems 

Keywords

Reinforcement Learning, Medical Robotics, Endovascular Navigation, Fluoroscopy, Simulation, Computer Vision, Robotics 

Daily Supervisor

Tianyuan Wang (https://www.ram.eemcs.utwente.nl/about-us/staff/tianyuan-wang)

References 

  1. Yao, T., Ban, M., Lu, B., Pei, Z., & Qi, P. (2025). Sim4EndoR: A Reinforcement Learning Centered Simulation Platform for Task Automation of Endovascular Robotics. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 824–830. 
  1. Yao, T., Wang, H., Lu, B., Ge, J., Pei, Z., Kowarschik, M., et al. (2025). Sim2Real Learning with Domain Randomization for Autonomous Guidewire Navigation in Robotic-Assisted Endovascular Procedures. IEEE Transactions on Automation Science and Engineering. 
  1. Xu, J., Li, B., Lu, B., Liu, Y. H., Dou, Q., & Heng, P. A. (2021). SurRoL: An Open-Source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1821–1828.