Robust Instrument Tracking for Endoscopic Ear Surgery Simulation
A recently developed endoscopic ear surgery simulator tracks physical instruments using a camera and AprilTag markers. Although this enables trainees to control a virtual endoscope and interact with an anatomical model, tracking can become unstable when markers are obscured or instruments move outside the camera’s favourable viewing angles.
This project investigates whether camera–IMU sensor fusion can improve the accuracy and continuity of instrument-tip tracking. The student will integrate IMUs into the endoscope and a second instrument, calibrate their relationship to the instrument tips, and develop a tracking method that combines inertial measurements with camera-based pose estimates. The camera will continue to provide position measurements relative to the anatomical model, while the IMUs may help maintain tracking during short periods of marker occlusion.
The student will compare the existing camera-based system with the fused system using an independent reference measurement. Evaluation will cover tip-position and orientation error, tracking loss and recovery, and response delay during representative training movements. The expected outcome is a working prototype and a quantitative assessment of when sensor fusion improves tracking—and where its limitations remain.
Suitable for: A student interested in robotics, sensor fusion, computer vision, embedded sensing, and experimental evaluation.