In the current video assisted thoracic surgery (VATS), the assistant of a surgeon controls the movement of the endoscopic camera. A head motion controlled endoscope system was developed enabling the surgeon to control the endoscope directly using his/her head motion. The image output obtained from the endoscope was displayed on the screen. A compensation of the image rotation is done to remove the rotation component in the motion of the output image. This is done to keep the directionality in the image is the same as the head movement of the user. The compensation was done using the angle of rotation of the servo encoder measurement at the image plane and the angle of rotation obtained from optical flow based estimation. However, the image rotation compensation for the image output was not accurate because the output obtained from optical flow based estimation was suffering from drift and the output obtained from servo motor was not accurate in finer motion.
During this Master’s assignment, a sensor fusion technique for obtaining a better image rotation compensation for the head motion controlled endoscope camera system has been developed and evaluated. The rotation obtained from the servo motor and the rotation obtained from the optical flow based estimation on the image output will be fused together. Complementary filter and the Extended Kalman Filter (EKF) are the two sensor fusion algorithms used in this study. The complete software implementation was done on ROS Kinetic.