Intra-Vision: Soft tissue deformation estimation by using SOFA and computer vision in robotic laparoscopic surgery

MSc assignment


Minimally invasive laparoscopic surgery has been widely applied in clinical practice. Stereovision endoscopes are used in robotic systems (like da Vinci robot), handheld endoscopes may also be used to provide 3D depth images. However, this superficial 3D perception cannot represent the actual transitions and deformations that happened inside the soft tissue. For example, dissecting the tumor from the liver still remains a huge challenge to dynamically locate and measure the position of the tumor embedded in the liver. Current methods highly rely on preoperative images, using elastic registration to match the preoperative images to the patient. However, these methods cannot provide a dynamic update on the deformation and transition of soft tissues, resulting in low accuracy in intraoperative tracking of the targeting tissue/organ during surgical procedures.  Dynamic tracking of the soft tissue is challenging, but the future is promising. Advancements in intraoperative soft tissue deformation sensing are paving a path to better, more predictable outcomes.


In this master thesis, an open-source simulation framework SOFA ( and surface tracking will be integrated into a soft tissue tracking system. Surface tracking provides a surface model of the target via computer vision or optical tracking technologies. The deformation of the surface model will be monitored in real-time to feed into SOFA model to estimate the deformation underlying the surface. The force applied to the soft tissue leads to deformation. Such information can also be included in the analysis, similar to the Finite element method. The estimated deformation and localization of the tumor therefore can be used for robots or clinicians to perform more accurate manipulations of surgical instruments and to dissect tumorous tissues.


  • to investigate the basic concept and technical feasibility;
  • to setup proper surface tracking approaches (computer vision, optical tracking, or LIDAR sensor) to be used in laparoscopic surgery;
  • to design a proper phantom to simulate the scenario of soft tissue deformation in laparoscopic surgery;
  • to develop a real-time soft tissue modelling approach in SOFA for tracking the inner tissue deformation;
  • to validate the proposed method on the phantoms.

Requirements for students:

  • hands-on experience on programming (c++ or python)
  • interests on computer vision and modedlling.
  • Experience in experimental design and phantom design