In a traditional visual servoing pipeline, waypoints are generated using a spline and given as a reference to a low-level controller, i.e. a model predictive controller. This approach has the primary limitation that it is not able to explore an environment to find an inspection target, and thus lacks high-level reasoning. On the contrary, diffusion models, which are AI generative models, have been proven effective in generating trajectories that can steer the robot to the desired target view even if the target is initially not visible. Solely relying on diffusion is not yet viable because they lack the robustness of traditional methods and require a high computational effort.
This thesis investigates the development of a control scheme for visual servoing that integrates the sampling capabilities of diffusion models and the performance guarantees of an MPC-based control approach. The former allows exploiting training to embed useful knowledge about the environment, e.g. the relative position of the target given the current image, while the latter allows imposing constraints for dynamical feasibility. The control scheme will be tested on a UAV equipped with an onboard camera. This thesis falls within the scope of the European AutoASSESS project aimed at developing UAVs that can autonomously perform contact aerial inspection in ballast water tanks of cargo ships.
Expected outcomes:
- A control scheme fusing the advantages of traditional MPC and latent diffusion approaches
- Simulation-based results showing the resilience of the proposed approach
- If feasible within the project timeframe, deployment and testing of the developed framework in real-world experiments