PIRATE Robot Autonomous Navigation through Complex Pipe Networks using Reinforcement Learning

Finished: 2021-09-28

MSc assignment

The approach considered in this project uses Reinforcement Learning (RL) for learning a navigation strategy for the PIRATE robot. This model chooses a setpoint value for the motor control taking into account the sensory readings from the robot.

Learning the optimal strategy with RL on the real robot is far from being an easy task. On each new training episode, the position and pose of the robot needs to be manually reset. Additionally, collecting the amount of data needed for learning a robust model from the interaction between the robot and the environment would take a lot of time. Thus, it is preferred to firstly train an initial navigation controller inside a simulation environment, by using the dynamic model of the robot. The resulting RL model obtained after this training can then be hopefully transferred to the real system.

Previous research, on which this project is based, proposes a Hierarchical Reinforcement Learning (HRL) control model for the navigation of the PIRATE robot in a navigation environment through straight pipes and 90 degrees turns.

The aim of this project is to improve upon the existing research. Firstly, the current model is adapted to allow navigation through T-junctions and pipe diameter changes. Secondly, the robustness of the model needs to be increased, such that the resulting navigation controller can be moved from a simulation environment on the real robot.