Author: Thomas Chaffre
Chaffre, Thomas, 2023 Reinforcement Learning and Sim-to-Real Transfer for Adaptive Control of AUV, Flinders University, College of Science and Engineering
Terms of Use: This electronic version is (or will be) made publicly available by Flinders University in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. You may use this material for uses permitted under the Copyright Act 1968. If you are the owner of any included third party copyright material and/or you believe that any material has been made available without permission of the copyright owner please contact copyright@flinders.edu.au with the details.
Sea disturbances vary unrelentingly making it necessary for autonomous underwater vehicles (AUVs) to continuously adapt their control systems to accommodate such changes. The complexity and dynamics of underwater processes are, however, difficult to be described analytically. While adaptive control systems could potentially be used to handle such changes, the field of adaptive control faces three main challenges: the dependency on the availability of a priori knowledge of the underlying processes; the need for well defined governing equations; and the implementation of these equations on physical systems. In the context of the underwater environment, these descriptions are not readily available. The objective of this thesis was to formalize a novel learning-based adaptive control using deep reinforcement learning and adaptive pole placement control to compensate for the known part of the process and to extract information on the unknown part directly from sensors feedback so as to compensate for the unobservable current disturbance. In addition, we proposed a novel experience replay mechanism that considers the characteristic of the biological replay mechanism. The methods were validated in simulation and in real life, demonstrating the benefits of combining both theories against using them separately.
Keywords: Adaptive Control, Deep Reinforcement Learning, Robotics, Underwater Vehicles
Subject: Engineering thesis
Thesis type: Doctor of Philosophy
Completed: 2023
School: College of Science and Engineering
Supervisor: Karl Sammut