Abstract: The complex dynamic characteristics and changeable marine environment pose great challenges to the design of the unmanned underwater vehicle (UUV) controller. In practical application, the parameter of the controller will be fixed after frequent manual debugging, which is unable to adapt to the changes of the environment during the control. In view of the above problems, a parameter adaptive S-plane control method based on the reinforcement learning is proposed referring to the adaptive control idea. The adaptive control method is used to optimize and automatically tune the controller parameter under different environments. This method is trained by Q-learning algorithm. The optimal mapping between the input status and output action is sought through the self-learning mechanism of Q-learning. Simulation results show that the proposed method can adjust the controller parameter in real time with excellent control effect and environment adaptability.

Key words: unmanned underwater vehicle (UUV), reinforcement learning, S-plane control, parameter adaptive

摘要： 复杂的动力学特性及多变的海洋环境对无人水下航行器（UUV）控制器的设计提出了巨大挑战,在实际应用中,控制器的参数经人工调试后便固化,在控制过程中无法适应环境的变化。针对上述难题,该文借鉴自适应控制思想,提出一种基于强化学习的参数自适应S面控制方法,采用自适应控制方式实现不同环境下控制器参数的优化和自动整定。该方法采用Q学习算法进行训练,通过Q学习的自学习机制寻找输入状态和输出动作间的最优映射。仿真试验表明,所提方法能对控制器的参数进行实时在线调整,具备良好的控制效果和环境自适应能力。

关键词: 无人水下航行器, 强化学习, S面控制, 参数自适应