Version 1
: Received: 30 September 2022 / Approved: 30 September 2022 / Online: 30 September 2022 (10:35:06 CEST)
How to cite:
Feng, W.; Han, C.; Lian, F.; Liu, X. A Data-efficiency Training Framework for Deep Reinforcement Learning. Preprints2022, 2022090483. https://doi.org/10.20944/preprints202209.0483.v1
Feng, W.; Han, C.; Lian, F.; Liu, X. A Data-efficiency Training Framework for Deep Reinforcement Learning. Preprints 2022, 2022090483. https://doi.org/10.20944/preprints202209.0483.v1
Feng, W.; Han, C.; Lian, F.; Liu, X. A Data-efficiency Training Framework for Deep Reinforcement Learning. Preprints2022, 2022090483. https://doi.org/10.20944/preprints202209.0483.v1
APA Style
Feng, W., Han, C., Lian, F., & Liu, X. (2022). A Data-efficiency Training Framework for Deep Reinforcement Learning. Preprints. https://doi.org/10.20944/preprints202209.0483.v1
Chicago/Turabian Style
Feng, W., Feng Lian and Xia Liu. 2022 "A Data-efficiency Training Framework for Deep Reinforcement Learning" Preprints. https://doi.org/10.20944/preprints202209.0483.v1
Abstract
Sparse reward long horizon task is a major challenge for deep reinforcement learning algorithm. One of the key barriers is data-inefficiency. Even in the simulation environment, it usually takes weeks to training the agent. In this study, a data-efficiency training framework is proposed, where a curriculum learning is design for the agent in the simulation scenario. Different distributions of the initial state are set for the agent to get more informative reward during the whole training process. A fine-tuning of the parameters in the output layer of the neural network for value function is conduct to bridge the gap between sim-to-real. An experiment of UAV maneuver control is conducted in the proposed training framework to verify the method more efficient. We demonstrate that data-efficiency is different for the same data in different training stages.
Keywords
deep reinforcement learning; data efficient; curriculum learning; transfer learning
Subject
Engineering, Control and Systems Engineering
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.