You are currently viewing a beta version of our website. If you spot anything unusual, kindly let us know.
Preprint
Article
Modeling an Inverted Pendulum via Differential Equations and Reinforcement Learning Techniques
Altmetrics
Downloads
2489
Views
503
Comments
0
This version is not peer-reviewed
Abstract
The prevalence of differential equations as a mathematical technique has refined the fields of control theory and constrained optimization due to the newfound ability to accurately model chaotic, unbalanced systems. However, in recent research, systems are increasingly more nonlinear and difficult to model using Differential Equations only. Thus, a newer technique is to use policy iteration and Reinforcement Learning, techniques that center around an action and reward sequence for a controller. Reinforcement Learning (RL) can be applied to control theory problems since a system can robustly apply RL in a dynamic environment such as the cartpole system (an inverted pendulum). This solution successfully avoids use of PID or other dynamics optimization systems, in favor of a more robust, reward-based control mechanism. This paper applies RL and Q-Learning to the classic cartpole problem, while also discussing the mathematical background and differential equations which are used to model the aforementioned system.
Keywords:
Subject: Computer Science and Mathematics - Mathematics
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Submitted:
09 May 2020
Posted:
10 May 2020
You are already at the latest version
Alerts
This version is not peer-reviewed
Submitted:
09 May 2020
Posted:
10 May 2020
You are already at the latest version
Alerts
Abstract
The prevalence of differential equations as a mathematical technique has refined the fields of control theory and constrained optimization due to the newfound ability to accurately model chaotic, unbalanced systems. However, in recent research, systems are increasingly more nonlinear and difficult to model using Differential Equations only. Thus, a newer technique is to use policy iteration and Reinforcement Learning, techniques that center around an action and reward sequence for a controller. Reinforcement Learning (RL) can be applied to control theory problems since a system can robustly apply RL in a dynamic environment such as the cartpole system (an inverted pendulum). This solution successfully avoids use of PID or other dynamics optimization systems, in favor of a more robust, reward-based control mechanism. This paper applies RL and Q-Learning to the classic cartpole problem, while also discussing the mathematical background and differential equations which are used to model the aforementioned system.
Keywords:
Subject: Computer Science and Mathematics - Mathematics
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions
Yi-Liang Yeh
et al.
Machines,
2021
Improved Q-Learning Method for Linear Discrete-Time Systems
Jian Chen
et al.
Processes,
2020
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated