Advanced Policy Learning Near-Optimal Regulation

Ding Wang; Xiangnan Zhong

doi:10.1109/JAS.2019.1911489

Volume 6 Issue 3

May 2019

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2019 > 6(3): 743-749

Ding Wang and Xiangnan Zhong, "Advanced Policy Learning Near-Optimal Regulation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 743-749, May 2019. doi: 10.1109/JAS.2019.1911489

Citation:

Ding Wang and Xiangnan Zhong, "Advanced Policy Learning Near-Optimal Regulation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 743-749, May 2019. doi: 10.1109/JAS.2019.1911489

Ding Wang and Xiangnan Zhong, "Advanced Policy Learning Near-Optimal Regulation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 743-749, May 2019. doi: 10.1109/JAS.2019.1911489

Citation:

Ding Wang and Xiangnan Zhong, "Advanced Policy Learning Near-Optimal Regulation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 743-749, May 2019. doi: 10.1109/JAS.2019.1911489

PDF( 3569 KB)

Advanced Policy Learning Near-Optimal Regulation

doi: 10.1109/JAS.2019.1911489

Ding Wang^1
,,
Xiangnan Zhong^2
,

1.
Faculty of Information Technology, Beijing University of Technology, and also with the Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, China
2.
Department of Electrical Engineering, University of North Texas, Denton TX 76203-5017 USA

Funds:

the National Natural Science Foundation of China 61773373

the National Natural Science Foundation of China U1501251

the National Natural Science Foundation of China 61533017

More Information

Author Bio:
Ding Wang (M'15) received the B.S. degree in mathematics from Zhengzhou University of Light Industry, Zhengzhou, China, in 2007, the M.S. degree in operations research and cybernetics from Northeastern University, Shenyang, China, in 2009, and the Ph.D. degree in control theory and control engineering from Institute of Automation, Chinese Academy of Sciences, Beijing, China, in 2012
From 2015 to 2017, he was a Visiting Scholar with the Department of Electrical, Computer, and Biomedical Engineering, University of Rhode Island, Kingston, RI, USA. He was an Associate Professor with The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. He is currently a Professor with the Faculty of Information Technology, Beijing University of Technology. He has authored or co-authored over 120 journal and conference papers and three monographs. His current research interests include adaptive and learning systems, computational intelligence, and intelligent control
Dr. Wang was a recipient of the Excellent Doctoral Dissertation Award of Chinese Academy of Sciences in 2013, and a nomination of the Excellent Doctoral Dissertation Award of Chinese Association of Automation in 2014. He was selected for the Young Elite Scientists Sponsorship Program by the China Association for Science and Technology in 2017, and also selected for the Youth Innovation Promotion Association of the Chinese Academy of Sciences in 2018. He was the Finance Chair of the 12th World Congress on Intelligent Control and Automation in 2016, and the Publications Chair of the 24th International Conference on Neural Information Processing in 2017. He currently or formerly serves as an Associate Editor of the IEEE Transactions on Neural Networks and Learning Systems, Neurocomputing, and Acta Automatica Sinica. (email: dingwang@bjut.edu.cn)

Xiangnan Zhong (M'17) received the B.S. degree in automation and the M.S. degree in control theory and control engineering from Northeastern University, Shenyang, China, in 2010 and 2012, respectively, and the Ph.D. degree in electrical, computer, and biomedical engineering from the University of Rhode Island (URI), Kingston, RI, USA, in 2017
She is currently an Assistant Professor with the Department of Electrical Engineering, University of North Texas, Denton, TX, USA. Her current research interests include computational intelligence, reinforcement learning, cyber physical systems, networked control systems, and intelligent control
Dr. Zhong was a recipient of the Chinese Government Award for Outstanding Students Abroad by the Chinese government in 2017, and the URI Enhancement of Graduate Research Award in 2016. She has been actively involved in numerous conference and workshop organization committees in the society. (email: xiangnan.zhong@unt.edu)
Corresponding author: Ding Wang, e-mail: dingwang@bjut.edu.cn
Received Date: 2018-07-03
Revised Date: 2018-08-06
Accepted Date: 2018-09-25

Abstract

Abstract

Designing advanced design techniques for feedback stabilization and optimization of complex systems is important to the modern control field. In this paper, a near-optimal regulation method for general nonaffine dynamics is developed with the help of policy learning. For addressing the nonaffine nonlinearity, a pre-compensator is constructed, so that the augmented system can be formulated as affine-like form. Different cost functions are defined for original and transformed controlled plants and then their relationship is analyzed in detail. Additionally, an adaptive critic algorithm involving stability guarantee is employed to solve the augmented optimal control problem. At last, several case studies are conducted for verifying the stability, robustness, and optimality of a torsional pendulum plant with suitable cost.
- Adaptive critic algorithm,
- learning control,
- neural approximation,
- nonaffine dynamics,
- optimal regulation

FullText(HTML)

References(18)

References

[1]	P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling, " in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992.
[2]	D. Liu, Q. Wei, D. Wang, X. Yang, and H. Li, Adaptive Dynamic Programming with Applications in Optimal Control. Cham, Switzerland: Springer, 2017.
[3]	D. Liu, Y. Xu, Q. Wei, and X. Liu, "Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming, " IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 36-46, 2018. doi: 10.1109/JAS.2017.7510739
[4]	Z. Fu, W. Xie, S. Rakheja, and J. Na, "Observer-based adaptive optimal control for unknown singularly perturbed nonlinear systems with input constraints, " IEEE/CAA J. Autom. Sinica, vol. 4, no. 1, pp. 48-57, 2017. doi: 10.1109/JAS.2017.7510322
[5]	X. Zhong, H. He, D. Wang, and Z. Ni, "Model-free adaptive control for unknown nonlinear zero-sum differential game, " IEEE Trans. Cybernetics, vol. 48, no. 5, pp. 1633-1646, 2018. doi: 10.1109/TCYB.2017.2712617
[6]	W. Gao and Z. P. Jiang, "Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems, " IEEE Trans. Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2614-2624, 2018. doi: 10.1109/TNNLS.2017.2761718
[7]	D. Wang, H. He, C. Mu, and D. Liu, "Intelligent critic control with disturbance attenuation for affine dynamics including an application to a microgrid system, " IEEE Trans. Industrial Electronics, vol. 64, no. 6, pp. 4935-4944, 2017. doi: 10.1109/TIE.2017.2674633
[8]	D. Wang, H. He, and D. Liu, "Intelligent optimal control with critic learning for a nonlinear overhead crane system, " IEEE Trans. Industrial Informatics, vol. 14, no. 7, pp. 2932-2940, 2018. doi: 10.1109/TII.2017.2771256
[9]	J. Li, T. Chai, F. L. Lewis, J. Fan, Z. Ding, and J. Ding, "Off-policy Q-learning: Set-point design for optimizing dual-rate rougher flotation operational processes, " IEEE Trans. Industrial Electronics, vol. 65, no. 5, pp. 4092-4102, 2018. doi: 10.1109/TIE.2017.2760245
[10]	D. Wang, H. He, and D. Liu, "Adaptive critic nonlinear robust control: A survey, " IEEE Trans. Cybernetics, vol. 47, no. 10, pp. 3429-3451, 2017. doi: 10.1109/TCYB.2017.2712188
[11]	Q. Y. Fan and G. H. Yang, "Adaptive nearly optimal control for a class of continuous-time nonaffine nonlinear systems with inequality constraints, " ISA Trans., vol. 66, pp. 122-133, 2017. doi: 10.1016/j.isatra.2016.10.019
[12]	B. Kiumarsi, W. Kang, and F. L. Lewis, "$H_{infty}$ control of nonaffine aerial systems using off-policy reinforcement learning, " Unmanned Systems, vol. 4, no. 1, pp. 51-60, 2016. doi: 10.1142/S2301385016400069
[13]	H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method, " IEEE Trans. Neural Networks, vol. 22, no. 12, pp. 2226-2236, 2011. doi: 10.1109/TNN.2011.2168538
[14]	X. Yang, D. Liu, and D. Wang, "Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints, " International J. Control, vol. 87, no. 3, pp. 553-566, 2014. doi: 10.1080/00207179.2013.848292
[15]	T. Bian, Y. Jiang, and Z. P. Jiang, "Adaptive dynamic programming and optimal control of nonlinear nonaffine systems, " Automatica, vol. 50, no. 10, pp. 2624-2632, 2014. doi: 10.1016/j.automatica.2014.08.023
[16]	C. Cox and R. Saeks, "Adaptive critic control and functional link neural networks, " in Proc. IEEE International Conference on Systems, Man, and Cybernetics, 1998, pp. 1652-1657.
[17]	D. Zhao, D. Liu, F. L. Lewis, J. C. Principe, and S. Squartini, "Special issue on deep reinforcement learning and adaptive dynamic programming, " IEEE Trans. Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2038-2041, 2018. doi: 10.1109/TNNLS.2018.2818878
[18]	T. Liu, B. Tian, Y. Ai, L. Li, D. Cao, and F. Y. Wang, "Parallel reinforcement learning: a framework and case study, " IEEE/CAA J. Autom. Sinica, vol. 5, no. 4, pp. 827-835, 2018. doi: 10.1109/JAS.2018.7511144

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(9) / Tables(1)

Get Citation

PDF

XML

Article Metrics

Article views (1474) PDF downloads(43)

Advanced Policy Learning Near-Optimal Regulation

doi: 10.1109/JAS.2019.1911489

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content