An Iterative Method for Optimal Feedback Control and Generalized HJB Equation

Xuesong Chen; Xin Chen

doi:10.1109/JAS.2017.7510706

Volume 5 Issue 5

Aug. 2018

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2018 > 5(5): 999-1006

Xuesong Chen and Xin Chen, "An Iterative Method for Optimal Feedback Control and Generalized HJB Equation," IEEE/CAA J. Autom. Sinica, vol. 5, no. 5, pp. 999-1006, Sept. 2018. doi: 10.1109/JAS.2017.7510706

Citation:

Xuesong Chen and Xin Chen, "An Iterative Method for Optimal Feedback Control and Generalized HJB Equation," IEEE/CAA J. Autom. Sinica, vol. 5, no. 5, pp. 999-1006, Sept. 2018. doi: 10.1109/JAS.2017.7510706

Citation:

PDF( 4611 KB)

An Iterative Method for Optimal Feedback Control and Generalized HJB Equation

doi: 10.1109/JAS.2017.7510706

Xuesong Chen^{1
,
,},
Xin Chen^2
,

1.
School of Applied Mathematics, Guangdong University of Technology, Guangzhou 510006, China
2.
School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou 510006, China

Funds:

the National Natural Science Foundation of China U1601202

the National Natural Science Foundation of China U1134004

the National Natural Science Foundation of China 91648108

the Natural Science Foundation of Guangdong Province 2015A030313497

the Natural Science Foundation of Guangdong Province 2015A030312008

the Project of Science and Technology of Guangdong Province 2015B010102014

the Project of Science and Technology of Guangdong Province 2015B010124001

the Project of Science and Technology of Guangdong Province 2015B010104006

the Project of Science and Technology of Guangdong Province 2018A030313505

More Information

Abstract

Abstract

In this paper, a new iterative method is proposed to solve the generalized Hamilton-Jacobi-Bellman (GHJB) equation through successively approximate it. Firstly, the GHJB equation is converted to an algebraic equation with the vector norm, which is essentially a set of simultaneous nonlinear equations in the case of dynamic systems. Then, the proposed algorithm solves GHJB equation numerically for points near the origin by considering the linearization of the non-linear equations under a good initial control guess. Finally, the procedure is proved to converge to the optimal stabilizing solution with respect to the iteration variable. In addition, it is shown that the result is a closed-loop control based on this iterative approach. Illustrative examples show that the update control laws will converge to optimal control for nonlinear systems.
- Generalized Hamilton-Jacobi-Bellman (HJB) equation,
- iterative method,
- nonlinear dynamic system,
- optimal control

FullText(HTML)

References(18)

References

[1]	D. E. Kirk, Optimal Control Theory: An Introduction. Mineola, USA: Courier Corporation, 2012.
[2]	F. L. Lewis and V. L. Syrmos, Optimal Control. New York, USA: Wiley, 1995.
[3]	R. Bellman, Dynamic Programming. New Jersey, USA: Princeton University Press, 1957.
[4]	D. Kleinman, "On an iterative technique for riccati equation computations, " IEEE Trans. Autom. Contr., vol. 13, no. 1, pp. 114-115, Feb. 1968. http://xueshu.baidu.com/s?wd=paperuri%3A%28c172df92b76abf7a3551f8ab95408337%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fci.nii.ac.jp%2Fnaid%2F30019918845&ie=utf-8&sc_us=16210463245255859084
[5]	F. Y. Wang, H. G. Zhang, and D. R. Liu, "Adaptive dynamic programming: An introduction, " IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009. http://xueshu.baidu.com/s?wd=paperuri%3A%282a885ce7daacb3621170ec72c98d3cf9%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Ficp.jsp%3Farnumber%3D4840325&ie=utf-8&sc_us=17790335501086901414
[6]	F. L. Lewis, D. Vrabie, and K. G. Vamvoudakis, "Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers, " IEEE Control Syst., vol. 32, no. 6, pp. 76-105, Dec. 2012. http://xueshu.baidu.com/s?wd=paperuri%3A%2860ee797c5b81e82150cf13d045f5fc09%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Ficp.jsp%3Farnumber%3D6315769&ie=utf-8&sc_us=15413527564144328925
[7]	W. Zhang, W. X. Liu, X. Wang, L. M. Liu, and F. Ferrese, "Online optimal generation control based on constrained distributed gradient algorithm, " IEEE Trans. Power Syst., vol. 30, no. 1, pp. 35-45, Jan. 2015. http://xueshu.baidu.com/s?wd=paperuri%3A%28d8e857a156c4bc17f4164e996d00b13a%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdx.doi.org%2F10.1109%2FTPWRS.2014.2319315&ie=utf-8&sc_us=4468568846486416817
[8]	N. Sakamoto and A. J. van der Schaft, "Analytical approximation methods for the stabilizing solution of the Hamilton-Jacobi equation, " IEEE Trans. Autom. Control, vol. 53, no. 10, pp. 2335-2350, Nov. 2008. http://www.mendeley.com/research/analytical-approximation-methods-stabilizing-solution-hamiltonjacobi-equation/
[9]	T. Bian, Y. Jiang, and Z. P. Jiang, "Adaptive dynamic programming and optimal control of nonlinear nonaffine systems, " Automatica, vol. 50, no. 10, pp. 2624-2632, Oct. 2014. http://xueshu.baidu.com/s?wd=paperuri%3A%287c5a4b93543a6cceb05ac88bb1e68464%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdialnet.unirioja.es%2Fservlet%2Farticulo%3Fcodigo%3D4854670&ie=utf-8&sc_us=3034413140015552710
[10]	C. O. Aguilar and A. J. Krener, "Numerical solutions to the Bellman equation of optimal control, " J. Optim. Theory Appl., vol. 160, no. 2, pp. 527-552, Feb. 2014. doi: 10.1007/s10957-013-0403-8
[11]	S. Cacace, E. Cristiani, M. Falcone, and A. Picarelli, "A patchy dynamic programming scheme for a class of Hamilton-Jacobi-Bellman equations, " SIAM J. Sci. Comput., vol. 34, no. 5, pp. A2625-A2649, Oct. 2012. http://xueshu.baidu.com/s?wd=paperuri%3A%286e34c72862da9ed0180ea341f0816b0c%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fadsabs.harvard.edu%2Fabs%2F2011arXiv1109.3577C&ie=utf-8&sc_us=9863127530291330464
[12]	N. Govindarajan, C. C. de Visser, and K. Krishnakumar, "A sparse collocation method for solving time-dependent HJB equations using multivariate B-splines, " Automatica, vol. 50, no. 9, pp. 2234-2244, Sep. 2014. http://xueshu.baidu.com/s?wd=paperuri%3A%28232ae8ead0425f554db5acf7747afd31%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdialnet.unirioja.es%2Fservlet%2Farticulo%3Fcodigo%3D4830522&ie=utf-8&sc_us=2303783704440044212
[13]	J. Markman and I. N. Katz, "An iterative algorithm for solving Hamilton-Jacobi type equations, " SIAM J. Sci. Comput., vol. 22, no. 1, pp. 312-329, Jul. 2000. doi: 10.1137/S1064827598344315
[14]	I. Smears and E. Süli, "Discontinuous Galerkin finite element approximation of Hamilton-Jacobi-Bellman equations with Cordes coefficients, "SIAM J. Numer. Anal., vol. 52, no. 2, pp. 993-1016, Apr. 2014. http://eprints.maths.ox.ac.uk/1671
[15]	R. W. Beard, G. N. Saridis, and J. T. Wen, "Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation, " J. Optim. Theory Appl., vol. 96, no. 3, pp. 589-626, Mar. 1998. http://xueshu.baidu.com/s?wd=paperuri%3A%28e806b8acd4f6c5306a9c2870839fa064%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fci.nii.ac.jp%2Fnaid%2F80010226858&ie=utf-8&sc_us=15790908467176106295
[16]	R. W. Beard, G. N. Saridis, and J. T. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation, " Automatica, vol. 33, no. 12, pp. 2159-2177, Dec. 1997. http://xueshu.baidu.com/s?wd=paperuri%3A%281e50ce49db952defceb5de4d82886749%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fdl.acm.org%2Fcitation.cfm%3Fid%3D280068%26amp%3Bpreflayout%3Dflat&ie=utf-8&sc_us=11738833470501958577
[17]	G. N. Saridis and C. S. G. Lee, "An approximation theory of optimal control for trainable manipulators, " IEEE Trans. Syst. Man Cybern. vol. 9, no. 3, pp. 152-159, Mar. 1979. http://xueshu.baidu.com/s?wd=paperuri%3A%28fafa925f04afc7b2d4167eec6f3a33b4%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fci.nii.ac.jp%2Fnaid%2F80013924674&ie=utf-8&sc_us=1501191005870337592
[18]	Q. Gong, W. Kang, and I. M. Ross, "A pseudospectral method for the optimal control of constrained feedback linearizable systems, " IEEE Trans. Autom. Control, vol. 51, no. 7, pp. 1115-1129, Jul. 2006. http://www.mendeley.com/catalog/pseudospectral-method-optimal-control-constrained-feedback-linearizable-systems/

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(5) / Tables(2)

Get Citation

PDF

XML

Article Metrics

Article views (1118) PDF downloads(30)

An Iterative Method for Optimal Feedback Control and Generalized HJB Equation

doi: 10.1109/JAS.2017.7510706

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content