Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

Zhijun Fu; Wenfang Xie; Subhash Rakheja; Jing Na

Volume 4 Issue 1

Jan. 2017

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2017 > 4(1): 48-57

Zhijun Fu, Wenfang Xie, Subhash Rakheja and Jing Na, "Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints," IEEE/CAA J. Autom. Sinica, vol. 4, no. 1, pp. 48-57, Jan. 2017.

Citation:

Zhijun Fu, Wenfang Xie, Subhash Rakheja and Jing Na, "Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints," IEEE/CAA J. Autom. Sinica, vol. 4, no. 1, pp. 48-57, Jan. 2017.

Citation:

PDF( 1040 KB)

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

Zhijun Fu^{1
,
,},
Wenfang Xie^2
,,
Subhash Rakheja^2
,,
Jing Na^3
,

1.
College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou, 310014, China
2.
Department of Mechanical & Industrial Engineering, Concordia University, Montreal H3G ZW1, Quebec CAN
3.
Faculty of Mechanical & Electrical Engineering, Kunming University of Science & Technology, Kunming 650500, China

Funds:

This work was supported by the Natural Sciences and Engineering Research Council of Canada N00892

in part by National Natural Science Foundation of China 51405436

in part by National Natural Science Foundation of China 51375452

in part by National Natural Science Foundation of China 61573174

More Information

Author Bio:
Wenfang Xie is a professor with the Department of Mechanical and Industrial Engineering at Concordia University, Montreal, Canada. She was an Industrial Research Fellowship holder from Natural Sciences and Engineering Research Council of Canada (NSERC) and served as a senior research engineer in In CoreTec, Inc. Canada before she joined Concordia University in 2003. She received her Ph.D. from The Hong Kong Polytechnic University in 1999 and her Master's degree from Beijing University of Aeronautics and Astronautics in 1991. Her research interests include nonlinear control and identification in mechatronics, artificial intelligent control, and advanced process control and system identification (e-mail: wfxie@encs.concordia.ca)

Subhash Rakheja received a Ph.D. in mechanical engineering from Concordia University. He joined the National Research Council soon after graduation and faculty in Concordia as a research assistant professor. He became associate professor in 1989 and Professor in 1994. He is a research chair in Vehicular Ergo-dynamics in Concordia University. He was also honored with the title of the University Research Fellow in 2005. He is a fellow of the CSME and of the ASME. He is the editor of the International Journal of Industrial Ergonomics and International Journal of Vehicle Performance and associate editor of the SAE Journal of Commercial Vehicles and the International Journal of Heavy Vehicle Systems. He continues to serve in the Canadian Advisory Council on International Standards. He has won the SAE Arch T. Colwell Merit Award, Alice Hamilton Award from Center for Disease Control (USA) and Liberty Mutual Award. His research expertise includes advanced transportation systems and highway safety, human responses to workplace vibration and driver-vehicle interactions (e-mail:subashrakheja@gmail.com)

Jing Na received the B.S. and Ph.D. degrees from the School of Automation, Beijing Institute of Technology, Beijing, China, in 2004 and 2010, respectively. In 2008, he was a visiting student with the Universitat Poli-téecnica de Catalunya (UPC), Barcelona, Spain. From 2008 to 2009, he was a research collaborator with the Department of Mechanical Engineering, University of Bristol, Bristol, UK. Since 2010, he has been with the Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming, China. Dr. Na was a postdoctoral fellow with the ITER Organization, Cadarache, France, from 2011 to 2012. His current research interests include adaptive control, time-delay systems, neural networks, parameter estimation, repetitive control, and nonlinear control and applications. He is a Marie Curie Fellow hosted in the Department of Mechanical Engineering, University of Bristol, Bristol, UK (email: najing25@163.com)
Corresponding author: Zhijun Fu is an assistant professor with the College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou, China. He received his Ph.D. degree from the Mechanical Engineering, University of Science and Technology Beijing, China, in June, 2013. From 2010 to 2012, he was a joint training doctoral students with the University of Concordia University, Canada. Now, he is carrying out his postdoctoral research in the Concordia University, Canada. His research interests include vehicle stability control, control theory and application, and intelligent control. Corresponding author of this paper (e-mail: fuzhijun@zjut.edu.cn)
Received Date: 2016-05-10
Accepted Date: 2016-08-02

Abstract

Abstract

This paper introduces an observer-based adaptive optimal control method for unknown singularly perturbed nonlinear systems with input constraints. First, a multi-time scales dynamic neural network (MTSDNN) observer with a novel updating law derived from a properly designed Lyapunov function is proposed to estimate the system states. Then, an adaptive learning rule driven by the critic NN weight error is presented for the critic NN, which is used to approximate the optimal cost function. Finally, the optimal control action is calculated by online solving the Hamilton-Jacobi-Bellman (HJB) equation associated with the MTSDNN observer and critic NN. The stability of the overall closed-loop system consisting of the MTSDNN observer, the critic NN and the optimal control action is proved. The proposed observer-based optimal control approach has an essential advantage that the system dynamics are not needed for implementation, and only the measured input/output data is needed. Moreover, the proposed optimal control design takes the input constraints into consideration and thus can overcome the restriction of actuator saturation. Simulation results are presented to confirm the validity of the investigated approach.
- Adaptive optimal control,
- multi-time scales neural network,
- nonlinear system,
- observer

FullText(HTML)

References(50)

References

[1]	P.Kokotovic and P.Sannuti, "Singular perturbation method for reducing the model order in optimal control design, "IEEE Trans.Autom.Control, vol.13, no.4, pp.377-384, Aug.1968. http://ieeexplore.ieee.org/document/1098927/
[2]	P.Sannuti and P.V.Kokotovic, "Near-optimum design of linear systems by a singular perturbation method, "IEEE Trans.Autom.Control, vol.14, no.1, pp.15-22, Feb.1969. http://ieeexplore.ieee.org/document/1099113/
[3]	E.Fridman, "A descriptor system approach to nonlinear singularly perturbed optimal control problem, "Automatica, vol.37, no.4, pp.543-549, Apr.2001. http://www.sciencedirect.com/science/article/pii/S0005109800001850
[4]	V.Dragan, H.Mukaidani, and P.Shi, "The linear quadratic regulator problem for a class of controlled systems modeled by singularly perturbed itô differential equations, "SIAM J.Control Optim., vol.50, no.1, pp.448-470, Feb.2012. https://www.researchgate.net/publication/262320202_The_Linear_Quadratic_Regulator_Problem_for_a_Class_of_Controlled_Systems_Modeled_by_Singularly_Perturbed_Ito_Differential_Equations?_sg=sHirOW_QzJW4c72TRh6hoa6wgzR1Kjmw71_MfpRaJD6NKBK7F92wpsbs5kaDETKpmM8FVQd8q0EiWGiAJh6D-g
[5]	V.Gaitsgory and S.Rossomakhine, "Averaging and linear programming in some singularly perturbed problems of optimal control, "Appl.Math. Optim., vol.71, no.2, pp.195-276, Apr.2015. http://dl.acm.org/citation.cfm?id=2749834
[6]	Y.Zhang, D.S.Naidu, C.X.Cai, and Y.Zou, "Singular perturbations and time scales in control theories and applications:an overview 2002-2012, "Int.J.Inform.Syst.Sci., vol.9, no.1, pp.1-36, 2014. http://www.d.umn.edu/~dsnaidu/Naidu_Survey_Zhang-Naidu_et_al_(IJISS)_2014.pdf
[7]	Y.H.Luo, Q.Y.Sun, H.G.Zhang, and L.L.Cui, "Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics, "Neurocomputing, vol.148, pp.200-208, Jan.2015. http://www.doc88.com/p-6877796860397.html
[8]	F.L.Lewis and D.R.Liu, Reinforcement Learning Approximate Dynamic Programming for Feedback Control.New York:John Wiley & Sons, Inc., 2013.
[9]	P.J.Werbos."Approximate dynamic programming for real-time control and neural modeling, "in Handbook of Intelligent Control:Neural, Fuzzy, and Adaptive Approaches, D.A.White and D.A.Sofge, Eds.New York:Van Nostrand Reinhold, 1992.
[10]	D.P.Bertsekas and J.N.Tsitsiklis.Neuro-Dynamic Programming. Belmont, MA:Athena Scientific, 1996.
[11]	P.J.Werbos."Reinforcement learning and approximate dynamic programming (RLADP)-foundations, common misconceptions, and the challenges ahead, "in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control.F.L.Lewis and D.Liu, Eds.Hoboken:John Wiley & Sons, Inc., 2013.
[12]	D.Vrabie, O.Pastravanu, M.Abu-Khalaf, and F.L.Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration, "Automatica, vol.45, no.2, pp.477-484, Feb.2009. http://www.sciencedirect.com/science/article/pii/S0005109808004469
[13]	D.Vrabie and F.Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems, " Neural Networks, vol.22, no.3, pp.237-246, Apr.2009. http://www.sciencedirect.com/science/article/pii/S0893608009000446
[14]	H.G.Zhang, C.B.Qin, B.Jiang, and Y.H.Luo, "Online adaptive policy learning algorithm for H_∞ state feedback control of unknown affine nonlinear discrete-time systems, "IEEE Trans.Cybern., vol.44, no.12, pp.2706-2718, Dec.2014. https://www.ncbi.nlm.nih.gov/pubmed/25095274
[15]	Q.L.Wei and D.R.Liu, "A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems, "Sci.China Inform.Sci., vol.58, no.12, pp.1-15, Dec.2015. doi: 10.1007/s11432-015-5462-z
[16]	Y.Jiang and Z.P.Jiang, "Robust adaptive dynamic programming with an application to power systems, "IEEE Trans.Neural Netw.Learn.Syst., vol.24, no.7, pp.1150-1156, Jul.2013. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6484168
[17]	J.Na and G.Herrmann, "Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems, "IEEE/CAA J.Autom.Sinica, vol.1, no.4, pp.412-422, Oct.2014. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7004668&filter%3DAND%28p_IS_Number%3A7004659%29
[18]	Z.Ni, H.B.He, and J.Y.Wen, "Adaptive learning in tracking control based on the dual critic network design, "IEEE Trans.Neural Netw. Learn.Syst., vol.24, no.6, pp.913-928, Jun.2013. https://www.ncbi.nlm.nih.gov/pubmed/24808473
[19]	H.B.He, Z.Ni, and J.Fu, "A three-network architecture for on-line learning and optimization based on adaptive dynamic programming, " Neurocomputing, vol.78, no.1, pp.3-13, Feb.2012. http://www.ele.uri.edu/faculty/he/PDFfiles/Three_ADP.pdf
[20]	Y.H.Luo, Q.Y.Sun, H.G.Zhang, and L.L.Cui, "Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics, "Neurocomputing, vol.148, pp.200-208, Jan.2015. http://www.doc88.com/p-6877796860397.html
[21]	J.L.Zhang, H.G.Zhang, B.R.Wang, and T.Y.Cai, "Nearly data-based optimal control for linear discrete model-free systems with delays via reinforcement learning, "Int.J.Syst.Sci., vol.47, no.7, pp.1563-1573, 2016. doi: 10.1080/00207721.2014.941147
[22]	Q.Yang, J.B.Vance, and S.Jagannathan, "Control of nonaffine nonlinear discrete-time systems using reinforcement-learning-based linearly parameterized neural networks, "IEEE Trans.Syst., Man, Cybern., Part B (Cybern.), vol.38, no.4, 994-1001, Aug.2008. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4567550
[23]	S.Bhasin, R.Kamalapurkar, M.Johnson, K.G.Vamvoudakis, F.L. Lewis, and W.E.Dixon, "A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems, "Automatica, vol.49, no.1, pp.82-92, Jan.2013. http://www.sciencedirect.com/science/article/pii/S0005109812004827
[24]	M.Abu-Khalaf and F.L.Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach, "Automatica, vol.41, no.5, pp.779-791, May 2005. http://www.sciencedirect.com/science/article/pii/S0005109805000105
[25]	X.Yang, D.R.Liu, and D.Wang, "Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints, "Int.J.Control, vol.87, no.3, pp.553-566, 2014. doi: 10.1080/00207179.2013.848292
[26]	X.Yang, D.R.Liu, and Y.Z.Huang, "Neural-network-based online optimal control for uncertain non-linear continuous-time systems with control constraints, "IET Control Theory Appl., vol.7, no.17, pp.2037-2047, Nov.2013. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6668837&sortType%3Dasc_p_Sequence%26filter%3DAND%28p_IS_Number%3A6667192%29
[27]	H.G.Zhang, Y.H.Luo, and D.R.Liu, "Neural-network-based nearoptimal control for a class of discrete-time affine nonlinear systems with control constraints, "IEEE Trans.Neural Netw.Learn.Syst., vol.20, no.9, pp.1490-1503, Sep.2009. http://ieeexplore.ieee.org/document/5191063/
[28]	D.R.Liu, X.Yang, D.Wang, and Q.L.Wei, "Reinforcement-learningbased robust controller design for continuous-time uncertain nonlinear systems subject to input constraints, "IEEE Trans.Cybern., vol.45, no.7, pp.1372-1385, Jul.2015. http://ieeexplore.ieee.org/document/7083712/
[29]	X.Yang, D.R.Liu, H.W.Ma, and Y.C.Xu, "Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems, "Inform.Sci., vol.328, pp.435-454, Jan.2016. http://www.sciencedirect.com/science/article/pii/S0020025515006556
[30]	D.R.Liu, Y.Z.Huang, D.Wang, and Q.L.Wei, "Neural-networkobserver-based optimal control for unknown nonlinear systems using adaptive dynamic programming, "Int.J.Control, vol.86, no.9, pp.1554-1566, 2013. doi: 10.1080/00207179.2013.790562
[31]	E.N.Sanchez, A.Y.Alanıs, and A.G.Loukianov, Discrete-Time High Order Neural Control:Trained with Kaiman Filtering.Berlin Heidelberg:Springer, 2008.
[32]	J.P.Xu, J.Na, Z.W.Liu, and H.B.Xiao, "Adaptive state observer for nonlinear MIMO systems with uncertain dynamics, "in Proc.Chinese Control and Decision Conf., Xuzhou, China, pp.1017-1022, 2010. http://ieeexplore.ieee.org/document/5498072/
[33]	A.S.Poznyak, E.N.Sanchez, and W.Yu, Differential Neural Networks for Robust Nonlinear Control:Identification, State Estimation and Trajectory Tracking.Singapore:World Scientific Publishing Co.Pte. Ltd., 2001.
[34]	I.Chairez, A.Poznyaks, and T.Poznyaks, "Practical stability analysis for DNN observation, "in Proc.47th IEEE Conf.Decision and Control, Cancun, Mexico, pp.2551-2556, 2008. http://ieeexplore.ieee.org/document/4738995/
[35]	W.Yu, M.A.Moreno, and X.Li, "Observer-based neuro identifier, "IEE Proc.-Control Theory Appl., vol.147, no.2, pp.145-152, Mar.2000. https://www.researchgate.net/publication/3352308_Observer-based_neuro_identifier
[36]	B.L.Walcott, M.J.Corless, and S.H.·Zak, "Comparative study of non-linear state-observation techniques, "Int.J.Control, vol.45, no.6, pp.2109-2132, 1987. doi: 10.1080/00207178708933870
[37]	S.Haykin, Kalman Filtering and Neural Networks.New York:John Wiley & Sons, 2001.
[38]	J.J.E.Slotine and W.P.Li, Applied Nonlinear Control.Englewood Cliffs, NJ:Prentice Hall, 1991.
[39]	A.Mezouar, M.K.Fellah, and S.Hadjeri, "Adaptive sliding mode observer for induction motor using two-time-scale approach, "Electr. Power Syst.Res., vol.77, no.5-6, pp.604-618, Apr.2007. https://www.researchgate.net/profile/Mohammed_Karim_Fellah/publication/245155571_Adaptive_sliding_mode_observer_for_induction_motor_using_two-time-scale_approach/links/54cf75f70cf298d65663759b.pdf?origin=publication_list
[40]	Z.S.Wang, E.L.Zhang, H.G.Zhang, and Z.Y.Ren, "Global stability analysis of multitime-scale neural networks, "Neural Comp. Appl., vol.22, no.2, pp.211-217, Feb.2013. doi: 10.1007/s00521-011-0680-9
[41]	A.Meyer-Bäse, G.Botella, and L.Rybarska-Rusinek, "Stochastic stability analysis of competitive neural networks with different time-scales, " Neurocomputing, vol.118, no.115-118, Oct.2013. http://www.sciencedirect.com/science/article/pii/S0925231213002580
[42]	W.Yu, X.O.Li, "Passivity analysis of dynamic neural networks with different time-scales, "Neural Process.Lett., vol.25, no.2, pp.143-155, Apr.2007. doi: 10.1007%2Fs11063-007-9034-0.pdf
[43]	Z.J.Fu, W.F.Xie, and W.D.Luo, "Robust on-line nonlinear systems identification using multilayer dynamic neural networks with two-time scales, "Neurocomputing, vol.113, pp.16-26, Aug.2013. http://www.sciencedirect.com/science/article/pii/S092523121300218X
[44]	Z.J.Fu, W.F.Xie, X.Han, and W.D.Luo, "Nonlinear systems identification and control via dynamic multitime scales neural networks, " IEEE Trans.Neural Netw.Learn.Syst., vol.24, no.11, pp.1814-1823, Nov.2013. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6553133&sortType%3Dasc_p_Sequence%26filter%3DAND%28p_IS_Number%3A6632919%29&reload=true
[45]	Z.J.Fu, W.F.Xie, and J.Na, "Robust adaptive nonlinear observer design via multi-time scales neural network, "Neurocomputing, vol.190, pp.217-225, May 2016. http://www.sciencedirect.com/science/article/pii/S092523121600062X
[46]	G.Cybenko, "Approximation by superpositions of a sigmoidal function, " Math.Control, Signals and Syst., vol.2, no.4, pp.303-314, Dec.1989. doi: 10.1007/BF02551274
[47]	W.Yu, A.S.Poznyak, X.O.Li, "Multilayer dynamic neural networks for non-linear system on-line identification, "Int.J.Control, vol.74, no.18, pp.1858-1864, 2001. doi: 10.1080/00207170110089816
[48]	G.Li, J.Na, D.P.Stoten, and X.M.Ren, "Adaptive neural network feedforward control for dynamically substructured systems, "IEEE Trans. Control Syst.Technol., vol.22, no.3, pp.944-954, May 2014. https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/5999/LIAdaptiveNeural2014Accepted.pdf;sequence=2
[49]	Y.F.Lv, J.Na, Q.M.Yang, X.Wu, and Y.Guo, "Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics, "Int.J.Control, vol.89, no.1, pp.99-112, 2016. doi: 10.1080/00207179.2015.1060362
[50]	F.L.Lewis, D.M.Dawson, and C.T.Abdallah, Robot Manipulator Control:Theory and Practice.New York:Marcel Dekker, USA, 2004.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(5)

Get Citation

PDF

XML

Article Metrics

Article views (1202) PDF downloads(31)

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content