Online Fault-Tolerant Tracking Control With Adaptive Critic for Nonaffine Nonlinear Systems

Ding Wang; Lingzhi Hu; Xiaoli Li; Junfei Qiao

doi:10.1109/JAS.2024.124989

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2025 > In Press, Accepted Manuscript

D. Wang, L. Hu, X. Li, and J. Qiao, “Online fault-tolerant tracking control with adaptive critic for nonaffine nonlinear systems,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 1, pp. 215–227, Jan. 2025. doi: 10.1109/JAS.2024.124989

Citation:

D. Wang, L. Hu, X. Li, and J. Qiao, “Online fault-tolerant tracking control with adaptive critic for nonaffine nonlinear systems,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 1, pp. 215–227, Jan. 2025. doi: 10.1109/JAS.2024.124989

Citation:

PDF( 1165 KB)

Online Fault-Tolerant Tracking Control With Adaptive Critic for Nonaffine Nonlinear Systems

doi: 10.1109/JAS.2024.124989

Funds: This work was supported in part by the National Natural Science Foundation of China (62222301, 62373012, 62473012, 62021003), the National Science and Technology Major Project (2021ZD0112302, 2021ZD0112301), and the Beijing Natural Science Foundation (JQ19013)

More Information

Author Bio:
Ding Wang (Senior Member, IEEE) received the Ph.D. degree in control theory and control engineering from the Institute of Automation, Chinese Academy of Sciences in 2012. He is currently a Full Professor with the School of Information Science and Technology, Beijing University of Technology. He has authored or co-authored over 150 journal and conference papers and five monographs. His current research interests include adaptive critic control with industrial applications, reinforcement learning, and intelligent systems. He currently serves as an Associate Editor of IEEE Transactions on Systems, Man, and Cybernetics: Systems, IEEE Transactions on Automation Science and Engineering, Neural Networks, Engineering Applications of Artificial Intelligence, ISA Transactions, International Journal of Robust and Nonlinear Control, and Acta Automatica Sinica

Lingzhi Hu received the B.E. degree in automation from Heilongjiang University of Science and Technology in 2019, and the M.E. degree in control science and engineering from Beijing University of Technology in 2023. He is pursuing the Ph.D. degree in control science and engineering with the School of Information Science and Technology, Beijing University of Technology.His research interests include adaptive dynamic programming, event-triggered control, reinforcement learning with industrial applications, and intelligent systems

Xiaoli Li received the B.Eng. and M.Eng. degrees in control science and engineering from Dalian University of Technology in 1994 and 1997, respectively, and the Ph.D. degree in control science and engineering from Northeastern University in 2000. From 2004 to 2015, he served as a Professor at the School of Automation and Electrical Engineering, University of Science and Technology Beijing. In 2016, he joined Beijing University of Technology, where he is currently a Professor at the School of Information Science and Technology. His research interests include artificial intelligence, multiple model control, and intelligent control

Junfei Qiao (Senior Member, IEEE) received the B.E. and M.E. degrees in control engineering from Liaoning Technical University in 1992 and 1995, respectively, and the Ph.D. degree from Northeastern University in 1998.He is currently a Professor with the School of Information Science and Technology, Beijing University of Technology, where he is also the Director of the Beijing Key Laboratory of Computational Intelligence and Intelligent Systems and Beijing Laboratory of Smart Environmental Protection. His current research interests include neural networks, intelligent systems, self-adaptive systems, and process control
Corresponding author: Ding Wang, e-mail: dingwang@bjut.edu.cn
Received Date: 2024-08-19
Accepted Date: 2024-10-19

Available Online: 2024-12-10

Abstract

Abstract

In this paper, a fault-tolerant-based online critic learning algorithm is developed to solve the optimal tracking control issue for nonaffine nonlinear systems with actuator faults. First, a novel augmented plant is constructed by fusing the system state and the reference trajectory, which aims to transform the optimal fault-tolerant tracking control design with actuator faults into the optimal regulation problem of the conventional nonlinear error system. Subsequently, in order to ensure the normal execution of the online learning algorithm, a stability criterion condition is created to obtain an initial admissible tracking policy. Then, the constructed model neural network (NN) is pretrained to recognize the system dynamics and calculate trajectory control. The critic and action NNs are constructed to output the approximate cost function and approximate tracking control, respectively. The Hamilton-Jacobi-Bellman equation of the error system is solved online through the action-critic framework. In theoretical analysis, it is proved that all concerned signals are uniformly ultimately bounded according to the Lyapunov principle. The tracking control law can approach the optimal tracking control within a finite approximation error. Finally, two experimental examples are conducted to indicate the effectiveness and superiority of the developed fault-tolerant tracking control scheme.
- Adaptive critic control,
- neural network (NN),
- nonaffine nonlinear systems,
- online fault-tolerant tracking design,
- uniform ultimate boundedness

FullText(HTML)

References(43)

References

[1]	Y. Li, Y. Liu, and S. Tong, “Observer-based neuro-adaptive optimized control of strict-feedback nonlinear systems with state constraints,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 7, pp. 3131–3145, Jul. 2022. doi: 10.1109/TNNLS.2021.3051030
[2]	L. Yang, X. Wei, and C. Wen, “A security defense method against eavesdroppers in the communication-based train control system,” Chin. J. Electron., vol. 32, no. 5, pp. 992–1001, Sept. 2023. doi: 10.23919/cje.2022.00.248
[3]	M. Zhao, D. Wang, J. Qiao, M. Ha, and J. Ren, “Advanced value iteration for discrete-time intelligent critic control: A survey,” Artif. Intell. Rev., vol. 56, no. 10, pp. 12315–12346, Oct. 2023. doi: 10.1007/s10462-023-10497-1
[4]	L. Wang, Q. Wu, W. Ma, and W. Tang, “Stability improvement for one cycle controlled boost converters based on energy balance principle,” Chin. J. Electron., vol. 32, no. 6, pp. 1293–1303, Nov. 2023. doi: 10.23919/cje.2021.00.204
[5]	L. Xia, Q. Li, R. Song, and H. Modares, “Optimal synchronization control of heterogeneous asymmetric input-constrained unknown nonlinear MASs via reinforcement learning,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 3, pp. 520–532, Mar. 2022. doi: 10.1109/JAS.2021.1004359
[6]	D. Wang, N. Gao, D. Liu, J. Li, and F. L. Lewis, “Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 1, pp. 18–36, Jan. 2024. doi: 10.1109/JAS.2023.123843
[7]	M. Ha, D. Wang, and D. Liu, “Neural-network-based discounted optimal control via an integrated value iteration with accuracy guarantee,” Neural Netw., vol. 144, pp. 176–186, Dec. 2021. doi: 10.1016/j.neunet.2021.08.025
[8]	J. Li, Z. Xiao, J. Fan, T. Chai, and F. L. Lewis, “Off-policy Q-learning: Solving Nash equilibrium of multi-player games with network-induced delay and unmeasured state,” Automatica, vol. 136, p. 110076, Feb. 2022. doi: 10.1016/j.automatica.2021.110076
[9]	D. Wang and J. Qiao, “Approximate neural optimal control with reinforcement learning for a torsional pendulum device,” Neural Netw., vol. 117, pp. 1–7, Sept. 2019. doi: 10.1016/j.neunet.2019.04.026
[10]	Z. Chen and S. Jagannathan, “Generalized Hamilton-Jacobi-Bellman formulation based neural network control of affine nonlinear discrete-time systems,” IEEE Trans. Neural Netw., vol. 19, no. 1, pp. 90–106, Jan. 2008. doi: 10.1109/TNN.2007.900227
[11]	M. Ha, D. Wang, and D. Liu, “Discounted iterative adaptive critic designs with novel stability analysis for tracking control,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 7, pp. 1262–1272, Jul. 2022. doi: 10.1109/JAS.2022.105692
[12]	A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, “Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof,” IEEE Trans. Syst. Man Cybern. B Cybern., vol. 38, no. 4, pp. 943–949, Aug. 2008. doi: 10.1109/TSMCB.2008.926614
[13]	D. Wang, J. Wang, M. Zhao, P. Xin, and J. Qiao, “Adaptive multi-step evaluation design with stability guarantee for discrete-time optimal learning control,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 9, pp. 1797–1809, Sept. 2023. doi: 10.1109/JAS.2023.123684
[14]	T. Dierks and S. Jagannathan, “Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1118–1129, Jul. 2012. doi: 10.1109/TNNLS.2012.2196708
[15]	H. Zhang, C. Qin, B. Jiang, and Y. Luo, “Online adaptive policy learning algorithm for H_∞ state feedback control of unknown affine nonlinear discrete-time systems,” IEEE Trans. Cybern., vol. 44, no. 12, pp. 2706–2718, Dec. 2014. doi: 10.1109/TCYB.2014.2313915
[16]	Q. Zhao, J. Si, and J. Sun, “Online reinforcement learning control by direct heuristic dynamic programming: From time-driven to event-driven,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 8, pp. 4139–4144, Aug. 2022. doi: 10.1109/TNNLS.2021.3053037
[17]	Y. Yang, W. Gao, H. Modares, and C. Z. Xu, “Robust actor-critic learning for continuous-time nonlinear systems with unmodeled dynamics,” IEEE Trans. Fuzzy Syst., vol. 30, no. 6, pp. 2101–2112, Jun. 2022. doi: 10.1109/TFUZZ.2021.3075501
[18]	J. Liu, N. Zhang, Y. Li, X. Xie, E. Tian, and J. Cao, “Learning-based event-triggered tracking control for nonlinear networked control systems with unmatched disturbance,” IEEE Trans. Syst. Man Cybern. Syst., vol. 53, no. 5, pp. 3230–3240, May 2023. doi: 10.1109/TSMC.2022.3224432
[19]	C. Li, J. Ding, F. L. Lewis, and T. Chai, “A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems,” Automatica, vol. 129, p. 109687, Jul. 2021. doi: 10.1016/j.automatica.2021.109687
[20]	D. Wang, M. Zhao, M. Ha, and L. Hu, “Adaptive-critic-based hybrid intelligent optimal tracking for a class of nonlinear discrete-time systems,” Eng. Appl. Artif. Intell., vol. 105, p. 104443, Oct. 2021. doi: 10.1016/j.engappai.2021.104443
[21]	Y. Du and Y. Chen, “Time optimal trajectory planning algorithm for robotic manipulator based on locally chaotic particle swarm optimization,” Chin. J. Electron., vol. 31, no. 5, pp. 906–914, Sept. 2022. doi: 10.1049/cje.2021.00.373
[22]	L. Hu, D. Wang, and J. Qiao, “Static/dynamic event-triggered learning control for constrained nonlinear systems,” Nonlinear Dyn., vol. 112, no. 16, pp. 14159–14174, Aug. 2024. doi: 10.1007/s11071-024-09778-3
[23]	K. G. Vamvoudakis and H. Ferraz, “Model-free event-triggered control algorithm for continuous-time linear systems with optimal performance,” Automatica, vol. 87, pp. 412–420, Jan. 2018. doi: 10.1016/j.automatica.2017.03.013
[24]	D. Wang, L. Hu, M. Zhao, and J. Qiao, “Dual event-triggered constrained control through adaptive critic for discrete-time zero-sum games,” IEEE Trans. Syst. Man Cybern. Syst., vol. 53, no. 3, pp. 1584–1595, Mar. 2023. doi: 10.1109/TSMC.2022.3201671
[25]	D. Liu, X. Yang, D. Wang, and Q. Wei, “Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints,” IEEE Trans. Cybern., vol. 45, no. 7, pp. 1372–1385, Jul. 2015. doi: 10.1109/TCYB.2015.2417170
[26]	O. Qasem, M. Davari, W. Gao, D. R. Kirk, and T. Chai, “Hybrid iteration ADP algorithm to solve cooperative, optimal output regulation problem for continuous-time, linear, multiagent systems: Theory and application in islanded modern microgrids with IBRs,” IEEE Trans. Ind. Electron., vol. 71, no. 1, pp. 834–845, Jan. 2024. doi: 10.1109/TIE.2023.3247734
[27]	M. Mottaghi and R. Chhabra, “Robust optimal output-tracking control of constrained mechanical systems with application to autonomous rovers,” IEEE Trans. Control Syst. Technol., vol. 31, no. 1, pp. 83–98, Jan. 2023. doi: 10.1109/TCST.2022.3171687
[28]	S. Zhao, J. Wang, H. Xu, and B. Wang, “Composite observer-based optimal attitude-tracking control with reinforcement learning for hypersonic vehicles,” IEEE Trans. Cybern., vol. 53, no. 2, pp. 913–926, Feb. 2023. doi: 10.1109/TCYB.2022.3192871
[29]	M. Lin, B. Zhao, and D. Liu, “Policy gradient adaptive critic designs for model-free optimal tracking control with experience replay,” IEEE Trans. Syst. Man Cybern. Syst., vol. 52, no. 6, pp. 3692–3703, Jun. 2022. doi: 10.1109/TSMC.2021.3071968
[30]	Y. Fu, C. Hong, J. Fu, and T. Chai, “Approximate optimal tracking control of nondifferentiable signals for a class of continuous-time nonlinear systems,” IEEE Trans. Cybern., vol. 52, no. 6, pp. 4441–4450, Jun. 2022. doi: 10.1109/TCYB.2020.3027344
[31]	T. Wang, Y. Wang, X. Yang, and J. Yang, “Further results on optimal tracking control for nonlinear systems with nonzero equilibrium via adaptive dynamic programming,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 4, pp. 1900–1910, Apr. 2023. doi: 10.1109/TNNLS.2021.3105646
[32]	S. Song, M. Zhu, X. Dai, and D. Gong, “Model-free optimal tracking control of nonlinear input-affine discrete-time systems via an iterative deterministic Q-learning algorithm,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 1, pp. 999–1012, Jan. 2024. doi: 10.1109/TNNLS.2022.3178746
[33]	J. Zhang, D. Yang, H. Zhang, Y. Wang, and B. Zhou, “Dynamic event-based tracking control of boiler turbine systems with guaranteed performance,” IEEE Trans. Autom. Sci. Eng., vol. 21, no. 3, pp. 4272–4282, Jul. 2024. doi: 10.1109/TASE.2023.3294187
[34]	H. Zhang, Q. Wei, and Y. Luo, “A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm,” IEEE Trans. Syst. Man Cybern. B Cybern., vol. 38, no. 4, pp. 937–942, Aug. 2008. doi: 10.1109/TSMCB.2008.920269
[35]	D. Wang, X. Li, M. Zhao, and J. Qiao, “Adaptive critic control design with knowledge transfer for wastewater treatment applications,” IEEE Trans. Industr. Inform., vol. 20, no. 2, pp. 1488–1497, Feb. 2024. doi: 10.1109/TII.2023.3278875
[36]	A. A. Ladel, A. Benzaouia, R. Outbib, and M. Ouladsine, “Integrated state/fault estimation and fault-tolerant control design for switched T-S fuzzy systems with sensor and actuator faults,” IEEE Trans. Fuzzy Syst., vol. 30, no. 8, pp. 3211–3223, Aug. 2022. doi: 10.1109/TFUZZ.2021.3107751
[37]	Y. Guo and X. He, “Active diagnosis of incipient actuator faults for stochastic systems,” IEEE Trans. Ind. Electron., vol. 71, no. 1, pp. 996–1005, Jan. 2024. doi: 10.1109/TIE.2023.3247778
[38]	D. Ye and G. H. Yang, “Adaptive fault-tolerant tracking control against actuator faults with application to flight control,” IEEE Trans. Control Syst. Technol., vol. 14, no. 6, pp. 1088–1096, Nov. 2006. doi: 10.1109/TCST.2006.883191
[39]	Q. Wei, H. Li, T. Li, and F. Wang, “A novel data-based fault-tolerant control method for multicontroller linear systems via distributed policy iteration,” IEEE Trans. Syst. Man Cybern. Syst., vol. 53, no. 5, pp. 3176–3186, May 2023. doi: 10.1109/TSMC.2022.3223910
[40]	T. Li, W. Bai, Q. Liu, Y. Long, and C. L. P. Chen, “Distributed fault-tolerant containment control protocols for the discrete-time multiagent systems via reinforcement learning method,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 8, pp. 3979–3991, Aug. 2023. doi: 10.1109/TNNLS.2021.3121403
[41]	B. F. Yue and W. W. Che, “Data-driven dynamic event-triggered fault-tolerant platooning control,” IEEE Trans. Industr. Inform., vol. 19, no. 7, pp. 8418–8426, Jul. 2023. doi: 10.1109/TII.2022.3217470
[42]	C. Treesatayapun, “Discrete-time robust event-triggered actuator fault-tolerant control based on adaptive networks and reinforcement learning,” Neural Netw., vol. 166, pp. 541–554, Sept. 2023. doi: 10.1016/j.neunet.2023.08.003
[43]	H. Lin, B. Zhao, D. Liu, and C. Alippi, “Data-based fault tolerant control for affine nonlinear systems through particle swarm optimized neural networks,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 4, pp. 954–964, Jul. 2020. doi: 10.1109/JAS.2020.1003225

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(5) / Tables(2)

Get Citation

PDF

XML

Article Metrics

Article views (36) PDF downloads(27)

Online Fault-Tolerant Tracking Control With Adaptive Critic for Nonaffine Nonlinear Systems

doi: 10.1109/JAS.2024.124989

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content