IEEE/CAA Journal of Automatica Sinica
Citation: | X. Yuan, W. Xu, Y. Wang, C. Yang, and W. Gui, “A deep residual PLS for data-driven quality prediction modeling in industrial process,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 8, pp. 1777–1785, Aug. 2024. doi: 10.1109/JAS.2024.124578 |
Partial least squares (PLS) model is the most typical data-driven method for quality-related industrial tasks like soft sensor. However, only linear relations are captured between the input and output data in the PLS. It is difficult to obtain the remaining nonlinear information in the residual subspaces, which may deteriorate the prediction performance in complex industrial processes. To fully utilize data information in PLS residual subspaces, a deep residual PLS (DRPLS) framework is proposed for quality prediction in this paper. Inspired by deep learning, DRPLS is designed by stacking a number of PLSs successively, in which the input residuals of the previous PLS are used as the layer connection. To enhance representation, nonlinear function is applied to the input residuals before using them for stacking high-level PLS. For each PLS, the output parts are just the output residuals from its previous PLS. Finally, the output prediction is obtained by adding the results of each PLS. The effectiveness of the proposed DRPLS is validated on an industrial hydrocracking process.
[1] |
D. Zheng, L. Zhou, and Z. Song, “Kernel generalization of multi-rate probabilistic principal component analysis for fault detection in nonlinear process,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 8, pp. 1465–1476, Aug. 2021. doi: 10.1109/JAS.2021.1004090
|
[2] |
J. Qian, L. Jiang, and Z. Song, “Locally linear back-propagation based contribution for nonlinear process fault diagnosis,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 3, pp. 764–775, May 2020. doi: 10.1109/JAS.2020.1003147
|
[3] |
X. Yuan, N. Xu, L. Ye, K. Wang, F. Shen, Y. Wang, C. Yang, and W. Gui, “Attention-based interval aided networks for data modeling of heterogeneous sampling sequences with missing values in process industry,” IEEE. Trans. Ind. Inf., vol. 20, no. 4, p. 5253–5262, Apr. 2024. doi: 10.1016/j.ces.2021.117299
|
[4] |
Y. Yang, X. Shi, X. Liu, and H. Li, “A novel MDFA-MKECA method with application to industrial batch process monitoring,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1446–1454, Sept. 2020. doi: 10.1109/JAS.2019.1911555
|
[5] |
M. Kano and Y. Nakagawa, “Data-based process monitoring, process control, and quality improvement: Recent developments and applications in steel industry,” Comput. Chem. Eng., vol. 32, no. 1–2, pp. 12–24, Jan. 2008. doi: 10.1016/j.compchemeng.2007.07.005
|
[6] |
P. Kadlec, B. Gabrys, and S. Strandt, “Data-driven soft sensors in the process industry,” Comput. Chem. Eng., vol. 33, no. 4, pp. 795–814, Apr. 2009. doi: 10.1016/j.compchemeng.2008.12.012
|
[7] |
X. Yuan, Y. Wang, C. Wang, L. Ye, K. Wang, Y. Wang, C. Yang, W. Gui, and F. Shen, “Variable correlation analysis-based convolutional neural network for far topological feature extraction and industrial predictive modeling,” IEEE Tyans. Instrum. Meas., vol. 73, no. 21, p. 3001110, 2024. doi: 10.1109/JSEN.2022.3196011
|
[8] |
L. Zhou, J. Zheng, Z. Ge, Z. Song, and S. Shan, “Multimode process monitoring based on switching autoregressive dynamic latent variable model,” IEEE Trans. Ind. Electron., vol. 65, no. 10, pp. 8184–8194, Oct. 2018. doi: 10.1109/TIE.2018.2803727
|
[9] |
B. Pan, H. Jin, L. Wang, B. Qian, X. Chen, S. Huang, and J. Li, “Just-in-time learning based soft sensor with variable selection and weighting optimized by evolutionary optimization for quality prediction of nonlinear processes,” Chem. Eng. Res. Des., vol. 144, pp. 285–299, Apr. 2019. doi: 10.1016/j.cherd.2019.02.004
|
[10] |
X. Yuan, L. Huang, L. Ye, Y. Wang, K. Wang, C. Yang, W. Gui, and F. Shen, “Quality prediction modeling for industrial processes using multiscale attention-based convolutional neural network,” IEEE Trans. Cybern., vol. 54, no. 5, pp. 2696–2707, May 2024. doi: 10.1021/acs.iecr.1c02768
|
[11] |
B. Pan, H. Jin, B. Yang, B. Qian, and Z. Zhao, “Soft sensor development for nonlinear industrial processes based on ensemble just-in-time extreme learning machine through triple-modal perturbation and evolutionary multiobjective optimization,” Ind. Eng. Chem. Res., vol. 58, no. 38, pp. 17991–18006, Sept. 2019. doi: 10.1021/acs.iecr.9b03702
|
[12] |
X. Yuan, S. Qi, Y. Wang, K. Wang, C. Yang, and L. Ye, “Quality variable prediction for nonlinear dynamic industrial processes based on temporal convolutional networks,” IEEE Sens. J., vol. 21, no. 18, pp. 20493–20503, Sept. 2021. doi: 10.1109/JSEN.2021.3096215
|
[13] |
P. Kadlec, R. Grbić, and B. Gabrys, “Review of adaptation mechanisms for data-driven soft sensors,” Comput. Chem. Eng., vol. 35, no. 1, pp. 1–24, Jan. 2011. doi: 10.1016/j.compchemeng.2010.07.034
|
[14] |
Q. Zhu, X. Zhang, Y. Wang, Y. Xu, and Y. He, “A novel intelligent model integrating PLSR with RBF-kernel based extreme learning machine: Application to modelling petrochemical process,” IFAC-PapersOnLine, vol. 52, no. 1, pp. 148–153, Jan. 2019. doi: 10.1016/j.ifacol.2019.06.052
|
[15] |
H. Abdi and L. J. Williams, “Principal component analysis,” WIREs Comput. Stat., vol. 2, no. 4, pp. 433–459, Jul.–Aug. 2010. doi: 10.1002/wics.101
|
[16] |
U. Depczynski, V. J. Frost, and K. Molt, “Genetic algorithms applied to the selection of factors in principal component regression,” Anal. Chim. Acta, vol. 420, no. 2, pp. 217–227, Sept. 2000. doi: 10.1016/S0003-2670(00)00893-X
|
[17] |
R. Rosipal and L. J. Trejo, “Kernel partial least squares regression in reproducing kernel hilbert space,” J. Mach. Learn. Res., vol. 2, pp. 97–123, Mar. 2002.
|
[18] |
J. Yang, A. F. Frangi, J. Y. Yang, D. Zhang, and Z. Jin, “KPCA plus LDA: A complete kernel Fisher discriminant framework for feature extraction and recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 2, pp. 230–244, Feb. 2005. doi: 10.1109/TPAMI.2005.33
|
[19] |
H. Kaneko and K. Funatsu, “Adaptive soft sensor based on online support vector regression and Bayesian ensemble learning for various states in chemical plants,” Chemom. Intell. Lab. Syst., vol. 137, pp. 57–66, Oct. 2014. doi: 10.1016/j.chemolab.2014.06.008
|
[20] |
V. E. Liong, J. Lu, and G. Wang, “Face recognition using deep PCA,” in Proc. 9th Int. Conf. Information, Communications and Signal Processing, Tainan, China, 2013, pp. 1–5.
|
[21] |
M. Ye, C. Ji, H. Chen, L. Lei, H. Lu, and Y. Qian, “Residual deep PCA-based feature extraction for hyperspectral image classification,” Neural Comput. Appl., vol. 32, no. 18, pp. 14287–14300, Sept. 2020. doi: 10.1007/s00521-019-04503-3
|
[22] |
X. Deng, X. Tian, S. Chen, and C. J. Harris, “Nonlinear process fault diagnosis based on serial principal component analysis,” IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 3, pp. 560–572, Mar. 2018. doi: 10.1109/TNNLS.2016.2635111
|
[23] |
X. Deng, X. Tian, S. Chen, and C. J. Harris, “Deep learning based nonlinear principal component analysis for industrial process fault detection,” in Proc. Int. Joint Conf. Neural Networks, Anchorage, USA, 2017, pp. 1237–1243.
|
[24] |
B. Shen, L. Yao, and Z. Ge, “Nonlinear probabilistic latent variable regression models for soft sensor application: From shallow to deep structure,” Control Eng. Pract., vol. 94, p. 104198, Jan. 2020. doi: 10.1016/j.conengprac.2019.104198
|
[25] |
X. Kong and Z. Ge, “Deep PLS: A lightweight deep learning model for interpretable and efficient data analytics,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 11, pp. 8923–8937, Nov. 2023. doi: 10.1109/TNNLS.2022.3154090
|
[26] |
Y. Chen and X. Deng, “A deep supervised learning framework based on kernel partial least squares for industrial soft sensing,” IEEE Trans. Ind. Inf., vol. 19, no. 3, pp. 3178–3187, Mar. 2023. doi: 10.1109/TII.2022.3182023
|
[27] |
Z. Chen, S. X. Ding, T. Peng, C. Yang, and W. Gui, “Fault detection for non-gaussian processes using generalized canonical correlation analysis and randomized algorithms,” IEEE Trans. Ind. Electron., vol. 65, no. 2, pp. 1559–1567, Feb. 2018. doi: 10.1109/TIE.2017.2733501
|
[28] |
D. Wang, J. Liu, and R. Srinivasan, “Data-driven soft sensor approach for quality prediction in a refining process,” IEEE Trans. Ind. Inf., vol. 6, no. 1, pp. 11–17, Feb. 2010. doi: 10.1109/TII.2009.2025124
|
[29] |
X. Yuan, J. Zhou, and Y. Wang, “A spatial-temporal LWPLS for adaptive soft sensor modeling and its application for an industrial hydrocracking process,” Chemom. Intell. Lab. Syst., vol. 197, p. 103921, Feb. 2020. doi: 10.1016/j.chemolab.2019.103921
|
[30] |
S. J. Qin and T.J. McAvoy, “Nonlinear PLS modeling using neural networks,” Comput. Chem. Eng., vol. 16, no. 4, pp. 379–391, Apr. 1992. doi: 10.1016/0098-1354(92)80055-E
|
[31] |
C. Ou, H. Zhu, Y. A. W. Shardt, L. Ye, X. Yuan, Y. Wang, and C. Yang, “Quality-driven regularization for deep learning networks and its application to industrial soft sensors,” IEEE Trans. Neural Netw. Learn. Syst., 2022. doi: 10.1109/TNNLS.2022.3144162
|
[32] |
X. Yuan, L. Feng, K. Wang, Y. Wang, and L. Ye, “Deep learning for data modeling of multirate quality variables in industrial processes,” IEEE Trans. Instrum. Meas., vol. 70, p. 2509611, May 2021.
|
[33] |
R. Rosipal and L. J. Trejo, “Kernel partial least squares regression in reproducing kernel hilbert space,” J. Mach. Learn. Res., vol. 2, pp. 97–123, Mar. 2002.
|