A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2025  > In Press, Accepted Manuscript
Z. Chen, L. Pan, Y. Ma, Z. Yang, L. Yao, J. Qian, and Z. Song, “E2AG: Entropy-regularized ensemble adaptive graph for industrial soft sensor modeling,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 4, pp. 745–760, Apr. 2025. doi: 10.1109/JAS.2024.124884
Citation: Z. Chen, L. Pan, Y. Ma, Z. Yang, L. Yao, J. Qian, and Z. Song, “E2AG: Entropy-regularized ensemble adaptive graph for industrial soft sensor modeling,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 4, pp. 745–760, Apr. 2025. doi: 10.1109/JAS.2024.124884
shu

E2AG: Entropy-Regularized Ensemble Adaptive Graph for Industrial Soft Sensor Modeling

doi: 10.1109/JAS.2024.124884
Funds:  This work was supported in part by the National Natural Science Foundation of China (NSFC) (62473103, 62203169, 62473121) and the Postdoctoral Science Foundation of Zhejiang Province (ZJ2023011)
More Information
    Abstract
  • Adaptive graph neural networks (AGNNs) have achieved remarkable success in industrial process soft sensing by incorporating explicit features that delineate the relationships between process variables. This article introduces a novel GNN framework, termed entropy-regularized ensemble adaptive graph (E2AG), aimed at enhancing the predictive accuracy of AGNNs. Specifically, this work pioneers a novel AGNN learning approach based on mirror descent, which is central to ensuring the efficiency of the training procedure and consequently guarantees that the learned graph naturally adheres to the row-normalization requirement intrinsic to the message-passing of GNNs. Subsequently, motivated by multi-head self-attention mechanism, the training of ensembled AGNNs is rigorously examined within this framework, incorporating an entropy regularization term in the learning objective to ensure the diversity of the learned graph. After that, the architecture and training algorithm of the model are then concisely summarized. Finally, to ascertain the efficacy of the proposed E2AG model, extensive experiments are conducted on real-world industrial datasets. The evaluation focuses on prediction accuracy, model efficacy, and sensitivity analysis, demonstrating the superiority of E2AG in industrial soft sensing applications.

     

    • FullText(HTML)
  • loading
    • 1 Supplementary Material of this paper can be found in link https://anonymous.4open.science/r/JA.
    References(54)
  • [1]
    F. Qian, Y. Jin, S. J. Qin, and K. Sundmacher, “Guest editorial special issue on deep integration of artificial intelligence and data science for process manufacturing,” IEEE Trans. Neural Networks and Learning Systems, vol. 32, no. 8, pp. 3294–3295, 2021. doi: 10.1109/TNNLS.2021.3092896
    [2]
    C. Liu, Y. Wang, C. Yang, and W. Gui, “Multimodal data-driven reinforcement learning for operational decision-making in industrial processes,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 1, pp. 252–254, 2024. doi: 10.1109/JAS.2023.123741
    [3]
    J. Qian, L. Jiang, and Z. Song, “Locally linear back-propagation based contribution for nonlinear process fault diagnosis,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 3, pp. 764–775, 2020. doi: 10.1109/JAS.2020.1003147
    [4]
    X. Jiang, X. Kong, and Z. Ge, “Augmented industrial data-driven modeling under the curse of dimensionality,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 6, pp. 1445–1461, 2023. doi: 10.1109/JAS.2023.123396
    [5]
    H. Xu, Z. Liu, H. Wang, C. Li, Y. Niu, W. Wang, and X. Liu, “Denoising diffusion straightforward models for energy conversion monitoring data imputation,” IEEE Trans. Industrial Informatics, pp. 1–11, 2024.
    [6]
    Q. Sun and Z. Ge, “Gated stacked target-related autoencoder: A novel deep feature extraction and layerwise ensemble method for industrial soft sensor application,” IEEE Trans. Cybern., vol. 52, no. 5, pp. 3457–3468, 2022. doi: 10.1109/TCYB.2020.3010331
    [7]
    K. Wang, C. Shang, L. Liu, Y. Jiang, D. Huang, and F. Yang, “Dynamic soft sensor development based on convolutional neural networks,” Industrial & Engineering Chemistry Research, vol. 58, no. 26, pp. 11521–11531, Jul. 2019.
    [8]
    C. Li, G. Li, Y. Song, Q. He, Z. Tian, H. Xu, and X. Liu, “Fast forest fire detection and segmentation application for uav-assisted mobile edge computing system,” IEEE Internet of Things J., vol. 11, no. 16, pp. 26690–26699, 2024. doi: 10.1109/JIOT.2023.3311950
    [9]
    L. Yao and Z. Ge, “Dynamic features incorporated locally weighted deep learning model for soft sensor development,” IEEE Trans. Instrumentation and Measurement, vol. 70, pp. 1–11, 2021.
    [10]
    C. Zhang, J. Yella, Y. Huang, X. Qian, S. Petrov, A. Rzhetsky, and S. Bom, “Soft sensing transformer: Hundreds of sensors are worth a single word,” in Proc. IEEE Int. Conf. Big Data, 2021, pp. 1999–2008.
    [11]
    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. 31st Int. Conf. Neural Inform. Processing Systems, 2017, pp. 6000–6010.
    [12]
    M. Li, A. Micheli, Y. G. Wang, S. Pan, P. Lió, G. S. Gnecco, and M. Sanguineti, “Guest editorial: Deep neural networks for graphs: Theory, models, algorithms, and applications,” IEEE Trans. Neural Networks and Learning Systems, vol. 35, no. 4, pp. 4367–4372, 2024. doi: 10.1109/TNNLS.2024.3371592
    [13]
    M. Jia, Y. Dai, D. Xu, T. Yang, Y. Yao, and Y. Liu, “Deep graph network for process soft sensor development,” in Proc. 8th Int. Conf. Inform., Cybern., and Computational Social Systems, 2021, pp. 1–6.
    [14]
    Y. Wang, P. Yan, and M. Gai, “Dynamic soft sensor for anaerobic digestion of kitchen waste based on SGSTGAT,” IEEE Sensors J., vol. 21, no. 17, pp. 19 198–19 208, 2021. doi: 10.1109/JSEN.2021.3090524
    [15]
    Y. Wang, Q. Sui, C. Liu, K. Wang, X. Yuan, and G. Dong, “Interpretable prediction modeling for froth flotation via stacked graph convolutional network,” IEEE Trans. Artificial Intelligence, vol. 5, no. 1, pp. 334–345, 2024. doi: 10.1109/TAI.2023.3240114
    [16]
    K. Zhu and C. Zhao, “Dynamic graph-based adaptive learning for online industrial soft sensor with mutable spatial coupling relations,” IEEE Trans. Industrial Electronics, vol. 70, no. 9, pp. 9614–9622, 2023. doi: 10.1109/TIE.2022.3215448
    [17]
    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “PyTorch: An imperative style, highperformance deep learning library,” in Proc. Advances in Neural Inform. Processing Systems, 2019, vol. 32, pp. 1–12.
    [18]
    J. Bradbury, R. Frostig, P. Hawkins, et al., “JAX: Autograd and XLA,” Astrophysics Source Code Library, pp. ascl-2111, 2021.
    [19]
    W. Liu, X. Zheng, J. Su, L. Zheng, C. Chen, and M. Hu, “Contrastive proxy kernel stein path alignment for cross-domain cold-start recommendation,” IEEE Trans. Knowledge and Data Engineering, vol. 35, no. 11, pp. 11 216–11 230, 2023. doi: 10.1109/TKDE.2022.3233789
    [20]
    M. Jing, Y. Zhu, T. Zang, J. Yu, and F. Tang, “Graph contrastive learning with adaptive augmentation for recommendation,” in Proc. Joint European Conf. Machine Learning and Knowledge Discovery in Databases, Springer, 2022, pp. 590–605.
    [21]
    Y. Jiang, C. Huang, and L. Huang, “Adaptive graph contrastive learning for recommendation,” in Proc. 29th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, 2023, pp. 4252–4261.
    [22]
    H. Wang, J. Fan, Z. Chen, H. Li, W. Liu, T. Liu, Q. Dai, Y. Wang, Z. Dong, and R. Tang, “Optimal transport for treatment effect estimation,” in Proc. Advances in Neural Inform. Proc. Systems, 2023, vol. 36, pp. 5404–5418.
    [23]
    H. Li, Y. Xiao, C. Zheng, P. Wu, Z. Geng, X. Chen, and P. Cui, “Debiased collaborative filtering with kernel-based causal balancing,” in Proc. Int. Conf. Learning Representations, 2024, pp. 1–12.
    [24]
    F. Bi, T. He, Y. Xie, and X. Luo, “Two-stream graph convolutional network-incorporated latent feature analysis,” IEEE Trans. Services Computing, vol. 16, no. 4, pp. 3027–3042, 2023. doi: 10.1109/TSC.2023.3241659
    [25]
    M. Jia, D. Xu, T. Yang, Y. Liu, and Y. Yao, “Graph convolutional network soft sensor for process quality prediction,” J. Process Control, vol. 123, pp. 12–25, 2023. doi: 10.1016/j.jprocont.2023.01.010
    [26]
    R. Zhai, J. Zeng, and Z. Ge, “Structured principal component analysis model with variable correlation constraint,” IEEE Trans. Control Systems Technology, vol. 30, no. 2, pp. 558–569, 2022. doi: 10.1109/TCST.2021.3069539
    [27]
    Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” in Proc. 28th Int. Joint Conf. Artificial Intelligence, 2019, pp. 1907–1913.
    [28]
    B. N. Oreshkin, A. Amini, L. Coyle, and M. Coates, “FC-GAGA: Fully connected gated graph architecture for spatio-temporal traffic forecasting,” in Proc. AAAI Conf. Artificial Intelligence, 2021, vol. 35, no. 10, pp. 9233–9241.
    [29]
    Y. Huang, C. Zhang, J. Yella, S. Petrov, X. Qian, Y. Tang, X. Zhu, and S. Bom, “GrassNet: Graph soft sensing neural networks,” in Proc. IEEE Int. Conf. Big Data, 2021, pp. 746–756.
    [30]
    H. Wang, Z. Wang, Y. Niu, Z. Liu, H. Li, Y. Liao, Y. Huang, and X. Liu, “An accurate and interpretable framework for trustworthy process monitoring,” IEEE Trans. Artificial Intelligence, pp. 1–12, 2023.
    [31]
    Z. Liu, Y. Cao, H. Xu, Y. Huang, Q. He, X. Chen, X. Tang, and X. Liu, “Hidformer: Hierarchical dual-tower transformer using multiscale mergence for long-term time series forecasting,” Expert Systems With Applications, vol. 239, p. 122412, 2024.
    [32]
    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proc. Int. Conf. Learning Representations, 2016, pp. 1–14.
    [33]
    X. Hong, T. Zhang, Z. Cui, and J. Yang, “Variational gridded graph convolution network for node classification,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 10, pp. 1697–1708, 2021. doi: 10.1109/JAS.2021.1004201
    [34]
    W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Proc. Advances in Neural Inform. Processing Systems, 2017, pp. 1025–1035.
    [35]
    P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in Proc. Int. Conf. Learning Representations, 2018, pp. 1–12.
    [36]
    Y. Ma and J. Tang, Deep Learning on Graphs. Cambridge, UK: Cambridge University Press, 2020.
    [37]
    J. Darzentas, “Problem complexity and method efficiency in optimization,” J. Operational Research Society, vol. 35, no. 5, pp. 455–455, 1984. doi: 10.1057/jors.1984.92
    [38]
    J. Shi, C. Liu, and L. Mackey, “Sampling with mirrored Stein operators,” in Proc. Int. Conf. Learning Representations, 2022, pp. 1–26.
    [39]
    S. Patterson and Y. W. Teh, “Stochastic gradient Riemannian langevin dynamics on the probability simplex,” Advances in Neural Inform. Proc. Systems, 2013, vol. 26, pp. 1–9.
    [40]
    Q. Liu and D. Wang, “Stein variational gradient descent: A general purpose bayesian inference algorithm,” in Proc. Advances in Neural Inform. Proc. Systems, 2016, vol. 29
    [41]
    Y. Li and R. E. Turner, “Gradient estimators for implicit models,” in Proc. Int. Conf. Learning Representations, 2018, pp. 1–13.
    [42]
    D. Wang and Q. Liu, “Nonlinear stein variational gradient descent for learning diversified mixture models,” in Proc. Int. Conf. Machine Learning, 2019, pp. 6576–6585.
    [43]
    D. P. Kingma and J. Ba, “ADAM: A method for stochastic optimization,” in Proc. Int. Conf. Learning Representations, 2015, pp. 1–8.
    [44]
    T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Ré, “FlashAttention: Fast and memory-efficient exact attention with IO-awareness,” in Proc. Advances in Neural Inform. Proc. Systems, 2022, pp. 1–16.
    [45]
    Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, “iTransformer: Inverted transformers are effective for time series forecasting,” in Proc. 12th Int. Conf. Learning Representations, 2024, pp. 1–22.
    [46]
    Y. Tay, D. Bahri, D. Metzler, D.-C. Juan, Z. Zhao, and C. Zheng, “Synthesizer: Rethinking self-attention for transformer models,” in Proc. Int. Conf. Machine Learning, 2021, pp. 10183–10192.
    [47]
    J. Chen, Y. Yuan, and X. Luo, “SDGNN: Symmetry-preserving dualstream graph neural networks,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 7, pp. 1717–1719, 2024. doi: 10.1109/JAS.2024.124410
    [48]
    C. Huang, M. Li, F. Cao, H. Fujita, Z. Li, and X. Wu, “Are graph convolutional networks with random weights feasible?” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 2751–2768, 2023. doi: 10.1109/TPAMI.2022.3183143
    [49]
    K. Huang, Y. G. Wang, M. Li, and P. Lio, “How universal polynomial bases enhance spectral graph neural networks: Heterophily, oversmoothing, and over-squashing,” in Proc. 41st Int. Conf. Machine Learning, 2024, vol. 235, pp. 20310–20330.
    [50]
    H. Dong, X. Wang, L. Yong, and T. Zhang, “Particle-based variational inference with preconditioned functional gradient flow,” in Proc. 11th Int. Conf. Learning Representations, 2022, pp. 1–26.
    [51]
    F. Wang, H. Zhu, C. Zhang, H. Zhao, and H. Qian, “GAD-PVI: A general accelerated dynamic-weight particle-based variational inference framework,” in Proc. AAAI Conf. Artificial Intelligence, 2023, vol. 37, pp. 1–29.
    [52]
    H. Wang, T.-W. Chang, T. Liu, J. Huang, Z. Chen, C. Yu, R. Li, and W. Chu, “ESCM2: Entire space counterfactual multi-task model for post-click conversion rate estimation,” in Proc. 45th Int. ACM SIGIR Conf. Research and Development in Information Retrieval, 2022, p. 363−372.
    [53]
    C. Liu and J. Zhu, “Riemannian stein variational gradient descent for bayesian inference,” in Proc. AAAI Conf. Artificial Intelligence, vol. 32, no. 1, 2018.
    [54]
    R. Zhang, Q. Liu, and X. Tong, “Sampling in constrained domains with orthogonal-space variational gradient descent,” in Proc. Advances in Neural Information Processing Systems, 2022, vol. 35, no. 37, pp. 37108–37120.
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (70) PDF downloads(12) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return