A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 10 Issue 2
Feb.  2023

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 17.6, Top 3% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2023  >  10(2): 524-546
W. T. Mao, G. S. Wang, L. L. Kou, and X. H. Liang, “Deep domain-adversarial anomaly detection with one-class transfer learning,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 2, pp. 524–546, Feb. 2023. doi: 10.1109/JAS.2023.123228
Citation: W. T. Mao, G. S. Wang, L. L. Kou, and X. H. Liang, “Deep domain-adversarial anomaly detection with one-class transfer learning,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 2, pp. 524–546, Feb. 2023. doi: 10.1109/JAS.2023.123228
shu

Deep Domain-Adversarial Anomaly Detection With One-Class Transfer Learning

doi: 10.1109/JAS.2023.123228
  • 1.

    School of Computer and Information Engineering, Henan Normal University, Xinxiang 453007, China

  • 2.

    Technology Department, Beijing Mass Transit Railway Operation Corp. Ltd., Beijing 100044, China

  • 3.

    Department of Mechanical Engineering, University of Manitoba, Winnipeg R3T 5V6, Canada

Funds:  This work was supported by the National Natural Science Foundation of China (NSFC) (U1704158), Henan Province Technologies Research and Development Project of China (212102210103), the NSFC Development Funding of Henan Normal University (2020PL09), and the University of Manitoba Research Grants Program (URGP)
More Information
    Abstract
  • Despite the big success of transfer learning techniques in anomaly detection, it is still challenging to achieve good transition of detection rules merely based on the preferred data in the anomaly detection with one-class classification, especially for the data with a large distribution difference. To address this challenge, a novel deep one-class transfer learning algorithm with domain-adversarial training is proposed in this paper. First, by integrating a hypersphere adaptation constraint into domain-adversarial neural network, a new hypersphere adversarial training mechanism is designed. Second, an alternative optimization method is derived to seek the optimal network parameters while pushing the hyperspheres built in the source domain and target domain to be as identical as possible. Through transferring one-class detection rule in the adaptive extraction of domain-invariant feature representation, the end-to-end anomaly detection with one-class classification is then enhanced. Furthermore, a theoretical analysis about the model reliability, as well as the strategy of avoiding invalid and negative transfer, is provided. Experiments are conducted on two typical anomaly detection problems, i.e., image recognition detection and online early fault detection of rolling bearings. The results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in terms of detection accuracy and robustness.

     

    • FullText(HTML)
  • loading
    • References(50)
  • [1]
    X. X. Ma, J. Wu, S. Xue, J. Yang, C. Zhou, Q. Z. Sheng, H. Xiong, and L. Akoglu, “A comprehensive survey on graph anomaly detection with deep learning,” IEEE Trans. Knowl. Data Eng., 2021. DOI: 10.1109/TKDE.2021.3118815.
    [2]
    X. X. Luo, J. Wu, A. Beheshti, J. Yang, X. K. Zhang, Y. Wang, and S. Xue, “ComGA: Community-aware attributed graph anomaly detection,” in Proc. Fifteenth ACM Int. Conf. Web Search and Data Mining, New York, USA, 2022, pp. 657–665.
    [3]
    J. Jabez and B. Muthukumar, “Intrusion detection system (IDS): Anomaly detection using outlier detection approach,” Procedia Comput. Sci., vol. 48, pp. 338–346, Dec. 2015. doi: 10.1016/j.procs.2015.04.191
    [4]
    G. Zhang, Z. Li, J. M. Huang, J. Wu, C. Zhou, J. Yang, and J. L. Gao, “eFraudcom: An E-commerce fraud detection system via competitive graph neural networks,” ACM Trans. Inf. Syst., vol. 40, no. 3, p. 47, Jul. 2022.
    [5]
    G. Zhang, J. Wu, J. Yang, A. Beheshti, S. Xue, C. Zhou, and Q. Z. Sheng, “FRAUDRE: Fraud detection dual-resistant to graph inconsistency and imbalance,” in Proc. IEEE Int. Conf. Data Mining, Auckland, New Zealand, 2021, pp. 867–876.
    [6]
    G. S. Pang, C. H. Shen, L. B. Cao, and A. Van Den Hengel, “Deep learning for anomaly detection: A review,” ACM Comput. Surv., vol. 54, no. 2, p. 38, Mar. 2022.
    [7]
    J. Park, J. H. Moon, N. Ahn, and K. A. Sohn, “What is wrong with one-class anomaly detection?” arXiv preprint arXiv: 2104.09793, 2021.
    [8]
    E. Hong and Y. Choe, “Latent feature decentralization loss for one-class anomaly detection,” IEEE Access, vol. 8, pp. 165658–165669, Sept. 2020. doi: 10.1109/ACCESS.2020.3022646
    [9]
    Y. B. Li, Y. T. Yang, X. Z. Wang, B. B. Liu, and X. H. Liang, “Early fault diagnosis of rolling bearings based on hierarchical symbol dynamic entropy and binary tree support vector machine,” J. Sound Vib., vol. 428, pp. 72–86, Aug. 2018. doi: 10.1016/j.jsv.2018.04.036
    [10]
    J. Saari, D. Strömbergsson, J. Lundberg, and A. Thomson, “Detection and identification of windmill bearing faults using a one-class support vector machine (SVM),” Measurement, vol. 137, pp. 287–301, Apr. 2019. doi: 10.1016/j.measurement.2019.01.020
    [11]
    Y. Chen, Z. Zhao, H. Wu, X. Chen, Q. Xiao, and Y. Yu, “Fault anomaly detection of synchronous machine winding based on isolation forest and impulse frequency response analysis,” Measurement, vol. 188, p. 110531, 2022.
    [12]
    Y. G. Lei, B. Yang, X. W. Jiang, F. Jia, N. P. Li, and A. K. Nandi, “Applications of machine learning to machine fault diagnosis: A review and roadmap,” Mech. Syst. Signal Process., vol. 138, p. 106587, Apr. 2020. doi: 10.1016/j.ymssp.2019.106587
    [13]
    X. S. Zhang, Y. Zhuang, F. Yan, and W. Wang, “Status and development of transfer learning based category-level object recognition and detection,” Acta Autom. Sinica, vol. 45, no. 7, pp. 1224–1243, Jul. 2019.
    [14]
    Z. J. Wang, J. Cui, W. N. Cai, and Y. F. Li, “Partial transfer learning of multidiscriminator deep weighted adversarial network in cross-machine fault diagnosis,” IEEE Trans. Instrum. Meas., vol. 71, p. 5010010, Apr. 2022.
    [15]
    W. T. Mao, J. Liu, J. X. Chen, and X. H. Liang, “An interpretable deep transfer learning-based remaining useful life prediction approach for bearings with selective degradation knowledge fusion,” IEEE Trans. Instrum. Meas., vol. 71, p. 3508616, Mar. 2022.
    [16]
    Y. S. Xiao, B. Liu, P. S. Yu, and Z. F. Hao, “A robust one-class transfer learning method with uncertain data,” Knowl. Inf. Syst., vol. 44, no. 2, pp. 407–438, Aug. 2015. doi: 10.1007/s10115-014-0765-8
    [17]
    H. Yu, J. W. Han, and K. C. C. Chang, “PEBL: Web page classification without negative examples,” IEEE Trans. Knowl. Data Eng., vol. 16, no. 1, pp. 70–81, Jan. 2004. doi: 10.1109/TKDE.2004.1264823
    [18]
    Y. Zhao, S. W. Wang, and F. Xiao, “Pattern recognition-based chillers fault detection method using support vector data description (SVDD),” Appl. Energy, vol. 112, pp. 1041–1048, Dec. 2013. doi: 10.1016/j.apenergy.2012.12.043
    [19]
    J. H. Chen, S. Sathe, C. C. Aggarwal, and D. S. Turaga, “Outlier detection with autoencoder ensembles,” in Proc. SIAM Int. Conf. Data Mining, Houston, USA, 2017, pp. 90–98.
    [20]
    G. S. Pang, C. Yan, C. H. Shen, A. van den Hengel, and X. Bai, “Self-trained deep ordinal regression for end-to-end video anomaly detection,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Seattle, USA, 2020, pp. 12173–12182.
    [21]
    L. Ruff, R. A. Vandermeulen, N. Görnitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, and M. Kloft, “Deep one-class classification,” in Proc. 35th Int. Conf. Machine Learning, Stockholm, Sweden, 2018, pp. 4393–4402.
    [22]
    C. Aytekin, X. Y. Ni, F. Cricri, and E. Aksu, “Clustering and unsupervised anomaly detection with l2 normalized deep auto-encoder representations,” in Proc. Int. Joint Conf. Neural Networks, Rio de Janeiro, Brazil, 2018, pp. 1–6.
    [23]
    P. P. Zheng, S. H. Yuan, X. T. Wu, J. Li, and A. D. Lu, “One-class adversarial nets for fraud detection,” in Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 1286–1293, Jul. 2019.
    [24]
    S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, “Domain adaptation via transfer component analysis,” IEEE Trans. Neural Netw., vol. 22, no. 2, pp. 199–210, Feb. 2011. doi: 10.1109/TNN.2010.2091281
    [25]
    J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in Proc. 27th Int. Conf. Neural Information Processing Systems, Montreal, Canada, 2014, pp. 3320–3328.
    [26]
    E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, “Deep domain confusion: Maximizing for domain invariance,” arXiv preprint arXiv: 1412.3474, 2014.
    [27]
    Y. J. Xue and P. Beauseroy, “Transfer learning for one class SVM adaptation to limited data distribution change,” Pattern Recognit. Lett., vol. 100, pp. 117–123, Dec. 2017. doi: 10.1016/j.patrec.2017.10.030
    [28]
    V. A. Sindagi and S. Srivastava, “Domain adaptation for automatic OLED panel defect detection using adaptive support vector data description,” Int. J. Comput. Vis., vol. 122, no. 2, pp. 193–211, Apr. 2017. doi: 10.1007/s11263-016-0953-y
    [29]
    J. X. Chen and X. M. Liu, “Transfer learning with one-class data,” Pattern Recognit. Lett., vol. 37, pp. 32–40, Feb. 2014. doi: 10.1016/j.patrec.2013.07.017
    [30]
    H. X. Xie, B. Liu, and Y. S. Xiao, “Transfer learning-based one-class dictionary learning for recommendation data stream,” Inf. Sci., vol. 547, pp. 526–538, Feb. 2021. doi: 10.1016/j.ins.2020.08.091
    [31]
    C. Q. Tan, F. C. Sun, T. Kong, W. C. Zhang, C. Yang, and C. F. Liu, “A survey on deep transfer learning,” in Proc. 27th Int. Conf. Artificial Neural Networks, Rhodes, Greece, 2018, pp. 270–279.
    [32]
    J. T. A. Andrews, T. Tanay, E. J. Morton, and L. D. Griffin, “Transfer representation-learning for anomaly detection,” in Proc. 33rd Int. Conf. Machine Learning, New York, USA, 2016.
    [33]
    W. T. Mao, L. Ding, S. Y. Tian, and X. H. Liang, “Online detection for bearing incipient fault based on deep transfer learning,” Measurement, vol. 152, p. 107278, Feb. 2020. doi: 10.1016/j.measurement.2019.107278
    [34]
    Y. Zhou, X. M. Liang, W. Zhang, L. R. Zhang, and X. Song, “VAE-based deep SVDD for anomaly detection,” Neurocomputing, vol. 453, pp. 131–140, Sept. 2021. doi: 10.1016/j.neucom.2021.04.089
    [35]
    P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, “Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Seattle, USA, 2020, pp. 4183–4192.
    [36]
    K. Sohn, C. L. Li, J. Yoon, M. Jin, and T. Pfister, “Learning and evaluating representations for deep one-class classification,” in Proc. 9th Int. Conf. Learning Representations, 2020.
    [37]
    T. Reiss, N. Cohen, L. Bergman, and Y. Hoshen, “PANDA: Adapting pretrained features for anomaly detection and segmentation,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Nashville, USA, 2021, pp. 2806–2814.
    [38]
    W. T. Mao, Y. M. Liu, L. Ding, A. Safian, and X. H. Liang, “A new structured domain adversarial neural network for transfer fault diagnosis of rolling bearings under different working conditions,” IEEE Trans. Instrum. Meas., vol. 70, p. 3509013, 2021.
    [39]
    H. L. Li, S. J. Pan, S. Q. Wang, and A. C. Kot, “Domain generalization with adversarial feature learning,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, pp. 5400–5409.
    [40]
    Z. Y. Yang, I. S. Bozchalooi, and E. Darve, “Anomaly detection with domain adaptation,” arXiv preprint arXiv: 2006.03689, 2020.
    [41]
    X. Han and S. H. Yuan, “Unsupervised cross-system log anomaly detection via domain adaptation,” in Proc. 30th ACM Int. Conf. Information & Knowledge Management, Virtual Event, Australia, 2021, pp. 3068–3072.
    [42]
    W. N. Lu, B. Liang, Y. Cheng, D. S. Meng, J. Yang, and T. Zhang, “Deep model based domain adaptation for fault diagnosis,” IEEE Trans. Ind. Electron., vol. 64, no. 3, pp. 2296–2305, Mar. 2017. doi: 10.1109/TIE.2016.2627020
    [43]
    W. T. Mao, S. Y. Tian, J. J. Fan, X. H. Liang, and A. Safian, “Online detection of bearing incipient fault with semi-supervised architecture and deep feature representation,” J. Manuf. Syst., vol. 55, pp. 179–198, Apr. 2020. doi: 10.1016/j.jmsy.2020.03.005
    [44]
    H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, USA, 2017, pp. 5385–5394.
    [45]
    P. Nectoux, R. Gouriveau, K. Medjaher, E. Ramasso, B. Chebel-Morello, N. Zerhouni, and C. Varnier, “PRONOSTIA: An experimental platform for bearings accelerated degradation tests,” in Proc. IEEE Int. Conf. Prognostics and Health Management, 2012, pp. 1–8.
    [46]
    B. Wang, Y. G. Lei, N. P. Li, and N. B. Li, “A hybrid prognostics approach for estimating remaining useful life of rolling element bearings,” IEEE Trans. Reliab., vol. 69, no. 1, pp. 401–412, Mar. 2020. doi: 10.1109/TR.2018.2882682
    [47]
    Y. B. Li, M. Q. Xu, X. H. Liang, and W. H. Huang, “Application of bandwidth EMD and adaptive multiscale morphology analysis for incipient fault diagnosis of rolling bearings,” IEEE Trans. Ind. Electron., vol. 64, no. 8, pp. 6506–6517, Aug. 2017. doi: 10.1109/TIE.2017.2650873
    [48]
    W. T. Mao, J. X. Chen, X. H. Liang, and X. M. Zhang, “A new online detection approach for rolling bearing incipient fault via self-adaptive deep feature matching,” IEEE Trans. Instrum. Meas., vol. 69, no. 2, pp. 443–456, Feb. 2020. doi: 10.1109/TIM.2019.2903699
    [49]
    W. N. Lu, Y. P. Li, Y. Cheng, D. S. Meng, B. Liang, and P. Zhou, “Early fault detection approach with deep architectures,” IEEE Trans. Instrum. Meas., vol. 67, no. 7, pp. 1679–1689, Jul. 2018. doi: 10.1109/TIM.2018.2800978
    [50]
    X. P. Guo, S. Y. Liu, and Y. Li, “Fault detection of multi-mode processes employing sparse residual distance,” Acta Autom. Sinica, vol. 45, no. 3, pp. 617–625, Mar. 2019.
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(19)  / Tables(5)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (914) PDF downloads(130) Cited by()

    Highlights

    • We propose a new deep transfer learning algorithm for one-class anomaly detection
    • We build a hypersphere adversarial training mechanism to transfer detection rule
    • We theoretically analyze model reliability to avoid invalid and negative transfer
    • This method can improve detection accuracy and robustness on insufficient and noisy data

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return