A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 8 Issue 4
Apr.  2021

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2021  >  8(4): 796-805
Di Wu, Xin Luo, "Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data," IEEE/CAA J. Autom. Sinica, vol. 8, no. 4, pp. 796-805, Apr. 2021. doi: 10.1109/JAS.2020.1003533
Citation: Di Wu, Xin Luo, "Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data," IEEE/CAA J. Autom. Sinica, vol. 8, no. 4, pp. 796-805, Apr. 2021. doi: 10.1109/JAS.2020.1003533
shu

Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data

doi: 10.1109/JAS.2020.1003533
  • 1.

    Chongqing Key Laboratory of Big Data and Intelligent Computing, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400714, China

  • 2.

    Chongqing School, University of Chinese Academy of Sciences, Chongqing 400714, China

Funds:  This work was supported in part by the National Natural Science Foundation of China (61702475, 61772493, 61902370, 62002337), in part by the Natural Science Foundation of Chongqing, China (cstc2019jcyj-msxmX0578, cstc2019jcyjjqX0013), in part by the Chinese Academy of Sciences “Light of West China” Program, in part by the Pioneer Hundred Talents Program of Chinese Academy of Sciences, and by Technology Innovation and Application Development Project of Chongqing, China (cstc2019jscx-fxydX0027)
More Information
    Abstract
  • High-dimensional and sparse (HiDS) matrices commonly arise in various industrial applications, e.g., recommender systems (RSs), social networks, and wireless sensor networks. Since they contain rich information, how to accurately represent them is of great significance. A latent factor (LF) model is one of the most popular and successful ways to address this issue. Current LF models mostly adopt L 2-norm-oriented Loss to represent an HiDS matrix, i.e., they sum the errors between observed data and predicted ones with L 2-norm. Yet L 2-norm is sensitive to outlier data. Unfortunately, outlier data usually exist in such matrices. For example, an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users. To address this issue, this work proposes a smooth L 1-norm-oriented latent factor (SL-LF) model. Its main idea is to adopt smooth L 1-norm rather than L 2-norm to form its Loss, making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix. Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.

     

    • FullText(HTML)
  • loading
    • 1 https://pan.baidu.com/s/1o_8sKP0HRluNH1a4IWHW8w, Code: t3sw
    References(58)
  • [1]
    R. Q. Lu, X. L. Jin, S. M. Zhang, M. K. Qiu, and X. D. Wu, “A study on big knowledge and its engineering issues,” IEEE Trans. Knowl. Data Eng., vol. 31, no. 9, pp. 1630–1644, Sep. 2019. doi: 10.1109/TKDE.2018.2866863
    [2]
    S. C. Gao, M. C. Zhou, Y. R. Wang, J. J. Cheng, H. Yachi, and J. H. Wang, “Dendritic neuron model with effective learning algorithms for classification, approximation, and prediction,” IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 2, pp. 601–614, Feb. 2019. doi: 10.1109/TNNLS.2018.2846646
    [3]
    D. P. Bertsekas, “Feature-based aggregation and deep reinforcement learning: A survey and some new implementations,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 1–31, Jan. 2019. doi: 10.1109/JAS.2018.7511249
    [4]
    H. Zahid, T. Mahmood, A. Morshed, and T. Sellis, “Big data analytics in telecommunications: Literature review and architecture recommendations,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 1, pp. 18–38, Jan. 2020.
    [5]
    Y. F. Ma, Z. Y. Wang, H. Yang, and L. Yang, “Artificial intelligence applications in the development of autonomous vehicles: A survey,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 2, pp. 315–329, Mar. 2020. doi: 10.1109/JAS.2020.1003021
    [6]
    D. Ryu, K. Lee, and J. Baik, “Location-based web service QoS prediction via preference propagation to address cold start problem,” IEEE Trans. Serv. Comput., to be published. DOI: 10.1109/TSC.2018.2821686
    [7]
    X. Luo, Z. G. Liu, S. Li, M. S. Shang, and Z. D. Wang, “A fast non-negative latent factor model based on generalized momentum method,” IEEE Trans. Syst., Man, Cybern.: Syst., to be published. DOI: 10.1109/TSMC.2018.2875452
    [8]
    J. D. Zhang, C. Y. Chow, and J. Xu, “Enabling kernel-based attribute-aware matrix factorization for rating prediction,” IEEE Trans. Knowl. Data Eng., vol. 29, no. 4, pp. 798–812, Apr. 2017. doi: 10.1109/TKDE.2016.2641439
    [9]
    J. Castro, J. Lu, G. G. Zhang, Y. C. Dong, and L. Martínez, “Opinion dynamics-based group recommender systems,” IEEE Trans. Syst., Man, Cybern.: Syst., vol. 48, no. 12, pp. 2394–2406, Dec. 2018. doi: 10.1109/TSMC.2017.2695158
    [10]
    B. Smith and G. Linden, “Two decades of recommender systems at Amazon.com,” IEEE Int. Comput., vol. 21, no. 3, pp. 12–18, May-Jun. 2017. doi: 10.1109/MIC.2017.72
    [11]
    M. G. Gong, X. M. Jiang, H. Li, and K. C. Tan, “Multiobjective sparse non-negative matrix factorization,” IEEE Trans. Cybern., vol. 49, no. 8, pp. 2941–2954, Aug. 2019. doi: 10.1109/TCYB.2018.2834898
    [12]
    X. N. He, J. H. Tang, X. Y. Du, R. C. Hong, T. W. Ren, and T. S. Chua, “Fast matrix factorization with nonuniform weights on missing data,” IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 8, pp. 2791–2804, Aug. 2020. doi: 10.1109/TNNLS.2018.2890117
    [13]
    X. Luo, M. C. Zhou, Y. N. Xia, and Q. S. Zhu, “An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems,” IEEE Trans. Ind. Inform., vol. 10, no. 2, pp. 1273–1284, May 2014. doi: 10.1109/TII.2014.2308433
    [14]
    Y. Koren, R. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” Computer, vol. 42, no. 8, pp. 30–37, Aug. 2009. doi: 10.1109/MC.2009.263
    [15]
    X. Luo, M. C. Zhou, S. Li, Z. H. You, Y. N. Xia, and Q. S. Zhu, “A nonnegative latent factor model for large-scale sparse matrices in recommender systems via alternating direction method,” IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 3, pp. 579–592, Mar. 2016. doi: 10.1109/TNNLS.2015.2415257
    [16]
    D. Wu, Q. He, X. Luo, M. S. Shang, Y. He, and G. Y. Wang, “A Posterior-neighborhood-regularized latent factor model for highly accurate web service QoS prediction,” IEEE Trans. Serv. Comput., to be published. DOI: 10.1109/TSC.2019.2961895
    [17]
    M. S. Shang, X. Luo, Z. G. Liu, J. Chen, Y. Yuan, and M. C. Zhou, “Randomized latent factor model for high-dimensional and sparse matrices from industrial applications,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 131–141, Jan. 2019. doi: 10.1109/JAS.2018.7511189
    [18]
    C. C. Leng, H. Zhang, G. R. Cai, I. Cheng, and A. Basu, “Graph regularized Lp smooth non-negative matrix factorization for data representation,” IEEE/CAA J. Automa. Sinica, vol. 6, no. 2, pp. 584–595, Mar. 2019. doi: 10.1109/JAS.2019.1911417
    [19]
    X. K. Zhu, X. Y. Jing, D. Wu, Z. Y. He, J. C. Cao, D. Yue, and L. N. Wang, “Similarity-maintaining privacy preservation and Location-aware Low-rank matrix factorization for QoS prediction based web service recommendation,” IEEE Trans. Serv. Comput., to be published. DOI: 10.1109/TSC.2018.2839741
    [20]
    L. Wang, M. D. Gordon, and J. Zhu, “Regularized least absolute deviations regression and an efficient algorithm for parameter tuning,” in Proc. 6th IEEE Int. Conf. Data Mining, Hong Kong, China, 2006, pp. 690–700.
    [21]
    W. T. Ma, N. Li, Y. H. Li, J. D. Duan, and B. D. Chen, “Sparse normalized least mean absolute deviation algorithm based on unbiasedness criterion for system Identification with noisy input,” IEEE Access, vol. 6, pp. 14379–14388, Feb. 2018. doi: 10.1109/ACCESS.2018.2800278
    [22]
    Q. F. Ke and T. Kanade, “Robust subspace computation using L1 norm: School of computer science,” Carnegie Mellon University, Pittsburgh, PA, CMU-CS-03-172, Aug. 2003.
    [23]
    R. Koenker and K. F. Hallock, “Quantile regression,” J. Econom. Perspect., vol. 15, no. 4, pp. 143–156, 2001. doi: 10.1257/jep.15.4.143
    [24]
    C. Wu, W. W. Qiu, Z. B. Zheng, X. Y. Wang, and X. H. Yang, “Qos prediction of web services based on two-phase K-means clustering,” in Proc. IEEE Int. Conf. Web Services, New York, USA, 2015, pp. 161–168.
    [25]
    B. Lakshminarayanan, G. Bouchard, and C. Archambeau, “Robust Bayesian matrix factorisation,” in Proc. the 14th Int. Conf. Artificial Intelligence and Statistics, Ft. Lauderdale, USA, 2011, pp. 425–433.
    [26]
    D. Wu, Y. He, X. Luo, M. S. Shang, and X. D. Wu, “Online feature selection with capricious streaming features: A general framework,” in Proc. IEEE Int. Conf. Big Data, Los Angeles, USA, 2019, pp. 683–688.
    [27]
    Y. Yuan, X. Luo, and M. S. Shang, “Effects of preprocessing and training biases in latent factor models for recommender systems,” Neurocomputing, vol. 275, pp. 2019–2030, Jan. 2018. doi: 10.1016/j.neucom.2017.10.040
    [28]
    Y. Koren and R. Bell, “Advances in collaborative filtering,” in Recommender Systems Handbook, F. Ricci, L. Rokach, B. Shapira, P. B. Kantor, Eds. Boston, USA: Springer, 2015, pp. 77–118.
    [29]
    P. Massa and P. Avesani, “Trust-aware recommender systems,” in Proc. ACM Conf. Recommender Systems, Minneapolis, USA, 2007, pp. 17–24.
    [30]
    L. Brozovsky and V. Petricek, “Recommender system for online dating service,” arXiv: cs/0703042, 2007.
    [31]
    Y. Shi, M. Larson, and A. Hanjalic, “Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges,” ACM Comput. Surv., vol. 47, no. 1, Article No. 3, May 2014.
    [32]
    K. Goldberg, T. Roeder, D. Gupta, and C. Perkins, “Eigentaste: A constant time collaborative filtering algorithm,” Inform. Retrieval, vol. 4, no. 2, pp. 133–151, Jul. 2001. doi: 10.1023/A:1011419012209
    [33]
    J. A. Konstan, B. N. Miller, D. Maltz, J. L. Herlocker, L. R. Gordon, and J. Riedl, “GroupLens: Applying collaborative filtering to Usenet news,” Commun. ACM, vol. 40, no. 3, pp. 77–87, Mar. 1997. doi: 10.1145/245108.245126
    [34]
    S. Zhang, L. N. Yao, A. X. Sun, and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” ACM Comput. Surv., vol. 52, no. 1, Article No. 5, Feb. 2019.
    [35]
    D. Wu, X. Luo, M. S. Shang, Y. He, G. Y. Wang, and X. D. Wu, “A data-characteristic-aware latent factor model for web services QoS prediction,” IEEE Trans. Knowl. Data Eng., to be published. DOI: 10.1109/TKDE.2020.3014302
    [36]
    P. Y. Zhang, S. Shu, and M. C. Zhou, “An online fault detection model and strategies based on SVM-Grid in clouds,” IEEE/CAA J. Autom. Sinica, vol. 5, no. 2, pp. 445–456, Mar. 2018. doi: 10.1109/JAS.2017.7510817
    [37]
    S. Sedhain, A. K. Menon, S. Sanner, and L. X. Xie, “AutoRec: Autoencoders meet collaborative filtering,” in Proc. the 24th Int. Conf. World Wide Web, Florence, Italy, 2015, pp. 111–112.
    [38]
    Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, Article no. 7553, pp. 436−444, May 2015.
    [39]
    Q. X. Wang, B. B. Peng, X. Y. Shi, T. Q. Shang, and M. S. Shang, “DCCR: Deep collaborative conjunctive recommender for rating prediction,” IEEE Access, vol. 7, pp. 60186–60198, May 2019. doi: 10.1109/ACCESS.2019.2915531
    [40]
    J. Demšar, “Statistical comparisons of classifiers over multiple data sets,” J. Mach. Learn. Res., vol. 7, pp. 1–30, Dec. 2006.
    [41]
    B. Rosner, R. J. Glynn, and M. L. T. Lee, “The Wilcoxon signed rank test for paired comparisons of clustered data,” Biometrics, vol. 62, no. 1, pp. 185–192, Mar. 2006. doi: 10.1111/j.1541-0420.2005.00389.x
    [42]
    D. Wu, X. Luo, G. Y. Wang, M. S. Shang, Y. Yuan, and H. Y. Yan, “A highly accurate framework for self-labeled semisupervised classification in industrial applications,” IEEE Trans. Ind. Inform., vol. 14, no. 3, pp. 909–920, Mar. 2018. doi: 10.1109/TII.2017.2737827
    [43]
    Z. H. Zhou and J. Feng, “Deep forest: Towards an alternative to deep neural networks,” in Proc. the 26th Int. Joint Conf. on Artificial Intelligence, Melbourne, Australia, 2017, pp. 3553–3559.
    [44]
    X. Y. Ren, M. N. Song, E. Haihong, and J. D. Song, “Context-aware probabilistic matrix factorization modeling for point-of-interest recommendation,” Neurocomputing, vol. 241, pp. 38–55, Jun. 2017. doi: 10.1016/j.neucom.2017.02.005
    [45]
    H. Wu, Z. X. Zhang, K. Yue, B. B. Zhang, J. He, and L. C. Sun, “Dual-regularized matrix factorization with deep neural networks for recommender systems,” Knowl.-Based Syst., vol. 145, pp. 46–58, Apr. 2018. doi: 10.1016/j.knosys.2018.01.003
    [46]
    C. Wang, Q. Liu, R. Z. Wu, E. H. Chen, C. R. Liu, X. P. Huang, and Z. Y. Huang, "Confidence-aware matrix factorization for recommender systems, " in Proc. the 32nd AAAI Conf. Artificial Intelligence, New Orleans, USA, 2018, pp. 434–442.
    [47]
    D. Wu, X. Luo, M. S. Shang, Y. He, G. Y. Wang, and M. C. Zhou, “A deep latent factor model for high-dimensional and sparse matrices in recommender systems,” IEEE Trans. Syst., Man, Cybern.: Syst., to be published. DOI: 10.1109/TSMC.2019.2931393
    [48]
    K. Yu, S. H. Zhu, J. Lafferty, and Y. H. Gong, “Fast nonparametric matrix factorization for large-scale collaborative filtering,” in Proc. the 32nd ACM SIGIR Int. Conf. Research and Development in Information Retrieval, Boston, USA, 2009, pp. 211–218.
    [49]
    X. N. He, L. Z. Liao, H. W. Zhang, L. Q. Nie, X. Hu, and T. S. Chua, “Neural collaborative filtering,” in Proc. the 26th Int. Conf. World Wide Web, Perth, Australia, 2017, pp. 173–182.
    [50]
    P. J. Li, Z. H. Wang, Z. C. Ren, L. D. Bing, and W. Lam, “Neural rating regression with abstractive tips generation for recommendation,” in Proc. the 40th Int. ACM SIGIR Conf. Research and Development in Information Retrieval, Shinjuku, Japan, 2017, pp. 345–354.
    [51]
    X. N. He and T. S. Chua, “Neural factorization machines for sparse predictive analytics,” in Proc. the 40th Int. ACM SIGIR Conf. Research and Development in Information Retrieval, Shinjuku, Japan, 2017, pp. 355–364.
    [52]
    J. Xiao, H. Ye, X. N. He, H. W. Zhang, F. Wu, and T. S. Chua, “Attentional factorization machines: Learning the weight of feature interactions via attention networks,” in Proc. the 26th Int. Joint. Conf. on Artificial Intelligence, Melbourne, Australia, 2017, pp. 3119–3125.
    [53]
    L. Zheng, V. Noroozi, and P. S. Yu, “Joint deep modeling of users and items using reviews for recommendation,” in Proc. the 10th ACM Int. Conf. Web Search and Data Mining, Cambridge, UK, 2017, pp. 425–434.
    [54]
    D. Kim, C. Park, J. Oh, S. Lee, and H. Yu, “Convolutional matrix factorization for document context-aware recommendation,” in Proc. the 10th ACM Conf. Recommender Systems, Boston, USA, 2016, pp. 233–240.
    [55]
    N. Y. Zeng, Z. D. Wang, H. Zhang, K. E. Kim, Y. R. Li and X. H. Liu, “An improved particle filter with a novel hybrid proposal distribution for quantitative analysis of gold immunochromatographic strips,” IEEE Trans. Nanotechnol., vol. 18, pp. 819–829, Aug. 2019. doi: 10.1109/TNANO.2019.2932271
    [56]
    N. Y. Zeng, Z. D. Wang, B. Zineddin, Y. R. Li, M. Du, L. Xiao, X. H. Liu, and T. Young, “Image-based quantitative analysis of gold immunochromatographic strip via cellular neural network approach,” IEEE Trans. Med. Imag., vol. 33, no. 5, pp. 1129–1136, May 2014. doi: 10.1109/TMI.2014.2305394
    [57]
    N. Y. Zeng, H. Li, Z. D. Wang, W. B. Liu, S. M. Liu, F. E. Alsaadi, and X. H. Liu, “Deep-reinforcement-learning-based images segmentation for quantitative analysis of gold immunochromatographic strip,” Neurocomputing, to be published. DOI: 10.1016/j.neucom.2020.04.001
    [58]
    Z. Q. Shu, X. J. Wu, C. Z. You, Z. Liu, P. Li, H. H. Fan, and F. Y. Ye, “Rank-constrained nonnegative matrix factorization for data representation,” Inform. Sci., vol. 528, pp. 133–146, Aug. 2020. doi: 10.1016/j.ins.2020.04.017
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(8)  / Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1026) PDF downloads(75) Cited by()

    Highlights

    • Proposing an SL-LF model with high robustness and accuracy in predicting the missing data of an HiDS matrix.
    • Performing a suite of theoretical analyses and algorithm designs for the proposed SL-LF model.
    • Conducting extensive empirical studies on eight HiDS matrices generated by industrial applications to evaluate the proposed model and other state-of-the-art ones.

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return