Graph Regularized <inline-formula><tex-math id="M1">\begin{document}$L_p$\end{document}</tex-math></inline-formula> Smooth Non-negative Matrix Factorization for Data Representation

Chengcai Leng; Hai Zhang; Guorong Cai; Irene Cheng; Anup Basu

doi:10.1109/JAS.2019.1911417

Volume 6 Issue 2

Mar. 2019

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2019 > 6(2): 584-595

Chengcai Leng, Hai Zhang, Guorong Cai, Irene Cheng and Anup Basu, "Graph Regularized \begin{document}$L_p$\end{document} Smooth Non-negative Matrix Factorization for Data Representation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 584-595, Mar. 2019. doi: 10.1109/JAS.2019.1911417

Citation:

Chengcai Leng, Hai Zhang, Guorong Cai, Irene Cheng and Anup Basu, "Graph Regularized

\begin{document}$L_p$\end{document}

Smooth Non-negative Matrix Factorization for Data Representation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 584-595, Mar. 2019. doi: 10.1109/JAS.2019.1911417

Citation:

Chengcai Leng, Hai Zhang, Guorong Cai, Irene Cheng and Anup Basu, "Graph Regularized

\begin{document}$L_p$\end{document}

Smooth Non-negative Matrix Factorization for Data Representation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 584-595, Mar. 2019. doi: 10.1109/JAS.2019.1911417

PDF( 2199 KB)

Graph Regularized $L_p$ Smooth Non-negative Matrix Factorization for Data Representation

doi: 10.1109/JAS.2019.1911417

1.
School of Mathematics, Northwest University, Xioan 710127, School of Mathematics and Information Sciences, Nanchang Hangkong University, Nanchang 330063, China, and the Department of Computing Science, University of Alberta, Edmonton, AB T6G 2E8, Canada
2.
School of Mathematics, Northwest University, Xioan 710127, and the Faculty of Information Technology & State Key Laboratory of Quality Research in Chinese Medicines, Macau University of Science and Technology, Macau, China
3.
College of Computer Engineering, Jimei University, Xiamen 361021, China
4.
Department of Computing Science, University of Alberta, Edmonton, AB T6G 2E8, Canada

Funds:

the National Natural Science Foundation of China 61702251

the National Natural Science Foundation of China 61363049

the National Natural Science Foundation of China 11571011

the State Scholarship Fund of China Scholarship Council (CSC) 201708360040

the Natural Science Foundation of Jiangxi Province 20161BAB212033

the Natural Science Basic Research Plan in Shaanxi Province of China 2018JM6030

the Doctor Scientific Research Starting Foundation of Northwest University 338050050

More Information

Author Bio:
Chengcai Leng received the Ph.D. degree in applied mathematics from Northwestern Polytechnical University, Xi'an, China, in 2012. From 2010 to 2011 and 2017 to 2018, he was a Visiting Student and Visiting Scholar with the Department of Computing Science, University of Alberta, Edmonton, Canada, respectively. He served as a Postdoctoral in the Institute of Automation, Chinese Academy of Sciences, Beijing, China from 2013 to 2016. He is currently an Associate Professor with the School of Mathematics, Northwest University, Xi'an, China. His current research interests include image processing, statistical analysis, computer vision, and optical molecular imaging.(e-mail: ccleng@nwu.edu.cn)

Hai Zhang received the Ph.D. degree in applied mathematics from Xi'an Jiaotong University, Xi'an, in 2012. He was a Visiting Scholar with the Department of Statistics, University of California, Berkeley, America from 2011 to 2012. He is currently a Professor and Chair in the Department of Financial Mathematics and Statistics at Northwest University. His research interests include statistical machine learning, high-dimensional statistics, and social network analysis.(e-mail: zhanghai@nwu.edu.cn)

Guorong Cai received the Ph.D. degree in Computer Science from Xiamen University, Fujian, China, in 2013. Currently, he is an Associate Professor with Computer Engineering College, Jimei University, Xiamen, China. His research interests include 3D reconstruction, machine learning, object detection/recognition, and image/video retrieval. (e-mail: guorongcai.jmu@gmail.com)

Irene Cheng received the Ph.D. degree from the University of Alberta, Edmonton, Canada. She is the Scientific Director of the Multimedia Research Centre, the Director of the Master with a Specialization in Multimedia Program, and an Adjunct Professor with the University of Alberta, Canada. She has been active in many professional organizations, which include General Chair (IEEE ICME 2011), Technical Program Chair (ICME 2013) and Technical Program Chair (SMC 2017). She is also an Associate Editor of the IEEE Transactions on Human-Machine Systems. Her research interests include human intelligence and perception in multimedia computing. (e-mail: locheng@ualberta.ca)

Anup Basu received the Ph.D. degree in computer science from the University of Maryland, College Park. He was a Visiting Professor with the University of California, Riverside, a Guest Professor with the Technical University of Austria, Graz, and the Director with the Hewlett-Packard Imaging Systems Instructional Laboratory, University of Alberta, Edmonton, Canada, since 1999, where he has been a Professor with the Department of Computing Science, and is currently an iCORE-NSERC Industry Research Chair. His current research interests include 3D/4D image processing and visualization especially for medical applications, multimedia in education and games, and wireless 3D multimedia transmission.(e-mail: basu@ualberta.ca)
Corresponding author: Chengcai Leng, e-mail: ccleng@nwu.edu.cn
Received Date: 2018-08-03
Revised Date: 2018-10-11
Accepted Date: 2018-11-28

Abstract

Abstract

This paper proposes a Graph regularized $L_p$ smooth non-negative matrix factorization (GSNMF) method by incorporating graph regularization and $L_p$ smoothing constraint, which considers the intrinsic geometric information of a data set and produces smooth and stable solutions. The main contributions are as follows: first, graph regularization is added into NMF to discover the hidden semantics and simultaneously respect the intrinsic geometric structure information of a data set. Second, the $L_p$ smoothing constraint is incorporated into NMF to combine the merits of isotropic ($L_{2}$-norm) and anisotropic ($L_{1}$-norm) diffusion smoothing, and produces a smooth and more accurate solution to the optimization problem. Finally, the update rules and proof of convergence of GSNMF are given. Experiments on several data sets show that the proposed method outperforms related state-of-the-art methods.
- Data clustering,
- dimensionality reduction,
- graph regularization,
- $L_p$ smooth non-negative matrix factorization (SNMF)

FullText(HTML)

References(40)

References

[1]	G. F. Lu, Y. Wang, and J. Zou, "Low-rank matrix factorization with adaptive graph regularizer, " IEEE Transactions on Image Processing, vol. 25, no. 5, pp. 2205-2196, 2016. http://dl.acm.org/citation.cfm?id=2930749
[2]	Z. Y. Zhang, and K. K. Zhao, "Low-rank matrix approximation with manifold regularization, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1717-1729, 2013. doi: 10.1109/TPAMI.2012.274
[3]	X. Luo, M. C. Zhou, Y. N. Xia, and Q. S. Zhu, "An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems, " IEEE Transactions on Industrial Informatics, vol. 10, no. 2, pp. 1273-1284, 2014. doi: 10.1109/TII.2014.2308433
[4]	F. H. Shang, L. C. Jiao, F. Wang, "Graph dual regularization non-negative matrix factorization for co-clustering, " Pattern Recognition, vol. 45, no. 6, pp. 2237-2250, 2012. doi: 10.1016/j.patcog.2011.12.015
[5]	D. Wang, X. B. Gao, X. M. Wang, "Semi-supervised nonnegative matrix factorization via constraint propagation, " IEEE Transactions on Cybernetics, vol. 46, no. 1, pp. 233-244, 2016. doi: 10.1109/TCYB.2015.2399533
[6]	L. He, N. Ray, Y. S. Guan, and H. Zhang, "Fast large-scale spectral clustering via explicit feature mapping, " IEEE Transactions on Cybernetics, DOI: 10.1109/TCYB.2018.2794998, 2018.
[7]	X. Q. Lu, H. Wu, Y. Yuan, P. K. Yan, and X. L. Li, "Manifold Regularized Sparse NMF for Hyperspectral Unmixing, " IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 5, pp. 2815-2826, 2013. doi: 10.1109/TGRS.2012.2213825
[8]	W. He, H. Y. Zhang, and L. P. Zhang, "Sparsity-regularized robust non-negative matrix factorization for hyperspectral unmixing, " IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 9, no. 9, pp. 4267-4279, 2016. doi: 10.1109/JSTARS.2016.2519498
[9]	Y. Ma, C. Li, X. G. Mei, C. Y. Liu, J. Y. Ma, "Robust sparse hyperspectral unmixing with l2, 1 norm, " IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 3, pp. 1227-1239, 2017. doi: 10.1109/TGRS.2016.2616161
[10]	F. Fan, Y. Ma, C. Li, X. G. Mei, J. Huang, J. Y. Ma, "Hyperspectral image denoising with superpixel segmentation and low-rank representation, " Information Sciences, vol. 397, pp. 48-68, 2017. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=33cc750f2d1d88ae9316d4e4569a86bb
[11]	Z. C. Li, J. Liu, and H. Q. Lu, "Structure preserving non-negative matrix factorization for dimensionality reduction, " Computer Vision and Image Understanding, vol. 117, no. 9, pp. 1175-1189, 2013. doi: 10.1016/j.cviu.2013.04.003
[12]	Y. W. Lu, C. Yuan, W. W. Zhu, and X. L. Li, "Structurally incoherent low-rank nonnegative matrix factorization for image classification, " IEEE Transactions on Image Processing, vol. 27, no. 11, pp. 5248-5260, 2018. doi: 10.1109/TIP.2018.2855433
[13]	X. Luo, M. C. Zhou, H. Leung, Y. N. Xia, Q. S. Zhu, Z. H. You, and S. Li, "An incremental-and-static-combined scheme for matrix-factorization- based collaborative filtering, " IEEE Transactions on Automation Science and Engineering, vol. 13, no. 1, pp. 333-343, 2016. doi: 10.1109/TASE.2014.2348555
[14]	Y. X. Wang, and Y. J. Zhang, "Nonnegative matrix factorization: a comprehensive review, " IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 6, pp. 1336-1353, 2013. doi: 10.1109/TKDE.2012.51
[15]	W. J. Hu, K. S. Choi, P. L. Wang, Y. L. Jiang, and S. T. Wang, "Convex nonnegative matrix factorization with manifold regularization, " Neural Networks, vol. 63, no. 1, pp. 94-103, 2015. http://www.sciencedirect.com/science/article/pii/S0893608014002615
[16]	J. Tenenbaum, V. de Silva, and J. Langford, "A global geometric framework for nonlinear dimensionality reduction, " Science, vol. 290, no. 5500, pp. 2319-2323, 2000. doi: 10.1126/science.290.5500.2319
[17]	S. Roweis, and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding, " Science, vol. 290, no. 5500, pp. 2323-2326, 2000. doi: 10.1126/science.290.5500.2323
[18]	M. Belkin, and P. Niyogi, "Laplacian eigenmaps and spectral techniques for embedding and clustering, " in Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 2001, vol. 14, pp. 585-591. http://www.numdam.org/item/M2AN_2009__43_4_721_0/
[19]	D. Cai, X. F. He, and J. W. Han, "Isometric projection", in Proceeding of the National Conference on Artificial Intelligence, Vancouver, BC, Canada, 2007, vol. 1, pp. 528-533.
[20]	A. Najafi, A. Joudaki, and E. Fatemizadeh, "Nonlinear dimensionality reduction via path-based isometric mapping, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 7, pp. 1452-1464, 2016. doi: 10.1109/TPAMI.2015.2487981
[21]	D. D. Lee, and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization, " Nature, vol. 401, no. 6755, pp. 788-791, 1999. doi: 10.1038/44565
[22]	L. Liu, A. L. Yang, W. J. Zhou, X. F. Zhang, M. R. Fei, and X. W. Tu, "Robust dataset classification approach based on neighbor searching and kernel fuzzy c-means, " IEEE/CAA Journal of Automatica Sinica, vol. 2, no. 3, pp. 235-247, 2015. doi: 10.1109/JAS.2015.7152657
[23]	B. Jiang, H. F. Zhao, J. Tang, and B. Luo, "A sparse nonnegative matrix factorization technique for graph matching problems, " Pattern Recognition, vol. 47, no. 2, pp. 736-747, 2014. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=22fdc3d3ed3cfae2f48671328cd56a63
[24]	Y. L. Tian, X. Li, K. F. Wang, and F.-Y. Wang, "Training and testing object detectors with virtual images, " IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 2, pp. 539-546, 2018. doi: 10.1109/JAS.2017.7510841
[25]	J. Q. Gu, H. F. Hu, and H. X. Li, "Local robust sparse representation for face recognition with single sample per person, " IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 2, pp. 547-554, 2018. doi: 10.1109/JAS.2017.7510658
[26]	W. He, H. Y. Zhang, L. P. Zhang, "Total variation regularized reweighted sparse nonnegative matrix factorization for hyperspectral unmixing, " IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 7, pp. 3909-3921, 2017. doi: 10.1109/TGRS.2017.2683719
[27]	C. C. Leng, G. R. Cai, D. D. Yu, Z. Y. Wang, "Adaptive total-variation for non-negative matrix factorization on manifold, " Pattern Recognition Letters, vol. 98, pp. 68-74, 2017. doi: 10.1016/j.patrec.2017.08.027
[28]	H. F. Liu, Z. H. Wu, X. L. Li, D. Cai, and T. S. Huang, "Constrained nonnegative matrix factorization for image representation, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1299-1311, 2012. doi: 10.1109/TPAMI.2011.217
[29]	E. Hosseini-Asl, J. M. Zurada, G. Gimelfarb, A. El-Baz, "3-D lung segmentation by incremental constrained nonnegative matrix factorization, " IEEE Transactions on Biomedical Engineering, vol. 63, no. 5, pp. 952-963, 2016. doi: 10.1109/TBME.2015.2482387
[30]	D. Cai, X. F. He, J. W. Han, and T. S. Huang, "Graph regularized nonnegative matrix factorization for data representation, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1548-1560, 2011. doi: 10.1109/TPAMI.2010.231
[31]	R. H. Shang, W. B. Wang, R. Stolkin, and L. C. Jiao, "Non-negative spectral learning and sparse regression-based dual-graph regularized feature selection, " IEEE Transactions on Cybernetics, vol. 48, no. 2, pp. 793-806, 2018. doi: 10.1109/TCYB.2017.2657007
[32]	X. Luo, M. C. Zhou, M. S. Shang, S. Li, and Y. N. Xia, "A novel approach to extracting non-negative latent factors from non-negative big sparse matrices, " IEEE Access, vol. 4, pp. 2649-2655, 2016. http://ieeexplore.ieee.org/document/7457202/
[33]	X. Luo, J. P. Sun, Z. D. Wang, S. Li, and M. S. Shang "Symmetric and nonnegative latent factor models for undirected, high-dimensional, and sparse networks in industrial applications, " IEEE Transactions on Industrial Informatics, vol. 13, no. 6, pp. 3098-3107, 2017. doi: 10.1109/TII.2017.2724769
[34]	X. Luo, M. C. Zhou, Y. N. Xia, Q. S. Zhu, A. C. Ammari, and A. Alabdulwahab, "Generating highly accurate predictions for missing QoS data via aggregating nonnegative latent factor models, " IEEE Transactions on Neural Networks and Learning Systems, vol. 27, no. 3, pp. 524-537, 2016. doi: 10.1109/TNNLS.2015.2412037
[35]	X. Luo, M. C. Zhou, S. Li, Y. N. Xia, Z. H. You, Q. S. Zhu, and H. Leung, "Incorporation of efficient second-order solvers into latent factor models for accurate prediction of missing QoS Data, " IEEE Transactions on Cybernetics, vol. 48, no. 4, pp. 1216-1228, 2018. doi: 10.1109/TCYB.2017.2685521
[36]	C. C. Leng, H. Zhang, and G. R. Cai, "A novel data clustering method based on smooth non-negative matrix factorization, " in Proceedings of the International Conference on Smart Multimedia, Toulon, France, 2018, pp. 406-414.
[37]	D. D. Lee, H. S. Seung, "Algorithms for non-negative matrix factorization, " Advances in Neural Information Processing Systems, 2000, vol. 13, pp. 556-562. http://d.old.wanfangdata.com.cn/OAPaper/oai_arXiv.org_1210.1190
[38]	F. R. K. Chung, Spectral Graph Theory, Providence RI: American Mathematical Society, 1997.
[39]	W. Xu, X. Liu, and Y. H. Gong, "Document clustering based on non-negative matrix factorization, " in Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2003, pp. 267-273.
[40]	D. Cai, X. F. He, and J. W. Han, "Document clustering using locality preserving indexing, " IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 12, pp. 1624-1637, 2005. doi: 10.1109/TKDE.2005.198

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(6) / Tables(5)

Get Citation

PDF

XML

Article Metrics

Article views (1146) PDF downloads(61)

Graph Regularized $L_p$ Smooth Non-negative Matrix Factorization for Data Representation

doi: 10.1109/JAS.2019.1911417

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content