IEEE/CAA Journal of Automatica Sinica
Citation: | Xin Luo, Wen Qin, Ani Dong, Khaled Sedraoui and MengChu Zhou, "Efficient and High-quality Recommendations via Momentum-incorporated Parallel Stochastic Gradient Descent-Based Learning," IEEE/CAA J. Autom. Sinica, vol. 8, no. 2, pp. 402-411, Feb. 2021. doi: 10.1109/JAS.2020.1003396 |
[1] |
J. Bobadilla, F. Ortega, A. Hernando, and J. Bernal, “Generalization of recommender systems: Collaborative filtering extended to groups of users and restricted to groups of items,” Expert Systems with Applications, vol. 39, no. 1, pp. 172–186, Jan. 2012. doi: 10.1016/j.eswa.2011.07.005
|
[2] |
J. Bobadilla, F. Serradilla, and J. Bernal, “A new collaborative filtering metric that improves the behavior of recommender systems,” Knowledge-Based Systems, vol. 23, no. 6, pp. 520–528, Aug. 2010. doi: 10.1016/j.knosys.2010.03.009
|
[3] |
X. Luo, M. C. Zhou, Y. N. Xia, Q. S. Zhu, A. C. Ammari, and A. Alabdulwahab, “Generating highly accurate predictions for missing QoS data via aggregating nonnegative latent factor models,” IEEE Trans. Neural Networks and Learning Systems, vol. 27, no. 3, pp. 524–537, 2016. doi: 10.1109/TNNLS.2015.2412037
|
[4] |
Z. B. Zheng, H. Ma, M. R. Lyu, and I. King, “QoS-aware web service recommendation by collaborative filtering,” IEEE Trans. Services Computing, vol. 4, no. 2, pp. 140–152, 2011. doi: 10.1109/TSC.2010.52
|
[5] |
Y. Shi, M. Larson, and A. Hanjalic, “Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges,” ACM Computing Surveys, vol. 47, no. 1, pp. 45, Jul. 2014.
|
[6] |
L. Yang, X. C. Cao, D. Jin, X. Wang, and D. Meng, “A unified semi-supervised community detection framework using latent space graph regularization,” IEEE Trans. Cybernetics, vol. 45, no. 11, pp. 2585–2598, 2015. doi: 10.1109/TCYB.2014.2377154
|
[7] |
M. S. Shang, X. Luo, Z. G. Liu, J. Chen, Y. Yuan, and M. C. Zhou, “Randomized latent factor model for high-dimensional and sparse matrices from industrial applications,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 131–141, Jan. 2019. doi: 10.1109/JAS.2018.7511189
|
[8] |
T. T. He, L. Hu, K. C. C. Chan, and P. W. Hu, “Learning latent factors for community identification and summarization,” IEEE Access, vol. 6, pp. 30137–30148, 2018. doi: 10.1109/ACCESS.2018.2843726
|
[9] |
Z. Ghahramani, “Probabilistic machine learning and artificial intelligence,” Nature, vol. 521, no. 7553, pp. 452–459, 2015. doi: 10.1038/nature14541
|
[10] |
N. Zerari, M. Chemachema, and N. Essounbouli, “Neural network based adaptive tracking control for a class of pure feedback nonlinear systems with input saturation,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 278–290, Jan. 2019. doi: 10.1109/JAS.2018.7511255
|
[11] |
X. Luo, M. C. Zhou, S. Li, Z. H. You, Y. N. Xia, Q. S. Zhu, and H. Leung, “An Efficient second-order approach to factorizing sparse matrices in recommender systems,” IEEE Trans. Industrial Informatics, vol. 11, no. 4, pp. 946–956, Aug. 2015. doi: 10.1109/TII.2015.2443723
|
[12] |
X. Luo, M. C. Zhou, H. Leung, Y. N. Xia, Q. S. Zhu, Z. H. You, and S. Li, “An incremental-and-static-combined scheme for matrix-factorization-based collaborative filtering,” IEEE Trans. Autom. Science and Engineering, vol. 13, no. 1, pp. 333–343, Jan. 2016. doi: 10.1109/TASE.2014.2348555
|
[13] |
M. F. Weng and Y. Y. Chuang, “Collaborative video reindexing via matrix factorization,” ACM Trans. Multimedia Computing Communications and Applications. Appl., vol. 8, no. 2, pp. 1–20, 2012.
|
[14] |
R. Salakhutdinov and A. Mnih, “Probabilistic matrix factorization,” Advances in Neural Information Processing Systems, vol. 20, pp. 1257–1264, 2008.
|
[15] |
X. Luo, H. Wu, M. C. Zhou, and H. Q Yuan, “Temporal pattern-aware QoS prediction via biased non-negative latent factorization of tensors,” IEEE Trans. Cybernetics, vol. 50, no. 5, pp. 1798–18.9, Mar. 2020. doi: 10.1109/TCYB.2019.2903736
|
[16] |
J. Sha, Y. Y. Du, and L. Qi, “A user requirement oriented web service discovery approach based on logic and threshold petri net,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 6, pp. 1528–1542, Nov. 2019. doi: 10.1109/JAS.2019.1911657
|
[17] |
A. Mahmood and M. Small, “Subspace based network community detection using sparse linear coding,” IEEE Trans. Knowledge and Data Engineering, vol. 28, no. 3, pp. 801–812, Mar. 2016. doi: 10.1109/TKDE.2015.2496345
|
[18] |
A. Almalaq, J. Hao, J. J. Zhang, and F. Y. Wang, “Parallel building: A complex system approach for smart building energy management,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 6, pp. 1452–1461, Nov. 2019.
|
[19] |
J. J. Pan, S. J. Pan, Y. Jie, L. M. Ni, and Y. Qiang, “Tracking mobile users in wireless networks via semi-supervised colocalization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, pp. 587–600, 2012. doi: 10.1109/TPAMI.2011.165
|
[20] |
G. Takács, I. Pilászy, Bottyán Németh, and D. Tikky, “Scalable collaborative filtering approaches for large recommender systems,” J. Machine Learning Research, vol. 10, pp. 623–656, Mar. 2009.
|
[21] |
Y. Koren, R. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” IEEE Computer., vol. 42, pp. 30–37, Aug. 2009.
|
[22] |
W. W. Wang, A. Cichocki, and J. A. Chambers, “A multiplicative algorithm for convolutive non-negative matrix factorization based on squared euclidean distance,” IEEE Trans. Signal Processing, vol. 57, no. 7, pp. 2858–2864, 2009. doi: 10.1109/TSP.2009.2016881
|
[23] |
H. Kim and H. Park, “Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis,” Bioinformatics, vol. 23, no. 12, pp. 1495–1502, 2007. doi: 10.1093/bioinformatics/btm134
|
[24] |
Z. Li, J. Tang, and T. Mei, “Deep collaborative embedding for social image understanding,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 41, no. 9, pp. 2070–2083, Sep. 2019. doi: 10.1109/TPAMI.2018.2852750
|
[25] |
C. J. Lin, “Projected gradient methods for nonnegative matrix factorization,” Neural Computation, vol. 19, no. 10, pp. 2756–2779, 2007. doi: 10.1162/neco.2007.19.10.2756
|
[26] |
Y. Chen, J. Bordes, and D. Filliat, “Comparison studies on active cross-situational object-word learning using non-negative matrix factorization and latent dirichlet allocation,” IEEE Trans. Cognitive and Developmental Systems, vol. 10, no. 4, pp. 1023–1034, Dec. 2018. doi: 10.1109/TCDS.2017.2725304
|
[27] |
M. Shiga and H. Mamitsuka, “Non-negative matrix factorization with auxiliary information on overlapping groups,” IEEE Trans. Knowledge and Data Engineering, vol. 27, no. 6, pp. 1615–1628, Jun. 2015. doi: 10.1109/TKDE.2014.2373361
|
[28] |
C. C. Leng, H. Zhang, G. R. Cai, I. Cheng, and A. Basu, “Graph regularized Lp smooth non-negative matrix factorization for data representation,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 584–595, Mar. 2019. doi: 10.1109/JAS.2019.1911417
|
[29] |
F. Niu, B. Recht, C. Ré, and S. J. Wright, “Hogwild!: A lock-free approach to parallelizing stochastic gradient descent,” Advances in Neural Information Processing Systems, pp. 693–701, 2011.
|
[30] |
R. Gemulla, E. Nijkamp, P. J. Haas, and Y. Sismaanis, “Large-scale matrix factorization with distributed stochastic gradient descent,” in Proc. 17th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Diego, California, USA, Aug, 2011, pp: 69−77.
|
[31] |
Y. Nesterov, “A method for unconstrained convex minimization problem with the rate of convergence,” Soviet Mathematics Doklady, vol. 269, pp. 543–547, 1983.
|
[32] |
H. Ma, D. Zhou, C. Liu, M. R. Lyu, and I. King, “Recommender systems with social regularization.” in Pro. 4th ACM Int. Conf. Web Search and Data Mining, pp: 287−296, 2014.
|
[33] |
T. Greenberg-Toledo, R. Mazor, A. Haj-Ali, and S. Kvatinsky, “Supporting the momentum training algorithm using a memristor-based synapse,” IEEE Trans. Circuits and Systems I:Regular Papers, vol. 66, no. 4, pp. 1571–1583, Apr. 2019. doi: 10.1109/TCSI.2018.2888538
|
[34] |
N. Qian, “On the momentum term in gradient descent learning algorithms,” Neural Networks:the Official J. Int. Neural Network Society, vol. 12, no. 1, pp. 145–151, 1999. doi: 10.1016/S0893-6080(98)00116-6
|
[35] |
C. K. Krzysztof, “Convergence of approximate and incremental subgradient methods for convex optimization,” SIAM J. Optimization, vol. 14, no. 3, pp. 807–840, 2004. doi: 10.1137/S1052623400376366
|
[36] |
H. Robbins and D. Siegmund, “A convergence theorem for non-negative almost super martingales and some applications,” Optimizing Methods in Statistics, pp. 233–257, 1971.
|
[37] |
J. A. Konstan, B. N. Miller, D. Maltz, J. L. Herlocker, L. R. Gordon, and J. Riedl, “Grouplens: applying collaborative filtering to UseNet news,” Communications of the ACM, vol. 40, no. 3, pp. 77–87, 1997. doi: 10.1145/245108.245126
|
[38] |
W. S. Chin, Y. Zhuang, Y. C. Juan, and C. J. Lin, “A fast parallel stochastic gradient method for matrix factorization in shared memory systems,” ACM Trans. Intelligent Systems and Technology, vol. 6, no. 1, pp. 24, Apr. 2015.
|
[39] |
G. Bhatnagar and Q. M. J. Wu, “A fractal dimension based framework for night vision fusion,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 220–227, Jan. 2019. doi: 10.1109/JAS.2018.7511102
|
[40] |
X. Luo, H. J. Liu, G. P. Gou, Y. N. Xia, and Q. S. Zhu, “A parallel matrix factorization based recommender by alternating stochastic gradient decent,” Engineering Applications of Artificial Intelligence, vol. 25, no. 7, pp. 1403–1412, Oct. 2012. doi: 10.1016/j.engappai.2011.10.011
|
[41] |
S. Sedhain, A. K. Menon, S. Sanner, and L. Xie, “AutoRec: autoencoders meet collaborative filtering,” in Proc. 24th Int. Conf. World Wide Web, 2015, pp: 111−112.
|
[42] |
Y. Guo, M. Wang, and X. Li, “Application of an improved apriori algorithm in a mobile e-commerce recommendation system,” Industrial Management and Data Systems, vol. 117, no. 2, pp. 287–303, 2017. doi: 10.1108/IMDS-03-2016-0094
|
[43] |
Q. Liu, S. Wu, D. Wang, Z. Li, and L. Wang, “Context-aware sequential recommendation,” in Proc. 16th IEEE Int. Conf. Data Mining, pp: 1053−1058, 2016.
|
[44] |
S. C. Gao, M. C. Zhou, Y. R. Wang, J. J. Cheng, H. Yachi, and J. H. Wang, “Dendritic neuron model with effective learning algorithms for classification, approximation and prediction,” IEEE Trans Neural Networks and Learning Systems, vol. 30, no. 2, pp. 601–614, Feb. 2019. doi: 10.1109/TNNLS.2018.2846646
|
[45] |
P. Y. Zhang and M. C. Zhou, “Dynamic Cloud Task Scheduling Based on a Two-stage Strategy,” IEEE Trans. Autom. Science and Engineering, vol. 15, no. 2, pp. 772–783, Apr. 2018. doi: 10.1109/TASE.2017.2693688
|
[46] |
P. Y. Zhang, M. C. Zhou, and G. Fortino, “Security and trust issues in fog computing: A survey,” Future Generation Computer Systems, vol. 88, pp. 16–27, Nov. 2018. doi: 10.1016/j.future.2018.05.008
|
[47] |
H. Duan, C. Liu, Q. Zeng and M. C. Zhou, “Refinement-Based Hierarchical Modeling and Correctness Verification of Cross-Organization Collaborative Emergency Response Processes,” IEEE Trans. Systems, Man, and Cybernetics: Systems, vol. 50, no. 8, pp. 2845-2859, Aug. 2020.
|
[48] |
X. Guo, M. C. Zhou, S. Liu, and L. Qi, “Lexicographic Multiobjective Scatter Search for the Optimization of Sequence-Dependent Selective Disassembly Subject to Multiresource Constraints,” IEEE Trans. Cybernetics, 50(7), pp. 3307-3317, Jul. 2020.
|
[49] |
Z. Huang, X. Xu, H. Zhu and M. C. Zhou, “An Efficient Group Recommendation Model with Multiattention-Based Neural Networks,” IEEE Trans Neural Networks and Learning Systems, vol. 31, no. 11, pp. 4461-4474, Nov. 2020.
|
[50] |
X. Luo, M. C. Zhou, S. Li, L. Hu, and M. Shang, “Non-Negativity Constrained Missing Data Estimation for High-Dimensional and Sparse Matrices from Industrial Applications,” IEEE Trans. Cybernetics, vol. 50, no. 5, pp. 1844-1855, May 2020.
|