IEEE/CAA Journal of Automatica Sinica
Citation:  C. Gong and Y. You, “Sparse reconstructive evidential clustering for multiview data,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 2, pp. 459–473, Feb. 2024. doi: 10.1109/JAS.2023.123579 
Although many multiview clustering (MVC) algorithms with acceptable performances have been presented, to the best of our knowledge, nearly all of them need to be fed with the correct number of clusters. In addition, these existing algorithms create only the hard and fuzzy partitions for multiview objects, which are often located in highlyoverlapping areas of multiview feature space. The adoption of hard and fuzzy partition ignores the ambiguity and uncertainty in the assignment of objects, likely leading to performance degradation. To address these issues, we propose a novel sparse reconstructive multiview evidential clustering algorithm (SRMVEC). Based on a sparse reconstructive procedure, SRMVEC learns a shared affinity matrix across views, and maps multiview objects to a 2dimensional humanreadable chart by calculating 2 newly defined mathematical metrics for each object. From this chart, users can detect the number of clusters and select several objects existing in the dataset as cluster centers. Then, SRMVEC derives a credal partition under the framework of evidence theory, improving the fault tolerance of clustering. Ablation studies show the benefits of adopting the sparse reconstructive procedure and evidence theory. Besides, SRMVEC delivers effectiveness on benchmark datasets by outperforming some stateoftheart methods.
[1] 
S. Wang, X. Lin, Z. Fang, S. Du, and G. Xiao, “Contrastive consensus graph learning for multiview clustering,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 11, pp. 2027–2030, 2022. doi: 10.1109/JAS.2022.105959

[2] 
C. Gong, Z. Su, P. Wang, and Q. Wang, “Cumulative belief peaks evidential knearest neighbor clustering,” Knowl. Based Syst., vol. 200, p. 105982, 2020. doi: 10.1016/j.knosys.2020.105982

[3] 
C. Gong, Z. Su, P. Wang, and Q. Wang, “An evidential clustering algorithm by finding beliefpeaks and disjoint neighborhoods,” Pattern Recognit., vol. 113, p. 107751, 2021. doi: 10.1016/j.patcog.2020.107751

[4] 
G. Chao, S. Sun, and J. Bi, “A survey on multiview clustering,” IEEE Trans. Artif. Intell., vol. 2, no. 2, pp. 146–168, 2021. doi: 10.1109/TAI.2021.3065894

[5] 
M. Worchel, R. Diaz, W. Hu, O. Schreer, I. Feldmann, and P. Eisert, “Multiview mesh reconstruction with neural deferred shading,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, New Orleans, USA, June 2022, pp. 6187–6197.

[6] 
S. Wang, X. Liu, L. Liu, W. Tu, X. Zhu, J. Liu, S. Zhou, and E. Zhu, “Highlyefficient incomplete largescale multiview clustering with consensus bipartite graph,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, New Orleans, USA, 2022, pp. 9776–9785.

[7] 
V. Mekthanavanh, T. Li, H. Meng, Y. Yang, and J. Hu, “Social web video clustering based on multiview clustering via nonnegative matrix factorization,” Int. J. Mach. Learn. Cybern., vol. 10, no. 10, pp. 2779–2790, 2019. doi: 10.1007/s13042018009025

[8] 
S. Bickel and T. Scheffer, “Multiview clustering,” in Proc. SIAM Int. Conf. Data Min., vol. 4, no. 2004. Washington, USA: Citeseer, 2004, pp. 19–26.

[9] 
G. F. Tzortzis and A. C. Likas, “Multiple view clustering using a weighted combination of exemplarbased mixture models,” IEEE Trans. Neural Netw., vol. 21, no. 12, pp. 1925–1938, 2010. doi: 10.1109/TNN.2010.2081999

[10] 
H. Wu, S. Huang, C. Tang, Y. Zhang, and J. Lv, “Pure graphguided multiview subspace clustering,” Pattern Recognit., vol. 136, p. 109187, 2023. doi: 10.1016/j.patcog.2022.109187

[11] 
S. Wang, Z. Chen, S. Du, and Z. Lin, “Learning deep sparse regularizers with applications to multiview clustering and semisupervised classification,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 9, pp. 5042–5055, 2021.

[12] 
G. Zhong and C.M. Pun, “Selftaught multiview spectral clustering,” Pattern Recognit., p. 109349, 2023.

[13] 
S. Shi, F. Nie, R. Wang, and X. Li, “Selfweighting multiview spectral clustering based on nuclear norm,” Pattern Recognit., vol. 124, p. 108429, 2022. doi: 10.1016/j.patcog.2021.108429

[14] 
Z. Lin, Z. Kang, L. Zhang, and L. Tian, “Multiview attributed graph clustering,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 2, pp. 1872–1880, 2023.

[15] 
L. Li and H. He, “Bipartite graph based multiview clustering,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 7, pp. 3111–3125, 2022.

[16] 
C. Zhang, Y. Liu, and H. Fu, “AE^{2}Nets: Autoencoder in autoencoder networks,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp. 2577–2585.

[17] 
M. Brbić and I. Kopriva, “Multiview lowrank sparse subspace clustering,” Pattern Recognit., vol. 73, pp. 247–258, 2018. doi: 10.1016/j.patcog.2017.08.024

[18] 
K. Zhou, M. Guo, and M. Jiang, “Evidential weighted multiview clustering,” in Proc. Int. Conf. Belief Func. Shanghai, China: Springer, 2021, pp. 22–32.

[19] 
J. Xu, Y. Ren, H. Tang, Z. Yang, L. Pan, Y. Yang, X. Pu, S. Y. Philip, and L. He, “Selfsupervised discriminative feature learning for deep multiview clustering,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 7, pp. 7470–7482, 2023.

[20] 
H. Zhao, Z. Ding, and Y. Fu, “Multiview clustering via deep matrix factorization,” in Proc. AAAI Conf. Artificial Intelligence, vol. 31, no. 1, San Francisco, USA, 2017.

[21] 
Y. Tian, Y. Feng, X. Zhang, and C. Sun, “A fast clustering based evolutionary algorithm for superlargescale sparse multiobjective optimization,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 4, pp. 1048–1063, 2023.

[22] 
D. Cai, X. He, and J. Han, “Using graph model for face analysis,” Tech. Rep., 2005.

[23] 
N. X. Vinh, J. Epps, and J. Bailey, “Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance,” J. Mach. Learn Res., vol. 11, pp. 2837–2854, 2010.

[24] 
W. J. Krzanowski and Y. Lai, “A criterion for determining the number of groups in a data set using sumofsquares clustering,” Biometrics, pp. 23–34, 1988.

[25] 
M. Syakur, B. Khotimah, E. Rochman, and B. D. Satoto, “Integration kmeans clustering method and elbow method for identification of the best customer profile cluster,” in Proc. IOP Conf. Ser.: Mater. Sci. Eng., vol. 336, no. 1, 2018, p. 012017.

[26] 
R. Tibshirani, G. Walther, and T. Hastie, “Estimating the number of clusters in a data set via the gap statistic,” J. R. Stat Soc.: Series B Stat. Methodol, vol. 63, no. 2, pp. 411–423, 2001. doi: 10.1111/14679868.00293

[27] 
M. Radovanovic, A. Nanopoulos, and M. Ivanovic, “Hubs in space: Popular nearest neighbors in highdimensional data,” J. Mach. Learn Res., vol. 11, no. sept, pp. 2487–2531, 2010.

[28] 
C. Gong, Y. Li, D. Fu, Y. Liu, P. Wang, and Y. You, “Selfreconstructive evidential clustering for highdimensional data,” in Proc. Int. Conf. Data Eng. Kuala Lumpur, Malaysia: IEEE, 2022, pp. 2099–2112.

[29] 
L. Czétány, V. Vámos, M. Horváth, Z. Szalay, A. MotaBabiloni, Z. DemeBélafi, and T. Csoknyai, “Development of electricity consumption profiles of residential buildings based on smart meter data clustering,” Energy Build., vol. 252, p. 111376, 2021. doi: 10.1016/j.enbuild.2021.111376

[30] 
Y. Wang, Q. Chen, C. Kang, and Q. Xia, “Clustering of electricity consumption behavior dynamics toward big data applications,” IEEE Trans. Smart Grid, vol. 7, no. 5, pp. 2437–2447, 2016. doi: 10.1109/TSG.2016.2548565

[31] 
C. Gong, Z. Su, P. Wang, Q. Wang, and Y. You, “A sparse reconstructive evidentialnearest neighbor classifier for highdimensional data,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 6, pp. 5563–5576, 2023.

[32] 
C. Lu, H. Min, Z. Zhao, L. Zhu, D. Huang, and S. Yan, “Robust and efficient subspace segmentation via least squares regression,” in Proc. Comput Vis ECCV. Florence, Italy: Springer, 2012, pp. 347–360.

[33] 
T. Denœux and M.H. Masson, “EVCLUS: Evidential clustering of proximity data,” IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 34, no. 1, pp. 95–109, 2004. doi: 10.1109/TSMCB.2002.806496

[34] 
T. Denoeux and O. Kanjanatarakul, “Evidential clustering: A review,” in Proc. Inter. Symp. IUKM. Da Nang, Vietnam: Springer, 2016, pp. 24–35.

[35] 
G. Shafer, “A mathematical theory of evidence,” in A Mathematical Theory of Evidence. Princeton university press, 1976.

[36] 
A. P. Dempster, “Upper and lower probabilities induced by a multivalued mapping,” in Classic Works of the DempsterShafer Theory of Belief Functions. Springer, 2008, pp. 57–72.

[37] 
G. Tzortzis and A. Likas, “Convex mixture models for multiview clustering,” in Proc. Int. Conf. Artif. Neural Netw. Limassol, Cyprus: Springer, 2009, pp. 205–214.

[38] 
X. Si, Q. Yin, X. Zhao, and L. Yao, “Consistent and diverse multiview subspace clustering with structure constraint,” Pattern Recognit., vol. 121, p. 108196, 2022. doi: 10.1016/j.patcog.2021.108196

[39] 
Z. Li, C. Tang, X. Liu, X. Zheng, W. Zhang, and E. Zhu, “Consensus graph learning for multiview clustering,” IEEE Trans. Multimedia., vol. 24, pp. 2461–2472, 2021.

[40] 
M.S. Chen, L. Huang, C.D. Wang, and D. Huang, “Multiview clustering in latent embedding space,” in Proc. AAAI Conf. Artificial Intelligence, vol. 34, no. 4, New York, USA, 2020, pp. 3513–3520.

[41] 
S. Zhou, X. Liu, M. Li, E. Zhu, L. Liu, C. Zhang, and J. Yin, “Multiple kernel clustering with neighborkernel subspace segmentation,” IEEE Trans. Neural Netw. Learn. Syst, vol. 31, no. 4, pp. 1351–1362, 2019.

[42] 
S. Du, Z. Liu, Z. Chen, W. Yang, and S. Wang, “Differentiable bisparse multiview coclustering,” IEEE Trans. Signal Process., vol. 69, pp. 4623–4636, 2021. doi: 10.1109/TSP.2021.3101979

[43] 
Z. Fang, S. Du, X. Lin, J. Yang, S. Wang, and Y. Shi, “DBONet: Differentiable bilevel optimization network for multiview clustering,” Inf. Sci., 2023.

[44] 
X. Liu, L. Liu, Q. Liao, S. Wang, Y. Zhang, W. Tu, C. Tang, J. Liu, and E. Zhu, “One pass late fusion multiview clustering,” in Proc. Int. Conf. Mach. Learn. PMLR, 2021, pp. 6850–6859.

[45] 
D. Pelleg, A. W. Moore et al., “Xmeans: Extending kmeans with efficient estimation of the number of clusters,” in Proc. Int. Conf. Mach. Learn, vol. 1, Stanford University, USA, 2000, pp. 727–734.

[46] 
J. Wang, “Consistent selection of the number of clusters via crossvalidation,” Biometrika, vol. 97, no. 4, pp. 893–904, 2010. doi: 10.1093/biomet/asq061

[47] 
S. Sarkar and A. K. Ghosh, “On perfect clustering of high dimension, low sample size data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 9, pp. 2257–2272, 2019.

[48] 
O. Basir and X. Yuan, “Engine fault diagnosis based on multisensor information fusion using dempster—Shafer evidence theory,” Inf. Fusion, vol. 8, no. 4, pp. 379–386, 2007. doi: 10.1016/j.inffus.2005.07.003

[49] 
I. Bloch, “Some aspects of dempstershafer evidence theory for classification of multimodality medical images taking partial volume effect into account,” Pattern Recognit. Lett., vol. 17, no. 8, pp. 905–919, 1996. doi: 10.1016/01678655(96)000396

[50] 
M. Kantardzic, C. Walgampaya, R. Yampolskiy, and R. J. Woo, “Click fraud prevention via multimodal evidence fusion by dempstershafer theory,” in Proc. Conf. Multisensor Fus. Integ. Salt Lake City, USA: IEEE, 2010, pp. 26–31.

[51] 
Y.T. Liu, N. R. Pal, A. R. Marathe, and C.T. Lin, “Weighted fuzzy dempster—Shafer framework for multimodal information integration,” IEEE Trans. Fuzzy Syst., vol. 26, no. 1, pp. 338–352, 2017.

[52] 
Z. Han, C. Zhang, H. Fu, and J. T. Zhou, “Trusted multiview classification with dynamic evidential fusion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 2, pp. 2551–2566, 2023.

[53] 
Y. Qin, D. Peng, X. Peng, X. Wang, and P. Hu, “Deep evidential learning with noisy correspondence for crossmodal retrieval,” in Proc. ACM Int. Conf. Multimedia, Lisbon, Portugal, 2022, pp. 4948–4956.

[54] 
C. Gong, P. Wang, and Z. Su, “An interactive nonparametric evidential regression algorithm with instance selection,” Soft Comput., vol. 24, pp. 3125–3140, 2020.

[55] 
L.Q. Huang, Z.G. Liu, and J. Dezert, “Crossdomain pattern classification with distribution adaptation based on evidence theory,” IEEE Trans. Cybernetics, vol. 53, no. 2, pp. 718–731, 2023.

[56] 
C. Gong, Z. Su, P. Wang, Q. Wang, and Y. You, “Evidential instance selection for knearest neighbor classification of big data,” Int. J. Approx Reason., vol. 138, pp. 123–144, 2021.

[57] 
C. Gong, Y. Li, Y. Liu, P. Wang, and Y. You, “Joint evidential knearest neighbor classification,” in Proc. Int. Conf. Data Eng. Kuala Lumpur, Malaysia: IEEE, 2022, pp. 2113–2126.

[58] 
S. Shi, F. Nie, R. Wang, and X. Li, “Multiview clustering via nonnegative and orthogonal graph reconstruction,” IEEE Trans. Neural Netw. Learn. Syst, vol. 34, no. 1, pp. 201–214, 2023.

[59] 
Q. Wang, Z. Tao, Q. Gao, and L. Jiao, “Multiview subspace clustering via structured multipathway network,” IEEE Trans. Neural Netw. Learn. Syst, 2022. DOI: 10.1109/TNNLS.2022.3213374

[60] 
J. Xu, H. Tang, Y. Ren, L. Peng, X. Zhu, and L. He, “Multilevel feature learning for contrastive multiview clustering,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, New Orleans, USA, 2022, pp. 16 051–16 060.

[61] 
D. Cai, X. He, and J. Han, “Document clustering using locality preserving indexing,” IEEE Trans. Knowl. Data Eng., vol. 17, no. 12, pp. 1624–1637, 2005. doi: 10.1109/TKDE.2005.198

[62] 
Z. Huang, Y. Ren, X. Pu, L. Pan, D. Yao, and G. Yu, “Dual selfpaced multiview clustering,” Neural Netw, vol. 140, pp. 184–192, 2021.

[63] 
Q. Yin, S. Wu, R. He, and L. Wang, “Multiview clustering via pairwise sparse subspace representation,” Neurocomputing, vol. 156, pp. 12–21, 2017.

[64] 
P. Zhu, B. Hui, C. Zhang, D. Du, L. Wen, and Q. Hu, “Multiview deep subspace clustering networks,” arXiv preprint arXiv: 1908.01978, 2019.

[65] 
J. Xu, Y. Ren, G. Li, L. Pan, C. Zhu, and Z. Xu, “Deep embedded multiview clustering with collaborative training,” Inf. Sci., vol. 573, pp. 279–290, 2021.

[66] 
Y. Xie, B. Lin, Y. Qu, C. Li, W. Zhang, L. Ma, Y. Wen, and D. Tao, “Joint deep multiview learning for image clustering,” IEEE Trans. Knowl. Data Eng., vol. 33, no. 11, pp. 3594–3606, 2020.

[67] 
Y. Ren, S. Huang, P. Zhao, M. Han, and Z. Xu, “Selfpaced and autoweighted multiview clustering,” Neurocomputing, vol. 383, pp. 248–256, 2020.

[68] 
C. Zhang, H. Fu, Q. Hu, X. Cao, Y. Xie, D. Tao, and D. Xu, “Generalized latent multiview subspace clustering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 1, pp. 86–99, 2020.

[69] 
Z. Kang, H. Pan, S. C. Hoi, and Z. Xu, “Robust graph learning from noisy data,” IEEE Trans. Cybernetics, vol. 50, no. 5, pp. 1833–1843, 2019.

[70] 
L. Van der Maaten and G. Hinton, “Visualizing data using tSNE,” J. Mach. Learn Res., vol. 9, no. 11, 2008.
