A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach

Xia Chen; Zhan-Li Sun; Kin-Man Lam; Zhigang Zeng

doi:10.1109/JAS.2020.1003006

Volume 7 Issue 5

Sep. 2020

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2020 > 7(5): 1455-1464

Xia Chen, Zhan-Li Sun, Kin-Man Lam and Zhigang Zeng, "A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1455-1464, Sept. 2020. doi: 10.1109/JAS.2020.1003006

Citation:

Xia Chen, Zhan-Li Sun, Kin-Man Lam and Zhigang Zeng, "A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1455-1464, Sept. 2020. doi: 10.1109/JAS.2020.1003006

Citation:

PDF( 1160 KB)

A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach

doi: 10.1109/JAS.2020.1003006

Funds: The work was supported by the National Natural Science Foundation of China (61972002), and Open Grant from Anhui Province Key Laboratory of Non-Destructive Evaluation (CGHBMWSJC07)

More Information

Author Bio:
Xia Chen is a Ph.D. candidate in the School of Electrical Engineering and Automation at Anhui University. Her research interests include machine learning and image and signal processing

Zhan-Li Sun (M’19) received the Ph.D. degree from the University of Science and Technology of China, in 2005. Since 2006, he has worked with The Hong Kong Polytechnic University, Nanyang Technological University, and National University of Singapore. He is currently a Professor with School of Electrical Engineering and Automation, Anhui University. His research interests include machine learning, and image and signal processing. Dr. Sun serves as an Associate Editor of IEEE Access

Kin-Man Lam (SM’14) received the associateship in electronic engineering with Distinction from The Hong Kong Polytechnic University (formerly called Hong Kong Polytechnic) in 1986, the M.Sc. degree in communication engineering from the Department of Electrical Engineering, Imperial College of Science, Technology and Medicine, U.K., in 1987, and the Ph.D. degree from the Department of Electrical Engineering, University of Sydney, Australia, in August 1996.From 1990 to 1993, he was a Lecturer at the Department of Electronic Engineering, The Hong Kong Polytechnic University. He joined the Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, as an Assistant Professor in October 1996, became an Associate Professor in 1999, and has been a Professor since 2010. He has been a Member of the organizing committee and program committee of many international conferences. Currently, Dr. Lam is a General Co-Chair of the IEEE International Conference on Signal Processing, Communications, and Computing (ICSPCC2012). He is a BoG Member of the Asia-Pacific Signal and Information Processing Association (APSIPA), and the Director-Student Services of the IEEE Signal Processing Society. Dr. Lam also serves as an Associate Editor of IEEE Transactions on Image Processing, APSIPA Transactions on Signal and Information Processing, and EURASIP International Journal on Image and Video Processing. His current research interests include human face recognition, image and video processing, and computer vision

Zhigang Zeng (SM’07–F’20) received the Ph.D. degree in systems analysis and integration from Huazhong University of Science and Technology in 2003. He is currently a Professor with the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, and also with the Key Laboratory of Image Processing and Intelligent Control of the Education Ministry of China. He has authored or coauthored more than 100 international journal papers. His current research interests include theory of functional differential equations and differential equations with discontinuous right-hand sides, and their applications to dynamics of neural networks, memristive systems, and control systems. Dr. Zeng was an Associate Editor for the IEEE Transactions on Neural Networks (2010–2011), and has been an Associate Editor for the IEEE Transactions on Cybernetics (since 2014), IEEE Transactions on Fuzzy Systems (since 2016), and a Member of the Editorial Board of Neural Networks (since 2012), Cognitive Computation (since 2010), and Applied Soft Computing (since 2013)
Corresponding author: K.-M. Lam is with the Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China (e-mail: enkmlam@polyu.edu.hk)
Received Date: 2019-07-16
Revised Date: 2019-08-23
Accepted Date: 2019-09-26

Available Online: 2019-10-16

Abstract

Abstract

In many traditional non-rigid structure from motion (NRSFM) approaches, the estimation results of part feature points may significantly deviate from their true values because only the overall estimation error is considered in their models. Aimed at solving this issue, a local deviation-constrained-based column-space-fitting approach is proposed in this paper to alleviate estimation deviation. In our work, an effective model is first constructed with two terms: the overall estimation error, which is computed by a linear subspace representation, and a constraint term, which is based on the variance of the reconstruction error for each frame. Furthermore, an augmented Lagrange multipliers (ALM) iterative algorithm is presented to optimize the proposed model. Moreover, a convergence analysis is performed with three steps for the optimization process. As both the overall estimation error and the local deviation are utilized, the proposed method can achieve a good estimation performance and a relatively uniform estimation error distribution for different feature points. Experimental results on several widely used synthetic sequences and real sequences demonstrate the effectiveness and feasibility of the proposed algorithm.
- Augmented Lagrange multipliers (ALM),
- column-spacefitting,
- non-rigid structure from motion (NRSFM)

FullText(HTML)

References(37)

References

[1]	X. Wang and H. Duan, “Hierarchical visual attention model for saliency detection inspired by avian visual pathways,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 540–552, 2019. doi: 10.1109/JAS.2017.7510664
[2]	Y. Tian, X. Li, K. Wang, and F.-Y. Wang, “Training and testing object detectors with virtual images,” IEEE/CAA J. Autom. Sinica, vol. 5, no. 2, pp. 539–546, 2018. doi: 10.1109/JAS.2017.7510841
[3]	Z. Q. Zhao, P. Zheng, S. T. Xu, and X. Wu, “Object detection with deep learning: A review,” IEEE Trans. Neural Networks and Learning Systems, 2019. doi: 10.1109/TNNLS.2018.2876865
[4]	D. Herrera, F. Roberti, M. Toibero M, and R. Carelli, “Human interaction dynamics for its use in mobile robotics: Impedance control for leaderfollower formation,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 696–703, 2017. doi: 10.1109/JAS.2017.7510631
[5]	Z. Q. Zhao, H. Glotin, Z. Xie, J. Gao, and X. Wu, “Cooperative sparse representation in two opposite directions for semi-supervised image annotation,” IEEE Trans. Image Processing, vol. 21, no. 9, pp. 4218–4231, 2012. doi: 10.1109/TIP.2012.2197631
[6]	D. S. Huang, H. H. S. Ip, and Z. Chi, “A neural root finder of polynomials based on root moments,” Neural Computation, vol. 16, no. 8, pp. 1721–1762, 2004. doi: 10.1162/089976604774201668
[7]	D. S. Huang, “A constructive approach for finding arbitrary roots of polynomials by neural networks,” IEEE Trans. Neural Networks, vol. 15, no. 2, pp. 477–491, 2004. doi: 10.1109/TNN.2004.824424
[8]	X. Sun, S. Shen, H. Cui, L. Hu, and Z. Hu, “Geographic, geometrical and semantic reconstruction of urban scene from high resolution oblique aerial images,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 118–130, 2019. doi: 10.1109/JAS.2017.7510673
[9]	N. Hao, H. Liao, Y. Qiu, and J. Yang, “Face super-resolution reconstruction and recognition using non-local similarity dictionary learning based algorithm,” IEEE/CAA J. Autom. Sinica, vol. 3, no. 2, pp. 213–224, 2016. doi: 10.1109/JAS.2016.7451109
[10]	Z. L. Sun, K. M. Lam, and Q. W. Gao, “An effective missing-data estimation approach for small-size image sequences,” IEEE Computational Intelligence Magazine, vol. 10, no. 3, pp. 10–18, 2015. doi: 10.1109/MCI.2015.2437311
[11]	S. Kumar, A. Cherian, Y. Dai, and H. Li, “Scalable dense non-rigid structure-from-motion: A grassmannian perspective,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018, pp. 254–263.
[12]	Z. Zhou, F. Shi, J. Xiao, and W. Wu, “Non-rigid structure-from-motion on degenerate deformations with low-rank shape deformation model,” IEEE Trans. Multimedia, vol. 17, no. 2, pp. 171–185, 2015. doi: 10.1109/TMM.2014.2384396
[13]	C. Bregler, A. Hertzmann, and H. Biermann, “Recovering non-rigid 3D shape from image streams,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp. 690–696.
[14]	J. Xiao, J. Chai, and T. Kanade, “A closed-form solution to non-rigid shape and motion recovery,” Int. J. Computer Vision, vol. 67, no. 2, pp. 233–246, 2006. doi: 10.1007/s11263-005-3962-9
[15]	L. Torresani, A. Hertzmann, and C. Bregler, “Nonrigid structure-frommotion: estimating shape and motion with hierarchical priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 878–892, 2008. doi: 10.1109/TPAMI.2007.70752
[16]	Z. L. Sun and M. K. Lam, “Depth estimation of face images based on the constrained ICA model,” IEEE Trans. Information Forensics and Security, vol. 6, no. 2, pp. 360–370, 2011. doi: 10.1109/TIFS.2011.2118207
[17]	Q. Dong and H. Wang, “Latent-smoothness nonrigid structure from motion by revisiting multilinear factorization,” IEEE Trans. Cybernetics, vol. 49, no. 9, pp. 3557–3570, 2018.
[18]	Y. Dai, H. Li, and M. He, “A simple prior-free method for non-rigid structure-from-motion factorization,” Int. J. Computer Vision, vol. 107, no. 2, pp. 101–122, 2014. doi: 10.1007/s11263-013-0684-2
[19]	S. Kumar, “Jumping manifolds: Geometry aware dense non-rigid structure from motion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2019, pp. 5346–5355.
[20]	J. W. Lu, G. Wang, W. H. Deng, and K. Jia, “Reconstruction-based metric learning for unconstrained face verification,” IEEE Trans. Information Forensics and Security, vol. 10, no. 1, pp. 79–89, 2015. doi: 10.1109/TIFS.2014.2363792
[21]	A. Agudo, F. Moreno-Noguer, B. Calvo, and J. M. M. Montiel, “Sequential non-rigid structure from motion using physical priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 38, no. 5, pp. 979–994, 2016. doi: 10.1109/TPAMI.2015.2469293
[22]	A. M. Gallardo, T. Collins, A. Bartoli, and F. Mathias, “Dense nonrigid structure-from-motion and shading with unknown,” in Proc. IEEE Int. Conf. Computer Vision, 2017, pp. 3884–3892.
[23]	A. Agudo, M. Pijoan, and F. Moreno-Noguer, “Dust: Dual union of spatio-temporal subspaces for monocular multiple object 3D reconstruction,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2017, pp. 1513–1521.
[24]	A. Agudo, M. Pijoan, and F. Moreno-Noguer, “Image collection popup: 3D reconstruction and clustering of rigid and non-rigid categories,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018, pp. 2607–2615.
[25]	I. Akhter, Y. Sheikh, and S. Khan, “In defense of orthonormality constraints for nonrigid structure from motion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 1534–1541.
[26]	I. Akhter, Y. A. Sheikh, S. Khan, and T. Kanade, “Trajectory space: A dual representation for nonrigid structure from motion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1442–1456, 2011. doi: 10.1109/TPAMI.2010.201
[27]	P. F. U. Gotardo and A. M. Martinez, “Computing smooth time trajectories for camera and deformable shape in structure from motion with occlusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 2051–2065, 2011.
[28]	P. F. U. Gotardo and A. M. Martinez, “Non-rigid structure from motion with complementary rank-3 spaces,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 3065–3072.
[29]	X. Zhou, M. Zhu, G. Pavlakos, S. Leonardos, K. G. Derpanis, and K. Daniilidis, “Monocap: Monocular human motion capture using a CNN coupled with a geometric prior,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 41, no. 4, pp. 901–914, 2018.
[30]	Y. Wang, X. Yan, M. Jiang, and J. Zheng, “3D Non-rigid structure from motion based on space approximation in trajectory space,” Int. J. Robotics and Automation, vol. 33, no. 2, pp. 111–117, 2018.
[31]	Z. Lin, M. Chen, and Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” UIUC Technicial Report UIUL-ENG-09-2214, 2010.
[32]	M. R. Hestenes, “Multiplier and gradient methods,” J. Optimization Theory and Applications, vol. 4, no. 5, pp. 303–320, 1969. doi: 10.1007/BF00927673
[33]	D. P. Bertsekas, “Constrained optimization and Lagrange multiplier methods,” New York: Academic Press, 2014.
[34]	C. Li, C. L. Wang, and J. Wang, “Convergence analysis of the augmented Lagrange multiplier algorithm for a class of matrix compressive recovery,” Applied Mathematics Letters, vol. 59, pp. 12–17, 2016. doi: 10.1016/j.aml.2016.02.022
[35]	O. C. Hamsici, P. F. U. Gotardo, and A. M. Martinez, “Learning spatially-smooth mappings in non-rigid structure from motion,” in Proc. European Conf. Computer Vision, 2012, pp. 260–273.
[36]	M. Lee, J. Cho, and S. Oh, “Consensus of non-rigid reconstructions”, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 4670–4678.
[37]	P. F. U. Gotardo, and A. M. Martinez, “Kernel non-rigid structure from motion”, in Proc. IEEE Int. Conf. Computer Vision, 2011, pp. 802–809.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(6) / Tables(4)

Get Citation

PDF

XML

Article Metrics

Article views (1256) PDF downloads(51)

Highlights

An effective model is constructed by considering both the overall estimation error and the variance of reconstruction errors for each frame.
An Augmented Lagrange Multipliers (ALM) iterative algorithm is developed to optimize the local deviation-constrained-based estimation model.
A convergence analysis is carried out in detail for the model optimization.

A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach

doi: 10.1109/JAS.2020.1003006

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Highlights

Export File

Citation

Format

Content