A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 11 Issue 12
Dec.  2024

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2024  >  11(12): 2463-2475
H. Liu, Y. Tong, and  Z. Zhang,  “Human observation-inspired universal image acquisition paradigm integrating multi-objective motion planning and control for robotics,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 12, pp. 2463–2475, Dec. 2024. doi: 10.1109/JAS.2024.124512
Citation: H. Liu, Y. Tong, and  Z. Zhang,  “Human observation-inspired universal image acquisition paradigm integrating multi-objective motion planning and control for robotics,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 12, pp. 2463–2475, Dec. 2024. doi: 10.1109/JAS.2024.124512
shu

Human Observation-Inspired Universal Image Acquisition Paradigm Integrating Multi-Objective Motion Planning and Control for Robotics

doi: 10.1109/JAS.2024.124512
Funds:  This work was supported in part by the National Natural Science Foundation of China (62303457, U21A20482), China Postdoctoral Science Foundation (2023M733737), and the National Key Research and Development Program of China (2022YFB3303800)
More Information
    Abstract
  • Image acquisition stands as a prerequisite for scrutinizing surfaces inspection in industrial high-end manufacturing. Current imaging systems often exhibit inflexibility, being confined to specific objects and encountering difficulties with diverse industrial structures lacking standardized computer-aided design (CAD) models or in instances of deformation. Inspired by the multidimensional observation of humans, our study introduces a universal image acquisition paradigm tailored for robotics, seamlessly integrating multi-objective optimization trajectory planning and control scheme to harness measured point clouds for versatile, efficient, and highly accurate image acquisition across diverse structures and scenarios. Specifically, we introduce an energy-based adaptive trajectory optimization (EBATO) method that combines deformation and deviation with dual-threshold optimization and adaptive weight adjustment to improve the smoothness and accuracy of imaging trajectory and posture. Additionally, a multi-optimization control scheme based on a meta-heuristic beetle antennal olfactory recurrent neural network (BAORNN) is proposed to track the imaging trajectory while addressing posture, obstacle avoidance, and physical constraints in industrial scenarios. Simulations, real-world experiments, and comparisons demonstrate the effectiveness and practicality of the proposed paradigm.

     

    • FullText(HTML)
  • loading
    • References(32)
  • [1]
    Y. Liu, J. Dong, Y. Li, X. Gong, and J. Wang, “A UAV-based aircraft surface defect inspection system via external constraints and deep learning,” IEEE Trans. Instrumentation and Measurement, vol. 71, pp. 1–15, 2022.
    [2]
    G. Dwivedi, L. Pensia, V. Lohchab, and R. Kumar, “Nondestructive inspection and quantification of soldering defects in PCB using an autofocusing digital holographic camera,” IEEE Trans. Instrumentation and Measurement, vol. 72, pp. 1–8, 2023. doi: 10.1109/TIM.2023.3298390
    [3]
    S. B. Block, R. D. da Silva, L. B. Dorini, and R. Minetto, “Inspection of imprint defects in stamped metal surfaces using deep learning and tracking,” IEEE Trans. Industrial Electronics, vol. 68, no. 5, pp. 4498–4507, 2021. doi: 10.1109/TIE.2020.2984453
    [4]
    J. Q. Yang, S. Zhou, D. van Le, D. Ho, and R. Tan, “Improving quality control with industrial aiot at HP factories: Experiences and learned lessons,” in Proc. 18th Annual IEEE Int. Conf. Sensing, Communication, and Networking, Rome, Italy, 2021, pp. 1–9.
    [5]
    P. Stavropoulos, K. Sabatakakis, A. Papacharalampopoulos, and D. Mourtzis, “Infrared (IR) quality assessment of robotized resistance spot welding based on machine learning,” The Int. J. Advanced Manufacturing Technology, vol. 119, no. 3–4, pp. 1785–1806, 1785.
    [6]
    D. Nakhaeinia, P. Payeur, and R. Laganiere, “A mode-switching motion control system for reactive interaction and surface following using industrial robots,” IEEE/CAA J. Autom. Sinica, vol. 5, no. 3, pp. 670–682, 2018. doi: 10.1109/JAS.2018.7511069
    [7]
    Y. Tong, J. Liu, Y. Liu, and Y. Yuan, “Analytical inverse kinematic computation for 7-DOF redundant sliding manipulators,” Mechanism and Machine Theory, vol. 155, p. 104006, 2021. doi: 10.1016/j.mechmachtheory.2020.104006
    [8]
    Q. Wang, W. Jiao, P. Wang, and Y. Zhang, “Digital twin for human-robot interactive welding and welder behavior analysis,” IEEE/CAA J. Autom. Sinica, vol. 8, no. 2, pp. 334–343, 2021. doi: 10.1109/JAS.2020.1003518
    [9]
    H. Chae, Y. Moon, K. Lee, S. Park, H. S. Kim, and T. Seo, “A tethered façade cleaning robot based on a dual rope windlass climbing mechanism: Design and experiments,” IEEE/ASME Trans. Mechatronics, vol. 27, no. 4, pp. 1982–1989, 2022. doi: 10.1109/TMECH.2022.3172689
    [10]
    J. Yang, X. Wang, and Y. Zhao, “Parallel manufacturing for industrial metaverses: A new paradigm in smart manufacturing,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 12, pp. 2063–2070, 2022. doi: 10.1109/JAS.2022.106097
    [11]
    R. Almadhoun, T. Taha, L. Seneviratne, J. Dias, and G. Cai, “A survey on inspecting structures using robotic systems,” Int. J. Advanced Robotic Systems, vol. 13, no. 6, p. 1729881416663664, 2016. doi: 10.1177/1729881416663664
    [12]
    I. D. Lee, J. H. Seo, Y. M. Kim, J. Choi, S. Han, and B. Yoo, “Automatic pose generation for robotic 3-D scanning of mechanical parts,” IEEE Trans. Robotics, vol. 36, no. 4, pp. 1219–1238, 2020. doi: 10.1109/TRO.2020.2980161
    [13]
    M. Xiao, Y. Ding, and G. Yang, “A model-based trajectory planning method for robotic polishing of complex surfaces,” IEEE Trans. Autom. Science and Engineering, vol. 19, no. 4, pp. 2890–2903, 2022. doi: 10.1109/TASE.2021.3095061
    [14]
    Y. Liu, W. Zhao, H. Liu, Y. Wang, and X. Yue, “Coverage path planning for robotic quality inspection with control on measurement uncertainty,” IEEE/ASME Trans. Mechatronics, vol. 27, no. 5, pp. 3482–3493, 2022. doi: 10.1109/TMECH.2022.3142756
    [15]
    N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa, “CHOMP: Gradient optimization techniques for efficient motion planning,” in Proc. IEEE Int. Conf. Robotics and Automation, 2009, pp. 489–494.
    [16]
    M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal, “STOMP: Stochastic trajectory optimization for motion planning,” in Proc. IEEE Int. Conf. Robotics and Automation, Shanghai, China, 2011, pp. 4569–4574.
    [17]
    A. H. Qureshi, A. Simeonov, M. J. Bency, and M. C. Yip, “Motion planning networks,” in Proc. Int. Conf. Robotics and Automation, 2019, pp. 2118–2124.
    [18]
    A. H. Qureshi, Y. Miao, A. Simeonov, and M. C. Yip, “Motion planning networks: Bridging the gap between learning-based and classical motion planners,” IEEE Trans. Robotics, vol. 37, no. 1, pp. 48–66, 2021. doi: 10.1109/TRO.2020.3006716
    [19]
    Z. Zhang, S. Chen, X. Deng, and J. Liang, “A circadian rhythms neural network for solving the redundant robot manipulators tracking problem perturbed by periodic noise,” IEEE/ASME Trans. Mechatronics, vol. 26, no. 6, pp. 3232–3242, 2021. doi: 10.1109/TMECH.2021.3056409
    [20]
    Y. Tong, J. Liu, X. Zhang, and Z. Ju, “Four-criterion-optimization-based coordination motion control of dual-arm robots,” IEEE Trans. Cognitive and Developmental Systems, vol. 15, no. 2, pp. 794–807, 2023. doi: 10.1109/TCDS.2022.3182534
    [21]
    Z. Xie, L. Jin, and X. Luo, “Kinematics-based motion-force control for redundant manipulators with quaternion control,” IEEE Trans. Autom. Science and Engineering, pp. 1–14, 2022.
    [22]
    Y. Tong, J. Liu, H. Zhou, Z. Ju, and X. Zhang, “Adaptive tracking control of robotic manipulators with unknown kinematics and uncertain dynamics,” IEEE Trans. Autom. Science and Engineering, 2023. DOI: 10.1109/TASE.2023.3309964.
    [23]
    A. H. Khan, S. Li, and X. Luo, “Obstacle avoidance and tracking control of redundant robotic manipulator: An RNN-based metaheuristic approach,” IEEE Trans. Industrial Informatics, vol. 16, no. 7, pp. 4670–4680, 2020. doi: 10.1109/TII.2019.2941916
    [24]
    M. Yang, Y. Zhang, N. Tan, and H. Hu, “Concise discrete ZNN controllers for end-effector tracking and obstacle avoidance of redundant manipulators,” IEEE Trans. Industrial Informatics, vol. 18, no. 5, pp. 3193–3202, 2022. doi: 10.1109/TII.2021.3109426
    [25]
    Z. Xu, X. Zhou, H. Wu, X. Li, and S. Li, “Motion planning of manipulators for simultaneous obstacle avoidance and target tracking: An RNN approach with guaranteed performance,” IEEE Trans. Industrial Electronics, vol. 69, no. 4, pp. 3887–3897, 2022. doi: 10.1109/TIE.2021.3073305
    [26]
    S. Wang, F. Qin, Y. Tong, X. Shang, and Z. Zhang, “Probabilistic boundary-guided point cloud primitive segmentation network,” IEEE Trans. Instrumentation and Measurement, vol. 72, p. 2529413, 2023. doi: 10.1109/TIM.2023.3322509
    [27]
    S. Wang, Y. Tong, X. Shang, and Z. Zhang, “Hierarchical viewpoint planning for complex surfaces in industrial product inspection,” IEEE/ASME Trans. Mechatronics, 2023. DOI: 10.1109/TMECH.2023.3340312.
    [28]
    C. Deng and H. Lin, “Progressive and iterative approximation for least squares b-spline curve and surface fitting,” Computer-Aided Design, vol. 47, pp. 32–44, 2014. doi: 10.1016/j.cad.2013.08.012
    [29]
    G. Wang, W. Li, C. Jiang, D. Zhu, Z. Li, W. Xu, H. Zhao, and H. Ding, “Trajectory planning and optimization for robotic machining based on measured point cloud,” IEEE Trans. Robotics, vol. 38, no. 3, pp. 1621–1637, 2022. doi: 10.1109/TRO.2021.3108506
    [30]
    M. Pourazady and X. Xu, “Direct manipulations of NURBS surfaces subjected to geometric constraints,” Computers & Graphics, vol. 30, no. 4, pp. 598–609, 2006.
    [31]
    X. Zhang, J. Liu, and Y. Li, “An obstacle avoidance algorithm for space hyper-redundant manipulators using combination of RRT and shape control method,” Robotica, vol. 40, no. 4, pp. 1036–1069, 2022. doi: 10.1017/S0263574721000928
    [32]
    J. Yan, L. Jin, Z. Yuan, and Z. Liu, “RNN for receding horizon control of redundant robot manipulators,” IEEE Trans. Industrial Electronics, vol. 69, no. 2, pp. 1608–1619, 2022. doi: 10.1109/TIE.2021.3062257
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(19)  / Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (78) PDF downloads(19) Cited by()

    Highlights

    • This article presents a universal image acquisition paradigm for industrial robots
    • This article proposes an improved BAORNN-based multi-optimization control scheme
    • This article introduces an energy-based adaptive trajectory optimization method
    • This article achieves point cloud-based clear imaging inspired by human observation

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return