A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2025  > In Press, Accepted Manuscript
H. Fang, S. Yuan, H. Ren, S. He, and S. S. Cheng, “ADAPT: A model-free adaptive optimal control for continuum robots,” IEEE/CAA J. Autom. Sinica, 2025. doi: 10.1109/JAS.2025.125183
Citation: H. Fang, S. Yuan, H. Ren, S. He, and S. S. Cheng, “ADAPT: A model-free adaptive optimal control for continuum robots,” IEEE/CAA J. Autom. Sinica, 2025. doi: 10.1109/JAS.2025.125183
shu

ADAPT: A Model-Free Adaptive Optimal Control for Continuum Robots

doi: 10.1109/JAS.2025.125183
Funds:  Research reported in this work was supported in part by Innovation and Technology Commission of Hong Kong (ITS/136/20, ITS/234/21, MHP/096/22, ITS/235/22), Multi-Scale Medical Robotics Center, InnoHK (Grant 8312051), Research Grants Council (RGC) of Hong Kong (CUHK 14217822, CUHK 14207823), The Chinese University of Hong Kong (CUHK) Direct Grant
More Information
    Abstract
  • Realizing optimal control performance for continuum robots (CRs) poses huge challenges on traditional model-based optimal control approaches due to their high degrees of freedom, complex nonlinear dynamics and soft continuum morphologies which are difficult to explicitly model. This paper proposes a model-free adaptive optimal control algorithm (ADAPT) for CRs. In our strategy, we consider CRs as a class of nonlinear continuous-time dynamical systems in the state space, wherein the position of the end-effector is considered as the state and the input torque is mapped as the control input. Then, the optimized Hamilton-Jacobi-Bellman (HJB) equation is derived by optimal control principles, and subsequently solved by the proposed ADAPT algorithm without requiring knowledge of the origin system dynamics. Under some mild assumptions, the global stability and convergence of the closed-loop control approach are guaranteed. Several simulation experiments are conducted on a magnetic CR (MCR) to demonstrate the practicality and effectiveness of the ADAPT algorithm.

     

    • FullText(HTML)
  • loading
    • References(55)
  • [1]
    M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot modeling and control. John Wiley & Sons, 2020.
    [2]
    R. J. Webster Ⅲ and B. A. Jones, “Design and kinematic modeling of constant curvature continuum robots: A review,” The Int. Journal of Robotics Research, vol. 29, no. 13, pp. 1661–1683, 2010. doi: 10.1177/0278364910368147
    [3]
    A. A. Alqumsan, S. Khoo, and M. Norton, “Robust control of continuum robots using Cosserat rod theory,” Mechanism and Machine Theory, vol. 131, pp. 48–61, 2019. doi: 10.1016/j.mechmachtheory.2018.09.011
    [4]
    J. Wang, J. Xue, S. Yuan, J. Tan, S. Song, and M. Q.-H. Meng, “Kine-matic modeling of magnetically-actuated robotic catheter in nonlinearly-coupled multi-field,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 8189–8196, 2021. doi: 10.1109/LRA.2021.3104620
    [5]
    F. Xu, H. Wang, Z. Liu, W. Chen, and Y. Wang, “Visual servoing pushing control of the soft robot with active pushing force regulation,” Soft Robotics, vol. 9, no. 4, pp. 690–704, 2022. doi: 10.1089/soro.2020.0178
    [6]
    Z. Wang, T. Wang, B. Zhao, Y. He, Y. Hu, B. Li, P. Zhang, and M. Q.-H. Meng, “Hybrid adaptive control strategy for continuum surgical robot under external load,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1407–1414, 2021. doi: 10.1109/LRA.2021.3057558
    [7]
    M. Khadem, J. O’Neill, Z. Mitros, L. Da Cruz, and C. Bergeles, “Autonomous steering of concentric tube robots via nonlinear model predictive control,” IEEE Trans. Robotics, vol. 36, no. 5, pp. 1595–1602, 2020. doi: 10.1109/TRO.2020.2991651
    [8]
    M. S. Xavier, A. J. Fleming, and Y. K. Yong, “Nonlinear estimation and control of bending soft pneumatic actuators using feedback linearization and ukf,” IEEE/ASME Trans. Mechatronics, vol. 27, no. 4, pp. 1919–1927, 2022. doi: 10.1109/TMECH.2022.3155790
    [9]
    P. Xiang, J. Zhang, D. Sun, K. Qiu, Q. Fang, X. Mi, Y. Wang, R. Xiong, and H. Lu, “Learning-based high-precision force estimation and compliant control for small-scale continuum robot,” IEEE Trans. Automation Science and Engineering, vol. 21, no. 4, pp. 5389–5401, 2024. doi: 10.1109/TASE.2023.3311179
    [10]
    A. Ghoul, K. Kara, M. Benrabah, and M. L. Hadjili, “Optimized nonlinear sliding mode control of a continuum robot manipulator,” Journal of Control, Automation and Electrical Systems, vol. 33, no. 5, pp. 1355–1363, 2022. doi: 10.1007/s40313-022-00914-1
    [11]
    X. Shao, P. Pustina, M. Stölzle, G. Sun, A. De Luca, L. Wu, and C. Della Santina, “Model-based control for soft robots with system uncertainties and input saturation,” IEEE Trans. Industrial Electronics, vol. 71, no. 7, pp. 7435–7444, 2024. doi: 10.1109/TIE.2023.3303636
    [12]
    C. Frazelle, J. Rogers, I. Karamouzas, and I. Walker, “Optimizing a continuum manipulator’s search policy through model-free reinforcement learning,” in 2020 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 5564–5571.
    [13]
    G. Ji, J. Yan, J. Du, W. Yan, J. Chen, Y. Lu, J. Rojas, and S. S. Cheng, “Towards safe control of continuum manipulator using shielded multiagent reinforcement learning,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7461–7468, 2021. doi: 10.1109/LRA.2021.3097660
    [14]
    P. Schegg, E. Menager, x E. Khairallah, D. Marchal, J. Dequidt, P. Preux, and C. Duriez, “Sofagym: An open platform for reinforcement learning based on soft robot simulations,” Soft Robotics, vol. 10, no. 2, pp. 410–430, 2023. doi: 10.1089/soro.2021.0123
    [15]
    D. Jakes, Z. Ge, and L. Wu, “Model-less active compliance for continuum robots using recurrent neural networks,” in 2019 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS). IEEE, 2019, pp. 2167–2173.
    [16]
    N. Tan, P. Yu, Z. Zhong, and Y. Zhang, “Data-driven control for continuum robots based on discrete zeroing neural networks,” IEEE Trans. Industrial Informatics, 2022.
    [17]
    D. Wu, X. T. Ha, Y. Zhang, M. Ourak, G. Borghesan, K. Niu, F. Trauzettel, J. Dankelman, A. Menciassi, and E. Vander Poorten, “Deep-learning-based compliant motion control of a pneumatically-driven robotic catheter,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 8853–8860, 2022. doi: 10.1109/LRA.2022.3186497
    [18]
    Y. Yang, J. Han, Z. Liu, Z. Zhao, and K.-S. Hong, “Modeling and adaptive neural network control for a soft robotic arm with prescribed motion constraints,” IEEE/CAA Journal of Automatica Sinica, vol. 10, no. 2, pp. 501–511, 2023. doi: 10.1109/JAS.2023.123213
    [19]
    G. G. Rigatos, “Model-based and model-free control of flexible-link robots: A comparison between representative methods,” Applied Mathematical Modelling, vol. 33, no. 10, pp. 3906–3925, 2009. doi: 10.1016/j.apm.2009.01.012
    [20]
    M. Jin, J. Lee, and N. G. Tsagarakis, “Model-free robust adaptive control of humanoid robots with flexible joints,” IEEE Trans. Industrial Electronics, vol. 64, no. 2, pp. 1706–1715, 2016.
    [21]
    H. Abouaïssa and S. Chouraqui, “On the control of robot manipulator: A model-free approach,” Journal of Computational Science, vol. 31, pp. 6–16, 2019. doi: 10.1016/j.jocs.2018.12.011
    [22]
    K. He, C. Dong, and Q. Wang, “Active disturbance rejection control for uncertain nonlinear systems with sporadic measurements,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 5, pp. 893–906, 2022. doi: 10.1109/JAS.2022.105566
    [23]
    R. Moradi, R. Berangi, and B. Minaei, “A survey of regularization strategies for deep models,” Artificial Intelligence Review, vol. 53, pp. 3947–3986, 2020. doi: 10.1007/s10462-019-09784-7
    [24]
    T. An, Y. Wang, G. Liu, Y. Li, and B. Dong, “Cooperative game-based approximate optimal control of modular robot manipulators for human–robot collaboration,” IEEE Trans. Cybernetics, vol. 53, no. 7, pp. 4691–4703, 2023. doi: 10.1109/TCYB.2023.3277558
    [25]
    P. M. Wensing, M. Posa, Y. Hu, A. Escande, N. Mansard, and A. Del Prete, “Optimization-based control for dynamic legged robots,” IEEE Trans. Robotics, doi: 10.1109/TRO.2023.3324580, 2023.
    [26]
    F. L. Lewis, D. Vrabie, and V. L. Syrmos, Optimal control. John Wiley & Sons, 2012.
    [27]
    Y. Yang, H. Modares, K. G. Vamvoudakis, and F. L. Lewis, “Cooperative finitely excited learning for dynamical games,” IEEE Trans. Cybernetics, vol. 54, no. 2, pp. 797–810, 2023.
    [28]
    F. Boyer, V. Lebastard, F. Candelier, F. Renda, and M. Alamir, “Statics and dynamics of continuum robots based on cosserat rods and optimal control theories,” IEEE Trans. Robotics, vol. 39, no. 2, pp. 1544–1562, 2022.
    [29]
    M. Samadi Khoshkho, Z. Samadikhoshkho, and M. G. Lipsett, “Distilled neural state-dependent riccati equation feedback controller for dynamic control of a cable-driven continuum robot,” Int. Journal of Advanced Robotic Systems, vol. 20, no. 3, p. 17298806231174737, 2023.
    [30]
    D. A. Haggerty, M. J. Banks, E. Kamenar, A. B. Cao, P. C. Curtis, I. Mezic, x and E. W. Hawkes, “Control of soft robots with inertial dynamics,” Science robotics, vol. 8, no. 81, p. eadd6864, 2023. doi: 10.1126/scirobotics.add6864
    [31]
    D. Bruder, X. Fu, R. B. Gillespie, C. D. Remy, and R. Vasudevan, “Data-driven control of soft robots using koopman operator theory,” IEEE Trans. Robotics, vol. 37, no. 3, pp. 948–961, 2020.
    [32]
    J. Chen, Y. Dang, and J. Han, “Offset-free model predictive control of a soft manipulator using the koopman operator,” Mechatronics, vol. 86, p. 102871, 2022. doi: 10.1016/j.mechatronics.2022.102871
    [33]
    D. Bruder, X. Fu, R. B. Gillespie, C. D. Remy, and R. Vasudevan, “Koopman-based control of a soft continuum manipulator under variable loading conditions,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 6852–6859, 2021. doi: 10.1109/LRA.2021.3095268
    [34]
    R. Luus, “Application of dynamic programming to high-dimensional non-linear optimal control problems,” Int. Journal of Control, vol. 52, no. 1, pp. 239–250, 1990. doi: 10.1080/00207179008953533
    [35]
    D. P. Bertsekas, “Dynamic programming and optimal control,” Belmont, MA: Athena Scientific, 2011.
    [36]
    L. Dong, Y. Tang, H. He, and C. Sun, “An event-triggered approach for load frequency control with supplementary adp,” IEEE Trans. Power Systems, vol. 32, no. 1, pp. 581–589, 2016.
    [37]
    Z. Lin, J. Duan, S. E. Li, H. Ma, Y. Yin, and B. Cheng, “Continuous-time finite-horizon adp for automated vehicle controller design with high efficiency,” in 2020 3rd Int. Conf. on Unmanned Systems (ICUS). IEEE, 2020, pp. 978–984.
    [38]
    H. Yang, Q. Hu, H. Dong, and X. Zhao, “ADP-based spacecraft attitude control under actuator misalignment and pointing constraints,” IEEE Trans. Industrial Electronics, vol. 69, no. 9, pp. 9342–9352, 2021.
    [39]
    S. Li, L. Ding, H. Gao, Y.-J. Liu, L. Huang, and Z. Deng, “ADP-based online tracking control of partially uncertain time-delayed nonlinear system and application to wheeled mobile robots,” IEEE Trans. Cybernetics, vol. 50, no. 7, pp. 3182–3194, 2019.
    [40]
    T. Sun and X.-M. Sun, “An adaptive dynamic programming scheme for nonlinear optimal control with unknown dynamics and its application to turbofan engines,” IEEE Trans. Industrial Informatics, vol. 17, no. 1, pp. 367–376, 2020.
    [41]
    Z. Li, L. Wu, Y. Xu, S. Moazeni, and Z. Tang, “Multi-stage real-time operation of a multi-energy microgrid with electrical and thermal energy storage assets: A data-driven MPC-ADP approach,” IEEE Trans. Smart Grid, vol. 13, no. 1, pp. 213–226, 2021.
    [42]
    Q. Wei, Z. Liao, R. Song, P. Zhang, Z. Wang, and J. Xiao, “Self-learning optimal control for ice-storage air conditioning systems via data-based adaptive dynamic programming,” IEEE Trans. Industrial Electronics, vol. 68, no. 4, pp. 3599–3608, 2020.
    [43]
    D. Wang, M. Zhao, M. Ha, and J. Ren, “Neural optimal tracking control of constrained nonaffine systems with a wastewater treatment application,” Neural Networks, vol. 143, pp. 121–132, 2021. doi: 10.1016/j.neunet.2021.05.027
    [44]
    Y. Yang, B. Kiumarsi, H. Modares, and C. Xu, “Model-free λ-policy iteration for discrete-time linear quadratic regulation,” IEEE Trans. Neural Networks and Learning Systems, vol. 34, no. 2, pp. 635–649, 2021.
    [45]
    W. Xue, B. Lian, J. Fan, P. Kolaric, T. Chai, and F. L. Lewis, “Inverse reinforcement Q-learning through expert imitation for discrete-time systems,” IEEE Trans. Neural Networks and Learning Systems, vol. 34, no. 5, pp. 2386–2399, 2023. doi: 10.1109/TNNLS.2021.3106635
    [46]
    Z. Xia, M. Hu, W. Dai, H. Yan, and X. Ma, “Q-learning-based multi-rate optimal control for process industries,” IEEE Trans. Circuits and Systems Ⅱ: Express Briefs, vol. 70, no. 6, pp. 2006–2010, 2022.
    [47]
    C. Zheng, Y. An, Z. Wang, H. Wu, X. Qin, B. Eynard, and Y. Zhang, “Hybrid offline programming method for robotic welding systems,” Robotics and Computer-Integrated Manufacturing, vol. 73, p. 102238, 2022. doi: 10.1016/j.rcim.2021.102238
    [48]
    J. Fitzgerald, R. M. A. Azad, and C. Ryan, “A bootstrapping approach to reduce over-fitting in genetic programming,” in Proc. the 15th annual conference companion on Genetic and evolutionary computation, 2013, pp. 1113–1120.
    [49]
    Y. Yang, Y. Pan, C.-Z. Xu, and D. C. Wunsch, “Hamiltonian-driven adaptive dynamic programming with efficient experience replay,” IEEE Trans. Neural Networks and Learning Systems, 2022.
    [50]
    V. N. Afanasiev, V. Kolmanovskii, and V. R. Nosov, Mathematical Theory of Control Systems Design. Springer Science & Business Media, 2013, vol. 341.
    [51]
    T. Bian, Y. Jiang, and Z.-P. Jiang, “Adaptive dynamic programming and optimal control of nonlinear nonaffine systems,” Automatica, vol. 50, no. 10, pp. 2624–2632, 2014. doi: 10.1016/j.automatica.2014.08.023
    [52]
    R. Dreyfus, Q. Boehler, and B. J. Nelson, “A simulation framework for magnetic continuum robots,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8370–8376, 2022. doi: 10.1109/LRA.2022.3187249
    [53]
    A. J. Petruska, J. Edelmann, and B. J. Nelson, “Model-based calibration for magnetic manipulation,” IEEE Trans. Magnetics, vol. 53, no. 7, pp. 1–6, 2017.
    [54]
    H. Fang, Y. Tu, H. Wang, S. He, F. Liu, Z. Ding, and S. S. Cheng, “Fuzzy-based adaptive optimization of unknown discrete-time nonlinear markov jump systems with off-policy reinforcement learning,” IEEE Trans. Fuzzy Systems, vol. 30, no. 12, pp. 5276–5290, 2022. doi: 10.1109/TFUZZ.2022.3171844
    [55]
    O. Qasem, W. Gao, and K. G. Vamvoudakis, “Adaptive optimal control of continuous-time nonlinear affine systems via hybrid iteration,” Automatica, vol. 157, p. 111261, 2023. doi: 10.1016/j.automatica.2023.111261
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(6)  / Tables(5)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (15) PDF downloads(5) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return