IEEE/CAA Journal of Automatica Sinica

Development and Control of Underwater Gliding Robots: A Review

Jian Wang, Zhengxing Wu, Huijie Dong, Min Tan, Junzhi Yu

2022, 9(9): 1543-1560. doi: 10.1109/JAS.2022.105671

Abstract(1653) HTML (57) PDF(354)

Abstract:
As one of the most effective vehicles for ocean development and exploration, underwater gliding robots (UGRs) have the unique characteristics of low energy consumption and strong endurance. Moreover, by borrowing the motion principles of current underwater robots, a variety of novel UGRs have emerged with improving their maneuverability, concealment, and environmental friendliness, which significantly broadens the ocean applications. In this paper, we provide a comprehensive review of underwater gliding robots, including prototype design and their key technologies. From the perspective of motion characteristics, we categorize the underwater gliding robots in terms of traditional underwater gliders (UGs), hybrid-driven UGs, bio-inspired UGs, thermal UGs, and others. Correspondingly, their buoyancy driven system, dynamic and energy model, and motion control are concluded with detailed analysis. Finally, we have discussed the current critical issues and future development. This review offers valuable insight into the development of next-generation underwater robots well-suited for various oceanic applications, and aims to gain more attention of researchers and engineers to this growing field.

Reinforcement Learning Behavioral Control for Nonlinear Autonomous System

Zhenyi Zhang, Zhibin Mo, Yutao Chen, Jie Huang

2022, 9(9): 1561-1573. doi: 10.1109/JAS.2022.105797

Abstract(782) HTML (127) PDF(217)

Abstract:
Behavior-based autonomous systems rely on human intelligence to resolve multi-mission conflicts by designing mission priority rules and nonlinear controllers. In this work, a novel two-layer reinforcement learning behavioral control (RLBC) method is proposed to reduce such dependence by trial-and-error learning. Specifically, in the upper layer, a reinforcement learning mission supervisor (RLMS) is designed to learn the optimal mission priority. Compared with existing mission supervisors, the RLMS improves the dynamic performance of mission priority adjustment by maximizing cumulative rewards and reducing hardware storage demand when using neural networks. In the lower layer, a reinforcement learning controller (RLC) is designed to learn the optimal control policy. Compared with existing behavioral controllers, the RLC reduces the control cost of mission priority adjustment by balancing control performance and consumption. All error signals are proved to be semi-globally uniformly ultimately bounded (SGUUB). Simulation results show that the number of mission priority adjustment and the control cost are significantly reduced compared to some existing mission supervisors and behavioral controllers, respectively.

Adaptive Consensus Quantized Control for a Class of High-Order Nonlinear Multi-Agent Systems With Input Hysteresis and Full State Constraints

Guoqiang Zhu, Haoqi Li, Xiuyu Zhang, Chenliang Wang, Chun-Yi Su, Jiangping Hu

2022, 9(9): 1574-1589. doi: 10.1109/JAS.2022.105800

Abstract(810) HTML (226) PDF(172)

Abstract:
For a class of high-order nonlinear multi-agent systems with input hysteresis, an adaptive consensus output-feedback quantized control scheme with full state constraints is investigated. The major properties of the proposed control scheme are: 1) According to the different hysteresis input characteristics of each agent in the multi-agent system, a hysteresis quantization inverse compensator is designed to eliminate the influence of hysteresis characteristics on the system while ensuring that the quantized signal maintains the desired value. 2) A barrier Lyapunov function is introduced for the first time in the hysteretic multi-agent system. By constructing state constraint control strategy for the hysteretic multi-agent system, it ensures that all the states of the system are always maintained within a predetermined range. 3) The designed adaptive consensus output-feedback quantization control scheme allows the hysteretic system to have unknown parameters and unknown disturbance, and ensures that the input signal transmitted between agents is the quantization value, and the introduced quantizer is implemented under the condition that only its sector bound property is required. The stability analysis has proved that all signals of the closed-loop are semi-globally uniformly bounded. The StarSim hardware-in-the-loop simulation certificates the effectiveness of the proposed adaptive quantized control scheme.

Hierarchical Cooperative Control of Connected Vehicles: From Heterogeneous Parameters to Heterogeneous Structures

Manjiang Hu, Lingkun Bu, Yougang Bian, Hongmao Qin, Ning Sun, Dongpu Cao, Zhihua Zhong

2022, 9(9): 1590-1602. doi: 10.1109/JAS.2022.105536

Abstract(837) HTML (38) PDF(148)

Abstract:
As one of the typical applications of connected vehicles (CVs), the vehicle platoon control technique has been proven to have the advantages of reducing emissions, improving traffic throughout and driving safety. In this paper, a unified hierarchical framework is designed for cooperative control of CVs with both heterogeneous model parameters and structures. By separating neighboring information interaction from local dynamics control, the proposed framework is designed to contain an upper-level observing layer and a lower-level tracking control layer, which helps address the heterogeneity in vehicle parameters and structures. Within the proposed framework, an observer is designed for following vehicles to observe the leading vehicle’s states using neighboring communication, while a tracking controller is designed to track the observed leading vehicle using local feedback control. Closed-loop stability in the absence and presence of communication time delay is analyzed, and the observer is further extended to a finite time convergent one to address string stability under general communication topology. Numerical simulation and field experiment verify the effectiveness of the proposed method.

Order-Preserved Preset-Time Cooperative Control: A Monotone System-Based Approach

Boda Ning, Qing-Long Han

2022, 9(9): 1603-1611. doi: 10.1109/JAS.2022.105440

Abstract(357) HTML (116) PDF(80)

Abstract:
This paper is concerned with order-preserved preset-time cooperative control of multi-agent systems with directed graphs. A novel monotone system-based approach is proposed to preserve the initial order of agents while guaranteeing the preset-time state agreement. Specifically, three different distributed controllers together with sufficient conditions are designed to realize leaderless consensus, leader-following consensus, and containment control, respectively. The proposed controllers facilitate preset-time deployment of agents in practical scenarios with collision avoidance requirement. Comparison studies through a numerical example are carried out to illustrate the effectiveness of the proposed controllers.

Multi-Modal Domain Adaptation Variational Autoencoder for EEG-Based Emotion Recognition

Yixin Wangs, Shuang Qiu, Dan Li, Changde Du, Bao-Liang Lu, Huiguang He

2022, 9(9): 1612-1626. doi: 10.1109/JAS.2022.105515

Abstract(892) HTML (69) PDF(163)

Abstract:
Traditional electroencephalograph (EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject, which restricts the application of the affective brain computer interface (BCI) in practice. We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples. To solve this problem, we propose a multi-modal domain adaptive variational autoencoder (MMDA-VAE) method, which learns shared cross-domain latent representations of the multi-modal data. Our method builds a multi-modal variational autoencoder (MVAE) to project the data of multiple modalities into a common space. Through adversarial learning and cycle-consistency regularization, our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge. Extensive experiments are conducted on two public datasets, SEED and SEED-IV, and the results show the superiority of our proposed method. Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.

Adaptive Containment Control for Fractional-Order Nonlinear Multi-Agent Systems With Time-Varying Parameters

Yang Liu, Huaguang Zhang, Yingchun Wang, Hongjing Liang

2022, 9(9): 1627-1638. doi: 10.1109/JAS.2022.105545

Abstract(535) HTML (54) PDF(99)

Abstract:
This paper investigates adaptive containment control for a class of fractional-order multi-agent systems (FOMASs) with time-varying parameters and disturbances. By using the bounded estimation method, the difficulty generated by the time-varying parameters and disturbances is overcome. The command filter is introduced to solve the complexity problem inherent in adaptive backstepping control. Meanwhile, in order to eliminate the effect of filter errors, a novel distributed error compensating scheme is constructed, in which only the local information from the neighbor agents is utilized. Then, a distributed adaptive containment control scheme for FOMASs is developed based on backstepping to guarantee that the outputs of all the followers are steered to the convex hull spanned by the leaders. Based on the extension of Barbalat’s lemma to fractional-order integrals, it can be proven that the containment errors and the compensating signals have asymptotic convergence. Finally, three simulation examples are given to show the feasibility and effectiveness of the proposed control method.

Autonomous Maneuver Decisions via Transfer Learning Pigeon-Inspired Optimization for UCAVs in Dogfight Engagements

Wanying Ruan, Haibin Duan, Yimin Deng

2022, 9(9): 1639-1657. doi: 10.1109/JAS.2022.105803

Abstract(678) HTML (68) PDF(107)

Abstract:
This paper proposes an autonomous maneuver decision method using transfer learning pigeon-inspired optimization (TLPIO) for unmanned combat aerial vehicles (UCAVs) in dogfight engagements. Firstly, a nonlinear F-16 aircraft model and automatic control system are constructed by a MATLAB/Simulink platform. Secondly, a 3-degrees-of-freedom (3-DOF) aircraft model is used as a maneuvering command generator, and the expanded elemental maneuver library is designed, so that the aircraft state reachable set can be obtained. Then, the game matrix is composed with the air combat situation evaluation function calculated according to the angle and range threats. Finally, a key point is that the objective function to be optimized is designed using the game mixed strategy, and the optimal mixed strategy is obtained by TLPIO. Significantly, the proposed TLPIO does not initialize the population randomly, but adopts the transfer learning method based on Kullback-Leibler (KL) divergence to initialize the population, which improves the search accuracy of the optimization algorithm. Besides, the convergence and time complexity of TLPIO are discussed. Comparison analysis with other classical optimization algorithms highlights the advantage of TLPIO. In the simulation of air combat, three initial scenarios are set, namely, opposite, offensive and defensive conditions. The effectiveness performance of the proposed autonomous maneuver decision method is verified by simulation results.

Interval Type-2 Fuzzy Hierarchical Adaptive Cruise Following-Control for Intelligent Vehicles

Hong Mo, Yinghui Meng, Fei-Yue Wang, Dongrui Wu

2022, 9(9): 1658-1672. doi: 10.1109/JAS.2022.105806

Abstract(798) HTML (52) PDF(116)

Abstract:
Intelligent vehicles can effectively improve traffic congestion and road traffic safety. Adaptive cruise following-control (ACFC) is a vital part of intelligent vehicles. In this paper, a new hierarchical vehicle-following control strategy is presented by synthesizing the variable time headway model, type-2 fuzzy control, feedforward + fuzzy proportion integration (PI) feedback (F+FPIF) control, and inverse longitudinal dynamics model of vehicles. Firstly, a traditional variable time headway model is improved considering the acceleration of the lead car. Secondly, an interval type-2 fuzzy logic controller (IT2 FLC) is designed for the upper structure of the ACFC system to simulate the driver’s operating habits. To reduce the nonlinear influence and improve the tracking accuracy for the desired acceleration, the control strategy of F+FPIF is given for the lower control structure. Thirdly, the lower control method proposed in this paper is compared with the fuzzy PI control and the traditional method (no lower controller for tracking desired acceleration) separately. Meanwhile, the proportion integration differentiation (PID), linear quadratic regulator (LQR), subsection function control (SFC) and type-1 fuzzy logic control (T1 FLC) are respectively compared with the IT2 FLC in control performance under different scenes. Finally, the simulation results show the effectiveness of IT2 FLC for the upper structure and F+FPIF control for the lower structure.

Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features

Wenzhang Liu, Lu Dong, Dan Niu, Changyin Sun

2022, 9(9): 1673-1686. doi: 10.1109/JAS.2022.105809

Abstract(878) HTML (160) PDF(141)

Abstract:
In multi-agent reinforcement learning (MARL), the behaviors of each agent can influence the learning of others, and the agents have to search in an exponentially enlarged joint-action space. Hence, it is challenging for the multi-agent teams to explore in the environment. Agents may achieve suboptimal policies and fail to solve some complex tasks. To improve the exploring efficiency as well as the performance of MARL tasks, in this paper, we propose a new approach by transferring the knowledge across tasks. Differently from the traditional MARL algorithms, we first assume that the reward functions can be computed by linear combinations of a shared feature function and a set of task-specific weights. Then, we define a set of basic MARL tasks in the source domain and pre-train them as the basic knowledge for further use. Finally, once the weights for target tasks are available, it will be easier to get a well-performed policy to explore in the target domain. Hence, the learning process of agents for target tasks is speeded up by taking full use of the basic knowledge that was learned previously. We evaluate the proposed algorithm on two challenging MARL tasks: cooperative box-pushing and non-monotonic predator-prey. The experiment results have demonstrated the improved performance compared with state-of-the-art MARL algorithms.