IEEE/CAA Journal of Automatica Sinica

Visuals to Text: A Comprehensive Review on Automatic Image Captioning

Yue Ming, Nannan Hu, Chunxiao Fan, Fan Feng, Jiangwan Zhou, Hui Yu

2022, 9(8): 1339-1365. doi: 10.1109/JAS.2022.105734

Abstract(4507) HTML (145) PDF(1078)

Abstract:
Image captioning refers to automatic generation of descriptive texts according to the visual content of images. It is a technique integrating multiple disciplines including the computer vision (CV), natural language processing (NLP) and artificial intelligence. In recent years, substantial research efforts have been devoted to generate image caption with impressive progress. To summarize the recent advances in image captioning, we present a comprehensive review on image captioning, covering both traditional methods and recent deep learning-based techniques. Specifically, we first briefly review the early traditional works based on the retrieval and template. Then deep learning-based image captioning researches are focused, which is categorized into the encoder-decoder framework, attention mechanism and training strategies on the basis of model structures and training manners for a detailed introduction. After that, we summarize the publicly available datasets, evaluation metrics and those proposed for specific requirements, and then compare the state of the art methods on the MS COCO dataset. Finally, we provide some discussions on open challenges and future research directions.

Networked Knowledge and Complex Networks: An Engineering View

Jinhu Lü, Guanghui Wen, Ruqian Lu, Yong Wang, Songmao Zhang

2022, 9(8): 1366-1383. doi: 10.1109/JAS.2022.105737

Abstract(1811) HTML (273) PDF(340)

Abstract:
Along with the development of information technologies such as mobile Internet, information acquisition technology, cloud computing and big data technology, the traditional knowledge engineering and knowledge-based software engineering have undergone fundamental changes where the network plays an increasingly important role. Within this context, it is required to develop new methodologies as well as technical tools for network-based knowledge representation, knowledge services and knowledge engineering. Obviously, the term “network” has different meanings in different scenarios. Meanwhile, some breakthroughs in several bottleneck problems of complex networks promote the developments of the new methodologies and technical tools for network-based knowledge representation, knowledge services and knowledge engineering. This paper first reviews some recent advances on complex networks, and then, in conjunction with knowledge graph, proposes a framework of networked knowledge which models knowledge and its relationships with the perspective of complex networks. For the unique advantages of deep learning in acquiring and processing knowledge, this paper reviews its development and emphasizes the role that it played in the development of knowledge engineering. Finally, some challenges and further trends are discussed.

Cyberbullying and Cyberviolence Detection: A Triangular User-Activity-Content View

Shuwen Wang, Xingquan Zhu, Weiping Ding, Amir Alipour Yengejeh

2022, 9(8): 1384-1405. doi: 10.1109/JAS.2022.105740

Abstract(969) HTML (120) PDF(126)

Abstract:
Recent years have witnessed the increasing popularity of mobile and networking devices, as well as social networking sites, where users engage in a variety of activities in the cyberspace on a daily and real-time basis. While such systems provide tremendous convenience and enjoyment for users, malicious usages, such as bullying, cruelty, extremism, and toxicity behaviors, also grow noticeably, and impose significant threats to individuals and communities. In this paper, we review computational approaches for cyberbullying and cyberviolence detection, in order to understand two major factors: 1) What are the defining features of online bullying users, and 2) How to detect cyberbullying and cyberviolence. To achieve the goal, we propose a user-activities-content (UAC) triangular view, which defines that users in the cyberspace are centered around the UAC triangle to carry out activities and generate content. Accordingly, we categorize cyberbully features into three main categories: 1) User centered features, 2) Content centered features, and 3) Activity centered features. After that, we review methods for cyberbully detection, by taking supervised, unsupervised, transfer learning, and deep learning, etc., into consideration. The UAC centered view provides a coherent and complete summary about features and characteristics of online users (their activities), approaches to detect bullying users (and malicious content), and helps defend cyberspace from bullying and toxicity.

Complex-Valued Neural Networks: A Comprehensive Survey

ChiYan Lee, Hideyuki Hasegawa, Shangce Gao

2022, 9(8): 1406-1426. doi: 10.1109/JAS.2022.105743

Abstract(2647) HTML (42) PDF(712)

Abstract:
Complex-valued neural networks (CVNNs) have shown their excellent efficiency compared to their real counterparts in speech enhancement, image and signal processing. Researchers throughout the years have made many efforts to improve the learning algorithms and activation functions of CVNNs. Since CVNNs have proven to have better performance in handling the naturally complex-valued data and signals, this area of study will grow and expect the arrival of some effective improvements in the future. Therefore, there exists an obvious reason to provide a comprehensive survey paper that systematically collects and categorizes the advancement of CVNNs. In this paper, we discuss and summarize the recent advances based on their learning algorithms, activation functions, which is the most challenging part of building a CVNN, and applications. Besides, we outline the structure and applications of complex-valued convolutional, residual and recurrent neural networks. Finally, we also present some challenges and future research directions to facilitate the exploration of the ability of CVNNs.

Position Encoding Based Convolutional Neural Networks for Machine Remaining Useful Life Prediction

Ruibing Jin, Min Wu, Keyu Wu, Kaizhou Gao, Zhenghua Chen, Xiaoli Li

2022, 9(8): 1427-1439. doi: 10.1109/JAS.2022.105746

Abstract(2571) HTML (38) PDF(300)

Abstract:
Accurate remaining useful life (RUL) prediction is important in industrial systems. It prevents machines from working under failure conditions, and ensures that the industrial system works reliably and efficiently. Recently, many deep learning based methods have been proposed to predict RUL. Among these methods, recurrent neural network (RNN) based approaches show a strong capability of capturing sequential information. This allows RNN based methods to perform better than convolutional neural network (CNN) based approaches on the RUL prediction task. In this paper, we question this common paradigm and argue that existing CNN based approaches are not designed according to the classic principles of CNN, which reduces their performances. Additionally, the capacity of capturing sequential information is highly affected by the receptive field of CNN, which is neglected by existing CNN based methods. To solve these problems, we propose a series of new CNNs, which show competitive results to RNN based methods. Compared with RNN, CNN processes the input signals in parallel so that the temporal sequence is not easily determined. To alleviate this issue, a position encoding scheme is developed to enhance the sequential information encoded by a CNN. Hence, our proposed position encoding based CNN called PE-Net is further improved and even performs better than RNN based methods. Extensive experiments are conducted on the C-MAPSS dataset, where our PE-Net shows state-of-the-art performance.

Consensus Control of Multi-Agent Systems Using Fault-Estimation-in-the-Loop: Dynamic Event-Triggered Case

Yamei Ju, Derui Ding, Xiao He, Qing-Long Han, Guoliang Wei

2022, 9(8): 1440-1451. doi: 10.1109/JAS.2021.1004386

Abstract(1077) HTML (168) PDF(156)

Abstract:
The paper develops a novel framework of consensus control with fault-estimation-in-the-loop for multi-agent systems (MASs) in the presence of faults. A dynamic event-triggered protocol (DETP) by adding an auxiliary variable is utilized to improve the utilization of communication resources. First, a novel estimator with a noise bias is put forward to estimate the existed fault and then a consensus controller with fault compensation (FC) is adopted to realize the demand of reliability and safety of addressed MASs. Subsequently, a novel consensus control framework with fault-estimation-in-the-loop is developed to achieve the predetermined consensus performance with the $l_{2}$-$l_{\infty}$ constraint by employing the variance analysis and the Lyapunov stability approaches. Furthermore, the desired estimator and controller gains are obtained in light of the solution to an algebraic matrix equation and a linear matrix inequality in a recursive way, respectively. Finally, a simulation result is employed to verify the usefulness of the proposed design framework.

Gradient-Based Differential kWTA Network With Application to Competitive Coordination of Multiple Robots

Mei Liu, Xiaoyan Zhang, Mingsheng Shang, Long Jin

2022, 9(8): 1452-1463. doi: 10.1109/JAS.2022.105731

Abstract(6632) HTML (42) PDF(81)

Abstract:
Aiming at the k-winners-take-all (kWTA) operation, this paper proposes a gradient-based differential kWTA (GD-kWTA) network. After obtaining the network, theorems and related proofs are provided to guarantee the exponential convergence and noise resistance of the proposed GD-kWTA network. Then, numerical simulations are conducted to substantiate the preferable performance of the proposed network as compared with the traditional ones. Finally, the GD-kWTA network, backed with a consensus filter, is utilized as a robust control scheme for modeling the competition behavior in the multi-robot coordination, thereby further demonstrating its effectiveness and feasibility.

Maneuvering Angle Rigid Formations With Global Convergence Guarantees

Liangming Chen, Zhiyun Lin, Hector Garcia de Marina, Zhiyong Sun, Mir Feroskhan

2022, 9(8): 1464-1475. doi: 10.1109/JAS.2022.105749

Abstract(654) HTML (137) PDF(56)

Abstract:
Angle rigid multi-agent formations can simultaneously undergo translational, rotational, and scaling maneuvering, therefore combining the maneuvering capabilities of both distance and bearing rigid formations. However, maneuvering angle rigid formations in 2D or 3D with global convergence guarantees is shown to be a challenging problem in the existing literature even when relative position measurements are available. Motivated by angle-induced linear equations in 2D triangles and 3D tetrahedra, this paper aims to solve this challenging problem in both 2D and 3D under a leader-follower framework. For the 2D case where the leaders have constant velocities, by using local relative position and velocity measurements, a formation maneuvering law is designed for the followers governed by double-integrator dynamics. When the leaders have time-varying velocities, a sliding mode formation maneuvering law is proposed by using the same measurements. For the 3D case, to establish an angle-induced linear equation for each tetrahedron, we assume that all the followers’ coordinate frames share a common Z direction. Then, a formation maneuvering law is proposed for the followers to globally maneuver Z-weakly angle rigid formations in 3D. The extension to Lagrangian agent dynamics and the construction of the desired rigid formations by using the minimum number of angle constraints are also discussed. Simulation examples are provided to validate the effectiveness of the proposed algorithms.

A Novel Multiobjective Fireworks Algorithm and Its Applications to Imbalanced Distance Minimization Problems

Shoufei Han, Kun Zhu, MengChu Zhou, Xiaojing Liu, Haoyue Liu, Yusuf Al-Turki, Abdullah Abusorrah

2022, 9(8): 1476-1489. doi: 10.1109/JAS.2022.105752

Abstract(471) HTML (30) PDF(52)

Abstract:
Recently, multimodal multiobjective optimization problems (MMOPs) have received increasing attention. Their goal is to find a Pareto front and as many equivalent Pareto optimal solutions as possible. Although some evolutionary algorithms for them have been proposed, they mainly focus on the convergence rate in the decision space while ignoring solutions diversity. In this paper, we propose a new multiobjective fireworks algorithm for them, which is able to balance exploitation and exploration in the decision space. We first extend a latest single-objective fireworks algorithm to handle MMOPs. Then we make improvements by incorporating an adaptive strategy and special archive guidance into it, where special archives are established for each firework, and two strategies (i.e., explosion and random strategies) are adaptively selected to update the positions of sparks generated by fireworks with the guidance of special archives. Finally, we compare the proposed algorithm with eight state-of-the-art multimodal multiobjective algorithms on all 22 MMOPs from CEC2019 and several imbalanced distance minimization problems. Experimental results show that the proposed algorithm is superior to compared algorithms in solving them. Also, its runtime is less than its peers’.

A Novel PDF Shape Control Approach for Nonlinear Stochastic Systems

Lingzhi Wang, Guo Xie, Fucai Qian, Jun Liu, Kun Zhang

2022, 9(8): 1490-1498. doi: 10.1109/JAS.2022.105755

Abstract(667) HTML (58) PDF(71)

Abstract:
In this work, a novel shape control approach of the probability density function (PDF) for nonlinear stochastic systems is presented. First, we provide the formula for the PDF shape controller without devising the control law of the controller. Then, based on the exact analytical solution of the Fokker-Planck-Kolmogorov (FPK) equation, the product function of the polynomial and the exponential polynomial is regarded as the stationary PDF of the state response. To validate the performance of the proposed control approach, we compared it with the exponential polynomial method and the multi-Gaussian closure method by implementing comparative simulation experiments. The results show that the novel PDF shape control approach is effective and feasible. Using an equal number of parameters, our method can achieve a similar or better control effect as the exponential polynomial method. By comparison with the multi-Gaussian closure method, our method has clear advantages in PDF shape control performance. For all cases, the integral of squared error and the errors of first four moments of our proposed method were very small, indicating superior performance and promising good overall control effects of our method. The approach presented in this study provides an alternative for PDF shape control in nonlinear stochastic systems.

A PD-Type State-Dependent Riccati Equation With Iterative Learning Augmentation for Mechanical Systems

Saeed Rafee Nekoo, José Ángel Acosta, Guillermo Heredia, Anibal Ollero

2022, 9(8): 1499-1511. doi: 10.1109/JAS.2022.105533

Abstract(847) HTML (39) PDF(128)

Abstract:
This work proposes a novel proportional-derivative (PD)-type state-dependent Riccati equation (SDRE) approach with iterative learning control (ILC) augmentation. On the one hand, the PD-type control gains could adopt many useful available criteria and tools of conventional PD controllers. On the other hand, the SDRE adds nonlinear and optimality characteristics to the controller, i.e., increasing the stability margins. These advantages with the ILC correction part deliver a precise control law with the capability of error reduction by learning. The SDRE provides a symmetric-positive-definite distributed nonlinear suboptimal gain K(x) for the control input law u = –R^–1(x)B^T(x)K(x)x. The sub-blocks of the overall gain R^–1(x)B^T(x)K(x), are not necessarily symmetric positive definite. A new design is proposed to transform the optimal gain into two symmetric-positive-definite gains like PD-type controllers as u = –K_SP(x)e–K_SD(x)ė. The new form allows us to analytically prove the stability of the proposed learning-based controller for mechanical systems; and presents guaranteed uniform boundedness in finite-time between learning loops. The symmetric PD-type controller is also developed for the state-dependent differential Riccati equation (SDDRE) to manipulate the final time. The SDDRE expresses a differential equation with a final boundary condition, which imposes a constraint on time that could be used for finite-time control. So, the availability of PD-type finite-time control is an asset for enhancing the conventional classical linear controllers with this tool. The learning rules benefit from the gradient descent method for both regulation and tracking cases. One of the advantages of this approach is a guaranteed-stability even from the first loop of learning. A mechanical manipulator, as an illustrative example, was simulated for both regulation and tracking problems. Successful experimental validation was done to show the capability of the system in practice by the implementation of the proposed method on a variable-pitch rotor benchmark.