This survey paper provides a review and perspective on intermediate and advanced reinforcement learning (RL) techniques in process industries. It offers a holistic approach by covering all levels of the process control hierarchy. The survey paper presents a comprehensive overview of RL algorithms, including fundamental concepts like Markov decision processes and different approaches to RL, such as value-based, policy-based, and actor-critic methods, while also discussing the relationship between classical control and RL. It further reviews the wide-ranging applications of RL in process industries, such as soft sensors, low-level control, high-level control, distributed process control, fault detection and fault tolerant control, optimization, planning, scheduling, and supply chain. The survey paper discusses the limitations and advantages, trends and new applications, and opportunities and future prospects for RL in process industries. Moreover, it highlights the need for a holistic approach in complex systems due to the growing importance of digitalization in the process industries.

This paper provides a comprehensive review of the current status, advancements, and future prospects of humanoid robots, highlighting their significance in driving the evolution of next-generation industries. By analyzing various research endeavors and key technologies, encompassing ontology structure, control and decision-making, and perception and interaction, a holistic overview of the current state of humanoid robot research is presented. Furthermore, emerging challenges in the field are identified, emphasizing the necessity for a deeper understanding of biological motion mechanisms, improved structural design, enhanced material applications, advanced drive and control methods, and efficient energy utilization. The integration of bionics, brain-inspired intelligence, mechanics, and control is underscored as a promising direction for the development of advanced humanoid robotic systems. This paper serves as an invaluable resource, offering insightful guidance to researchers in the field, while contributing to the ongoing evolution and potential of humanoid robots across diverse domains.

The power grid is undergoing a transformation from synchronous generators (SGs) toward inverter-based resources (IBRs). The stochasticity, asynchronicity, and limited-inertia characteristics of IBRs bring about challenges to grid resilience. Virtual power plants (VPPs) are emerging technologies to improve the grid resilience and advance the transformation. By judiciously aggregating geographically distributed energy resources (DERs) as individual electrical entities, VPPs can provide capacity and ancillary services to grid operations and participate in electricity wholesale markets. This paper aims to provide a concise overview of the concept and development of VPPs and the latest progresses in VPP operation, with the focus on VPP scheduling and control. Based on this overview, we identify a few potential challenges in VPP operation and discuss the opportunities of integrating the multi-agent system (MAS)-based strategy into the VPP operation to enhance its scalability, performance and resilience.

Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output (MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning control (ILC) scheme based on the zeroing neural networks (ZNNs) is proposed. First, the equivalent dynamic linearization data model is obtained by means of dynamic linearization technology, which exists theoretically in the iteration domain. Then, the iterative extended state observer (IESO) is developed to estimate the disturbance and the coupling between systems, and the decoupled dynamic linearization model is obtained for the purpose of controller synthesis. To solve the zero-seeking tracking problem with inherent tolerance of noise, an ILC based on noise-tolerant modified ZNN is proposed. The strict assumptions imposed on the initialization conditions of each iteration in the existing ILC methods can be absolutely removed with our method. In addition, theoretical analysis indicates that the modified ZNN can converge to the exact solution of the zero-seeking tracking problem. Finally, a generalized example and an application-oriented example are presented to verify the effectiveness and superiority of the proposed process.

Platooning represents one of the key features that connected automated vehicles may possess as it allows multiple automated vehicles to be maneuvered cooperatively with small headways on roads. However, a critical challenge in accomplishing automated vehicle platoons is to deal with the effects of intermittent and sporadic vehicle-to-vehicle data transmissions caused by limited wireless communication resources. This paper addresses the co-design problem of dynamic event-triggered communication scheduling and cooperative adaptive cruise control for a convoy of automated vehicles with diverse spacing policies. The central aim is to achieve automated vehicle platooning under various gap references with desired platoon stability and spacing performance requirements, while simultaneously improving communication efficiency. Toward this aim, a dynamic event-triggered scheduling mechanism is developed such that the inter-vehicle data transmissions are scheduled dynamically and efficiently over time. Then, a tractable co-design criterion on the existence of both the admissible event-driven cooperative adaptive cruise control law and the desired scheduling mechanism is derived. Finally, comparative simulation results are presented to substantiate the effectiveness and merits of the obtained results.

A dandelion algorithm (DA) is a recently developed intelligent optimization algorithm for function optimization problems. Many of its parameters need to be set by experience in DA, which might not be appropriate for all optimization problems. A self-adapting and efficient dandelion algorithm is proposed in this work to lower the number of DA’s parameters and simplify DA’s structure. Only the normal sowing operator is retained; while the other operators are discarded. An adaptive seeding radius strategy is designed for the core dandelion. The results show that the proposed algorithm achieves better performance on the standard test functions with less time consumption than its competitive peers. In addition, the proposed algorithm is applied to feature selection for credit card fraud detection (CCFD), and the results indicate that it can obtain higher classification and detection performance than the-state-of-the-art methods.

Pan-sharpening aims to seek high-resolution multispectral (HRMS) images from paired multispectral images of low resolution (LRMS) and panchromatic (PAN) images, the key to which is how to maximally integrate spatial and spectral information from PAN and LRMS images. Following the principle of gradual advance, this paper designs a novel network that contains two main logical functions, i.e., detail enhancement and progressive fusion, to solve the problem. More specifically, the detail enhancement module attempts to produce enhanced MS results with the same spatial sizes as corresponding PAN images, which are of higher quality than directly up-sampling LRMS images. Having a better MS base (enhanced MS) and its PAN, we progressively extract information from the PAN and enhanced MS images, expecting to capture pivotal and complementary information of the two modalities for the purpose of constructing the desired HRMS. Extensive experiments together with ablation studies on widely-used datasets are provided to verify the efficacy of our design, and demonstrate its superiority over other state-of-the-art methods both quantitatively and qualitatively. Our code has been released at

.

DC-DC converter-based multi-bus DC microgrids (MGs) in series have received much attention, where the conflict between voltage recovery and current balancing has been a hot topic. The lack of models that accurately portray the electrical characteristics of actual MGs while is controller design-friendly has kept the issue active. To this end, this paper establishes a large-signal model containing the comprehensive dynamical behavior of the DC MGs based on the theory of high-order fully actuated systems, and proposes distributed optimal control based on this. The proposed secondary control method can achieve the two goals of voltage recovery and current sharing for multi-bus DC MGs. Additionally, the simple structure of the proposed approach is similar to one based on droop control, which allows this control technique to be easily implemented in a variety of modern microgrids with different configurations. In contrast to existing studies, the process of controller design in this paper is closely tied to the actual dynamics of the MGs. It is a prominent feature that enables engineers to customize the performance metrics of the system. In addition, the analysis of the stability of the closed-loop DC microgrid system, as well as the optimality and consensus of current sharing are given. Finally, a scaled-down solar and battery-based microgrid prototype with maximum power point tracking controller is developed in the laboratory to experimentally test the efficacy of the proposed control method.

In this paper, the issues of stochastic stability analysis and fault estimation are investigated for a class of continuous-time Markov jump piecewise-affine (PWA) systems against actuator and sensor faults. Firstly, a novel mode-dependent PWA iterative learning observer with current feedback is designed to estimate the system states and faults, simultaneously, which contains both the previous iteration information and the current feedback mechanism. The auxiliary feedback channel optimizes the response speed of the observer, therefore the estimation error would converge to zero rapidly. Then, sufficient conditions for stochastic stability with guaranteed

performance are demonstrated for the estimation error system, and the equivalence relations between the system information and the estimated information can be established via iterative accumulating representation. Finally, two illustrative examples containing a class of tunnel diode circuit systems are presented to fully demonstrate the effectiveness and superiority of the proposed iterative learning observer with current feedback.

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation, which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units (RSU) or unmanned aerial vehicles (UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning (MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization (MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers (e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

The pursuit-evasion game models the strategic interaction among players, attracting attention in many realistic scenarios, such as missile guidance, unmanned aerial vehicles, and target defense. Existing studies mainly concentrate on the cooperative pursuit of multiple players in two-dimensional pursuit-evasion games. However, these approaches can hardly be applied to practical situations where players usually move in three-dimensional space with a three-degree-of-freedom control. In this paper, we make the first attempt to investigate the equilibrium strategy of the realistic pursuit-evasion game, in which the pursuer follows a three-degree-of-freedom control, and the evader moves freely. First, we describe the pursuer’s three-degree-of-freedom control and the evader’s relative coordinate. We then rigorously derive the equilibrium strategy by solving the retrogressive path equation according to the Hamilton-Jacobi-Bellman-Isaacs (HJBI) method, which divides the pursuit-evasion process into the navigation and acceleration phases. Besides, we analyze the maximum allowable speed for the pursuer to capture the evader successfully and provide the strategy with which the evader can escape when the pursuer’s speed exceeds the threshold. We further conduct comparison tests with various unilateral deviations to verify that the proposed strategy forms a Nash equilibrium.

Although many multi-view clustering (MVC) algorithms with acceptable performances have been presented, to the best of our knowledge, nearly all of them need to be fed with the correct number of clusters. In addition, these existing algorithms create only the hard and fuzzy partitions for multi-view objects, which are often located in highly-overlapping areas of multi-view feature space. The adoption of hard and fuzzy partition ignores the ambiguity and uncertainty in the assignment of objects, likely leading to performance degradation. To address these issues, we propose a novel sparse reconstructive multi-view evidential clustering algorithm (SRMVEC). Based on a sparse reconstructive procedure, SRMVEC learns a shared affinity matrix across views, and maps multi-view objects to a 2-dimensional human-readable chart by calculating 2 newly defined mathematical metrics for each object. From this chart, users can detect the number of clusters and select several objects existing in the dataset as cluster centers. Then, SRMVEC derives a credal partition under the framework of evidence theory, improving the fault tolerance of clustering. Ablation studies show the benefits of adopting the sparse reconstructive procedure and evidence theory. Besides, SRMVEC delivers effectiveness on benchmark datasets by outperforming some state-of-the-art methods.

In recent years, a large number of approaches to constrained multi-objective optimization problems (CMOPs) have been proposed, focusing on developing tweaked strategies and techniques for handling constraints. However, an overly fine-tuned strategy or technique might overfit some problem types, resulting in a lack of versatility. In this article, we propose a generic search strategy that performs an even search in a promising region. The promising region, determined by obtained feasible non-dominated solutions, possesses two general properties. First, the constrained Pareto front (CPF) is included in the promising region. Second, as the number of feasible solutions increases or the convergence performance (i.e., approximation to the CPF) of these solutions improves, the promising region shrinks. Then we develop a new strategy named even search, which utilizes the non-dominated solutions to accelerate convergence and escape from local optima, and the feasible solutions under a constraint relaxation condition to exploit and detect feasible regions. Finally, a diversity measure is adopted to make sure that the individuals in the population evenly cover the valuable areas in the promising region. Experimental results on 45 instances from four benchmark test suites and 14 real-world CMOPs have demonstrated that searching evenly in the promising region can achieve competitive performance and excellent versatility compared to 11 most state-of-the-art methods tailored for CMOPs.

Unmanned aerial vehicles (UAVs) have gained significant attention in practical applications, especially the low-altitude aerial (LAA) object detection imposes stringent requirements on recognition accuracy and computational resources.  In this paper, the LAA images-oriented tensor decomposition and knowledge distillation-based network (TDKD-Net) is proposed, where the TT-format TD (tensor decomposition) and equal-weighted response-based KD (knowledge distillation) methods are designed to minimize redundant parameters while ensuring comparable performance.  Moreover, some robust network structures are developed, including the small object detection head and the dual-domain attention mechanism, which enable the model to leverage the learned knowledge from small-scale targets and selectively focus on salient features.  Considering the imbalance of bounding box regression samples and the inaccuracy of regression geometric factors, the focal and efficient IoU (intersection of union) loss with optimal transport assignment (F-EIoU-OTA) mechanism is proposed to improve the detection accuracy.  The proposed TDKD-Net is comprehensively evaluated through extensive experiments, and the results have demonstrated the effectiveness and superiority of the developed methods in comparison to other advanced detection algorithms, which also present high generalization and strong robustness.  As a resource-efficient precise network, the complex detection of small and occluded LAA objects is also well addressed by TDKD-Net, which provides useful insights on handling imbalanced issues and realizing domain adaptation.

Binaural rendering is of great interest to virtual reality and immersive media. Although humans can naturally use their two ears to perceive the spatial information contained in sounds, it is a challenging task for machines to achieve binaural rendering since the description of a sound field often requires multiple channels and even the metadata of the sound sources. In addition, the perceived sound varies from person to person even in the same sound field. Previous methods generally rely on individual-dependent head-related transferred function (HRTF) datasets and optimization algorithms that act on HRTFs. In practical applications, there are two major drawbacks to existing methods. The first is a high personalization cost, as traditional methods achieve personalized needs by measuring HRTFs. The second is insufficient accuracy because the optimization goal of traditional methods is to retain another part of information that is more important in perception at the cost of discarding a part of the information. Therefore, it is desirable to develop novel techniques to achieve personalization and accuracy at a low cost. To this end, we focus on the binaural rendering of ambisonic and propose 1) channel-shared encoder and channel-compared attention integrated into neural networks and 2) a loss function quantifying interaural level differences to deal with spatial information. To verify the proposed method, we collect and release the first paired ambisonic-binaural dataset and introduce three metrics to evaluate the content information and spatial information accuracy of the end-to-end methods. Extensive experimental results on the collected dataset demonstrate the superior performance of the proposed method and the shortcomings of previous methods.

The modular system can change its physical structure by self-assembly and self-disassembly between modules to dynamically adapt to task and environmental requirements. Recognizing the adaptive capability of modular systems, we introduce a modular reconfigurable flight array (MRFA) to pursue a multifunction aircraft fitting for diverse tasks and requirements, and investigate the attitude control and the control allocation problem by using the modular reconfigurable flight array as a platform. First, considering the variable and irregular topological configuration of the modular array, a center-of-mass-independent flight array dynamics model is proposed to allow control allocation under over-actuated situations. Secondly, in order to meet the stable, fast and accurate attitude tracking performance of the MRFA, a fixed-time convergent sliding mode controller with state-dependent variable exponent coefficients is proposed to ensure fast convergence rate both away from and near the system equilibrium point without encountering the singularity. It is shown that the controller also has fixed-time convergent characteristics even in the presence of external disturbances. Finally, simulation results are provided to demonstrate the effectiveness of the proposed modeling and control strategies.

Due to its flexibility and complementarity, the multi-UAVs system is well adapted to complex and cramped workspaces, with great application potential in the search and rescue (SAR) and indoor goods delivery fields. However, safe and effective path planning of multiple unmanned aerial vehicles (UAVs) in the cramped environment is always challenging: conflicts with each other are frequent because of high-density flight paths, collision probability increases because of space constraints, and the search space increases significantly, including time scale, 3D scale and model scale. Thus, this paper proposes a hierarchical collaborative planning framework with a conflict avoidance module at the high level and a path generation module at the low level. The enhanced conflict-base search (ECBS) in our framework is improved to handle the conflicts in the global path planning and avoid the occurrence of local deadlock. And both the collision and kinematic models of UAVs are considered to improve path smoothness and flight safety. Moreover, we specifically designed and published the cramped environment test set containing various unique obstacles to evaluating our framework performance thoroughly. Experiments are carried out relying on Rviz, with multiple flight missions: random, opposite, and staggered, which showed that the proposed method can generate smooth cooperative paths without conflict for at least 60 UAVs in a few minutes. The benchmark and source code are released in

.