Stochastic Iterative Learning Control With Faded Signals

Ⅰ. INTRODUCTION

Iterative learning control (ILC) is an intelligent control strategy applied to repetitive systems. When it comes to repetitive work, humans can learn from past experiences and correct operations for the next attempt so that the desired goal can be gradually achieved. Inspired by this concept of learning, Arimoto et al. proposed ILC in 1984 [1]. The basic mechanism is to use the previous control profiles and tracking errors for correcting control effect, so that system output can converge to the desired trajectory asymptotically as the iteration number increases. After more than three decades of rapid developments, ILC has made significant progress in both theory and applications [2]–[5], such as randomly varying iteration lengths system [6]–[9], networked ILC [10]–[12], multi-agent systems [13], [14], quantized ILC [15]–[17]. It can be seen from this literature that ILC is fairly effective in solving repetitive tracking task because of the introduction of learning concept to conventional control methodologies.

With the rapid developments of network communication technology, an continuously increasing number of systems have adopted a networked control structure. Networked control structure means that the plants and controllers are located in different places and they communicate with each other through wireless or wired networks. This structure allows the controller to work at a distance from the plants, thus keeps operators away from dangerous and terrible operation environments. In addition, networked control structure can improve flexibility, stability, and robustness of the whole control system. However, this control structure also brings the risk of data destruction during data transmission through remote communications. In other words, data transmission over wireless or wired networks suffers from various forms of data corruption such as packet loss and data fading.

In this paper, we focus on the case that data are transmitted over fading channels, resulting in imprecise signals modeled by multiplicative randomness [18]–[22]. Fading channels, which refer to the attenuation of transmission signal amplitude caused by channel changes, are common in wireless communication networks due to reflection, refraction, and diffraction during propagation travel. This topic has been investigated in the existing literature using conventional control strategies. In [18], a remote mean-square stabilization problem was studied for a MIMO (multi-input-multi-output) system, where each actuator and sensor had independent fading channels. Considering state feedback control, a mean-square capacity for stabilization of a single-input system was given by a sum of logarithmic magnitudes of plant's unstable eigenvalues. Extensions to a MIMO system with multiple fading channels at the input side were conducted in [19] using a linear control strategy. In addition, robust stability for multi-topic uncertain systems was presented in [23] and input power constraints were studied in [20] for linear systems in the presence of fading randomness. Because of the destruction caused by fading channels, the received signals usually deviate from their original values in a multiplicative form. If received signals are not corrected before usage, the controlled systems may become unstable. The observation motivates the Kalman filtering issue with faded measurements [21], [22]. Generally, boundedness of filtered signals and convergence to steady states are addressed in the existing studies. In [21], The authors verified the boundedness of the expected error covariance matrix generated by a conventional Kalman filter and proved its convergence to a stable state. In addition, for scalar measurements with several specific fading distributions, the paper provided upper bounds on the expected error covariance. The Tobit Kalman filtering problem was studied in [22] for discrete-time systems with both censored and fading measurements. In summary of this literature, it is seen that the research mainstream is yet limited in several directions. Currently, few results have been reported on the learning tracking control in the presence of fading channels. This observation motivates us to propose an effective ILC scheme to achieve acceptable tracking performance.

In consideration of fading channels, an intuitive difficulty is to treat randomness caused by faded signals besides possible system noises. To solve this difficulty, it is of significant importance to select a proper learning gain matrix according to specified system performance. In this case, the traditional fixed learning gain matrix is not suitable for achieving a perfect tracking performance in the presence of various types of randomness. For example, the update law in [9] with Arimoto-like and causal type of invariant gain matrices is able to ensure almost sure and mean-square convergence in the presence of randomly varying iteration lengths. However, we observe from simulations that the fixed gain matrix fails to converge when additional random noises are involved in the system. Indeed, there are a lot of papers addressing the design issue of varying learning gain matrix regarding different control topics. In [24], the author proposed a recursive calculation method for the gain matrix by minimizing the input error covariance with respect to it, which actually is a Kalman filtering framework. This framework can handle various randomness in discrete time systems effectively. For example, in [25], [26], the authors successfully applied the Kalman filtering framework for stochastic linear systems in the presence of random data dropouts. Experimental results show that the output can converge to the desired trajectory as long as the data can be transmitted with a certain probability. It is observed from the above literature that the Kalman filtering theory is useful to enhance the tracking performance against stochastic factors. This observation motivates us to employ the framework in [24]–[26] to solve learning tracking problem of stochastic linear systems through fading channels. We would note that adaptive critic control is another learning mechanism for nonlinear systems with uncertainties [27], [28], while the approach in this paper focuses on the iteration-wise utilization of the previous experience rather than the time-wise improvement in [27], [28].

In this paper, we study an ILC problem for stochastic linear systems through fading channels. The fading randomness is modeled by a random variable with known mean and variance. A recursive learning gain matrix is derived from the Kalman filter technique and then is used in the learning control algorithm to generate input sequences. The resulted tracking error is proved convergent to zero in mean-square sense asymptotically along the iteration axis. Differing from the existing results regarding the fading phenomenon, this paper proposes a primary result on the learning control issue over fading channels using the Kalman filtering method. Both output and input fading cases are accounted for. Strict analysis and illustrative simulations are provided to demonstrate the effectiveness of the proposed scheme.

The main contributions can be summarized as follows:

1) The learning tracking problem for stochastic linear systems through fading channels is studied for the first time. This point differs from the existing results of fading channels concerning for filtering and stability.

2) The fading channels at both output and input sides are addressed separately with a detailed analysis of the differences between the two cases. This point helps understand the influence of fading randomness involved with output and input, respectively.

3) A two-dimensional Kalman filtering approach is used to establish a recursive computation of the learning gain matrix along both time and iteration axes.

The rest of this paper is organized as follows: We give problem formulation in Section Ⅱ. In Section Ⅲ, we focus on output fading case. The computation of iterative learning gain matrix and the convergence of corresponding learning control algorithms are detailed. In the Section Ⅳ, the results of input fading case are presented. Extensions to the general case is given in Section Ⅴ. Illustrative simulations are presented in Section Ⅵ. Section Ⅶ concludes the paper.

Notations: The set of real numbers and the $n$-dimensional space are denoted by $\mathbb{R}$ and $\mathbb{R}^n$, respectively. ${P}$ denotes the probability of its indicated event and ${E}$ is the mathematical expectation operator.

Ⅱ. PROBLEM FORMULATION

Consider the following MIMO stochastic linear time-varying system:

where subscript $k=1, 2, \ldots$ is the iteration number, and argument $t$ is the time label ($t=1, 2, \ldots, N$, where $N$ is the iteration length). Variables $x_k(t)\in\mathbb{R}^n$, $u_k(t)\in\mathbb{R}^p$, and $y_k(t)$ $\in$ $\mathbb{R}^q$ are state, input, and output, respectively. System matrices $A(t)$, $B(t)$, and $C(t)$ are with appropriate dimensions with respect to system variables. Variables $\omega_k(t)$ and $\upsilon_k(t)$ are system noise and measurement noise, respectively. Without loss of generality, we assume that the matrix $C(t+1)B(t)$ is of full column rank, implying that the system relative degree is one and the input dimension is not greater than the output dimension. This assumption is commonly satisfied by many practical systems and necessary to a certain extent [29].

Before further analysis, we need the following assumptions.

Assumption 1: The desired trajectory $y_d(t)$ is realizable in the sense that there exist suitable initial state $x_d(0)$ and desired input $u_d(t)$ such that

Thus, we can obtain a unique input regarding the desired trajectory as follows:

Assumption 2: The initial input error $u_d(t)-u_0(t)$, $\forall t$, and the initial state error $x_d(0)-x_k(0)$, $\forall k$, are assumed to be zero-mean white noise. Moreover, ${E}((u_d(t)-u_0(t))(u_d(t)-u_0(t))^T)$ is a symmetrical positive-definite matrix and ${E}((x_d(0)-x_k(0))(x_d(0)-x_k(t))^T)$ is a positive-semidefinite matrix. In addition, $x_d(0)-x_k(0)$ is uncorrelated with $u_d(t)-u_0(t)$, $\omega_k(0)$ and $\upsilon_k(0)$.

Assumption 3: The system noise $\omega_k(t)$ and measurement noises $\upsilon_k(t)$ are sequences of independent zero-mean white Gaussian noise such that ${E}(\omega_k(t)\omega_k^T(t))= Q_{11, t}$ is a positive-semidefinite matrix and ${E}(\upsilon_k(t)\upsilon_k^T(t))= Q_{22, t}$ is a positive-definite matrix for all $k$. Moreover, $\omega_k(t)$ and $\upsilon_k(t)$ are uncorrelated.

The networked system setup is illustrated in Fig. 1, where plant and controller communicate with each other by fading networks. In this implementation, the signals can be affected during the transmission over fading channels. The fading effect in this paper is modeled by a random variable referred to as a fading gain, which is multiplied to original signals. Assume that the channels undergo stationary fading such that statistical information of this fading gain is known and compensated for at the receiver. Specifically, we assume the random fading gain to be independent and identically distributed with respect to $k$ and $t$, according to a continuous fading distribution function $f(\theta)$ such that ${P}(\theta_k(t)> 0)=1$. In addition, we assume that $\theta_k(t)$ has the following statistical properties

Figure  1.  Block diagram of ILC with fading networks

DownLoad: Full-Size Img PowerPoint

Remark 1: In this paper, we assume that the statistics of the fading channel are known. This hypothesis has been widely used in many papers. The fading problem is mainly determined by network equipment and physical environments. Therefore, it is reasonable for us to obtain relatively precise information of the fading gain $\theta_k(t)$. If statistical information is unavailable in advance, we can estimate it through some online identification strategies such as sending a unit signal at the being of each iteration and calculating the empirical estimation. However, this topic is beyond the scope of this paper and we omit it for the sake of brevity.

Because fading channels introduce multiplicative randomness to signals making the received signal inaccurate; thus, the learning control algorithm and system operation are affected by these inaccurate signals. Consequently, it is important to provide a suitable learning process against such randomness. The control objective is to design an optimal learning gain matrix associated with necessary correction for received signals, such that the system output converges to the desired trajectory asymptotically as the iteration number increases. However, because of the existence of random noises in system operation and measurements, it is impossible to derive the conclusion $y_k(t)$ $\rightarrow y_d(t)$ as $k\rightarrow \infty$, $\forall t$. The best achievable performance is that output can converge to the desired trajectory if noises are eliminated. This objective can be achieved as long as we can ensure $u_k(t)\rightarrow u_d(t)$ as $k \rightarrow \infty$, $\forall t$. This is the analysis objective in this paper.

In this paper, we consider two fading cases in turn: the fading channel at the output side and that at the input side. The detailed design and analysis procedures are given in the following two sections, respectively. The separation of fading channels is to make the layout concise and our main idea easy to follow. Extensions to the general case that output and input fading are included can be completed following the same method.

Ⅲ. OUTPUT FADING CASE

In this section, we present results for the output fading case. In particular, only the channel from plant to controller suffers from random fading, and the channel from controller to plant is assumed to work without fading.

A Algorithm Design

The actual output is denoted by $y_k(t)$. The received signal by the controller is denoted by $y^\circ_k(t)$. Then, according to the fading model, we have

It can be seen that the received output signal cannot be directly applied in learning algorithms because it has deviated from the true value. To establish an effective learning algorithm, we modify the received signal by multiplying a mean inverse of the fading gain

Thus, $\hat{y}_k(t)$ is corrected output signal. The learning update is given by

where $K_k(t)$ is a learning control gain matrix with dimension $p\times q$. In order to simplify the writing, we denote input and state error vectors by $\Delta u_k(t)= u_d(t)-u_k(t)$ and $\Delta x_k(t)= x_d(t)-x_k(t)$, respectively. For (1), (2) and (6), we can deduce the expressions of input error and state error.

The state error is shown as follows:

and the input error is given as follows:

Then, we can combine (7) and (8) into a two-dimensional Roessor model

where, for convenience, we define $K_k= K_k(t)$, $C^+= C(t+1)$, $B= B(t)$, $A= A(t)$, $\Phi_1= I-K_kC^+B$, and $\Phi_2= K_k(-C^+A)$. In addition, we introduce $\tilde{\theta}_k(t)=1-\mu^{-1}\theta_k(t)$. It is obvious that $\tilde{\theta}_k(t)$ is a zero-mean random variable with variance $\tilde{\sigma}=\sigma^2/\mu^2$.

Writing (9) in a compact form, we have

where

are $(n+p)\times1$ vectors, and

are a $(n+q)\times1$ vector and a $(n+p)\times1$ vector, respectively.

are a $(n+p)\times(n+p)$ matrix and a $(n+p)\times(n+q)$ matrix, respectively.

are a $(n+p)\times(n+p)$ matrix and a $(n+p)\times(n+q)$ matrix, respectively.

It is assumed that initial state errors and input errors at the initial iteration are of zero-mean, then the right-hand side of (10) is of zero-mean. It is proper to refer to $H^+={E}[X^+{X^+}^T]$ as a covariance matrix. Our objective is to find an optimal learning gain matrix $K_k$ minimizing the trace of this error covariance matrix. The error covariance matrix $H^+$ is as follows:

Let

where

and

As a consequence, $H$ and $V$ are symmetric positive-semidefinite matrices. Moreover, we have

where the zero sub-matrices $Q_{11, t}$ and $Q_{22, t+1}$ are due to zero cross-correlation between $\omega_k(t)$ and $\upsilon_k(t)$.

We expand the right-hand side of (11) to get (12)-(15) at the bottom of the next page. Then, we can deduce the trace of $H^+$ in (11) as

Expanding and rearranging the terms on the right-hand side of this equation, we have

Define

and

then (17) is transformed as

Note that the learning gain matrix $K_k(t)$ is used to update the input for the next batch. Therefore, the value of $K_k(t)$ is related to the input error at time instant $t$ of the $(k+1)$th iteration and does not affect the state error at time instant $t+1$ of the $k$th iteration. Then,

To find $K_k$ such that the trace of input error covariance matrix is minimized, we can set the derivative of trace of $H^+$ with respect to $K_k$ as follows:

Note that $H$, $V$, and $Q_{11, t}$ are positive-semidefinite matrices, and $Q_{22, t+1}$ is a positive-definite matrix. Then, we obtain that $\tilde{\sigma}DVD^T+DHD^T+(\tilde{\sigma}+1)(C^+Q_{11, t}{C^+}^T+Q_{22, t+1})$ is nonsingular. Let $\frac{\partial ({\rm{tr}}(H^+))}{\partial K_k}=0$ yielding an optimal learning gain matrix $K_k$

B Convergence Analysis

From (6) and (7), we have

and

where

and

Noting all terms on the right-hand side of both (19) and (20), we can conclude that $\Delta x_k(0)$, $\Delta u_0(t)$, $\omega_k(0\leq m\leq t-1)$, and $g_{t, 0\leq m\leq k-1}$ are all uncorrelated. In addition, these terms are also uncorrelated with $\Delta u_k(0\leq m\leq t-1)$ and $\Delta x_{0\leq m\leq k-1}(t)$. Because of $\Delta u_k(0\leq m\leq t-1)$ and $\Delta x_{0\leq m\leq k-1}(t)$ cannot be represented by each other, thus they are uncorrelated. Consequently, we get $H_{12, k}=0$.

Next, it is noticed that the covariance between $x_k(t)$ and $u_k(t)$ is the same as the covariance between $\Delta x_k(t)$ and $\Delta u_k(t)$. Since the last two terms $\Delta u_k(0\leq m\leq t-1)$ and $\Delta x_{0\leq m\leq k-1}(t)$ are uncorrelated, hence $u_k(0\leq m\leq t-1)$ and $x_{0\leq m\leq k-1}(t)$ are uncorrelated. Therefore, $V_{12, k}=0$ and $V^T_{12, k}=0$.

Then, $K_k$ is reduced to

From (8), we obtain the input error covariance matrix

Define

Expanding (22), we have

Substituting (21) into (23) yields

Theorem 1: Consider system (1) with output fading and assume that Assumptions 1-3 hold. The learning algorithm, presented by (6), (21), and (24), guarantees that the input covariance matrix converges to zero asymptotically in the mean-square sense. That is, $H_{11, k}\rightarrow 0$ as $k\rightarrow \infty$.

Proof: Rewrite (21) as follows,

Note that $H_{22, k}$, $V_{11, k}$, $V_{22, k}$ and $Q_{11, t}$ are positive-semidefinite matrices and $Q_{22, t+1}$ is positive definite, thus $S_1$ is positive definite. Combining (21) and (22), we have

If $H_{11, k} > 0$, it can be examined from that $H_{11, k+1}\geq K_kS_1K_k^T > 0$, since $S_1 > 0$ and $M$ is a full-column rank matrix. It means that $H_{11, 0} > 0$ leads to $H_{11, k} > 0$. Then, taking inverse of both sides of (24) with the well known matrix inversion method yields to

Then, we have

From the above analysis, we can readily deduce that $H_{11, k+1} < H_{11, k}$, $\forall k$ and $\lim_{k \rightarrow \infty} H_{11, k} = 0$.

The proof is inspired by [30]. Theorem 1 reveals the asymptotical convergence of the generated input sequence to the desired control value as iteration number increases in the mean-square sense. The fundamental reason is that Kalman filtering theory enhances the performance of ILC against various randomness. It can be seen from Theorem 1 that $K_k$ depends on the variance of the fading variable. If the variance is zero (i.e., the output is not affected by fading), the algorithm proposed in this study turns into the one provided in [24]. This observation connects the results in this study with the existing literature.

Corollary 1: Consider system (1) with output fading and assume that Assumptions 1-3 hold. In addition, assume that there is no system noise and initial state error, i.e., $\omega_k(t)=0$ and $\Delta x_k(0)=0$. Then, the learning algorithm, presented by (6), (21), and (24), guarantees that $\|H_{22, k}\|\rightarrow 0$ as $k\rightarrow \infty$.

Proof: Because none of system noise and initial state error exist in the dynamics, (19) is reduced to

Then, we have

Using the basic inequality, we have

Because $\lim_{k\rightarrow \infty}H_{11, k}=0,$ $\forall 0\leq m\leq t-1$, $\lim_{k\rightarrow \infty} {E}[\Delta x_k(t)\Delta x_k(t)^T]=0$.

Remark 2: We note that the proposed learning control algorithm (21) requires the state error covariance matrix for computation. This covariance matrix can be obtained according (7) as follows:

The pseudo code for the proposed algorithm is given in Algorithm 1.

Algorithm 1: The algorithm with fading channel at the output side
Step 1: a) Without loss of generality, the initial condition is set as $H_{11, 0}(t)$ and $H_{22, k}(0)$; b) using (21), compute learning gain $K_k$; c) using (26), compute $H_{22, k}(t+1)$; d) using (6), update the control $u_{k+1}(t)$; e) using (24), update $H_{11, k+1}$; Step 2: $k=k+1$, repeat whole process.s

 | Show Table

DownLoad: CSV

Ⅳ. INPUT FADING CASE

In this section, we discuss the input fading case. In particular, the channel from controller to plant suffers from random fading, and the channel from plant to controller is assumed to work without any fading. Compared with the previous section, the input fading may lead to an unstable operation because the received input is incorrect.

A Algorithm Design

The generated input by controller is $u_k(t)$. The received signal by plant is denoted by $u^\circ_k(t)$. Then, according to the fading model

where $\theta_k(t)$ is the random fading variable. In the rest of this section, we retain most notation same to the previous section to save notation.

Clearly, the received input signal is not suitable for driving the system because of its deviation from the computed one. The mean inverse of the fading variable is multiplied to correct the signal

where $\hat{u}_k(t)$ denotes the corrected output signal. Then, the dynamics is given by

We apply the following update law

The state error and input error can be expressed as follows:

and

We can combine (31) and (32) into a Roessor model

Writing (33) in a compact form, we have

where

is a $(n+p)\times(n+q)$ matrix and the other notation is same as in the previous section.

Because initial state errors and input errors at the initial iterations are of zero-mean, the expectation of $X^+$ is zero. Denote $P^+$ $={E}[X^+{X^+}^T]$ as a covariance matrix. Out objective is to find an optimal learning gain matrix $K_k$ minimizing the trace of $P^+$. To this end, we expand $P^+$ as follows:

Then, we have

Setting

and

the trace of $P^+$ is reduced to

To find $K_k$ minimizing the trace of $P^+$, we calculate the derivative of trace of $P^+$ with respect to $K_k$ as follows:

Note that $H$, $V_{11, k}$ and $Q_{11, t}$ are positive-semidefinite matrices and $Q_{22, t+1}$ is positive definite, thus $\tilde{\sigma}(C^+B)V_{11, k}(C^+B)^T+DHD^T+(C^+Q_{11, t}{C^+}^T)+Q_{22, t+1}$ is nonsingular. Letting $\frac{\partial ({\rm{tr}}{H^+})}{\partial K_k}=0$, we can obtain a solution of $K_k$ as follows:

B Convergence Analysis

From (30) and (31), we have

and

where

Considering all terms at the right-hand side of (38) and (39), we can conclude that $\Delta x_k(0)$, $\Delta u_0(t)$, $f_{0\leq m\leq t-1, k}$, and $g_{t, 0\leq m\leq k-1}$ are all uncorrelated. In addition, these terms are also uncorrelated with $\Delta u_k(0\leq m\leq t-1)$ and $\Delta x_{0\leq m\leq k-1}(t)$. Because of $\Delta u_k(0\leq m\leq t-1)$ and $\Delta x_{0\leq m\leq k-1}(t)$ cannot be represented by each other, they are uncorrelated. Consequently, we get $H_{12, k}=0$.

Next, it is noticed that the covariance between $x_k(t)$ and $u_k(t)$ is the same as the covariance between $\Delta x_k(t)$ and $\Delta u_k(t)$. Since the last two terms $\Delta u_k(0\leq m\leq t-1)$ and $\Delta x_{0\leq m\leq k-1}(t)$ are uncorrelated, hence $u_k(0\leq m\leq t-1)$ and $x_{0\leq m\leq k-1}(t)$ are uncorrelated. Therefore, $V_{12, k}=0$ and $V^T_{12, k}=0$. Then, $K_k$ is reduced to

From (32), we obtain the input error covariance matrix

Define

Expanding (41), we have

Substituting (40) into (42) yields

Theorem 2: Consider system (1) with input fading and assume that Assumptions 1-3 hold. The learning algorithm, presented by (30), (40), and (43), guarantees that the input covariance matrix converges to zero asymptotically in the mean-square sense. That is, $H_{11, k}\rightarrow 0$ as $k\rightarrow \infty$.

Proof: From the above analysis, we can readily deduce that

It means that $H_{11, 0} > 0$ leads to $H_{11, k} > 0$. Then, taking inverse of both sides of (43) with the well known matrix inversion method yields to

Then, we have

From the above analysis, we can readily deduce that $H_{11, k+1} < H_{11, k}$, $\forall k$ and $\lim_{k \rightarrow \infty} H_{11, k} = 0$.

Corollary 2: Consider system (1) with output fading and assume that Assumptions 1-3 hold. In addition, assume that there is no system noise and initial state error, i.e., $\omega_k(t)=0$ and $\Delta x_k(0)=0$. Then, the learning algorithm, presented by (30), (40), and (43), guarantees that $\|H_{22, k}\|\rightarrow 0$ as $k\rightarrow \infty$.

Proof: The proof is the same as the corresponding part of the proof of Corollary 1.

Remark 3: To apply the proposed learning control algorithm, we need the state error covariance matrix. It can be derived from (31) as follows:

The pseudo code for the proposed algorithm is given in Algorithm 2.

Algorithm 2: The algorithm with fading channel at the input side
Step 1: a) Without loss of generality, the initial condition is set as $H_{11, 0}(t)$ and $H_{22, k}(0)$; b) using (40), compute learning gain $K_k$; c) using (44) compute $H_{22, k}(t+1)$; d) using (30), update the control $u_{k+1}(t)$; e) using (43), update $H_{11, k+1}$; Step 2: $k=k+1$, repeat whole process.

 | Show Table

DownLoad: CSV

Ⅴ. EXTENSION TO GENERAL CASE

In the previous sections, we consider the output fading and input fading separately to clarify our main idea and highlight the difference between the two scenarios. However, in many practical applications, the transmission of the output and input signals usually shares the same communication networks. Thus, the output and input fading may coexist. In this section, we clarify the extension to this general case briefly.

The input fading variable and the output fading variable are denoted by $\theta_k^{(1)}(t)$ and $\theta_k^{(2)}(t)$, respectively. Moreover, $\theta_k^{(1)}(t)$ is independent of $\theta_k^{(2)}(t)$. In addition, we assume that $\theta_k^{(*)}(t)$ has the following statistical properties

where the subscript "$*$" indicates $1$ or $2$ according to the input fading case or the output fading case.

Define

Again, we denote $C^+ = C(t+1)$, $B = B(t)$, $A=A(t)$, $K_k$ $=$ $K_k(t)$, the state error $\Delta x_{k}(t) = x_d(t) - x_k(t)$, and the input error vector $\Delta u_{k}(t) = u_d(t) - u_k(t)$. According to the system (1) and reference model (2), we have that

where the correction to the input is imposed. Therefore, the input error is given by

We can combine (45) and (46) into a Roessor model

where $\star=-K_kC^+B[\tilde{\theta}_k^{(1)}(t)+\tilde{\theta}_k^{(2)}(t)\mu_1^{-1}{\theta}_k^{(1)}(t)]$.

Borrowing the results from Section Ⅲ, we find that the learning gain that minimizes the trace of the input error covariance matrix is similar to the one presented by (21); particularly, we have

The input error covariance matrix becomes

Therefore, similar to Sections Ⅲ and Ⅳ, one can conclude the following theorem.

Theorem 3: Consider system (1) with both output and input fading and assume that Assumptions 1-3 hold. The learning algorithm, presented by (6), (48), and (49), guarantees that guarantees that the input covariance matrix converges to zero asymptotically in the mean-square sense. That is, $H_{11, k}\rightarrow 0$ as $k\rightarrow \infty$.

Proof: The proof is similar to that of Theorem 1. We omit it for saving space.

Ⅵ. ILLUSTRATIVE SIMULATIONS

A Numerical Example

Consider a discrete time-variant linear multiple-input-multiple-output (MIMO) system, where the system matrices are given as follows:

For the initial state, it is assumed that $x_k(0)=x_d(0)$ for all $k$. The random variable $\theta_k(t)\sim N(0.95, 0.1^2)$. The system noises $\omega_k(t)\sim N(0, 0.1^2)$ and measurement noises $\upsilon_k(t)\sim N(0, 0.1^2)$. Without loss of generality, the initial conditions are set to $H_{11, 0}(t)={\rm{diag}}\{20, 20\}$ and $H_{22, k}(0)={\rm{diag}}\{0, 0,$ $0\}$. The operation is repeated for $100$ iterations.

The desired output within the time interval $t\in[0, 50]$ is given as follows:

The output tracking performance and the convergence of the input error are demonstrated in Figs. 2 and 3. In Fig. 2(a) and 2(b) for output fading and input fading, respectively, it can be seen that output trajectories at the 40th iteration almost overlap with the desired trajectory, demonstrating a satisfied tracking performance after several learning iterations. The evolution of averaged 2-norm of input errors (defined as $(\sum_{t=0}^{N-1}{\|\Delta u_k(t)\|}^2)/N$ for the $k$th iteration) along the iteration axis is demonstrated in Fig. 3(a) and 3(b) for output fading and input fading, respectively. The profiles in both figures decreases rapidly along the iteration axis, implying that the proposed algorithms have favorite performance in handling stochastic systems over fading channels.

Figure  2.  The desired trajectory and tracking profiles at the 2nd and 40th iteration: (a) output fading case; (b) input fading case

DownLoad: Full-Size Img PowerPoint

Figure  3.  Evolution of the averaged mean-square input error profile along the iteration axis: (a) output fading case; (b) input fading case

DownLoad: Full-Size Img PowerPoint

To show the better performance of the proposed algorithms than the conventional $P$-type learning algorithm, we also simulate the algorithms with fixed learning gain matrices. In particular, the selected gain matrix $K$ for both output fading and input fading selects

It can be calculated that these selections satisfy a contraction condition that $\|I-KC^+B\|<1$ for both cases. The mean-square input error profile for this algorithm along the iteration axis is demonstrated in Fig. 4 (see the dotted lines in both upper and lower figures). It is apparent that this algorithm with a fixed gain matrix fails to continuously improve the input after several iterations because of the existence of various random noises. This observation discloses the distinct advantages of the proposed approach in handling various randomness and achieving competitive tracking performance.

Figure  4.  Evolution of the averaged mean-square input error profile along the iteration axis for different learning control algorithms: (a)output fading case; (b) input fading case

DownLoad: Full-Size Img PowerPoint

In order to show the effect of fading channel, we also simulate the framework in the absence of fading randomness. In other words, the algorithm in [24] is simulated as a comparison. The corresponding mean-square input error profile in the iteration domain is demonstrated in Fig. 4. For both output and input cases, it is observed that the framework without fading leads to a faster convergence speed. This is in agreement with our intuitive understanding that the fading randomness may slow down the convergence rate. Moreover, the convergence rate of the output case is slightly faster than that of the input case. This is because the fading at the input signal would lead a fluctuation of control inputs and then worsen the tracking performance.

In addition, we are interested in whether different fading distributions have significant effects on convergence. To clarify this point, we consider the fading gain $\theta_k(t)$ for three distributions as follows: $\mu= 0.95$ and $\sigma = 0.1$, $\mu = 0.85$ and $\sigma = 0.1$, and $\mu = 0.95$ and $\sigma = 0.05$. The output and input fading cases are shown in Fig. 5(a) and 5(b), respectively. It can be seen that variation of fading gain mean would not bring much difference to convergence performance, while variation of fading gain variance can greatly change the specific convergence speed and tracking precision. In particular, a smaller variance of fading gains leads to a better performance of the proposed algorithms. These observations coincides with our common sense.

Figure  5.  Averaged input error profiles for different fading parameters: (a) output fading case; (b) input fading case

DownLoad: Full-Size Img PowerPoint

B PMLM Model

To verify effectiveness and convergence properties of the proposed scheme, a permanent magnet linear motor (PMLM) model is utilised. The discretized model of PMLM is given as follows [31]:

where the sampling period $\Delta$ is selected as $0.01$ s, $x$ and $\nu$ denote the motor position and rotor velocity, $R$ = 8.6, $m4 = 1.635$ kg, $\psi_f= 0.35$ Wb are the resistance of the stator, rotor mass and flux linkage, respectively, $k_1=\pi/\tau$ and $k_2= 1.5\pi/\tau$ where $\tau = 0.031$ m is the pole pitch. The initial values satisfy $H_{11, k}(0) = \{0\}$ and $H_{22, 0}(t)=\{10\}$. The fading gain is assumed to be $\theta_k(t)\sim N(0.95, 0.1^2)$. The system noises $\omega_k(t)\sim N(0, 0.05^2)$ and measurement noises $\upsilon_k(t)\sim N(0, 0.05^2)$ are used. The desired trajectory is given as $y_d(t) = -1.2\sin(0.05t\pi)+0.2-0.2\cos(0.02t)$. The operation is repeated for 1000 iterations. Only the output fading is accounted for the sake of brevity. The results for input fading are similar.

Fig. 6 displays the tracking performance of the proposed scheme at the 2nd and 100th iterations and the averaged spectral radius profile of input error covariance matrices along the iteration axis. It can be seen from Fig. 6(a) that the tracking performance for the 2nd iteration is far away from satisfaction but that for the 100th iteration is already acceptable for practical applications. The different continuously decreasing trend of the input error profile in Fig. 6(b) illustrates effectiveness of different learning control algorithm. The results show similar conclusions to the previous subsection, and thus we omit the details of saving space.

Figure  6.  The output fading case for the PMLM model: (a) desired output trajectory and actual output trajectories at the 2nd and 100th iterations; (b) evolution of the mean-square input error profiles

DownLoad: Full-Size Img PowerPoint

Ⅶ. CONCLUSION

We studied the ILC problem for stochastic systems through fading channels at output side and input side, respectively. A Kalman filtering-based framework is used to generate the optimal learning gain matrix under various randomness by optimizing the trace of input error covariance matrices. Differing from the results without fading channels, the results in this paper indicate that fading introduces multiplicative randomness to the involved quantities and results in careful analysis in the relation among different variables. In particular, for both output fading and input fading, we presented a computable algorithm for generating the optimal learning gain matrix and then presented a learning algorithm for producing input signals along the iteration axis. As a result, the tracking performance can be gradually improved. Specific convergence performance has been proved in mean-square sense. Although we separated discussions for output fading and input fading to reveal different effects of both fading cases, the presented framework is also effective for the general case that both output and input fading exist simultaneously. For further research, it is of great interest to investigate the ILC problem of nonlinear systems in the presence of random fading.