An Incremental Model Transfer Method for Complex Process Fault Diagnosis

Ⅰ. Introduction

With the rapid growth of integration scale of the modern industry, the production process is growing more complex for its dynamic, nonlinear, time-delay, large inertia and strong coupling [1]. Even if a tiny fault is ignored, it may lead to paralysis of the whole system and seriously threaten the safety of life and property. So it is crucial to detect the fault of the whole production process accurately and predict the abnormal working conditions in advance to prevent more losses. It attracts many researchers to devote themselves to this field, which results in an increasing number of fault diagnosis methods being proposed [2], [3]. As a result of the rapid development of computer technology in recent years, a large number of process data have been collected and stored. Thus, the methods based on data-driven have their unique advantages. These methods are based solely on collected data rather than an accurate input-output model for the controlled process. They are suitable for the fault diagnosis of a modern complex process whose mathematical model is difficult to establish. Therefore, the methods based on data-driven have been widely researched and used to the complex industrial fault diagnosis [4]-[7].

With the increasing complexity of the industrial processes, some traditional machine learning methods, such as Support Vector Machine (SVM), neural networks, reinforcement learning, have been used for fault diagnosis [8]-[10]. These methods have a good effect on the fault diagnosis of many processes, but there is a strict premise that the process data of training and test should be independent and they have the same distribution. Unfortunately, in complex industrial processes, there will be a distribution difference between the old process data and the new ones if there are some changes in the device parameters, the production environment, even the raw materials. Since the new process can not meet the requirements of the old process, the old model can not be directly applied to the fault diagnosis of the new process. In the meanwhile, the data accumulated in the new process is too few to rebuild a new diagnosis model. In this case, traditional machine learning methods are no longer applicable. And it becomes a research hotspot that how to effectively use the old model and the limited new data to build a new model. Transfer learning is a new learning framework to solve the problem [11], [12], it relaxes the assumption that the training and testing samples need to meet the independent and identically distribution, and it focuses on solving the problem of the different distribution between old process and the new one. Domain adaptation of two processes is an effective way to improve the efficiency of transfer learning [13].

Transfer learning has involved in many fields, mainly in text classification, image classification, artificial intelligence planning and so on. In recent years, transfer learning begins to apply to fault diagnosis field [14], [15], and it performs well. However, transfer learning does not take account of the fact that the differences between the two processes may lead to some new faults in the new process, and the number of new faults can not be estimated accurately. Using known faults knowledge to learn the new faults, this demand often exists but it is nearly untouched in transfer learning. For other fault diagnosis methods, there are several algorithms can detect unknown faults on certain conditions. Hu et al. [16] proposed a fault diagnosis method based on kernel fuzzy C-means (KFCM) for unknown faults in satellite reaction wheels. The unknown faults which can be identified in the method must be a combination of two or more known faults. Kuzborskij et al. [17] solved the problem of image classification and they also recognized a new class by transfer learning. If this method is applied to fault diagnosis, it can only identify an unknown fault. But it is necessary to predict all the new unknown faults in the actual process. However, identifying all the N+M faults is not simply iterating M times of N $\rightarrow$ N+1. With the increase of unknown faults, it will inevitably bring about a series of problems, which hampered scholars' study. Under the inspiration of literature [17], this paper proposes an incremental model transfer fault diagnosis method for complex process, which can identify all the faults, including the new faults in the new process. For convenience, we call our method incremental model transfer learning (IMTL). Prior to current fault diagnosis methods, the presented algorithm can identify the new faults in a new process with the advantage of transfer learning. Besides, the method gets the results of fault detection and classification simultaneously. The main contributions of this paper are as follows:

1) Introducing model transfer into fault diagnosis field, the method achieves fault diagnosis with very few labeled samples. And fault detection and classification are achieved at the same time.

2) Combining model transfer with incremental learning, the proposed method achieves the extension from identifying the existing N faults in the old process to N+M faults in the new process, where M is the number of new faults in the new process.

3) Breaking the limit of [17] that the new process can only have one more class of faults than the old process, the method can identify M faults more in the new process.

4) This method gives a solution to a series of problems caused by the increase of fault classes.

The rest of the paper is organized as follows: the related work of model transfer is introduced in Section Ⅱ. Our learning framework and fault diagnosis process are presented in Section Ⅲ. We introduce typical industrial processes and describe our experimental results in Section Ⅳ. Finally, we conclude the paper in Section Ⅴ.

Ⅱ. Related Works

Now we have a strategy to identify new faults, but how can we establish a diagnosis model quickly for a new industrial process with a few labeled samples? This is a practical and meaningful question. Using the data of similar processes to help to learn new process tasks, transfer learning solves the problem. As long as there exists the similarity between old process and the new one, the old knowledge can be used in the new process. Moreover, the transfer efficiency mainly depends on the degree of the similarity. The more common features there are, the better effect it will get.

Lu et al. [18] proposed the concept of model transfer learning first. It is based on an established model and there should be a similarity between the two processes. Then it uses less experimental data and the model parameters of the similar process to build a new model. The previous machine learning methods need a lot of time and samples to build a diagnosis model, while the model transfer methods focus on exploiting the existing model and a few new process data to build a diagnosis model rapidly. The model transfer learning does not require remodeling and greatly reduces the time and cost of modeling as well as inherits the advantages of the old method. However, model transfer has its own drawback: the ability of local changes between the old model and the new one is limited and model transfer mainly focuses on global regularization. This paper makes up for the drawback with a local regularization term.

Most model transfer algorithms focus on the visual object categorization or natural language processing. Segevetal et al. [19] proposed three model transfer algorithms with random forest. The structure expansion reduction (SER) algorithm uses a few samples to expand or cut the structure of the source domain (old process) decision tree to establish target domain (new process) decision tree model; the STRUT (Structure Transfer) algorithm uses several samples to establish the decision model of target domain by modifying the decision tree threshold; the mix SER and STRUT (MIX) algorithm combines the two random forests to get the final classification result by the majority of votes. Tommasi et al. [20] presented a multi-model knowledge transfer algorithm for image binary classification, and the method can tune parameters automatically. On the basis of the original least squares support vector machine (LS-SVM) optimization problem, Chen et al. [21] achieved the purpose of model transfer by adding the penalty function and constraint condition of the auxiliary set, respectively, in the original objective function and constraint condition. Cawley gave a detailed exposition of the leave-one-out method and cross-validation of weighted LSSVM [22]. Luo et al. [23] used model transfer to classify object with a method called Multi-Kernel Transfer Learning, and the results indicated the great effect of the method. The model transfer learning methods mentioned above can only identify known classes. The method in [17] can identify a new class and is the only method in the current model transfer method that can identify new classes, but its extendibility is limited. Prior to the method in [17], the method proposed in this paper identities all the new classes, it also solves the problem that the difference between the new model and the original model increases gradually with the class extension. So it has higher practicability. Moreover, the proposed method achieves the domain adaption through transfer component analysis (TCA) [24], which improves the efficiency of model transfer learning. Besides, the method proposed in this paper applies model transfer to the field of fault diagnosis, which is rarely studied until now.

Ⅲ. Fault Diagnosis Based on Incremental Transfer Learning

A Incremental Transfer Learning

1)Problem Settings and Definitions: In the paper, column vectors and matrices are denoted by small and capital bold letters, respectively, e.g. $\mathbf{m} = [a_1, a_2, \ldots, a_d]^T$. We denote the input sample sets as $\mathbf{X} = [\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_m]\in \mathbf{R}^{d\times m}$, where m is the number of samples and d is the characteristic number of each sample. Supposing we have m samples with n classes $(m\gg n)$ that need to be classified, the labels are denoted as $y_i\in \mathbf{Y} = \{1, 2, \ldots, n\}$ with $i\in \{1, 2, \ldots, n\}$. For convenience, we set the label matrix as $\mathbf{Y}\in \mathbf{B}^{m\times n}$, in which ${\mathbf{Y}}_{in} = \left\{ {\begin{array}{*{20}{c}} {1, }\\ {0, } \end{array}} \right.\left. {\; \; \; \; \begin{array}{*{20}{c}} {{y_i} = n}\\ \rm {otherwise} \end{array}} \right\}$. We want to learn a function $f(\mathbf{x}) = \mathbf{W}^T\cdot\phi(\mathbf{x})+b$, which assigns the classes of input vector. $\phi(\cdot)$ is used to map $\mathbf{x}$ to a high dimensional feature space to solve the nonlinear problems.

2) Model Transfer Learning Based on LS-SVM: Firstly, we extend LS-SVM to a multiclass form through one-vs-all (OVA) as the base model and expressed as

Solving (1) we can obtain an N-hyperplanes set from the source (old process) N classes and the N-hyperplanes set can be expressed as classification models. So we regard $\mathbf{W}$ as the source optimal model $\mathbf{W}' = [\mathbf{W}_1', \ldots, \mathbf{W}_N']$. When facing a target (new process) task, we can add a global regularization term to (1) to make the new hyperplanes set $\mathbf{W}$ close to $\mathbf{W}'$, thus the result is

For binary classifier, there is only one hyperplane, so $\mathbf{W}$ is a vector and constraint term uses $L_2$ norm so as not to transfer too much. As for multiclass, $\mathbf{W}$ is a hyperplane set and then it is a matrix. Since the form of $L_2$ norm is similar to the F norm, the $L_2$ norm focuses on the vector and the F norm focuses on the matrix, so we choose the F norm constraint. To avoid the optimization problem caused by too many parameters, we simplify the weights of global regularization terms to 1/2 and so do all the following global regularization terms and local regularization terms. Of course, in practical applications, the optimization algorithms of parameters can be designed according to the actual situations. Here, we mainly explained the usage of each global regularization term.

Until now, we have just made a simple approximation of the source model and target model, which allows the source knowledge to transfer to the target domain as much as possible. The reason why we do not replace $\frac{1}{2} \left\|\mathbf{W}\right\|_F^2$ with $\frac{1}{2} \left\|\mathbf{W}-\mathbf{W}'\right\|_F^2$ like other model transfer methods is that minimizing $\frac{1}{2} \left\|\mathbf{W}\right\|_F^2$ can reduce the weight vector, decrease the model complexity, avoid over-fitting, and improve the generalization performance. However, we cannot ignore what is unique to the target task. We will explain it in the next section.

3) From N to N+1: Incremental Transfer Learning: Incremental transfer learning aims at finding a new hyperplanes set $\mathbf{W} = [\mathbf{W}_1, \ldots, \mathbf{W}_N], \mathbf{w}_{N+1}$ to achieve two goals. The first one is to identify the new class in the target domain by transferring the source models (N classes). While the other one is to make sure the performance on source N classes does not decline too much compared with the former. In other transfer learning methods, the second goal is often ignored. Because of the linearity of the model, we can force to shorten the distance between the two classifiers to make the classifiers performance similar. And both of the goals can be implemented by distance-based regularization terms.

For the new hyperplane $\mathbf{w}_{N+1}$, we enforce it to get close to a linear combination of the source models $\mathbf{W}' = [\mathbf{W}_1', \ldots, \mathbf{W}_N']$. Therefore, the local regularization term $\frac{1}{2} \left\|\mathbf{w}_{N+1}-\boldsymbol{\beta}\mathbf{W}'\right\|_F^2$ is added to achieve the first goal. Meanwhile, the coefficient vector $\boldsymbol{\beta} = [\beta_1, \ldots, \beta_N]^T$ is used to weigh the transfer amount of every source model. And $\boldsymbol{\beta}$ can be calculated automatically with the algorithm later. Similarly, we can use the global regularization term $\frac{1}{2} \left\|\mathbf{W}-\mathbf{W}'\right\|_F^2$ to achieve the second goal by forcing new hyperplane set $\mathbf{W}$ to be close to the prior hyperplanes $\mathbf{W}'$. It can also prevent negative transfer of the whole model. To achieve the two goals, in the model transfer learning with LS-SVM, we obtain the function

Let us reiterate the role of each item. The global regularization term $\frac{1}{2} \left\|\mathbf{W}-\mathbf{W}'\right\|_F^2$ is to avoid negative transfer. While local regularization term $\frac{1}{2} \left\|\mathbf{w}_{N+1}-\boldsymbol{\beta}\mathbf{W}'\right\|_F^2$ allows local modification of the model and avoids the problem that model transfer usually focuses on global regularization and ignore what is unique to the target model. Minimizing $\frac{1}{2} \left\|\mathbf{W}\right\|_F^2$ is to avoid over-fitting. The last term is a relaxation variable for LS-SVM. The Lagrangian to (3) can be expressed as

Then, we can get

Putting $\mathbf{W}_n$ from (5), $\mathbf{w}_{N+1}$ from (6), $\xi_i$ from (7) into (8), we can get

In matrix form we have

where ${{\mathbf{K}}_{ki}} = \phi ^T{({{\mathbf{x}}_k})}\phi ({{\mathbf{x}}_i})$, and $\mathbf{\mathop Y\limits^ \sim} = [\begin{array}{*{20}{c}} {\frac{1}{2}{{\mathbf{X}}^T}{\mathbf{W}}'} & {\frac{1}{2}{{\mathbf{X}}^T}{\mathbf{W}}' {\boldsymbol{\beta }}} \end{array}], \; {\mathbf{A}} = {\mathbf{A}}' - \left[{\begin{array}{*{20}{c}} {{\mathbf{A}}''} & {{\mathbf{A}}''β} \end{array}} \right]$, ${\mathbf{b}} = {\mathbf{b}}' - \left[{\begin{array}{*{20}{c}} {{\mathbf{b}}''} & {{\mathbf{b}}''β} \end{array}} \right]$, Let ${\mathbf{M}} = \left[{\begin{array}{*{20}{c}} {\frac{1}{2}{\mathbf{K}} + \frac{1}{C}{\mathbf{I}}} & {\mathbf{1}}\\ {{{\mathbf{1}}^T}} & {\mathbf{0}} \end{array}} \right]$, so

Through (5), (6), (11), (12), we can get the new hyperplane set $\mathbf{W} = [\mathbf{W}_1, \ldots, \mathbf{W}_N], \mathbf{w}_{N+1}$, and the transfer learning problem can be solved as long as the parameter $\boldsymbol{\beta}$ is determined. The main thought of tuning the value of the parameter automatically is as follow: the label leave-one-out (LOO) prediction $\mathbf{Y(\boldsymbol{ \pmb{\mathsf{ β}}})}$ of test samples is measured by the hinge loss, and then the parameter $\boldsymbol{\beta}$ is self-adjusted by solving the minimization problem of LOO error. Finally, LOO prediction is presented as ${\mathop {\mathbf{Y}}\limits^ \wedge _{in}} = {{\mathbf{Y}}_{in}} - {{{{\mathbf{A}}_{in}}}}/{{{\mathbf{M}}_{ii}^{ - 1}}}$. The function of minimizing the LOO error is as follow.

4) From N to N+M: IMTL Algorithm With Few Example: Based on the method above and the thought of incremental learning with SVM, we complete the extension from N to N+M with LS-SVM. Incremental learning here refers to training a new class at each iteration. The steps of the incremental algorithm with LS-SVM is summarized in Algorithm 1.

Algorithm 1: IMTL: Incremental Model Transfer Learning
Input: Training set ${\mathbf{X}}$, with $m$-2 subsets adding a class in turn ${{\mathbf{X}}_1}, \ldots, {{\mathbf{X}}_{M - 2}}$, the number of the classes from 3 to m.
Output: An LS-SVM classifier model based on ${\mathbf{X}}$.
Step 1: Take the subset ${\mathbf{X}_1}$ for training, the hyperplane set ${\mathbf{W}_1}$ classifies class 1 with the other classes, the hyperplane set ${\mathbf{W}_2}$ classifies class 2 with the other classes, and the hyperplane set classifies ${\mathbf{W}_3}$ class 3 with the other classes.
Step 2: Train hyperplane set ${{\mathbf{W}}_1}, {{\mathbf{W}}_2}, {{\mathbf{W}}_3}$ and subset ${\mathbf{X}_2}$, the hyperplane set ${\mathbf{W}_4}$ was obtained, specific algorithms have been introduced above.
Step 3: Repeat step 2 to train the obtained new hyperplane set and data subset ${\mathbf{X}_i}$, over and over again, until the training set ${\mathbf{X}_{M-2}}$. The hyperplane set ${\mathbf{W}_M}$ trained by ${\mathbf{W}_{M-2}}$ and ${\mathbf{X}_{M-2}}$ is the classification model of the whole training set ${\mathbf{X}}$, then output ${\mathbf{W}_M}$.
Step 4: End the algorithm.

 | Show Table

DownLoad: CSV

From the point of solving problem, the proposed method aims at finding a new hyperplane set $\mathbf{W} = [\mathbf{W}_1, \ldots, \mathbf{W}_N, \mathbf{w}_{N+1}, \ldots, \mathbf{w}_{N+p-1}, \mathbf{w}_{N+p}], \mathbf{w}_{N+p+1}$. Then our learning framework IMTL can be expressed as

where $\mathbf{W}' = [\mathbf{W}_1, \ldots, \mathbf{W}_N, \mathbf{w}_{N+1}, \ldots, \mathbf{w}_{N+p-1}, \mathbf{w}_{N+p}]$, ${\boldsymbol{\beta}} = {\left[{{\beta _1}, \ldots, {\beta _{N + { {\rm p}}}}} \right]^T}$.

The solution of the problem is similar to that of (3), and it can be regarded as the iteration of the solving problem (3). In other words, we can calculate $\mathbf{w}_{N+1}$ first, and then regard $\mathbf{W} = [\mathbf{W}_1, \ldots, \mathbf{W}_N], \mathbf{w}_{N+1}$ as $\mathbf{W}'$ to find $\mathbf {w}_{N+2}$, repeatedly and iteratively, until $\mathbf {w}_{N+p+1}$ is calculated.

In theory, problems maybe appear as the classes increase: the prediction error rate may have a corresponding cumulative effect. Even though we use a global regularization term $\frac{1}{2} \left\|\mathbf{W}-\mathbf{W}'\right\|_F^2$ to correct the model in each iteration to solve $\mathbf {w}_{N+p+1}$, with the classes increasing in the target domain, the difference between the new model and the initial model will get greater, which causes a lower accuracy. The number of classes that the model can identify effectively depends on the complexity of the process data. To weaken the negative effects of the situation, we use $\sum\limits_{n = 1}^N {\sum\limits_{t = 1}^n {\frac{1}{2}\Big\| {{{\mathbf{W}}_n} - {{\mathbf{W}}_n}_{{ {\rm - }}t}'} \Big\|_F^2} }$ to replace $\sum\limits_{n = 1}^{N + p} {\frac{1}{2}\Big\| {{{\mathbf{W}}_n} - {{\mathbf{W}}_n}'} \Big\|_F^2}$, and then add $\sum\limits_{n = 1}^{N + p} \sum\limits_{t = 1}^n \frac{1}{2}\Big\| {{\mathbf{w}}_{n + p + 1}} -$ $\beta {{\mathbf{W}}_{n{ {\rm - t}}}}' \Big\|_F^2$ to replace $\sum\limits_{n = 1}^{N + p} {\frac{1}{2}\Big\| {{{\mathbf{w}}_{n + p + 1}} - \beta {{\mathbf{W}}_n}'} \Big\|_F^2}$ when the error rate drops suddenly, to decrease the difference among new hyperplanes, the original model hyperplanes and the every pre-model.

The problem is the combination of (3) and (14), we can get

Then

Then $\mathbf{W} = [\mathbf{W}_1, \ldots, \mathbf{W}_N, \mathbf{w}_{N+1}, \ldots, \mathbf{w}_{N+p}], \mathbf{w}_{N+p+1}$ and the prediction matrix $\mathbf{Y}'$ can be got. As for computational complexity, with the classes increasing, it increases linearly. However, we can also use the learning framework presented in (14), it can work well within certain range and is easier in implementation, since it does not need to be iterated. So we use the method presented in (15) to correct the diagnosis when the accuracy of method presented in (14) suddenly drops a lot.

B Fault Diagnosis Processes

The old fault diagnosis model has been built by the old process data $\mathbf{x_s}$ and its corresponding label matrix ${{\mathbf{Y}}_s} \in \{ normal, faul{t_1}, \ldots, faul{t_{(N - 1)}}\}$. Considering the fault diagnosis of the new processes, we use the old model and the new process data to build a new model in the new complex process, and the ${{\mathbf{Y}}_t} \in \{ normal, faul{t_1}, \ldots, faul{t_{(N +M- 1)}}\}$ is the new process data label matrix. The procedures are as follows.

1) TCA Process: As mentioned earlier, the efficiency of transfer learning increases as the distribution difference between old process data and new process data decreases. So we need to seek a mapping to minimize the difference in a subspace for a good transferring effect. First, Maximum Mean Difference (MMD) based on the Reproduction Kernel Hilbert Space (RKHS) is used to represent the distribution difference of two datasets, and let one dataset $\mathbf{X}$ satisfy $P$ distribution, ${\mathbf{X}^s} = \{ {\mathbf{x}}_1^s, \ldots, {\mathbf{x}}_{{n_1}}^s\}$ and the other dataset satisfy $Q$ distribution which is set as ${\mathbf{X}^t} = \{ {\mathbf{x}}_1^t, \ldots, {\mathbf{x}}_{{n_1}}^t\}$ When n and m tend to infinity, the MMD between data set ${{\mathbf{X}}^s}$ and ${{\mathbf{X}}^t}$ can be expressed as

As for process data, the distribution difference between the old process data and the new process data can be expressed as

The nonlinear mapping function can be obtained by minimizing (17). With the method presented in [24], (17) could be converted into

where ${\mathbf{K}} = \left[{\begin{array}{*{20}{c}} {{{\mathbf{K}}_{S, S}}} & {{{\mathbf{K}}_{S, T}}}\\ {{{\mathbf{K}}_{T, S}}} & {{{\mathbf{K}}_{T, T}}} \end{array}} \right]$.

${{\mathbf{l}}_{i, j}} = \left\{ {\begin{array}{*{20}{c}} {\frac{1}{{m_s^2}}, {{\mathbf{x}}_i}, {{\mathbf{x}}_j} \in {{\mathbf{X}}_{src}}}\\ {\frac{1}{{m_t^2}}, {{\mathbf{x}}_i}, {{\mathbf{x}}_j} \in {{\mathbf{X}}_{tar}}}\\ { - \frac{1}{{{m_s}{m_t}}}, {\rm others}} \end{array}} \right.$

$\mu$: Trade-off parameter.

${\mathbf{H}} = {{\mathbf{I}}_{{n_1} + {n_2}}} - \frac{1}{{{n_1} + {n_2}}}{\mathbf{1}}{{\mathbf{1}}^T}$.

${{\mathbf{I}}_{{n_1} + {n_2}}}$: Constant value matrix.

The solution of (18) is the first d eigenvalues of ${({\mathbf{I}} + \mu {\mathbf{KLK}})^{ - 1}}{\mathbf{KLK}}$. And the processed process data ${\mathbf{X}} \in {\mathbf{R}^{d \times m}}$ is obtained after the treatment. The processed data of the old process and the new process are used in the future modeling process.

2) Fault Diagnosis Model of the Old Process: Since LS-SVM framework makes it possible to write the LOO error in a closed form, we use it to express the old process model. And the LOO error is an unbiased estimator of the classifier generalization error and it can be used in model selection [20]. Consider the problem of LS-SVM

where ${\xi _i}$ is a slack variable, minimizing $\frac{C}{2}\sum\limits_{i = 1}^M {{\xi _i}^2}$ to allow as many samples as possible to satisfy the constraint, and the model parameters $({\mathbf{W}}, {\mathbf{b}})$ are found by introducing Lagrangian function and solving the optimization problem. The optimal ${\mathbf{W}}$ is expressed as ${\mathbf{W}} = \sum\limits_{i = {\mathbf{1}}}^M {{{\mathbf{ \pmb{\mathsf{ α}} }}_i}} \phi ({{\mathbf{x}}_{i}})$. And $({\boldsymbol{\alpha }}, b)$ are found to solve $\left[{\begin{array}{*{20}{c}} {{\mathbf{K}} + \frac{1}{C}{\mathbf{I}}} & {\mathbf{1}}\\ {{{\mathbf{1}}^T}} & 0 \end{array}} \right]\; \left[{\begin{array}{*{20}{c}} {\mathbf{A}}\\ {\mathbf{B}} \end{array}} \right] = \left[{\begin{array}{*{20}{c}} {\mathbf{Y}}\\ {\mathbf{0}} \end{array}} \right]$. Then we get the optimal hyperplanes parameters ${\mathbf{W'}}$ in the old process.

3) Build A New Diagnosis Model in the New Process: After the two steps above, we have got what we need to build new diagnosis model as (15). After solving the optimization problem (15), the optimal hyperplane parameter ${\mathbf{W}}$ and ${\mathbf{b}}$ of the new process will be obtained.

4) Get Fault Diagnosis Results: The diagnosis results of the new process data can be shown as:

$\mathop {\mathbf{Y}}\limits^ \wedge = {{\mathbf{W}}^T}{\mathbf{X}} + {\mathbf{b}}$. For each sample ${{\mathbf{x}}_i}$, ${y_i}$ is the corresponding class, such as normal, fault 2, fault N, and so on. Therefore, we can achieve fault detection and fault classification simultaneously.

Ⅳ. SIMULATION EXPERIMENT

The experimental process is divided into two parts. Firstly, we validate the effectiveness and feasibility of the method with an accepted simulation platform (TE process). Then we verify the performance of the algorithm with the actual complex process (ore grinding classification process). In the simulation, all experiments are based on class expansion, 4 classes are taken for example, each class is used as the unknown class of the new process in turn, and the other three classes are used as known classes of the old process. The algorithm is verified by the cyclic cross validation. Similarly, we also expanded 3 classes to 4, 5, 6 classes in the new process.

A Algorithm Validation in Tennessee-Eastman (TE) Process

TE process is a real simulation of the manufacturing facilities of Tennessee Eastman chemical company. It is often used to test and verify the fault detection and diagnosis methods. The process is subjected to an indefinite Gauss noise perturbation so that we can study the control of the process. TE process consists of 12 operation variables and 41 measurement variables, and a total of preset 21 faults are included. There is a zero vector which can generate 20 kinds of faults (the last fault is not included) in the disturbance module. From [25], [26], faults 3, 4, 5, 9, 15 are usually difficult to detect. Since TE process is an actual industrial production process, it has six modes of operation to meet the market demand. The data generated by different operation modes are used to represent the different distributions between the old process and the new process. The classes of each expanded experimental process are shown in Table Ⅰ.

Table  Ⅰ.  CLASS DISTRIBUTION OF EACH CASE IN THE TE EXPERIMENTS

The total number of classes	Class distribution
3 classes	normal, fault 1, fault 2
4 classes	normal, fault 1, fault 2, fault 3
5 classes	normal, fault 1, fault 2, fault 3, fault 4
6 classes	normal, fault 1, fault 2, fault 3, fault 4, fault 5

 | Show Table

DownLoad: CSV

This experiment is used to compare the performance of IMTL and the traditional fault diagnosis method KPCA (Kernel Principal Component Analysis). In both old process and new process, fault 1 is introduced from the 100th sample point, and fault 2 is introduced from the 200th sample point. Fig. 1 is the result of fault diagnosis by KPCA in the old process. Fig. 2 is the diagnosis result of the new process by using the old KPCA model. SPE and T2 are the monitoring indices of KPCA, and dotted line is the control limit. If the monitoring index does not exceed the control limit, the process is normal, otherwise, the process is in some fault states. Obviously, the method has a good fault detection effect in old process. However, the discernment of fault 2 in the new process is bad because of the distribution difference. And the fault 1 is not detected exactly. Thus, the traditional method of fault diagnosis is no longer applicable when distribution difference exists. The simulation result of IMTL is shown in Fig. 3. In the simulation graph, The $Y$-axis represents the fault diagnosis result matrix, and the 3 kinds of lines represent the distributions of 3 columns of the data in the result matrix. Because our model is based on LS-SVM, for each sample, the predictive value of the correct class should be close to 1 and the wrong one should be close to $-1$. So in the picture, the highest point represents the current class of each sample.

Figure  1.  KPCA detecting results of the old process.

DownLoad: Full-Size Img PowerPoint

Figure  2.  KPCA detecting results of the new process.

DownLoad: Full-Size Img PowerPoint

Figure  3.  The diagnosis result of IMTL.

DownLoad: Full-Size Img PowerPoint

300 new process examples and 300 old process examples are selected to verify the superiority of IMTL compared with the method MULTIpLE presented in [17]. When MULTIpLE is used to classify the process data, we must extend it with kernel function first because of the nonlinearity of process data. The final simulation results are shown in Fig. 4, and the result of IMTL has been shown in Fig. 3. We can see that MULTIpLE can diagnose faults in general, but the boundary fault points can not be distinguished very well. However IMTL works well without this problem. Figs. 5 and 6 are the results of merely adding a global regularization term and a TCA term to MULTIpLE, respectively. After domain adaption by TCA, the classification graph is smoother and the accuracy is higher. So it is very necessary to dispose data in the model transfer with TCA. The global regularization term reduces graphic fluctuation. It optimizes the whole algorithm.

Figure  4.  The diagnosis result of MULTIpLE.

DownLoad: Full-Size Img PowerPoint

Figure  5.  The result of adding global regularization.

DownLoad: Full-Size Img PowerPoint

Figure  6.  The result of adding TCA.

DownLoad: Full-Size Img PowerPoint

The old process model and a few labeled sample of new process are used to train the new process model. We use 100 sets of examples to test the model, Figs. 7-9 represent using 10, 20, and 100 new examples to train new model respectively. We do verification experiments ten times, and the line chart of accuracy rate is shown in Fig. 10. Comparing the three groups of experiments, we come to the conclusion that the more the new process examples are, the better the results can be acquired. But when a new process which has only 20 or 10 labeled examples, the diagnosis results of IMTL also have a great guidance significance.

Figure  7.  The result with 10 labeled samples.

DownLoad: Full-Size Img PowerPoint

Figure  8.  The result with 20 labeled samples.

DownLoad: Full-Size Img PowerPoint

Figure  9.  The result with 100 labeled samples.

DownLoad: Full-Size Img PowerPoint

Figure  10.  The accuracy rate of 10 experiments.

DownLoad: Full-Size Img PowerPoint

We do a series of experiments on multi-faults classification and obtain the simulation results when there are some new faults in the new process. There are 3 classes in the old process, Fig. 11 is the diagnosis effect of the original faults in the new process. Figs. 12-14 indicate that the fault classes in the new process extend from 3 to 4, 5, and 6, respectively. For TE process, there are 3 known classes in old process, the effect of fault classification is not very desirable when the class of the new process increases to 6. In this case, we use the method described in (15) to adjust the model, and Fig. 15 is the adjusted effect. In addition, for the fault 3 and fault 4 in TE process, which are difficult to distinguish with many other methods, our algorithm can separate it correctly.

Figure  11.  The diagnosis result of 3 classes in the new process.

DownLoad: Full-Size Img PowerPoint

Figure  12.  The diagnosis result of 4 classes in the new process.

DownLoad: Full-Size Img PowerPoint

Figure  13.  The diagnosis result of 5 classes in the new process.

DownLoad: Full-Size Img PowerPoint

Figure  14.  The diagnosis result of 6 classes in the new process.

DownLoad: Full-Size Img PowerPoint

Figure  15.  The diagnosis effect after the model adjustment.

DownLoad: Full-Size Img PowerPoint

B Ore Grinding Process Fault Diagnosis

Hydro-Cyclone grading is an important part of ore grinding classification process. Because of the nonlinearity and the serious coupling among the variables, hydro-cyclone grading process is often with high failure rate and must be monitored. Typical process is shown in Fig. 16. During the process, the crushed row ore will be sent to the ball mill by adjusting the feeding device. At the same time, supplementary water and steel ball are added according to a certain proportion. After the ball mill rotating, the ore pulp is discharged and enters the slime dam together with the supplementary water. The mixed ore pulp is conveyed to hydro-cyclone under the action of the slurry pump with a certain hydro-cyclone pressure. The ore pulp passes into the hydro-cyclone and forms a rotary current inside the hydro-cyclone. Then the tiny particles are discharged through the overflow of the hydro-cyclone and become the final products. The coarse particles (grit) are discharged from the ore discharge outlets of the hydro-cyclone, and then the grit is returned to the ball mill to carry out cyclic grinding.

Figure  16.  Hydro-Cyclone classification technological process.

DownLoad: Full-Size Img PowerPoint

In ore grinding classification process, the detectable variables mainly include ore feeding flow (FIT2003), the ore feed pressure of hydro-cyclones (PIT2001), pump sump level (LIT2004 and LIT2005), ore feed concentration of hydro-cyclones (DIT2001), overflow granularity (AT2001), the overflow concentration (DIT2002), flow-rate of overflow (FIT2005), and the amount of the returning sand (RS). The controllable variables mainly include the amount of supplementary water (FIT2004) in slime dam and the rotational speed of turning pump (ZJB2DL). The 11 main variables above are selected as monitor data in the experiments. Because there exist serious coupling and nonlinearity among the liquid level control of the slime dam, the control of the hydro-cyclone pressure, and the control of the cyclone concentration, many kinds of faults occur frequently during the process. The typical faults and the causes are shown in Table Ⅱ.

Table  Ⅱ.  TYPICAL FAULTS AND CAUSES

Monitoring variables	The creative phenomenon	The possible reasons
Pressure of hydro-cyclone	Data abnormal	a) Hydro-cyclone blocking
		b) Liquid level of slim dam abnormal
		c) Concentration abnormal
		d) Slurry pump abnormal
Feed concentration of hydro-cyclone	Data abnormal	a) Supplementary water abnormal
		b) Liquid level of slim dam abnormal
Liquid level of slime dam	Data abnormal	a) Pressure of hydro-cyclone abnormal
		b) Supplementary water abnormal
		c) Production load abnormal
Supplementary water of slime dam	Data abnormal	a) Fault of water supply valve
		b) Fault of pipeline
Overflow granularity of hydro-cyclone	Data abnormal	a) Pressure of hydro-cyclone abnormal
		b) Feed concentration of hydro-cyclone abnormal
		c) Change of ore quality
		d) Fault of granularity analyzer

 | Show Table

DownLoad: CSV

In our experiments, the normal data is collected in regular producing process and the fault data is collected in debugging state. The experiments use the data of 5 kinds of fault states and 1 normal state. The fault 1 is caused by the blockage of the outlet of the hydro-cyclone. The fault 2 is caused by the abnormal of supplementary water. The fault 3 is caused by the abnormal of production load. The fault 4 is caused by the fault of feed water valve. And the fault 5 is caused by the granularity analyzer fault. The label is defined according to the experienced workers. The empirical values of each variable in the six typical states are shown in Table Ⅲ.

Table  Ⅲ.  THE EMPIRICAL VALUES OF VARIABLES IN EACH MODE

	Normal	Fault 1	Fault 2	Fault 3	Fault 4	Fault 5
ZJB2DL	235-308	0	218-253	0	240-308	218-321
FIT2003	587-694	0.029-523	259-336	0.058-326	671.3-676	327-521
DIT2001	56-60	0.028-52.8	0.028-52.8	57.4-57.9	57.47-58.71	57.80-60.01
PIT2001	0.049-0.070	0.002-0.007	0.003-0.078	0-0.02	0-0.04	0.03-0.06
FIT2005	301-438	12.73-451	227-338	12.73-338	0.05-0.70	87.0-247.7
DIT2002	30.1-42.39	9.08-57.84	9.14-24.65	99.83	308-335	44.0-58.07
LIT2004	4.16-4.20	4.17-4.2	4.16-4.18	1.25-4.17	1.15-4.16	1.66-2.34
LIT2005	1.10-1.34	1.10-1.34	1.18-4.18	2.72-2.77	2.73-2.77	1.79-1.85
FIT2004	93.66-95.25	93.66-95.25	93.66-94.10	99.83	99.83	98.0-99.57
AT2001	89.76-90.10	89.76-90.10	89.76-90.2	0.29	98.0-99.6	0.29
RS	280-300	430-480	280-300	280-300	280-300	280-300

 | Show Table

DownLoad: CSV

Firstly, we use the method MULTIpLE and IMTL to diagnose ore grinding classification process. The effects are shown in Figs. 17 and 18. Even if the MULTIpLE method is extended to a nonlinear method, the effect is much worse than that of IMTL, which fully demonstrates the necessity of our improvement.

Figure  17.  The diagnosis result of MULTIpLE.

DownLoad: Full-Size Img PowerPoint

Figure  18.  The diagnosis result of IMTL.

DownLoad: Full-Size Img PowerPoint

Figs. 19 and 20 are the results of adding the following two terms to objective function on the MULTIpLE. Term 1 is a global regularization term and term 2 is a TCA term. After adding the two items, the figure has a smaller fluctuation and a smoother waveform compared with Fig. 17. Fig. 21 is the contribution of the two items in the error rate. From the result, there is a marked drop in error rate when only adding term 1 or term 2, and the effect of the two terms in the TE process is more obvious. The error rate of our method IMTL is much lower than that of MUTIpLE when adding the two items.

Figure  19.  The result of adding global regularization.

DownLoad: Full-Size Img PowerPoint

Figure  20.  The result of adding TCA.

DownLoad: Full-Size Img PowerPoint

Figure  21.  Error rate comparison.

DownLoad: Full-Size Img PowerPoint

In the simulation results, Figs. 22-25 are the simulation result of training new model by using old model and a small amount of samples in the new process. The results of training with 10, 20 and 100 labeled samples are shown in Figs. 22-25 is a line chart of the corresponding accuracy. We can see that there will be a high accuracy even though the labeled samples are rare in new process. So the algorithm can be applied to a new process without many labeled data, which is also a superiority of the model transfer learning. It greatly shortens the modeling time and cost.

Figure  22.  The result with 10 labeled samples.

DownLoad: Full-Size Img PowerPoint

Figure  23.  The result with 20 labeled samples.

DownLoad: Full-Size Img PowerPoint

Figure  24.  The result with 100 labeled samples.

DownLoad: Full-Size Img PowerPoint

Figure  25.  The accuracy rate of 10 experiments.

DownLoad: Full-Size Img PowerPoint

Similar to the TE process simulation, we also use the ore grinding classification process data to do a series of fault classification experiments. When the old process has 3 known classes, the simulation results of 3, 4, 5, 6 classes in new process are correspondingly shown in Figs. 27-29, respectively.

Figure  26.  The diagnosis result of 3 classes in new process.

DownLoad: Full-Size Img PowerPoint

Figure  27.  The diagnosis result of 4 classes in new process.

DownLoad: Full-Size Img PowerPoint

Figure  28.  The diagnosis result of 5 classes in new process.

DownLoad: Full-Size Img PowerPoint

Figure  29.  The diagnosis result of 6 classes in new process.

DownLoad: Full-Size Img PowerPoint

The classification effect of the first 5 classes is better. And with the increase of new classes, the overall performance almost keeps unchanged. After adding the sixth new category, the overall performance decreases. It indicates that the data of fault 5 is similar to the data of other faults in some variables. Some classes are not affected, which indicates that these classes are less coupled with fault 5. In these simulation results, the more new classes the new process owns, or the more complex the couplings among the faults are, the worse effect of overall fault separation will be. At this moment, we need to modify the model to decrease the coupling. The adjusted effect is shown in Fig. 30. It can be seen that each fault can be diagnosed accurately after adjustment.

Figure  30.  The diagnosis effect after the model adjustment.

DownLoad: Full-Size Img PowerPoint

Ⅴ. CONCLUSIONS

A promising method IMTL of fault diagnosis is presented in this paper. For the process data with different distributions, an effective domain adaptation processing strategy is used, which creates good conditions for model reconstruction. Aiming at the problem that new faults may occur in new process, because of the same feature space, new faults can be regarded as a linear combination of the original faults, thus we obtain the representation of each new fault and realize the diagnosis of all faults in the new process. Moreover, the proposed method can realize fault detection and fault classification synchronously. The results based on complex process data demonstrate the effectiveness and practicability of the method. In future research, we expected to explore the setting and optimization strategies for the weights of regularized terms in the model.