Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter

Transcription

1 Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter (Chair) STF - China Fellow francesco.dimaio@polimi.it

2 Fault Detection, Diagnostics and Prognostics f 1 f 2 Forcing functions Measured signals

3 Fault Detection, Diagnostics and Prognostics Diagnostic system f 1 f 2 FAULT DETECTION (early recognition) Normal operation Forcing functions Measured signals

4 Fault Detection, Diagnostics and Prognostics Diagnostic system f 1 f 2 FAULT DETECTION (early recognition) Normal operation Forcing functions Measured signals f 1 FAULT CLASSIFICATION (correct assignment) f 2

5 Fault Detection, Diagnostics and Prognostics Diagnostic system f 1 f 2 FAULT DETECTION (early recognition) Normal operation Forcing functions Lifetime estimation Prognostic system Measured signals PROGNOSIS FAULT CLASSIFICATION (correct assignment) f 1 f 2

6 Diagnostic System: Elements { Mean residual Maimum wavelet coeff. Minimum wavelet coeff Plant Sensors T1 T2... M Data Pre- Processing 1 window s1 s2... s n Feature Selection 1 s f 2 s f... h s f h < n Classification Fault Type

7 Problem Statement: Challenges Comple Plants and Systems Large number of monitored signals with non-linear dependences Difficulties in Detecting and Classyfying Faults We resort to empirical models, rather than to analytical models Soft Computing Techniques: Artificial Neural Networks Support/Relevance Vector Machines Fuzzy Classifiers

8 Artificial Neural Networks

9 What Are (Artificial) Neural Networks? Multivariate, non linear interpolators Techniques capable of reconstructing the underlying comple I/O nonlinear relations by combining multiple simple functions (modeling & computational advantage) Empirical model built by training on input/output data (modeling advantage)

10 What Are (Artificial) Neural Networks? In an artificial neural network, variables are associated with nodes in a graph.

11 What Artificial Do Neural Networks Do? Regression y,, y y deterministic stochastic f, wˆ ~ y ~ y regression prediction

12 What Do Artificial Neural Networks Do? Classification (, c) c 1 Discriminant function c 2

13 What Are (Artificial) Neural Networks? The output (u) of a node is the result of nonlinear transformations (f) on the input variables (i) transmitted along the links of the graph. i 1 i 2 i n f u

14 What is the mathematical basis behind Artificial Neural Networks? Neural networks are universal approimators of multivariate non-linear functions. KOLMOGOROV (1957): For any real function f ( continuous in [0,1] n 1, 2,..., n), n2, there eist n(2n+1) functions ml () continuous in [0,1] such that f ( 2n1 n 1, 2,..., n ) l ml( m) l1 m1 where the 2n+1 functions l s are real and continuous. Thus, a total of (2n+1) functions l () and n(2n+1) functions ml () of one variable represent a function of n variables

15 What is the mathematical basis behind Artificial Neural Networks? Neural networks are universal approimators of multivariate non-linear functions. CYBENKO (1989): Let ( ) be a sigmoidal continuous function. The linear combinations are dense in [0,1] n. N n j j0 i1 i w ij j In other words, any function f: [0,1] n can be approimated by a linear combination of sigmoidal functions. Note that N is not specified.

16 ARTIFICIAL NEURAL NETWORKS INPUT Number of connections =117 OUTPUT

17 Forward Calculation (input-hidden) Multilayered Feedforward NN INPUT LAYER: each k-th node (k=1, 2,, n i ) receives the (normalized) value of the k-th component of the input vector and delivers the same value HIDDEN LAYER: each j-th node (j=1, 2,, n h ) receives and delivers z j f n i k 1 k w jk w j n i k1 k w jk w j0 0 with f typically sigmoidal

18 Forward Calculation (hidden-output) OUTPUT LAYER: each l-th node (l=1, 2,, n o ) receives and delivers u l f n h j1 z j w lj n h j1 z j w lj w l0 f typically linear or sigmoidal w l0

19 Setting the NN Parameters: Training Phase Available input/output patterns 1 (1), 2 (1) t (1) 1 (2), 2 (2) t (2) Synapsis Weights 1 (p), 2 (p) t (p) W 1 (np), 2 (np) t (np)

20 Setting The NN Parameters: Error Backpropagation Initialize weights to random values Update weights so as to minimize the average squared output deviation error (also called Energy Function): np n 1 o E u t 2 o 2 pl pl nn p o p1 l1 TRUE l-th output of the p-th pattern Computed l-th output of the p-th pattern Minimization by gradient descent algorithms

21 Error Backpropagation (output-hidden) Without loss of generality, set np=1 and define as a user-defined output importance (usually equal to 1) Updating weight w lj (output-hidden connections) being w lj 1 h ( n) lu j wlj( n 1) n o Learning coefficient Momentum

22 Error Backpropagation (hidden-input) Similarly to the updating of the output-hidden weigths, Updating weight w jk (hidden-input connections) being 1 i wjk ( n) ju j wjk ( n 1) n o Learning coefficient Momentum

23 Utilization of the Neural Network After training: Synaptic weights fied New input retrieval of information in the weights output Capabilities: Nonlinearity of sigmoids NN can learn nonlinear mappings Each node independent and relies only on local info (synapses) Parallel processing and fault-tolerance

24 CONCLUSIONS Advantages: No physical/mathematical modelling efforts. Automatic parameters adjustment through a training phase based on available input/output data. Adjustments such as to obtain the best interpolation of the functional relation between input and output. Disadvantages: black bo : difficulties in interpreting the underlying physical model.

25 Application: Detection of malfunctions in the secondary system of a nuclear power plant by neural networks

26 Objective Build and train a neural network to classify different malfunctions in the secondary system of a boiling water reactor

27 Boiling Water Reactor

28 Secondary System Turbine Vessel Condenser Pumps

29 Outputs Class 1: Leakage through the second high-pressure preheater Class 2: Leakage in the first high-pressure preheater to the drain tank Class 3: Leakage through the first high-pressure preheater drain back-up valve to the condenser Class 4: Leakage through high-pressure preheaters bypass valve Class 5: Leakage through the second high-pressure preheater drain back-up valve to the feedwater tank

30 Transients Class 2: Valve EA2 Class 1: Valve EA1 Class 4: Valve VB7 Class 5: Valve VA25 Class 3: Valve VB20

31 Variables inputs Variable Signal Unit 1 Position level for control valve EA1 % 2 Position level for control valve EB1 % 3 Temperature drain before VB3 ºC 4 Temperature feedwater after EA2 train A ºC 5 Temperature feedwater after EB2 train B ºC 6 Temperature drain 6 after VB1 ºC 7 Temperature drain 5 after VB2 ºC 8 Position level control valve before EA2 % 9 Position level control valve before EB2 % 10 Temperature feedwater before EB2 train B ºC

32 Sensors position Measurement: 36 sampling instants in [80, 290]s, one each 6 s Var 6 (ºC) Var 7 (ºC) Var 4 (ºC) Var 8 (%) Var 3 (ºC) Var 1 (%) Var 5 (ºC) Var 9 (%) Var 10 (ºC) Var 2 (%)

33 Input/Output patterns Input: The class assignment is performed dynamically as a two-step time window of the measured signals shifts to the successive time t+1 Output: number of the class to which the variables belong

34 Training set A training set was constructed containing 8 transients for each class. For a transient of a given class the 35 patterns used to train the network take the following form ( t) ( t 1) (2) (36) ( t 1) 1 ( t) (1) (35) 2 ( t) ( t 1) (2) (36) 2 ( t 1) (1) ( t) (35) (2) ( t) ( t 1) (36) 10 (1) ( t 1) ( t) (35) class class class class

35 Network architecure N h, η, α optimized with grid optimization Optimal values N h : 17 η: 0.55 α : 0.6

36 Test results class Neural Network and True values.test DATA a b c d e input pattern

37 Comments The real-valued outputs provided by the neural network closely approimate the class integer values since early times. As time progresses, the network output class assignment approaches more and more closely the true class integer.

38 New transients Response of the neural network performance desirable when a new transient is found Identification that something happens Classification as don t know

39 New transients New class Class 6: Steam line valve to the second highpressure preheater closing Class 6: Valve VA6

40 New transients Class Class Neural Network and True values.class 6 Close to Class 2 but with a larger deviation a Value provided by the neural network close to input pattern 4b

41 New transient signal value Temperature drain after VB2 the large output error is due to the fact that some of the inputs which are fed into the network fall outside the ranges of the input values of the training transients second transient upper training level lower training level input patterns

42 Conclusions A neural network has been constructed and trained to classify different malfunctions in the secondary system of a boiling water reactor. The network uses as input patterns the information provided by signals that are monitored in the plant and provides as output an estimate of the class assignment. When a transient belonging to a different class is fed to the network, it is capable of detecting that the pattern does not belong to any of the classes for which it was trained