Advanced Methods for Fault Detection

Size: px
Start display at page:

Download "Advanced Methods for Fault Detection"

Transcription

1 Advanced Methods for Fault Detection Piero Baraldi Agip KCO Introduction Piping and long to eploration distance pipelines activities Piero Baraldi

2 Maintenance Intervention Approaches & PHM Maintenance Intervention Unplanned Planned Corrective Replacement or repair of failed units Scheduled Replacement or Repair following a predefined schedule Conditionbased Monitor the health of the component and then decide on repair actions based on the degradation level assessed Predictive Predict the Remaining Useful Life (RUL) of the component and then decide on repair actions based on the predicted RUL PHM

3 3 Fault Detection Piero Baraldi 3 Agip KCO Introduction Piping and long to eploration distance pipelines activities 3 Piero Baraldi

4 Fault Detection: objective 4 Equipment Measured signals Piero Baraldi

5 Fault Detection: objective 5 5 f Automatic algorithm Normal condition Equipment f Forcing functions Measured signals Piero Baraldi 5

6 Fault Detection: analogy 6 6 f Automatic algorithm Normal condition Equipment f Forcing functions Measured signals f f Piero Baraldi 6

7 7 Methods for Fault Detection: Limit-based Data-driven Piero Baraldi 7

8 8 Methods for Fault Detection: Limit-based Data-driven Piero Baraldi 8

9 Data & Information for fault detection (I) 9 Normal operation ranges of key signals Eample: Pressurizer of a PWR nuclear reactor Water level. m 3.8 m Abnormal condition Abnormal condition time Normal operation range

10 Methods for fault detection (I) Normal operation ranges of key signals Limit Value-Based Fault Detection Eample: Pressurizer of a PWR nuclear reactor Water level. m 3.8 m Abnormal condition Abnormal condition time Normal operation range

11 Methods for fault detection (I) Normal operation ranges of key signals Limit Value-Based Fault Detection Eample: Pressurizer of a PWR nuclear reactor Water level Drawbacks: comple plant too many signals to monitor no early detection control systems operations may hide small anomalies (the signal remains in the normal range although there is a process anomaly) not applicable to fault detection during operational transients. m 3.8 m Abnormal condition Abnormal condition time Normal operation range

12 Methods for Fault Detection: Limit-based Data-driven

13 Data & Information for fault detection (III) 3 Normal operation ranges of key signals Historical signal measurements in normal condition Eample: Liquid Steam Pressuretemperattemperat ure ure Spray flow Surge line flow Heaters power Level Pressure Water level

14 s The fault detection approach Real measurements MODEL OF COMPONENT BEHAVIOR IN NORMAL CONDITION ŝ ŝ t 4 Signal reconstructions s t Pb. t COMPARISON t s ŝ s ŝ Residuals Pb. t DECISION NORMAL CONDITION: No maintenance ABNORMAL CONDITION: maintenance required Piero Baraldi 4

15 s The fault detection approach Real measurements MODEL OF COMPONENT BEHAVIOR IN NORMAL CONDITION ŝ ŝ t 5 Signal reconstructions s t Pb. t COMPARISON t s ŝ s ŝ Residuals Pb. t DECISION NORMAL CONDITION: No maintenance ABNORMAL CONDITION: maintenance required Piero Baraldi 5

16 6 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

17 7 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

18 Model Requirements () 8 NORMAL CONDITION Real measurements Signal reconstructions t t M O D E L nc ˆ nc ˆ t t ACCURACY: ˆ nc 8

19 Model Requirements () 9 ABNORMAL CONDITION Real measurements Signal reconstructions t M O D E nc ˆ nc ˆ t OK t L t ROBUSTNESS ˆ nc nc 9

20 Model Requirements (3) ABNORMAL CONDITION Real measurements Signal reconstructions t t M O D E L nc ˆ nc ˆ t t NO OK NO SPILL-OVER

21 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

22 PRINCIPAL COMPONENT ANALYSIS (PCA) ˆ t t MODEL OF COMPONENT BEHAVIOR IN NORMAL CONDITIONS ˆ t t Piero Baraldi

23 PCA for fault detection Auto-associative model n Auto- Associative Model nc ˆ nc ˆ nc ˆn nc ˆ i f,,, i,, n n Piero Baraldi

24 PCA for fault detection Auto-associative model n Auto- Associative Model nc ˆ nc ˆ nc ˆn nc ˆ i f,,, i,, n n Empirical model built using training patterns = historical signal measurements in normal plant condition Signal X nc k N nc nc j kj Nj nc n kn nc Nn Observation Piero Baraldi

25 PCA for fault detection 5 Training pattern Test pattern: input Test pattern: output X nc k N nc nc nc ˆ ( ˆ j kj Nj (,, ) nc,, ˆ X nc n n ) nc nc n kn nc Nn = historical signal measurements in normal plant condition = measured signals at current time = signal reconstructions (epected values of the signals in normal condition) n PCA nc ˆ nc ˆ nc ˆn Piero Baraldi

26 What is PCA? 6 PCA: space transformation from an n-dimensional space to a λ-dimensional space (λ<n) Retaining most of the information (loosing the least information). 3 Piero Baraldi

27 PCA Eample 7 B (,) A (,) X C (, ) Piero Baraldi

28 PCA Eample 8 B (,) ' A (,) X C (, ) ' U Piero Baraldi

29 PCA Eample 9 B (,) ' A (,) X C (, ) ' U Piero Baraldi

30 Basic Concepts 3 Two signals are highly correlated or dependent One is enough! u s s Piero Baraldi

31 Basic Concepts 3 Two signals are highly correlated or dependent key underlying phenomena One is enough! Areas of variance in data Focus on directions along which the ervations have high variance u 8 6 s - -. s s s Piero Baraldi

32 What does PCA do? 3 ) Find Principal Components: PC is the direction of maimum variance PC,, PC n are orthogonal to PC and describe maimum residual variance s PC PC s - Piero Baraldi

33 What does PCA do? 33 ) Find Principal Components: PC is the direction of maimum variance PC,, PC n are orthogonal to PC and describe maimum residual variance 3 Piero Baraldi

34 What does PCA do? 34 Step ) Find Principal Components: PC is the direction of maimum variance PC,, PC n are orthogonal to PC and describe maimum residual variance s PC PC Step ) PCA approimation: ignore the PCs of lower significance s Lost small information number of dimensions from n= to = 4 Piero Baraldi

35 How does PCA works? (step ) 35 ) Find principal components: V Compute the covariance matri of X nc 8 V nc nc T nc nc X X X X Empirical mean matri N i i i V Piero Baraldi

36 How does PCA works? (step ) 36 ) Find principal components: V Compute the covariance matri of X nc V nc nc T nc nc X X X X N i i i V Piero Baraldi

37 Piero Baraldi PCA Eample : Variance Matri 37 X X X X X X V T T A (,) ), ( C (,) B X X

38 How does PCA works? (step ) 38 8 ) Find principal components: V Compute the covariance matri of nc X, p,, p n n Find the n eigenvectors of and the corresponding eigenvalues : p V Vp p Piero Baraldi

39 Piero Baraldi PCA Eample : Eigenvalue 39 V Eigenvalues from: 4 ) det( 4 4 det det ) det( I V I V p Vp

40 PCA Eample: Eigenvectors 4 Vp p V 4 First eigenvector: p p p Vp p p p p 4 p p p p p 4 p 4 p p p p p p Piero Baraldi

41 Piero Baraldi PCA Eample: Eigenvectors 4 Second eigenvector: V p Vp p p p p p p p p p p p p p p Vp p p p 4

42 How does PCA works? (step ) 4 8 ) Find principal components: V Compute the covariance matri of nc X, p,, p n n Find the n eigenvectors of and the corresponding eigenvalues : p V Vp p p p T.8 T 9.5 highest.4 smallest Piero Baraldi

43 Eample: the trasformed basis p p p p T.8 T Piero Baraldi

44 How does PCA works? (step ) 44 8 ) Find principal components: V Compute the covariance matri of nc X, p,, p n n Find the n eigenvectors of and the corresponding eigenvalues : p V Vp p p p T.8 T 9.5 highest.4 smallest P p p, Piero Baraldi

45 How does PCA works? (step ) 45 ) Find principal components: V Compute the covariance matri of Find the n eigenvalues of and the corresponding eigenvectors p, p,, p n P represents an orthonormal basis: pi p i p PP T I j i j X n nc V Piero Baraldi

46 Piero Baraldi PCA Eample: P 46 V p Vp 4 p p P

47 Piero Baraldi PCA eample: P is an orthonormal basis 47 i i i p p p p p I PP T p p p p p

48 How does PCA works? (step ) 48 ) Find principal components: V Compute the covariance matri of V nc nc nc T nc nc X X X X P ( n, n) Find the n eigenvectors of and the corresponding eigenvalues : X V n P represents an orthonormal basis U the projection of X -nc on the new basis is given by: U X nc P The inverse relation is: nc T X UP Piero Baraldi

49 PCA Eample : projection of X 49 B (,) A (,) C (, ) X Piero Baraldi

50 PCA Eample : projection of X 5 C B (,) (, ) p p PC A (,) PC Piero Baraldi

51 PCA Eample : projection of X 5 PC PC C B (,) (, ) p A (,) p C ( ) A ( ) B (,) PC PC U XP Piero Baraldi

52 X Eample: projection of on the new basis 5 nc X nc U X nc P PC p p PC Piero Baraldi

53 X Eample: projection of on the new basis 53 nc U X nc P PC p p nc T X UP PC Piero Baraldi

54 Properties of the PCs 54 PCs are uncorrelated p i p j Data can be transformed from the original to the trasformed bases and viceversa without any loss of information (multiplication for P and P T ) PCs are ordered the first few retains the most of the variation present in the data. % Var PC i i i,, n i Piero Baraldi

55 Why does PCA work? () 55 PC arg w ma Var X nc w Var w ma T X w X w X w T X nc nc T X T T Var X w L( w, ) w Vw w w nc nc nc w w T Vw nc Lagrange multiplier To be maimized! L( w, ) Vw w Eigenvectors T w are solutions of Which eigenvector is maimizing the variance w T Vw? Piero Baraldi L( w, )

56 Why does PCA work? () 56 Eigenvectors w T are solutions of Vw w Which eigenvector is maimizing the variance w T Vw? T T w Vw w w The eigenvector associated to the largest eigenvalue Piero Baraldi

57 How does PCA work? (Step ) 57 ) PCA approimation = ignore the PCs of lower significance map the ervations in a subspace identified by the first eigenvectors p n,, p XP 8 nc X nc X P PC 8.8 X nc P PC PC P p p p Piero Baraldi

58 How does PCA work? (Step 3) 58 ) PCA approimation = map the ervations in a subspace n identified by the first n eigenvectors p,, p XP ) Antitrasform: XP P T PC Piero Baraldi

59 Piero Baraldi 59 PCA for fault detection: Historical data nc Nn Nj nc N kn kj k nc n j nc nc X P nc X Find from

60 PCA for fault detection: 6 Historical data nc k N nc nc Measured signals at present time: X j kj Nj nc n kn nc Nn Find (,, n ) P from X nc Trasform and project P I m looking at the measurements considering only the directions that are most meaningful in normal condition (directions of maimum variance) Antitrasform ˆ nc P P T Signal reconstructions ˆ ˆ nc normal condition nc abnormal condition Piero Baraldi I loose only the irrelevant noise The process is changed

61 Eercise Measured signals at present time: Signal reconstructions? Normal or abnormal condition? (, ) available historical signal measurements in normal plant condition Piero Baraldi

62 Eercise : Solution Measured signals at present time: p Step : find principal components:, (, p ) p p available historical signal measurements in normal plant condition Piero Baraldi

63 Eercise : Solution Measured signals at present time: p Step : find principal components, p (, ) p p PC Piero Baraldi

64 Eercise : Solution Measured signals at present time: p Step : find principal components, (, Step (PCA approimation): keep only PC of p ),i.e. p p p PC Piero Baraldi

65 Eercise : Solution Measured signals at present time: Step : find principal components (, Step (PCA approimation): keep only PC of Step 3 (antitrasform): 8 6 measuraments reconstruction ˆ nc P P T ),i.e. p ˆ Piero Baraldi nc normal condition

66 Eercise Measured signals at present time: Signal reconstructions? Normal or abnormal condition? (, ) available historical signal measurements in normal plant condition Piero Baraldi

67 Eercise : Solution Measured signals at present time: Step : find principal components p (, p Step (PCA approimation): keep only PC of ),i.e. p p p PC Piero Baraldi

68 Eercise : Solution Measured signals at present time: Step : find principal components p (, p Step (PCA approimation): keep only PC of ),i.e. p p p PC Piero Baraldi

69 Eercise : Solution Measured signals at present time: Step : find principal components (, Step (PCA approimation): keep only PC of Step 3 (antitrasform): 8 ˆ nc P P T ),i.e. p 6 4 measuraments reconstruction ˆ Piero Baraldi nc Abnormal condition

70 PCA remarks 7 Computational time: Training time = computational time necessary to find the Principal Components is proportional to the number of measured signals (n) Eecution time: very short (only matri multiplications) OK for online applications Piero Baraldi

71 PCA remarks 7 Performance: Accuracy = satisfactory t t M O D E L ˆ t t Piero Baraldi

72 PCA remarks 7 Performance: Accuracy = satisfactory Low robustness and spillover effects on highly correlated signals t t M O D E L ˆ t t Unsatisfactory for dataset characterized by highly non linear relationships. Piero Baraldi

73 73 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

74 Auto Associative Kernel Regression (AAKR)

75 What is AAKR? Auto-associative model n Auto- Associative Model ˆ ˆ ˆn ˆ i f,,, i,, n n Empirical model built using training patterns = historical signal measurements in normal plant condition Signal nc nc j n nc X k kj kn nc nc N Nj Nn Observation 75

76 76 How does AAKR work? Training pattern Test pattern: input Output X nc k N nc nc j kj Nj (,, n ) nc nc nc ˆ ( ˆ,, ˆ ) X n nc nc n kn nc Nn = historical signal measurements in normal plant condition = measured signals at current time = signal reconstructions (epected values of the signals in normal condition) nc ˆ n AAKR nc ˆ nc ˆn 76

77 77 How does AAKR work? Training pattern X Test pattern: input Test pattern: output nc k N nc nc j kj Nj (,, n ) nc nc nc ˆ ( ˆ,, ˆ ) n nc n kn nc Nn = historical signal measurements in normal plant condition = measured signals at current time = weighted sum of the training patterns: 77

78 78 How does AAKR work? Training pattern X Test pattern: input Test pattern: output nc k N nc nc j kj Nj (,, n ) nc nc nc ˆ ( ˆ,, ˆ ) n nc n kn nc Nn = historical signal measurements in normal plant condition = measured signals at current time = weighted sum of the training patterns: On all the training pattern ˆ nc j N k w( k) N k w( k) nc kj 78

79 79 How does AAKR work? Output nc ˆ ( ˆ nc,, ˆ nc n ) = weighted sum of the training patterns: On all the training pattern ˆ nc j N k w( k) k nc kj weights w(k) = similarity measures between (the test and the k-th training pattern): n w N w( k) d ( k ) h ( k) e h and nc k nc with d ( k) ( j ) Euclidean distance between and j kj low weight nc k high weight h = bandwidth parameter 79

80 Bandwidth parameter d= w=.4/h d=h w=.4/h d=h w=.5/h d=3h w=.4/h w d h w( d 3h) w 4 w 8 6 h=. h= d 8

81 Eample Signal values at current time: Signal reconstructions? Normal or abnormal condition? (,, n ) available historical signal measurements in normal plant condition 8

82 Eample : Solution Signal values at current time: ( ˆ nc nc ˆ,, (,, n nc ˆ n ) Signal reconstructions: based on the available historical signal measurements in normal plant condition ) ˆ nc normal condition 8

83 Eample Signal values at current time: Signal reconstructions? Normal or abnormal condition? (,, n ) available historical signal measurements in normal plant condition 83

84 84 Eample : Solution Signal values at current time: ( ˆ nc nc ˆ,, (,, n nc ˆ n ) Signal reconstructions: based on the available historical signal measurements in normal plant condition ) ˆ nc abnormal condition available historical signal measurements in normal plant condition 84

85 85 AAKR: Computational Time Computational time: No training of the model Test: computational time depends on the number of training patterns (N) and on the number of signals (n) d ( k) n j ( j nc kj ) 85

86 86 AAKR Performance: Accuracy Accuracy: depends on the training set: N Accuracy 86

87 87 AAKR Performance: Accuracy () Accuracy: depends on the training set: N Accuracy Few patterns and not well distributed in the training space Inaccurate reconstruction 87

88 AAKR Robustness and resistance to the spill-over effect 88

89 Case study: Monitoring a Turbine for Energy Production 89 6 Temperature Sensors in different position Highly Correlated Signals Abnormal conditions occur on the first.95 signal

90 Residual Residual Case study: Traditional AAKR Traditional AAKR Simulated abnormal residuals conditions Time Traditional AAKR Simulated abnormal residuals conditions Time DELAY IN THE DETECTION IMPOSSIBILITY TO IDENTIFY THE SIGNALS TRIGGERING THE ABNORMAL BEHAVIOR

91 Our Contribution: A modified AAKR method 9 Ԧ - Δ Ԧ Fault Detection Fault No Fault Modified AAKR Ԧ nc Ԧ Pre-processing ψ Ԧ Kernel-Based Reconstruction Ԧ nc Projecting the measurement and the historical data in a new space Ԧ, Ԧ hist k ψ Ԧ, ψ Ԧ hist k Ԧ hist Ԧhist Ԧ Malfunctions causing variations of a small number of signals are more frequent than those causing variations of a large number of signals ψ Ԧ ψ Ԧ hist ψ Ԧ hist Ԧ Ԧ hist Ԧ hist Dilatation matri ψ Ԧ hist = D Ԧp P perm Ԧ hist Permutation matri Modification of the loci of equisimilarity points

92 Residual Residual Results Modified AAKR Traditional AAKR Simulated abnormal conditions Time Modified AAKR Traditional AAKR Simulated abnormal conditions Time Early Detection 3 Correct Diagnosis of the signal that triggers the alarms More Accurate 4 5 6

93 Application to a real case study 93

94 Application to a real case study 94

95 Application to a real case study 95

96 Visual Comparison with historical normal condition values 96

97 Visual Comparison with historical normal condition values 97

98 Correlation coefficients 98 Historical Values Modified AAKR reconstruction Traditional AAKR reconstruction

99 Modified AAKR 99 Traditional AAKR Modify AAKR Loci of equisimilarity points Ԧ Ԧ hist Ԧ hist Ԧ Ԧ hist Ԧ hist Accuracy OK! OK! Robustness NO! Especially with correlated signals. Robust reconstruction of the values epected in normal conditions. Correct identification of signals affected by abnormal condition 3. Good performance with correlated signals

100 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

101 Problem Statement (EdF): hundreds of signals Grouping AUTO-ASSOCIATIVE MODEL OF PLANT BEHAVIOR IN NORMAL CONDITION MEASURED SIGNALS COMPARISON MODEL MODEL MODEL m

102 Challenge: Optimal Grouping MEASURED SIGNALS COMPARISON HOW TO GROUP SIGNALS? MODEL MODEL MODEL m

103 Case Study (EdF) 3 COMPONENT TO BE MONITORED Reactor Coolant Pumps of a PWR Nuclear Power Plant 4 MEASURED 94 signals, 48 of which considered SIGNALS as most important by EdF eperts RECONSTRUCTION Auto-Associative Kernel Regression MODEL (AAKR) historical measurements test measurements reconstruction

104 Approaches to signal grouping 4 HOW TO SPLIT THE SIGNALS INTO SUBGROUPS? A-PRIORI CRITERIA (signals divided on the basis of ) location in the power plant correlation physical homogeneity others

105 Results 5 BEST A-PRIORI GROUPING: correlation signals WHITE: high correlation (.7 ] 33 GREY: medium correlation (.4.7] BLACK: low correlation [.4] signal belonging to group signal belonging to group signal belonging to group 3 signal belonging to group 4 signal belonging to group 5 4 signals CRITICALITY: how to group the signals which have a low degree of correlation with all the others?

106 Approaches to Signal Grouping 6 HOW TO SPLIT THE SIGNALS INTO SUBGROUPS? A-PRIORI CRITERIA (signals divided on the basis of ) location in the power plant correlation physical homogeneity others WRAPPER APPROACH n SIGNALS SEARCH ENGINE SEARCH ALGORITHM CANDIDATE GROUPS AAKR PERFORMANCE EVALUATION OPTIMAL GROUPING

107 The Wrapper Approach: Genetic Algorithms 7 SEARCH ENGINE Genetic Algorithms CHROMOSOME n genes = n signals GENE signal signal signal 3 signal i signal n GROUP LABEL (integer number) PERFORMANCE EVALUATION fitness = Accuracy Robustness

108 Results 8 OBTAINED GROUPINGS correlation GA MAIN DIFFERENCE: signals which have a low degree of correlation are divided in groups and/or mied with signals with a high degree of correlation

109 Results 9 correlation GA Accuracy = = Robustness NO Spill-Over = =

110 Results: Wrapper Approach AUTO-ASSOCIATIVE MODEL OF PLANT BEHAVIOR IN NORMAL CONDITIONS ŝ t COMPARISON MEASURED SIGNALS s s ŝ (SEAL OUTCOMING FLOW) t DECISION t ABNORMAL CONDITION: seal deterioration NORMAL CONDITION ABNORMAL CONDITION

111 Conclusions groups of correlated signals accurate AAKR reconstructions; big groups robust AAKR reconstructions; the wrapper GA search finds the optimal groups also for those signals with low degree of correlation with all the others.

112 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

113 Model based fault detection 3 SIGNAL RECONSTRUCTIONS ŝ MEASURED SIGNALS s MODEL OF COMPONENT BEHAVIOR IN NORMAL CONDITIONS ŝ t t t s COMPARISON t RESIDUALS Pb. s ŝ s ŝ t DECISION t NORMAL CONDITION: No maintenance ABNORMAL CONDITION: maintenance required 3

114 Abnormal condition detection: decision Basics of the decision: residual analysis r ˆ r r Normal condition Abnormal condition Methods Threshold-based approaches Stochastic approaches 4

115 Abnormal condition detection: decision Basics of the decision: residual analysis r ˆ r r Normal condition Abnormal condition Methods Threshold-based approaches Stochastic approaches 5

116 Threshold-based approaches r Abnormal condition detection t r Too small thresholds high false alarm rates (α) Normal conditions t Too large thresholds high missing alarm rates (β) r Abnormal conditions t 6

117 Stochastic approaches Residual (r)= random variable described by a probability law The probability law is different in case of normal/abnormal condition.5 Normal condition.5 Abnormal condition r r ~ N(, ) r r~ N(, ) 7

118 Stochastic approach: Sequential Probability Ratio Test (SPRT) R r r,, sequence of residuals at time t,t,,t N n, r N Binary hypothesis test: Null hypothesis (H ) Normal condition ~ i i, r N(, ) f r, N r Alternative hypothesis (H ) Abnormal condition ~ i i, r N(, ) f r, N r 8

119 SPRT: the decision L n P P R R n n H H is true is true f f r fr frn r fr frn ln L n H is true B Not enough information A H is true 9

120 SPRT Theorem False alarm B A ln ln P P D H is true D H is true Missing alarm

121 SPRT for the positive mean test Null hypothesis (H ) Normal condition Alternative hypothesis (H ) Abnormal condition ), N( r ~ ), ( N r ~ Sequential Formula! = ) ln( ln n n n n k k n k k n r L r r r L. ),,, ( ),,, ( n k k n k r k r n n n e e H r r r P H r r r P L

122 Eample: SPRT B ln ln L n H is true L ln ln ln( L L lnl L lnl ) r r 3 Not enough information Not enough information t A ln H is true

123 SPRT: parameters to be set the residual variance in normal condition (σ ) the epected offset amplitude (μ ) the maimum acceptable false alarm rate (α) the maimum acceptable missing alarm rate (β) 3

124 Signal value Eample: Reactor Coolant Pumps in a PWR Time interval Offset [-] No [-4] Yes (amplitude =.) [4-6] Yes (amplitude =.3) [6-8] Yes (amplitude =.34) [8-] Yes (amplitude =.46) erved True signal value Reconstruction Time 4

125 Residuals Eample: residuals Time No offset Maimum offset 5

126 L ln ln Eample: SPRT Parameter Value α. Β. μ μ.46 σ. L L L L r r H B ln is true SPRT inde Upper Threshold Lower Threshold SPRT value A ln H is true Time No offset Maimum offset 6

127 SPRT: performance Average Sample Number (ASN) necessary to deliver a decision Time interval Offset Estimated ASN Number of times in which a normal condition has been detected Number of times in which an abnormal condition has been detected [-] No. 5 [-4] amplitude = [4-6] amplitude = [6-8] amplitude = [8-] amplitude =

128 8 Data-driven fault detection The approach Modeling the component behaviour in normal condition Model requirements Modeling techniques Principal Component Analysis (PCA) AutoAssociative Kernel Regression (AAKR) Eample of application Analysis of the residuals: Sequential Probability Ratio Test (SPRT) Some Conclusions

129 9 Fault detection challenges Fault detection systems are currently used to monitor component of comple industrial plants (e.g. electric power plants, transportation systems, process plants) Commercial software for fault detection is available A Challenge: Adaptability 9

130 Fault Detection challenge: adaptability of the methods 3 Eample: monitoring the turbine of an electric power plant t Identify historical signal measurements in normal plant operation P P Develop the model False alarm rate Plant operations slowly change with time T T t t time A New Model is necessary 3

131 Fault Detection challenge: adaptability of the methods 3 Eample: monitoring the turbine of an electric power plant t Identify historical signal measurements in normal plant operation P P Develop the model False alarm rate Plant operations slowly change with time T T t t time t Identify recent historical signal measurements in normal plant operation P Develop the new model P A New Model is necessary False alarm rates T T t t time Periodic Human Interventions for developing new models! high costs! 3

132 Fault detection challenge: adaptability of the methods 3 The detection model should be able to follow the process changes: Incremental learning of the new data that gradually becomes available No necessity of human intervention for: selecting recent normal operation data building the new model New data are coming P P T T Automatic updating of the model P T 3

Modern Reliability and Maintenance Engineering for Modern Industry

Modern Reliability and Maintenance Engineering for Modern Industry Modern Reliability and Maintenance Engineering for Modern Industry Piero Baraldi, Francesco Di Maio, Enrico Zio Politecnico di Milano, Department of Energy, Italy ARAMIS Srl, Italy LOW SMART KID POSITIONING

More information

Fault Detection in Nuclear Power Plants Components by a Combination of Statistical Methods

Fault Detection in Nuclear Power Plants Components by a Combination of Statistical Methods Fault Detection in Nuclear Power Plants Components by a Combination of Statistical Methods Francesco Di Maio, Piero Baraldi, Enrico Zio, Rédouane Seraoui To cite this version: Francesco Di Maio, Piero

More information

Condition Monitoring of Electrical Power. Plant Components During Operational

Condition Monitoring of Electrical Power. Plant Components During Operational Author manuscript, published in "Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability (2012) 568583" DOI : 10.1177/1748006X12463502 Condition Monitoring of Electrical

More information

Using Principal Component Analysis Modeling to Monitor Temperature Sensors in a Nuclear Research Reactor

Using Principal Component Analysis Modeling to Monitor Temperature Sensors in a Nuclear Research Reactor Using Principal Component Analysis Modeling to Monitor Temperature Sensors in a Nuclear Research Reactor Rosani M. L. Penha Centro de Energia Nuclear Instituto de Pesquisas Energéticas e Nucleares - Ipen

More information

Principal Components Analysis (PCA)

Principal Components Analysis (PCA) Principal Components Analysis (PCA) Principal Components Analysis (PCA) a technique for finding patterns in data of high dimension Outline:. Eigenvectors and eigenvalues. PCA: a) Getting the data b) Centering

More information

Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter

Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter (Chair) STF - China Fellow francesco.dimaio@polimi.it

More information

Introduction to Machine Learning

Introduction to Machine Learning 10-701 Introduction to Machine Learning PCA Slides based on 18-661 Fall 2018 PCA Raw data can be Complex, High-dimensional To understand a phenomenon we measure various related quantities If we knew what

More information

MACHINE LEARNING ADVANCED MACHINE LEARNING

MACHINE LEARNING ADVANCED MACHINE LEARNING MACHINE LEARNING ADVANCED MACHINE LEARNING Recap of Important Notions on Estimation of Probability Density Functions 2 2 MACHINE LEARNING Overview Definition pdf Definition joint, condition, marginal,

More information

Lecture 3: Latent Variables Models and Learning with the EM Algorithm. Sam Roweis. Tuesday July25, 2006 Machine Learning Summer School, Taiwan

Lecture 3: Latent Variables Models and Learning with the EM Algorithm. Sam Roweis. Tuesday July25, 2006 Machine Learning Summer School, Taiwan Lecture 3: Latent Variables Models and Learning with the EM Algorithm Sam Roweis Tuesday July25, 2006 Machine Learning Summer School, Taiwan Latent Variable Models What to do when a variable z is always

More information

1 Principal Components Analysis

1 Principal Components Analysis Lecture 3 and 4 Sept. 18 and Sept.20-2006 Data Visualization STAT 442 / 890, CM 462 Lecture: Ali Ghodsi 1 Principal Components Analysis Principal components analysis (PCA) is a very popular technique for

More information

A Data-driven Approach for Remaining Useful Life Prediction of Critical Components

A Data-driven Approach for Remaining Useful Life Prediction of Critical Components GT S3 : Sûreté, Surveillance, Supervision Meeting GdR Modélisation, Analyse et Conduite des Systèmes Dynamiques (MACS) January 28 th, 2014 A Data-driven Approach for Remaining Useful Life Prediction of

More information

Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation)

Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) PCA transforms the original input space into a lower dimensional space, by constructing dimensions that are linear combinations

More information

Robust signal reconstruction for condition monitoring of industrial components via a modified Auto Associative Kernel Regression method

Robust signal reconstruction for condition monitoring of industrial components via a modified Auto Associative Kernel Regression method Robust signal reconstruction for condition monitoring of industrial components via a modified Auto Associative Kernel Regression method Piero Baraldi, Francesco Di Maio, Pietro Turati, Enrico Zio To cite

More information

MACHINE LEARNING ADVANCED MACHINE LEARNING

MACHINE LEARNING ADVANCED MACHINE LEARNING MACHINE LEARNING ADVANCED MACHINE LEARNING Recap of Important Notions on Estimation of Probability Density Functions 22 MACHINE LEARNING Discrete Probabilities Consider two variables and y taking discrete

More information

Advances in Military Technology Vol. 8, No. 2, December Fault Detection and Diagnosis Based on Extensions of PCA

Advances in Military Technology Vol. 8, No. 2, December Fault Detection and Diagnosis Based on Extensions of PCA AiMT Advances in Military Technology Vol. 8, No. 2, December 203 Fault Detection and Diagnosis Based on Extensions of PCA Y. Zhang *, C.M. Bingham and M. Gallimore School of Engineering, University of

More information

Condition Monitoring for Maintenance Support

Condition Monitoring for Maintenance Support Journal of Marine Science and Application, Vol.6, No., January 26, PP***-*** Condition Monitoring for Maintenance Support BEERE William, BERG Øivind, WINGSTEDT Emil SAVOLAINEN Samuli 2, and LAHTI Tero

More information

Covariance and Correlation Matrix

Covariance and Correlation Matrix Covariance and Correlation Matrix Given sample {x n } N 1, where x Rd, x n = x 1n x 2n. x dn sample mean x = 1 N N n=1 x n, and entries of sample mean are x i = 1 N N n=1 x in sample covariance matrix

More information

Dimensionality Reduction

Dimensionality Reduction Lecture 5 1 Outline 1. Overview a) What is? b) Why? 2. Principal Component Analysis (PCA) a) Objectives b) Explaining variability c) SVD 3. Related approaches a) ICA b) Autoencoders 2 Example 1: Sportsball

More information

Issues and Techniques in Pattern Classification

Issues and Techniques in Pattern Classification Issues and Techniques in Pattern Classification Carlotta Domeniconi www.ise.gmu.edu/~carlotta Machine Learning Given a collection of data, a machine learner eplains the underlying process that generated

More information

Face Recognition and Biometric Systems

Face Recognition and Biometric Systems The Eigenfaces method Plan of the lecture Principal Components Analysis main idea Feature extraction by PCA face recognition Eigenfaces training feature extraction Literature M.A.Turk, A.P.Pentland Face

More information

Principal Component Analysis

Principal Component Analysis B: Chapter 1 HTF: Chapter 1.5 Principal Component Analysis Barnabás Póczos University of Alberta Nov, 009 Contents Motivation PCA algorithms Applications Face recognition Facial expression recognition

More information

Principal Component Analysis

Principal Component Analysis CSci 5525: Machine Learning Dec 3, 2008 The Main Idea Given a dataset X = {x 1,..., x N } The Main Idea Given a dataset X = {x 1,..., x N } Find a low-dimensional linear projection The Main Idea Given

More information

Independent Component Analysis for Redundant Sensor Validation

Independent Component Analysis for Redundant Sensor Validation Independent Component Analysis for Redundant Sensor Validation Jun Ding, J. Wesley Hines, Brandon Rasmussen The University of Tennessee Nuclear Engineering Department Knoxville, TN 37996-2300 E-mail: hines2@utk.edu

More information

PCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani

PCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani PCA & ICA CE-717: Machine Learning Sharif University of Technology Spring 2015 Soleymani Dimensionality Reduction: Feature Selection vs. Feature Extraction Feature selection Select a subset of a given

More information

Lecture 24: Principal Component Analysis. Aykut Erdem May 2016 Hacettepe University

Lecture 24: Principal Component Analysis. Aykut Erdem May 2016 Hacettepe University Lecture 4: Principal Component Analysis Aykut Erdem May 016 Hacettepe University This week Motivation PCA algorithms Applications PCA shortcomings Autoencoders Kernel PCA PCA Applications Data Visualization

More information

Machine Learning Lecture 2

Machine Learning Lecture 2 Machine Perceptual Learning and Sensory Summer Augmented 6 Computing Announcements Machine Learning Lecture 2 Course webpage http://www.vision.rwth-aachen.de/teaching/ Slides will be made available on

More information

Karhunen-Loéve (PCA) based detection of multiple oscillations in multiple measurement signals from large-scale process plants

Karhunen-Loéve (PCA) based detection of multiple oscillations in multiple measurement signals from large-scale process plants Karhunen-Loéve (PCA) based detection of multiple oscillations in multiple measurement signals from large-scale process plants P. F. Odgaard and M. V. Wickerhauser Abstract In the perspective of optimizing

More information

CS4495/6495 Introduction to Computer Vision. 8B-L2 Principle Component Analysis (and its use in Computer Vision)

CS4495/6495 Introduction to Computer Vision. 8B-L2 Principle Component Analysis (and its use in Computer Vision) CS4495/6495 Introduction to Computer Vision 8B-L2 Principle Component Analysis (and its use in Computer Vision) Wavelength 2 Wavelength 2 Principal Components Principal components are all about the directions

More information

Synergy between Data Reconciliation and Principal Component Analysis.

Synergy between Data Reconciliation and Principal Component Analysis. Plant Monitoring and Fault Detection Synergy between Data Reconciliation and Principal Component Analysis. Th. Amand a, G. Heyen a, B. Kalitventzeff b Thierry.Amand@ulg.ac.be, G.Heyen@ulg.ac.be, B.Kalitventzeff@ulg.ac.be

More information

Principal Component Analysis

Principal Component Analysis Principal Component Analysis Yingyu Liang yliang@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison [based on slides from Nina Balcan] slide 1 Goals for the lecture you should understand

More information

Process-Quality Monitoring Using Semi-supervised Probability Latent Variable Regression Models

Process-Quality Monitoring Using Semi-supervised Probability Latent Variable Regression Models Preprints of the 9th World Congress he International Federation of Automatic Control Cape own, South Africa August 4-9, 4 Process-Quality Monitoring Using Semi-supervised Probability Latent Variable Regression

More information

Metric-based classifiers. Nuno Vasconcelos UCSD

Metric-based classifiers. Nuno Vasconcelos UCSD Metric-based classifiers Nuno Vasconcelos UCSD Statistical learning goal: given a function f. y f and a collection of eample data-points, learn what the function f. is. this is called training. two major

More information

ON MODEL SELECTION FOR STATE ESTIMATION FOR NONLINEAR SYSTEMS. Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof

ON MODEL SELECTION FOR STATE ESTIMATION FOR NONLINEAR SYSTEMS. Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof ON MODEL SELECTION FOR STATE ESTIMATION FOR NONLINEAR SYSTEMS Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof Delft Center for Systems and Control, Delft University of Technology, Mekelweg 2, 2628 CD

More information

Likelihood Bounds for Constrained Estimation with Uncertainty

Likelihood Bounds for Constrained Estimation with Uncertainty Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 5 Seville, Spain, December -5, 5 WeC4. Likelihood Bounds for Constrained Estimation with Uncertainty

More information

Data Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Data Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Data Mining Dimensionality reduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 42 Outline 1 Introduction 2 Feature selection

More information

FAULT DETECTION AND ISOLATION WITH ROBUST PRINCIPAL COMPONENT ANALYSIS

FAULT DETECTION AND ISOLATION WITH ROBUST PRINCIPAL COMPONENT ANALYSIS Int. J. Appl. Math. Comput. Sci., 8, Vol. 8, No. 4, 49 44 DOI:.478/v6-8-38-3 FAULT DETECTION AND ISOLATION WITH ROBUST PRINCIPAL COMPONENT ANALYSIS YVON THARRAULT, GILLES MOUROT, JOSÉ RAGOT, DIDIER MAQUIN

More information

Kernel-Based Principal Component Analysis (KPCA) and Its Applications. Nonlinear PCA

Kernel-Based Principal Component Analysis (KPCA) and Its Applications. Nonlinear PCA Kernel-Based Principal Component Analysis (KPCA) and Its Applications 4//009 Based on slides originaly from Dr. John Tan 1 Nonlinear PCA Natural phenomena are usually nonlinear and standard PCA is intrinsically

More information

6.867 Machine learning

6.867 Machine learning 6.867 Machine learning Mid-term eam October 8, 6 ( points) Your name and MIT ID: .5.5 y.5 y.5 a).5.5 b).5.5.5.5 y.5 y.5 c).5.5 d).5.5 Figure : Plots of linear regression results with different types of

More information

Machine Learning Lecture 2

Machine Learning Lecture 2 Announcements Machine Learning Lecture 2 Eceptional number of lecture participants this year Current count: 449 participants This is very nice, but it stretches our resources to their limits Probability

More information

Principal Component Analysis

Principal Component Analysis Principal Component Analysis Introduction Consider a zero mean random vector R n with autocorrelation matri R = E( T ). R has eigenvectors q(1),,q(n) and associated eigenvalues λ(1) λ(n). Let Q = [ q(1)

More information

Principal Component Analysis and Singular Value Decomposition. Volker Tresp, Clemens Otte Summer 2014

Principal Component Analysis and Singular Value Decomposition. Volker Tresp, Clemens Otte Summer 2014 Principal Component Analysis and Singular Value Decomposition Volker Tresp, Clemens Otte Summer 2014 1 Motivation So far we always argued for a high-dimensional feature space Still, in some cases it makes

More information

Experimental Design and Data Analysis for Biologists

Experimental Design and Data Analysis for Biologists Experimental Design and Data Analysis for Biologists Gerry P. Quinn Monash University Michael J. Keough University of Melbourne CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv I I Introduction 1 1.1

More information

EECS 275 Matrix Computation

EECS 275 Matrix Computation EECS 275 Matrix Computation Ming-Hsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 6 1 / 22 Overview

More information

7. Variable extraction and dimensionality reduction

7. Variable extraction and dimensionality reduction 7. Variable extraction and dimensionality reduction The goal of the variable selection in the preceding chapter was to find least useful variables so that it would be possible to reduce the dimensionality

More information

Semi-Supervised Laplacian Regularization of Kernel Canonical Correlation Analysis

Semi-Supervised Laplacian Regularization of Kernel Canonical Correlation Analysis Semi-Supervised Laplacian Regularization of Kernel Canonical Correlation Analsis Matthew B. Blaschko, Christoph H. Lampert, & Arthur Gretton Ma Planck Institute for Biological Cbernetics Tübingen, German

More information

APPENDIX 1 NEYMAN PEARSON CRITERIA

APPENDIX 1 NEYMAN PEARSON CRITERIA 54 APPENDIX NEYMAN PEARSON CRITERIA The design approaches for detectors directly follow the theory of hypothesis testing. The primary approaches to hypothesis testing problem are the classical approach

More information

Classification. The goal: map from input X to a label Y. Y has a discrete set of possible values. We focused on binary Y (values 0 or 1).

Classification. The goal: map from input X to a label Y. Y has a discrete set of possible values. We focused on binary Y (values 0 or 1). Regression and PCA Classification The goal: map from input X to a label Y. Y has a discrete set of possible values We focused on binary Y (values 0 or 1). But we also discussed larger number of classes

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning Christoph Lampert Spring Semester 2015/2016 // Lecture 12 1 / 36 Unsupervised Learning Dimensionality Reduction 2 / 36 Dimensionality Reduction Given: data X = {x 1,..., x

More information

Introduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin

Introduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin 1 Introduction to Machine Learning PCA and Spectral Clustering Introduction to Machine Learning, 2013-14 Slides: Eran Halperin Singular Value Decomposition (SVD) The singular value decomposition (SVD)

More information

Linear Algebra Methods for Data Mining

Linear Algebra Methods for Data Mining Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 Linear Discriminant Analysis Linear Algebra Methods for Data Mining, Spring 2007, University of Helsinki Principal

More information

Advanced Introduction to Machine Learning CMU-10715

Advanced Introduction to Machine Learning CMU-10715 Advanced Introduction to Machine Learning CMU-10715 Principal Component Analysis Barnabás Póczos Contents Motivation PCA algorithms Applications Some of these slides are taken from Karl Booksh Research

More information

Principal component analysis

Principal component analysis Principal component analysis Angela Montanari 1 Introduction Principal component analysis (PCA) is one of the most popular multivariate statistical methods. It was first introduced by Pearson (1901) and

More information

ECE 521. Lecture 11 (not on midterm material) 13 February K-means clustering, Dimensionality reduction

ECE 521. Lecture 11 (not on midterm material) 13 February K-means clustering, Dimensionality reduction ECE 521 Lecture 11 (not on midterm material) 13 February 2017 K-means clustering, Dimensionality reduction With thanks to Ruslan Salakhutdinov for an earlier version of the slides Overview K-means clustering

More information

A methodology for fault detection in rolling element bearings using singular spectrum analysis

A methodology for fault detection in rolling element bearings using singular spectrum analysis A methodology for fault detection in rolling element bearings using singular spectrum analysis Hussein Al Bugharbee,1, and Irina Trendafilova 2 1 Department of Mechanical engineering, the University of

More information

Keywords: Multimode process monitoring, Joint probability, Weighted probabilistic PCA, Coefficient of variation.

Keywords: Multimode process monitoring, Joint probability, Weighted probabilistic PCA, Coefficient of variation. 2016 International Conference on rtificial Intelligence: Techniques and pplications (IT 2016) ISBN: 978-1-60595-389-2 Joint Probability Density and Weighted Probabilistic PC Based on Coefficient of Variation

More information

Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di Milano Italy

Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di Milano Italy Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di Milano Italy 2 Piero Baraldi Data Industry 4.0 2 Digitalization 2.8 Trillion GD (ZD)

More information

Strategy for signal classification to improve data quality for Advanced Detectors gravitational-wave searches

Strategy for signal classification to improve data quality for Advanced Detectors gravitational-wave searches Strategy for signal classification to improve data quality for Advanced Detectors gravitational-wave searches E. Cuoco 1, A.Torres-Forné 2,J.A. Font 2,7, J.Powell 3, R.Lynch 4, D.Trifiró 5 2 Universitat

More information

A Data-Driven Model for Software Reliability Prediction

A Data-Driven Model for Software Reliability Prediction A Data-Driven Model for Software Reliability Prediction Author: Jung-Hua Lo IEEE International Conference on Granular Computing (2012) Young Taek Kim KAIST SE Lab. 9/4/2013 Contents Introduction Background

More information

STA 414/2104: Lecture 8

STA 414/2104: Lecture 8 STA 414/2104: Lecture 8 6-7 March 2017: Continuous Latent Variable Models, Neural networks Delivered by Mark Ebden With thanks to Russ Salakhutdinov, Jimmy Ba and others Outline Continuous latent variable

More information

MRC: The Maximum Rejection Classifier for Pattern Detection. With Michael Elad, Renato Keshet

MRC: The Maximum Rejection Classifier for Pattern Detection. With Michael Elad, Renato Keshet MRC: The Maimum Rejection Classifier for Pattern Detection With Michael Elad, Renato Keshet 1 The Problem Pattern Detection: Given a pattern that is subjected to a particular type of variation, detect

More information

Basics of Multivariate Modelling and Data Analysis

Basics of Multivariate Modelling and Data Analysis Basics of Multivariate Modelling and Data Analysis Kurt-Erik Häggblom 2. Overview of multivariate techniques 2.1 Different approaches to multivariate data analysis 2.2 Classification of multivariate techniques

More information

Lecture 13. Principal Component Analysis. Brett Bernstein. April 25, CDS at NYU. Brett Bernstein (CDS at NYU) Lecture 13 April 25, / 26

Lecture 13. Principal Component Analysis. Brett Bernstein. April 25, CDS at NYU. Brett Bernstein (CDS at NYU) Lecture 13 April 25, / 26 Principal Component Analysis Brett Bernstein CDS at NYU April 25, 2017 Brett Bernstein (CDS at NYU) Lecture 13 April 25, 2017 1 / 26 Initial Question Intro Question Question Let S R n n be symmetric. 1

More information

Preprocessing & dimensionality reduction

Preprocessing & dimensionality reduction Introduction to Data Mining Preprocessing & dimensionality reduction CPSC/AMTH 445a/545a Guy Wolf guy.wolf@yale.edu Yale University Fall 2016 CPSC 445 (Guy Wolf) Dimensionality reduction Yale - Fall 2016

More information

Introduction to Signal Detection and Classification. Phani Chavali

Introduction to Signal Detection and Classification. Phani Chavali Introduction to Signal Detection and Classification Phani Chavali Outline Detection Problem Performance Measures Receiver Operating Characteristics (ROC) F-Test - Test Linear Discriminant Analysis (LDA)

More information

Independent Component Analysis. Slides from Paris Smaragdis UIUC

Independent Component Analysis. Slides from Paris Smaragdis UIUC Independent Component Analysis Slides from Paris Smaragdis UIUC 1 A simple audio problem 2 Formalizing the problem Each mic will receive a mi of both sounds Sound waves superimpose linearly We ll ignore

More information

PDEEC Machine Learning 2016/17

PDEEC Machine Learning 2016/17 PDEEC Machine Learning 2016/17 Lecture - Model assessment, selection and Ensemble Jaime S. Cardoso jaime.cardoso@inesctec.pt INESC TEC and Faculdade Engenharia, Universidade do Porto Nov. 07, 2017 1 /

More information

CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION

CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION 59 CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION 4. INTRODUCTION Weighted average-based fusion algorithms are one of the widely used fusion methods for multi-sensor data integration. These methods

More information

Failure Prognostics with Missing Data Using Extended Kalman Filter

Failure Prognostics with Missing Data Using Extended Kalman Filter Failure Prognostics with Missing Data Using Extended Kalman Filter Wlamir Olivares Loesch Vianna 1, and Takashi Yoneyama 2 1 EMBRAER S.A., São José dos Campos, São Paulo, 12227 901, Brazil wlamir.vianna@embraer.com.br

More information

Machine Learning. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Machine Learning. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Machine Learning Dimensionality reduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1395 1 / 47 Table of contents 1 Introduction

More information

Time Series Classification

Time Series Classification Distance Measures Classifiers DTW vs. ED Further Work Questions August 31, 2017 Distance Measures Classifiers DTW vs. ED Further Work Questions Outline 1 2 Distance Measures 3 Classifiers 4 DTW vs. ED

More information

CHAPTER 3 FEATURE EXTRACTION USING GENETIC ALGORITHM BASED PRINCIPAL COMPONENT ANALYSIS

CHAPTER 3 FEATURE EXTRACTION USING GENETIC ALGORITHM BASED PRINCIPAL COMPONENT ANALYSIS 46 CHAPTER 3 FEATURE EXTRACTION USING GENETIC ALGORITHM BASED PRINCIPAL COMPONENT ANALYSIS 3.1 INTRODUCTION Cardiac beat classification is a key process in the detection of myocardial ischemic episodes

More information

PCA and admixture models

PCA and admixture models PCA and admixture models CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar, Alkes Price PCA and admixture models 1 / 57 Announcements HW1

More information

Plant-wide Root Cause Identification of Transient Disturbances with Application to a Board Machine

Plant-wide Root Cause Identification of Transient Disturbances with Application to a Board Machine Moncef Chioua, Margret Bauer, Su-Liang Chen, Jan C. Schlake, Guido Sand, Werner Schmidt and Nina F. Thornhill Plant-wide Root Cause Identification of Transient Disturbances with Application to a Board

More information

Problem Set 3 Due Oct, 5

Problem Set 3 Due Oct, 5 EE6: Random Processes in Systems Lecturer: Jean C. Walrand Problem Set 3 Due Oct, 5 Fall 6 GSI: Assane Gueye This problem set essentially reviews detection theory. Not all eercises are to be turned in.

More information

Lecture 7: Con3nuous Latent Variable Models

Lecture 7: Con3nuous Latent Variable Models CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 7: Con3nuous Latent Variable Models All lecture slides will be available as.pdf on the course website: http://www.cs.toronto.edu/~urtasun/courses/csc2515/

More information

Building knowledge from plant operating data for process improvement. applications

Building knowledge from plant operating data for process improvement. applications Building knowledge from plant operating data for process improvement applications Ramasamy, M., Zabiri, H., Lemma, T. D., Totok, R. B., and Osman, M. Chemical Engineering Department, Universiti Teknologi

More information

STA 414/2104: Lecture 8

STA 414/2104: Lecture 8 STA 414/2104: Lecture 8 6-7 March 2017: Continuous Latent Variable Models, Neural networks With thanks to Russ Salakhutdinov, Jimmy Ba and others Outline Continuous latent variable models Background PCA

More information

Bayes Decision Theory - I

Bayes Decision Theory - I Bayes Decision Theory - I Nuno Vasconcelos (Ken Kreutz-Delgado) UCSD Statistical Learning from Data Goal: Given a relationship between a feature vector and a vector y, and iid data samples ( i,y i ), find

More information

Machine Learning for Software Engineering

Machine Learning for Software Engineering Machine Learning for Software Engineering Dimensionality Reduction Prof. Dr.-Ing. Norbert Siegmund Intelligent Software Systems 1 2 Exam Info Scheduled for Tuesday 25 th of July 11-13h (same time as the

More information

Dynamic Data-Driven Adaptive Sampling and Monitoring of Big Spatial-Temporal Data Streams for Real-Time Solar Flare Detection

Dynamic Data-Driven Adaptive Sampling and Monitoring of Big Spatial-Temporal Data Streams for Real-Time Solar Flare Detection Dynamic Data-Driven Adaptive Sampling and Monitoring of Big Spatial-Temporal Data Streams for Real-Time Solar Flare Detection Dr. Kaibo Liu Department of Industrial and Systems Engineering University of

More information

CS281 Section 4: Factor Analysis and PCA

CS281 Section 4: Factor Analysis and PCA CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we

More information

PCA, Kernel PCA, ICA

PCA, Kernel PCA, ICA PCA, Kernel PCA, ICA Learning Representations. Dimensionality Reduction. Maria-Florina Balcan 04/08/2015 Big & High-Dimensional Data High-Dimensions = Lot of Features Document classification Features per

More information

Impeller Fault Detection for a Centrifugal Pump Using Principal Component Analysis of Time Domain Vibration Features

Impeller Fault Detection for a Centrifugal Pump Using Principal Component Analysis of Time Domain Vibration Features Impeller Fault Detection for a Centrifugal Pump Using Principal Component Analysis of Time Domain Vibration Features Berli Kamiel 1,2, Gareth Forbes 2, Rodney Entwistle 2, Ilyas Mazhar 2 and Ian Howard

More information

Machine learning for pervasive systems Classification in high-dimensional spaces

Machine learning for pervasive systems Classification in high-dimensional spaces Machine learning for pervasive systems Classification in high-dimensional spaces Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Version

More information

Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations.

Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations. Previously Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations y = Ax Or A simply represents data Notion of eigenvectors,

More information

Interacting multiple-models, state augmented Particle Filtering for Fault Diagnostics

Interacting multiple-models, state augmented Particle Filtering for Fault Diagnostics Interacting multiple-models, state augmented Particle Filtering for Fault Diagnostics Michele Compare 1,2, Piero Baraldi 1, Pietro Turati 1, Enrico Zio 1,2,3,* 1 Energy Department, Politecnico di Milano,

More information

Discriminant analysis and supervised classification

Discriminant analysis and supervised classification Discriminant analysis and supervised classification Angela Montanari 1 Linear discriminant analysis Linear discriminant analysis (LDA) also known as Fisher s linear discriminant analysis or as Canonical

More information

Info-Greedy Sequential Adaptive Compressed Sensing

Info-Greedy Sequential Adaptive Compressed Sensing Info-Greedy Sequential Adaptive Compressed Sensing Yao Xie Joint work with Gabor Braun and Sebastian Pokutta Georgia Institute of Technology Presented at Allerton Conference 2014 Information sensing for

More information

Machine Learning 4771

Machine Learning 4771 Machine Learning 4771 Instructor: Tony Jebara Topic 7 Unsupervised Learning Statistical Perspective Probability Models Discrete & Continuous: Gaussian, Bernoulli, Multinomial Maimum Likelihood Logistic

More information

Data Preprocessing Tasks

Data Preprocessing Tasks Data Tasks 1 2 3 Data Reduction 4 We re here. 1 Dimensionality Reduction Dimensionality reduction is a commonly used approach for generating fewer features. Typically used because too many features can

More information

A Modified Incremental Principal Component Analysis for On-Line Learning of Feature Space and Classifier

A Modified Incremental Principal Component Analysis for On-Line Learning of Feature Space and Classifier A Modified Incremental Principal Component Analysis for On-Line Learning of Feature Space and Classifier Seiichi Ozawa 1, Shaoning Pang 2, and Nikola Kasabov 2 1 Graduate School of Science and Technology,

More information

Neuroscience Introduction

Neuroscience Introduction Neuroscience Introduction The brain As humans, we can identify galaxies light years away, we can study particles smaller than an atom. But we still haven t unlocked the mystery of the three pounds of matter

More information

Data reduction for multivariate analysis

Data reduction for multivariate analysis Data reduction for multivariate analysis Using T 2, m-cusum, m-ewma can help deal with the multivariate detection cases. But when the characteristic vector x of interest is of high dimension, it is difficult

More information

Robot Image Credit: Viktoriya Sukhanova 123RF.com. Dimensionality Reduction

Robot Image Credit: Viktoriya Sukhanova 123RF.com. Dimensionality Reduction Robot Image Credit: Viktoriya Sukhanova 13RF.com Dimensionality Reduction Feature Selection vs. Dimensionality Reduction Feature Selection (last time) Select a subset of features. When classifying novel

More information

Least Squares with Examples in Signal Processing 1. 2 Overdetermined equations. 1 Notation. The sum of squares of x is denoted by x 2 2, i.e.

Least Squares with Examples in Signal Processing 1. 2 Overdetermined equations. 1 Notation. The sum of squares of x is denoted by x 2 2, i.e. Least Squares with Eamples in Signal Processing Ivan Selesnick March 7, 3 NYU-Poly These notes address (approimate) solutions to linear equations by least squares We deal with the easy case wherein the

More information

COS 429: COMPUTER VISON Face Recognition

COS 429: COMPUTER VISON Face Recognition COS 429: COMPUTER VISON Face Recognition Intro to recognition PCA and Eigenfaces LDA and Fisherfaces Face detection: Viola & Jones (Optional) generic object models for faces: the Constellation Model Reading:

More information

Dimension Reduction Techniques. Presented by Jie (Jerry) Yu

Dimension Reduction Techniques. Presented by Jie (Jerry) Yu Dimension Reduction Techniques Presented by Jie (Jerry) Yu Outline Problem Modeling Review of PCA and MDS Isomap Local Linear Embedding (LLE) Charting Background Advances in data collection and storage

More information

Bearing fault diagnosis based on EMD-KPCA and ELM

Bearing fault diagnosis based on EMD-KPCA and ELM Bearing fault diagnosis based on EMD-KPCA and ELM Zihan Chen, Hang Yuan 2 School of Reliability and Systems Engineering, Beihang University, Beijing 9, China Science and Technology on Reliability & Environmental

More information

PRINCIPAL COMPONENT ANALYSIS

PRINCIPAL COMPONENT ANALYSIS PRINCIPAL COMPONENT ANALYSIS 1 INTRODUCTION One of the main problems inherent in statistics with more than two variables is the issue of visualising or interpreting data. Fortunately, quite often the problem

More information

Applied Machine Learning for Biomedical Engineering. Enrico Grisan

Applied Machine Learning for Biomedical Engineering. Enrico Grisan Applied Machine Learning for Biomedical Engineering Enrico Grisan enrico.grisan@dei.unipd.it Data representation To find a representation that approximates elements of a signal class with a linear combination

More information