Lessons learned from the theory and practice of. change detection. Introduction. Content. Simulated data - One change (Signal and spectral densities)

Size: px
Start display at page:

Download "Lessons learned from the theory and practice of. change detection. Introduction. Content. Simulated data - One change (Signal and spectral densities)"

Transcription

1 Lessons learned from the theory and practice of change detection Simulated data - One change (Signal and spectral densities) - Michèle Basseville IRISA / CNRS, Rennes, France michele.basseville@irisa.fr Introduction Detection of changes Content Stochastic models (static, dynamic) uncertainties Parameterized models (physical interpretation, diagnostics) Damage deviation in the parameter vector Changes in the mean Changes in the spectrum Many changes of interest are small Changes in the system dynamics and vibration-based SHM Early detection of (small) deviations is useful Key issue (): which function of the data should be handled? Example: flutter monitoring Key issue (): how to make the decision? 3 4

2 Changes in the mean Hypothesis testing Key concepts - Independent data Likelihood p θ (y i ) Log-likelihood ratio s i = ln p θ (y i ) p θ (y i ) E θ (s i ) < E θ (s i ) > p θ (Y N Likelihood ratio Λ N = ) p θ (Y N) = i p θ (y i ) i p θ (y i ) Log-likelihood ratio S N = ln ΛN = N i= s i Hypotheses H H Simple θ θ Known parameter values Composite Θ Θ Unknown parameter values Simple hypotheses : Likelihood ratio test If Λ N λ/s N h : decide H, H otherwise Composite hypotheses : Generalized likelihood ratio (GLR) Λ N = sup θ Θ p θ (Y N) sup θ Θ p θ (Y N) = p θ (Y N ) p θ (Y N ) Maximize the likelihoods w.r.t. unknown values of θ and θ 6 5 On-line change detection, unknown onset time Simple case: Known θ - CUSUM algorithm θ o t o θ? n Ratio of likelihoods under H and H : t i= p θ (y i ) ki= p θ (y i ) ki=t p θ (y i ) = ki=t p θ (y i ) ki=t p θ (y i ) = Λk t Hypothesis H θ = θ known ( i k) Hypothesis H t s.t. θ = θ ( i<t ) θ (t i k) Alarm time t a : t a =min{k :g k h} decision function Maximize over the unknown onset time t : ( t ) k = arg max j k = arg max j k = arg max j k j i= p θ (y i ) Λ k j S k j, k i=j p θ (y i ) Sk j =ln Λk j Estimated onset time: t g k = max j k Sk j = lnλ kˆt 7 8

3 CUSUM algorithm (Contd.) Lesson CUSUM algorithm (Contd.) - Gaussian example g k = max j k Sk j = S k min j k Sj = Sk m k, m k = min j k Sj N (µ, σ ), θ = µ, p θ (y) = σ π exp (y i µ) σ s i = ln p µ (y i ) p µ (y i ) t a = min{k :S k m k + h} Adaptative threshold = ( (yi σ µ ) (y i µ ) ) g k = (g k + s k ) + g k = ( S k k N k +) +, Nk = Nk I(g k )+ = ν σ y i µ ν, ν = µ µ ( t ) k = t a N ta + Sliding window with adaptive size 9 S k involves k i= y i : Integrator (with adaptive threshold) CUSUM algorithm - Gaussian example (Contd.) Composite case: Unknown θ Modified CUSUM algorithms y k Minimum magnitude of change Weighted CUSUM GLR algorithm -4 S k Double maximization g k = max j k sup θ S k j (θ ) g k Gaussian case, additive faults: second maximization explicit.

4 Unknown θ - Gaussian example (Contd.) Changes in the spectrum Minimum magnitude of change ν m Lesson Key concepts - Dependent data Decreasing mean T k = k i= y i µ + ν m Increasing mean U k = k i= y i µ ν m Conditional likelihood p θ (y i Y i ) Log-likelihood ratio s i = ln p θ (y i Y i ) p θ (y i Y i ) E θ (s i ) < M k = max j k T j m k = min j k U j t a =min{k :M k T k h} t a =min{k :U k m k h} E θ (s i ) > p θ (Y N Likelihood ratio Λ N = ) p θ (Y N) = i p θ (y i Y i ) i p θ (y i Y i ) Log-likelihood ratio S N = ln ΛN = N i= s i 3 4 Key concepts - Dependent data (Contd.) Key concepts - Dependent data (Contd.) θ : reference parameter, known (or identified) Which residuals? Lesson 3 Y k : N-size sample of new measurements Likelihood ratio computationally complex Build a residual ζ significantly non zero when damage Efficient score : likelihood sensitivity w.r.t. parameter vector Other estimating fonctions Residual Estimating function ζ N (θ, Y N ) Characterized by: E θ ζ N (θ, Y N )= θ = θ Innovation not sufficient for monitoring the dynamics. Mean sensitivity J (θ ) and covariance Σ(θ ) of ζ N (θ ) 5 6

5 Key concepts - Dependent data (Contd.) Changes in the dynamics and vibration-based SHM The residual is asymptotically Gaussian ζ N (θ ) N (, Σ(θ )) if P θ N ( J (θ ) δθ, Σ(θ )) if P θ + δθ N (On-board) χ -test in the residual small deviations Structural monitoring : Eigenstructure monitoring X k+ = F X k + V k F Φ λ = λ Φ λ Y k = H X k ϕ λ = H Φλ ζ T N Σ J (J T Σ J ) J T Σ ζ N h Canonical parameter : θ = Λ vec Φ modes mode shapes Noises and uncertainty on θ taken into account. 7 8 Example: flutter monitoring Detecting structural changes Monitoring a damping coefficient Reference data covariances Hankel matrix H Left null space S s.t. S T H = (S T O(θ )=) Fresh data covariances Hankel matrix H Check if ζ = S T H ζ asympt. Gaussian, test: χ in ζ Test ρ ρ c against ρ<ρ c Write the subspace-based residual ζ as a cumulative sum Introduce a minimum change magnitude (actual change magnitude unknown) 9 Run two CUSUM tests in parallel (actual change direction unknown)

6 Flutter monitoring (Contd) Application to real datasets Ariane booster launcher during a launch scenario on the ground (not during a real flight test) Running the test with critical values With Laurent Mével, Maurice Goursat, Albert Benveniste Free COSMAD Toolbox to used with Scilab Automatic identification. Each symbol: processing 5 sec. data. Conclusion Advanced statistical signal processing mandatory for SHM On-line test for ρ c = ρ = ρ () c. A statistical framework enlightens the meaning and increases the power of a number of familiar operations integration, averaging, sensitivity, adaptive thresholds & windows Change detection useful for (vibration-based) SHM On-line test for ρ c = ρ = ρ () c <ρ () c. Bottom: g n reflects ρ<ρ c. Top: g + n reflects ρ>ρ c. Current investigations flutter monitoring (with Dassault, Airbus), handling the temperature effect on civil structures (with LCPC) 3 4

Uncertainty Quantification for Stochastic Subspace Identification of Multi-Setup Measurements

Uncertainty Quantification for Stochastic Subspace Identification of Multi-Setup Measurements Uncertainty Quantification for Stochastic Subspace Identification of Multi-Setup Measurements Michael Döhler, Xuan-Binh Lam, Laurent Mevel INRIA Centre Rennes Bretagne Atlantique, Rennes, France Palle

More information

SISTHEM. Inférence Statistique pour la Surveillance d Intégrité de Structures. 5th April 2004

SISTHEM. Inférence Statistique pour la Surveillance d Intégrité de Structures. 5th April 2004 SISTHEM Statistical Inference for STructural HEalth Monitoring Inférence Statistique pour la Surveillance d Intégrité de Structures Proposal for a joint Cnrs - Inria - U. Rennes 1 project-team at Irisa

More information

Données, SHM et analyse statistique

Données, SHM et analyse statistique Données, SHM et analyse statistique Laurent Mevel Inria, I4S / Ifsttar, COSYS, SII Rennes 1ère Journée nationale SHM-France 15 mars 2018 1 Outline 1 Context of vibration-based SHM 2 Modal analysis 3 Damage

More information

Subspace-based damage detection on steel frame structure under changing excitation

Subspace-based damage detection on steel frame structure under changing excitation Subspace-based damage detection on steel frame structure under changing excitation M. Döhler 1,2 and F. Hille 1 1 BAM Federal Institute for Materials Research and Testing, Safety of Structures Department,

More information

Change Detection Algorithms

Change Detection Algorithms 5 Change Detection Algorithms In this chapter, we describe the simplest change detection algorithms. We consider a sequence of independent random variables (y k ) k with a probability density p (y) depending

More information

Automated Modal Parameter Estimation For Operational Modal Analysis of Large Systems

Automated Modal Parameter Estimation For Operational Modal Analysis of Large Systems Automated Modal Parameter Estimation For Operational Modal Analysis of Large Systems Palle Andersen Structural Vibration Solutions A/S Niels Jernes Vej 10, DK-9220 Aalborg East, Denmark, pa@svibs.com Rune

More information

A CUSUM approach for online change-point detection on curve sequences

A CUSUM approach for online change-point detection on curve sequences ESANN 22 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges Belgium, 25-27 April 22, i6doc.com publ., ISBN 978-2-8749-49-. Available

More information

Uncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department

Uncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department Decision-Making under Statistical Uncertainty Jayakrishnan Unnikrishnan PhD Defense ECE Department University of Illinois at Urbana-Champaign CSL 141 12 June 2010 Statistical Decision-Making Relevant in

More information

GARCH Models Estimation and Inference

GARCH Models Estimation and Inference GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in

More information

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)? ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we

More information

Vibration Based Health Monitoring for a Thin Aluminum Plate: Experimental Assessment of Several Statistical Time Series Methods

Vibration Based Health Monitoring for a Thin Aluminum Plate: Experimental Assessment of Several Statistical Time Series Methods Vibration Based Health Monitoring for a Thin Aluminum Plate: Experimental Assessment of Several Statistical Time Series Methods Fotis P. Kopsaftopoulos and Spilios D. Fassois Stochastic Mechanical Systems

More information

Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems

Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Jeremy S. Conner and Dale E. Seborg Department of Chemical Engineering University of California, Santa Barbara, CA

More information

Composite Hypotheses and Generalized Likelihood Ratio Tests

Composite Hypotheses and Generalized Likelihood Ratio Tests Composite Hypotheses and Generalized Likelihood Ratio Tests Rebecca Willett, 06 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve

More information

Project-Team sigma2. Signaux, modèles et algorithmes

Project-Team sigma2. Signaux, modèles et algorithmes INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE Project-Team sigma2 Signaux, modèles et algorithmes Rennes THEME 4A d' ctivity eport 2003 Table of contents 1. Team 1 2. Overall Objectives

More information

Detection and Estimation of Multiple Fault Profiles Using Generalized Likelihood Ratio Tests: A Case Study

Detection and Estimation of Multiple Fault Profiles Using Generalized Likelihood Ratio Tests: A Case Study Preprints of the 16th IFAC Symposium on System Identification The International Federation of Automatic Control Detection and Estimation of Multiple Fault Profiles Using Generalized Likelihood Ratio Tests:

More information

Handling parametric and non-parametric additive faults in LTV Systems

Handling parametric and non-parametric additive faults in LTV Systems 1 / 16 Handling parametric and non-parametric additive faults in LTV Systems Qinghua Zhang & Michèle Basseville INRIA & CNRS-IRISA, Rennes, France 9th IFAC SAFEPROCESS, Paris, France, Sept. 2-4, 2015 2

More information

Asynchronous Multi-Sensor Change-Point Detection for Seismic Tremors

Asynchronous Multi-Sensor Change-Point Detection for Seismic Tremors Asynchronous Multi-Sensor Change-Point Detection for Seismic Tremors Liyan Xie, Yao Xie, George V. Moustakides School of Industrial & Systems Engineering, Georgia Institute of Technology, {lxie49, yao.xie}@isye.gatech.edu

More information

Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary

Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary ECE 830 Spring 207 Instructor: R. Willett Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we saw that the likelihood

More information

Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models Introduction to General and Generalized Linear Models Generalized Linear Models - part II Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs.

More information

Multi-Order Covariance Computation for Estimates in Stochastic Subspace Identification Using QR Decompositions

Multi-Order Covariance Computation for Estimates in Stochastic Subspace Identification Using QR Decompositions Preprints of the 19th World Congress The International Federation of Automatic Control Multi-Order Covariance Computation for Estimates in Stochastic Subspace Identification Using QR Decompositions Michael

More information

Detection theory 101 ELEC-E5410 Signal Processing for Communications

Detection theory 101 ELEC-E5410 Signal Processing for Communications Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off

More information

MATH5745 Multivariate Methods Lecture 07

MATH5745 Multivariate Methods Lecture 07 MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction

More information

Machine Learning Linear Classification. Prof. Matteo Matteucci

Machine Learning Linear Classification. Prof. Matteo Matteucci Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)

More information

P U B L I C A T I O N I N T E R N E N o MERGING SENSOR DATA FROM MULTIPLE TEMPERATURE SCENARIOS FOR VIBRATION MONITORING OF CIVIL STRUCTURES

P U B L I C A T I O N I N T E R N E N o MERGING SENSOR DATA FROM MULTIPLE TEMPERATURE SCENARIOS FOR VIBRATION MONITORING OF CIVIL STRUCTURES I R I P U B L I C A T I O N I N T E R N E N o 1824 S INSTITUT DE RECHERCHE EN INFORMATIQUE ET SYSTÈMES ALÉATOIRES A MERGING SENSOR DATA FROM MULTIPLE TEMPERATURE SCENARIOS FOR VIBRATION MONITORING OF CIVIL

More information

Lecture 7 Introduction to Statistical Decision Theory

Lecture 7 Introduction to Statistical Decision Theory Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7

More information

Machine Learning. Lecture 3: Logistic Regression. Feng Li.

Machine Learning. Lecture 3: Logistic Regression. Feng Li. Machine Learning Lecture 3: Logistic Regression Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2016 Logistic Regression Classification

More information

10. Composite Hypothesis Testing. ECE 830, Spring 2014

10. Composite Hypothesis Testing. ECE 830, Spring 2014 10. Composite Hypothesis Testing ECE 830, Spring 2014 1 / 25 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve unknown parameters

More information

Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics)

Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming

More information

Bayesian inference. Justin Chumbley ETH and UZH. (Thanks to Jean Denizeau for slides)

Bayesian inference. Justin Chumbley ETH and UZH. (Thanks to Jean Denizeau for slides) Bayesian inference Justin Chumbley ETH and UZH (Thanks to Jean Denizeau for slides) Overview of the talk Introduction: Bayesian inference Bayesian model comparison Group-level Bayesian model selection

More information

GARCH Models Estimation and Inference. Eduardo Rossi University of Pavia

GARCH Models Estimation and Inference. Eduardo Rossi University of Pavia GARCH Models Estimation and Inference Eduardo Rossi University of Pavia Likelihood function The procedure most often used in estimating θ 0 in ARCH models involves the maximization of a likelihood function

More information

ON THE APPLICATION OF A SUBSPACE-BASED FAULT DETECTION METHOD *

ON THE APPLICATION OF A SUBSPACE-BASED FAULT DETECTION METHOD * ON THE APPLICATION OF A SUBSPACE-BASED FAULT DETECTION METHOD * L.Mevel, M.Basseville, A.Benveniste, M.Goursat, M.Abdelghani IRISA-INRIA, Campus de Beaulieu, 35042 Rennes cedex, France! L.Hermans LMS-International,

More information

Parameter Estimation and Fitting to Data

Parameter Estimation and Fitting to Data Parameter Estimation and Fitting to Data Parameter estimation Maximum likelihood Least squares Goodness-of-fit Examples Elton S. Smith, Jefferson Lab 1 Parameter estimation Properties of estimators 3 An

More information

Parametric Output Error Based Identification and Fault Detection in Structures Under Earthquake Excitation

Parametric Output Error Based Identification and Fault Detection in Structures Under Earthquake Excitation Parametric Output Error Based Identification and Fault Detection in Structures Under Earthquake Excitation J.S. Sakellariou and S.D. Fassois Department of Mechanical & Aeronautical Engr. GR 265 Patras,

More information

Parameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn!

Parameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn! Parameter estimation! and! forecasting! Cristiano Porciani! AIfA, Uni-Bonn! Questions?! C. Porciani! Estimation & forecasting! 2! Cosmological parameters! A branch of modern cosmological research focuses

More information

APPLICATION OF A SUBSPACE-BASED FAULT DETECTION METHOD TO INDUSTRIAL STRUCTURES

APPLICATION OF A SUBSPACE-BASED FAULT DETECTION METHOD TO INDUSTRIAL STRUCTURES Mechanical Systems and Signal Processing (1999) 00(0), 000}000 Article No. mssp.1999.1247, available online at http://www.idealibrary.com on APPLICATION OF A SUBSPACE-BASED FAULT DETECTION METHOD TO INDUSTRIAL

More information

EECS564 Estimation, Filtering, and Detection Exam 2 Week of April 20, 2015

EECS564 Estimation, Filtering, and Detection Exam 2 Week of April 20, 2015 EECS564 Estimation, Filtering, and Detection Exam Week of April 0, 015 This is an open book takehome exam. You have 48 hours to complete the exam. All work on the exam should be your own. problems have

More information

Novel spectrum sensing schemes for Cognitive Radio Networks

Novel spectrum sensing schemes for Cognitive Radio Networks Novel spectrum sensing schemes for Cognitive Radio Networks Cantabria University Santander, May, 2015 Supélec, SCEE Rennes, France 1 The Advanced Signal Processing Group http://gtas.unican.es The Advanced

More information

VARIANCE COMPUTATION OF MODAL PARAMETER ES- TIMATES FROM UPC SUBSPACE IDENTIFICATION

VARIANCE COMPUTATION OF MODAL PARAMETER ES- TIMATES FROM UPC SUBSPACE IDENTIFICATION VARIANCE COMPUTATION OF MODAL PARAMETER ES- TIMATES FROM UPC SUBSPACE IDENTIFICATION Michael Döhler 1, Palle Andersen 2, Laurent Mevel 1 1 Inria/IFSTTAR, I4S, Rennes, France, {michaeldoehler, laurentmevel}@inriafr

More information

NONUNIFORM SAMPLING FOR DETECTION OF ABRUPT CHANGES*

NONUNIFORM SAMPLING FOR DETECTION OF ABRUPT CHANGES* CIRCUITS SYSTEMS SIGNAL PROCESSING c Birkhäuser Boston (2003) VOL. 22, NO. 4,2003, PP. 395 404 NONUNIFORM SAMPLING FOR DETECTION OF ABRUPT CHANGES* Feza Kerestecioğlu 1,2 and Sezai Tokat 1,3 Abstract.

More information

Bayesian inference J. Daunizeau

Bayesian inference J. Daunizeau Bayesian inference J. Daunizeau Brain and Spine Institute, Paris, France Wellcome Trust Centre for Neuroimaging, London, UK Overview of the talk 1 Probabilistic modelling and representation of uncertainty

More information

A Dynamic Threshold Based Algorithm for Change Detection in Autonomous Systems

A Dynamic Threshold Based Algorithm for Change Detection in Autonomous Systems Preprints, 12th IFAC International Workshop on Adaptation and Learning in Control and Signal Processing, A Dynamic Threshold Based Algorithm for Change Detection in Autonomous Systems Alexander Kalmuk

More information

A Simple Example Binary Hypothesis Testing Optimal Receiver Frontend M-ary Signal Sets Message Sequences. possible signals has been transmitted.

A Simple Example Binary Hypothesis Testing Optimal Receiver Frontend M-ary Signal Sets Message Sequences. possible signals has been transmitted. Introduction I We have focused on the problem of deciding which of two possible signals has been transmitted. I Binary Signal Sets I We will generalize the design of optimum (MPE) receivers to signal sets

More information

On the Complexity of Best Arm Identification with Fixed Confidence

On the Complexity of Best Arm Identification with Fixed Confidence On the Complexity of Best Arm Identification with Fixed Confidence Discrete Optimization with Noise Aurélien Garivier, Emilie Kaufmann COLT, June 23 th 2016, New York Institut de Mathématiques de Toulouse

More information

Monitoring Wafer Geometric Quality using Additive Gaussian Process

Monitoring Wafer Geometric Quality using Additive Gaussian Process Monitoring Wafer Geometric Quality using Additive Gaussian Process Linmiao Zhang 1 Kaibo Wang 2 Nan Chen 1 1 Department of Industrial and Systems Engineering, National University of Singapore 2 Department

More information

Variance estimation of modal parameters from input/output covariance-driven subspace identification

Variance estimation of modal parameters from input/output covariance-driven subspace identification Variance estimation of modal parameters from input/output covariance-driven subspace identification Philippe Mellinger, Michael Döhler, Laurent Mevel To cite this version: Philippe Mellinger, Michael Döhler,

More information

Confidence Intervals of Modal Parameters during Progressive Damage Test

Confidence Intervals of Modal Parameters during Progressive Damage Test Confidence Intervals of Modal Parameters during Progressive Damage Test Michael Döhler, Falk Hille, Xuan-Binh Lam, Laurent Mevel and Werner Rücker Abstract In Operational Modal Analysis, the modal parameters

More information

GPS Multipath Detection Based on Sequence of Successive-Time Double-Differences

GPS Multipath Detection Based on Sequence of Successive-Time Double-Differences 1 GPS Multipath Detection Based on Sequence of Successive-Time Double-Differences Hyung Keun Lee, Jang-Gyu Lee, and Gyu-In Jee Hyung Keun Lee is with Seoul National University-Korea University Joint Research

More information

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University PRINCIPAL COMPONENT ANALYSIS DIMENSIONALITY

More information

Introduction to Statistical Inference

Introduction to Statistical Inference Structural Health Monitoring Using Statistical Pattern Recognition Introduction to Statistical Inference Presented by Charles R. Farrar, Ph.D., P.E. Outline Introduce statistical decision making for Structural

More information

Linear Models for Regression CS534

Linear Models for Regression CS534 Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict

More information

Surveillance of Infectious Disease Data using Cumulative Sum Methods

Surveillance of Infectious Disease Data using Cumulative Sum Methods Surveillance of Infectious Disease Data using Cumulative Sum Methods 1 Michael Höhle 2 Leonhard Held 1 1 Institute of Social and Preventive Medicine University of Zurich 2 Department of Statistics University

More information

Hierarchical sparse Bayesian learning for structural health monitoring. with incomplete modal data

Hierarchical sparse Bayesian learning for structural health monitoring. with incomplete modal data Hierarchical sparse Bayesian learning for structural health monitoring with incomplete modal data Yong Huang and James L. Beck* Division of Engineering and Applied Science, California Institute of Technology,

More information

Practical Statistics

Practical Statistics Practical Statistics Lecture 1 (Nov. 9): - Correlation - Hypothesis Testing Lecture 2 (Nov. 16): - Error Estimation - Bayesian Analysis - Rejecting Outliers Lecture 3 (Nov. 18) - Monte Carlo Modeling -

More information

Sequential Detection. Changes: an overview. George V. Moustakides

Sequential Detection. Changes: an overview. George V. Moustakides Sequential Detection of Changes: an overview George V. Moustakides Outline Sequential hypothesis testing and Sequential detection of changes The Sequential Probability Ratio Test (SPRT) for optimum hypothesis

More information

Parameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn

Parameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Cosmological parameters A branch of modern cosmological research focuses on measuring

More information

Elements of Multivariate Time Series Analysis

Elements of Multivariate Time Series Analysis Gregory C. Reinsel Elements of Multivariate Time Series Analysis Second Edition With 14 Figures Springer Contents Preface to the Second Edition Preface to the First Edition vii ix 1. Vector Time Series

More information

Bayesian inference J. Daunizeau

Bayesian inference J. Daunizeau Bayesian inference J. Daunizeau Brain and Spine Institute, Paris, France Wellcome Trust Centre for Neuroimaging, London, UK Overview of the talk 1 Probabilistic modelling and representation of uncertainty

More information

PATTERN RECOGNITION AND MACHINE LEARNING

PATTERN RECOGNITION AND MACHINE LEARNING PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 3: Detection Theory January 2018 Heikki Huttunen heikki.huttunen@tut.fi Department of Signal Processing Tampere University of Technology Detection theory

More information

TMA 4275 Lifetime Analysis June 2004 Solution

TMA 4275 Lifetime Analysis June 2004 Solution TMA 4275 Lifetime Analysis June 2004 Solution Problem 1 a) Observation of the outcome is censored, if the time of the outcome is not known exactly and only the last time when it was observed being intact,

More information

Hypothesis testing for Stochastic PDEs. Igor Cialenco

Hypothesis testing for Stochastic PDEs. Igor Cialenco Hypothesis testing for Stochastic PDEs Igor Cialenco Department of Applied Mathematics Illinois Institute of Technology igor@math.iit.edu Joint work with Liaosha Xu Research partially funded by NSF grants

More information

Data-Driven Anomaly Detection based on a Bias Change

Data-Driven Anomaly Detection based on a Bias Change Preprints of the 19th World Congress The International Federation of Automatic Control Cape Town, South Africa. August 24-29, 214 Data-Driven Anomaly Detection based on a Bias Change André Carvalho Bittencourt

More information

Lecture 8: Information Theory and Statistics

Lecture 8: Information Theory and Statistics Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang

More information

Applications of Information Geometry to Hypothesis Testing and Signal Detection

Applications of Information Geometry to Hypothesis Testing and Signal Detection CMCAA 2016 Applications of Information Geometry to Hypothesis Testing and Signal Detection Yongqiang Cheng National University of Defense Technology July 2016 Outline 1. Principles of Information Geometry

More information

Scalar and Vector Time Series Methods for Vibration Based Damage Diagnosis in a Scale Aircraft Skeleton Structure

Scalar and Vector Time Series Methods for Vibration Based Damage Diagnosis in a Scale Aircraft Skeleton Structure Scalar and Vector Time Series Methods for Vibration Based Damage Diagnosis in a Scale Aircraft Skeleton Structure Fotis P. Kopsaftopoulos and Spilios D. Fassois Stochastic Mechanical Systems & Automation

More information

P Values and Nuisance Parameters

P Values and Nuisance Parameters P Values and Nuisance Parameters Luc Demortier The Rockefeller University PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics CERN, Geneva, June 27 29, 2007 Definition and interpretation of p values;

More information

Online Seismic Event Picking via Sequential Change-Point Detection

Online Seismic Event Picking via Sequential Change-Point Detection Online Seismic Event Picking via Sequential Change-Point Detection Shuang Li, Yang Cao, Christina Leamon, Yao Xie H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology

More information

Real-Time Detection of Hybrid and Stealthy Cyber-Attacks in Smart Grid

Real-Time Detection of Hybrid and Stealthy Cyber-Attacks in Smart Grid 1 Real-Time Detection of Hybrid and Stealthy Cyber-Attacks in Smart Grid Mehmet Necip Kurt, Yasin Yılmaz, Member, IEEE, and Xiaodong Wang, Fellow, IEEE Abstract For a safe and reliable operation of the

More information

LECTURE 10: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING. The last equality is provided so this can look like a more familiar parametric test.

LECTURE 10: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING. The last equality is provided so this can look like a more familiar parametric test. Economics 52 Econometrics Professor N.M. Kiefer LECTURE 1: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING NEYMAN-PEARSON LEMMA: Lesson: Good tests are based on the likelihood ratio. The proof is easy in the

More information

Testing for Anomalous Periods in Time Series Data. Graham Elliott

Testing for Anomalous Periods in Time Series Data. Graham Elliott Testing for Anomalous Periods in Time Series Data Graham Elliott 1 Introduction The Motivating Problem There are reasons to expect that for a time series model that an anomalous period might occur where

More information

Model-based Fault Diagnosis Techniques Design Schemes, Algorithms, and Tools

Model-based Fault Diagnosis Techniques Design Schemes, Algorithms, and Tools Steven X. Ding Model-based Fault Diagnosis Techniques Design Schemes, Algorithms, and Tools Springer Notation XIX Part I Introduction, basic concepts and preliminaries 1 Introduction 3 1.1 Basic concepts

More information

Lecture 2 Machine Learning Review

Lecture 2 Machine Learning Review Lecture 2 Machine Learning Review CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago March 29, 2017 Things we will look at today Formal Setup for Supervised Learning Things

More information

Digital Transmission Methods S

Digital Transmission Methods S Digital ransmission ethods S-7.5 Second Exercise Session Hypothesis esting Decision aking Gram-Schmidt method Detection.K.K. Communication Laboratory 5//6 Konstantinos.koufos@tkk.fi Exercise We assume

More information

Identification Methods for Structural Systems

Identification Methods for Structural Systems Prof. Dr. Eleni Chatzi Lecture 13-29 May, 2013 Courtesy of Prof. S. Fassois & Dr. F. Kopsaftopoulos, SMSA Group, University of Patras Statistical methods for SHM courtesy of Prof. S. Fassois & Dr. F. Kopsaftopoulos,

More information

Sparsity Models. Tong Zhang. Rutgers University. T. Zhang (Rutgers) Sparsity Models 1 / 28

Sparsity Models. Tong Zhang. Rutgers University. T. Zhang (Rutgers) Sparsity Models 1 / 28 Sparsity Models Tong Zhang Rutgers University T. Zhang (Rutgers) Sparsity Models 1 / 28 Topics Standard sparse regression model algorithms: convex relaxation and greedy algorithm sparse recovery analysis:

More information

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem

More information

ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals

ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals Jingxian Wu Department of Electrical Engineering University of Arkansas Outline Matched Filter Generalized Matched Filter Signal

More information

Prompt Network Anomaly Detection using SSA-Based Change-Point Detection. Hao Chen 3/7/2014

Prompt Network Anomaly Detection using SSA-Based Change-Point Detection. Hao Chen 3/7/2014 Prompt Network Anomaly Detection using SSA-Based Change-Point Detection Hao Chen 3/7/2014 Network Anomaly Detection Network Intrusion Detection (NID) Signature-based detection Detect known attacks Use

More information

WHEN IS A MAXIMAL INVARIANT HYPOTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred O. Hero

WHEN IS A MAXIMAL INVARIANT HYPOTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred O. Hero WHEN IS A MAXIMAL INVARIANT HYPTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred. Hero Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 489-222 ABSTRACT

More information

Some New Aspects of Dose-Response Models with Applications to Multistage Models Having Parameters on the Boundary

Some New Aspects of Dose-Response Models with Applications to Multistage Models Having Parameters on the Boundary Some New Aspects of Dose-Response Models with Applications to Multistage Models Having Parameters on the Boundary Bimal Sinha Department of Mathematics & Statistics University of Maryland, Baltimore County,

More information

Parametric Unsupervised Learning Expectation Maximization (EM) Lecture 20.a

Parametric Unsupervised Learning Expectation Maximization (EM) Lecture 20.a Parametric Unsupervised Learning Expectation Maximization (EM) Lecture 20.a Some slides are due to Christopher Bishop Limitations of K-means Hard assignments of data points to clusters small shift of a

More information

Linear Models for Regression CS534

Linear Models for Regression CS534 Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict

More information

An introduction to Bayesian inference and model comparison J. Daunizeau

An introduction to Bayesian inference and model comparison J. Daunizeau An introduction to Bayesian inference and model comparison J. Daunizeau ICM, Paris, France TNU, Zurich, Switzerland Overview of the talk An introduction to probabilistic modelling Bayesian model comparison

More information

CMU-Q Lecture 24:

CMU-Q Lecture 24: CMU-Q 15-381 Lecture 24: Supervised Learning 2 Teacher: Gianni A. Di Caro SUPERVISED LEARNING Hypotheses space Hypothesis function Labeled Given Errors Performance criteria Given a collection of input

More information

Exercises Chapter 4 Statistical Hypothesis Testing

Exercises Chapter 4 Statistical Hypothesis Testing Exercises Chapter 4 Statistical Hypothesis Testing Advanced Econometrics - HEC Lausanne Christophe Hurlin University of Orléans December 5, 013 Christophe Hurlin (University of Orléans) Advanced Econometrics

More information

Notes on the Multivariate Normal and Related Topics

Notes on the Multivariate Normal and Related Topics Version: July 10, 2013 Notes on the Multivariate Normal and Related Topics Let me refresh your memory about the distinctions between population and sample; parameters and statistics; population distributions

More information

F2E5216/TS1002 Adaptive Filtering and Change Detection. Course Organization. Lecture plan. The Books. Lecture 1

F2E5216/TS1002 Adaptive Filtering and Change Detection. Course Organization. Lecture plan. The Books. Lecture 1 Adaptive Filtering and Change Detection Bo Wahlberg (KTH and Fredrik Gustafsson (LiTH Course Organization Lectures and compendium: Theory, Algorithms, Applications, Evaluation Toolbox and manual: Algorithms,

More information

Variations. ECE 6540, Lecture 10 Maximum Likelihood Estimation

Variations. ECE 6540, Lecture 10 Maximum Likelihood Estimation Variations ECE 6540, Lecture 10 Last Time BLUE (Best Linear Unbiased Estimator) Formulation Advantages Disadvantages 2 The BLUE A simplification Assume the estimator is a linear system For a single parameter

More information

Modeling the Goodness-of-Fit Test Based on the Interval Estimation of the Probability Distribution Function

Modeling the Goodness-of-Fit Test Based on the Interval Estimation of the Probability Distribution Function Modeling the Goodness-of-Fit Test Based on the Interval Estimation of the Probability Distribution Function E. L. Kuleshov, K. A. Petrov, T. S. Kirillova Far Eastern Federal University, Sukhanova st.,

More information

Likelihood-Based Methods

Likelihood-Based Methods Likelihood-Based Methods Handbook of Spatial Statistics, Chapter 4 Susheela Singh September 22, 2016 OVERVIEW INTRODUCTION MAXIMUM LIKELIHOOD ESTIMATION (ML) RESTRICTED MAXIMUM LIKELIHOOD ESTIMATION (REML)

More information

Decentralized Sequential Hypothesis Testing. Change Detection

Decentralized Sequential Hypothesis Testing. Change Detection Decentralized Sequential Hypothesis Testing & Change Detection Giorgos Fellouris, Columbia University, NY, USA George V. Moustakides, University of Patras, Greece Outline Sequential hypothesis testing

More information

Statistics 203: Introduction to Regression and Analysis of Variance Course review

Statistics 203: Introduction to Regression and Analysis of Variance Course review Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying

More information

Parameter Estimation in a Moving Horizon Perspective

Parameter Estimation in a Moving Horizon Perspective Parameter Estimation in a Moving Horizon Perspective State and Parameter Estimation in Dynamical Systems Reglerteknik, ISY, Linköpings Universitet State and Parameter Estimation in Dynamical Systems OUTLINE

More information

Multi-Baseline SAR interferometry

Multi-Baseline SAR interferometry ulti-baseline SAR interferometry A. onti Guarnieri, S. Tebaldini Dipartimento di Elettronica e Informazione - Politecnico di ilano G. Fornaro, A. Pauciullo IREA-CNR ESA cat-1 project, 3173. Page 1 Classical

More information

Introduction to Signal Detection and Classification. Phani Chavali

Introduction to Signal Detection and Classification. Phani Chavali Introduction to Signal Detection and Classification Phani Chavali Outline Detection Problem Performance Measures Receiver Operating Characteristics (ROC) F-Test - Test Linear Discriminant Analysis (LDA)

More information

Gaussian with mean ( µ ) and standard deviation ( σ)

Gaussian with mean ( µ ) and standard deviation ( σ) Slide from Pieter Abbeel Gaussian with mean ( µ ) and standard deviation ( σ) 10/6/16 CSE-571: Robotics X ~ N( µ, σ ) Y ~ N( aµ + b, a σ ) Y = ax + b + + + + 1 1 1 1 1 1 1 1 1 1, ~ ) ( ) ( ), ( ~ ), (

More information

Hyung So0 Kim and Alfred 0. Hero

Hyung So0 Kim and Alfred 0. Hero WHEN IS A MAXIMAL INVARIANT HYPOTHESIS TEST BETTER THAN THE GLRT? Hyung So0 Kim and Alfred 0. Hero Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 48109-2122

More information

SIGNAL AND IMAGE RESTORATION: SOLVING

SIGNAL AND IMAGE RESTORATION: SOLVING 1 / 55 SIGNAL AND IMAGE RESTORATION: SOLVING ILL-POSED INVERSE PROBLEMS - ESTIMATING PARAMETERS Rosemary Renaut http://math.asu.edu/ rosie CORNELL MAY 10, 2013 2 / 55 Outline Background Parameter Estimation

More information

SEQUENTIAL CHANGE-POINT DETECTION WHEN THE PRE- AND POST-CHANGE PARAMETERS ARE UNKNOWN. Tze Leung Lai Haipeng Xing

SEQUENTIAL CHANGE-POINT DETECTION WHEN THE PRE- AND POST-CHANGE PARAMETERS ARE UNKNOWN. Tze Leung Lai Haipeng Xing SEQUENTIAL CHANGE-POINT DETECTION WHEN THE PRE- AND POST-CHANGE PARAMETERS ARE UNKNOWN By Tze Leung Lai Haipeng Xing Technical Report No. 2009-5 April 2009 Department of Statistics STANFORD UNIVERSITY

More information

Appendix A: The time series behavior of employment growth

Appendix A: The time series behavior of employment growth Unpublished appendices from The Relationship between Firm Size and Firm Growth in the U.S. Manufacturing Sector Bronwyn H. Hall Journal of Industrial Economics 35 (June 987): 583-606. Appendix A: The time

More information

Hypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA

Hypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA Hypothesis Testing Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA An Example Mardia et al. (979, p. ) reprint data from Frets (9) giving the length and breadth (in

More information