Second-Order Inference for Gaussian Random Curves
|
|
- Sharleen Day
- 5 years ago
- Views:
Transcription
1 Second-Order Inference for Gaussian Random Curves With Application to DNA Minicircles Victor Panaretos David Kraus John Maddocks Ecole Polytechnique Fédérale de Lausanne Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 1 / 48
2 Are they different? Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 2 / 48
3 Outline 1 DNA minicircles 2 Functional Data Analysis background 3 Testing procedures 4 Analysis of DNA minicircles 5 Summary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 3 / 48
4 Outline 1 DNA minicircles 2 Functional Data Analysis background 3 Testing procedures 4 Analysis of DNA minicircles 5 Summary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 4 / 48
5 Geometry of DNA molecules, base-pairs sequences Goal: study of mechanical properties of DNA molecules (via their geometry) Question: influence of base pair sequences 158 base-pairs, 18 bp fragment different (TATA vs CAP) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 5 / 48
6 Electron microscope images 50 nm layer of ice at 170 C, images tilted ±15 Minicircles diam 17 nm Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 6 / 48
7 Original data y z z x x y Each curve 200 xyz-coordinates Curves not directly comparable Adjustment Centering (center of mass = 0) Scaling (length = 1) Not sufficient, further alignment necessary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 7 / 48
8 Registration of functional data Standard alignment methods 1 Landmark alignment 2 Warping Standard methods cannot be used Landmark alignment not applicable: no landmarks available Warping inappropriate: don t want to change the shape, need a rigid method Curves have no beginning/end, no orientation Rotate each curve to make them as close as possible Global optimization over n = 99 orthogonal transformations would be difficult Instead, rotate each curve separately Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 8 / 48
9 Moments of inertia tensor Consider an object in R 3 with distribution of mass µ For DNA minicircles, µ is the uniform measure supported on the curve Consider an axis given by a unit vector u R 3 ( u = 1) Moment of inertia tensor defined as J (u) = r 2 (u, x)µ(dx) = R 3 (I uu T )x 2 µ(dx) R 3 (integrated squared distance from the axis given by u) Interpretation: J (u) measures how difficult it is to rotate the object around the axis u In matrix form J (u) = u T Ju, where J = (x T xi xx T )µ(dx) R 3 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 9 / 48
10 Principal axes of inertia J (u) = u T Ju is positive semidefinite, hence it possesses nonnegative eigenvalues and orthonormal eigenvectors The first eigenvector w 1 determines the axis around which the curve is most difficult to rotate (J (u) = u T Ju is maximized at u = w 1 ) The projection of the curve on the plane orthogonal to w 1 is most spread The second eigenvector w 2 determines the axis within the first principal plane around which the projected curve is most difficult to rotate Within the first principal plane, the projection on the line orthogonal to w 2 is most spread The axes given by w 1, w 2, w 3 are called principal axes of inertia (PAI1, PAI2, PAI3) PAI3 carries the most spatial information, PAI1 contains the smallest amount of information Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 10 / 48
11 Moments of inertia alignment Each curve aligned separately (no averaging over the sample) For each curve, the principal axes of inertia are determined and the curve is rotated so that the PAI s agree with the axes of the coordinate system (i.e., the curves are projected on PAI s) The procedure is similar to the balancing of a tyre (adjusting the distribution of mass of a wheel such that its PAI is aligned with the axle) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 11 / 48
12 Aligned DNA minicircles TATA CAP PAI3 PAI3 PAI2 PAI1 PAI1 PAI3 PAI2 PAI1 PAI1 PAI3 PAI2 PAI2 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 12 / 48
13 From curves to functions Curves have no starting point The intersection of the projection on the first principal plane and the horizontal (PAI2) positive semi-axis Curves have no orientation Counterclockwise in the first principal plane No correspondence between points on the curves Parametrization by arc length PAI PAI2 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 13 / 48
14 PAI coordinates of aligned DNA minicircles Principal Axis 3, TATA Principal Axis 2, TATA Principal Axis 1, TATA Principal Axis 3, CAP Principal Axis 2, CAP Principal Axis 1, CAP Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 14 / 48
15 Outline 1 DNA minicircles 2 Functional Data Analysis background 3 Testing procedures 4 Analysis of DNA minicircles 5 Summary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 15 / 48
16 Functional data Each minicircle curve is modelled as the realisation of a stochastic process indexed by [0, 1], taking values in R 3 X = {X(t), t [0, 1]} X is seen as a random element of the Hilbert space L 2 [0, 1] of coordinate-wise square-integrable R 3 -valued functions with the inner product f, g = 1 0 f (t), g(t) dt Wlog assume that the mean function is zero µ(t) = E X(t) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 16 / 48
17 Covariance operator Denote the covariance function (kernel) R(s, t) = cov(x(s), X(t)) = E(X(s)X(t) T ) The covariance operator is defined as Equivalently R : L 2 [0, 1] L 2 [0, 1] R(f ) = cov( X, f X) = 1 0 R = E(X X) R(, t)f (t)dt, The tensor product of a, b L 2 [0, 1] is defined as the operator (a b) : L 2 [0, 1] L 2 [0, 1], (a b)(f ) = 1 Multivariate analog: a b = ab T for a, b R p 0 a( ) b(t), f (t) dt Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 17 / 48
18 Karhunen Loève decomposition The covariance kernel admits the representation R(s, t) = λ k ϕ k (s)ϕ k (t) T, k=1 where λ k 0 are nonincreasing eigenvalues and ϕ k orthonormal eigenfunctions of R, i.e., R(ϕ k ) = λ k ϕ k The process X can be represented as X(t) = X, ϕ k ϕ k (t) = k=1 k=1 λ 1/2 k ξ k ϕ k (t) where the Fourier coefficients ξ k = λ 1/2 k X, ϕ k are uncorrelated random variables with zero mean and unit variance If the process is Gaussian, the scores ξ k, k 1 are iid standard Gaussian Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 18 / 48
19 Truncated KL decomposition, dimension reduction The first K eigenelements λ k, ϕ k, k = 1,..., K provide the optimal rank K approximation in the sense min ϕ 1,...,ϕ K orthonormal E X min E X X ϕ 1,...,ϕ K orthonormal, λ 1,...,λ K 0 max ϕ 1,...,ϕ K orthonormal K 2 X, ϕ k ϕ k k=1 2 K λ k (ϕ k ϕ k ) k=1 K var( X, ϕ k ) k=1 2 HS Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 19 / 48
20 Functional Principal Component Analysis Functional Principal Component Analysis is the empirical version of the Karhunen Loève decomposition Empirical covariance operator ˆR = 1 n (X i n X) (X i X) i=1 Functional eigenproblem ˆR ˆϕ k = ˆλ k ˆϕ k Usually, observations are represented in a basis, X i (t) = c ij ψ j (t) j=1 If the basis {ψ j } is orthonormal (such as the Fourier basis for periodic data like minicircles), then functional PCA is usual PCA of the coefficient matrix C = (c ij ) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 20 / 48
21 Outline 1 DNA minicircles 2 Functional Data Analysis background 3 Testing procedures 4 Analysis of DNA minicircles 5 Summary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 21 / 48
22 Means of PAI coordinates Principal Axis 3, TATA Principal Axis 3, all Principal Axis 3, CAP Principal Axis 2, TATA Principal Axis 2, all Principal Axis 2, CAP Principal Axis 1, TATA Principal Axis 1, all Principal Axis 1, CAP Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 22 / 48
23 Means on the principal plane Proj. on Prin. Plane 1, TATA Proj. on Prin. Plane 1, all Proj. on Prin. Plane 1, CAP PAI PAI PAI PAI PAI PAI2 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 23 / 48
24 Covariance functions covariance 0 covariance Covariance of PAI3, TATA Covariance of PAI3, CAP PAI3 PAI3 PAI3 PAI3 PAI3 0 3e 04 2e 04 1e e e 04 2e 04 1e 04 3e PAI e e e e e 04 5e 05 1e PAI3 PAI3 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 24 / 48
25 Testing problem Situation: X 1,..., X n1, Y 1,..., Y n2 independent samples of Gaussian stochastic processes with means µ X, µ Y and covariance operators R X, R Y Mean functions appear to be equal Focus on covariance operators Hypothesis testing problem H 0 : R X = R Y vs H 1 : R X R Y Use of a statistic like ˆR 1 X Instead, use the difference ˆR Y impossible (noninvertibility) ˆR X ˆR Y which should be close to the zero operator under the null Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 25 / 48
26 Hilbert Schmidt operator norm Need to measure the distance of ˆRX ˆR Y from a zero operator Need an operator norm The Hilbert Schmidt operator norm is defined as A 2 HS = i A e i 2 2 = i,j e i, A e j 2 (multivariate analog: A 2 F = i,j a2 ij for a matrix A) The distribution of ˆR X ˆR Y 2 HS intractable Perform dimension reduction, focus on the projected operators Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 26 / 48
27 Projection, truncation of the HS norm Let f 1,..., f K be some orthonormal L 2 functions Let K π K = f k f k k=1 be the projection operator onto the span of f 1,..., f K The test will based on ˆR K X ˆR K Y 2 HS where ˆR K X = π K ˆR X π K, ˆRK Y = π K ˆRY π K Need a common basis (the same π K for X, Y ) (a common reference coordinate system) We use π K = ˆπ K projecting on ˆϕ XY 1,..., ˆϕXY K (eigenfunctions of the pooled-sample estimator ˆR = n 1 n ˆRX + n 2 n ˆRY ) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 27 / 48
28 Test statistic The statistic is The terms ˆR K X ˆR K Y 2 HS = K ˆλ X,XY kj K k=1 j=1 ˆϕ XY k = ˆϕ XY k, ˆRX ˆϕ XY j, ˆλ Y,XY kj, ( ˆR X ˆR Y ) ˆϕ XY j 2 = ˆϕ XY k, ˆRY ˆϕ XY j are empirical var/cov of the X and Y scores w.r.t. the common basis Their asymptotic var/cov under H 0 is 2 δ kj λ k λ j The test statistic based on standardized components is T = n 1n 2 2n K K k=1 j=1 (ˆλ X,XY kj ˆλ Y,XY kj ) 2 ˆλ k ˆλ j Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 28 / 48
29 Asymptotics Under H 0 and Gaussian assumption, T D n χ2 K (K +1)/2 Sketch of the proof Using consistency of ˆϕ XY k, replace ˆπ K by π K By the Hilbert space CLT, ˆR X = 1 n 1 X i with X i = X i X i n 1 i=1 is asymptotically a Gaussian random operator (random element in the space of operators with Gaussian fdd s) Investigation of the covariance operator of the limit (an operator on operators on L 2 ) yields that the components of T are asymptotically independent Gaussian Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 29 / 48
30 Modifications of the test statistic Diagonal statistic T 1 = n 1n 2 2n K k=1 (ˆλ X,XY kk ˆλ Y,XY kk ) 2 ˆλ 2 k (compares only eigenvalues, might be good when eigenfunctions equal) Variance-stabilizing transformations log of the diagonal (variance) terms Fisher s z-transformation of the off-diagonal (covariance) terms Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 30 / 48
31 Selection of K 1 Scree plots, cumulative explained proportion of variance,... 2 Minimization of the penalized fit criterion PFC(K ) = GOF X (K )+GOF Y (K )+ n 1 n PEN X (K )+ n 2 n PEN Y (K ) (no formal result on the post-selection test) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 31 / 48
32 Simulations: design Simulated processes are combinations of Fourier basis functions with independent Gaussian coefficients Mimicking the elbow effect : 3 or 4 dominating components and several components with smaller variance n 1 = n 2 = 50 Nominal level 5 % Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 32 / 48
33 Simulations: level Scenario A Equal covariance operators K Test K off-diag diag The tests maintain the nominal level when K rank This is true also for K. The selection criterion aims at estimating the effective complexity of the distributions, it does not optimize the power, does not reflect validity or invalidity of H 0. Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 33 / 48
34 Simulations: power Scenario B The same sequence of eigenfunctions (in the same order) The first eigenvalues differ K Test K off-diag diag The power decreases (difference only in the first component) Diagonal better (the same eigenfunctions) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 34 / 48
35 Simulations: power Scenario E The same set of eigenfunctions in a different order (permuted) The first eigenfunctions equal The same eigenvalues K Test K off-diag diag No power with K = 1 (equal first eigenelements) Lower power for the diagonal test (the same eigenvalues) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 35 / 48
36 Simulations: power Scenario F Completely different eigenfunctions (sines vs time-shifted sines) Equal eigenvalues K Test K off-diag diag Lower power for the diagonal test Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 36 / 48
37 Outline 1 DNA minicircles 2 Functional Data Analysis background 3 Testing procedures 4 Analysis of DNA minicircles 5 Summary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 37 / 48
38 Outliers? Principal Axis 3, TATA Principal Axis 2, TATA Principal Axis 1, TATA Prin. Plane, TATA Principal Axis 3, CAP Principal Axis 2, CAP PAI3 PAI2 Principal Axis 1, CAP Prin. Plane, CAP PAI PAI2 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 38 / 48
39 Spatial median, outlier detection The functional spatial median is defined as the solution to min m L 2 n X i m 2 i=1 or n i=1 m X i m X i 2 = 0 The solution ˆm can be written as the weighted sum ˆm = n w i X i, w i 0, i=1 n w i = 1 i=1 Outliers have small weights w i, large values of 1/w i indicate outliers Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 39 / 48
40 Inverse spatial median weights Inverse median weights, PAI3 Index 1/w Inverse median weights, PAI2 Index 1/w Inverse median weights, PAI1 Index 1/w Inverse median weights, PAI1,2,3 Index 1/w Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 40 / 48
41 Outliers Principal Axis 3, TATA Principal Axis 2, TATA Principal Axis 1, TATA Prin. Plane, TATA Principal Axis 3, CAP Principal Axis 2, CAP PAI3 PAI2 Principal Axis 1, CAP Prin. Plane, CAP PAI PAI2 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 41 / 48
42 Joint PCA of PAI2,3: coordinates of eigenfunctions PAI2 in PAI2,3, phi 1 PAI2 in PAI2,3, phi 2 PAI2 in PAI2,3, phi 3 PAI2 in PAI2,3, phi PAI3 in PAI2,3, phi PAI3 in PAI2,3, phi PAI3 in PAI2,3, phi 3 PAI3 in PAI2,3, phi Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 42 / 48
43 Joint PCA of PAI2,3: eigencircles PCA function 1 (Percentage of variability 26.1 ) PCA function 2 (Percentage of variability 22.4 ) PCA function 3 (Percentage of variability 12.4 ) PCA function 4 (Percentage of variability 11 ) PCA function 1 (Percentage of variability 28.1 ) PCA function 2 (Percentage of variability 20.1 ) PCA function 3 (Percentage of variability 12.5 ) PCA function 4 (Percentage of variability 11.6 ) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 43 / 48
44 Joint PCA of PAI2,3: eigenvalues Lambda for PAI2, Var prop for PAI2, Cum var prop for PAI2, Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 44 / 48
45 Selection of K (joint analysis of PAI2,3) Scree plot w.r.t. the common (pooled sample) eigenbasis Lambda for PAI2, Var prop for PAI2, Cum var prop for PAI2, Plots suggest K = 6 or 7 Automatic choice K = 7 Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 45 / 48
46 Test results Test statistic: off-diagonal, transformed, χ 2 approximation p-value K PAI3 PAI2 PAI1 PAI2, S A S S A A (S = Selection based on scree plots, A = Automatic selection) Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 46 / 48
47 Outline 1 DNA minicircles 2 Functional Data Analysis background 3 Testing procedures 4 Analysis of DNA minicircles 5 Summary Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 47 / 48
48 Summary DNA minicircle data, alignment,... Functional data approach Tests based on an approximation of the Hilbert Schmidt distance between empirical covariance operators, asymptotics, simulations,... Minicircle data analaysis (outlier detection, order selection, testing,... ) Outlook Robustification Normality testing... Panaretos, Kraus, Maddocks (EPFL) Second-Order Inference for Gaussian Random Curves 48 / 48
Second-Order Comparison of Gaussian Random Functions and the Geometry of DNA Minicircles
Supplementary materials for this article are available online. Please click the JASA link at http://pubs.amstat.org. Second-Order Comparison of Gaussian Random Functions and the Geometry of DNA Minicircles
More informationIndependent component analysis for functional data
Independent component analysis for functional data Hannu Oja Department of Mathematics and Statistics University of Turku Version 12.8.216 August 216 Oja (UTU) FICA Date bottom 1 / 38 Outline 1 Probability
More informationTests for separability in nonparametric covariance operators of random surfaces
Tests for separability in nonparametric covariance operators of random surfaces Shahin Tavakoli (joint with John Aston and Davide Pigoli) April 19, 2016 Analysis of Multidimensional Functional Data Shahin
More informationChapter 4: Factor Analysis
Chapter 4: Factor Analysis In many studies, we may not be able to measure directly the variables of interest. We can merely collect data on other variables which may be related to the variables of interest.
More informationStructure in Data. A major objective in data analysis is to identify interesting features or structure in the data.
Structure in Data A major objective in data analysis is to identify interesting features or structure in the data. The graphical methods are very useful in discovering structure. There are basically two
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More informationStatistics for Applications. Chapter 9: Principal Component Analysis (PCA) 1/16
Statistics for Applications Chapter 9: Principal Component Analysis (PCA) 1/16 Multivariate statistics and review of linear algebra (1) Let X be a d-dimensional random vector and X 1,..., X n be n independent
More informationSTATISTICAL LEARNING SYSTEMS
STATISTICAL LEARNING SYSTEMS LECTURE 8: UNSUPERVISED LEARNING: FINDING STRUCTURE IN DATA Institute of Computer Science, Polish Academy of Sciences Ph. D. Program 2013/2014 Principal Component Analysis
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,
More informationTesting the Equality of Covariance Operators in Functional Samples
Scandinavian Journal of Statistics, Vol. 4: 38 5, 3 doi:./j.467-9469..796.x Board of the Foundation of the Scandinavian Journal of Statistics. Published by Blackwell Publishing Ltd. Testing the Equality
More informationDistances and inference for covariance operators
Royal Holloway Probability and Statistics Colloquium 27th April 2015 Distances and inference for covariance operators Davide Pigoli This is part of joint works with J.A.D. Aston I.L. Dryden P. Secchi J.
More informationPrincipal Component Analysis
CSci 5525: Machine Learning Dec 3, 2008 The Main Idea Given a dataset X = {x 1,..., x N } The Main Idea Given a dataset X = {x 1,..., x N } Find a low-dimensional linear projection The Main Idea Given
More informationBasic Operator Theory with KL Expansion
Basic Operator Theory with Karhunen Loéve Expansion Wafa Abu Zarqa, Maryam Abu Shamala, Samiha Al Aga, Khould Al Qitieti, Reem Al Rashedi Department of Mathematical Sciences College of Science United Arab
More informationA Note on Hilbertian Elliptically Contoured Distributions
A Note on Hilbertian Elliptically Contoured Distributions Yehua Li Department of Statistics, University of Georgia, Athens, GA 30602, USA Abstract. In this paper, we discuss elliptically contoured distribution
More informationLinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Principal Component Analysis 3 Factor Analysis
More informationMaximum variance formulation
12.1. Principal Component Analysis 561 Figure 12.2 Principal component analysis seeks a space of lower dimensionality, known as the principal subspace and denoted by the magenta line, such that the orthogonal
More informationPCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
PCA & ICA CE-717: Machine Learning Sharif University of Technology Spring 2015 Soleymani Dimensionality Reduction: Feature Selection vs. Feature Extraction Feature selection Select a subset of a given
More informationFUNCTIONAL DATA ANALYSIS. Contribution to the. International Handbook (Encyclopedia) of Statistical Sciences. July 28, Hans-Georg Müller 1
FUNCTIONAL DATA ANALYSIS Contribution to the International Handbook (Encyclopedia) of Statistical Sciences July 28, 2009 Hans-Georg Müller 1 Department of Statistics University of California, Davis One
More informationReview (Probability & Linear Algebra)
Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint
More informationModeling Multi-Way Functional Data With Weak Separability
Modeling Multi-Way Functional Data With Weak Separability Kehui Chen Department of Statistics University of Pittsburgh, USA @CMStatistics, Seville, Spain December 09, 2016 Outline Introduction. Multi-way
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 4 for Applied Multivariate Analysis Outline 1 Eigen values and eigen vectors Characteristic equation Some properties of eigendecompositions
More informationConcentration Ellipsoids
Concentration Ellipsoids ECE275A Lecture Supplement Fall 2008 Kenneth Kreutz Delgado Electrical and Computer Engineering Jacobs School of Engineering University of California, San Diego VERSION LSECE275CE
More informationNoise & Data Reduction
Noise & Data Reduction Paired Sample t Test Data Transformation - Overview From Covariance Matrix to PCA and Dimension Reduction Fourier Analysis - Spectrum Dimension Reduction 1 Remember: Central Limit
More informationLecture 11. Multivariate Normal theory
10. Lecture 11. Multivariate Normal theory Lecture 11. Multivariate Normal theory 1 (1 1) 11. Multivariate Normal theory 11.1. Properties of means and covariances of vectors Properties of means and covariances
More informationDimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
More informationCollocation based high dimensional model representation for stochastic partial differential equations
Collocation based high dimensional model representation for stochastic partial differential equations S Adhikari 1 1 Swansea University, UK ECCM 2010: IV European Conference on Computational Mechanics,
More informationUnsupervised Learning: Dimensionality Reduction
Unsupervised Learning: Dimensionality Reduction CMPSCI 689 Fall 2015 Sridhar Mahadevan Lecture 3 Outline In this lecture, we set about to solve the problem posed in the previous lecture Given a dataset,
More informationData Preprocessing Tasks
Data Tasks 1 2 3 Data Reduction 4 We re here. 1 Dimensionality Reduction Dimensionality reduction is a commonly used approach for generating fewer features. Typically used because too many features can
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationSmooth Common Principal Component Analysis
1 Smooth Common Principal Component Analysis Michal Benko Wolfgang Härdle Center for Applied Statistics and Economics benko@wiwi.hu-berlin.de Humboldt-Universität zu Berlin Motivation 1-1 Volatility Surface
More informationEECS 275 Matrix Computation
EECS 275 Matrix Computation Ming-Hsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 6 1 / 22 Overview
More information5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More informationRegularized Discriminant Analysis and Reduced-Rank LDA
Regularized Discriminant Analysis and Reduced-Rank LDA Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Regularized Discriminant Analysis A compromise between LDA and
More informationPrincipal Components Theory Notes
Principal Components Theory Notes Charles J. Geyer August 29, 2007 1 Introduction These are class notes for Stat 5601 (nonparametrics) taught at the University of Minnesota, Spring 2006. This not a theory
More informationA Vector-Space Approach for Stochastic Finite Element Analysis
A Vector-Space Approach for Stochastic Finite Element Analysis S Adhikari 1 1 Swansea University, UK CST2010: Valencia, Spain Adhikari (Swansea) Vector-Space Approach for SFEM 14-17 September, 2010 1 /
More informationPrincipal Component Analysis!! Lecture 11!
Principal Component Analysis Lecture 11 1 Eigenvectors and Eigenvalues g Consider this problem of spreading butter on a bread slice 2 Eigenvectors and Eigenvalues g Consider this problem of stretching
More information3.1. The probabilistic view of the principal component analysis.
301 Chapter 3 Principal Components and Statistical Factor Models This chapter of introduces the principal component analysis (PCA), briefly reviews statistical factor models PCA is among the most popular
More informationKarhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques
Institut für Numerische Mathematik und Optimierung Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques Oliver Ernst Computational Methods with Applications Harrachov, CR,
More informationPrincipal Component Analysis. Applied Multivariate Statistics Spring 2012
Principal Component Analysis Applied Multivariate Statistics Spring 2012 Overview Intuition Four definitions Practical examples Mathematical example Case study 2 PCA: Goals Goal 1: Dimension reduction
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More information9.1 Orthogonal factor model.
36 Chapter 9 Factor Analysis Factor analysis may be viewed as a refinement of the principal component analysis The objective is, like the PC analysis, to describe the relevant variables in study in terms
More informationHands-on Matrix Algebra Using R
Preface vii 1. R Preliminaries 1 1.1 Matrix Defined, Deeper Understanding Using Software.. 1 1.2 Introduction, Why R?.................... 2 1.3 Obtaining R.......................... 4 1.4 Reference Manuals
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationFocus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations.
Previously Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations y = Ax Or A simply represents data Notion of eigenvectors,
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationExercises * on Principal Component Analysis
Exercises * on Principal Component Analysis Laurenz Wiskott Institut für Neuroinformatik Ruhr-Universität Bochum, Germany, EU 4 February 207 Contents Intuition 3. Problem statement..........................................
More informationPrincipal Component Analysis -- PCA (also called Karhunen-Loeve transformation)
Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) PCA transforms the original input space into a lower dimensional space, by constructing dimensions that are linear combinations
More informationKarhunen-Loève decomposition of Gaussian measures on Banach spaces
Karhunen-Loève decomposition of Gaussian measures on Banach spaces Jean-Charles Croix jean-charles.croix@emse.fr Génie Mathématique et Industriel (GMI) First workshop on Gaussian processes at Saint-Etienne
More informationTAMS39 Lecture 10 Principal Component Analysis Factor Analysis
TAMS39 Lecture 10 Principal Component Analysis Factor Analysis Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content - Lecture Principal component analysis
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationCramér-von Mises Gaussianity test in Hilbert space
Cramér-von Mises Gaussianity test in Hilbert space Gennady MARTYNOV Institute for Information Transmission Problems of the Russian Academy of Sciences Higher School of Economics, Russia, Moscow Statistique
More informationLecture 4: Principal Component Analysis and Linear Dimension Reduction
Lecture 4: Principal Component Analysis and Linear Dimension Reduction Advanced Applied Multivariate Analysis STAT 2221, Fall 2013 Sungkyu Jung Department of Statistics University of Pittsburgh E-mail:
More informationStatistical Data Analysis
DS-GA 0 Lecture notes 8 Fall 016 1 Descriptive statistics Statistical Data Analysis In this section we consider the problem of analyzing a set of data. We describe several techniques for visualizing the
More informationData Mining Lecture 4: Covariance, EVD, PCA & SVD
Data Mining Lecture 4: Covariance, EVD, PCA & SVD Jo Houghton ECS Southampton February 25, 2019 1 / 28 Variance and Covariance - Expectation A random variable takes on different values due to chance The
More informationChapter 7: Symmetric Matrices and Quadratic Forms
Chapter 7: Symmetric Matrices and Quadratic Forms (Last Updated: December, 06) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved
More informationStatistical Convergence of Kernel CCA
Statistical Convergence of Kernel CCA Kenji Fukumizu Institute of Statistical Mathematics Tokyo 106-8569 Japan fukumizu@ism.ac.jp Francis R. Bach Centre de Morphologie Mathematique Ecole des Mines de Paris,
More informationNonparametric Inference In Functional Data
Nonparametric Inference In Functional Data Zuofeng Shang Purdue University Joint work with Guang Cheng from Purdue Univ. An Example Consider the functional linear model: Y = α + where 1 0 X(t)β(t)dt +
More informationLecture 13. Principal Component Analysis. Brett Bernstein. April 25, CDS at NYU. Brett Bernstein (CDS at NYU) Lecture 13 April 25, / 26
Principal Component Analysis Brett Bernstein CDS at NYU April 25, 2017 Brett Bernstein (CDS at NYU) Lecture 13 April 25, 2017 1 / 26 Initial Question Intro Question Question Let S R n n be symmetric. 1
More informationMultivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
More informationApproximate Kernel PCA with Random Features
Approximate Kernel PCA with Random Features (Computational vs. Statistical Tradeoff) Bharath K. Sriperumbudur Department of Statistics, Pennsylvania State University Journées de Statistique Paris May 28,
More informationCSE 554 Lecture 7: Alignment
CSE 554 Lecture 7: Alignment Fall 2012 CSE554 Alignment Slide 1 Review Fairing (smoothing) Relocating vertices to achieve a smoother appearance Method: centroid averaging Simplification Reducing vertex
More informationVector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar
More informationStatistical signal processing
Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable
More informationGaussian vectors and central limit theorem
Gaussian vectors and central limit theorem Samy Tindel Purdue University Probability Theory 2 - MA 539 Samy T. Gaussian vectors & CLT Probability Theory 1 / 86 Outline 1 Real Gaussian random variables
More information7. Variable extraction and dimensionality reduction
7. Variable extraction and dimensionality reduction The goal of the variable selection in the preceding chapter was to find least useful variables so that it would be possible to reduce the dimensionality
More informationNumerical Methods I Singular Value Decomposition
Numerical Methods I Singular Value Decomposition Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 9th, 2014 A. Donev (Courant Institute)
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen PCA. Tobias Scheffer
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen PCA Tobias Scheffer Overview Principal Component Analysis (PCA) Kernel-PCA Fisher Linear Discriminant Analysis t-sne 2 PCA: Motivation
More information1 Inner Product and Orthogonality
CSCI 4/Fall 6/Vora/GWU/Orthogonality and Norms Inner Product and Orthogonality Definition : The inner product of two vectors x and y, x x x =.., y =. x n y y... y n is denoted x, y : Note that n x, y =
More informationNoise & Data Reduction
Noise & Data Reduction Andreas Wichert - Teóricas andreas.wichert@inesc-id.pt 1 Paired Sample t Test Data Transformation - Overview From Covariance Matrix to PCA and Dimension Reduction Fourier Analysis
More informationModeling Repeated Functional Observations
Modeling Repeated Functional Observations Kehui Chen Department of Statistics, University of Pittsburgh, Hans-Georg Müller Department of Statistics, University of California, Davis Supplemental Material
More informationLearning Eigenfunctions: Links with Spectral Clustering and Kernel PCA
Learning Eigenfunctions: Links with Spectral Clustering and Kernel PCA Yoshua Bengio Pascal Vincent Jean-François Paiement University of Montreal April 2, Snowbird Learning 2003 Learning Modal Structures
More informationPrincipal Component Analysis
Principal Component Analysis Laurenz Wiskott Institute for Theoretical Biology Humboldt-University Berlin Invalidenstraße 43 D-10115 Berlin, Germany 11 March 2004 1 Intuition Problem Statement Experimental
More informationMultivariate Statistics (I) 2. Principal Component Analysis (PCA)
Multivariate Statistics (I) 2. Principal Component Analysis (PCA) 2.1 Comprehension of PCA 2.2 Concepts of PCs 2.3 Algebraic derivation of PCs 2.4 Selection and goodness-of-fit of PCs 2.5 Algebraic derivation
More informationEmpirical Gramians and Balanced Truncation for Model Reduction of Nonlinear Systems
Empirical Gramians and Balanced Truncation for Model Reduction of Nonlinear Systems Antoni Ras Departament de Matemàtica Aplicada 4 Universitat Politècnica de Catalunya Lecture goals To review the basic
More informationPolynomial Chaos and Karhunen-Loeve Expansion
Polynomial Chaos and Karhunen-Loeve Expansion 1) Random Variables Consider a system that is modeled by R = M(x, t, X) where X is a random variable. We are interested in determining the probability of the
More informationIV. Matrix Approximation using Least-Squares
IV. Matrix Approximation using Least-Squares The SVD and Matrix Approximation We begin with the following fundamental question. Let A be an M N matrix with rank R. What is the closest matrix to A that
More informationPreprocessing & dimensionality reduction
Introduction to Data Mining Preprocessing & dimensionality reduction CPSC/AMTH 445a/545a Guy Wolf guy.wolf@yale.edu Yale University Fall 2016 CPSC 445 (Guy Wolf) Dimensionality reduction Yale - Fall 2016
More informationPrinciple Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA
Principle Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA Principle Components Analysis: Uses one group of variables (we will call this X) In
More informationAlignment and Analysis of Proteomics Data using Square Root Slope Function Framework
Alignment and Analysis of Proteomics Data using Square Root Slope Function Framework J. Derek Tucker 1 1 Department of Statistics Florida State University Tallahassee, FL 32306 CTW: Statistics of Warpings
More informationFunctional principal component analysis of aircraft trajectories
Functional principal component analysis of aircraft trajectories Florence Nicol To cite this version: Florence Nicol. Functional principal component analysis of aircraft trajectories. ISIATM 0, nd International
More informationHigh Dimensional Covariance and Precision Matrix Estimation
High Dimensional Covariance and Precision Matrix Estimation Wei Wang Washington University in St. Louis Thursday 23 rd February, 2017 Wei Wang (Washington University in St. Louis) High Dimensional Covariance
More informationFactor Analysis Continued. Psy 524 Ainsworth
Factor Analysis Continued Psy 524 Ainsworth Equations Extraction Principal Axis Factoring Variables Skiers Cost Lift Depth Powder S1 32 64 65 67 S2 61 37 62 65 S3 59 40 45 43 S4 36 62 34 35 S5 62 46 43
More informationKernel-Based Contrast Functions for Sufficient Dimension Reduction
Kernel-Based Contrast Functions for Sufficient Dimension Reduction Michael I. Jordan Departments of Statistics and EECS University of California, Berkeley Joint work with Kenji Fukumizu and Francis Bach
More informationPrincipal component analysis
Principal component analysis Angela Montanari 1 Introduction Principal component analysis (PCA) is one of the most popular multivariate statistical methods. It was first introduced by Pearson (1901) and
More informationFrank C Porter and Ilya Narsky: Statistical Analysis Techniques in Particle Physics Chap. c /9/9 page 147 le-tex
Frank C Porter and Ilya Narsky: Statistical Analysis Techniques in Particle Physics Chap. c08 2013/9/9 page 147 le-tex 8.3 Principal Component Analysis (PCA) 147 Figure 8.1 Principal and independent components
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationMultivariate Statistics
Multivariate Statistics Chapter 2: Multivariate distributions and inference Pedro Galeano Departamento de Estadística Universidad Carlos III de Madrid pedro.galeano@uc3m.es Course 2016/2017 Master in Mathematical
More informationPrincipal Component Analysis CS498
Principal Component Analysis CS498 Today s lecture Adaptive Feature Extraction Principal Component Analysis How, why, when, which A dual goal Find a good representation The features part Reduce redundancy
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
More informationMachine Learning - MT & 14. PCA and MDS
Machine Learning - MT 2016 13 & 14. PCA and MDS Varun Kanade University of Oxford November 21 & 23, 2016 Announcements Sheet 4 due this Friday by noon Practical 3 this week (continue next week if necessary)
More informationCS 143 Linear Algebra Review
CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see
More informationConvergence of Eigenspaces in Kernel Principal Component Analysis
Convergence of Eigenspaces in Kernel Principal Component Analysis Shixin Wang Advanced machine learning April 19, 2016 Shixin Wang Convergence of Eigenspaces April 19, 2016 1 / 18 Outline 1 Motivation
More informationPrincipal Component Analysis-I Geog 210C Introduction to Spatial Data Analysis. Chris Funk. Lecture 17
Principal Component Analysis-I Geog 210C Introduction to Spatial Data Analysis Chris Funk Lecture 17 Outline Filters and Rotations Generating co-varying random fields Translating co-varying fields into
More informationIntelligent Data Analysis. Principal Component Analysis. School of Computer Science University of Birmingham
Intelligent Data Analysis Principal Component Analysis Peter Tiňo School of Computer Science University of Birmingham Discovering low-dimensional spatial layout in higher dimensional spaces - 1-D/3-D example
More informationFunctional Latent Feature Models. With Single-Index Interaction
Generalized With Single-Index Interaction Department of Statistics Center for Statistical Bioinformatics Institute for Applied Mathematics and Computational Science Texas A&M University Naisyin Wang and
More informationIntroduction to Machine Learning
10-701 Introduction to Machine Learning PCA Slides based on 18-661 Fall 2018 PCA Raw data can be Complex, High-dimensional To understand a phenomenon we measure various related quantities If we knew what
More information1 Principal Components Analysis
Lecture 3 and 4 Sept. 18 and Sept.20-2006 Data Visualization STAT 442 / 890, CM 462 Lecture: Ali Ghodsi 1 Principal Components Analysis Principal components analysis (PCA) is a very popular technique for
More informationData Mining and Analysis: Fundamental Concepts and Algorithms
Data Mining and Analysis: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA
More informationSTATISTICAL SHAPE MODELS (SSM)
STATISTICAL SHAPE MODELS (SSM) Medical Image Analysis Serena Bonaretti serena.bonaretti@istb.unibe.ch ISTB - Institute for Surgical Technology and Biomechanics University of Bern Overview > Introduction
More informationModel Selection and Geometry
Model Selection and Geometry Pascal Massart Université Paris-Sud, Orsay Leipzig, February Purpose of the talk! Concentration of measure plays a fundamental role in the theory of model selection! Model
More information