Distances and inference for covariance operators

Transcription

1 Royal Holloway Probability and Statistics Colloquium 27th April 2015 Distances and inference for covariance operators Davide Pigoli

2 This is part of joint works with J.A.D. Aston I.L. Dryden P. Secchi J. S. Coleman P. Z. Hadjipantelis

3 OUTLINE: Introduction Metrics for covariance operators Permutation tests Interpolation and extrapolation Future directions

4 OUTLINE: Demonstrative applications: Introduction Metrics for covariance operators Permutation tests Geometrical features of the Internal Carotid Artery for patients with risk of aneurysms Interpolation and extrapolation Future directions Acoustic Phonetic Data in comparative linguistics

5 INTRODUCTION: FUNCTIONAL DATA AND THEIR COVARIANCE A dataset is labeled as functional when it can be seen as multiple realizations of a smooth random process. In practice, this means having data with an internal ordering (e.g. data collected over time or space) and a high signal to noise ratio. We do not usually want to assume a parametric structure for these functions, although some constrains may be applied (for example, monotonicity). Here, we are interest in the inference for the covariance operators of these random processes.

6 INTRODUCTION : ANEURISK DATASET Example from to biomedical dataset described in Sangalli et al. (2009) [ available at ] Observational Study conducted relative to 65 patients hospitalized at Ospedale Niguarda Ca Granda Milano from September 2002 to October Geometric features of the Internal Carotid Artery (as function of the abscissa along the centerline): - geometry of the centerline - radius - curvature -

7 INTRODUCTION : ANEURISK DATASET Example from to biomedical dataset described in Sangalli et al. (2009) [ available at ] Derivatives of the centerline coordinates as function of the curvilinear abscissa Radius of the artery as function of the curvilinear abscissa Curvature of the artery as function of the curvilinear abscissa

8 METRICS FOR COVARIANCE OPERATORS

9 METRICS FOR COVARIANCE OPERATORS The aim is to develop a framework for the inference on covariance operators of functional random processes. The properties of the random process reflect the regularity of covariance operator. The focus is on the covariance function associated covariance operator and the where is a functional random variable taking values in s.t.

10 METRICS FOR COVARIANCE OPERATORS It is a self-adjoint, trace class compact operator on with non negative eigenvalues. trace class: eigenvalues The focus is on the covariance function associated covariance operator and the where is a functional random variable taking values in s.t.

11 METRICS FOR COVARIANCE OPERATORS Note that all matrix distances based on inverse matrix (Affine invariant Riemannian distance) or matrix logarithm are not suitable, since

12 METRICS FOR COVARIANCE OPERATORS Note that all matrix distances based on inverse matrix (Affine invariant Riemannian distance) or matrix logarithm are not suitable, since Hilbert-Schmidt (Euclidean) distance: Spectral distance: ( largest eigenvalue of in absolute value)

13 METRICS FOR COVARIANCE OPERATORS Note that all matrix distances based on inverse matrix (Affine invariant Riemannian distance) or matrix logarithm are not suitable, since Hilbert-Schmidt (Euclidean) distance: Spectral distance: ( largest eigenvalue of in absolute value) Square root distance Procrustes distance

14 METRICS FOR COVARIANCE OPERATORS Theoretical results for Procrustes (and Square root) distance Proposition 1: It is minimized for the unitary operator defined by where, come from the SVD

15 METRICS FOR COVARIANCE OPERATORS Theoretical results for Procrustes (and Square root) distance Proposition 1: It is minimized for the unitary operator defined by where, come from the SVD Proposition 2: where are singular values of the operator

16 METRICS FOR COVARIANCE OPERATORS Theoretical results for Procrustes (and Square root) distance Proposition 1: It is minimized for the unitary operator defined by where, come from the SVD Proposition 2: where are singular values of the operator Proposition 3: Let be an orthonormal basis for and

17 PERMUTATION TESTS FOR COMPARISON OF COVARIANCE OPERATORS BETWEEN DIFFERENT GROUPS

18 PERMUTATION TESTS realizations of a random process with mean and covariance operator realizations of a random process with mean and covariance operator Aim: testing the hypothesis Under the null hypothesis, curves observations are exchangeable. Let C 1, C 2 be sample covariance operators, big values of are evidence against the null hypothesis.

19 PERMUTATION TESTS We can set up a permutation test Let us take M permutations of the original (n 1 + n 2 ) curves. For m=1,,m, let C 1m,C 2 m be the sample covariance operators for the permuted curves. Then the p-value of the test will be

20 PERMUTATION TESTS We can set up a permutation test Let us take M permutations of the original (n 1 + n 2 ) curves. For m=1,,m, let C 1m,C 2 m be the sample covariance operators for the permuted curves. Then the p-value of the test will be If the two groups have different means, this procedure can still be applied to the centered curves. The resulting testing procedure is asymptotically exact.

21 PERMUTATION TESTS Simulations study: We simulate two samples of curves on [0,1] from a process with mean and covariance operators and respectively.

22 PERMUTATION TESTS Simulations study: We simulate two samples of curves on [0,1] from a process with mean and covariance operators and respectively.

23 PERMUTATION TESTS Simulations study: We simulate two samples of curves on [0,1] from a process with mean and covariance operators and respectively.

24 PERMUTATION TESTS Existing methods used for comparison, based on the Karhunen-Loève expansion:

25 PERMUTATION TESTS Existing methods used for comparison, based on the Karhunen-Loève expansion: 1) Panaretos et al. (2010) introduce the test statistic Eigenfunctions of the pooled covariance operator where Projections of the two dataset on the eigendirections If the data are generated by a Gaussian process, the asymptotic distribution of this test statistic is a chi-square with K (K+1)/2 degrees of freedom.

26 PERMUTATION TESTS Existing methods used for comparison, based on the Karhunen-Loève expansion: 2) Fremdt et al. (2013) propose an alternative, in the same spirit, for the non Gaussian case. Let be the difference of the variability of the two groups in the direction of and a vectorization of. Then, we can obtain the covariance matrix of (it has an explicit, although complicated, expression). Finally, has asymptotically a chi-square distribution with K (K+1)/2 degrees of freedom.

27 PERMUTATION TESTS Gaussian error Square root permutation Procrustes permutation Panaretos et al. test Fremdt et al. test Spectral permutation Hilbert-Schmidt permutation

28 PERMUTATION TESTS Non Gaussian error: Multivariate t with 6 df Square root permutation Procrustes permutation Panaretos et al. test Fremdt et al. test Spectral permutation Hilbert-Schmidt permutation

29 PERMUTATION TESTS : ANEURISK DATASET Radius and curvature of the Internal Carotid Artery are considered for 3 groups of patients: Patients with no aneurysm Patients with aneurysm before ICA bifurcation (Lower Group) Patients with aneurysm after ICA bifurcation (Upper Group) Questions: No aneurysm and Lower groups are usually treated as a single group. Is this supported by the analysis of covariance structure? Is there a difference between No aneurysm + Lower group and Upper group?

30 Curvature Radius PERMUTATION TESTS : ANEURISK DATASET Lower Group No Aneurysm Group Permutation test based on Square Root distance between covariance operators p-value: Lower Group No Aneurysm Group p-value: 0.61

31 Curvature Radius PERMUTATION TESTS : ANEURISK DATASET Lower+ No Aneurysm Group Upper Group Permutation test based on Square Root distance between covariance operators p-value <0.001 Lower+ No Aneurysm Group Upper Group p-value:0.005

32 PERMUTATION TESTS Can we generalize this procedure to multiple groups? number of groups Test statistic centerpoint of a set of covariance operators

33 PERMUTATION TESTS Fréchet averaging: Sample Fréchet mean Hilbert-Schmidt distance: Traditional pooled covariance function Square root distance: Procrustes distance: Iterative Procrustes algorithm [Gower, 1975]

34 PERMUTATION TESTS Test statistic Let us take M permutations of the original labels. For m=1,,m, let be the sample covariance operators for the permuted groups and their sample Fréchet mean. Permutation statistic Then the p-value of the test will be

35 PERMUTATION TESTS Test statistic If the groups have different means, the procedure needs to be applied to the centered observations Let us take M permutations of the original labels. For m=1,,m, let be the sample covariance operators for the permuted groups and their sample Fréchet mean. Permutation statistic Then the p-value of the test will be

36 APPLICATION: ACOUSTIC PHONETIC DATA

37 APPLICATION: ACOUSTIC PHONETIC DATA Historical and comparative linguistics: discipline that studies linguistic changes and evolution at different time scales. based on evidence coming from history, archeology, genetics, philology and similarity between existing languages. Our goal: to provide tools to explore phonetic changes between languages, based on acoustic data in place of phonetic transcriptions. We focus on a case where considerable information is already available: Romance languages (i.e., languages with the common root of Latin).

38 APPLICATION: ACOUSTIC PHONETIC DATA Example Spanish French Portuguese um [ˈu ] uno [ uno] un [œ ] uno [ uno] American Spanish uno [ uno] Italian We want to include the information from speech recordings!

39 APPLICATION: ACOUSTIC PHONETIC DATA Raw acoustic data Local Fourier Transform Smoothing Log-Spectrogram Time warping

40 APPLICATION: ACOUSTIC PHONETIC DATA Data: Log-spectrograms of recorded speech for 23 speakers of 5 different Romance languages, pronouncing 10 different words (numbers from 1 to 10). : 2 dimensional log-spectrogram for the language L, i-th word and k-th speaker. Residual log-spectrogram: We consider the residuals as replicates from the acoustic process: Total number of records for language L

41 APPLICATION: ACOUSTIC PHONETIC DATA How can we deal with this large amount of information? Significant phonetic features of the language are caught by relationships among different frequencies. Working hypothesis: existence of a language covariance structure, common to all words. The focus is on the covariance operator between frequencies, i.e. the relationships between frequencies in the spoken language. We consider this a descriptor of what a language sounds like, without considering the individual words.

42 APPLICATION: ACOUSTIC PHONETIC DATA What do we mean by frequency covariance operator? Let us suppose that the covariance function for each language logspectrogram has a separable structure, i.e. Frequency covariance function

43 APPLICATION: ACOUSTIC PHONETIC DATA What do we mean by frequency covariance operator? Let us suppose that the covariance function for each language logspectrogram has a separable structure, i.e. Possible estimator: Frequency covariance function

44 APPLICATION: ACOUSTIC PHONETIC DATA Spanish French Portuguese Is there a significant difference? American Spanish Italian

45 APPLICATION: ACOUSTIC PHONETIC DATA Let us check the assumptions we made about the frequency covariance operators: Are the covariance operators common to all words? Are the covariance operators significantly different between languages?

46 APPLICATION: ACOUSTIC PHONETIC DATA Let us check the assumptions we made about the frequency covariance operators: Are the covariance operators common to all words? P-values of the permutation test using words as groups (M=1000, Square root distance): French Italian Portuguese American Spanish Iberian Spanish Are the covariance operators significantly different between languages?

47 APPLICATION: ACOUSTIC PHONETIC DATA Let us check the assumptions we made about the frequency covariance operators: Are the covariance operators common to all words? P-values of the permutation test using words as groups (M=1000, Square root distance): French Italian Portuguese American Spanish Iberian Spanish Are the covariance operators significantly different between languages? P-values of the permutation test using languages as groups (M=1000, Square root distance): <0.001 Yes!

48 APPLICATION: ACOUSTIC PHONETIC DATA The same happens for the time covariance operators: Are the covariance operators common to all words? P-values of the permutation test using words as groups (M=1000, Square root distance): French Italian Portuguese American Spanish Iberian Spanish Are the covariance operators significantly different between languages? P-values of the permutation test using languages as groups (M=1000, Square root distance): <0.001 Yes!

49 APPLICATION: ACOUSTIC PHONETIC DATA This suggests a model with language-dependant covariance and worddependant means: Sound surface for a French Speaker pronouncing the word un (one) Average for one Frequency covariance operator 1 2 Time covariance operator 1 2 Projection of the French speaker in the population of Portuguese speakers

50 APPLICATION: ACOUSTIC PHONETIC DATA This suggests a model with language-dependant covariance and worddependant means: Sound surface for a French Speaker pronouncing the word un (one) Interpolated changing path? Projection of the French speaker in the population of Portuguese speakers

51 INTERPOLATION AND EXTRAPOLATION FOR COVARIANCE OPERATORS

52 INTERPOLATION AND EXTRAPOLATION Hilbert-Schmidt distance Square root distance Procrustes distance

53 APPLICATION: ACOUSTIC PHONETIC DATA Example of interpolation of frequency covariance operator (Procrustes distance) : French frequency covariance function Spanish frequency covariance function

54 APPLICATION: ACOUSTIC PHONETIC DATA Example of extrapolation of frequency covariance operator: P S F Comparison between estimated Portuguese covariance operator and the geodesic extrapolation from the two Spanish varieties. Non valid covariance operator

55 APPLICATION: ACOUSTIC PHONETIC DATA Example of extrapolation of frequency covariance operator: P S F Comparison between estimated Portuguese covariance operator and the geodesic extrapolation from the two Spanish varieties. Behavior of small eigenvalues in the Square root and Procrustes geodesics: In the Hilbert- Schmidt space Extrapolated covariance operators

56 FUTURE DIRECTIONS

57 FUTURE DIRECTIONS Spatial dependence

58 FUTURE DIRECTIONS Spatial dependence Comparison of covariance and time warping. Curves are usually aligned within each group but this means any across-group comparison may reflect mainly phase variability

59 FUTURE DIRECTIONS Spatial dependence Comparison of covariance and time warping. For the analysis of acoustic phonetic data: We assumed the covariance operators of the log-spectrograms are separable and we focused on the covariance between frequency. However, this assumption also needs to be checked. The next step will be to study the evolution of the covariance structure (and associated population of sounds) between languages, using geodesic paths.

60 BIBLIOGRAPHY Dryden, I.L., Koloydenko, A., Zhou, D. (2009) Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging, The Annals of Applied Statistics, 3, Fremdt, S., Steinebach, J.G., Horváth, L. and Kokoszka, P. (2013), Testing the Equality of Covariance Operators in Functional Samples,Scandinavian Journal of Statistics, 40, Panaretos, V. M., Kraus, D. and Maddocks, J.H. (2010) Second-Order Comparison of Gaussian Random Functions and the Geometry of DNA Minicircles, Journal of American Statistical Association, 105, Pigoli, D., Aston, J.A.D., Dryden, I.L., Secchi, P. (2014) Distances and Inference for Covariance Operators, Biometrika, 101, Pigoli, D., Hadjipantelis, P. Z., Coleman, J.S. and Aston, J.A.D. (2014) Exploring changes in Acoustic Phonetic Data: The spoken digits in Romance languages., in preparation. Sangalli, L.M., Secchi, P., Vantini, S., Veneziani, A. (2009), A Case Study in Exploratory Functional Data Analysis: Geometrical Features of the Internal Carotid Artery, Journal of the American Statistical Association, 104,