An Information Geometrical View of Stationary Subspace Analysis
|
|
- Brendan Henry
- 5 years ago
- Views:
Transcription
1 An Information Geometrica View of Stationary Subspace Anaysis Motoaki Kawanabe, Wojciech Samek, Pau von Bünau, and Frank C. Meinecke Fraunhofer Institute FIRST, Kekuéstr. 7, Berin, Germany Berin Institute of Technoogy, Frankinsr. 28/29, Berin, Germany Abstract. Stationary Subspace Anaysis (SSA) 3 is an unsupervised earning method that finds subspaces in which data distributions stay invariant over time. It has been shown to be very usefu for studying non-stationarities in various appications 5, 10, 4, 9. In this paper, we present the first SSA agorithm based on a fu generative mode of the data. This new derivation reates SSA to previous work on finding interesting subspaces from high-dimensiona data in a simiar way as the three easy routes to independent component anaysis 6, and provides an information geometric view. Keywords: stationary subspace anaysis, generative mode, maximum ikeihood estimation, Kuback-Leiber divergence, information geometry 1 Introduction Finding subspaces which contain interesting structures in high-dimensiona data is an important preprocessing step for efficient information processing. A cassica exampe is Principa Component Anaysis (PCA) that extracts ow-dimensiona subspaces which capture as much variance of the observations as possibe. In contrast, more recent approaches consider other criteria than maximum variance. For instance, non-gaussian component anaysis (NGCA) 2, 7, a genera semiparametric framework incuding projection pursuit 8, extracts non-gaussian structures, e.g. subspaces which revea custers or heavy taied distributions. Another recent direction is coored subspace anaysis 13 which searches for inear projections having tempora correations. What a these methods have in common is that i.i.d. (white) Gaussian random variabes are considered as noise, and each method seeks to maximize a specific deviation from it to uncover an informative subspace. This fact reminded us of the thought-provoking paper 6 showing three easy routes to Independent Component Anaysis (ICA). Independent components can be extracted either by using non-gaussianity, or by non-whiteness of spectrum, or by non-stationarity 11 of the variance. The née Wojcikiewicz
2 2 Kawanabe, Samek, von Bünau and Meinecke goa of this paper is to discuss projection methods corresponding to the third route, i.e. based on finding non-stationary structures. Recenty, Bünau et a. 3 proposed a nove technique caed Stationary Subspace Anaysis (SSA) which finds the ow-dimensiona projections having stationary distributions from high-dimensiona observations. This is an instrumenta anaysis whenever the observed data is conceivaby generated as a mixture of underying stationary and non-stationary components: a singe non-stationary source mixed into the observed signas can make the whoe data appear nonstationary; and non-stationary sources with ow power can remain hidden among strong stationary components. Uncovering these two groups of sources is therefore a key step towards understanding the data. Stationary Subspace Anaysis has been appied successfuy to biomedica signa processing 5, Computer Vision 10, high-dimensiona change point detection 4 and domain adaptation probems 9. However, the SSA agorithm 3 is based on finding the stationary sources and does not mode the non-stationary components, i.e. unike 11, the SSA agorithm is not derived from any generative mode of the observed data. In this paper, we assume a inear mixing mode with uncorreated stationary and nonstationary sources and compute the maximum ikeihood estimators of the projections onto both subspaces. That is, we aso expicity mode the non-stationary part. The objective function turns out to be a combination of the origina SSA objective and a term which penaizes cross-covariances between the stationary and non-stationary sources. The remainder of this paper is organized as foows. After expaining our mode assumptions in Section 2, we derive the objective function of the maximum ikeihood approach in Section 3 and provide a geometrica iustration. Then, we present the resuts of the numerica simuations in Section 4. 2 Bock Gaussian Mode for SSA In the SSA mode 3, we assume that the system of interest consists of d stationary source signas s s (t) = s 1 (t),s 2 (t),...,s d (t) (caed s-sources) and D d non-stationary source signas s n (t) = s d+1 (t),s d+2 (t),...,s D (t) (aso n- sources) where the observed signas x(t) are a inear superposition of the sources, x(t) = As(t) = A s A n s s (t) s n, (1) (t) and A is an invertibe matrix. Note that we do not assume that the sources s(t) are independent. We refer to the spaces spanned by A s and A n as the s- and n-space respectivey. The goa is to factorize the observed signas x(t) according to Eq. (1), i.e. to find a inear transformation  1 that separates the s-sources from the n-sources. Given this mode, the s-sources and the n-space are uniquey identifiabe whereas the n-sources and the s-space are not (see 3 for detais). In order to invert the mixing of stationary and non-stationary sources, we divide the time series x(t) of ength T into L epochs defined by the index sets
3 An Information Geometric View of SSA 3 T 1,...,T L. Even though the SSA mode incudes both the stationary and the non-stationary sources, the SSA agorithm 3 optimizes ony the stationarity of the estimated stationary sources and does not optimize the non-stationary source estimates: the approach is to find the projection such that the difference between the mean and covariance in each epoch is identica to the average mean and covariance over a epochs. In this paper, we derive a maximum ikeihood estimator for the SSA mode under the foowing additiona assumptions. The sources s are independent in time. The s-sourcess s aredrawnfrom a d-dimensiona GaussianN(µ s,σ s ), which is constant in time In each epoch, the n-sources s n are drawn from a (D d)-dimensiona Gaussian N(µ n,σn ), i.e. their distribution varies over epochs The s- and n-sources are group-wise uncorreated. Thus the true distribution of the observed signas x(t) in epoch is N(m,R ), where m = Aµ, R = AΣ A and µ s Σ µ = µ n, Σ = s 0 0 Σ n. (2) The unknown parameters of the mode are the mixing matrix A, the mean and covariance of the s-sources (µ s,σ s ) and those of the n-sources {(µ n,σn )}L =1. 3 Maximum Likeihood and its Information Geometric Interpretation In this section, we derive the objective function of the maximum ikeihood SSA (MLSSA). Under the bock Gaussian mode, the negative og ikeihood to be minimized becomes =1 t T ogp(x(t) A,µ s,µ n,σs,σ n ) ogdetσ = T og deta + TD 2 og2π =1 t T { +Tr Σ 1 A 1 (x(t) m )(x(t) m ) A }. (3) It is known that the maximum ikeihood estimation can be regarded as the minimum Kuback-Leiber-divergence (KLD) projection 1. Let x = 1 t T x(t), R = 1 t T (x(t) x )(x(t) x ) (4)
4 4 Kawanabe, Samek, von Bünau and Meinecke be the sampe mean and covariance of the -th chunk, where denotes its ength. Indeed, the ikeihood L ML can be expressed as D KL N(x, R ) N(m,R ) +const. (5) =1 In the foowing, we derive the estimators of the moment parameters expicity as functions of the mixing matrix A which eads to an objective function for estimating A. Since the KL-divergence is invariant under inear transformations, the divergence between the two distributions of the observation x is equa to that between the corresponding distributions of the sources s = A 1 x, i.e. D KL N(x, R ) N(m,R ) = D KL N(A 1 x,a 1 R A ) N(µ,Σ ). (6) The centra idea of this paper is to regard the KL-divergence as a distance measure in the space of Gaussian probabiity distributions, which eads to an information geometric viewpoint of the SSA objective. The divergence in Equation (6) is the distance between the empirica distribution of the demixed signas and the true underying sources. According to our assumption the true mode ies on a manifod M defined by Equation (2), i.e. { } M := N(µ,Σ) Σ Σ = s 0 0 Σ n. (7) Therefore it is convenient to spit the divergence(6) into the foowing two parts, 1. an orthogona projection D 1 onto the manifod M, D 1 = D KL N(A 1 x,a 1 R A ) N( µ, Σ ), 2. and a component D 2 in the manifod, D 2 = D KL N( µ, Σ ) N(µ,Σ ). This decomposition is iustrated in Figure 1. It is aso known as the generaized pythagorean theorem in information geometry 1. The orthogona projection onto the manifod M is given by µ = µ s µ n = Σs Σ = 0 0 Σ n ( A 1 ) s x (A 1 ) n x = A 1 x, (8) ( A 1 )s { (A ) } 1 s R 0 = ( 0 A 1 )n { (A ) } 1 n, (9) R where ( A 1) s and ( A 1 ) n are the projection matrices to the stationary and non-stationary sources respectivey. The projection onto the manifod M does
5 (A 1 x,a 1 R A ) An Information Geometric View of SSA 5 D 1 D s 2 ( µ s µ n Σ s 0, 0 Σ n M ) ( µ, Σ ) D n 2 D 2 ( µ s Σ µ n, s 0 0 Σ n ) (µ,σ ) Fig.1. A geometrica view of the divergence decomposition. The divergence D 1 corresponds to the orthogona projection onto the manifod of modes and D 2 is the divergence on the manifod, which can be further decomposed according to the bock diagona structure of the source covariance matrix. not affect the mean, it merey sets the off diagona bocks of the covariance matrix to zero. Conversey, on the manifod M, ony the diagona bocks vary, thus the above projection is orthogona w.r.t. the Fisher information matrix. For more detais, see 1. Since on M the stationary and non-stationary sources are independent by definition, we can further decompose the distance D 2 from N( µ, Σ ) to the true distribution N(µ,Σ ) into two independent parts, D 2 = D s 2 +D n 2 = D KL N( µ s, Σ s ) N(µ s,σ s ) +D KL N( µ n, Σ ) n N(µ n,σ) n. This decomposition eads to the foowing expression for the ikeihood, =1 {D KL N(A 1 x,a 1 R A ) +D KL N( µ s, Σ s ) N(µ s,σ s ) N( µ, Σ ) +D KL N( µ n, Σ n ) N(µ n,σ n ) }. (10) The moment parameters (µ s,σ s ) of the s-sources appear ony in the second term, whie those of the n-sources {(µ n,σn )}L =1 are incuded ony in the third term. Therefore, for fixed A, the moment estimators (denoted with hats) minimizing the ikeihood L ML can be cacuated from {( µ, Σ )} L =1 : for the n-sources, ( µ n, Σ n ) = ( µn, Σ n ), which means that Dn 2 = 0,
6 6 Kawanabe, Samek, von Bünau and Meinecke the mean µ s of the s-sources is the weighted average L =1 T µs or equivaenty ( A 1) s x, where x = 1 T T t=1x(t) is the goba mean, the covariance Σ s of the s-sources is the projection of the goba covariance R = 1 T T t=1 (x(t) x)(x(t) x) ontothes-space,i.e. ( { (A A 1)s 1 R ) } s By substituting these moment estimators, we obtain an objective function to determine the mixing matrix A, =1 {D KL N(A 1 x,a 1 R A ) + D KL N( µ s, Σ s ) N( µ, Σ ) N( µ s, Σ } s ). (11) Moreover, since the soution is undetermined up to scaing, sign and inear transformations within the s- and n-space, we set the estimated s-sources to zero mean and unit variance by centering and whitening of the data, i.e. we write the estimated demixing matrix as A 1 = BŴ where Ŵ = R 1/2 is a whitening matrix and B is an orthogona matrix. Let y = Ŵx and V = Ŵ R Ŵ be the mean and covariance of the centerized and whitened data y(t) = Ŵx(t) in the -th epoch. Then, the objective function becomes =1 {D KL N(By,B V B ) + D KL N(B s y,b s V (B s ) ) N(By,b-diagB s V (B s ),B n V (B n ) ) } N(0,I d ), (12) where B = (B s ),(B n ) and b-diag denotes the bock diagona matrix with given sub-matrices. The second term coincides with the origina SSA objective. The first term comes from the orthogonaity assumption and can be regarded as a joint bock diagonaization criterion. The impication of this joint objective is iustrated in Section 4. The objective function (12) can be further simpified as =1 2 { ogdet( Σ ) ogdet(σ n )+( µ s ) µ } s +const. (13) where µ s = B s y, Σs = B s V (B s ) and Σ := B V B = A 1 R A. As in the SSA agorithm 3, we optimize the objective function (12) using a gradient descent agorithm taking into account the Lie group structure of the set of orthogona matrices Numerica Exampe We compare the performance of our nove method with the origina SSA agorithm 3. In particuar, we study the case where the epoch covariance matrices are exacty bock diagonaizabe as this matches the assumptions of our method.
7 An Information Geometric View of SSA 7 In Figure 2, we compare MLSSA and SSA over varying numbers of epochs using 5 stationary sources and a 20-dimensiona input space, i.e. d = 5 and D = 20. For each epoch we randomy sampe a bock diagonaizabe covariance matrix and a mean vector and mix them with a random, but we-conditioned matrix A. Note that no estimation error is introduced as we sampe both moments directy. As performance measure we use the median ange to the true n-subspace over 100 trias and represent the 25% and 75% quanties by error bars. For each tria, in order to avoid oca minima, we restart the optimization procedure five times and seect the soution with the owest objective function vaue. From the resuts we see that our method yieds a much ower error than the origina approach, especiay when a sma number of epochs is used. For instance, when using MLSSA (red ine) 10 epochs are sufficient to achieve negigibe errors (around 0.01 degree) whereas for the origina SSA agorithm (bue ine) more than 30 epochs are required. Athough it seems that thanks to the first term in Eq. (12), i.e. joint bock diagonaization, MLSSA aows a much faster convergence than the origina method, more experiments wi be needed to obtain a fu picture. Ange to the true n subspace in degrees SSA MLSSA Number of epochs Fig.2. Performance Comparison between the proposed method (MLSSA) and the origina SSA for a varying number of epochs. The red curve shows the performance, measured as the median ange to the true n-subspace over 100 trias, of our method whereas the bue curve stands for the resuts obtained with SSA. The 25% and 75% quanties are represented by the error bars. Note that we use randomy samped bock diagonaizabe covariance matrices in our experiments and set the dimensionaity of the stationary subspace to 5 (out of 20 dimensions). Our proposed method significanty outperforms SSA in this exampe, especiay when ess than 30 epochs are used. MLSSA obviousy expoits (by the first term in Eq. (12)) the bock diagonaizabe structure of the covariance matrices and thus reconstructs the true stationary subspace much faster (i.e. with ess epochs) than SSA.
8 8 Kawanabe, Samek, von Bünau and Meinecke 5 Concusions In this paper, we deveoped the first generative version of SSA. The generative mode is a bock Gaussian mode and the objective function of the maximum ikeihood approach turns out to be a combination of the origina SSA and a joint bock diagonaization term. This new derivation not ony heps the theoretica understanding of the procedure in the information geometrica framework, but the agorithm aso yieds competitive resuts. Moreover, the ikeihood formuation aows future extension towards mode seection methods and for incorporating prior knowedge. Acknowedgement The authors acknowedge K.-R. Müer, F. J. Kiráy and D. Bythe for vauabe discussions. References 1. Amari, S.: Differentia-geometrica methods in statistics. Lecture notes in statistics, Springer-Verag, Berin (1985) 2. Banchard, G., Sugiyama, M., Kawanabe, M., Spokoiny, V., Müer, K.R.: In search of non-gaussian components of a high-dimensiona distribution. Journa of Machine Learning Research 7, (2006) 3. von Bünau, P., Meinecke, F.C., Kiráy, F., Müer, K.R.: Finding stationary subspaces in mutivariate time series. Physica Review Letters 103, (2009) 4. von Bünau, P., Meinecke, F.C., Müer, J.S., Lemm, S., Müer, K.R.: Boosting high-dimensiona change point detection with stationary subspace anaysis. In: Workshop on Tempora Segmentation at NIPS 2009 (2009) 5. von Bünau, P., Meinecke, F.C., Schoer, S., Müer, K.R.: Finding stationary brain sources in EEG data. In: Proceedings of the 32nd Annua Conference of the IEEE EMBS. pp (2010) 6. Cardoso, J.F.: The three easy routes to independent component anaysis; contrasts and geometry. In: Proc. ICA pp. 1 6 (2001) 7. Diederichs, E., Juditsky, A., Spokoiny, V., Schtte, C.: Sparse non-gaussian component anaysis. IEEE Trans. Inform. Theory 56, (2010) 8. Friedman, J.H., Tukey, J.W.: A projection pursuit agorithm for exporatory data anaysis. IEEE Trans. Computers 23, (1974) 9. Hara, S., Kawahara, Y., Washio, T., von Bünau, P.: Stationary subspace anaysis as a generaized eigenvaue probem. In: Proc of 17th internationa conference on Neura information processing: theory and agorithms - Voume Part I. pp ICONIP 10, Springer-Verag (2010) 10. Meinecke, F., von Bünau, P., Kawanabe, M., Müer, K.R.: Learning invariances with stationary subspace anaysis. In: Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th Internationa Conference on. pp (2009) 11. Pham, D.T., Cardoso, J.F.: Bind separation of instantaneous mixtures of non stationary sources. In: Proc. ICA pp Hesinki, Finand (2000) 12. Pumbey, M.D.: Geometrica methods for non-negative ica: Manifods, ie groups and tora subagebras. Neurocomputing 67( ) (2005) 13. Theis, F.: Coored subspace anaysis: Dimension reduction based on a signa s autocorreation structure. IEEE Trans. Circuits & Systems I 57(7), (2010)
FRST Multivariate Statistics. Multivariate Discriminant Analysis (MDA)
1 FRST 531 -- Mutivariate Statistics Mutivariate Discriminant Anaysis (MDA) Purpose: 1. To predict which group (Y) an observation beongs to based on the characteristics of p predictor (X) variabes, using
More informationA Brief Introduction to Markov Chains and Hidden Markov Models
A Brief Introduction to Markov Chains and Hidden Markov Modes Aen B MacKenzie Notes for December 1, 3, &8, 2015 Discrete-Time Markov Chains You may reca that when we first introduced random processes,
More informationA. Distribution of the test statistic
A. Distribution of the test statistic In the sequentia test, we first compute the test statistic from a mini-batch of size m. If a decision cannot be made with this statistic, we keep increasing the mini-batch
More informationStochastic Variational Inference with Gradient Linearization
Stochastic Variationa Inference with Gradient Linearization Suppementa Materia Tobias Pötz * Anne S Wannenwetsch Stefan Roth Department of Computer Science, TU Darmstadt Preface In this suppementa materia,
More informationFirst-Order Corrections to Gutzwiller s Trace Formula for Systems with Discrete Symmetries
c 26 Noninear Phenomena in Compex Systems First-Order Corrections to Gutzwier s Trace Formua for Systems with Discrete Symmetries Hoger Cartarius, Jörg Main, and Günter Wunner Institut für Theoretische
More informationStatistics for Applications. Chapter 7: Regression 1/43
Statistics for Appications Chapter 7: Regression 1/43 Heuristics of the inear regression (1) Consider a coud of i.i.d. random points (X i,y i ),i =1,...,n : 2/43 Heuristics of the inear regression (2)
More informationAn Algorithm for Pruning Redundant Modules in Min-Max Modular Network
An Agorithm for Pruning Redundant Modues in Min-Max Moduar Network Hui-Cheng Lian and Bao-Liang Lu Department of Computer Science and Engineering, Shanghai Jiao Tong University 1954 Hua Shan Rd., Shanghai
More informationASummaryofGaussianProcesses Coryn A.L. Bailer-Jones
ASummaryofGaussianProcesses Coryn A.L. Baier-Jones Cavendish Laboratory University of Cambridge caj@mrao.cam.ac.uk Introduction A genera prediction probem can be posed as foows. We consider that the variabe
More informationSUPPLEMENTARY MATERIAL TO INNOVATED SCALABLE EFFICIENT ESTIMATION IN ULTRA-LARGE GAUSSIAN GRAPHICAL MODELS
ISEE 1 SUPPLEMENTARY MATERIAL TO INNOVATED SCALABLE EFFICIENT ESTIMATION IN ULTRA-LARGE GAUSSIAN GRAPHICAL MODELS By Yingying Fan and Jinchi Lv University of Southern Caifornia This Suppementary Materia
More informationAppendix of the Paper The Role of No-Arbitrage on Forecasting: Lessons from a Parametric Term Structure Model
Appendix of the Paper The Roe of No-Arbitrage on Forecasting: Lessons from a Parametric Term Structure Mode Caio Ameida cameida@fgv.br José Vicente jose.vaentim@bcb.gov.br June 008 1 Introduction In this
More informationOptimality of Inference in Hierarchical Coding for Distributed Object-Based Representations
Optimaity of Inference in Hierarchica Coding for Distributed Object-Based Representations Simon Brodeur, Jean Rouat NECOTIS, Département génie éectrique et génie informatique, Université de Sherbrooke,
More informationarxiv: v1 [cs.lg] 31 Oct 2017
ACCELERATED SPARSE SUBSPACE CLUSTERING Abofaz Hashemi and Haris Vikao Department of Eectrica and Computer Engineering, University of Texas at Austin, Austin, TX, USA arxiv:7.26v [cs.lg] 3 Oct 27 ABSTRACT
More informationExpectation-Maximization for Estimating Parameters for a Mixture of Poissons
Expectation-Maximization for Estimating Parameters for a Mixture of Poissons Brandon Maone Department of Computer Science University of Hesini February 18, 2014 Abstract This document derives, in excrutiating
More information(This is a sample cover image for this issue. The actual cover is not yet available at this time.)
(This is a sampe cover image for this issue The actua cover is not yet avaiabe at this time) This artice appeared in a journa pubished by Esevier The attached copy is furnished to the author for interna
More informationA Simple and Efficient Algorithm of 3-D Single-Source Localization with Uniform Cross Array Bing Xue 1 2 a) * Guangyou Fang 1 2 b and Yicai Ji 1 2 c)
A Simpe Efficient Agorithm of 3-D Singe-Source Locaization with Uniform Cross Array Bing Xue a * Guangyou Fang b Yicai Ji c Key Laboratory of Eectromagnetic Radiation Sensing Technoogy, Institute of Eectronics,
More informationA Solution to the 4-bit Parity Problem with a Single Quaternary Neuron
Neura Information Processing - Letters and Reviews Vo. 5, No. 2, November 2004 LETTER A Soution to the 4-bit Parity Probem with a Singe Quaternary Neuron Tohru Nitta Nationa Institute of Advanced Industria
More informationMultilayer Kerceptron
Mutiayer Kerceptron Zotán Szabó, András Lőrincz Department of Information Systems, Facuty of Informatics Eötvös Loránd University Pázmány Péter sétány 1/C H-1117, Budapest, Hungary e-mai: szzoi@csetehu,
More informationSome Measures for Asymmetry of Distributions
Some Measures for Asymmetry of Distributions Georgi N. Boshnakov First version: 31 January 2006 Research Report No. 5, 2006, Probabiity and Statistics Group Schoo of Mathematics, The University of Manchester
More informationMelodic contour estimation with B-spline models using a MDL criterion
Meodic contour estimation with B-spine modes using a MDL criterion Damien Loive, Ney Barbot, Oivier Boeffard IRISA / University of Rennes 1 - ENSSAT 6 rue de Kerampont, B.P. 80518, F-305 Lannion Cedex
More informationExplicit overall risk minimization transductive bound
1 Expicit overa risk minimization transductive bound Sergio Decherchi, Paoo Gastado, Sandro Ridea, Rodofo Zunino Dept. of Biophysica and Eectronic Engineering (DIBE), Genoa University Via Opera Pia 11a,
More informationA Statistical Framework for Real-time Event Detection in Power Systems
1 A Statistica Framework for Rea-time Event Detection in Power Systems Noan Uhrich, Tim Christman, Phiip Swisher, and Xichen Jiang Abstract A quickest change detection (QCD) agorithm is appied to the probem
More informationTarget Location Estimation in Wireless Sensor Networks Using Binary Data
Target Location stimation in Wireess Sensor Networks Using Binary Data Ruixin Niu and Pramod K. Varshney Department of ectrica ngineering and Computer Science Link Ha Syracuse University Syracuse, NY 344
More informationLeast-squares Independent Component Analysis
Neura Computation, vo.23, no., pp.284 30, 20. Least-squares Independent Component Anaysis Taiji Suzuki Department of Mathematica Informatics, The University of Tokyo 7-3- Hongo, Bunkyo-ku, Tokyo 3-8656,
More informationA proposed nonparametric mixture density estimation using B-spline functions
A proposed nonparametric mixture density estimation using B-spine functions Atizez Hadrich a,b, Mourad Zribi a, Afif Masmoudi b a Laboratoire d Informatique Signa et Image de a Côte d Opae (LISIC-EA 4491),
More informationBayesian Learning. You hear a which which could equally be Thanks or Tanks, which would you go with?
Bayesian Learning A powerfu and growing approach in machine earning We use it in our own decision making a the time You hear a which which coud equay be Thanks or Tanks, which woud you go with? Combine
More informationBayesian Unscented Kalman Filter for State Estimation of Nonlinear and Non-Gaussian Systems
Bayesian Unscented Kaman Fiter for State Estimation of Noninear and Non-aussian Systems Zhong Liu, Shing-Chow Chan, Ho-Chun Wu and iafei Wu Department of Eectrica and Eectronic Engineering, he University
More informationBALANCING REGULAR MATRIX PENCILS
BALANCING REGULAR MATRIX PENCILS DAMIEN LEMONNIER AND PAUL VAN DOOREN Abstract. In this paper we present a new diagona baancing technique for reguar matrix pencis λb A, which aims at reducing the sensitivity
More informationStatistical Learning Theory: A Primer
Internationa Journa of Computer Vision 38(), 9 3, 2000 c 2000 uwer Academic Pubishers. Manufactured in The Netherands. Statistica Learning Theory: A Primer THEODOROS EVGENIOU, MASSIMILIANO PONTIL AND TOMASO
More informationActive Learning & Experimental Design
Active Learning & Experimenta Design Danie Ting Heaviy modified, of course, by Lye Ungar Origina Sides by Barbara Engehardt and Aex Shyr Lye Ungar, University of Pennsyvania Motivation u Data coection
More informationAlberto Maydeu Olivares Instituto de Empresa Marketing Dept. C/Maria de Molina Madrid Spain
CORRECTIONS TO CLASSICAL PROCEDURES FOR ESTIMATING THURSTONE S CASE V MODEL FOR RANKING DATA Aberto Maydeu Oivares Instituto de Empresa Marketing Dept. C/Maria de Moina -5 28006 Madrid Spain Aberto.Maydeu@ie.edu
More informationA SIMPLIFIED DESIGN OF MULTIDIMENSIONAL TRANSFER FUNCTION MODELS
A SIPLIFIED DESIGN OF ULTIDIENSIONAL TRANSFER FUNCTION ODELS Stefan Petrausch, Rudof Rabenstein utimedia Communications and Signa Procesg, University of Erangen-Nuremberg, Cauerstr. 7, 958 Erangen, GERANY
More information6.434J/16.391J Statistics for Engineers and Scientists May 4 MIT, Spring 2006 Handout #17. Solution 7
6.434J/16.391J Statistics for Engineers and Scientists May 4 MIT, Spring 2006 Handout #17 Soution 7 Probem 1: Generating Random Variabes Each part of this probem requires impementation in MATLAB. For the
More informationOn a geometrical approach in contact mechanics
Institut für Mechanik On a geometrica approach in contact mechanics Aexander Konyukhov, Kar Schweizerhof Universität Karsruhe, Institut für Mechanik Institut für Mechanik Kaiserstr. 12, Geb. 20.30 76128
More informationExtended SMART Algorithms for Non-Negative Matrix Factorization
Extended SMART Agorithms for Non-Negative Matrix Factorization Andrzej CICHOCKI 1, Shun-ichi AMARI 2 Rafa ZDUNEK 1, Rau KOMPASS 1, Gen HORI 1 and Zhaohui HE 1 Invited Paper 1 Laboratory for Advanced Brain
More informationFORECASTING TELECOMMUNICATIONS DATA WITH AUTOREGRESSIVE INTEGRATED MOVING AVERAGE MODELS
FORECASTING TEECOMMUNICATIONS DATA WITH AUTOREGRESSIVE INTEGRATED MOVING AVERAGE MODES Niesh Subhash naawade a, Mrs. Meenakshi Pawar b a SVERI's Coege of Engineering, Pandharpur. nieshsubhash15@gmai.com
More informationDIGITAL FILTER DESIGN OF IIR FILTERS USING REAL VALUED GENETIC ALGORITHM
DIGITAL FILTER DESIGN OF IIR FILTERS USING REAL VALUED GENETIC ALGORITHM MIKAEL NILSSON, MATTIAS DAHL AND INGVAR CLAESSON Bekinge Institute of Technoogy Department of Teecommunications and Signa Processing
More informationSteepest Descent Adaptation of Min-Max Fuzzy If-Then Rules 1
Steepest Descent Adaptation of Min-Max Fuzzy If-Then Rues 1 R.J. Marks II, S. Oh, P. Arabshahi Λ, T.P. Caude, J.J. Choi, B.G. Song Λ Λ Dept. of Eectrica Engineering Boeing Computer Services University
More information4 Separation of Variables
4 Separation of Variabes In this chapter we describe a cassica technique for constructing forma soutions to inear boundary vaue probems. The soution of three cassica (paraboic, hyperboic and eiptic) PDE
More informationThe EM Algorithm applied to determining new limit points of Mahler measures
Contro and Cybernetics vo. 39 (2010) No. 4 The EM Agorithm appied to determining new imit points of Maher measures by Souad E Otmani, Georges Rhin and Jean-Marc Sac-Épée Université Pau Veraine-Metz, LMAM,
More informationScalable Spectrum Allocation for Large Networks Based on Sparse Optimization
Scaabe Spectrum ocation for Large Networks ased on Sparse Optimization innan Zhuang Modem R&D Lab Samsung Semiconductor, Inc. San Diego, C Dongning Guo, Ermin Wei, and Michae L. Honig Department of Eectrica
More informationA Robust Voice Activity Detection based on Noise Eigenspace Projection
A Robust Voice Activity Detection based on Noise Eigenspace Projection Dongwen Ying 1, Yu Shi 2, Frank Soong 2, Jianwu Dang 1, and Xugang Lu 1 1 Japan Advanced Institute of Science and Technoogy, Nomi
More informationStochastic Automata Networks (SAN) - Modelling. and Evaluation. Paulo Fernandes 1. Brigitte Plateau 2. May 29, 1997
Stochastic utomata etworks (S) - Modeing and Evauation Pauo Fernandes rigitte Pateau 2 May 29, 997 Institut ationa Poytechnique de Grenobe { IPG Ecoe ationae Superieure d'informatique et de Mathematiques
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part IX The EM agorithm In the previous set of notes, we taked about the EM agorithm as appied to fitting a mixture of Gaussians. In this set of notes, we give a broader view
More informationA Comparison Study of the Test for Right Censored and Grouped Data
Communications for Statistica Appications and Methods 2015, Vo. 22, No. 4, 313 320 DOI: http://dx.doi.org/10.5351/csam.2015.22.4.313 Print ISSN 2287-7843 / Onine ISSN 2383-4757 A Comparison Study of the
More informationProbabilistic Graphical Models
Schoo of Computer Science Probabiistic Graphica Modes Gaussian graphica modes and Ising modes: modeing networks Eric Xing Lecture 0, February 0, 07 Reading: See cass website Eric Xing @ CMU, 005-07 Network
More informationAn approximate method for solving the inverse scattering problem with fixed-energy data
J. Inv. I-Posed Probems, Vo. 7, No. 6, pp. 561 571 (1999) c VSP 1999 An approximate method for soving the inverse scattering probem with fixed-energy data A. G. Ramm and W. Scheid Received May 12, 1999
More informationLecture 6: Moderately Large Deflection Theory of Beams
Structura Mechanics 2.8 Lecture 6 Semester Yr Lecture 6: Moderatey Large Defection Theory of Beams 6.1 Genera Formuation Compare to the cassica theory of beams with infinitesima deformation, the moderatey
More informationAvailable online at ScienceDirect. Procedia Computer Science 96 (2016 )
Avaiabe onine at www.sciencedirect.com ScienceDirect Procedia Computer Science 96 (206 92 99 20th Internationa Conference on Knowedge Based and Inteigent Information and Engineering Systems Connected categorica
More informationBP neural network-based sports performance prediction model applied research
Avaiabe onine www.jocpr.com Journa of Chemica and Pharmaceutica Research, 204, 6(7:93-936 Research Artice ISSN : 0975-7384 CODEN(USA : JCPRC5 BP neura networ-based sports performance prediction mode appied
More informationIdentification of macro and micro parameters in solidification model
BULLETIN OF THE POLISH ACADEMY OF SCIENCES TECHNICAL SCIENCES Vo. 55, No. 1, 27 Identification of macro and micro parameters in soidification mode B. MOCHNACKI 1 and E. MAJCHRZAK 2,1 1 Czestochowa University
More informationMode in Output Participation Factors for Linear Systems
2010 American ontro onference Marriott Waterfront, Batimore, MD, USA June 30-Juy 02, 2010 WeB05.5 Mode in Output Participation Factors for Linear Systems Li Sheng, yad H. Abed, Munther A. Hassouneh, Huizhong
More informationSTABILITY OF A PARAMETRICALLY EXCITED DAMPED INVERTED PENDULUM 1. INTRODUCTION
Journa of Sound and Vibration (996) 98(5), 643 65 STABILITY OF A PARAMETRICALLY EXCITED DAMPED INVERTED PENDULUM G. ERDOS AND T. SINGH Department of Mechanica and Aerospace Engineering, SUNY at Buffao,
More informationPartial permutation decoding for MacDonald codes
Partia permutation decoding for MacDonad codes J.D. Key Department of Mathematics and Appied Mathematics University of the Western Cape 7535 Bevie, South Africa P. Seneviratne Department of Mathematics
More informationTHE THREE POINT STEINER PROBLEM ON THE FLAT TORUS: THE MINIMAL LUNE CASE
THE THREE POINT STEINER PROBLEM ON THE FLAT TORUS: THE MINIMAL LUNE CASE KATIE L. MAY AND MELISSA A. MITCHELL Abstract. We show how to identify the minima path network connecting three fixed points on
More informationEfficiently Generating Random Bits from Finite State Markov Chains
1 Efficienty Generating Random Bits from Finite State Markov Chains Hongchao Zhou and Jehoshua Bruck, Feow, IEEE Abstract The probem of random number generation from an uncorreated random source (of unknown
More informationTwo view learning: SVM-2K, Theory and Practice
Two view earning: SVM-2K, Theory and Practice Jason D.R. Farquhar jdrf99r@ecs.soton.ac.uk Hongying Meng hongying@cs.york.ac.uk David R. Hardoon drh@ecs.soton.ac.uk John Shawe-Tayor jst@ecs.soton.ac.uk
More informationSequential Decoding of Polar Codes with Arbitrary Binary Kernel
Sequentia Decoding of Poar Codes with Arbitrary Binary Kerne Vera Miosavskaya, Peter Trifonov Saint-Petersburg State Poytechnic University Emai: veram,petert}@dcn.icc.spbstu.ru Abstract The probem of efficient
More informationSVM-based Supervised and Unsupervised Classification Schemes
SVM-based Supervised and Unsupervised Cassification Schemes LUMINITA STATE University of Pitesti Facuty of Mathematics and Computer Science 1 Targu din Vae St., Pitesti 110040 ROMANIA state@cicknet.ro
More informationPARSIMONIOUS VARIATIONAL-BAYES MIXTURE AGGREGATION WITH A POISSON PRIOR. Pierrick Bruneau, Marc Gelgon and Fabien Picarougne
17th European Signa Processing Conference (EUSIPCO 2009) Gasgow, Scotand, August 24-28, 2009 PARSIMONIOUS VARIATIONAL-BAYES MIXTURE AGGREGATION WITH A POISSON PRIOR Pierric Bruneau, Marc Gegon and Fabien
More informationEnhanced Trajectory Based Similarity Prediction with Uncertainty Quantification
Enhanced Trajectory Based Simiarity Prediction with Uncertainty Quantification Jack Lam 1, Shankar Sankararaman 2, and Bryan Stewart 3 1,3 Nava Surface Warfare Center Port Hueneme Division, Port Hueneme,
More informationPublished in: Proceedings of the Twenty Second Nordic Seminar on Computational Mechanics
Aaborg Universitet An Efficient Formuation of the Easto-pastic Constitutive Matrix on Yied Surface Corners Causen, Johan Christian; Andersen, Lars Vabbersgaard; Damkide, Lars Pubished in: Proceedings of
More informationA Novel Learning Method for Elman Neural Network Using Local Search
Neura Information Processing Letters and Reviews Vo. 11, No. 8, August 2007 LETTER A Nove Learning Method for Eman Neura Networ Using Loca Search Facuty of Engineering, Toyama University, Gofuu 3190 Toyama
More informationAlgorithms to solve massively under-defined systems of multivariate quadratic equations
Agorithms to sove massivey under-defined systems of mutivariate quadratic equations Yasufumi Hashimoto Abstract It is we known that the probem to sove a set of randomy chosen mutivariate quadratic equations
More informationSoft Clustering on Graphs
Soft Custering on Graphs Kai Yu 1, Shipeng Yu 2, Voker Tresp 1 1 Siemens AG, Corporate Technoogy 2 Institute for Computer Science, University of Munich kai.yu@siemens.com, voker.tresp@siemens.com spyu@dbs.informatik.uni-muenchen.de
More informationMoreau-Yosida Regularization for Grouped Tree Structure Learning
Moreau-Yosida Reguarization for Grouped Tree Structure Learning Jun Liu Computer Science and Engineering Arizona State University J.Liu@asu.edu Jieping Ye Computer Science and Engineering Arizona State
More informationApproximation and Fast Calculation of Non-local Boundary Conditions for the Time-dependent Schrödinger Equation
Approximation and Fast Cacuation of Non-oca Boundary Conditions for the Time-dependent Schrödinger Equation Anton Arnod, Matthias Ehrhardt 2, and Ivan Sofronov 3 Universität Münster, Institut für Numerische
More informationAkaike Information Criterion for ANOVA Model with a Simple Order Restriction
Akaike Information Criterion for ANOVA Mode with a Simpe Order Restriction Yu Inatsu * Department of Mathematics, Graduate Schoo of Science, Hiroshima University ABSTRACT In this paper, we consider Akaike
More informationModel-based Clustering by Probabilistic Self-organizing Maps
EEE TRANATN N NERA NETRK, V. XX, N., ode-based ustering by robabiistic ef-organizing aps hih-ian heng, Hsin-hia u, ember, EEE, and Hsin-in ang, ember, EEE Abstract n this paper, we consider the earning
More informationKernel pea and De-Noising in Feature Spaces
Kerne pea and De-Noising in Feature Spaces Sebastian Mika, Bernhard Schokopf, Aex Smoa Kaus-Robert Muer, Matthias Schoz, Gunnar Riitsch GMD FIRST, Rudower Chaussee 5, 12489 Berin, Germany {mika, bs, smoa,
More informationV.B The Cluster Expansion
V.B The Custer Expansion For short range interactions, speciay with a hard core, it is much better to repace the expansion parameter V( q ) by f( q ) = exp ( βv( q )), which is obtained by summing over
More informationpp in Backpropagation Convergence Via Deterministic Nonmonotone Perturbed O. L. Mangasarian & M. V. Solodov Madison, WI Abstract
pp 383-390 in Advances in Neura Information Processing Systems 6 J.D. Cowan, G. Tesauro and J. Aspector (eds), Morgan Kaufmann Pubishers, San Francisco, CA, 1994 Backpropagation Convergence Via Deterministic
More informationPhysics 235 Chapter 8. Chapter 8 Central-Force Motion
Physics 35 Chapter 8 Chapter 8 Centra-Force Motion In this Chapter we wi use the theory we have discussed in Chapter 6 and 7 and appy it to very important probems in physics, in which we study the motion
More informationWidth of Percolation Transition in Complex Networks
APS/23-QED Width of Percoation Transition in Compex Networs Tomer Kaisy, and Reuven Cohen 2 Minerva Center and Department of Physics, Bar-Ian University, 52900 Ramat-Gan, Israe 2 Department of Computer
More informationSTA 216 Project: Spline Approach to Discrete Survival Analysis
: Spine Approach to Discrete Surviva Anaysis November 4, 005 1 Introduction Athough continuous surviva anaysis differs much from the discrete surviva anaysis, there is certain ink between the two modeing
More informationInternational Journal of Mass Spectrometry
Internationa Journa of Mass Spectrometry 280 (2009) 179 183 Contents ists avaiabe at ScienceDirect Internationa Journa of Mass Spectrometry journa homepage: www.esevier.com/ocate/ijms Stark mixing by ion-rydberg
More informationA Ridgelet Kernel Regression Model using Genetic Algorithm
A Ridgeet Kerne Regression Mode using Genetic Agorithm Shuyuan Yang, Min Wang, Licheng Jiao * Institute of Inteigence Information Processing, Department of Eectrica Engineering Xidian University Xi an,
More informationIncremental Reformulated Automatic Relevance Determination
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 60, NO. 9, SEPTEMBER 22 4977 Incrementa Reformuated Automatic Reevance Determination Dmitriy Shutin, Sanjeev R. Kukarni, and H. Vincent Poor Abstract In this
More informationPath planning with PH G2 splines in R2
Path panning with PH G2 spines in R2 Laurent Gajny, Richard Béarée, Eric Nyiri, Oivier Gibaru To cite this version: Laurent Gajny, Richard Béarée, Eric Nyiri, Oivier Gibaru. Path panning with PH G2 spines
More informationAST 418/518 Instrumentation and Statistics
AST 418/518 Instrumentation and Statistics Cass Website: http://ircamera.as.arizona.edu/astr_518 Cass Texts: Practica Statistics for Astronomers, J.V. Wa, and C.R. Jenkins, Second Edition. Measuring the
More informationProduct Cosines of Angles between Subspaces
Product Cosines of Anges between Subspaces Jianming Miao and Adi Ben-Israe Juy, 993 Dedicated to Professor C.R. Rao on his 75th birthday Let Abstract cos{l,m} : i cos θ i, denote the product of the cosines
More informationFitting affine and orthogonal transformations between two sets of points
Mathematica Communications 9(2004), 27-34 27 Fitting affine and orthogona transformations between two sets of points Hemuth Späth Abstract. Let two point sets P and Q be given in R n. We determine a transation
More informationTechnical Appendix for Voting, Speechmaking, and the Dimensions of Conflict in the US Senate
Technica Appendix for Voting, Speechmaking, and the Dimensions of Confict in the US Senate In Song Kim John Londregan Marc Ratkovic Juy 6, 205 Abstract We incude here severa technica appendices. First,
More informationSupervised i-vector Modeling - Theory and Applications
Supervised i-vector Modeing - Theory and Appications Shreyas Ramoji, Sriram Ganapathy Learning and Extraction of Acoustic Patterns LEAP) Lab, Eectrica Engineering, Indian Institute of Science, Bengauru,
More informationSubstructuring Preconditioners for the Bidomain Extracellular Potential Problem
Substructuring Preconditioners for the Bidomain Extraceuar Potentia Probem Mico Pennacchio 1 and Vaeria Simoncini 2,1 1 IMATI - CNR, via Ferrata, 1, 27100 Pavia, Itay mico@imaticnrit 2 Dipartimento di
More informationInverse-Variance Weighting PCA-based VRE criterion to select the optimal number of PCs
Preprints of the 18th IFAC Word Congress Miano (Itay) August 28 - September 2, 211 Inverse-Variance Weighting PCA-based VRE criterion to seect the optima number of PCs Baigh Mnassri E Mostafa E Ade Mustapha
More informationDo Schools Matter for High Math Achievement? Evidence from the American Mathematics Competitions Glenn Ellison and Ashley Swanson Online Appendix
VOL. NO. DO SCHOOLS MATTER FOR HIGH MATH ACHIEVEMENT? 43 Do Schoos Matter for High Math Achievement? Evidence from the American Mathematics Competitions Genn Eison and Ashey Swanson Onine Appendix Appendix
More informationUniformly Reweighted Belief Propagation: A Factor Graph Approach
Uniformy Reweighted Beief Propagation: A Factor Graph Approach Henk Wymeersch Chamers University of Technoogy Gothenburg, Sweden henkw@chamers.se Federico Penna Poitecnico di Torino Torino, Itay federico.penna@poito.it
More informationCryptanalysis of PKP: A New Approach
Cryptanaysis of PKP: A New Approach Éiane Jaumes and Antoine Joux DCSSI 18, rue du Dr. Zamenhoff F-92131 Issy-es-Mx Cedex France eiane.jaumes@wanadoo.fr Antoine.Joux@ens.fr Abstract. Quite recenty, in
More informationC. Fourier Sine Series Overview
12 PHILIP D. LOEWEN C. Fourier Sine Series Overview Let some constant > be given. The symboic form of the FSS Eigenvaue probem combines an ordinary differentia equation (ODE) on the interva (, ) with a
More informationarxiv: v1 [cs.db] 1 Aug 2012
Functiona Mechanism: Regression Anaysis under Differentia Privacy arxiv:208.029v [cs.db] Aug 202 Jun Zhang Zhenjie Zhang 2 Xiaokui Xiao Yin Yang 2 Marianne Winsett 2,3 ABSTRACT Schoo of Computer Engineering
More informationTwo-sample inference for normal mean vectors based on monotone missing data
Journa of Mutivariate Anaysis 97 (006 6 76 wwweseviercom/ocate/jmva Two-sampe inference for norma mean vectors based on monotone missing data Jianqi Yu a, K Krishnamoorthy a,, Maruthy K Pannaa b a Department
More informationMONTE CARLO SIMULATIONS
MONTE CARLO SIMULATIONS Current physics research 1) Theoretica 2) Experimenta 3) Computationa Monte Caro (MC) Method (1953) used to study 1) Discrete spin systems 2) Fuids 3) Poymers, membranes, soft matter
More informationNEW DEVELOPMENT OF OPTIMAL COMPUTING BUDGET ALLOCATION FOR DISCRETE EVENT SIMULATION
NEW DEVELOPMENT OF OPTIMAL COMPUTING BUDGET ALLOCATION FOR DISCRETE EVENT SIMULATION Hsiao-Chang Chen Dept. of Systems Engineering University of Pennsyvania Phiadephia, PA 904-635, U.S.A. Chun-Hung Chen
More informationT.C. Banwell, S. Galli. {bct, Telcordia Technologies, Inc., 445 South Street, Morristown, NJ 07960, USA
ON THE SYMMETRY OF THE POWER INE CHANNE T.C. Banwe, S. Gai {bct, sgai}@research.tecordia.com Tecordia Technoogies, Inc., 445 South Street, Morristown, NJ 07960, USA Abstract The indoor power ine network
More informationStrauss PDEs 2e: Section Exercise 1 Page 1 of 7
Strauss PDEs 2e: Section 4.3 - Exercise 1 Page 1 of 7 Exercise 1 Find the eigenvaues graphicay for the boundary conditions X(0) = 0, X () + ax() = 0. Assume that a 0. Soution The aim here is to determine
More informationPrimal and dual active-set methods for convex quadratic programming
Math. Program., Ser. A 216) 159:469 58 DOI 1.17/s117-15-966-2 FULL LENGTH PAPER Prima and dua active-set methods for convex quadratic programming Anders Forsgren 1 Phiip E. Gi 2 Eizabeth Wong 2 Received:
More informationMA 201: Partial Differential Equations Lecture - 10
MA 201: Partia Differentia Equations Lecture - 10 Separation of Variabes, One dimensiona Wave Equation Initia Boundary Vaue Probem (IBVP) Reca: A physica probem governed by a PDE may contain both boundary
More informationSource and Relay Matrices Optimization for Multiuser Multi-Hop MIMO Relay Systems
Source and Reay Matrices Optimization for Mutiuser Muti-Hop MIMO Reay Systems Yue Rong Department of Eectrica and Computer Engineering, Curtin University, Bentey, WA 6102, Austraia Abstract In this paper,
More informationTraffic data collection
Chapter 32 Traffic data coection 32.1 Overview Unike many other discipines of the engineering, the situations that are interesting to a traffic engineer cannot be reproduced in a aboratory. Even if road
More informationFast Blind Recognition of Channel Codes
Fast Bind Recognition of Channe Codes Reza Moosavi and Erik G. Larsson Linköping University Post Print N.B.: When citing this work, cite the origina artice. 213 IEEE. Persona use of this materia is permitted.
More information