3.3 Variational Characterization of Singular Values
|
|
- Lindsay Elliott
- 6 years ago
- Views:
Transcription
1 3.3. Variational Characterization of Singular Values Variational Characterization of Singular Values Since the singular values are square roots of the eigenvalues of the Heritian atrices A A and AA, the singular values inherit the variational characterizations that were explored in Section 2.2. For exaple, v A Av 1/2 u AA u 1/2, σ 1 = ax = ax v n v v u u u with the leading right and left singular vectors v 1 and u 1 being unit vectors that attain these axia. However, the singular values also satisfy a subtler variational property that incorporates both left and right singular vectors at the sae tie. Consider, for unit vectors u and v n, the quantity u Av uav = σ 1, using the Cauchy Schwarz inequality and the definition of the induced atrix 2-nor. On the other hand, if u 1 and v 1 unit vectors that give Hence u 1Av 1 = u 1(σ 1 u 1 ) = σ 1. σ 1 = u Av ax u,v n uv. We can characterize subsequent singular values the sae way. Recall the dyadic version of the singular value decoposition, A = r σ j u j vj for A of rank r. If we restrict unit vectors u and v such to be orthogonal to u 1 and v 1,then u Av = r σ j u u j vj v = r j=2 σ j u u j v j v = u r j=2 σ j u j v j v. Hence u Av σ 2, with the inequality attained when u = u 1 and v = v 1. process gives the following analogue of Theore 2.2. Continuing this
2 62 Chapter 3. Singular Value Decoposition Theore 3.4. For any A n, σ k = u Av ax u span{u 1,...,u k 1 } uv. v span{v 1,...,v k 1 } 3.4 Principal Coponent Analysis Matrix theory enables the analysis of the volues of data that now so coonly arise fro applications ranging fro basic science to public policy. Such easured data often depends on any factors, and we seek to identify those that are ost critical. Within this real of ultivariate statistics, principal coponent analysis (PCA) is a fundaental tool. Linear algebraists often say, PCA is the SVD in this section, we will explain what this eans, and soe of the subtleties involved Variance and covariance To understand principal coponent analysis, we need soe basic notions fro statistics, described in any basic textbook. For a general description of PCA along with nuerous applications, see the text by Jolliffe [Jol02], whose presentation shaped parts of our discussion here. The expected value, or ean, of a rando variable X is denoted E[X]. The expected value is a linear function, so for any constants α, β, E[αX + β] =αe[x]+β. The variance of X describes how uch X is expected to deviate fro its ean, Var(X) =E[(X E[X]) 2 ], which, using linearity of the expected value, takes the equivalent for Var(X) =E[X 2 ] E[X] 2. The covariance between two (potentially correlated) rando variables X and Y is Cov(X, Y )=E[(X E[X])(Y E[Y ])] = E[XY ] E[X]E[Y ]. with Cov(X,X) = Var(X). These definitions of variance and covariance are the bedrock concepts underneath PCA, for with the we can understand the variance present in a linear cobination of several rando variables.
3 3.4. Principal Coponent Analysis 63 Suppose we have a set of real-valued rando variables X 1,...,X n in which we suspect there ay be soe redundancy. Perhaps soe of these variables can be expressed as linear cobinations of the others either exactly, or nearly so. At the other extree, there ay be soe way to cobine X 1,...,X n that captures uch of the variance in one (or a few) aggregate rando variables. In particular, we shall seek scalars γ 1,...,γ n such that n γ j X j has the largest possible variance. The definitions of variance and covariance, along with the linearity of the expected value, lead to a forula for the variance of a linear cobination of rando variables: n n n Var γ j X j = γ j γ k Cov(X j,x k ). (3.1) k=1 You have seen double sus like this before. atrix C n n having (j, k) entry c j,k = Cov(X j,x k ), If we define the covariance and let v =[γ 1,...,γ n ] T, then the variance of the cobined variable is just a Rayleigh quotient: n Var γ j X j = v Cv. Since the covariance function is syetric: Cov(X, Y ) = Cov(Y,X), the atrix C is Heritian; it is also positive seidefinite. Why? Variance, by its definition as the expected value of the square of a real rando variable, is always nonnegative. Thus the forula (3.1), which derives fro the linearity of the expected value, ensures that v Cv 0. (Under what circustances can this quantity be zero?) We can write C in another convenient way. Collect the rando variables into the vector X 1 X =.. X n Then the (j, k) entry of E[XX ] E[X]E[X ]is and so E[X j X k ] E[X j ]E[X k ]=Cov(X j,x k )=c j,k, C = E[XX ] E[X]E[X ].
4 64 Chapter 3. Singular Value Decoposition Derived variables that axiize variance Return now to the proble of axiizing the variance of v Cv. Without constraint on v, this quantity can be arbitrarily large (assuing C is nonzero); thus we shall require that k γ2 j = v2 =1. With this noralization, you iediately see how to axiize the variance v Cv: v should be a unit eigenvector associated with the largest agnitude eigenvalue of C; call this vector v 1. The associated variance, of course, is the largest eigenvalue of C; call it λ 1 = v1cv v Cv 1 = ax v n v v. The eigenvector v 1 encodes the way to cobine X 1,...,X n to axiize variance. The new variable the leading principal coponent is v 1X = n γ j X j. You are already suspecting that a unit eigenvector associated with the second largest eigenvalue, v 2 with λ 2 = v 2 Cv 2, ust encode the second-largest way to axiize variance. Let us explore this intuition. To find the second-best way to cobine the variables, we should insist that the next new variable, for now call it w X, should be independent of the first, i.e., Cov(v 1X, w X)=0. However, using linearity of expectation and the fact that, e.g., w X = X w for real vectors, Cov(v 1X, w X)=E[(v 1X)(w X)] E[v 1X]E[w X] = E[(v 1XX w] E[v 1X]E[X w] = v 1E[XX ]w v 1E[X]E[X ]w = v 1(E[XX ] E[X]E[X ])w = v 1Cw = λ 1 v 1w. Hence (assuing λ 1 = 0), for the cobined variables v1 X and w X to be independent, the vectors v 1 and w ust be orthogonal, perfectly confiring your intuition: the second-best way to cobine the variables is to pick
5 3.4. Principal Coponent Analysis 65 w to be a unit eigenvector v 2 of C corresponding to the second largest eigenvalue a direct result of the variational characterization of eigenvalues in Theore 2.2. The associated variance of v 2 X is λ 2 = w Cw ax w span{u 1 } w w. Of course, in general, the kth best way to cobine the variables is given by the eigenvector v k of C associated with the kth largest eigenvalue. We learn uch about our variables fro the relative size of the variances (eigenvalues) λ 1 λ 2 λ n 0. If soe of the latter eigenvalues are very sall, that indicates that the set of n rando variables can be well approxiated by a fewer nuber of aggregate variables. These aggregate variables are the principal coponents of X 1,...,X n Approxiate PCA fro epirical data In practical situations, one often seeks to analyze epirical data drawn fro soe unknown distribution: the expected values and covariances are not available. Instead, we will estiate these fro the easured data. Suppose, as before, that we are considering n rando variables, X 1,...,X n, with saples of each: x j,k, k =1,...,, i.e., x j,k is the kth saple of the rando variable X j. The expected value has a the failiar unbiased estiate µ j = 1 x j,k. Siilarly, we can approxiate the covariance Cov(X j,x k )=E[(X j E[X j ])(X k E[X k )]. One ight naturally estiate this as 1 (x j, µ j )(x k, µ k ). =1
6 66 Chapter 3. Singular Value Decoposition However, replacing the true expected values E[X j ] and E[X k ]withtheepirical estiates µ j and µ k introduces soe slight bias into this estiate. This bias can be reoved by applying Bessel s correction [??], replacing 1/ by 1/( 1) to get the unbiased estiate If we let s j,k = 1 1 (x j, µ j )(x k, µ k ), j,k =1,...,n. =1 x j = x j,1. x j,, j =1,...,n, then each covariance estiate is just an inner product s j,k = 1 1 (x j µ j ) (x k µ k ). Thus, if we center the saples of each variable about its epirical ean, we can write the epirical covariance atrix S =[s j,k ] as a atrix product. Let X := [ (x 1 µ 1 ) (x 2 µ 2 ) (x n µ n )] n, so that S = 1 1 X X. Now conduct principal coponent analysis just as before, but with the epirical covariance atrix S replacing the true covariance atrix C. The eigenvectors of S now lead to saple principal coponents. Note that there is no need to explicitly for the atrix S: instead, we can siply perfor the singular value decoposition of the data atrix X. This is why soe say, PCA is just the SVD. We suarize the details step-by-step. 1. Collect saples of each of n rando variables, x j,k for j =1,..., and k =1,...,n.(Weneed>1, and, generally expect n.) 2. Copute the epirical eans, µ k =( x j,k)/. 3. Stacking the saples of the kth variable in the vector x k, construct the ean-centered data atrix X =[(x 1 µ 1 ) (x 2 µ 2 ) (x n µ n )] n.
7 3.4. Principal Coponent Analysis Copute the (skinny) singular value decoposition X = UΣV,with U n, Σ = diag(σ 1,...,σ n ) n n, and V =[v 1 v n ] n n. 5. The kth saple principal coponent is given by v k X,whereX = [X 1,...,X n ] is the vector of rando variables. A word of caution: when conducting principal coponent analysis, the scale of each colun atters. For exaple, if the rando variables sapled in each colun of X are easureents of physical quantities, they can differ considerably in agnitude depending on the units of easureent. By changing units of easureent, you can significantly alter the principal coponents.
Introduction to Machine Learning. Recitation 11
Introduction to Machine Learning Lecturer: Regev Schweiger Recitation Fall Seester Scribe: Regev Schweiger. Kernel Ridge Regression We now take on the task of kernel-izing ridge regression. Let x,...,
More informationFeature Extraction Techniques
Feature Extraction Techniques Unsupervised Learning II Feature Extraction Unsupervised ethods can also be used to find features which can be useful for categorization. There are unsupervised ethods that
More informationPrincipal Components Analysis
Principal Coponents Analysis Cheng Li, Bingyu Wang Noveber 3, 204 What s PCA Principal coponent analysis (PCA) is a statistical procedure that uses an orthogonal transforation to convert a set of observations
More informationBlock designs and statistics
Bloc designs and statistics Notes for Math 447 May 3, 2011 The ain paraeters of a bloc design are nuber of varieties v, bloc size, nuber of blocs b. A design is built on a set of v eleents. Each eleent
More informationPhysics 215 Winter The Density Matrix
Physics 215 Winter 2018 The Density Matrix The quantu space of states is a Hilbert space H. Any state vector ψ H is a pure state. Since any linear cobination of eleents of H are also an eleent of H, it
More informationBootstrapping Dependent Data
Bootstrapping Dependent Data One of the key issues confronting bootstrap resapling approxiations is how to deal with dependent data. Consider a sequence fx t g n t= of dependent rando variables. Clearly
More informationTopic 5a Introduction to Curve Fitting & Linear Regression
/7/08 Course Instructor Dr. Rayond C. Rup Oice: A 337 Phone: (95) 747 6958 E ail: rcrup@utep.edu opic 5a Introduction to Curve Fitting & Linear Regression EE 4386/530 Coputational ethods in EE Outline
More informationLecture 9 November 23, 2015
CSC244: Discrepancy Theory in Coputer Science Fall 25 Aleksandar Nikolov Lecture 9 Noveber 23, 25 Scribe: Nick Spooner Properties of γ 2 Recall that γ 2 (A) is defined for A R n as follows: γ 2 (A) = in{r(u)
More informationA Simple Regression Problem
A Siple Regression Proble R. M. Castro March 23, 2 In this brief note a siple regression proble will be introduced, illustrating clearly the bias-variance tradeoff. Let Y i f(x i ) + W i, i,..., n, where
More informationEstimating Parameters for a Gaussian pdf
Pattern Recognition and achine Learning Jaes L. Crowley ENSIAG 3 IS First Seester 00/0 Lesson 5 7 Noveber 00 Contents Estiating Paraeters for a Gaussian pdf Notation... The Pattern Recognition Proble...3
More informationMachine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
More information13.2 Fully Polynomial Randomized Approximation Scheme for Permanent of Random 0-1 Matrices
CS71 Randoness & Coputation Spring 018 Instructor: Alistair Sinclair Lecture 13: February 7 Disclaier: These notes have not been subjected to the usual scrutiny accorded to foral publications. They ay
More informationSlide10. Haykin Chapter 8: Principal Components Analysis. Motivation. Principal Component Analysis: Variance Probe
Slide10 Motivation Haykin Chapter 8: Principal Coponents Analysis 1.6 1.4 1.2 1 0.8 cloud.dat 0.6 CPSC 636-600 Instructor: Yoonsuck Choe Spring 2015 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 How can we
More informationThe Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Parameters
journal of ultivariate analysis 58, 96106 (1996) article no. 0041 The Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Paraeters H. S. Steyn
More informationOBJECTIVES INTRODUCTION
M7 Chapter 3 Section 1 OBJECTIVES Suarize data using easures of central tendency, such as the ean, edian, ode, and idrange. Describe data using the easures of variation, such as the range, variance, and
More informationMulti-Scale/Multi-Resolution: Wavelet Transform
Multi-Scale/Multi-Resolution: Wavelet Transfor Proble with Fourier Fourier analysis -- breaks down a signal into constituent sinusoids of different frequencies. A serious drawback in transforing to the
More informationA remark on a success rate model for DPA and CPA
A reark on a success rate odel for DPA and CPA A. Wieers, BSI Version 0.5 andreas.wieers@bsi.bund.de Septeber 5, 2018 Abstract The success rate is the ost coon evaluation etric for easuring the perforance
More informationCSE525: Randomized Algorithms and Probabilistic Analysis May 16, Lecture 13
CSE55: Randoied Algoriths and obabilistic Analysis May 6, Lecture Lecturer: Anna Karlin Scribe: Noah Siegel, Jonathan Shi Rando walks and Markov chains This lecture discusses Markov chains, which capture
More informationarxiv: v1 [cs.ds] 3 Feb 2014
arxiv:40.043v [cs.ds] 3 Feb 04 A Bound on the Expected Optiality of Rando Feasible Solutions to Cobinatorial Optiization Probles Evan A. Sultani The Johns Hopins University APL evan@sultani.co http://www.sultani.co/
More informationQuantum algorithms (CO 781, Winter 2008) Prof. Andrew Childs, University of Waterloo LECTURE 15: Unstructured search and spatial search
Quantu algoriths (CO 781, Winter 2008) Prof Andrew Childs, University of Waterloo LECTURE 15: Unstructured search and spatial search ow we begin to discuss applications of quantu walks to search algoriths
More informationIn this chapter, we consider several graph-theoretic and probabilistic models
THREE ONE GRAPH-THEORETIC AND STATISTICAL MODELS 3.1 INTRODUCTION In this chapter, we consider several graph-theoretic and probabilistic odels for a social network, which we do under different assuptions
More informationTesting equality of variances for multiple univariate normal populations
University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Inforation Sciences 0 esting equality of variances for ultiple univariate
More informationPhysics 139B Solutions to Homework Set 3 Fall 2009
Physics 139B Solutions to Hoework Set 3 Fall 009 1. Consider a particle of ass attached to a rigid assless rod of fixed length R whose other end is fixed at the origin. The rod is free to rotate about
More informationCOS 424: Interacting with Data. Written Exercises
COS 424: Interacting with Data Hoework #4 Spring 2007 Regression Due: Wednesday, April 18 Written Exercises See the course website for iportant inforation about collaboration and late policies, as well
More informationMultivariate Methods. Matlab Example. Principal Components Analysis -- PCA
Multivariate Methos Xiaoun Qi Principal Coponents Analysis -- PCA he PCA etho generates a new set of variables, calle principal coponents Each principal coponent is a linear cobination of the original
More informationProc. of the IEEE/OES Seventh Working Conference on Current Measurement Technology UNCERTAINTIES IN SEASONDE CURRENT VELOCITIES
Proc. of the IEEE/OES Seventh Working Conference on Current Measureent Technology UNCERTAINTIES IN SEASONDE CURRENT VELOCITIES Belinda Lipa Codar Ocean Sensors 15 La Sandra Way, Portola Valley, CA 98 blipa@pogo.co
More informationON THE TWO-LEVEL PRECONDITIONING IN LEAST SQUARES METHOD
PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical and Matheatical Sciences 04,, p. 7 5 ON THE TWO-LEVEL PRECONDITIONING IN LEAST SQUARES METHOD M a t h e a t i c s Yu. A. HAKOPIAN, R. Z. HOVHANNISYAN
More informationInteractive Markov Models of Evolutionary Algorithms
Cleveland State University EngagedScholarship@CSU Electrical Engineering & Coputer Science Faculty Publications Electrical Engineering & Coputer Science Departent 2015 Interactive Markov Models of Evolutionary
More information1 Proving the Fundamental Theorem of Statistical Learning
THEORETICAL MACHINE LEARNING COS 5 LECTURE #7 APRIL 5, 6 LECTURER: ELAD HAZAN NAME: FERMI MA ANDDANIEL SUO oving te Fundaental Teore of Statistical Learning In tis section, we prove te following: Teore.
More informationIntelligent Systems: Reasoning and Recognition. Artificial Neural Networks
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley MOSIG M1 Winter Seester 2018 Lesson 7 1 March 2018 Outline Artificial Neural Networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationChapter 6 1-D Continuous Groups
Chapter 6 1-D Continuous Groups Continuous groups consist of group eleents labelled by one or ore continuous variables, say a 1, a 2,, a r, where each variable has a well- defined range. This chapter explores:
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lessons 7 20 Dec 2017 Outline Artificial Neural networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationChapter 6 The Singular Value Decomposition
Matrix Methods for Computational Modeling and Data Analytics Virginia Tech Spring 018 Chapter The Singular Value Decomposition Mark Embree embree@vt.edu Ax=b version of April 018 The singular value decomposition
More informationPolygonal Designs: Existence and Construction
Polygonal Designs: Existence and Construction John Hegean Departent of Matheatics, Stanford University, Stanford, CA 9405 Jeff Langford Departent of Matheatics, Drake University, Des Moines, IA 5011 G
More informationCh 12: Variations on Backpropagation
Ch 2: Variations on Backpropagation The basic backpropagation algorith is too slow for ost practical applications. It ay take days or weeks of coputer tie. We deonstrate why the backpropagation algorith
More informationAlgebraic Montgomery-Yang problem: the log del Pezzo surface case
c 2014 The Matheatical Society of Japan J. Math. Soc. Japan Vol. 66, No. 4 (2014) pp. 1073 1089 doi: 10.2969/jsj/06641073 Algebraic Montgoery-Yang proble: the log del Pezzo surface case By DongSeon Hwang
More informationLinear Algebra (I) Yijia Chen. linear transformations and their algebraic properties. 1. A Starting Point. y := 3x.
Linear Algebra I) Yijia Chen Linear algebra studies Exaple.. Consider the function This is a linear function f : R R. linear transforations and their algebraic properties.. A Starting Point y := 3x. Geoetrically
More informationProbability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
More informationCombining Classifiers
Cobining Classifiers Generic ethods of generating and cobining ultiple classifiers Bagging Boosting References: Duda, Hart & Stork, pg 475-480. Hastie, Tibsharini, Friedan, pg 246-256 and Chapter 10. http://www.boosting.org/
More information1 Bounding the Margin
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #12 Scribe: Jian Min Si March 14, 2013 1 Bounding the Margin We are continuing the proof of a bound on the generalization error of AdaBoost
More informationAnalyzing Simulation Results
Analyzing Siulation Results Dr. John Mellor-Cruey Departent of Coputer Science Rice University johnc@cs.rice.edu COMP 528 Lecture 20 31 March 2005 Topics for Today Model verification Model validation Transient
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 11 10/15/2008 ABSTRACT INTEGRATION I
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 11 10/15/2008 ABSTRACT INTEGRATION I Contents 1. Preliinaries 2. The ain result 3. The Rieann integral 4. The integral of a nonnegative
More informationarxiv: v1 [stat.ot] 7 Jul 2010
Hotelling s test for highly correlated data P. Bubeliny e-ail: bubeliny@karlin.ff.cuni.cz Charles University, Faculty of Matheatics and Physics, KPMS, Sokolovska 83, Prague, Czech Republic, 8675. arxiv:007.094v
More informationThe Weierstrass Approximation Theorem
36 The Weierstrass Approxiation Theore Recall that the fundaental idea underlying the construction of the real nubers is approxiation by the sipler rational nubers. Firstly, nubers are often deterined
More informationSupplementary Material for Fast and Provable Algorithms for Spectrally Sparse Signal Reconstruction via Low-Rank Hankel Matrix Completion
Suppleentary Material for Fast and Provable Algoriths for Spectrally Sparse Signal Reconstruction via Low-Ran Hanel Matrix Copletion Jian-Feng Cai Tianing Wang Ke Wei March 1, 017 Abstract We establish
More informationADVANCES ON THE BESSIS- MOUSSA-VILLANI TRACE CONJECTURE
ADVANCES ON THE BESSIS- MOUSSA-VILLANI TRACE CONJECTURE CHRISTOPHER J. HILLAR Abstract. A long-standing conjecture asserts that the polynoial p(t = Tr(A + tb ] has nonnegative coefficients whenever is
More informationLecture 20 November 7, 2013
CS 229r: Algoriths for Big Data Fall 2013 Prof. Jelani Nelson Lecture 20 Noveber 7, 2013 Scribe: Yun Willia Yu 1 Introduction Today we re going to go through the analysis of atrix copletion. First though,
More informationModel Fitting. CURM Background Material, Fall 2014 Dr. Doreen De Leon
Model Fitting CURM Background Material, Fall 014 Dr. Doreen De Leon 1 Introduction Given a set of data points, we often want to fit a selected odel or type to the data (e.g., we suspect an exponential
More informationGeneral Properties of Radiation Detectors Supplements
Phys. 649: Nuclear Techniques Physics Departent Yarouk University Chapter 4: General Properties of Radiation Detectors Suppleents Dr. Nidal M. Ershaidat Overview Phys. 649: Nuclear Techniques Physics Departent
More informationE0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
More informationCS Lecture 13. More Maximum Likelihood
CS 6347 Lecture 13 More Maxiu Likelihood Recap Last tie: Introduction to axiu likelihood estiation MLE for Bayesian networks Optial CPTs correspond to epirical counts Today: MLE for CRFs 2 Maxiu Likelihood
More informationBoosting with log-loss
Boosting with log-loss Marco Cusuano-Towner Septeber 2, 202 The proble Suppose we have data exaples {x i, y i ) i =... } for a two-class proble with y i {, }. Let F x) be the predictor function with the
More informationThe proofs of Theorem 1-3 are along the lines of Wied and Galeano (2013).
A Appendix: Proofs The proofs of Theore 1-3 are along the lines of Wied and Galeano (2013) Proof of Theore 1 Let D[d 1, d 2 ] be the space of càdlàg functions on the interval [d 1, d 2 ] equipped with
More informationLecture 13 Eigenvalue Problems
Lecture 13 Eigenvalue Probles MIT 18.335J / 6.337J Introduction to Nuerical Methods Per-Olof Persson October 24, 2006 1 The Eigenvalue Decoposition Eigenvalue proble for atrix A: Ax = λx with eigenvalues
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016 Lessons 7 14 Dec 2016 Outline Artificial Neural networks Notation...2 1. Introduction...3... 3 The Artificial
More informationINNER CONSTRAINTS FOR A 3-D SURVEY NETWORK
eospatial Science INNER CONSRAINS FOR A 3-D SURVEY NEWORK hese notes follow closely the developent of inner constraint equations by Dr Willie an, Departent of Building, School of Design and Environent,
More informationMeasures of average are called measures of central tendency and include the mean, median, mode, and midrange.
CHAPTER 3 Data Description Objectives Suarize data using easures of central tendency, such as the ean, edian, ode, and idrange. Describe data using the easures of variation, such as the range, variance,
More informationRisk & Safety in Engineering. Dr. Jochen Köhler
Risk & afety in Engineering Dr. Jochen Köhler Contents of Today's Lecture Introduction to Classical Reliability Theory tructural Reliability The fundaental case afety argin Introduction to Classical Reliability
More informationLecture 21 Nov 18, 2015
CS 388R: Randoized Algoriths Fall 05 Prof. Eric Price Lecture Nov 8, 05 Scribe: Chad Voegele, Arun Sai Overview In the last class, we defined the ters cut sparsifier and spectral sparsifier and introduced
More informationDistributed Subgradient Methods for Multi-agent Optimization
1 Distributed Subgradient Methods for Multi-agent Optiization Angelia Nedić and Asuan Ozdaglar October 29, 2007 Abstract We study a distributed coputation odel for optiizing a su of convex objective functions
More informationTEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES
TEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES S. E. Ahed, R. J. Tokins and A. I. Volodin Departent of Matheatics and Statistics University of Regina Regina,
More informationA new type of lower bound for the largest eigenvalue of a symmetric matrix
Linear Algebra and its Applications 47 7 9 9 www.elsevier.co/locate/laa A new type of lower bound for the largest eigenvalue of a syetric atrix Piet Van Mieghe Delft University of Technology, P.O. Box
More informationSupport recovery in compressed sensing: An estimation theoretic approach
Support recovery in copressed sensing: An estiation theoretic approach Ain Karbasi, Ali Horati, Soheil Mohajer, Martin Vetterli School of Coputer and Counication Sciences École Polytechnique Fédérale de
More informationESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS. A Thesis. Presented to. The Faculty of the Department of Mathematics
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS A Thesis Presented to The Faculty of the Departent of Matheatics San Jose State University In Partial Fulfillent of the Requireents
More informationEnsemble Based on Data Envelopment Analysis
Enseble Based on Data Envelopent Analysis So Young Sohn & Hong Choi Departent of Coputer Science & Industrial Systes Engineering, Yonsei University, Seoul, Korea Tel) 82-2-223-404, Fax) 82-2- 364-7807
More informationOptimal Jamming Over Additive Noise: Vector Source-Channel Case
Fifty-first Annual Allerton Conference Allerton House, UIUC, Illinois, USA October 2-3, 2013 Optial Jaing Over Additive Noise: Vector Source-Channel Case Erah Akyol and Kenneth Rose Abstract This paper
More informationMeta-Analytic Interval Estimation for Bivariate Correlations
Psychological Methods 2008, Vol. 13, No. 3, 173 181 Copyright 2008 by the Aerican Psychological Association 1082-989X/08/$12.00 DOI: 10.1037/a0012868 Meta-Analytic Interval Estiation for Bivariate Correlations
More informationAn Introduction to Meta-Analysis
An Introduction to Meta-Analysis Douglas G. Bonett University of California, Santa Cruz How to cite this work: Bonett, D.G. (2016) An Introduction to Meta-analysis. Retrieved fro http://people.ucsc.edu/~dgbonett/eta.htl
More informatione-companion ONLY AVAILABLE IN ELECTRONIC FORM
OPERATIONS RESEARCH doi 10.1287/opre.1070.0427ec pp. ec1 ec5 e-copanion ONLY AVAILABLE IN ELECTRONIC FORM infors 07 INFORMS Electronic Copanion A Learning Approach for Interactive Marketing to a Custoer
More informationLower Bounds for Quantized Matrix Completion
Lower Bounds for Quantized Matrix Copletion Mary Wootters and Yaniv Plan Departent of Matheatics University of Michigan Ann Arbor, MI Eail: wootters, yplan}@uich.edu Mark A. Davenport School of Elec. &
More informationlecture 37: Linear Multistep Methods: Absolute Stability, Part I lecture 38: Linear Multistep Methods: Absolute Stability, Part II
lecture 37: Linear Multistep Methods: Absolute Stability, Part I lecture 3: Linear Multistep Methods: Absolute Stability, Part II 5.7 Linear ultistep ethods: absolute stability At this point, it ay well
More informationReed-Muller Codes. m r inductive definition. Later, we shall explain how to construct Reed-Muller codes using the Kronecker product.
Coding Theory Massoud Malek Reed-Muller Codes An iportant class of linear block codes rich in algebraic and geoetric structure is the class of Reed-Muller codes, which includes the Extended Haing code.
More informationpaper prepared for the 1996 PTRC Conference, September 2-6, Brunel University, UK ON THE CALIBRATION OF THE GRAVITY MODEL
paper prepared for the 1996 PTRC Conference, Septeber 2-6, Brunel University, UK ON THE CALIBRATION OF THE GRAVITY MODEL Nanne J. van der Zijpp 1 Transportation and Traffic Engineering Section Delft University
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee227c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee227c@berkeley.edu October
More informationBest Procedures For Sample-Free Item Analysis
Best Procedures For Saple-Free Ite Analysis Benjain D. Wright University of Chicago Graha A. Douglas University of Western Australia Wright s (1969) widely used "unconditional" procedure for Rasch saple-free
More informationAnisotropic reference media and the possible linearized approximations for phase velocities of qs waves in weakly anisotropic media
INSTITUTE OF PHYSICS PUBLISHING JOURNAL OF PHYSICS D: APPLIED PHYSICS J. Phys. D: Appl. Phys. 5 00 007 04 PII: S00-770867-6 Anisotropic reference edia and the possible linearized approxiations for phase
More informationKeywords: Estimator, Bias, Mean-squared error, normality, generalized Pareto distribution
Testing approxiate norality of an estiator using the estiated MSE and bias with an application to the shape paraeter of the generalized Pareto distribution J. Martin van Zyl Abstract In this work the norality
More informationKinetic Theory of Gases: Elementary Ideas
Kinetic Theory of Gases: Eleentary Ideas 17th February 2010 1 Kinetic Theory: A Discussion Based on a Siplified iew of the Motion of Gases 1.1 Pressure: Consul Engel and Reid Ch. 33.1) for a discussion
More informationUsing EM To Estimate A Probablity Density With A Mixture Of Gaussians
Using EM To Estiate A Probablity Density With A Mixture Of Gaussians Aaron A. D Souza adsouza@usc.edu Introduction The proble we are trying to address in this note is siple. Given a set of data points
More informationConsistent Multiclass Algorithms for Complex Performance Measures. Supplementary Material
Consistent Multiclass Algoriths for Coplex Perforance Measures Suppleentary Material Notations. Let λ be the base easure over n given by the unifor rando variable (say U over n. Hence, for all easurable
More informationNon-Parametric Non-Line-of-Sight Identification 1
Non-Paraetric Non-Line-of-Sight Identification Sinan Gezici, Hisashi Kobayashi and H. Vincent Poor Departent of Electrical Engineering School of Engineering and Applied Science Princeton University, Princeton,
More informationFourier Series Summary (From Salivahanan et al, 2002)
Fourier Series Suary (Fro Salivahanan et al, ) A periodic continuous signal f(t), - < t
More informationBiostatistics Department Technical Report
Biostatistics Departent Technical Report BST006-00 Estiation of Prevalence by Pool Screening With Equal Sized Pools and a egative Binoial Sapling Model Charles R. Katholi, Ph.D. Eeritus Professor Departent
More informationare equal to zero, where, q = p 1. For each gene j, the pairwise null and alternative hypotheses are,
Page of 8 Suppleentary Materials: A ultiple testing procedure for ulti-diensional pairwise coparisons with application to gene expression studies Anjana Grandhi, Wenge Guo, Shyaal D. Peddada S Notations
More informationA1. Find all ordered pairs (a, b) of positive integers for which 1 a + 1 b = 3
A. Find all ordered pairs a, b) of positive integers for which a + b = 3 08. Answer. The six ordered pairs are 009, 08), 08, 009), 009 337, 674) = 35043, 674), 009 346, 673) = 3584, 673), 674, 009 337)
More informationarxiv: v1 [stat.ml] 31 Jan 2018
Increental kernel PCA and the Nyströ ethod arxiv:802.00043v [stat.ml] 3 Jan 208 Fredrik Hallgren Departent of Statistical Science University College London London WCE 6BT, United Kingdo fredrik.hallgren@ucl.ac.uk
More informationNORMAL MATRIX POLYNOMIALS WITH NONSINGULAR LEADING COEFFICIENTS
NORMAL MATRIX POLYNOMIALS WITH NONSINGULAR LEADING COEFFICIENTS NIKOLAOS PAPATHANASIOU AND PANAYIOTIS PSARRAKOS Abstract. In this paper, we introduce the notions of weakly noral and noral atrix polynoials,
More informationUsing a De-Convolution Window for Operating Modal Analysis
Using a De-Convolution Window for Operating Modal Analysis Brian Schwarz Vibrant Technology, Inc. Scotts Valley, CA Mark Richardson Vibrant Technology, Inc. Scotts Valley, CA Abstract Operating Modal Analysis
More informationLeast Squares Fitting of Data
Least Squares Fitting of Data David Eberly, Geoetric Tools, Redond WA 98052 https://www.geoetrictools.co/ This work is licensed under the Creative Coons Attribution 4.0 International License. To view a
More informationKinetic Theory of Gases: Elementary Ideas
Kinetic Theory of Gases: Eleentary Ideas 9th February 011 1 Kinetic Theory: A Discussion Based on a Siplified iew of the Motion of Gases 1.1 Pressure: Consul Engel and Reid Ch. 33.1) for a discussion of
More informationSupplementary Information for Design of Bending Multi-Layer Electroactive Polymer Actuators
Suppleentary Inforation for Design of Bending Multi-Layer Electroactive Polyer Actuators Bavani Balakrisnan, Alek Nacev, and Elisabeth Sela University of Maryland, College Park, Maryland 074 1 Analytical
More informationMath 1600A Lecture 3, Section 002
Math 1600 Lecture 3 1 of 5 Math 1600A Lecture 3, Section 002 Announceents: More texts, solutions anuals and packages coing soon. Read Section 1.3 for next class. Work through recoended hoework questions.
More informationASSUME a source over an alphabet size m, from which a sequence of n independent samples are drawn. The classical
IEEE TRANSACTIONS ON INFORMATION THEORY Large Alphabet Source Coding using Independent Coponent Analysis Aichai Painsky, Meber, IEEE, Saharon Rosset and Meir Feder, Fellow, IEEE arxiv:67.7v [cs.it] Jul
More informationA Theoretical Analysis of a Warm Start Technique
A Theoretical Analysis of a War Start Technique Martin A. Zinkevich Yahoo! Labs 701 First Avenue Sunnyvale, CA Abstract Batch gradient descent looks at every data point for every step, which is wasteful
More information12 Towards hydrodynamic equations J Nonlinear Dynamics II: Continuum Systems Lecture 12 Spring 2015
18.354J Nonlinear Dynaics II: Continuu Systes Lecture 12 Spring 2015 12 Towards hydrodynaic equations The previous classes focussed on the continuu description of static (tie-independent) elastic systes.
More information2 Q 10. Likewise, in case of multiple particles, the corresponding density in 2 must be averaged over all
Lecture 6 Introduction to kinetic theory of plasa waves Introduction to kinetic theory So far we have been odeling plasa dynaics using fluid equations. The assuption has been that the pressure can be either
More informationA note on the multiplication of sparse matrices
Cent. Eur. J. Cop. Sci. 41) 2014 1-11 DOI: 10.2478/s13537-014-0201-x Central European Journal of Coputer Science A note on the ultiplication of sparse atrices Research Article Keivan Borna 12, Sohrab Aboozarkhani
More informationNOTES AND CORRESPONDENCE. Two Extra Components in the Brier Score Decomposition
752 W E A T H E R A N D F O R E C A S T I N G VOLUME 23 NOTES AND CORRESPONDENCE Two Extra Coponents in the Brier Score Decoposition D. B. STEPHENSON School of Engineering, Coputing, and Matheatics, University
More informationSupplementary to Learning Discriminative Bayesian Networks from High-dimensional Continuous Neuroimaging Data
Suppleentary to Learning Discriinative Bayesian Networks fro High-diensional Continuous Neuroiaging Data Luping Zhou, Lei Wang, Lingqiao Liu, Philip Ogunbona, and Dinggang Shen Proposition. Given a sparse
More informationStayability Evaluation as a Categorical Trait and by Considering Other Traits
University of Nebrasa - Lincoln DigitalCoons@University of Nebrasa - Lincoln Faculty Papers and Publications in Anial Science Anial Science Departent January 1980 Stayability Evaluation as a Categorical
More information