Slide10. Haykin Chapter 8: Principal Components Analysis. Motivation. Principal Component Analysis: Variance Probe
|
|
- Audrey Barber
- 5 years ago
- Views:
Transcription
1 Slide10 Motivation Haykin Chapter 8: Principal Coponents Analysis cloud.dat 0.6 CPSC Instructor: Yoonsuck Choe Spring How can we project the given data so that the variance in the projected points is axiized? 1 2 Principal Coponent Analysis: Variance Probe X: -diensional rando vector (vector rando variable following a certain probability distribution). Assue EX 0. Projection of a unit vector q ((qq T ) 1/2 1) onto X: A X T q q T X. We know EA Eq T X q T EX 0. The variance can also be calculated: σ 2 EA 2 E(q T X)(X T q) q T EXX T covariance atrix q q T Rq. 3 Principal Coponent Analysis: Variance Probe (cont d) This is sort of a variance probe: ψ(q) q T Rq. Using different unit vectors q for the projection of the input data points will result in saller or larger variance in the projected points. With this, we can ask which vector direction does the variance probe ψ(q) has exteral value? The solution to the question is obtained by finding unit vectors satisfying the following condition: Rq λq, where λ is a scaling factor. This is basically an eigenvalue proble. 4
2 PCA With an covariance atrix R, we can get eigenvectors and eigenvalues: Rq j λ j q j, j 1, 2,..., We can sort the eigenvectors/eigenvalues according to the eigenvalues, so that λ 1 > λ 2 >... > λ. PCA: Suary The eigenvectors of the covariance atrix R of zero-ean rando input vector X define the principal directions q j along with the variance of the projected inputs have extreal values. The associated eigenvaluess define the extreal values of the variance probe. and arrange the eigenvectors in a colun-wise atrix Q q 1, q 2,..., q. Then we can write RQ Qλ where λ diag(λ 1, λ 2,..., λ ). Q is orthogonal, so that QQ T I. That is, Q 1 Q T. 5 6 PCA: Usage Project input x to the principal directions: a Q T x. We can also recover the input fro the projected point a: x (Q T ) 1 a Qa. Note that we don t need all principal directions, depending on how uch variance is captured in the first few eigenvalues: We can do diensionality reduction. PCA: Diensionality Reduction Encoding: We can use the first l eigenvectors to encode x. a 1, a 2,..., a l T q 1, q 2,..., q l T x. Note that we only need to calculate l projections a 1, a 2,..., a l, where l. Decoding: Once a 1, a 2,..., a l T is obtained, we want to reconstruct the full x 1, x 2,..., x l,..., x T. x Qa q 1, q 2,..., q l a 1, a 2,..., a l T ˆx. Or, alternatively ˆx Qa 1, a 2,..., a l, 0, 0,..., 0 T. l zeros 7 8
3 PCA: Total Variance The total variance of th e coponents of the data vector is σ 2 j j1 j1 λ j. The truncated version with the first l coponents have variance l σ 2 j j1 j1 The larger the variance in the truncated version, i.e., the saller the variance in the reaining coponents, the ore accurate the diensionality reduction. 9 l λ j PCA Exaple line 1 line 2 line 3 line inprandn(800,2)/9+0.5;randn(1000,2)/6+ones(1000,2); Q λ PCA s Relation to Neural Networks: Hebbian-Based Maxiu Eigenfilter How does all the above relate to neural networks? A rearkable result by Oja (1982) shows that a single linear neuron with Hebbian synapse can evolve into a filter for the first principal coponent of the input distribution! Activation: y i1 Learning rule: w i (n + 1) ( w i x i w i (n) + ηy(n)x i (n) i1 w i(n) + ηy(n)x i (n) 2 ) 1/2 Hebbian-Based Maxiu Eigenfilter Expanding the denoinator as a power series, dropping the higher order ters, etc., we get w i (n+1) w i (n)+ηy(n)x i (n) y(n)w i (n)+o(η 2 ), with O(η 2 ) including the second- and higher-order effects of η, which we can ignore for sall η. Based on that, we get w i (n + 1) w i (n) + ηy(n)x i (n) y(n)w i (n) w i (n) + η y(n)xi (n) y(n) 2 w i (n) Hebbian ter Stabilization ter
4 Matrix Forulation of the Algorith Asyptotic Stability Theore Activation y(n) x T (n)w(n) w T (n)x(n) To ease the analysis, we rewrite the learning rule as w(n + 1) w(n) + η(n)h(w(n), x(n)). Learning w(n + 1) w(n) + ηy(n)x(n) y(n)w(n) Cobining the above, w(n + 1) w(n) + ηx(n)x T (n)w(n) w T (n)x(n)x T (n)w(n)w(n) represents a nonlinear stochastic difference equation, which is hard to analyze., The goal is to associate a deterinistic ordinary differential equation (ODE) with the stochastic equation. Under certain reasonable conditions on η, h(, ), and w, we get the asyptotic stability theore stating that infinitely often with probability 1. li w(n) q 1 n Conditions for Stability Stability Analysis of Maxiu Eigenfilter 1. η(n) is a decreasing sequence of positive real nubers such that η(n) n1, n1 ηp (n) < for p > 1, η(n) 0 as n. 2. Sequence of paraeter vectors w( ) is bounded with probability The update function h(w, x) is continuously differentiable w.r.t. w and x, and it derivatives are bounded in tie. 4. The liit h(w) li n Eh(w, X) exists for each w, where X is a rando vector. 5. There is a locally asyptotically stable solution to the ODE d w(t) ĥ(w(t)). 6. Let q 1 denote the solution to the ODE above with a basin of attraction B(q). The paraeter vector w(n) enters the copact subset A of B(q) infinitely often with prob Set it up to satisfy the conditions of the asyptotic stability theore: Set the learning rate to be η(n) 1/n. Set h(, ) to h(w, x) Taking expectaion over all x, x(n)y(n) y 2 w(n) x(n)x T (n)w(n) w T (n)x(n)x T (n)w(n)w(n) h lin EX(n)X T (n)w(n) (w T (n)x(n)x T (n)w(n))w(n) Rw( ) w T ( )Rw( ) w( ) Substituting h into the ODE, d w(t) h(w(t)) Rw(t) w T (t)rw(t)w(t). 16
5 Stability Analysis of Maxiu Eigenfilter Expanding w(t) with the eigenvectors of R, w(t) and using basic definitions we get (see next slide for derivation) q k θ k (t)q k, Rq k λ k q, q T k Rq k λ k λ k θ k (t)q k λ l θ 2 l (t) l1 θ k (t)q k. Equating the RHS s of the following dw(t) d ( θ k (t)q k ) d w(t) h(w(t)) Rw(t) w T (t)rw(t)w(t). we get q k λ k θ k (t)q k λ l θ 2 l (t) l1, θ k (t)q k First, we show Rw(t) λ kθ k (t)q k, using Rq k λ k q. Rw(t) R θ k (t)q k θ k (t)rq k λ k θ k (t)q k 19 Next, we show w T (t)rw(t)w(t) l1 λ l θ2 l (t) θ k (t)q k. w T (t)rw(t)w(t) w (t)rw(t) T θ k (t)q k ( ) l1 θ l R( (t)qt l θ k (t)q k ( l1 l1 l1 ) ( θ l (t)q T l R θ k ( k)) (t)q ) θ l (t)qt l Rθ k (t)q k ( ) θ l (t)θ k (t)qt l Rq k ( ( θ k (t)q k θ k (t)q k θ k (t)q k θ k (t)q k ) l1 θ l (t)θ k (t)qt l (λ k q k ) θ k (t)q k ) l1 θ l (t)θ k (t)λ k qt l q k ) θ k (t)q k { Inner su disappears since q T l q k 0 for l k and 1 for l k} l1 θ l (t)θ l (t)λ l θ k (t)q k l1 θ2 l (t)λ l θ k (t)q k 20
6 Factoring out q k, we get λ k θ k (t) l1 λ l θ 2 l (t) θ k (t). We can analyze the above in two cases (details in following slides): Case I: k 1 In this case, α k (t) θ k(t) Case II: k 1 θ 1 (t) above to derive dα k(t) 0 as t, by using In this case, θ 1 (t) ±1 as t, fro dθ 1 (t) λ 1 θ 1 (t) 1 θ 2 1 (t). 21 (λ 1 λ k )α k (t). positive! Case I (in detail): k 1 Given λ k θ k (t) Define α k (t) θ k (t) θ 1 (t). Derive dα k (t) l1 1 θ 1 (t) λ l θ 2 l (t) θ k (t).(1) θ k(t) dθ 1 (t) θ1 2(t) (2) Plug in (1) above into (2). (Both / and dθ 1 (k)/.) Finally, we get: dα k (t) t. (λ 1 λ k )α k (t), so α k (t) 0 as 22 Case II: k 1 dθ 1 (t) λ 1 θ 1 (t) l1 λ l θ 2 l (t) θ k (t) λ 1 θ 1 (t) λ 1 θ 3 1 (t) θ 1(t) l2 λ l θ 2 l (t) λ 1 θ 1 (t) λ 1 θ 3 1 (t) θ3 1 (t) λ l α 2 l (t) l2 Using results fro Case I (α l 0 for l 1 and t ), θ 1 (t) ±1 as t, fro dθ 1 (t) λ 1 θ 1 (t) 1 θ 2 1 (t). 23 Recalling the original expansion w(t) we can conclude that θ k (t)q k, w(t) q 1, as t. where q 1 is the noralized eigenvector associated with the largest eigenvalue λ 1 of the covariance atrix R. Other conditions of stability can also be shown to hold (see the textbook). 24
7 Suary of Hebbian-Based Maxiu Eigenfilter Hebbian-based linear neuron converges with probability 1 to a fixed point, which is characterized as follows: Generalized Hebbian Algorith for full PCA Sanger (1989) showed how to construct a feedfoward network to learn all the eigenvectors of R. Variance of output approaches the largest eigenvalue of the covariance atrix R (y(n) is the output): li n σ2 (n) li EY 2 (n) λ 1 n Activation y j (n) i1 w ji (n)x i (n), j 1, 2,..., l Synaptic weight vector approaches the associated eigenvector with li n w(n) q 1 li w(n) 1. n Learning w ji (n) η y j (n)x i (n) y j (n) j i 1, 2,...,, j 1, 2,..., l. w ki (n)y k (n), 25 26
3.3 Variational Characterization of Singular Values
3.3. Variational Characterization of Singular Values 61 3.3 Variational Characterization of Singular Values Since the singular values are square roots of the eigenvalues of the Heritian atrices A A and
More informationFeature Extraction Techniques
Feature Extraction Techniques Unsupervised Learning II Feature Extraction Unsupervised ethods can also be used to find features which can be useful for categorization. There are unsupervised ethods that
More informationIntroduction to Machine Learning. Recitation 11
Introduction to Machine Learning Lecturer: Regev Schweiger Recitation Fall Seester Scribe: Regev Schweiger. Kernel Ridge Regression We now take on the task of kernel-izing ridge regression. Let x,...,
More informationMechanics Physics 151
Mechanics Physics 5 Lecture Oscillations (Chapter 6) What We Did Last Tie Analyzed the otion of a heavy top Reduced into -diensional proble of θ Qualitative behavior Precession + nutation Initial condition
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016/2017 Lessons 9 11 Jan 2017 Outline Artificial Neural networks Notation...2 Convolutional Neural Networks...3
More informationPrincipal Components Analysis
Principal Coponents Analysis Cheng Li, Bingyu Wang Noveber 3, 204 What s PCA Principal coponent analysis (PCA) is a statistical procedure that uses an orthogonal transforation to convert a set of observations
More informationBlock designs and statistics
Bloc designs and statistics Notes for Math 447 May 3, 2011 The ain paraeters of a bloc design are nuber of varieties v, bloc size, nuber of blocs b. A design is built on a set of v eleents. Each eleent
More informationMachine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
More informationSynaptic Plasticity. Introduction. Biophysics of Synaptic Plasticity. Functional Modes of Synaptic Plasticity. Activity-dependent synaptic plasticity:
Synaptic Plasticity Introduction Dayan and Abbott (2001) Chapter 8 Instructor: Yoonsuck Choe; CPSC 644 Cortical Networks Activity-dependent synaptic plasticity: underlies learning and memory, and plays
More informationEstimating Parameters for a Gaussian pdf
Pattern Recognition and achine Learning Jaes L. Crowley ENSIAG 3 IS First Seester 00/0 Lesson 5 7 Noveber 00 Contents Estiating Paraeters for a Gaussian pdf Notation... The Pattern Recognition Proble...3
More informationMulti-Scale/Multi-Resolution: Wavelet Transform
Multi-Scale/Multi-Resolution: Wavelet Transfor Proble with Fourier Fourier analysis -- breaks down a signal into constituent sinusoids of different frequencies. A serious drawback in transforing to the
More information1 Bounding the Margin
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #12 Scribe: Jian Min Si March 14, 2013 1 Bounding the Margin We are continuing the proof of a bound on the generalization error of AdaBoost
More informationThe Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Parameters
journal of ultivariate analysis 58, 96106 (1996) article no. 0041 The Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Paraeters H. S. Steyn
More information13.2 Fully Polynomial Randomized Approximation Scheme for Permanent of Random 0-1 Matrices
CS71 Randoness & Coputation Spring 018 Instructor: Alistair Sinclair Lecture 13: February 7 Disclaier: These notes have not been subjected to the usual scrutiny accorded to foral publications. They ay
More informationCovariance and Correlation Matrix
Covariance and Correlation Matrix Given sample {x n } N 1, where x Rd, x n = x 1n x 2n. x dn sample mean x = 1 N N n=1 x n, and entries of sample mean are x i = 1 N N n=1 x in sample covariance matrix
More informationUnsupervised Learning: Dimension Reduction
Unsupervised Learning: Diension Reduction by Prof. Seungchul Lee isystes Design Lab http://isystes.unist.ac.kr/ UNIST Table of Contents I.. Principal Coponent Analysis (PCA) II. 2. PCA Algorith I. 2..
More information1 Principal Components Analysis
Lecture 3 and 4 Sept. 18 and Sept.20-2006 Data Visualization STAT 442 / 890, CM 462 Lecture: Ali Ghodsi 1 Principal Components Analysis Principal components analysis (PCA) is a very popular technique for
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE7C (Spring 018: Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee7c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee7c@berkeley.edu October 15,
More informationThe Hebb rule Neurons that fire together wire together.
Unsupervised learning The Hebb rule Neurons that fire together wire together. PCA RF development with PCA Classical Conditioning and Hebbʼs rule Ear A Nose B Tongue When an axon in cell A is near enough
More informationUsing EM To Estimate A Probablity Density With A Mixture Of Gaussians
Using EM To Estiate A Probablity Density With A Mixture Of Gaussians Aaron A. D Souza adsouza@usc.edu Introduction The proble we are trying to address in this note is siple. Given a set of data points
More informationOrder Recursion Introduction Order versus Time Updates Matrix Inversion by Partitioning Lemma Levinson Algorithm Interpretations Examples
Order Recursion Introduction Order versus Tie Updates Matrix Inversion by Partitioning Lea Levinson Algorith Interpretations Exaples Introduction Rc d There are any ways to solve the noral equations Solutions
More informationThis model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.
CS 493: Algoriths for Massive Data Sets Feb 2, 2002 Local Models, Bloo Filter Scribe: Qin Lv Local Models In global odels, every inverted file entry is copressed with the sae odel. This work wells when
More informationLean Walsh Transform
Lean Walsh Transfor Edo Liberty 5th March 007 inforal intro We show an orthogonal atrix A of size d log 4 3 d (α = log 4 3) which is applicable in tie O(d). By applying a rando sign change atrix S to the
More informationma x = -bv x + F rod.
Notes on Dynaical Systes Dynaics is the study of change. The priary ingredients of a dynaical syste are its state and its rule of change (also soeties called the dynaic). Dynaical systes can be continuous
More informationRandom Process Review
Rando Process Review Consider a rando process t, and take k saples. For siplicity, we will set k. However it should ean any nuber of saples. t () t x t, t, t We have a rando vector t, t, t. If we find
More informationSupplementary to Learning Discriminative Bayesian Networks from High-dimensional Continuous Neuroimaging Data
Suppleentary to Learning Discriinative Bayesian Networks fro High-diensional Continuous Neuroiaging Data Luping Zhou, Lei Wang, Lingqiao Liu, Philip Ogunbona, and Dinggang Shen Proposition. Given a sparse
More informationCh 12: Variations on Backpropagation
Ch 2: Variations on Backpropagation The basic backpropagation algorith is too slow for ost practical applications. It ay take days or weeks of coputer tie. We deonstrate why the backpropagation algorith
More informationSupport recovery in compressed sensing: An estimation theoretic approach
Support recovery in copressed sensing: An estiation theoretic approach Ain Karbasi, Ali Horati, Soheil Mohajer, Martin Vetterli School of Coputer and Counication Sciences École Polytechnique Fédérale de
More informationIntroduction to Optimization Techniques. Nonlinear Programming
Introduction to Optiization echniques Nonlinear Prograing Optial Solutions Consider the optiization proble in f ( x) where F R n xf Definition : x F is optial (global iniu) for this proble, if f( x ) f(
More informationSupporting Information for Supression of Auger Processes in Confined Structures
Supporting Inforation for Supression of Auger Processes in Confined Structures George E. Cragg and Alexander. Efros Naval Research aboratory, Washington, DC 20375, USA 1 Solution of the Coupled, Two-band
More informationCS Lecture 13. More Maximum Likelihood
CS 6347 Lecture 13 More Maxiu Likelihood Recap Last tie: Introduction to axiu likelihood estiation MLE for Bayesian networks Optial CPTs correspond to epirical counts Today: MLE for CRFs 2 Maxiu Likelihood
More informationProblem Set 2 Due Sept, 21
EE6: Rando Processes in Sstes Lecturer: Jean C. Walrand Proble Set Due Sept, Fall 6 GSI: Assane Guee This proble set essentiall reviews notions of conditional epectation, conditional distribution, and
More informationCOS 424: Interacting with Data. Written Exercises
COS 424: Interacting with Data Hoework #4 Spring 2007 Regression Due: Wednesday, April 18 Written Exercises See the course website for iportant inforation about collaboration and late policies, as well
More informationIntelligent Systems: Reasoning and Recognition. Artificial Neural Networks
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley MOSIG M1 Winter Seester 2018 Lesson 7 1 March 2018 Outline Artificial Neural Networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationProbability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
More informationCSE525: Randomized Algorithms and Probabilistic Analysis May 16, Lecture 13
CSE55: Randoied Algoriths and obabilistic Analysis May 6, Lecture Lecturer: Anna Karlin Scribe: Noah Siegel, Jonathan Shi Rando walks and Markov chains This lecture discusses Markov chains, which capture
More informationA remark on a success rate model for DPA and CPA
A reark on a success rate odel for DPA and CPA A. Wieers, BSI Version 0.5 andreas.wieers@bsi.bund.de Septeber 5, 2018 Abstract The success rate is the ost coon evaluation etric for easuring the perforance
More informationA method to determine relative stroke detection efficiencies from multiplicity distributions
A ethod to deterine relative stroke detection eiciencies ro ultiplicity distributions Schulz W. and Cuins K. 2. Austrian Lightning Detection and Inoration Syste (ALDIS), Kahlenberger Str.2A, 90 Vienna,
More informationReed-Muller Codes. m r inductive definition. Later, we shall explain how to construct Reed-Muller codes using the Kronecker product.
Coding Theory Massoud Malek Reed-Muller Codes An iportant class of linear block codes rich in algebraic and geoetric structure is the class of Reed-Muller codes, which includes the Extended Haing code.
More informationA Note on the Applied Use of MDL Approximations
A Note on the Applied Use of MDL Approxiations Daniel J. Navarro Departent of Psychology Ohio State University Abstract An applied proble is discussed in which two nested psychological odels of retention
More informationPhysics 215 Winter The Density Matrix
Physics 215 Winter 2018 The Density Matrix The quantu space of states is a Hilbert space H. Any state vector ψ H is a pure state. Since any linear cobination of eleents of H are also an eleent of H, it
More informationStatistics and Probability Letters
Statistics and Probability Letters 79 2009 223 233 Contents lists available at ScienceDirect Statistics and Probability Letters journal hoepage: www.elsevier.co/locate/stapro A CLT for a one-diensional
More informationLecture 9 November 23, 2015
CSC244: Discrepancy Theory in Coputer Science Fall 25 Aleksandar Nikolov Lecture 9 Noveber 23, 25 Scribe: Nick Spooner Properties of γ 2 Recall that γ 2 (A) is defined for A R n as follows: γ 2 (A) = in{r(u)
More informationE0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lessons 7 20 Dec 2017 Outline Artificial Neural networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationSlide05 Haykin Chapter 5: Radial-Basis Function Networks
Slide5 Haykin Chapter 5: Radial-Basis Function Networks CPSC 636-6 Instructor: Yoonsuck Choe Spring Learning in MLP Supervised learning in multilayer perceptrons: Recursive technique of stochastic approximation,
More informationSupport Vector Machines. Machine Learning Series Jerry Jeychandra Blohm Lab
Support Vector Machines Machine Learning Series Jerry Jeychandra Bloh Lab Outline Main goal: To understand how support vector achines (SVMs) perfor optial classification for labelled data sets, also a
More informationPhysics 139B Solutions to Homework Set 3 Fall 2009
Physics 139B Solutions to Hoework Set 3 Fall 009 1. Consider a particle of ass attached to a rigid assless rod of fixed length R whose other end is fixed at the origin. The rod is free to rotate about
More informationApproximation in Stochastic Scheduling: The Power of LP-Based Priority Policies
Approxiation in Stochastic Scheduling: The Power of -Based Priority Policies Rolf Möhring, Andreas Schulz, Marc Uetz Setting (A P p stoch, r E( w and (B P p stoch E( w We will assue that the processing
More informationTopic 5a Introduction to Curve Fitting & Linear Regression
/7/08 Course Instructor Dr. Rayond C. Rup Oice: A 337 Phone: (95) 747 6958 E ail: rcrup@utep.edu opic 5a Introduction to Curve Fitting & Linear Regression EE 4386/530 Coputational ethods in EE Outline
More informationFeedforward Networks
Feedforward Networks Gradient Descent Learning and Backpropagation Christian Jacob CPSC 433 Christian Jacob Dept.of Coputer Science,University of Calgary CPSC 433 - Feedforward Networks 2 Adaptive "Prograing"
More informationThe proofs of Theorem 1-3 are along the lines of Wied and Galeano (2013).
A Appendix: Proofs The proofs of Theore 1-3 are along the lines of Wied and Galeano (2013) Proof of Theore 1 Let D[d 1, d 2 ] be the space of càdlàg functions on the interval [d 1, d 2 ] equipped with
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee227c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee227c@berkeley.edu October
More informationChapter 6 1-D Continuous Groups
Chapter 6 1-D Continuous Groups Continuous groups consist of group eleents labelled by one or ore continuous variables, say a 1, a 2,, a r, where each variable has a well- defined range. This chapter explores:
More informationChaotic Coupled Map Lattices
Chaotic Coupled Map Lattices Author: Dustin Keys Advisors: Dr. Robert Indik, Dr. Kevin Lin 1 Introduction When a syste of chaotic aps is coupled in a way that allows the to share inforation about each
More informationSolutions of some selected problems of Homework 4
Solutions of soe selected probles of Hoework 4 Sangchul Lee May 7, 2018 Proble 1 Let there be light A professor has two light bulbs in his garage. When both are burned out, they are replaced, and the next
More informationEE5900 Spring Lecture 4 IC interconnect modeling methods Zhuo Feng
EE59 Spring Parallel LSI AD Algoriths Lecture I interconnect odeling ethods Zhuo Feng. Z. Feng MTU EE59 So far we ve considered only tie doain analyses We ll soon see that it is soeties preferable to odel
More informationInteractive Markov Models of Evolutionary Algorithms
Cleveland State University EngagedScholarship@CSU Electrical Engineering & Coputer Science Faculty Publications Electrical Engineering & Coputer Science Departent 2015 Interactive Markov Models of Evolutionary
More informationPoornima University, For any query, contact us at: , 18
AIEEE//Math S. No Questions Solutions Q. Lets cos (α + β) = and let sin (α + β) = 5, where α, β π, then tan α = 5 (a) 56 (b) 9 (c) 7 (d) 5 6 Sol: (a) cos (α + β) = 5 tan (α + β) = tan α = than (α + β +
More informationExtension of CSRSM for the Parametric Study of the Face Stability of Pressurized Tunnels
Extension of CSRSM for the Paraetric Study of the Face Stability of Pressurized Tunnels Guilhe Mollon 1, Daniel Dias 2, and Abdul-Haid Soubra 3, M.ASCE 1 LGCIE, INSA Lyon, Université de Lyon, Doaine scientifique
More informationList Scheduling and LPT Oliver Braun (09/05/2017)
List Scheduling and LPT Oliver Braun (09/05/207) We investigate the classical scheduling proble P ax where a set of n independent jobs has to be processed on 2 parallel and identical processors (achines)
More informationIntroduction to Discrete Optimization
Prof. Friedrich Eisenbrand Martin Nieeier Due Date: March 9 9 Discussions: March 9 Introduction to Discrete Optiization Spring 9 s Exercise Consider a school district with I neighborhoods J schools and
More informationFeedforward Networks. Gradient Descent Learning and Backpropagation. Christian Jacob. CPSC 533 Winter 2004
Feedforward Networks Gradient Descent Learning and Backpropagation Christian Jacob CPSC 533 Winter 2004 Christian Jacob Dept.of Coputer Science,University of Calgary 2 05-2-Backprop-print.nb Adaptive "Prograing"
More informationDonald Fussell. October 28, Computer Science Department The University of Texas at Austin. Point Masses and Force Fields.
s Vector Moving s and Coputer Science Departent The University of Texas at Austin October 28, 2014 s Vector Moving s Siple classical dynaics - point asses oved by forces Point asses can odel particles
More informationlecture 36: Linear Multistep Mehods: Zero Stability
95 lecture 36: Linear Multistep Mehods: Zero Stability 5.6 Linear ultistep ethods: zero stability Does consistency iply convergence for linear ultistep ethods? This is always the case for one-step ethods,
More informationOptimal Jamming Over Additive Noise: Vector Source-Channel Case
Fifty-first Annual Allerton Conference Allerton House, UIUC, Illinois, USA October 2-3, 2013 Optial Jaing Over Additive Noise: Vector Source-Channel Case Erah Akyol and Kenneth Rose Abstract This paper
More informationHebb rule book: 'The Organization of Behavior' Theory about the neural bases of learning
PCA by neurons Hebb rule 1949 book: 'The Organization of Behavior' Theory about the neural bases of learning Learning takes place in synapses. Synapses get modified, they get stronger when the pre- and
More informationRECOVERY OF A DENSITY FROM THE EIGENVALUES OF A NONHOMOGENEOUS MEMBRANE
Proceedings of ICIPE rd International Conference on Inverse Probles in Engineering: Theory and Practice June -8, 999, Port Ludlow, Washington, USA : RECOVERY OF A DENSITY FROM THE EIGENVALUES OF A NONHOMOGENEOUS
More informationASSUME a source over an alphabet size m, from which a sequence of n independent samples are drawn. The classical
IEEE TRANSACTIONS ON INFORMATION THEORY Large Alphabet Source Coding using Independent Coponent Analysis Aichai Painsky, Meber, IEEE, Saharon Rosset and Meir Feder, Fellow, IEEE arxiv:67.7v [cs.it] Jul
More informationarxiv: v1 [cs.ds] 3 Feb 2014
arxiv:40.043v [cs.ds] 3 Feb 04 A Bound on the Expected Optiality of Rando Feasible Solutions to Cobinatorial Optiization Probles Evan A. Sultani The Johns Hopins University APL evan@sultani.co http://www.sultani.co/
More informationIterative face image feature extraction with Generalized Hebbian Algorithm and a Sanger-like BCM rule
Iterative face image feature extraction with Generalized Hebbian Algorithm and a Sanger-like BCM rule Clayton Aldern (Clayton_Aldern@brown.edu) Tyler Benster (Tyler_Benster@brown.edu) Carl Olsson (Carl_Olsson@brown.edu)
More informationLecture 21 Principle of Inclusion and Exclusion
Lecture 21 Principle of Inclusion and Exclusion Holden Lee and Yoni Miller 5/6/11 1 Introduction and first exaples We start off with an exaple Exaple 11: At Sunnydale High School there are 28 students
More informationLec 05 Arithmetic Coding
Outline CS/EE 5590 / ENG 40 Special Topics (7804, 785, 7803 Lec 05 Arithetic Coding Lecture 04 ReCap Arithetic Coding About Hoework- and Lab Zhu Li Course Web: http://l.web.ukc.edu/lizhu/teaching/06sp.video-counication/ain.htl
More informationDetection and Estimation Theory
ESE 54 Detection and Estiation Theory Joseph A. O Sullivan Sauel C. Sachs Professor Electronic Systes and Signals Research Laboratory Electrical and Systes Engineering Washington University 11 Urbauer
More informationQuantum algorithms (CO 781, Winter 2008) Prof. Andrew Childs, University of Waterloo LECTURE 15: Unstructured search and spatial search
Quantu algoriths (CO 781, Winter 2008) Prof Andrew Childs, University of Waterloo LECTURE 15: Unstructured search and spatial search ow we begin to discuss applications of quantu walks to search algoriths
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationKernel Methods and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley ENSIAG 2 / osig 1 Second Seester 2012/2013 Lesson 20 2 ay 2013 Kernel ethods and Support Vector achines Contents Kernel Functions...2 Quadratic
More informationA Simple Regression Problem
A Siple Regression Proble R. M. Castro March 23, 2 In this brief note a siple regression proble will be introduced, illustrating clearly the bias-variance tradeoff. Let Y i f(x i ) + W i, i,..., n, where
More informationFeedforward Networks
Feedforward Neural Networks - Backpropagation Feedforward Networks Gradient Descent Learning and Backpropagation CPSC 533 Fall 2003 Christian Jacob Dept.of Coputer Science,University of Calgary Feedforward
More informationOn Lotka-Volterra Evolution Law
Advanced Studies in Biology, Vol. 3, 0, no. 4, 6 67 On Lota-Volterra Evolution Law Farruh Muhaedov Faculty of Science, International Islaic University Malaysia P.O. Box, 4, 570, Kuantan, Pahang, Malaysia
More informationRandom Process Examples 1/23
ando Process Eaples /3 E. #: D-T White Noise Let ] be a sequence of V s where each V ] in the sequence is uncorrelated with all the others: E{ ] ] } 0 for This DEFINES a DT White Noise Also called Uncorrelated
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016 Lessons 7 14 Dec 2016 Outline Artificial Neural networks Notation...2 1. Introduction...3... 3 The Artificial
More informationModeling Chemical Reactions with Single Reactant Specie
Modeling Cheical Reactions with Single Reactant Specie Abhyudai Singh and João edro Hespanha Abstract A procedure for constructing approxiate stochastic odels for cheical reactions involving a single reactant
More informationBoosting with log-loss
Boosting with log-loss Marco Cusuano-Towner Septeber 2, 202 The proble Suppose we have data exaples {x i, y i ) i =... } for a two-class proble with y i {, }. Let F x) be the predictor function with the
More informationRecovering Data from Underdetermined Quadratic Measurements (CS 229a Project: Final Writeup)
Recovering Data fro Underdeterined Quadratic Measureents (CS 229a Project: Final Writeup) Mahdi Soltanolkotabi Deceber 16, 2011 1 Introduction Data that arises fro engineering applications often contains
More informationLeast Squares Fitting of Data
Least Squares Fitting of Data David Eberly, Geoetric Tools, Redond WA 98052 https://www.geoetrictools.co/ This work is licensed under the Creative Coons Attribution 4.0 International License. To view a
More informationPrincipal Component Analysis
CSci 5525: Machine Learning Dec 3, 2008 The Main Idea Given a dataset X = {x 1,..., x N } The Main Idea Given a dataset X = {x 1,..., x N } Find a low-dimensional linear projection The Main Idea Given
More informationSupport Vector Machines MIT Course Notes Cynthia Rudin
Support Vector Machines MIT 5.097 Course Notes Cynthia Rudin Credit: Ng, Hastie, Tibshirani, Friedan Thanks: Şeyda Ertekin Let s start with soe intuition about argins. The argin of an exaple x i = distance
More informationTEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES
TEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES S. E. Ahed, R. J. Tokins and A. I. Volodin Departent of Matheatics and Statistics University of Regina Regina,
More informationVulnerability of MRD-Code-Based Universal Secure Error-Correcting Network Codes under Time-Varying Jamming Links
Vulnerability of MRD-Code-Based Universal Secure Error-Correcting Network Codes under Tie-Varying Jaing Links Jun Kurihara KDDI R&D Laboratories, Inc 2 5 Ohara, Fujiino, Saitaa, 356 8502 Japan Eail: kurihara@kddilabsjp
More informationTutorial on Principal Component Analysis
Tutorial on Principal Component Analysis Copyright c 1997, 2003 Javier R. Movellan. This is an open source document. Permission is granted to copy, distribute and/or modify this document under the terms
More informationSTOPPING SIMULATED PATHS EARLY
Proceedings of the 2 Winter Siulation Conference B.A.Peters,J.S.Sith,D.J.Medeiros,andM.W.Rohrer,eds. STOPPING SIMULATED PATHS EARLY Paul Glasseran Graduate School of Business Colubia University New Yor,
More informationarxiv: v1 [stat.ml] 31 Jan 2018
Increental kernel PCA and the Nyströ ethod arxiv:802.00043v [stat.ml] 3 Jan 208 Fredrik Hallgren Departent of Statistical Science University College London London WCE 6BT, United Kingdo fredrik.hallgren@ucl.ac.uk
More informationMultivariate Methods. Matlab Example. Principal Components Analysis -- PCA
Multivariate Methos Xiaoun Qi Principal Coponents Analysis -- PCA he PCA etho generates a new set of variables, calle principal coponents Each principal coponent is a linear cobination of the original
More informationarxiv: v1 [stat.ot] 7 Jul 2010
Hotelling s test for highly correlated data P. Bubeliny e-ail: bubeliny@karlin.ff.cuni.cz Charles University, Faculty of Matheatics and Physics, KPMS, Sokolovska 83, Prague, Czech Republic, 8675. arxiv:007.094v
More informationINNER CONSTRAINTS FOR A 3-D SURVEY NETWORK
eospatial Science INNER CONSRAINS FOR A 3-D SURVEY NEWORK hese notes follow closely the developent of inner constraint equations by Dr Willie an, Departent of Building, School of Design and Environent,
More informationNB1140: Physics 1A - Classical mechanics and Thermodynamics Problem set 2 - Forces and energy Week 2: November 2016
NB1140: Physics 1A - Classical echanics and Therodynaics Proble set 2 - Forces and energy Week 2: 21-25 Noveber 2016 Proble 1. Why force is transitted uniforly through a assless string, a assless spring,
More informationLinear Transformations
Linear Transforations Hopfield Network Questions Initial Condition Recurrent Layer p S x W S x S b n(t + ) a(t + ) S x S x D a(t) S x S S x S a(0) p a(t + ) satlins (Wa(t) + b) The network output is repeatedly
More informationA New Class of APEX-Like PCA Algorithms
Reprinted fro Proceedings of ISCAS-98, IEEE Int. Syposiu on Circuit and Systes, Monterey (USA), June 1998 A New Class of APEX-Like PCA Algoriths Sione Fiori, Aurelio Uncini, Francesco Piazza Dipartiento
More informationPhysics 221B: Solution to HW # 6. 1) Born-Oppenheimer for Coupled Harmonic Oscillators
Physics B: Solution to HW # 6 ) Born-Oppenheier for Coupled Haronic Oscillators This proble is eant to convince you of the validity of the Born-Oppenheier BO) Approxiation through a toy odel of coupled
More information