Fast Convolutional Sparse Coding (FCSC)
|
|
- Amber Neal
- 5 years ago
- Views:
Transcription
1 Fat Convolutional Spare Coding (FCSC) Bailey ong Department of Computer Science Univerity of California, Irvine Charle C. Fowlke Department of Computer Science Univerity of California, Irvine Introduction Convolutional pare coding remedie the hortcoming of the traditional (patch-baed) pare coding by modeling hift invariance directly. Thi yield more parimoniou coding cheme but increae complexity by coupling the coding problem for neighboring image patche where they overlap. Here we decribe an optimization approach which exploit the eparability of convolution acro band in the frequency domain in order to make dictionary learning efficient. Our preentation follow the notation in the paper of [Britow et al. IEEE CVPR ] but include additional implementation detail and correct ome inaccuracie and typo. Convolutional coding in the frequency domain The objective for convolutional pare coding i arg min d,z x d k z k + z k.t. d k k =,...,, where d k R M i the k-th filter, z k R D i the correponding pare feature map, and x R D M+ i an image. (Note that M D.) For implicity, we dicu uing -dimenional image ignal, but thi generalize to -dimenional image ignal a well. It i well known that convolution correpond to element-wie multiplication in the frequency domain, o it i more efficient to rewrite thi objective in the frequency domain. Unfortunately thi cannot be done due to the l penalty term, which i not rotation invariant. One way to work around thi problem i by introducing two auxiliary variable and t o that we can optimize part of the problem in the frequency domain and another part in the patial domain. The objective can be rewritten a arg min ˆd,,ẑ,t D ˆx ˆd k ẑ k + t k.t. = F ˆdk k =,..., k =,..., t k = F ẑ k k =,..., = C k =,...,, where ˆx, ˆd k, ẑ k C D,, t k R D, and F i the invere dicrete Fourier tranform matrix. The original variable d and z are moved to the frequency domain, and are now denoted a ˆd and ẑ, while the auxiliary variable and t take their place in the patial domain and contraint are added to couple them together. The careful oberver may alo noticed that although originally d k wa M dimenional, i now D dimenional. The mapping matrix C zero out any entrie of which are outide the deired patial upport o the final contraint aure the learned dictionary element have mall patial extent. Notation: F i the dicrete Fourier tranform matrix and F i the invere DFT matrix defined a F = D F. Since the tranform matrix F i orthogonal, the quared l -norm in the patial and frequency domain are equivalent up to a cale factor (i.e., D = ŝ ). The matrix C i the mapping matrix that truncate a vector R D R M and pad the truncation with zero back to R D. Thi i can be defined a the product of two matrice C = PT, where P i our padding matrix (a D M identity matrix) and T i our truncation matrix (a M D identity matrix). For the vector d,, z, and t where the ubcript Thi work wa upported by NSF DBI-4 and NSF DBI- () ()
2 i dropped, we refer to a uper vector with component tacked together (e.g. = [ T,..., T ]T ). The ame applie for the vector in the frequency domain, ŝ = [(F ),..., (F ). Optimization by ADMM To olve thi problem with variable in both the frequency and patial domain, the Alternating Direction Method of Multiplier (ADMM) approach i ued. We begin by writing the augmented Lagrangian that incorporate the equality contraint coupling the auxiliary variable, t to the original variable d, z. L(ˆd,, ẑ, t, λ, λ t ) = D ˆx ˆd k ẑ k + t k + + F ˆdk + F ẑ k t k +.t. k =,..., = C k =,...,, λ T k(f ˆdk ) λ T tk(f ẑ k t k ) () where λ k,, λ tk, and are the Lagrange multiplier aociated with and t repectively. Subproblem z: ẑ ẑ ẑ ẑ L(ẑ; ˆd,, t, λ, λ t ) D ˆx ˆd k ẑ k + F ẑ k t k + D ˆx ˆDẑ + D ẑ ˆt + D ˆλ t(ẑ ˆt) = ( ˆD ˆD + µt I) ( ˆD ˆx + ˆt ˆλ t ), λ T tk(f ẑ k t k ) (4) where ˆD = [diag(ˆd ),..., diag(ˆd )] i D D. Although the matrix ˆD ˆD i quite large, D D, it i pare banded and ha an efficient variable reordering uch that ẑ can be found by olving D independent linear ytem. Subproblem t: t k t k L(t k ; ˆd k,, ẑ k, λ k, λ tk ) t k t k t k + F ẑ k t k + λ T tk(f ẑ k t k ) t k + z k t k + λ T tk(z k t k ) () Given z k and λ tk in the patial domain, it turn out that each coefficient in t k i independent of all other coefficient, t ki t ki (z ki t ki ) + λ tki (z ki t ki ) + t ki. () Which can be efficiently compute in cloed form uing the hrinkage (oft-threholding) function, ( t ki = ign z ki + λ ) { tki max z ki + λ tki },. ()
3 Subproblem d: ˆd ˆd ˆd ˆd L(ˆd;, ẑ, t, λ, λ t ) D ˆx ˆd k ẑ k + F ˆdk + D ˆx Ẑˆd + D ˆd ŝ + D ˆλ (ˆd ŝ) = (Ẑ Ẑ + I) (Ẑ ˆx + ŝ ˆλ ) λ T k(f ˆdk ) where Ẑ = [diag(ẑ ),..., diag(ẑ )] i D D. Like ubproblem z, there i an efficient variable reordering uch that ˆd can be found by olving D independent linear ytem. Subproblem : k L( ; ˆd k, ẑ k, t k, λ k, λ tk ).t. = C F ˆdk + λ T k(f ˆdk ) d k + λ T k(d k ) An equivalent formulation i to drop the patial upport contraint on and intead preerve only the M coefficient of d k and λ k aociated with the mall patial upport. k.t.. Cd k + (Cλ k ) T (Cd k ) The coefficient of outide the patial upport (where Cd k i ) are automatically driven to by the l norm (ee appendix for detail). We can olve the uncontrained verion of thi problem by pulling the linear term into the quadratic, completing the quare k Cd k + (Cλ k ) T (Cd k ) d k + λ T k( d k ) where d k = Cd k, λ k = Cλ k ( dk + ) λk µ d k + λk + d kt dk + λ T d k k ( dk + ) λk µ The optimal olution to the uncontrained problem (denoted by ) i thu given in cloed form by (8) (9) () = Cd k + Cλ k, () Since thi objective i iotropic, we can imply enforce the norm contraint by projecting the uncontrained olution back on to the contraint et: { k k = if. () otherwie Lagrange Multiplier Update: λ (i+) λ (i+) t = λ (i) + (d (i+) (i+) ) () = λ (i) t + (z (i+) t (i+) ) (4)
4 Penalty Update: µ (i+) = { τµ (i) if µ (i) < µ max µ (i) otherwie () Algorithm Convolutional Spare Coding uing Fourier Subproblem and ADMM : Initialize (), z (), t (), λ (), λ () t, µ (), µ () t : Perform FFT (), z (), t (), λ (), λ () : i = 4: repeat : dictionary update: t ŝ (), ẑ (), ˆt (), : Solve for ˆd (i+) given ŝ (i), ẑ (i) (i), ˆλ uing Eq. 8 : Perform invere FFT ˆd (i+) d (i+) 8: Solve for (i+) given d (i+) uing Eq. 9: Solve for (i+) by projecting (i+) uing Eq. : code update: () () ˆλ, ˆλ t : Solve for ẑ (i+) given ˆt (i) (i), ˆλ t, ˆd (i+) uing Eq. 4 : Perform invere FFT ẑ (i+) z (i+) : Solve for t (i+) given z (i+), λ (i) t uing Eq. 4: Update Lagrange multiplier vector uing Eq. and Eq. 4 : Update penaltie uing Eq. : Perform FFT, t, λ, λ t ŝ, ˆt, ˆλ, ˆλ t : i = i + 8: until ˆd,, ẑ, t ha converged 4 Effect of ADMM parameter on optimization Thi ection aim to provide ome intuition regarding the hyper-parameter of FCSC optimization, pecifically we want to undertand how the hyper-parameter interact with one another in term of the convergence rate and the objective value. In all our experiment, we ue image that are contrat normalized. The convergence criteria i that we ve reach a feaible olution (i.e., = F ˆd and t = F ẑ). The FCSC algorithm ha three hyper-parameter,,, and. can be thought of a the parameter that control the convergence rate of the dictionary learning, a it i the tep-ize at which we move toward a feaible olution. Likewie, i the ame for the coding. The hyper-parameter in our original objective control the parity of the code found. (a) (b) (c) Figure : (a) Example A. odoratum gra pollen grain image (SEM, x). (b) Viualization of pare coding of pollen texture. Peudo-color indicate dictionary element with the larget magnitude coefficient (c) Correponding dictionary learned from image of pollen texture uing convolutional pare coding with = 4
5 4. Effect of tep-ize parameter on coding Firt we look at the coding hyper-parameter; uing a pre-trained dictionary, we vary and. Since control the parity of the code, we expect (and the reult how) that a it increae the recontruction error would alo increae. Perhap not urpriingly, the ame behavior i oberved w.r.t. the objective value. A increae the code become more pare to the point where they re all zero. At that point the objective value i jut the recontruction error with a vector of zero. We ee that both the recontruction error and the objective value at convergence increae lowly a increae. In another experiment (not hown) we have een the objective value be roughly the ame for all value of and =, when a minimum number of iteration i enforced (in thoe experiment it wa ). Thee iteration are neceary to overcome the memory of the initial value of t in the ADMM iteration. In term of the convergence rate, for all value of we generally ee that the number of iteration required before convergence decreae a increae. Thi i alo to be expected a we previouly decribed a the tep-ize toward a feaible olution. Ignoring =, we generally oberve that the number of iteration goe down a increae for any particular value of. Thi i a bit urpriing, but can poibly be explained by conidering how parity i induced. We know that the hrinkage function i ued to olve the t-ubproblem, and that at each iteration the code are hrunk by toward zero. If i large enough, then doe not grow enough at the early iteration uch that additional coefficient of z are not zeroed out. Thi in turn magnifie the coefficient of λ t to make z more like t, thu leading to fater convergence. 4. Effect of tep-ize and parity parameter on dictionary learning In the econd experiment, we fix the tep-ize and imultaneouly learn the dictionary and the code. Here the reult are quite different from before. We ee that mut be trictly greater than for FCSC to learn anything meaningful. When = both the recontruction error and the objective value are largely unchanged w.r.t.. Finally in term of convergence, we continue to oberve that a increae the number of iteration to convergence decreae for all value of, conitent with our firt experiment. 4. Effect of tep-ize on runtime and accuracy In the third experiment, we fix the hyper-parameter equal to and look at how and interact. With fixed, we ee that the objective value increae a and increae in the domain > and >. Thi eem to be a reult of the optimization enforcing contraint too quickly. For mall value of and (not hown) we alo ee erratic behavior. If we ignore the region where i le than, we ee that both the objective and the recontruction error are mooth w.r.t. the hyper-parameter. Thi i quite nice, a we can chooe a computational trade-off between lowering the objective value and the number of iteration to convergence. We may alo ak whether the optimization produce qualitatively good filter. A good learned filter houldn t reemble the initialization and in general we expect to ee Gabor-like oriented edge filter. After examining ome learned filter that were ubjectively good/bad, we defined goodne a having a tandard deviation greater than.. Much higher than the flat initialization which ha a tandard deviation of. and other bad looking filter that had tandard deviation le than. in the cae we examined. While not a perfect meaure by any mean, we can ee in the next et of figure that the recontruction error i indeed correlated with the goodne meaure. x 4... Original objective x 4 4 Recontruction error 8 4 Iter. to convergence Figure : Effect of, on coding with fixed dictionary.
6 x 4. Original objective x 4 Recontruction error Iter. to convergence Figure : Effect of, on joint coding and dictionary learning ( = ). Appendix A - Leat-quare filter with contrained patial upport To ee why Eq. i equivalent to the previou argmin (Eq. 9), conider the following problem = d + λ T (d ).t. = C [ ] d + d + λ T (d ) + λ T (d ),.t. =, where we plit each vector into two ub-vector: the part that correpond to mall patial upport (denoted with ) and the part that doe not (denoted with ). By plitting into and, we can think about each a eparate problem a there are no term that contain both. d + λ T (d ) Td + Tλ T (Td ) d + λ T (d ).t. = So the problem amount to an uncontrained linear leat quare; the problem can be tranformed into an equivalent uncontrained problem with a trivial olution when certain condition hold. Thee condition are that d and λ are null vector. The matrix C doe exactly thi, a it preerve the coefficient of any vector correponding to the mall patial upport and zero out the ret. Original objective Iter. to convergence Good Figure 4: Joint effect of, on olution quality and run time ( = )
7 Appendix B - Implementation detail During our dicuion of ubproblem z and d, we mentioned an efficient variable reordering to find the optimal olution by olving D independent linear ytem. Firt we define the following matrice Ẑ = [ẑ,..., ẑ, ˆT = [ˆt,..., ˆt, ˆD = [ˆd,..., ˆd, Ŝ = [ŝ,..., ŝ ˆΛ t = [ˆλt,..., ˆλ t, ˆΛ = [ˆλ,..., ˆλ ˆ Dx = [ˆd ˆx,..., ˆd ˆx, ˆ Zx = [ẑ ˆx,..., ẑ ˆx and denote Ẑi, ˆT i, ˆD i, Ŝi, ˆΛ ti, ˆΛ i, Dxi ˆ, and Zx ˆ i to be the i-th column of their repective matrice. We can then olve the ubproblem z with and ubproblem d with Ẑ i = ( ˆD i ˆD i + I) ( Dx ˆ i + ˆTi + ˆΛ ti ), ˆD i = (ẐiẐ i + I) ( ˆ Zx i + Ŝ i + ˆΛ i ). In Eq. 8, we olved for ˆd k which the invere Fourier tranform need to be applied to get d k. However, ince only a ub-vector of d k i needed computing the invere of the entire vector i wateful. If we look at the definition of C, we ee that we can define a new matrix Φ = TF that will invert jut the part of the vector we need. And the maller M i with repect to D, the greater the aving we gain. We can redefine Eq. and Eq. a = Φˆd k + Φˆλ k. () { k k = P if otherwie () Our entire dicuion thu far ha revolved around a ingle image, to handle multiple image i quite traightforward and only require u to change Eq. 8, N N ˆd = ( Ẑ xẑx + I) ( Ẑ xˆx x + ŝ ˆλ ). (8) x= x= Note that ˆT i unrelated to th truncation matrix T
Chapter 2 Sampling and Quantization. In order to investigate sampling and quantization, the difference between analog
Chapter Sampling and Quantization.1 Analog and Digital Signal In order to invetigate ampling and quantization, the difference between analog and digital ignal mut be undertood. Analog ignal conit of continuou
More informationSocial Studies 201 Notes for November 14, 2003
1 Social Studie 201 Note for November 14, 2003 Etimation of a mean, mall ample ize Section 8.4, p. 501. When a reearcher ha only a mall ample ize available, the central limit theorem doe not apply to the
More informationCompact finite-difference approximations for anisotropic image smoothing and painting
CWP-593 Compact finite-difference approximation for aniotropic image moothing and painting Dave Hale Center for Wave Phenomena, Colorado School of Mine, Golden CO 80401, USA ABSTRACT Finite-difference
More informationThe Hassenpflug Matrix Tensor Notation
The Haenpflug Matrix Tenor Notation D.N.J. El Dept of Mech Mechatron Eng Univ of Stellenboch, South Africa e-mail: dnjel@un.ac.za 2009/09/01 Abtract Thi i a ample document to illutrate the typeetting of
More informationSocial Studies 201 Notes for March 18, 2005
1 Social Studie 201 Note for March 18, 2005 Etimation of a mean, mall ample ize Section 8.4, p. 501. When a reearcher ha only a mall ample ize available, the central limit theorem doe not apply to the
More informationLecture 7: Testing Distributions
CSE 5: Sublinear (and Streaming) Algorithm Spring 014 Lecture 7: Teting Ditribution April 1, 014 Lecturer: Paul Beame Scribe: Paul Beame 1 Teting Uniformity of Ditribution We return today to property teting
More informationLecture 21. The Lovasz splitting-off lemma Topics in Combinatorial Optimization April 29th, 2004
18.997 Topic in Combinatorial Optimization April 29th, 2004 Lecture 21 Lecturer: Michel X. Goeman Scribe: Mohammad Mahdian 1 The Lovaz plitting-off lemma Lovaz plitting-off lemma tate the following. Theorem
More information7.2 INVERSE TRANSFORMS AND TRANSFORMS OF DERIVATIVES 281
72 INVERSE TRANSFORMS AND TRANSFORMS OF DERIVATIVES 28 and i 2 Show how Euler formula (page 33) can then be ued to deduce the reult a ( a) 2 b 2 {e at co bt} {e at in bt} b ( a) 2 b 2 5 Under what condition
More informationOne Class of Splitting Iterative Schemes
One Cla of Splitting Iterative Scheme v Ciegi and V. Pakalnytė Vilniu Gedimina Technical Univerity Saulėtekio al. 11, 2054, Vilniu, Lithuania rc@fm.vtu.lt Abtract. Thi paper deal with the tability analyi
More informationThe Laplace Transform , Haynes Miller and Jeremy Orloff
The Laplace Tranform 8.3, Hayne Miller and Jeremy Orloff Laplace tranform baic: introduction An operator take a function a input and output another function. A tranform doe the ame thing with the added
More informationBogoliubov Transformation in Classical Mechanics
Bogoliubov Tranformation in Claical Mechanic Canonical Tranformation Suppoe we have a et of complex canonical variable, {a j }, and would like to conider another et of variable, {b }, b b ({a j }). How
More informationSampling and the Discrete Fourier Transform
Sampling and the Dicrete Fourier Tranform Sampling Method Sampling i mot commonly done with two device, the ample-and-hold (S/H) and the analog-to-digital-converter (ADC) The S/H acquire a CT ignal at
More informationClustering Methods without Given Number of Clusters
Clutering Method without Given Number of Cluter Peng Xu, Fei Liu Introduction A we now, mean method i a very effective algorithm of clutering. It mot powerful feature i the calability and implicity. However,
More informationPreemptive scheduling on a small number of hierarchical machines
Available online at www.ciencedirect.com Information and Computation 06 (008) 60 619 www.elevier.com/locate/ic Preemptive cheduling on a mall number of hierarchical machine György Dóa a, Leah Eptein b,
More informationProperties of Z-transform Transform 1 Linearity a
Midterm 3 (Fall 6 of EEG:. Thi midterm conit of eight ingle-ided page. The firt three page contain variou table followed by FOUR eam quetion and one etra workheet. You can tear out any page but make ure
More informationAlternate Dispersion Measures in Replicated Factorial Experiments
Alternate Diperion Meaure in Replicated Factorial Experiment Neal A. Mackertich The Raytheon Company, Sudbury MA 02421 Jame C. Benneyan Northeatern Univerity, Boton MA 02115 Peter D. Krau The Raytheon
More informationA Constraint Propagation Algorithm for Determining the Stability Margin. The paper addresses the stability margin assessment for linear systems
A Contraint Propagation Algorithm for Determining the Stability Margin of Linear Parameter Circuit and Sytem Lubomir Kolev and Simona Filipova-Petrakieva Abtract The paper addree the tability margin aement
More informationinto a discrete time function. Recall that the table of Laplace/z-transforms is constructed by (i) selecting to get
Lecture 25 Introduction to Some Matlab c2d Code in Relation to Sampled Sytem here are many way to convert a continuou time function, { h( t) ; t [0, )} into a dicrete time function { h ( k) ; k {0,,, }}
More informationCDMA Signature Sequences with Low Peak-to-Average-Power Ratio via Alternating Projection
CDMA Signature Sequence with Low Peak-to-Average-Power Ratio via Alternating Projection Joel A Tropp Int for Comp Engr and Sci (ICES) The Univerity of Texa at Autin 1 Univerity Station C0200 Autin, TX
More informationAssignment for Mathematics for Economists Fall 2016
Due date: Mon. Nov. 1. Reading: CSZ, Ch. 5, Ch. 8.1 Aignment for Mathematic for Economit Fall 016 We now turn to finihing our coverage of concavity/convexity. There are two part: Jenen inequality for concave/convex
More informationCHEAP CONTROL PERFORMANCE LIMITATIONS OF INPUT CONSTRAINED LINEAR SYSTEMS
Copyright 22 IFAC 5th Triennial World Congre, Barcelona, Spain CHEAP CONTROL PERFORMANCE LIMITATIONS OF INPUT CONSTRAINED LINEAR SYSTEMS Tritan Pérez Graham C. Goodwin Maria M. Serón Department of Electrical
More informationUnavoidable Cycles in Polynomial-Based Time-Invariant LDPC Convolutional Codes
European Wirele, April 7-9,, Vienna, Autria ISBN 978--87-4-9 VE VERLAG GMBH Unavoidable Cycle in Polynomial-Baed Time-Invariant LPC Convolutional Code Hua Zhou and Norbert Goertz Intitute of Telecommunication
More informationLecture 10 Filtering: Applied Concepts
Lecture Filtering: Applied Concept In the previou two lecture, you have learned about finite-impule-repone (FIR) and infinite-impule-repone (IIR) filter. In thee lecture, we introduced the concept of filtering
More informationCHAPTER 8 OBSERVER BASED REDUCED ORDER CONTROLLER DESIGN FOR LARGE SCALE LINEAR DISCRETE-TIME CONTROL SYSTEMS
CHAPTER 8 OBSERVER BASED REDUCED ORDER CONTROLLER DESIGN FOR LARGE SCALE LINEAR DISCRETE-TIME CONTROL SYSTEMS 8.1 INTRODUCTION 8.2 REDUCED ORDER MODEL DESIGN FOR LINEAR DISCRETE-TIME CONTROL SYSTEMS 8.3
More informationBy Xiaoquan Wen and Matthew Stephens University of Michigan and University of Chicago
Submitted to the Annal of Applied Statitic SUPPLEMENTARY APPENDIX TO BAYESIAN METHODS FOR GENETIC ASSOCIATION ANALYSIS WITH HETEROGENEOUS SUBGROUPS: FROM META-ANALYSES TO GENE-ENVIRONMENT INTERACTIONS
More informationSuggestions - Problem Set (a) Show the discriminant condition (1) takes the form. ln ln, # # R R
Suggetion - Problem Set 3 4.2 (a) Show the dicriminant condition (1) take the form x D Ð.. Ñ. D.. D. ln ln, a deired. We then replace the quantitie. 3ß D3 by their etimate to get the proper form for thi
More informationAn Inequality for Nonnegative Matrices and the Inverse Eigenvalue Problem
An Inequality for Nonnegative Matrice and the Invere Eigenvalue Problem Robert Ream Program in Mathematical Science The Univerity of Texa at Dalla Box 83688, Richardon, Texa 7583-688 Abtract We preent
More informationPDF hosted at the Radboud Repository of the Radboud University Nijmegen
PDF hoted at the Radboud Repoitory of the Radboud Univerity Nijmegen The following full text i an author' verion which may differ from the publiher' verion. For additional information about thi publication
More informationμ + = σ = D 4 σ = D 3 σ = σ = All units in parts (a) and (b) are in V. (1) x chart: Center = μ = 0.75 UCL =
Our online Tutor are available 4*7 to provide Help with Proce control ytem Homework/Aignment or a long term Graduate/Undergraduate Proce control ytem Project. Our Tutor being experienced and proficient
More informationChapter 4: Applications of Fourier Representations. Chih-Wei Liu
Chapter 4: Application of Fourier Repreentation Chih-Wei Liu Outline Introduction Fourier ranform of Periodic Signal Convolution/Multiplication with Non-Periodic Signal Fourier ranform of Dicrete-ime Signal
More informationCorrection for Simple System Example and Notes on Laplace Transforms / Deviation Variables ECHE 550 Fall 2002
Correction for Simple Sytem Example and Note on Laplace Tranform / Deviation Variable ECHE 55 Fall 22 Conider a tank draining from an initial height of h o at time t =. With no flow into the tank (F in
More informationLecture 8: Period Finding: Simon s Problem over Z N
Quantum Computation (CMU 8-859BB, Fall 205) Lecture 8: Period Finding: Simon Problem over Z October 5, 205 Lecturer: John Wright Scribe: icola Rech Problem A mentioned previouly, period finding i a rephraing
More informationConvergence criteria and optimization techniques for beam moments
Pure Appl. Opt. 7 (1998) 1221 1230. Printed in the UK PII: S0963-9659(98)90684-5 Convergence criteria and optimization technique for beam moment G Gbur and P S Carney Department of Phyic and Atronomy and
More information3.1 The Revised Simplex Algorithm. 3 Computational considerations. Thus, we work with the following tableau. Basic observations = CARRY. ... m.
3 Computational conideration In what follow, we analyze the complexity of the Simplex algorithm more in detail For thi purpoe, we focu on the update proce in each iteration of thi procedure Clearly, ince,
More informationFactor Analysis with Poisson Output
Factor Analyi with Poion Output Gopal Santhanam Byron Yu Krihna V. Shenoy, Department of Electrical Engineering, Neurocience Program Stanford Univerity Stanford, CA 94305, USA {gopal,byronyu,henoy}@tanford.edu
More informationV = 4 3 πr3. d dt V = d ( 4 dv dt. = 4 3 π d dt r3 dv π 3r2 dv. dt = 4πr 2 dr
0.1 Related Rate In many phyical ituation we have a relationhip between multiple quantitie, and we know the rate at which one of the quantitie i changing. Oftentime we can ue thi relationhip a a convenient
More informationStochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions
Stochatic Optimization with Inequality Contraint Uing Simultaneou Perturbation and Penalty Function I-Jeng Wang* and Jame C. Spall** The John Hopkin Univerity Applied Phyic Laboratory 11100 John Hopkin
More informationLecture 9: Shor s Algorithm
Quantum Computation (CMU 8-859BB, Fall 05) Lecture 9: Shor Algorithm October 7, 05 Lecturer: Ryan O Donnell Scribe: Sidhanth Mohanty Overview Let u recall the period finding problem that wa et up a a function
More informationHomework 12 Solution - AME30315, Spring 2013
Homework 2 Solution - AME335, Spring 23 Problem :[2 pt] The Aerotech AGS 5 i a linear motor driven XY poitioning ytem (ee attached product heet). A friend of mine, through careful experimentation, identified
More informationCOHOMOLOGY AS A LOCAL-TO-GLOBAL BRIDGE
COHOMOLOGY AS A LOCAL-TO-GLOBAL BRIDGE LIVIU I. NICOLAESCU ABSTRACT. I dicu low dimenional incarnation of cohomology and illutrate how baic cohomological principle lead to a proof of Sperner lemma. CONTENTS.
More informationEfficient Methods of Doppler Processing for Coexisting Land and Weather Clutter
Efficient Method of Doppler Proceing for Coexiting Land and Weather Clutter Ça gatay Candan and A Özgür Yılmaz Middle Eat Technical Univerity METU) Ankara, Turkey ccandan@metuedutr, aoyilmaz@metuedutr
More informationConvex Hulls of Curves Sam Burton
Convex Hull of Curve Sam Burton 1 Introduction Thi paper will primarily be concerned with determining the face of convex hull of curve of the form C = {(t, t a, t b ) t [ 1, 1]}, a < b N in R 3. We hall
More informationNew bounds for Morse clusters
New bound for More cluter Tamá Vinkó Advanced Concept Team, European Space Agency, ESTEC Keplerlaan 1, 2201 AZ Noordwijk, The Netherland Tama.Vinko@ea.int and Arnold Neumaier Fakultät für Mathematik, Univerität
More informationLecture 17: Analytic Functions and Integrals (See Chapter 14 in Boas)
Lecture 7: Analytic Function and Integral (See Chapter 4 in Boa) Thi i a good point to take a brief detour and expand on our previou dicuion of complex variable and complex function of complex variable.
More informationSingular perturbation theory
Singular perturbation theory Marc R. Rouel June 21, 2004 1 Introduction When we apply the teady-tate approximation (SSA) in chemical kinetic, we typically argue that ome of the intermediate are highly
More informationCONGESTION control is a key functionality in modern
IEEE TRANSACTIONS ON INFORMATION TEORY, VOL. X, NO. X, XXXXXXX 2008 On the Connection-Level Stability of Congetion-Controlled Communication Network Xiaojun Lin, Member, IEEE, Ne B. Shroff, Fellow, IEEE,
More informationProblem Set 8 Solutions
Deign and Analyi of Algorithm April 29, 2015 Maachuett Intitute of Technology 6.046J/18.410J Prof. Erik Demaine, Srini Devada, and Nancy Lynch Problem Set 8 Solution Problem Set 8 Solution Thi problem
More informationSuggested Answers To Exercises. estimates variability in a sampling distribution of random means. About 68% of means fall
Beyond Significance Teting ( nd Edition), Rex B. Kline Suggeted Anwer To Exercie Chapter. The tatitic meaure variability among core at the cae level. In a normal ditribution, about 68% of the core fall
More informationTheoretical Computer Science. Optimal algorithms for online scheduling with bounded rearrangement at the end
Theoretical Computer Science 4 (0) 669 678 Content lit available at SciVere ScienceDirect Theoretical Computer Science journal homepage: www.elevier.com/locate/tc Optimal algorithm for online cheduling
More informationLearning Multiplicative Interactions
CSC2535 2011 Lecture 6a Learning Multiplicative Interaction Geoffrey Hinton Two different meaning of multiplicative If we take two denity model and multiply together their probability ditribution at each
More informationDetermination of the local contrast of interference fringe patterns using continuous wavelet transform
Determination of the local contrat of interference fringe pattern uing continuou wavelet tranform Jong Kwang Hyok, Kim Chol Su Intitute of Optic, Department of Phyic, Kim Il Sung Univerity, Pyongyang,
More informationMATEMATIK Datum: Tid: eftermiddag. A.Heintz Telefonvakt: Anders Martinsson Tel.:
MATEMATIK Datum: 20-08-25 Tid: eftermiddag GU, Chalmer Hjälpmedel: inga A.Heintz Telefonvakt: Ander Martinon Tel.: 073-07926. Löningar till tenta i ODE och matematik modellering, MMG5, MVE6. Define what
More informationReal Sources (Secondary Sources) Phantom Source (Primary source) LS P. h rl. h rr. h ll. h lr. h pl. h pr
Ecient frequency domain ltered-x realization of phantom ource iet C.W. ommen, Ronald M. Aart, Alexander W.M. Mathijen, John Gara, Haiyan He Abtract A phantom ound ource i a virtual ound image which can
More informationOnline Appendix for Managerial Attention and Worker Performance by Marina Halac and Andrea Prat
Online Appendix for Managerial Attention and Worker Performance by Marina Halac and Andrea Prat Thi Online Appendix contain the proof of our reult for the undicounted limit dicued in Section 2 of the paper,
More informationLDPC Convolutional Codes Based on Permutation Polynomials over Integer Rings
LDPC Convolutional Code Baed on Permutation Polynomial over Integer Ring Marco B. S. Tavare and Gerhard P. Fettwei Vodafone Chair Mobile Communication Sytem, Dreden Univerity of Technology, 01062 Dreden,
More informationTMA4125 Matematikk 4N Spring 2016
Norwegian Univerity of Science and Technology Department of Mathematical Science TMA45 Matematikk 4N Spring 6 Solution to problem et 6 In general, unle ele i noted, if f i a function, then F = L(f denote
More informationMarch 18, 2014 Academic Year 2013/14
POLITONG - SHANGHAI BASIC AUTOMATIC CONTROL Exam grade March 8, 4 Academic Year 3/4 NAME (Pinyin/Italian)... STUDENT ID Ue only thee page (including the back) for anwer. Do not ue additional heet. Ue of
More informationMulticolor Sunflowers
Multicolor Sunflower Dhruv Mubayi Lujia Wang October 19, 2017 Abtract A unflower i a collection of ditinct et uch that the interection of any two of them i the ame a the common interection C of all of
More information5. Fuzzy Optimization
5. Fuzzy Optimization 1. Fuzzine: An Introduction 135 1.1. Fuzzy Memberhip Function 135 1.2. Memberhip Function Operation 136 2. Optimization in Fuzzy Environment 136 3. Fuzzy Set for Water Allocation
More informationIEOR 3106: Fall 2013, Professor Whitt Topics for Discussion: Tuesday, November 19 Alternating Renewal Processes and The Renewal Equation
IEOR 316: Fall 213, Profeor Whitt Topic for Dicuion: Tueday, November 19 Alternating Renewal Procee and The Renewal Equation 1 Alternating Renewal Procee An alternating renewal proce alternate between
More informationThe type 3 nonuniform FFT and its applications
Journal of Computational Phyic 206 (2005) 1 5 Short Note The type 3 nonuniform FFT and it application June-Yub Lee a, *, Lelie Greengard b a Department of Mathematic, Ewha Woman Univerity, 11-1 Daehyundong,
More informationOptimal Coordination of Samples in Business Surveys
Paper preented at the ICES-III, June 8-, 007, Montreal, Quebec, Canada Optimal Coordination of Sample in Buine Survey enka Mach, Ioana Şchiopu-Kratina, Philip T Rei, Jean-Marc Fillion Statitic Canada New
More informationEE 508 Lecture 16. Filter Transformations. Lowpass to Bandpass Lowpass to Highpass Lowpass to Band-reject
EE 508 Lecture 6 Filter Tranformation Lowpa to Bandpa Lowpa to Highpa Lowpa to Band-reject Review from Lat Time Theorem: If the perimeter variation and contact reitance are neglected, the tandard deviation
More informationApplication of Gradient Projection for Sparse Reconstruction to Compressed Sensing for Image Reconstruction of Electrical Capacitance Tomography
Journal of Electrical and Electronic Engineering 08; 6(): 46-5 http://www.ciencepublihinggroup.com/j/jeee doi: 0.648/j.jeee.08060. ISS: 39-63 (Print); ISS: 39-605 (Online) Application of Gradient Projection
More informationTHE SPLITTING SUBSPACE CONJECTURE
THE SPLITTING SUBSPAE ONJETURE ERI HEN AND DENNIS TSENG Abtract We anwer a uetion by Niederreiter concerning the enumeration of a cla of ubpace of finite dimenional vector pace over finite field by proving
More informationGain and Phase Margins Based Delay Dependent Stability Analysis of Two- Area LFC System with Communication Delays
Gain and Phae Margin Baed Delay Dependent Stability Analyi of Two- Area LFC Sytem with Communication Delay Şahin Sönmez and Saffet Ayaun Department of Electrical Engineering, Niğde Ömer Halidemir Univerity,
More informationDIFFERENTIAL EQUATIONS
DIFFERENTIAL EQUATIONS Laplace Tranform Paul Dawkin Table of Content Preface... Laplace Tranform... Introduction... The Definition... 5 Laplace Tranform... 9 Invere Laplace Tranform... Step Function...4
More informationAdvanced Digital Signal Processing. Stationary/nonstationary signals. Time-Frequency Analysis... Some nonstationary signals. Time-Frequency Analysis
Advanced Digital ignal Proceing Prof. Nizamettin AYDIN naydin@yildiz.edu.tr Time-Frequency Analyi http://www.yildiz.edu.tr/~naydin 2 tationary/nontationary ignal Time-Frequency Analyi Fourier Tranform
More informationFUNDAMENTALS OF POWER SYSTEMS
1 FUNDAMENTALS OF POWER SYSTEMS 1 Chapter FUNDAMENTALS OF POWER SYSTEMS INTRODUCTION The three baic element of electrical engineering are reitor, inductor and capacitor. The reitor conume ohmic or diipative
More informationSolving Differential Equations by the Laplace Transform and by Numerical Methods
36CH_PHCalter_TechMath_95099 3//007 :8 PM Page Solving Differential Equation by the Laplace Tranform and by Numerical Method OBJECTIVES When you have completed thi chapter, you hould be able to: Find the
More informationRoot Locus Diagram. Root loci: The portion of root locus when k assume positive values: that is 0
Objective Root Locu Diagram Upon completion of thi chapter you will be able to: Plot the Root Locu for a given Tranfer Function by varying gain of the ytem, Analye the tability of the ytem from the root
More informationProximal Iteratively Reweighted Algorithm with Multiple Splitting for Nonconvex Sparsity Optimization
Proceeding of the Twenty-Eighth AAAI Conference on Artificial Intelligence Proimal Iteratively Reweighted Algorithm with Multiple Splitting for Nonconve Sparity Optimization Canyi Lu 1, Yunchao Wei, Zhouchen
More informationThe continuous time random walk (CTRW) was introduced by Montroll and Weiss 1.
1 I. CONTINUOUS TIME RANDOM WALK The continuou time random walk (CTRW) wa introduced by Montroll and Wei 1. Unlike dicrete time random walk treated o far, in the CTRW the number of jump n made by the walker
More informationCalculation of the temperature of boundary layer beside wall with time-dependent heat transfer coefficient
Ŕ periodica polytechnica Mechanical Engineering 54/1 21 15 2 doi: 1.3311/pp.me.21-1.3 web: http:// www.pp.bme.hu/ me c Periodica Polytechnica 21 RESERCH RTICLE Calculation of the temperature of boundary
More informationPIPELINING AND PARALLEL PROCESSING. UNIT 4 Real time Signal Processing
PIPELINING AND PARALLEL PROCESSING UNIT 4 Real time Signal Proceing Content Introduction Pipeling of FIR Digital Filter Parallel proceing Low power Deign FIR Digital Filter A FIR Filter i defined a follow:
More informationMoment of Inertia of an Equilateral Triangle with Pivot at one Vertex
oment of nertia of an Equilateral Triangle with Pivot at one Vertex There are two wa (at leat) to derive the expreion f an equilateral triangle that i rotated about one vertex, and ll how ou both here.
More informationHalliday/Resnick/Walker 7e Chapter 6
HRW 7e Chapter 6 Page of Halliday/Renick/Walker 7e Chapter 6 3. We do not conider the poibility that the bureau might tip, and treat thi a a purely horizontal motion problem (with the peron puh F in the
More informationQuestion 1 Equivalent Circuits
MAE 40 inear ircuit Fall 2007 Final Intruction ) Thi exam i open book You may ue whatever written material you chooe, including your cla note and textbook You may ue a hand calculator with no communication
More informationJul 4, 2005 turbo_code_primer Revision 0.0. Turbo Code Primer
Jul 4, 5 turbo_code_primer Reviion. Turbo Code Primer. Introduction Thi document give a quick tutorial on MAP baed turbo coder. Section develop the background theory. Section work through a imple numerical
More informationLTV System Modelling
Helinki Univerit of Technolog S-72.333 Potgraduate Coure in Radiocommunication Fall 2000 LTV Stem Modelling Heikki Lorentz Sonera Entrum O heikki.lorentz@onera.fi Januar 23 rd 200 Content. Introduction
More informationNonlinear Single-Particle Dynamics in High Energy Accelerators
Nonlinear Single-Particle Dynamic in High Energy Accelerator Part 6: Canonical Perturbation Theory Nonlinear Single-Particle Dynamic in High Energy Accelerator Thi coure conit of eight lecture: 1. Introduction
More informationLecture #9 Continuous time filter
Lecture #9 Continuou time filter Oliver Faut December 5, 2006 Content Review. Motivation......................................... 2 2 Filter pecification 2 2. Low pa..........................................
More informationEE 508 Lecture 16. Filter Transformations. Lowpass to Bandpass Lowpass to Highpass Lowpass to Band-reject
EE 508 Lecture 6 Filter Tranformation Lowpa to Bandpa Lowpa to Highpa Lowpa to Band-reject Review from Lat Time Theorem: If the perimeter variation and contact reitance are neglected, the tandard deviation
More information11.2 Stability. A gain element is an active device. One potential problem with every active circuit is its stability
5/7/2007 11_2 tability 1/2 112 tability eading Aignment: pp 542-548 A gain element i an active device One potential problem with every active circuit i it tability HO: TABIITY Jim tile The Univ of Kana
More informationChapter 5 Consistency, Zero Stability, and the Dahlquist Equivalence Theorem
Chapter 5 Conitency, Zero Stability, and the Dahlquit Equivalence Theorem In Chapter 2 we dicued convergence of numerical method and gave an experimental method for finding the rate of convergence (aka,
More informationAn Image-encoded Mach-Zehnder Joint Transform Correlator for Polychromatic Pattern Recognition with Multi-level Quantized Reference Functions
Proceeding of the International MultiConference of Engineer and Computer Scientit 008 Vol I IMECS 008, 19-1 March, 008, Hong Kong An Image-encoded Mach-Zehnder Joint Tranform Correlator for Polychromatic
More informationUSPAS Course on Recirculated and Energy Recovered Linear Accelerators
USPAS Coure on Recirculated and Energy Recovered Linear Accelerator G. A. Krafft and L. Merminga Jefferon Lab I. Bazarov Cornell Lecture 6 7 March 005 Lecture Outline. Invariant Ellipe Generated by a Unimodular
More informationThe Impact of Imperfect Scheduling on Cross-Layer Rate. Control in Multihop Wireless Networks
The mpact of mperfect Scheduling on Cro-Layer Rate Control in Multihop Wirele Network Xiaojun Lin and Ne B. Shroff Center for Wirele Sytem and Application (CWSA) School of Electrical and Computer Engineering,
More informationONLINE APPENDIX: TESTABLE IMPLICATIONS OF TRANSLATION INVARIANCE AND HOMOTHETICITY: VARIATIONAL, MAXMIN, CARA AND CRRA PREFERENCES
ONLINE APPENDIX: TESTABLE IMPLICATIONS OF TRANSLATION INVARIANCE AND HOMOTHETICITY: VARIATIONAL, MAXMIN, CARA AND CRRA PREFERENCES CHRISTOPHER P. CHAMBERS, FEDERICO ECHENIQUE, AND KOTA SAITO In thi online
More informationImproving the Efficiency of a Digital Filtering Scheme for Diabatic Initialization
1976 MONTHLY WEATHER REVIEW VOLUME 15 Improving the Efficiency of a Digital Filtering Scheme for Diabatic Initialization PETER LYNCH Met Éireann, Dublin, Ireland DOMINIQUE GIARD CNRM/GMAP, Météo-France,
More informationDigital Control System
Digital Control Sytem - A D D A Micro ADC DAC Proceor Correction Element Proce Clock Meaurement A: Analog D: Digital Continuou Controller and Digital Control Rt - c Plant yt Continuou Controller Digital
More informationA generalized mathematical framework for stochastic simulation and forecast of hydrologic time series
WATER RESOURCES RESEARCH, VOL. 36, NO. 6, PAGES 1519 1533, JUNE 2000 A generalized mathematical framework for tochatic imulation and forecat of hydrologic time erie Demetri Koutoyianni Department of Water
More informationNoise Effects on Two Excitable Systems: Semiconductor Laser and Neuron Network Oscillations
Noie Effect on Two Excitable Sytem: Semiconductor Laer and Neuron Network Ocillation Athena Pan, Wei-Mien Mendy Hu Department of Phyic, Univerity of California, San Diego I. Introduction An excitable ytem
More informationModeling the scalar wave equation with Nyström methods
GEOPHYSICS, VOL. 71, NO. 5 SEPTEMBER-OCTOBER 200 ; P. T151 T158, 7 FIGS. 10.1190/1.2335505 Modeling the calar wave equation with Nytröm method Jing-Bo Chen 1 ABSTRACT High-accuracy numerical cheme for
More informationPhysics 741 Graduate Quantum Mechanics 1 Solutions to Final Exam, Fall 2014
Phyic 7 Graduate Quantum Mechanic Solution to inal Eam all 0 Each quetion i worth 5 point with point for each part marked eparately Some poibly ueful formula appear at the end of the tet In four dimenion
More information0 of the same magnitude. If we don t use an OA and ignore any damping, the CTF is
1 4. Image Simulation Influence of C Spherical aberration break the ymmetry that would otherwie exit between overfocu and underfocu. One reult i that the fringe in the FT of the CTF are generally farther
More informationPractice Problems - Week #7 Laplace - Step Functions, DE Solutions Solutions
For Quetion -6, rewrite the piecewie function uing tep function, ketch their graph, and find F () = Lf(t). 0 0 < t < 2. f(t) = (t 2 4) 2 < t In tep-function form, f(t) = u 2 (t 2 4) The graph i the olid
More informationControl Systems Analysis and Design by the Root-Locus Method
6 Control Sytem Analyi and Deign by the Root-Locu Method 6 1 INTRODUCTION The baic characteritic of the tranient repone of a cloed-loop ytem i cloely related to the location of the cloed-loop pole. If
More informationLAPLACE TRANSFORM REVIEW SOLUTIONS
LAPLACE TRANSFORM REVIEW SOLUTIONS. Find the Laplace tranform for the following function. If an image i given, firt write out the function and then take the tranform. a e t inh4t From #8 on the table:
More informationCONTROL SYSTEMS, ROBOTICS AND AUTOMATION Vol. VIII Decoupling Control - M. Fikar
DECOUPLING CONTROL M. Fikar Department of Proce Control, Faculty of Chemical and Food Technology, Slovak Univerity of Technology in Bratilava, Radlinkého 9, SK-812 37 Bratilava, Slovakia Keyword: Decoupling:
More information