Variable Fractional Delay FIR Filters with Sparse Coefficients

Size: px
Start display at page:

Download "Variable Fractional Delay FIR Filters with Sparse Coefficients"

Transcription

1 Variable Fractional Delay FIR Filters with Sparse Coefficients W.-S. Lu T. Hinamoto Dept. of Electrical & Computer Engineering Graduate School of Engineering University of Victoria Hiroshima University Victoria, Canada Hiroshima, Japan

2 Outline Design Problem and Related Work Significance Design Method at a Glance A Design Example Concluding Remarks

3 . Design Problem and Related Work Design Problem Find a VFD FIR digital filter of order (N, K) N K n n n nk n= k= H (, zp) = a( pz ), a( p) = a p k with p and sparse A = {a ij } ( N ) ( K ) R + + that optimally approximates a desired frequency response H ( ω, p) = e ω in a weighted least-squares sense. π d 3 j ( D+ p) J ( A) = W( ω, p) H( ω, p) H ( ω, p) dpdω subject to: sparsity( A) = N z d

4 Related Work A. Tarczynski, G. D. Cain, E. Hermanowicz and M. Rojewki, 997 W.-S. Lu and T.-B. Deng, 999. T.-B. Deng,. C.-C. Tseng, 4. T.-B. Deng and Y. Lian, 6. Linear programming algorithms for sparse FIR filters (T. Baran, D. Wei, and A.V. Oppenheim, March ) l -minimization algorithms for sparse -D and -D FIR filters (W.-S. Lu and T. Hinamoto,, ) 4

5 . Significance Implementing a VFD filter in Farrow model is costly as each coefficient of a VFD filter is a polynomial rather than a scalar. Hence VFD digital filters with sparse coefficients are of interest because the sparsity implies reduced implementation complexity (cost), hence real-time application potential. VFD filters are not sparse in general. 5

6 3.Design Method at a Glance Define T jω jnω K ω = e e and p = p p and assume a separable weighting function W( ω, p) = W( ω) W( p), then up to a constant one can write where π J ( A) = W( ω, p) H( ω, p) H ( ω, p) dpdω T = tr( PA ΩA) tr( SA) d T P S π T T = W( p) dp, Re W( ω) dω pp Ω = ωω π T T T jω ( D+ p) = W( p) pdp, p Re W( ω) e dω pω ω = ω 6

7 To deal with large condition numbers of P and Ω, Cholesky T T decompositions P = P P, Ω = ΩΩ are used, and up to a constant J(A) can be written as T J ( a) = Γa y, Γ = P Ω, y= Γ s where a and s are the vectors generated by concatenating the columns of A and S, and is the Kronecker product. Design Phase To identify an index set of the most appropriate locations in a to be set to zero in order to satisfy a target sparsity. jω This is done subject to maintaining closeness of H ( e, p) to H ( ω, p). d 7

8 The target sparsity is achieved by introducing sparsity promoting l -norm of a into an objective function: minimize μ a + Γ a y () a where a n = i= a i. Problem () is a convex, for which many fast algorithms are available. E.g. using FISTA, () can be solved with a small number of iterations. Input: Data Γ and y, parameter μ and iteration number M. Step. Compute A = Γ y, a = A (:). Set b = a, t =, and m =. Step. Compute a m = S μ/l {b m ( /L)Γ T (Γb m y)}, where Sα ( u) = sgn( u) max{ u α,} Step 3. Update t m+ = (+ + 4 t m ) / Step 4. Update b m+ = a m + (( tm ) / t m + ) (a m a m ). Step 5. If m < M, set m = m + and repeat from Step ; otherwise stop and output a m as solution â. 8

9 Hard thresholding is applied to vector â with an appropriate value of threshold ε so that the length of the index set I = {, i aˆ () i < ε } equals to target sparsity N z. The index set I is a key ingredient of design phase. Design Phase To find a coefficient matrix A that minimizes the WLS error J(A) subject to the sparsity constraint. This part of the design is carried out by solving the convex problem 9 Γ minimize J ( a) = a y a subject to: ai ( ) = for i I By simply substituting the constraints into the objective function, the above problem becomes an unconstrained least square problem whose solution is given by

10 ( I ) Γ s and ( I ) a = y a = where I denotes the set of index not in I and Γ s is composed of those columns of Γ with indices in set I. 4. A Design Example The Design Problem Design a VFD FIR filter of order (N = 65 and K = 7) with cutoff ωc =.9π. Design performance is evaluated in terms of maximum error emax = max { e( ω, p), ω.9 π, p } with e( ω, p) = log H( ω, p) H ( ω, p) and L-error d

11 .9π e = H( ω, p) Hd ( ω, p) dpdω / The weighting function was set to W( ω, p) = W( ω) W( p) with W (p) = for p in [, ] and for ω [,.88 π ) W ( ω) = 3 for ω [.88 π,.8994 π) for ω [.88 π, π] The target sparsity was set to N z = 98 which means a 37.5% of coefficients were set to zero. To achieve this sparsity, the two key parameters in phase- were set to μ 5 3 =, ε =. It took 6 FISTA iterations for the algorithm in phase to converge.

12 Phase of the design then produced an optimal A* with sparsity(a*) = 98. Below are the numerical evaluation results: maximum error e max =.. L-error e = 75.5 db. ======== Comparison ======= Compare with an equivalent nonsparse VFD filter of order (N = 65, K = 4) with the same specifications: maximum error e max =.69. L-error e = 45.8 db.

13 Profile of frequency response error e( ω, p) over ω.9 π, p : the sparse VFD filter: 5 e(,p) (db) Fractional delay p Normalized frequency 3

14 Profile of frequency response error e( ω, p) over ω.9 π, p : the nonsparse VFD filter: 5 e(,p) (db) Fractional delay p Normalized frequency 4

15 ======== Comparison ======= To justify phase of the design, we compare the above result with the following: We design a conversional (nonsparse) VFD filter of order (N = 65, K = 7), then applied hard thresholding to generate exactly 98 locations that may be considered appropriate to set to zero. We then went on the carried out phase to yield a sparse VFD filter. It was found that maximum error e max =.5 (vs. with phase ) L-error e = 73.4 db (vs 75.5 db with phase ). 5

16 Profile of frequency response error e( ω, p) over ω.9 π, p : the sparse VFD filter without phase : 5 e(,p) (db) Fractional delay p Normalized frequency 6

17 ======== Comparison 3 ======= Fractional delay over ω.9 π, p. The sparse filter:.8 Fractional delay Fractional delay p Normalized frequency.8 7

18 Fractional delay over ω.9 π, p. The equivalent nonsparse filter:.8 Fractional delay Fractional delay p Normalized frequency.8 8

19 5. Concluding Remarks A two-phase technique for the WLS design of VFD FIR filters subject to a target coefficient sparsity constraint has been proposed. The design algorithm is easy to implement abd computationally efficient because it is based on l l convex optimization. The performance of the filter appears to be satisfactory compared with its nonsparse counterpart. 9

Direct Design of Orthogonal Filter Banks and Wavelets

Direct Design of Orthogonal Filter Banks and Wavelets Direct Design of Orthogonal Filter Banks and Wavelets W.-S. Lu T. Hinamoto Dept. of Electrical & Computer Engineering Graduate School of Engineering University of Victoria Hiroshima University Victoria,

More information

Enhanced Steiglitz-McBride Procedure for. Minimax IIR Digital Filters

Enhanced Steiglitz-McBride Procedure for. Minimax IIR Digital Filters Enhanced Steiglitz-McBride Procedure for Minimax IIR Digital Filters Wu-Sheng Lu Takao Hinamoto University of Victoria Hiroshima University Victoria, Canada Higashi-Hiroshima, Japan May 30, 2018 1 Outline

More information

Two-Dimensional State-Space Digital Filters with Minimum Frequency-Weighted L 2 -Sensitivity

Two-Dimensional State-Space Digital Filters with Minimum Frequency-Weighted L 2 -Sensitivity wo-dimensional State-Space Digital Filters with Minimum Frequency-Weighted L -Sensitivity under L -Scaling Constraints. Hinamoto,. Oumi, O. I. Omoifo W.-S. Lu Graduate School of Engineering Electrical

More information

A Unified Approach to the Design of Interpolated and Frequency Response Masking FIR Filters

A Unified Approach to the Design of Interpolated and Frequency Response Masking FIR Filters A Unified Approach to the Design of Interpolated and Frequency Response Masking FIR Filters Wu Sheng Lu akao Hinamoto University of Victoria Hiroshima University Victoria, Canada Higashi Hiroshima, Japan

More information

Minimax Design of Complex-Coefficient FIR Filters with Low Group Delay

Minimax Design of Complex-Coefficient FIR Filters with Low Group Delay Minimax Design of Complex-Coefficient FIR Filters with Low Group Delay Wu-Sheng Lu Takao Hinamoto Dept. of Elec. and Comp. Engineering Graduate School of Engineering University of Victoria Hiroshima University

More information

-Digital Signal Processing- FIR Filter Design. Lecture May-16

-Digital Signal Processing- FIR Filter Design. Lecture May-16 -Digital Signal Processing- FIR Filter Design Lecture-17 24-May-16 FIR Filter Design! FIR filters can also be designed from a frequency response specification.! The equivalent sampled impulse response

More information

Iterative reweighted l 1 design of sparse FIR filters

Iterative reweighted l 1 design of sparse FIR filters Iterative reweighted l 1 design of sparse FIR filters Cristian Rusu, Bogdan Dumitrescu Abstract Designing sparse 1D and 2D filters has been the object of research in recent years due mainly to the developments

More information

DIGITAL filters capable of changing their frequency response

DIGITAL filters capable of changing their frequency response IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: ANALOG AND DIGITAL SIGNAL PROCESSING, VOL 46, NO 8, AUGUST 1999 1035 An Improved Weighted Least-Squares Design Variable Fractional Delay FIR Filters Wu-Sheng

More information

Design of Projection Matrix for Compressive Sensing by Nonsmooth Optimization

Design of Projection Matrix for Compressive Sensing by Nonsmooth Optimization Design of Proection Matrix for Compressive Sensing by Nonsmooth Optimization W.-S. Lu T. Hinamoto Dept. of Electrical & Computer Engineering Graduate School of Engineering University of Victoria Hiroshima

More information

Design of Stable IIR filters with prescribed flatness and approximately linear phase

Design of Stable IIR filters with prescribed flatness and approximately linear phase Design of Stable IIR filters with prescribed flatness and approximately linear phase YASUNORI SUGITA Nagaoka University of Technology Dept. of Electrical Engineering Nagaoka city, Niigata-pref., JAPAN

More information

Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks

Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks 1 / 45 Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks Jie Yan Department of Electrical and Computer Engineering University of Victoria April 16, 2010 2 / 45 OUTLINE 1 INTRODUCTION

More information

Part 4: IIR Filters Optimization Approach. Tutorial ISCAS 2007

Part 4: IIR Filters Optimization Approach. Tutorial ISCAS 2007 Part 4: IIR Filters Optimization Approach Tutorial ISCAS 2007 Copyright 2007 Andreas Antoniou Victoria, BC, Canada Email: aantoniou@ieee.org July 24, 2007 Frame # 1 Slide # 1 A. Antoniou Part4: IIR Filters

More information

Weighted Least-Squares Method for Designing Variable Fractional Delay 2-D FIR Digital Filters

Weighted Least-Squares Method for Designing Variable Fractional Delay 2-D FIR Digital Filters 114 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: ANALOG AND DIGITAL SIGNAL PROCESSING, VOL 47, NO 2, FEBRUARY 2000 Weighted Least-Squares Method for Designing Variable Fractional Delay 2-D FIR Digital

More information

Recovery of Sparse Signals from Noisy Measurements Using an l p -Regularized Least-Squares Algorithm

Recovery of Sparse Signals from Noisy Measurements Using an l p -Regularized Least-Squares Algorithm Recovery of Sparse Signals from Noisy Measurements Using an l p -Regularized Least-Squares Algorithm J. K. Pant, W.-S. Lu, and A. Antoniou University of Victoria August 25, 2011 Compressive Sensing 1 University

More information

Optimal Design of Real and Complex Minimum Phase Digital FIR Filters

Optimal Design of Real and Complex Minimum Phase Digital FIR Filters Optimal Design of Real and Complex Minimum Phase Digital FIR Filters Niranjan Damera-Venkata and Brian L. Evans Embedded Signal Processing Laboratory Dept. of Electrical and Computer Engineering The University

More information

Compressed Sensing and Neural Networks

Compressed Sensing and Neural Networks and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications

More information

Quadrature-Mirror Filter Bank

Quadrature-Mirror Filter Bank Quadrature-Mirror Filter Bank In many applications, a discrete-time signal x[n] is split into a number of subband signals { v k [ n]} by means of an analysis filter bank The subband signals are then processed

More information

Learning Gaussian Graphical Models with Unknown Group Sparsity

Learning Gaussian Graphical Models with Unknown Group Sparsity Learning Gaussian Graphical Models with Unknown Group Sparsity Kevin Murphy Ben Marlin Depts. of Statistics & Computer Science Univ. British Columbia Canada Connections Graphical models Density estimation

More information

LINEAR SYSTEMS (11) Intensive Computation

LINEAR SYSTEMS (11) Intensive Computation LINEAR SYSTEMS () Intensive Computation 27-8 prof. Annalisa Massini Viviana Arrigoni EXACT METHODS:. GAUSSIAN ELIMINATION. 2. CHOLESKY DECOMPOSITION. ITERATIVE METHODS:. JACOBI. 2. GAUSS-SEIDEL 2 CHOLESKY

More information

Solution: K m = R 1 = 10. From the original circuit, Z L1 = jωl 1 = j10 Ω. For the scaled circuit, L 1 = jk m ωl 1 = j10 10 = j100 Ω, Z L

Solution: K m = R 1 = 10. From the original circuit, Z L1 = jωl 1 = j10 Ω. For the scaled circuit, L 1 = jk m ωl 1 = j10 10 = j100 Ω, Z L Problem 9.9 Circuit (b) in Fig. P9.9 is a scaled version of circuit (a). The scaling process may have involved magnitude or frequency scaling, or both simultaneously. If R = kω gets scaled to R = kω, supply

More information

Neural Network Algorithm for Designing FIR Filters Utilizing Frequency-Response Masking Technique

Neural Network Algorithm for Designing FIR Filters Utilizing Frequency-Response Masking Technique Wang XH, He YG, Li TZ. Neural network algorithm for designing FIR filters utilizing frequency-response masking technique. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 4(3): 463 471 May 009 Neural Network

More information

Filter Design Problem

Filter Design Problem Filter Design Problem Design of frequency-selective filters usually starts with a specification of their frequency response function. Practical filters have passband and stopband ripples, while exhibiting

More information

BlockMatrixComputations and the Singular Value Decomposition. ATaleofTwoIdeas

BlockMatrixComputations and the Singular Value Decomposition. ATaleofTwoIdeas BlockMatrixComputations and the Singular Value Decomposition ATaleofTwoIdeas Charles F. Van Loan Department of Computer Science Cornell University Supported in part by the NSF contract CCR-9901988. Block

More information

Linear Programming Algorithms for Sparse Filter Design

Linear Programming Algorithms for Sparse Filter Design Linear Programming Algorithms for Sparse Filter Design The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher

More information

Digital Signal Processing

Digital Signal Processing COMP ENG 4TL4: Digital Signal Processing Notes for Lecture #21 Friday, October 24, 2003 Types of causal FIR (generalized) linear-phase filters: Type I: Symmetric impulse response: with order M an even

More information

The basic structure of the L-channel QMF bank is shown below

The basic structure of the L-channel QMF bank is shown below -Channel QMF Bans The basic structure of the -channel QMF ban is shown below The expressions for the -transforms of various intermediate signals in the above structure are given by Copyright, S. K. Mitra

More information

Stochastic Optimization: First order method

Stochastic Optimization: First order method Stochastic Optimization: First order method Taiji Suzuki Tokyo Institute of Technology Graduate School of Information Science and Engineering Department of Mathematical and Computing Sciences JST, PRESTO

More information

sparse and low-rank tensor recovery Cubic-Sketching

sparse and low-rank tensor recovery Cubic-Sketching Sparse and Low-Ran Tensor Recovery via Cubic-Setching Guang Cheng Department of Statistics Purdue University www.science.purdue.edu/bigdata CCAM@Purdue Math Oct. 27, 2017 Joint wor with Botao Hao and Anru

More information

Lecture 9: September 28

Lecture 9: September 28 0-725/36-725: Convex Optimization Fall 206 Lecturer: Ryan Tibshirani Lecture 9: September 28 Scribes: Yiming Wu, Ye Yuan, Zhihao Li Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These

More information

Electronic Circuits EE359A

Electronic Circuits EE359A Electronic Circuits EE359A Bruce McNair B26 bmcnair@stevens.edu 21-216-5549 Lecture 22 578 Second order LCR resonator-poles V o I 1 1 = = Y 1 1 + sc + sl R s = C 2 s 1 s + + CR LC s = C 2 sω 2 s + + ω

More information

FIR BAND-PASS DIGITAL DIFFERENTIATORS WITH FLAT PASSBAND AND EQUIRIPPLE STOPBAND CHARACTERISTICS. T. Yoshida, Y. Sugiura, N.

FIR BAND-PASS DIGITAL DIFFERENTIATORS WITH FLAT PASSBAND AND EQUIRIPPLE STOPBAND CHARACTERISTICS. T. Yoshida, Y. Sugiura, N. FIR BAND-PASS DIGITAL DIFFERENTIATORS WITH FLAT PASSBAND AND EQUIRIPPLE STOPBAND CHARACTERISTICS T. Yoshida, Y. Sugiura, N. Aikawa Tokyo University of Science Faculty of Industrial Science and Technology

More information

Designing Information Devices and Systems I Discussion 13B

Designing Information Devices and Systems I Discussion 13B EECS 6A Fall 7 Designing Information Devices and Systems I Discussion 3B. Orthogonal Matching Pursuit Lecture Orthogonal Matching Pursuit (OMP) algorithm: Inputs: A set of m songs, each of length n: S

More information

CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6

CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6 CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 6 GENE H GOLUB Issues with Floating-point Arithmetic We conclude our discussion of floating-point arithmetic by highlighting two issues that frequently

More information

EFFICIENT REMEZ ALGORITHMS FOR THE DESIGN OF NONRECURSIVE FILTERS

EFFICIENT REMEZ ALGORITHMS FOR THE DESIGN OF NONRECURSIVE FILTERS EFFICIENT REMEZ ALGORITHMS FOR THE DESIGN OF NONRECURSIVE FILTERS Copyright 2003- Andreas Antoniou Victoria, BC, Canada Email: aantoniou@ieee.org July 24, 2007 Frame # 1 Slide # 1 A. Antoniou EFFICIENT

More information

Compressive Imaging by Generalized Total Variation Minimization

Compressive Imaging by Generalized Total Variation Minimization 1 / 23 Compressive Imaging by Generalized Total Variation Minimization Jie Yan and Wu-Sheng Lu Department of Electrical and Computer Engineering University of Victoria, Victoria, BC, Canada APCCAS 2014,

More information

III. Applications in convex optimization

III. Applications in convex optimization III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization

More information

Towards Global Design of Orthogonal Filter Banks and Wavelets

Towards Global Design of Orthogonal Filter Banks and Wavelets Towards Global Design of Orthogonal Filter Banks and Wavelets Jie Yan and Wu-Sheng Lu Department of Electrical and Computer Engineering University of Victoria Victoria, BC, Canada V8W 3P6 jyan@ece.uvic.ca,

More information

Provable Alternating Minimization Methods for Non-convex Optimization

Provable Alternating Minimization Methods for Non-convex Optimization Provable Alternating Minimization Methods for Non-convex Optimization Prateek Jain Microsoft Research, India Joint work with Praneeth Netrapalli, Sujay Sanghavi, Alekh Agarwal, Animashree Anandkumar, Rashish

More information

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Adrien Todeschini Inria Bordeaux JdS 2014, Rennes Aug. 2014 Joint work with François Caron (Univ. Oxford), Marie

More information

Lecture 8: February 9

Lecture 8: February 9 0-725/36-725: Convex Optimiation Spring 205 Lecturer: Ryan Tibshirani Lecture 8: February 9 Scribes: Kartikeya Bhardwaj, Sangwon Hyun, Irina Caan 8 Proximal Gradient Descent In the previous lecture, we

More information

Algebra C Numerical Linear Algebra Sample Exam Problems

Algebra C Numerical Linear Algebra Sample Exam Problems Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric

More information

Approximation. Inderjit S. Dhillon Dept of Computer Science UT Austin. SAMSI Massive Datasets Opening Workshop Raleigh, North Carolina.

Approximation. Inderjit S. Dhillon Dept of Computer Science UT Austin. SAMSI Massive Datasets Opening Workshop Raleigh, North Carolina. Using Quadratic Approximation Inderjit S. Dhillon Dept of Computer Science UT Austin SAMSI Massive Datasets Opening Workshop Raleigh, North Carolina Sept 12, 2012 Joint work with C. Hsieh, M. Sustik and

More information

Power-Efficient Linear Phase FIR Notch Filter Design Using the LARS Scheme

Power-Efficient Linear Phase FIR Notch Filter Design Using the LARS Scheme Wei Xu, Jiaxiang Zhao, Hongie Wang, Chao Gu Power-Efficient Linear Phase FIR Notch Filter Design Using the LARS Scheme WEI XU Tianin Polytechnic University Institute of Electronics and Information Engineering

More information

Dictionary Learning for photo-z estimation

Dictionary Learning for photo-z estimation Dictionary Learning for photo-z estimation Joana Frontera-Pons, Florent Sureau, Jérôme Bobin 5th September 2017 - Workshop Dictionary Learning on Manifolds MOTIVATION! Goal : Measure the radial positions

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

( ) John A. Quinn Lecture. ESE 531: Digital Signal Processing. Lecture Outline. Frequency Response of LTI System. Example: Zero on Real Axis

( ) John A. Quinn Lecture. ESE 531: Digital Signal Processing. Lecture Outline. Frequency Response of LTI System. Example: Zero on Real Axis John A. Quinn Lecture ESE 531: Digital Signal Processing Lec 15: March 21, 2017 Review, Generalized Linear Phase Systems Penn ESE 531 Spring 2017 Khanna Lecture Outline!!! 2 Frequency Response of LTI System

More information

OWL to the rescue of LASSO

OWL to the rescue of LASSO OWL to the rescue of LASSO IISc IBM day 2018 Joint Work R. Sankaran and Francis Bach AISTATS 17 Chiranjib Bhattacharyya Professor, Department of Computer Science and Automation Indian Institute of Science,

More information

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms François Caron Department of Statistics, Oxford STATLEARN 2014, Paris April 7, 2014 Joint work with Adrien Todeschini,

More information

Sparse Subspace Clustering

Sparse Subspace Clustering Sparse Subspace Clustering Based on Sparse Subspace Clustering: Algorithm, Theory, and Applications by Elhamifar and Vidal (2013) Alex Gutierrez CSCI 8314 March 2, 2017 Outline 1 Motivation and Background

More information

A direct formulation for sparse PCA using semidefinite programming

A direct formulation for sparse PCA using semidefinite programming A direct formulation for sparse PCA using semidefinite programming A. d Aspremont, L. El Ghaoui, M. Jordan, G. Lanckriet ORFE, Princeton University & EECS, U.C. Berkeley Available online at www.princeton.edu/~aspremon

More information

Bayesian Regularization and Nonnegative Deconvolution for Time Delay Estimation

Bayesian Regularization and Nonnegative Deconvolution for Time Delay Estimation University of Pennsylvania ScholarlyCommons Departmental Papers (ESE) Department of Electrical & Systems Engineering December 24 Bayesian Regularization and Nonnegative Deconvolution for Time Delay Estimation

More information

Two Dimensional Linear Phase Multiband Chebyshev FIR Filter

Two Dimensional Linear Phase Multiband Chebyshev FIR Filter Two Dimensional Linear Phase Multiband Chebyshev FIR Filter Vinay Kumar, Bhooshan Sunil To cite this version: Vinay Kumar, Bhooshan Sunil. Two Dimensional Linear Phase Multiband Chebyshev FIR Filter. Acta

More information

SGN Advanced Signal Processing Project bonus: Sparse model estimation

SGN Advanced Signal Processing Project bonus: Sparse model estimation SGN 21006 Advanced Signal Processing Project bonus: Sparse model estimation Ioan Tabus Department of Signal Processing Tampere University of Technology Finland 1 / 12 Sparse models Initial problem: solve

More information

ECEN 615 Methods of Electric Power Systems Analysis Lecture 18: Least Squares, State Estimation

ECEN 615 Methods of Electric Power Systems Analysis Lecture 18: Least Squares, State Estimation ECEN 615 Methods of Electric Power Systems Analysis Lecture 18: Least Squares, State Estimation Prof. om Overbye Dept. of Electrical and Computer Engineering exas A&M University overbye@tamu.edu Announcements

More information

Sparse PCA with applications in finance

Sparse PCA with applications in finance Sparse PCA with applications in finance A. d Aspremont, L. El Ghaoui, M. Jordan, G. Lanckriet ORFE, Princeton University & EECS, U.C. Berkeley Available online at www.princeton.edu/~aspremon 1 Introduction

More information

Network Localization via Schatten Quasi-Norm Minimization

Network Localization via Schatten Quasi-Norm Minimization Network Localization via Schatten Quasi-Norm Minimization Anthony Man-Cho So Department of Systems Engineering & Engineering Management The Chinese University of Hong Kong (Joint Work with Senshan Ji Kam-Fung

More information

A Chebyshev-based two-stage iterative method as an alternative to the direct solution of linear systems

A Chebyshev-based two-stage iterative method as an alternative to the direct solution of linear systems A Chebyshev-based two-stage iterative method as an alternative to the direct solution of linear systems Mario Arioli m.arioli@rl.ac.uk CCLRC-Rutherford Appleton Laboratory with Daniel Ruiz (E.N.S.E.E.I.H.T)

More information

Reconstruction of Block-Sparse Signals by Using an l 2/p -Regularized Least-Squares Algorithm

Reconstruction of Block-Sparse Signals by Using an l 2/p -Regularized Least-Squares Algorithm Reconstruction of Block-Sparse Signals by Using an l 2/p -Regularized Least-Squares Algorithm Jeevan K. Pant, Wu-Sheng Lu, and Andreas Antoniou University of Victoria May 21, 2012 Compressive Sensing 1/23

More information

A DESIGN OF FIR FILTERS WITH VARIABLE NOTCHES CONSIDERING REDUCTION METHOD OF POLYNOMIAL COEFFICIENTS FOR REAL-TIME SIGNAL PROCESSING

A DESIGN OF FIR FILTERS WITH VARIABLE NOTCHES CONSIDERING REDUCTION METHOD OF POLYNOMIAL COEFFICIENTS FOR REAL-TIME SIGNAL PROCESSING International Journal of Innovative Computing, Information and Control ICIC International c 23 ISSN 349-498 Volume 9, Number 9, September 23 pp. 3527 3536 A DESIGN OF FIR FILTERS WITH VARIABLE NOTCHES

More information

Key words. conjugate gradients, normwise backward error, incremental norm estimation.

Key words. conjugate gradients, normwise backward error, incremental norm estimation. Proceedings of ALGORITMY 2016 pp. 323 332 ON ERROR ESTIMATION IN THE CONJUGATE GRADIENT METHOD: NORMWISE BACKWARD ERROR PETR TICHÝ Abstract. Using an idea of Duff and Vömel [BIT, 42 (2002), pp. 300 322

More information

9. Numerical linear algebra background

9. Numerical linear algebra background Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

More information

Denosing Using Wavelets and Projections onto the l 1 -Ball

Denosing Using Wavelets and Projections onto the l 1 -Ball 1 Denosing Using Wavelets and Projections onto the l 1 -Ball October 6, 2014 A. Enis Cetin, M. Tofighi Dept. of Electrical and Electronic Engineering, Bilkent University, Ankara, Turkey cetin@bilkent.edu.tr,

More information

Reluctance/Inductance Matrix under

Reluctance/Inductance Matrix under Generating Stable and Sparse Reluctance/Inductance Matrix under Insufficient Conditions Y. Tanji, Kagawa University, Japan T. Watanabe, The University it of Shizuoka, Japan H. Asai, Shizuoka University,

More information

Chapter 3 Salient Feature Inference

Chapter 3 Salient Feature Inference Chapter 3 Salient Feature Inference he building block of our computational framework for inferring salient structure is the procedure that simultaneously interpolates smooth curves, or surfaces, or region

More information

LU Factorization. LU Decomposition. LU Decomposition. LU Decomposition: Motivation A = LU

LU Factorization. LU Decomposition. LU Decomposition. LU Decomposition: Motivation A = LU LU Factorization To further improve the efficiency of solving linear systems Factorizations of matrix A : LU and QR LU Factorization Methods: Using basic Gaussian Elimination (GE) Factorization of Tridiagonal

More information

Proximal Gradient Descent and Acceleration. Ryan Tibshirani Convex Optimization /36-725

Proximal Gradient Descent and Acceleration. Ryan Tibshirani Convex Optimization /36-725 Proximal Gradient Descent and Acceleration Ryan Tibshirani Convex Optimization 10-725/36-725 Last time: subgradient method Consider the problem min f(x) with f convex, and dom(f) = R n. Subgradient method:

More information

Design of Fractional Delay Filter Using Hermite Interpolation Method

Design of Fractional Delay Filter Using Hermite Interpolation Method 1458 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: REGULAR PAPERS, VOL. 59, NO. 7, JULY 2012 Design of Fractional Delay Filter Using Hermite Interpolation Method Chien-Cheng Tseng, Senior Member, IEEE,

More information

Digital Signal Processing 2/ Advanced Digital Signal Processing, Audio/Video Signal Processing Lecture 10, Frequency Warping, Example

Digital Signal Processing 2/ Advanced Digital Signal Processing, Audio/Video Signal Processing Lecture 10, Frequency Warping, Example Digital Signal Processing 2/ Advanced Digital Signal Processing, Audio/Video Signal Processing Lecture 10, Gerald Schuller, TU Ilmenau Frequency Warping, Example Example: Design a warped low pass filter

More information

Supplementary Materials to Learning Sparse Causal Gaussian Networks With Experimental Intervention: Regularization and Coordinate Descent

Supplementary Materials to Learning Sparse Causal Gaussian Networks With Experimental Intervention: Regularization and Coordinate Descent Supplementary Materials to Learning Sparse Causal Gaussian Networks With Experimental Intervention: Regularization and Coordinate Descent Fei Fu and Qing Zhou 1 Proofs 1.1 Proof of Proposition 1 [ ( We

More information

Gaussian Elimination for Linear Systems

Gaussian Elimination for Linear Systems Gaussian Elimination for Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 3, 2011 1/56 Outline 1 Elementary matrices 2 LR-factorization 3 Gaussian elimination

More information

Analog and Digital Filter Design

Analog and Digital Filter Design Analog and Digital Filter Design by Jens Hee http://jenshee.dk October 208 Change log 28. september 208. Document started.. october 208. Figures added. 6. october 208. Bilinear transform chapter extended.

More information

SOR- and Jacobi-type Iterative Methods for Solving l 1 -l 2 Problems by Way of Fenchel Duality 1

SOR- and Jacobi-type Iterative Methods for Solving l 1 -l 2 Problems by Way of Fenchel Duality 1 SOR- and Jacobi-type Iterative Methods for Solving l 1 -l 2 Problems by Way of Fenchel Duality 1 Masao Fukushima 2 July 17 2010; revised February 4 2011 Abstract We present an SOR-type algorithm and a

More information

Approximate Message Passing Algorithms

Approximate Message Passing Algorithms November 4, 2017 Outline AMP (Donoho et al., 2009, 2010a) Motivations Derivations from a message-passing perspective Limitations Extensions Generalized Approximate Message Passing (GAMP) (Rangan, 2011)

More information

PS403 - Digital Signal processing

PS403 - Digital Signal processing PS403 - Digital Signal processing 6. DSP - Recursive (IIR) Digital Filters Key Text: Digital Signal Processing with Computer Applications (2 nd Ed.) Paul A Lynn and Wolfgang Fuerst, (Publisher: John Wiley

More information

(Refer Slide Time: 01:28 03:51 min)

(Refer Slide Time: 01:28 03:51 min) Digital Signal Processing Prof. S. C. Dutta Roy Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture 40 FIR Design by Windowing This is the 40 th lecture and our topic for

More information

Lecture 1: September 25

Lecture 1: September 25 0-725: Optimization Fall 202 Lecture : September 25 Lecturer: Geoff Gordon/Ryan Tibshirani Scribes: Subhodeep Moitra Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have

More information

Maximally Flat Lowpass Digital Differentiators

Maximally Flat Lowpass Digital Differentiators Maximally Flat Lowpass Digital Differentiators Ivan W. Selesnick August 3, 00 Electrical Engineering, Polytechnic University 6 Metrotech Center, Brooklyn, NY 0 selesi@taco.poly.edu tel: 78 60-36 fax: 78

More information

Electronic Circuits EE359A

Electronic Circuits EE359A Electronic Circuits EE359A Bruce McNair B26 bmcnair@stevens.edu 21-216-5549 Lecture 22 569 Second order section Ts () = s as + as+ a 2 2 1 ω + s+ ω Q 2 2 ω 1 p, p = ± 1 Q 4 Q 1 2 2 57 Second order section

More information

Lecture Notes 10: Matrix Factorization

Lecture Notes 10: Matrix Factorization Optimization-based data analysis Fall 207 Lecture Notes 0: Matrix Factorization Low-rank models. Rank- model Consider the problem of modeling a quantity y[i, j] that depends on two indices i and j. To

More information

Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013

Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 Machine Learning for Signal Processing Sparse and Overcomplete Representations Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 1 Key Topics in this Lecture Basics Component-based representations

More information

Iterative Methods. Splitting Methods

Iterative Methods. Splitting Methods Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition

More information

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline

More information

EE482: Digital Signal Processing Applications

EE482: Digital Signal Processing Applications Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 05 IIR Design 14/03/04 http://www.ee.unlv.edu/~b1morris/ee482/

More information

Dual Proximal Gradient Method

Dual Proximal Gradient Method Dual Proximal Gradient Method http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/19 1 proximal gradient method

More information

Robust Sparse Recovery via Non-Convex Optimization

Robust Sparse Recovery via Non-Convex Optimization Robust Sparse Recovery via Non-Convex Optimization Laming Chen and Yuantao Gu Department of Electronic Engineering, Tsinghua University Homepage: http://gu.ee.tsinghua.edu.cn/ Email: gyt@tsinghua.edu.cn

More information

2.161 Signal Processing: Continuous and Discrete Fall 2008

2.161 Signal Processing: Continuous and Discrete Fall 2008 IT OpenCourseWare http://ocw.mit.edu 2.161 Signal Processing: Continuous and Discrete all 2008 or information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. assachusetts

More information

Proximal Newton Method. Zico Kolter (notes by Ryan Tibshirani) Convex Optimization

Proximal Newton Method. Zico Kolter (notes by Ryan Tibshirani) Convex Optimization Proximal Newton Method Zico Kolter (notes by Ryan Tibshirani) Convex Optimization 10-725 Consider the problem Last time: quasi-newton methods min x f(x) with f convex, twice differentiable, dom(f) = R

More information

ECE 410 DIGITAL SIGNAL PROCESSING D. Munson University of Illinois Chapter 12

ECE 410 DIGITAL SIGNAL PROCESSING D. Munson University of Illinois Chapter 12 . ECE 40 DIGITAL SIGNAL PROCESSING D. Munson University of Illinois Chapter IIR Filter Design ) Based on Analog Prototype a) Impulse invariant design b) Bilinear transformation ( ) ~ widely used ) Computer-Aided

More information

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear

More information

9. Numerical linear algebra background

9. Numerical linear algebra background Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

More information

Problem structure in semidefinite programs arising in control and signal processing

Problem structure in semidefinite programs arising in control and signal processing Problem structure in semidefinite programs arising in control and signal processing Lieven Vandenberghe Electrical Engineering Department, UCLA Joint work with: Mehrdad Nouralishahi, Tae Roh Semidefinite

More information

Computer-Aided Design of Digital Filters. Digital Filters. Digital Filters. Digital Filters. Design of Equiripple Linear-Phase FIR Filters

Computer-Aided Design of Digital Filters. Digital Filters. Digital Filters. Digital Filters. Design of Equiripple Linear-Phase FIR Filters Computer-Aided Design of Digital Filters The FIR filter design techniques discussed so far can be easily implemented on a computer In addition, there are a number of FIR filter design algorithms that rely

More information

Vasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks

Vasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks C.M. Bishop s PRML: Chapter 5; Neural Networks Introduction The aim is, as before, to find useful decompositions of the target variable; t(x) = y(x, w) + ɛ(x) (3.7) t(x n ) and x n are the observations,

More information

Multirate signal processing

Multirate signal processing Multirate signal processing Discrete-time systems with different sampling rates at various parts of the system are called multirate systems. The need for such systems arises in many applications, including

More information

Sparsity of Matrix Canonical Forms. Xingzhi Zhan East China Normal University

Sparsity of Matrix Canonical Forms. Xingzhi Zhan East China Normal University Sparsity of Matrix Canonical Forms Xingzhi Zhan zhan@math.ecnu.edu.cn East China Normal University I. Extremal sparsity of the companion matrix of a polynomial Joint work with Chao Ma The companion matrix

More information

Optimization methods

Optimization methods Lecture notes 3 February 8, 016 1 Introduction Optimization methods In these notes we provide an overview of a selection of optimization methods. We focus on methods which rely on first-order information,

More information

Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization

Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization Rapid, Robust, and Reliable Blind Deconvolution via Nonconvex Optimization Shuyang Ling Department of Mathematics, UC Davis Oct.18th, 2016 Shuyang Ling (UC Davis) 16w5136, Oaxaca, Mexico Oct.18th, 2016

More information

Machine Learning for Signal Processing Sparse and Overcomplete Representations

Machine Learning for Signal Processing Sparse and Overcomplete Representations Machine Learning for Signal Processing Sparse and Overcomplete Representations Abelino Jimenez (slides from Bhiksha Raj and Sourish Chaudhuri) Oct 1, 217 1 So far Weights Data Basis Data Independent ICA

More information

Methods for sparse analysis of high-dimensional data, II

Methods for sparse analysis of high-dimensional data, II Methods for sparse analysis of high-dimensional data, II Rachel Ward May 26, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 55 High dimensional

More information

MATH 425-Spring 2010 HOMEWORK ASSIGNMENTS

MATH 425-Spring 2010 HOMEWORK ASSIGNMENTS MATH 425-Spring 2010 HOMEWORK ASSIGNMENTS Instructor: Shmuel Friedland Department of Mathematics, Statistics and Computer Science email: friedlan@uic.edu Last update April 18, 2010 1 HOMEWORK ASSIGNMENT

More information