Optimization transfer approach to joint registration / reconstruction for motion-compensated image reconstruction

Size: px
Start display at page:

Download "Optimization transfer approach to joint registration / reconstruction for motion-compensated image reconstruction"

Transcription

1 Optimization transfer approach to joint registration / reconstruction for motion-compensated image reconstruction Jeffrey A. Fessler EECS Dept. University of Michigan ISBI Apr. 5, 00 0

2 Introduction Image reconstruction of moving objects with unknown motion Joint estimation of motion parameters and object Jacobson et al., IEEE NSS 003 Taguchi et al. SPIE 007 Odille et al. MRM Jul. 008 Also super-resolution problems with unknown motion... (cf starting with low-resolution images vs starting with sinograms or k-space) Computational challenge: motion operator in forward model. We use optimization transfer to put motion estimation step in image domain

3 M frames (defined generally) Measurement model y m = A m x m + ε m, m =,...,M y m measured data for mth frame A m system matrix for mth frame x m unknown image for mth frame ε m measurement noise for mth frame Nominal goal: reconstruct image frames {x m } from measured data {y m }.

4 Object model Assume each frame is a spatial transformation of one base image: x m = T(α m ) c α m motion parameters for mth frame T( ) nonrigid warp operator c base image coefficient vector (e.g., for B-splines) x 0 = T(0) c Motion-compensated image reconstruction goal: reconstruct base image coefficients c and motion parameters {α m } from measured data {y m }. 3

5 Joint registration/reconstruction Combined measurement model / object model: y m = A m T(α m ) c+ ε m, m =,...,M Stacking, where α (α,..., α M ): y y. y M A A... y = A T(α) c+ ε A M T(α) T(α ). T(α M ) ε ε. ε M Penalized weighted least-squares (PWLS) estimation: (ĉc, ˆα) = argmin c,α Ψ(c, α) Ψ(c, α) = y A T(α) c W + R (c)+r (α) 4

6 Optimization by alternation Initialize base image c 0 and motion parameters α 0. Alternating updates: c n+ = argmin c α n+ = argmin α Ψ(α n, c) Ψ ( α, c n+) Update c using standard image reconstruction methods: c n+ = argmin c y A T(αn ) c W + R (c) Updating motion parameters α is challenging: α n+ = argmin α ψ(α), ψ(α) = y AT(α)c n+ W + R (α) 5

7 Optimization transfer / Majorize-minimize Alternative to minimizing cost function ψ(α) directly: S-step: find a surrogate function (majorizer) φ(α; α n ) s.t. φ(α; α n ) ψ(α), φ(α n ; α n ) = ψ(α n ) M-step: minimize surrogate function: α n+ = argmin α φ(α; α n ). α Guaranteed to decrease cost function ψ(α n ) monotonically. 6

8 Optimization Transfer Illustrated Ψ(x) and φ(x, x n ) Surrogate function Cost function x n x n+ 7

9 Optimization Transfer in d 8

10 Separable quadratic surrogate for WLS L(x) = y Ax W = i w i (y i [Ax] i ) = L(x n )+(x x n ) g n + (x xn ) A W A(x x n ) L(x n )+(x x n ) g n + (x xn ) D(x x n ) D /[ x ( x n + D g n)] Q(x; x n ), when D is any matrix that satisfies D A W A, where g n L(x n ) = A W (y Ax n ). Useful choice (Erdoğan and Fessler, PMB 999): ( D = diag{d j }, d j i w i a i j k a ik ). 9

11 Surrogate for motion parameters Recall: ψ(α) = y A } T(α)c {{ n+ } + R (α) W x Using result for WLS, the following majorizes ψ: φ(α, α n ) Q(T(α) c n, T(α n ) c n )+R (α) = D /[ T(α) c n ( T(α n ) c n + D g n)] + R (α), where the gradient depends on previous estimates: g n L(x n ) = A W (y A T(α n ) c n ). M-step is entirely image-domain operations: α n+ = argmin α φ(α, α n ). (Guobao Wang & Jinyi Qi did something similar for PET kinetics in ISBI 008) 0

12 Simulation Example MRI with randomly ordered k y,k z phase encodes True Object

13 Simulation Example K space data for 3 frame (log scale) Zero fill IFFT reconstructions, and their average True motion: horizontal translation: -, 4, 0

14 Joint motion estimation / reconstruction Joint estimate: Oracle reconstruction (3 alternations, 9 CG) ( known motion) Joint estimate using Optimization Transfer Oracle estimate using true motion Estimated horizontal translation: -.4, 4.4, 0.4 True horizontal translation: -, 4, 0 3

15 Discussion Proposed optimization transfer approach to joint image reconstruction and motion estimation Puts motion estimation step in image domain akin to image registration avoids expensive forward projections in registration step promotes software modularity (e.g., importance sampling, Bhagalia et al., IEEE T-MI, Aug. 009) Optimization transfer approach may require more iterations Usual nonconvexity issues (multi-resolution...) Generalizes readily to non-quadratic log-likelihood functions (e.g., PET, polyenergetic CT,...) Awaits evaluation with real data... Awaits comparisons with nonlinear CG Open problem: using ordered-subsets approaches efficiently 4

Reconstruction from Digital Holograms by Statistical Methods

Reconstruction from Digital Holograms by Statistical Methods Reconstruction from Digital Holograms by Statistical Methods Saowapak Sotthivirat Jeffrey A. Fessler EECS Department The University of Michigan 2003 Asilomar Nov. 12, 2003 Acknowledgements: Brian Athey,

More information

Part 3. Algorithms. Method = Cost Function + Algorithm

Part 3. Algorithms. Method = Cost Function + Algorithm Part 3. Algorithms Method = Cost Function + Algorithm Outline Ideal algorithm Classical general-purpose algorithms Considerations: nonnegativity parallelization convergence rate monotonicity Algorithms

More information

Noise properties of motion-compensated tomographic image reconstruction methods

Noise properties of motion-compensated tomographic image reconstruction methods IEEE TRANSACTIONS ON MEDICAL IMAGING, ACCEPTED FOR PUBLICATION 1 Noise properties of motion-compensated tomographic image reconstruction methods Se Young Chun, Member, IEEE, and Jeffrey A. Fessler, Fellow,

More information

A Study of Numerical Algorithms for Regularized Poisson ML Image Reconstruction

A Study of Numerical Algorithms for Regularized Poisson ML Image Reconstruction A Study of Numerical Algorithms for Regularized Poisson ML Image Reconstruction Yao Xie Project Report for EE 391 Stanford University, Summer 2006-07 September 1, 2007 Abstract In this report we solved

More information

Scan Time Optimization for Post-injection PET Scans

Scan Time Optimization for Post-injection PET Scans Presented at 998 IEEE Nuc. Sci. Symp. Med. Im. Conf. Scan Time Optimization for Post-injection PET Scans Hakan Erdoğan Jeffrey A. Fessler 445 EECS Bldg., University of Michigan, Ann Arbor, MI 4809-222

More information

Optimized first-order minimization methods

Optimized first-order minimization methods Optimized first-order minimization methods Donghwan Kim & Jeffrey A. Fessler EECS Dept., BME Dept., Dept. of Radiology University of Michigan web.eecs.umich.edu/~fessler UM AIM Seminar 2014-10-03 1 Disclosure

More information

Two-Material Decomposition From a Single CT Scan Using Statistical Image Reconstruction

Two-Material Decomposition From a Single CT Scan Using Statistical Image Reconstruction / 5 Two-Material Decomposition From a Single CT Scan Using Statistical Image Reconstruction Yong Long and Jeffrey A. Fessler EECS Department James M. Balter Radiation Oncology Department The University

More information

Relaxed linearized algorithms for faster X-ray CT image reconstruction

Relaxed linearized algorithms for faster X-ray CT image reconstruction Relaxed linearized algorithms for faster X-ray CT image reconstruction Hung Nien and Jeffrey A. Fessler University of Michigan, Ann Arbor The 13th Fully 3D Meeting June 2, 2015 1/20 Statistical image reconstruction

More information

Objective Functions for Tomographic Reconstruction from. Randoms-Precorrected PET Scans. gram separately, this process doubles the storage space for

Objective Functions for Tomographic Reconstruction from. Randoms-Precorrected PET Scans. gram separately, this process doubles the storage space for Objective Functions for Tomographic Reconstruction from Randoms-Precorrected PET Scans Mehmet Yavuz and Jerey A. Fessler Dept. of EECS, University of Michigan Abstract In PET, usually the data are precorrected

More information

Sparsity Regularization

Sparsity Regularization Sparsity Regularization Bangti Jin Course Inverse Problems & Imaging 1 / 41 Outline 1 Motivation: sparsity? 2 Mathematical preliminaries 3 l 1 solvers 2 / 41 problem setup finite-dimensional formulation

More information

Analytical Approach to Regularization Design for Isotropic Spatial Resolution

Analytical Approach to Regularization Design for Isotropic Spatial Resolution Analytical Approach to Regularization Design for Isotropic Spatial Resolution Jeffrey A Fessler, Senior Member, IEEE Abstract In emission tomography, conventional quadratic regularization methods lead

More information

Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems)

Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Donghwan Kim and Jeffrey A. Fessler EECS Department, University of Michigan

More information

Sparse Regularization via Convex Analysis

Sparse Regularization via Convex Analysis Sparse Regularization via Convex Analysis Ivan Selesnick Electrical and Computer Engineering Tandon School of Engineering New York University Brooklyn, New York, USA 29 / 66 Convex or non-convex: Which

More information

Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors

Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors Sean Borman and Robert L. Stevenson Department of Electrical Engineering, University of Notre Dame Notre Dame,

More information

A Computational Framework for Total Variation-Regularized Positron Emission Tomography

A Computational Framework for Total Variation-Regularized Positron Emission Tomography Numerical Algorithms manuscript No. (will be inserted by the editor) A Computational Framework for Total Variation-Regularized Positron Emission Tomography Johnathan M. Bardsley John Goldes Received: date

More information

Variable Metric Forward-Backward Algorithm

Variable Metric Forward-Backward Algorithm Variable Metric Forward-Backward Algorithm 1/37 Variable Metric Forward-Backward Algorithm for minimizing the sum of a differentiable function and a convex function E. Chouzenoux in collaboration with

More information

Recovering overcomplete sparse representations from structured sensing

Recovering overcomplete sparse representations from structured sensing Recovering overcomplete sparse representations from structured sensing Deanna Needell Claremont McKenna College Feb. 2015 Support: Alfred P. Sloan Foundation and NSF CAREER #1348721. Joint work with Felix

More information

Strengthened Sobolev inequalities for a random subspace of functions

Strengthened Sobolev inequalities for a random subspace of functions Strengthened Sobolev inequalities for a random subspace of functions Rachel Ward University of Texas at Austin April 2013 2 Discrete Sobolev inequalities Proposition (Sobolev inequality for discrete images)

More information

Optimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30

Optimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained

More information

4F3 - Predictive Control

4F3 - Predictive Control 4F3 Predictive Control - Lecture 2 p 1/23 4F3 - Predictive Control Lecture 2 - Unconstrained Predictive Control Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 2 p 2/23 References Predictive

More information

Optimization by General-Purpose Methods

Optimization by General-Purpose Methods c J. Fessler. January 25, 2015 11.1 Chapter 11 Optimization by General-Purpose Methods ch,opt Contents 11.1 Introduction (s,opt,intro)...................................... 11.3 11.1.1 Iterative optimization

More information

A Majorize-Minimize subspace approach for l 2 -l 0 regularization with applications to image processing

A Majorize-Minimize subspace approach for l 2 -l 0 regularization with applications to image processing A Majorize-Minimize subspace approach for l 2 -l 0 regularization with applications to image processing Emilie Chouzenoux emilie.chouzenoux@univ-mlv.fr Université Paris-Est Lab. d Informatique Gaspard

More information

Inverse problem and optimization

Inverse problem and optimization Inverse problem and optimization Laurent Condat, Nelly Pustelnik CNRS, Gipsa-lab CNRS, Laboratoire de Physique de l ENS de Lyon Decembre, 15th 2016 Inverse problem and optimization 2/36 Plan 1. Examples

More information

Information geometry for bivariate distribution control

Information geometry for bivariate distribution control Information geometry for bivariate distribution control C.T.J.Dodson + Hong Wang Mathematics + Control Systems Centre, University of Manchester Institute of Science and Technology Optimal control of stochastic

More information

Linear & nonlinear classifiers

Linear & nonlinear classifiers Linear & nonlinear classifiers Machine Learning Hamid Beigy Sharif University of Technology Fall 1396 Hamid Beigy (Sharif University of Technology) Linear & nonlinear classifiers Fall 1396 1 / 44 Table

More information

Lecture 6: CS395T Numerical Optimization for Graphics and AI Line Search Applications

Lecture 6: CS395T Numerical Optimization for Graphics and AI Line Search Applications Lecture 6: CS395T Numerical Optimization for Graphics and AI Line Search Applications Qixing Huang The University of Texas at Austin huangqx@cs.utexas.edu 1 Disclaimer This note is adapted from Section

More information

Course Notes: Week 4

Course Notes: Week 4 Course Notes: Week 4 Math 270C: Applied Numerical Linear Algebra 1 Lecture 9: Steepest Descent (4/18/11) The connection with Lanczos iteration and the CG was not originally known. CG was originally derived

More information

CSC321 Lecture 9: Generalization

CSC321 Lecture 9: Generalization CSC321 Lecture 9: Generalization Roger Grosse Roger Grosse CSC321 Lecture 9: Generalization 1 / 26 Overview We ve focused so far on how to optimize neural nets how to get them to make good predictions

More information

Super-Resolution. Dr. Yossi Rubner. Many slides from Miki Elad - Technion

Super-Resolution. Dr. Yossi Rubner. Many slides from Miki Elad - Technion Super-Resolution Dr. Yossi Rubner yossi@rubner.co.il Many slides from Mii Elad - Technion 5/5/2007 53 images, ratio :4 Example - Video 40 images ratio :4 Example Surveillance Example Enhance Mosaics Super-Resolution

More information

A memory gradient algorithm for l 2 -l 0 regularization with applications to image restoration

A memory gradient algorithm for l 2 -l 0 regularization with applications to image restoration A memory gradient algorithm for l 2 -l 0 regularization with applications to image restoration E. Chouzenoux, A. Jezierska, J.-C. Pesquet and H. Talbot Université Paris-Est Lab. d Informatique Gaspard

More information

AN NONNEGATIVELY CONSTRAINED ITERATIVE METHOD FOR POSITRON EMISSION TOMOGRAPHY. Johnathan M. Bardsley

AN NONNEGATIVELY CONSTRAINED ITERATIVE METHOD FOR POSITRON EMISSION TOMOGRAPHY. Johnathan M. Bardsley Volume X, No. 0X, 0X, X XX Web site: http://www.aimsciences.org AN NONNEGATIVELY CONSTRAINED ITERATIVE METHOD FOR POSITRON EMISSION TOMOGRAPHY Johnathan M. Bardsley Department of Mathematical Sciences

More information

Lecture 5: September 12

Lecture 5: September 12 10-725/36-725: Convex Optimization Fall 2015 Lecture 5: September 12 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Barun Patra and Tyler Vuong Note: LaTeX template courtesy of UC Berkeley EECS

More information

Sparsity Regularization for Image Reconstruction with Poisson Data

Sparsity Regularization for Image Reconstruction with Poisson Data Sparsity Regularization for Image Reconstruction with Poisson Data Daniel J. Lingenfelter a, Jeffrey A. Fessler a,andzhonghe b a Electrical Engineering and Computer Science, University of Michigan, Ann

More information

Regularizing inverse problems using sparsity-based signal models

Regularizing inverse problems using sparsity-based signal models Regularizing inverse problems using sparsity-based signal models Jeffrey A. Fessler William L. Root Professor of EECS EECS Dept., BME Dept., Dept. of Radiology University of Michigan http://web.eecs.umich.edu/

More information

Iterative regularization of nonlinear ill-posed problems in Banach space

Iterative regularization of nonlinear ill-posed problems in Banach space Iterative regularization of nonlinear ill-posed problems in Banach space Barbara Kaltenbacher, University of Klagenfurt joint work with Bernd Hofmann, Technical University of Chemnitz, Frank Schöpfer and

More information

CSC321 Lecture 9: Generalization

CSC321 Lecture 9: Generalization CSC321 Lecture 9: Generalization Roger Grosse Roger Grosse CSC321 Lecture 9: Generalization 1 / 27 Overview We ve focused so far on how to optimize neural nets how to get them to make good predictions

More information

System Modeling for Gamma Ray Imaging Systems

System Modeling for Gamma Ray Imaging Systems System Modeling for Gamma Ray Imaging Systems Daniel J. Lingenfelter and Jeffrey A. Fessler COMMUNICATIONS & SIGNAL PROCESSING LABORATORY Department of Electrical Engineering and Computer Science The University

More information

Seismic imaging and optimal transport

Seismic imaging and optimal transport Seismic imaging and optimal transport Bjorn Engquist In collaboration with Brittany Froese, Sergey Fomel and Yunan Yang Brenier60, Calculus of Variations and Optimal Transportation, Paris, January 10-13,

More information

Conjugate Gradient algorithm. Storage: fixed, independent of number of steps.

Conjugate Gradient algorithm. Storage: fixed, independent of number of steps. Conjugate Gradient algorithm Need: A symmetric positive definite; Cost: 1 matrix-vector product per step; Storage: fixed, independent of number of steps. The CG method minimizes the A norm of the error,

More information

Motion Estimation (I) Ce Liu Microsoft Research New England

Motion Estimation (I) Ce Liu Microsoft Research New England Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion

More information

Advanced computational methods X Selected Topics: SGD

Advanced computational methods X Selected Topics: SGD Advanced computational methods X071521-Selected Topics: SGD. In this lecture, we look at the stochastic gradient descent (SGD) method 1 An illustrating example The MNIST is a simple dataset of variety

More information

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu

More information

We consider the problem of finding a polynomial that interpolates a given set of values:

We consider the problem of finding a polynomial that interpolates a given set of values: Chapter 5 Interpolation 5. Polynomial Interpolation We consider the problem of finding a polynomial that interpolates a given set of values: x x 0 x... x n y y 0 y... y n where the x i are all distinct.

More information

Physics-based Prior modeling in Inverse Problems

Physics-based Prior modeling in Inverse Problems Physics-based Prior modeling in Inverse Problems MURI Meeting 2013 M Usman Sadiq, Purdue University Charles A. Bouman, Purdue University In collaboration with: Jeff Simmons, AFRL Venkat Venkatakrishnan,

More information

Design of Optimal Quantizers for Distributed Source Coding

Design of Optimal Quantizers for Distributed Source Coding Design of Optimal Quantizers for Distributed Source Coding David Rebollo-Monedero, Rui Zhang and Bernd Girod Information Systems Laboratory, Electrical Eng. Dept. Stanford University, Stanford, CA 94305

More information

ANALYSIS OF p-norm REGULARIZED SUBPROBLEM MINIMIZATION FOR SPARSE PHOTON-LIMITED IMAGE RECOVERY

ANALYSIS OF p-norm REGULARIZED SUBPROBLEM MINIMIZATION FOR SPARSE PHOTON-LIMITED IMAGE RECOVERY ANALYSIS OF p-norm REGULARIZED SUBPROBLEM MINIMIZATION FOR SPARSE PHOTON-LIMITED IMAGE RECOVERY Aramayis Orkusyan, Lasith Adhikari, Joanna Valenzuela, and Roummel F. Marcia Department o Mathematics, Caliornia

More information

SPATIAL RESOLUTION PROPERTIES OF PENALIZED WEIGHTED LEAST-SQUARES TOMOGRAPHIC IMAGE RECONSTRUCTION WITH MODEL MISMATCH

SPATIAL RESOLUTION PROPERTIES OF PENALIZED WEIGHTED LEAST-SQUARES TOMOGRAPHIC IMAGE RECONSTRUCTION WITH MODEL MISMATCH SPATIAL RESOLUTION PROPERTIES OF PENALIZED WEIGHTED LEAST-SQUARES TOMOGRAPHIC IMAGE RECONSTRUCTION WITH MODEL MISMATCH Jeffrey A. Fessler COMMUNICATIONS & SIGNAL PROCESSING LABORATORY Department of Electrical

More information

Lecture 6 : Projected Gradient Descent

Lecture 6 : Projected Gradient Descent Lecture 6 : Projected Gradient Descent EE227C. Lecturer: Professor Martin Wainwright. Scribe: Alvin Wan Consider the following update. x l+1 = Π C (x l α f(x l )) Theorem Say f : R d R is (m, M)-strongly

More information

Statistics 203: Introduction to Regression and Analysis of Variance Course review

Statistics 203: Introduction to Regression and Analysis of Variance Course review Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying

More information

A New Trust Region Algorithm Using Radial Basis Function Models

A New Trust Region Algorithm Using Radial Basis Function Models A New Trust Region Algorithm Using Radial Basis Function Models Seppo Pulkkinen University of Turku Department of Mathematics July 14, 2010 Outline 1 Introduction 2 Background Taylor series approximations

More information

SYSTEMS OF NONLINEAR EQUATIONS

SYSTEMS OF NONLINEAR EQUATIONS SYSTEMS OF NONLINEAR EQUATIONS Widely used in the mathematical modeling of real world phenomena. We introduce some numerical methods for their solution. For better intuition, we examine systems of two

More information

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 9. Alternating Direction Method of Multipliers

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 9. Alternating Direction Method of Multipliers Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 9 Alternating Direction Method of Multipliers Shiqian Ma, MAT-258A: Numerical Optimization 2 Separable convex optimization a special case is min f(x)

More information

Motion Estimation (I)

Motion Estimation (I) Motion Estimation (I) Ce Liu celiu@microsoft.com Microsoft Research New England We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion

More information

Statistical Methods for Image Reconstruction

Statistical Methods for Image Reconstruction Statistical Methods for Image Reconstruction Jeffrey A. Fessler EECS Department The University of Michigan Johns Hopkins University: Short Course May 11, 2007 0.0 Image Reconstruction Methods (Simplified

More information

Statistical Methods for Image Reconstruction

Statistical Methods for Image Reconstruction Statistical Methods for Image Reconstruction Jeffrey A. Fessler EECS Department The University of Michigan NSS-MIC Oct. 19, 2004 0.0 Image Reconstruction Methods (Simplified View) Analytical (FBP) Iterative

More information

Stability Analysis of Discrete time Recurrent Neural Networks

Stability Analysis of Discrete time Recurrent Neural Networks Stability Analysis of Discrete time Recurrent Neural Networks Department of Mathematics North Dakota State University Fargo,ND November 8, 2014 2014 AMS Fall Southeastern Sectional Meeting University of

More information

Reconstruction from Fourier Samples (Gridding and alternatives)

Reconstruction from Fourier Samples (Gridding and alternatives) Chapter 6 Reconstruction from Fourier Samples (Gridding and alternatives) ch,four The customer of sampling and reconstruction technology faces a large gap between the practice of (non-uniform) sampling

More information

Constructing Approximation Kernels for Non-Harmonic Fourier Data

Constructing Approximation Kernels for Non-Harmonic Fourier Data Constructing Approximation Kernels for Non-Harmonic Fourier Data Aditya Viswanathan aditya.v@caltech.edu California Institute of Technology SIAM Annual Meeting 2013 July 10 2013 0 / 19 Joint work with

More information

Scan Time Optimization for Post-injection PET Scans

Scan Time Optimization for Post-injection PET Scans Scan Time Optimization for Post-injection PET Scans Hakan Erdogan and Jeffrey. Fessler 445 EECS Bldg., University of Michigan, nn rbor, MI 4809-222 bstract Previous methods for optimizing the scan times

More information

Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix

Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen, Denmark, May 2002 Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix René Vidal Dept of EECS, UC Berkeley Berkeley,

More information

An Introduction to Sparse Approximation

An Introduction to Sparse Approximation An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,

More information

Composite nonlinear models at scale

Composite nonlinear models at scale Composite nonlinear models at scale Dmitriy Drusvyatskiy Mathematics, University of Washington Joint work with D. Davis (Cornell), M. Fazel (UW), A.S. Lewis (Cornell) C. Paquette (Lehigh), and S. Roy (UW)

More information

Generalized Orthogonal Matching Pursuit- A Review and Some

Generalized Orthogonal Matching Pursuit- A Review and Some Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents

More information

Properties of MM Algorithms on Convex Feasible Sets: Extended Version

Properties of MM Algorithms on Convex Feasible Sets: Extended Version Properties of MM Algorithms on Convex Feasible Sets: Extended Version Mattthew W. Jacobson Jeffrey A. Fessler November 30, 2004 Abstract We examine some properties of the Majorize-Minimize (MM) optimization

More information

DNNs for Sparse Coding and Dictionary Learning

DNNs for Sparse Coding and Dictionary Learning DNNs for Sparse Coding and Dictionary Learning Subhadip Mukherjee, Debabrata Mahapatra, and Chandra Sekhar Seelamantula Department of Electrical Engineering, Indian Institute of Science, Bangalore 5612,

More information

Unbiased Risk Estimation as Parameter Choice Rule for Filter-based Regularization Methods

Unbiased Risk Estimation as Parameter Choice Rule for Filter-based Regularization Methods Unbiased Risk Estimation as Parameter Choice Rule for Filter-based Regularization Methods Frank Werner 1 Statistical Inverse Problems in Biophysics Group Max Planck Institute for Biophysical Chemistry,

More information

Proximal tools for image reconstruction in dynamic Positron Emission Tomography

Proximal tools for image reconstruction in dynamic Positron Emission Tomography Proximal tools for image reconstruction in dynamic Positron Emission Tomography Nelly Pustelnik 1 joint work with Caroline Chaux 2, Jean-Christophe Pesquet 3, and Claude Comtat 4 1 Laboratoire de Physique,

More information

Chapter 4. Unconstrained optimization

Chapter 4. Unconstrained optimization Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file

More information

6. Proximal gradient method

6. Proximal gradient method L. Vandenberghe EE236C (Spring 2016) 6. Proximal gradient method motivation proximal mapping proximal gradient method with fixed step size proximal gradient method with line search 6-1 Proximal mapping

More information

6. Proximal gradient method

6. Proximal gradient method L. Vandenberghe EE236C (Spring 2013-14) 6. Proximal gradient method motivation proximal mapping proximal gradient method with fixed step size proximal gradient method with line search 6-1 Proximal mapping

More information

The Proximal Gradient Method

The Proximal Gradient Method Chapter 10 The Proximal Gradient Method Underlying Space: In this chapter, with the exception of Section 10.9, E is a Euclidean space, meaning a finite dimensional space endowed with an inner product,

More information

Uniform Quadratic Penalties Cause Nonuniform Spatial Resolution

Uniform Quadratic Penalties Cause Nonuniform Spatial Resolution Uniform Quadratic Penalties Cause Nonuniform Spatial Resolution Jeffrey A. Fessler and W. Leslie Rogers 3480 Kresge 111, Box 0552, University of Michigan, Ann Arbor, MI 48109-0552 ABSTRACT This paper examines

More information

Randomness-in-Structured Ensembles for Compressed Sensing of Images

Randomness-in-Structured Ensembles for Compressed Sensing of Images Randomness-in-Structured Ensembles for Compressed Sensing of Images Abdolreza Abdolhosseini Moghadam Dep. of Electrical and Computer Engineering Michigan State University Email: abdolhos@msu.edu Hayder

More information

Ch. 10 Vector Quantization. Advantages & Design

Ch. 10 Vector Quantization. Advantages & Design Ch. 10 Vector Quantization Advantages & Design 1 Advantages of VQ There are (at least) 3 main characteristics of VQ that help it outperform SQ: 1. Exploit Correlation within vectors 2. Exploit Shape Flexibility

More information

Lecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods.

Lecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods. Lecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods. Linear models for classification Logistic regression Gradient descent and second-order methods

More information

Three-scale Radar Backscattering Model of the Ocean Surface Based on Second-order Scattering

Three-scale Radar Backscattering Model of the Ocean Surface Based on Second-order Scattering PIERS ONLINE, VOL. 4, NO. 2, 2008 171 Three-scale Radar Backscattering Model of the Ocean Surface Based on Second-order Scattering Ying Yu 1, 2, Xiao-Qing Wang 1, Min-Hui Zhu 1, and Jiang Xiao 1, 1 National

More information

Nonlinear Optimization: What s important?

Nonlinear Optimization: What s important? Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global

More information

Numerical Methods. Rafał Zdunek Underdetermined problems (2h.) Applications) (FOCUSS, M-FOCUSS,

Numerical Methods. Rafał Zdunek Underdetermined problems (2h.) Applications) (FOCUSS, M-FOCUSS, Numerical Methods Rafał Zdunek Underdetermined problems (h.) (FOCUSS, M-FOCUSS, M Applications) Introduction Solutions to underdetermined linear systems, Morphological constraints, FOCUSS algorithm, M-FOCUSS

More information

Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 16 Advanced topics in computational statistics 18 May 2017 Computer Intensive Methods (1) Plan of

More information

Efficient Data-Driven Learning of Sparse Signal Models and Its Applications

Efficient Data-Driven Learning of Sparse Signal Models and Its Applications Efficient Data-Driven Learning of Sparse Signal Models and Its Applications Saiprasad Ravishankar Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor Dec 10, 2015

More information

On Irreducible Polynomial Remainder Codes

On Irreducible Polynomial Remainder Codes 2011 IEEE International Symposium on Information Theory Proceedings On Irreducible Polynomial Remainder Codes Jiun-Hung Yu and Hans-Andrea Loeliger Department of Information Technology and Electrical Engineering

More information

Machine Learning & Data Mining Caltech CS/CNS/EE 155 Hidden Markov Models Last Updated: Feb 7th, 2017

Machine Learning & Data Mining Caltech CS/CNS/EE 155 Hidden Markov Models Last Updated: Feb 7th, 2017 1 Introduction Let x = (x 1,..., x M ) denote a sequence (e.g. a sequence of words), and let y = (y 1,..., y M ) denote a corresponding hidden sequence that we believe explains or influences x somehow

More information

(a) Write down the value of q and of r. (2) Write down the equation of the axis of symmetry. (1) (c) Find the value of p. (3) (Total 6 marks)

(a) Write down the value of q and of r. (2) Write down the equation of the axis of symmetry. (1) (c) Find the value of p. (3) (Total 6 marks) 1. Let f(x) = p(x q)(x r). Part of the graph of f is shown below. The graph passes through the points ( 2, 0), (0, 4) and (4, 0). (a) Write down the value of q and of r. (b) Write down the equation of

More information

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 5, MAY

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 5, MAY IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 23, NO. 5, MAY 2004 591 Emission Image Reconstruction for Roms-Precorrected PET Allowing Negative Sinogram Values Sangtae Ahn*, Student Member, IEEE, Jeffrey

More information

Lecture 5: September 15

Lecture 5: September 15 10-725/36-725: Convex Optimization Fall 2015 Lecture 5: September 15 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Di Jin, Mengdi Wang, Bin Deng Note: LaTeX template courtesy of UC Berkeley EECS

More information

Stabilization of a 3D Rigid Pendulum

Stabilization of a 3D Rigid Pendulum 25 American Control Conference June 8-, 25. Portland, OR, USA ThC5.6 Stabilization of a 3D Rigid Pendulum Nalin A. Chaturvedi, Fabio Bacconi, Amit K. Sanyal, Dennis Bernstein, N. Harris McClamroch Department

More information

Linear Regression (continued)

Linear Regression (continued) Linear Regression (continued) Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 6, 2017 1 / 39 Outline 1 Administration 2 Review of last lecture 3 Linear regression

More information

Eigenvalues of Collapsing Domains and Drift Laplacian

Eigenvalues of Collapsing Domains and Drift Laplacian Eigenvalues of Collapsing Domains and Drift Laplacian Zhiqin Lu Dedicate to Professor Peter Li on his 60th Birthday Department of Mathematics, UC Irvine, Irvine CA 92697 January 17, 2012 Zhiqin Lu, Dept.

More information

ADMM and Fast Gradient Methods for Distributed Optimization

ADMM and Fast Gradient Methods for Distributed Optimization ADMM and Fast Gradient Methods for Distributed Optimization João Xavier Instituto Sistemas e Robótica (ISR), Instituto Superior Técnico (IST) European Control Conference, ECC 13 July 16, 013 Joint work

More information

Greedy Signal Recovery and Uniform Uncertainty Principles

Greedy Signal Recovery and Uniform Uncertainty Principles Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles

More information

MMSE Denoising of 2-D Signals Using Consistent Cycle Spinning Algorithm

MMSE Denoising of 2-D Signals Using Consistent Cycle Spinning Algorithm Denoising of 2-D Signals Using Consistent Cycle Spinning Algorithm Bodduluri Asha, B. Leela kumari Abstract: It is well known that in a real world signals do not exist without noise, which may be negligible

More information

Problems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B

Problems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2

More information

On nonstationary preconditioned iterative regularization methods for image deblurring

On nonstationary preconditioned iterative regularization methods for image deblurring On nonstationary preconditioned iterative regularization methods for image deblurring Alessandro Buccini joint work with Prof. Marco Donatelli University of Insubria Department of Science and High Technology

More information

ECE521 lecture 4: 19 January Optimization, MLE, regularization

ECE521 lecture 4: 19 January Optimization, MLE, regularization ECE521 lecture 4: 19 January 2017 Optimization, MLE, regularization First four lectures Lectures 1 and 2: Intro to ML Probability review Types of loss functions and algorithms Lecture 3: KNN Convexity

More information

An interior-point gradient method for large-scale totally nonnegative least squares problems

An interior-point gradient method for large-scale totally nonnegative least squares problems An interior-point gradient method for large-scale totally nonnegative least squares problems Michael Merritt and Yin Zhang Technical Report TR04-08 Department of Computational and Applied Mathematics Rice

More information

Option 1: Landmark Registration We can try to align specific points. The Registration Problem. Landmark registration. Time-Warping Functions

Option 1: Landmark Registration We can try to align specific points. The Registration Problem. Landmark registration. Time-Warping Functions The Registration Problem Most analyzes only account for variation in amplitude. Frequently, observed data exhibit features that vary in time. Option 1: Landmark Registration We can try to align specific

More information

Math 4329: Numerical Analysis Chapter 03: Fixed Point Iteration and Ill behaving problems. Natasha S. Sharma, PhD

Math 4329: Numerical Analysis Chapter 03: Fixed Point Iteration and Ill behaving problems. Natasha S. Sharma, PhD Why another root finding technique? iteration gives us the freedom to design our own root finding algorithm. The design of such algorithms is motivated by the need to improve the speed and accuracy of

More information

CE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review

CE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley

More information

1 Conjugate gradients

1 Conjugate gradients Notes for 2016-11-18 1 Conjugate gradients We now turn to the method of conjugate gradients (CG), perhaps the best known of the Krylov subspace solvers. The CG iteration can be characterized as the iteration

More information

Lecture 6: Communication Complexity of Auctions

Lecture 6: Communication Complexity of Auctions Algorithmic Game Theory October 13, 2008 Lecture 6: Communication Complexity of Auctions Lecturer: Sébastien Lahaie Scribe: Rajat Dixit, Sébastien Lahaie In this lecture we examine the amount of communication

More information