Approximate Message Passing with Built-in Parameter Estimation for Sparse Signal Recovery
|
|
- Samson Singleton
- 5 years ago
- Views:
Transcription
1 Approimate Message Passing with Built-in Parameter Estimation for Sparse Signal Recovery arxiv: v1 [cs.it] Jun 016 Shuai Huang, Trac D. Tran Department of Electrical and Computer Engineering Johns Hopkins University, Baltimore, MD, USA {shuaihuang, Abstract The approimate message passing AMP) algorithm shows advantage over conventional conve optimization methods in recovering under-sampled sparse signals. AMP is analytically tractable and has a much lower compleity. However, it requires that the true parameters of the input and output channels are known. In this paper, we propose an AMP algorithm with built-in parameter estimation that jointly estimates the sparse signals along with the parameters by treating them as unknown random variables with simple priors. Specifically, the maimum a posterior MAP) parameter estimation is presented and shown to produce estimations that converge to the true parameter values. Eperiments on sparse signal recovery show that the performance of the proposed approach matches that of the oracle AMP algorithm where the true parameter values are known. 1 Introduction Sparse signal recovery SSR) is the key topic in Compressive Sensing CS) [1 4], it lays the foundation for applications such as dictionary learning [5], sparse representation-based classification [6], etc. Specifically, SSR tries to recover the sparse signal R N given a M N sensing matri A and a measurement vector y = A + w R M, where M < N and w R M is the unknown noise introduced in this process. Although the problem itself is ill-posed, perfect recovery is still possible provided that is sufficiently sparse and A is incoherent enough [1]. Lasso [7], a.k.a l 1 -minimization, is one of most popular approaches proposed to solve this problem: arg min y A + γ 1, 1) where y A is the data-fidelity term, 1 is the sparsity-promoting term, and γ balances the trade-off between them. The authors would like to thank NSF-CCF , NSF-CCF and NSF-EECS for funding. 1
2 p j λ) R N z R M y R M py i z i, θ) A M N Separable Input channel Linear transformation matri Separable Output channel Figure 1: A probabilistic view of the sparse signal recovery [8]: The signal is estimated given the output vector y, the channel transition probability functions p j λ), py i z i, θ) and the transformation matri A. {λ, θ} denote the parameters of the probability models and are usually unknown. From a probabilistic view, Lasso is equivalent to a maimum likelihood ML) estimation of the signal under the assumption that the entries of are i.i.d. distributed following the Laplace distribution p j ) ep λ j ), and those of w are i.i.d. distributed following the Gaussian distribution pw i ) ep w i/θ). Let z = A, we have py i ) ep y i z i ) /θ). The ML estimation is then arg ma p, y), which is essentially the same as 1). In general, SSR can be described by the Bayesian model from [8], as is shown in Fig. 1. Under the Bayesian setting it is possible to design efficient iterative algorithms to compute either the maimum a posterior MAP) or minimum mean square error MMSE) estimate of the signal. Most notable among them is the approimate message passing AMP) [9 11]. The AMP algorithm performs probabilistic inferences on the corresponding factor graph using Gaussian and quadratic approimations of loopy belief propagation loopy BP), a.k.a. message passing [1]. Based on different inference tasks, loopy BP has two variants: sumproduct message passing for the MMSE estimate of and ma-sum message passing for the MAP estimate of [8]. Although AMP is analytically tractable and has low compleity [9 11], it requires the parameters {λ, θ} in the input and output channels are known eactly, which is often not satisfied in practice. Various methods have been proposed to include parameter estimation in the AMP algorithm using Epectation-Maimization [13, 14] and adaptive methods [15]. In this paper, we propose an etension to the generalized approimate message passing GAMP) [8] by treating the parameters {λ, θ} as unknown random variables with prior distributions and estimating them jointly with the signal. Using the sum-product GAMP as an eample, we give the message passing updates between the factor nodes and the variable nodes, which serves as the basis to write the state evolution equations of the GAMP algorithms. The MAP parameter estimation can then be derived by maimizing the approimated posterior marginals pλ y) and pθ y). Following the analysis in [15], we can show that the estimated parameter values from the MAP estimation also converge to the true parameter values if they are computed eactly. To compare the proposed GAMP with built-in parameter estimation PE-GAMP) with oracle-gamp where the true parameters are known, we run sparse signal recovery eperiments with Bernoulli-Gaussian BG) input channel and additive white Gaussian noise AWGN) output channel. The eperiments show that the performance of PE-GAMP does match that of oracle-gamp.
3 GAMP with Built-in Parameter Estimation The generalized factor graph for the proposed PE-GAMP that treats the parameters as random variables is shown in Fig.. Here we adopt the same notations used by [8]. Take the messages being passed between the factor node Φ m and the variable node n for eample, Φm n is the message from Φ m to n, and Φm n is the message from n to Φ m. Both Φm n and Φm n can be viewed as functions of n. In the following section.1 and., we give the messages being passed on the generalized factor graph in log domain for the sum-product message passing algorithm and the ma-sum message passing algorithm respectively..1 Sum-product Message Passing In the following, we first present the sum-product message passing in the t + 1)-th iteration. Φ m n = const + log Φ m y m,, θ) ep j n t) Φ m j + ) v t) Φ m θ v ) \ n,θ Φ m n = const + t) Ω n n + i m t) Φ i n 3) ) Ω n n = const + log Ω n n, λ) ep u t) Ω n λ u 4) λ Ω n n = const + i t) Φ i n, 5) where Φ m y m,, θ) = py m, θ) and Ω n n, λ) = p n λ). Let Γ n ) denote the factor nodes in the neighborhood of n. The posterior marginal of n is: p n y) ep t) n = ep t) Ω n n + ) Φ m Γ n) t) Φ m n. 6) Using p y), the MMSE estimate of can then be computed: ˆ n = E [ n y] = n p n y). 7) n Similarly, we can write the message passing updates involving λ, θ as follows: Ω n λ l = const + log Ω n n, λ) ep t) Ω n n + ) u l t) Ω n λ u n,λ\λ l Ω n λ l = const + j n t) Ω j λ l + log pλ l ) 9) Φ m θ k = const + log Φ m y m,, θ) ep j t) Φ m j + ) v k t) Φ m θ v 10) θ\θ k, Φ m θ k = const + i m t) Φ i θ k + log pθ k ), 11) where pλ l ), pθ k ) are the pre-specified prior on the parameters. In general, if we don t have any knowledge about how the parameters are distributed, we can fairly assume a uniform 8) 3
4 Ω 1 1 Φ 1 Λ 1 λ 1.. Λ L λ L Ω.. Φ. θ 1. θ K Θ 1. Θ K N Φ M Ω N Figure : The factor graph for the proposed PE-GAMP. represents the factor node, and represents the variable node. λ = {λ 1,, λ L } and θ = {θ 1,, θ K } are the parameters. = [ 1,, N ] T is the sparse signal. prior and treat pλ l ), pθ k ) as constants. Additionally, we also have the following posterior marginals of λ l, θ k : pλ l y) ep t) λ l = ep log pλ l ) + ) Ω n Γλ l ) t) Ω n λ l 1) pθ k y) ep t) θ k = ep log pθ k ) + ) Φ m Γθ k ) t) Φ m θ k. 13). Ma-sum Message Passing For the ma-sum message passing, the message updates from the variable nodes to the factor nodes are the same as the aforementioned sum-product message passing, i.e. 3, 5, 9, 11). We only need to change the message updates from the factor nodes to the variable nodes by replacing with ma: Φ m n = ma \ n,θ log Φ m y m,, θ) + j n t) Φ m j + v t) Φ m θ v 14) Ω n n = ma log Ω n n, λ) + u λ t) Ω n λ u 15) Ω n λ l = ma log Ω n n, λ) + t) Ω n n + u l t) Ω n λ u 16) n,λ\λ l Φ m θ k = ma θ\θ k, log Φ m y m,, θ) + j t) Φ m j + v k t) Φ m θ v. 17) The MAP estimate of is then: ˆ n = arg ma t) n n = arg ma n t) Ω n n + Φ m Γ n) t) Φ m n. 18) 4
5 3 MAP Parameter Estimation for Sum-product Message Passing In this paper we focus on using the sum-product message passing algorithm for sparse signal recovery. Take ) in sum-product message passing for eample, it contains integrations with respect to the parameters θ, and they might be difficult to compute. However, the posterior marginals of θ can be directly estimated in 1). Instead of doing a complete integration w.r.t θ k, we can use the following scalar MAP estimate of θ k to simplify the message updates: ˆθ t) k = arg ma θk pθ k y) = arg ma θk t) θ k. 19) Similarly, we also have the MAP estimate of λ l as follows: ˆλ t) l = arg ma λl pλ l y) = arg ma λl t) λ l. 0) The GAMP algorithm with MAP parameter estimation can be summarized in Algorithm 1. Algorithmus 1 Sum-product GAMP with MAP parameter estimation Require: y, A, pλ), p λ), pθ), pw θ) 1: Initialize λ 0), θ 0) : for t = {1,, } do 3: Perform GAMP [8]; t) 4: Compute ˆλ l for l = 1,, L; t) 5: Compute ˆθ k for k = 1,, K; 6: Compute the MMSE estimate ˆ t) = E[ y]; 7: if ˆ t) reaches convergence then 8: ˆ = ˆ t) ; 9: break; 10: end if 11: end for 1: return Output ˆ; Discussion: The EM-GAMP [13, 14] and the adaptive-gamp [15] are both ML estimations. Specifically, EM-GAMP tries to maimize E[log p, w; λ, θ) y, λ t), θ t) ] iteratively using the Epectation-Maimization EM) algorithm [16]; adaptive-gamp tries to maimize the log-likelihood of two new random variables r, p introduced in the original GAMP: log pr λ), log py, p θ). For the proposed MAP parameter estimation, we have the following using Bayes rule: pλ y) py λ)pλ) pθ y) py θ)pθ). 1a) 1b) If the priors pλ), pθ) are chosen to be uniform distributions, the MAP estimation and ML estimation would have the same solutions since they were both maimizing py λ) and py θ). 5
6 For the state evolution analysis, we make the same assumptions as [8, 11, 15] about the PE-GAMP algorithm. In general, the estimation problem is indeed by the dimensionality N of, and the output dimensionality M satisfies lim N N/M = β for some β > 0. Specifically, 19,0) can be viewed as different adaptation functions defined in [15]. The empirical convergence of, λ, θ can then be obtained directly from the conclusion of Theorem in [15]. Following the analysis in [15], we also assume the MAP adaptation functions arg ma λ pλ y), arg ma θ pθ y) satisfy the week pseudo-lipschitz continuity property [11] over y. Using Theorem 3 in [15], we can see that the MAP estimations converge to the true parameters as N when they are computed eactly. 3.1 MAP Estimation of λ, θ We net show how to compute the MAP estimates in 19,0). For clarification purposes, we will remove the superscript t) from the notations. Starting with some initial solutions λ l 0), θ k 0), we will maimize the quadratic approimations of λl, θk iteratively. Take λl for eample, it is approimated as follows in the h + 1)-th iteration: λl λl h) + λ l h) λ l λ l h)) + 1 λ l h) λ l λ l h)), ) where λ l h) is the solution in the h-th iteration. If λ l h) λ l h + 1) using Newton s method: < 0, we can get the following λ l h + 1) = λ l h) λ l h). 3) λ l h) If λ l h) > 0, Newton s method will produce local minimum solution. In this case we will use the line search method. This iterative approach can be summarized in Algorithm. The MAP estimation θ k can also be computed in a similar way. Algorithmus MAP parameter estimation Require: λl 1: Initialize λ l 0) : for h = {0, 1, } do 3: Compute λ l h), λ l h) ; 4: if λ l h) < 0 then λ lh + 1) = λ l h) λ l h) λ l h) end if; 5: if λ l h) 0 then Perform line search for λ lh + 1) end if; 6: if λ l h + 1) reaches convergence then 7: λ l = λ l h + 1); 8: break; 9: end if 10: end for 11: return Output λ l ; 6
7 3. MAP estimation for Special Input and Output Channels We then give the following three eamples on how to perform MAP estimation of the parameters in the input and output channels Bernoulli-Gaussian Input Channel In this case, the sparse signal can be modeled as a miture of Bernoulli and Gaussian distribution: p j λ) = 1 λ 1 )δ j ) + λ 1 N j ; λ, λ 3 ), 4) where δ ) is Dirac delta function, λ 1 [0, 1] is the sparsity rate, λ R is the nonzero coefficient mean and λ 3 R + is the nonzero coefficient variance. The AMP algorithm [8,9] uses quadratic approimation of the loopy BP. In the t+1)-th iteration, we have the following from [8]: 1 Ωn n τω n t) n ) + const, 5) n n where τω n n, t) n correspond to the variance and mean of n in the t-th iteration respectively. Take λ 1 for eample, Ω n λ 1 can be computed as follows: c n ) = ep t) n τω n n ) d n ) = τ Ωn n λ t) 3 + τ Ω n n ep λt) t) n ) λ t) 3 + τω n n ) 6) 7) Ω n λ 1 = log 1 λ 1 )c n ) + λ 1 d n )) + const, 8) where λ t), λ t) 3 are the MAP estimations in the t-th iteration. λ 1 can then be computed accordingly for the MAP estimation of λ t+1) 1 in the t + 1)-th iteration. λ t+1), λ t+1) 3 can be updated similarly. 3.. Laplace Input Channel Laplace input channel assumes the signal follows the Laplace distribution: p j λ) = λ 1 ep λ 1 j ) 9) where λ 1 0, ). We can compute Ω n λ 1 similarly using the quadratic approimation of Ωn n from [8]: ) c n ) = λ )) 1 τ ep Ωn n λ 1 t) n λ 1 τ Ωn n π 1 erf t) n λ 1 τω n n 30) τ Ωn n 7
8 ) d n ) = λ 1 τ ep Ωn n λ 1 + n t) λ 1 τ Ωn n π 1 + erf t) n )) + λ 1 τω n n τ Ωn n Ω n λ 1 = logc n ) + d n )) + const, 3) where erf ) is the error function. λ 1 can then be computed accordingly for the MAP estimation of λ t+1) 1 in the t + 1)-th iteration Additive White Gaussian Noise Output Channel AWGN output channel assumes the noise w is white Gaussian noise: 31) pw i θ) = N w i ; 0, θ 1 ), 33) where θ 1 R + is the noise variance. Using the quadratic approimation from [8], we can get: where τ p Φ m, p t) Φ m iteration respectively. θ 1 Φ m θ 1 = 1 log ) θ 1 + τ p 1 Φ m y m p t) Φ m ) θ 1 + τ p + const, 34) Φ m ) are from [8] and correspond to the variance and mean of z m in the t-th θ t+1) 1 in the t + 1)-th iteration. 4 Numerical Eperiments can then be computed accordingly for the MAP estimation of In this section, we use the proposed GAMP algorithm with built-in parameter estimation PE-GAMP) to recover sparse signal from the under-sampled measurement y = A + w. Specifically, we will use BG input channel and AWGN output channel to generate, y. Since we don t have any knowledge about the priors of λ, θ, we will fairly choose the uniform prior for each parameter. The recovered signal ˆ is then computed using the MMSE estimator E [ y] from sum-product message passing, and we will compare the recovery performance with the oracle-gamp that knows the true parameters. 4.1 Noiseless Sparse Signal Recovery We first perform noiseless sparse signal recovery eperiments and draw the empirical phase transition curves PTC) of PE-GAMP, oracle-gamp. We fi N = 1000 and vary the oversampling ratio σ = M N [0.05, 0.1, 0.15,, 0.95] and the under-sampling ratio ρ = S M [0.05, 0.1, 0.15,, 0.95], where S is the sparsity of the signal, i.e. the number of nonzero coefficients. For each combination of σ and ρ, we randomly generate 100 pairs of {, A}. The entries of A and the nonzero entries of are i.i.d. Gaussian N 0, 1). Given the measurement vector y = A and the sensing matri A, we try to recover the sparse signal. If ɛ = ˆ / < 10 3, the recovery is considered to be a success. Based on the 100 trials, we compute the success rate for each combination of σ and ρ. 8
9 ρ = S M ρ = S M PE GAMP orcale GAMP theoretical Lasso σ = M N σ = M N a) b) Figure 3: a) The absolute difference of the success rates between PE-GAMP and oracle- GAMP; b)the phase transition curves of the PE-GAMP, oracle-gamp and theoretical Lasso. The absolute difference of the success rates between PE-GAMP and oracle-gamp are shown in Fig. 3a). The PTC curve is the contour that correspond to the 0.5 success rate in the domain σ, ρ) 0, 1), it divides the domain into a success phase lower right) and a failure phase upper left). The PTC curves of the two GAMP methods along with the theoretic Lasso [9] are shown in Fig. 3b). We can see from Fig. 3 that the performance of PE-GAMP generally matches that of the oracle-gamp. PE-GAMP is able to estimate the parameters fairly well while recovering the sparse signals. 4. Noisy Sparse Signal Recovery We net try to recover the sparse signal from a noisy measurement vector y. In this case, we would like to see how the two algorithms behave when an increasing amount of noise is added to the measurement. Specifically, S = 100, M = 500, N = 1000 are fied, and y is generated as follows: y = A + νw, 35) where we choose ν {0.1, 0.,, 1}. For each ν, we randomly generate 100 triples of {, A, w}. The entries of A, w and the nonzero entries of are i.i.d Gaussian N 0, 1). ɛ = ˆ / is used to evaluate the performances of the algorithms. The mean ± standard deviation of the ɛs from the 100 trials are shown in Fig. 4. We can see that the proposed PE-GAMP is able to perform as well as the oracle-gamp in recovering noisy sparse signals. 9
10 ε 0.04 method PE GAMP oracle GAMP ν Figure 4: Comparison on the recovery of noisy sparse signals at different noise levels ν, the error bar represents the mean ± standard deviation of the ɛ. 5 Conclusion and Future Work In this paper we proposed the GAMP algorithm with built-in parameter estimation to recover under-sampled sparse signals. The parameters are treated as random variables with prespecified priors, their posterior marginals can then be directly approimated by loopy belief propagation. This allows us to perform MAP estimation of the parameters and update to the recovered signals jointly. Following the same assumptions made by the original GAMP, the proposed PE-GAMP converges empirically to the true parameter values, as is evident from a series of noiseless and noisy sparse signal recovery eperiments. We have mainly focused on the GAMP algorithm based on sum-product message passing here. In the future we would like to include the ma-sum message passing and conduct analysis on its convergence and asymptotic consistency behavior. References [1] E. Candès and T. Tao, Decoding by linear programming, IEEE Trans. on Information Theory, vol. 511), pp , 005. [] E.J. Candes, J. Romberg, and T. Tao, Robust uncertainty principles: Eact signal reconstruction from highly incomplete frequency information, IEEE Trans. on Information Theory, vol. 5), pp , 006. [3] D.L. Donoho, Compressed sensing, IEEE Trans. on Information Theory, vol. 5, no. 4, pp , 006. [4] E.J. Candes and T. Tao, Near-optimal signal recovery from random projections: Universal encoding strategies?, IEEE Trans. on Information Theory, vol. 5, no. 1, pp ,
11 [5] M. Aharon, M. Elad, and A. Bruckstein, K-svd: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, vol. 54, no. 11, pp , 006. [6] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, Robust face recognition via sparse representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no., pp. 10 7, 009. [7] Robert Tibshirani, Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, vol. 58, pp , [8] S. Rangan, Generalized approimate message passing for estimation with random linear miing, in Information Theory Proceedings ISIT), 011 IEEE International Symposium on, July 011, pp [9] D.L. Donoho, A. Maleki, and A. Montanari, Message-passing algorithms for compressed sensing, Proc. Nat. Acad. Sci., vol. 106, no. 45, pp , 009. [10] D.L. Donoho, A. Maleki, and A. Montanari, Message-passing algorithms for compressed sensing: I. motivation and construction, Proc. Inform. Theory Workshop, Jan 010. [11] M. Bayati and A. Montanari, The dynamics of message passing on dense graphs, with applications to compressed sensing, IEEE Transactions on Information Theory, vol. 57, no., pp , Feb 011. [1] M. J. Wainwright and M. I. Jordan, Graphical models, eponential families, and variational inference, Found. Trends Mach. Learn., vol. 1, no. 1-, Jan [13] J. Vila and P. Schniter, Epectation-maimization bernoulli-gaussian approimate message passing, in Conf. Rec. 45th Asilomar Conf. Signals, Syst. Comput, Nov 011, pp [14] J. P. Vila and P. Schniter, Epectation-maimization gaussian-miture approimate message passing, IEEE Transactions on Signal Processing, vol. 61, no. 19, pp , Oct 013. [15] U. S. Kamilov, S. Rangan, A. K. Fletcher, and M. Unser, Approimate message passing with consistent parameter estimation and applications to sparse learning, IEEE Transactions on Information Theory, vol. 60, no. 5, pp , May 014. [16] A. P. Dempster, N. M. Laird, and D. B. Rubin, Maimum likelihood from incomplete data via the em algorithm, Journal of the Royal Statistical Society, vol. 39, no. 1, pp. 1 38,
Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems
1 Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems Alyson K. Fletcher, Mojtaba Sahraee-Ardakan, Philip Schniter, and Sundeep Rangan Abstract arxiv:1706.06054v1 cs.it
More informationCompressive Sensing under Matrix Uncertainties: An Approximate Message Passing Approach
Compressive Sensing under Matrix Uncertainties: An Approximate Message Passing Approach Asilomar 2011 Jason T. Parker (AFRL/RYAP) Philip Schniter (OSU) Volkan Cevher (EPFL) Problem Statement Traditional
More informationApproximate Message Passing Algorithms
November 4, 2017 Outline AMP (Donoho et al., 2009, 2010a) Motivations Derivations from a message-passing perspective Limitations Extensions Generalized Approximate Message Passing (GAMP) (Rangan, 2011)
More informationColor Scheme. swright/pcmi/ M. Figueiredo and S. Wright () Inference and Optimization PCMI, July / 14
Color Scheme www.cs.wisc.edu/ swright/pcmi/ M. Figueiredo and S. Wright () Inference and Optimization PCMI, July 2016 1 / 14 Statistical Inference via Optimization Many problems in statistical inference
More informationOn convergence of Approximate Message Passing
On convergence of Approximate Message Passing Francesco Caltagirone (1), Florent Krzakala (2) and Lenka Zdeborova (1) (1) Institut de Physique Théorique, CEA Saclay (2) LPS, Ecole Normale Supérieure, Paris
More informationPhil Schniter. Collaborators: Jason Jeremy and Volkan
Bilinear Generalized Approximate Message Passing (BiG-AMP) for Dictionary Learning Phil Schniter Collaborators: Jason Parker @OSU, Jeremy Vila @OSU, and Volkan Cehver @EPFL With support from NSF CCF-,
More informationApproximate Message Passing for Bilinear Models
Approximate Message Passing for Bilinear Models Volkan Cevher Laboratory for Informa4on and Inference Systems LIONS / EPFL h,p://lions.epfl.ch & Idiap Research Ins=tute joint work with Mitra Fatemi @Idiap
More informationHybrid Approximate Message Passing for Generalized Group Sparsity
Hybrid Approximate Message Passing for Generalized Group Sparsity Alyson K Fletcher a and Sundeep Rangan b a UC Santa Cruz, Santa Cruz, CA, USA; b NYU-Poly, Brooklyn, NY, USA ABSTRACT We consider the problem
More informationBayesian Methods for Sparse Signal Recovery
Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Jason Palmer, Zhilin Zhang and Ritwik Giri Motivation Motivation Sparse Signal Recovery
More informationMismatched Estimation in Large Linear Systems
Mismatched Estimation in Large Linear Systems Yanting Ma, Dror Baron, Ahmad Beirami Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC 7695, USA Department
More informationInferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines. Eric W. Tramel. itwist 2016 Aalborg, DK 24 August 2016
Inferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines Eric W. Tramel itwist 2016 Aalborg, DK 24 August 2016 Andre MANOEL, Francesco CALTAGIRONE, Marylou GABRIE, Florent
More informationPerformance Regions in Compressed Sensing from Noisy Measurements
Performance egions in Compressed Sensing from Noisy Measurements Junan Zhu and Dror Baron Department of Electrical and Computer Engineering North Carolina State University; aleigh, NC 27695, USA Email:
More informationMotivation Sparse Signal Recovery is an interesting area with many potential applications. Methods developed for solving sparse signal recovery proble
Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Zhilin Zhang and Ritwik Giri Motivation Sparse Signal Recovery is an interesting
More informationSTA 414/2104, Spring 2014, Practice Problem Set #1
STA 44/4, Spring 4, Practice Problem Set # Note: these problems are not for credit, and not to be handed in Question : Consider a classification problem in which there are two real-valued inputs, and,
More informationSparse Algorithms are not Stable: A No-free-lunch Theorem
Sparse Algorithms are not Stable: A No-free-lunch Theorem Huan Xu Shie Mannor Constantine Caramanis Abstract We consider two widely used notions in machine learning, namely: sparsity algorithmic stability.
More informationNew Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
More informationAn equivalence between high dimensional Bayes optimal inference and M-estimation
An equivalence between high dimensional Bayes optimal inference and M-estimation Madhu Advani Surya Ganguli Department of Applied Physics, Stanford University msadvani@stanford.edu and sganguli@stanford.edu
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationMachine Learning Basics III
Machine Learning Basics III Benjamin Roth CIS LMU München Benjamin Roth (CIS LMU München) Machine Learning Basics III 1 / 62 Outline 1 Classification Logistic Regression 2 Gradient Based Optimization Gradient
More informationType II variational methods in Bayesian estimation
Type II variational methods in Bayesian estimation J. A. Palmer, D. P. Wipf, and K. Kreutz-Delgado Department of Electrical and Computer Engineering University of California San Diego, La Jolla, CA 9093
More informationApproximate Message Passing
Approximate Message Passing Mohammad Emtiyaz Khan CS, UBC February 8, 2012 Abstract In this note, I summarize Sections 5.1 and 5.2 of Arian Maleki s PhD thesis. 1 Notation We denote scalars by small letters
More informationebay/google short course: Problem set 2
18 Jan 013 ebay/google short course: Problem set 1. (the Echange Parado) You are playing the following game against an opponent, with a referee also taking part. The referee has two envelopes (numbered
More informationMachine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013
Machine Learning for Signal Processing Sparse and Overcomplete Representations Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 1 Key Topics in this Lecture Basics Component-based representations
More informationCompressed Sensing and Related Learning Problems
Compressed Sensing and Related Learning Problems Yingzhen Li Dept. of Mathematics, Sun Yat-sen University Advisor: Prof. Haizhang Zhang Advisor: Prof. Haizhang Zhang 1 / Overview Overview Background Compressed
More informationThe Minimax Noise Sensitivity in Compressed Sensing
The Minimax Noise Sensitivity in Compressed Sensing Galen Reeves and avid onoho epartment of Statistics Stanford University Abstract Consider the compressed sensing problem of estimating an unknown k-sparse
More informationTurbo-AMP: A Graphical-Models Approach to Compressive Inference
Turbo-AMP: A Graphical-Models Approach to Compressive Inference Phil Schniter (With support from NSF CCF-1018368 and DARPA/ONR N66001-10-1-4090.) June 27, 2012 1 Outline: 1. Motivation. (a) the need for
More informationSparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference
Sparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference Shunsuke Horii Waseda University s.horii@aoni.waseda.jp Abstract In this paper, we present a hierarchical model which
More informationBhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego
Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego 1 Outline Course Outline Motivation for Course Sparse Signal Recovery Problem Applications Computational
More informationState Space and Hidden Markov Models
State Space and Hidden Markov Models Kunsch H.R. State Space and Hidden Markov Models. ETH- Zurich Zurich; Aliaksandr Hubin Oslo 2014 Contents 1. Introduction 2. Markov Chains 3. Hidden Markov and State
More informationScale Mixture Modeling of Priors for Sparse Signal Recovery
Scale Mixture Modeling of Priors for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Jason Palmer, Zhilin Zhang and Ritwik Giri Outline Outline Sparse
More informationOutline. Spring It Introduction Representation. Markov Random Field. Conclusion. Conditional Independence Inference: Variable elimination
Probabilistic Graphical Models COMP 790-90 Seminar Spring 2011 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Outline It Introduction ti Representation Bayesian network Conditional Independence Inference:
More information1-Bit Compressive Sensing
1-Bit Compressive Sensing Petros T. Boufounos, Richard G. Baraniuk Rice University, Electrical and Computer Engineering 61 Main St. MS 38, Houston, TX 775 Abstract Compressive sensing is a new signal acquisition
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationCompressed Sensing Using Bernoulli Measurement Matrices
ITSchool 11, Austin Compressed Sensing Using Bernoulli Measurement Matrices Yuhan Zhou Advisor: Wei Yu Department of Electrical and Computer Engineering University of Toronto, Canada Motivation Motivation
More informationScalable Inference for Neuronal Connectivity from Calcium Imaging
Scalable Inference for Neuronal Connectivity from Calcium Imaging Alyson K. Fletcher Sundeep Rangan Abstract Fluorescent calcium imaging provides a potentially powerful tool for inferring connectivity
More informationCompressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery
Compressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery Jorge F. Silva and Eduardo Pavez Department of Electrical Engineering Information and Decision Systems Group Universidad
More informationLearning MMSE Optimal Thresholds for FISTA
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Learning MMSE Optimal Thresholds for FISTA Kamilov, U.; Mansour, H. TR2016-111 August 2016 Abstract Fast iterative shrinkage/thresholding algorithm
More informationEstimating LASSO Risk and Noise Level
Estimating LASSO Risk and Noise Level Mohsen Bayati Stanford University bayati@stanford.edu Murat A. Erdogdu Stanford University erdogdu@stanford.edu Andrea Montanari Stanford University montanar@stanford.edu
More informationVector Approximate Message Passing. Phil Schniter
Vector Approximate Message Passing Phil Schniter Collaborators: Sundeep Rangan (NYU), Alyson Fletcher (UCLA) Supported in part by NSF grant CCF-1527162. itwist @ Aalborg University Aug 24, 2016 Standard
More informationRisk and Noise Estimation in High Dimensional Statistics via State Evolution
Risk and Noise Estimation in High Dimensional Statistics via State Evolution Mohsen Bayati Stanford University Joint work with Jose Bento, Murat Erdogdu, Marc Lelarge, and Andrea Montanari Statistical
More informationReconstruction from Anisotropic Random Measurements
Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013
More informationGWAS IV: Bayesian linear (variance component) models
GWAS IV: Bayesian linear (variance component) models Dr. Oliver Stegle Christoh Lippert Prof. Dr. Karsten Borgwardt Max-Planck-Institutes Tübingen, Germany Tübingen Summer 2011 Oliver Stegle GWAS IV: Bayesian
More informationCSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
More informationMachine Learning for Signal Processing Sparse and Overcomplete Representations
Machine Learning for Signal Processing Sparse and Overcomplete Representations Abelino Jimenez (slides from Bhiksha Raj and Sourish Chaudhuri) Oct 1, 217 1 So far Weights Data Basis Data Independent ICA
More informationLeast Squares Regression
CIS 50: Machine Learning Spring 08: Lecture 4 Least Squares Regression Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may not cover all the
More informationLecture 1c: Gaussian Processes for Regression
Lecture c: Gaussian Processes for Regression Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London c.archambeau@cs.ucl.ac.uk
More informationLeast Squares Regression
E0 70 Machine Learning Lecture 4 Jan 7, 03) Least Squares Regression Lecturer: Shivani Agarwal Disclaimer: These notes are a brief summary of the topics covered in the lecture. They are not a substitute
More informationON CONVERGENCE PROPERTIES OF MESSAGE-PASSING ESTIMATION ALGORITHMS. Justin Dauwels
ON CONVERGENCE PROPERTIES OF MESSAGE-PASSING ESTIMATION ALGORITHMS Justin Dauwels Amari Research Unit, RIKEN Brain Science Institute, Wao-shi, 351-0106, Saitama, Japan email: justin@dauwels.com ABSTRACT
More informationMixture Models and EM
Mixture Models and EM Goal: Introduction to probabilistic mixture models and the expectationmaximization (EM) algorithm. Motivation: simultaneous fitting of multiple model instances unsupervised clustering
More informationNear Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing
Near Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas M.Vidyasagar@utdallas.edu www.utdallas.edu/ m.vidyasagar
More informationOn the Lagrangian Biduality of Sparsity Minimization Problems
On the Lagrangian Biduality of Sparsity Minimization Problems Dheeraj Singaraju Roberto Tron Ehsan Elhamifar Allen Yang S. Shankar Sastry Electrical Engineering and Computer Sciences University of California
More informationPhil Schniter. Supported in part by NSF grants IIP , CCF , and CCF
AMP-inspired Deep Networks, with Comms Applications Phil Schniter Collaborators: Sundeep Rangan (NYU), Alyson Fletcher (UCLA), Mark Borgerding (OSU) Supported in part by NSF grants IIP-1539960, CCF-1527162,
More informationSparse representation classification and positive L1 minimization
Sparse representation classification and positive L1 minimization Cencheng Shen Joint Work with Li Chen, Carey E. Priebe Applied Mathematics and Statistics Johns Hopkins University, August 5, 2014 Cencheng
More informationLecture Note 2: Estimation and Information Theory
Univ. of Michigan - NAME 568/EECS 568/ROB 530 Winter 2018 Lecture Note 2: Estimation and Information Theory Lecturer: Maani Ghaffari Jadidi Date: April 6, 2018 2.1 Estimation A static estimation problem
More informationMobile Robot Localization
Mobile Robot Localization 1 The Problem of Robot Localization Given a map of the environment, how can a robot determine its pose (planar coordinates + orientation)? Two sources of uncertainty: - observations
More information2 Statistical Estimation: Basic Concepts
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Lecture Notes Fall 2009 November, 2009 Byoung-Ta Zhang School of Computer Science and Engineering & Cognitive Science, Brain Science, and Bioinformatics Seoul National University
More informationA new method on deterministic construction of the measurement matrix in compressed sensing
A new method on deterministic construction of the measurement matrix in compressed sensing Qun Mo 1 arxiv:1503.01250v1 [cs.it] 4 Mar 2015 Abstract Construction on the measurement matrix A is a central
More informationCompressed Sensing and Neural Networks
and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications
More informationExpectation Maximization Deconvolution Algorithm
Epectation Maimization Deconvolution Algorithm Miaomiao ZHANG March 30, 2011 Abstract In this paper, we use a general mathematical and eperimental methodology to analyze image deconvolution. The main procedure
More informationPerformance Trade-Offs in Multi-Processor Approximate Message Passing
Performance Trade-Offs in Multi-Processor Approximate Message Passing Junan Zhu, Ahmad Beirami, and Dror Baron Department of Electrical and Computer Engineering, North Carolina State University, Email:
More informationDNNs for Sparse Coding and Dictionary Learning
DNNs for Sparse Coding and Dictionary Learning Subhadip Mukherjee, Debabrata Mahapatra, and Chandra Sekhar Seelamantula Department of Electrical Engineering, Indian Institute of Science, Bangalore 5612,
More informationStatistical Image Recovery: A Message-Passing Perspective. Phil Schniter
Statistical Image Recovery: A Message-Passing Perspective Phil Schniter Collaborators: Sundeep Rangan (NYU) and Alyson Fletcher (UC Santa Cruz) Supported in part by NSF grants CCF-1018368 and NSF grant
More information6.867 Machine learning
6.867 Machine learning Mid-term eam October 8, 6 ( points) Your name and MIT ID: .5.5 y.5 y.5 a).5.5 b).5.5.5.5 y.5 y.5 c).5.5 d).5.5 Figure : Plots of linear regression results with different types of
More informationBayesian Approach 2. CSC412 Probabilistic Learning & Reasoning
CSC412 Probabilistic Learning & Reasoning Lecture 12: Bayesian Parameter Estimation February 27, 2006 Sam Roweis Bayesian Approach 2 The Bayesian programme (after Rev. Thomas Bayes) treats all unnown quantities
More informationMessage passing and approximate message passing
Message passing and approximate message passing Arian Maleki Columbia University 1 / 47 What is the problem? Given pdf µ(x 1, x 2,..., x n ) we are interested in arg maxx1,x 2,...,x n µ(x 1, x 2,..., x
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
More informationProbabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Gaussian graphical models and Ising models: modeling networks Eric Xing Lecture 0, February 5, 06 Reading: See class website Eric Xing @ CMU, 005-06
More informationWiener Filters in Gaussian Mixture Signal Estimation with l -Norm Error
Wiener Filters in Gaussian Mixture Signal Estimation with l -Norm Error Jin Tan, Student Member, IEEE, Dror Baron, Senior Member, IEEE, and Liyi Dai, Fellow, IEEE Abstract Consider the estimation of a
More informationSingle-letter Characterization of Signal Estimation from Linear Measurements
Single-letter Characterization of Signal Estimation from Linear Measurements Dongning Guo Dror Baron Shlomo Shamai The work has been supported by the European Commission in the framework of the FP7 Network
More informationThe Minimum Message Length Principle for Inductive Inference
The Principle for Inductive Inference Centre for Molecular, Environmental, Genetic & Analytic (MEGA) Epidemiology School of Population Health University of Melbourne University of Helsinki, August 25,
More informationRobust Principal Component Analysis
ELE 538B: Mathematics of High-Dimensional Data Robust Principal Component Analysis Yuxin Chen Princeton University, Fall 2018 Disentangling sparse and low-rank matrices Suppose we are given a matrix M
More informationNotes on Discriminant Functions and Optimal Classification
Notes on Discriminant Functions and Optimal Classification Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Discriminant Functions Consider a classification problem
More informationNUMERICAL COMPUTATION OF THE CAPACITY OF CONTINUOUS MEMORYLESS CHANNELS
NUMERICAL COMPUTATION OF THE CAPACITY OF CONTINUOUS MEMORYLESS CHANNELS Justin Dauwels Dept. of Information Technology and Electrical Engineering ETH, CH-8092 Zürich, Switzerland dauwels@isi.ee.ethz.ch
More informationDoes l p -minimization outperform l 1 -minimization?
Does l p -minimization outperform l -minimization? Le Zheng, Arian Maleki, Haolei Weng, Xiaodong Wang 3, Teng Long Abstract arxiv:50.03704v [cs.it] 0 Jun 06 In many application areas ranging from bioinformatics
More informationLearning Multiple Related Tasks using Latent Independent Component Analysis
Learning Multiple Related Tasks using Latent Independent Component Analysis Jian Zhang, Zoubin Ghahramani, Yiming Yang School of Computer Science Gatsby Computational Neuroscience Unit Cargenie Mellon
More informationA NEW FRAMEWORK FOR DESIGNING INCOHERENT SPARSIFYING DICTIONARIES
A NEW FRAMEWORK FOR DESIGNING INCOERENT SPARSIFYING DICTIONARIES Gang Li, Zhihui Zhu, 2 uang Bai, 3 and Aihua Yu 3 School of Automation & EE, Zhejiang Univ. of Sci. & Tech., angzhou, Zhejiang, P.R. China
More informationCSC 576: Variants of Sparse Learning
CSC 576: Variants of Sparse Learning Ji Liu Department of Computer Science, University of Rochester October 27, 205 Introduction Our previous note basically suggests using l norm to enforce sparsity in
More informationProbabilistic image processing and Bayesian network
robabilistic imae processin and Bayesian network Kazuyuki Tanaka Graduate School o Inormation Sciences Tohoku University kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ Reerences K.
More informationSparse signals recovered by non-convex penalty in quasi-linear systems
Cui et al. Journal of Inequalities and Applications 018) 018:59 https://doi.org/10.1186/s13660-018-165-8 R E S E A R C H Open Access Sparse signals recovered by non-conve penalty in quasi-linear systems
More informationSparse Superposition Codes for the Gaussian Channel
Sparse Superposition Codes for the Gaussian Channel Florent Krzakala (LPS, Ecole Normale Supérieure, France) J. Barbier (ENS) arxiv:1403.8024 presented at ISIT 14 Long version in preparation Communication
More informationA parametric approach to Bayesian optimization with pairwise comparisons
A parametric approach to Bayesian optimization with pairwise comparisons Marco Co Eindhoven University of Technology m.g.h.co@tue.nl Bert de Vries Eindhoven University of Technology and GN Hearing bdevries@ieee.org
More informationEECS 545 Project Progress Report Sparse Kernel Density Estimates B.Gopalakrishnan, G.Bellala, G.Devadas, K.Sricharan
EECS 545 Project Progress Report Sparse Kernel Density Estimates B.Gopalakrishnan, G.Bellala, G.Devadas, K.Sricharan Introduction Density estimation forms the backbone for numerous machine learning algorithms
More informationNoisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
More informationLecture 3: Pattern Classification. Pattern classification
EE E68: Speech & Audio Processing & Recognition Lecture 3: Pattern Classification 3 4 5 The problem of classification Linear and nonlinear classifiers Probabilistic classification Gaussians, mitures and
More informationOn the Role of the Properties of the Nonzero Entries on Sparse Signal Recovery
On the Role of the Properties of the Nonzero Entries on Sparse Signal Recovery Yuzhe Jin and Bhaskar D. Rao Department of Electrical and Computer Engineering, University of California at San Diego, La
More informationLecture : Probabilistic Machine Learning
Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning
More informationMessage Passing Algorithms for Compressed Sensing: II. Analysis and Validation
Message Passing Algorithms for Compressed Sensing: II. Analysis and Validation David L. Donoho Department of Statistics Arian Maleki Department of Electrical Engineering Andrea Montanari Department of
More informationMachine Learning 4771
Machine Learning 4771 Instructor: Tony Jebara Topic 7 Unsupervised Learning Statistical Perspective Probability Models Discrete & Continuous: Gaussian, Bernoulli, Multinomial Maimum Likelihood Logistic
More informationVariational Bayes. A key quantity in Bayesian inference is the marginal likelihood of a set of data D given a model M
A key quantity in Bayesian inference is the marginal likelihood of a set of data D given a model M PD M = PD θ, MPθ Mdθ Lecture 14 : Variational Bayes where θ are the parameters of the model and Pθ M is
More informationSteepest descent on factor graphs
Steepest descent on factor graphs Justin Dauwels, Sascha Korl, and Hans-Andrea Loeliger Abstract We show how steepest descent can be used as a tool for estimation on factor graphs. From our eposition,
More informationImproving Approximate Message Passing Recovery of Sparse Binary Vectors by Post Processing
10th International ITG Conference on Systems, Communications and Coding (SCC 2015) Improving Approximate Message Passing Recovery of Sparse Binary Vectors by Post Processing Martin Mayer and Norbert Goertz
More informationLecture 3: More on regularization. Bayesian vs maximum likelihood learning
Lecture 3: More on regularization. Bayesian vs maximum likelihood learning L2 and L1 regularization for linear estimators A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting
More informationAn Overview of Multi-Processor Approximate Message Passing
An Overview of Multi-Processor Approximate Message Passing Junan Zhu, Ryan Pilgrim, and Dror Baron JPMorgan Chase & Co., New York, NY 10001, Email: jzhu9@ncsu.edu Department of Electrical and Computer
More informationof Orthogonal Matching Pursuit
A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement
More informationRobustly Stable Signal Recovery in Compressed Sensing with Structured Matrix Perturbation
Robustly Stable Signal Recovery in Compressed Sensing with Structured Matri Perturbation Zai Yang, Cishen Zhang, and Lihua Xie, Fellow, IEEE arxiv:.7v [cs.it] 4 Mar Abstract The sparse signal recovery
More informationCompressive Phase Retrieval From Squared Output Measurements Via Semidefinite Programming
Compressive Phase Retrieval From Squared Output Measurements Via Semidefinite Programming Henrik Ohlsson, Allen Y. Yang Roy Dong S. Shankar Sastry Department of Electrical Engineering and Computer Sciences,
More informationSparse Linear Models (10/7/13)
STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine
More informationBayesian Machine Learning
Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 2: Bayesian Basics https://people.orie.cornell.edu/andrew/orie6741 Cornell University August 25, 2016 1 / 17 Canonical Machine Learning
More information