An Efficient Lloyd-Max Quantizer for Matching Pursuit Decompositions
|
|
- Augustus Bryan
- 6 years ago
- Views:
Transcription
1 BRAZILVI ITERATIOAL TELECOMMUICATIOS SYMPOSIUM (ITS6), SEPTEMBER 3-6, 6, FORTALEZA-CE, BRAZIL A Efficiet Lloyd-Max Quatizer for Matchig Pursuit Decompositios Lisadro Lovisolo, Eduardo A B da Silva ad Paulo S R Diiz Abstract Several applicatios are usig the Matchig Pursuit algorithm for sigal ad video compressio. The Matchig Pursuit approximates sigals iteratively usig liear combiatios of pre-defied atoms of a dictioary. I compressio applicatios Matchig Pursuits coefficiets, which multiply the atoms i the liear combiatio, eed to be quatized. The Lloyd- Max quatizer is kow to be the best quatizer for a give source. However, to desig a Lloyd-Max quatizer the statistics of the source eed to be kow. The statistics of Matchig Pursuit coefficiets are difficult to model. I this paper, startig from the observatio that the statistics of the agles betwee the residues ad the atoms preset little variatio alog Matchig Pursuit iteratios, we propose to use these statistics to model the oes of Matchig Pursuit coefficiets. This permits the desig of Lloyd-Max quatizers for Matchig Pursuit coefficiets. The Lloyd-Max quatize is compared to a state-of-the-art off-loop Matchig Pursuit quatizatio scheme. Results show that the proposed scheme has good rate-distortio performace, specially at low rates. Idex Terms Matchig Pursuit, Quatizatio, Compressio. After M iteratios, the MP obtais the M-term approximatio ˆx, or simply M-term, give by ˆx = γ g i(), (3) = which has distortio x ˆx = r M x,themth residue. The performace of the MP for sigal compressio applicatios depeds heavily o two aspects: i) Dictioary it should iclude atoms that are good matches to the possible compoets of the sigals to compress. I additio, the dictioary cardiality affects the data rate. ii) Quatizatio for compressio the coefficiets γ must be quatized. Usig a coefficiet quatizatio rule Q[ the compressed sigal is retrieved by the quatized M-term I. Itroductio The Matchig Pursuit algorithm [ (MP) approximates sigals iteratively. The approximatio is obtaied usig atoms (pre-defied sigals) g k from a dictioary D (the collectio of possible atoms). The MP is beig used i compressio schemes for -dimesioal [, [3 ad - dimesioal sigals [, [4. The MP works as follows. Let D = {g k } with k [,...,C(D), such that g k = k, C(D) is the dictioary cardiality (umber of elemets i D). At each iteratio, the MP searches for the atom g i(), i() [...C(D), with largest ier product with the residual sigal r x [, [5, which is give by γ = r x, g i(). () Observe that the first residue is equal to the sigal to be decomposed, that is r x = x. The ier product γ,which is a measure of how much of g i() is icluded i r x,is used to compute the ext residue r x = r x γ g i(). () Lisadro Lovisolo (lisadro@uerj.br) is with DETEL-FE-UERJ, Eduardo A B da Silva ad Paulo S R Diiz (eduardo, diiz@lps.ufrj.br) are with POLI/COPPE-UFRJ. ˆx q = Q[γ g i(). (4) = I order to desig efficiet quatizers Q[, oe eeds a statistical model for MP coefficiets. However, MP coefficiets are difficult to model. For example, i [6 it has bee observed that MP residues have a chaotic behavior. Here, istead of searchig for a good model for MP coefficiets, our approach is based o a statistical model for the MP agles. It relies o the observatio that the agles betwee the atom chose ad the residue beig decomposed alog MP iteratios have well behaved statistics. Defie, at each MP iteratio, the agle betwee the residue r x ad the selected atom g i() as ( r θ = arccos x, g i() r x ). (5) We have verified, experimetally (see sectio II), that the statistics of θ are approximately idepedet of. Here, it is cojectured that the agles betwee the residues ad the atoms i MP iteratios ca be statistically modeled as idepedet ad idetically distributed. SBrT 97
2 BRAZILVI ITERATIOAL TELECOMMUICATIOS SYMPOSIUM (ITS6), SEPTEMBER 3-6, 6, FORTALEZA-CE, BRAZIL II. Agles i Matchig Pursuit Iteratios Usig the defiitio of θ, eq. (5), the MP algorithm is such that γ = x cos (θ ), (6) γ = x si (θ )cos(θ ), (7). γ = x si (θ i )cos(θ ). (8) i= However, if D icludes the elemets g k ad g k for ay k, forcig the ier products to be positive, the γ = x si (θ i )cos(θ ). (9) i= This work assumes that g k ad g k belog to D obtaiig always positive coefficiets, ad all subsequet refereces to dictioaries cosider that. That is, if D does ot iclude g k, the g k is icluded i D ad C(D) is updated accordigly. A. Statistics of the Agles i MP Iteratios I [6, it has bee observed that MP residues have a chaotic behavior, thus it would be reasoable to assume that, after some MP iteratios, the residues ca have ay orietatio. I additio, oe may assume that these orietatios may have uiform probability desity fuctio o the uit-ball. This is equivalet to assume that the residues are realizatios from a memoryless idepedet ad idetically distributed (iid) Gaussia source. That is, the sigal source is such that if a outcome is x = [x(),x(),...,x(), the the x(j) have the same Gaussia distributio (,σ ).This source does ot privilege ay sigal orietatio. Therefore, we use this source to ivestigate MP agle statistics. Fig. shows the relative frequecy histograms of the RVs Θ, which correspod to the agles θ that result from decompositios of realizatios of a Gaussia source usig a dictioary composed of 6 ormalized radom atoms also draw from a Gaussia source i R 4.These histograms were obtaied usig a esemble of 5, MP decompositios of radom sigals from a Gaussia source. Such dictioaries, composed of C(D) ormalized sigals draw from a -dimesioal Gaussia source, are referred here as GS D(C(D),) ad thus the former dictioary is deoted by GSD(6, 4). It should be oted that this dictioary has its cardiality doubled i order to obtai just positive coefficiets. I Fig. oe otes that the histograms of the RVs Θ have very similar shape for all. This leads to the cojecture that the pdfs f Θ (θ )are idepedet ad idetically distributed at ay iteratio. The results preseted i [6 corroborate this assumptio, where it is show that, uder specific coditios, MP residues have chaotic behavior. Fig. shows the mea ad the variace of Θ for several. Fig. depicts also the covariaces betwee MP agles i.5..5 ormalized histogram of Θ θ.5..5 ormalized histogram of Θ θ ormalized histogram of Θ θ ormalized histogram of Θ θ.5..5 ormalized histogram of Θ θ ormalized histogram of Θ θ Fig.. Relative frequecy histograms of Θ for a Gaussia source i R 4 usig the GSD(6, 4). 4 Expected value of Θ Cov[Θ Θ 5 5 Cov[Θ 5 Θ Variace of of Θ Cov[Θ Θ 5 5 Cov[Θ Θ 5 5 Fig.. Mea, variace ad covariace of Θ for a Gaussia source i R 4 usig the GSD(6, 4). differet steps. From it oe observes that Cov[Θ i Θ k =, i k, that is, the agles are ucorrelated. I this work we assume that the agles are idepedet RVs. This is ot a ureasoable assumptio, sice it does ot cotradicts the behavior observed i Fig.. The results preseted so far make use of a dictioary of relatively low dimesio ad cardiality C(D). I practice, the MP is commoly used i large dimesioal spaces with dictioaries such as the Gabor oe [5. The elemets of this dictioary are defied by traslatios, modulatios ad dilatios of a prototype sigal. The most commo choice for the prototype f() is the Gaussia widow. We aalyze here a real Gabor dictioary composed of atoms π with predefied phases i multiples of V.Eachatomis the give by [5 8 δ(), j = >< «p j g() = K (j,p,v) f cos kπ j + πv, j (,L), >: j V, j = L () where f() = 4 e π, is the sample, K (j,p,v) provides uit-orm atoms, ad v [,...,V. Above, j defies the atom scale, p defies the time shift, ad k defies the atom modulatio, ad for L =log () scales their rages are [ j [,L, p [, j ),k [, j ), ad v [,V. SBrT 98
3 BRAZILVI ITERATIOAL TELECOMMUICATIOS SYMPOSIUM (ITS6), SEPTEMBER 3-6, 6, FORTALEZA-CE, BRAZIL 3 Fig. 3 shows f Θ (θ ), for some, obtaied for a esemble of 8, decompositios of Gaussia sigals i R 64 usig the Gabor dictioary with four phases. Fig. 4showsf Θ (θ ), for some, obtaied for a esemble of 8, decompositios of sigals drive from a source that has gamma distributed coordiates i R 64 for the same dictioary. ote that the agle statistics show for each sigal source differ strogly oly at the first MP iteratio, beig visually very similar for the other iteratios. It ca be oted that for this dictioary the agles i differet MP steps have similar statistics eve for very differet sources. ote that, although the statistics of Θ are ot exactly equal to the statistics obtaied i the first MP iteratio for a Gaussia source, they are reasoably close to these; therefore f Θ (θ ), for >, ca be reasoably approximated by the f Θ (θ ) obtaied for a memoryless Gaussia source. This is a reasoable assumptio, sice the memoryless Gaussia source does ot privilege ay orietatio, what seem to be the case for the residues r x for >. B. Discussio of Results It was verified that the statistics of the agles, after the first MP iteratio, ca be cosidered to be ivariat with respect to the step umber. For some dictioaries, for example the Gabor oe, the agle pdf, after some steps, is slightly differet from the agle pdf obtaied for a memoryless Gaussia source. However, oe ca still use a Gaussia source to obtai good estimates of the pdf of the agle i MP iteratios. The differece i the statistics meas that, although the residues of MP iteratios have similar orietatios, these orietatios are ot uiformly distributed. evertheless, the iid statistical model is ot too bad a assumptio. I the sequel, this model is used to desig Lloyd-Max quatizers of MP coefficiets. This permits to verify the validity of the statistical model for the agles i MP iteratios. It should be oted that the statistics of the first MP agle, ad therefore of the first coefficiet, are much more source depedet tha they are at further MP steps. Therefore, i order, to make a appropriate use of the preseted agle model, the first coefficiet, γ, will be quatized with egligible error ad ecoded as side iformatio. ote that this value replaces x, the value of the sigal orm which is usually trasmitted as side iformatio i practical MP based compressio schemes [, [4. III. Quatized Matchig Pursuit Decompositios I compressio applicatios, oe ecodes the quatized versios Q[γ of the coefficiets γ of M-terms. Oe way to obtai the Q[γ is to quatize the coefficiets offloop, i.e. first the whole decompositio is obtaied ad the the coefficiets are quatized. Aother strategy is iloop quatizatio [4, i which the quatized coefficiet is used to compute the residue, ad due to that i-loop quatizatio might result i atoms i M-term that deped o the quatizer used. 3 x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ Fig. 3. ormalized histograms of MP agles for a Gaussia source i R 64, usig bi, at = {, 8, 6, 3, 64, 7}, for the 4-phase Gabor dictioary i R x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ x 6 ormalized histogram of Θ Fig. 4. ormalized histograms of MP agles for a source with coordiates drive from a gamma distributio, usig bi, at = {, 8, 6, 3, 64, 7}, for the 4-phase Gabor dictioary i R 64. We choose off-loop quatizatio sice it allows a simple rate distortio (RD) optimizatio procedure, i which differet quatizers are tried out i order to fid oe that meets a prescribed RD criterio. I cotrast, i-loop quatizatio would require, for RD optimizatio, several MP sigal decompositios, what would greatly icrease the computatioal demads. Sice i off-loop quatizatio the residues are computed without kowledge of the quatizer, the quatizatio error i oe step caot be compesated i subsequet steps. However, these quatizatio errors ca be miimized by desigig appropriate quatizers [. I this work, MP coefficiet quatizers desig is achieved by employig the iid statistical model of MP agles to desig Lloyd-Max quatizers for the MP coefficiets. The quatizers desiged make use of dead-zoes because some advatage i RD performace ca be gaied by quatizig small coefficiets to zero [, [4: either the coefficiets quatized to zero or their atoms idices are SBrT 99
4 BRAZILVI ITERATIOAL TELECOMMUICATIOS SYMPOSIUM (ITS6), SEPTEMBER 3-6, 6, FORTALEZA-CE, BRAZIL 4 set, reducig the data rate. For trasmissio, it is importat that if coefficiets ad/or idices are lost, the decoder ca just successfully igore the lost terms whe recostructig the sigal. Therefore, it would be highly desirable for the quatizer for a give γ to be idepedet of the quatized values of other γ m (m ). A. Distortio Due to Off-Loop Quatizatio Whe M-terms are quatized off-loop, each coefficiet γ is replaced by its quatized versio Q[γ ad the sigal approximatio is retrieved usig eq. (4). The distortio criterio employed for MP quatizer desig ca be: i) the eergy of the error relative to the actual sigal d = x ˆx q ; () ii) the eergy of the error relative to the M-term d M = ˆx ˆx q = (γ Q[γ )g i(). () = I this work, the secod oe is used, sice we desig oe Lloyd-Max quatizer for each umber of coded terms M. The distortio per sample of the quatized M-term is give by d M = d M,thatis, d M = (γ Q[γ )(γ m Q[γ m ) g i(), g i(m), = m= (3) where is the sigal legth. D is composed of uit orm vectors ( g i() = ). Thus, defiig the quatizatio error e q (γ )=γ Q[γ, (4) it follows that [ d M = M e q (γ )+ (5) = e q (γ )e q (γ m ) g i(), g i(m). = m=, For a M-term ˆx, the miimizatio of eq. (5) implies also the miimizatio of the distortio per sample (see eq. ()) give by d = d = x ˆx q. (6) ote that if d M is equal to zero the d = r M x /. ) Distortio for a Give Source: For a give sigal source X, oe may cosider the expected value of d M { E[d M = M E [ e q(γ ) + (7) = E [ e q (Γ )e q (Γ m ) g i(), g i(m). = m=,m Each Γ is a radom variable (RV) that correspods to γ,for M, for sigals draw from X. I eq. (7), E [ e q(γ ) stads for the expected value of the squared quatizatio errors of the RV Γ. B. MP Coefficiet Quatizatio Usig MP Agles I eq. (9) the value of x is required to compute the MP coefficiets from the MP agles. Accordig to thediscussioattheedofsectioiiweusethefirst coefficiet γ istead of x, the eq. (9) ca be rewritte as γ = γ δ, δ =ta(θ ) si (θ i )cos(θ ),. (8) i= Thus, the pdfs of the coefficiets γ ca be computed from the pdfs of the agles Θ, assumig that the Θ are idepedet. The pdfs of the θ, >, ca be obtaied as explaied i sectio II. For a kow γ, the pdf of the RV Γ,for, is give by f (γ γ )=f (γ δ γ ), where is the RV whose outcome is δ, see eq. (8). If a optimal quatizer Q is desiged for the RV Y, the the optimal quatizer for Z = cy (c is a costat) is simply a scaled versio of Q. Thus,forγ kow, the quatizatio of δ istead of γ,fromeq.(7),gives { E[d M γ = γ M E [ e q ( ) + (9) = E [ e q ( )e q ( m ) g i(), g i(m). = m=,m Oce the pdfs of the RVs are kow E[d M γ cabe computed for ay quatizatio rule applied to.sice the quatizatio is applied to δ istead of γ, the value of γ is required at the decoder. I this work we use γ to defie γ,for. The use of γ to compute γ ( ) guaratees its correct value at the decoder. For quatizer desig usig eq. (9) the pdfs of are eeded which ca be computed from the RVs Θ i, i [,...,, correspodig to the agles betwee the residues ad the atoms selected i MP steps, as i eq. (8). IV. Lloyd-Max Quatizers for MP Coefficiets For a give first coefficiet γ the quatizatio of MP coefficiets should aim to miimize the distortio i eq. (9). I this case, oe should desig quatizers for,. Therefore, if just oe quatizer is to be used the it shall be desiged for = M =,thervgiveby the uio of the ( ). Sice MP iteratios are disjoit evets the pdf of = M = is give by f (δ) = M M = f (δ ). A. Distortio for a Optimal Quatizer The distortio per sample i eq. (9) has two terms. The first term is the sum of the squared quatizatio errors of, whereas the secod cotais a sum of ier products betwee dictioary atoms weighted by the products of the quatizatio errors of the atoms ivolved i the ier products. As verified, i sectio II, the RVs of the agles Θ ca be assumed to be ucorrelated. Although ad m may be correlated, whe desigig a quatizer for = M = the assumptio that the quatizatio errors e q ( ) are ucorrelated is reasoable. It is also SBrT 9
5 BRAZILVI ITERATIOAL TELECOMMUICATIOS SYMPOSIUM (ITS6), SEPTEMBER 3-6, 6, FORTALEZA-CE, BRAZIL 5 reasoable to assume that the quatizatio errors products e q ( )e q ( m ) are ot correlated to the ier products g i(), g i(m). Usig the assumptios above, the secod term i eq. (9) becomes E [e q ( ) E [e q ( m ) E [ g i(), g i(m). = m=,m () The atoms selected at differet MP steps may be correlated. But due to the ivariat ature of the agles statistics i differet MP steps, oe ca cosider the expected value of the ier product betwee the atoms selected i ay two differet MP steps ad m to be also ivariat. That is E [ g i(), g i(m) = c. Therefore, otig that E [e q ( ) = eq. () yields E [e q( ) = (M )E [e q( ) M m=,m E [e q ( ), () = = m=,m E [e q( m) c = () E [e q( m) c. The optimal quatizatio of leads to E [e q ( ) =, so that the expressio above vaishes. As a result eq. (9) becomes E[d M γ = γ E[( Q[ ), (3) = This result is a sum of terms E[( Q[ ), thus E[d M γ = γ (δ Q[δ ) f (δ )dδ. (4) = Sice the same quatizer Q[ is applied to all, M, oe has that E[d M γ =γ M (δ Q[δ) f (δ)dδ, (5) with f (δ) = f (δ ) (6) M = If = M = ad defiig eq. (5) becomes MSE( ) = E [ ( Q[ ) (7) E[d M γ = γ (M ) MSE( ). (8) The desig of a optimal quatizer for = M =,that miimizes eq. (8), is accomplished by Lloyd-Max quatizers [7 (). ote that sice Lloyd-Max quatizers are ubiased estimates of the iput the E[e q ( ) =, ad thus the term i eq. () vaishes validatig eq. (3). B. Lloyd-Max Quatizer Desig The desig of Lloyd-Max quatizers requires f (δ ), for M; these are estimated from f Θ (θ ). I tur, f Θ (θ ) is estimated by applyig oe MP decompositio step to a large set of sigals draw from a memoryless Gaussia source. As discussed i sectio II, this is a acceptable procedure sice all Θ,, have statistics that are similar to the oes obtaied for a memoryless Gaussia source. The estimated f (δ ) are the used to calculate f (δ), ad f (δ) istheusedtoobtaithe of b coef bits (L = b coef levels). The quatizer thresholds ad recostructio levels are calculated usig a iterative algorithm [7. The f (δ ), that are used to compute f (δ), are obtaied recursively from a estimate of f Θ (θ ) that is i tur obtaied usig the agle i the first MP step for a esemble of C(D) realizatios from a Gaussia source. The same quatizer law is used for all coefficiets γ. ote that f (δ) varies with M, therefore each M leads to a differet quatizer. The quatizer desig is idepedet of γ ad it suffices to desig quatizers for γ =, storig copies of the quatizers i both ecoder ad decoder. The ecoder seds γ, the umber of bits of the quatizer, ad the umber of terms of the decompositio (M), i a header, to the decoder. The parameter γ is used to scale the quatizer i both coder ad decoder, a simple strategy that makes good use of resources. V. Lloyd-Max Quatizers Performace The state-of-the-art for off-loop quatizatio of MP coefficiets is preseted i [. There, uiform quatizers whose umber of quatizatio levels ad rage adapt accordig to the coefficiets of the M-term are used. Here, this quatizatio scheme is referred to as adaptive bouded uiform quatizer (). The implemets a bit-allocatio per coefficiet that relies o the kow result that MP coefficiets magitudes decrease o average at each MP iteratio. I the, previous to quatizatio all the coefficiets eed to be sorted i decreasig magitude ad the is fed with the coefficiets i this order. For each coefficiet, the employs a uiform quatizer of differet rage ad umber of levels; the quatizer rage for the l th coefficiet depeds othequatizedvalueofthe(l ) th coefficiet, ad the umber of levels of each coefficiet quatizer is decided usig a criterio based o a Lagragia multiplier a bitallocatio procedure. For decodig the quatized decompositio the umber of bits used to quatize the secod coefficiet as well as the larger coefficiet are set as side iformatio. The total rate of MP quatized decompositios is give by R = S [log (C(D)) + r coef, (9) where S is the umber of terms that remai after quatizatio, ad r coef is the rate icurred i codig the quatized coefficiets. Therefore, the rate i bits per sample is give by R/, where is the sigal space dimesio. The SBrT 9
6 BRAZILVI ITERATIOAL TELECOMMUICATIOS SYMPOSIUM (ITS6), SEPTEMBER 3-6, 6, FORTALEZA-CE, BRAZIL 6 strategy employed to geerate the coded bitstream is to etropy code the differeces betwee the quatizatio idices of successive coefficiets. I the comparisos betwee Lloyd-Max quatizatio ad preseted here, this strategy is also employed to code the Lloyd-Max quatized M-terms. Fig. 5 shows the RD curves of quatized MP expasios arrivig from three differet radom sources i R (a memoryless Gaussia, a memoryless uiform ad a memoryless Gamma distributed source) usig both the Lloyd- Max quatizatio ad the ad a GSD(8, ). For each distict source the results are averages over a esemble of quatized MP decompositios of sigals from each source. For this experimet the s were desiged with bit-depth ragig from to 8. It ca be see i Fig. 5 that both quatizers have similar performace for the three sigal sources; however the teds to be slightly better at low rates (below 8 ). Fig. 6 shows the RD curves of quatized MP expasios arrivig from three differet radom sources i R 64 for the ad the for the Gabor dictioary of 4 phases i R 64, see eq. (). For that purpose the s were desiged with bit-depth ragig from to 6. The decompositios to be coded allowed a maximum of 56 terms. For each distict source the results are averages over a esemble of quatized MP decompositios of sigals from each source. The decompositios quatized with the require a larger bitrate because its bitallocatio scheme does ot maages to cotrol the rate as the shape of f Θ (θ ) for this dictioary implies a low decay rate of the coefficiets ivolved i M-terms obtaied usig this dictioary. VI. Coclusio The agles betwee the residues ad the selected atoms i Matchig Pursuit iteratios ca be statistically modeled as idepedet ad idetically distributed (iid) at each iteratio. That is, the agles i Matchig Pursuit steps ca be cosidered statistically ivariat with respect to the decompositio step. As a result, the statistics of Matchig Pursuit agles ca be obtaied from the statistics of the first Matchig Pursuit agle for sigals draw from a memoryless Gaussia source. Based o the iid statistical model for Matchig Pursuit agles, Lloyd-Max quatizers for Matchig Pursuit coefficiets were desiged. The Lloyd-Max quatizer is desiged to be the same for all the coefficiets i a M-term, ad its desig requires both the umber of terms to be coded ad the probability desity fuctio of the first Matchig Pursuit agle for a memoryless Gaussia source, resultig i dictioary depedet quatizers. It is importat to poit out that if the source is ot Gaussia the first MP agle has differet statistics tha the MP agles i other steps. However, eve i this case, the Lloyd-Max quatizers obtaied for the Gaussia source ca still be employed sice the first coefficiet is ot quatized but trasmitted with egligible error as side iformatio. E[d E[d E[d Memoryless Gaussia Source Memoryless Uiform Source Memoryless Gamma Distributed Source Fig. 5. ad RDs for three differet radom sources usig the GSD(8, ), the distortio is defied i eq. (6). E[d E[d E[d Memoryless Gaussia Source Memoryless Uiform Source Memoryless Gamma Distributed Source Fig. 6. ad RDs for three differet radom sources usig the 4-phase Gabor dictioary i R 64, the distortio is defied i eq. (6). The Lloyd-Max quatizatio preseted was compared to the state-of-the-art off-loop quatizatio scheme i [ ad both quatizatio schemes showed similar rate distortio performace. Refereces [ S. Mallat ad Z. Zhag, Matchig pursuits with time-frequecy dictioaries, IEEE Tras. o Sigal Processig, vol. 4, o., pp , December 993. [ P.Frossard,P.Vadergheyst,R.M.FiguerasIVetura,ad M. Kut, A posteriori quatizatio of progressive matchig pursuit streams, IEEE Tras. o Sigal Processig, vol. 5, o., pp , February 4. [3 L.Lovisolo,M.A.M.Rodrigues,E.A.B.daSilva,adP.S.R. Diiz, Efficiet coheret decompositios of power systems sigals usig damped siusoids, IEEE Tras. o Sigal Processig, vol. 53, pp , 5. [4 R. eff ad A. Zakhor, Modulus quatizatio for matchigpursuit video codig, IEEE Tras. o Circuits ad Systems for Video Techology, vol., pp ,. [5 S. Mallat, A Wavelet Tour of Sigal Processig, sted. Sa Diego, Califoria, USA: Academic Press, 998. [6 G. Davis, S. Mallat, ad M. Avellaeda, Adaptive greedy approximatios, Joural of Costructive Approximatio, vol. 3, pp , 997. [7 A. K. Jai, Fudametals of Digital Image Processig, 3rded., ser. Pretice Hall Iformatio Ad System Scieces Series. Eglewood Cliffs, J 763: Pretice Hall, 989. SBrT 9
Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE
Geeral e Image Coder Structure Motio Video (s 1,s 2,t) or (s 1,s 2 ) Natural Image Samplig A form of data compressio; usually lossless, but ca be lossy Redudacy Removal Lossless compressio: predictive
More informationOPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE
Joural of ELECTRICAL EGIEERIG, VOL. 56, O. 7-8, 2005, 200 204 OPTIMAL PIECEWISE UIFORM VECTOR QUATIZATIO OF THE MEMORYLESS LAPLACIA SOURCE Zora H. Perić Veljo Lj. Staović Alesadra Z. Jovaović Srdja M.
More informationVector Quantization: a Limiting Case of EM
. Itroductio & defiitios Assume that you are give a data set X = { x j }, j { 2,,, }, of d -dimesioal vectors. The vector quatizatio (VQ) problem requires that we fid a set of prototype vectors Z = { z
More informationVector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE
Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2004 Parma, Italy, October 10 13, 2004 Vector Permutatio Code Desig Algorithm Dailo SILVA ad Weiler A. FINAMORE Cetro de Estudos em Telecomuicações
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationSensitivity Analysis of Daubechies 4 Wavelet Coefficients for Reduction of Reconstructed Image Error
Proceedigs of the 6th WSEAS Iteratioal Coferece o SIGNAL PROCESSING, Dallas, Texas, USA, March -4, 7 67 Sesitivity Aalysis of Daubechies 4 Wavelet Coefficiets for Reductio of Recostructed Image Error DEVINDER
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationLecture 11: Channel Coding Theorem: Converse Part
EE376A/STATS376A Iformatio Theory Lecture - 02/3/208 Lecture : Chael Codig Theorem: Coverse Part Lecturer: Tsachy Weissma Scribe: Erdem Bıyık I this lecture, we will cotiue our discussio o chael codig
More informationFormation of A Supergain Array and Its Application in Radar
Formatio of A Supergai Array ad ts Applicatio i Radar Tra Cao Quye, Do Trug Kie ad Bach Gia Duog. Research Ceter for Electroic ad Telecommuicatios, College of Techology (Coltech, Vietam atioal Uiversity,
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationUC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 17 Lecturer: David Wagner April 3, Notes 17 for CS 170
UC Berkeley CS 170: Efficiet Algorithms ad Itractable Problems Hadout 17 Lecturer: David Wager April 3, 2003 Notes 17 for CS 170 1 The Lempel-Ziv algorithm There is a sese i which the Huffma codig was
More informationECE 564/645 - Digital Communication Systems (Spring 2014) Final Exam Friday, May 2nd, 8:00-10:00am, Marston 220
ECE 564/645 - Digital Commuicatio Systems (Sprig 014) Fial Exam Friday, May d, 8:00-10:00am, Marsto 0 Overview The exam cosists of four (or five) problems for 100 (or 10) poits. The poits for each part
More informationModule 5 EMBEDDED WAVELET CODING. Version 2 ECE IIT, Kharagpur
Module 5 EMBEDDED WAVELET CODING Versio ECE IIT, Kharagpur Lesso 4 SPIHT algorithm Versio ECE IIT, Kharagpur Istructioal Objectives At the ed of this lesso, the studets should be able to:. State the limitatios
More informationThe Maximum-Likelihood Decoding Performance of Error-Correcting Codes
The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationGeometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT
OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture 9: Pricipal Compoet Aalysis The text i black outlies mai ideas to retai from the lecture. The text i blue give a deeper uderstadig of how we derive or get
More informationElement sampling: Part 2
Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationThe multiplicative structure of finite field and a construction of LRC
IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science. BACKGROUND EXAM September 30, 2004.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Departmet of Electrical Egieerig ad Computer Sciece 6.34 Discrete Time Sigal Processig Fall 24 BACKGROUND EXAM September 3, 24. Full Name: Note: This exam is closed
More informationTHE SYSTEMATIC AND THE RANDOM. ERRORS - DUE TO ELEMENT TOLERANCES OF ELECTRICAL NETWORKS
R775 Philips Res. Repts 26,414-423, 1971' THE SYSTEMATIC AND THE RANDOM. ERRORS - DUE TO ELEMENT TOLERANCES OF ELECTRICAL NETWORKS by H. W. HANNEMAN Abstract Usig the law of propagatio of errors, approximated
More informationw (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.
2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For
More informationOrthogonal Gaussian Filters for Signal Processing
Orthogoal Gaussia Filters for Sigal Processig Mark Mackezie ad Kiet Tieu Mechaical Egieerig Uiversity of Wollogog.S.W. Australia Abstract A Gaussia filter usig the Hermite orthoormal series of fuctios
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationModule 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
More informationDiscrete-Time Systems, LTI Systems, and Discrete-Time Convolution
EEL5: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we begi our mathematical treatmet of discrete-time s. As show i Figure, a discrete-time operates or trasforms some iput sequece x [
More informationADVANCED DIGITAL SIGNAL PROCESSING
ADVANCED DIGITAL SIGNAL PROCESSING PROF. S. C. CHAN (email : sccha@eee.hku.hk, Rm. CYC-702) DISCRETE-TIME SIGNALS AND SYSTEMS MULTI-DIMENSIONAL SIGNALS AND SYSTEMS RANDOM PROCESSES AND APPLICATIONS ADAPTIVE
More informationTime-Domain Representations of LTI Systems
2.1 Itroductio Objectives: 1. Impulse resposes of LTI systems 2. Liear costat-coefficiets differetial or differece equatios of LTI systems 3. Bloc diagram represetatios of LTI systems 4. State-variable
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationLecture 15: Learning Theory: Concentration Inequalities
STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 5: Learig Theory: Cocetratio Iequalities Istructor: Ye-Chi Che 5. Itroductio Recall that i the lecture o classificatio, we have see that
More informationReliability and Queueing
Copyright 999 Uiversity of Califoria Reliability ad Queueig by David G. Messerschmitt Supplemetary sectio for Uderstadig Networked Applicatios: A First Course, Morga Kaufma, 999. Copyright otice: Permissio
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationStatistical Noise Models and Diagnostics
L. Yaroslavsky: Advaced Image Processig Lab: A Tutorial, EUSIPCO2 LECTURE 2 Statistical oise Models ad Diagostics 2. Statistical models of radom iterfereces: (i) Additive sigal idepedet oise model: r =
More informationProbability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More informationOFDM Precoder for Minimizing BER Upper Bound of MLD under Imperfect CSI
MIMO-OFDM OFDM Precoder for Miimizig BER Upper Boud of MLD uder Imperfect CSI MCRG Joit Semiar Jue the th 008 Previously preseted at ICC 008 Beijig o May the st 008 Boosar Pitakdumrogkija Kazuhiko Fukawa
More information1 Hash tables. 1.1 Implementation
Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationMath 216A Notes, Week 5
Math 6A Notes, Week 5 Scribe: Ayastassia Sebolt Disclaimer: These otes are ot early as polished (ad quite possibly ot early as correct) as a published paper. Please use them at your ow risk.. Thresholds
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.
More informationChapter Vectors
Chapter 4. Vectors fter readig this chapter you should be able to:. defie a vector. add ad subtract vectors. fid liear combiatios of vectors ad their relatioship to a set of equatios 4. explai what it
More informationInformation-based Feature Selection
Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with
More informationLecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound
Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee
More informationOptimum LMSE Discrete Transform
Image Trasformatio Two-dimesioal image trasforms are extremely importat areas of study i image processig. The image output i the trasformed space may be aalyzed, iterpreted, ad further processed for implemetig
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More information4.3 Growth Rates of Solutions to Recurrences
4.3. GROWTH RATES OF SOLUTIONS TO RECURRENCES 81 4.3 Growth Rates of Solutios to Recurreces 4.3.1 Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer.
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationECEN 655: Advanced Channel Coding Spring Lecture 7 02/04/14. Belief propagation is exact on tree-structured factor graphs.
ECEN 655: Advaced Chael Codig Sprig 014 Prof. Hery Pfister Lecture 7 0/04/14 Scribe: Megke Lia 1 4-Cycles i Gallager s Esemble What we already kow: Belief propagatio is exact o tree-structured factor graphs.
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationSession 5. (1) Principal component analysis and Karhunen-Loève transformation
200 Autum semester Patter Iformatio Processig Topic 2 Image compressio by orthogoal trasformatio Sessio 5 () Pricipal compoet aalysis ad Karhue-Loève trasformatio Topic 2 of this course explais the image
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationON POINTWISE BINOMIAL APPROXIMATION
Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 57-66 ON POINTWISE BINOMIAL APPROXIMATION BY w-functions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationLecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
More informationFilter banks. Separately, the lowpass and highpass filters are not invertible. removes the highest frequency 1/ 2and
Filter bas Separately, the lowpass ad highpass filters are ot ivertible T removes the highest frequecy / ad removes the lowest frequecy Together these filters separate the sigal ito low-frequecy ad high-frequecy
More informationMonte Carlo Integration
Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce
More informationReport on Private Information Retrieval over Unsynchronized Databases
Report o Private Iformatio Retrieval over Usychroized Databases Lembit Valgma Supervised by Vitaly Skachek May 25, 217 1 Problem Statemet There are may challeges cocerig olie privacy. Private iformatio
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More information6.867 Machine learning, lecture 7 (Jaakkola) 1
6.867 Machie learig, lecture 7 (Jaakkola) 1 Lecture topics: Kerel form of liear regressio Kerels, examples, costructio, properties Liear regressio ad kerels Cosider a slightly simpler model where we omit
More informationOlli Simula T / Chapter 1 3. Olli Simula T / Chapter 1 5
Sigals ad Systems Sigals ad Systems Sigals are variables that carry iformatio Systemstake sigals as iputs ad produce sigals as outputs The course deals with the passage of sigals through systems T-6.4
More informationLossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding
ossy Compressio via Sparse iear Regressio: Computatioally Efficiet Ecodig ad Decodig Ramji Vekataramaa Dept of Egieerig, Uiv of Cambridge ramjiv@egcamacuk Tuhi Sarkar Dept of EE, IIT Bombay tuhi91@gmailcom
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationRandom Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices
Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio
More information5.1 Review of Singular Value Decomposition (SVD)
MGMT 69000: Topics i High-dimesioal Data Aalysis Falll 06 Lecture 5: Spectral Clusterig: Overview (cotd) ad Aalysis Lecturer: Jiamig Xu Scribe: Adarsh Barik, Taotao He, September 3, 06 Outlie Review of
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More information6.883: Online Methods in Machine Learning Alexander Rakhlin
6.883: Olie Methods i Machie Learig Alexader Rakhli LECTURES 5 AND 6. THE EXPERTS SETTING. EXPONENTIAL WEIGHTS All the algorithms preseted so far halluciate the future values as radom draws ad the perform
More informationOblivious Transfer using Elliptic Curves
Oblivious Trasfer usig Elliptic Curves bhishek Parakh Louisiaa State Uiversity, ato Rouge, L May 4, 006 bstract: This paper proposes a algorithm for oblivious trasfer usig elliptic curves lso, we preset
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationThe Minimum Distance Energy for Polygonal Unknots
The Miimum Distace Eergy for Polygoal Ukots By:Johaa Tam Advisor: Rollad Trapp Abstract This paper ivestigates the eergy U MD of polygoal ukots It provides equatios for fidig the eergy for ay plaar regular
More informationThe DOA Estimation of Multiple Signals based on Weighting MUSIC Algorithm
, pp.10-106 http://dx.doi.org/10.1457/astl.016.137.19 The DOA Estimatio of ultiple Sigals based o Weightig USIC Algorithm Chagga Shu a, Yumi Liu State Key Laboratory of IPOC, Beijig Uiversity of Posts
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationStatistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons
Statistical Aalysis o Ucertaity for Autocorrelated Measuremets ad its Applicatios to Key Comparisos Nie Fa Zhag Natioal Istitute of Stadards ad Techology Gaithersburg, MD 0899, USA Outlies. Itroductio.
More informationShannon s noiseless coding theorem
18.310 lecture otes May 4, 2015 Shao s oiseless codig theorem Lecturer: Michel Goemas I these otes we discuss Shao s oiseless codig theorem, which is oe of the foudig results of the field of iformatio
More informationLecture 11: Decision Trees
ECE9 Sprig 7 Statistical Learig Theory Istructor: R. Nowak Lecture : Decisio Trees Miimum Complexity Pealized Fuctio Recall the basic results of the last lectures: let X ad Y deote the iput ad output spaces
More information5. Fast NLMS-OCF Algorithm
5. Fast LMS-OCF Algorithm The LMS-OCF algorithm preseted i Chapter, which relies o Gram-Schmidt orthogoalizatio, has a compleity O ( M ). The square-law depedece o computatioal requiremets o the umber
More informationModified Logistic Maps for Cryptographic Application
Applied Mathematics, 25, 6, 773-782 Published Olie May 25 i SciRes. http://www.scirp.org/joural/am http://dx.doi.org/.4236/am.25.6573 Modified Logistic Maps for Cryptographic Applicatio Shahram Etemadi
More informationChandrasekhar Type Algorithms. for the Riccati Equation of Lainiotis Filter
Cotemporary Egieerig Scieces, Vol. 3, 00, o. 4, 9-00 Chadrasekhar ype Algorithms for the Riccati Equatio of Laiiotis Filter Nicholas Assimakis Departmet of Electroics echological Educatioal Istitute of
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More information