12. Maximum Entropy and Spectrum Estimation
|
|
- Griffin Lawrence
- 5 years ago
- Views:
Transcription
1 : Maxu Etropy ad pectru Estato Maru Jutt Overvew Maxu etropy dstrbutos are studed The proble of spectru estato s troduced ad axu etropy based solutos for t are llustrated ource The ateral s aly based o Chapter of the course boo [] Teleco Laboratory Course Overvew Basc cocepts ad tools Itroducto Etropy, relatve etropy ad utual forato 3 Asyptotc equpartto property 4 Etropy rates of a stochastc process ource codg or data copresso 5 Data copresso Chael capacty 8 Chael capacty 9 Dfferetal etropy 0 The Gaussa chael Other applcatos Maxu etropy ad spectral estato 3 Rate dstorto theory 4 Networ forato theory Teleco Laboratory Outle of the Lecture Revew of the last lecture Itroducto Maxu etropy dstrbutos pectru estato Etropy rates of a Gaussa process Burg s axu etropy theore uary Refereces Chael put at te X Revew of Last Lecture The Gaussa chael wth dscrete te dex ad cotuous apltude The ost portat cotuous alphabet chael s the Gaussa chael Wthout costrats o the sgal-to-ose rato (NR) the capacty would be fte Usually a power costrat s appled for a codeword of legth : x P = Gaussa ose at te Z ~N(0,σ Z ); depedet o put X Σ Chael output at te Y = X + Z Teleco Laboratory 3 Teleco Laboratory 4
2 Capacty wth Nyqust Badwdth hao Boud for Codg Methods: Probablty of Error Boud vs NR per Bt The u badwdth for relable (tersybol terferece free) coucato [Dgtal Coucatos] s the Nyqust badwdth: W Nyq = R b /R The capacty per te ut wth P = R b E b becoes P RbE b C = W log + = W log + N 0W N 0W The capacty per sgle chael use wth W = W Nyq = R b /R: P RbE b RE b C = log log log + N0W = + = + N0Rb /R N 0 Teleco Laboratory 5 Teleco Laboratory 6 Requred NR per Bt vs Code Rate Mu NR for Relable Coucatos If tae the ltg case of fte badwdth (W ): RbE b RbE b C = W log + log e, W N 0W N 0 The faous lower boud for NR per bt for relable coucatos: RbE b C = loge Rb N E b N = 59 db loge Teleco Laboratory 7 Teleco Laboratory 8
3 pectral Effcecy Oe ca slarly derve a boud for spectral effcecy r = R b /W [(bts/s)/hz]: E r b N r 0 Teleco Laboratory 9 Parallal Gaussa Chael Model Chael put vector: T [ ] X = X X X Nose vector: T Z = Z Z Z ~ N 0, Chael output vector: [ ] Y = Y Y Y = X + Nose covarace: K = dag σz,, σz,,, σz, Ucorrelated for the te beg The coo power costrat: E X P = K T [ ] Z [ ] Z ~ N (0, σ X Y Z ~ N (0, σ X Parallel Gaussa chaels Teleco Laboratory 0 Σ Σ Z, Z, X Y ) ) Z ~ N (0, σz, ) Σ Y Power Allocato Optzato: Water-Fllg oluto Optal Power Allocato oluto wth Colored Gaussa Nose P = = ( ν σ ) ( ν σ ) + Z, + Z, = P, x, x 0 0, x < 0 + ( x ) =, Power P P P 3 σ Z, σ Z, σ Z,3 P 4 =0 σ Z,4 Ch # Ch # Ch #3 Ch #4 ce for ay atrces B ad C tr(bc) = tr(cb), the trace the power costrat becoes tr(a) = tr(q T K X Q) = tr(qq T K X ) = tr(k X ) Maxze A+Λ subject to tr(a) P Apply Hadaard equalty: K K wth equalty f ad oly f K s dagoal A + Λ A = ( + λ ) larly to the water-fllg, the soluto s + ( ν λ ), A P A = = = Teleco Laboratory Teleco Laboratory
4 Feedbac Capacty The capacty wth feedbac per oe trassso: ( ) C K ax X + Z,FB = log + bts, ( ) ( ) C tr K P K X where the capacty wthout feedbac C s ( ) ( ) ax log KX + KZ C = tr K P K X Ay achevable code satsfes ( ) K X + Z R = log + ε, ε 0, K Z Z Z Itroducto Maxu etropy dstrbutos have bee cosdered therodyacs to characterze the olecule veloctes or crostates gas Maxwell-Boltza dstrbuto Maxu etropy dstrbutos ca be utlzed paraetrc spectru estato ad other slar estato probles elect such a estate value whch assues the least about the data, e, axzes the etropy Robustess Teleco Laboratory 3 Teleco Laboratory 4 Maxu Etropy Dstrbutos Cosder the proble of selectg the dstrbuto so that the dfferetal etropy s axzed Gaussa PDF s the soluto wth covarace costrat Maxze the dfferetal etropy h(f) over all PDF s satsfyg (r(x) s a arbtrary fucto used oet costrat) f(x) 0, wth equalty outsde the support set f ( x ) dx =, f 3 = oluto approaches: calculus forato equalty ( x ) r ( x ) dx α, =,,, Approach #: Calculus Dfferetal etropy s a cocave fucto over a covex set For fuctoal: J ( f ) = f l f + λ 0 f + λ fr = Dfferetate wth respect to f(x), the copoet of f: J = lf ( x ) + λ 0 + λr ( x ) f ( x ) = ettg the dervatve to zero yelds: λ0 + λ r ( x ) = f ( x ) = e, x, where λ 0, λ,, λ are chose to satsfy the costrats Teleco Laboratory 5 Teleco Laboratory 6
5 Approach #: Iforato Iequalty The calculus approach above dd ot prove that the solved PDF yelds deed the axu etropy, but t rather just suggest a approprate for for the correct soluto Iforato equalty ca be used for a rgorous dervato of the axu etropy dstrbuto Ths s foralzed the followg theore Theore: Maxu Etropy Dstrbuto Let where λ 0, λ,, λ, are chose so that f* satsfes f(x) 0, wth equalty outsde the support set 3 λ 0 + λ r ( x ) = f =, x, f x x e = f λ f Fucto f* uquely axzes h(f) over all PDF s satsfyg the costrats The sae approach ad slar soluto hold also for dscrete etropes, ad ultvarate dstrbutos ( x ) dx =, ( x ) r ( x ) dx = α, =,,, Teleco Laboratory 7 Teleco Laboratory 8 Proof Let g be ay PDF whch satsfes the costrats 3 above g h ( g ) = g lg = g l f = D ( g f ) g lf f Equalty f ad oly f g(x) = f*(x) g lf = g l exp λ 0 + λr ( x ) = = f λ 0 + λr ( x ) = = f l f = h f by o-egatvty of relatve etropy by defto of f* both f* ad g satsfy the costrats Teleco Laboratory 9 Exaple #: Frst ad ecod Moet Costrats Cosder the case = (, ) = R Let the costrats be E(X) = 0 ad E(X ) = σ Falar varace costrat The soluto s the Gaussa (oral) dstrbuto N(0,σ ) To verfy t, let us ote that E(X) = 0 ad E(X ) = σ E ( X ) = 0 xf ( x ) dx = 0, Accordg to the prevous theore: ( X ) E = σ x f x dx = σ f x = e λ 0 +λ x +λ x Teleco Laboratory 0
6 Exaple #: No Costrats ad Closed Iterval Cosder the case = [a,b] Assue o costrats except the trval frst two b oes: f ( x ) dx =, The soluto s of the for Ufor dstrbuto a λ = [ ] f x e 0, x a, b Exaple #3: No Costrats vs the Frst ad ecod Moet Costrats Cosder the case = (, ) = R Assue o costrats The axu etropy s fte ad there s o axu etropy PDF Cosder, eg, Gaussa PDF wth σ Let the costrats be geeral frst ad secod oet costrats E(X) = µ ad E(X ) = α The soluto s the Gaussa (oral) dstrbuto N(µ,α µ ) Follows drectly fro the fact that Gaussa PDF axzes the dfferetal etropy uder covarace costrat Teleco Laboratory Teleco Laboratory Exaple #4: Multvarate ecod Moet Costrats Cosder the case = R wth the secod oet costrats E(X X ) j = K j λ 0 + λj x x j, j = The axu etropy PDF: f ( x) = e Aga, sce the expoet s of a quadratc for, t s clear by the earler results that the soluto ust be the ultvarate Gaussa PDF: T f ( x) = f ( x, x,, x ) = exp, x K x K ( π) h ( f ) = h ( X, X,, X ) = log ( πe ) K Teleco Laboratory 3 Exaple #5: A Aoalous Maxu Etropy Proble () Let us cosder a slar set-up as Exaple #: Now we add a thrd costrat there: The soluto ust be of the for: f x = e However, f λ 3 0 f x dx = The PDF caot be oralzed correctly xf x dx = α xf x dx = α 3 λ 0 +λ x +λ x +λ3 x, Teleco Laboratory 4 3 x f x dx = α 3
7 Exaple #5: A Aoalous Maxu Etropy Proble () Four equatos ad three varables I geeral, the soluto does ot exst The dfferetal etropy has a upper boud gve by the Gaussa PDF, but t s ot possble to atta t However, we ca get arbtrarly close Cocluso: suph(f) = h(n(µ,α α )) = ½lπe(α α ) Exaple #6: Dce Cosder dce [oppa, Fsh] so that = {,,3,4,5,6} Dscrete RV dscrete etropy s cosdered Wth far dce wth p(x) = /6, the etropy s axzed wth the ufor dstrbuto Wth a costrat E(X) = α, [Boltza] the proble becoes ore trcy λ e The soluto s [, Exaple 3]: p =, 6 λ where λ s such that 6 e p = α = = Teleco Laboratory 5 Teleco Laboratory 6 pectru Estato Autocorrelato fucto of a statoary stochastc process: R ( ) = E( X X + ) Power spectral desty (Fourer trasfor): jλ λ = R e, π λ π = pectru estato attepts to estate the power spectral desty or equvaletly the autocorrelato everal ethods exst A deeper treatet the course Coucato gal Processg I pectru Estato Methods Proble: Fro a fte-legth data record, detere a estate of the power spectral desty Noparaetrc ethods: No paraetrc odel of the syste assued Methods: perodogra-based ethods ultple wdow ethod Paraetrc ethods: Assue a paraetrc stochastc odel Methods: odel detfcato procedures u varace dstortoless respose ethod egedecoposto-based ethods Teleco Laboratory 7 Teleco Laboratory 8
8 Paraetrc Methods Assue a paraetrc stochastc odel Model detfcato procedures Mu varace dstortoless respose ethod 3 Egedecoposto-based ethods Noparaetrc Methods No assuptos tartg pot: the fudaetal equato Two dfferet approaches ca be dstgushed Perodogra-based ethods perodogra U N (ω) /N s used as the startg pot Multple wdow ethod the fudaetal equato appled pectra coputed usg the paraetrc ethods ted to have sharper peas ad hgher resoluto tha those obtaed fro the oparaetrc ethods Teleco Laboratory 9 Teleco Laboratory 30 Perodogra Method = aple estate: Rˆ = X X + splest Perodogra ethod: estate the spectru based o saple estates More saple values for low lags (sall values of ) tha hgh lags (large values of ) If correlato coverges slowly to zero, the spectru estate wll be based Poor estates et large lag autocorrelatos to zero, use wdowg Maxu Etropy Method The axu etropy ethod (MEM) or Burg s ethod s based o forato theoretc arguets To ze the ucertaty related to the easureets, choose the ost upredctable estate, e, the oe whch axzes the etropy (or ucertaty) of the uderlyg rado varable Bypasses the probles of wdowg fuctos perodc exteso of the data or assupto of zero data outsde the observato terval Noegatve at all frequeces Teleco Laboratory 3 Teleco Laboratory 3
9 Etropy Rates of a Gaussa Process Etropy rate of a dscrete valued stochastc process was studed earler It ca be exteded for a cotuous-valued stochastc processes as well Dfferetal etropy rate (f the lt exsts): h ( X, X,, X ) h ( X) = l For statoary processes: h( X) = l h( X X,, X, X) For a statoary Gaussa stochastc process: ( ) h ( X, X,, X ) = log ( πe ) det( K ), where K () s covarace atrx wth K () j =R( -j ) Etropy Rate of a Gaussa Process () As, the egevalues of the covarace atrx approach the spectru of the stochastc process Etropy rate of a statoary Gaussa stochastc process: π hx = log( πe ) + log[ ( λ) ] dλ 4π π Varace of the best estator gve the fte past: h( X σ ) = πe Teleco Laboratory 33 Teleco Laboratory 34 Burg s Maxu Etropy Theore The axu etropy rate stochastc process {X} satsfyg E X X = α, = 0,,, p +, s the pth order Gauss-Marov process: p X = a X + Z, = where Z are IID zero ea Gaussa RV s wth varace σ Costats are chose to satsfy the costrats, ad ca be solved by Yule-Waler equatos Rears Rear: {X } s ot assued to a) have zero ea b) be Gaussa c) Be statoary Proof: ee textboo [, ect 6] for coplete proof The dfferetal etropy of a fte seget of a stochastc s bouded above by the etropy of a seget of a Gaussa rado process wth the sae covarace structure Ths s tur bouded above by the al order Gauss- Marov process Characterzed by Yule-Waler equatos Teleco Laboratory 35 Teleco Laboratory 36
10 Yule-Waler Equatos ad the Maxu Etropy pectru Estate The pth order Gauss-Marov process s characterzed by the well-ow Yule-Waler p equatos: R ( 0 ) = a R ( ) + σ, p+ equatos ad uows a uque soluto exsts Fast soluto by Levso algorth The spectru of the axu etropy process: σ ( l ) = p jl + a e = p R l = a R l, l =,, = = Teleco Laboratory 37 uary Maxu etropy dstrbuto: Maxu etropy spectru estate: λ 0 + λ r ( x ) = = =,, f x f x e x λ ( l ) = p = jl The paraeters are solved fro Yule-Waler equatos Ca be used other probles as well Exaple: rado chael odelg σ + a e Teleco Laboratory 38 Refereces o Hay, Adaptve Flter Theory, 3rd ed Pretce Hall, 996 IBN: X P toca & R Moses, Itroducto to pectral Aalyss Pretce-Hall, 997 IBN M Jutt, Coucato Processg I Lecture otes, Uversty of Oulu, Dept Electrcal ad Ifor Eg, Teleco Laboratory Teleco Laboratory 39
5. Data Compression. Review of Last Lecture. Outline of the Lecture. Course Overview. Basics of Information Theory: Markku Juntti
: Markku Jutt Overvew The deas of lossless data copresso ad source codg are troduced ad copresso lts are derved. Source The ateral s aly based o Sectos 5. 5.5 of the course book []. Teleco. Laboratory
More informationBasics of Information Theory: Markku Juntti. Basic concepts and tools 1 Introduction 2 Entropy, relative entropy and mutual information
: Maru Jutt Overvew he propertes of adlmted Gaussa chaels are further studed, parallel Gaussa chaels ad Gaussa chaels wth feedac are solved. Source he materal s maly ased o Sectos.4.6 of the course oo
More information3. Basic Concepts: Consequences and Properties
: 3. Basc Cocepts: Cosequeces ad Propertes Markku Jutt Overvew More advaced cosequeces ad propertes of the basc cocepts troduced the prevous lecture are derved. Source The materal s maly based o Sectos.6.8
More informationSome Different Perspectives on Linear Least Squares
Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,,
More informationDepartment of Mathematics UNIVERSITY OF OSLO. FORMULAS FOR STK4040 (version 1, September 12th, 2011) A - Vectors and matrices
Deartet of Matheatcs UNIVERSITY OF OSLO FORMULAS FOR STK4040 (verso Seteber th 0) A - Vectors ad atrces A) For a x atrx A ad a x atrx B we have ( AB) BA A) For osgular square atrces A ad B we have ( )
More informationAlgorithms behind the Correlation Setting Window
Algorths behd the Correlato Settg Wdow Itroducto I ths report detaled forato about the correlato settg pop up wdow s gve. See Fgure. Ths wdow s obtaed b clckg o the rado butto labelled Kow dep the a scree
More informationA New Method for Solving Fuzzy Linear. Programming by Solving Linear Programming
ppled Matheatcal Sceces Vol 008 o 50 7-80 New Method for Solvg Fuzzy Lear Prograg by Solvg Lear Prograg S H Nasser a Departet of Matheatcs Faculty of Basc Sceces Mazadara Uversty Babolsar Ira b The Research
More informationRelations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels
Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October
More informationPRACTICAL CONSIDERATIONS IN HUMAN-INDUCED VIBRATION
PRACTICAL CONSIDERATIONS IN HUMAN-INDUCED VIBRATION Bars Erkus, 4 March 007 Itroducto Ths docuet provdes a revew of fudaetal cocepts structural dyacs ad soe applcatos hua-duced vbrato aalyss ad tgato of
More informationLecture 8 IEEE DCF Performance
Lecture 8 IEEE82. DCF Perforace IEEE82. DCF Basc Access Mechas A stato wth a ew packet to trast otors the chael actvty. If the chael s dle for a perod of te equal to a dstrbuted terfrae space (DIFS), the
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationA Bivariate Distribution with Conditional Gamma and its Multivariate Form
Joural of Moder Appled Statstcal Methods Volue 3 Issue Artcle 9-4 A Bvarate Dstrbuto wth Codtoal Gaa ad ts Multvarate For Sue Se Old Doo Uversty, sxse@odu.edu Raja Lachhae Texas A&M Uversty, raja.lachhae@tauk.edu
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 17
CS434a/54a: Patter Recogto Prof. Olga Vesler Lecture 7 Today Paraetrc Usupervsed Learg Expectato Maxato (EM) oe of the ost useful statstcal ethods oldest verso 958 (Hartley) seal paper 977 (Depster et
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationA Penalty Function Algorithm with Objective Parameters and Constraint Penalty Parameter for Multi-Objective Programming
Aerca Joural of Operatos Research, 4, 4, 33-339 Publshed Ole Noveber 4 ScRes http://wwwscrporg/oural/aor http://ddoorg/436/aor4463 A Pealty Fucto Algorth wth Obectve Paraeters ad Costrat Pealty Paraeter
More informationStationary states of atoms and molecules
Statoary states of atos ad olecules I followg wees the geeral aspects of the eergy level structure of atos ad olecules that are essetal for the terpretato ad the aalyss of spectral postos the rotatoal
More informationDATA DOMAIN DATA DOMAIN
3//6 Coprght otce: Most ages these sldes are Gozalez ad oods Pretce-Hall Note: ages are [spatall] ostatoar sgals. e eed tools to aalze the locall at dfferet resolutos e ca do ths the data doa or sutable
More informationChapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
More information3.1 Introduction to Multinomial Logit and Probit
ES3008 Ecooetrcs Lecture 3 robt ad Logt - Multoal 3. Itroducto to Multoal Logt ad robt 3. Estato of β 3. Itroducto to Multoal Logt ad robt The ultoal Logt odel s used whe there are several optos (ad therefore
More informationBasics of Information Theory: Markku Juntti. Basic concepts and tools 1 Introduction 2 Entropy, relative entropy and mutual information
: Markku Jutt Overvew Te basc cocepts o ormato teory lke etropy mutual ormato ad EP are eeralzed or cotuous-valued radom varables by troduc deretal etropy ource Te materal s maly based o Capter 9 o te
More informationOn Probability of Undetected Error for Hamming Codes over Q-ary Symmetric Channel
Joural of Coucato ad Coputer 8 (2 259-263 O Probablty of Udetected Error for Hag Codes over Q-ary Syetrc Chael Mash Gupta, Jaskar Sgh Bhullar 2 ad O Parkash Vocha 3. D.A.V. College, Bathda 5, Ida 2. Malout
More informationConstruction of Composite Indices in Presence of Outliers
Costructo of Coposte dces Presece of Outlers SK Mshra Dept. of Ecoocs North-Easter Hll Uversty Shllog (da). troducto: Oftetes we requre costructg coposte dces by a lear cobato of a uber of dcator varables.
More informationCoherent Potential Approximation
Coheret Potetal Approxato Noveber 29, 2009 Gree-fucto atrces the TB forals I the tght bdg TB pcture the atrx of a Haltoa H s the for H = { H j}, where H j = δ j ε + γ j. 2 Sgle ad double uderles deote
More informationSolutions to problem set ); (, ) (
Solutos to proble set.. L = ( yp p ); L = ( p p ); y y L, L = yp p, p p = yp p, + p [, p ] y y y = yp + p = L y Here we use for eaple that yp, p = yp p p yp = yp, p = yp : factors that coute ca be treated
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationKURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS. Peter J. Wilcoxen. Impact Research Centre, University of Melbourne.
KURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS by Peter J. Wlcoxe Ipact Research Cetre, Uversty of Melboure Aprl 1989 Ths paper descrbes a ethod that ca be used to resolve cossteces
More information7.0 Equality Contraints: Lagrange Multipliers
Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse
More informationLecture 4 Sep 9, 2015
CS 388R: Radomzed Algorthms Fall 205 Prof. Erc Prce Lecture 4 Sep 9, 205 Scrbe: Xagru Huag & Chad Voegele Overvew I prevous lectures, we troduced some basc probablty, the Cheroff boud, the coupo collector
More informationD. L. Bricker, 2002 Dept of Mechanical & Industrial Engineering The University of Iowa. CPL/XD 12/10/2003 page 1
D. L. Brcker, 2002 Dept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/XD 2/0/2003 page Capactated Plat Locato Proble: Mze FY + C X subject to = = j= where Y = j= X D, j =, j X SY, =,... X 0, =,
More informationUnsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationD. VQ WITH 1ST-ORDER LOSSLESS CODING
VARIABLE-RATE VQ (AKA VQ WITH ENTROPY CODING) Varable-Rate VQ = Quatzato + Lossless Varable-Legth Bary Codg A rage of optos -- from smple to complex A. Uform scalar quatzato wth varable-legth codg, oe
More informationA Conventional Approach for the Solution of the Fifth Order Boundary Value Problems Using Sixth Degree Spline Functions
Appled Matheatcs, 1, 4, 8-88 http://d.do.org/1.4/a.1.448 Publshed Ole Aprl 1 (http://www.scrp.org/joural/a) A Covetoal Approach for the Soluto of the Ffth Order Boudary Value Probles Usg Sth Degree Sple
More informationAnalysis of Lagrange Interpolation Formula
P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal
More information2/20/2013. Topics. Power Flow Part 1 Text: Power Transmission. Power Transmission. Power Transmission. Power Transmission
/0/0 Topcs Power Flow Part Text: 0-0. Power Trassso Revsted Power Flow Equatos Power Flow Proble Stateet ECEGR 45 Power Systes Power Trassso Power Trassso Recall that for a short trassso le, the power
More informationNon-degenerate Perturbation Theory
No-degeerate Perturbato Theory Proble : H E ca't solve exactly. But wth H H H' H" L H E Uperturbed egevalue proble. Ca solve exactly. E Therefore, kow ad. H ' H" called perturbatos Copyrght Mchael D. Fayer,
More informationBounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy
Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More informationSome results and conjectures about recurrence relations for certain sequences of binomial sums.
Soe results ad coectures about recurrece relatos for certa sequeces of boal sus Joha Cgler Faultät für Matheat Uverstät We A-9 We Nordbergstraße 5 Joha Cgler@uveacat Abstract I a prevous paper [] I have
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationArithmetic Mean and Geometric Mean
Acta Mathematca Ntresa Vol, No, p 43 48 ISSN 453-6083 Arthmetc Mea ad Geometrc Mea Mare Varga a * Peter Mchalča b a Departmet of Mathematcs, Faculty of Natural Sceces, Costate the Phlosopher Uversty Ntra,
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationStandard Deviation for PDG Mass Data
4 Dec 06 Stadard Devato for PDG Mass Data M. J. Gerusa Retred, 47 Clfde Road, Worghall, HP8 9JR, UK. gerusa@aol.co, phoe: +(44) 844 339754 Abstract Ths paper aalyses the data for the asses of eleetary
More informationDerivation of 3-Point Block Method Formula for Solving First Order Stiff Ordinary Differential Equations
Dervato of -Pot Block Method Formula for Solvg Frst Order Stff Ordary Dfferetal Equatos Kharul Hamd Kharul Auar, Kharl Iskadar Othma, Zara Bb Ibrahm Abstract Dervato of pot block method formula wth costat
More informationCapacitated Plant Location Problem:
. L. Brcker, 2002 ept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/ 5/29/2002 page CPL/ 5/29/2002 page 2 Capactated Plat Locato Proble: where Mze F + C subect to = = =, =, S, =,... 0, =, ; =,
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationTHE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE
THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE PAPER I STATISTICAL THEORY The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More informationSUBCLASS OF HARMONIC UNIVALENT FUNCTIONS ASSOCIATED WITH SALAGEAN DERIVATIVE. Sayali S. Joshi
Faculty of Sceces ad Matheatcs, Uversty of Nš, Serba Avalable at: http://wwwpfacyu/float Float 3:3 (009), 303 309 DOI:098/FIL0903303J SUBCLASS OF ARMONIC UNIVALENT FUNCTIONS ASSOCIATED WIT SALAGEAN DERIVATIVE
More information5 Short Proofs of Simplified Stirling s Approximation
5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationTHE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5
THE ROYAL STATISTICAL SOCIETY 06 EAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 The Socety s provdg these solutos to assst cadtes preparg for the examatos 07. The solutos are teded as learg ads ad should
More informationA Family of Non-Self Maps Satisfying i -Contractive Condition and Having Unique Common Fixed Point in Metrically Convex Spaces *
Advaces Pure Matheatcs 0 80-84 htt://dxdoorg/0436/a04036 Publshed Ole July 0 (htt://wwwscrporg/oural/a) A Faly of No-Self Mas Satsfyg -Cotractve Codto ad Havg Uque Coo Fxed Pot Metrcally Covex Saces *
More informationVARIABLE-RATE VQ (AKA VQ WITH ENTROPY CODING)
VARIABLE-RATE VQ (AKA VQ WITH ENTROPY CODING) Varable-Rate VQ = Quatzato + Lossless Varable-Legth Bary Codg A rage of optos -- from smple to complex a. Uform scalar quatzato wth varable-legth codg, oe
More informationEstimation of Stress- Strength Reliability model using finite mixture of exponential distributions
Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur
More informationCS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 7 Lear regresso Mlos Hauskrecht los@cs.ptt.edu 59 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear cobato of put copoets f + + + K d d K k - paraeters eghts
More informationLecture 12 APPROXIMATION OF FIRST ORDER DERIVATIVES
FDM: Appromato of Frst Order Dervatves Lecture APPROXIMATION OF FIRST ORDER DERIVATIVES. INTRODUCTION Covectve term coservato equatos volve frst order dervatves. The smplest possble approach for dscretzato
More informationDistributed Fusion Filter for Asynchronous Multi-rate Multi-sensor Non-uniform Sampling Systems
Dstrbuted Fuso Flter for Asychroous Mult-rate Mult-sesor No-ufor Saplg Systes Jg Ma, Hogle L School of Matheatcs scece Helogjag Uversty Harb, Helogjag Provce, Cha ajg47@gal.co, lhogle8@63.co Shul Su Departet
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationDebabrata Dey and Atanu Lahiri
RESEARCH ARTICLE QUALITY COMPETITION AND MARKET SEGMENTATION IN THE SECURITY SOFTWARE MARKET Debabrata Dey ad Atau Lahr Mchael G. Foster School of Busess, Uersty of Washgto, Seattle, Seattle, WA 9895 U.S.A.
More informationUniform DFT Filter Banks 1/27
.. Ufor FT Flter Baks /27 Ufor FT Flter Baks We ll look at 5 versos of FT-based flter baks all but the last two have serous ltatos ad are t practcal. But they gve a ce trasto to the last two versos whch
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationA Model Reduction Technique for linear Model Predictive Control for Non-linear Large Scale Distributed Systems
A Model Reducto Techque for lear Model Predctve Cotrol for No-lear Large Scale Dstrbuted Systes Weguo Xe ad Costatos Theodoropoulos School of Checal Egeerg ad Aalytcal Scece Uversty of Machester, Machester
More informationCODING & MODULATION Prof. Ing. Anton Čižmár, PhD.
CODING & MODULATION Prof. Ig. Ato Čžmár, PhD. also from Dgtal Commucatos 4th Ed., J. G. Proaks, McGraw-Hll It. Ed. 00 CONTENT. PROBABILITY. STOCHASTIC PROCESSES Probablty ad Stochastc Processes The theory
More informationLecture 02: Bounding tail distributions of a random variable
CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome
More informationQueueing Networks. γ 3
Queueg Networks Systes odeled by queueg etworks ca roughly be grouped to four categores. Ope etworks Custoers arrve fro outsde the syste are served ad the depart. Exaple: acket swtched data etwork. γ µ
More information3D Reconstruction from Image Pairs. Reconstruction from Multiple Views. Computing Scene Point from Two Matching Image Points
D Recostructo fro Iage ars Recostructo fro ultple Ves Dael Deetho Fd terest pots atch terest pots Copute fudaetal atr F Copute caera atrces ad fro F For each atchg age pots ad copute pot scee Coputg Scee
More informationThe theoretical background of
he theoretcal backgroud of -echologes he theoretcal backgroud of FactSage he followg sldes gve a abrdged overvew of the ajor uderlyg prcples of the calculatoal odules of FactSage. -echologes he bbs Eergy
More informationEstimation of KL Divergence: Optimal Minimax Rate
stato of KL Dvergece: Optal Max Rate Yuheg Bu Shaofeg Zou Ygb Lag Veugopal V. Veeravall Uversty of Illos at Urbaa-Chapag Syracuse Uversty al: bu3@llos.edu, szou3@llos.edu, ylag06@syr.edu, vvv@llos.edu
More informationfor each of its columns. A quick calculation will verify that: thus m < dim(v). Then a basis of V with respect to which T has the form: A
Desty of dagoalzable square atrces Studet: Dael Cervoe; Metor: Saravaa Thyagaraa Uversty of Chcago VIGRE REU, Suer 7. For ths etre aer, we wll refer to V as a vector sace over ad L(V) as the set of lear
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationSymmetry of the Solution of Semidefinite Program by Using Primal-Dual Interior-Point Method
Syetry of the Soluto of Sedefte Progra by Usg Pral-Dual Iteror-Pot Method Yoshhro Kao Makoto Ohsak ad Naok Katoh Departet of Archtecture ad Archtectural Systes Kyoto Uversty Kyoto 66-85 Japa kao@s-jarchkyoto-uacjp
More informationOrder Nonlinear Vector Differential Equations
It. Joural of Math. Aalyss Vol. 3 9 o. 3 39-56 Coverget Power Seres Solutos of Hgher Order Nolear Vector Dfferetal Equatos I. E. Kougas Departet of Telecoucato Systes ad Networs Techologcal Educatoal Isttute
More informationIS 709/809: Computational Methods in IS Research. Simple Markovian Queueing Model
IS 79/89: Comutatoal Methods IS Research Smle Marova Queueg Model Nrmalya Roy Deartmet of Iformato Systems Uversty of Marylad Baltmore Couty www.umbc.edu Queueg Theory Software QtsPlus software The software
More informationLecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More informationECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
More informationSolving the fuzzy shortest path problem on networks by a new algorithm
Proceedgs of the 0th WSEAS Iteratoal Coferece o FUZZY SYSTEMS Solvg the fuzzy shortest path proble o etworks by a ew algorth SADOAH EBRAHIMNEJAD a, ad REZA TAVAKOI-MOGHADDAM b a Departet of Idustral Egeerg,
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More informationChapter 3 Sampling For Proportions and Percentages
Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys
More informationCOV. Violation of constant variance of ε i s but they are still independent. The error term (ε) is said to be heteroscedastic.
c Pogsa Porchawseskul, Faculty of Ecoomcs, Chulalogkor Uversty olato of costat varace of s but they are stll depedet. C,, he error term s sad to be heteroscedastc. c Pogsa Porchawseskul, Faculty of Ecoomcs,
More informationLogistic regression (continued)
STAT562 page 138 Logstc regresso (cotued) Suppose we ow cosder more complex models to descrbe the relatoshp betwee a categorcal respose varable (Y) that takes o two (2) possble outcomes ad a set of p explaatory
More informationA CHARACTERIZATION OF THE CLIFFORD TORUS
PROCEEDINGS OF THE AERICAN ATHEATICAL SOCIETY Volue 17, Nuber 3, arch 1999, Pages 819 88 S 000-9939(99)05088-1 A CHARACTERIZATION OF THE CLIFFORD TORUS QING-ING CHENG AND SUSUU ISHIKAWA (Coucated by Chrstopher
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationA New Measure of Probabilistic Entropy. and its Properties
Appled Mathematcal Sceces, Vol. 4, 200, o. 28, 387-394 A New Measure of Probablstc Etropy ad ts Propertes Rajeesh Kumar Departmet of Mathematcs Kurukshetra Uversty Kurukshetra, Ida rajeesh_kuk@redffmal.com
More information. The set of these sums. be a partition of [ ab, ]. Consider the sum f( x) f( x 1)
Chapter 7 Fuctos o Bouded Varato. Subject: Real Aalyss Level: M.Sc. Source: Syed Gul Shah (Charma, Departmet o Mathematcs, US Sargodha Collected & Composed by: Atq ur Rehma (atq@mathcty.org, http://www.mathcty.org
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More information2648 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 64, NO. 4, APRIL 2018
648 I TRANSACTIONS ON INFORMATION THORY, VOL 64, NO 4, APRIL 08 stato of KL Dvergece: Optal Max Rate Yuheg Bu, Studet Meber, I, Shaofeg Zou, Meber, I, Ygb Lag, Seor Meber, I, ad Veugopal V Veeravall, Fellow,
More informationLecture 7: Linear and quadratic classifiers
Lecture 7: Lear ad quadratc classfers Bayes classfers for ormally dstrbuted classes Case : Σ σ I Case : Σ Σ (Σ daoal Case : Σ Σ (Σ o-daoal Case 4: Σ σ I Case 5: Σ Σ j eeral case Lear ad quadratc classfers:
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationInterval extension of Bézier curve
WSEAS TRANSACTIONS o SIGNAL ROCESSING Jucheg L Iterval exteso of Bézer curve JUNCHENG LI Departet of Matheatcs Hua Uversty of Huates Scece ad Techology Dxg Road Loud cty Hua rovce 47 R CHINA E-al: ljucheg8@6co
More informationAnalytical Study of Fractal Dimension Types in the Context of SPC Technical Paper. Noa Ruschin Rimini, Irad Ben-Gal and Oded Maimon
Aalytcal Study of Fractal Deso Types the Cotext of SPC Techcal Paper oa Rusch R, Irad Be-Gal ad Oded Mao Departet of Idustral Egeerg, Tel-Avv Uversty, Tel-Avv, Israel Ths paper provdes a aalytcal study
More information