Discrete Memoryless Channels
|
|
- Jonah Lee
- 6 years ago
- Views:
Transcription
1 Dscrete Meorless Channels Source Channel Snk Aount of Inforaton avalable Source Entro Generall nos, dstorted and a be te varng ow uch nforaton s receved? ow uch s lost? Introduces error and lts the rate at whch data can be transferred Ma.Caact? C. In general, the channel ncludes odulator, transsson edu, deodulator and channel decoder. Source Channel Snk Alhabet of M nut sbols Iarent causes error n the detected sbol Sae alhabet of M sbols The dscrete channel s odeled b,,,3,..., M : source sbol robablt,.e. th robablt that the nut to the channel s the sbol. Therefore, we have,,,..., 3 M C.
2 th, where,,,3,..., M : robablt that the th sbol s sent and the sbol of the alhabet s receved at the outut of the channel. That s. Therefore, we have,,...,,.,,...,,... These robabltes deend on the araeters of the odulator, transsson eda, nose, and deodulator. C.3 Two-sbol source For a two-sbol source, the odel s as follows: Source : ; Snk : and and due to nose. aths reresent correct receton. aths reresent erroneous receton C.4
3 Bnar Setr Channel BSC error,, Assuton: the occurrence of an error durng a bt nterval does not affect the sste durng other bt ntervals..e. assue channel to be eorless Dscrete Meorless channel If - Bnar setr channel BSC.e. C.5 M-sbol source Model for a -sbol source s: Source : Snk : 3 C.6
4 M-sbol source Note that for a artcular value of, and for a artcular value of.. error The odel ght have nut sbols and n outut sbols, and usuall n C.7 Source entro Entro of the nut source entro s defned as log bts/sbol where source sbol robablt C.8
5 C.9 Snk entro Entro of the outut snk entro s defned as bts/sbol where snk sbol robablt usuall unknowns log C. Condtonal entro The effect of nose on the sbols s to cause uncertant n the receved sbol. The aount of uncertant s gven b the condtonal entro or error entro and equvocaton. log log
6 C. Condtonal entro - easures the uncertant of about a receved bt based on a transtted bt. C. Condtonal entro It s also ossble to defne another condtonal entro n ters of the condtonal robabltes log log
7 Condtonal entro - reresents how uncertan we are of, on the average, when we know. In other words, t reresents the aount of uncertant reanng about the channel nut after the channel outut has been observed. C.3 Eale Source : Snk : 3 3 [ log [ log [ 3 log log log log 3 3log 3 log 3 log 3 33 log 3] 3 ] 33] C.4
8 C.5 Eale log log log 3 3 where C.6 Eale When the channel s noseless, the sbols are receved wthout error. and - NO uncertant about the outut when the nut s known. - NO nforaton s lost. f... log
9 Eale Source : Snk : C.7 Eale When the channel s nos that the outut s statstcall ndeendent of the nut, That s are equal for all and. and then / C.8
10 Eale log log log log { log } log C.9 Eale If BSC, bt/sbol. > bt uncertant. Therefore, No nforaton s conveed. C.
11 3 4 4? Eale 3 Gven : / 3; / 3 log log.98 bts,, log log C. Eale 3 Slarl,? Thus, f or, there s no uncertant about, but f?, we have uncertant about. C.
12 Rate of Inforaton transsson A dscrete sbol eorless channel s accetng sbols fro an M-sbol source at a rate of sbols /second. r s log bts/sbol The average rate at whch nforaton s gong nto the channel s D r bts/sec. n s owever, soe nforaton s lost due to nose n ractce. C.3 Eale 4 Suose two sbols {,} are transtted at sbols/sec wth /, /, D n bts/sec. Let the channel be setrc wth robablt of errorless transsson.95. What s the rate of transsson of nforaton? a. 95 bts/sec? b. >95bts/sec? c. <95bts/sec? C.4
13 Eale 5 For eale,.5, then /, / rresectve of what s actuall beng transtted ver nose condton. Now the sbols receved are correct due to chance alone. In fact, we can dsconnect the ath and guessng the receved sbol to be ether a '' or ''. C.5 Mutual Inforaton reresents the aount of nforaton n that one cannot rel on. Thus the aount of nforaton at the snk ust be reduced b the aount of uncertant that gve the true aount of nforaton receved at the snk. We defne Mutual Inforaton as I ; Alternatvel, I ; can be defned b notng that the nforaton etted b the source,, s reduced b the loss nforaton caused b nose n the channel. I ; C.6
14 Eale 6 Recall fro eale and, Noseless channel I ; or I ; Ver nos channel or I ; I ; C.7 Eale 7 A bnar setrc channel s shown below. Fnd the rate of nforaton transsson over ths channel when.9,.8,.6; assue that the sbol rate bt rate n ths case s sbols/sec. log log log / / bt/sbol C.8
15 Eale 7 Average rate whch nforaton transsson over the channel s bts/sec. r s The rate of nforaton transsson over the channel s gven b the utual nforaton I ;. I ; or I ; r s C.9 Eale 7 Usng I ; / / [ log log ] [ log log ] [ log log ] [ log log ] C.3
16 Eale I ; I ; 53 bts/sec 78 bts/sec 9 bts/sec r s I ; r s decreases radl as the robablt of error /. Note that data rate bts/sec s not the sae as the nforaton rate I ;. r s C.3 Suar Source Entro: log bts/sbol Snk Entro: log bts/sbol Error Entro or Condtonal Entro: where log C.3
17 where Source sbol robabltes Snk sbol robabltes and log C.33 Channel Caact: Dscrete Meorless Channel DMC The concet of utual nforaton can be suarzed as follows: I ; In ractce,, are fed for a gven,,..., dscrete channel unless the nose characterstcs are tevarng. The utual nforaton of a channel deends not onl on the channel but also on the wa n whch the channel s used. The nut robablt dstrbuton { } s ndeendent of the channel. We can then aze the utual nforaton of the channel wth resect to. { } C.34
18 Channel caact The caact of a nos dscrete, eorless channel s defned as the au ossble rate of nforaton transsson over the channel. The au rate of transsson occurs when the source s "atched" to the channel. C Ma I ; { } { } Ma bts /sbol Note that the channel caact s a functon onl of the transton robablt, whch defne the channel. The calculaton of C nvolves azaton of the utual nforaton over varables subect to two constrants: { } { } for all C.35 In general, C s a functon of the source and the nose statstcs. The channel redundanc and channel effcent are defned as Channel Redundanc C I ; bts/sbol C I ; % C I ; Channel Effcenc % C Note that C s eressed n bts/sbol. If the sbol rate s τ s sbol/s, C can be eressed n bts/s b ultlng τ s. C.36
19 Eale 8 Bnar Setrc Channel robablt of a sbol beng receved correctl s q - sae for each sbol. robablt of a sbol beng receved ncorrectl s - sae for each sbol. q q q C.37 Let log log q log q log log q log q log log log log log q log q C.38
20 C.39 log log log where log log C.4 Thus The results show that au caact s when ndeendent of log log ; I { } { } and / when Ma / - - / Ma / log log then and C
21 BSC s a usual odel whch aroates the behavour of an ractcal bnar channels. C.4 Other channel odels Bnar Erasure Channel BEC q e erasure q C.4
22 Other channel odels Setrc Erasure Channel SEC, a cobnaton of BSC and BEC q r r q e erasure C.43 Contnuous Channel Modulator Modulator Deodulator Shannon's Theore Contnuous Channel Gven a source of M equall lkel essages, wth M >>, whch s generatng nforaton at a rate of R bts er second. If R C, the channel caact, there ests a channel codng technque such that the councaton sste wll transt nforaton wth an arbtrar sall robablt of error. C.44
23 Shannon-artle Theore For a whte, bandlted Gaussan channel, the channel caact s C B log S N bt/sec Note: S/N not the 'db' value where S - average sgnal ower at outut of contnuos channel N - average nose ower at outut of contnuos channel C.45 η η N B ηb Watts Shannon-artle Theore : two-sded ower sectral denst sd of the nose n watt/z sd B - channel bandwdth. η / df B B frequenc Nose altude C.46
24 Soe secal cases C B log bt/sec gves uer lt for relable data transsson over Gaussan channel For eale: Bandwdth of a telehone lne 3kz, S/N 3dB C 3log 3kb / sec S N Echange of S/N for Bandwdth B As N S / N C C, therefore no nose, C.47 As B, does C? No. As B ncreases so does the nose. In the resence of nose, C reaches a fnte uer lt as bandwdth ncreases, for a fed sgnal ower η Nose ower N B ηb S C B log N S η S B log η S ηb S log η S ηb ηb S C.48
25 l / e S S l C log e.44 B η η C.49 Eale 9 Consder C bt/s Bandwdth 3z, S/N? 3 log S / N S / N Bandwdth z, S/N? log S / N S / N For the sae C, bandwdth can be reduced fro kz to 3kz f we ncrease S/N 9 tes. C.5
26 Eale If If S / N 7, B 4kz C kbt / s S / N 5, B 3kz C kbt / s 3 Wth a 3 kz bandwdth, the nose ower ηb wll be as 4 large as wth a 4kz. Snce, N3 η3 3 and S / N 3 5 N η 4 4 S / N 7 4 S S The sgnal ower s ncreased b.6 tes to gve the sae caact when the bandwdth s reduced fro 4 to 3 kz 5% 4 C.5 Eale Can we transt an analogue sgnal of bandwdth over a channel havng a bandwdth less than? f f z Suose salng rate s 3 Nqust rate and nuber of quantzaton level s M Nuber of levels er sbol Data rate R 6 f log M bts/s Sa M 64, R 36 f bt/s and bandwdth of channel B, we can then work out S/N, rovdng R C. C.5
27 Eale Sa M 64, R 36 f bt/s Let C R but the channel bandwdth s f / 36 f 7 log S N f log 7 S N S N 7dB C.53
Source-Channel-Sink Some questions
Source-Channel-Snk Soe questons Source Channel Snk Aount of Inforaton avalable Source Entro Generall nos and a be te varng Introduces error and lts the rate at whch data can be transferred ow uch nforaton
More informationChapter 7 Channel Capacity and Coding
Chapter 7 Channel Capacty and Codng Contents 7. Channel models and channel capacty 7.. Channel models Bnary symmetrc channel Dscrete memoryless channels Dscrete-nput, contnuous-output channel Waveform
More informationChapter 7 Channel Capacity and Coding
Wreless Informaton Transmsson System Lab. Chapter 7 Channel Capacty and Codng Insttute of Communcatons Engneerng atonal Sun Yat-sen Unversty Contents 7. Channel models and channel capacty 7.. Channel models
More informationDeparture Process from a M/M/m/ Queue
Dearture rocess fro a M/M// Queue Q - (-) Q Q3 Q4 (-) Knowledge of the nature of the dearture rocess fro a queue would be useful as we can then use t to analyze sle cases of queueng networs as shown. The
More informationLecture 3: Shannon s Theorem
CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationMultipoint Analysis for Sibling Pairs. Biostatistics 666 Lecture 18
Multpont Analyss for Sblng ars Bostatstcs 666 Lecture 8 revously Lnkage analyss wth pars of ndvduals Non-paraetrc BS Methods Maxu Lkelhood BD Based Method ossble Trangle Constrant AS Methods Covered So
More informationExcess Error, Approximation Error, and Estimation Error
E0 370 Statstcal Learnng Theory Lecture 10 Sep 15, 011 Excess Error, Approxaton Error, and Estaton Error Lecturer: Shvan Agarwal Scrbe: Shvan Agarwal 1 Introducton So far, we have consdered the fnte saple
More informationÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE
ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE School of Computer and Communcaton Scences Handout 0 Prncples of Dgtal Communcatons Solutons to Problem Set 4 Mar. 6, 08 Soluton. If H = 0, we have Y = Z Z = Y
More informationCOS 511: Theoretical Machine Learning
COS 5: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #0 Scrbe: José Sões Ferrera March 06, 203 In the last lecture the concept of Radeacher coplexty was ntroduced, wth the goal of showng that
More informationA Mathematical Theory of Communication. Claude Shannon s paper presented by Kate Jenkins 2/19/00
A Mathematcal Theory of Communcaton Claude hannon s aer resented by Kate Jenkns 2/19/00 Publshed n two arts, July 1948 and October 1948 n the Bell ystem Techncal Journal Foundng aer of Informaton Theory
More informationThe Decibel and its Usage
The Decbel and ts Usage Consder a two-stage amlfer system, as shown n Fg.. Each amlfer rodes an ncrease of the sgnal ower. Ths effect s referred to as the ower gan,, of the amlfer. Ths means that the sgnal
More informationError Probability for M Signals
Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal
More informationAn application of generalized Tsalli s-havrda-charvat entropy in coding theory through a generalization of Kraft inequality
Internatonal Journal of Statstcs and Aled Mathematcs 206; (4): 0-05 ISS: 2456-452 Maths 206; (4): 0-05 206 Stats & Maths wwwmathsjournalcom Receved: 0-09-206 Acceted: 02-0-206 Maharsh Markendeshwar Unversty,
More informationFermi-Dirac statistics
UCC/Physcs/MK/EM/October 8, 205 Fer-Drac statstcs Fer-Drac dstrbuton Matter partcles that are eleentary ostly have a type of angular oentu called spn. hese partcles are known to have a agnetc oent whch
More informationSystem in Weibull Distribution
Internatonal Matheatcal Foru 4 9 no. 9 94-95 Relablty Equvalence Factors of a Seres-Parallel Syste n Webull Dstrbuton M. A. El-Dacese Matheatcs Departent Faculty of Scence Tanta Unversty Tanta Egypt eldacese@yahoo.co
More informationPHYS 342L NOTES ON ANALYZING DATA. Spring Semester 2002
PHYS 34L OTES O AALYZIG DATA Sprng Seester 00 Departent of Phscs Purdue Unverst A ajor aspect of eperental phscs (and scence n general) s easureent of soe quanttes and analss of eperentall obtaned data.
More information1 Review From Last Time
COS 5: Foundatons of Machne Learnng Rob Schapre Lecture #8 Scrbe: Monrul I Sharf Aprl 0, 2003 Revew Fro Last Te Last te, we were talkng about how to odel dstrbutons, and we had ths setup: Gven - exaples
More informationMulti-dimensional Central Limit Theorem
Mult-dmensonal Central Lmt heorem Outlne ( ( ( t as ( + ( + + ( ( ( Consder a sequence of ndependent random proceses t, t, dentcal to some ( t. Assume t = 0. Defne the sum process t t t t = ( t = (; t
More information1 Definition of Rademacher Complexity
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #9 Scrbe: Josh Chen March 5, 2013 We ve spent the past few classes provng bounds on the generalzaton error of PAClearnng algorths for the
More informationSOME NOISELESS CODING THEOREM CONNECTED WITH HAVRDA AND CHARVAT AND TSALLIS S ENTROPY. 1. Introduction
Kragujevac Journal of Mathematcs Volume 35 Number (20, Pages 7 SOME NOISELESS COING THEOREM CONNECTE WITH HAVRA AN CHARVAT AN TSALLIS S ENTROPY SATISH KUMAR AN RAJESH KUMAR 2 Abstract A new measure L,
More informationCHAPTER-5 INFORMATION MEASURE OF FUZZY MATRIX AND FUZZY BINARY RELATION
CAPTER- INFORMATION MEASURE OF FUZZY MATRI AN FUZZY BINARY RELATION Introducton The basc concept of the fuzz matr theor s ver smple and can be appled to socal and natural stuatons A branch of fuzz matr
More informationEGR 544 Communication Theory
EGR 544 Communcaton Theory. Informaton Sources Z. Alyazcoglu Electrcal and Computer Engneerng Department Cal Poly Pomona Introducton Informaton Source x n Informaton sources Analog sources Dscrete sources
More informationLossy Compression. Compromise accuracy of reconstruction for increased compression.
Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost
More informationSolving Fuzzy Linear Programming Problem With Fuzzy Relational Equation Constraint
Intern. J. Fuzz Maeatcal Archve Vol., 0, -0 ISSN: 0 (P, 0 0 (onlne Publshed on 0 Septeber 0 www.researchasc.org Internatonal Journal of Solvng Fuzz Lnear Prograng Proble W Fuzz Relatonal Equaton Constrant
More information+, where 0 x N - n. k k
CO 745, Mdterm Len Cabrera. A multle choce eam has questons, each of whch has ossble answers. A student nows the correct answer to n of these questons. For the remanng - n questons, he checs the answers
More informationManaging Capacity Through Reward Programs. on-line companion page. Byung-Do Kim Seoul National University College of Business Administration
Managng Caacty Through eward Programs on-lne comanon age Byung-Do Km Seoul Natonal Unversty College of Busness Admnstraton Mengze Sh Unversty of Toronto otman School of Management Toronto ON M5S E6 Canada
More informationWhat is LP? LP is an optimization technique that allocates limited resources among competing activities in the best possible manner.
(C) 998 Gerald B Sheblé, all rghts reserved Lnear Prograng Introducton Contents I. What s LP? II. LP Theor III. The Splex Method IV. Refneents to the Splex Method What s LP? LP s an optzaton technque that
More informationPulse Coded Modulation
Pulse Coded Modulaton PCM (Pulse Coded Modulaton) s a voce codng technque defned by the ITU-T G.711 standard and t s used n dgtal telephony to encode the voce sgnal. The frst step n the analog to dgtal
More informationModule 2. Random Processes. Version 2 ECE IIT, Kharagpur
Module Random Processes Lesson 6 Functons of Random Varables After readng ths lesson, ou wll learn about cdf of functon of a random varable. Formula for determnng the pdf of a random varable. Let, X be
More informationA Performance Model of Space-Division ATM Switches with Input and Output Queueing *
A Perforance Model of Sace-Dvson ATM Swtches wth Inut and Outut Queueng * Guogen Zhang Couter Scence Deartent Unversty of Calforna at Los Angeles Los Angeles, CA 94, USA Wlla G. Bulgren Engneerng Manageent
More informationEE513 Audio Signals and Systems. Statistical Pattern Classification Kevin D. Donohue Electrical and Computer Engineering University of Kentucky
EE53 Audo Sgnals and Systes Statstcal Pattern Classfcaton Kevn D. Donohue Electrcal and Couter Engneerng Unversty of Kentucy Interretaton of Audtory Scenes Huan erceton and cognton greatly eceeds any couter-based
More information6 Supplementary Materials
6 Supplementar Materals 61 Proof of Theorem 31 Proof Let m Xt z 1:T : l m Xt X,z 1:t Wethenhave mxt z1:t ˆm HX Xt z 1:T mxt z1:t m HX Xt z 1:T + mxt z 1:T HX We consder each of the two terms n equaton
More informationXiangwen Li. March 8th and March 13th, 2001
CS49I Approxaton Algorths The Vertex-Cover Proble Lecture Notes Xangwen L March 8th and March 3th, 00 Absolute Approxaton Gven an optzaton proble P, an algorth A s an approxaton algorth for P f, for an
More informationConfidence intervals for weighted polynomial calibrations
Confdence ntervals for weghted olynomal calbratons Sergey Maltsev, Amersand Ltd., Moscow, Russa; ur Kalambet, Amersand Internatonal, Inc., Beachwood, OH e-mal: kalambet@amersand-ntl.com htt://www.chromandsec.com
More informationHidden Markov Model Cheat Sheet
Hdden Markov Model Cheat Sheet (GIT ID: dc2f391536d67ed5847290d5250d4baae103487e) Ths document s a cheat sheet on Hdden Markov Models (HMMs). It resembles lecture notes, excet that t cuts to the chase
More informationTHE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY. William A. Pearlman. References: S. Arimoto - IEEE Trans. Inform. Thy., Jan.
THE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY Wllam A. Pearlman 2002 References: S. Armoto - IEEE Trans. Inform. Thy., Jan. 1972 R. Blahut - IEEE Trans. Inform. Thy., July 1972 Recall
More informationPhysics 3A: Linear Momentum. Physics 3A: Linear Momentum. Physics 3A: Linear Momentum. Physics 3A: Linear Momentum
Recall that there was ore to oton than just spee A ore coplete escrpton of oton s the concept of lnear oentu: p v (8.) Beng a prouct of a scalar () an a vector (v), oentu s a vector: p v p y v y p z v
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede. . For P such independent random variables (aka degrees of freedom): 1 =
Fall Analss of Epermental Measurements B. Esensten/rev. S. Errede More on : The dstrbuton s the.d.f. for a (normalzed sum of squares of ndependent random varables, each one of whch s dstrbuted as N (,.
More information( ) 2 ( ) ( ) Problem Set 4 Suggested Solutions. Problem 1
Problem Set 4 Suggested Solutons Problem (A) The market demand functon s the soluton to the followng utlty-maxmzaton roblem (UMP): The Lagrangean: ( x, x, x ) = + max U x, x, x x x x st.. x + x + x y x,
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationOn the Construction of Polar Codes
On the Constructon of Polar Codes Ratn Pedarsan School of Coputer and Councaton Systes, Lausanne, Swtzerland. ratn.pedarsan@epfl.ch S. Haed Hassan School of Coputer and Councaton Systes, Lausanne, Swtzerland.
More informationMathematical Models for Information Sources A Logarithmic i Measure of Information
Introducton to Informaton Theory Wreless Informaton Transmsson System Lab. Insttute of Communcatons Engneerng g Natonal Sun Yat-sen Unversty Table of Contents Mathematcal Models for Informaton Sources
More informationMatching Dyadic Distributions to Channels
Matchng Dyadc Dstrbutons to Channels G. Böcherer and R. Mathar Insttute for Theoretcal Informaton Technology RWTH Aachen Unversty, 5256 Aachen, Germany Emal: {boecherer,mathar}@t.rwth-aachen.de Abstract
More informationOn the Construction of Polar Codes
On the Constructon of Polar Codes Ratn Pedarsan School of Coputer and Councaton Systes, Lausanne, Swtzerland. ratn.pedarsan@epfl.ch S. Haed Hassan School of Coputer and Councaton Systes, Lausanne, Swtzerland.
More informationy new = M x old Feature Selection: Linear Transformations Constraint Optimization (insertion)
Feature Selecton: Lnear ransforatons new = M x old Constrant Optzaton (nserton) 3 Proble: Gven an objectve functon f(x) to be optzed and let constrants be gven b h k (x)=c k, ovng constants to the left,
More informationI - Information theory basics
I - Information theor basics Introduction To communicate, that is, to carr information between two oints, we can emlo analog or digital transmission techniques. In digital communications the message is
More informationWhat Independencies does a Bayes Net Model? Bayesian Networks: Independencies and Inference. Quick proof that independence is symmetric
Bayesan Networks: Indeendences and Inference Scott Daves and ndrew Moore Note to other teachers and users of these sldes. ndrew and Scott would be delghted f you found ths source materal useful n gvng
More informationThe Impact of the Earth s Movement through the Space on Measuring the Velocity of Light
Journal of Appled Matheatcs and Physcs, 6, 4, 68-78 Publshed Onlne June 6 n ScRes http://wwwscrporg/journal/jap http://dxdoorg/436/jap646 The Ipact of the Earth s Moeent through the Space on Measurng the
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationXII.3 The EM (Expectation-Maximization) Algorithm
XII.3 The EM (Expectaton-Maxzaton) Algorth Toshnor Munaata 3/7/06 The EM algorth s a technque to deal wth varous types of ncoplete data or hdden varables. It can be appled to a wde range of learnng probles
More informationRevision: December 13, E Main Suite D Pullman, WA (509) Voice and Fax
.9.1: AC power analyss Reson: Deceber 13, 010 15 E Man Sute D Pullan, WA 99163 (509 334 6306 Voce and Fax Oerew n chapter.9.0, we ntroduced soe basc quanttes relate to delery of power usng snusodal sgnals.
More informatione i is a random error
Chapter - The Smple Lnear Regresson Model The lnear regresson equaton s: where + β + β e for,..., and are observable varables e s a random error How can an estmaton rule be constructed for the unknown
More informationNaïve Bayes Classifier
9/8/07 MIST.6060 Busness Intellgence and Data Mnng Naïve Bayes Classfer Termnology Predctors: the attrbutes (varables) whose values are used for redcton and classfcaton. Predctors are also called nut varables,
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationGenerative classification models
CS 675 Intro to Machne Learnng Lecture Generatve classfcaton models Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Data: D { d, d,.., dn} d, Classfcaton represents a dscrete class value Goal: learn
More informationLECTURE :FACTOR ANALYSIS
LCUR :FACOR ANALYSIS Rta Osadchy Based on Lecture Notes by A. Ng Motvaton Dstrbuton coes fro MoG Have suffcent aount of data: >>n denson Use M to ft Mture of Gaussans nu. of tranng ponts If
More informationOn the number of regions in an m-dimensional space cut by n hyperplanes
6 On the nuber of regons n an -densonal space cut by n hyperplanes Chungwu Ho and Seth Zeran Abstract In ths note we provde a unfor approach for the nuber of bounded regons cut by n hyperplanes n general
More informationAssignment 2. Tyler Shendruk February 19, 2010
Assgnment yler Shendruk February 9, 00 Kadar Ch. Problem 8 We have an N N symmetrc matrx, M. he symmetry means M M and we ll say the elements of the matrx are m j. he elements are pulled from a probablty
More informationLecture 20: Hypothesis testing
Lecture : Hpothess testng Much of statstcs nvolves hpothess testng compare a new nterestng hpothess, H (the Alternatve hpothess to the borng, old, well-known case, H (the Null Hpothess or, decde whether
More informationSTATISTICAL MECHANICS
STATISTICAL MECHANICS Thermal Energy Recall that KE can always be separated nto 2 terms: KE system = 1 2 M 2 total v CM KE nternal Rgd-body rotaton and elastc / sound waves Use smplfyng assumptons KE of
More informationPreference and Demand Examples
Dvson of the Huantes and Socal Scences Preference and Deand Exaples KC Border October, 2002 Revsed Noveber 206 These notes show how to use the Lagrange Karush Kuhn Tucker ultpler theores to solve the proble
More informationExam. Econometrics - Exam 1
Econometrcs - Exam 1 Exam Problem 1: (15 ponts) Suppose that the classcal regresson model apples but that the true value of the constant s zero. In order to answer the followng questons assume just one
More informationThe Dirac Equation. Elementary Particle Physics Strong Interaction Fenomenology. Diego Bettoni Academic year
The Drac Equaton Eleentary artcle hyscs Strong Interacton Fenoenology Dego Betton Acadec year - D Betton Fenoenologa Interazon Fort elatvstc equaton to descrbe the electron (ncludng ts sn) Conservaton
More information1.3 Hence, calculate a formula for the force required to break the bond (i.e. the maximum value of F)
EN40: Dynacs and Vbratons Hoework 4: Work, Energy and Lnear Moentu Due Frday March 6 th School of Engneerng Brown Unversty 1. The Rydberg potental s a sple odel of atoc nteractons. It specfes the potental
More informationPriority Queuing with Finite Buffer Size and Randomized Push-out Mechanism
ICN 00 Prorty Queung wth Fnte Buffer Sze and Randomzed Push-out Mechansm Vladmr Zaborovsy, Oleg Zayats, Vladmr Muluha Polytechncal Unversty, Sant-Petersburg, Russa Arl 4, 00 Content I. Introducton II.
More informationAn Application of Fuzzy Hypotheses Testing in Radar Detection
Proceedngs of the th WSES Internatonal Conference on FUZZY SYSEMS n pplcaton of Fuy Hypotheses estng n Radar Detecton.K.ELSHERIF, F.M.BBDY, G.M.BDELHMID Department of Mathematcs Mltary echncal Collage
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationOn Pfaff s solution of the Pfaff problem
Zur Pfaff scen Lösung des Pfaff scen Probles Mat. Ann. 7 (880) 53-530. On Pfaff s soluton of te Pfaff proble By A. MAYER n Lepzg Translated by D. H. Delpenc Te way tat Pfaff adopted for te ntegraton of
More informationON THE NUMBER OF PRIMITIVE PYTHAGOREAN QUINTUPLES
Journal of Algebra, Nuber Theory: Advances and Applcatons Volue 3, Nuber, 05, Pages 3-8 ON THE NUMBER OF PRIMITIVE PYTHAGOREAN QUINTUPLES Feldstrasse 45 CH-8004, Zürch Swtzerland e-al: whurlann@bluewn.ch
More informationTwo Conjectures About Recency Rank Encoding
Internatonal Journal of Matheatcs and Coputer Scence, 0(205, no. 2, 75 84 M CS Two Conjectures About Recency Rank Encodng Chrs Buhse, Peter Johnson, Wlla Lnz 2, Matthew Spson 3 Departent of Matheatcs and
More informationStrong Markov property: Same assertion holds for stopping times τ.
Brownan moton Let X ={X t : t R + } be a real-valued stochastc process: a famlty of real random varables all defned on the same probablty space. Defne F t = nformaton avalable by observng the process up
More informationWhat would be a reasonable choice of the quantization step Δ?
CE 108 HOMEWORK 4 EXERCISE 1. Suppose you are samplng the output of a sensor at 10 KHz and quantze t wth a unform quantzer at 10 ts per sample. Assume that the margnal pdf of the sgnal s Gaussan wth mean
More informationQuantum Particle Motion in Physical Space
Adv. Studes Theor. Phys., Vol. 8, 014, no. 1, 7-34 HIKARI Ltd, www.-hkar.co http://dx.do.org/10.1988/astp.014.311136 Quantu Partcle Moton n Physcal Space A. Yu. Saarn Dept. of Physcs, Saara State Techncal
More informationPhysicsAndMathsTutor.com
PhscsAndMathsTutor.com phscsandmathstutor.com June 005 5. The random varable X has probablt functon k, = 1,, 3, P( X = ) = k ( + 1), = 4, 5, where k s a constant. (a) Fnd the value of k. (b) Fnd the eact
More informationDigital Modems. Lecture 2
Dgtal Modems Lecture Revew We have shown that both Bayes and eyman/pearson crtera are based on the Lkelhood Rato Test (LRT) Λ ( r ) < > η Λ r s called observaton transformaton or suffcent statstc The crtera
More informationModelli Clamfim Equazione del Calore Lezione ottobre 2014
CLAMFIM Bologna Modell 1 @ Clamfm Equazone del Calore Lezone 17 15 ottobre 2014 professor Danele Rtell danele.rtell@unbo.t 1/24? Convoluton The convoluton of two functons g(t) and f(t) s the functon (g
More informationOne-sided finite-difference approximations suitable for use with Richardson extrapolation
Journal of Computatonal Physcs 219 (2006) 13 20 Short note One-sded fnte-dfference approxmatons sutable for use wth Rchardson extrapolaton Kumar Rahul, S.N. Bhattacharyya * Department of Mechancal Engneerng,
More informationAnswers Problem Set 2 Chem 314A Williamsen Spring 2000
Answers Problem Set Chem 314A Wllamsen Sprng 000 1) Gve me the followng crtcal values from the statstcal tables. a) z-statstc,-sded test, 99.7% confdence lmt ±3 b) t-statstc (Case I), 1-sded test, 95%
More informationLinear system of the Schrödinger equation Notes on Quantum Mechanics
Lnear sstem of the Schrödnger equaton Notes on Quantum Mechancs htt://quantum.bu.edu/notes/quantummechancs/lnearsstems.df Last udated Wednesda, October 9, 003 :0:08 Corght 003 Dan Dll (dan@bu.edu) Deartment
More informationOn Syndrome Decoding of Punctured Reed-Solomon and Gabidulin Codes 1
Ffteenth Internatonal Workshop on Algebrac and Cobnatoral Codng Theory June 18-24, 2016, Albena, Bulgara pp. 35 40 On Syndroe Decodng of Punctured Reed-Soloon and Gabduln Codes 1 Hannes Bartz hannes.bartz@tu.de
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationBAYESIAN CURVE FITTING USING PIECEWISE POLYNOMIALS. Dariusz Biskup
BAYESIAN CURVE FITTING USING PIECEWISE POLYNOMIALS Darusz Bskup 1. Introducton The paper presents a nonparaetrc procedure for estaton of an unknown functon f n the regresson odel y = f x + ε = N. (1) (
More information6. Hamilton s Equations
6. Hamlton s Equatons Mchael Fowler A Dynamcal System s Path n Confguraton Sace and n State Sace The story so far: For a mechancal system wth n degrees of freedom, the satal confguraton at some nstant
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationClassification Bayesian Classifiers
lassfcaton Bayesan lassfers Jeff Howbert Introducton to Machne Learnng Wnter 2014 1 Bayesan classfcaton A robablstc framework for solvng classfcaton roblems. Used where class assgnment s not determnstc,.e.
More informationSpectral method for fractional quadratic Riccati differential equation
Journal of Aled Matheatcs & Bonforatcs vol2 no3 212 85-97 ISSN: 1792-662 (rnt) 1792-6939 (onlne) Scenress Ltd 212 Sectral ethod for fractonal quadratc Rccat dfferental equaton Rostay 1 K Kar 2 L Gharacheh
More informationApplied Mathematics Letters
Appled Matheatcs Letters 2 (2) 46 5 Contents lsts avalable at ScenceDrect Appled Matheatcs Letters journal hoepage: wwwelseverco/locate/al Calculaton of coeffcents of a cardnal B-splne Gradr V Mlovanovć
More informationProfessor Chris Murray. Midterm Exam
Econ 7 Econometrcs Sprng 4 Professor Chrs Murray McElhnney D cjmurray@uh.edu Mdterm Exam Wrte your answers on one sde of the blank whte paper that I have gven you.. Do not wrte your answers on ths exam.
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 3.
More informationUniversal communication part II: channels with memory
Unversal councaton part II: channels wth eory Yuval Lontz, Mer Feder Tel Avv Unversty, Dept. of EE-Systes Eal: {yuvall,er@eng.tau.ac.l arxv:202.047v2 [cs.it] 20 Mar 203 Abstract Consder councaton over
More informationLogistic regression with one predictor. STK4900/ Lecture 7. Program
Logstc regresson wth one redctor STK49/99 - Lecture 7 Program. Logstc regresson wth one redctor 2. Maxmum lkelhood estmaton 3. Logstc regresson wth several redctors 4. Devance and lkelhood rato tests 5.
More informationOur focus will be on linear systems. A system is linear if it obeys the principle of superposition and homogenity, i.e.
SSTEM MODELLIN In order to solve a control syste proble, the descrptons of the syste and ts coponents ust be put nto a for sutable for analyss and evaluaton. The followng ethods can be used to odel physcal
More informationLecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem.
Lecture 14 (03/27/18). Channels. Decodng. Prevew of the Capacty Theorem. A. Barg The concept of a communcaton channel n nformaton theory s an abstracton for transmttng dgtal (and analog) nformaton from
More informationChapter 1. Probability
Chapter. Probablty Mcroscopc propertes of matter: quantum mechancs, atomc and molecular propertes Macroscopc propertes of matter: thermodynamcs, E, H, C V, C p, S, A, G How do we relate these two propertes?
More informationStatistical Inference. 2.3 Summary Statistics Measures of Center and Spread. parameters ( population characteristics )
Ismor Fscher, 8//008 Stat 54 / -8.3 Summary Statstcs Measures of Center and Spread Dstrbuton of dscrete contnuous POPULATION Random Varable, numercal True center =??? True spread =???? parameters ( populaton
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More information