ECE594I Notes set 4: More Math: Expectations of 1-2 R.V.'s

Transcription

1 C594I Notes set 4: More Math: pectations o - R.V.'s Mark Rodwell Universit o Caliornia, Santa Barbara rodwell@ece.ucsb.edu , a

2 Reerences and Citations: Sources / Citations : Kittel and Kroemer : Thermal Phsics Van der Ziel : Noise in Solid - State Devices Papoulis : Probabil it and Random Variables hard, comprehens ive Peton Z. Peebles : Probabili t, Random Variables, Random Signal Principle s introduct or Wozencrat & Jacobs : Principle s o Communicat ions ngineeri ng. Motchenbak er : Low Noise lectroni c Design Inormation or lecture notes : Thomas Cover, Stanord, circa 98 Probabilit lecture notes : Martin Hellman, Stanord, circa 98 National Semiconductor Linear Applicatio ns Notes : Noise in circuits. Suggested reerences or stud. td Van der Ziel, Wozencrat & Jacobs, Peebles, Kittel and Kroemer Papers b Fukui device noise, Smith & Personik optical receiver design National Semi. App. Notes! Cover and Williams : lements o Inormation Theor

3 random variables and pectations

4 Recall: Distribution Function o Random Variable During an eperiment, a random variable takes on a particular value. The probabilit that lies between and is P{ < < } d is probabilit distributi on unction.

5 Mean values and epectations pectatio n o a unction g o random variable [ g ] g d Mean Vl Value o [ ] d pected value o [ ] d

6 Variance The variance σ rom its average σ o is value its root - mean - square [ ] deviation d The standard d square deviation i root o σ o variance is simpl

7 Returning to Gaussian Distribution The notation describing Gaussian ep πσ σ should now be clear. distributi on : ~σ

8 Variance vs pectation o Square σ σ The variance is epectatio n o square minus square o epectatio n.

9 ample o pectation: Mean Kinetic nerg Our particle with a rmal velocit distributi on : v V v ep where v σ π σ v σ v [ v ] v v dv 0 V [ v ] v v this So, V computes σ v dv kt / m variance o a Gaussian [ Kinetic energ] [ mv / ] kt / [ ] kt / kt / m skip proo

10 ample o pectation: Shot Noise Bernoulli Trial p q p probabilit 0 probabilit p p q p p p q p 0 ] [ 0 ] [ pq p p p p q p ] [ ] [ 0 ] [ σ

11 ample o pectation: Quantization "Noise" I we stipulate over some large that V A range, is itsel sa an Δ, ε is uniorml distribute d over [ Δ The quantizati on error is ± n Δ / d ] σ ε [ ε ] 0, [ ε Note we that ε is an R.V. must be cautious in Δ / onl i N RV R.V. also an ε Δ treating VA Δ is n /, Δ distribute d R.V., also an / ] uni orml quantizati on error with R.V., i.e. quantizati on error as noise.

12 Pairs o Random Variables To understand random processes, we must irst understand pairs o random variables. In an eperiment, a pair o random variables and takes on speciic particular values and. Their joint behavior is described b joint distributi on, P { A < < B and C < DB < D }, dd C A

13 Pairs o Random Variables Marginal distributi ons B P { A < < B }, dd A B and similarl or : A d D must also P { C < < D }, dd C D C d be deined

14 Statistical Independence In case where,, variables are said to be statistica ll independen t. This is not generall epected.

15 Conditional Densities Arises in revising distributi on o an R.V. ater making some observation. Consider t he conidtiona l probabilit that R.V. is less P[ than a speciic value given P[ B] B] F P [ B ] occurence o event B B This is cumulative distributi on o unction o given B. B. d Distributi on unction : / B B F B B d

16 Conditional Densities II Given a pair o random variables,, what is distibution o given that has some particular value? / /,, I, n /

17 pectations o a pair o random variables The epectation o a unction g, o random variables and is [ g, ] g,, dd pectation o : [ ], dd d pectation o [ ], dd d...and similarl or and.

18 Correlation between random variables The correlatio n o R The and is [ ], covariance o and dd dd is [ ] [ ] C R Note that correlatio n and covariance are same i eir or have zero mean values.

19 Correlation versus Covariance When we are working with voltages and currents, mean value DC bias rom time - varing we usuall separate component. The random variables n have zero mean. Correlatio n is n equal to covariance. It is reore common in circuit noise to use two terms interchang abl. analsis But, nonzero mean values can return when we e.g. calculate conditiona l distributi ons. Be careul.

20 Correlation Coeicient The correlatio n coeicien t o ρ C / σ σ and is Nt Note between t standard d conusion in terminolog correlatio n and covariance.

21 Sum o TWO Random Variables Sum o two random variables : Z [ ] [ ] [ Z ] [ ] [ ] R I and both have zero means [ ] [ ] [ Z ] C This emphasizes role o correlatio n.

22 Uncorrelated Variables. Uncorrelat ed : C 0 Statisticati ti ll id independend t, : Independen ce implies zero correlatio n. Zero correlatio n does not impl id independend ce. For JGRV' s, uncorrelat ed does impl independen ce

23 Summing o Noise Random Voltages Two voltages are applied to resistor it R The power dissipated in resistor is a random variable P [ P ] P V V V VV V R R V CV V V R R R V σ V V V R R R V V ρv V σ V σ V R R R V VV V R R R The noise powers o two random generators do not add - - a correllationon term must be included. V V R The instantane ous time values o random noise voltages do add.

24 Shot Noise as a Random Variable The iber has transmiss ion probabilit p. Send one photon, and call # o received photons N. [ N ] N p and [ N ] p so σ [ N ] N N p p I we now send man photons M o m, transmis sion o each is statistica i ll id independend t, so calling # o received photons N, [ N ] M [ N ] Mp and σ M σ M p p N N p Now suppose M >>, p <<, and Mp >>, σ N N The variance o count approaches mean value o count.

25 Thermal Noise as a Random Variable A capacitor C is connected to a resistor R. The resistor is in equilibriu m with a "reservoir" a warm room at tempera ture T R can echange energ with room in orm o heat. C can dissipate no power : it establishe s rmal equilibrium with room via resistor. From rmodnamics, an independen t degree o reedom o a sstem at tempera ture T has mean energ kt/, hence kt / CV / kt / R V kt / C The noise voltage has variance kt/c.

26 Distribution o Sums and Jointl Gaussian RV's

27 Distribution o a Sum o Independent Random Variables Sum o two * indendent * random variables : Z F Z z P [ z ] z, d d But z Z z d dz F Z d z, so d Z z z d, convolution o & z.

28 ample: Digital Transmission N rmal noise, Gaussian R received signal T N distribution. T t / δ t / δ t t is transmitted signal, not time N n ep n π σ σσ n n

29 ample o Convolution o Distributions: Communication Sum o two * indendent * random variables : Z F Z z P [ z ] z, d d But z Z z d dz F Z d z, so d

30 Distribution o a Sum o Man Independent RV's Given Z z z d, convolutionol o & z, we can see that convolving,,4,,,,... identical uniorm distributi ons will slowl lead to a orm similar to a Gaussian. This gives some crude sense o central limit orem.

31 Distribution o a Sum o a Few Random Variables Sum o two * perhaps not id indendentd * random variables ibl : Z F Z z d z P [ z ] d, d and Z z F Z dz z This is a major diicult or circuit it & sstem design. Given a random process random unction o time V t, and a linear ilter V t a V t 0 τ a V t τ a V t τ... out 0 in in output V out unction requires computing convolution integrals. in is a sum o random variables, and to ind its distributi on in Jointl Gaussian random variables net avoid this diicult.

32 Pairs o Jointl Gaussian Random Variables I and are Jointl Gaussian :, πσ σ ρ ep ρ σ σ σ σ p This deinition can be etended to a larger # o variables. In general, we can have a Jointl Gaussian random vector,, L, n which is speciied means i, variances b a set o [ ], and covariance s [ ] i i i j

33 Linear Operations on JGRV's I and are Jointl Gaussian, and i we V a b and W c d Then V and W are also Jointl Gaussian. deine This is stated without proo; result arises convolution o Gaussian unctions produces a Gaussian unction. because The result holds or JGRVs o an number.

34 Probabilit distribution ater a Linear Operation on JGRV's V [ ] [ a b ] V a b and W c d σ σ C V W VW [ ] V a [ ] b [ ] ab [ ] a [ ] W c [ ] d [ ] cd [ ] c V W [ ] VW [ a b c d ] VW VW [ ac ad bc bd ] VW ac[ ] ad bc [ ] bd [ ] a b c b d d ted dious det tails We can now calculate joint distributi on o V and W. VW v, w πσ V σ W ρ VW v v v v w w w w ep ρvw σv σvσw σw

35 Jointl Gaussian Random Variables in N Dimensions Deine a set o a covariance C random variables matri C M N is a matri o C : [ ] T correlatio n coeicien ts. ep a random vector C C M C N and N / π det[ C ] / ] C C... O T C C C N N N Ke point : linear operations on jointl Gaussian RV' s, such as L, lead to jointl Gaussian R.V.' s, with C LC L T.

36 Wh are JGRV's Important? The math on last slide was tedious but re is a clear conclusion : With JGRV' s subjected to linear operations, it is suicient to keep track o means, correlatio ns, and variances. With this inormatio n, distributi on unctions can alwas be simpl ound. This vastl simpliies calculatio ns o noise propagatio n in linear sstems linear circuits.

37 Linear Filtering Operations From Nquist' s sampling orem, a bandlimite d uniquel determined b its samples at time points waveorm is n τ We can reore analze a linear ilter in discrete time Vout t a Vin t 0 τ av in t τ av We write signals as vectors : V V out in t τ 0 in [ V out... 0 τ, V [ V in, V in out τ,..., V,..., V in, N ] T outn Hence in some inite time window, n τ ] T [ V V MV out, V out...,..., V Time waveorms are vectors. Filters are linear tra nsormatio ns. out in outn ] T

38 Linear Filtering Operations Noise propagatin g through linear circuits & sstems linear tra nsormatio ns. Sums o R.V.' s are ormed. undergoes I R.V.' s are Jointl Gaussian, n iltered R.V.' s are also jointl Gaussian. Hence, we must onl calculate means & variances and covariance s to determine probabilit distributi ons. I R.V' s are not jointl Gaussian, probabilit distributi ons iltered R.V.' s must be determined b convolution integrals. o It is ortunate that central limit orem causes processes to be jointl Gaussian. man random

39 stimation

40 Conditional Densities Again / /,, Conditional pectati on [ g ] g / d pectatio n o given a particular [ ] / d pec ted...important t or estimation. value that RV o has taken RV on, given value value. o that we : have observed

41 stimation Given an RV, we wish to estimate its value ˆ. Possible measure o qualit o this estimate : mean - square M.S.. [ ˆ ] [ ] ˆ ˆ [ ] error Minimum mean squared error estimate MMS : ˆ [ ] picked to minimize [ ˆ ] Minimum mean squared error MMS MMS [ ] [ ] [ ] [ ] σ

42 stimation I we observe R.V., and use this observation to estimate value o presumabl correlated RV, n ˆ MMS estimate o given that we observe. ˆ [ ] MMS σ [ ˆ ]

43 stimation with JGRV's For a pair o JGRV' s and analsis simpliies : [ ] C σ