Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider a radom variable x ad the correspodig probability distributio p(x). For coveiece, we cosider the case of a discrete radom variable, although the geeralizatio to cotiuous radom variables is straightforward. Give p(x), oe ca easily compute the expectatio value ad the variace, E(x) µ = x xp(x), () Var(x) = x (x µ) 2 p(x) = E(x 2 ) [E(x)] 2. (2) The stadard deviatio of x is deoted by σ Var(x). I the real world, p(x) is usually ukow, i which case µ ad σ are ukow. However, oe ca perform experimets to measure x. Suppose measuremets are made, ad the values x, x 2,...x are obtaied. Ideally, we would like to recostruct the probability distributio p(x) from the data, but here we are iterested i determiig the expectatio value µ ad the stadard deviatio σ from the experimetal results. We ca regard x, x 2,...x as idepedet ad idetically distributed radom variables (ofte abbreviated as iid or IID radom variables). These are idepedet, sice separate measuremets of x are idepedet of each other. These are idetically distributed, sice the experimet is measurig the same radom variable x each time (although, of course, the outcome of each measuremet will ot be the same). This meas that E(x i ) = µ ad Var(x i ) =, for i =,2,3,...,. Of course, the above iformatio is ot very practical, sice a priori we do ot kow the values of µ ad σ. Havig made idepedet measuremets, we would like to ascertai the best possible estimates for µ ad σ. I class, we defied the sample average x ad the sample variace Σ 2 by x x i, (3) Σ 2 (x i x) 2. (4)
These quatities are easily computed from the data. We ow assert that the sample average x provides a best estimate for the actual mea µ ad the sample variace Σ 2 provides a best estimate for the actual variace. I the mathematical statistics literature, there is some debate as to the meaig of the word best. I the preset cotext, the word best simply meas that the estimates are ubiased, that is E(x) = µ ad E(Σ 2 ) =, (5) where the expectatio values are computed assumig for a momet that we do kow the uderlyig probability distributio p(x). Let us verify eq. (5) explicitly. First, recallig that E(cx) = ce(x) ad E(x+y) = E(x)+E(y), we have E(x) = E(x i ) = µ = µ = µ. I the Appedix, we demostrate that E(Σ 2 ) =. To reiterate, x provides a best estimate of the ukow µ, which is the expectatio value of the radom variable x. Similarly, Σ 2 provides a best estimate of the ukow, which is the expectatio value of (x µ) 2. 2. The stadard deviatio of the mea Although x provides a best estimate of the ukow µ, its determiatio does ot tell us howlikely it is that themeasured valuexis close toµ. After all, if Iperformadditioal measuremets of x, I would expect the value of the average x to chage (although the chage is expected to be small oce is large eough). Thus, what we would really like to kow is the probability distributio of the radom variable x. Of course, sice we do ot kow i geeral the expectatio value ad variace of x, we also do ot kow i geeral the expectatio value ad variace of x. Ideed, we have already see that E(x) = µ, which we do ot kow. Likewise, we ca compute Var(x) as follows: Var(x) = Var ( ) x i = Var(x 2 i ) = 2 = 2 σ2 = σ2, (6) which depeds o the ukow. However, we do have a best estimate for based o our data, amely Σ 2 defied i eq. (4). Hece, we shall defie the stadard deviatio of the mea (also called the stadard error) to be σ m, where m Σ2 = () (x i x) 2. (7) Theexperimetalistowcocludesaftertakigdataadobtaiigthevaluesx,x 2,...,x after measuremets, that the best estimate of the mea is x±σ m. 2
If there is a theoretical value of µ to compare this to, the experimetalist ca ow make statemets ivolvig cofidece itervals (e.g., the probability that the data is cosistet with the theoretical expectatio), as discussed i Boas. It is very importat to distiguish σ m, which is obtaied from data ad σ which is the ukow stadard deviatio of the radom variable x. We have = Var(x), which is determied by the probability distributio p(x) ad does ot deped o the umber measuremets performed by the experimetalist. The experimetalist ca make a estimate for, amely Σ 2 give by eq. (4). It may look like Σ 2 depeds o, but the depedece is pretty weak (if is large). After all, x also depeds weakly o (if is large), which provides the best estimate for µ. However, σm 2 = Σ2 / depeds strogly o. The more measuremets that are made, the smaller σm 2 is. This is ot surprisig, sice oe expects that the larger is, the better x is as a estimate for µ. As emphasized above, σm 2 is a estimate of the variace of x, which is obviously ot the same as the variace of x [they differ by a factor of as show i eq. (6)]. Equivaletly, Σ is a estimate of the ucertaity i a sigle measuremet of the radom variable x, whereas σ m is a estimate o the ucertaity of the mea value of the radom variable x as determied by measuremets. A simple example illustrates the above discussio. Suppose that p(x) is the biomial distributio with probability p that a tossed coi will lad o heads. Defie the radom variable, { x =, the coi lads o heads, x = x = 0, the coi lads o tails. Give this coi, the experimetalist is asked to determie the mea µ = p ad the variace = p( p) by flippig the coi times. After flips, the experimetalist obtais a data set, x, x 2,...,x, which is a series of s ad 0s. From this data, the experimetalist computes x which is equal to the umber of heads divided by. The experimetalist also computes Σ usig eq. (4) ad σ m usig eq. (7). The experimetalist cocludes that the probability p of the coi (i.e, the true mea µ) is x±σ m, where the error bars represet a 68% cofidece iterval, correspodig to a oe stadard deviatio of the mea ucertaity. Clearly, the large is (i.e. more coi flips), the smaller the correspodig stadard error σ m, ad cosequetly the more reliable x is as a estimate of the probability p of the coi. Likewise, the best estimate for is give by Σ 2. By the way, the latter determiatio also has a error associated with it, which I briefly discuss i Sectio 3 of these otes. Refereces ad 2 provide a coget discussio of the differeces betwee stadard deviatio ad the stadard deviatio of the mea. I particular, referece is a superb treatmet of error aalysis writte specifically for physicists at a elemetary level. 3
3. The stadard deviatio of the variace Although Σ 2 provides a best estimate of the ukow, this does ot tell us how likely it is that the measured value Σ 2 is close to. After all, if I perform additioal measuremets of x, I would expect the value of the average Σ 2 to chage (although the chage is expected to be small oce is large eough). Thus, what we would really like to kow is the probability distributio of the radom variable Σ 2. Of course, sice we do ot kow i geeral the expectatio value ad variace of x, we also do ot kow i geeral the expectatio value ad variace of Σ 2. Ideed, we have already see that E(Σ 2 ) =, which we do ot kow. Likewise, oe ca compute Var(Σ 2 ). The result depeds o ad o E(x 4 ) which we have ot discussed i this course. However, it may be of some iterest to cosider the case of a ormal distributio, sice the cetral limit theorem ca be applied if is large eough. I this case, it is straightforward to show that E(x 4 ) = 3σ 4, i which case Var(Σ 2 ) depeds oly o σ. The result (obtaied i Appedix E of referece ad Appedix C of referece 3) is: Var(Σ 2 ) = 2(), which agai depeds o the ukow. However, we ca agai employ best estimate for based o our data, amely Σ 2. Thus, we coclude that uder the assumptio that p(x) is the ormal distributio of ukow mea ad variace, the the best estimate of the stadard deviatio of the variace of the radom variable x obtaied from our data is give by σ v, where v = Σ 2 2() = 2() 2 (x i x) 2. As i the case of σ m, we see that σ v also ca be reduced i size by performig more measuremets (i.e. by takig larger). However, i practice σ v (sometimes called the error of the error ) is ot ofte employed i experimetal aalyses. Refereces. Joh R. Taylor, A Itroductio to Error Aalysis: the study of ucertaities i physical measuremets, 2d editio (Uiversity Sciece Books, Sausalito, CA, 997). 2. David L. Streier, Maitaiig Stadards: Differeces betwee the Stadard Deviatio ad the Stadard Error, ad Whe to Use Each, Caadia Joural of Psychiatry, 4 (996) pp. 498 502. 3. Jörg W. Müller, Some Secod Thoughts o Error Statemets, Nuclear Istrumets ad Methods 63 (979) 24 25. 4
APPENDIX: Proof that E(Σ 2 ) = Startig with eq. (4), we shall compute E(Σ 2 ) = It is coveiet to rewrite the above equatio by otig that after usig E[(x i x) 2 ]. (8) Var(x i x) = E[(x i x) 2 ] [E(x i x)] 2 = E[(x i x) 2 ], Thus, eq. (8) ca be rewritte as E(x i x) = E(x i ) E(x) = µ µ = 0. E(Σ 2 ) = Var(x i x). To evaluate the above expressio, we shall use Var(cx) = c 2 Var(x) ad Var(x + y) = Var(x)+Var(y), where the latter holds uder the assumptio that x ad y are idepedet radom variables. Sice x i ad x are ot idepedet radom variables (sice x cotais x i i its defiitio), we must perform the followig maipulatio, x i x = x i ( ) x i = x i x j. Cosequetly, E(Σ 2 ) = = = = = = [ ( ) ] Var x i x j ( ) 2 Var(x i )+ Var(x 2 j ) ( ) 2 + 2 ( ) 2 ( ) + 2 2 [+] = σ2 = σ2, which completes the proof. Note that this computatio justifies the presece of the deomiator factor rather tha i eq. (4). 5