Basis for simulation techniques


 Karen Anderson
 9 months ago
 Views:
Transcription
1 Basis for simulatio techiques M. Veeraraghava, March 7, 004 Estimatio is based o a collectio of experimetal outcomes, x, x,, x, where each experimetal outcome is a value of a radom variable. x i. Defiitios [3], page 47 Radom sample: The set of radom variables,,, is said to costitute a radom sample of size from the populatio with the distributio fuctio F( x) provided they are mutually idepedet ad idetically distributed with the distributio fuctio ( x) Fx ( ) for all i ad x. Statistic: Ay fuctio T (,,, ) of the observatios,,, is a statistic. Estimator: Ay statistic Θˆ Θ(, ˆ,, ) used to estimate the value of a parameter θ of the populatio is called a estimator of θ. A observed value θˆ θˆ ( x, x,, x ) is kow as a estimate of θ. F i. Estimatio methods. Method of momets  poit estimate Suppose oe or more parameters of the distributio of are to be estimated based o a radom sample of size. Defie the k th sample momet of to be: M k ' k k,,. () k The k th populatio momet is: µ k ' E [ k ] k,,, which is a fuctio of ukow parameters. ()
2 The method of momets cosists of equatig the first few populatio momets with the correspodig sample momets. Use as may equatios as ukows ad solve simultaeous equatios.. Cofidece itervals ([3], page 484)  iterval estimate We describe four ways of obtaiig cofidece itervals i the followig subsectios... Chebyshev s iequality Var[ Θˆ ] P( Θˆ ε < θ < Θˆ + ε) where Θˆ is the estimator of the parameter θ. This ca be used for ay estimator. ε (3) As a example, applyig this iequality to obtai the populatio mea θ µ, usig the sample mea as a estimator, Θˆ, ad the parameter beig estimated is the mea θ µ, the assumig the populatio variace is, Var[ ] σ, the iequality becomes: σ P ( ε < θ < + ε) Var[ ] ε σ P ( ε < θ < + ε) ε (4) (5) See Sectio 3. for why Var[ ] σ... Uderlyig radom variable has a ormal distributio + populatio variace is kow I geeral, cofidece itervals obtaied by Chebyshev s iequality ca be improved if the distributio of s kow. Here are the steps for obtaiig a cofidece iterval for parameter θ :. Fid a radom variable that is a fuctio of,,, : T T(,,, ; θ) (6) such that the distributio of T is kow.. Fid umbers a ad b such that: Pa ( < T< b) γ. (7)
3 3. After samplig the values x i of, determie the rage of values that θ ca take o while maitaiig the coditio This rage is a 00γ% cofidece iterval of θ. a < t( θ) < b, where (8) t( θ) Tx (, x,, x ; θ). (9) Thus, if s kow to have a ormal distributio, N ( µσ, ), the the sample mea is N ( µσ, ) ad Z (( µ ) ( σ ) ) has the stadard ormal distributio N( 0, ) (see Theorem 3.6 below). Therefore, Pa ( < Z< b) γ is the 00γ % CI (ote Z is the T fuctio of (6)). (0) x µ a < Z< b or a < < b or x bσ < µ < x a σ () σ By choosig a z ad b z, which is the umber of stadar deviatios from the mea oe must go i order to cotai 00γ % of the probability mass, we get the equatio i howtosimulate.doc: M zσ M µ M + zσ M () where σ M σ ( ) ad z is the umber of stadard deviatios from the mea oe must go i a ormal distributio N( 0, ) to cotai 00γ % of the probability mass. I other words PM ( zσ M µ M+ zσ M ) γ. So if γ 0.95, z.96. I other words, we are 95% sure that parameter µ, which is beig estimated, lies i that rage. Theorem 3.6 [3], page 70: Let,,, be mutually idepedet radom variables such that N ( µ i, σ i ), i,,,. The S is ormally distributed, that is S N ( µ, σ ), where
4 µ µ σ σ i i (3) Example 3.30 [3]: Sample mea S. It ca be show that sice S has the distributio S Nµ (, σ ) ad f f S ( x), has the distributio N ( µσ, ) ad the radom variable (( µ ) ( σ ) ) has the distributio N( 0, ). To prove the statemet i the above example, use the results from Sectio 3., where is a radom variable that is a fuctio of aother radom variable S, whose pdf is kow (by theorem 3.6 above). Sice S, we ca write f f S ( x) usig the results of Sectio 3.. We kow that S Nµ (, σ ). Therefore, we have: f ( x) e π σ ( x µ ) σ (4) f ( x) e πσ ( x µ ) ( σ ). (5) Therefore, N ( µσ, ) I Sectio 3., we showed that the Var[ ] beig equal to σ does ot require the s to be ormally distributed. Here, whe the s are ormally distributed, we agai see that the variace of the r.v. s σ...3 Uderlyig radom variable has a ukow distributio but populatio variace is kow Eve without the s beig ormally distributed, because of the Cetral Limit Theorem, the sample mea, as a fuctio of radom variables, has a ormal distributio as. Cetral Limit Theorem ([3], page 7):
5 Let,,, be mutually idepedet radom variables with a fiite mea E [ i ] µ i ad a fiite variace Var[ ] σ i,,,,. We form the ormalized radom variable: Z so that EZ [ ] 0 ad Var[ Z ]. The, uder certai regularity coditios, the limitig distributio of Z is stadard ormal deoted Z N( 0, ), i.e. σ i µ i (6) lim () t PZ ( t) F Z t e y dy π (7) Special case: Let,,, be iid with a commo mea µ E [ i ] ad commo variace σ Var[ ], the (6) becomes Z ( µ ) σ (8) where is the sample mea. Therefore the sample mea from radom samples ted toward ormality as the sample size icreases. Give the Cetral Limit Theorem, the same cofidece iterval as i Sectio.. works here: PM ( zσ M µ M+ zσ M ) γ (9) where σ M σ ( )...4 Cofidece iterval whe populatio variace is ukow Example 3.35 of page 78 [3]: Assume that,,, be mutually idepedet idetically distributed radom variables such that N ( µσ, ). The it follows that V ( µ ) σ (0)
6 has the stadard ormal distributio (see example o first page). From example 3.33, ( )S W σ σ () has the chisquared distributio with degrees of freedom (prove this?). It follows that: T V ( µ )(( ) σ) has the distributio with W S ( µ ) t S ( ) ( ) σ degrees of freedom, where ( ) S () ( µ ) P t ; α < <, or (3) S ( ) t ; α α t ; α S µ t M + ; α σs M (4) where S M S ( ) ad t is the value of a tdistributed radom variable with ; α degrees of freedom such that 00( α) % of the probability mass of the radom variable is cotaied betwee ( t t ; α,. ; α ) Theorem 3.7 [3], page 7: If,,, is a sequece of mutually idepedet, stadard ormal variables, the Y (5) has the gamma distributio, GAM(, ), or the chisquare distributio with degrees of freedom Y χ.
7 If,,, are ot ormal, the above theorem does ot hold as strogly as does the CLT for []. Theorem 3.0. If V ad W are idepedet radom variables such that V N( 0, ) ad W χ, the the radom variable: T V W (6) has the t distributio with degrees of freedom. Example 3.3: Let,,, be a sequece of mutually idepedet, ormal variables N µσ (, ). The radom variables Z i µ i are stadard ormal. Therefore σ Y Z i ( µ ) σ (7) has the chisquare distributio with degrees of freedom Y χ. We typically do ot kow µ, the populatio mea. Therefore, replace µ by the sample mea i. Defie a ra dom variable Referece [] gives a theorem: S σ σ (8) Theorem 6.: Let Let,,, be a sequece of mutually idepedet, ormal variables N ( µσ, ) for all i. Let  i be the sample mea, ad (9)
8 The: S be the sample variace. (30) ( i ). ad S are idepedet.. ( )S σ Chi square( ) σ Questio:. V is stadard ormal eve if s are ot ormally distributed provided is large because of CLT, but does Theorem 3.7 hold eve if Note: oormal distributios []: s are ot ormally distributed? The tbased cofidece iterval procedures are ofte applied whe,,, are ot draw from a ormal distributio. This is acceptable i large samples if the distributio of is reasoably symmetric. However the procedures are ot valid for highly skewed distributios. What about Pareto? File sizes follow the Pareto distributio!,,, Exercise: Work out Problem o page 49 of [3] i class...5 Depedet samples [3], page 503: I all the previous subsectios to fid cofidece itervals, we assumed that the samples were idepedet. But i most of our simulatios, this is ot likely to be true. I this case, the variace is o loger. If we assume that the sequece is widesese statioary, the autocovariace fuctio σ K j i E µ [( )( j µ )] Cov(, j ) (3) is fiite ad is a fuctio of oly i j. The variace of the sample mea is:
9 Var[ ]  Var[ ] + ( i, j ) ( i j) Cov(, j ) (3) because Var[ + Y] E[ (( + Y) E [ + Y] ) ] Var[ + Y] E[ (( + Y) E [ ] EY [ ]) ] Var[ + Y] E[ ( E[ ] ) + ( Y E[ Y] ) + ( E[ ] )( Y E[ Y] )] Var[ + Y] E[ ( E[ ] ) ] + E[ ( Y E[ Y] ) ] + E[ ( E[ ] )( Y E[ Y] )] (33) (34) (35) (36) Comig back to (3): Var[ + Y] Var[ ] + Var[ Y] + Cov(, Y) (37) Write out secod term i (3): Var[ ] σ j Kj (38) Cov (, ) j Cov( ) + Cov( 3 ) + + Cov( ) + ( ij, ) ( i j) (39) Cov( ) + Cov( 3 ) + + Cov( ) + + Cov( ) + Cov( ) + + Cov( ) Cosider how may terms have i j of. This is o the first lie, o the secod, for each subsequet lie util last but oe. The last lie agai has oly oe such term. Therefore, K occurs Cosider the multiplicative factor for + ( ). (40) K i (38). It is
10 . (4) ( ) Give the factor i the deomiator i (3), this checks out. Cosider the multiplicative factor of K. From (39), we see that the first two rows oly have such term as do the last two rows. The itermediate rows will have two such terms, e.g., 3 ad 3 5. Thus from (39), the multiplicative factor of K is + ( 4) + 4. From (38), we see this factor is: Checks out! So (38) is equivalet to (3). As,. (4) ( ) lim Var[ ] σ + K j aσ K, where a j (43) σ j It ca be show that uder rather geeral coditios, the statistic: j µ σ a  (44) of the correlated data approaches the stadard ormal distributio. Therefore, the 00( α) % CI for µ is give by: ± σz α a  (45) We ca avoid havig to estimate σ a by usig the method of idepedet replicatios. Replicate a experimet m times, with each experimet cotaiig observatios. If the seed i the m experimets are chose radomly, the the results of differet experimets will be idepedet though the observatios i a experimet will be depedet. Let the i th observatio i the j th
11 experimet be () j. Let the sample mea ad sample variace of the j th experimet be j () ad S () j, where: j ()  i () j ad (46) S () j [ i () j j ()] (47) Sice ( ), ( ), m ( ) are idepedet (ad idetically distributed) From the idividual sample meas, we obtai a estimator for the populatio mea µ as m  j () m j m m i () j j (48) m V [ j () ] m j ( j ()) m m ( ) m Sice a estimate of the variace is used, we use the tdistributio. Therefore the statistic m j (49) ( µ )(( m) V) is approximately tdistributed with ( m ) degrees of freedom. The 00( α) % CI is x ± (50) m This is actually oe of six approaches to hadle this depedece of samples problem [8] (see basic.pdf ad others.pdf file posted o web site, which are extracted pages from [8]). Also see [0]. They call this approach replicatio/deletio ad suggest deletig some iitial sample poits to get rid of trasiet behavior. t m ; α v Batch meas method: Igore the first few sample poits for trasiets. The take the whole (steady state) process of sample poits ad divide them ito k batches with size of k sam
12 ples. The "sample meas" of those batches/segmets ca roughly be treated as idepedet, if k is sufficiet large. The key for usig batch meas is to select large eough batch sizes. See [9] for rule of thumb o how large to make the batch sizes. The PostNotes3.doc file [9] o CI calculatio explais the details. I that file, the umber of observatios per batch k is suggested to be at least 4t where choice of t is umber of observatios before correlatio dies out (decays to almost zero). This meas that correlatio should be computed ad the batch size determied. I simulatios that oe of my research studets ra, correlatio died out i 00 samples; this meas each batch size should be 800. Spectrum aalysis method [8]: ˆ Obtai a estimator for Var[ ] i (38) by replacig K j with a estimator K j obtaied from the samples: K ˆ j j [ ( )][ + j ( )] j (5) ad a estimator for σ i (38) as S M ( ). Plug K ˆ istead of ad istead of N j K j S σ ito (38) ad get a estimate v of the Var[ ]. The 00( α) % CI for the mea is x ± (5) m Matlab correlatio fuctios (maybe corr) ca be used to compute correlatio. Correlatio ρ Y Cov( x, Y) Var( )Var( Y) t m ; α v A example: For our research work o VBLS, here is what we did to determie how log to apply the replicatio/deletio approach. Each simulatio ru was executed for 6000sec because of the followig computatio. Our largest file size was GB file ad miimum badwidth was 00Mbps, which meas the maximum trasfer time is 80s. If we wat 00 such samples, we eed to at least simulate 00*80 sec 6000 sec. With a call arrival rate of 50 calls/sec, we eed at least 50*6000 or files. The first 0% of the data was dropped. This was arbitrary. My studet said it did t
13 impact results greatly. He could have dropped just 0%. The umber of replicatios was 5. I other words, my studet executed 5 rus for each lambda (call arrival rate). He computed the mea of file trasfer delays from all the sample poits i each ru. The he took these 5 meas ad computed aother mea ad the CI usig the tdistributio formula with 4 as the degrees of freedom. The five meas ca be assumed to be idepedet sice they are from differet rus. Builti matlab fuctio ormfit ca be used for CI calculatio. 3. Appedices 3. Appedix I: Fid the pdf ad CDF of a fuctio of a radom variable whose pdf ad CDF are kow Theorem 3. (page 40, [3]): Let be a cotiuous r.v. with desity that is ozero o a subset I of real umbers (that is f ( x) > 0, x I ad f ( x) 0 for x I. Let Φ be a differetiable mootoic fuctio whose domai is I ad whose rage is the set of reals. The Y Φ( ) is a f cotiuous radom variable with desity f Y give by: f Y ( y) f [ Φ ( y) ][ ( Φ )'( y) ] y Φ() I. (53) 0 otherwise Proof: Assume Φ( ) is a icreasig fuctio. F Y ( y) PY ( y) P( Φ( ) y) P ( Φ ( y) ) F ( Φ ( y) ) (54) dy dy du To get the desity fuctio (use chai rule: ): dx du dx f Y ( y) d FY ( y) d F ( Φ ( y) ) dy dy d d F ( Φ ( y) ) ( ) dy Φ y d Φ ( y) (55) f Y ( y) f [ Φ ( y) ][ ( Φ )'( y) ] (56) If Y a+ b, the
14 f Y ( y) y b f a a y ai + b 0 otherwise (57) The proof is similar for the case whe Φ( ) is decreasig. There whe P( Φ( ) y) P ( Φ ( y) ) F ( Φ ( y) ). But whe we take the derivatives we will get (53). The reaso for takig the absolute value is because pdf is +ve. 3. Appedix II: Derivatio for Var[Sample mea] Derivatio of Var[ ] σ : Var[ + Y] Var[ ] + Var[ Y] if ad Y are idepedet. Therefore Var Var[ ] σ ad Var[ ] Var i. ( σ ) σ This from page 93 of [3]. Here is how I verified it. Let Z Y, where Y Usig (57), sice Z Y, Var[ Z] EZ [ ] ( EZ [ ]) z f Z ( z) dz ( EZ [ ]) (58) f Z ( z) f Y ( z). (59) Therefore (58) becomes: Var[ Z]  fz ( z) dz ( EZ [ ]) f Y ( z) dz ( EZ [ ]) y y f Y ( z) d  y ( EZ [ ]) (60) y Var[ Z]  ( y f Y ( y) dy) ( EZ [ ]) (6) y EZ [ ] zf Z ( z) dz  f ( z ) d  y  yf Y Y ( y) dy EY [ ] (6)
15 Var[ Z] E[ Y EY [ ] ] σ Var ( Y) σ (63) 3.3 Distributios 3.3. Normal distributio If N ( µσ, ), its pdf is: f( x) exp  x µ < x < σ π σ (64) F Z ( z) z e y dy, where Z N( 0, ) (65) π x µ F ( x) F Z σ (66) 3.3. Gamma, chisquare ad t distributios Gamma pdf: GAM( λα, ), pg. 7/8 [3] f() t λ α t α e λt α > 0 t > 0. (67) Γα ( ) Γα ( ) x α e x dx, Γα ( ) ( α )Γ( α ) ad (68) 0 Γ( ) ( )! if is a positive iteger (69) The chisquare distributio is a special case of gamma distributio with α  ad λ , where is a positive iteger. Thus if GAM(, ), the it is said to have a chisquared distributio with degrees of freedom χ. Studett pdf (oe parameter ): f T () t Γ πγ + t ( + )  < t <  (70)
16 Refereces [] K. S. Trivedi, Probability, Statistics with Reliability, Queueig ad Computer Sciece Applicatios, Secod Editio, Wiley, 00, ISBN [] R. Yates ad D. Goodma, Probability ad Stochastic Processes, Wiley, ISBN [3] K. S. Trivedi, Probability, Statistics with Reliability, Queueig ad Computer Sciece Applicatios, First Editio, Pretice Hall, 98, ISBN r. [4] D. Bertsekas ad R. Gallager, Data Networks, Pretice Hall, 986, ISBN [5] Prof. Boorsty s otes, Polytechic Uiversity, NY. [6] A. Leo Garcia ad I. Widjaja, Commuicatio Networks, McGraw Hill, 000, First Editio. [7] Mischa Schwartz, Telecommuicatios Networks, Protocols, Modelig ad Aalysis, Addiso Wesley, 987. [8] Averill M. Law, W. David Kelto, Simulatio modelig ad aalysis, McGraw Hill, 000. [9] Output Aalysis of a Sigle System, IE 305 Simulatio, distributed /9/03 (posted o web site as PostNotes3.doc). [0] Prof. William Saders, UIUC otes (posted o web site). [] R. Fewster, Stats 0, chapter 6.
Probability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chisquare Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chisquare Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationChapter 11 Output Analysis for a Single Model. Banks, Carson, Nelson & Nicol DiscreteEvent System Simulation
Chapter Output Aalysis for a Sigle Model Baks, Carso, Nelso & Nicol DiscreteEvet System Simulatio Error Estimatio If {,, } are ot statistically idepedet, the S / is a biased estimator of the true variace.
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationJoint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }
UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig
More informationStatisticians use the word population to refer the total number of (potential) observations under consideration
6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More information71. Chapter 4. Part I. Sampling Distributions and Confidence Intervals
71 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7 Sectio 1. Samplig Distributio 73 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationThe Sample Variance Formula: A Detailed Study of an Old Controversy
The Sample Variace Formula: A Detailed Study of a Old Cotroversy Ky M. Vu PhD. AuLac Techologies Ic. c 00 Email: kymvu@aulactechologies.com Abstract The two biased ad ubiased formulae for the sample variace
More informationConfidence Level We want to estimate the true mean of a random variable X economically and with confidence.
Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationOutput Analysis and RunLength Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad RuLegth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationConfidence Intervals for the Population Proportion p
Cofidece Itervals for the Populatio Proportio p The cocept of cofidece itervals for the populatio proportio p is the same as the oe for, the samplig distributio of the mea, x. The structure is idetical:
More informationKLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions
We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationNYU Center for Data Science: DSGA 1003 Machine Learning and Computational Statistics (Spring 2018)
NYU Ceter for Data Sciece: DSGA 003 Machie Learig ad Computatioal Statistics (Sprig 208) Brett Berstei, David Roseberg, Be Jakubowski Jauary 20, 208 Istructios: Followig most lab ad lecture sectios, we
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral LidebergFeller CLT, it is most stadard
More informationSampling Distributions, ZTests, Power
Samplig Distributios, ZTests, Power We draw ifereces about populatio parameters from sample statistics Sample proportio approximates populatio proportio Sample mea approximates populatio mea Sample variace
More informationSection 14. Simple linear regression.
Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo
More informationR. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State
Bayesia Cotrol Charts for the Twoparameter Expoetial Distributio if the Locatio Parameter Ca Take o Ay Value Betwee Mius Iity ad Plus Iity R. va Zyl, A.J. va der Merwe 2 Quitiles Iteratioal, ruaavz@gmail.com
More informationStatistical inference: example 1. Inferential Statistics
Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either
More informationPH 425 Quantum Measurement and Spin Winter SPINS Lab 1
PH 425 Quatum Measuremet ad Spi Witer 23 SPIS Lab Measure the spi projectio S z alog the zaxis This is the experimet that is ready to go whe you start the program, as show below Each atom is measured
More informationDS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
More informationFirst Year Quantitative Comp Exam Spring, Part I  203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I1 Part I  203A A radom variable X is distributed with the margial desity: >
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationMA131  Analysis 1. Workbook 2 Sequences I
MA3  Aalysis Workbook 2 Sequeces I Autum 203 Cotets 2 Sequeces I 2. Itroductio.............................. 2.2 Icreasig ad Decreasig Sequeces................ 2 2.3 Bouded Sequeces..........................
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σalgebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationMOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.
XI1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI2 (1075) STATISTICAL DECISION MAKING Advaced
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More informationSequences I. Chapter Introduction
Chapter 2 Sequeces I 2. Itroductio A sequece is a list of umbers i a defiite order so that we kow which umber is i the first place, which umber is i the secod place ad, for ay atural umber, we kow which
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationLesson 10: Limits and Continuity
www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationLinear Regression Models
Liear Regressio Models Dr. Joh MellorCrummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationON POINTWISE BINOMIAL APPROXIMATION
Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 5766 ON POINTWISE BINOMIAL APPROXIMATION BY wfunctions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece
More informationSimple Linear Regression
Simple Liear Regressio 1. Model ad Parameter Estimatio (a) Suppose our data cosist of a collectio of pairs (x i, y i ), where x i is a observed value of variable X ad y i is the correspodig observatio
More informationInstructor: Judith Canner Spring 2010 CONFIDENCE INTERVALS How do we make inferences about the population parameters?
CONFIDENCE INTERVALS How do we make ifereces about the populatio parameters? The samplig distributio allows us to quatify the variability i sample statistics icludig how they differ from the parameter
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationPRACTICE FINAL/STUDY GUIDE SOLUTIONS
Last edited December 9, 03 at 4:33pm) Feel free to sed me ay feedback, icludig commets, typos, ad mathematical errors Problem Give the precise meaig of the followig statemets i) a f) L ii) a + f) L iii)
More informationStatistics 20: Final Exam Solutions Summer Session 2007
1. 20 poits Testig for Diabetes. Statistics 20: Fial Exam Solutios Summer Sessio 2007 (a) 3 poits Give estimates for the sesitivity of Test I ad of Test II. Solutio: 156 patiets out of total 223 patiets
More informationLecture 4. Random variable and distribution of probability
Itroductio to theory of probability ad statistics Lecture. Radom variable ad distributio of probability dr hab.iż. Katarzya Zarzewsa, prof.agh Katedra Eletroii, AGH email: za@agh.edu.pl http://home.agh.edu.pl/~za
More informationMonte Carlo Integration
Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce
More information5.1 A mutual information bound based on metric entropy
Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local
More informationIE 230 Probability & Statistics in Engineering I. Closed book and notes. No calculators. 120 minutes.
Closed book ad otes. No calculators. 120 miutes. Cover page, five pages of exam, ad tables for discrete ad cotiuous distributios. Score X i =1 X i / S X 2 i =1 (X i X ) 2 / ( 1) = [i =1 X i 2 X 2 ] / (
More informationAdvanced Engineering Mathematics Exercises on Module 4: Probability and Statistics
Advaced Egieerig Mathematics Eercises o Module 4: Probability ad Statistics. A survey of people i give regio showed that 5% drak regularly. The probability of death due to liver disease, give that a perso
More informationSubject: Differential Equations & Mathematical ModelingIII
Power Series Solutios of Differetial Equatios about Sigular poits Subject: Differetial Equatios & Mathematical ModeligIII Lesso: Power series solutios of differetial equatios about Sigular poits Lesso
More information4.1 Sigma Notation and Riemann Sums
0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas
More information1 Constructing and Interpreting a Confidence Interval
Itroductory Applied Ecoometrics EEP/IAS 118 Sprig 2014 WARM UP: Match the terms i the table with the correct formula: Adrew CraeDroesch Sectio #6 5 March 2014 ˆ Let X be a radom variable with mea µ ad
More informationSTA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:
STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform largesample ifereces (hypothesis test ad cofidece itervals) to compare two populatio
More informationIt should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.
Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig
More informationCommutativity in Permutation Groups
Commutativity i Permutatio Groups Richard Wito, PhD Abstract I the group Sym(S) of permutatios o a oempty set S, fixed poits ad trasiet poits are defied Prelimiary results o fixed ad trasiet poits are
More informationProbability, Expectation Value and Uncertainty
Chapter 1 Probability, Expectatio Value ad Ucertaity We have see that the physically observable properties of a quatum system are represeted by Hermitea operators (also referred to as observables ) such
More informationLecture 9: September 19
36700: Probability ad Mathematical Statistics I Fall 206 Lecturer: Siva Balakrisha Lecture 9: September 9 9. Review ad Outlie Last class we discussed: Statistical estimatio broadly Pot estimatio BiasVariace
More informationWHAT IS THE PROBABILITY FUNCTION FOR LARGE TSUNAMI WAVES? ABSTRACT
WHAT IS THE PROBABILITY FUNCTION FOR LARGE TSUNAMI WAVES? Harold G. Loomis Hoolulu, HI ABSTRACT Most coastal locatios have few if ay records of tsuami wave heights obtaied over various time periods. Still
More informationMedian and IQR The median is the value which divides the ordered data values in half.
STA 666 Fall 2007 Webbased Course Notes 4: Describig Distributios Numerically Numerical summaries for quatitative variables media ad iterquartile rage (IQR) 5umber summary mea ad stadard deviatio Media
More informationRecall the study where we estimated the difference between mean systolic blood pressure levels of users of oral contraceptives and nonusers, x  y.
Testig Statistical Hypotheses Recall the study where we estimated the differece betwee mea systolic blood pressure levels of users of oral cotraceptives ad ousers, x  y. Such studies are sometimes viewed
More informationSome Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;
More informationStat 200 Testing Summary Page 1
Stat 00 Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece
More informationG. R. Pasha Department of Statistics Bahauddin Zakariya University Multan, Pakistan
Deviatio of the Variaces of Classical Estimators ad Negative Iteger Momet Estimator from Miimum Variace Boud with Referece to Maxwell Distributio G. R. Pasha Departmet of Statistics Bahauddi Zakariya Uiversity
More informationMAS111 Convergence and Continuity
MAS Covergece ad Cotiuity Key Objectives At the ed of the course, studets should kow the followig topics ad be able to apply the basic priciples ad theorems therei to solvig various problems cocerig covergece
More informationTable 12.1: Contingency table. Feature b. 1 N 11 N 12 N 1b 2 N 21 N 22 N 2b. ... a N a1 N a2 N ab
Sectio 12 Tests of idepedece ad homogeeity I this lecture we will cosider a situatio whe our observatios are classified by two differet features ad we would like to test if these features are idepedet
More informationDiscrete probability distributions
Discrete probability distributios I the chapter o probability we used the classical method to calculate the probability of various values of a radom variable. I some cases, however, we may be able to develop
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Crosssectioal data. 2. Time series data.
More informationNCSS Statistical Software. Tolerance Intervals
Chapter 585 Itroductio This procedure calculates oe, ad two, sided tolerace itervals based o either a distributiofree (oparametric) method or a method based o a ormality assumptio (parametric). A twosided
More informationBHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13
BHW # /5 ENGR Probabilistic Aalysis Beautiful Homework # Three differet roads feed ito a particular freeway etrace. Suppose that durig a fixed time period, the umber of cars comig from each road oto the
More informationUCLA STAT 110B Applied Statistics for Engineering and the Sciences
UCLA STAT 110B Applied Statistics for Egieerig ad the Scieces Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistats: Bria Ng, UCLA Statistics Uiversity of Califoria, Los Ageles,
More informationINFINITE SEQUENCES AND SERIES
11 INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES 11.4 The Compariso Tests I this sectio, we will lear: How to fid the value of a series by comparig it with a kow series. COMPARISON TESTS
More informationESTIMATION AND PREDICTION BASED ON KRECORD VALUES FROM NORMAL DISTRIBUTION
STATISTICA, ao LXXIII,. 4, 013 ESTIMATION AND PREDICTION BASED ON KRECORD VALUES FROM NORMAL DISTRIBUTION Maoj Chacko Departmet of Statistics, Uiversity of Kerala, Trivadrum 695581, Kerala, Idia M. Shy
More informationPROBABILITY AND MATHEMATICAL STATISTICS. Prasanna Sahoo Department of Mathematics University of Louisville Louisville, KY USA
PROBABILITY AND MATHEMATICAL STATISTICS Prasaa Sahoo Departmet of Mathematics Uiversity of Louisville Louisville, KY 409 USA THIS BOOK IS DEDICATED TO AMIT SADHNA MY PARENTS, TEACHERS AND STUDENTS v vi
More informationTopic 6 Sampling, hypothesis testing, and the central limit theorem
CSE 103: Probability ad statistics Fall 2010 Topic 6 Samplig, hypothesis testig, ad the cetral limit theorem 61 The biomial distributio Let X be the umberofheadswhe acoiofbiaspistossedtimes The distributio
More information10.1 Sequences. n term. We will deal a. a n or a n n. ( 1) n ( 1) n 1 2 ( 1) a =, 0 0,,,,, ln n. n an 2. n term.
0. Sequeces A sequece is a list of umbers writte i a defiite order: a, a,, a, a is called the first term, a is the secod term, ad i geeral eclusively with ifiite sequeces ad so each term Notatio: the sequece
More informationElement sampling: Part 2
Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig
More informationRiemann Sums y = f (x)
Riema Sums Recall that we have previously discussed the area problem I its simplest form we ca state it this way: The Area Problem Let f be a cotiuous, oegative fuctio o the closed iterval [a, b] Fid
More informationPaired Data and Linear Correlation
Paired Data ad Liear Correlatio Example. A group of calculus studets has take two quizzes. These are their scores: Studet st Quiz Score ( data) d Quiz Score ( data) 7 5 5 0 3 0 3 4 0 5 5 5 5 6 0 8 7 0
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCIB609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationMOMENTMETHOD ESTIMATION BASED ON CENSORED SAMPLE
Vol. 8 o. Joural of Systems Sciece ad Complexity Apr., 5 MOMETMETHOD ESTIMATIO BASED O CESORED SAMPLE I Zhogxi Departmet of Mathematics, East Chia Uiversity of Sciece ad Techology, Shaghai 37, Chia. Email:
More informationEksamen 2006 H Utsatt SENSORVEILEDNING. Problem 1. Settet består av 9 delspørsmål som alle anbefales å telle likt. Svar er gitt i <<.. >>.
Eco 43 Eksame 6 H Utsatt SENSORVEILEDNING Settet består av 9 delspørsmål som alle abefales å telle likt. Svar er gitt i . Problem a. Let the radom variable (rv.) X be expoetially distributed with
More informationClosed book and notes. No calculators. 60 minutes, but essentially unlimited time.
IE 230 Seat # Closed book ad otes. No calculators. 60 miutes, but essetially ulimited time. Cover page, four pages of exam, ad Pages 8 ad 12 of the Cocise Notes. This test covers through Sectio 4.7 of
More informationHOMEWORK #10 SOLUTIONS
Math 33  Aalysis I Sprig 29 HOMEWORK # SOLUTIONS () Prove that the fuctio f(x) = x 3 is (Riema) itegrable o [, ] ad show that x 3 dx = 4. (Without usig formulae for itegratio that you leart i previous
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationCentral Limit Theorem the Meaning and the Usage
Cetral Limit Theorem the Meaig ad the Usage Covetio about otatio. N, We are usig otatio X is variable with mea ad stadard deviatio. i lieu of sayig that X is a ormal radom Assume a sample of measuremets
More informationThe Sampling Distribution of the Maximum. Likelihood Estimators for the Parameters of. BetaBinomial Distribution
Iteratioal Mathematical Forum, Vol. 8, 2013, o. 26, 12631277 HIKARI Ltd, www.mhikari.com http://d.doi.org/10.12988/imf.2013.3475 The Samplig Distributio of the Maimum Likelihood Estimators for the Parameters
More informationReal Analysis Fall 2004 Take Home Test 1 SOLUTIONS. < ε. Hence lim
Real Aalysis Fall 004 Take Home Test SOLUTIONS. Use the defiitio of a limit to show that (a) lim si = 0 (b) Proof. Let ε > 0 be give. Defie N >, where N is a positive iteger. The for ε > N, si 0 < si
More information1036: Probability & Statistics
036: Probability & Statistics Lecture 0 Oe ad TwoSample Tests of Hypotheses 0 Statistical Hypotheses Decisio based o experimetal evidece whether Coffee drikig icreases the risk of cacer i humas. A perso
More information