Monte Carlo Simulation and Generation of Random Numbers
|
|
- Anthony Phelps
- 6 years ago
- Views:
Transcription
1 S Postgraduate Course n Radocommuncatons Sprng 000 Monte Carlo Smulaton and Generaton of Random umbers Dmtr Foursov Dmtr.Foursov@noka.com
2 Contents. Introducton. Prncple of Monte Carlo Smulaton 3. Random umber Generaton Generaton of Unform Random umbers Methods of Generatng Random umbers from an Arbtrary pdf Generatng Gaussan Random Varables Sum-of- Method Box-Muller Method Generatng Independent Random Sequence Whte Gaussan ose Random Bnary Sequences and Random Bnary Waveforms Pseudorandom Bnary Sequences M-ary Pseudonose Sequences 9 4. Generaton of Correlated Random Sequences Correlated Gaussan Sequences Autoregressve and Movng Average (ARMA) Models Spectral Factorzaton Method 0 5. Testng of Random umber Generators 5.. Statonarty and Uncorrelatedness 6. Summary. Introducton Smulaton of a communcaton system requres the generaton of sampled values of all the nput waveforms, processng these samples through models of functonal blocks n the system, and estmatng system performance from the output samples at varous ponts n the model. The focus of the presentaton s on how to generate sampled values of these nput waveforms.. Prncple of Monte Carlo Smulaton Mechancal random number generators were frst used to smulate games of chance, and smulaton technques usng random number generators bear the name of the cty of Monte Carlo, the home of the world famous casno. Whle there are many varatons of the Monte Carlo technque, t bascally nvolves the smulaton of a random experment usng artfcal means,.e., wthout fully repeatng the underlyng physcal experment. In the context of estmatng the bt error rate n a communcaton system, we can defne Monte Carlo smulaton as follows. Wth respect to fg., we are gven a model of a communcaton system and a descrpton of the nput sgnals U(t), V(t), and W(t), whch are assumed to random processes. Our objectve s to fnd the statstcal propertes of Y(t) or the expected value of some functon g(y(t)) of Y(t). If we solve ths problem by emulatng the system ncludng the tme evoluton of all the waveforms n the system, then we have a pure Monte Carlo smulaton. Ths
3 3 mples generatng sampled values of all the nput processes, lettng the models of functonal blocks n the communcaton system operate on them, and observng the output waveforms. Ideally the Monte Carlo smulaton s a one-to-one correspondence wth the real system wthn the lmts of modelng assumptons and approxmaton. U(t) V(t) W(t) Model of a Communcaton System Y(t) Fg.. Defnton of Monte Carlo smulaton The expected value E{g(Y(t))} s estmated from MC smulaton accordng to E{ g( Yˆ( t))} = = g( Y ( )) where the caret ndcates estmated value and s the number of samples smulated. In the context of the specfc example shown n the fgure, Monte Carlo smulaton for estmatng the bt error rate n a dgtal communcaton system nvolves the followng steps:. Generate sampled values of the nput bt sequence {A(k)}, k =,,, and nose samples {(j)}, j =,,, m (the samplng rate s m samples per bt).. Process these samples through the models of the functonal blocks and generate output sequence Y(k). 3. Estmate E(g(Y(k))) as Pˆ e = k = g( Y ( k)) where g(y(k)) = f Y(k) A(k) and g(y(k)) = 0 f Y(k) = A(k).
4 4 Bnary PAM Modulator X(j) Flter Y(j) onlnear Amplfer U(j) Channel Sample & Decde W(j) Flter V(j) Compare & Count Errors Pˆ e = k = g( Y ( k)) Fg.. Smulaton model of a communcaton system The accuracy of estmates obtaned va MC smulaton wll depend on the estmaton procedure used, sample sze, the ablty to reproduce sampled values of the nput process accurately, and modelng assumptons and approxmatons. ot all smulatons are Monte Carlo or pure Monte Carlo smulatons. Monte Carlo mples that at least one random process s emulated and pure Monte Carlo mples all the nput processes are emulated. In many smulaton applcatons t s not necessary to smulate all the random processes present n the system. In the example shown n fg. the error rate performance of the system wll depend on the nose and the cumulatve dstorton ntroduced by the nonlnearty and the flters. Snce the nose s Gaussan and addtve at the nput, ts net effect at the output of the flter and hence on the decson metrc wll also be addtve and Gaussan. In the absence of dstorton, the effect of AWG can be handled analytcally wthout smulaton. Even wth dstorton, the effect of AWG can stll be characterzed analytcally for a gven value of dstorton. MC smulatons n whch only some nput processes nto the system are smulated explctly whle the effects of other processes are handled usng analytcal technques are called partal MC or quasanalytcal (QA) smulatons. The man advantage of a QA smulaton s that t wll requre the smulaton of a fewer samples than a pure MC smulaton to produce estmates wth the same accuracy. 3. Random umber Generaton The key to mplementng a Monte Carlo smulaton s the generaton of sequences of random numbers whch represent the sampled values of the nput random processes. In modelng and smulaton of random processes often a fundamental assumpton s made: that the underlyng random processes are ergodc. Ergodcty assumpton s the key for applyng smulaton results obtaned from one sample functon of a process to the entre ensemble of the process.
5 5 Random number generaton on computers starts wth a formula for generatng random numbers that are unformly dstrbuted n the nterval [0, ]. By applyng approprate transformatons to the sequence of unformly dstrbuted random numbers we can generate random numbers wth arbtrary dstrbutons and temporal propertes. 3.. Generaton of Unform Random umbers The preferred approach for generatng random numbers nvolves the use of recursve formulas that are computatonally effcent. One of the popular methods, called the congruental, uses the followng recursve formula: X ( k) = [ αx ( k )] mod M, k =,,K where α s a carefully chosen nteger between and M, and M s a large prme number or an nteger power of a prme number p m. The random number generaton s started by usng an ntal seed value X(0). Ths generator wll produce ntegers that are unformly dstrbuted between and M. If M s very large and α s chosen properly, then the numbers n the sequence wll appear random and the sequence wll have a maxmum perod of M. For generatng random numbers on computers wth a word sze of 3 bts for nteger arthmetc, the followng formula s recommended: X ( k) = [6807X ( k )] mod ( Wchman-Hll algorthm s used to generate sequences wth longer perod. It nvolves lnearly combnng the outputs of dfferent random number generators wth shorter perods. If we sum two perodc waveforms wth perods and, the resultng waveform wll have a perod = lcm(, ). 3 If and are relatvely prme wth respect to each other, then ) = The Wchman-Hll algorthm s based on ths prncple and t combnes the outputs of three random number generators accordng to and X ( n) = 7X ( n ) mod 3069 Y ( n) = 7Y ( n ) mod Z( n) = 70Z( n ) mod 3033 X ( n) Y ( n) Z( n) U ( n) = + + mod
6 6 The perod of the sequence produced by ths algorthm s p = The man advantage s that t s portable (machne-ndependent), and t guarantees maxmum perod by constructon and there s no need to verfy ts perod emprcally, whch s very dffcult to do when perod s very long. The algorthm, ntroduced by Marsagla and Zaman, s a lnear recursve algorthm and t employs longer memory and carry and borrow operaton n the recurson. There are two smlar versons of ths algorthm, the subtract-wth-borrow and the add-wth-carry algorthm. The subtract-wth-borrow algorthm has the form: Z = X r X s C Z f Z X = Z + b f Z 0 f Z 0 C = f Z < 0 The borrow bt C s ntally set to zero and s toggled between 0 and dependng on whether the recurson produces a postve or negatve nteger Z. The parameters that defne ths generator nclude the base b and lags r and s. It s shown that, n order to guarantee maxmum perod M, the constants b, r and s must be chosen such that M = b r b s + s a prme wth b a prmtve root mod M. For 3-bt machnes, b = 3 5, r = 43, and s = wll produce a perod 0 < 0 M Methods of Generatng Random umbers from an Arbtrary pdf Transform method (analytcal) By applyng a smple transformaton to a unform random varable U, we can generate a random varable Z wth an arbtrary pdf f z (z) as follows: Consder the transformaton Y = F z (Z) where F z (Z) s the cumulatve dstrbuton functon (CDF) of the random varable Z. We can use ths fact to generate Z usng Z = F z - (Y), where Y has a unform pdf:. Generate U unformly dstrbuted n [0, ].. Output Z = F z - (U); Z has the pdf f z (z). If the cdf and the nverse of the cdf of Z can be expressed n closed form, then step n the transform method s easy to mplement. Otherwse both F z (.) and the nverse F z - (.) have to be computed usng numercal methods. Transform method (emprcal) When the nverse transform cannot be expressed n closed form, then the followng emprcal search algorthm can be used to mplement the transform method. If Z s a contnuous random varable, then the dstrbuton s frst quantzed. If p, p,
7 7, p are probabltes of the cells of the unformly quantzed dstrbuton, then the followng algorthm can be used to generate samples of the random varable Z:. Generate U, unform n [0, ].. Let F = j = p j, = 0,,,,, wth F 0 =0. 3. Fnd the smallest value of that satsfes F - < U F, =,,, 4. Output Z = z - + (U F - )/C and return to step. Besdes these two basc methods the other methods can be used among whch are Transform Method for Dscrete Random Varables, Acceptance/Rejecton Method of Generatng Random umbers, and Modfed Acceptance/Rejecton Method Generatng Gaussan Random Varables Sum-of- Method The smplest method of generatng random numbers wth a standard Gaussan pdf s va the use of the central lmt theorem accordng to the formula Y = U ( k) 6.0 k= where U(k), k =,,,, s a set of ndependent, dentcally dstrbuted random varables wth unform dstrbuton n the nterval [0, ]. The number s tradtonal and represents some compromse between speed and accuracy, but there s no reason to lmt k to Box-Muller Method It s well-known that f X and Y are ndependent zero-mean Gaussan varables, then R = (X + Y ) / and θ = tan - {Y/X} have Raylegh and unform pdfs, respectvely. Ths fact can be used to generate two samples of a Gaussan varable. If U and U are two ndependent varables unformly dstrbuted n the nterval, then X = [ ln( U )] cos(πu Y = [ ln( U)] sn(πu ) are ndependent Gaussan random varables wth mean zero and varance. ) 3.4. Generatng Independent Random Sequence Whte Gaussan ose Whte nose has a constant power spectral densty (PSD) for all frequences
8 8 0 S ( f ) =, for < f < However, practcal systems have fnte bandwdths, B, and the samplng rate for smulaton of f s s chosen to be greater than B. For smulaton purposes, sampled values of Gaussan whte nose are generated. The sampled PSD leads to an autocorrelaton functon wth zero crossng at τ = kt s, T s = / f s. The varance of the samples s 0 f s /. Hence, sampled values of bandlmted whte Gaussan nose can be smulated usng an ndependent sequence of Gaussan varables wth zero mean and varance σ 0 f s s = Such a sequence can be generated by multplyng the output of a zero mean, unt-varance Gaussan random number generator σ Random Bnary Sequences and Random Bnary Waveforms A random bnary sequence {b k }, b k = 0 or, can be generated from a unform sequence {U k } by b k = 0 f U f U k k > p p 0 where p 0 = P[b k = 0] Pseudorandom Bnary Sequences A random bnary sequences conssts of a statstcally ndependent sequence of 0 s and s each occurrng wth probablty of ½. A pseudonose (P) or pseudorandom sequence s a perodc bnary sequence wth an autocorrelaton functon that resembles the autocorrelaton functon of a random bnary sequence. The P sequence s generated usng a feedback shft regster arrangement. The feedback shft regster arrangement conssts of bnary storage elements and feedback logc. A feedback shft regster s called lnear f the feedback logc conssts entrely of modulo adders. An m-stage lnear feedback shft regster generates P sequences accordng to S n = cm Sn cm Sn L cs n m+ c0 S n m where S n s the value of the sequence at tme n, and the coeffcents c are bnaryvalued. Let us assocate wth the coeffcents a polynomal of degree m, P m m X ) = X + cm X m + K+ c X + ( c 0
9 9 Then t can be shown that the sequence {S n } wll be a maxmal-length sequence f and only f P m (.) s a prmtve polynomal. Very mportant property of P sequences s that a maxmal-length sequence has pattern of all possble m-bt combnatons each of whch appears only once wthn each perod, except for a pattern of m zeros. Ths property s useful for smulatng ntersymbol nterference (ISI). To smulate the effect of ISI, we need to drve the system wth a bnary sequence that has all possble combnatons of m bts, where m s the memory length of the system measured n bt ntervals M-ary Pseudonose Sequences Many dgtal communcaton systems use M-ary waveforms (M-ary phase, frequency, and ampltude). To smulate the performance of these systems, we may use M-ary P sequences whch can also be generated usng feedback shft regsters. The generaton of such sequences can be done by a generalzaton of the bnary case. Consder the degree-m monc polynomal P m m X ) = X + am X m + K+ a X + ( a 0 wth coeffcents a n a fnte feld of q elements. Fnte felds are called Galos felds and a fnte feld of q elements s denoted GF(q). In M-ary communcaton we normally use values of M = k. Hence, we want to set q = M = k and develop the arthmetc n GF( k ). 4. Generaton of Correlated Random Sequences In many applcatons there s often the need to generate a random sequence whch has a specfed autocorrelaton functon R(k) or R(τ) or power spectral densty S(f). When R(τ) 0 for τ = kt s, k = ±, ±,, the process s correlated at multples of the samplng nterval and hence samples of the random process wll be correlated. In order to smulate the sampled values of such a process, we need algorthms for generatng correlated sequences of random numbers wth a gven autocorrelaton functon R(kT s ). 4.. Correlated Gaussan Sequences An uncorrelated Gaussan sequence can be transformed nto a correlated Gaussan sequence through a lnear transformaton or flterng whch preserves the Gaussan dstrbuton, but alters the correlaton propertes. The coeffcents of the lnear transformaton can be obtaned from the specfed correlaton functon of the output.
10 Autoregressve and Movng Average (ARMA) Models A correlated Gaussan sequence Y(n) can be generated from an uncorrelated Gaussan sequence X(n) usng an autoregressve movng average [ARMA (p, q)] model of the form p = Y ( n) = φ Y ( n ) + θ p q j= qj X ( n j) + X ( n) Autoregressve part Movng average part where X(n) s the nput sequence that s an uncorrelated Gaussan sequence wth zero mean and varance σ X, Y(n) s the output sequence, and φ p, =,,, p and θ qj, j =,,, q, are the parameters of the autoregressve and the movng average parts of the model, respectvely. The model gven s a lnear tme-nvarant dscretetme flter wth a transfer functon H ( f ) + = = q + p = φ θ p q exp( jπf) exp( jπf) and produces an output power spectral densty of the form S YY ( f ) p + = = S XX ( f ) H ( f ) = σ X p + = φ φ p p exp( jπf) exp( jπf) When we mplement an ARMA model we need p ntal values of Y(k) to get the recurson started. If these ntal condtons are chosen arbtrary, then the output wll have a transent n the begnnng and the frst few samples (of the order of 0p) should be gnored Spectral Factorzaton Method We can also desgn a flter n the frequency doman to transform an uncorrelated Gaussan sequence nto a correlated sequence. When we pass an ndependent Gaussan sequence through a flter, the output Gaussan process wll have a power spectral densty gven by S ( f ) S ( f ) H ( f YY = XX ) In many cases the output spectral densty may be gven n emprcal form or t may be n an analytcal form that s not conductve to factorng. In these cases, one of two approaches can be taken:. Emprcally ft an analytcal form to the gven psd and then apply the spectral factorzaton.
11 . Drectly synthesze an FIR flter by settng the flter transfer functon to be equal to the square root of the gven psd. 5. Testng of Random umber Generators 5.. Statonarty and Uncorrelatedness The output of RGs should be statonary and uncorrelated. Smple test for statonarty nvolve the followng steps. Generate a long sequence of samples.. Dvde t nto a long sequence nonoverlappng segments and compute the means and varances of each segment. 3. Test for the qualty of the means and varances. The followng procedure can be used to test for uncorrelatedness and perodcty:. Generate a very long sequence.. Dvde the data nto overlappng segments of ponts wth overlap of /. Let X 0, X, X, etc., represent the data vectors. 3. Compute the normalzed cross correlaton between X0 and X 0, X, X, 4. Plot ρ ˆ ( k) and check for peaks. XX Fgure shows some typcal results. Durbn-Watson Test for Correlaton Ths s more rgorous procedure for testng the hypothess that adjacent members of the sequence X(n) are uncorrelated. The test statstc used s D n= = [ X ( n) X ( n )] n= [ X ( n)] If X(n) and X(n-) are uncorrelated then D would have an expected value of. The value of D would be much smaller f there s strong postve correlaton and would approach 4 f there s strong negatve correlaton. 6. Summary Random varables and random processes are used to model sgnals, nose, and nterference n communcaton systems and also for modelng the randomly tmevaryng behavor of components such as certan types of rado communcaton channels. The startng pont for random number generaton s a computatonally effcent algorthm for generatng an uncorrelated sequence of unformly dstrbuted random numbers. The output of such a generator can be used to produce both correlated sequences and arbtrary dstrbutons by applyng lnear and nonlnear transformatons.
12 Random sequences that represent the output of dgtal sources can be generated usng lnear feedback shft regster arrangement whch s computatonally very effcent. Fg. Example of a correlated sequence. (a) Sampled functon. (b) perodogram of x(t). (c) autocorrelaton of x(t)
13 Problem. Compare the computatonal effcency of the Gaussan random number generators (Sum-of- and Box-Muller),.e. generate say 0000 samples usng each method and compare the run tmes.
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationSimulation and Random Number Generation
Smulaton and Random Number Generaton Summary Dscrete Tme vs Dscrete Event Smulaton Random number generaton Generatng a random sequence Generatng random varates from a Unform dstrbuton Testng the qualty
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationModeling and Simulation NETW 707
Modelng and Smulaton NETW 707 Lecture 5 Tests for Random Numbers Course Instructor: Dr.-Ing. Magge Mashaly magge.ezzat@guc.edu.eg C3.220 1 Propertes of Random Numbers Random Number Generators (RNGs) must
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationDurban Watson for Testing the Lack-of-Fit of Polynomial Regression Models without Replications
Durban Watson for Testng the Lack-of-Ft of Polynomal Regresson Models wthout Replcatons Ruba A. Alyaf, Maha A. Omar, Abdullah A. Al-Shha ralyaf@ksu.edu.sa, maomar@ksu.edu.sa, aalshha@ksu.edu.sa Department
More informationChapter 7 Channel Capacity and Coding
Wreless Informaton Transmsson System Lab. Chapter 7 Channel Capacty and Codng Insttute of Communcatons Engneerng atonal Sun Yat-sen Unversty Contents 7. Channel models and channel capacty 7.. Channel models
More informationChapter 7 Channel Capacity and Coding
Chapter 7 Channel Capacty and Codng Contents 7. Channel models and channel capacty 7.. Channel models Bnary symmetrc channel Dscrete memoryless channels Dscrete-nput, contnuous-output channel Waveform
More informationError Probability for M Signals
Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationNotes prepared by Prof Mrs) M.J. Gholba Class M.Sc Part(I) Information Technology
Inverse transformatons Generaton of random observatons from gven dstrbutons Assume that random numbers,,, are readly avalable, where each tself s a random varable whch s unformly dstrbuted over the range(,).
More informationStatistical analysis using matlab. HY 439 Presented by: George Fortetsanakis
Statstcal analyss usng matlab HY 439 Presented by: George Fortetsanaks Roadmap Probablty dstrbutons Statstcal estmaton Fttng data to probablty dstrbutons Contnuous dstrbutons Contnuous random varable X
More information4 Analysis of Variance (ANOVA) 5 ANOVA. 5.1 Introduction. 5.2 Fixed Effects ANOVA
4 Analyss of Varance (ANOVA) 5 ANOVA 51 Introducton ANOVA ANOVA s a way to estmate and test the means of multple populatons We wll start wth one-way ANOVA If the populatons ncluded n the study are selected
More informationSee Book Chapter 11 2 nd Edition (Chapter 10 1 st Edition)
Count Data Models See Book Chapter 11 2 nd Edton (Chapter 10 1 st Edton) Count data consst of non-negatve nteger values Examples: number of drver route changes per week, the number of trp departure changes
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More informationLossy Compression. Compromise accuracy of reconstruction for increased compression.
Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost
More informationConvergence of random processes
DS-GA 12 Lecture notes 6 Fall 216 Convergence of random processes 1 Introducton In these notes we study convergence of dscrete random processes. Ths allows to characterze phenomena such as the law of large
More informationLecture 3: Shannon s Theorem
CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More information1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands
Content. Inference on Regresson Parameters a. Fndng Mean, s.d and covarance amongst estmates.. Confdence Intervals and Workng Hotellng Bands 3. Cochran s Theorem 4. General Lnear Testng 5. Measures of
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationIRO0140 Advanced space time-frequency signal processing
IRO4 Advanced space tme-frequency sgnal processng Lecture Toomas Ruuben Takng nto account propertes of the sgnals, we can group these as followng: Regular and random sgnals (are all sgnal parameters determned
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationMarkov Chain Monte Carlo Lecture 6
where (x 1,..., x N ) X N, N s called the populaton sze, f(x) f (x) for at least one {1, 2,..., N}, and those dfferent from f(x) are called the tral dstrbutons n terms of mportance samplng. Dfferent ways
More informationRELIABILITY ASSESSMENT
CHAPTER Rsk Analyss n Engneerng and Economcs RELIABILITY ASSESSMENT A. J. Clark School of Engneerng Department of Cvl and Envronmental Engneerng 4a CHAPMAN HALL/CRC Rsk Analyss for Engneerng Department
More informationConsider the following passband digital communication system model. c t. modulator. t r a n s m i t t e r. signal decoder.
PASSBAND DIGITAL MODULATION TECHNIQUES Consder the followng passband dgtal communcaton system model. cos( ω + φ ) c t message source m sgnal encoder s modulator s () t communcaton xt () channel t r a n
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationHere is the rationale: If X and y have a strong positive relationship to one another, then ( x x) will tend to be positive when ( y y)
Secton 1.5 Correlaton In the prevous sectons, we looked at regresson and the value r was a measurement of how much of the varaton n y can be attrbuted to the lnear relatonshp between y and x. In ths secton,
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationAppendix B: Resampling Algorithms
407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VIII LECTURE - 34 ANALYSIS OF VARIANCE IN RANDOM-EFFECTS MODEL AND MIXED-EFFECTS EFFECTS MODEL Dr Shalabh Department of Mathematcs and Statstcs Indan
More informationProbability Theory (revisited)
Probablty Theory (revsted) Summary Probablty v.s. plausblty Random varables Smulaton of Random Experments Challenge The alarm of a shop rang. Soon afterwards, a man was seen runnng n the street, persecuted
More informationProbability and Random Variable Primer
B. Maddah ENMG 622 Smulaton 2/22/ Probablty and Random Varable Prmer Sample space and Events Suppose that an eperment wth an uncertan outcome s performed (e.g., rollng a de). Whle the outcome of the eperment
More informationProbability Theory. The nth coefficient of the Taylor series of f(k), expanded around k = 0, gives the nth moment of x as ( ik) n n!
8333: Statstcal Mechancs I Problem Set # 3 Solutons Fall 3 Characterstc Functons: Probablty Theory The characterstc functon s defned by fk ep k = ep kpd The nth coeffcent of the Taylor seres of fk epanded
More informationSignal space Review on vector space Linear independence Metric space and norm Inner product
Sgnal space.... Revew on vector space.... Lnear ndependence... 3.3 Metrc space and norm... 4.4 Inner product... 5.5 Orthonormal bass... 7.6 Waveform communcaton system... 9.7 Some examples... 6 Sgnal space
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationSIO 224. m(r) =(ρ(r),k s (r),µ(r))
SIO 224 1. A bref look at resoluton analyss Here s some background for the Masters and Gubbns resoluton paper. Global Earth models are usually found teratvely by assumng a startng model and fndng small
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationTime-Varying Systems and Computations Lecture 6
Tme-Varyng Systems and Computatons Lecture 6 Klaus Depold 14. Januar 2014 The Kalman Flter The Kalman estmaton flter attempts to estmate the actual state of an unknown dscrete dynamcal system, gven nosy
More informationModule 2. Random Processes. Version 2 ECE IIT, Kharagpur
Module Random Processes Lesson 6 Functons of Random Varables After readng ths lesson, ou wll learn about cdf of functon of a random varable. Formula for determnng the pdf of a random varable. Let, X be
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationClassification as a Regression Problem
Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/8.40J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) Our focus: effcency of
More informationNote 10. Modeling and Simulation of Dynamic Systems
Lecture Notes of ME 475: Introducton to Mechatroncs Note 0 Modelng and Smulaton of Dynamc Systems Department of Mechancal Engneerng, Unversty Of Saskatchewan, 57 Campus Drve, Saskatoon, SK S7N 5A9, Canada
More informationWhy Monte Carlo Integration? Introduction to Monte Carlo Method. Continuous Probability. Continuous Probability
Introducton to Monte Carlo Method Kad Bouatouch IRISA Emal: kad@rsa.fr Wh Monte Carlo Integraton? To generate realstc lookng mages, we need to solve ntegrals of or hgher dmenson Pel flterng and lens smulaton
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationarxiv:cs.cv/ Jun 2000
Correlaton over Decomposed Sgnals: A Non-Lnear Approach to Fast and Effectve Sequences Comparson Lucano da Fontoura Costa arxv:cs.cv/0006040 28 Jun 2000 Cybernetc Vson Research Group IFSC Unversty of São
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationDigital Signal Processing
Dgtal Sgnal Processng Dscrete-tme System Analyss Manar Mohasen Offce: F8 Emal: manar.subh@ut.ac.r School of IT Engneerng Revew of Precedent Class Contnuous Sgnal The value of the sgnal s avalable over
More informationParametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010
Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton
More informationCHAPTER 4 SPEECH ENHANCEMENT USING MULTI-BAND WIENER FILTER. In real environmental conditions the speech signal may be
55 CHAPTER 4 SPEECH ENHANCEMENT USING MULTI-BAND WIENER FILTER 4.1 Introducton In real envronmental condtons the speech sgnal may be supermposed by the envronmental nterference. In general, the spectrum
More informationStatistics for Economics & Business
Statstcs for Economcs & Busness Smple Lnear Regresson Learnng Objectves In ths chapter, you learn: How to use regresson analyss to predct the value of a dependent varable based on an ndependent varable
More informationComputation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models
Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,
More information/ n ) are compared. The logic is: if the two
STAT C141, Sprng 2005 Lecture 13 Two sample tests One sample tests: examples of goodness of ft tests, where we are testng whether our data supports predctons. Two sample tests: called as tests of ndependence
More informationUncertainty in measurements of power and energy on power networks
Uncertanty n measurements of power and energy on power networks E. Manov, N. Kolev Department of Measurement and Instrumentaton, Techncal Unversty Sofa, bul. Klment Ohrdsk No8, bl., 000 Sofa, Bulgara Tel./fax:
More informationMath Review. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University
Math Revew CptS 223 dvanced Data Structures Larry Holder School of Electrcal Engneerng and Computer Scence Washngton State Unversty 1 Why do we need math n a data structures course? nalyzng data structures
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationprinceton univ. F 13 cos 521: Advanced Algorithm Design Lecture 3: Large deviations bounds and applications Lecturer: Sanjeev Arora
prnceton unv. F 13 cos 521: Advanced Algorthm Desgn Lecture 3: Large devatons bounds and applcatons Lecturer: Sanjeev Arora Scrbe: Today s topc s devaton bounds: what s the probablty that a random varable
More informationIdentification of Linear Partial Difference Equations with Constant Coefficients
J. Basc. Appl. Sc. Res., 3(1)6-66, 213 213, TextRoad Publcaton ISSN 29-434 Journal of Basc and Appled Scentfc Research www.textroad.com Identfcaton of Lnear Partal Dfference Equatons wth Constant Coeffcents
More informationBasic Statistical Analysis and Yield Calculations
October 17, 007 Basc Statstcal Analyss and Yeld Calculatons Dr. José Ernesto Rayas Sánchez 1 Outlne Sources of desgn-performance uncertanty Desgn and development processes Desgn for manufacturablty A general
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationA note on almost sure behavior of randomly weighted sums of φ-mixing random variables with φ-mixing weights
ACTA ET COMMENTATIONES UNIVERSITATIS TARTUENSIS DE MATHEMATICA Volume 7, Number 2, December 203 Avalable onlne at http://acutm.math.ut.ee A note on almost sure behavor of randomly weghted sums of φ-mxng
More informationMarkov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement
Markov Chan Monte Carlo MCMC, Gbbs Samplng, Metropols Algorthms, and Smulated Annealng 2001 Bonformatcs Course Supplement SNU Bontellgence Lab http://bsnuackr/ Outlne! Markov Chan Monte Carlo MCMC! Metropols-Hastngs
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationAn Application of Fuzzy Hypotheses Testing in Radar Detection
Proceedngs of the th WSES Internatonal Conference on FUZZY SYSEMS n pplcaton of Fuy Hypotheses estng n Radar Detecton.K.ELSHERIF, F.M.BBDY, G.M.BDELHMID Department of Mathematcs Mltary echncal Collage
More informationPulse Coded Modulation
Pulse Coded Modulaton PCM (Pulse Coded Modulaton) s a voce codng technque defned by the ITU-T G.711 standard and t s used n dgtal telephony to encode the voce sgnal. The frst step n the analog to dgtal
More information( ) = ( ) + ( 0) ) ( )
EETOMAGNETI OMPATIBIITY HANDBOOK 1 hapter 9: Transent Behavor n the Tme Doman 9.1 Desgn a crcut usng reasonable values for the components that s capable of provdng a tme delay of 100 ms to a dgtal sgnal.
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationNegative Binomial Regression
STATGRAPHICS Rev. 9/16/2013 Negatve Bnomal Regresson Summary... 1 Data Input... 3 Statstcal Model... 3 Analyss Summary... 4 Analyss Optons... 7 Plot of Ftted Model... 8 Observed Versus Predcted... 10 Predctons...
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationis the calculated value of the dependent variable at point i. The best parameters have values that minimize the squares of the errors
Multple Lnear and Polynomal Regresson wth Statstcal Analyss Gven a set of data of measured (or observed) values of a dependent varable: y versus n ndependent varables x 1, x, x n, multple lnear regresson
More informationx i1 =1 for all i (the constant ).
Chapter 5 The Multple Regresson Model Consder an economc model where the dependent varable s a functon of K explanatory varables. The economc model has the form: y = f ( x,x,..., ) xk Approxmate ths by
More informationCS-433: Simulation and Modeling Modeling and Probability Review
CS-433: Smulaton and Modelng Modelng and Probablty Revew Exercse 1. (Probablty of Smple Events) Exercse 1.1 The owner of a camera shop receves a shpment of fve cameras from a camera manufacturer. Unknown
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationComparison of Regression Lines
STATGRAPHICS Rev. 9/13/2013 Comparson of Regresson Lnes Summary... 1 Data Input... 3 Analyss Summary... 4 Plot of Ftted Model... 6 Condtonal Sums of Squares... 6 Analyss Optons... 7 Forecasts... 8 Confdence
More informationSuppose that there s a measured wndow of data fff k () ; :::; ff k g of a sze w, measured dscretely wth varable dscretzaton step. It s convenent to pl
RECURSIVE SPLINE INTERPOLATION METHOD FOR REAL TIME ENGINE CONTROL APPLICATIONS A. Stotsky Volvo Car Corporaton Engne Desgn and Development Dept. 97542, HA1N, SE- 405 31 Gothenburg Sweden. Emal: astotsky@volvocars.com
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More informationPolynomial Regression Models
LINEAR REGRESSION ANALYSIS MODULE XII Lecture - 6 Polynomal Regresson Models Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Test of sgnfcance To test the sgnfcance
More informationLecture 4 Hypothesis Testing
Lecture 4 Hypothess Testng We may wsh to test pror hypotheses about the coeffcents we estmate. We can use the estmates to test whether the data rejects our hypothess. An example mght be that we wsh to
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationDS-GA 1002 Lecture notes 5 Fall Random processes
DS-GA Lecture notes 5 Fall 6 Introducton Random processes Random processes, also known as stochastc processes, allow us to model quanttes that evolve n tme (or space n an uncertan way: the trajectory of
More information( ) ( ) ( ) ( ) STOCHASTIC SIMULATION FOR BLOCKED DATA. Monte Carlo simulation Rejection sampling Importance sampling Markov chain Monte Carlo
SOCHASIC SIMULAIO FOR BLOCKED DAA Stochastc System Analyss and Bayesan Model Updatng Monte Carlo smulaton Rejecton samplng Importance samplng Markov chan Monte Carlo Monte Carlo smulaton Introducton: If
More informationSimulation and Probability Distribution
CHAPTER Probablty, Statstcs, and Relablty for Engneers and Scentsts Second Edton PROBABILIT DISTRIBUTION FOR CONTINUOUS RANDOM VARIABLES A. J. Clark School of Engneerng Department of Cvl and Envronmental
More information4DVAR, according to the name, is a four-dimensional variational method.
4D-Varatonal Data Assmlaton (4D-Var) 4DVAR, accordng to the name, s a four-dmensonal varatonal method. 4D-Var s actually a drect generalzaton of 3D-Var to handle observatons that are dstrbuted n tme. The
More informationQuantifying Uncertainty
Partcle Flters Quantfyng Uncertanty Sa Ravela M. I. T Last Updated: Sprng 2013 1 Quantfyng Uncertanty Partcle Flters Partcle Flters Appled to Sequental flterng problems Can also be appled to smoothng problems
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationComparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method
Appled Mathematcal Scences, Vol. 7, 0, no. 47, 07-0 HIARI Ltd, www.m-hkar.com Comparson of the Populaton Varance Estmators of -Parameter Exponental Dstrbuton Based on Multple Crtera Decson Makng Method
More information