Math493 - Fall HW 2 Solutions

Size: px
Start display at page:

Download "Math493 - Fall HW 2 Solutions"

Transcription

1 Math493 - Fall HW 2 Solutions Renato Feres - Wash. U. Preliminaries. In this assignment you will do a few more simulations in the style of the first assignment to explore conditional probability, Bayes theorem, and also play with the concept of random variables. Some useful R tricks are introduced below: creating your own functions, and plotting graphs and histograms. I also elaborate a bit on the notion of random variables. Conditional probability. To illustrate the idea of conditional probability consider the following problem. From a bowl containing five red, three white, and seven blue chips, select four at random and without replacement. Compute the conditional probability of one red, zero white and three blue chips, given that there are at least three blue chips in this sample of four chips. Suppose that the bowl has a total of N chips of which N r are red, N w are white, and N b are blue. Therefore, N r + N w + N b = N. We select a sample of size n from the bowl, at random without replacement, and ask for the probability that there are n r red, n w white, and n b blue chips in the sample. So n r + n w + n b = n. I will indicate this event simply by (n r,n w,n b ). Then an elementary counting argument already used in Homework Set 1 gives the probability P(n r,n w,n b ) = ( Nr )( Nw )( Nb n r n w ( N ). n n b ) It is not hard to see how this generalizes to any number of colors. Keep this example in mind as a model for the general (multivariable) hypergeometric distribution. This is the key information you need to solve the problem by hand, in addition to the definition of conditional probability: P(A B) P(A B) =. P(B) (Hint: A = (1,0,3) and B is the union of the events (1,0,3), (0,1,3), and (0,0,4).) I won t give away the answer, but if you d like to check your calculation, try to run a simulation of the problem. The following script does this. It also illustrates how conditional probability may be approximated by Monte Carlo simulation. (You may need to go back to the R tutorial in the preliminaries section of the first homework assignment to make sense of the below programs.) Chips=c(1,1,1,1,1,2,2,2,3,3,3,3,3,3,3) #1 for red, 2 for white, 3 for blue N= #Number of trials of the experiment Total_s=0 #Counts the number of successes (right number #red, white and blue chips in sample of 4) Total_c=0 #Counts the number of trials with least 3 blue #chips in sample of 4.

2 for(i in 1:N){ a=sample(chips,4,replace=false) #Select 4 chips #Count number of red, white and blue chips in sample r=sum(a==1) w=sum(a==2) b=sum(a==3) if (b>=3){ Total_c=Total_c+1 #Success means: 1 red, 0 white and 3 blue chips: Total_s=Total_s+(r==1 && w==0 && b==3) Total_s/Total_c #Relative frequency approximates probability One run of this program gave me the number for the conditional probability that the sample of size 4 contains one red, zero white and three blue chips given that it contains at least three blue chips. Bayesian probability. Bayes theorem is a very simple but extremely significant idea. It is at the heart of many subjects, from machine learning to Bayesian statistics, which is a method of inference in which Bayes theorem is used to revise our estimate of some hypothesis as additional data are acquired. The following is a standard textbook example. (It may serve as a model for one of the homework problems, below.) Let A be the event that a given person has a disease for which a clinical test is available. Let B be the event that the test gives a positive reading. We may have prior information as to how reliable the test is, so we may already know the conditional probability P(B A) that if the test is given to a person who is known to have the disease, the reading will be positive, and similarly P(B A c ), of a positive reading if the person is healthy (i.e., a false positive). We may also know how common or rare the disease is in the population at large, so the prior probability, P(A), that a person has the disease may be known. Assume that: 3% of the population has the disease, that the probability of a diseased person being tested positive is 98%, and of a healthy person being tested positive is 1%. 1. What is the probability of actually having the disease given that the test is positive? By Bayes formula, this probability is given by Substituting the numerical values: This is approximately 75%. P(A B) = P(A B) = P(B A)P(A) P(B A)P(A) + P(B A c )P(A c ) = What is the probability of having the disease given that the test is negative? Similarly, P(A B c ) = P(B c A)P(A) P(B c A)P(A) + P(B c A c )P(A c ). 2

3 Substituting the numerical values: This is approximately 0.06%. P(A B c ) = (1 0.98) 0.03 (1 0.98) (1 0.01) 0.97 = We can confirm the above results by stochastic simulation. This amounts to running many independent trials (say N, which we think of as the number of persons tested) of the following random experiment: (1) to each person assign the condition of having the disease or not at random and independently, with the given probability for having the disease; (2) independently, assign to this person a test result, with the given probabilities for a positive or negative test result; (3) keep a counter for the number of times the test is positive, and for the number of times that both the test is positive and the person has the disease. The probability that the person has the disease given that the test is positive is then approximated by the fraction of individuals with the disease in the group of individuals tested positive. The following program implements this in R. N = #Number of persons selected (the larger the better) #Each person is randomly "assigned" a value 1 (has the disease) or 0 (does not #have it). So D[j]=1 if person j has the disease and 0 otherwise. D = sample(c(0,1),n,replace=true,c(0.97,0.03)) #Initialize the vector T of test results: T[j]=1 if test is positive #for person j and 0 if negative. T = 0*c(1:N) #Arbitrarily giving test result 0 to all persons, simply to initialize T. #Now person j is tested (this will assign T[j] with a test result value): for (j in 1:N){ if (D[j]==1) { T[j]=sample(c(0,1),1,replace=TRUE,c(0.02,0.98)) else { T[j]=sample(c(0,1),1,replace=TRUE,c(0.99,0.01)) #Now create a vector TD whose jth entry is 1 if jth person both has the #disease and was tested positive, and 0 otherwise. Note that this vector #is simply the elementwise product of T and D: TD = T*D f = sum(td)/sum(t) #This fraction approximates the conditional probability we want. f One run of the script gave me the value f = This seems to support the result obtained analytically. The following script is equivalent, but avoids the for loop. It uses the R operation which and employs vector operations more fully. This helps to speed up the program considerably. (Observe that I have used a much bigger N than in the first version. Run the two versions to see the difference in speed.) N = #Number of persons selected #Each person is randomly "assigned" a value 1 (has the disease) or 0 (does not #have it). So D[j]=1 if person j has the disease and 0 otherwise. D = sample(c(0,1),n,replace=true,c(0.97,0.03)) #Initialize the vector T of test results: T[j]=1 if test is positive #for person j and 0 if negative. 3

4 T = 0*c(1:N) #Arbitrarily giving test result value 0 to all the persons. #Now look for the indices of those persons with and without the disease: d1 = which(d==1) d2 = which(d==0) n1 = sum(d==1) n2 = N-n1 #Assign test values to the persons: T[d1] = sample(c(0,1),n1,replace=true,c(0.02,0.98)) T[d2] = sample(c(0,1),n2,replace=true,c(0.99,0.01)) #Now create a vector TD whose jth entry is 1 if jth person both has the #disease and was tested positive, and 0 otherwise. Note that this vector #is simply the elementwise product of T and D: TD = T*D f = sum(td)/sum(t) #This fraction approximates the conditional probability we want. f This gave me the value f = I then ran the same script with N = (it took about one minute on my Mac) and the result was This agrees with the analytical solution (part 1 of the problem) up to 4 decimal places. User defined functions in R. A very useful device in R is to create your own functions. The following examples should give you the basic idea. #We define three functions, f, g, h. #Note that h is a piecewise defined function. #You may enter these function definitions directly on the command line console in R. f=function(x) x^2+3*x-5 g=function(x) 2*sin(3*x)-5*cos(5*x) h=function(x) { f(x)*(x<0)+g(x)*(x>=0) #Now we calculate a few values: > f(4) [1] 23 > g(-2) [1] > f(g(1)) [1] > h(-1)-f(-1) [1] 0 #We can plot a graph of h over the interval [-5,2*pi] as follows. x=seq(from=-5, to=2*pi, by=(2*pi+5)/1000) #The sequence x consists of 1001 equally spaced numbers between -5 and 2*pi. #The following shows how to plot h(x) over this sequence, adding labels and title. #You can choose different types of line. The simplest choice is "type= l " for line. plot(x,h(x),type= l,main= Graph of the function h,xlab= x variable, ylab= y variable ) 4

5 grid() #Add a grid. #The resulting graph is shown below. Graph of the function h y variable x variable Random variables. I will need to get a little abstract for a moment. Please bear with me here as there is a concrete and very useful point about computer simulation worth noting in the end. (See the main remark towards the end of the next page.) The concept of random variable is introduced in Chapter 4 of the textbook. I will present it here in a more mathematically formal fashion. Let (X,F,P) be a probability space, where X is a sample space, or set of outcomes (it is simply a set; I had used the symbol X for this in class, but I ll reserve X for random variables here), F is a σ-algebra (whose elements are subsets of X referred to as events), and P is a probability measure on B. Recall the axioms for F and P: Axioms of the σ-algebra of events F 1. X F ; 2. if A F then A c F ; 3. if A 1, A 2, F then A i F. In class I used the term algebra for F, but σ-algebra ( sigma-algebra ) is the technically proper name. Axioms of the probability function P : F [0,1] 1. P(X ) = 1; 2. if A 1, A 2,... are mutually disjoint events, then P( A i ) = A i. One often thinks of (X,F,P) as the mathematical specification of a random experiment. Now let Y be some other set and X : X Y a function. (I like to indicate functions by such arrows. This means that X is the domain of X and X (x) Y for each x X.) A random variable is simply any such function. (But see a refinement of this definition below.) A couple of points to observe. First note that Y can be given its own σ-algebra, which I will denote by F X, defined as follows: A subset D of Y belongs to F X if (by definition) X 1 (D) := {x X : X (x) D 5

6 is an event in the σ-algebra F. (I use the symbol := to indicate that the left-hand side of this equality is defined by the right-hand side.) Furthermore, X induces a probability measure P X on F X as follows. For any D F X (hence D Y ) P X (D) := P ( X 1 (D) ). (1) Notice that we are using the probability measure P on F and the function X to define a probability measure P X on F X. The measure P X is often called the law of X. The following exercise is optional, but think about it for a few minutes! Exercise 1. Show that (Y,F X,P X ) satisfies the axioms of a probability space. It may be the case, that Y already had a σ-algebra of its own before we chose X. Let F denote this σ-algebra. Then for X : X Y to be called a random variable, and for the probability measure P X to make logical sense, we require that X 1 (D) be an event in F for every event D F. In other words, we require F F X. For example, if the random variable X takes values in R, then we always consider the σ-algebra of subsets in R generated from intervals through the use of basic set operations applied at most countably many times. This is called the Borel σ-algebra. (More on this in the next bullet point.) In this case, we say that the function X : X R is a random variable if every event of the form X 1 (I ) belongs to F, where I is any interval. Random numbers in [0, 1]. Consider the experiment of choosing a random (real) number between 0 and 1 so that all numbers are equally likely to be chosen. This experiment is formally represented by the probability space ([0,1],B,λ), where B is the Borel σ-algebra (whose elements are called Borel sets), which is the σ-algebra generated by the open intervals of R intersected with the unit interval; and λ is the length measure, also called the Lebesgue measure, that assigns value b a to an interval [a,b] [0,1]. That is, λ([a,b]) = b a. (Technical point: one typically throws into B also all those subsets of [0,1] which are contained in Borel sets of zero length. This process of adding all sets that ought to be assigned measure 0 but may not have been in B initially is called completion of the probability space.) Finally, here s the main remark: It turns out that, for essentially anything we can possibly want do in this course (as well in more advanced graduate level courses in probability theory), the domain of our random variables (say, taking on numerical values) can be assumed simply to be ([0,1],B,λ). This makes the concept of a random variable very concrete. The moral of this long story is this: If we have any means of producing random numbers in [0, 1] with the uniform probability distribution, then we can simulate essentially any other numerical random variable of interest by constructing an appropriate function X from [0,1] into R. Other target spaces for X that will be useful to us are the n-dimensional coordinate spaces R n. The precise statement and proof of this general claim belongs to a more advanced course, but the following discussion should hint at why this may be true. Discrete random variables and the indicator function of sets. In R, the random experiment of producing a uniformly distributed random number between 0 and 1 amounts to invoking the function runif. Thus, for example, to produce a sequence of 5 such random numbers: > runif(5) [1] We can define a (mathematical) fair coin as a random variable X : [0,1] {0,1, where 0 stands for Heads and 1 for Tails, as follows: 0 if x [0,1/2) X (x) = 1 if x [1/2,1] 6

7 Then flipping a coin amounts to choosing a random number x [0, 1] with probability distribution λ (uniform distribution) and evaluating X (x). More generally, if p is the probability of heads, then the random experiment of flipping the possibly biased coin amounts to the function 0 if x [0, p) X (x) = 1 if x [p,1] That this does what is expected can be shown as follows. P X ({0) = λ(x 1 ({0)) = λ({x [0,1] : X (x) = 0) = λ([0, p)) = p 0 = p. Similarly, we have P X ({1) = 1 p. This random variable may be implemented as an R-function as follows. Coin=function(n,p) { x=runif(n) 1*(x>=p) If I want 5 independent tosses of a fair coin: > Coin(5,1/2) [1] Suppose now that X is the random variable taking on values in {a 1,..., a k with probabilities p 1,..., p k, respectively, where the a i are simply numbers. There is a convenient way to write X using the so-called indicator function of a set. (Another name for it is the characteristic function of the set.) If A is a (Borel) subset of [0,1], we let 1 A (x) represent the function which is 1 if x A and 0 if not: 1 if x A 1 A (x) = 0 if x A c We want to write X as a linear combination of indicator functions. Let A 1, A 2,..., A k form a partition of [0,1] such that λ(a i ) = p i. Any partition will do so long as the lengths are equal to p i. For example, let A i be the intervals A 1 = [0, p 1 ), A 2 = [p 1, p 1 + p 2 ), A 3 = [p 1 + p 2, p 1 + p 2 + p 3 ),..., A k = [p p k 1,1]. Note that these sets indeed form a partition of [0,1] (meaning that they are disjoint Borel sets and their union is all of [0,1]) and each A i is an interval of length λ(a i ) = p p i 1 + p i (p p i 1 ) = p i. 7

8 Now, the punchline of this story is that any discrete random variable X with k possible values can be written as k X (x) = a j 1 A j (x). (2) j =1 In fact, it was not important that k be finite; the same argument works also for k =. It is also easy to check from the definition of P X that P X ({a j ) = λ(a j ) = p j for each j. Allowing k to be infinite, equation 2 gives in fact the general form of a discrete random variable. Indicator functions are easy to implement in R using the logical relations: == Equals! = Not equal > Greater than >= Greater than or equal to < Less than <= Less than or equal to Or Or (use with vectors and matrices) && & And And (use with vectors and matrices) As an example, suppose that X is a random variable taking on the possible values 2,5,6 with probabilities 1/2,1/3,1/6, respectively. Here is a user defined R function implementing this random variable. #Defining the indicator functions of the three intervals Ind1=function(x) (x>=0 & x<1/2) Ind2=function(x) (x>=1/2 & x<1/2+1/3) Ind3=function(x) (x>=1/2+1/3 & x<=1) #Now define the random variable X X=function(x) -2*Ind1(x)+5*Ind2(x)+6*Ind3(x) To see the graph of X : #Create a sequence in the interval [0,1] x=seq(from=0,to=1,by=1/1000) plot(x,x(x),type= l,main= Graph of the random variable X ) grid() #The graph is shown next. Note that X is discontinuous at 1/2 and 5/6. All we need to know about X is already contained in its probability distribution P X. From the definition of X we have P X ( 2) = 1/2, P X (5) = 1/3, P X (6) = 1/6. It is possible to approximate this distribution by Monte Carlo simulation. We simply draw a large number of independent values of X and plot the relative frequency of the three values 2,5,6. Such a frequency plot is also called a histogram. I will use this example as a model. The idea is to generate a large number of sample values of X and then represent the frequency (or relative frequency) of each value in { 2,5,6 by the height of a bar in a bar chart. Here it is: #First I will generate, say random values of X with the given 8

9 Graph of the random variable X X(x) x #probabilities. #As noted above this amounts to generating random numbers in [0,1] #with the uniform distribution, then evaluating the function X at #those numbers. x=runif(10000) Xvalues=X(x) #Now the hist command. It requires specifying the break points of the #bins, or intervals, containing the values of X. This breakpoints=c(-2.5,-1.5,-0.5,0.5, 1.5, 2.5,3.5,4.5,5.5,6.5) hist(xvalues,breaks=breakpoints,prob=true) #prob=true indicates that we want the relative frequencies rather #than absolute counts. #The histogram is shown below. Histogram of Xvalues Density Xvalues It is good to know how to translate more or less verbatim mathematical expressions into R as we just did, but in 9

10 practice we could have used the built-in function sample for generating random values of X. Here is how this would look: #Create a vector of values x=c(-2,5,6) #Create a vector of probabilities p=c(1/2,1/3,1/6) #Now generate sample values of X Xvalues=sample(x,10000,replace=TRUE,p) #From this we could go on to plot the histogram just as above. Continuous random variables. The above idea of representing discrete random variables as a linear combination of indicator functions of sets in [0, 1] suggests that one could similarly approximate continuous random variables by such stepwise functions. We will explore this idea later in the course. For now I just note the conclusion that continuous random variables can also be represented as functions X : [0,1] R, now no longer taking on values in a discrete set. Just as in the discrete case, to find sample values of X we choose random numbers x in [0,1] with the uniform probability distribution and evaluate X (x). The probability distribution P X is just as given in the general definition above. (See the section on Random variables. ) Continuous random variables are characterized by the property that P X has a probability density. This means that there is a (often continuous) function f (x) on the range of values of X such that P X (A) = f (x)d x. A We can visualize the density function f (x) in a similar way as we did above for the discrete random variables using a histogram. By making the break points of the histogram bins (the base of the rectangular bars) sufficiently close together we can make it look as continuous as we want. (An alternative is to use the R function density.) This is illustrated with the next example. #First I create a continuous random variable. This is an arbitrary example: X=function(x) { 100*x*(1-x)*(x-0.5)^2 #Let us see first what this function looks like by plotting its graph: x=seq(from=0,to=1,by=1/1000) plot(x,x(x),type= l ) 10

11 X(x) x #We now generate (this number is arbitrary) independent values of X x=runif( ) Xvalues=X(x) #We now produce a histogram plot approximating the probability density of X. #In the variable breaks now I simply specify the number of bins rather than exact breakpoints. hist(xvalues,breaks=100,prob=true) #The plot is below. Histogram of Xvalues Density Xvalues Can you explain qualitatively the shape of this histogram? Note that the two peaks seem to coincide with the values of X where the derivative of X equals 0. 11

12 Problems 1. Each bag in a large box contains 25 tulip bulbs. It is known that 60% of the bags contain bulbs for 5 red and 20 yellow tulips, while the remaining 40% of the bags contain bulbs for 15 red and 10 yellow tulips. A bag is selected at random and a bulb taken at random from this bag is planted. (a) What is the probability that it will be a yellow tulip? (b) Given that it is yellow, what is the conditional probability it comes from a bag that contained 5 red and 20 yellow bulbs? Solve this problem by hand. (That is, not using computer simulation.) Solution. Part (a). Let Y be the event that the bulb selected will be a yellow tulip, and R the event that it will be red. Let B 1 the event that the bulb is taken from the first kind of bag and B 2 that it was taken from the second kind. From the data given we can say: Then the probability of Y can be obtained using P(Y B 1 ) = 20 25, P(Y B 2) = 10 25, P(B 1) = 0.6, P(B 2 ) = 0.4. P(Y ) = P(Y B 1 )P(B 1 ) + P(Y B 2 )P(B 2 ). Therefore, P(Y ) = = Part (b). The problem is to find P(B 1 Y ). Here we need to invert a conditional probability, which calls for Bayes theorem. P(B 1 Y ) = P(Y B 1)P(B 1 ) P(Y ) where we have used the value of P(Y ) obtained in part (a). = = Confirm your solution to part (a) of the previous problem by doing a computer simulation. In other words, simulate the experiment of selecting a bag at random with the given probabilities, then choosing a bulb at random from that bag. This experiment should be repeated a large number of times from which you compute the fraction of times bulbs for yellow tulips were selected. This fraction approximates the probability P(Y ). What approximate value do you obtain? (Make your approximate answer close to the exact one to at least two digits after the decimal point but choosing a sufficiently large number of trials of the experiment.) Solution. The following program does it. The number of times the experiment is repeated (N = ) is somewhat arbitrary. It is big enough to give a precise solution but not so big that I have to wait a long time to get the answer. The problem of how big is sufficient for some desired precision is a statistical problem that we will (in principle, even if we may not have the time to get to it) be able to answer using the methods of this course. N= #Number of times experiment is repeated Y=0 #Initializing the count of yellow tulips. for (j in 1:N){ bag=sample(c(1,2),1,replace=true,c(0.6,0.4)) if (bag==1){ #Here we pick a bulb from bag 1. 12

13 #a==1 means yellow, 2 means red. a=sample(c(1,2),1,replace=true,c(20/25,5/25)) Y=Y+(a==1) else { #Here we pick a bulb from bag 2. Again, #a==1 means yellow, 2 means red. a=sample(c(1,2),1,replace=true,c(10/25,15/25)) Y=Y+(a==1) #The fraction of yellow tulips is Y/N The value I obtained in one run of this problem is This is in good agreement with the exact answer A chemist wishes to detect an impurity in a certain compound that she is making. There is a test that detects an impurity with probability 0.90; however, this test indicates that an impurity is there when it is not about 5% of the time. The chemist produces compounds with the impurity about 20% of the time. A compound is selected at random from the chemist s output. The test indicates that an impurity is present. What is the conditional probability that the compound actually has the impurity? (a) Find the exact solution for the conditional probability. (b) Obtain the solution by simulation, as in the above example of the clinical test. Your solution should be correct to at least two decimal places. Solution. Part (a). This is a standard application of Bayes theorem. Let D be the event that the test detects the impurity, and I the event that impurity is present in the compound. The event that impurity is not present will be denoted by I c (the complementary event). From the problem s data we have P(D I ) = 0.9, P(D I c ) = 0.05, P(I ) = 0.2. We wish to find P(I D), the probability that impurity is present given that the test was positive. This probability of a positive test is P(D) = P(D I )P(I ) + P(D I c )P(I c ) = = We can now use Bayes formula: P(D I )P(I ) P(I D) = = P(D) 0.22 Part (b). The program I used is a simple modification of the one for the clinical test example. N = #Number of trial compounds pd = 0.9 #Probability that test detects impurity pfp = 0.05 #Probability of a false positive pci = 0.2 #Probability a compound has the impurity #Each compound is "assigned" a value 1 (has the impurity) or 0 (does not 13

14 #have it). So C[j]=1 if compound j has the impurity and 0 otherwise. C = sample(c(0,1),n,replace=true,c(1-pci,pci)) #Initialize the vector T of test results: T[j]=1 if test is positive #for compound j and 0 if negative. T = 0*c(1:N) #Arbitrarily giving test result value 0 to all compounds. #Now compound j is tested (this will assign T[j] with a test result value): for (j in 1:N){ if (C[j]==1) { #If compound j has the impurity T[j]=sample(c(0,1),1,replace=TRUE,c(1-pD,pD)) else { T[j]=sample(c(0,1),1,replace=TRUE,c(1-pFP,pFP)) #Now create a vector TD whose jth entry is 1 if jth person both has the #disease and was tested positive, and 0 otherwise. Note that this vector #is simply the elementwise product of T and D: TC = T*C f = sum(tc)/sum(t) #This fraction approximates the condition probability we want. f One run gave me the value This is certainly in the ballpark of what was obtained by hand. I also tried N = (10 times bigger number of trials) and obtained the value , which gives the correct value to 4 decimal places. 4. Histogram. A random variable X : [0,1] {1,2,3,4,5 on the Lebesgue probability space ([0,1],B,λ) has the following expression: X (x) = 1 [0,1/5] (x) + 21 (1/5,1/4] (x) + 31 (1/4,1/3] (x) + 41 (1/3,1/2] (x) + 51 (1/2,1] (x). (Recall that 1 I (x) is my notation for the indicator function of a set I [0,1]. So, for example, X (0.8) = 5.) (a) Draw the graph of the function X using R along the lines of the example given in the tutorial. (b) What is the value of the probability P X ({1,3,5)? (This is to be done by hand.) (c) Using R obtain a histogram plot for 1000 independent sample values of X. Solution. Part (a). Here is the script for part (a): #I first define the indicator functions for the intervals Ind1=function(x) x>0 & x<= 1/5 Ind2=function(x) x>1/5 & x<= 1/4 Ind3=function(x) x>1/4 & x<= 1/3 Ind4=function(x) x>1/3 & x<= 1/2 Ind5=function(x) x>1/2 & x<= 1 #The function X is a linear combination of the indicator functions X=function(x) Ind1(x)+2*Ind2(x)+3*Ind3(x)+4*Ind4(x)+5*Ind5(x) #Now, plotting the graph of X x=seq(from=0,to=1,by=1/1000) 14

15 plot(x,x(x),type= l ) grid() #The plot is shown below. X(x) x Part (b). The probability P X of the set {1,3,5 is the sum of the lengths of the intervals [0,1/5], (1/4,1/3], and (1/2,1]. This sum is P X ({1,3,5) = ( ) ( ) = Part (c). To obtain independent values of the random variable X, I will use the function X from part (a). x=runif(1000) Xvalues=X(x) hist(xvalues,breaks=c(0.5,1.5,2.5,3.5,4.5,5.5),prob=true) 15

16 Histogram of Xvalues Density Xvalues 5. Density plots. Produce density plots using the histogram operation in R (as in the above example) for the following random variables defined by functions on [0,1]. (a) X (x) = log(x) (b) X (x) = 0.5 x 0.5 Note: in R the absolute value function is abs. For example, abs(-2) gives 2. Solution. Part (a). X=function(x) -log(x) x=runif( ) Xvalues=X(x) hist(xvalues,breaks=50,prob=true) #The plot is below. Histogram of Xvalues Density Xvalues 16

17 Part (b). X=function(x) 0.5-abs(x-0.5) x=runif( ) Xvalues=X(x) hist(xvalues,breaks=50,prob=true) #The plot is below. Histogram of Xvalues Density Xvalues 17

4. Conditional Probability

4. Conditional Probability 1 of 13 7/15/2009 9:25 PM Virtual Laboratories > 2. Probability Spaces > 1 2 3 4 5 6 7 4. Conditional Probability Definitions and Interpretations The Basic Definition As usual, we start with a random experiment

More information

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten

More information

LECTURE 1. 1 Introduction. 1.1 Sample spaces and events

LECTURE 1. 1 Introduction. 1.1 Sample spaces and events LECTURE 1 1 Introduction The first part of our adventure is a highly selective review of probability theory, focusing especially on things that are most useful in statistics. 1.1 Sample spaces and events

More information

Homework set 2 - Solutions

Homework set 2 - Solutions Homework set 2 - Solutions Math 495 Renato Feres Simulating a Markov chain in R Generating sample sequences of a finite state Markov chain. The following is a simple program for generating sample sequences

More information

Randomized Algorithms

Randomized Algorithms Randomized Algorithms Prof. Tapio Elomaa tapio.elomaa@tut.fi Course Basics A new 4 credit unit course Part of Theoretical Computer Science courses at the Department of Mathematics There will be 4 hours

More information

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 10

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 10 EECS 70 Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 10 Introduction to Basic Discrete Probability In the last note we considered the probabilistic experiment where we flipped

More information

2.4. Conditional Probability

2.4. Conditional Probability 2.4. Conditional Probability Objectives. Definition of conditional probability and multiplication rule Total probability Bayes Theorem Example 2.4.1. (#46 p.80 textbook) Suppose an individual is randomly

More information

Statistical Inference

Statistical Inference Statistical Inference Lecture 1: Probability Theory MING GAO DASE @ ECNU (for course related communications) mgao@dase.ecnu.edu.cn Sep. 11, 2018 Outline Introduction Set Theory Basics of Probability Theory

More information

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section

More information

1. Discrete Distributions

1. Discrete Distributions Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 1. Discrete Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space Ω.

More information

Sample Space: Specify all possible outcomes from an experiment. Event: Specify a particular outcome or combination of outcomes.

Sample Space: Specify all possible outcomes from an experiment. Event: Specify a particular outcome or combination of outcomes. Chapter 2 Introduction to Probability 2.1 Probability Model Probability concerns about the chance of observing certain outcome resulting from an experiment. However, since chance is an abstraction of something

More information

STAT:5100 (22S:193) Statistical Inference I

STAT:5100 (22S:193) Statistical Inference I STAT:5100 (22S:193) Statistical Inference I Week 3 Luke Tierney University of Iowa Fall 2015 Luke Tierney (U Iowa) STAT:5100 (22S:193) Statistical Inference I Fall 2015 1 Recap Matching problem Generalized

More information

MATH MW Elementary Probability Course Notes Part I: Models and Counting

MATH MW Elementary Probability Course Notes Part I: Models and Counting MATH 2030 3.00MW Elementary Probability Course Notes Part I: Models and Counting Tom Salisbury salt@yorku.ca York University Winter 2010 Introduction [Jan 5] Probability: the mathematics used for Statistics

More information

Notes Week 2 Chapter 3 Probability WEEK 2 page 1

Notes Week 2 Chapter 3 Probability WEEK 2 page 1 Notes Week 2 Chapter 3 Probability WEEK 2 page 1 The sample space of an experiment, sometimes denoted S or in probability theory, is the set that consists of all possible elementary outcomes of that experiment

More information

Probability. Lecture Notes. Adolfo J. Rumbos

Probability. Lecture Notes. Adolfo J. Rumbos Probability Lecture Notes Adolfo J. Rumbos October 20, 204 2 Contents Introduction 5. An example from statistical inference................ 5 2 Probability Spaces 9 2. Sample Spaces and σ fields.....................

More information

Math 416 Lecture 3. The average or mean or expected value of x 1, x 2, x 3,..., x n is

Math 416 Lecture 3. The average or mean or expected value of x 1, x 2, x 3,..., x n is Math 416 Lecture 3 Expected values The average or mean or expected value of x 1, x 2, x 3,..., x n is x 1 x 2... x n n x 1 1 n x 2 1 n... x n 1 n 1 n x i p x i where p x i 1 n is the probability of x i

More information

Lecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability

Lecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability Lecture Notes 1 Basic Probability Set Theory Elements of Probability Conditional probability Sequential Calculation of Probability Total Probability and Bayes Rule Independence Counting EE 178/278A: Basic

More information

1. When applied to an affected person, the test comes up positive in 90% of cases, and negative in 10% (these are called false negatives ).

1. When applied to an affected person, the test comes up positive in 90% of cases, and negative in 10% (these are called false negatives ). CS 70 Discrete Mathematics for CS Spring 2006 Vazirani Lecture 8 Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According to clinical trials,

More information

2011 Pearson Education, Inc

2011 Pearson Education, Inc Statistics for Business and Economics Chapter 3 Probability Contents 1. Events, Sample Spaces, and Probability 2. Unions and Intersections 3. Complementary Events 4. The Additive Rule and Mutually Exclusive

More information

Math493 - Fall HW 4 Solutions

Math493 - Fall HW 4 Solutions Math493 - Fall 2017 - HW 4 Solutions Renato Feres - Wash. U. Preliminaries We have up to this point ignored a central aspect of the Monte Carlo method: How to estimate errors? Clearly, the larger the sample

More information

Chapter 2 PROBABILITY SAMPLE SPACE

Chapter 2 PROBABILITY SAMPLE SPACE Chapter 2 PROBABILITY Key words: Sample space, sample point, tree diagram, events, complement, union and intersection of an event, mutually exclusive events; Counting techniques: multiplication rule, permutation,

More information

Probability Theory and Applications

Probability Theory and Applications Probability Theory and Applications Videos of the topics covered in this manual are available at the following links: Lesson 4 Probability I http://faculty.citadel.edu/silver/ba205/online course/lesson

More information

Conditional Probability, Independence and Bayes Theorem Class 3, Jeremy Orloff and Jonathan Bloom

Conditional Probability, Independence and Bayes Theorem Class 3, Jeremy Orloff and Jonathan Bloom Conditional Probability, Independence and Bayes Theorem Class 3, 18.05 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of conditional probability and independence of events. 2.

More information

P (E) = P (A 1 )P (A 2 )... P (A n ).

P (E) = P (A 1 )P (A 2 )... P (A n ). Lecture 9: Conditional probability II: breaking complex events into smaller events, methods to solve probability problems, Bayes rule, law of total probability, Bayes theorem Discrete Structures II (Summer

More information

7.1 What is it and why should we care?

7.1 What is it and why should we care? Chapter 7 Probability In this section, we go over some simple concepts from probability theory. We integrate these with ideas from formal language theory in the next chapter. 7.1 What is it and why should

More information

Sample Spaces, Random Variables

Sample Spaces, Random Variables Sample Spaces, Random Variables Moulinath Banerjee University of Michigan August 3, 22 Probabilities In talking about probabilities, the fundamental object is Ω, the sample space. (elements) in Ω are denoted

More information

ELEG 3143 Probability & Stochastic Process Ch. 1 Probability

ELEG 3143 Probability & Stochastic Process Ch. 1 Probability Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 1 Probability Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Applications Elementary Set Theory Random

More information

Lecture 1: Probability Fundamentals

Lecture 1: Probability Fundamentals Lecture 1: Probability Fundamentals IB Paper 7: Probability and Statistics Carl Edward Rasmussen Department of Engineering, University of Cambridge January 22nd, 2008 Rasmussen (CUED) Lecture 1: Probability

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE Most of the material in this lecture is covered in [Bertsekas & Tsitsiklis] Sections 1.3-1.5

More information

Lecture 8: Probability

Lecture 8: Probability Lecture 8: Probability The idea of probability is well-known The flipping of a balanced coin can produce one of two outcomes: T (tail) and H (head) and the symmetry between the two outcomes means, of course,

More information

Chapter 2 Random Variables

Chapter 2 Random Variables Stochastic Processes Chapter 2 Random Variables Prof. Jernan Juang Dept. of Engineering Science National Cheng Kung University Prof. Chun-Hung Liu Dept. of Electrical and Computer Eng. National Chiao Tung

More information

2. AXIOMATIC PROBABILITY

2. AXIOMATIC PROBABILITY IA Probability Lent Term 2. AXIOMATIC PROBABILITY 2. The axioms The formulation for classical probability in which all outcomes or points in the sample space are equally likely is too restrictive to develop

More information

CIS 2033 Lecture 5, Fall

CIS 2033 Lecture 5, Fall CIS 2033 Lecture 5, Fall 2016 1 Instructor: David Dobor September 13, 2016 1 Supplemental reading from Dekking s textbook: Chapter2, 3. We mentioned at the beginning of this class that calculus was a prerequisite

More information

POISSON PROCESSES 1. THE LAW OF SMALL NUMBERS

POISSON PROCESSES 1. THE LAW OF SMALL NUMBERS POISSON PROCESSES 1. THE LAW OF SMALL NUMBERS 1.1. The Rutherford-Chadwick-Ellis Experiment. About 90 years ago Ernest Rutherford and his collaborators at the Cavendish Laboratory in Cambridge conducted

More information

Single Maths B: Introduction to Probability

Single Maths B: Introduction to Probability Single Maths B: Introduction to Probability Overview Lecturer Email Office Homework Webpage Dr Jonathan Cumming j.a.cumming@durham.ac.uk CM233 None! http://maths.dur.ac.uk/stats/people/jac/singleb/ 1 Introduction

More information

Introduction and basic definitions

Introduction and basic definitions Chapter 1 Introduction and basic definitions 1.1 Sample space, events, elementary probability Exercise 1.1 Prove that P( ) = 0. Solution of Exercise 1.1 : Events S (where S is the sample space) and are

More information

Statistics 1 - Lecture Notes Chapter 1

Statistics 1 - Lecture Notes Chapter 1 Statistics 1 - Lecture Notes Chapter 1 Caio Ibsen Graduate School of Economics - Getulio Vargas Foundation April 28, 2009 We want to establish a formal mathematic theory to work with results of experiments

More information

1 What is the area model for multiplication?

1 What is the area model for multiplication? for multiplication represents a lovely way to view the distribution property the real number exhibit. This property is the link between addition and multiplication. 1 1 What is the area model for multiplication?

More information

Basic counting techniques. Periklis A. Papakonstantinou Rutgers Business School

Basic counting techniques. Periklis A. Papakonstantinou Rutgers Business School Basic counting techniques Periklis A. Papakonstantinou Rutgers Business School i LECTURE NOTES IN Elementary counting methods Periklis A. Papakonstantinou MSIS, Rutgers Business School ALL RIGHTS RESERVED

More information

Discrete Probability. Chemistry & Physics. Medicine

Discrete Probability. Chemistry & Physics. Medicine Discrete Probability The existence of gambling for many centuries is evidence of long-running interest in probability. But a good understanding of probability transcends mere gambling. The mathematics

More information

Naive Bayes classification

Naive Bayes classification Naive Bayes classification Christos Dimitrakakis December 4, 2015 1 Introduction One of the most important methods in machine learning and statistics is that of Bayesian inference. This is the most fundamental

More information

Origins of Probability Theory

Origins of Probability Theory 1 16.584: INTRODUCTION Theory and Tools of Probability required to analyze and design systems subject to uncertain outcomes/unpredictability/randomness. Such systems more generally referred to as Experiments.

More information

Introduction to Probability. Ariel Yadin. Lecture 1. We begin with an example [this is known as Bertrand s paradox]. *** Nov.

Introduction to Probability. Ariel Yadin. Lecture 1. We begin with an example [this is known as Bertrand s paradox]. *** Nov. Introduction to Probability Ariel Yadin Lecture 1 1. Example: Bertrand s Paradox We begin with an example [this is known as Bertrand s paradox]. *** Nov. 1 *** Question 1.1. Consider a circle of radius

More information

Problems from Probability and Statistical Inference (9th ed.) by Hogg, Tanis and Zimmerman.

Problems from Probability and Statistical Inference (9th ed.) by Hogg, Tanis and Zimmerman. Math 224 Fall 2017 Homework 1 Drew Armstrong Problems from Probability and Statistical Inference (9th ed.) by Hogg, Tanis and Zimmerman. Section 1.1, Exercises 4,5,6,7,9,12. Solutions to Book Problems.

More information

CS 361: Probability & Statistics

CS 361: Probability & Statistics February 12, 2018 CS 361: Probability & Statistics Random Variables Monty hall problem Recall the setup, there are 3 doors, behind two of them are indistinguishable goats, behind one is a car. You pick

More information

Compatible probability measures

Compatible probability measures Coin tossing space Think of a coin toss as a random choice from the two element set {0,1}. Thus the set {0,1} n represents the set of possible outcomes of n coin tosses, and Ω := {0,1} N, consisting of

More information

MATH 556: PROBABILITY PRIMER

MATH 556: PROBABILITY PRIMER MATH 6: PROBABILITY PRIMER 1 DEFINITIONS, TERMINOLOGY, NOTATION 1.1 EVENTS AND THE SAMPLE SPACE Definition 1.1 An experiment is a one-off or repeatable process or procedure for which (a there is a well-defined

More information

Probability and Discrete Distributions

Probability and Discrete Distributions AMS 7L LAB #3 Fall, 2007 Objectives: Probability and Discrete Distributions 1. To explore relative frequency and the Law of Large Numbers 2. To practice the basic rules of probability 3. To work with the

More information

HW2 Solutions, for MATH441, STAT461, STAT561, due September 9th

HW2 Solutions, for MATH441, STAT461, STAT561, due September 9th HW2 Solutions, for MATH44, STAT46, STAT56, due September 9th. You flip a coin until you get tails. Describe the sample space. How many points are in the sample space? The sample space consists of sequences

More information

Probability Notes (A) , Fall 2010

Probability Notes (A) , Fall 2010 Probability Notes (A) 18.310, Fall 2010 We are going to be spending around four lectures on probability theory this year. These notes cover approximately the first three lectures on it. Probability theory

More information

Chapter 2 Classical Probability Theories

Chapter 2 Classical Probability Theories Chapter 2 Classical Probability Theories In principle, those who are not interested in mathematical foundations of probability theory might jump directly to Part II. One should just know that, besides

More information

Fundamentals of Probability CE 311S

Fundamentals of Probability CE 311S Fundamentals of Probability CE 311S OUTLINE Review Elementary set theory Probability fundamentals: outcomes, sample spaces, events Outline ELEMENTARY SET THEORY Basic probability concepts can be cast in

More information

Week 2: Probability: Counting, Sets, and Bayes

Week 2: Probability: Counting, Sets, and Bayes Statistical Methods APPM 4570/5570, STAT 4000/5000 21 Probability Introduction to EDA Week 2: Probability: Counting, Sets, and Bayes Random variable Random variable is a measurable quantity whose outcome

More information

MAT 271E Probability and Statistics

MAT 271E Probability and Statistics MAT 7E Probability and Statistics Spring 6 Instructor : Class Meets : Office Hours : Textbook : İlker Bayram EEB 3 ibayram@itu.edu.tr 3.3 6.3, Wednesday EEB 6.., Monday D. B. Bertsekas, J. N. Tsitsiklis,

More information

the time it takes until a radioactive substance undergoes a decay

the time it takes until a radioactive substance undergoes a decay 1 Probabilities 1.1 Experiments with randomness Wewillusethetermexperimentinaverygeneralwaytorefertosomeprocess that produces a random outcome. Examples: (Ask class for some first) Here are some discrete

More information

MA : Introductory Probability

MA : Introductory Probability MA 320-001: Introductory Probability David Murrugarra Department of Mathematics, University of Kentucky http://www.math.uky.edu/~dmu228/ma320/ Spring 2017 David Murrugarra (University of Kentucky) MA 320:

More information

Measure and integration

Measure and integration Chapter 5 Measure and integration In calculus you have learned how to calculate the size of different kinds of sets: the length of a curve, the area of a region or a surface, the volume or mass of a solid.

More information

DS-GA 1003: Machine Learning and Computational Statistics Homework 7: Bayesian Modeling

DS-GA 1003: Machine Learning and Computational Statistics Homework 7: Bayesian Modeling DS-GA 1003: Machine Learning and Computational Statistics Homework 7: Bayesian Modeling Due: Tuesday, May 10, 2016, at 6pm (Submit via NYU Classes) Instructions: Your answers to the questions below, including

More information

Statistical Theory 1

Statistical Theory 1 Statistical Theory 1 Set Theory and Probability Paolo Bautista September 12, 2017 Set Theory We start by defining terms in Set Theory which will be used in the following sections. Definition 1 A set is

More information

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 16. Random Variables: Distribution and Expectation

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 16. Random Variables: Distribution and Expectation CS 70 Discrete Mathematics and Probability Theory Spring 206 Rao and Walrand Note 6 Random Variables: Distribution and Expectation Example: Coin Flips Recall our setup of a probabilistic experiment as

More information

Dept. of Linguistics, Indiana University Fall 2015

Dept. of Linguistics, Indiana University Fall 2015 L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 34 To start out the course, we need to know something about statistics and This is only an introduction; for a fuller understanding, you would

More information

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019 Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial

More information

Probabilistic models

Probabilistic models Probabilistic models Kolmogorov (Andrei Nikolaevich, 1903 1987) put forward an axiomatic system for probability theory. Foundations of the Calculus of Probabilities, published in 1933, immediately became

More information

PLEASE MARK YOUR ANSWERS WITH AN X, not a circle! 1. (a) (b) (c) (d) (e) 2. (a) (b) (c) (d) (e) (a) (b) (c) (d) (e) 4. (a) (b) (c) (d) (e)...

PLEASE MARK YOUR ANSWERS WITH AN X, not a circle! 1. (a) (b) (c) (d) (e) 2. (a) (b) (c) (d) (e) (a) (b) (c) (d) (e) 4. (a) (b) (c) (d) (e)... Math 020, Exam II October, 206 The Honor Code is in effect for this examination. All work is to be your own. You may use a calculator. The exam lasts for hour 5 minutes. Be sure that your name is on every

More information

Lecture 16. Lectures 1-15 Review

Lecture 16. Lectures 1-15 Review 18.440: Lecture 16 Lectures 1-15 Review Scott Sheffield MIT 1 Outline Counting tricks and basic principles of probability Discrete random variables 2 Outline Counting tricks and basic principles of probability

More information

Lecture 10: Bayes' Theorem, Expected Value and Variance Lecturer: Lale Özkahya

Lecture 10: Bayes' Theorem, Expected Value and Variance Lecturer: Lale Özkahya BBM 205 Discrete Mathematics Hacettepe University http://web.cs.hacettepe.edu.tr/ bbm205 Lecture 10: Bayes' Theorem, Expected Value and Variance Lecturer: Lale Özkahya Resources: Kenneth Rosen, Discrete

More information

STAT 516: Basic Probability and its Applications

STAT 516: Basic Probability and its Applications Lecture 3: Conditional Probability and Independence Prof. Michael September 29, 2015 Motivating Example Experiment ξ consists of rolling a fair die twice; A = { the first roll is 6 } amd B = { the sum

More information

Module 1. Probability

Module 1. Probability Module 1 Probability 1. Introduction In our daily life we come across many processes whose nature cannot be predicted in advance. Such processes are referred to as random processes. The only way to derive

More information

MAT 271E Probability and Statistics

MAT 271E Probability and Statistics MAT 71E Probability and Statistics Spring 013 Instructor : Class Meets : Office Hours : Textbook : Supp. Text : İlker Bayram EEB 1103 ibayram@itu.edu.tr 13.30 1.30, Wednesday EEB 5303 10.00 1.00, Wednesday

More information

Probability Distributions

Probability Distributions CONDENSED LESSON 13.1 Probability Distributions In this lesson, you Sketch the graph of the probability distribution for a continuous random variable Find probabilities by finding or approximating areas

More information

Writing proofs for MATH 51H Section 2: Set theory, proofs of existential statements, proofs of uniqueness statements, proof by cases

Writing proofs for MATH 51H Section 2: Set theory, proofs of existential statements, proofs of uniqueness statements, proof by cases Writing proofs for MATH 51H Section 2: Set theory, proofs of existential statements, proofs of uniqueness statements, proof by cases September 22, 2018 Recall from last week that the purpose of a proof

More information

Chapter 2 Class Notes

Chapter 2 Class Notes Chapter 2 Class Notes Probability can be thought of in many ways, for example as a relative frequency of a long series of trials (e.g. flips of a coin or die) Another approach is to let an expert (such

More information

Probability Theory. Introduction to Probability Theory. Principles of Counting Examples. Principles of Counting. Probability spaces.

Probability Theory. Introduction to Probability Theory. Principles of Counting Examples. Principles of Counting. Probability spaces. Probability Theory To start out the course, we need to know something about statistics and probability Introduction to Probability Theory L645 Advanced NLP Autumn 2009 This is only an introduction; for

More information

Be able to define the following terms and answer basic questions about them:

Be able to define the following terms and answer basic questions about them: CS440/ECE448 Section Q Fall 2017 Final Review Be able to define the following terms and answer basic questions about them: Probability o Random variables, axioms of probability o Joint, marginal, conditional

More information

Construction of a general measure structure

Construction of a general measure structure Chapter 4 Construction of a general measure structure We turn to the development of general measure theory. The ingredients are a set describing the universe of points, a class of measurable subsets along

More information

Conditional Probability

Conditional Probability Conditional Probability Idea have performed a chance experiment but don t know the outcome (ω), but have some partial information (event A) about ω. Question: given this partial information what s the

More information

Notes 1 Autumn Sample space, events. S is the number of elements in the set S.)

Notes 1 Autumn Sample space, events. S is the number of elements in the set S.) MAS 108 Probability I Notes 1 Autumn 2005 Sample space, events The general setting is: We perform an experiment which can have a number of different outcomes. The sample space is the set of all possible

More information

STAT509: Probability

STAT509: Probability University of South Carolina August 20, 2014 The Engineering Method and Statistical Thinking The general steps of engineering method are: 1. Develop a clear and concise description of the problem. 2. Identify

More information

With Question/Answer Animations. Chapter 7

With Question/Answer Animations. Chapter 7 With Question/Answer Animations Chapter 7 Chapter Summary Introduction to Discrete Probability Probability Theory Bayes Theorem Section 7.1 Section Summary Finite Probability Probabilities of Complements

More information

If the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r).

If the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r). Caveat: Not proof read. Corrections appreciated. Combinatorics In the following, n, n 1, r, etc. will denote non-negative integers. Rule 1 The number of ways of ordering n distinguishable objects (also

More information

Probability- describes the pattern of chance outcomes

Probability- describes the pattern of chance outcomes Chapter 6 Probability the study of randomness Probability- describes the pattern of chance outcomes Chance behavior is unpredictable in the short run, but has a regular and predictable pattern in the long

More information

Discrete Random Variable

Discrete Random Variable Discrete Random Variable Outcome of a random experiment need not to be a number. We are generally interested in some measurement or numerical attribute of the outcome, rather than the outcome itself. n

More information

Continuum Probability and Sets of Measure Zero

Continuum Probability and Sets of Measure Zero Chapter 3 Continuum Probability and Sets of Measure Zero In this chapter, we provide a motivation for using measure theory as a foundation for probability. It uses the example of random coin tossing to

More information

Chapter 2. Conditional Probability and Independence. 2.1 Conditional Probability

Chapter 2. Conditional Probability and Independence. 2.1 Conditional Probability Chapter 2 Conditional Probability and Independence 2.1 Conditional Probability Example: Two dice are tossed. What is the probability that the sum is 8? This is an easy exercise: we have a sample space

More information

Probability COMP 245 STATISTICS. Dr N A Heard. 1 Sample Spaces and Events Sample Spaces Events Combinations of Events...

Probability COMP 245 STATISTICS. Dr N A Heard. 1 Sample Spaces and Events Sample Spaces Events Combinations of Events... Probability COMP 245 STATISTICS Dr N A Heard Contents Sample Spaces and Events. Sample Spaces........................................2 Events........................................... 2.3 Combinations

More information

Probability and distributions. Francesco Corona

Probability and distributions. Francesco Corona Probability Probability and distributions Francesco Corona Department of Computer Science Federal University of Ceará, Fortaleza Probability Many kinds of studies can be characterised as (repeated) experiments

More information

Math 511 Exam #1. Show All Work No Calculators

Math 511 Exam #1. Show All Work No Calculators Math 511 Exam #1 Show All Work No Calculators 1. Suppose that A and B are events in a sample space S and that P(A) = 0.4 and P(B) = 0.6 and P(A B) = 0.3. Suppose too that B, C, and D are mutually independent

More information

Notes on the Point-Set Topology of R Northwestern University, Fall 2014

Notes on the Point-Set Topology of R Northwestern University, Fall 2014 Notes on the Point-Set Topology of R Northwestern University, Fall 2014 These notes give an introduction to the notions of open and closed subsets of R, which belong to the subject known as point-set topology.

More information

I - Probability. What is Probability? the chance of an event occuring. 1classical probability. 2empirical probability. 3subjective probability

I - Probability. What is Probability? the chance of an event occuring. 1classical probability. 2empirical probability. 3subjective probability What is Probability? the chance of an event occuring eg 1classical probability 2empirical probability 3subjective probability Section 2 - Probability (1) Probability - Terminology random (probability)

More information

Event A: at least one tail observed A:

Event A: at least one tail observed A: Chapter 3 Probability 3.1 Events, sample space, and probability Basic definitions: An is an act of observation that leads to a single outcome that cannot be predicted with certainty. A (or simple event)

More information

tossing a coin selecting a card from a deck measuring the commuting time on a particular morning

tossing a coin selecting a card from a deck measuring the commuting time on a particular morning 2 Probability Experiment An experiment or random variable is any activity whose outcome is unknown or random upfront: tossing a coin selecting a card from a deck measuring the commuting time on a particular

More information

1. Exercise on Page 8: 2. Exercise on Page 8: (b)c 1 (C 2 C 3 ) and (C 1 C 2 ) (C 1 C 3 ) (C 1 C 2 ) (C 2 C 3 ) Problem Set 1 Fall 2017

1. Exercise on Page 8: 2. Exercise on Page 8: (b)c 1 (C 2 C 3 ) and (C 1 C 2 ) (C 1 C 3 ) (C 1 C 2 ) (C 2 C 3 ) Problem Set 1 Fall 2017 . Exercise.2. on Page 8: Find the union C C 2 and the intersection C C 2 of the two sets C and C 2, where (a) C = {0,, 2}, C 2 = {2, 3, 4} Answer(a): C C 2 = {0,, 2, 3, 4}, C C 2 = {2} (b) C = {x : 0

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Lecture 3: Probability, Bayes Theorem, and Bayes Classification Peter Belhumeur Computer Science Columbia University Probability Should you play this game? Game: A fair

More information

Statistical methods in recognition. Why is classification a problem?

Statistical methods in recognition. Why is classification a problem? Statistical methods in recognition Basic steps in classifier design collect training images choose a classification model estimate parameters of classification model from training images evaluate model

More information

Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides

Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides slide 1 Inference with Bayes rule: Example In a bag there are two envelopes one has a red ball (worth $100) and a black ball one

More information

Introduction to Stochastic Processes

Introduction to Stochastic Processes Stat251/551 (Spring 2017) Stochastic Processes Lecture: 1 Introduction to Stochastic Processes Lecturer: Sahand Negahban Scribe: Sahand Negahban 1 Organization Issues We will use canvas as the course webpage.

More information

Announcements. Topics: To Do:

Announcements. Topics: To Do: Announcements Topics: In the Probability and Statistics module: - Sections 1 + 2: Introduction to Stochastic Models - Section 3: Basics of Probability Theory - Section 4: Conditional Probability; Law of

More information

HW MATH425/525 Lecture Notes 1

HW MATH425/525 Lecture Notes 1 HW MATH425/525 Lecture Notes 1 Definition 4.1 If an experiment can be repeated under the same condition, its outcome cannot be predicted with certainty, and the collection of its every possible outcome

More information

Outline Conditional Probability The Law of Total Probability and Bayes Theorem Independent Events. Week 4 Classical Probability, Part II

Outline Conditional Probability The Law of Total Probability and Bayes Theorem Independent Events. Week 4 Classical Probability, Part II Week 4 Classical Probability, Part II Week 4 Objectives This week we continue covering topics from classical probability. The notion of conditional probability is presented first. Important results/tools

More information

Computational Cognitive Science

Computational Cognitive Science Computational Cognitive Science Lecture 9: A Bayesian model of concept learning Chris Lucas School of Informatics University of Edinburgh October 16, 218 Reading Rules and Similarity in Concept Learning

More information