Special Topic: Bayesian Finite Population Survey Sampling

Transcription

1 Special Topic: Bayesian Finite Population Survey Sampling Sudipto Banerjee Division of Biostatistics School of Public Health University of Minnesota April 2, Special Topic

2 Overview Scientific survey sampling (Neyman, 1934) represents one of statistics greatest contribution to science 2 Special Topic

3 Overview Scientific survey sampling (Neyman, 1934) represents one of statistics greatest contribution to science It provides the remarkable ability to obtain useful inferences about large populations from modest samples, with measurable uncertainty 2 Special Topic

4 Overview Scientific survey sampling (Neyman, 1934) represents one of statistics greatest contribution to science It provides the remarkable ability to obtain useful inferences about large populations from modest samples, with measurable uncertainty It forms the backbone of data collection in science and government 2 Special Topic

5 Overview Scientific survey sampling (Neyman, 1934) represents one of statistics greatest contribution to science It provides the remarkable ability to obtain useful inferences about large populations from modest samples, with measurable uncertainty It forms the backbone of data collection in science and government Traditional (and widely favoured) approach: Randomization-based 2 Special Topic

6 Overview Scientific survey sampling (Neyman, 1934) represents one of statistics greatest contribution to science It provides the remarkable ability to obtain useful inferences about large populations from modest samples, with measurable uncertainty It forms the backbone of data collection in science and government Traditional (and widely favoured) approach: Randomization-based Advocating Bayes for survey sampling is like swimming upstream : Modelling assumptions of any kind are anathema here, let alone priors and further subjectivity that Bayes brings along! 2 Special Topic

7 Fundamental Notions For a population with N units, let Y = (Y 1,..., Y N ) denote the population variable 3 Special Topic

8 Fundamental Notions For a population with N units, let Y = (Y 1,..., Y N ) denote the population variable Y i is the value of unit i in the population 3 Special Topic

9 Fundamental Notions For a population with N units, let Y = (Y 1,..., Y N ) denote the population variable Y i is the value of unit i in the population Let I = (I 1,..., I N ) be the set of inclusion indicator variables 3 Special Topic

10 Fundamental Notions For a population with N units, let Y = (Y 1,..., Y N ) denote the population variable Y i is the value of unit i in the population Let I = (I 1,..., I N ) be the set of inclusion indicator variables I i = 1 if unit i is selected in the sample; I i = 0 otherwise 3 Special Topic

11 Fundamental Notions For a population with N units, let Y = (Y 1,..., Y N ) denote the population variable Y i is the value of unit i in the population Let I = (I 1,..., I N ) be the set of inclusion indicator variables I i = 1 if unit i is selected in the sample; I i = 0 otherwise Let I denote the complement of I, obtained by switching the 1 s to 0 s and 0 s to 1 s in I 3 Special Topic

12 Fundamental Notions For a population with N units, let Y = (Y 1,..., Y N ) denote the population variable Y i is the value of unit i in the population Let I = (I 1,..., I N ) be the set of inclusion indicator variables I i = 1 if unit i is selected in the sample; I i = 0 otherwise Let I denote the complement of I, obtained by switching the 1 s to 0 s and 0 s to 1 s in I Thus, I represents the exclusion indicator variables, indexing units not selected in the sample. 3 Special Topic

13 Simple random sampling Simple Random Sampling With Replacement (SRSWR): We draw a unit from the population, put it back and draw again. 4 Special Topic

14 Simple random sampling Simple Random Sampling With Replacement (SRSWR): We draw a unit from the population, put it back and draw again. Simple Random Sampling Without Replacement (SRSWOR): We draw a unit from the population, keep it aside and draw again 4 Special Topic

15 Simple random sampling Simple Random Sampling With Replacement (SRSWR): We draw a unit from the population, put it back and draw again. Simple Random Sampling Without Replacement (SRSWOR): We draw a unit from the population, keep it aside and draw again SRSWR leads to binomial distribution; chances of units being drawn remain the same between draws 4 Special Topic

16 Simple random sampling Simple Random Sampling With Replacement (SRSWR): We draw a unit from the population, put it back and draw again. Simple Random Sampling Without Replacement (SRSWOR): We draw a unit from the population, keep it aside and draw again SRSWR leads to binomial distribution; chances of units being drawn remain the same between draws SRSWOR leads to hypergeometric distribution: chances of units being drawn change from draw to draw 4 Special Topic

17 Simple random sampling Simple Random Sampling With Replacement (SRSWR): We draw a unit from the population, put it back and draw again. Simple Random Sampling Without Replacement (SRSWOR): We draw a unit from the population, keep it aside and draw again SRSWR leads to binomial distribution; chances of units being drawn remain the same between draws SRSWOR leads to hypergeometric distribution: chances of units being drawn change from draw to draw Key questions: What are the random variables? 4 Special Topic

18 Simple random sampling Simple Random Sampling With Replacement (SRSWR): We draw a unit from the population, put it back and draw again. Simple Random Sampling Without Replacement (SRSWOR): We draw a unit from the population, keep it aside and draw again SRSWR leads to binomial distribution; chances of units being drawn remain the same between draws SRSWOR leads to hypergeometric distribution: chances of units being drawn change from draw to draw Key questions: What are the random variables? What are the parameters? 4 Special Topic

19 Estimating the population mean Let Y I be the collection of units selected in the sample; we often write Y I as y = (y 1,..., y n ). 5 Special Topic

20 Estimating the population mean Let Y I be the collection of units selected in the sample; we often write Y I as y = (y 1,..., y n ). IMPORTANT: The lower-case y i s simply index the sample; thus y i does not necessarily represent Y i, but the i-th member of the sample. 5 Special Topic

21 Estimating the population mean Let Y I be the collection of units selected in the sample; we often write Y I as y = (y 1,..., y n ). IMPORTANT: The lower-case y i s simply index the sample; thus y i does not necessarily represent Y i, but the i-th member of the sample. Let Ȳ = 1 N N i=1 Y i be the population mean 5 Special Topic

22 Estimating the population mean Let Y I be the collection of units selected in the sample; we often write Y I as y = (y 1,..., y n ). IMPORTANT: The lower-case y i s simply index the sample; thus y i does not necessarily represent Y i, but the i-th member of the sample. Let Ȳ = 1 N N i=1 Y i be the population mean Let ȲI = ȳ = 1 n n i=1 y i be the sample mean; i.e. the mean of Y I 5 Special Topic

23 Estimating the population mean Let Y I be the collection of units selected in the sample; we often write Y I as y = (y 1,..., y n ). IMPORTANT: The lower-case y i s simply index the sample; thus y i does not necessarily represent Y i, but the i-th member of the sample. Let Ȳ = 1 N N i=1 Y i be the population mean Let ȲI = ȳ = 1 n n i=1 y i be the sample mean; i.e. the mean of Y I Let Y I be the collection of units excluded from the sample; let ȲI be the mean of Y I. 5 Special Topic

24 Estimating the population mean Let Y I be the collection of units selected in the sample; we often write Y I as y = (y 1,..., y n ). IMPORTANT: The lower-case y i s simply index the sample; thus y i does not necessarily represent Y i, but the i-th member of the sample. Let Ȳ = 1 N N i=1 Y i be the population mean Let ȲI = ȳ = 1 n n i=1 y i be the sample mean; i.e. the mean of Y I Let Y I be the collection of units excluded from the sample; let ȲI be the mean of Y I. Then Ȳ = fȳ + (1 f)ȳi ; f = n N is the sampling fraction 5 Special Topic

25 Estimating the population mean Key questions answered: What are the random variables? Answer: The I i s What are the parameters? Answer: The Y i s. 6 Special Topic

26 Estimating the population mean Key questions answered: What are the random variables? Answer: The I i s What are the parameters? Answer: The Y i s. For SRSWOR: E[I i ] = P (I i = 1) = (N 1 n 1) = ( N n) n N for each i. 6 Special Topic

27 Estimating the population mean Key questions answered: What are the random variables? Answer: The I i s What are the parameters? Answer: The Y i s. For SRSWOR: E[I i ] = P (I i = 1) = (N 1 n 1) = ( N n) n N for each i. Unbiasedness of the sample mean ȳ: Write ȳ = 1 N n i=1 I iy i and see: E[ȳ] = 1 n N E[I i ]Y i = 1 n i=1 N i=1 n N Y i = Ȳ. 6 Special Topic

28 Estimating the population mean Key questions answered: What are the random variables? Answer: The I i s What are the parameters? Answer: The Y i s. For SRSWOR: E[I i ] = P (I i = 1) = (N 1 n 1) = ( N n) n N for each i. Unbiasedness of the sample mean ȳ: Write ȳ = 1 N n i=1 I iy i and see: E[ȳ] = 1 n N E[I i ]Y i = 1 n i=1 N i=1 n N Y i = Ȳ. For SRSWR: ȳ = N i=1 W iy i, where W i is the number of times unit i appears in the sample. Then, W i Bin(n, 1 N ), so E[W i ] = n N, hence E[ȳ] = 1 n N i=1 E[W i]y i = 1 n N i=1 n N Y i = Ȳ 6 Special Topic

29 Bayesian modelling We need a prior for each population unit: Y i 7 Special Topic

30 Bayesian modelling We need a prior for each population unit: Y i Suppose Y i iid N(µ, σ 2 ) 7 Special Topic

31 Bayesian modelling We need a prior for each population unit: Y i Suppose Y i iid N(µ, σ 2 ) Let us first assume σ 2 is known and P (µ) 1. 7 Special Topic

32 Bayesian modelling We need a prior for each population unit: Y i Suppose Y i iid N(µ, σ 2 ) Let us first assume σ 2 is known and P (µ) 1. What is seen? D = (Y I, I) 7 Special Topic

33 Bayesian modelling We need a prior for each population unit: Y i Suppose Y i iid N(µ, σ 2 ) Let us first assume σ 2 is known and P (µ) 1. What is seen? D = (Y I, I) Note that P (D µ, σ 2 ) = P (I)P (Y I σ 2 ) P (Y I σ 2 ); so likelihood is product of n independent N(µ, σ 2 ). 7 Special Topic

34 Bayesian modelling We need a prior for each population unit: Y i Suppose Y i iid N(µ, σ 2 ) Let us first assume σ 2 is known and P (µ) 1. What is seen? D = (Y I, I) Note that P (D µ, σ 2 ) = P (I)P (Y I σ 2 ) P (Y I σ 2 ); so likelihood is product of n independent N(µ, σ 2 ). We need the posterior distribution P (Ȳ D, σ2 ) = P (Ȳ D, µ, σ2 )P (µ D, σ 2 )dµ. 7 Special Topic

35 Bayesian modelling Let us take a closer look at P (Ȳ D, σ2 ). 8 Special Topic

36 Bayesian modelling Let us take a closer look at P (Ȳ D, σ2 ). Observe that Ȳ = fȳ + (1 f)ȳi. 8 Special Topic

37 Bayesian modelling Let us take a closer look at P (Ȳ D, σ2 ). Observe that Ȳ = fȳ + (1 f)ȳi. Given D, ȳ is fixed; so the posterior distribution of ȲI will determine the posterior distribution of Ȳ. 8 Special Topic

38 Bayesian modelling Let us take a closer look at P (Ȳ D, σ2 ). Observe that Ȳ = fȳ + (1 f)ȳi. Given D, ȳ is fixed; so the posterior distribution of ȲI will determine the posterior distribution of Ȳ. So, what is the posterior distribution P (ȲI D, σ2 )? 8 Special Topic

39 Bayesian modelling Let us take a closer look at P (Ȳ D, σ2 ). Observe that Ȳ = fȳ + (1 f)ȳi. Given D, ȳ is fixed; so the posterior distribution of ȲI will determine the posterior distribution of Ȳ. So, what is the posterior distribution P (ȲI D, σ2 )? P (ȲI D, σ2 ) = P (ȲI µ, σ2 )P (µ D, σ 2 )dµ, where we use P (ȲI ( D, µ, σ2 ) = P (ȲI µ, σ2 ). Note P (ȲI µ, σ2 σ ) = N µ, 2 N(1 f) 8 Special Topic

40 Bayesian modelling Let us take a closer look at P (Ȳ D, σ2 ). Observe that Ȳ = fȳ + (1 f)ȳi. Given D, ȳ is fixed; so the posterior distribution of ȲI will determine the posterior distribution of Ȳ. So, what is the posterior distribution P (ȲI D, σ2 )? P (ȲI D, σ2 ) = P (ȲI µ, σ2 )P (µ D, σ 2 )dµ, where we use P (ȲI ( D, µ, σ2 ) = P (ȲI µ, σ2 ). Note P (ȲI µ, σ2 σ ) = N µ, 2 N(1 f) Can we simplify this posterior without integrating? 8 Special Topic

41 Bayesian modelling What is the posterior distribution P (µ D, σ 2 )? 9 Special Topic

42 Bayesian modelling What is the posterior distribution P (µ D, σ 2 )? Note that P (µ D, σ 2 ) P (D µ, σ 2 ) 9 Special Topic

43 Bayesian modelling What is the posterior distribution P (µ D, σ 2 )? Note that P (µ D, σ 2 ) P (D µ, σ 2 ) ( = P (µ D, σ 2 ) = N ȳ, σ2 n ). 9 Special Topic

44 Bayesian modelling What is the posterior distribution P (µ D, σ 2 )? Note that P (µ D, σ 2 ) P (D µ, σ 2 ) ( = P (µ D, σ 2 ) = N ȳ, σ2 n ). Conditional on D, µ and σ 2, we can write: Ȳ = fȳ + (1 f)ȳi ; ( σ 2 ) Ȳ I = µ + u 1 ; u 1 N 0, ; N(1 f) Ȳ = fȳ + (1 f)µ + (1 f)u 1 ( Also, conditional on D and σ 2, µ = ȳ + u 2 ; u 2 N 0, σ2 n ). 9 Special Topic

45 Bayesian modelling Then: Ȳ = ȳ + (1 f)u 2 + (1 f)u 1 10 Special Topic

46 Bayesian modelling Then: Ȳ = ȳ + (1 f)u 2 + (1 f)u 1 The posterior is P (Ȳ D, σ2 ) is: N ) (ȳ, σ2 (1 f) n 10 Special Topic

47 Bayesian modelling Then: Ȳ = ȳ + (1 f)u 2 + (1 f)u 1 The posterior is P (Ȳ D, σ2 ) is: N ) (ȳ, σ2 (1 f) n Let σ 2 be unknown. Prior: P (µ, σ 2 ) 1 σ 2 10 Special Topic

48 Bayesian modelling Then: Ȳ = ȳ + (1 f)u 2 + (1 f)u 1 The posterior is P (Ȳ D, σ2 ) is: N ) (ȳ, σ2 (1 f) n Let σ 2 be unknown. Prior: P (µ, σ 2 ) 1 σ 2 Then we reproduce classical result: Ȳ ȳ s 2n (1 f) t n 1 where s 2 = 1 n 1 n i=1 (y i ȳ) 2 is the sample variance. 10 Special Topic

49 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. 11 Special Topic

50 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. We can perform exact posterior sampling as follows. For l = 1,..., L do the following: 11 Special Topic

51 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. We can perform exact posterior sampling as follows. For l = 1,..., L do the following: ( ) Draw σ(l) 2 IG n 1 2, (n 1)s Special Topic

52 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. We can perform exact posterior sampling as follows. For l = 1,..., L do the following: ( ) Draw σ(l) 2 IG n 1 2, (n 1)s2 2 ( ) Draw µ (l) N ȳ, σ2 (l) n 11 Special Topic

53 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. We can perform exact posterior sampling as follows. For l = 1,..., L do the following: ( ) Draw σ(l) 2 IG n 1 2, (n 1)s2 2 ( ) Draw µ (l) N ȳ, σ2 (l) n ) Draw Ȳ (l) I (µ N (l), σ 2 (l) N(1 f) 11 Special Topic

54 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. We can perform exact posterior sampling as follows. For l = 1,..., L do the following: ( ) Draw σ(l) 2 IG n 1 2, (n 1)s2 2 ( ) Draw µ (l) N ȳ, σ2 (l) n ) Draw Ȳ (l) I (µ N (l), σ 2 (l) N(1 f) Set Ȳ (l) = fȳ + (1 f)ȳ (l) I. 11 Special Topic

55 Bayesian computation Note that the marginal ( posterior ) distribution P (σ 2 D) = IG n 1 2, (n 1)s2 2, where s 2 = 1 n n 1 i=1 (y i ȳ) 2 is the sample variance. We can perform exact posterior sampling as follows. For l = 1,..., L do the following: ( ) Draw σ(l) 2 IG n 1 2, (n 1)s2 2 ( ) Draw µ (l) N ȳ, σ2 (l) n ) Draw Ȳ (l) I (µ N (l), σ 2 (l) N(1 f) Set Ȳ (l) = fȳ + (1 f)ȳ (l) I. The resulting collection {Ȳ (l) } L l=1 is a sample from the posterior distribution of the population mean. 11 Special Topic

56 Homework HOMEWORK A simple random sample of 30 households was drawn from a township containing 576 households. The numbers of persons per household in the sample were as follows: 5, 6, 3, 3, 2, 3, 3, 3, 4, 4, 3, 2, 7, 4, 3, 5, 4, 4, 3, 3, 4, 3, 3, 1, 2, 4, 3, 4, 2, 4 Use a non-informative Bayesian analysis to estimate the total number of people in the area and compute the posterior probability that the population total lies within 10% of the sample estimate. From a list of 468 small 2-year colleges in the North-Eastern United States, a simple sample of 100 colleges was drawn. Data for the number of students (y) and the number of teachers (x) for these colleges were summarized as follows: The total number of students in the sample was: 44, 987 The total number of teachers in the sample was: 3, 079 Also given are the sample sums of squares: P n i=1 yi 2 = 29, 881, 219 and P ni=1 x 2 i = 111, 090. Assuming a non-informative Bayesian setting and where the population of students and teachers are independent, find the posterior mean and 95% credible interval for the student-teacher ratio in the population (i.e. all the 468 colleges combined). 12 Special Topic

57 Missing Data Inference A general theory for Bayesian missing data analysis 13 Special Topic

58 Missing Data Inference A general theory for Bayesian missing data analysis Let y = (y 1,..., y N ) be the units in the population. We assume a likelihood for this population, say p(y θ), where θ are the parameters in the model. 13 Special Topic

59 Missing Data Inference A general theory for Bayesian missing data analysis Let y = (y 1,..., y N ) be the units in the population. We assume a likelihood for this population, say p(y θ), where θ are the parameters in the model. Let I = (I 1,..., I N ) be the set of inclusion indicators. 13 Special Topic

60 Missing Data Inference A general theory for Bayesian missing data analysis Let y = (y 1,..., y N ) be the units in the population. We assume a likelihood for this population, say p(y θ), where θ are the parameters in the model. Let I = (I 1,..., I N ) be the set of inclusion indicators. Let us denote by obs the index set {i : I i = 1} and by mis the index set {i; I i = 0} 13 Special Topic

61 Missing Data Inference A general theory for Bayesian missing data analysis Let y = (y 1,..., y N ) be the units in the population. We assume a likelihood for this population, say p(y θ), where θ are the parameters in the model. Let I = (I 1,..., I N ) be the set of inclusion indicators. Let us denote by obs the index set {i : I i = 1} and by mis the index set {i; I i = 0} So, y obs is the set of observed data points and y mis is for the unobserved data. We will often write y = (y obs, y mis ). 13 Special Topic

62 Buliding the models We will break the joint probability model into two parts 14 Special Topic

63 Buliding the models We will break the joint probability model into two parts The model for the complete data p(y θ) including observed and missing data 14 Special Topic

64 Buliding the models We will break the joint probability model into two parts The model for the complete data p(y θ) including observed and missing data The model for the inclusion vector I, say p(i y, φ) 14 Special Topic

65 Buliding the models We will break the joint probability model into two parts The model for the complete data p(y θ) including observed and missing data The model for the inclusion vector I, say p(i y, φ) We define the complete-likelihood as: p(y, I θ, φ) = p(y θ)p(i y, φ) 14 Special Topic

66 Buliding the models We will break the joint probability model into two parts The model for the complete data p(y θ) including observed and missing data The model for the inclusion vector I, say p(i y, φ) We define the complete-likelihood as: p(y, I θ, φ) = p(y θ)p(i y, φ) Parameters: Scientific interest usually revolves around estimation of θ (sometimes called the super-population parameters) 14 Special Topic

67 Buliding the models We will break the joint probability model into two parts The model for the complete data p(y θ) including observed and missing data The model for the inclusion vector I, say p(i y, φ) We define the complete-likelihood as: p(y, I θ, φ) = p(y θ)p(i y, φ) Parameters: Scientific interest usually revolves around estimation of θ (sometimes called the super-population parameters) The parameters φ index the missingness model (often called design parameters). 14 Special Topic

68 Buliding the models The complete-likelihood is not the data likelihood as it involves the missing data component as well. 15 Special Topic

69 Buliding the models The complete-likelihood is not the data likelihood as it involves the missing data component as well. The actual information available is (y obs, I). So the data likelihood is: p(y obs, I θ, φ) = p(y, I θ, φ)dy mis 15 Special Topic

70 Buliding the models The complete-likelihood is not the data likelihood as it involves the missing data component as well. The actual information available is (y obs, I). So the data likelihood is: p(y obs, I θ, φ) = p(y, I θ, φ)dy mis The joint posterior distribution is p(θ, φ y obs, I) p(θ, φ)p(y obs, I θ, φ) p(θ, φ) p(y θ)p(i y, φ)dy mis 15 Special Topic

71 Buliding the models The complete-likelihood is not the data likelihood as it involves the missing data component as well. The actual information available is (y obs, I). So the data likelihood is: p(y obs, I θ, φ) = p(y, I θ, φ)dy mis The joint posterior distribution is p(θ, φ y obs, I) p(θ, φ)p(y obs, I θ, φ) p(θ, φ) p(y θ)p(i y, φ)dy mis 15 Special Topic

72 Buliding the models These integrals are evaluated by drawing samples (y mis, θ, φ) from p(y mis, θ, φ y obs, I) 16 Special Topic

73 Buliding the models These integrals are evaluated by drawing samples (y mis, θ, φ) from p(y mis, θ, φ y obs, I) Typically, we first compute samples (θ (l), φ (l) ) from p(θ, φ y obs, I) 16 Special Topic

74 Buliding the models These integrals are evaluated by drawing samples (y mis, θ, φ) from p(y mis, θ, φ y obs, I) Typically, we first compute samples (θ (l), φ (l) ) from p(θ, φ y obs, I) Then we draw y (l) mis p(y mis y obs, I, θ (l), φ (l) ) 16 Special Topic

75 Buliding the models These integrals are evaluated by drawing samples (y mis, θ, φ) from p(y mis, θ, φ y obs, I) Typically, we first compute samples (θ (l), φ (l) ) from p(θ, φ y obs, I) Then we draw y (l) mis p(y mis y obs, I, θ (l), φ (l) ) Computations often simplify from assumptions: p(i y, φ) = p(i y obs, φ) 16 Special Topic

76 Buliding the models These integrals are evaluated by drawing samples (y mis, θ, φ) from p(y mis, θ, φ y obs, I) Typically, we first compute samples (θ (l), φ (l) ) from p(θ, φ y obs, I) Then we draw y (l) mis p(y mis y obs, I, θ (l), φ (l) ) Computations often simplify from assumptions: p(i y, φ) = p(i y obs, φ) p(i y, φ) = p(i φ) = p(i) 16 Special Topic

77 Buliding the models These integrals are evaluated by drawing samples (y mis, θ, φ) from p(y mis, θ, φ y obs, I) Typically, we first compute samples (θ (l), φ (l) ) from p(θ, φ y obs, I) Then we draw y (l) mis p(y mis y obs, I, θ (l), φ (l) ) Computations often simplify from assumptions: p(i y, φ) = p(i y obs, φ) p(i y, φ) = p(i φ) = p(i) p(φ θ) = p(φ) p(y mis θ (l), y obs ) = p(y mis θ (l) ). 16 Special Topic