Herd Management Science

Transcription

1 Herd Management Science Exercises and Supplementary Reading 2010 Edition Compiled on July 27, 2010 Anders Ringgaard Kristensen University of Copenhagen Faculty of Life Sciences

2 2

3 3 Preface These exercises have been compiled for the Advanced Herd Management course at University of Copenhagen in Several people have contributed to the exercises. Thus, Cécile Cornou is responsible for the exercises related to Chapters 7 and 8. Michael Höhle has originally composed the exercises of Sections 11.3, 12.5 and Finally, Nils Toft has further developed the exercises of Sections 12.5 and Unless otherwise stated, all references to specific sections, figures and tables refer to the textbook Herd Management Science, Volumes I and II, by Kristensen et al. (2010a,b). The exercises are organized in accordance with the chapters of the textbook. In order to avoid ambiguity, all internal numbering of tables, figures and equations in the exercises is done with the prefix E for Exercises. Thus, for instance, Eq. (E7.1) should be looked up in this book - not in the textbook. For some of the exercises, solutions will be provided through the home page of the course at For the exercises, the following general software systems are used: Excel (with the solver installed) R (statistical software package). In addition to the general software systems, a special Java based software package is provided for the course. A Windows Installer package, Advanced.msi, is available from the home page for simultaneous installation of the following software programs: The TV Show simulation demo A simple Markov chain simulation tool Sow monitor application demo The SimFlock simulation model The Esthauge LIMID software system The MLHMP software The installer package also installs the Java Runtime Environment version 1.6. Having installed the package, an entry called "Advanced Herd Management" is available on the Windows "Start" menu. It is recommended to install the package. A few additional programs may be used in the course. Copenhagen, July 27, 2010 Anders Ringgaard Kristensen

4 4 This book has a home page at URL: ark/book.htm

5 Contents 4 Classical production theory Optimal feeding level of dairy cows The dairy cow replacement problem: Classical approach Optimal slaughter weight for bull calves: Classical approach Basic production monitoring Basic monitoring: Litter size results Basic monitoring: Mortality results Basic monitoring: Daily gain in slaughter pigs From registration to information Value of information Categorical measurements: Sensitivity and specificity Daily gain in slaughter pigs Dynamic production monitoring by classical methods Control charts Plotting of the data Create a Shewart Control Chart for the time series k Calculate the EWMA Create a EWMA Control Chart Use the EWMA as a predictor for autocorrelated data Optimize the value of lambda Monitoring cell counts by classical methods Use the classical methods Shewart control chart Shewart control chart for the prediction errors Find the alarms Discussion Dynamic production monitoring by state space models Dynamic Linear Models A simple DLM in R Implement a simple DLM in R

6 6 CONTENTS Plotting Implement a simple DLM using a Discount Factor Analysis of the forecast errors Monitoring revisited Monitoring cell counts by state space models Dynamic Linear Model DLM with discount factor Intervention Discussion Modeling and sensitivity analysis by Linear Programming Ration formulation using linear programming Data Envelopment Analysis, DEA Introduction - relative efficiency of sow herds Implementation in R Implementation in Excel Assignment One factor and one product One factor and two products Two factors and two products Limitations of DEA Bayesian networks Bayes Theorem and the inseminated cow Mastitis diagnosis and d-separation Johnson s insemination strategy Johnson s heat detection Use of a Bayesian network to model litter sizes The general multivariate normal distribution Effect of genotype Effect of body condition score (BCS) Effect of correlated body condition scores Decision graphs Background and objective of the study The Bayesian network Risk factors Clinical signs Hypothesis variables Structure of the network Conditional probabilities Assignment: Bayesian network The shoulder lesion problem as a decision graph The basic model without information edges

7 CONTENTS Value of information from BCS and clinical signs Assignment: Value of information The pig breeders problem The milk test problem Adding a second milk test Soluble LIMIDs Dynamic programming and Markov decision processes Optimization algorithms using Excel Value Iteration Policy iteration Optimization algorithms using R Value Iteration Policy iteration Hierarchical Markov decision processes The MLHMP software Basic concepts and model use Model comprehension and optimization Extending the model A hierarchical model Reduced sow model with Bayesian updating Short description Assignment The ewe replacement model Background and problem formulation Assignment Acknowledgements Monte Carlo simulation Modeling a batch of slaughter pigs Description of the production The simulation model Model type State of nature Decision strategy Modeling a pig Modeling a batch A simulation run Functions returning output variables, Ω Assignment Simulation of a scavenging chicken production system The SimFlock user interface: Testing alternative decision rules Editing the hyper distribution

8 8 CONTENTS Defining a simulation job Creating an initial flock Creating multiple states of nature Testing each of the decision rules Modifying the R program Running the R program Exercise Exercise 2. Smallholder egg production? Background Problems Exercise 3. Providing shelter Background Problems A Probabilities and distributions 107 A.1 Basic probability calculus A.2 Conditional probability calculus A.3 Discrete distributions A.4 The normal distribution C Linear algebra 111 C.1 Matrix and vector operations C.2 Linear regression and matrices D Advanced topics from statistics 115 D.1 The multivariate normal distribution D.2 Hyper distributions Bayesian Networks and Decision Graphs by Finn V. Jensen 120

9 Chapter 4 Classical production theory The exercises for Chapters 4 and 5 all use the spreadsheet CompSes1.xls, which should be saved to disk and opened in Excel. The sheets of the spreadsheet contain lots of numbers and formulas, but in general, cells that are supposed to be edited are green, whereas a pink cell contains an important result. In several of the exercises, the Scandinavian Feed Unit (SFU) is used for energy. It is a net energy based system where 1 SFU is defined as the net energy content of 1 kg barley. 4.1 Optimal feeding level of dairy cows For this exercise you must use the sheet Optimal feeding level. You may also refer to Exercise in the textbook. The idea of this exercise is to illustrate the use of the classical method for determination of optimal input level as described in Section Our goal is to determine the optimal feeding level for a group of dairy cows during the first 24 weeks of lactation. Thysen (1983) identified the following relation between average daily milk yield, Y, and daily energy intake, X, in SFU: Y = b 0 + b 1 X + b 2 X 2 (E4.1) for constant protein/energy and fat/energy rates. For given values of the parameters b 0, b 1 and b 2 (calculated from estimates reported by Thysen, 1983) the milk yield has been calculated for energy levels in the interval from 12.0 to 16.5 SFU. Also the marginal milk yield, dy dx = b 1 + 2b 2 X, has been calculated in a column. The milk yield as a function of energy intake has been plotted in the sheet. The marginal milk yield is simply the slope of the tangent of the curve. This is illustrated graphically:

10 The dairy cow replacement problem: Classical approach By entering an energy level, X, in Cell B10, the marginal yield is returned in Cell E10. Furthermore, the tangent to the milk yield curve at X is added to the plot. Your job is now to determine the optimal energy level for the dairy cows by use of the methods described in Section 4.1.4: 1. Create a plot like Figure 4.3 including total revenue, marginal revenue, average revenue and factor price (just add one column for each and make a plot). 2. Identify the 3 regions I, II and III. 3. At what energy level does Region II begin and at what level does it end? 4. Identify the optimal feeding level. 5. Is the optimum energy level sensitive to price changes? 4.2 The dairy cow replacement problem: Classical approach For this exercise you must use the sheet Optimal replacement. In addition, the basic parameters as estimated by Kristensen (1986) are hidden in the sheet Parameters (opt.rep.) (yellow tab). You may also refer to Section and Figure 4.7. There are many examples in older literature of applications of the classical replacement theory to dairy cows (van Arendonk, 1984). The data used for this exercise is based on a study by Kristensen (1986). The input to the model is: Herd level of milk yield: Defined as total kg ECM week 1-40 for 1st lactation cows. (Cell B14). Feed (energy) efficiency: Base level is 0.9. Adjusting to e.g will increase feed intake by the factor 0.95/0.90. (Cell B18). Prices: Heifer for replacement, milk (per kg ECM), basic food for maintenance, embryo and gain (per SFU), production food for milk (per SFU), calf, culled cow (per kg live weight). The price may be specified in any currency (the initial prices are in DKK). (Cells I13-I18). Output from the model is: A plot of marginal and average profit (just like Figure 4.7). Average daily profit returned in Cell I20 (under the optimal solution).

11 4.3 Optimal slaughter weight for bull calves: Classical approach 11 The optimal replacement time returned in Cell I21. Assignment: 1. Enter relevant input for herd level (feed efficiency) and prices. 2. Identify the optimal replacement time under the conditions you specified. 3. Try to increase all prices (one at the time) by 30%. For each of the price changes, notice the consequences for average daily revenue and optimal replacement time, respectively: (a) Which price changes is the optimal replacement time sensitive to? (b) Which price changes is the average daily profit sensitive to? 4. When producing under a milk quota, a more relevant criterion of optimality is average revenue per kg ECM produced. Try to solve the replacement problem under that criterion instead Optimal slaughter weight for bull calves: Classical approach For this exercise you must use the sheet Optimal slaughter weight. You may also refer to Exercise Based on experimental results, Holstein Friesian bull calves fed concentrates ad libitum, are assumed at the live weight, W, to have a daily energy intake, X, X = 3.56 ln(w ) (E4.2) where X is measured in SFU, and a daily gain G, G = X (2.17 exp( w )) (E4.3) where G is in kg. We wish to determine the optimal slaughter weight. Two criteria (I and II) may be used. Under Criterion I, the calf is slaughtered when the marginal (daily) revenue equals the marginal cost. Under Criterion II, the calf is slaughtered when the marginal (daily) profit equals the average profit (from insertion to slaughtering). The input to the model is: Calf price: Price of a 60 kg calf. (Cell D7). Feed price: The price is per SFU. (Cell D8). 1 Hint: Create columns for cumulated milk yield, average profit per kg milk based on cumulated milk yield and cumulated revenue, and finally, marginal profit per kg milk based on milk yield and revenue for the individual period. Plot the two last columns.

12 Optimal slaughter weight for bull calves: Classical approach Slaughter price: The price is per kg live weight (Cell D9). Output from the model is: A plot of marginal revenue and marginal cost. A plot of marginal and average profit (corresponding to Figure 4.7). Optimal slaughter weight under Criterion I. (Cell I7). Optimal slaughter weight under Criterion II. (Cell I8). Assignment: 1. Enter realistic prices in Cells D7, D8 and D9. What is the optimal slaughter weight under each of the two criteria? Try to explain the difference. 2. What is the consequence of increased calf price under Criterion I? Can you explain what you observe? 3. What is the consequence of increased feed price under Criterion I? 4. What is the consequence of increased slaughter price under Criterion I? 5. What is the consequence of increased calf price under Criterion II? Can you explain what you observe (compared to the result for Criterion I)? 6. What is the consequence of increased feed price under Criterion II? 7. What is the consequence of increased slaughter price under Criterion II? 8. Explain in words, what is maximized under each of the two criteria. 9. Under what circumstances is Criterion I relevant? 10. Under what circumstances is Criterion II relevant?

13 Chapter 5 Basic production monitoring The exercises for Chapter 5 again use the spreadsheet CompSes1.xls, which should be saved to disk and opened in Excel. The sheets of the spreadsheet contain lots of numbers and formulas, but in general, cells that are supposed to be edited are green, whereas a pink cell contains an important result. 5.1 Basic monitoring: Litter size results For this exercise you may use the sheet Litter size. In a sow herd the expected average litter size was 14.0 live born piglets. The production report for the most recent quarter showed that the observed value was only 13.7 for the 375 farrowings. In order to see whether or not we should be worried, we follow the guidelines from Section of the textbook: 1. How is the key figure calculated from data? 2. Specify an approximative distribution for the data. 3. Use standard rules from statistics (Appendix A) to determine the distribution of the key figure. 4. What is the null hypothesis to be tested? 5. Calculate confidence limits for the key figure. 6. Should we be worried? 7. Assume that the average was based on only 200 farrowings. Should we then be worried?

14 Basic monitoring: Mortality results 5.2 Basic monitoring: Mortality results For this exercise you may use the sheet Mortality. In a sow herd the expected piglet mortality before weaning was 11.0%. The production report for the most recent quarter showed that the observed value was 11.9% based on 5138 live born piglets. In order to see whether or not we should be worried, we follow the guidelines from Section of the textbook: 1. What is the null hypothesis to be tested? 2. How is the key figure calculated from data? 3. Specify an approximative distribution for the data. 4. What is the null hypothesis to be tested? 5. Calculate confidence limits for the number of dead piglets. (Or, equivalently, calculate the p value - which in this case is easier). 6. Should we be worried? 5.3 Basic monitoring: Daily gain in slaughter pigs For this exercise you must use the sheet Daily gain. You may also refer to Exercise This exercise deals with interpretation of a measured value of the "daily gain" key figure in slaughter pigs. It is basically a solution to Exercise even though it only covers Problems e and f of the exercise in the textbook. It is recommended to read the text of the exercise. The idea is to illustrate the influence of different methods for obtaining the valuation weights 1 which are used for calculation of the key figure. Two basic principles may be applied: 1. Weighing a representative fraction of the pigs: either 1 (or a few) randomly selected from each pen, or, a number of pigs selected completely at random. 2. Visual estimation of the weight. The input to the model is: Choice of basic method: Weighing a fraction versus visual estimation. (Cell A6). Choice of sampling method for the fraction: Selection systematically from each pen versus at random from the entire herd. (Cell A14). 1 The valuation weight is the total live weight of all pigs in the herd on the date of valuation.

15 5.3 Basic monitoring: Daily gain in slaughter pigs 15 Number of pigs to sample from pen/herd: How many should we sample from each pen (Cell B14) or how many from the entire herd (Cell B15). Standard deviation of daily gain within herd (kg): This figure represents the natural biological variation. (Cell L6). Standard deviation of live weight within pen (kg): Typically pigs in the same pen are rather homogeneous. (Cell L7). Standard deviation of live weight within herd (kg) : This variation reflects age differences etc. (Cell L8). Measurement error, visual estimation of live weight: How good is the farmer at estimating the valuation weight visually? (Cell L9) Output from the model is: A plot of the measured daily gain (with standard deviation) compared to the target. The standard deviation of the calculated daily gain under the assumptions specified. (Cell H27) The deviation from the target measured with the standard deviation as unit. As a rule of thumb, the deviation is significant if this value is numerically greater than 2. (Cell G30) The calculated daily gain is 796 g (Cell G27) and the target (expected value) is 820 g (Cell G28). The basic question to be considered is: Should we be worried over this deviation. In order to answer the question, you should: 1. Make adjustments to the input values concerning standard deviations (if you consider them unrealistic). 2. Try to vary the settings for obtaining the valuation weights and study the consequences for the standard deviation of the daily gain. Under what circumstances should we be worried about the result (796 g daily gain)? 3. Under the systematical sampling, what is the benefit of selecting more than one pig from each pen? 4. How many pigs must we sample at random from the entire herd in order to have the same precision as by sampling one pig systematically from each pen? 5. What happens if the standard deviation is the same within pen as within herd?

16 Basic monitoring: Daily gain in slaughter pigs

17 Chapter 6 From registration to information 6.1 Value of information 1. In Example 6.1 of Section the simultaneous distribution of litter size Y = (Y 1i, Y 2i ) in first and second parity of a sow was described by the bivariate normal distribution ( Y1i Y 2i ) N (( ) ( , )). (E6.1) For a particular herd, however, Toft and Jørgensen (2002) estimated the following bivariate normal distribution: ( Y1i Y 2i ) N (( ) ( , )). (E6.2) (a) Describe in words the difference between the two herds as expressed by Eqs. (E6.1) and (E6.2). (b) Assume that Y 1i = 11 has been observed for sow i. What is then the expected litter size of second parity: in the herd described by (E6.1) in the herd described by (E6.2) (c) In Example 6.1 the value of observation of first litter size results was calculated as 0.28 piglets in the herd described by (E6.1). Do you think that it will be higher or lower than 0.28 in the herd described by (E6.2)? Explain why. (d) Calculate the value of observations of first litter sizes in the herd described by (E6.2). The calculation is done exactly as shown in Example 6.1.

18 Categorical measurements: Sensitivity and specificity 2. In a dairy herd, the simultaneous distribution of milk yield Y = (Y 1i, Y 2i ) in first and second lactation of a cow is assumed to be described by the bivariate normal distribution ( Y1i Y 2i ) N (( ) ( , )). (E6.3) (a) What is the conditional distribution of Y 2i given Y 1i? (b) Assume that we have a two-cow problem, i.e. we can only keep one of the cows (due to capacity constraints). What is then the value (measured in kg milk) of knowing the milk yield of first lactation? 6.2 Categorical measurements: Sensitivity and specificity Ucar et al. (2004) suggested use of milk progesterone assay for early pregnancy diagnosis in Anatolian water buffalos where clinical signs of oestrus are described as weak. The following exercise is based on a simplified interpretation of the results reported by the authors. The idea was to measure the progesterone content of milk on day 21 after mating. In non-pregnant buffalos, the progesterone level is expected to be low due to the absence of a corpus luteum. For pregnant buffalos the distribution of the progesterone level was found to be H + N (8.04, ). For non-pregnant buffalos the distribution of the corresponding hormone level was not clearly described in the article, but based on a number of plots, it seemed to be something like H N (5.01, ). In Figure E6.1 the two distributions are plotted. Assume that we are able to measure the progesterone level on day 21 automatically. For a given measured progesterone level H = h we must conclude whether or not the buffalo is pregnant. The principles of Example 6.3 of Section may be used. 1. What threshold value τ do you suggest to use based on Figure E6.1? 2. What are the sensitivity and specificity of the resulting pregnancy test? 3. Based on the calculated sensitivity and specificity, do you then wish to adjust the threshold value? A closer examination of the non-pregnant buffalos showed that they actually fell in two distinct groups: Healthy buffalos that (just by accident) didn t conceive at mating. For these buffalos, the progesterone level H H could be described by the distribution N (0.88, ) according to Ucar et al. (2004).

19 6.2 Categorical measurements: Sensitivity and specificity 19 0,16 0,14 0,12 0,1 0,08 0,06 0,04 0, Progesterone level (ng/ml) Pregnant Non-pregnant Figure E6.1: Distribution of progesterone level 21 days after mating for pregnant and non-pregnant water buffalos. Buffalos with reproductive disorders as for instance persistent corpora lutea. The distribution of the progesterone level of these animals was not directly reported by Ucar et al. (2004), but it seems to be something like H D N (10, 3 2 ). The distributions for the 3 groups of buffalos are shown in Figure E6.2. Now, assume that what you are really interested in is to find healthy buffalos that have not conceived. With this purpose in mind, consider the following questions: 4. What threshold value τ do you suggest to use based on Figure E6.2? 5. What are the sensitivity and specificity of the resulting pregnancy test (with no distinction between pregnant and non-pregnant with disorders )? In order to calculate the sensitivity you may assume that the base probabilities of the three groups are 0.05 for non-pregnant with disorders, 0.45 for pregnant and 0.50 for non-pregnant, but healthy. 6. Ucar et al. (2004) end their article with the following statement In conclusion, milk progesterone assay is a fast and reliable tool to verify oestrus time and to detect non-pregnant buffalos with a 100% accuracy. Do you agree with the authors?

20 Daily gain in slaughter pigs 1,2 1 0,8 0,6 0,4 0, Progesterone level (ng/ml) Pregnant Non-pregnant, healthy Non-pregnant, disorders Figure E6.2: Distribution of progesterone level in water buffalos 21 days after mating for pregnant, non-pregnant (but healthy) and non-pregnant with reproductive disorders Daily gain in slaughter pigs The following exercise is based on Example 6.4 of Section For the pigs of a slaughter pig herd we assume that the individual daily gain of an arbitrarily selected pig is DG N (850, ). The intended live weight at slaughtering is 100 kg, and the pigs are inserted at a live weight of 30 kg. On average, the pigs need (100 30)/ days to grow from 30 to 100 kg. For a pen with 16 pigs, the true daily gain of each pig is assumed to be as shown in Table E Assume that the 16 pigs are fattened in a herd producing under a very strict all-in/all-out policy where all pigs are initially inserted and later slaughtered on exactly the same date 82 days later. Under this assumption, the average daily gain DG H may (according to Example 6.4) be calculated as DG H = 16 i=1 DG i. 16 So, what is the average daily gain that would appear in the production report? 2. Assume that the 16 pigs are fattened in a herd producing under a very strict continuous policy where any pig is slaughtered exactly when it reaches the intended live weight of 100 kg. Under this assumption, the average daily

21 6.3 Daily gain in slaughter pigs 21 Table E6.1: Individual true daily gain for 16 slaughter pigs in a pen. Pig # DG i 1/DG i Sum gain DG H may (according to Example 6.4) be calculated as DG H = i=1 DG i So, what is the average daily gain that would appear in the production report? 3. Describe the consequences of comparing calculated daily gain in systems with all-in/all-out production with results from continuous systems.

22 Daily gain in slaughter pigs

23 Chapter 7 Dynamic production monitoring by classical methods Cécile Cornou 7.1 Control charts The data set for this exercise is taken from the Course Textbook (Chapter 7, Table 7.1). The Excel file Exercise1.csv contains the average daily milk yields of a dairy herd for the first lactation (milk1) and other parities (milk2). Open and run the document FUNCTIONS.R. In this document you will find the functions used for the exercises of Chapter 7 and 8. Open and save the document Ex1.R where you will edit your R codes Plotting of the data Set your own working directory by writing the R code below (adjust it to your own path) or using "File - Change dir..." in the tool bar. setwd("c:/documents and Settings/...") Import the data set saved as a csv file and save it as table1. table1 <- read.csv2("exercise1.csv") To print the table on the R console, and have a look at the variables use table1 Create an empty vector k and assign the values of average milk of the other parities to the vector k

24 Control charts k <- c() k <- table1$milk2 Note that the dollars sign $ is used to extract and store only the variable milk2 of table1, into the vector object k. Make a first plot of the data using plot(k, type="l") The option type indicates the type of plot. Here type="l" indicates "lines" - the defaults is "p", plotting points Create a Shewart Control Chart for the time series k Assign the corresponding values for each element of the Shewart control chart to the objects cl, ucl and lcl. Use the target value and a = 2. We assume that the number of cows behind the average is constant for every quarter, i.e. n t = 275. Use this number to calculate sd1, the corresponding standard deviation. Note that the code for calculating the square root in R is sqrt(..); the equation for calculating the standard deviation is found in Example 7.1 of the text book. To make sure that all the lines appear on the graph, add the option ylim=c(31,34) to the plotting function as plot(k, type="l", ylim=c(31,34)) To add the elements of the control chart use abline(h=cl). Note that abline(h=...) is used to plot an additional horizontal line on your existing plot. Plot both control limits (using a=2) and warning limits (using a=1.5). To identify warning limits, you can change the type of line by using the option lty=2 (or add colors e.g. using col="red") such as abline(h=uclw,lty=2)

25 7.1 Control charts Calculate the EWMA Implement the EWMA as defined in (E7.1) (corresponding to (7.14), Page 55 in textbook) z t = λk t + (1 λ)z t 1 (E7.1) The function FUNEWMA, found in the FUNCTIONS.R document, makes the implementation flexible for changes in λ. Store the values calculated by FUNEWMA in a vector z. To understand this function, and if you are not familiar with programming, follow the note below. Note 1. Elements of a vector are selected by using squared brackets at the end of the vector. For instance, k[1] is used to extract the first element of the variable k; k[1:10] extracts the first 10 elements. Create an empty vector z and assign the first element of the variable k to the first element of the vector z. z <- c() z[1] <- k[1] 2. Use a for loop to calculate the results of the EWMA (from i=2 to i=n) using equation (E7.1). The for loop should have the same number of elements as the vector k. To calculate the number of elements (n) of the vector k use the function length(). n <- length(k) 3. Assign the value 0.6 to lambda and run the for loop. lambda <- 0.6 for (i in 2:n) {z[i] <- lambda*k[i]+(1-lambda)*z[i-1]} 4. In order to quickly calculate the exponentially moving average for different values of λ, it is easier to create a function. The function FUNEWMA(k,lambda) is presented below. FUNEWMA <- function(k,lambda) { n <- length(k);

26 Control charts z <- c(); z[1]<-k[1] for (i in 2:n) {z[i] <- lambda*k[i]+(1-lambda)*z[i-1]} return(z); } Assign the value 0.6 to lambda, apply the function and store the results in the vector z, as z <- FUNEWMA(k,lambda) Plot the observation vector k and add the result of the EWMA using the command lines(z). Try it for different values of lambda, for instance 0.1, 0.5, 0.9, You can store the results in different vectors z1, z2... and use different colors for the lines (e.g. lines(z1, col="blue")) Create a EWMA Control Chart Create a EWMA control chart as you have done for the Shewart control chart. Use the same Center Line but change the Control Limits using the standard deviation sde calculated as sde <- sd1 * sqrt(lambda/(2-lambda)) where sd1 is the standard deviation used for calculating the control limits of the Shewart Control Chart Use the EWMA as a predictor for autocorrelated data As mentioned in your Textbook in Section 7.6.3, the EWMA is a good one-stepahead predictor for autocorrelated data. Here, we want to create a Shewart control chart for the prediction errors e t, such as e t = k t z t 1. Use the function FUNERRORS(z) to calculate the prediction errors. Store the results in a vector errors. Create the Shewart control chart for the prediction errors. Plot the prediction errors (errors); use other limits for the vertical axis, i.e. ylim=c(-0.5,0.5); use 0 as Center Line. Use the standard deviation of the prediction errors sdr to calculate the 2σControl limits sdr <- sd(errors)

27 7.2 Monitoring cell counts by classical methods Optimize the value of lambda The value of lambda is optimized by minimizing the sum of the squares of the errors. Here we create a vector with λ values varying from 0.1 to 1 by interval of The for loop shown below calculates, for each value of λ: 1) the EWMA, 2) the errors and 3) the sum of the squared errors (stored in the vector res). vector <- seq(0.1,1,by=0.02) res <- c() i<-1 for (lambda in vector) { z <- FUNEWMA(k,lambda) errors <- FUNERRORS(z) res[i] <- sum(errors^2) i<- i+1; } To find out the value of λ where the sum of the square is minimum, run the for loop and use vector[which(res==min(res))] This indicates which element of res is the minimum, and the value of λ corresponding to this element. Create a 2σ control chart for the prediction errors, using the optimum value of λ. 7.2 Monitoring cell counts by classical methods For the mandatory report, you will analyze a time series of cell counts for cows who have more than one lactation - celle2.csv. The report should be around 3-5 pages (included graph). Send the text document (notepad) with the R codes that consisted the basis of your analysis, on a file separated from your report Use the classical methods Shewart control chart Implement a 2σ Shewart control chart, using θ = 240 and σ = 65; for plotting, it is suggested to use the option ylim=c(0,500).

28 Monitoring cell counts by classical methods Shewart control chart for the prediction errors Implement a Shewart control chart for the prediction errors from the EWMA. Estimate and use the optimum λ. You can use Section (of the exercises) as guideline. Find the alarms For the Shewart Control Chart, use the function FUNALARM (found in the document FUNCTIONS.R) to print which week there is an alarm (if any). This function indicates the weeks where the errors are above UCL or below LCL. First, extract the time series week from your data set, as week<- table$week Discussion Discuss the results and the methods: What does represent lambda in the EWMA? What does represent the value of the EWMA? Why do we look at the prediction errors? Are these methods satisfying to analyze your data sets?

29 Chapter 8 Dynamic production monitoring by state space models Cécile Cornou 8.1 Dynamic Linear Models As for Chapter 7, open and run the document FUNCTIONS.R. In this document you will find the functions used for the exercises of Chapter 7 and 8. Open and save the document Ex2.R where you will edit your R codes. Data for this exercise describe the activity of a sow around farrowing, and are found in the text file Exercise2.txt. Acceleration data have been collected for individual sows in a farrowing house. Data were measured in 3-dimensions, and collected four times per second, during 8 days. Figure E8.1 shows the acceleration data collected for sow 7, previously averaged per 10 seconds acceleration, in g F1 x,y,z (blue,red,green) Time, in days Figure E8.1: Acceleration data measured in 3-dimensions and averaged per 10 sec. Data for sow 1, collected from the 28 May to the 2 June The vertical dotted line indicates the onset of farrowing. In a first step, the series of acceleration measurements were analyzed using a Multi Process Kalman Filter (MPKF) of class I. This aimed at classifying sows

30 Dynamic Linear Models activity types. From the quantitative series, presented in Figure E8.1, a qualitative series is obtained. For every 2 min observations, a sow activity is described as either High Active, Medium Active or Passive (which includes Lying on the right side, left side and sternally). The series of this exercise describes the difference of the number of 2 min observations classified as High Active (HA) for an hour (t), as compared to the same hour the day before (t-24), such as: Y t = HA t HA t 24. (E8.1) In other words, the series describes the evolution of the periods where the sow is highly active. This series is previously standardized. The objective of this exercise is to implement a simple DLM on this series of observations, as introduced in the second Lecture. Recall the form of the simple DLM Observation equation Y t = µ t + ν t, v t N(0, V t ) (E8.2) System equation µ t = µ t 1 + ω t, w t N(0, W t ) (E8.3) where Y t is the time series observed. The objective of using a DLM combined with a Kalman Filter (KF) is to estimate the true underlying level (µ t ), by mean of its mean m t and variance C t. To do so, we use the updating equations of the KF, which are shown in Equations (E8.4 to E8.11). P rior mean a t = m t 1 (E8.4) P rior variance R t = C t 1 + W t (E8.5) One step forecast mean f t = a t (E8.6) One step forecast variance Q t = R t + V t (E8.7) Adaptive coefficient matrix A t = R t /Q t (E8.8) A simple DLM in R One step forecast error e t = Y t f t (E8.9) T rue underlying level m t = a t + A t e t (E8.10) F iltered variance C t = A t V t (E8.11) The function FUNFILTER(Yt,mu0,C0,V,W) is created from scratch using the Equations (E8.4 to E8.11). This function calculates the true underlying level of the time series Yt. The initial information needed for the function is mu0, C0

31 8.1 Dynamic Linear Models 31 and the variances V and W. The codes printed on the next page show you how this function is build. You can use these lines if you want to create the function yourself. Otherwise, it is available in the document FUNCTIONS.R. FUNFILTER <- function(yt, mu0, C0, V, W){ n <- length(yt); at <- c(); # Define the vectors Rt <- c();... mt[1] <- mu0; Ct[1] <- C0; # Set the initial values for(i in 2:n) { at[i] < }; return(list( at=at,.. mt=mt, Ct=Ct)); }; Implement a simple DLM in R Import the data from the file Exercise2.txt and assign the variable x (activity data) to the observed vector Yt. Use (µ 0 D 0 ) N(m 0, C 0 ) = N(0.2, 0.3), V = 0.5 and W = as initial information; assign these values to mu0, C0, V and W. Apply the function FUNFILTER(Yt,mu0,C0,V,W) and store the results in Res The variables stored in the object Res can be affected to a new object to ease their access as

32 Dynamic Linear Models mt <- Res$mt ft <- Res$ft At <- Res$At Plotting Plot the observations Yt. Add the true underlying level mt, and the forecast ft using another color or type of line. Illustrate graphically the development of A t. According to the equation (8.13) of the textbook, what does this evolution of A t indicate? Implement a simple DLM using a Discount Factor Sometimes the system variance Wt is unknown. In that case, we can use a discount factor δ, 0 δ 1. Typical values of δ are between 0.8 and 1. In that case the system variance is given by a fixed percentage of the uncertainty of the level-estimate as W t = 1 δ C t 1 δ (E8.12) which means that R t = C t 1 δ (E8.13) In the function FUNFILTERD (from FUNCTIONS.R) you can see that R t is modified and that in the function specification, the argument W is replaced by the value of the discount factor. Try the filter for 3 different values of δ: 0.05, 0.5 and 0.9. Apply the function FUNFILTERD for these values and store the results in the objects Res1, Res2 and Res3. Make a plot of the observations Yt and of the corresponding true underlying levels mt1, mt2 and mt3. Plot these results on the graph using different line colors or types. Interpret the results Analysis of the forecast errors Around farrowing, a sow will typically perform nest building behavior. We can therefore expect to observe an increase of the number of 2 min observations of

33 8.2 Monitoring revisited 33 High active type. In order to see whether the behaviour of the sow is changing around farrowing, we look at the forecast errors of our model. Run the function FUNFILTER as done in 8.1.2, i.e. with the same initial values and variances, and extract the forecast errors et and the variance associated Qt. Then calculate the standardized errors ut, such as u t = e t Qt (E8.14) and calculate the cumulative sum of the standardized errors using the standard function cumsum. Store the results in the variable cu, such as cu <- cumsum(ut[2:n]) Plot the cumulative sum cu and add the farrowing time (t=90), using abline(v=90,col="green") Use the function FUNALARM2 using a threshold/upper limit value lim = 9. Add an horizontal line that indicates the threshold value and a vertical line at the alarm time. Interpret the results. 8.2 Monitoring revisited Today we will, for a while, turn the computers off and only rely on paper and pencil. The aim of the exercises is to illustrate how the graphical language used with Bayesian networks can be used to visualize the other monitoring techniques presented in the course. Another aim is to practice translation from formulas to graphs (DAGs) and vice versa. 1. As we have seen in the lecture slides, an ordinary first order polynomial DLM may be drawn as a Bayesian network as the one in Figure E8.2. In formulas, such a model is defined by the observation equation and the system equation κ t = θ t + v t, v t N (0, V ) θ t = θ t 1 + w t, w t N (0, W ). Are the two representations (the equations versa the figure) completely analogous, or are there any differences?

34 Monitoring cell counts by state space models Figure E8.2: An ordinary first order polynomial DLM (6 observations) drawn as a Bayesian network. 2. Draw the linear growth DLM of Section as a Bayesian network with for instance 5 observations. 3. Draw the seasonal model of Example 8.12 as a Bayesian network with at least 5 observations. 4. A Multi-process DLM of Class II as the one described in Example 8.21 may be drawn as shown in Figure E8.3. Change the figure so that it corresponds to a multi-process DLM of Class I instead. 5. In the lecture a DLM for monitoring daily gain of slaughter pigs with data from online automatic weighing systems. The observation equation was ( ŵ(t) ŵ 0.16 ) = ( y(t) 1 0 y(t) 1 1 ) L e(t) ρ(t) + ( ɛt τ t ) and the system equation was: L e(t) ρ(t) = α 0 t 0 0 t 1 L e(t 1) ρ(t 1) + 0 η t 0. Each time ŵ(t) and ŵ 0.16 are observed, the estimates of L, e(t) and ρ(t) are updated. Draw the model as a Bayesian network. 8.3 Monitoring cell counts by state space models For the mandatory report, you will analyze a time series of cell counts for cows who have more than one lactation - celle2.csv. The report should be around 3-5 pages (included graph). Send the text document (notepad) with the R codes that consisted the basis of your analysis, on a file separated from your report.

35 8.3 Monitoring cell counts by state space models 35 Figure E8.3: A multi-process DLM of class II as in Example 8.21 (rotated 90 degrees). The prefix Mx: in the node label indicates that the node belongs to Model x, x {1, 2, 3}, corresponding to Normal, Outlier and Level shift, respectively. The pt nodes at the bottom hold the probabilities that the observation at time t are drawn from each of the three models.

36 Monitoring cell counts by state space models Dynamic Linear Model DLM with discount factor Implement a DLM with discount factor δ = 0.8. Use (µ 0 D 0 ) N(m 0, C 0 ) = N(240, 4000) and V = 50 as initial information. Make a plot of the observation vector and the underlying level. Make a plot of the Adaptive coefficient. Intervention Consider that an event occurred at time t = 14 (the 14th observation of your time series). You have no information about the effect of this event. This event may produce an unwanted false alarm. You decide to realize an intervention. Try to implement an intervention of type II as mentioned in the lecture II. Help: To implement it, replace the scalar delta by a vector delta where delta[14] has a lower value than the other values. delta <- rep(0.8,length(k)) delta[14] <- 0.2 Modify the function FUNFILTERD by replacing delta by the indexed vector delta[i] inside the function. Implement the intervention. Compare the underlying mean level and the one step forecast mean with and without intervention. Look at the Adaptive coefficient. How does it change around the time of intervention? How would that influence the one-step forecast errors? Discussion Discuss the results and the methods: Are these methods satisfying to analyze your data sets? What is main difference between the DLM and the classical methods (from the previous chapter)? Could you suggest any improvement for the DLM? What alarm system could you implement after modeling your data set with a DLM? (hint: some possibilities are described in the last section of Chapter 8 in your textbook.)

37 Chapter 10 Modeling and sensitivity analysis by Linear Programming 10.1 Ration formulation using linear programming For this exercise you will need: Excel with the Problem solver installed. A feedstuff table. A table of requirements for your favorite animal species. The idea of this exercise is that you define and solve a more realistic ration formulation problem than the simplified example used in the lecture. If you wish, you can (for inspiration) download the CowFeeding.xls spreadsheet containing the example from the lecture. Refer also to Kristensen et al. (2010b) for background information. 1. Make a decision concerning the animal species and the group of animals (e.g. Slaughter pigs, Pregnant sows, Bull calves, First lactation dairy cows, beginning of lactation ). 2. Based on your knowledge about the group of animals chosen (and the table of requirements ) you define a number of nutritional demands on the feed ration. It is recommended not to include too many demands (at least not initially). You should for instance omit demands for vitamins and minerals. 3. Choose (from the feedstuff table) a number of relevant feedstuffs. The selection of feedstuffs should be based on your knowledge about feeding. 4. Define realistic prices for your feedstuffs. 5. Use Excel to set up a linear programming problem calculating a least cost ration for your group of animals.

38 Data Envelopment Analysis, DEA 6. Use the Problem solver to find the least costs ration. If it turns out that no solution exists, you can either add some more feedstuffs, or you can relax some of the demands. 7. If you have initially defined fewer demands than you find relevant (from a feeding point of view), you can add some additional constraints. 8. Identify the shadow prices of the limiting constraints. If some of the shadow prices appear to be high, you may consider to relax some of your constraints. 9. Identify for each feedstuff the price interval where the price may vary without changing the optimal composition of the ration Data Envelopment Analysis, DEA Using data from 10 sow herds (the herds of Examples ) we shall estimate the relative efficiency under various conditions. The exercise can be solved in R or in Excel Introduction - relative efficiency of sow herds Implementation in R If you use R, the package linprog must be installed. The package solves by default a linear programming problem of which all constraints are less than,, and all variables are non-negative. Thus we don t need to indicate that explicitly. It should be noticed, that here we change the variable returns to scale constraint λ λ 10 = 1 to two less than constraints, i.e. λ λ 10 1 and λ 1... λ You must download the R program DEA.R and the data file SowHerds.txt". The R program performs the following steps: Loads the package linprog (if it has been installed). Reads (and prints) the data file. Defines a number of functions. Those functions don t need to be modified in the exercises. They are just left as they are. The functions are: definea: Defines the matrix of coefficients for the constraints. defineb: Defines the values of the constraints (the right hand sides of the inequalities). definep: Defines the coefficients of the objective function (the prices ). addnames: Adds explanatory names to the solution of the linear programming problem.

39 10.2 Data Envelopment Analysis, DEA 39 DEA: Perform Data Envelopment Analysis by linear programming. Defines (and prints) a vector of logical values defining whether a certain factor or product should be included in the analysis. The vector has 4 elements, and if it is defined as include = c(t, F, T, F) it means that the first factor is included, the second is excluded, the first product is included, and the second product is excluded. Defines and prints some examples of resulting vectors and matrices. Performs the DEA analyses, you define. To run an analysis, you use the DEA function as follows: 1. Define the include vector as desired. 2. Choose a herd. 3. Decide the assumption concerning returns to scale. 4. Call DEA and print the result. Suppose you wish to run an analysis for Herd 5 including sows and piglets assuming constant returns to scale. The following code will do it for you (the third argument to the DEA function is always 2): include = c(t, F, T, F) herd = 5 x = DEA(herds, herd, 2, include, F) x The code produces the following output: > include = c(t, F, T, F) > herd = 5 > x = DEA(herds, herd, 2, include, F) > x Results of Linear Programming / Linear Optimization Objective function (Maximum): Iterations in phase 1: 0 Iterations in phase 2: 2 Solution opt Relative efficiency of Herd Herd Herd Herd Herd Herd Herd Herd Herd Herd Herd Basic Variables opt Constraints actual dir bvec free dual dual.reg

40 Data Envelopment Analysis, DEA Sows 517 <= e Piglets 0 <= e All Variables (including slack variables) opt cvec min.c max.c marg marg.reg Relative efficiency of Herd Inf NA NA Herd Inf e-01 Herd Inf e+00 Herd Inf e-01 Herd Inf NA NA Herd Inf e+00 Herd Inf e+00 Herd Inf e+00 Herd Inf e+00 Herd Inf e-01 Herd Inf e+00 S Inf e+02 S Inf e+04 > As it is seen from the output, the relative efficiency of Herd 5 is 1.10 and it is simply benchmarked against Herd 4 as expected when we assume constant returns to scale. Implementation in Excel If you use Excel the Problem solver must be installed. You must download the spreadsheet Dea.xls. It contains 4 sheets: DEA solution - LP problem: The parameters of the linear programming problem used to estimate the relative efficiency. Plot, one factor, one product: A plot corresponding to Figure Plot, one factor, two products: A plot corresponding to Figure Raw data: The raw data with results from the 10 sow herds (and a few other data series necessary for the plots). Data for two factors (sows and feed) and two products (sows slaughtered and piglets produced) are included. In the exercise only the first-mentioned sheet is used. It is shown in Figure E10.1. The number, i, of the herd to benchmark against the other herds is entered in the green cell, and the solution F i, λ 1,..., λ 10 will appear in the pink cells. Five constraints have been predefined: two factor constraints, two product constraints, and a constraint for λ λ 10 = 1 indicating whether we assume constant returns to scale (constraint deactivated) or variable returns to scale (constraint activated). The linear programming problem is defined in the solver. The cell to be maximized is N19, and the cells to be edited (the variables) are in Cells B5-L5. We specify the problem to be a maximization problem and in the advanced settings, the problem is defined as a linear problem and the checkbox indicating a non-negative solution must be checked. The constraints are defined according to the problem. It is possible to run One factor - one product analyses,

41 10.2 Data Envelopment Analysis, DEA 41 Figure E10.1: The spreadsheet holding the the parameters of the linear programming problem used for Data Envelopment Analysis of 10 sow herds. One factor - two products analyses, Two factors - one product analyses, Two factors - two products analyses. All 4 analyses may be performed assuming constant or variable returns to scale. If you assume variable returns to scale, the constraint in Row 16 must be activated. Under constant returns to scale it is, accordingly, deactivated Assignment One factor and one product The factor is sows and the product is piglets. Thus the constraints for sows and piglets must be activated.initially, we will assume constant returns to scale, so the constraint for variable returns to scale must be deactivated. 1. Select Herd 1 and try to reproduce the results described in Example Select a few other herds (one by one) and see whether the results are in agreement with Table 10.5 Now, activate the constraint for variable returns to scale. 3. Select Herd 1 and try to reproduce the results described in Example Select Herd 6. What is the relative efficiency of Herd 6? Explain why. 5. Take a look at Figure Which herds do you expect that Herd 8 will be benchmarked against? 6. Select Herd 8 and run an optimization. Does the result confirm what you expected?

42 Data Envelopment Analysis, DEA Sows sla aughtered Piglets produced Figure E10.2: Figure 10.6 modified so that the axes intersect at (0, 0). One factor and two products The factor is again sows and the products are piglets and sows slaughtered. Thus the constraints for sows, piglets and sows slaughtered must be activated. Initially, we will assume constant returns to scale, so the constraint for variable returns to scale must be deactivated. 7. Take a look at Figure Which herds do you expect that Herd 5 will be benchmarked against? 8. Select Herd 5 and run an optimization. Does the result confirm what you expected? 9. Select a few other herds (one by one) and see whether the results are in agreement with Table 10.6 Figure E10.2 is identical to Figure 10.8 except that the intersection of the two axes is now (0, 0). Draw a straight line (use a ruler) from (0, 0) through the mark for Herd 7 and extend the line so that it crosses the production frontier. 10. Use the figure (with the new line) to answer: (a) Where (i.e. between which herds) does the line cross the production frontier?

43 10.2 Data Envelopment Analysis, DEA 43 Table E10.1: Relative efficiency of 10 sow herds evaluated on two products (piglets produced and sows slaughtered) and two factor (sows and feed). The efficiency is shown under two assumptions: variable returns to scale and constant returns to scale. Herd Sows Feed Piglets Sows slaughtered Relative efficiency, F i i x 1i x 1i Y 1i Y 2i Variable returns Constant returns (b) Measure the distance, a, from (0, 0) to the production frontier (along the line). (c) Measure the distance, b, from (0, 0) to the mark for Herd 7. (d) Calculate the ratio a/b. 11. Select Herd 7 and run an optimization in the solver. (a) Compare the efficiency to the ratio a/b. What do you notice? (b) Which herds are Herd 7 benchmarked against? Now, activate the constraint in Row 16 so that variable returns to scale are assumed. 12. Select a few herds (one by one) and see whether the results are in agreement with Table Two factors and two products Now, all the constraints must be activated. 13. Calculate the relative efficiency of all 10 herds under both assumptions regarding returns to scale, and complete the two last columns of Table E10.1. Limitations of DEA 14. Are there any implicit or explicit assumptions behind DEA that you find questionable?

44 Data Envelopment Analysis, DEA

45 Chapter 11 Bayesian networks The primary aim of this computer exercise session is to make you familiar with the Esthauge LIMID software system, which is the program we use for Bayesian networks and decision graphs. A secondary aim is to illustrate theoretical concepts like Bayes Theorem and the notion of d-separation. In addition to the computer program you will probably need the textbook chapters from Jensen (2001). Also Appendix A of the textbook Herd Management Science by Kristensen et al. (2010a) may be relevant. Finally, the example files ShowBN.xbn, PregnantCow.xbn, Pregnancy.xbn, mastitis.xbn, insem.xbn and insemheat.xbn are useful Bayes Theorem and the inseminated cow In the lecture the very simple Bayesian network shown in Figure E11.1 was created. The probability tables were as shown in Tables E11.1 and E11.2. Table E11.2 specifies the sensitivity and specificity of the heat detection test. It is valid for a three-week period. Figure E11.1: The pregnancy diagnosis network.

46 Bayes Theorem and the inseminated cow Table E11.1: Initial probability of pregnancy (the cow has been inseminated). Pregnant = Yes Pregnant = No Table E11.2: Conditional probabilities of heat detection. Condition Heat = Yes Heat = No Pregnant = Yes Pregnant = No Figure E11.2: The pregnancy diagnosis network with evidence for Heat = No.

47 11.1 Bayes Theorem and the inseminated cow 47 From the lecture we know that given that we have observed a cow not to be in heat, the probability of the cow being pregnant increases to 0.71 (cf. Figure E11.2). This new probability was calculated by use of Bayes theorem. In general terms, Bayes theorem may be written as follows (where S is the entire sample space so that P(S) = 1): Bayes theorem: Let A 1, A 2,..., A n be pairwise disjoint events so that n i=1 A i = S. Let B be an event so that P(B) > 0. Then P(A i B) = P(B A i )P(A i ) n j=1 P(B A j)p(a j ). In order to re-do the calculations we shall first define the following events using the notation of the theorem (notice that A 1 A 2 = S): A 1 : Pregnant = Yes A 2 : Pregnant = No B: Heat = No By use of Bayes theorem we have P(A 1 B) = and P(A 2 B) = P(B A 1 )P(A 1 ) P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ) = = 0.71 P(B A 2 )P(A 2 ) P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ) = = 0.29 In this exercise you are asked to: 1. Download and save the example file PregnantCow.xbn to disk (make sure that the extension.xbn is kept). Open the file in the Esthauge LIMID software system. 2. Double-click each of the nodes and see that a node is described by a (unique) name, a descriptive label, the number of st states and a list of descriptive state labels. In general it is not recommended to edit the name of a node - just accept the name given by the system. Instead the label is used to add a description that makes sense to humans. 3. Right-click on each of the nodes and select the menu Edit node table. Compare the tables displayed with Tables E11.1 and E Compile the model, open the monitor windows, and enter evidence for Heat = No as shown in Figure E11.2. What is the probability that the cow is pregnant given that we observed the cow, but concluded that it was not in heat?

48 Mastitis diagnosis and d-separation 5. Re-label the node Heat to Heat 1 and add an additional node labeled Heat 2 in order to model that the cow is observed for heat during 2 consecutive three-week periods. Add also a directed edge from Pregnant to Heat Assume that the second heat detection test has the same sensitivity and specificity as the first one (i.e. as it appears from Table E11.2). Enter the values in the table belonging to Heat Compile the model again and open monitor windows for all three nodes. 8. Enter evidence for Heat 1 = No. What is the probability, that the cow is pregnant? Enter evidence for Heat 2 = No. What is now the probability, that the cow is pregnant? 9. Use Bayes theorem to calculate manually the probability that the cow is pregnant given the evidence that both heat detections are negative. Do you agree with the net? An extended example Pregnancy.xbn (referring to a sow) with three consecutive heat observations and a pregnancy diagnosis is available as an example. If you have time, you can download it and open it in the software Mastitis diagnosis and d-separation In this exercise we work with the small mastitis diagnosis example 1 mastitis.xbn. The net is shown in Figure E11.3. The network has been designed with an automatic milking system with online measurement of conductivity and temperature in mind. The variables Previous cases, Milk yield and Mastitis index (breeding value) are risk factors for Mastitis. The variables Conductivity and Temperature are symptoms. The variable Heat is an other possible explanation of slightly increased temperature. Study the structure of the model carefully, and identify: 1. Variables that are d-separated given observation of a node (i.e. variables that become independent given observation of a node). Verify your findings by actually entering evidence. 2. Examples of converging connections where two nodes become dependent if a common child is observed. Verify your findings by actually entering evidence. 3. Examples of convergent connections where two nodes become dependent if a common descendent is observed. Verify your findings by actually entering evidence. 1 Originally created by 2 students as part of their Advanced Herd Management course.

49 11.3 Johnson s insemination strategy 49 Figure E11.3: The mastitis diagnosis network Johnson s insemination strategy Michael Höhle In this exercise we will use a Bayesian Network to perform calculations on the number of pregnant sows obtained by some fixed insemination strategy. Johnson is a farmer, who according to his week-schedule needs 10 pregnant sows each week. Assuming a pregnancy rate of 80% he decides to inseminate 12 sows each week. The network insem.xbn contains a Bayesian Network reflecting the above situation and contains the following nodes. HLvl The pregnancy herd level. For Johnson it is 80%. P i Pregnancy state of sow no. i. It can either be non-pregnant (state=0) or pregnant (state=1). S Variable counting the number of pregnant sows, i.e. S = 12 i=1 P i. 1. Download the network and open it in the Esthauge LIMID editor. Have a look at the nodes and conditional probability tables to get an impression of what the network does. 2. Compile the model and open monitor windows for the nodes HLvl and S. 3. Use a herd level of 80% and calculate P (S) in this case. Have also a look at the mean and standard deviation. 4. What impact would a large Var(S) have for the farmer?

50 Johnson s heat detection 5. Which known discrete probability distribution is S following? 6. Try to play around with different herd levels Johnson s heat detection Michael Höhle After approximately 21 days, sows that are not pregnant will get into heat again. The problem for Johnson is to detect this heat as he has better things to do than running around in the gestation house. To illustrate the consequences for him, use the Bayesian Network insem-heat.xbn, which extends the former network with the following nodes. HDQ Heat detection quality, can either be good or bad. HD i A variable indicating whether Johnson detects sow i to be in heat or not. Download the file and keep using a herd level pregnancy rate of 80%. 1. What is the sensitivity and specificity of the heat detection test in the HDQ = good case? Note: Specificity is defined as the probability of correctly detecting a truly non-pregnant sow to be in heat. Sensitivity is the opposite, i.e. correctly detecting no heat for a truly pregnant sow. Are the probabilities reasonable? Compare with the HDQ = bad case. 2. Assume that Johnson, using the low quality heat detection scheme, manages to detect two sows in heat. For the remaining 10 he does not detect heat. What is P (S e) in this case? 3. What would P (S e) be had he used the high quality scheme instead? Interpret the results. 4. Johnson also has the possibility to perform an ultra-sound check of each sow, e.g. if he did not detect heat. Let the sensitivity and specificity of the ultrasound test be 95% and 80%, respectively. Add UST i nodes to the diagram incorporating the outcome of this additional test for each sow. 5. Johnson tested the 10 sows not showing heat (using HDQ = bad) and got 10 positive outcomes of the pregnancy tests. How does P (S e) look now? 6. Try to reduce the complexity of the S node in the network by using the modeling trick divorcing.

51 11.5 Use of a Bayesian network to model litter sizes Use of a Bayesian network to model litter sizes The general multivariate normal distribution Litter size in sheep is of course a discrete variable, but in this assignment we shall assume that litter size at weaning X n of parity n is determined by an underlying continuous variable Y n, so that 0, Y n 0.5 X n = 1, 0, 5 < Y n 1.5 (E11.1) 2, 1.5 < Y n We shall furthermore assume that (Y 1, Y 2,..., Y 10 ) has a 10 dimensional normal distribution N (µ, Σ), where for Danish Texel, and, (in excerpt) Σ = µ = (E11.2) (E11.3) Thus, we assume litter sizes to be correlated over time for the same ewe. In Figure E11.4 the general model is shown as a Bayesian network Effect of genotype An underlying reason for the observed correlation is the genetic properties of the ewe. Taking the genotypes of the ewes into account, we may formulate the following revised litter size model for parity n of ewe i: Y in = µ n + A i + ɛ ni, (E11.4)

52 Use of a Bayesian network to model litter sizes Figure E11.4: A model for litter size over the first 10 parities in sheep assuming a multidimensional normal distribution.

53 11.5 Use of a Bayesian network to model litter sizes 53 where µ n is the average litter size for parity n, A i N (0, σa 2 ) is the random effect of ewe, and ɛ ni N (0, σn) 2 is a random residual. It should be noticed, that the model (E11.4) is not in any way in conflict with the original assumption of a multidimensional normal distribution. Exercise 1: As it is seen from Eq. (E11.3) we assume for n > 2 that Var(Y in ) = 0.16 and Cov(Y i,n 1, Y in ) = If those values are true for the model (E11.4), what are then (for n > 2) the values of σ 2 A and σ2 n? Exercise 2: Draw a Bayesian network corresponding to the litter size model (E11.4). Exercise 3: Specify the conditional probability distribution for (Y i4 A i ). Calculate the probabilities of having 0, 1 and 2 lambs given that A i = 0.04, A i = 0.00 and A i = 0.04 (average litter size at parity 4 is E(Y i4 ) = 1.5 as it is seen in Eq. (E11.2)) Effect of body condition score (BCS) Practical experience show that the body condition score at mating has a considerable effect on litter size. According to Ranvig (1998), a body condition score of 3.5 will on average improve the litter size by 0.3 lamb compared to a score of only 2.5. Taking BCS into account, we may extend the litter size model accordingly. The litter size for parity n of ewe i may then be described as: Y in = µ n + A i + B in + ɛ ni, (E11.5) where B in is the body condition score at mating of ewe i before parity n. It is initially assumed that the body condition scores of a ewe are independent over parities. Exercise 4: Draw a Bayesian network corresponding to the litter size model (E11.5). (You are only supposed to draw the structure - not to define the parameters). Exercise 5: Describe in what way observation of body condition scores will influence our belief in the genetic level of the ewe Effect of correlated body condition scores Assume now, that the body condition scores B i1, B i2,..., B i10 form a stationary first order autoregressive time series with mean B, i.e. that for a ewe i: B in = B + α(b i,n 1 B) + e in, (E11.6) where e in N (0, σ 2 e), and 0 α 1.

54 Use of a Bayesian network to model litter sizes Exercise 6: Draw a Bayesian network corresponding to the litter size model (E11.5) where the body condition scores are dependent over time as described by the model (E11.6) (You are only supposed to draw the structure - not to define the parameters).

55 Chapter 12 Decision graphs For these exercises you will need the Esthauge LIMID Software System. Furthermore, you will need the XML files with the Bayesian network shoulder.xbn and the decision graph shoulderdg.xbn Background and objective of the study If a shoulder lesion of grade 3 or 4 is found at the post-mortem examination at the abattoir, the finding is reported to the police, and the sow farmer will typically receive a fine. The result of the post-mortem examination is, of course, not known in advance when the sow is still in the herd. The farmer therefore has to rely only on clinical signs when the destiny of a sow is decided (destruction, treatment, slaughter). In an abattoir survey Strathe (2007) studied the relation between ante-mortem clinical signs and post-mortem registrations of shoulder lesions in sows. In total, 775 sow shoulders were examined before and after slaughter. One of the objectives of the study was to construct a Bayesian network for prediction of notifiable shoulder lesions based on relevant risk factors and clinical signs. Furthermore, it was the hope of the author that it would be possible to identify combinations of simple clinical signs as markers of pathological shoulder lesions The Bayesian network Risk factors The only risk factor considered in the study was the body condition score (BCS), which was observed on the usual integer scale from 1 to 5 (where 5 is very fat). In the Bayesian network only two states were considered: Thin indicating a BCS of 1 or 2, and Normal/fat for BCS 3.

56 The Bayesian network Clinical signs The following clinical signs were registered ante-mortem: Skin : Normal, Crust or Scar. Erythema (i.e. red skin ): Yes or No. Swelling : None, Soft or Firm. Adherence to the tuber: Yes or No. Diameter of crust/scar : Measured in mm by callipers. Diameter of swelling : Measured in mm by callipers. If both a swelling and a crust/scar were observed, only the diameter of the swelling was registered Hypothesis variables Basically, only one hypothesis variable is of interest: The grading of the shoulder lesion with 5 states: Grade 0 (no lesion), Grade 1, Grade 2, Grade 3 and Grade 4. Grades 3 and 4 are notifiable Structure of the network The diagnostic network for shoulder lesions was defined according to the usual principles, i.e. with a directed edge from the risk factor to the hypothesis variable and directed edges from the hypothesis variable to each of the clinical signs ( symptoms ). The resulting Bayesian network is shown in Figure E12.1. In addition to the straight forward edges, the edges into the two diameter nodes need explanation: D Swell. (diameter of swelling): This variable is only relevant if a swelling exist. In case of no swelling, it has has the value (state) Not relevant with probability 1. D Skin (diameter of crust/scar): This variable is only relevant if 1) a crust or scar exists and 2) if there is no swelling. If there is no crust/scar or there is a swelling, this diameter has the value (state) Not relevant with probability Conditional probabilities All conditional probabilities of the network have been estimated from the data of the abattoir survey study by Strathe (2007). The value of the hypothesis variable Grading was observed through post-mortem examination of the sows previously examined ante-mortem. For details about the estimation technique reference is made to Strathe (2007).

57 12.2 The Bayesian network 57 Figure E12.1: The shoulder lesion network Assignment: Bayesian network The following assignment is based on the Bayesian network provided as shoulder.xbn. 1. We may regard all nodes (except the Grade hypothesis variable) as test decisions (they don t influence anything). What is the obvious intervening action to consider as a consequence of the results of the test decisions? 2. Add three new nodes to the network in order to model the intervening action: The action itself, the consequence (as a categorical outcome) and the consequence of the categorical outcome as a numerical value (in money). Use ordinary (yellow) chance nodes for the first two variables and a (purple) utility node for the monetary outcome. Make realistic assumptions concerning the monetary consequences. 3. Identify (and describe) combinations of clinical signs (and BCS) where you recommend the farmer to send the sow to the slaughterhouse. 4. Identify (and describe) combinations of clinical signs (and BCS) where you recommend the farmer not to send the sow to the slaughterhouse.

58 The shoulder lesion problem as a decision graph Figure E12.2: The shoulder lesion network as a decision graph The shoulder lesion problem as a decision graph The basic model without information edges In Figure E12.2 the Bayesian network has been changed to a decision graph with one intervening decision. The decision graph is available in the file shoulderdg.xbn. As the model is presented in Figure E12.2, the decision is global (i.e. the same decision is chosen no matter what clinical signs are observed. In order to run an optimization we must first compile the model and afterwards select the Net Single Policy Updating menu item. For the basis model, the optimal policy and its consequences may be illustrated as in Figure E12.3. In the figure, the monitor windows for the action, the categorical consequence and the monetary consequence are shown for the optimal policy. It is seen that the optimal decision is to slaughter the sow. From the Conseq. window we see that this in 1.29% of the cases results in a fine. The expected revenue is DKK as it is seen from the Money monitor window. The optimal policy is seen by rightclicking the Action node and afterwards selecting the Edit node table option from the popup menu. As expected, the optimal action (when no information is available) is to slaughter the sow Value of information from BCS and clinical signs Adding information edges to the graph will enable us to investigate the value of information from each of the clinical signs. The procedure is as follows:

59 12.3 The shoulder lesion problem as a decision graph 59 Figure E12.3: The optimal policy and its consequences. Add the desired information edge(s) into the action node (i.e. from the BCS and/or the clinical signs. Delete any undesired information edges. Compile the model. Run Single Policy Updating. Read the expected utility, U, from the Money monitor window. The value of the information provided by the selected clinical signs is simply U Assignment: Value of information 5. Determine (one by one) the economic value of information from the BCS node and the clinical sign nodes. For each of the variables, enter the results in the table below (the results for the BCS node have already been entered). 6. Determine the value of using all information (edges from BCS and all clinical signs). 7. Inspect the optimal policy (using all information). 8. Is the utility node a fair representation of a sow farmer s utility function?

60 The pig breeders problem Node Exp. utility Value Killed (%) Slaughtered (%) Fine (%) BCS D. Swell. Swell. D. Skin Skin Eryth. Adhere All variables 12.4 The pig breeders problem Previous exercises will have introduced the software system when used for Bayesian networks and single-action Decision Graphs. We will now introduce more advanced Decision Graphs. Here we use an example by Lauritzen and Nilsson (2000): A pig breeder is growing pigs for a period of four months and subsequently selling them. During this period the pigs may or may not develop a certain disease. If the pig has the disease at the time it must be sold, the pig must be sold for slaughtering, and its expected market price is then 300 DKK (Danish kroner). If it is disease free, its expected market price as a breeding animal is 1000 DKK. Once a month, a veterinary doctor sees the pig and makes a test for presence of the disease. If the pig is ill, the test will indicate this with probability 0.80, and if the pig is healthy, the test will indicate this with probability At each monthly visit, the doctor may or may not treat the pig for the disease by injecting a certain drug. The cost of an injection is 100 DKK. A pig has the disease in the first month with probability A healthy pig develops the disease in the subsequent month with probability 0.20 without injection, whereas a healthy and treated pig develops the disease with probability 0.10, so the injection has some preventive effect. An untreated pig that is unhealthy will remain so in the subsequent month with probability 0.90, whereas the similar probability is 0.50 for an unhealthy pig that is treated. Thus spontaneous cure is possible, but treatment is beneficial on average. 1. Identify the variables (including decisions and utilities) of the pig breeder example. 2. Identify the causal relations between the variables.

61 12.5 The milk test problem Build the decision graph as a LIMID using the Esthauge LIMID software system. 4. Define a decision node ordering to be used in the optimization. 5. Calculate a strategy for medical treatment of a pig. Notice the expected utility of the near-optimal solution. 6. Add edges to the LIMID until it satisfies the "no forgetting" assumptions (be careful only to add edges into decision nodes!) 7. Check for solubility and determine the minimal reduction. Calculate an optimal solution. 8. Compare the expected utility with the one obtained in The milk test problem Michael Höhle and Nils Toft Consider the milk test problem from Jensen (2001) Milk from a cow may be infected. To detect whether the milk is infected, you have a test, which may give either a positive or a negative test result. The test is not perfect. It may give a positive result on clean milk as well as a negative result on infected milk. 1. Assume that the test has sensitivity (Se = P(T + I+) = 0.99) and specificity (Sp = P(T I ) = 0.99). To estimate P(I+) we need to make a couple of assumptions, because the farm which we seek to apply the test to, has no experience with actually testing the milk from each cow with a perfect test. Define λ = P(I+), then we assume that: 1) the daily λ is constant for all cows; 2) outbreaks of infected milk for the cows on the farm are independent. Assume that the farm in question has 50 cows and that milk from all 50 cows is poured into a container and transported to the dairy, where the milk is tested with a perfect test. The farmer s experience is that on average the milk is reported to be infected once a month, i.e., on 29 out of 30 days the milk from all 50 cows is clean. This gives: which yields the estimate λ = 1 (1 λ) 50 = ( ) 29 1/ Based on the above information, construct a Bayesian network for the milk test for the farm with 50 cows as defined above.

62 Adding a second milk test 2. At the dairy, the container with the milk from the farm is tested with a perfect test and if the milk is infected it is thrown away and the farmer receives no money. Milk from each cow is, if uninfected, worth $2. A natural application of the test is to use it to test the milk from the individual cows before pouring it into the container. If the test is positive, the milk from the cow is thrown away. The cost of the test is 6 cents. Assume the milk from the other 49 cows is clean. If the farmer pours the milk from the last cow into the container he will gain $100 if it is not infected and he will gain nothing if the milk is infected. If he throws the milk away, he will gain $98 no matter the state of the milk from the last cow. Based on this information, augment the Bayesian network from the previous question to a decision graph (as an ID) representation of the decision scenario described above, i.e., test and subsequently throw away test-positive milk. 3. Is the test worth using? 4. What is the maximum cost of the test, where it should still be used? 12.6 Adding a second milk test Michael Höhle and Nils Toft 1. Assume that there is another test available, with Se=Sp=0.999, but that this test costs 20 cents. Extend the ID from the previous exercise with this additional test by assuming that the order of the tests is the cheap test from the previous exercise followed by the new test. 2. Find, the strategy for combining the two tests which yields the maximum expected utility Soluble LIMIDs 1. In the lecture, one of the slides displayed the decision graph of Figure E12.4. As we concluded, the decision graph is not a Markov decision process, and interpreted as a LIMID, it was not soluble. Change the figure, so that the LIMID becomes soluble (even though it is still not a decision graph.

63 12.7 Soluble LIMIDs 63 Figure E12.4: A decision graph for sheep replacement given observed litter sizes (compare with Bayesian network of third mandatory report).

64 Soluble LIMIDs

65 Chapter 13 Dynamic programming and Markov decision processes 13.1 Optimization algorithms using Excel The primary aim of this computer exercise session is to become familiar with the two most important optimization algorithms for Markov decision processes: Value iteration and Policy iteration. Both exercises deal with the (very) simple dairy cow replacement model presented in Section Value Iteration In the textbook, value iteration is described in Section , where Eq. (13.6) specifies the functional equations under the criterion (13.3) maximizing the expected present value, and Eq. (13.9) provides the corresponding functional equations under criterion (13.4) maximizing average rewards per stage (year). Under this criterion m d i = 1 for all i and d, so the equations simplify to f i (n) = max d rd i + u p d ijf j (n 1). j=1 (E13.1) Your job in this exercise is to implement value iteration under one of these criteria of your choice. Now, let us define the function fi 1 (n) as u fi 1 (n) = ri 1 + β p 1 ijf j (n 1) j=1 (E13.2) and the function fi 2 (n) as

66 Optimization algorithms using Excel Figure E13.1: The parameters of the simple dairy cow replacement model as they have been entered into the spreadsheet. f 2 i (n) = r 2 i + β u p 2 ijf j (n 1). j=1 (E13.3) Since we only have two decisions (d = 1: Keep and d = 2: Replace ), we notice that the functional equations (13.6) may be written as f i (n) = max d rd i + β u p d ijf j (n 1) = max { fi 1 (n), fi 2 (n) }. j=1 In other words, implementing value iteration is a question of calculating the value of fi 1(n) and f i 2 (n) for all states at all iteration stages, and, at each iteration to select the maximum value. The optimal decision, d, for state i at iteration n is simply identified as the decision satisfying f i (n) = f d i (n). We will use the spreadsheet MarkovIntro.xls, which should be saved to disk and opened in Excel. The sheets of the spreadsheet contain lots of numbers and formulas, but in general, cells that are used for basic input are green, whereas a pink cell contains an important result. The sheet called Value iteration has been prepared for the implementation. The parameters of the simple dairy cow replacement model have already been entered into the spreadsheet as it is shown in Figure E13.1. Comparing with Tables 13.1 and 13.3 will make it easy to identify the various parameters. It is recommended that you leave the transition probabilities as they are, whereas you are welcome to edit the rewards and/or the interest rate as you like (those parameters appear in green cells). When you have implemented the algorithm, the optimal policy under infinite planning horizon is returned in the pink cells K10-K12 (it is actually 100 iterations that we consider as infinite ). What you need to do in order to finish the implementation is to enter the necessary formulas for calculation of fi 1(n) and f i 2 (n). Columns C-D, H-I and M-N

67 13.1 Optimization algorithms using Excel 67 are left empty for that purpose. Assignment: 1. Calculate the values of the functions f 1 1 (n), f 2 1 (n), f 1 2 (n), f 2 2 (n), f 1 3 (n), f 2 3 (n) for n = 2,..., Compare the results with Table 13.4 or 13.5 (depending on the criterion of optimality). If you use the original interest rate and rewards, the results should be exactly as in the table. 3. Select the tab labeled Plot, value it. and study the convergence of the value functions f i (n). Does the plot confirm Eq. (13.7) or (13.10) (depending on the criterion of optimality). 4. This question only makes sense under criterion (13.3): What are the converged values f i ( ) for i = 1, 2, 3? For practical purposes, you may just assume that f i ( ) f i (100). Try to explain in words how the value f i ( ) should be interpreted. What is the interpretation of the difference f 2 ( ) f 1 ( )? What is the effect of changing the interest rate? 5. This question only makes sense under criterion (13.4): What is the average rewards per stage (year) under the optimal policy? What is the interpretation of the difference f 2 (n) f 1 (n) for large n? Policy iteration In the textbook, policy iteration is described in Section , where Table 13.6 contains the necessary formulas for implementation of the algorithm under three different criteria. In this exercise, we shall only implement Criterion (13.3). Implementation of the policy iteration algorithm is a little more complicated than value iteration, because each iteration involves solution of a set of linear simultaneous equations as shown in Table Define the vectors f 1 = f 1 1 f 1 2 f 1 3, f 2 = f 2 1 f 2 2 f 2 3, r 1 = r 1 1 r 1 2 r 1 3, r 2 = r 2 1 r 2 2 r 2 3, and the matrices P 1 = p 1 11 p 1 12 p 1 13 p 1 21 p 1 22 p 1 23 p 1 31 p 1 32 p 1 33, P 2 = p 2 11 p 2 12 p 2 13 p 2 21 p 2 22 p 2 23 p 2 31 p 2 32 p 2 33 Thus, f 1, r 1 and P 1 relate to the decision Keep, and f 2, r 2 and P 2 to the decision Replace. For a given policy s, we define the corresponding vectors f s and r s and.

68 Optimization algorithms using Excel the matrix P s as constructed in such a way that each row is taken from either the corresponding vector/matrix for keep or for replace. If, for instance, s(1) = 1, s(2) = 1 and s(3) = 2, we have f s = f 1 1 f 1 2 f 2 3, r s = r 1 1 r 1 2 r 2 3, P s = p 1 11 p 1 12 p 1 13 p 1 21 p 1 22 p 1 23 p 2 31 p 2 32 p For any policy, s, the linear equations for Criterion (13.3) in Table 13.6 may be written in matrix notation as f s = r s + βp s f s Solving for f s gives f s = (I βp s ) 1 r s, where I is the identity matrix. The sheet called Policy iteration has been prepared for the implementation. Again, the parameters of the simple dairy cow replacement model have already been entered into the spreadsheet as it was shown in Figure E13.1. For convenience, a calculator capable of solving systems of linear equations has already been implemented in the spreadsheet. The calculator works as follows (with reference to Figure E13.2): 1. At the top (labeled 1 ), the original vectors r 1 and r 2 as well as the matrices P 1 and P 2 have been specified (feel free to edit the rewards). These values do not change during use. 2. The desired policy, s, is specified at 1 (green cells). 3. The resulting vector, r s, and matrix, P s, are instantaneously created in the middle (labeled 2 ). 4. Intermediate results are shown at The resulting f s vector is returned at 4 (pink cells). The calculator is used manually each time it is needed (i.e. once for each iteration). Assignment: 1. Enter an initial arbitrary policy in Cells B30-D Solve the equations for the policy chosen (i.e. enter the policy in Cells K10- K12 and read the results in Cells K22-K24.

69 13.2 Optimization algorithms using R Figure E13.2: The parameters of the simple dairy cow replacement model as they have been entered into the spreadsheet. 3. Write the solution in Cells E30-G30 (don t copy-paste - just write the results using the keyboard). 4. Enter formulas for the expressions specified in Table 13.6 for Criterion (13.3) in Cells H30-M The maximum is automatically identified for each state, and a new policy to be used for the next iteration is automatically identified and displays in Cells B31-D Continue through the iterations until the same policy is found in two consecutive iterations Compare the results with the corresponding results from the value iteration method. Is f s for the optimal policy equal to (f 1 ( ), f 2 ( ), f 3 ( )) from value iteration under the same criterion? Are the optimal policies identical Optimization algorithms using R The primary aim of this computer exercise session is to become familiar with the two most important optimization algorithms for Markov decision processes: Value iteration and Policy iteration. Both exercises deal with the (very) simple dairy cow replacement model presented in Section When solving the exercise in R it will be beneficial to use matrix notation. Define the vectors

70 Optimization algorithms using R f 1 = f 1 1 f 1 2 f 1 3, f 2 = f 2 1 f 2 2 f 2 3, r 1 = r 1 1 r 1 2 r 1 3, r 2 = r 2 1 r 2 2 r 2 3, and the matrices P 1 = p 1 11 p 1 12 p 1 13 p 1 21 p 1 22 p 1 23 p 1 31 p 1 32 p 1 33, P 2 = p 2 11 p 2 12 p 2 13 p 2 21 p 2 22 p 2 23 p 2 31 p 2 32 p 2 33 Thus, f 1, r 1 and P 1 relate to the decision Keep, and f 2, r 2 and P 2 to the decision Replace. For a given policy s, we define the corresponding vectors f s and r s and the matrix P s as constructed in such a way that each row is taken from either the corresponding vector/matrix for keep or for replace. If, for instance, s(1) = 1, s(2) = 1 and s(3) = 2, we have. f s = f 1 1 f 1 2 f 2 3, r s = r 1 1 r 1 2 r 2 3, P s = p 1 11 p 1 12 p 1 13 p 1 21 p 1 22 p 1 23 p 2 31 p 2 32 p (E13.4) The file markov-1.r contains R code defining the matrices involved in the model: ## First we define our matrices and other parameters # The KEEP transition matrix \\ P1 = matrix(c(0.6, 0.2, 0.1, 0.3, 0.6, 0.3, 0.1, 0.2, 0.6), 3) # The REPLACE transition matrix \\ P2 = matrix(c(1/3, 1/3, 1/3, 1/3, 1/3, 1/3, 1/3, 1/3, 1/3), 3) # The KEEP reward matrix (column vector) \\ r1 = matrix(c(10000, 12000, 14000), 3) # The REPLACE reward matrix (column vector)\\ r2 = matrix(c(9000, 11000, 13000), 3) # The format of a policy - this policy says KEEP ("1") in all states policy = c(1, 1, 1) # The discount factor. Under Criterion (13.4) beta must be 1.00\\ beta = 0.85 You should particularly notice that a policy is defined as a vector of which each element specifies the decision for the corresponding state. Thus, the policy (1, 1, 2) says Keep for States 1 and 2 and Replace for State 3. A function, getparms has also been predefined. It takes a policy vector and two matrices as arguments, and it returns a matrix of which some rows are taken from the first matrix argument and some from the second matrix. The ith row of the returned matrix is the is the ith row of the first matrix if the ith element of the

71 13.2 Optimization algorithms using R 71 policy vector is 1. If the latter element is 2, the ith row is taken from the second matrix. In other words, the function returns mixed matrices as those in Eqs. (E13.4). The function is defined as follows: # For a given policy, this function creates a parameter matrix by picking # rows from the KEEP and REPLACE matrices, respectively getparms = function(policy, m1, m2) { j = length(m1) res = matrix(rep(0, j), 3) for (i in 1:length(policy)) { if (policy[i]==1) { res[i,] = m1[i,] } else { res[i,] = m2[i,] } } return(res) } If we define the policy (1, 1, 2) and call getparms using this policy and the two transition matrices P 1 and P 2, R will respond as follows by picking 1st and 2nd row from P 1 and 3rd row from P 2 : > policy = c(1, 1, 2) > getparms(policy, P1, P2) [,1] [,2] [,3] [1,] [2,] [3,] > Value Iteration In the textbook, value iteration is described in Section , where Eq. (13.6) specifies the functional equations under the criterion (13.3) maximizing the expected present value, and Eq. (13.9) provides the corresponding functional equations under criterion (13.4) maximizing average rewards per stage (year). Under this criterion m d i = 1 for all i and d, so the equations simplify to f i (n) = max d rd i + u p d ijf j (n 1). j=1 (E13.5) Your job in this exercise is to implement value iteration under one of these criteria of your choice. Now, let us define the function fi 1 (n) as u fi 1 (n) = ri 1 + β p 1 ijf j (n 1) j=1 (E13.6) and the function fi 2 (n) as

72 Optimization algorithms using R u fi 2 (n) = ri 2 + β p 2 ijf j (n 1). j=1 (E13.7) If we define the vector functions f 1 (n) = (f1 1(n), f 2 1(n), f 3 1(n)) and f 2 (n) = (f1 2(n), f 2 2(n), f 3 2(n)) we notice that Eqs. (E13.6) and (E13.7) may be written in matrix notation as f 1 (n) = r 1 (n) + βp 1 f(n) (E13.8) and f 2 (n) = r 2 (n) + βp 2 f(n) (E13.9) Since we only have two decisions (d = 1: Keep and d = 2: Replace ), we notice that the functional equations (13.6) may be written as f i (n) = max d rd i + β u p d ijf j (n 1) = max { fi 1 (n), fi 2 (n) }. j=1 In other words, implementing value iteration is a question of calculating the value of fi 1(n) and f i 2 (n) for all states at all iteration stages, and, at each iteration to select the maximum value. The optimal decision, d, for state i at iteration n is simply identified as the decision satisfying Assignment: f i (n) = f d i (n). 1. Write an R function determinepolicy taking a vector f, the two matrices P 1 and P 2, the two vectors r 1 and r 2, and the scalar β as arguments and returning a new policy determined by Eq. (13.6). The new policy must be a vector of decisions as for instance (1, 2, 2) for the policy Keep in State 1 and replace in States 2 and 3. You may for instance use the template below: # For given value vector f, KEEP matrices P1 and R1, REPLACE matrices P2 and R2 # and discount factor beta, a new policy is defined as in Eq. (13.6) determinepolicy = function(f, P1, P2, R1, R2, beta) { res = rep(1, length(f)) # Create a vector for the resulting policy... (your own code)... } return(res) # Return the policy vector 2. Write an R function determinevalue taking the same arguments as the previously defined determinepolicy supplemented by a policy and returning an updated value of the vector f as expressed by Eq. (13.6). You may for instance use the template below:

73 13.2 Optimization algorithms using R 73 # Having defined the policy, this function returns the corresponding values # i.e. the vector of f-values at the left hand side of Eq. (13.6) determinevalue = function(f, P1, P2, R1, R2, beta, policy) {... (your own code)... } return(res) # Return value 3. By use of the two functions defined above, it is now rather trivial to define an R function valueiteration taking a predefined number of iterations as argument together with the parameter matrices P 1, P 2, r 1, r 2 and the discount factor β. The function returns as output a table of optimal policies and optimal values like those shown in Table 13.4 or 13.5 (depending on the criterion of optimality). It may for instance be specified as follows: # Performs value iteration over ite iteration for the parameters specified valueiteration = function(ite, P1, P2, R1, R2, beta) { res = matrix(rep(0, (ite*length(r1)*2)), ite) # Create a matrix for the results f = rep(0, length(r1)); # Initial values: zero for (n in 1:ite) { # Iterate over stages from 1 to ite policy = determinepolicy(f, P1, P2, R1, R2, beta) # Find the policy of this stage f = determinevalue(f, P1, P2, R1, R2, beta, policy) # Find the corresponding values line = rep(0, length(r1)*2) # Create a line in the result table for (j in 1:length(R1)) { # corresponding to table 13.4 j1 = ((j-1)*2+1) # For each state we specifyt j2 = j *2 # line[j1] = policy[j] #... first the decision line[j2] = f[j] #... then the value } res[n,] = line # Insert the line in the result table } return(res) # Return the table } Try to understand what goes on in the code. The returned table has the same format as Tables 13.4 and Run the valueiteration function with 100 iterations and print the results. 5. Compare the results with Table 13.4 or 13.5 (depending on the criterion of optimality). If you use the original interest rate and rewards, the results should be exactly as in the table. 6. Create plots where you show the convergence of f 1 (n), f 2 (n) and f 3 (n) for n = 1,..., 100. Do the plots confirm Eq. (13.7) or (13.10) (depending on the criterion of optimality). 7. This question only makes sense under criterion (13.3): What are the converged values f i ( ) for i = 1, 2, 3? For practical purposes, you may just assume that f i ( ) f i (100). Try to explain in words how the value f i ( ) should be interpreted. What is the interpretation of the difference f 2 ( ) f 1 ( )? What is the effect of changing the interest rate? 8. This question only makes sense under criterion (13.4): What is the average rewards per stage (year) under the optimal policy? What is the interpretation of the difference f 2 (n) f 1 (n) for large n?

74 Optimization algorithms using R Policy iteration In the textbook, policy iteration is described in Section , where Table 13.6 contains the necessary formulas for implementation of the algorithm under three different criteria. In this exercise, we shall only implement Criterion (13.3). Implementation of the policy iteration algorithm is a little more complicated than value iteration, because each iteration involves solution of a set of linear simultaneous equations as shown in Table For any policy, s, the linear equations for Criterion (13.3) in Table 13.6 may be written in matrix notation as Solving for f s gives where I is the identity matrix. Assignment: f s = r s + βp s f s f s = (I βp s ) 1 r s, (E13.10) 1. Write an R function findpolicyvalue taking a specified policy and the parameters of the problem as arguments and returning the vector f s according to Eq. (E13.10). You may for instance use the template below: # Find the values for a specified policy and model parameters by # solving the equations of table 13.6 (criterion (13.3)) findpolicyvalue = function(policy, P1, P2, r1, r2, beta) { P = getparms(policy, P1, P2) # Define the transition matrix from the policy r = getparms(policy, r1, r2) # Define the reward matrix from the policy }... (your own code) The previously defined function determinepolicy may be used for maximization of the expression mentioned in the rightmost column of Table Use that function and the findpolicyvalue function to finish the policyiteration function below: # Perform policy iteration until convergence using the specified initial policy policyiteration = function(policy, P1, P2, r1, r2, beta) { policy2 = policy policies = policy2 # A matrix to store the successive policies policy1 = c(-1, -1, -1) # Make sure that policy1 is different # Continue as long as the policy differs from the previous: while(length(policy2[policy2 == policy1]) < length(r1)) { policy1 = policy2 # Use the policy from previous iteration # Solve the matrix equation and find the values... (your own code)... # Determine an improved policy (policy2) # (step 3 of the algorithm) policies = rbind(policies, policy2) } return(policies) } # Add the most recent policy # Return the matrix of policies In order to have the values, we also need a function returning the corresponding values. The following function takes care of that: # Calculate the values corresponding to the policies of the iterations findallpolicyvalues = function(policies, P1, P2, r1, r2, beta) { ite = length(policies[,1]) # How many iterations were there?

75 13.3 Hierarchical Markov decision processes 75 res = matrix(rep(0, ite*3), ite) # Create a matrix for the results for (n in 1:ite) { # For each iteration... res[n,] = findpolicyvalue(policies[n,], P1, P2, r1, r2, beta) #... find the values } return(res) # Return the results } 3. Run the policyiteration function using an arbitrary initial policy and print the resulting optimized policies. Run also the findallpolicyvalues function and compare the results with the corresponding results from the value iteration method. Is f s for the optimal policy equal to (f 1 ( ), f 2 ( ), f 3 ( )) from value iteration under the same criterion? Are the optimal policies identical. 4. Create plots where you show the convergence of f 1, f 2 and f 3 over the iterations Hierarchical Markov decision processes The primary aim of the first exercises of this computer session is to make you familiar with the MLHMP software system, which is the program we use for Markov decision processes (dynamic programming). For the last exercises the aim is to illustrate the concepts of hierarchical Markov decision processes, Bayesian updating and Markov chain simulation. In addition to the computer program you will probably need the textbook Kristensen et al. (2010b, Chapter 13). Finally, the example files cow12.hmp, cow36.hmp and simple-cow.hmp as well as some parameter and sow data files are used The MLHMP software A special Java based software for multi-level hierarchical Markov processes is used. It is installed as part of the Advanced Herd Management package. Unfortunately, the users guide corresponds to a previous version of the software, but most of the features described still work in the same way. Open the program by selecting Advanced Herd Management MLHMP - Multi-level Hierarchical Markov Processes from the Windows Start menu (the manual is also opened from the Start menu). The software has three windows referred to as the Process tree, the Optimization log and the Result table. The window to be displayed is selected by clicking one of the tabs just above the button bar (alternatively, the window is selected from the Window menu). The three windows are shown in Figures E13.3, E13.4 and E13.5. The three windows each serve a different purpose: Process tree (Figure E13.3): This is the model browser and editor. It is used for showing (and editing) the structure of the model. Also the numerical content may be browsed and edited from this window.

76 Hierarchical Markov decision processes Figure E13.3: The process tree window. Figure E13.4: The optimization log window.

77 13.3 Hierarchical Markov decision processes 77 Figure E13.5: The result table window. Optimization log (Figure E13.4): When an optimization (or simulation) has been run, the value of the objective function at each iteration step is shown here. The upper half part of the window shows the result of the most recent optimization (or simulation), whereas the lower half part contains a continuous log, which can be saved to disk as a text file. Result table (Figure E13.5): After an optimization, the optimal policy is shown in this table. It can be saved to disk as a comma separated text file that can be imported in e.g. Excel. The table is also able to show the present (or relative) value of each action Basic concepts and model use Before we turn to hierarchical models, we implement the very simple dairy cow replacement model of Section in MLHMP: 1. Select the New empty model item from the New submenu of the "File menu. 2. Enter basic model properties: Model name (e.g. Dairy cow replacement model, 3 states) and interest rate ( ) 1 - leave other fields unchanged. 3. Rename Main stage to Lactation cycle", State 0 to Low milk yield and Action 0 to Keep". 1 This strange value ensures that you get exactly the same results as in the textbook

78 Hierarchical Markov decision processes 4. Left-click the Edit model structure button in order to allow for extending the model. 5. Select the Lactation cycle stage and click the + button twice in order to add two new states. 6. Rename the two new states to Average milk yield and High milk yield", respectively. Rename also the actions defined for the new states to Keep in both cases. 7. Select the three states one by one and press the + button in order to add an additional action in each case. Rename the new actions to Replace" so that each state has a Keep and a Replace action. 8. Press the Build model button in order to create the designed model. 9. Select Parameter values from the right most choice field. 10. Enter the parameter values from Section for each of the 2 x 3 actions. 11. Select Labels only from the right most choice field. 12. Select Probability check from the Run menu in order to ensure that all probability distributions sum to 1. In case of errors, correct them Model comprehension and optimization Having built the model, we are now ready to use it: 1. Select the Discounting criterion of optimality from the left most choice field. 2. Select Value iteration from the Run menu. 3. Select the Optimization log from the Windows menu. Compare the values of the upper table with Table 13.4 of the textbook. What is the interpretation of the figures shown in the table? 4. Select the Results from the Windows menu. Compare with Table Select the Average rewards over time criterion from the left most choice field. 6. Select Value iteration from the Run menu. Compare the values of the upper table with Table 13.5 of the textbook. What is the interpretation of the figures shown in the table? 7. Select the Results from the Windows menu. Compare with Table 13.5.

79 13.3 Hierarchical Markov decision processes Select the Discounting criterion of optimality from the left most choice field. 9. Select Optimization (i.e. policy iteration) from the Run menu. 10. Press the R button. What is the interpretation of the figures shown in the right most column? 11. Press the button without label. What is the interpretation of the figures shown in the right most column? 12. Run in turn optimizations for all three criteria of optimality. In each case: (a) Inspect the Optimization log and make sure that you understand all figures shown in the upper table. (b) View the Results and try to explain why the optimal policy differs (or doesn t differ) between criteria. (c) Press the button without label and try to explain why the values differ between criteria. 13. Close the model Extending the model 1. Open the model cow12.hmp (File Open...). 2. Inspect the model structure in the process tree window. 3. Take a look at some of the parameters. 4. Optimize the model under all three criteria and try to explain the optimal policies and the differences between the criteria. 5. Close the model. 6. Open the model cow36.hmp (File Open...). 7. Inspect the model structure in the process tree window. 8. Take a look at some of the parameters. 9. Optimize the model under all three criteria and try to explain the optimal policies and the differences between the criteria. 10. Remember the optimal average rewards under both average criteria (for Exercise ). 11. Close the model

80 Hierarchical Markov decision processes A hierarchical model 1. Open the model simple-cow.hmp (File Open...). 2. Inspect the model structure in the process tree window. 3. Take a look at some of the parameters (notice the size of the parameter tables). 4. Optimize the model under all three criteria and try to explain the optimal policies and the differences between the criteria. 5. Compare the average rewards values to those calculated for the 36 state simple model. 6. Select one of the average criteria. 7. Optimize the model. 8. Remember the optimal average rewards value (Optimization log: Last row of upper table). 9. Go to the Results window and edit the policy. 10. Select Simulation from the Run menu. 11. Compare the calculated average rewards with the optimal value. 12. Calculate by simulation the average milk yield per cow per year under an optimal policy Reduced sow model with Bayesian updating Short description The sow model was originally developed by Kristensen and Søllested (2004a,b) who present it as follows in the abstracts of the two articles: Several replacement models have been presented in literature. In other applicational areas like dairy cow replacement, various methodological improvements like hierarchical Markov processes and Bayesian updating have been implemented, but not in sow models. Furthermore, there are methodological improvements like multi-level hierarchical Markov processes with decisions on multiple time scales, efficient methods for parameter estimations at herd level and standard software that has been hardly implemented at all in any replacement model. The aim of this study is to present a sow replacement model that really uses all these methodological improvements. In this paper, the biological model describing the performance and feed intake of

81 13.3 Hierarchical Markov decision processes 81 sows is presented. In particular, estimation of herd specific parameters is emphasized. The optimization model is described in a subsequent paper. Recent methodological improvements in replacement models comprising multi-level hierarchical Markov processes and Bayesian updating have hardly been implemented in any replacement model and the aim of this study is to present a sow replacement model that really uses these methodological improvements. The biological model of the replacement model is described in a previous paper and in this paper the optimization model is described. The model is developed as a prototype for use under practical conditions. The application of the model is demonstrated using data from two commercial Danish sow herds. It is concluded that the Bayesian updating technique and the hierarchical structure decrease the size of the state space dramatically. Since parameter estimates vary considerably among herds it is concluded that decision support concerning sow replacement only makes sense with parameters estimated at herd level. It is argued that the multi-level formulation and the standard software comprise a flexible tool and a shortcut to working prototypes. The model has 3 levels (founder and 3 child levels) and the model is an example of integration of a Dynamic Linear Model (DLM) with a hierarchical Markov decision process. Instead of forming a state space as the cartesian product of litter size results (as in the ewe model) an abstract underlying production potential of the sow is estimated (learned) as observations are done over parities. The model has been built for use with herd specific parameters estimated from the herd s own records. Parameters from two different herds are provided in the text files 480.txt and 580.txt. They are loaded from the initial model settings dialog as shown in Figure E13.6. The model is created directly by the MLHMP software system (File New New: Reduced sow model with Bayesian updating). When it is created it installs a separate menu, Sow model, in the menu bar as shown in Figure E13.7. When an optimal policy for a herd has been calculated, the sow records from the herd may be read from a text file by selecting the menu item Read sow herd data. The sow records from both herds are available in the files bes_480.dat and bes_580.dat. Having read the sows, the optimal decisions and future profitabilities of the sows in the herd may be shown by selecting Show herd results. Assignment 1. Load the settings for either herd 480 or herd 580 and create the model. 2. Calculate an optimal policy maximizing average net returns over time:

82 Hierarchical Markov decision processes Figure E13.6: The dialog for editing the settings of the sow model. The settings for a specific herd are loaded by pushing the Load button. Two herd specific settings are provided: 480.txt and 580.txt. Figure E13.7: The Sow model menu installed by the plug-in creating the model.

83 13.4 The ewe replacement model 83 (a) What is the average lifetime of a sow? (b) What is average litter size? (c) What is the distribution of farrowings over parities? (d) What is the average net returns per piglet produced? 3. Take a look at the optimal policy (the Result table ). When should sows be replaced in that herd? 4. Import the sow records (from the same herd as the parameters originate from). 5. Show the results for the individual sows. How should we interpret the numbers shown? 13.4 The ewe replacement model This model has only been developed for teaching. Nevertheless it is rather realistic and it is a good model to start with, because it is at the same time simple and representative for animal replacement models. It is created directly by the MLHMP software system (File New New: Ewe replacement model) Background and problem formulation Litter size in sheep is of course a discrete variable, but in this assignment we shall assume that litter size at weaning X n of parity n is determined by an underlying continuous variable Y n, so that 0, Y n 0.5 X n = 1, 0, 5 < Y n 1.5 (E13.11) 2, 1.5 < Y n We shall furthermore assume that (Y 1, Y 2,..., Y 10 ) has a 10 dimensional normal distribution N (µ, Σ), where for Danish Texel, µ = (E13.12)

84 The ewe replacement model Table E13.1: Economic key figures (DKK) for Danish Texel Revenues & costs Ewe and 0 lamb Ewe and 1 lamb Ewe and 2 lambs Lamb(s) for slaughter Subsidy, EU Feed costs, maintenance Feed costs, embryo Feed costs, lactation Feed costs, lambs and, (in excerpt) Σ = (E13.13) Economical key figures for Danish Texel are presented in Table E13.1. The carcass value of a ewe is assumed to be 231 DKK for 1st parity ewes and 298 DKK for older ewes. For convenience, it is assumed that ewes are only replaced voluntarily (i.e. no forced replacements due to death or disease). The price of a young replacement ewe is assumed to be 700 DKK Assignment 1. Describe the structure of the sheep replacement problem as a 2-level hierarchical Markov process. For the structural description, you must for each level define the stages, states and actions. Notice that in order to (approximately) satisfy the Markov property, a memory variable is defined in the state space of parity n. 2. Describe how to calculate the parameters from the information given in the previous section. For the transition probabilities, use the conditional distribution (Y n+1 Y n 1, Y n ). Describe the principles and show in details the calculation of at least one transition probability from 3rd to 4th parity. Describe also how to calculate the rewards (and show it in details for both actions in at least one state of parity 3). 3. Calculate an optimal replacement policy under the following three criteria of optimality: (a) Maximum present value (discounting).

85 13.4 The ewe replacement model 85 (b) Maximum average net returns per ewe per year. (c) Maximum average net returns per lamb produced. 4. Compare the three optimal policies and explain the differences (if any). 5. For the optimal policy maximizing average net returns per ewe per year, calculate by Markov chain simulation some technical and economical key figures describing the optimal policy. You may for instance calculate: Average lifetime of an ewe. Average number of lambs per ewe per year. Average number of lambs produced over the lifetime of an ewe. The distribution of annual income from the three sources (EU subsidy, lambs sold, ewes slaughtered). 6. Evaluate the model. Is it realistic? Could it be improved? Acknowledgements The information regarding the multi-dimensional normal distribution is due to Jens Noesgaard Jørgensen, and the data in Table E13.1 is provided by Anita Meldgaard Hansen.

86 The ewe replacement model

87 Chapter 14 Monte Carlo simulation 14.1 Modeling a batch of slaughter pigs In this exercise we shall use R to build a Monte Carlo simulation model, Sim- Batch, of a batch of slaughter pigs. In addition to the assignment you will need the R system and the file SimBatch.R with the R code in a deterministic version. The file SimBatchSolution.R contains the R code with stochastic functions. As the name implies the file should be regarded as a possible solution to the exercises Description of the production We shall model an all-in/all-out production. At time 0 a batch consisting of a specified number of piglets are inserted. They are fed ad libitum and when they have reached a certain weight they are sent to a Danish slaughterhouse. The strategy for picking out pigs for slaughter is decided by the farmer. In Denmark the price obtained for a slaughter pig basically depends on the slaughter weight, where the highest price per kg is obtained in an interval from 70 to 84 kg (referred to as the optimal interval in the following). For higher as well as lower slaughter weight the price is reduced proportionally with the deviation from the optimal interval. It is therefore important that as many pigs as possible are delivered with a slaughter weight in the optimal interval. Since the slaughter weight is not observable by the farmer, the decision to deliver pigs for slaughter is made on the basis of observed live weight and observed growth rate. Thus, the decision is made under uncertainty concerning slaughter weight The simulation model Model type SimBatch is a dynamic time stepping model with daily updating of the state of the batch and the pigs. Since it models the batch by modeling each individual pig, it

88 Modeling a batch of slaughter pigs is also a mechanistic model. Initially, the model is deterministic, but as this exercise progresses, it becomes more and more stochastic. Since the basic simulation mechanism is based on drawing random numbers from relevant distributions, the model also classifies as a Monte Carlo simulation model. State of nature The state of nature, Φ 0, is represented as a list of parameters. The following R code defines (and prints) the state of nature. # State of nature information stateofnature = list(initialaverage = 30, initialstddev = 2, averagedg = 0.900, stddevdg = 0.100, autocorrelationdg = 0.99, diseaserisk = 0.01, diseaseeffectave = 0.250, diseaseeffectstd = 0.030, diseasedurationav = 5, livetoslaughterav = 0.76, livetoslaughterstd = 1.4, loweroptimal = 70, upperoptimal = 84, deliverevery = 7) # Average weight at insertion # Std. deviation of weight at insertion # Average daily gain # Std. deviation of daily gain between pigs # Autocorrelation for individual daily gain # Risk of disease any day # Average effect of disease # Standard deviation of disease effect # Average duration of disease # Live weight to slaughter weight conversion # Standard deviation of conversion # Lower bound of optimal slaughter weight # Upper bound of optimal slaughter weight # Days between delivery (7 = weekly, 1 = daily) stateofnature Having defined the state of nature, stateofnature, we can afterwards refer to specific parameters using the $ symbol. For instance, stateofnature$averagedg will give us the average daily gain defined in the state of nature. Decision strategy The decision strategy, Θ, is also represented as a list of decision parameters reflecting the management of the batch. In this simple example they all refer to the delivery policy applied to the batch. The following R code defines (and prints) the decision strategy. # Decision strategy information decisionstrategy = list(thresholdweight = 100, # Intended live weight at slaughter minimumdelivery = 10, # Minimum number of pigs to send to maximumunderweight = 2, # Accept this number of pigs below threshold minimumstock = 50, # Minimum acceptable stock: Send all if below maximumage = 90) # Maximum age before slaughter decisionstrategy Having defined the decision strategy, decisionstrategy, we can afterwards refer to specific parameters using the $ symbol. For instance, decisionstrategy$maximumage will give us the maximum time in the department before slaughter. The interpretation of the decision parameters is as follows:

89 14.1 Modeling a batch of slaughter pigs 89 thresholdweight: Any pig of which the observed live weight exceeds the threshold is selected for potential delivery. minimumdelivery: Due to transportation restrictions pigs are only sent to slaughter if we can send at least this number. If there are enough pigs exceeding the above threshold, they are sent. If the number of pigs exceeding the threshold is smaller than minimumdelivery we may supplement with the heaviest of the pigs below the threshold. maximumunderweight: Only this number of pigs below the threshold is allowed. minimumstock: If the number of remaining pigs is below this value, they are all sent to slaughter independently of their weight. maximumage: If the pigs have been in the department this number of days, they are all sent to slaughter in order to make space for a new batch of piglets. Different strategies may be defined. If we wish to define a strategy with individual delivery and no other restrictions, we can do it like this: # Strategy with completely individual delivery individualstrategy = list(thresholdweight = 100, # Intended live weight at slaughter minimumdelivery = 1, # Minimum number of pigs to send to maximumunderweight = 0, # Accept this number of pigs below threshold minimumstock = 0, # Minimum acceptable stock: Send all if below maximumage = 9999) # Maximum age before slaughter individualstrategy A strategy, were all pigs are slaughtered at the same time could be defined like this: # Strategy with slaughter at a given age agestrategy = list(thresholdweight = 9999, minimumdelivery = 9999, maximumunderweight = 0, minimumstock = 0, maximumage = 90) # Intended live weight at slaughter # Minimum number of pigs to send to # Accept this number of pigs below threshold # Minimum acceptable stock: Send all if below # Maximum age before slaughter agestrategy Modeling a pig Representation of a pig A pig is defined in the model by 4 state variables which are updated every day: 1. The live weight in kg. 2. The gain today in kg (if healthy). 3. An integer, i, defining the status of the pig:

90 Modeling a batch of slaughter pigs i = 2: The pig has been slaughtered. i = 1: The pig is healthy and still present in the herd. i 0: The pig is diseased and will stay diseased i days yet. 4. The effect of the disease (if any) on daily gain. In R, a pig is simply represented by a vector with 4 elements. A valid pig object (vector) could be created as follows: > pig = c(45.0, 0.956, 3, 0.123) > pig [1] The above code generates (and prints) a pig with a live weight of 45.0 kg, a gain until tomorrow of 956 g (if healthy). It is, however, diseased and will stay diseased for 3 days. The effect of the disease is a reduction in daily gain of 123 g. Simple functions used with a pig object The following functions have been defined to be used with a pig object. Initially, they are all deterministic, but during this exercise they are changed in order to reflect random variation through random number generation. They all (except getobservedweight) take the state of nature as parameter. getstartweight(stateofnature): A start weight (in kg) is returned. Initially, it only returns the initial average from the state of nature. getinitialdg(stateofnature): An initial daily gain (in kg) for a pig is returned. Initially, it only returns the average daily gain from the state of nature. ispigdiseased(stateofnature): Returns a logical value. If the pig gets diseased it returns TRUE, otherwise FALSE. Initially, it always returns FALSE. getdiseaseeffect(stateofnature): Returns the reduction (in kg) of daily gain during a disease period. Initially it only returns the average disease effect from the state of nature. getdiseaseduration(stateofnature): Returns the duration (in days) of a new disease. Initially, it only returns the average duration from the state of nature. getslaughterweight(liveweight, stateofnature): Transforms the specified input parameter liveweight to slaughter weight (in kg). Initially it just multiplies the live weight with the conversion ratio from the state of nature.

91 14.1 Modeling a batch of slaughter pigs 91 getobservedweight(liveweight): Returns the observed live weight of a pig with the specified true live weight. Initially, it just returns the true live weight (ignoring the observation error). updatedg(olddg, stateofnature): Returns a new daily gain for the next 24 hours from the specified old daily gain (from the previous 24 hours). Initially, it just returns the old value (without any transform). Complex functions used with a pig object The following functions are used for simulation of a pig over time. It is not necessary to change them during the exercise. The first function creates a new 30 kg piglet: # Create a "30 kg" piglet at random createpiglet = function(son) { piglet = rep(na, 4) piglet[1] = getstartweight(son) piglet[2] = getinitialdg(son) piglet[3] = -1 piglet[4] = 0 return(piglet) } # Draw initial live weight # Draw initial daily gain # Number of days remaining if diseased # If "-1": Healthy. If "-2": Slaughtered # Disease effect on daily gain The following function updates the specified pig corresponding to a one day time step. In other words, the pig becomes one day older and increases in weight accordingly. The gain until tomorrow is determined as well, and the disease states are updated. # Updates the state of a pig updatepig = function(pig, son) { pignew = rep(na, 4) if (pig[3] > -2) { pignew[1] = pig[1] + pig[2] - pig[4] pignew[2] = updatedg(pig[2], son) if (pig[3] == -1) { if (ispigdiseased(son)) { pignew[3] = getdiseaseduration(son) pignew[4] = getdiseaseeffect(son) } else { pignew[3] = -1 pignew[4] = 0 } } else { if (pig[3] > 0) { pignew[3] = pig[3] - 1; pignew[4] = pig[4] } else { pignew[3] = -1; pignew[4] = 0 } } } # Is the pig still present? # Yes, calculate new weight # Update daily gain # Is the pig healthy now? # Yes it was. Should it change state? # Yes it should. Draw duration of disease # Draw effect of disease # No, the pig remains healthy # Define it as healthy # No effect of disease # No, the pig is already diseased # Has it still at least one day left of diseas # Yes, reduce the number of days left by one # Keep the disease effect on daily gain # No, it is the last day of the disease # Change state to healthy # No disease effect next time

92 Modeling a batch of slaughter pigs else { pignew = pig } return(pignew) } # The pig is already slaughtered # Leave it as it is The last function simulates the life of the specified piglet over the specified number of days. It is only used for illustration. # Simulates a pig from a specified piglet over the specified number of days simulatepig = function(piglet, days, son) { piglife = matrix(rep(na, length(piglet)*days), days) # Create a matrix for the results piglife[1,] = piglet pigday = piglet for (n in 2:days) { pigday = updatepig(pigday, son) piglife[n,] = pigday } return(piglife) } Modeling a batch Representation of a batch on a given day A batch is only characterized by the number of pigs it contains. On a given day the batch is simply represented by an array of pigs. The following code would produce a batch of 5 piglets on the day of insertion: > pig1 = createpiglet(stateofnature) > pig2 = createpiglet(stateofnature) > pig3 = createpiglet(stateofnature) > pig4 = createpiglet(stateofnature) > pig5 = createpiglet(stateofnature) > batchday = array(na, dim =c(5, length(pig1))) > batchday[1,] = pig1 > batchday[2,] = pig2 > batchday[3,] = pig3 > batchday[4,] = pig4 > batchday[5,] = pig5 > batchday [,1] [,2] [,3] [,4] [1,] [2,] [3,] [4,] [5,] Functions used with a batch None of the functions used at batch level need modification. They are left as they are throughout the exercise. Just like a pig object is updated at daily time steps, the same applies to a batch. The function responsible for updating (by updating the pigs one by one) is defined as follows:

93 14.1 Modeling a batch of slaughter pigs 93 # Updates the state of a batch updatebatch = function(batchday, son) { newbatchday = array(na, dim = c(length(batchday[,1]), length(batchday[1,]))) for (i in 1:length(batchDay[,1])) { # Iterate over pigs pigday = batchday[i,] # Take the i th pig pigday = updatepig(pigday, son) # Update it newbatchday[i,] = pigday # Insert it in the updated batch } return(newbatchday) # Return the updated batch } The following function just counts how many of the pigs in the batch that are still present in the herd (i.e. not yet slaughtered): # Counts the number of pigs still present in the batch (not slaughtered) pigsleft = function(batchday) { left = 0 for (i in 1:length(batchDay[,1])) { if (batchday[i,3] > -2) { left = left + 1 } } return(left) } The decision strategy is applied at batch level through the following function picking out pigs for slaughter: # Applies the decision strategy on the batch a given day selectforslaughter = function(batchday, age, dec) { correctedday = array(na, dim=c(length(batchday[,1]), length(batchday[1,]))) observedday = rep(na, length(batchday[,1])) for (i in 1:length(batchDay[,1])) { correctedday[i,] = batchday[i,] # Copy pigs to new array observedday[i] = getobservedweight(batchday[i,1]) } leftpigs = pigsleft(batchday) # How many pigs are still present if (leftpigs < dec$minimumstock # Fewer than minimum? age > dec$maximumage) { # Maximum age exceeded? for (i in 1:length(batchDay[,1])) { # Yes, correctedday[i,3] = -2 #... cull the rest } } readyindexes = which(observedday >= dec$thresholdweight) # Find the pigs with observed weights exceeding threshold readycount = length(readyindexes) # How many were there if (readycount < dec$minimumdelivery) { # Are there two few? if (readycount > dec$minimumdelivery - dec$maximumunderweight) { # Yes, but are there so many that we slaughter anyway? sortedvector = sort(observedday, decreasing = TRUE) # Yes, find the biggest pigs realthres = sortedvector[dec$minimumdelivery] # Find the real threshold readyindexes = which(observedday >= realthres) # Pick all pigs exceeding the real threshold for (i in readyindexes) { correctedday[i,3] = -2 # Mark them as slaughtered } } } else { # No, there were enough pigs for (i in readyindexes) { # Pick them correctedday[i,3] = -2 # Mark them as slaughtered } } return(correctedday) # Return the pigs with the slaughtered marked } A simulation run In order to run a simulation (one complete calculation for a batch) the user must specify the number of pigs in the batch, the number of days to simulate, the state

94 Modeling a batch of slaughter pigs of nature and the decision strategy. The following function performs a simulation run: # Simulates a batch with the specified number of pigs over the specified number of days simulatebatch = function(pigs, days, son, dec) { piglet = createpiglet(son) # Create a piglet (just to know its dimension...) batchstructure = array(na, dim=c(pigs, days, length(piglet))) # Create a data structure for the results for (i in 1:pigs) { # Create "pigs" piglets piglet = createpiglet(son) # The i th piglet batchstructure[i,1,] = piglet # Insert it } for (n in 2:days) { # Iterate over the simulation period day by day batchstructure[,n,] = updatebatch(batchstructure[,n-1,], son) # if (n %% son$deliverevery == 0) { # Is it a delivery day? batchstructure[,n,] = selectforslaughter(batchstructure[,n,], n, dec) # Yes, apply decision strategy } } return(batchstructure) # Return full data structure } Output from the function (simulation run) is a data structure containing the values of all state variables of all pigs at all days during the simulation. In other words, the data structure is identical to Φ s = (Φ s1, Φ s2,..., Φ st,..., Φ st ) in the textbook. If we wish to simulate a batch with 100 pigs for a period of 120 days using the defined state of nature (stateofnature) and the decision strategy (decisionstrategy), we may write as follows in R: batch = simulatebatch(100, 120, stateofnature, decisionstrategy) Afterwards, the variable batch (corresponding to Φ s ) contains all information about the simulation. A visual illustration of the simulation results is provided through the plotbatchweights function. It takes the simulation data as input together with an integer parameter, skip. If skip = 1, it means that all pigs are plotted. If skip = 10, it means that every 10th pig is plotted etc. # Plot growth curves of pigs plotbatchweights = function(batch, skip) { pigweights = batch[1,,] days = 1:length(pigWeights[,1]) plot(days, pigweights[,1], type="l", ylim=c(25, 110), col="white", xlab="days", ylab="kg live weight") for (i in 1:length(batch[,1,1])) { piglife = batch[i,,] liveindexes = which(piglife[,3] > -2) pigweights = piglife[liveindexes,1] if ((i-1) %% skip == 0) { lines(pigweights) } } } The function produces a plot like the one in Figure E14.1.

95 14.1 Modeling a batch of slaughter pigs 95 Figure E14.1: Simulated growth curves for a batch of pigs. Functions returning output variables, Ω For simplicity we will only look at a few possible output variables. For that purpose a number of output functions are available. Since it is important from an economical point of view to deliver pigs in the optimal interval, the following two functions have been defined: # Checks whether a pig has been delivered in the optimal interval inoptimalinterval = function(liveweight, son) { slaughterweight = getslaughterweight(liveweight, son) return(slaughterweight >= son$loweroptimal && slaughterweight <= son$upperoptimal) } # Count the number of pigs delivered in the optimal interval deliveredinoptimalinterval = function(batch, son) { count = 0 for (i in 1:length(batch[,1,1])) { n = 1 while (batch[i,n,3]!= -2) { n = n + 1 } if (inoptimalinterval(batch[i,n,1], son)) { count = count +1 } } return(count) } The function inoptimalinterval is not called directly. It is just a function used by deliveredinoptimalinterval. The following function calculates the true average daily gain of the batch:

96 Modeling a batch of slaughter pigs # Calculate (true) daily gain dailygain = function(batch) { count = 0 totgain = 0.0 for (i in 1:length(batch[,1,1])) { n = 1 while (batch[i,n,3]!= -2) { count = count + 1 totgain = totgain + batch[i,n,2] - batch[i,n,4] n = n + 1 } } return(totgain/count) } Assignment 1. Copy the entire simulation program to the R console and run it. (a) Why does the plot look so different from Figure E14.1? (b) How many pigs were delivered in the optimal interval? Explain the result. (c) What was the true average daily gain? Explain the result. 2. We shall now work with the piglet creation function createpiglet. (a) Create a piglet by issuing the command createpiglet(stateofnature). (b) Describe the properties of the created piglet (weight, gain, disease state). (c) Create a new piglet by issuing the command createpiglet(stateofnature). (d) Is the new piglet in any way different from the first one? Explain why (not). (e) The createpiglet function uses two other functions, getstartweight and getinitialdg. Modify getstartweight so that it returns random numbers instead of just average values. Use the relevant mean and standard deviation of the state of nature to draw a random normally distributed number using the rnorm function. (f) Create two new piglets by issuing the command createpiglet(stateofnature) twice. Do the two piglets now differ? Explain why (not). (g) Run a new batch simulation by issuing the commands > batch = simulatebatch(100, 120, stateofnature, decisionstrategy) > plotbatchweights(batch, 1) Explain what you see in the plot. (h) Modify also the getinitialdg function so that it returns random numbers instead of just average values. Use the relevant mean and standard deviation of the state of nature to draw a random normally distributed number using the rnorm function.

97 14.1 Modeling a batch of slaughter pigs 97 (i) Create two new piglets by issuing the command createpiglet(stateofnature) twice. Explain what you see. (j) Run a new batch simulation by issuing the commands > batch = simulatebatch(100, 120, stateofnature, decisionstrategy) > plotbatchweights(batch, 1) Explain what you see in the plot. 3. We shall now extend the model in order to simulate the changes over time in the growth process. Let g t denote the live weight gain at time t. The basic assumption behind the development of growth rate over time is that it follows a first order autoregressive time series, i.e. g t = µ g + α(g t 1 µ g ) + ɛ t where α is the autocorrelation coefficient, ɛ t N (0, (1 α 2 )σ 2 g), µ g is the average daily gain for the batch and σ g is the standard deviation of daily gain between pigs. (a) Modify the updatedg function so that it reflects the first order autoregressive time series as described above. Use the relevant mean, standard deviation and autocorrelation coefficient of the state of nature to draw a random normally distributed number using the rnorm function. (b) Run a new batch simulation by issuing the commands > batch = simulatebatch(100, 120, stateofnature, decisionstrategy) > plotbatchweights(batch, 1) Explain what you see in the plot. 4. We shall now extend the model in order to simulate the effect of disease. Every day the ispigdiseased function is called for each pig in order to simulate outbreak of a disease in the pig. If it is diseased, a disease effect (reduced daily gain) created by getdiseaseeffect and a disease duration created by getdiseaseduration must be determined. (a) Modify the ispigdiseased function so that it returns the value TRUE with the probability specified in the state of nature 1 (b) Modify the getdiseaseeffect function so that it returns random numbers instead of just average values. Use the relevant mean and standard deviation of the state of nature to draw a random normally distributed number using the rnorm function. (c) Modify also the getdiseaseduration function so that it returns a random number drawn from a Poisson distribution to represent the duration of the disease. Use the relevant mean of the state of nature to draw a random Poisson distributed number using the rpois function. 1 Hint: Use the runif function in R.

98 Modeling a batch of slaughter pigs (d) Run a new batch simulation by issuing the commands > batch = simulatebatch(100, 120, stateofnature, decisionstrategy) > plotbatchweights(batch, 1) Explain what you see in the plot. 5. We shall now extend the model in order to include an observation error for the live weight of pigs. It is assumed that the weighing method is unbiased, i.e. that the observation error is N (0, σ 2 o), where σ o is the standard deviation of the observation error. The value of σ o depends heavily on the weighing method. If a scale is used, we may assume that σ o = 0, but for visual weight assessment it is perhaps as big as σ o = 5 kg. (a) Modify first the getslaughterweight function in order to represent the uncertainty of the resulting slaughter weight for a given (true) live weight. Use the relevant standard deviation of the state of nature to draw a random normally distributed number using the rnorm function. (b) Run a new batch simulation (now with 500 pigs) by issuing the commands > batch = simulatebatch(500, 120, stateofnature, decisionstrategy) > plotbatchweights(batch, 1) > deliveredinoptimalinterval(batch, stateofnature) These results corresponds to weighing pigs with a scale (i.e. no observation error). How many pigs were delivered in the optimal interval? (c) Modify the getobservedweight function in order to reflect weighing by visual weight assessment. Use the rnorm function to draw normally distributed observed live weights with observation error. (d) Run a new batch simulation by issuing the commands > batch = simulatebatch(500, 120, stateofnature, decisionstrategy) > plotbatchweights(batch, 1) > deliveredinoptimalinterval(batch, stateofnature) Explain what you see in the plot. These results corresponds to weighing of pigs by visual assessment. How many pigs were delivered in the optimal interval? 6. We shall now study the influence of delivery policy on calculated daily gain. (a) Define the decision strategy individualstrategy defined on Page 89. Also, change the state of nature so that it allows delivery every day, i.e. issue the command stateofnature$deliverevery = 1. (b) Run a new batch simulation with 500 pigs using this strategy by issuing the commands > batch = simulatebatch(500, 120, stateofnature, individualstrategy) > plotbatchweights(batch, 1) > deliveredinoptimalinterval(batch, stateofnature) > dailygain(batch)

99 14.2 Simulation of a scavenging chicken production system 99 Explain what you see in the plot. How many pigs are delivered in the optimal interval? What is the daily gain? (c) Define the decision strategy agestrategy defined on Page 89. (d) Run a new batch simulation with 500 pigs using this strategy by issuing the commands > batch = simulatebatch(500, 120, stateofnature, agestrategy) > plotbatchweights(batch, 1) > deliveredinoptimalinterval(batch, stateofnature) > dailygain(batch) Explain what you see in the plot. How many pigs are delivered in the optimal interval? What is the daily gain? (e) Explain why the daily gain is different under the two strategies. 7. Feel free to run all the simulations you can come up with! 14.2 Simulation of a scavenging chicken production system The exercises are based on the SimFlock model by McAinsh and Kristensen (2004). Details about the state of nature of the model are described by Kristensen and Pedersen (2003) The SimFlock user interface: Testing alternative decision rules In order to test alternative decision rules, you must go through the following steps: 1. Edit the hyper distribution (the f(x) icon). 2. Define a simulation job (the gear wheel icon). 3. Create an initial flock (from the menu). 4. Create multiple states of nature (from the menu). 5. For each decision rule: (a) Define it (the farmer icon). (b) Run the simulation job (from the menu). (c) Save the results to disk. (d) Save the states of nature to disk (only for the first decision rule) 6. Modify the R program to read the results from files. 7. Run the R program. 8. Conclude.

100 Simulation of a scavenging chicken production system Figure E14.2: The simulation job definition window Editing the hyper distribution In exercise 3 it is necessary to edit the hyper distribution. As long as you work with Exercises 1 & 2 you can skip this step and just rely on the default settings. The hyper distribution is edited either by clicking the f(x) icon or from the Menu: Edit Hyper parameters. Defining a simulation job The simulation job is defined either by clicking the gear wheel icon or from the Menu: Edit Simulation job. The editing window is shown in Figure E14.2. You should always (at least in these exercises) enable multiple states of nature. When setting the other parameters you will probably need to set the values lower than desired, because we have limited time for the exercises (simulation is rather time consuming). Defining the job with the following settings is a reasonable compromise between precision and time available: Number of states of nature to generate: 20 (corresponding to 20 different flocks drawn at random). Number of replications per state of nature: 30 (the more replications the higher precision). Number of days to simulate per replication: 3650 (corresponding to 10 years). Burn-in period (days) 730 (ignoring results from the first 2 years). Creating an initial flock This is done from the Menu: Create Initial random flock. The initial flock is used as starting point for the simulation. Since we are not interested in this particular flock we usually define a burn-in period as described in the previous section.

101 14.2 Simulation of a scavenging chicken production system 101 Figure E14.3: The decision rule definition window Creating multiple states of nature This is done from the Menu: Create Multiple states of nature. It is important that it is done whenever the hyper distribution has been modified, because a number of states of nature (corresponding to different flocks) is drawn at random from the hyper distribution. The number of states of nature generated corresponds to the number defined in the simulation job. On the other hand, it is very important that you don t generate new states of nature for each new decision rule. Since there may be interactions between the states of nature (flocks) and the decision rules, it is important that all the rules are tested on the same states of nature. Testing each of the decision rules Defining the decision rules The first step is to define the decision rule. It is edited either by clicking the farmer icon or from the Menu: Edit Farmer. The window for editing decision rules is shown in Figure E14.3 The fields Farmer name and Need for meat can be ignored. The other fields define intended flock size, egg removal policy and policy for buying young hens and cocks for replacement: Intended flock size: This is just specified as the number of hens and cocks that the farmer wish to have. Don t try to specify realistic flock sizes under Danish conditions - the model will not be able to handle that. Egg removal policy: By default, no eggs are removed from the nests (i.e. all eggs are incubated). It is possible to remove eggs according to two different principles:

102 Simulation of a scavenging chicken production system Figure E14.4: The status window showing the progress of a running simulation job. Removal of all eggs during the first n days of the laying cycle of each hen. If eggs are removed for the first three weeks the field Days removing eggs should be set to 21. Removal of all eggs from all hens at a particular season of the year. If eggs are removed from March 1st until June 1st, the field Remove all eggs from day (season) should be set to 60 and the field Remove all eggs until day (season) should be set to 152. Policy for buying replacements: By default dead hens and cocks are replaced by pullets and cockerels from the farmer s own flock. Because of the very high mortality this is not always possible, so there is a considerable risk of dying out. The policy for buying replacement hens is specified through two fields. A yes/no field defines whether the farmer will buy replacements at all and a buy limit defines when to buy hens. The actual number of hens is every day compared to the buy limit. If the actual number is lower than the buy limit (and no pullets are available neither) the missing hens are bought at the market. Running the simulation job Next, the simulation job must be run. This is done from the Menu: Run Simulation job. The progress of the simulation job is shown in a status window as in Figure E14.4. While the simulation job is running you can pick your nose, have a cup of coffee or whatever you like. Saving results to disk When a simulation job has terminated and you have closed the status window, the results are available for inspection. This is done by clicking the table icon which displays a window like the one shown in Figure E14.5. By selecting an output variable from the upper field and afterwards pressing the Show button, you can browse the results for that particular output variable

103 14.2 Simulation of a scavenging chicken production system 103 Figure E14.5: The window from which the results can be browsed or saved to disk. in a large table. This can be useful for a quick overview, but for a more thorough analysis it is better to save all results to disk and afterwards analyze them in SAS. In order to save all results to disk you press the Save results button and specify a file name and location for a text file containing the data in SAS readable format. Saving the states of nature to disk Since all decision rules should be tested under the same states of nature, you only need to save them after the first simulation run (i.e. for the first decision rule). This is done by clicking the Save States of Nature button and specifying a file name and location for a text file containing the state of nature data in SAS readable format. It is not necessary to create a SAS program, since an R program is provided separately for the purpose of this exercise. Modifying the R program The modification of the R program to read the results from files as a minimum consists of changing the file names used in the read.table statements to reflect the names you specified when you saved the data. If you want, you can also add other relevant statistical analyses to the program. Running the R program You are assumed to be familiar with running an R program - otherwise ask. The program uses the doby package which must be installed on your computer (install it directly from a CRAN mirror or download it from the home page of this exercise). By default, the program reads the states of nature and the results from two different decision rules (policies) from disk. If you have defined more than two different decision rules you must modify the program accordingly.

104 Simulation of a scavenging chicken production system Having read the simulation results to data frames, a common data frame containing all results (under both/all policies) is generated. Then the states of nature a read to a data frame and a new variable Combined.survival is defined and calculated. It expresses the probability that a chicken survives the chick and grower period. The results and the states of nature are then merged into a common data frame, and a couple of supplementary results are calculated. A summaryby statement calculates average results for all combinations of state of nature and policy for 4 selected output variables ( Chickens.produced, Kg.meat.produced, Eggs.removed..sold. and Young.hens.bought ). Finally the simulation results are analyzed by plots and analysis of variance (these analyses can be extended). The program produces the following output: A print of the first 5 observations of the combined result data frame. This is just for identification of the variable names. A print of the first 5 observations of the state of nature data frame (for identification of variable names). A print of the average results for the 4 selected output variables chosen to describe the combinations of states of nature and policies. For the variable Chickens.produced : A plot comparing the two policies. An analysis of variance estimating the effect of policy on number of chickens produced. For the variable Young.hens.bought : A plot comparing the two policies. An analysis of variance estimating the effect of policy on number of young hens bought Exercise 1 The aim of this exercise is to make you familiar with the SimFlock simulation model (part of the Advanced Herd Management package) and its user interface. Try to run some simulations and study the results Exercise 2. Smallholder egg production? Background In Zimbabwe, most smallholders keep hens in order to have a source of meat for household consumption at special occasion and in order to have a source of monetary income by selling chickens at the market. It is not usual neither to sell eggs nor to use them for household consumption.

105 14.2 Simulation of a scavenging chicken production system 105 By use of SimFlock, we would like to explore the opportunities for egg production based on the scavenging production system. Clicking the Farmer icon will allow you to define the egg removal policy to be used. There are basically two options. You can either remove all eggs from a hen for a specified number of days (calculated from the start of the current ovulation period, or you can remove all eggs in a specified period of the year (i.e. from day d 1 to day d 2, where 0 d 1 < d 2 365). One of the pre-defined output variables is Eggs removed (i.e. sold). That output variable will tell you how many eggs it is possible to produce under the specified egg removal policy. Logically, however, the removal of eggs will influence other production results as well, so it is a question whether it (from an over-all point of view) is beneficial to collect eggs from the hens. Problems 1. Study the list of pre-defined output variables in SimFlock (by clicking the Simulation Results icon). Which ones do you expect will be influenced by the egg removal policy? 2. Define one or more egg removal policies and test them using the procedure described in Section Is there a potential for egg production? Exercise 3. Providing shelter Background Compared to production systems in developed countries, the smallholder scavenging production system has a very high mortality, and only very few birds originally hatched will actually survive. In order to reduce the mortality it has been tested whether a kind of shelter protecting from weather (and lions?) for the birds can improve the survival rate. Mortality in SimFlock is represented in the state of nature by daily survival rates for each group of birds (i.e. chicks, growers, pullets and cockerels). Since they are mutually correlated, these survival rates are drawn from the hyper distribution by use of a logistic model with random effects and interactions: ( ) pij logit(p ij ) = ln = µ + α j + F i + (αf ) ij 1 p ij where p ij is the survival rate for the entire period (as chick, grower etc.) in flock i, group j (j = 1,..., 4 for chick, grower, pullet and cockerel). The parameter µ is a general level, α i is the systematic effect of group, F j is the random effect of flock, and (αf ) ij is the random interaction between flock and group. In the hyper parameters the current settings for α 1,..., α 4 as well as the variance of F i

106 Simulation of a scavenging chicken production system and (αf ) ij may be found. Since µ = 0, 2526, α 1 = 0, 5590 and α 1 = 0, 3981, we conclude that the average survival rate (across states of natures) of chicks p 1 is given by ( ) p1 y 1 = logit(p 1 ) = ln = = p 1 Solving for p 1 yields p 1 = 1 exp( y 1 ) + 1 = 1 exp( ) + 1 = The mortality rate for the whole chick period (3 weeks) is therefore on average = Analogous calculations reveal that the average mortality rate for growers (7 weeks) is = Assume that experiments have shown that providing shelter can decrease chick mortality by 20% and grower mortality by 10%. This means that the average chick mortality across all states of nature when using shelter is = In other words, the revised average survival rate becomes p 1 = The logit transform is therefore ( ) ( ) p logit(p 1 ) = ln p = ln = Since the logit transform must satisfy logit(p 1 ) = µ + α 1, we logically conclude that α 1 = logit(p 1 ) µ = = We have then determined the new hyper parameter specifying the chick effect. Problems 1. Calculate the revised hyper parameter α2 for the corresponding grower effect when providing shelter. 2. Enter both hyper parameters into the simulation model. 3. Repeat the analyses from exercise Will providing shelter improve the possibilities for egg production? 2 This is for the whole chick period. The daily average survival rate used in the simulation is then =

107 Appendix A Probabilities and distributions A.1 Basic probability calculus 1. What is the probability of observing the exact sequence of outcomes 3, 2, 4, 1, 5 when rolling a (fair) dice 5 times? 2. What is the probability of observing a value bigger than 4 when rolling a (fair) dice? 3. What is the probability of observing a 3 and a 6 when rolling two dices simultaneously? 4. Effectiveness of contraceptive methods is measured as the average number of pregnancies per year among 100 couples using the method. Assume that the effectiveness of a method is 0.05 or 5% (i.e. the average number of pregnancies among 100 couples using the method is 5 per year). (a) What is the probability of at least one undesired pregnancy for a couple using the method for 5 years? (b) What is the probability of at least one undesired pregnancy for a couple using the method for 10 years? (c) What is the probability of at least one undesired pregnancy for a couple using the method for 15 years? (d) Assume instead that the effectiveness is What is then the probability of at least one undesired pregnancy for a couple using the method for 5, 10 or 15 years? 5. In a dairy herd, the conception rate is assumed to be 0.40 per insemination. What is the probability of a cow needing at least 5 inseminations before pregnancy is achieved? 6. Assume that people s birthdays are evenly distributed over the 365 days of a year (i.e. the probability that a randomly selected person has birthday on a

108 108 A.2 Conditional probability calculus randomly selected day is 1/365). What is the probability that at least 2 pupils in a school class of 25 have birthday on the same day? A.2 Conditional probability calculus 1. A diagnostic test for a certain disease has the sensitivity 0.80 and the specificity The prevalence of the disease in the population is 0.01 (i.e. one percent of the individuals is suffering from the disease). We shall define S + as the event that an individual is suffering from the disease. Correspondingly, the event S implies that the individual is healthy. Furthermore, let T + and T represent the two possible outcomes of the test. In symbols, we may express our prior knowledge as P(S + ) = 0.01, P(S ) = = 0.99, P(T + S + ) = 0.80, P(T S + ) = = 0.20, P(T S ) = 0.95 and P(T + S ) = = (a) What is the probability that a person being diagnosed as diseased is actually diseased? (b) What is the probability of a wrong diagnose? (c) What is the probability of a wrong diagnose if we just declare all individuals for healthy? (d) What can be concluded about correct use of a diagnostic test as the one described (initially)? Is it suitable for screening of entire populations? 2. A dairy farmer has inseminated a cow and 3 weeks later he concludes that it does not show heat. Given the assumptions of Example A.6 of Section A.1.6, the probability that the cow is pregnant is After further some weeks a pregnancy test is performed. Make realistic assumptions about the sensitivity and specificity of the pregnancy test and calculate the probability that the cow is pregnant given: (a) A positive outcome of the pregnancy test. (b) A negative outcome of the pregnancy test. A.3 Discrete distributions 1. A pig farmer mates 20 sows in a given week. The conception rate is assumed to be (a) What is the expected number of sows farrowing 115 days later? (b) What is the variance and the standard deviation? (c) What assumptions have you made in order to answer the two previous questions?

109 A.4 The normal distribution Assume that the number of leg disorders per year among sows of a sow herd is poisson distributed with parameter 10. (a) What is the expected number of leg disorders for the next year? (b) What is the variance and the standard deviation? (c) What assumptions must we make in order to assume that the number of leg disorders is poisson distributed? (d) Would it be realistic to assume that the number of cases of lung diseases among slaughter pigs is poisson distributed? A.4 The normal distribution 1. The annual milk yield, X, of the cows in a certain herd may be regarded as N (9000, ). Assume that the herd at present consists of 100 cows. (a) What is the expected number of cows (among the 100) that have a higher milk yield than 8000 kg, 9000 kg, kg, respectively? (b) The farmer intends to calculate the average milk yield of his 100 cows. Denote the average yield as X. What is the distribution of X? (c) What is the distribution of the total milk yield of all 100 cows?

110 110 A.4 The normal distribution

111 Appendix C Linear algebra C.1 Matrix and vector operations 1. Let A = ( ), B = Calculate the following expressions if they make sense: (a) A + B (b) A + B (c) A + 4B (d) AB (e) BA (f) A B (g) A Let A = ( ). Calculate first det(a), and afterwards, A 1. Verify that A 1 A = I and AA 1 = I. 3. Assume that we observe a herd every week and classify the health status of the animals as being either good, normal or bad. If the condition this week is good, the probability of being good, normal or bad next week is 0.50, 0.30 and 0.10, respectively. Furthermore, the probability of death is For a normal animal, the probabilities of transition to good, normal, bad and dead are 0.15, 0.50, 0.15 and 0.20, respectively. The corresponding probabilities for a bad animal are 0.00, 0.30, 0,40 and

112 112 C.2 Linear regression and matrices (Naturally, a dead animal remains dead with probability 1, so we will not have a state for dead animals). Every week, 10 new normal animals are inserted into the herd. The inspection this week, t, revealed that there were 10 animals in state good, 30 in state normal and 10 in state bad. We shall express the present counts by the (row) vector n t = (10, 30, 10). The possible transitions may be represented by the state transition matrix P = Since all new animals are normal, we may represent the animals inserted by the vector a = (0, 10, 0). The expected number of animals distributed over states for week t + 1 is calculated as. n t+1 = a + n t P. (E3.1) (a) What is the expected number of animals in each state in week n + 1? (b) What is the expected number of animals in each state in week n + 2? (c) Assume that if we wait for sufficiently long time, the expected distribution over states will not change any more (because the herd has reached an equilibrium). If the equilibrium is reached, we logically have that n t+1 = n t = n. According to Eq. (E3.1), we therefore have at equilibrium: n = a + n P. (E3.2) Use Eq. (E3.2) to find an expression for n. (d) Find the numerical value of n. C.2 Linear regression and matrices 1. Refer to Example C.2 of Section C.3 of Kristensen et al. (2010b) and define the vector Y and the matrix x as in Eq. (C.2). Use R or Excel as you prefer. (a) Estimate the regression coefficient ˆβ and the residual variance s by use of Eqs. (C.3) and (C.4). (b) Perform a linear regression (using the built in linear regression function of R or Excel). (c) Compare the matrix algebra result to the result of the linear regression. 2. Include sex in the model and redefine the x matrix as described in Example C.2. (a) Estimate the regression coefficient ˆβ and the residual variance s by use of Eqs. (C.3) and (C.4).

113 C.2 Linear regression and matrices 113 (b) Perform a linear regression (using the built in linear regression function of R or Excel). (c) Compare the matrix algebra result to the result of the linear regression.

114 114 C.2 Linear regression and matrices

115 Appendix D Advanced topics from statistics D.1 The multivariate normal distribution The distribution of litter size, Y n, of an nth parity sow is of course discrete, but for practical purposes it is often convenient to approximate it by a normal distribution, N (µ n, σ 2 ). We then calculate the probability of having k piglets as ( P(k) = P k 1 2 Y < k + 1 ) = Φ 2 ( k µ σ ) ( k 1 2 Φ µ ) σ We may then consider the mutual distribution of the first 12 parities of a sow Y = (Y 1,..., Y 12 ) as a 12 dimensional normal distribution, i.e. Y N (µ, Σ), where µ = (µ 1,..., µ 12 ) is the mean vector, and Σ is a variance-covariance matrix. Based on estimation results by Toft and Jørgensen (2002) the distribution for a particular herd had the following parameters: µ = Σ = The mean vector µ and the variance-covariance matrix Σ have been pre-entered into the spreadsheet LitterSize.xls. Assume that we have observed the first 5 parities of a sow, and that Y 1 5 = y1 5 = (12, 14, 15, 12, 16). 1. Find an expression for Y 6 12 y 1 5 (cf. Eqs. (D.4) and (D.5)). 2. What is the conditional mean vector of Y 6 12 y 1 5. Use Excel to calculate it.

116 116 D.2 Hyper distributions 3. What is the conditional variance-covariance matrix of Y 6 12 y 1 5. Use Excel to calculate it. 4. What is the decrease in standard deviation of Y 6 caused by the conditioning on y 1 5? 5. Try to change the observations for litter size for parity 1-5 (i.e. the values of y 1 5 ): (a) What happens to the conditional expectation? (b) What happens to the conditional variance-covariance matrix? D.2 Hyper distributions Assume that in a dairy herd we wish to estimate the proportion of cows conceiving within the first 13 weeks of lactation. Pregnancy diagnoses are made regularly on all cows 7 weeks after the most recent insemination. Thus, 20 weeks after calving, the result (pregnant or not pregnant) is known for each cow. In every week during the year, the farmer makes two observations: The number of cows, n t, being 20 weeks after calving. The number of cows C t (among the n t ) that are pregnant. Thus, 0 C t n t. The farmer logically assumes that the random variable C t is binomially distributed with parameters p and n t, where p is the underlying true rate of cows conceiving within the first 13 weeks of lactation. He would like to estimate the value of p as an indication of his over-all performance concerning reproduction management in his herd. He has asked you to help him setting up a dynamic estimation procedure for p. The Excel spreadsheet pregnant.xls contains observations for the a period of 200 weeks (or close to 4 years). 1. Use the Beta distribution to specify your prior belief in p by setting the parameters of the beta distribution to values reflecting your belief (cf. Section D.4.2). 2. Update the parameters of the Beta distribution week by week, and, calculate in each week the expected value of the beta distribution as the current estimate for p. Calculate also (in each week) a lower and an upper 95% limit for the value of p. 3. Create a graphical illustration of the weekly estimates together with the lower and upper limits.

117 D.2 Hyper distributions Since the data used in this exercise has been simulated, the true p value is known. What do you think it is? 5. Try to set the parameters of the initial Beta distribution to something completely unrealistic, and observe the consequences. 6. What is your opinion about a dynamic estimation system as the one developed? Are there any assumptions behind that are unrealistic?

118 118 D.2 Hyper distributions

119 Bibliography Jensen, F. V., Bayesian Networks and Decision Graphs. Springer, New York. Kristensen, A. R., A model to describe and predict the production and food intake of the dairy cow. Acta Agriculturæ Scandinavica 36, Kristensen, A. R., Jørgensen, E., Toft, N., 2010a. Herd Management Science. I. Basic concepts. Sl books, Copenhagen. Kristensen, A. R., Jørgensen, E., Toft, N., 2010b. Herd Management Science. II. Advanced topics. Sl books, Copenhagen. Kristensen, A. R., Pedersen, C. V., Representation of uncertainty in a monte carlo simulation model of a scanvenging chicken production system. In: EFITA Fourth European Conference of the European Federation for Information Technology in Agriculture, Food and the Environment. Debrecen, Hungary, pp Kristensen, A. R., Søllested, T. A., 2004a. A sow replacement model using bayesian updating in a 3-level hierarchic markov process. i. biological model. Livestock Production Science 87, Kristensen, A. R., Søllested, T. A., 2004b. A sow replacement model using bayesian updating in a 3-level hierarchic markov process. ii. optimization model. Livestock Production Science 87, Lauritzen, S. L., Nilsson, D., Limids of decision problems. Management Science 47, McAinsh, C. V., Kristensen, A. R., Dynamic modelling of a traditional african chicken production system. Tropical Animal Health and Production 36, Ranvig, H., Fodring af får. DSR Forlag, Frederiksberg C. Strathe, J., Shoulder lesions in danish sows - an abattoir survey with emphasis on the relation between clinical signs and post-mortem registrations. Master s thesis, Department of Large Animal Sciences, Faculty of Life Sciences, University of Copenhagen.

120 120 BIBLIOGRAPHY Thysen, I., Alternative fremgangsmåder ved optimering af foderrationer. In: Østergaard, V., Neimann-Sørensen, A. (Eds.), Optimum feeding of the dairy cow. Feed composition, feed intake, matabolism and production. Vol. 551 of Report from the National Institute of Animal Science. København, Ch. 17, pp , in Danish. Toft, N., Jørgensen, E., Estimation of farm specific parameters in a longitudinal model for litter size with variance components and random dropout. Livestock Production Science 77, Ucar, M., Kücükkebapci, M., Gündogan, M., Saban, E., Using milk progesterone assay at the time of oestrus and post-mating for diagnosing early pregnancy in anatolian water buffalos. Turkish Journal of Veterinary and Animal Science 28, van Arendonk, J. A. M., Studies on the replacement policies in dairy cattle. i. evaluation of techniques to determine the optimum time for replacement and to rank cows on future profitability. Zeitschrift für Tierzüchtung und Züchtungsbiologie 101,

121 Finn V. Jensen Bayesian Networks and Decision Graphs Springer, New York, pages. Selected pages: 3 28 (from Chapter 1) (from Chapter 2)

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175