Hierarchical Generalized Linear Models for Multiple QTL Mapping

Transcription

1 Genetics: Published Articles Ahead of Print, published on January 1, 009 as /genetics Hierarchical Generalized Linear Models for Multiple QTL Mapping Nengun Yi 1,* and Samprit Baneree 1 Department of Biostatistics, Section on Statistical Genetics, University of Alabama at Birmingham, Birmingham, Alabama 3594, Divison of Biostatistics and Epidemiology, Department of Public Health, Weill Medical College of Cornell University, New York, NY-1001 * Corresponding author: Nengun Yi Department of Biostatistics University of Alabama at Birmingham Birmingham, AL Phone: Fax: nyi@ms.soph.uab.edu Key words: Bayesian methods, Generalized linear models, Interactions, Quantitative trait loci, Shrinkage Running title: Generalized QTL models 1

2 ABSTRACT We develop hierarchical generalized linear models and computationally efficient algorithms for genome-wide analysis of quantitative trait loci (QTL) for various types of phenotypes in experimental crosses. The proposed models can fit a large number of effects, including covariates, main effects of numerous loci, gene-gene (epistasis) and gene-environment (G E) interactions. The key to the approach is the use of continuous prior distribution on coefficients that favors sparseness in the fitted model and facilitates computation. We develop a fast expectation-maximization (EM) algorithm to fit models by estimating posterior modes of coefficients. We incorporate our algorithm into the iteratively weighted least squares for classical generalized linear models as implemented in the package R. We propose a model search strategy to build a parsimonious model. Our method takes advantage of the special correlation structure in QTL data. Simulation studies demonstrate reasonable power to detect true effects, while controlling the rate of false positives. We illustrate with three real datasets and compare our method to existing methods for multiple QTL mapping. Our method has been implemented in our freely available package R/qtlbim ( providing a valuable addition to our previous Markov chain Monte Carlo (MCMC) approach.

3 INTRODUCTION Most complex traits are influenced by interacting networks of multiple quantitative trait loci (QTL) and environmental factors (Carlborg and Haley 004). Mapping QTL is to infer which genomic loci are strongly associated with the complex trait, and to estimate genetic effects of these loci, i.e., main effects, gene-gene (epistasis) and gene-environment (G E) interactions. Due to the multi-locus nature of complex traits, it is desirable to simultaneously analyze multiple loci rather than one (or a few) locus at a time. However, QTL mapping studies usually genotype hundreds or thousands of genomic loci (markers), leading to numerous variables and a huge number of possible models, and the dependence of genotypes on a chromosome results in many correlated variables. Therefore, mapping multiple QTL requires sophisticated methods that can handle problems with high-dimensional correlated variables. The popular approaches to mapping multiple QTL are some form of variable selection. Such techniques involve identifying a subset of all possible genetic effects (a multiple QTL model) that best explains the phenotypic variation. Classical variable selection methods use forward or stepwise search procedures and selection criteria such as BIC (Bayesian information criteria) or modified versions to find a multiple QTL model (Kao et al. 1999; Broman and Speed 00; Bogdan et al. 004; Baierl et al. 006). Bayesian methods proceed by setting up a likelihood function for observed data and prior distributions on unobserved quantities. Two types of prior distributions have been suggested for multiple QTL mapping. The first assumes a two-component mixture distribution for each genetic effect, typically a normal distribution with known or unknown variance and a point mass at zero. This discrete prior allows each effect to have 3

4 positive probability of dropping out of the model (Yi 004; Yi and Shriner 008). The second formulation takes continuous prior distributions for genetic effects that favor a sparse structure with many of the effects having values close to zero and few with large values (Meuwissen et al. 001; Xu 003; Yi and Xu 008). These Bayesian models are computed using Markov chain Monte Carlo (MCMC) algorithms to sample from the posterior distribution. Due to the recent development of MCMC algorithms and associated computer software, Bayesian methods have become increasingly popular in QTL mapping (Yi et al. 005; Yi et al. 007; Yandell et al. 007a, b; Yi and Shriner 008). The Bayesian MCMC approaches can provide comprehensive information, but they are computationally intensive in interacting QTL analysis. Further, the existing methods rely primarily upon normal linear models. We present a unified methodology for mapping multiple QTL based upon the generalized linear model framework. Generalized linear model is the main tool for routine statistical analysis, enoying a body of well-developed theory, algorithms and software and including various models as special cases. However, classical generalized linear models cannot simultaneously handle many correlated variables. We propose a Bayesian generalized linear model approach, placing continuous prior distributions on genetic effects, to dealing with the high-dimensional problem. Although various priors can be used for this purpose (Griffin and Brown 007; Park and Casella 008), we consider the well-known Student-t distribution and recommend the choice of hyperparameters to yield a sharp mode at zero and to induce a sparse model. Our unified framework incorporates the advantages of generalized linear models and hierarchical modeling into multiple QTL mapping, allowing us to deal with various types of 4

5 continuous and discrete phenotypes and to simultaneously analyze covariates, main effects of numerous loci, epistatic and G E interactions. In principle, we can fit our hierarchical generalized linear model by adapting the MCMC algorithms of Yi and Xu (008) for normal regressions. However, it is desirable to have a quick calculation that estimates the posterior mode of genetic effects (a point estimate that maximizes the posterior distribution). Although the posterior mode estimate provides less information than a fully Bayesian MCMC analysis, such analyses are similar to classical QTL practice, easy to understand, and can be valuable for identifying significant variables. Various methods have been proposed for the purpose, but they are built upon special models and optimization algorithms (Tibshirani 1996; Figueiredo 003; Kiiveri 003; Efron et al. 004; Genkin et al. 007) and cannot be easily set up for multiple QTL mapping. Recently, some mode-finding methods for mapping multiple QTL have been suggested (Zhang and Xu 005; Xu 007). However, these methods have dealt with only continuous traits and have not been effectively implemented in computer software. We present a fast expectation-maximization (EM) algorithm to fit our hierarchical generalized linear models by estimating the posterior modes of coefficients. We develop our algorithm by making use of the two-level formulation of the Student-t density and expressing the prior information on coefficients as additional data points. This strategy allows us to incorporate our algorithm into the usual iteratively weighted least squares for generalized linear models as implemented in R (R Proect 006). This implementation takes advantage of the well-developed existing algorithm and software, extending available tools for fitting generalized linear models to multiple QTL mapping. 5

6 Although in principle our hierarchical model and algorithm can fit all possible effects in a single model, we don t recommend this and rather propose a novel model search method to build a parsimonious model by seeking significant genetic effects. This procedure provides a flexible and convenient way to deal with large-scale QTL data, and also has the advantage of accommodating the correlation structure in QTL data. Simulation studies demonstrate that our method provides reasonable power to detect true effects and can control the rate of extraneous effects. Real data analyses show that the proposed approach compares favorably to existing sophisticated methods. METHODS Generalized linear models of multiple QTL: We consider experimental crosses derived from two inbred lines (for example, F, backcross and recombinant inbred lines). Observed data in QTL studies consist of phenotypic values of a complex trait, genetic markers across the genome, and/or some relevant environmental factors (covariates). The marker data include the genotypes and the genomic positions of markers. Mapping QTL is to identify genomic loci that associate with the phenotype and to estimate their genetic effects. For simplicity, we describe our methods by considering observed markers as potential QTL. The methods can be extended to consider loci between the observed markers. We use the generalized linear model framework to analyze various types of complex traits. A generalized linear model consists of three components: the linear predictor, the link function, and the distribution of the outcome variable (McCullagh and Nelder 1989; Gelman et al. 003). We simultaneously fit environmental effects, main 6

7 effects of markers, epistatic effects, and gene-environment (G E) interactions. Therefore, the linear predictor can be expressed as η = β + X β + X β + X β + X β Xβ, (1) 0 E E G G GG GG GE GE where β 0 is the intercept, βe and β G represent the vectors of environmental effects and all possible main effects, respectively, β GG and β GE represent the vectors of all possible epistatic and G E interactions, respectively, and X E, X G, X GG, and X GE are the corresponding design matrices of effect predictors. We describe the construction of these design matrices in the next subsection. The invertible link function g relates the linear predictor η to the mean of the outcome variable y: 1 1 E( y X) = g ( η) = g ( Xβ ), () T where the vector y = ( y, y,, y n ) represent the phenotypic values. The distribution of 1 y depends on the linear predictor X β and also a dispersion (or variance) parameterφ (but some models do not requireφ ), and can be expressed as n py ( Xβ, φ) = py ( Xβ, φ), (3) i= 1 i i where X iβ = ηi is the linear predictor for the i th individual. Generalized linear modeling provides a unified framework for statistical analysis; by choosing appropriate link functions and data distributions, some commonly used models, e.g., normal linear (Gaussian), logistic, probit and Poisson regressions, become special cases. 7

8 Construction of the design matrices: There is some discussion in the QTL mapping literature on the best way to construct contrasts for genetic predictors (Zeng et al. 005). Among those, due to its orthogonal property, the Cockerham genetic model is widely used in QTL mapping and is applied in this study. For a backcross design, the Cockerham model defines the values of the main-effect contrast as -0.5 or 0.5 for two genotypes at any locus. For an intercross (F ) design, there are two types of main effects, called additive and dominance effects, and the original Cockerham model defines additive contrast as -1, 0 and 1 for three genotypes and dominance contrast as -0.5 and 0.5 for homozygotes and heterozygote, respectively. To make additive and dominance contrasts have a common scale, however, we rescale additive contrast as , 0 and For loci with missing genotypic values, we replace the values of the above contrasts by their conditional expectations given the observed marker data (Haley and Knott 199). The conditional expectations can be calculated using the multipoint method (Jiang and Zeng 1997). For each covariate, we transform the raw values to have a mean of 0 and a standard deviation of 0.5, by subtracting the mean and dividing by *sd (the standard deviation of the raw values) (Gelman et al. 008). This transformation standardizes all the covariates to a common scale, the scale of all the genetic main effects described above. Finally, the epistatic contrasts multiplying two corresponding main-effect contrasts. X GG or the G E contrasts X GE are constructed by Prior and posterior distributions: Mapping QTL is equivalent to estimating coefficients β in the above model, including environmental effects, main effects of markers, epiatstic and G E interactions. The number of coefficients in the model can be 8

9 large and the predictors are highly correlated, precluding the use of classical maximum likelihood methods. This problem could be solved by setting up a prior distribution on β to capture the notion that most of the components of β are likely to be zero or at least negligible; such prior distributions are often referred to as shrinkage priors. For the intercept β 0 and the dispersion parameterφ if present, we use the uniform priors, p( β0) 1 and p( φ) 1. We assume independent Student-t priors tν s on coefficients β s, with ν (0, ) and s chosen to give each coefficient a high probability of being near zero while still allowing for occasional large effects. We are motivated to use the t distribution because it allows for robust inference, shrinkage estimation and easy computation (Gelman et al. 003). There is no easy way to estimate the coefficients directly using the t densities, but it is straightforward to deal with the two-level formulation of t distribution (Gelman et al. 003; Gelman et al. 008). The t distribution tν s can be expressed as a mixture of (0, ) normal distributions with mean 0 and variance distributed as scaled inverse- χ : β τ τ, ~ N(0, ) τ ~ Inv- χ ( ν, s ), = 1,, J, (4) where J is the number of the coefficients, and the hyperparameters ν > 0 and s > 0 represent the degrees of freedom and the scale of the distribution, respectively. The priors (4) introduce coefficient-specific variances, resulting in distinct shrinkage for different coefficients. The variancesτ s are not the parameters of interest, but they are useful intermediate quantities to allow easy and efficient computation. The hyperparametersν and s affect the amount of shrinkage in the coefficient estimates and 9

10 should be carefully chosen. We shall discuss our choice of ν and s after describing our computational algorithm. Our algorithm highlights how these hyperparameters affect the estimates of the parameters. With the above prior distributions, we can express the log-posterior distribution of the parameters ( β, φτ, ) as n J J log p( βφτ,, yx, ) log py ( i Xiβφ, ) + log p( β τ ) + log p( τ ν, s) i= 1 = 1 = 1 n J 1 β J ν ν ν s log py ( i Xiβφ, ) logτ + + log s ( 1) log + τ i= 1 = 1 τ = 1 τ, (5) whereτ = ( τ 1,, τ J ). EM algorithm for computing the posterior mode: Our hierarchical generalized linear model can be fit using MCMC algorithms, applied to the oint posterior distribution p ( βφτ,, yx, ) (Gelman et al. 003). However, it is desirable to have a faster computation that estimates the posterior mode of β and φ by maximizing the marginal posterior p( β, φ yx, ) rather than the full posterior distribution. The two-level hierarchical model described above allows us to obtain the posterior mode by using the EM algorithm (Gelman et al. 003; Gelman et al. 008). Following Gelman et al. (008), we develop our approach built upon the standard algorithm for classical generalized linear models. The analysis of classical generalized linear models uses iterative weighted linear regression to obtain approximate maximum likelihood estimates for the parameters (as implemented in the R routine glm, for example). The basic method is to approximate the generalized linear model by a normal linear model and then apply the algorithm for 10

11 normal linear models to estimate the parameters ( β, φ ) (Gelman et al. 003; Gelman et al. 008). At each iteration, the algorithm proceeds by calculating pseudo-data z i and pseudo-variances σ i for each observation i based on the current estimates of the parameters( ˆ β, ˆ φ ), approximating the generalized likelihood py ( Xβ, φ ) by the weighted normal likelihood N( z X β, σ ), and then updating the parameters ( β, φ ) by i i i weighted linear regression. The iteration proceeds until convergence. The pseudo-data z i and pseudo-variances σ i are calculated by L'( y ˆ, ˆ i ηi φ) 1 z ˆ i = ηi, σi =, (6) L''( y ˆ η, ˆ φ) L''( y ˆ η, ˆ φ) i i i i whereη = X ˆ β, Ly ( ˆ η, ˆ φ) = log py ( X ˆ β, ˆ φ), L'( y η, φ) = dl( y η, φ)/ dη, ˆi i i i i i i i i i i i i i i i L''( y η, φ) = d L( y η, φ)/ dη, and ˆ β and ˆ φ are the current estimates of β and φ, i i respectively. Given the variancesτ, the prior information on β (i.e., β τ ~ N(0, τ )) can be included in the classical generalized linear model as J additional observations of value 0, with predictor matrix I J and residual variance matrix Σ β = diag( τ,, τ J ), where I J is the J J identity matrix (Gelman et al. 003; Gelman et al. 008). Thus, the generalized linear model with the normal prior on β can be approximated by: 1 z X, β, φ ~ N( X β, Σ ), (7) * * * * z where z * = ( ) 1 0 n + J, X* X =, * I J ( n+ J) J Σz Σ = 0 0 Σ β with Σ = diag( σ,, σ ). Thus, z 1 n we can use the standard iteratively weighted least squares computation to estimate ( β, φ ) 11

12 by performing a weighed regression of the augmented data vector z * on the augmented predictor matrix X * with augmented weight vector w* = ( σ1,, σn, τ1,, τj ). To implement the EM algorithm, we treat the unknown variancesτ = ( τ 1,, τ J ) as missing data, and average over them by replacing the terms involving bothτ and β in the oint posterior (5) by their expected values conditional on the current estimate of ( ˆ, ˆ β φ ). Thus, we must evaluate the conditional expectations of 1/ τ for all. It can be easily shown that the conditional posterior distributions of τ is Inv-χ 1 + ν, ˆ ν s + β, and thus the conditional expectations of 1+ ν 1/ τ equals ˆ ν s + β 1 ν + variances by 1. Therefore, the E-step of our EM algorithm is equivalent to replacing the ˆ τ ˆ ν s + β = 1+ ν, = 1,, J, (8) where ˆ β is the current estimate of β. We have incorporated our algorithm into the standard package glm in R for fitting generalized linear models. We altered the glm function by inserting the steps for calculating the augmented data (7) and updating the variances (8) into the iterative procedure in the glm function. Our algorithm is initialized by setting eachτ to a small value (say τ = 0.1) and ( β, φ ) to the starting value provided by the glm function. At each step of our algorithm, we average over the variances τ = ( τ 1,, τ J ) and then 1

13 update ( β, φ ) by maximizing the posterior density (5). In summary, our algorithm proceeds as follows: 1. Based on the current value of ( β, φ ), calculate pseudo-data z and pseudo-variances σ using (6);. E-step: replace each variance τ by its conditional expectation using (8); 3. M-step: perform the weighted least squares regression based on the augmented data to obtain the estimate ( ˆ β, ˆ φ ) using (7); 4. Repeat steps 1-3 until convergence. We apply the criterion in the glm function to assess convergence. In practice our algorithm converges rapidly. At convergence of the algorithm, we obtain all the outputs produced by the glm function, including the latest estimate ˆ β, their standard deviations and p-values (for testing β = 0 ), and some additional values (e.g., for the variances). The outputs are automatically stored as a standard glm obect. Choosing the hyperparameters: Before discussing our choice of the hyperparameters ν and s we discuss the improper Jeffreys prior on the variances, π ( τ ) τ, which is equivalent to the uniform distribution on logτ and is the limiting case of the scaled inverse- χ density withν = 0. The Jeffreys prior yields an improper posterior on each β with an infinite mode at zero. As β increases, however, a second finite local mode appears away from zero (e.g., Griffin and Brown 007; ter Braak et al 005). The property of a model with the Jeffreys prior may make it problematic to fully explore the posterior, but still formally allows the analysis of posterior modes (see Griffin 13

14 and Brown 007). Much work has shown that the Jeffreys prior leads to good performance in MCMC and mode-finding analyses, yielding strong shrinkage for very small effects but weak shrinkage for large effects (Xu 003; Bae and Mallick 004; Figueiredo 003; Kiiveri 003; Griffin and Brown 007). We choose ν and s to retain the good features of the Jeffreys prior while avoiding the drawback of impropriety of both prior and posterior. This can be achieved by setting ν and s to be small enough so that the variances ˆ τ obtained by the equation (8) are close to zero for near-zero coefficients ( β 0), but approximately equals β for ˆ large coefficients ( β ν s ˆ ). We set ν = 0.01 and s = 10-4 in our application. This prior works well and stably. However, we find that small changes from this default value (e.g., ν from 0 to 0.1, s from 0 to 10-3 ) do not affect the results. The small positive value of ν and s yields a proper prior on β sharply peaked near zero but approximately uniform for the range away from zero, and thus leads to strong sparseness in the fitted model while allowing for occasional large coefficients. Finally, our scaled ˆ inverse- χ prior includes uniform priors for β as a special case ( ν = s = ), and thus we can perform no shrinkage for certain variables (e.g., relevant covariates). Model search strategy: In principle the above hierarchical models can handle any number of variables (see the augmented regression (8), where the number of data points (n + J) is larger than the number of variables J), and thus can simultaneously estimate genetic effects of a large number of markers. For biological interpretability, however, we prefer using only a subset of the genetic effects in the model. Furthermore, including all possible effects requires large memory and intensive computation. 14

15 Therefore, we propose a model search strategy to build a parsimonious model, beginning with a model with no genetic effect but some relevant covariates if any, gradually adding different types of genetic effects into the model and then fitting the new model. It is expected that many variables have no effects on the phenotype and should be pruned from the model. We set a very small threshold value t 1 (say 10-8 ) and delete genetic effects satisfying ˆ β < t 1 from the model. Our approach differs from classical forward stepwise methods by simultaneously adding many correlated variables and deleting many near-zero variables. As described below, our search strategy can take advantage of the special correlation structure in QTL data: genotypes on same chromosomes are correlated. In summary, our approach proceeds as follows: 1. Searching for main effects: for each chromosome c ( c = 1,, ), simultaneously add all possible main effects for markers on chromosome c into the current model, fit the model and then delete some main effects based on the above mentioned criterion ;. Searching for epistatic effects among the included main effects: simultaneously add all possible epistatic interactions among the included main effects into the current model, fit the model and delete some epistatic effects; 3. Searching for epistatic effects between the excluded and the included main effects: for chromosome c ( c = 1,, ), simultaneously add all possible epistatic interactions between the remaining main effects (not included in the current model) on chromosome c and the included main effects into the current model, fit the model and delete some epistatic effects added in this step; 15

16 4. Searching for interactions between the covariate(s) and main effects: for chromosome c ( c = 1,, ), simultaneously add all possible interactions between the covariate(s) and all possible main effects on chromosome c into the current model, fit the model and delete some G E interactions added in this step. After these steps, we obtain a final model with preset covariates and genetic effects satisfying ˆ β > t 1. With our shrinkage prior, the maority of genetic effects are shrunk to zero, and thus the final model includes only a limited number of variables. As shown in our simulation studies, our model search procedure picks up true effects with reasonable high probability, but can occasionally include some extraneous effects. However, inclusion of spurious effects has little effect on detecting and estimating true or strong effects, and the estimates of spurious effects (and the corresponding p-values) are smaller (larger) than those of the true effects, allowing us to easily identify strong causal effects. Although not necessary, however, it could be useful to have a formal procedure to remove extraneous effects from the final model. A simple way is to use the p-values for testing β = 0 (equivalent to the likelihood ratio test) in the final model; we set a small threshold value t (say 0.001) and remove genetic effects with the p-values larger than t from the model. As described later, a value of t = can greatly reduce the rate of false positives. However, the p-values should be interpreted with caution, as they are based on the final model. Implementation in R/qtlbim: The freely available package R/qtlbim ( is an extensible and interactive environment for multiple QTL mapping in experimental crosses (Yandell et al. 007), built on top of the widely used R/qtl (Broman et al. 003). The previous version of R/qtlbim performs fully Bayesian 16

17 analyses via MCMC algorithms to simultaneously handle covariates, main effects, epistatic and gene-environment interactions for continuous traits, binary and ordinal traits. We have incorporated the proposed method into R/qtlbim, by creating new functions to implement our model fitting algorithm and model search procedure. The outputs for the final model are stored as a standard glm obect and thus can be summarized and displayed conventionally. SIMULATION STUDIES Simulation design: Although our method can handle various types of phenotypes and models, we here evaluate the performance of the method for a normal continuous trait and a binary trait. We expect that the basic conclusions hold for other cases. In our simulations, we generated the continuous phenotypes based on different multiple QTL models and the binary trait by transforming the continuous values into two categories with the proportions of 40 and 60%, respectively. We generated a genome with 19 chromosomes, each of length 100 cm. Five percent of the marker genotypes were assumed to be randomly missing. The first experiment was an F intercross composed of 400 progenies and having 11 equally spaced markers (placed 10 cm apart) on each chromosome. We considered a model with one binary covariate and five QTL with only main effects. The covariate effect explained ~ 4% of the variance of the continuous phenotype. Table 1a presents the simulated positions of five QTL, their additive or dominance effects, and the approximate heritabilities (proportion of the phenotypic variation explained by an effect). As can be seen, two of five simulated QTL (Q and Q4) were positioned at markers, and others 17

18 were in marker intervals. The second simulation was the same as the first one except that each chromosome had 51 equally spaced markers (placed cm apart). Therefore, these two simulations had 09 and 969 total markers, respectively. Our third experiment was a backcross (BC) with 400 progenies and 11 equally spaced markers on each chromosome. We simulated one binary covariate and eight QTL with complex interactions; among the eight simulated QTL, five had main effects while other three had no main effects but interacted with other QTL or the covariate. The covariate effect explained ~ % of the variance of the continuous phenotype. Table 1b presents the positions of simulated QTL, the genetic effects (main effects, epistasis and G E), and the approximate heritabilities. The fourth simulation was the same as the third except that each chromosome had 51 equally spaced markers. (insert Table 1 here) Analysis and evaluation: Under each scenario, we carried out 00 simulation replicates. Each of the F simulated data sets was analyzed using two models; the first included a covariate term and main effects of all markers, and the second allowed not only the covariate and main effects but also epistatic and G E interactions. The aim of the second analysis is to examine whether or not we detect interactions if they are not present. For each of the simulated BC data sets, we fit a model with a covariate effect, main effects of all markers, epistatic and G E interactions. For all analyses, the continuous and the binary traits were analyzed using Gaussian and Probit models, respectively. We tried several values for the hyperparameters ν from 0 to 0.1 and s from 0 to 10-3, and got similar results. We displayed the results with ( ν, s ) = (0.01, 10-4 ). To 18

19 investigate the influence of two thresholds t 1 and t on our model search method (see the last section), we analyzed the data sets using several values (t 1 = 10-4, 10-6, 10-8, 10-10, 10-1, and t = 1, 0.01, 0.005, 0.001). We evaluated the performance of our method by examining the frequency of each effect included in the final model, true positives (the frequency of true effects included in the final model), and false positives (the frequency of false effects included in the final model) over the 00 simulation replicates. The inclusion frequency of an effect presents the empirical power to detect the effect. A detected main or G E effect (episttic effect) is considered as correct if the associated position(s) is within 10 cm of any true QTL. Results: The threshold t 1 controls the minimal size of genetic effects to enter the models. We found that the final models almost picked up the same sets of effects for the above values of t 1 in all the simulation experiments, and thus the choice of t 1 had little influence on our model search method. Therefore, we only present the results for t 1 = 10-8 below. Figure 1 displays the genetic effects included in the final models with frequencies greater than 0.01 for the four values of t in the first simulation experiment. We can see that no false effect (defined above) was included the final models with frequency greater than For the continuous trait, the simulated main effects located at markers, (1@60)d and (@50)a, were detected with frequencies close to 1. As expected, the analyses of the binary trait had lower power than the continuous trait, especially for the two linked QTL on chromosome. For the simulated effects located within the marker intervals, our analyses picked up the corresponding effects of the flanking markers in the final models; for example, for the simulated effect (@3)a, the final models included 19

20 or For these simulated effects, the true positives that roughly equal the sum of the inclusion frequencies of the corresponding flanking marker effects were also high. Although no interactions were present, the analyses allowing epistatic and G E effects did not reduce the power for detecting the main effects and did not include any interactions with high frequency. Finally, it can be seen that a smaller threshold t slightly reduced the power. However, this threshold controls the rate of false positives which will be illustrated shortly. (insert Figure 1 here) Figure shows the results of the analyses for the second experiment. The models including or excluding epistatic and G E interactions produced similar results again. Our second experiment simulated 51 markers evenly distributed on each chromosome, resulting in much higher correlation among the variables. Consequently, the simulated effects were included with lower frequencies than in the first experiment. However, we were still able to identify the true effects with reasonable high power and true positives for both the continuous and the binary traits. For example, the first simulated effect (1@1)a was identified with power of ~ 60% (40%) and true positives of ~ 90% (80%) for the continuous (binary) trait. (insert Figure here) The results for the BC simulations are illustrated in Figure 3. For the experiment with 11 markers on each chromosome, no false effect was included the final models with frequency greater than For the continuous trait, all the simulated main effects, epistatic and G E interactions were identified almost in all 00 replicates. For all the simulated main effects and epistatic interactions, the analyses of the binary trait had 0

21 lower power; the reduction of the power was larger for the main effects of the two relatively tightly linked QTL on chromosome and epistasis. For the fourth experiment where 51 markers were evenly placed on each chromosome, most of the simulated effects were included in the final models with frequencies greater than 60% for the continuous trait. Higher correlation among the variables resulted in the method to occasionally pick up effects with associated positions near to the true effects. Again, the true positives for all the simulated effects were high. Finally, it was observed that the threshold t had little influence on the power of detecting the true effects. (insert Figure 3 here) Figure 4 shows the false positives for the BC experiments under different p-values t. The other experiments gave similar results. With a large value t, the models included extraneous effects with high frequencies in the simulation replicates. Data with dense marker maps generally produced lower false positives. False effects were removed from the models rapidly as the threshold t decreased, indicating that false effects were associated with larger p-values. For all the cases, we observed that the estimates of the simulated effects were close to the corresponding true values. But the estimates of spurious effects were much smaller. (insert Figure 4 here) REAL DATA EXAMPLES We illustrate our method with three real QTL mapping experiments. These data sets have been extensively analyzed previously, and thus serve as good examples to compare our method with the existing methods and to illustrate the advantages of our 1

22 method in terms of statistical and computational efficiencies and modeling flexibility. We used several values for ν (between 0 and 0.1) and s (between 0 and 10-3 ), and obtained identical results. We displayed the results by setting the thresholds t 1 = 10-8 and t = 1. Barley dataset: The barley dataset (Tinker et al. 1996) was reanalyzed for demonstration. The data were collected from a doubled haploid population that contained n = 145 lines; each was grown in 5 different environments. The phenotype analyzed was the average value of kernel weight across environments. A total of 17 markers spanning seven chromosomes were genotyped. The data contained ~5% missing marker genotypes. We used a Gaussian model to fit main effects of all markers and their epistatic interactions. The total number of effects was 819 (17 main effects and 17 16/ = 8001 epistatic effects). Our analysis took ~0.05 minutes on a P4 computer. We obtained a final model with eight main effects, three on chromosome 1, two on chromosome, one on chromosomes 3, 4 and 7, respectively. We did not detect epistatic effects. The estimates of the main effects are displayed in the top panel of Figure 5. The p-values for these effects were 1.3e-07, 3.3e-15, 1.4e-03, 4.3e-05, 6.9e-04, 9.e-05, 3.3e-03, and 3.1e- 4, respectively. If we set t = 0.001, the effects 1@1.1 and 4@145.5 are removed from the final model. For comparison, we reanalyzed the data using the MCMC method (Yi et al. 005; Yi et al. 007a) implemented in R/qtlbim (Yandell et al. 007). Our MCMC analysis also did not detect epistatic effects, but multiple main effects with positions overlapping or close to those in the proposed method analysis (see the bottom panel of Figure 5). (insert Figure 5 here)

23 This dataset has been extensively analyzed using different methods, including mode-finding algorithms (Xu 007; Xu and Jia 007) and shrinkage MCMC methods (Xu 003; Yi and Xu 008). Xu (007) (also see Xu and Jia 007) analyzed this dataset using an empirical Bayes method and the LASSO of Tibshirani (1996). Both the analyses took ~ minutes and obtained the results similar to ours. The shrinkage MCMC algorithms also gave the similar results, but took a few hours (Xu 003; Yi and Xu 008). Therefore, our method performs comparably with the alternative sophisticated techniques, and is computationally faster. Listeria monocytogenes dataset: As a second study we illustrate the application of our method by re-analyses of the mouse data of Boyartchuk et al. (001). This dataset consisted of 116 female mice from an intercross (F ) between the BALB/cByJ and C57BL/6ByJ strains. Each animal was genotyped at 133 markers spanning 0 chromosomes. The phenotype of interest was the time to death following infection with Listeria monocytogenes. The animals that died from infection had a mean survival of hours. Roughly 30% of the mice recovered from the infection and survived past the 64-hour time point. Several methods have been applied to this dataset, including twopart model (Broman 003), parametric proportional hazards model (Diao et al. 004) and a unified semi-parametric framework approach (Jin et al. 007). These analyses detected significant QTL on chromosomes 1, 5 and 13, and suggestive QTL on chromosome 15. We applied our method to the data using the two-part model of Broman (003) rather than developing a new model for this type of phenotype. The two-part model decomposes the phenotype into a binary trait, indicating whether the subect survived, and a continuous trait for time to death for those dying within 64 hours (Broman 003). 3

24 We used a Probit regression model for the binary phenotype and a Gaussian model for the logarithm of the continuous phenotype, and fit a model with main effects of all markers and their epistatic interactions. The total number of effects was (133 additive effects, 133 dominance effects, and 3511 epistatic effects). Our analyses took ~0.0 and 0.03 minutes on a P4 computer, for the continuous and the binary traits, respectively. Our results were consistent to the previous ones ( Figure 6). For the continuous trait, the final model included two additive effects on chromosomes 1 and 15, respectively, and one dominance effect on chromosome 13. The p-values for these effects were 3.8e-05, 1.8e-0 and 1.6e-03, respectively. For the binary trait, the final model included two additive effects on chromosomes 5 and 13, with the p- values being 1.7e-07 and 1.9e-04, respectively. We did not detect epistatic effects. (insert Figure 6 here) We also analyzed the data using the Bayesian MCMC method (Yi et al. 005; Yi et al. 007a, b; Yandell et al. 007). The MCMC algorithm detected additive effects on chromosomes 1 and 15 and dominance effect on chromosome 13 for the continuous trait, and additive effects on chromosomes 5 and 13 for the binary trait (not shown here). The positions of these QTL were close to the markers detected by the proposed method. In addition these main effects, the fully Bayesian analysis found evidence of epistatic effects; for the continuous trait, QTL on chromosomes 1, 5 and 9 showed evidence of epistasis, and for the binary trait, chromosomes 6 and 13 harbored epistatic QTL. However, these epistatic effects were not strong. The MCMC analyses took ~ 8 and 10 minutes for the continuous and the binary traits, respectively. 4

25 Obesity dataset: We also applied our method to an obesity dataset of backcross mice. This dataset has been extensively analyzed using Bayesian MCMC methods (Yi et al. 005; Yi et al. 006; Yi et al. 007a). The cross was produced from two highly divergent strains: M16i, consisting of large and moderately obese mice, and CAST/Ei, a wild strain of small mice with lean bodies. CAST/Ei males were mated to M16i females, and F 1 males were backcrossed to M16i females, resulting in 41 mice (13 males and 08 females) reaching 1 wk of age. All mice were genotyped for 9 markers located on 19 autosomal chromosomes. We analyzed a continuous trait Fat, the sum of right gonadal and hindlimb subcutaneous fat pads. We included sex and body weight at the age of 1 weeks as covariates, and permitted the inclusion of epistatic and gene-sex interactions in a Gaussian regression model. The total number of genetic effects was 4370, including 9 main effects, 4186 epistatic effects and 9 gene-sex interactions. Using the MCMC algorithm (took ~15 minutes), Yi et al. (007a) detected evidence of QTL on 9 chromosomes (1,, 6, 8, 13, 14, 15, 18, and 19). These QTL showed a complex network of interactions. The strongest main-effect QTL was found on chromosome, and QTL on chromosomes 8, 13, 14, 15, and 19 also had main effects. Strong epistatic interactions between chromosomes 15 and 19 and chromosomes 1 and 18 were detected. Evidence of epistasis was found between a few other chromosome pairs. The QTL on chromosome was found to interact with the covariate sex. Our hierarchical generalized linear model analysis took ~0.15 minutes, and obtained a final model including 1 main effects, five epistatic effects, and two gene-sex interactions. The estimates of the genetic effects and their p-values are displayed in Figure 7. Our results were basically consistent to the previous full Bayesian analysis; all 5

26 strong effects were detected by the both analyses. All the main-effect QTL detected in Yi et al. (007a) were also identified in our reanalysis, with positions in the regions detected previously. The p-values for most of the main effects were smaller than Main effects of two linked markers on chromosome were simultaneously included in the model, which were the largest among the detected main effects. The epistatic interactions, and were the strongest, which were also detected previously. Main effects on both chromosomes 1 and 18 were not included in the final model, indicating that our method can detect QTL with weak main effects but strong epistasis. Two gene-sex interactions, (@73):SEX and (15@1):SEX, were included in the final model with p-values being 1.7e-03 and 4.4e-03, respectively. (insert Figure 7 here) DISCUSSION We propose a new statistical framework for genome-wide multiple QTL mapping in experimental crosses. Our method is developed based on the generalized linear model framework and thus can deal with various phenotypes that the standard generalized linear models can analyze. The proposed models assume sparseness-inducing priors and thus can simultaneously fit a large number of effects, including environmental effects, main effects of numerous genomic loci, epistatic and G E interactions. By looking for the modes of the posterior distribution via the EM algorithm we are able to quickly identify significant effects. Experiments with extensive simulation studies and real datasets have shown good performance. We assign independent normal distributions with unknown variances to the coefficients, and devote our attention to the scaled inverse- χ distribution for the 6

27 variance parameters. Our prior includes the noninformative Jeffreys prior as a special case, which has the form of a scaled inverse- χ density with 0 degrees of freedom. As already shown in previous studies (Figueiredo 003; Kiiveri 003; Xu 003; Bae and Mallick 004; Griffin and Brown 007) and in our experimental results, the Jeffreys prior usually yields good performance, inducing strong sparseness in the fitted model. However, we recommend choosing the hyperparameters in the scaled inverse- χ to yield a proper prior on coefficients sharply peaked near zero but approximately uniform for the range away from zero. Our prior induces strong shrinkage for near-zero effects but weak shrinkage for large effects. Our method can be in principle extended to models with other prior distributions, for example, the exponential distributions with fixed hyperparameters. However, it would be appealing to treat the hyper-parameters as unknowns and estimate them from the data so that the model can shrink the coefficients as much as can be ustified by the data. Yi and Xu (008) developed MCMC algorithms to ointly estimate all hyperparameters and model parameters, and found that such procedures usually outperform those with fixed hyperparameters. Future research will extend the proposed EM algorithm to estimate hyperparameters. Gelman et al. (008) proposed independent Cauchy distribution with mean 0 and scale.5 (i.e., β τ ~ N( µ, τ ) and τ ~ Inv- χ (1,.5) ) for routine data analysis, and implemented a procedure to fit generalized linear models with this prior by incorporating an EM algorithm into the usual iteratively weighted least squares. However, this Cauchy prior cannot shrink small effects to zero and thus cannot be used for our high-dimensional models. Our algorithm is similar to Gelman et al. (008), but differs in regarding the variances rather than the coefficients as missing data. The trick of the algorithm is to 7

28 express the prior information on coefficients as additional data points, allowing us to take advantage of the existing iteratively weighted least squares algorithm. Various other efficient methods have been proposed recently, but are built upon special algorithms and thus more difficult to implement (e.g., Tibshirani 1996; Efron et al. 004; Genkin et al. 007; Xu 007). Rather than fitting all possible effects in a large model, we propose a novel model search strategy to build a parsimonious model. Our method begins with a model with no genetic effect, and then adds different types of genetic effects into the model and deletes very small effects. Unlike traditional variable selection approach, we simultaneously add or delete many variables and take care of the correlation among the variables. Kiiveri (003) and Zhang and Xu (005) adopted a similar criterion for pruning effects from the model, but used backward selection procedure. Our search strategy avoids the need of large memory and computational problems when the number of effects is huge, and thus facilitates the genome-wide interacting QTL analysis. The fully Bayesian approach, employing the MCMC simulation to generate posterior samples from the oint posterior distribution of all unknowns, can provide more comprehensive posterior inferences (Gelman et al. 003). In QTL mapping, the fully Bayesian methods can easily treat missing genotypes and positions of QTL as unknowns and use model averaging to induce robustness and stability (Yi et al. 005; Yi et al. 007a, b; Yi and Shriner 008). We describe our method by allowing only observed markers as potential QTL and replacing missing marker genotypes by their conditional expectations. We can extend our method to detect QTL within marker intervals by inserting loci within each marker intervals (pseudo-markers) as possible positions of 8

29 QTL. It may deserve further investigation on how the number and the spacing of pseudomarkers affects the performance of the method. The proposed method has computational advantages over the fully Bayesian MCMC methods. High-order interactions could be implemented using our approach, but this would substantially increase the model space to be explored and thus may need further investigation. Future research will extend our method to genome-wide association analysis and gene expression QTL (eqtl) mapping. Fast and efficient algorithms are an essential feature for the practical analysis of these more complicated cases. ACKNOWLEDGMENTS This work was supported by the National Institutes of Health (NIH) Grants R01 GM to NY and CTSC GRANT UL1-RR04996 to SB. LITERATURE CITED Baierl, A., M. Bogdan, F. Frommlet and A. Futschik, 006 On locating multiple interacting quantitative trait loci in intercross designs. Genetics 173: Bogdan, M., Ghosh, J. K., and R. W. Doerge, 004 Modifying the Schwarz Bayesian information criterion to locate multiple interacting quantitative trait loci. Genetics 167: Broman, K. W. and T. P. Speed, 00 A model selection approach for identification of quantitative trait loci in experimental crosses. J. R. Statist. Soc. B. 64: Broman, K. W., Wu, H., Sen, Ś., and G. A. Churchill, 003 R/qtl: QTL mapping in experimental crosses. Bioinformatics 19: Broman, K. W., 003 Mapping quantitative trait loci in the case of a spike in the phenotype distribution. Genetics 163:

30 Bao, K. and B. K. Mallick, 004 Gene selection using a two-level hierarchical Bayesian model. Bioinformatics 0: Boyartchuk, V. L., Broman, K. W., Mosher, R. E., D'Orazio S. E. F., Starnbach, M. N. and Dietrich, W. F., 001 Multigenic control of Listeria monocytogenes susceptibility in mice. Nature Genetics 7: Carlborg, Ö. and C. Haley, 004 Epistasis: too often neglected in complex trait studies? Nat. Rev. Genet. 5: Diao, G., Lin, D. Y., and F. Zou, 004 Mapping quantitative trait loci with censored observations. Genetics 168: Efron, B., Hastie, T., Johnstone, I., and R. Tibshirani, 004 Least angle regression. Annals of Statistics 3: Figueiredo, M. A. T., 003 Adaptive sparseness for supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 5: Gelman, A., Carlin, J., Stern, H., and D. Rubin, 003 Bayesian data analysis. Chapman and Hall, London. Gelman, A., Jakulin, A., Pittau, M. G., and Y. S. Su, 008 A weakly informative default prior distribution for logistic and other regression models. Annals of Applied Statistics (in press). Genkin, A., Lewis, D. D., and D. Madigan, 007 Large-scale Bayesian logistic regression for text categorization. Technometrics 49: Griffin, J. E. and P. J. Brown, 007 Bayesian adaptive lassos with non-convex penalization. Technical report, University of Warwick. Haley, C. S. and S. A. Knott, 199 A simple regression method for mapping quantitative trait loci in line crosses using flanking markers. Heredity 69: Jiang, C., and Z-B. Zeng, 1997 Mapping quantitative trait loci with dominant and missing markers in various crosses from two inbred lines. Genetica 101: Jin, C., Fine, J. P., and B. S. Yandell, 007 A unified semiparametric framework for quantitative trait loci analysis, with application to spike phenotypes. Journal of the American Statistical Association 10: Kao, C. H., Zeng, Z-B., and R. D. Teasdale, 1999 Multiple interval mapping for quantitative trait loci. Genetics 15: