An efficient sampling method for stochastic inverse problems

Transcription

1 Comput Optim Appl (2007) 37: DOI /s An efficient sampling method for stochastic inverse problems Pierre Ngnepieba M.Y. Hussaini Received: 23 June 2005 / Revised: 15 November 2005 / Published online: 1 May 2007 Springer Science+Business Media, LLC 2007 Abstract A general framework is developed to treat inverse problems with parameters that are random fields. It involves a sampling method that exploits the sensitivity derivatives of the control variable with respect to the random parameters. As the sensitivity derivatives are computed only at the mean values of the relevant parameters, the related extra cost of the present method is a fraction of the total cost of the Monte Carlo method. The effectiveness of the method is demonstrated on an example problem governed by the Burgers equation with random viscosity. It is specifically shown that this method is two orders of magnitude more efficient compared to the conventional Monte Carlo method. In other words, for a given number of samples, the present method yields two orders of magnitude higher accuracy than its conventional counterpart. Keywords Monte Carlo method Data assimilation Error covariance matrix Sensitivity derivatives Burgers equation 1 Introduction Optimization problems under uncertain conditions have become a subject of increasing attention among scientists and engineers in the recent times [6, 9, 11]. First, such problems are more realistic and of real-world application value considering that the physical parameters, the initial and boundary conditions, and the data for validation or assimilation are random in nature and hence sources of uncertainty in reality. Second, despite the extra dimension of complexity due to randomness (which prohibited P. Ngnepieba Department of Mathematics, Florida A&M University, Tallahassee, FL 32307, USA M.Y. Hussaini ( ) School of Computational Science, Florida State University, Tallahassee, FL , USA susan@cespr.fsu.edu

2 122 P. Ngnepieba, M.Y. Hussaini attempts at solutions of such problems till recently), these problems have become tractable because of the high efficiency of the current numerical algorithms coupled with the high performance of the present day computing machinery. A general method of solving optimization problems with uncertain parameters is obviously a Monte Carlo method. It requires thousands of samples, and each sample in the present context requires solving partial differential equations. Its convergence is inversely proportional to the square root of the number of samples and directly proportional to the variance of the integrand. Various variance reduction methods have been proposed to accelerate convergence of these methods. Recently, Cao et al. [1, 2] proposed a sensitivity derivative based Monte Carlo method, which coupled with other traditional variance reduction technique provides several orders of magnitude improvement in convergence. The problem under consideration is an inverse problem involving the minimization of the cost function, which is the norm of the difference between the numerical solution and the observation, by controlling the initial condition. Optimal control of initial conditions is crucial for the predictive methodologies, for example, in meteorology. Since the original work of Le Dimet and Talagrand [7], it has become ubiquitous in atmospheric and oceanic sciences [4] where it is called the variational data assimilation problem solved by an optimal control technique. However, it should be pointed out that the present formulation carries over in a straightforward fashion to any inverse problem governed by partial differential equations (with random parameters) where the control variables are boundary conditions or physical parameters. A semidiscrete version of such problems reduces to the form discussed in the next section. The proposed method embeds the optimization process inside the outer Monte Carlo process. Then it leverages the (sensitivity derivative) information from the inner process to accelerate the convergence of the outer Monte Carlo sampling process. In other words, the proposed method efficiently computes the sensitivity derivative of the control variable (which is the optimal initial condition in the present instance) with respect to the random parameter. It uses the optimal initial condition and its sensitivity derivative to obtain the evolution of the state variable and its sensitivity derivative. On the other hand, Cao et al. [1, 2] assume the availability of the sensitivity derivative of the state variable with respect to the random parameter and exploit it to achieve convergence acceleration of Monte Carlo method by employing the residual in the first-order Taylor series expansion of the cost functional in terms of the stochastic parameter rather than the cost functional itself. 2 Problem statement and solution technique Consider the evolution equation dx = F(X,ξ), t (0,T), dt X(0) = u (2.1) where ξ is a parameter, X(t; u, ξ) is the state variable that belongs, for any t, to a Hilbert space X, u X is the control variable, and F is a nonlinear operator mapping X into X.

3 An efficient sampling method for stochastic inverse problems 123 We pose an inverse problem whose goal is to find a control u so that X is close to the observation/data X obs. To this end, we define the cost function J : T J(u,ξ)= 1 CX(t; u, ξ) X obs 2 2 X obs dt (2.2) 0 where the observation space X obs is a Hilbert space, and C : X X obs is a linear bounded operator. The cost function defines the discrepancy between the simulated value X(t; u, ξ) and the corresponding observation X obs, and implicitly depends on u and ξ through X. Then we define the following inverse problem: For a given ξ, find u X such that dx = F(X,ξ), t (0,T), dt X(0) = u, (2.3) J(u,ξ)= inf J(v,ξ). v 2.1 Problem statement If the parameter ξ in the above inverse (data assimilation) problem (2.3) is a random field defined on the probability space (, F, P), how does one quantify the uncertainty in the optimal initial condition u and X? In other words, given the pdf of the random field ξ, how does one determine the pdf of u and X, or their statistics such as their expectations and variances. An obvious approach is to combine an optimization method with a statistical method for uncertainty quantification. Such an approach would nest the optimization process of the control problem (inner loop) in the uncertainty quantification process (outer loop), say, a conventional Monte Carlo process. This becomes prohibitively expensive as the convergence of the Monte Carlo process may require a huge number of samples. However, it is possible to leverage information from the inner process/iteration to accelerate the outer process/iteration and vice versa. We propose an efficient sampling method to address this strategy. Efficiency stems from the exploitation of the so-called sensitivity derivative information, specifically the Jacobian of the optimal initial condition with respect to the random parameter, which is computed during the inner iteration process. Before proceeding to describe the method, we provide here, for the purpose of completeness, a brief description of optimality system. 2.2 Solution technique Let ξ denote a sample of real-valued random field ξ generated according to the pdf P. For this value of ξ, the optimization problem defined by (2.3) is deterministic. If J has a minimum then the optimality condition u J(u, ξ)= 0 holds. The gradient of the cost function J is obtained by using the adjoint model [7, 8]: dx = F(X, ξ), t (0,T), dt X(0) = u, (2.4)

4 124 P. Ngnepieba, M.Y. Hussaini dx [ ] F T + dt X (X, ξ) X = C T (CX X obs), t (0,T), X (T ) = 0, u J(u, ξ)= X (0) (2.5) with the unknowns X, X, and u, where [ F X (X, ξ)] T is the adjoint to the Frechet derivative of F, and C T is the adjoint to C defined by (CX, Y ) Xobs = (X, C T Y) X, X X, Y X obs. The optimality condition becomes u J(u, ξ)= X (0) = 0. (2.6) The solution of the minimization problem defined by (2.3) may be found by using the Newton s method: u k+1 = u k [ 2 u J(u k, ξ) ] 1. u J(u k,ξ) (2.7) where u k is the current estimate. The minimization procedure employs the limited-memory quasi-newton minimization algorithm [5] based on the BFGS (Broyden Fletcher Goldfarb Shanno) update formula for the inverse Hessian. The solution u ( ξ) of the optimality system ( ) (if it exists) is implicitly dependent on ξ through X. In a standard Monte Carlo method, samples {ξ i } N i=1 are generated by pdf P. For each ξ j, the solution of the optimality system yields the corresponding optimal initial condition u j u (ξ j ). The sample mean of the u j s is a natural estimator of E[u ]: E[u ]= 1 N N u j. It also yields histograms of the u j s that are the natural estimator of pdf of u and other statistics of u. Simultaneously, the evolution equation (2.4) is solved with the initial condition u j, which yields the mean and variance of the state variable X in some norm. For satisfactory convergence, a prohibitively large number of samples are required. A drawback is that the optimality system has to be solved prohibitively large number of times, which is not feasible for realistic problems. The present method first obtains the mean and covariance (or variance) of the optimal initial condition by efficiently constructing an approximate Jacobian matrix ξ u around the mean value of the random field ξ. Using this mean optimal initial condition and its Jacobian, the evolution equations for X and X ξ are solved, where from the mean and covariance of X are obtained. The procedure is as follows. Linearizing the random field ξ around its mean value E[ξ] (first-order Taylor expansion), we have j=1 u (ξ) u (E[ξ]) + [ ξ u (E[ξ]) ] (ξ E[ξ]). (2.8) Taking the expectation of both sides of (2.8) using the pdf of ξ, we get E[u ]=u (E[ξ]). (2.9)

5 An efficient sampling method for stochastic inverse problems 125 An approximate covariance matrix for u is obtained using (2.8): Cov(u ) = E [ (u E[u ])(u E[u ]) T ] =[ ξ u ] [ ξ u ] T (2.10) where = E[(ξ E[ξ])(ξ E[ξ]) T ] is the error covariance matrix of ξ. Once the Jacobian matrix ξ u is known, the pdf P u of the optimal control u can be approximated by the Gaussian distribution with the mean E[u ] defined by (2.9) and the covariance matrix given by (2.10). To obtain the mean E[X] of the state variable, we expand the state variable X(t; u (ξ), ξ) in a first-order Taylor series around the mean value E[ξ], X(t; u (ξ), ξ) X ( t; u (E[ξ]), E[ξ] ) + [ X ξ (t; u (E[ξ]), E[ξ]) ] (ξ E[ξ]) (2.11) and take the expectation of both sides of the equation, which yields E[X]=X ( t; u (E[ξ]), E[ξ] ) (2.12) where X(t; u (E[ξ]), E[ξ]) is the solution of the evolution equation, dx = F(X,E[ξ]), t (0,T), dt X(0) = u (E[ξ]). We obtain Cov(X) by using (2.11) in the definition of covariance: Cov(X) = E [( X ( t; u (ξ), ξ ) X ( t; u (E[ξ]), E[ξ] )) ( X ( t; u (ξ), ξ ) X ( t; u (E[ξ]), E[ξ] )) T ] = [ X ξ ( t; u (E[ξ]), E[ξ] )] [ X ξ ( t; u (E[ξ]), E[ξ] )] T (2.13) where X ξ (t; u (E[ξ]), E[ξ]) is the solution of the linear equation, [ dx ξ F ( = X(t; u (E[ξ]), E[ξ]) )] X ξ dt X [ F ( + X(t; u (E[ξ]), E[ξ]) )], t (0,T), ξ X ξ (0) = [ ξ u (E[ξ]) ]. The next section describes the computation of the Jacobian matrix ξ u Approximation of the Jacobian matrix ξ u Equation (2.8) can be written as P = BQ (2.14) where B is the Jacobian matrix ξ u, P ={p (j) }={u (ξ j ) u (E[ξ])}, j = 1, 2,...,m, and Q ={q (j) }={ξ j E[ξ]}, j = 1, 2,...,m. The vectors p (j) =

6 126 P. Ngnepieba, M.Y. Hussaini u (ξ j ) u (E[ξ]), j = 1, 2,...,m are obtained by solving the optimization problems (2.3) with ξ = ξ j, j = 1, 2,...,m; q (j) = ξ j E[ξ], j = 1, 2,...,mare simply obtained by sampling of ξ using its pdf P. The algorithm to compute the matrix B is explained in [3]. It is briefly described here. To find B that approximately maps Q into P, we find a least square solution to the matrix equation (2.14), by solving the following convex quadratic programming problem: Minimize P BQ 2 F (2.15) B where X 2 F = trace(xxt ) denotes the Frobenius norm of X. The column vectors b (j) of the matrix B are found by solving the linear system QQ T b (j) = PQ T (j) with PQ T (j) jth columns of the matrix PQ T. 3 Applications The present methodology is applied to two cases. The first case is a linear model problem which has an analytical solution. This simple example demonstrates that the sensitivity of the optimal initial condition to the random parameter is highly nonlinear even though the problem is linear. It also serves to validate our methodology. Second, we consider an inverse problem governed by the Burgers equation. 3.1 Linear model Consider the following data assimilation problem where the state variable is the solution of a linear equation dx = ξx, t (0, 1), dt X(0) = u, (3.1) J(u,ξ)= inf J(v,ξ) v where ξ is scalar real-valued random variable with known pdf P, and the cost function J is defined by 1 J(u,ξ)= 1 (X(t; u, ξ) X obs ) 2 dt. 2 0 The optimality system associated with the minimization of the cost function J is dx = ξx, t (0, 1) dt X(0) = u, dx + ξx (3.2) = (X X obs ), t (0, 1), dt X (1) = 0, u J(u,ξ)= X (0) = 0

7 An efficient sampling method for stochastic inverse problems 127 where X is the adjoint of X. The explicit solution of (3.2) is X(t;,u,ξ)= ue ξt, (3.3) X (t; u, ξ) = u 2ξ The gradient of J with respect of u is given by ( e ξt e ξt e 2ξ ) X obs ( 1 e ξt e ξ ). (3.4) ξ u J(u,ξ)= X (0; u, ξ) = u 2ξ ( e 2ξ 1 ) + X obs ( 1 e ξ ). ξ Thus, the optimal initial condition u (which is the solution of u J(X(u,ξ))= 0) is u = u (ξ) = 2X obs e ξ =: f(ξ). (3.5) + 1 In order to obtain the pdf, P u, of the optimal initial condition u, we consider the cumulative density function (cdf), F u, of the optimal initial condition u, which is defined as (we choose X obs = 1 for convenience) F u (z) = P u (u (ξ) z) = P(f (ξ) z) = 1 P(f (ξ) > z) = 1 P ( ξ<f 1 (z) ) = 1 F ξ ( f 1 (z) ) since f 1 is monotone decreasing on R. This implies The explicit derivation gives P u (x) = { 1 F ξ (f 1 (x)) }. P u (x) = P ( f 1 (x) ) (f 1 ) (x). Substituting (f 1 ) and f 1, we obtain P u (x) = 2 P log((2 x)/x)), x (0, 2). (3.6) x(2 x) Equation (3.6) gives the exact expression for the pdf of u which depends on the pdf of the random variable ξ. For instance, if ξ is normally distributed with mean μ and variance σ 2, then P(x) = e ((x μ)2 /(2σ 2 )) σ, x R. 2π Using (3.6), we get the pdf of the optimal initial condition as 2e (log((2 x)/x) μ)2 /(2σ 2 ) P u (x) = σ, x (0, 2) 2πx(2 x) 0, otherwise. (3.7)

8 128 P. Ngnepieba, M.Y. Hussaini The exact expressions for the mean and the variance of the optimal initial condition are E[u ]= 2 2 σ e 1 ( log((2 x)/x) μ ) 2 2 σ dx, 2π 0 (2 x) V(u ) = 2 2 σ (x E[u ]) 2 e 1 ( log((2 x)/x) μ ) 2 (3.8) 2 σ dx. 2π x(2 x) Validation of the present method 0 Choosing ξ N (μ, σ 2 ), μ = 2, σ = 0.1, the exact values of the mean and variance (computing the integrals in (3.8) using quadrature (trapezoidal) with 1000 points) are E ex [u ]= , V ex (u ) = E 4. The present method yields the mean E ap [u ] (using (2.9) and (3.5)) E ap [u 2 ]= e E[ξ] = (independent of samples), (3.9) and the variance V ap (u ) (using (2.10)) 4e 2E[ξ] V ap (u ) = [ f (E[ξ]) ] 2 V(ξ) = V(ξ). (3.10) (e E[ξ] 4 + 1) The absolute error in mean E mean = E ex [u ] E ap [u ] = 7.9E 4. Using the exact value of [f (E[ξ])], we find V ap (u ) = E 4, the absolute error in variance E V = V ex (u ) V ap (u ) = E 6. Here, we use the explicit expressions for the optimal initial condition as a function of the random parameter and its derivative with respect to ξ at its mean value, i.e., u (ξ) = f(ξ)= 2 e ξ + 1 and u ξ (E[ξ]) = f (E[ξ]) = 2eE[ξ] (e E[ξ] + 1) 2. Thus, these error estimates are the lower bound of approximation errors of the present method Approximation of the derivative [f (E[ξ])] using samples The present method provides the following approximation of the sensitivity of u with respect to ξ: [ N ][ f PM (E[ξ]) = N p k q k ( q k ) ] 1 2 (3.11) k=1 where p k = u (ξ k ) u (E[ξ]) and q k = ξ k E[ξ], k= 1, 2,...,N.Using(3.11), the variance of u is V PM (u ) = [ f PM (E[ξ])] 2 V(ξ). (3.12) Figure 1 plots the error E var = V ex (u ) V(u ) as a function of the sample size N, where V(u ) is obtained by the Monte Carlo method and the present method. For k=1

9 An efficient sampling method for stochastic inverse problems 129 Fig. 1 Error in variance of optimal initial condition vs. sample size a given sample size, the corresponding absolute value of the error (in both cases) is computed with an ensemble average of 100 realizations of ξ. It is apparent that the error in both methods decays at the expected 1/ N rate. The present method achieves an order of magnitude reduction in error. However, as N increases beyond (N 10 4 ), the error (swamped by the linear tangent approximation error) remains constant at E Probability density function of the optimal initial condition u (ξ) The exact pdf of u is known from (3.7): 2 x ((log( 2e x ) μ)2 /(2σ 2 )) P u (x) = σ, x (0, 2), 2πx(2 x) 0, otherwise where μ = 2 and σ = 0.1. The present method yields the mean of the optimal initial condition (3.9): E PM [u ]=E ap [u ]= 2 e E[ξ] + 1 = (independent of samples) and the values of variance V PM = E 4, E 4, and E 4 for N = 10 3,10 4, and 10 5 respectively. Using this mean and these values of variance and

10 130 P. Ngnepieba, M.Y. Hussaini assuming a normal distribution, the present method constructs the pdf of the optimal initial condition u, which is compared with the exact pdf of u (3.7)inFig.2. The pdf of u obtained by the present method at N = 10 3,10 4, and 10 5 are indistinguishable, which is not surprising in that the difference in the values of variance is negligibly small. The agreement with the exact pdf is excellent except for the slight discrepancy at the tail Probability density function of the state variable X(t; u (ξ), ξ) We have from (3.3) X(t; u (ξ), ξ) = u (ξ)e tξ = 2etξ e ξ + 1 =: g t(ξ), t (0, 1). Using the present method, we have from (2.12) the mean of X: 2etE[ξ] E[X]=X ( t; u (E[ξ]), E[ξ] ) = e (E[ξ]) + 1 (3.13) and from (2.13) the variance of X: V(X) = [ g t (E[ξ])] 2 V(ξ) (3.14) Fig. 2 Probability density function of u

11 An efficient sampling method for stochastic inverse problems 131 where g t (x) = 2ext (e x + 1) 2 (ex (t 1) + t). Then, the exact pdf of the state variable is given by P t X (z) = P( g 1 t (z) )( gt 1 ) (z) P(g 1 t (z)) = g t(g 1 t (z)). (3.15) The inverse (g 1 t (z)) cannot be obtained analytically. However, it can be computed to any order of accuracy, say, by Newton iteration. Once P t X (z) is known, we compute ( numerically exactly ) the mean and the variance of the state variable: { E[X]= zp t X (z) dz, V(X) = (z E[X]) 2 P t X (z) dz. (3.16) Figure 3 compares the mean of the state variable obtained by the present method (3.13) with the exact mean (3.16) as a function of time. Figure 4 provides similar comparison for the variance of the state variable as a function of time. We note that the agreement is excellent. Fig. 3 Expectation of state variable X

12 132 P. Ngnepieba, M.Y. Hussaini Fig. 4 Variance of state variable X The mean and variance of the state variable are time dependent. At instants t = 0.25, t = 0.5, t = 0.75, and t = 0.95, the present method gives (E[X] = , V(X) = E 4), (E[X]= , V(X) = E 4), (E[X]= , V(X) = E 4), and (E[X]= , V(X) = E 4), respectively. These values of mean and variance of the state variable are used to construct its Gaussian pdf and are plotted in Fig. 5 along with the exact pdf of the state variable at exactly the same instants. Once again, the agreement is found to be very good. 3.2 Burgers equation We seek the solution of the stochastic Burgers equation, u t + 1 u 2 2 x ( ν(x) u ) = 0, t,x (0, 1), x x u(0,x)= 2 x 2, u(t,0) = u(t, 1) = 0. The viscosity parameter ν is assumed to be a Gaussian random field with mean E[ν] = x; its variance is assumed isotropic with an exponential autocorrelation length (see, e.g., [6, 10]) defined by C(x,y) = σ 2 e ( x y /L), where L = 1 and σ 2 = The observation X obs (t, x) is the solution of the model with viscosity ν(x) = x.

13 An efficient sampling method for stochastic inverse problems 133 Fig. 5 Probability density function of state variable The objective is to quantify the uncertainty in optimal initial condition u and its evolution. In other words, we would like to obtain the statistics, such as the mean and variance, of the optimal initial condition u and the state variable X Numerical results and discussion The Burgers equation is discretized with centered finite-difference scheme in space with 100 points and forward Euler in time. The unconstrained minimization algorithm of the quasi-newton limited memory type [5] with the convergence criterion either on the number of iterations or on the gradient norm of the cost function is used to determine the optimal initial condition. The decay is about 10 5 in 19 iterations. This shows the convergence of the identification process at the mean value of the viscosity. The standard Monte Carlo simulation employed a sample size of 400,000 for the viscosity parameter to compute the mean and covariance of the optimal initial condition and the state variable for the purpose of validating the results of the present method. It took approximately 4 weeks on Dell PC (P4, 2.26 GHZ, 512 MB). In the present approach, the error covariance matrix is approximated by = 1 N 1 N (ν j E[ν])(ν j E[ν]) T j=1

14 134 P. Ngnepieba, M.Y. Hussaini Fig. 6 Gradient of cost function J (at the mean value of viscosity E[ν]= x) with N = 35 realizations of the random viscosity. Approximation of the statistical first and second moments is carried out using (2.9) for the mean and (2.10) forthe covariance matrix with 2500 samples of the viscosity and the optimal initial condition. It took 1.6 h on Dell PC (P4, 2.26 GHZ, 512 MB). Figure 6 presents the decay of the gradient norm J(u,E[ν]) 2 = ( 100 ( k J(u,E[ν]) ) 2 k=1 ) 1/2 of the objective functional versus the number of iterations for viscosity ν = E[ν]. It decays very fast to about 10 5 in 19 iterations. Figure 7 shows the comparison between the mean of the optimal initial condition E[u ]= 1 n ni=1 u (ν i ) obtained by the standard Monte Carlo simulation and the present method at the mean value of the viscosity u (E[ν]). We observe that the agreement is excellent. Figure 8 displays the approximate covariance matrix Cov(u ) of the optimal initial condition. The diagonal of this covariance matrix represents the variance V(u );this variance is compared with the variance obtained by the standard Monte Carlo method in Fig. 9. The agreement is good except towards the end of the space distribution. The discrepancy is probably due to the inadequacy of the first-order Taylor expansion and the nonlinearity of the problem. Figure 10 presents the l 2 norm of the variance of the optimal initial condition, V 2 = ( 100 k=1 Vk 2 ) 1/2, where V k = V(u (x k,ξ)). The present method is found to

15 An efficient sampling method for stochastic inverse problems 135 Fig. 7 Optimal initial condition for E[ν] (continuous line) and mean of optimal initial condition obtained form Monte Carlo simulation (diamonds) Fig. 8 Covariance matrix Cov(u )

16 136 P. Ngnepieba, M.Y. Hussaini Fig. 9 Variance of u Fig. 10 l 2 norm of the variance of u

17 An efficient sampling method for stochastic inverse problems 137 Fig. 11 Comparison of errors (l 2 norm) converge initially at a rate propositional to 1/N 2 whereas the convergence rate of the standard Monte Carlo method is found proportional to 1/N. Thus, The convergence rate of the present method is two orders of magnitude faster than the Monte Carlo method. However, the accuracy of the converged l 2 norm of the variance in the present method is affected by the approximation in the construction of the Jacobian of the optimal initial condition with respect to the random parameter. The approximation in the minimization procedure equally affects both methods. The variance of the optimal initial condition V(u ) determined by the MC simulation with a million samples is taken as the reference for computing the l 2 norm of error E var = (V ref (u ) V(u )): E var 2 = ( 100 ) 1/2 V k V k 2 ref. k=1 Figure 11 displays the l 2 norm of error E V as a function of sample size N for the Monte Carlo simulation and the present method. It is to be noted that the convergence rate for the Monte Carlo method is proportional to the expected 1/ N, and the present method s convergence rate is proportional to 1/N 2 till it stagnates affected by the error inherent in the linear tangent approximation.

18 138 P. Ngnepieba, M.Y. Hussaini 4 Conclusions An efficient sampling method is developed to solve stochastic inverse problems with a random parameter. It involves the sensitivity derivatives of the control variable with respect to the random parameter. Specifically, the present method quantifies the uncertainty (due to the random parameter) of the optimal initial condition of a data assimilation problem and its impact on the evolution of the state variable. The paper discusses a linear example that permits analytical construction of the pdf of the optimal initial condition given the pdf of the random parameter. It provides insight into the nonlinear sensitivity of the optimal initial condition to the random parameter even in the linear case, and it further validated the present method. Then the effectiveness of the present method is demonstrated in the nonlinear example of an inverse problem governed by the Burgers equation. Acknowledgement This work was partially supported by NASA Ames Research Center (contract NCC2-1349). It is a pleasure to acknowledge the comments of Professor William Hager. References 1. Cao, Y., Hussaini, M.Y., Zang, T.A.: An efficient Monte Carlo method for optimal control problem with uncertainty. Comput. Optim. Appl. 26, (2003) 2. Cao, Y., Hussaini, M.Y., Zang, T.A.: Exploitation of sensitivity derivatives for improving sampling methods. AIAA J. 42(4), (2004) 3. Fletcher, R., Grothey, A., Leyffer, S., Computing sparse Hessian and Jacobian approximations with optimal hereditary properties. In: Biegler, L.T., Coleman, T.F., Conn, A.R., Santosa, F.N. (eds.) Large- Scale Optimization with Applications, Part II: Optimal Design and Control. Springer, Berlin (1997). 4. Ghil, M., Ide, K.: Introduction. In: Ghil, M., et al. (eds.) Data Assimilation in Meteorology and Oceanography: Theory and Practice, pp. i iii. Meteorological Society of Japan and Universal Academy Press, Tokyo (1997) 5. Gilbert, J.C., Lemarechal, C.: Some numerical experiments with variable-storage quasi-newton algorithms. Math. Program. 45, (1989) 6. Huyse, L., Walters, R.W.: Random field solutions including boundary condition uncertainty for the steady-state generalized Burgers equation. NASA/CR , ICASE report, No (2001) 7. Le Dimet, F.-X., Talagrand, O.: Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects. Tellus 38A, (1986) 8. Lions, J.L.: Optimal Control of Systems Governed by Partial Differential Equations. Springer, Berlin (1971) (Translated by Mitter, S.K.) 9. Putko, M.M., Newman, P.A., Taylor III, A.C., Green, L.L.: Approach for uncertainty propagation and robust design in CFD using sensitivity derivatives. AIAA paper, No (2001) 10. Tarantola, A.: Inverse Problem Theory: Methods for Data Fitting and Model Parameters Estimation. Elsevier, Amsterdam (1987) 11. Walters, R.W., Huyse, L.: Uncertainty analysis for fluid mechanics with applications. NASA/CR , ICASE report, No (2002)