LEAST SQUARES PROBLEMS WITH INEQUALITY CONSTRAINTS AS QUADRATIC CONSTRAINTS

Transcription

1 LEAST SQUARES PROBLEMS WITH INEQUALITY CONSTRAINTS AS QUADRATIC CONSTRAINTS JODI MEAD AND ROSEMARY A RENAUT. Introduction. The linear least squares problems discussed here are often used to incorporate observations into mathematical models. For example in inversion, imaging and data assimilation in medical and geophysical applications. In many of these applications the variables in the mathematical models are known to lie within prescribed intervals. This leads to a bound constrained least squares problem: (.) min Ax b α x β, where x, α, β R n, A R mxn, and b R m. If the matrix A has full column rank, then this problem has a unique solution for any vector b [3]. In Section, we also consider the case when A does not have full column rank. Successful approaches to solving bound-constrained optimization problems for general linear or nonlinear objective functions can be found in [6], [3], [8], [4] and the MATLAB function fmincon. Approaches which are specific to least squares problem are described in [], [9] and [5] and the MATLAB function lsqlin. In this work, we implement a novel approach to solving the bound constrained least squares problem by writing the constraints in quadratic form, and solving the corresponding unconstrained least squares problem. Most methods for for solutions of bound-constrained least squares problems of the form (.) can be catagorized as active-set or interior point methods. In active-set methods, a sequence of equality constrained problems are solved with efficient solution methods. The equality constrained problem involves those variables x i which belong to the active set, i.e. those which are known to satisfy the equality constraint [8]. It is difficult to know the active set a priori but algorithms for it include Bounded Variable Least Squares (BVLS) given in [3]. These methods can be expensive for large-scale problems, and a popular alternative to them are interior point methods. Supported by NSF grant EPS , Boise State University, Department of Mathematicss, Boise, ID Tel: , Fax: jmead@boisestate.edu Supported by NSF grants DMS 534 and DMS Arizona State University, Department of Mathematics and Statistics, Tempe, AZ Tel: , Fax: renaut@asu.edu

2 Interior point methods use variants of Newton s method to solve the KKT equality conditions for (.). In addition, the search directions are chosen so the inequalities in the KKT conditions are are satisfied at each iteration. These methods can have slow convergence, but if high-accuracy solutions are not necessary, they are a good choice for large scale applications [8]. In this work we write the inequality constraints as quadratic constraints and solve the optimization problem with a penalty-type method that is commonly used for equality constrained problems. When A is not full rank, regularized solutions are necessary for both the constrained and unconstrained problem. A popular approach is Tikhonov regularization [5] (.) min Ax b + λ L(x x ), where x is an initial parameter estimate and L is typically chosen to yield approximations to the l th order derivative, l =,,. There are different methods for choosing the regularization parameter λ; the most popular of which include L-curve, Generalized Cross-Validation (GCV) and the Discrepancy principle [5]. In this work, we will use a χ method introduced in [] and further developed in []. The efficient implementation of this χ approach for choosing λ compliments the solution of bound-constrained least squares problem with quadratic constraints. The rest of the paper is organized as follows. In Section we write the bound-constraint least squares problem with quadratic constraints, and describe how to solve the problem with quadratic, inequality constraints using the weighted [7] or penalty [8] method. In Section 3 we apply the chi-squared method to the bound-constrained problem. In Section 4 we give some numerical results on a test problem, and on a Hydrological application. Section 5 gives some conclusions.. Bound-Constrained Least Squares... Quadratic Constraints. The unconstrained regularized least squares problem (.) can be viewed as a least squares problem with a quadratic constraint []: (.) min Ax b subject to L(x x ) δ. Equivalence of (.) and (.) can be seen by the fact that the necessary and sufficient KKT conditions for a feasible point x to be a solution of (.) are (A T A λ L T L)x = A T b

3 3 λ λ ( L(x x ) δ) = δ L(x x ) Here λ is the Lagrange multiplier, and equivalence with problem (.) corresponds to λ = λ. The advantage of viewing the problem in a quadratically constrained least squares formulation rather than as Tikhonov regularization, is that the constrained formulation can give the problem physical meaning. In [] they give the example in image restoration where δ represents the energy of the target image. For a more general set of problems, the authors in [] successfully find the regularization parameter λ by solving the quadratically constrained least squares problem. Here we introduce an approach whereby the bound constrained problem is written with n quadratic inequality constraints, i.e. (.) becomes (.) min Ax b subject to (x i x i ) σ i i =,..., n where x = [x i ; i =,..., n] T is the midpoint of the interval [α, β], i.e. x = (β + α)/ and σ = (β α)/. The necessary and sufficient KKT conditions for a feasible point x to be a solution of (.) are: (.3) (.4) (.5) (A T A + λ )x = λ x + A T b (λ ) i i =,..., n (λ ) i [σ i (x i ˆx i ) ] = i =,..., n (.6) σ i (x i ˆx i ) i =,..., n where λ = diag((λ ) i ). Reformulating the box constraints α x β as quadratic constraints (x i x i ) σi, i =,..., n effectively circumscribes an ellipsoid constraint around the original box constraint. In [9] box constraints were reformulated in exactly the same manner, however the optimization problem was not solved with the penalty or weighted approach as is done here, and described in the next section. Rather, in [9] parameters were found which ensure there is a convex combination of the objective function and the constraints. This ensures the ellipsoid defined by the objective function intersects that defined by the inequality constraints.

4 4.. Penalty or Weighted Approach. The penalty [8] or weighted [7] approach is typically used to solve a least squares problem with n equality constraints. A simple application of this approach would be to solve (.7) min Ax b subject to x = x. The objective function and constraints are combined in the following manner: (.8) min ɛ Ax b + x x, and a sequence of unconstrained problems are solved for ɛ. By examination, one can see that as ɛ, infinite weight is added to the contraint. More formally, in [7] they prove that if x is the solution of the least squares problem (.8) and ˆx the solution of the equality constrained problem (.7) then x(ɛ) ˆx η ˆx with η being machine precision. Note that (.8) is equivalent to Tikhonov regularization (.) with λ = ɛ, L = I and x =. We apply the penalty or weighted approach to the quadratic, inequality constrained problem (.). In this case, the penalty term x x must be multiplied by a matrix which represents the values of the inequality constraints, rather than a scalar involving λ or ɛ. This matrix will come from the first KKT condition (.3), which is the solution of the least squares problem: (.9) min Ax b + λ / (x x). We view the inequality constraints as a penalty term by choosing the value of λ to be C = diag((σ) i ). Since the quadratic constraints circumscribe the box constraints, a sequence of probems for decreasing ɛ are solved which effectively decreases the radius of the ellipsoid until the constraints are satisfied, i.e. solve (.) min Ax b + C / ɛ (x x), where C ɛ = ɛc. Starting with ɛ =, the penalty parameter ɛ decreases until the solution of the inequality constrained problem (.) is identified. Since ɛ solves the equality constrained problem, these iterations are guarenteed to converge when A is full rank.

5 ALGORITHM. Solve least squares problem with box constraints as quadratic constraints. Initialization: x = (β + α)/,c ɛ = diag((/(β i α i )) ), Z = {j : j =,..., n}, J = NULL count= Do (until all constraints are satisfied) Solve (A T A + C ɛ )y = A T (b A x) for y x = x + y if α j x j β j j P, else j Z if Z :=NULL, end ɛ = /( + count/ɛ if j Z, (C ɛ ) jj = ɛ(c ɛ ) jj End This algorithm performs poorly because it over-smoothes the solution x. In particular, if the interval is small, then the standard deviation is small, and the solution is heavily weighted towards the mean, x. This approach to inequality constraints is not recommended unless prior information about the parameters, or a regularization term is included in the optimization as described in the next section..3. Regularization and quadratic constraints. The above algorithm is not useful for well-conditioned or full rank matrices A because it over-smoothes the solution. In addition, for rank deficient or ill-conditioned Awe may not be able to calculate the least squares solution x to (.) 5 x = x + (AA T + C ɛ ) A T (b A x). because as ɛ decreases, (A T A + C ɛ ) will eventually become ill-conditioned. Thus the approach to inequality constraints in this work should be used with regularized methods, when typical solution techniques do not yield solutions in the desired interval. The most typical way to regularize a problem is with Tikhonov regularization: (.) min Ax b + λ L(x x ). As mentioned in the Introduction, there are many methods for choosing the parameter λ, and here we focus on a chi-squared method similar to the discrepancy principle [6] introduced in [], and further developed in []. This chi-squared method is based on the fact that C b and C x are invertible covariance matrices on the mean zero errors in data b and initial parameter estimates x, respectively, the minimum value of the functional J = (b Ax) T C b (b Ax)+(x x ) T C x (x x )

6 6 is a random variable which follows a χ distribution with m degrees of freedom [, ]. In particular, given value for C b and C x, the difference J( x) m is an estimate of confidence that C b and C x are accurate weighting matrices. Mead [] noted these observations, and suggested a matrix C x can be found by requiring that J( x), to within a specified ( α) confidence interval, is a χ random variable with m degrees of freedom, namely such that (.) m mz α/ < r T (AC x A T + C b ) r < m + mz α/, where z α/ is the relevant z-value for a χ -distribution with m degrees of freedom. If C x = λ I in [] it was shown that for accurate C b, this χ approach is more efficient and gives better results than the discrepancy principle, the L-curve and generalized cross validation (GCV) [5]..3.. χ regularization with quadratic constraints. Applying quadratic constraints to the chi-squared method amounts to solving the following problem: (.3) min J ɛ x where J ɛ = C / b (Ax b) + C / x (x x ) + C / ɛ (x x). First it must be the case that the last two terms in the objective function J ɛ are consistent. That is, Dx = f must be consistent where [ ] [ C / x D = C /, f = ɛ i.e. rank(d)=rank(d f). C / x x x C / ɛ This formulation has three terms in the objective function and seeks a solution which lies in an intersection of three ellipsoids. However it is possible, but not necessary, to write the regularization term and the inequality constrained term as one. The minimum value of J ɛ occurs at x ɛ = x + (A T C b A + C x + C ɛ where x = x x. The minimum value of J ɛ is J ɛ ( x ɛ ) = r T P ɛ r + x T Q x P ɛ = A(C x ] ) (A T C r + C x), + C ɛ ) A T + C b Q = C ɛ + (A T C b A + C x ). b, ɛ

7 7 Now r T P ɛ r is a random variable, while x T Q x is not. Define k ɛ = P / ɛ r, then k ɛ is normally distributed if r is, or if there are a large number of data m. It has zero mean, i.e. E(k ɛ ) =, if Ax is the mean of b, while its covariance This implies that E(k ɛ k T ɛ ) = P / ɛ (AC x A T + C b )P / ɛ. k = (AC x A T + C b )P / ɛ k ɛ has the identity as it s covariance thus kk T has the properties of a χ distribution. Note that k does not depend on the constraint term, and this is the typical χ property (.)..3.. Other regularization methods and generalized implementation of quadratic constraints. Any regularization method can be used to implement box constraints as quadratic constraints. In addition to χ regularization with quadratic constraints, in Section 3 we will show results where C x is found by the L-curve and maximum likelihood estimation. The L-curve approach finds the parameter λ in (.), for specified L by plotting the weighted parameter misfit L(x x ) versus the data misfit b Ax. The data misfit may also be weighted by the inverse covariance of the data error, i.e. C b. When plotted in a log-log scale the curve is typically in the shape of an L, and the solution is the parameter values at the corner. These parameter values are optimal in the sense that the error in the weighted parameter misfit and data misfit are balanced. The L-curve method can assume that the data contain random noise. Maximum likelihood estimation (MLE) and the χ regularization method not only assume that the data contain noise, but so do the initial parameter estimates. In MLE the data b are random, assumed to be independent and identically distributed, following a normal distribution with probability density function { ρ(b) = const exp } (.4) (b Ax)T C b (b Ax), with Ax the expected value of b and C b the corresponding covariance matrix. In addition, it is assumed that the parameter values x are also random following a normal distribution with probability density function { ρ(x) = const exp } (.5) (x x ) T C m (m m ),

8 8 with x the expected value of x and C x the corresponding covariance matrix. If the data and parameters are independent then their joint distribution is ρ(b, x) = ρ(b)ρ(x). In order to maximize the probability that the data were in fact observed we find x where the probability density is maximum. The maximum likelihood estimate of the parameters occurs when the joint probability density function is maximum, i.e. optimal parameter values are found by solving (.6) min x { (b Ax) T C b (b Ax) + (x x ) T C x (x x ) }. This optimal parameter estimate is found under the assumption that the data and parameters follow a normal distribution and are independent and identically distributed. The χ regularization method is based on this idea, but the assumptions are relaxed, see [] []. In addition the χ regularization method is an approach for finding C x (or C b ), while MLE assumes they are known. Note that the L-curve is equivalent to MLE when If C x = λ I. The advantage of MLE is when C x is not a constant matrix and hence the weights on the parameter misfits vary. Moreover, when C x has off diagonal elements, correlation in initial parameter estimate errors can be modeled. The disadvantage of MLE is that a priori information is needed. On the other hand the χ regularization method is an approach for finding elements of C x, thus matrices rather than parameters may be used for regularization, but not as much a priori information is needed as with MLE. ALGORITHM. Solve regularized least squares problem with box constraints as quadratic constraints. Initialization: x = (β + α)/,c ɛ = diag((/(β i α i )) ), Z = {j : j =,..., n}, J = NULL Find C x by any regularization method. count= Do (until all constraints are satisfied) Solve (A T A + C ɛ + C x x ɛ = x + y ɛ if α j x j β j j P, else j Z if Z :=NULL, end ɛ = /( + count/)ɛ if j Z, (C ɛ count = count + End ) jj = ɛ(c ɛ )y ɛ = (A T C r + C x) for y ɛ ) jj b ɛ

9 9 In the next two sections we give numerical results where the box constrained least squares problem (.) is solved by (.3), i.e. by implementing the box constraints a quadratic constraints using Algorithm. 3. Experiments. For validation of the algorithm against other standard approaches we present a series of representative results using benchmark cases, phillips, shaw, and wing from the Regularization Tool Box [5]. In addition, we present the results for a real model from hydrology. The box constraints for the benchmark cases were set arbitrarily, while for the hydrological model, parameter constraints from the literature were used. 3.. Benchmark Problems: Experimental Design. System matrices A, right hand side data b and solutions x are obtained for a specific benchmark problem from the Regularization Tool Box [5]. In all cases we generate a random matrix Θ of size m 5, with columns Θ c, c = : 5, using the Matlab R function randn. Then setting b c = b + level b Θ c / Θ c, for c = : 5, generates 5 copies of the right hand vector b with normally distributed noise, dependent on the chosen level. Results are presented for level =.. An example of the error distribution for one case of phillips with n = 8 is illustrated in Figure??. Because the noise depends on the right hand side b the actual error, as measured by the mean of b b c / b over all c, is.8. To obtain the weighting matrix W b, the resulting covariance C b between the measured components is calculated directly for the entire data set B with rows (b c ) T. Because of the design, C b is close to diagonal, C b diag(σb i ) and the noise is colored. In all experiments, regardless of parameter selection method, the same covariance matrix is used. The a priori reference solution x is generated using the exact known solution and noise added with level =. in the same way as for modifying b. The same reference solution x is used for all right hand side vectors b c, see Figure Benchmark Problems: Results. Algorithm was implemented with the χ regularization method, the L-curve and maximum likelihood estimation (MLE). For the benchmark results, we know the true solution, thus we can implement maximum likelihood estimation exactly, which is not true in real applications. Thus the MLE is an exact version of the χ regularization method because the covariance matrix C x is known, while in the χ regularization method it is solved for using the properties of a χ distribution. For comparison the Matlab R function lsqlin was used to implement the box constraints in the linear least squares problem. As evidenced in Figures 3.3(b), 3.5(b) and 3.7(b), the lsqlin solution stays

10 4 3 Problem Phillips Right Hand Side exact noise Problem Phillips Right Hand Side exact noise.8 3 (a) (b).. Problem Wing Right Hand Side exact noise (c) FIG. 3.. Illustration of the noise in the right hand side for problem (a)phillips, (b) shaw (c) wing.

11 .8.6 Problem Phillips Reference Solution exact noise.8.5 Problem Phillips Reference Solution exact noise (a) (b).4. Problem Phillips Reference Solution exact noise (c) FIG. 3.. Illustration of the reference solution x for problem (a)phillips, (b) shaw (c) wing.

12 .4. Solutions without constraints x MLE Quadratically constrained solutions,. < x < x MLE x Lcurve x χ x Lcurve x χ x lsqlin...4 (a) (b) FIG Phillips (a) unconstrained and (b) constrained solutions within the correct constraints, but does not retain the shape of the curve. Figures (3.4), (3.6) and (3.8) show that for any regularization method, the significant advantage of implementing the box constraints as quadratic constraints and solving (.3), is that we retain the shape of the curve Estimating data error: Example from Hydrology. The χ -curve method is used to find the weight σ b on field measurements b. The initial parameter estimates x and their respective covariance C x = diag(σx i ) are based on laboratory measurements, and specified a priori. In other words, the regularization parameter or initial parameter misfit weight is taken from laboratory measurements while the data weight is obtained by the χ -curve method. In nature, the coupling of terrestrial and atmospheric systems happens through soil moisture. Water must pass through soil on its way to groundwater and streams, and soil moisture feeds back to the atmosphere via evapotranspiration. Consequently, methods to quantify the movement of water through unsaturated soils are essential at all hydrologic scales. Traditionally, soil moisture movement is simulated using Richards [] equation for unsaturated

13 L curve with and without quadratic constraints x Lcurve.< x Lcurve <.6 Maximum likelihood with and without quadratic constraints. x MLE. < x MLE < (a) (b) Regularized χ with and without quadratic constraints.8.6 x χ.< x χ < (c) FIG Phillips test problem of (a) L-curve, (b) maximum likelihood estimation (c) regularized χ method.

14 x MLE x Lcurve x χ Solutions without constraints Quadratically constrained solutions,.5 < x <.5.5 x MLE x Lcurve x χ.5 x lsqlin.5.5 (a) (b) FIG Shaw (a) unconstrained and (b) constrained solutions flow: (3.) θ t = (K h) + K z. θ is the volumetric moisture content, h pressure head, and K(h) hydraulic conductivity. Solution of Richards equation requires reliable inputs for θ(h) and K(h). These relationships are collectively referred to as soil-moisture characteristic curves, and they are typically highly nonlinear functions. These curves are commonly parameterized using the van Genuchten [6] and Mualem [7] relationships: (3.) θ(h) = θ r + (θ s θ r ) ( + αh n ) m h < ( K(h) = K s θe l ( θe /m ) m) θ e = θ(h) θ r θ s θ r. h < These equations contain seven independent parameters: θ r and θ s are the residual and saturated water contents (cm 3 cm 3 ), respectively, α (cm ), n and m (commonly set = /n)

15 L curve with and without quadratic constraints x Lcurve.5 < x Lcurve <.5 Maximum likelihood with and without quadratic constraints x MLE.5 < x MLE < (a) (b) Regularized χ with and without quadratic constraints.5 x χ.5 < x χ < (c) FIG Shaw test problem of (a) L-curve, (b) maximum likelihood esitmation (c) regularized χ method.

16 6.8 Solutions without constraints. Quadratically constrained solutions, < x <..6.4 x MLE x Lcurve x χ..8 x MLE x Lcurve x χ..6 x lsqlin (a) (b) FIG Wing (a) unconstrained and (b) constrained solutions are empirical fitting parameters (-), K s is the saturated hydraulic conductivity (cm/sec) and l is the pore connectivity parameter (-). (Note that the variables used in this application, n, m, α, r, s, are common notation for the parameters in the model, and do not represent the size of the problem, or any previous referenced variables.) We focus on parameter estimates for θ(h) to be used in Richards equation. Field studies and laboratory measurements of soil moisture content and pressure head in the Dry Creek catchment near Boise, Idaho are used to obtain θ(h) estimates. The north facing slope is currently instrumented with a weather station, two soil pits recording temperature and moisture content at four depths, and four additional soil pits recording moisture content and pressure head with TDR/tensiometer pairs. We will show soil water retention curves from two of these pits: NU 5 and SU5 5. They represent pits upstream from a weir and 5 meters, respectively, both 5 meters from the surface. We rely on the laboratory measurements for good first estimates of the parameters x = [θ r, θ s, α, n] and their standard deviations σ xi. It takes -3 weeks to obtain one set of labora-

17 L curve with and without quadratic constraints x Lcurve < x Lcurve <. Maximum likelihood with and without quadratic constraints.4. x MLE < x. MLE < (a) (b) Regularized χ with and without quadratic constraints x χ < x χ <. (c) FIG Wing test problem of (a) L-curve, (b) maximum likelihood esitmation (c) regularized χ method.

18 8 tory measurements, but this procedure is done multiple times from which we obtain standard deviation estimates and form C x = diag(σ θ r, σ θ s, σ α, σ n). These standard deviations account for measurement technique or error. however, measurements on this core may not accurately reflect soils in entire watershed region. The quadratically constrained method described here requires a linear model, and we use the following technique to simulate the nonlinear behavior of (3.). Let θ(h, x) θ(h, x ) + θ (3.3) x (x x ) x=x (3.4) = A(h, x )x + q(h, x ). Then b i = θ(h i, x) q(h i, x ) is of dimension m, while A has dimension m 5. An optimal ˆx is found by the χ -curve method, x is updated with it, and (3.4) is iterated. Table (3.) gives parameter ranges (or constraints) in the form soil class averages found in [7]. In addition, the parameter values for both pits both constrained to fit into those class averages or ranges, and the unconstrained values are given. For both pits, the only parameter that did not fit into the appropriate range is θ s. This parameter represents the soil moisture content when the ground is nearly saturated. In the semi-arid environment of the Dry Creek Watershed, the soil does not typically come near saturation. We ve concluded from these results that the class averages, with a minimum value of θ s =.3, do not reflect the soils found in this region. A more appropriate minimum would be θ s =.. NU 5 SU5 5 Parameter Ranges Unconstrained Constrained Unconstrained Constrained log α [.86,.96] log n [.4,.68] θ s [.3,.568] θ r [.5,.3] TABLE 3. Hydrological Parameters Figure (3.9) shows the constrained and unconstrained results with θ(ψ) on the horizontal axis. The van Genuchten equation is typically plotted in this manner, and the curve is called the soil moisture retention curve. Near saturation, i.e. for ψ near, the soil moisture falls below.5 further indicating the the soil class averages found in [7] are not appropriate for this region and should not be used as a constraints. In Table 3. we see that the unconstrained parameters are appropriate when θ s is allowed to lie within a range with lower bound.. REFERENCES

19 9 SD5_5 NU_ Measurements Unconstrained Constrained 5 4 Measurements Unconstrained Constrained log ψ log ψ θ(ψ) θ(ψ) (a) (b) FIG Unconstrained and constrained soil moisture retention curves for (a) SD5 5 and (b) NU 5. [] Bennett, A., 5 Inverse Modeling of the Ocean and Atmosphere (Cambridge University Press) p 34. [] Bierlaire, M., Toint, Ph.L. and Tuyttens, On iterative algorithms for linear least squares problems with bound constraints, Lin. Alg. Appl., Vol. 43 () p. -43, 99. [3] Björck, Å, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, pp.48, 996. [4] Figueras, J., and M. M. Gribb, A new user-friendly automated multi-step outflow test apparatus, Vadose Zone Journal (in preparation). [5] Hansen, P. C., 994, Regularization Tools: A Matlab Package for Analysis and Solution of Discrete Ill-posed Problems, Numerical Algorithms [6] Huyer, W. and A. Neumaier, Global optimization by multilevel coordinate search, J. Global Optimization 4 (999), [7] Lawson, C.L. and Hanson, R.J. Solving Least Squares Problems, Prentic-Hall, p.34, 974. [8] Lin, C-J and J. Moreé Newton s method for large bound-constrained optimization problems, SIAM Journal on Optimization, Volume 9, Number 4, pp. -7, 999. [9] Lötstedt, P. Solving the minimal least squares problem subject to bounds on the variables, BIT 4 p. 6-4, 984. [] McNamara, J. P., Chandler, D. G., Seyfried, M., and Achet, S. 5. Soil moisture states, lateral flow, and streamflow generation in a semi-arid, snowmelt-driven catchment. Hydrological Processes, 9, [] Mead J., 8, A priori weighting for parameter estimation, J. Inv. Ill-posed Problems, to appear. [] J.L. Mead and R.A. Renaut, A Newton root-finding algorithm for estimating the regularization parameter for solving ill-conditioned least squares problems, submitted to Inverse Problems. [3] Michalewicz, Z. and C. Z. Janikow, GENOCOP: a genetic algorithm for numerical optimization problems with linear constraints, Comm. ACM, Volume 39, Issue es (December 996) [4] Mockus J., Bayesian Approach to Global Optimization.Kluwer Academic Publishers, Dordrecht, 989. [5] Morigi, S, Reichel, L., Sgallari, F., and Zama, F., An iterative method for linear discrete ill-posed problems with box constraints, J. Comp. Appl. Math. 98, p. 55-5, 7.

20 [6] V.A. Morozov, On the solution of functional equations by the method of regularization, Soviet Math. Dokl. 7 (966) [7] Mualem, Y A new model for predicting thehydraulic conductivity of unsaturated porous media. Water Resour. Res. :53. [8] Nocedal, J. and Wright S, Numerical Optimization, Springer-Verlag, New York, pp. 636, 999. [9] Pierce, J.E. and Rust, B.W. Constrained Least Squares Interval Estimation, SIAM Sci. Stat. Comput., Vol. 6, No. 3, p , 985. [] Rao, C. R., 973, Linear Statistical Inference and its applications, Wiley, New York. [] Richards, L.A. Capillary conduction of liquids in porous media, Physics, [] M. Rojas and D.C. Sorensen, A Trust-Region Approach to the Regularization of Large-Scale Discrete Forms of Ill-Posed Problems, SISC, Vol. 3, No. 6, p ,. [3] P.B. Stark and R.L. Parker, Bounded-Variable Least-Squares: An Algorithm and Applications, Computational Statistics, :9-4, 995. [4] Tarantola A 5 Inverse Problem Theory and Methods for Model Parameter Estimation (SIAM). [5] Tikhonov, A.N., Regularization of incorreclty posed problems, Soviet Math., 4, pp.64-67, 963. [6] van Genuchten, M.Th. 98. A closed-form equation for predicting the hydraulic conductivity of unsaturated soils. Soil Sci. Soc. Am. J. 44:89?898. [7] Warrick, A.W. Soil Water Dynamics, Oxford University Press, 3, p 39.