Overview of ridge regression estimators in survey sampling

Transcription

1 Overview of ridge regreion etimator in urvey ampling Camelia Goga and Muhammad Ahmed Shehzad IMB, Univerité de Bourgogne, DIJON - France camelia.goga@u-bourgogne.fr, M uhammad Ahmed.Shehzad@u-bourgogne.fr December 2, 2011 Abtract The ridge regreion i a biaed etimation method ued to circumvent the intability in the regreion etimator obtained by ordinary leat quare method in the preence of multicollinearity. Thi method ha been ued in urvey ampling in order to cope with negative or extremely large weight reulted when a very large number of calibration or balancing contraint wa impoed. In thi paper, we give a review and ome new interpretation of the ridge-type etimator in a urvey ampling framework. Key Word: calibration, model-baed etimator, model-aited etimator, multicollinearity, penalized regreion. 1 Introduction Regreion technique are widely ued in practice due to their large and eae applicability. They are baed on ordinary leat quare method. Neverthele, in preence of multicollinearity of data, thi etimator can have extremely large variance even if it ha the deirable property of being the minimum variance etimator in the cla of linear unbiaed etimator (the Gau-Markov theorem). Biaed etimator have been uggeted to cope with problem and the ridge regreion i one of them. Hoerl and Kennard (1970) ugget in a eminal paper the ridge regreion and how that for uitable value of the penalty parameter, the ridge etimator ha maller mean quared error that the ordinary leat quare etimator. The method ha been applied in many field. The book of Vinod and Ullah (1981) give a comprehenive decription on thi topic a well a many example. In a urvey ampling etting, weighted etimator uing auxiliary information are built in order to give precie etimation about parameter of interet uch a total, mean, ratio and o on. Uually, thee weighted etimator are equivalent to regreion etimator but it happen that, in 1

2 the preence of a large amount of information, the weight are very untable, negative or very large. Moreover, data may contain many zero or the ample ize may be maller than the number of auxiliary variable a for the mall domain cauing problem of matrix invertibility. The paper i tructured a follow. Section 2 recall the contruction of the ridge etimator for the regreion coefficient a introduced by Hoerl and Kennard, (1970) in a claical regreion etting. At thi occaion, we give the equivalent interpretation of thi etimator uch a the contrained minimization problem and the Bayeian point of view. The ridge etimator depend on a penalty parameter that control the trade-off between the bia and variance of the etimator. We recall briefly the ridge trace a a method to find the penalty parameter. Section 3 give a detailed preentation of the application of ridge principle in urvey ampling. Thi preentation include the derivation of the penalized etimator under the model-baed approach given in ection 3.1 a well a under the calibration approach, ection 3.2. Section 3.3 exhibit the partial calibration or balancing. When we attribute a prior on previou etimation, we may ue the Bayeian interpretation to contruct ridge regreion type etimator. Deville (1999) conidered it a a calibration on an uncertain ource. We decribe it in ection 3.4. Finally, ection 3.5 give the tatitic propertie of the cla of penalized etimator and we finih by concluding remark and ome further work. 2 Ridge regreion Let X = (X 1,..., X p ) be a n p matrix of tandardized known regreor X i = (X ki ) n k=1 i = 1,..., p. Conider the following linear model, for all y = 1 nβ 0 + Xβ + ε, (1) where y = (y k ) n k=1 i the n 1 vector of obervation and ε = (ε k) n k=1 i the n 1 vector of error. We aume that X i a non-tochatic matrix of regreor with X X of full rank matrix (i.e the rank of X i p). We uppoe alo that the error ε k are independent with zero mean and variance Var(ε k ) = σ 2 for all k = 1,..., n. The ordinary leat quare (OLS) etimator ˆβ of β minimize the error um of quare (ESS), ESS = (y Xβ) (y Xβ) yielding the following etimator, ˆβ OLS = (X X) 1 X y. 2

3 2.1 Multicollinearity, ill-conditioning and conequence on the OLS etimator Zero or no dependence among the explanatory variable i one of the aumption of claical linear regreion model. The ubject of multicollinearity i widely referred to the ituation where there i either exact or approximately exact linear relationhip among the explanatory variable (Gujarati, 2002). Gunt and Maon (1977) dicriminate between the exitence and the degree of the multicollinearity found in the auxiliary variable. They tate that the cloer the linear combination between the column of X are to zero, the tronger are the multicolinearitie and the more damaging are their effect on the leat quare etimator. It hould be kept in mind while detecting the multicollinearity that the quetion hould be of the degree/intenity of multicollinearity and not of kind of the multicollinearity. Small eigenvalue and their correponding eigenvector help to identify the multicollinearitie. Let λ 1,..., λ p be the eigenvalue of X X in decreaing order, λ max = λ 1 λ 2... λ p = λ min > 0 and their correponding eigenvector V 1,..., V p. If we write (Gunt and Maon, 1977), λ j = V jx XV j = (XV j ) (XV j ), j = 1,..., p we obtain that for mall eigenvalue λ j of X X, (XV j ) (XV j ) 0 XV j 0 which mean that there i an approximately linear relationhip between the column of X. The element of the correponding eigenvector V j allow to identify the coefficient ued in the linear dependency. The multicollinearity i one form of ill-conditioning. More general, a meaure of ill-conditioning i the conditioning number K given by K = λ max /λ min. For λ min 0, we have K, and o, a large K implie an ill-conditioned matrix X. The multicollinearity or the ill-conditioning of X have eriou conequence on the OLS etimator. The mean quare error (MSE) of any etimator ˆβ of β i given by MSE( ˆβ) = E(( ˆβ β) ( ˆβ β)) 3

4 and for the OLS etimator ˆβ OLS, it become MSE( ˆβ OLS ) = σ 2 Trace(X X) 1 = σ 2 p i 1 λ i (2) The above expreion implie that the maller the eigenvalue are, the greater are the variance of ˆβ OLS and the average value of the quared ditance from ˆβ OLS to β. Thi reult in wider confidence interval and therefore lead to accept more often the N ull Hypothei (i.e. the true population coefficient i zero). Moreover, in cae of ill-conditionning, the OLS olution i untable meaning that the regreion coefficient are enitive to mall change in the y or X data (ee Marquardt and Snee, 1975 and Vinod and Ullah, 1981). Hoerl and Kennard (1970) dicu the cae when the leat quare coefficient can be both too large in abolute value and incorrect with repect to ign. Roundoff error tend to occur into leat quare calculation while X X i computed. Obviouly, any error in X X may be maximized during the calculation of β or any other related computation. The danger of roundoff error in X X i particularly magnified when (a) the determinant of X X i cloe to zero and/or (b) X X ha the element ubtantially different in order of magnitude. Method dealing with uch data conit in (1) uing priori information (Bayeian approach), (2) omitting highly collinear variable, (3) obtaining additional or new data and (4) uing ridge regreion. Thee method can be ued individually or together depending upon the countered ituation. Our dicuion however remain limited toward the ridge regreion which i an important tool to deal with multicollinearity. 2.2 Definition of the ridge etimator Ridge regreion wa firt ued by Hoerl and Kennard (1962) and then by Hoerl and Kennard (1970) a a olution to the biaed etimation for nonorthogonal data problem. A a purpoe to control intability linked to the leat quare etimate, Hoerl and Kennard (1962) and Hoerl and Kennard (1968) uggeted an alternative etimate of the regreion coefficient a obtained by adding a poitive contant k to the diagonal element of the leat quare etimator ˆβ OLS, ˆβ k = (X X + ki p ) 1 X y (3) where I p i the p-dimenional identity matrix. Since the contant k i arbitrary, we obtain a cla of etimator ˆβ k for the regreion etimator β rather than a unique etimator. For k = 0, we obtain the OLS etimator and a k, ˆβ k 0, the null vector. 4

5 The relationhip between the ridge etimator and the OLS etimator i given by (Hoerl and Kennard, 1970), ˆβ k = (I p + k(x X) 1 ) 1 ˆβ OLS. Let conider again the latent eigenvalue of (X X) 1, λ 1,..., λ p with the correponding eigenvector V 1,..., V p. Hence, the OLS etimator may be written a ˆβ OLS = p j=1 V j X y λ j V j. The fact of adding a mall contant to the diagonal of X X will have a conequence the increae of it eigenvalue with the ame quantity and dramatically decreae in thi way the conditioning number K. So, the matrix X X + ki ha eigenvalue λ 1 + k,..., λ p + k with the ame eigenvector V 1,..., V p and the ridge etimator may be written a follow ˆβ k = p j=1 V j X y λ j + k V j. The effect of the mallet eigenvalue may not be entirely eliminated by thi etimator ˆβ k but their effect on the parameter etimate are ignificantly leened. Hoerl and Kennard (1970) how alo that for k 0, the length of the ridge etimator ˆβ k i horter than that of ˆβ OLS, namely ˆβ ˆβ k k < ˆβ ˆβ OLS OLS. Let tudy now the tatitical propertie of the ridge etimator. It i important to note that the ridge etimator ˆβ k i a biaed etimator of β unle k = 0. The bia i given by E( ˆβ k ) β = k(x X + ki) 1 β = p (V j k j λ j + k which depend on the unknown β and on k. It appear that ˆβ k can be ued to improve the mean quare error of the OLS etimator, and the magnitude of thi improvement increae with an increae in pread of the eigenvalue pectrum. The ridge regreion come up with the objective of developing table et of coefficient which will do a reaonable job for predicting future obervation. Conniffe and Stone (1973) however criticized the ˆβ k ince it propertie depend on the non-tochatic choice of k. Hoerl and Kennard (1970) and Hoerl, Kennard and Baldwin (1975) how that an improvement j=1 5

6 of the MSE can be obtained uing ˆβ k. Conider for that the MSE of ˆβ k, MSE( ˆβ k ) = σ 2 p j=1 λ j (λ j + k) 2 + k2 p j=1 (V j β)2 (λ j + k) 2 = Trace(Var( ˆβ k )) + (Bia( ˆβ k )) (Bia( ˆβ k )) = A + B (4) Theorem 1 (exitence theorem, Hoerl and Kennard, 1970) There alway exit a k > 0 uch that MSE( ˆβ k ) < MSE( ˆβ) = σ 2 p j 1 λ j. Moreover, the above inequality i valid for all 0 < k < k max = of (V 1,..., V p )β. σ2 α 2 max where α max i the larget value The proof i baed on the fact that the variance term A from relation (4) i a continuou, monotonically decreaing function of k and the quared bia term B i a continuou, monotonically increaing function of k. Their firt derivative are alway non-poitive and non-negative, repectively. Thu, a neceary condition to prove the theorem i to how that it alway exit a k > 0 uch that the firt derivative of MSE( ˆβ k ) i non-poitive. Thi i poible for all 0 < k < σ 2 /αmax. 2 However, Theobald (1974) criticized the MSE criteria ued by Hoerl and Kennard (1970) and uggeted a more general criteria. Theobald (1974) uggeted minimizing the weighted mean quare error (WMSE) defined by ( ) W MSE(ˆβ) = E ( ˆβ β) W( ˆβ β) for any non-negative definite matrix W. For W = I the identity matrix, we obtain the MSE criteria. He howed that minimizing the WMSE, for all non-negative definite matrix W i equivalent to minimizing the mean quare error matrix (MMSE), ( MMSE(ˆβ) = E ( ˆβ β)( ˆβ β) ). Theorem 2 (Theobald, 1974) The ridge etimator ˆβ k i better than ˆβ OSL in the ene that MMSE(ˆβ k ) MMSE(ˆβ OSL ) i a poitive-definite matrix whatever 0 < k < k max = 2σ2 β β. 6

7 Vinod and Ullah (1981) give a different proof for the Theobald reult. A neceary and ufficient condition for MMSE(ˆβ k ) MMSE(ˆβ OSL ) to be a poitive-definite matrix given by Swindel and Chapman, (1973) i 0 < k < 2/[ min(0, η)] where η i the minimum eigenvalue of (X X) 1 (ββ /σ 2 ). The ridge trace We can remark that the ˆβ k depend upon the unknown parameter k which make it impoible to calculate. Hoerl and Kennard (1970) uggeted the ridge trace method to acquire the uitable value for the ridge parameter k. The ridge trace i a graphical tool that plot the component of the ridge regreion coefficient ˆβ k veru k. It aim at finding an appropriate value of k which provide a et of coefficient ˆβ k with maller MSE than that of the leat quare olution ˆβ OLS. The intability of the ridge trace indicate intercorrelation among regreor ariing from multicollinearity and hence, the ridge olution in the untable region of ridge trace are more eriouly affected by multicollinearity. Marquardt and Snee (1975) conider the ridge trace a one of the major advantage of the ridge regreion. It i clear that thi method do not yield a ingle automatic olution to the etimation problem, but rather, a family of olution. Variou rule for chooing k have been uggeted in the literature (ee Vinod and Ullah, 1981). Conniffe and Stone (1973) criticized the ridge trace method becaue it may need iteration over a relatively long range of k and doubt the lack of improvement of the leat quare etimator via any particular choice of k. Intead direct examination of eigenvalue may be preferred Other interpretation of the ridge regreion etimator The ridge regreion etimator a a olution of a contrained minimization problem The ridge etimator can alo be een a a olution of contrained optimization problem. Hoerl and Kennard (1970) conider the error um of quare due to any etimate β of β, ESS( β) = (y X β) (y X β) = ESS( ˆβ OLS ) + ( β ˆβ OLS ) X X( β ˆβ OLS ) which achieve it minimum only when β = ˆβ OLS. Relation (2) prove that on the average the 7

8 ditance between β and ˆβ OLS increae with the preence of ill-conditioning in X X but without an appreciable increae in the error um of quare. Hoerl and Kennard (1970) therefore, require finding the etimator β of minimum length that belong to the hyperellipoid centered at the OLS ) ) etimator and defined by the equation ( β ˆβOLS X X ( β ˆβOLS = Φ = contant. Figure 1 illutrate the geometry of the ridge regreion when β = (β 1, β 2 ) i a two-dimenional parameter (Marquart and Snee, 1975). We can remark that ˆβ k i the hortet vector that give a reidual um of quare a mall a the Φ value anywhere on the mall ellipe. β 2 ˆβ k 0 ˆβ OLS β 1 Figure 1: Geometry of ridge regreion In an equivalent way, we may minimize ESS( β) for a fixed length of β ay r. Thi i equivalent to finding the ellipe contour that i a cloe a poible to the circle centered in zero of ray equal to r. Uing the Lagrangian principle (Izenman, 2008), the optimization problem may be preented a min β(y X β) (y X β) + k( β β r 2 ) or equivalently, min β: β 2 r 2 (y X β) (y X β) (5) where i the Euclidian norm. In order to attribute the ame influence of the contraint from (5), it i adviable to tandardize the regreor. With no-tandardized variable, one may ue ome other norm (Kapat and Goel, 2010) or the generalized ridge regreion when each diagonal element of X X i modified differently (Hoerl and Kennard, 1970). 8

9 Bayeian or Mixed Regreion Interpretation of Ridge Coefficient The Bayeian approach treat the parameter β a a random variable with a prior probability denity which may be baed on ome ubjective prior information about β. The goal i to determine the poterior probability denity of β which i done by combining the prior probability denity with the ample information given by the likelihood function. A ridge etimator can be een alo a a Baye etimator when β take a uitable normal prior ditribution with mean β 0 and variance covariance matrix σβ 2 Ω (Vinod and Ullah, 1981, Izenman, 2008). Vinod and Ullah (1981) advocate that the Bayeian interpretation of the ridge regreion coefficient ˆβ k implie deriving the prior ditribution of β for which ˆβ k i the poterior mean. They alo tate that the Bayeian method imply that the poterior mean i the optimal etimator when uing the MSE a expected lo. We conider the model given in (1) with the following upplementary aumption: the error ɛ are normally ditributed with mean zero and variance covariance matrix σ 2 I p with σ 2 a known contant and I p i the p dimenional identity matrix. In other word, y i normally ditributed N(Xβ, σ 2 I p ). We uppoe that the prior normal ditribution on β i alo normal with known mean β 0 and known variance σ 2 β Ω. The poterior denity of β i therefore normal with mean β a follow β = (X X + αω 1 ) 1 (X Xˆβ OLS + αω 1 β 0 ) (6) = β 0 + (X X + αω 1 ) 1 X X(ˆβ OLS β 0 ) (7) of variance covariance matrix given by σ 2 Ω = σ 2 (X X + α 2 Ω 1 ) 1 and α = σ 2 /σ 2 β i the ratio of the variance. Relation (6) or (7) how that if the prior information i uele, i.e. σ 2 β, then α 0 and β = ˆβ OLS. On the other hand, for σ 2 β 0, we have β = β 0. Vinod and Ullah (1981) remark that the etimator β given by formula (6) may be written a a weighted matrix combination of the OLS or the maximum likelihood etimator ˆβ OLS and the prior mean β 0, β = Aˆβ OLS + (I p A)β 0 (8) where A i given by ( A = Var(ˆβ OLS ) 1 + αvar(β 0 ) 1) 1 Var(ˆβOLS ) 1 (9) ( 1 = I p Var(ˆβ OLS ) Var(ˆβ OLS ) + α 1 Var(β 0 )) (10) So, the normalized weight for ˆβ OLS and β 0 are given by the preciion matrix. The ame reult i obtained if one deire to compute the bet etimator from the minimum variance point of view of 9

10 β being a matrix combination of ˆβ OLS and β 0 namely, ) A = argminãvar (ÈβOLS + (I p Ã)β 0 One can remark from (6), that for Ω = k 1 I p and β 0 = 0, we get the ordinary ridge etimator ˆβ k given by (3). A Vinod and Ullah (1981) remarked, ome Bayeian feel that thi prior i unrealitic and a non null prior mean hould be ued but in abence of prior knowledge on β 0, it i often conervative to hrink toward the zero vector. When a prior knowledge about β 0 exit, then one hrink the ridge etimator toward thi known prior. Neverthele, the drawback i that different choice of the prior lead to different ridge etimator. It i worth mentioning that the Baye etimator of β given by (7) correpond to the etimator of the regreion coefficient for the mixed regreion model (Vinod and Ullah, 1981 ), y = Xβ + ε β = β 0 + η with E(η) = 0 and Var(η) = σ β Ω. Conditionally on β 0, the value of β given by (6) i then obtained by minimization with repect of β of 1 σ 2 (y Xβ) (y Xβ) + 1 σβ 2 (β β 0 ) Ω(β β 0 ) Even if the two approache lead to the ame olution, the interpretation are different. Bayeian model, β i a random variable, wherea in the mixed regreion model it i not. In the We have uppoed in thi ection that the regreor X j, for j = 1,..., p are tandardized and the vector of error ε i homocedatic. With heterocedaticity in the model (1), namely for Var(ε) = σ 2 V, where V = diag(v1 2, v2 2,..., v2 n) i the known poitive definite variance-covariance matrix, one can eaily tranform the heterocedatic model into a homocedatic one by multiplying the model (1) by a matrix G atifying the condition G G = V 1. Trenkler (1984) dicue the performance of biaed etimator in the linear regreion model under the heterocedaticity aumption. In what concern the tandardization of the regreor, the problem i more delicate and it i not alway very obviou when one hould tandardize the X-variable. The tandardization i not neceary for mot theoretical reult (Vinod and Ullah, 1981). However, it i adviable to 10

11 tandardize data before computing the ridge etimator pecially when there are large variation between regreor and they are meaured in different cale. An additional advantage of the tandardization i that it make the numerical magnitude of the component of β comparable with each other. A Kapat and Goel (2010) remarked, different olution for the ridge etimator ˆβ k may be obtained depending on the nature of the regreor, tandardized or not, and on the contrained norm. Thu, it i important to ditinguih between the olution of thee problem in order to avoid confuion. 3 Ue of ridge principle in urvey In thi ection, we undertake a detailed preentation of the ue of ridge principle in urvey ampling etting. Even if reult are omewhat imilar, the way they are derived i different from the claical tatitic and thi i motly due the fact that in urvey ampling framework, the main goal i to make inference about a function of y and not on the vector y or equivalently on the regreion coefficient, β. The implet cae, which will be conidered here, i the etimation of the finite population total t y = y k k U of the variable of interet Y of value y k. Here, U denote a finite population containing N element, U = {a 1,..., a k,..., a N } = {1,..., k,..., N} with the uppoition that a population unit i identifiable uniquely by it label k. Furthermore, a ample of ize n i elected from U and the vector y i known only on the ample individual. Uually, the finite population total t y i etimated by a weighted etimator ˆt w, ˆt w = w k y k (11) where the weight w k are derived uually uing auxiliary information by mean of a uperpopulation model (model-baed or model-aited approach) or by calibration. Uually, with multipurpoe urvey, weight hould not depend on the tudy variable in order to etimate mean or total of a very large number of variable. They hould alo be poitive and depend only on the auxiliary information. The weight necearily hould produce internally conitent etimator and if they are uitably choen, thee weight will produce etimator with maller variance than the etimator without uing the weight. 11

12 The idea of ridge etimation wa ued for the firt time in a urvey ampling framework in order to eliminate negative or extremely large weight obtained when a too retrictive condition of unbiaedne wa impoed. The latter ituation may caue inefficient reult rather than improving the etimator. So, weight are crucial in urvey ampling theory. From (11), the weight vector w = (w k ) k i the unknown parameter to be found. The role of β i taken now by w. In ection 3.1 and 3.2 we give in detail the derivation of ridge weight in urvey ampling a olution of penalized optimization problem (ection 2.2.1). The ame etimator may be obtained by uing a uperpopulation linear model depending on a parameter and the cla of model-baed or modelaited etimator for the finite population total. Thi way of computing ridge etimator in urvey ampling i the direct application of ridge principle from the claical regreion decribed in ection 2.2 and we preent it below. When we attribute a prior on previou etimation, we may ue the Bayeian interpretation to contruct ridge regreion type etimator. Deville (1999) conidered it a a calibration on an uncertain ource. We decribe it in ection 3.4. Suppoe that the relationhip between the variable of interet Y and the auxiliary variable X 1,..., X p i given by a uperpopulation model denoted by ξ in the urvey literature: ξ : y = Xβ + ε. (12) The explicative variable are not tandardized now. In order to ditinguih the population from the ample, let y = (y 1,..., y N ) be a N 1 vector of and let X = (X 1,..., X p ) be the N p matrix with x k = (X k1,..., X kp ) a row. The error ε k, for all k U are independent one of each other, of mean zero and variance Var(ε k ) = σ 2 vk 2. Let Var ξ(ε) = σ 2 V with V = diag(vk 2) k U and v k are poitive known contant. Some further notation are needed. Let X = (x k ) k, repectively y = (y k ) k, be the retriction of X, repectively of y, on the ample. Let alo Var ξ (ε ) = σ 2 V be the variance of ε, the retriction of ε on the ample, and Var ξ (ε ) = σ 2 V be the variance of ε, the retriction of ε on = U. The population variance V may be written a ( ) V 0 V = n (N n) 0 (N n) n V Without auxiliary information, t y i etimated by the Horvitz and Thompon (1952) etimator given by ˆt y,d = d k y k = y k π k (13) 12

13 where π k i the firt order incluion probability of the individual k U. The model (12) i then ued to improve the etimation of ˆt y,d by taking into account the auxiliary information given by X 1,..., X p. Uing the model ξ, one etimate the regreion parameter β and after, plug-in a model baed etimator, abbreviated a MB below, ˆt MB = y k + U x k β (14) or in a generalized difference etimator, abbreviated a DIFF below, ˆt DIF F = ( y k x k ) x k β. (15) π k π k U Thi mean that ˆt MB and ˆt DIF F rely on the etimation of the regreion coefficient β : bet linear unbiaed etimator of β for the MB (Royall, 1976) and the bet deign-baed etimator of β for the MA (Särndal, 1980). In a model-baed etting and uing the generalized leat quare (GLS) etimation under the model ξ, the etimator of the regreion coefficient β i obtained a olution of the optimization problem (P1) : ˆβGLS, = argmin β (y X β) V 1 (y X β) (16) yielding the etimator ˆβ GLS, = (X V 1 X ) 1 X V 1 y auming that (X V 1 X ) 1 exit.the bet linear unbiaed etimator (BLUE) of t y from the ξ-variance point of view i given by (Royall,1976) ˆt BLUE = y k + U x k ˆβ GLS, (17) If the matrix X V 1 X ha eigenvalue cloe to zero, then it i adviable to perturb it diagonal before inverting it. We obtain the ridge etimator of β a follow ˆβ MBR, = ( X V 1 X + D ) 1 X V 1 where D i a p p diagonal matrix with poitive quantitie on the diagonal. The ridge MB etimator given in (14) become y 13

14 ˆt MBR = ( ) y k + ˆβ MBR, (18) U x k A imilar reaoning may be ued in a deign-baed approach. The deign-baed etimator ˆβ π of the regreion coefficient β i the olution of the following optimization problem (Särndal, 1980), (P2) : ˆβπ = argmin β (y X β) V 1 Π 1 (y X β) where Π = diag(π k ) k. Thi optimization problem yield the following etimator for β, ˆβ π = ( X V 1 Π 1 X ) 1 X V 1 Π 1 y and the total t y i etimated by the well known GREG etimator (Särndal, 1980), ˆt GREG = ( y k x k ) x k ˆβ π k π π. (19) k U The ridge etimator of β become ( ˆβ π,r = X V 1 Π 1 X + D ) 1 X V 1 Π 1 y (20) for ome poitive diagonal matrix D and plugging-in (15), we obtain ˆt GREG,R = ( y k x k ) x k ˆβ π,r. (21) π k π k U The ridge etimator of β i ξ-biaed but i more table in preence of multicollinearity. 3.1 Ridge regreion under the model-baed approach Bardley and Chamber (1984) explored the relationhip between the unbalanced ample and multicollinearity. We call a balanced ample a ample for which the following relation i atified w k x k = U On the oppoite ituation, we have an unbalanced ample. A Bardley and Chamber (1984) tated, in multipurpoe ample urvey for which a large number of finite population total or mean are to be etimated, it i very difficult or even impoible to have a fully pecified model underlying each variable. In uch ituation, balanced ampling may protect from model mipecification (Royall x k. 14

15 and Heron, 1973). In the model-baed etting for unbalanced ample, excluion of variable may increae the bia and incluion of too many variable may reult in a overpecified model and the etimate will be untable and inefficient even if they are unbiaed. Alo thee variable can linearly be related with each other, and hence can caue multicollinearity. The trategy uggeted by Bardley and Chamber (1984) i to conider a many variable a they exit but to relax the balancing condition which i in fact the unbiaedne condition of the etimator under the model. Thi i equivalent to deriving a biaed etimator but with a maller prediction error and thi i why, it lead naturally to a ridge type etimator. Bardley and Chamber (1984) ugget finding the weight w = (w k ) k uch that the weighted etimator ˆt w = w ky k ha minimum ξ-mean quared error among the cla of bounded biaed etimator, (P3) : w MB,R = argmin w (w 1 ) V (w 1 ) + B CB (22) where B = w kx k U x k i the ξ-bia of ˆt w, C i ome diagonal cot matrix and 1 i the n-dimenional vector of one. The equality B = 0 mean that the etimator ˆt w i ξ-unbiaed or that the deign i exactly balanced. The optimization problem (P3) given by (22) mean that we look for weight w k that explain the bet the vector 1 according to a pecific metric and uch that the weighted etimator i not very far away from the true total. The metric employed here ue the ample variance V a we are in the cae of a model-baed approach. The minimization problem from above can alo be written a (P3 ) : w MB,R = argmin w, B 2 C r2(w 1 ) V (w 1 ) for the norm B 2 C = B CB which mean that we penalize large value of the bia B. Solving thi minimization problem, we obtain the weight w MB,R given by w MB,R = 1 V 1 ( X X V 1 X + C 1) 1 (1 X 1 UX) (23) leading to the model-baed ridge etimator ˆt w,mb = w MB,R y, ˆt w,mb = y k + ( U x k ) ˆβ w,r (24) 15

16 with ˆβ w,r = ( X V 1 X + C 1) 1 X V 1 y. Remark that we obtain an etimator imilar to the one obtained in (14). We have mentioned before that the weight vector w take in a way the place of the regreion coefficient β. We have een in ection 2.2 that introducing a penalty parameter reduce the length of β. The ame reult i true for the weight vector (Bardley and Chamber, 1984). Conider the particular cae V = I n and C 1 = ki p, w MB,Rw MB,R 1 UX(X X + ki p ) 1 X X (X X + ki p ) 1 X 1 U = p ηi 2 λ i (λ i + k) 2 j=1 where λ i, i = 1,..., p are the eigenvalue of X X, η = (η i ) p i=1 = PX 1 U and P i the matrix of eigenvector aociated to the eigenvalue of X X. Following the ame argument, we obtain that w MBw MB 1 UX(X X ) 1 X 1 U = p 1 ηi 2 λ j=1 i Since for any k > 0, we alway have 1 λ i > λ i (λ i +k) 2, we get that w MB,R w MB,R < w MB w MB. Thi prove that the catter of ridge weight i maller and more table under perturbation of X than that of BLUE weight. Thi i in concordance with the ridge principle. We can ee alo that in the preence of multicollinearity, repectively of ill-conditionning, λ min = minλ i are cloe to zero, repectively the conditionning number K = λ max /λ min i very large, which entail negative or extremely large calibration weight. It i worth mentioning two extreme value of ˆt w,mb. A C (i.e. infinite cot aociated with the bia B), we obtain w MB,R = 1 V 1 X ( X V 1 ) 1 X X V 1 y and ˆt w,mb i the minimum variance unbiaed linear etimator (Royall, 1970). Thi mean that the contraint B = 0 i exactly atified. On the oppoite cae, a C 0, we obtain w MB,R = 1 and ˆt w,mb = y k which i equivalent to removing the contraint from the optimization problem. The derivation of model-baed ridge etimator depend on the cot matrix C. Conidering that C = k 1 C, Bardley and Chamber (1984) and Chamber (1996) ue the ridge trace to determine the appropriate k. C i a fixed cot matrix providing a correct relative weighting of the component of the relative bia vector (diag(x 1 U )) 1 B. Thi tranformation wa needed becaue of the large difference in cale between the predictor in X and it i a kind of tandardization of variable. 16

17 3.2 Ridge under the calibration approach or penalized calibration Without auming a uperpopulation model, one can ue the calibration method (Deville and Särndal, 1992) which conit in deriving a weighted etimator ˆt w = w k y k with weight minimizing a peudo-ditance, ubject to calibration contraint (i.e. all the auxiliary variable total are exactly etimated). Uually a chi-quare ditance i ued, the calibration weight w c = (w c k ) k (P4) : w c = argmin w (w d ) Π (w d ) ubject to (w c ) X = 1 UX (w k d k ) 2 d k q k, yielding where Π = diag(q 1 k d 1 k ) k and q k are poitive contant. Mot of the time, we conider q k = 1 for all k. The calibration weight thu get the following hape, w c = d 1 Π X (X 1 Π X ) 1 (d X 1 UX) For q k = 1/vk 2, the calibrated etimator ˆt yw = (w) c y i equal to the GREG etimator given by (19). Moreover, remark that in thi cae we have Π = V Π, which mean that the optimization problem (P2) ue the invere of the weight matrix employed in (P4). For a more general ditance function, Deville and Särndal (1992) how that under certain condition the calibrated etimator i aymptotically equivalent to the model-aited or GREG etimator ˆt GREG. Thi equivalence i in the ene that N 1 (ˆt yw ˆt GREG ) = O p (n 1 ). Thi fact will conequently lead to the aymptotic equivalence of the variance of both etimator. From a geometrical point of view, we earch the weight w k which explain the bet the Horvitz- Thompon weight d k = 1/π k and that lie in the orthogonal of the contraint pace given by the kernel of the matrix X. The contraint pace i of dimenion n p, o increaing the number of auxiliary variable will decreae the number of degree of freedom for w k (Guggemo and Tillé, 2010). The imilar reaoning by Silva and Skinner (1997) proved that by increaing the number of calibration variable after a certain number may increae the variance up to a harmful level. Guggemo and Tillé (2010) called it over-calibration and uggeted not calibrating on thoe variable which are le correlated with the variable of interet. Another iue with the calibration weight i the fact that they may not atify range retriction (i.e. pre-pecified lower and upper bound) epecially when the number of calibration or benchmark 17

18 contraint i large. Satifying uch condition i deirable epecially for avoiding the inflation of the ampling error of etimate in mall to moderate domain (Beaumont and Bocci, 2008). A Deville and Särndal (1992) tated, negative weight may occur when the chi-quared ditance i employed. For the other ditance ued in their paper, the poitivene of weight i guaranteed but unrealitic or extreme weight may alo occur. To cope with thi iue, everal modification have been uggeted in the literature. However, all thee method are iterative and may not yield a olution even if the range retriction are mild (Rao and Singh, 1997 and Beaumont and Bocci, 2008). Thi i more likely to happen when they are many contraint, and o multicollinearity of data, or when the ample ize i mall. So, how to avoid negative or extremely large weight? Chamber (1996) and Rao and Singh (1997) anwer thi quetion by uggeting to relax the calibration contraint. Suppoe we have non-negative contant C j with j = 1,..., p repreenting the cot of the weighted etimator which doe not atify the calibration equation. The cot C j can alo be the cot of the rik aociated with the calibration equation not to be atified. The objective function can be given a, (P5) : w c R, = argmin w (w d ) Π (w d ) + 1 λ (w X 1 UX)C(w X 1 UX) (25) Rao and Singh (1997) conider the objective function without the contant λ. Writing the problem (P5) a a contrained optimization problem, put into evidence that we leen the calibration equation correponding to thoe variable which are omehow unable to atify the calibration contraint but not too much ince we penalize the large value of w X 1 UX. In thi way we eliminate the poibility of having very large or negative weight. Simply, we can ay that the ridge etimator perform a a variable election tool. The weight are given by w c R, = d 1 Π X (X 1 Π X + λc 1 ) 1 (X d X 1 U ) (26) which yield the ridge calibration etimator or the penalized calibration of the population total t y, where ˆβ λ = (X ˆt y,rw = (w c R,) y = d y (X d X 1 U ) ˆβλ 1 Π X + λc 1 ) 1 X = ˆt y,d (ˆt x,d t x ) ˆβλ (27) 1 Π y and ˆt x,d i the Horvitz-Thompon etimator for the total t x. Thi approach i equivalent to contruct a GREG etimator of population total with the 18

19 regreion coefficient etimated by a ridge etimator (Hoerl and Kennard, 1970). More preciely, ˆβ λ i in fact ˆβ π,r from (20) for λc 1 = D and Π 1 = V 1 Π 1. The ridge etimator given by (27) can be written a a linear combination of the Horvitz-Thompon etimator and the GREG etimator (Rao and Singh, 1997) a follow, ˆt y,rw = (1 α)ˆt y,d + αˆt GREG where ˆt GREG,R = ˆt y,d (ˆt x,d t x ) ˆβπ i the GREG etimator given by (19) and α i given by, ( α = y 1 Π X X 1 Π X + λc 1) [ 1 ( ) ] (tx ˆt x,d ) y 1 Π X X Π X (tx ˆt x,d ) A for the model-baed approach, the Horvitz-Thompon a well a the GREG etimator are two limit value of ˆt y,rw. More exactly, conider relation (27) for a fixed cot matrix C and let λ vary from 0 to. The ridge calibration etimator i a continuo function of λ. For λ = 0, then α = 1 and an infinite cot i attributed to all contraint meaning that they are all exactly atified. It implie that ˆt y,rw i the GREG etimator which i ξ-unbiaed for the population total t y. Ridge weight with trictly poitive biaing parameter λ mean that the weight do not atify exactly the calibration equation. In thi cae, the etimator ˆt y,rw i ξ-biaed but the weight w c R, are more table (Chamber, 1996) and implied a reduction in MSE (Bardley and Chamber, 1984 ). Value of λ producing weight larger or equal to 1 are accepted by Chamber (1996). A λ, α 0 and the ridge calibrated etimator ˆt y,rw goe to the Horvitz-Thompon etimator. In thi cae, we do not ue any of the auxiliary variable for the etimation of the finite population total of the variable of interet. It i of interet to ee how ˆt y,rw change when a pecific cot C j varie from 0 to. The zero cot C j = 0 mean that the contraint correponding to the total t Xj i dicarded and the large or infinite cot C j =, that the correponding calibration contraint i exactly atified. In the latter ituation, the weight are computed uing (25) with the cot matrix C 1 having 0 on the j-th diagonal element. Uing the ame jutification given by Hoerl and Kennard (1970) for obtaining the ridge regreion coefficient a a olution of contrained minimization (ee ection 2.2.1), the weight w c R, 19

20 atifying the optimization problem (P5) atify alo the following optimization problem, (P6) : w c R, = argmin w (w X 1 UX)C(w X 1 UX) + λ(w d ) Π (w d ) = argmin w, w d 2 e Π r 2(w X 1 UX)C(w X 1 UX) which mean that we find the mallet ditance between the weighted etimator ˆt y,rw and the total t y with weight atifying the range retriction. The geometric interpretation are omehow imilar to thoe given in ection and uing Figure 1. Beaumont and Bocci (2008) give another jutification of thi reult and ugget a biection algorithm to find w k knowing that the maximum value of (w X 1 U X)C(w X 1 U X) i reached for w = d while atifying the range retriction. The minimum value i reached for the GREG etimator but without repecting necearily the retriction on w. Neverthele, a Beaumont and Bocci remarked, thi algorithm may be time-conuming. 3.3 Partially penalized ridge or partially penalized calibration In a model baed approach, Bardley and Chamber (1984) uggeted to divide the p variable in the data matrix X into two et of variable X 1 and X 2 baed on the fact that variable in X 1 contain much more importance than the variable in X 2 in the ene that they can contribute more influentially in the etimation proce. We may conider that the matrix X ha the following expreion after re-ordering the variable X 1,..., X p, ( X = X1, X ) 2 where X 1 = [X 1,..., X q ] and X 2 = [X q+1,..., X p ]. The variable contained in X 1 may be related for example to ocio-demographic criteria. Bardley and Chamber (1984) attach the importance to the variable in term of cot which are in fact penaltie aociated to the variable. Let C be the diagonal matrix of nonnegative cot which can meaure the acceptable level of error while etimating the total of variable from the X matrix, ( C1 0 C = (q,p q) 0 (p q,p) C 2 ) where C 1, repectively C 2, i the relative diagonal cot matrix of ize q q aociated to X 1, repectively of ize (p q) (p q) aociated to X 2. A dicued in the above ection, allowing an infinite cot C j mean that the aociated contraint i 20

21 exactly atified. Bardley and Chamber (1984) conider the cae when contraint correponding to X 1,..., X q are all exactly atified. Thi mean C 1 = and hence, weight may be derived uing relation (23) with C 1 1 = 0 (q q). The weight uing thi partially penalized ridge regreion and abbreviated a w ppr below can be written a, w ppr = ( 1 V 1 X 1, V 1 ) ( X X 1 V 1 2 X 2 V 1 X 1 X 1 X 1 V 1 X 2 V 1 X 2 X 2 + C 1 2 ) 1 ( X 1 1 X 1 1 U X 2 1 X 2 1 U ) (28) where X 1, repectively X 2, i the ample retriction of X 1, repectively of X 2. Uing a calibration approach, the weight are derived uing the above formula with V replaced by Π and 1 by d. In particular, we have w ppr X 1 = 1 X U 1. Now, if the cot matrix C 2 alo goe to infinity, then the contraint correponding to variable in X 2 are alo exactly atified. Hence, the etimator uing the weight o derived i again nothing ele than the bet linear unbiaed etimator ˆt BLUE given by (17) and derived under the model ξ that ue the whole matrix X. Moreover, in the cae C 2 0 (p q,p q) the variable included in X 2 are dicarded from the contraint and thu the model will include only the calibration variable from X 1, w ppr w ppr (1) = h V 1 ( ) 1 X 1 X 1 V 1 X 1 (1 X1 1 X U 1 ) The penalized etimator become the bet unbiaed etimator baed under the retricted model that ue only the matrix X 1. Since ˆt BLUE baed on the whole model ξ a well a on the retricted model with X 1 are two extreme etimator a C 2 varie from to 0, Bardley and Chamber (1984) called the etimator that ue weight w ppr an interpolated etimator between the two extreme. So, the penalized ridge etimator may be conidered a a trade-off between an over-pecified model and an under-pecified model. One can how that the ridge weight w ppr verifying the optimization problem (P3) with the invere matrix cot C 1 = ( 0(q,q) 0 (q,p q) 0 (p q,p) C 1 2 ) (29) 21

22 may be obtained a a olution of the following optimization problem (P7) : w ppr = argmin w (w 1 ) V 1 (w 1 ) + (w X 2 1 X U 2 )C 1 2 (w X 2 1 X U 2 ) w X 1 = 1 U X 1 (30) Thi kind of optimization problem wa ued by Park and Yang (2010) and Guggemo and Tillé (2010). Uing the model ξ given by (12) with intercept, Park and Yang (2010) aim at etimating the mean y U = U y k/n of the variable of interet Y uing a Hajek-type etimator. Thi mean that they ue a weighted etimator with weight that um up to unity and being a cloe a poible to the Hajek weight, α i = π 1 i 1 π i Thi mean that the optimization problem (P7) i ued with 1 replaced by α = (α i ) i. They build two partially penalized etimator. In the firt cae, X1 = 1 U and in the econd cae, X 1 = (1 U, X 2,..., X q ). Weight may be derived uing relation (28). Slightly implified formula are obtained ince 1 α 1 U 1 U/N = 0. In a linear regreion context, it i not very common to conider the penalty or the cot matrix C 1 given by (29). Thi i more likely to happen with a mixed model. Guggemo and Tillé (2010) conider the following mixed model ξ : y = X 1 B + X 2 u + η and the calibration approach, namely we replace in the objective function from the optimization problem (P7) from (30) the matrix V by Π, repectively 1 by d. Guggemo and Tillé conider alo that the econd term of the objectif function depend on a penalty parameter and they ugget the Fiher coring algorithm to compute it. The value of the penalty parameter i obtained at the convergence of the Fiher coring algorithm. They give alo application of the penalized calibration for etimation of finite population total in a mall area context. 3.4 Calibration on uncertain auxiliary information In preence of everal extern etimation which may be conidered a uncertain, Deville (1999) uggeted another contruction which ue in fact the Bayeian interpretation of the ridge etimator given in ection Conider that another etimation ˆt x,d = d X baed on the auxiliary information X i available from external ource uch a previou urvey. We have alo the 22

23 current etimation baed on X. We uppoe that the variance of ˆt x,d and ˆt x,d are known and the covariance between the two ource i zero. We uppoe alo that the covariance between ˆt x,d and ˆt y,d i alo zero. Deville look for linear weighted etimator for t y of the form ˆt w = d y + (d X d X )β = ˆt y,d + (ˆt x,d ˆt x,d ) β. (31) The optimal value of the unknown parameter β i the one that minimize the ampling variance of ˆt w. We find β opt = ( Var(ˆt x,d) + Var(ˆt x,d ) ) 1 Cov(ty,d, t x,d ) and the ame value may be derived by uing a variance minimization criteria a in Montanari (1987) plu a penalty term, namely (P8) : β opt = argmin β (y X β) (y X β) + β X X β (32) where = ( π ij π i π j π ij ) i,j U. We remark that the penalty i now on the variance of ˆt x,d. The etimation of t y given by (31) computed for β = β opt may be improved by replacing ˆt x,d with the bet unbiaed linear etimator of ˆt x,d and ˆt x,d. Thi i equivalent to determine the poterior etimation knowing that the priori etimation given by the auxiliary information i ˆt x,d and the actual etimation i ˆt x,d. One may ue relation (8) to find the poterior etimation a ˆt opt x,x = (I p A) ˆt x,d + Aˆt x,d where A i a quared p-dimenional matrix given by A = I p Var(ˆt x,d ) ( Var(ˆt x,d ) + Var(ˆt x,d) ) 1. Then, one can derive the etimator ˆt opt y of t y from relation (31) with ˆt x,d replaced with ˆt opt x,x, ˆt opt y = ˆt y,d + (ˆt opt x,x ˆt x,d ) (Var(ˆt x,d )) 1 Cov(t y,d, t x,d ) One can eaily obtain that for y i = x i, we obtain ˆt opt x,x or equivalently, the etimator i calibrated on ˆt opt x,x. If the variance covariance Cov(t y,d, t x,d ) i etimated by the uual Horvitz-Thompon etimator, ˆt opt y i a linear etimator in y i with weight w i given by w i = d i + (ˆt x,d ˆt x,d ) (Var(ˆt x,d )) 1 z i d i, 23

24 where z i = ij x j j π ij π j. The main advantage of Deville contruction i that it doe not need to determine a penalty parameter a it wa the cae before. All we need i the variance of the externe etimation. Deville (1999) give alo a practical implementation and generalization to everal external etimation. 3.5 Statitical propertie of ridge etimator with urvey data Ridge-type etimator are biaed etimator uggeted in claical regreion in order to diminih the model mean quare error. Bardley and Chamber (1984) affirm that the model-baed ridge etimator ha maller prediction variance than the bet linear unbiaed etimator ˆt BLUE but they do not give a rigorou proof. Bellhoue (1987) how that a predictor Ŷ (1) = y k + (N n)ˆµ (1) of the finite population total t y i better than another predictor Ŷ (2) = y k + (N n)ˆµ (2) repect to the mean quare error under the model ξ and the ampling deign p if, for every ample of fixed ize n, ˆµ (1) i better than ˆµ (2) in the ene that E ξ (ˆµ (1) µ n ) 2 E ξ (ˆµ (2) µ n ) 2 where µ n i the unknown prediction of the non ampled mean of Y. Uing thi reult and the ame argument a in Vinod and Ullah (1981), one can get that for any penalty contant k atifying with 0 < k < 2σ 2 /β β, E ξ E p (ˆt w,mb t y ) 2 < E ξ E p (ˆt BLUE t y ) 2 where ˆt w,mb i the ridge model baed etimator given by (24) for C 1 = ki p and ˆt BLUE i the bet linear unbiaed etimator given by (17). A neceary and ufficient condition for the ridge etimator to be more efficient than the leat quare etimator i 0 < k < 2/[ min(0, ψ)] where ψ i the minimum eigenvalue of (X X ) 1 (ββ /σ 2 ) (Swindel, and Chapman, 1973). Duntan and Chamber (1986) derived confidence interval for finite population total etimated uing the ridge model-baed procedure and robut model-baed variance etimator. In a deign-baed etting, the concern i about aymptotic propertie of ˆt y,rw given by (27) with repect to the ampling deign p. A Rao and Singh (1997) tated, an important requirement while relaxing benchmark contraint i that for given tolerance level, the calibration method hould enure deign conitency like the generalized regreion method. The aymptotic deign unbiaedne and 24

25 conitency of ˆt y,rw are derived uing the equivalence with GREG etimator even if ˆt y,rw ha been obtained a a olution of penalized calibration problem. Under broad aumption (Fuller, 2002), the deign-baed ridge etimator ˆβ λ of β tend in probability to β λ = (X X + λc 1 ) 1 X y and the ridge etimator ˆt y,rw i aymptotically equivalent to ˆt y,rw d y (X d X 1 U ) β λ = d (y X β λ ) + 1 UXβ λ which implie that the ˆt y,rw i aymptotically deign unbiaed and conitent under abroad aumption that provide the deign unbiaedne and conitency of the Horvitz-Thompon etimator d y and d X (Rao and Singh, 1997 and Théberge, 2000). The aymptotic variance under the ampling deign may thu be deduced a being the Horvitz-Thompon variance applied to reidual y k x k β λ. 4 Concluion and extenion In thi paper, we have undertaken an overview of the application of ridge-type etimator in urvey ampling theory. Even if the paper of Bardley and Chamber (1984) ha not received much attention at the beginning, we ait now of an increaingly interet on thi ubject. Thi i motly due to the fact that nowaday, we face more and more information and thi kind of iue i more often encountered in practice than before. To ue thi cla of etimator, two practical iue hould be treated carefully. The firt one i the computation of the penalty parameter. Several algorithm have been uggeted in the literature uch a the ridge trace (Bardley and Chamber, 1984), the Ficher coring algorithm (Guggemo and Tillé, 2010) or the biection algorithm (Beaumont and Bocci, 2008). Beaumont and Bocci (2008) compare the ridge calibration with the method of Chen et al. (2002) howing the uperiority of the ridge calibration method. Neverthele, it would be intereting having a comparion between all thee algorithm. Their i another important point that we would like to tre. All the paper dealing with ridge-type etimator in urvey ampling give few detail about the tandardization of the auxiliary variable if any ha been done. Or, a mentioned at the end of ection 2.2.1, it i important to know what kind of tandardization i ued ince different method lead to different ridge etimator. The cot matrix ued in the objective function from the optimization problem (P3) and (P5) may be interpreted a a tandardization matrix. 25

26 Finally, ome other alternative method for dealing with huge data et mut be invetigated. We mention here the lao method which conit in conidering a penalty with the abolute value intead of the euclidian norm. We are not aware of the exitence of uch application in urvey ampling. The regreion on principal component analyi i another intereting alternative. Thi method conit in conidering the principal component of X X which reduce the number of auxiliary variable while keeping maximum of information. For huge urvey data, Goga et al. (2011) ugget calibration on the et of thee new variable which i in general of much maller dimenion than the initial one. Bibliography Bardley, P. and Chamber, R.L. (1984), Applied Statitic, 33, Multipurpoe etimation from unbalanced ample, Beaumont, J.-F. and Bocci, C. (2008), Another look at ridge calibration, Metron-International Journal of Statitic, LXVI, Bellhoue, D. R. (1987), Model-baed etimation in finite population ampling, Journal of the American Statitical Aociation, 41, Chamber, R.L. (1996), Robut Cae-Weighting for Multipurpoe Etablihment Survey, Journal of Official tatitic, 12, Chen, J., Sitter, R.R. and Wu, C. (2002), Uing empirical likelihood method to obtain range retricted weight in regreion etimator for urvey, Biometrika, 89, Conniffe, D. and Stone, J. (1973), A critical view of ridge regreion, The Statitician, 22, Deville, J.C. (1992), Contrained ample, conditional inference, weighting: three apect of the utilization of auxiliary information, In Proceeding of the Workhop on the Ue of Auxiliary Information in Survey, rebro (Sweden). Deville, J.C. (1999), Calage imultané de pluieur enquête, Recueil du Sympoium 99 de Statitique Canada, mai, Deville, J.-C., and Särndal, C.-E. (1992), Calibration etimation in urvey ampling, Journal of the American Statitical Aociation, 418,