Acta Universitatis Palackianae Olomucensis. Facultas Rerum Naturalium. Mathematica Lubomír Kubáček; Eva Tesaříková Underparametrization of weakly nonlinear regression models Acta Universitatis Palackianae Olomucensis. Facultas Rerum Naturalium. Mathematica, Vol. 49 21, No. 2, 83--93 Persistent URL: http://dml.cz/dmlcz/141419 Terms of use: Palacký University Olomouc, Faculty of Science, 21 Institute of Mathematics of the Academy of Sciences of the Czech Republic provides access to digitized documents strictly for personal use. Each copy of any part of this document must contain these Terms of use. This paper has been digitized, optimized for electronic delivery and stamped with digital signature within the project DML-CZ: The Czech Digital Mathematics Library http://project.dml.cz
Acta Univ. Palacki. Olomuc., Fac. rer. nat., Mathematica 49, 2 21 83 93 Underparametrization of Weakly Nonlinear Regression Models * Lubomír KUBÁČEK 1, Eva TESAŘÍKOVÁ 2 1 Department of Mathematical Analysis and Applications of Mathematics Faculty of Science, Palacký University 17. listopadu 12, 771 46 Olomouc, Czech Republic e-mail: kubacekl@inf.upol.cz 2 Department of Algebra and Geometry, Faculty of Science, Palacký University 17. listopadu 12, 771 46 Olomouc, Czech Republic e-mail: tesariko@inf.upol.cz Received January 12, 21 Abstract A large number of parameters in regression models can be serious obstacle for processing and interpretation of experimental data. One way how to overcome it is an elimination of some parameters. In some cases it need not deteriorate statistical properties of estimators of useful parameters and can help to interpret them. The problem is to find conditions which enable us to decide whether such favourable situation occurs. Key words: Weakly nonlinear regression model, underparameterization,mse,blue. 2 Mathematics Subject Classification: 62J5, 62F1 Introduction Real events and processes in many professions can be modelled adequately in many situations by help of a large number of parameters. If all of them can be interpreted by a professional language, then it is no reason to neglect them. Their elimination can lead to a misinterpretation of other parameters, since estimators of them are influenced by the underparametrization of the model. Sometimes not all parameters can be interpreted in professional language and then a problem arises whether noninterpretable parameters can be neglected. * Supported by the Council of the Czech Government MSM 6 198 959 214. 83
84 Lubomír KUBÁČEK, Eva TESAŘÍKOVÁ 1 Prerequisities Let Y n [ fβ, γ, Σ ] means that Y is an n-dimensional random vector with the mean value EY equal to fβ, γ andwiththecovariancematrixvary equal to Σ, which is assumed to be known and p.d. positive definite. The function f, : R k+l R n R s means the s-dimensional linear vector space can be, with sufficiently high accuracy, expressed as fβ, γ =f + F + S + 1 2 κ,, where f = fβ, γ, β, γ are approximate values of the true values of the unknown vectors β, γ, = β β, = γ γ, F = fβ, γ / β, S = fβ, γ / γ, κ, = [ κ 1,,...,κ n, ], κ i, =, Fi,1,1, F i,1,2, F i,2,1, F i,2,2 Fi,1,1, F F i = i,1,2, F i,2,1, F i,2,2 F i,1,1 = 2 f i β, γ β β, F i,1,2 = 2 f i β, γ β γ, F i,2,1 = 2 f i β, γ γ β, F i,2,2 = 2 f i β, γ γ γ, i =1,...,n. Let the rank rf, S of the matrix F, S be rf, S =k + l, wherek is the dimension of the vector β and l is the dimension of the vector γ. Lemma 1.1 In the underparametrized and linearized model Y f n F, Σ the BLUE best linear unbiased estimator of is under =F Σ 1 F 1 F Σ 1 Y f. Its bias in the model Y f n [F, S + 12 ] κ,, Σ 1 is b = C 1 F Σ 1 S + 1 2 C 1 F Σ 1 κ,, where C = F Σ 1 F. Proof is elementary and therefore it is omitted in more detail cf. [2] and [6].
Underparametrization of weakly nonlinear regression models 85 The symbol I means the identity matrix and A + is the Moore Penrose generalized inverse [7] of the matrix A. The projection matrix on the column space MA m,n = {Au: u R n } of the matrix A is denoted as P A, i.e. P A = AA + and M A = I P A. In the following text different approaches cf. also [4] and [5] are described, which enable us to decide whether the parameter can or cannot be neglected. 2 The case of a single function Let a single linear function h β = h β + h, be under consideration in the model 1. Then the following theorem can be stated. Theorem 2.1 If A h, then Here A h [ u v A h = n i=1 A h = Var h under +h b 2 < Var h true. { u : u R v k, v R l, [ u + 1 ] v 2 A+ h + 1 ] 2 A+ h +, a u hm Ah 1 } v 4, a ha + h c h, { } 1 2 h C 1 F Σ 1 c h = F i, M Ah = I A h A + h, a h = h C 1 F Σ 1 S, i h C 1 F Σ 1 S [ S M F ΣM F + S ] 1 S Σ 1 FC 1 h and true is the BLUE of the vector in the model ] Y f n [F, S, Σ. 2 Proof Since the BLUE of in the model 2 is true = C 1 F Σ 1 Y f C 1 F Σ 1 S [ S M F ΣM F + S ] 1 [ Y f FC 1 F Σ 1 Y f ], where C = F Σ 1 F and the vectors C 1 F Σ 1 Y f and Y f FC 1 F Σ 1 Y f are noncorrelated, it is valid that Var true =C 1 + C 1 F Σ 1 S [ S M F ΣM F + S ] 1 S Σ 1 FC 1.
86 Lubomír KUBÁČEK, Eva TESAŘÍKOVÁ Thus Varh true =Varh under + h C 1 F Σ 1 S [ S M F ΣM F + S ] 1 S Σ 1 FC 1 h. The bias of the estimator h under in the model 1 is h b = a h +, A h. The linear quadratic form on the right hand side of the last equality, can be expressed as follows. In general / MA h =MA + h. Therefore the vector is decomposed, i.e. = P Ah + M Ah, where P Ah a h MA h.thus and h b =, a hm Ah +, a hp Ah +, A h, a h [ since The assumption + 1 2 A+ h +, A h ] +, a hm Ah A + h P A h a h = A + h a h [ = + 1 ] 2 A+ h A h 1 4, a ha + h, and A h A + h = P A h. A h implies h b <c h and Var h true Var h under = c 2 h > h b 2 implies the statement of the theorem.
Underparametrization of weakly nonlinear regression models 87 Remark 2.2 The semiaxes of the quadratic on the left hand side of the following equality [ + 1 ] [ 2 A+ h A h + 1 ] 2 A+ h = c h + 1 4, a ha + h, a hm Ah depends on the vector M Ah which is orthogonal to MA h. Let u = P Ah, v = M Ah. Then the boundery of the set A h is { u : v [ u + 1 ] [ 2 A+ h A h u + 1 ] 2 A+ h = c h + 1 4, a ha + h }, a hv, i.e. it is a paraboloid with the section for v = equal to { [ u: u + 1 ] [ 2 A+ h A h u + 1 ] 2 A+ h = c h + 1 4, a ha + h }. Another approach to a problem of a single function is as follows. Since h b = h C 1 F Σ 1 S +, A h, two sets, i.e. and { A 1,h = R k : h C 1 F Σ 1 S 2 ε } h C 1 h A 2,h = can be constructed. Thus the set { :, A h ε } h C 2 1 h C h = A 1,h A 2,h is the set with the property C h h b ε h C 1 h,
88 Lubomír KUBÁČEK, Eva TESAŘÍKOVÁ where h is given by the function hb =h b, b R k, considered. The statement is obvious in the view of the Scheffé inequality [8] {h R k } h b ε h C 1 h iff b Cb ε. Instead of the matrix C 1 =Var under,thematrix C 1 + C 1 F Σ 1 S [ S M F ΣM F + S ] 1 S Σ 1 FC 1 =Var true can be used. 3 The case of the whole vector The bias of the estimator under in the model 1 is composed of the two terms, i.e. C 1 F Σ 1 S which is due to neglecting the parameter in the model and 1 2 C 1 F Σ 1 κ, which is due to the nonlinearity of the model. The first term is suppressed sufficiently if ε 2 S Σ 1 FC 1 CC 1 F Σ 1 S, 2 where ε> is sufficiently small number. In view of the Scheffé inequality it is equivalent to the relation {h R k } h b ε 2 h C 1 h. Let { } ε 2 A 1 = R k : S Σ 1 FC 1 CC 1 F Σ 1 S. 2 The second term can be analyzed by a small modification of the Bates and Watts parametric curvature cf. [1] 1 C par 4 β, γ =sup κ Σ 1 FC 1 CC 1 F Σ 1 κ : R k+l F, S Σ 1 F, S a simple algorithm for a numerical determination of C par β, γ is given in [1]. It is valid that in more detail cf. [3] and [6] A 2 { u = :u v, v F Σ 1 F, F Σ 1 } S u ε/2 S Σ 1 F, S Σ 1 S v C par β, γ {h R k } 1 h 2 C 1 F Σ 1 κ, 2 ε h C 1 h. Thus the following theorem can be stated.
Underparametrization of weakly nonlinear regression models 89 Theorem 3.1 The model Y f n [F, S + 12 ] κ,, Σ can be substituted by the model Y f n F, Σ if A 1 A 2. In this case {h R k } h b ε h C 1 h. 4 Numerical example Let y i = β 1 exp β 2 x i +γx i + ε i, i 1 2 3 4 5 6 x i 1 2 3 4 5 6 Y =Y 1,Y 2,...,Y 6, VarY =σ 2 I =.1 2 I, = 1 1, γ 2 β f =5.6788, 5.3534, 6.4979, 8.1832, 1.674, 12.248, F = EY β 1,β 2 = exp β 2x 1, x 1 β 1 exp β 2 x 1... exp β 2 x 6, x 6 β 1 exp β 2 x 6.3679, 3.6788.1353, 2.767 =.498, 1.4936.183,.7326,.67,.3369.25,.1487 S =1, 2, 3, 4, 5, 6, 2 EY F i = = β β,γ γ i =1, 2,...,6,, x i exp β 2 x i, x i exp β 2 x i,x 2 i β 1 exp β 2 x i,,,,
9 Lubomír KUBÁČEK, Eva TESAŘÍKOVÁ.,.368,..,.271,. F 1 =.368, 3.679,., F 2 =.271, 5.413,.,.,.,..,.,..,.149,..,.73,. F 3 =.149, 4.481,., F 4 =.73, 2.931,.,.,.,..,.,..,.34,..,.15,. F 5 =.34, 1.684,., F 6 =.15,.892,.,.,.,..,.,. C = 1 σ 2 F F = 1.157, 1.81.1 2. 1.81, 23.763 i The case of a single function h =1, = 6 { }.,.,. 1 A h = 2 1, F F 1 F F i =., 12.584,., i=1 i.,.,. a h =1, F F 1 F S = 29.169, 1 c h = σ 1, F F 1 F SS M F S 1 S FF F 1 =.37713, 1 2 A+ h = 1,,, M a Ah =,,, h,, 1 1 =, A h = { cf. also Fig. 1 4,a h A + h a h [ [ + 1 ] 2 A+ h { : R 2, R 1, c h + 1 4,a h A + h :, a h + 1 ] 2 A+ h A h,a h M Ah,.,.., 12.584,.,.,. } }.377 + 29.2,
Underparametrization of weakly nonlinear regression models 91 4 2 1 1 1 1 5 2 5 1 Fig. 1 The set A h Another approach is characterized by the sets A 1,h and A 2,h for ε =.4 and σ =.1. A 1,h = R 2 { : 1, F F 1 F S ε } {F 2 σ F 1 } 1,1 A 2,h = = R 2 { : 5.12 1 5 }, { :,A h ε2 } {F σ F 1 } 1,1.,.,. = :,., 12.584,..,.,. C h = A 1,h A 2,h..1446, iithecaseofthewholevector C par β,γ = 1 4 sup σ κ,p F κ, F, F, F : R S 3 =.12652, S F, S S { } A 1 = R 2 εσ : 4S FF F 1 F S = R 2 { : 3.5818 1 5},
92 Lubomír KUBÁČEK, Eva TESAŘÍKOVÁ { A 2 = : F, F, F S σ 2 } ε S F, S S 2C par β,γ.157, 1.81,.91 = :, 1.81, 23.763, 19.8 4.16524 1 5.91, 19.8, 91. for ε =.4 and σ =.1. The set A 1,h has the same shape as the set A 1 in Fig. 2b and 2c however it is wider 5.12/3.5818 = 1.399 times. The section of the set A 1 A 2 cf. on Fig. 2. 4 x 1 3 3 2 1 2 1 2 3 A 2 4.5.5 1 Fig. 2a The section of the set A 2 in coordinates 1 and 2.8.6.4.2.2.4.6.8 1 x 1 3 A 1 A 2 1.2.1.1.2 1 Fig. 2b The section of the set A 1 A 2 in coordinates 1 and
Underparametrization of weakly nonlinear regression models 93.8.6.4.2.2.4.6.8 1 x 1 3 A 1 A 2 1 2 1 1 2 2 x 1 3 Fig. 2c The section of the set A 1 A 2 in coordinates 2 and The criterior for neglecting the parameter γ is rigorous. If the function fβ 1,β 2, γ =β 1 is estimated, the value β 2 must be smaller than.1 and γ must be smaller than.51, i.e. in fact they cannot be neglected. If the whole vector is estimated, the situation is even worse. The model considered cannot be reduced. References [1] Bates, D. M., Watts, D. G.: Relative curvature measures of nonlinearity. J.Roy.Stat. Soc. B42198, 1 25. [2] Kubáček, L., Kubáčková, L., Volaufová, J.: Statistical Models with Linear Structures. Veda, Bratislava, 1995. [3] Kubáček, L., Kubáčková, L.: Regression Models with a weak Nonlinearity. Technical Report Nr. 1998.1, Universität Stuttgart, Stuttgart, 1998, 1 67. [4] Kubáček, L.: Underparametrization in a regression model with constraints II. Math. Slovaca 55 25, 579 596. [5] Kubáček, L.: Underparametrized regression models. Tatra Mt. Math. Publ. 14 26, 241 254. [6] Kubáček, L., Tesaříková, E.: Weakly Nonlinear Regression Models. Vyd. Univerzity Palackého, Olomouc, 28. [7] Rao, C. R., Mitra, S. K.: Generalized Inverse of Matrices and its Applications. Wiley, New York London Sydney Toronto, 1971. [8] Scheffé, H.: The Analysis of Variance. Wiley, New York London Sydney, 1967, fifth printing.