PENALIZED LEAST SQUARES FITTING. Manfred von Golitschek and Larry L. Schumaker

Similar documents
S. S. Dragomir. 2, we have the inequality. b a

The Regulated and Riemann Integrals

Theoretical foundations of Gaussian quadrature

Abstract inner product spaces

Czechoslovak Mathematical Journal, 55 (130) (2005), , Abbotsford. 1. Introduction

S. S. Dragomir. 1. Introduction. In [1], Guessab and Schmeisser have proved among others, the following companion of Ostrowski s inequality:

Best Approximation in the 2-norm

g i fφdx dx = x i i=1 is a Hilbert space. We shall, henceforth, abuse notation and write g i f(x) = f

ODE: Existence and Uniqueness of a Solution

Math Advanced Calculus II

Some estimates on the Hermite-Hadamard inequality through quasi-convex functions

AN INEQUALITY OF OSTROWSKI TYPE AND ITS APPLICATIONS FOR SIMPSON S RULE AND SPECIAL MEANS. I. Fedotov and S. S. Dragomir

f(x)dx . Show that there 1, 0 < x 1 does not exist a differentiable function g : [ 1, 1] R such that g (x) = f(x) for all

New Expansion and Infinite Series

Best Approximation. Chapter The General Case

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS.

Advanced Calculus: MATH 410 Notes on Integrals and Integrability Professor David Levermore 17 October 2004

Lecture 1. Functional series. Pointwise and uniform convergence.

Lecture 6: Singular Integrals, Open Quadrature rules, and Gauss Quadrature

Journal of Inequalities in Pure and Applied Mathematics

1 The Lagrange interpolation formula

Physics 116C Solution of inhomogeneous ordinary differential equations using Green s functions

(4.1) D r v(t) ω(t, v(t))

Advanced Calculus: MATH 410 Uniform Convergence of Functions Professor David Levermore 11 December 2015

Main topics for the First Midterm

1. Gauss-Jacobi quadrature and Legendre polynomials. p(t)w(t)dt, p {p(x 0 ),...p(x n )} p(t)w(t)dt = w k p(x k ),

Regulated functions and the regulated integral

KRASNOSEL SKII TYPE FIXED POINT THEOREM FOR NONLINEAR EXPANSION

MATH 174A: PROBLEM SET 5. Suggested Solution

QUADRATURE is an old-fashioned word that refers to

II. Integration and Cauchy s Theorem

Inner-product spaces

Exam 2, Mathematics 4701, Section ETY6 6:05 pm 7:40 pm, March 31, 2016, IH-1105 Instructor: Attila Máté 1

8 Laplace s Method and Local Limit Theorems

Partial Derivatives. Limits. For a single variable function f (x), the limit lim

ASYMPTOTIC BEHAVIOR OF INTERMEDIATE POINTS IN CERTAIN MEAN VALUE THEOREMS. II

We partition C into n small arcs by forming a partition of [a, b] by picking s i as follows: a = s 0 < s 1 < < s n = b.

SOLUTIONS FOR ANALYSIS QUALIFYING EXAM, FALL (1 + µ(f n )) f(x) =. But we don t need the exact bound.) Set

Numerical Integration

1.9 C 2 inner variations

Math 426: Probability Final Exam Practice

1. On some properties of definite integrals. We prove

NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by.

Math 1B, lecture 4: Error bounds for numerical methods

Discrete Least-squares Approximations

Notes on length and conformal metrics

Frobenius numbers of generalized Fibonacci semigroups

ON A CONVEXITY PROPERTY. 1. Introduction Most general class of convex functions is defined by the inequality

A HELLY THEOREM FOR FUNCTIONS WITH VALUES IN METRIC SPACES. 1. Introduction

The Banach algebra of functions of bounded variation and the pointwise Helly selection theorem

Polynomial Approximations for the Natural Logarithm and Arctangent Functions. Math 230

WHEN IS A FUNCTION NOT FLAT? 1. Introduction. {e 1 0, x = 0. f(x) =

Orthogonal Polynomials

p-adic Egyptian Fractions

Lecture 19: Continuous Least Squares Approximation

Presentation Problems 5

Numerical Methods I Orthogonal Polynomials

Keywords : Generalized Ostrowski s inequality, generalized midpoint inequality, Taylor s formula.

Properties of the Riemann Integral

Chapter 3 Polynomials

Orthogonal Polynomials and Least-Squares Approximations to Functions

Review of Calculus, cont d

NEW INEQUALITIES OF SIMPSON S TYPE FOR s CONVEX FUNCTIONS WITH APPLICATIONS. := f (4) (x) <. The following inequality. 2 b a

A basic logarithmic inequality, and the logarithmic mean

Lecture 14: Quadrature

Convergence of Fourier Series and Fejer s Theorem. Lee Ricketson

1 The Riemann Integral

ON THE C-INTEGRAL BENEDETTO BONGIORNO

Euler, Ioachimescu and the trapezium rule. G.J.O. Jameson (Math. Gazette 96 (2012), )

MATH 185: COMPLEX ANALYSIS FALL 2009/10 PROBLEM SET 5 SOLUTIONS. cos t cos at dt + i

Problem Set 4: Solutions Math 201A: Fall 2016

p(x) = 3x 3 + x n 3 k=0 so the right hand side of the equality we have to show is obtained for r = b 0, s = b 1 and 2n 3 b k x k, q 2n 3 (x) =

UNIFORM CONVERGENCE MA 403: REAL ANALYSIS, INSTRUCTOR: B. V. LIMAYE

AN INTEGRAL INEQUALITY FOR CONVEX FUNCTIONS AND APPLICATIONS IN NUMERICAL INTEGRATION

Functional Analysis I Solutions to Exercises. James C. Robinson

W. We shall do so one by one, starting with I 1, and we shall do it greedily, trying

P 3 (x) = f(0) + f (0)x + f (0) 2. x 2 + f (0) . In the problem set, you are asked to show, in general, the n th order term is a n = f (n) (0)

ACM 105: Applied Real and Functional Analysis. Solutions to Homework # 2.

INEQUALITIES FOR TWO SPECIFIC CLASSES OF FUNCTIONS USING CHEBYSHEV FUNCTIONAL. Mohammad Masjed-Jamei

MAA 4212 Improper Integrals

Math 520 Final Exam Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

State space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies

The First Fundamental Theorem of Calculus. If f(x) is continuous on [a, b] and F (x) is any antiderivative. f(x) dx = F (b) F (a).

CLOSED EXPRESSIONS FOR COEFFICIENTS IN WEIGHTED NEWTON-COTES QUADRATURES

4.4 Areas, Integrals and Antiderivatives

CMDA 4604: Intermediate Topics in Mathematical Modeling Lecture 19: Interpolation and Quadrature

UNIFORM CONVERGENCE. Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3

Asymptotic behavior of intermediate points in certain mean value theorems. III

ODE: Existence and Uniqueness of a Solution

Euler-Maclaurin Summation Formula 1

Quadratic Forms. Quadratic Forms

2 Fundamentals of Functional Analysis

TRAPEZOIDAL TYPE INEQUALITIES FOR n TIME DIFFERENTIABLE FUNCTIONS

p n m q m s m. (p q) n

1 Linear Least Squares

Fourier series. Preliminary material on inner products. Suppose V is vector space over C and (, )

POSITIVE SOLUTIONS FOR SINGULAR THREE-POINT BOUNDARY-VALUE PROBLEMS

Definite integral. Mathematics FRDIS MENDELU

Section 6.1 INTRO to LAPLACE TRANSFORMS

MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1

Transcription:

Serdic Mth. J. 28 2002), 329-348 PENALIZED LEAST SQUARES FITTING Mnfred von Golitschek nd Lrry L. Schumker Communicted by P. P. Petrushev Dedicted to the memory of our collegue Vsil Popov Jnury 4, 942 My 3, 990 Abstrct. Bounds on the error of certin penlized lest squres dt fitting methods re derived. In ddition to generl results in firly bstrct setting, more detiled results re included for severl prticulrly interesting specil cses, including splines in both one nd severl vribles.. Introduction. We begin with n bstrct definition of wht we men by penlized lest squres fit. Suppose X, Y nd S re liner spces over IR, where S Y X. Let X : X IR nd Y : Y IR be semi-norms induced by semi-definite inner products, on X nd [, ] on Y, respectively. Given f X nd > 0, suppose there exists s f,s) in S such tht Φs f,s)) = min u S Φu), 2000 Mthemtics Subject Clssifiction: Primry 4A5; Secondry 4A25. Key words: Penlized lest squres, Tikhonov regulriztion, Spline pproximtion. * Supported by the Army Reserch Office under grnt DAAD-9-02-0059.

330 Mnfred von Golitschek nd Lrry L. Schumker where Φu) := f u 2 X + u 2 Y. Then we cll s f,s) penlized lest squres fit of f corresponding to. If there exists unique minimizer of Φ in S for ech f X, then Q : X S defined by Q f := s f,s) defines liner opertor which is not in generl liner projection. Our im in this pper is to investigte the behvior of f Q f X s function of both nd the pproximtion properties of S. The pper is orgnized s follows. In Section 2 we give detiled tretment of the penlized lest squres problem for pproximtion by trigonometric polynomils. The next section is devoted to Tikhonov regulriztion with weights) for univrite functions defined on n intervl. The results include error bounds for certin clsses of functions. In Section 4 we derive n explicit formul for the Tikhonov regulriztion of rbitrry functions in C[,π]. Our min L 2 error bounds for penlized lest squres fits of univrite functions re contined in Section 5. Generl penlized lest squres is treted in Section 6. We conclude the pper by outlining two typicl pplictions: univrite splines nd bivrite splines on tringultions. 2. Penlized Fourier series. Throughout this section, we tke X = π L 2 [,π] with the inner product f,g = fg, nd set Y = W2 2 [T] with the semi-definite inner product [f, g] = π f g, where T denotes the circle. We let S = T n be the set of ll rel-vlued trigonometric polynomils of degree t most n, with n IN fixed. Clerly, for ech f L 2 [,π] nd > 0, there exists unique miniml solution s,n := s,n f) of the penlized lest squres problem { π min f T) 2 dx + T T n π } T ) 2 dx. Our first theorem compres s,n with the n-th prtil sum s n f) := s 0,n f) of the Fourier series of f. Theorem 2.. Suppose f L 2 [,π], nd tht fx) = 0 + ) k cos kx) + b k sin kx)

Penlized Lest Squres Fitting 33 is its Fourier series. Then for ech > 0 nd n IN, s,n = s f, T n ) is given by n ) 2.) s,n x) = 0 + + k 4 k cos kx) + b k sin kx), or equivlently, 2.2) s n f) = s,n + s 4),n. Proof. The miniml solution s,n is chrcterized by the orthogonlity reltions π π 2.3) f s,n )T = s,n T, for ll T T n. A simple computtion shows tht the function s,n given by 2.) is the only function in T n which stisfies 2.3). This implies tht s,n is the unique miniml solution. The proof of 2.2) is eqully simple. Tking the limit s n in Theorem 2., we get Corollry 2.2. Let f be s in Theorem 2.. Then for ech > 0, the function s, x) = 0 + ) + k 4 k cos kx) + b k sin kx), x IR, lies in W 4 2 T), nd f = s, + s 4),. Moreover, s, is the unique solution of the minimum problem { π min f g) 2 dx + π g ) 2 dx, g W 2 2 T) }. Let 2 nd denote the L 2 nd uniform norms on [,π], respectively. Since f s n f) 2 f T 2 for ll T T n, we hve f s,n 2 f s n f) 2. In the next theorem we will see tht the L 2 -norm f s,n f) 2 chnges very little for incresing n > /4. Hence, for numericl purposes, it is not worthwhile to compute s,n f) for n lrger thn /4.

332 Mnfred von Golitschek nd Lrry L. Schumker Theorem 2.3. Let f L 2 [,π] nd 0 <. Let N = /4 be the lrgest integer smller thn or equl to /4. Then 2.4) ε,n f s,n f) 2 ε,n, 2 where ε,n := ε,n f) is defined by { ε,n := f sn f) 2 2 + 2 s 4) n f) 2 2, if n N, ε,n, if n > N. P r o o f. We pply the orthogonlity reltion π f s n f))t = 0, T T n, π for T = s 4),n = s4),n f). This gives f s n f))s 4),n = 0, nd 2.2) implies 2.5) f s,n 2 2 = f s nf) + s 4),n 2 2 = f s nf) 2 2 + 2 s 4),n 2 2. It follows from 2.) tht s 4),n 2 2 = n If n N, this implies If n > N, then k 4 ) 2 n + k 4 k cos kx) + b k sin kx) = π 2 s 4),n 2 2 π s 4),n 2 2 π 4 n n s 4),n 2 2 = s 4),N 2 2 + π s 4),N 2 2 + π k 8 2 k + b2 k ) = s4) n 2 2 k 8 2 k + b2 k ) = 4 s4) n 2 2. n k=n + n k=n + k 8 2 k + b2 k ) + k 4 ) 2 2 k + b2 k 2 = s 4),N 2 2 + 2 f s N 2 2 f s n 2 2 k 8 2 k + b2 k ) + k 4 ) 2. ).

Penlized Lest Squres Fitting 333 Moreover, if n > N, then s 4),n 2 2 = s 4),N 2 2 + π s 4),N 2 2 + π n k=n + n k=n + k 8 2 k + b2 k ) + k 4 ) 2 2 k + b2 k 4 2 = s 4),N 2 2 + ) 4 2 f s N 2 2 f s n 2 2. Inserting these estimtes into 2.5) yields 2.4). 3. Tikhonov regulriztion on [, b] with weights. Suppose w : [,b] IR is piecewise continuous nonnegtive function on [,b] with In this section we tke X = L 2,b) with the inner product f,g = set Y = S = W2 2 b [,b] with the semi-definite inner product [f,g] = w > 0. wfg, nd f g. Definition 3.. Let f L 2,b) nd > 0. We cll φ,w = φ f,w) W2 2 [,b] the nonperiodic) Tikhonov regulriztion of f corresponding to nd w provided it minimizes 3.) with respect to ll u W 2 2 [,b]. ) 2 wx) f u dx + u ) 2 dx For the weight w =, Rgozin [5] proved tht the Tikhonov regulriztion φ stisfies f q) φ q) 2 C q 2 q)/4 f 2, q = 0,,2, for some bsolute constnts C q > 0. The next theorem sttes, mong other things, tht C 0 nd C 2 2 for the weight w =. Theorem 3.2. Let f C[,b] nd > 0. Let φ := φ f,w) be the Tikhonov regulriztion of f corresponding to,w. Then φ C 3 [,b], φ 4) is piecewise continuous on [, b], nd

334 Mnfred von Golitschek nd Lrry L. Schumker ) wx)[fx) φ x)] = φ 4) x), for ll x [,b] where w is continuous, 2) φ ) ) = φ ) b) = φ 3) If f W2 2[,b], then φ ) 2 f 2 nd ) = φ3) b) = 0. wf φ ) 2) /2 /2 f 2. 3.2) Proof. Let f C[,b]. The orthogonlity reltions for 3.) re wf φ )u We define the function F by Fx) := x t φ ) u = 0, for ll u W 2 2 [,b]. ) ws) fs) φ s) ds dt, x b. Then, F) = F ) = 0. Applying 3.2) for u = nd ux) = x yields wf φ )dx = fx) φ x))xdx = 0, nd thus F b) = 0 nd Fb) = 0. Therefore, integrting by prts gives Hence 3.2) is equivlent to nd thus This implies tht Differentiting twice yields wf φ )u = Fu. F φ ) )u = 0, for ll u W 2 2 [,b], F φ ) )g = 0, for ll g L 2 [,b]. Fx) = φ ) x), x b. wx)fx) φ x)) = φ 4) x)

Penlized Lest Squres Fitting 335 t ll points x [,b] where w is continuous. Then F = φ ) nd F) = Fb) = 0 imply φ ) ) = φ ) b) = 0, while F = φ 3) nd F ) = F b) = 0 imply φ 3) ) = φ3) b) = 0. Let f W2 2 [,b]. We denote the miniml vlue in 3.) by M. Then 3.) nd 3.2) imply tht M = wf φ )f u) + Tking u = f, it follows tht M = φ ) u, for ll u W 2 2 [,b]. φ ) f φ ) 2 f 2. Therefore, nd thus wf φ ) 2 + φ ) 2 2 φ ) 2 f 2, φ ) 2 f 2, wf φ ) 2 /2 f 2. This concludes the proof. Theorem 3.3. Let w =. Let f W 4 2 [,b] nd f ) = f b) = f 3) ) = f 3) b) = 0. Then the Tikhonov regulriztion φ of f corresponding to > 0 stisfies f φ 2 f 4) 2, f φ ) 2 /2 f 4) 2, φ 4) 2 f 4) 2. Proof. We write U := {u W 4 2 [,b] : u ) = u b) = u 3) ) = u 3) b) = 0}. Clerly, f U by ssumption nd φ U by Theorem 3.2. For ech u U we obtin by integrtion by prts tht f φ )u 4) = f φ ) )u.

336 Mnfred von Golitschek nd Lrry L. Schumker Inserting the identity f φ = φ 4), it follows tht 3.3) f φ ) )u φ 4) u4) = 0, for ll u U. But 3.3) re the orthogonlity reltions for the minimum problem 3.4) M := min u U f u 2 2 + u 4) 2 2, nd so φ is the miniml solution of 3.4). Since f U, it follows tht M f 4) 2 2, nd thus Finlly, we hve φ 4) 2 f 4) 2, f φ ) 2 /2 f 4) 2. f φ 2 = φ 4) 2 f 4) 2, which concludes the proof of the theorem. 4. An explicit formul for the Tikhonov regulriztion In this section we consider the inner product spces X = L 2,b) nd Y = S = [,b] of Section 3 with the weight function w = on [,b], tht is, f,g = W2 2 fg, [f,g] = f g. In Theorem 2. we hve given n explicit formul for the periodic Tikhonov regulriztion. In this section we wnt to pply the methods of Section 2 to the nonperiodic cse. We my ssume for convenience tht the intervl is [,b] = [,π]. Otherwise, under the liner trnsformtion xt) = + b )t + π), t π, 2π of the intervl [,π] onto [,b], the Tikhonov regulriztions φ x) on [,b] of f nd ψ µ t) on [,π] of gt) := fxt)) re relted by 4.) ψ µ t) = φ xt)), = b ) 4µ. 2π To prove 4.), we simply compre the orthogonlity reltions 4.2) fx) φ x))ux)dx = φ ) x)u x)dx, u W 2 2 [,b],

Penlized Lest Squres Fitting 337 nd 4.3) π π gt) ψ µ t))vt)dt = µ ψ µ t)v t)dt, v W2 2 [,π], where the equivlence of 4.2) nd 4.3) follows from the bijection vt) = uxt)) of W 2 2 [,b] onto W 2 2 [,π]. The min ide of this section is contined in the next lemm. Lemm 4. Ech function v C [,π] hs unique representtion of the form 4.4) vt) = c t + c 2 t 2 + wt), c IR, c 2 IR, w C T). In other words, 4.4) defines bijection from IR 2 C T) onto C [,π]. Proof. Given c,c 2 IR nd w C T), the function v in 4.4) lies in C [,π]. Conversely, for function v C [,π], the condition w) = wπ) is equivlent to nd thus to v) + c π c 2 π 2 = vπ) c π c 2 π 2, c = vπ) v). 2π The condition w ) = w π) is equivlent to v )+2c 2 π = v π) 2c 2 π, nd thus to c 2 = v π) v ). 4π This concludes the proof of the lemm. As corollry, ech function f C [,π] hs unique representtion 4.5) ft) = c t + c 2 t 2 + 0 + k cos kt) + b k sin kt)), where c = fπ) f) 2π, c 2 = f π) f ). 4π

338 Mnfred von Golitschek nd Lrry L. Schumker Theorem 4.2. Let f C[,π] be given by 4.5). If f C [,π], then we tke c 2 = 0. Then the nonperiodic Tikhonov regulriztion φ of f corresponding to is given by φ t) = γ t + γ 2 t 2 + α 0 + α k cos kt) + β k sin kt)) with 4.6) 4.7) 4.8) 4.9) 4.0) where γ = c ) k k 3 b k 2B) + k 4, ) ) k k 2 k γ 2 = 4A)c 2 + 2 + 4A) + k 4, α 0 = 0 + c 2 γ 2 )π 2, 3 α k = + k 4 β k = + k 4 A) := k + 4 )k c 2 γ 2 ) k 2 b k + 2 )k c γ ) k ), k, ), k, + k 4, B) := k 2 + k 4. Proof. The Fourier series of t nd t 2 on,π) re 2 ) k sin kt) t =, < t < π, k t 2 = π2 3 + 4 ) k cos kt) k 2, t π, with pointwise convergence. From Theorem 3.2, we know tht f = φ + φ 4) Compring the Fourier coefficients of f nd φ + φ 4) ft) = c + 0 + 2 ) k sinkt) k + c 2 π 2 3 + k cos kt) + b k sin kt)) in the expnsions 4 ) k cos kt) ) k 2.

Penlized Lest Squres Fitting 339 nd φ t) + φ 4) t) = γ we obtin the equlities +α 0 + 2 ) k sin kt) k + γ 2 π 2 3 + + k 4 )α k cos kt) + β k sin kt)), 4 ) k cos kt) ) k 2 c 2 π 2 + 0 = γ 2π 2 + α 0 3 3 4c 2 ) k k 2 + k = 4γ 2 ) k k 2 + + k 4 )α k, k, 2c ) k + b k = 2γ ) k + + k 4 )β k, k k k, which re equivlent to 4.8) 4.0). From Theorem 3.2 we know tht φ ) π) = 0 nd φ 3) π) = 0, nd thus, φ ) π) = 2γ 2 ) k k 2 α k = 0, 4.) φ 3) Inserting 4.8) 4.0) into 4.) yields nd 0 = 2γ 2 = π) = ) k k 3 β k = 0. ) k k 3 β k = ) k k 2 k + k 4 + 4c 2 γ 2 ) These two conditions re equivlent to 4.2) nd 4.3) 2γ 2 = ) k k 3 b k + k 4 2c γ ) + k 4 ) k k 2 k + k 4 + 4c 2 γ 2 )A) 2c γ )B) = ) k k 3 b k + k 4. k 2 + k 4.

340 Mnfred von Golitschek nd Lrry L. Schumker Now 4.6) follows from 4.2), nd 4.7) follows from 4.3). 5. L 2 -error bounds for penlized lest squres fitting. In this section we consider the inner product spces X = L 2,b) nd Y = W2 2 [,b] of Section 4 with the sme inner products f, g = fg nd [f,g] = f g, but we now tke S to be proper subspce of Y. For f L 2 [,b] nd > 0, we compre the penlized lest squres fit φ Y of Section 3 with weight function w = ) stisfying { } f φ ) 2 + φ )2 = min f u) 2 + u ) 2 u Y with the penlized lest squres fit s = Q f) from S stisfying { } f s ) 2 + s )2 = min f u) 2 + u ) 2. u S Theorem 5.. Let f W2 2[,b] nd > 0. Then s minimizes the expression 5.) φ u 2 2 + φ ) u 2 2 with respect to ll u S. Proof. s is chrcterized by the orthogonlity reltions 5.2) f s )u s ) u = 0, for ll u S. Since f = φ + φ 4) φ s )u + by Theorem 3.2, 5.2) is equivlent to φ 4) u s ) u ) = 0, for ll u S, nd since φ ) = φ b) = φ3) ) = φ3) b) = 0 by Theorem 3.2, integrtion by prts shows tht 5.2) is equivlent to 5.3) φ s )u + φ s ) )u = 0, for ll u S. But 5.3) re the orthogonlity reltions for the minimum problem 5.).

Penlized Lest Squres Fitting 34 Corollry 5.2. Let S W2 2 [,b] be liner spce. Suppose tht for ech g W2 2 [,b] there exists u S such tht 5.4) g u 2 Ch 2 g 2, g u 2 C g 2, for positive numbers C nd h independent of g. Then the penlized lest squres fit s of f in S corresponding to stisfies f s 2 /2 + C ) 5.5) h 4 + f 2 nd 5.6) s ) 2 + C ) h 4 + f 2. Proof. By Theorem 3.2, the Tikhonov regulriztion φ of f stisfies f φ 2 /2 f 2. Let M 0 be the minimum in 5.). Applying 5.4) for g := φ, it follows tht M 0 C 2 ) h 4 + φ ) 2 2. Then, since φ ) 2 f 2 by Theorem 3.2, f s 2 f φ 2 + φ s 2 /2 f 2 + M /2 0 /2 + C ) h 4 + f 2, which yields 5.5). Since φ ) 2 f 2 by Theorem 3.2, while 5.7) φ ) s ) 2 2 M 0, it follows tht s ) 2 φ ) 2 + /2 M /2 0 [ ] /2 ) + C h 4 + f 2, which proves 5.6).

342 Mnfred von Golitschek nd Lrry L. Schumker Corollry 5.3. Let S W2 2 [,b] be liner spce. Suppose tht for ech g W2 4 [,b] there exists u S such tht 5.8) g u 2 Ch 4 g 4) 2, g u 2 Ch 2 g 4) 2, for positive numbers C nd h independent of g. Let f W2 4[,b] nd f ) = f b) = f 3) ) = f 3) b) = 0. Then the penlized lest squres fit s of f in S corresponding to stisfies ) 5.9) f s 2 + C[ + h 4 ] f 4) 2, 5.0) f s ) 2 /2 + Ch 2 [h 4 + ] /2) f 4) 2, Proof. By Theorem 3.3, the Tikhonov regulriztion φ of f stisfies f φ 2 f 4) 2. Let M 0 be the minimum in 5.). Applying 5.8) for g := φ, it follows tht M 0 C 2 h 8 + h 4) φ 4) 2 2. Then, since φ 4) 2 f 4) 2 by Theorem 3.3, f s 2 f φ 2 + φ s 2 f 4) 2 + M /2 0 + Ch 2 ) h 4 + f 4) 2, which implies 5.9). Since f φ ) 2 /2 f 4) 2 by Theorem 3.3, using 5.7) gives f s ) 2 /2 f 4) 2 + /2 M /2 0, nd 5.0) follows. 6. A generl penlized lest squres problem. Keeping in mind the problems nd results of Sections 2 5, we now investigte the more generl

Penlized Lest Squres Fitting 343 setting described in the introduction. Let X, Y nd S be liner spces on IR, where S Y X; X : X IR nd Y : Y IR re semi-norms induced by semi-definite inner products, on X nd [, ] on Y, respectively. For simplicity, we ssume throughout the rest of this pper tht S hs dimension n nd tht the restriction X onto S is norm on S. This implies tht for ny f X nd > 0, there exists unique s := s f,s) := Q f in S with the property { } f s 2 X + s 2 Y = inf f u 2 X + u S u 2 Y. We denote the nonpenlized lest squres fit in S by s 0, i.e., f s 0 2 X = min u S f u 2 X. The penlized lest squres pproximtion s of f is chrcterized by the orthogonlity reltions 6.) f s,u = [s,u], for ll u S, while s 0 is chrcterized by 6.2) f s 0,u = 0, for ll u S. By 6.) 6.2), we hve 6.3) s 0 s,u = [s,u], for ll u S, which proves tht s = Q s 0, tht is s 0 s 2 X + s 2 Y = min u S { } s 0 u 2 X + u 2 Y. In wht follows, we will discuss the error function s 0 s which indictes the consequences of the penlty, nd is esier to nlyse thn the error function f s itself. This suffices since f s 0 X f s X f s 0 X + s 0 s X. In nlogy with Theorem 3.2, we hve

344 Mnfred von Golitschek nd Lrry L. Schumker Theorem 6.. For ech f X nd > 0, s := s f,s) stisfies 6.4) nd 6.5) Proof. Becuse of 6.3), s Y s 0 Y s 0 s X s 0 Y. Ms 0,) := s 0 s 2 X + s 2 Y = s 0 s,s 0 u + [s,u], for ll u S. Inserting u = s 0 yields nd inequlities 6.4) nd 6.5) follow. tht Ms 0,) = [s,s 0 ] s Y s 0 Y, Our next result provides n improvement of 6.5) for smll which implies s 0 s X = O) s 0. Theorem 6.2. For ech f X nd > 0, s := s f,s) stisfies 6.6) where s 0 s X K S s 0 Y. { } u Y K S := sup : u S, u 0. u X Proof. Coupling the orthogonlity reltion 6.3) for u = s 0 s with the Cuchy-Schwrz inequlity nd 6.4), we hve which implies 6.6). s 0 s 2 X = s 0 s,s 0 s = [s,s 0 s ] s Y s 0 s Y s 0 Y K S s 0 s X, We conclude this section by exmining the specil cse where X, Y, S re function spces on some set Ω IR d, d IN. Our next result gives bound on s 0 s L Ω) which implies tht s 0 s L Ω) = O) s 0.

Penlized Lest Squres Fitting 345 Theorem 6.3. Suppose X L Ω), nd let { } u L Ω) κ S := sup : u S, u 0 <. u X Then } s 0 s L Ω) κ S s0 Y min {,K S. Proof. The ssertion follows directly from 6.5), Theorem 6.2, nd the definition of κ S. 7. Applictions. 7.. Cubic splines on n intervl. Let X = C[,b], Y = W2 2[,b], nd suppose S := S 4 ) C 2 [,b] is the liner spce of cubic splines with simple knots : = x 0 < x < < x n = b. Suppose tht D := {t i } m i= re distinct points in [,b] so tht the restriction of S 4 ) onto D hs full dimension n + 3. We cn now pply the results of Section 6 for the inner products f,g = m m ft i )gt i ), i= f,g C[,b] nd [f,g] = In the nottion of Sect. 6, f g, f,g W 2 2 [,b]. ) /2 m u X = ut i ) 2, u S. m i= We write 2 for the L 2 -norm on [,b] nd suppose tht D nd re such tht { } u 2 δ := sup : u S, u 0 u X is finite. Let h 0 := h 0 ) := min{x j+ x j : j = 0,...,n }. Since u 2 δ u X, u S, it follows tht for some bsolute constnt C 0, u Y = u 2 C 0 h 2 0 u 2 C 0 δ h 2 0 u X,

346 Mnfred von Golitschek nd Lrry L. Schumker nd thus K S C 0 δ h 2 0. Moreover, for some other bsolute constnt C, κ S C δ h /2 0. For given rel numbers {y i } m i=, let s 0 S be such tht m m i= 2 y i s 0 t i )) = min u S m m i= y i ut i )) 2. Recll tht the penlized lest squres spline s minimizes the expression m m y i ut i )) 2 + i= u t)) 2 dt. The following result follows immeditely from Theorems 6. 6.3. Moreover, Theorem 7.. For ll > 0, s 0 s 2 δ min{,k S } s 0 2. s 0 s L [,b] C δ h /2 0 min {,C0 δ h 2 0 } s 0 2. 7.2. Bivrite spline spces with stble locl bses. Let Ω be bounded set in IR 2 with polygonl boundry, nd suppose tht X L Ω), Y = W2 2 Ω). Let S X be liner spce of polynomil splines defined on regulr tringultion of Ω. For results on the pproximtion properties of these types of spces, see [, 2, 4]. Suppose tht D = {t i } m i= re distinct points in Ω so tht the restriction of S onto D hs the sme dimension s S. We pply the results of Section 6 for the inner products f,g = m m ft i )gt i ), f,g X, i=

Penlized Lest Squres Fitting 347 nd [f,g] := Ω f xx g xx + 2f xy g xy + f yy g yy ) dxdy, f,g Y. For given rel numbers z i ) m i=, s 0 S is the unique function in S which stisfies m m i= In the nottion of Sect. 6, 2 z i s 0 t i )) = min u S m m i= z i ut i )) 2. ) /2 m u X = ut i ) 2, u S. m i= We hve investigted properties of s 0 S in [3]. Let 2 denote the L 2 -norm on Ω, nd suppose tht D nd re such tht { } u 2 δ := sup : u S, u 0 u X is finite. Let h 0 := h 0 ) be the minimum side length of the tringles in. Since u 2 δ u X, u S, it follows tht for some bsolute constnt C 0 so tht u Y C 0 h 2 0 u 2 C 0 δ h 2 0 u X K S C 0 δ h 2 0. Moreover, for some other bsolute constnt C, κ S C δ h 0. As for univrite splines, we cn now pply Theorems 6. 6.3 to get bounds on s 0 s in both the L 2 nd uniform norms on Ω. REFERENCES [] C. K. Chui, D. Hong, R.-Q. Ji. Stbility of optiml order pproximtion by bivrite splines over rbitrry tringultions. Trns. Amer. Mth. Soc. 347 995), 330 338.

348 Mnfred von Golitschek nd Lrry L. Schumker [2] O. Dvydov, L. L. Schumker. On stble locl bses for bivrite polynomil spline spces. Constr. Approx. 8 200), 87 6. [3] Mnfred v. Golitschek, Lrry L. Schumker. Bounds on projections onto bivrite polynomil spline spces with stble locl bsis. Constr. Approx. 8 2002), 24 254. [4] M. J. Li, L. L. Schumker. On the pproximtion power of bivrite splines. Advnces in Comp. Mth. 9 998), 25 279. [5] Dvid L. Rgozin. Error bounds for derivtive estimtes bsed on spline smoothing for exct or noisy dt. J. Approx. Theory 37 983) 335 355. [6] John Rice, Murry Rosenbltt. Integrted men squred error of smoothing spline; J. Approx. Theory 33 98), 353 369. Mnfred von Golitschek Institut für Angewndte Mthemtik und Sttistik der Universität Würzburg, 97074 Würzburg, Germny e-mil: goli@mthemtik.uni-wuerzburg.de Lrry L. Schumker Deprtment of Mthemtics Vnderbilt University, Nshville TN 37240, USA e-mil: s@mrs.cs.vnderbilt.edu Received September 9, 2002