The Method of Reflection for Solving the Time-Optimal Hamilton-Jacobi-Bellman Equation on the Interval Using Wavelets

Transcription

1 The Method of eflection for Solving the Time-Optimal Hamilton-Jacobi-Bellman Equation on the Interval Using Wavelets S. Jain and P. Tsiotras Georgia Institute of Technology, Atlanta, GA 333-5, USA Abstract In this paper we use the antiderivatives of wavelets to efficiently represent functions which are defined over a bounded interval and which satisfy a boundary condition in the interior of this interval. The functions we want to approimate are typically non smooth at the origin. Such functions appear as solutions to Hamilton-Jacobi-Bellman (HJB) equations for time-optimal control problems. We first give the degree of approimation of the antiderivatives and then propose a Wavelet eflection Algorithm (WA) to solve numerically the time-optimal HJB equation on the interval. Several numerical eamples demonstrate the advantages of the technique developed in this paper over polynomial epansions. I. INTODUCTION Wavelets are basis functions (actually, frames) which allow efficient representations of (otherwise regular) functions with isolated singularities, owing to their nice localization properties both in space and frequency domains. Moreover, the wavelet representation of such functions is sparse, in the sense that most of the wavelet coefficients are very small or zero. This property of wavelets allows for compact functional approimations by ignoring the coefficients that are smaller than a prescribed threshold []. Wavelets have been used in the recent past for solving hyperbolic, elliptic and parabolic partial differential equations [], [3]. As a matter of fact, the advantages of wavelets for solving pdes have been noticed early on [], [4], [5]. Some of the most recent results in this contet have appeared in []. Glowinski et al, for eample, formulated a Galerkin-wavelet method for various boundary value problems []. They applied their method to the heat equation and to Burger s equation. However, their methodology may encounter some difficulties. First, Daubechies wavelets of low order cannot be used due to their lack of sufficient regularity. Second, Dirichlet boundary conditions cannot be applied directly without further modifications. elated results have appeared in [7]. In order to overcome the previous difficulties, Xu and Shann [4] constructed a set of basis functions using the antiderivatives of wavelets. They applied this set of bases to two-point boundary value problems and obtained numerical results of high consistency. The work reported in this paper is a continuation of our recent work Graduate Student. School of Aerospace Engineering, sachin jain@ae.gatech.edu. Professor, School of Aerospace Engineering, p.tsiotras@ae.gatech.edu. Corresponding author. on the use of wavelets for the solution of optimal control problems [8], [9], []. Standard numerical methods for solving pde s may not be used directly for solving the Hamilton-Jacobi-Bellman pde which arises in the solution of optimal feedback control problems. For instance, the solution (value function) to the time-optimal HJB equation may have discontinuous derivatives. In addition, the value function needs to satisfy a boundary condition in the interior of the domain. To address these two problems, in [] we constructed a frame for the Sobolev Space W, (Ω), (where Ω denotes an open interval) using the antiderivatives of wavelets. Working with the antiderivatives we automatically ensure that the interior boundary condition is satisfied, while at the same time we keep some of the nice properties of wavelets (e.g., a multiresolution decomposition of the solution space). In this paper, we provide the degree of approimation of the frame proposed in [] and propose yet another method for solving the time-optimal HJB equation. The paper is organized as follows. First, we offer a brief summary of the wavelets and their properties. We then introduce the antiderivatives of wavelets and investigate their multi-resolution and approimation properties in the Sobolev space W, (Ω). In the second part of the paper we apply this frame in order to find solutions to the timeoptimal HJB equation. The method of reflection is introduced to efficiently solve problems for which it is known that the solution is even. Finally, we offer several numerical eamples to demonstrate the proposed method. It is shown that the proposed approach offers major advantages in terms of the approimation error and compactness of the representation when compared to polynomial approimations. A. Nomenclature II. PELIMINAIES The following notation is used in this paper. ) If G, then by cl(g) we denote the closure of G in. ) Let Ω be an open set. For any nonnegative integer s, C s (Ω) is the vector space consisting of all functions f which, together with all their derivatives of order less than or equal to s, are continuous on Ω. 3) By + we denote the set + := : >}. 4) By y we mean Cy, where, C is some positive constant.

2 5) The inner product in L (Ω) will be denoted by f,g L (Ω) := Ω f()g()d. Definition (Sobolev Spaces): Let Ω be an open set. Then the Sobolev space W s, (Ω) is defined by W s, (Ω) := f L d α } f (Ω) : d α L (Ω), α s with norm f W s, (Ω) := s α= dα f d α L (Ω). The semi-norm of f W s, (Ω) is given by f W s, (Ω) := ds f d For simplicity, we denote the norm s W L (Ω). s, (Ω) by s,,ω and the semi-norm W s, (Ω) by s,,ω. Note that,,ω =,,Ω. In the following we assume that Ω is an open interval of with the origin in its interior. The space of interest in this paper is the Sobolev space W, (Ω) that is, the space of functions which are square integrable, having square integrable first derivative over Ω. In addition, we will deal with functions in W, (Ω) which are zero at the origin, that is, W, (Ω) := f W, (Ω) : f() = }. It can be readily shown that W, (Ω) is a subspace of W, (Ω). B. Wavelet Fundamentals In this section, we give a brief overview of the wavelets and their properties. For the details we refer the reader, for eample, to [], []. Wavelets are basis functions which induce a multiresolution decomposition of L (). This is the main property making wavelets attractive in applications. Specifically, wavelets induce the following nested sequence of subspaces j j+ L () such that j= j is dense in L (), that is, cl ( j= j) = L (). The base (or coarse-resolution) subspace is spanned by integer translates of the scaling function φ: = cl(spanφ( k)} k Z ). The higherresolution subspaces j are spanned by dilated versions of the scaling function: j =cl(spanφ( j k)} k Z ), j. The orthogonal complement of j in the larger subspace j+ is denoted by W j and it is spanned by the wavelets: W j = cl(spanψ( j k)} k Z ), j, where ψ is the mother wavelet, which spans the space W =. Let us define the two-parameter family of functions φ( k), for j =, ψ j,k () := j ψ( j k), for j, where k Z. The following fact is crucial for the approimating properties of wavelet decompositions. Theorem (anishing Moments []): The following are equivalent: ) The wavelet has m vanishing moments, i.e., l ψ()d =, l =,,,m () ) All polynomials of degree up to m can be epressed as a linear combination of shifted scaling functions at any scale. A wavelet is of order m if it has m vanishing moments. C. Wavelets as frames for L (Ω) Definition ([4]): Let ϕ n } n= be a subset of a Banach Space (B, B ). Let spanϕ n } be the set of all elements of the form α n ϕ n, (α n ) which converge (strongly) in B. Then ϕ n } n= is said to be a frame of B if spanϕ n } = B. oughly speaking, frames are redundant (linearly dependent) bases. In the following we use the notation Ω=(, +) and Ω + =(,). The proofs of the following results can be found in []. Theorem ([]): Let supp ψ, = supp ψ, =(,). Then the set ψ j,k Ω : j,k I j } where I j := k Z k, j = } ( j +) k j, j forms a frame for L (Ω). Corollary ([]): Let the space J (Ω) := spanψ j,k Ω : j<j, k I j }. () Then J (Ω) J+ (Ω) and J= J(Ω) is dense in L (Ω). Theorem 3 ([]): The semi-norm,,ω is equivalent to the norm,,ω in W, (Ω). Lemma ([]): Let ψ j,k Ω : j, k I j } be a frame for L (Ω). Then, for j and k I j, the set } Ψ j,k () := ψ j,k (s)ds, Ω W, (Ω) forms a frame for W, (Ω). In the net section, we give the degree of approimation when using the antiderivatives of wavelets to represent functions in W, (Ω). III. THE DEGEE OF APPOXIMATION Definition 3 ([3]): Let Ω be an open set. An etension operator T for W s, (Ω) is a bounded linear operator T : W s, (Ω) W s, () such that Tz Ω = z for every z W s, (Ω).

3 Lemma : Let ψ be a wavelet of order at least s. Forj and s<, let f W s, () and α j,k be given by α j,k = f,ψ j,k L (), where k Z. Then where, S j,k = supp ψ j,k. α j,k js f s,,sj,k Proof: Let f W s, () and α j,k = f,ψ j,k L ().For any polynomial q() of degree less than equal to s and a wavelet of order greater than or equal to s, Theorem implies that q()ψ j,k ()d = Therefore, we can write α j,k = fψ j,k d qψ j,k d = (f q)ψ j,k d S j,k or that α j,k = f q, ψ j,k L (Sj,k ). α j,k = f q, ψ j,k L (Sj,k ) f q,,sj,k ψ j,k,,sj,k = f q,,sj,k ψ j,k,, = f q,,sj,k where the last equality follows from the fact that ψ j,k,, =. Therefore, α j,k inf f q,,s q j,k, j, k Z. (3) Using the Bramble-Hilbert Lemma [4] we have f q,,sj,k S j,k s f s,,sj,k js f s,,sj,k since S j,k = / j. Furthermore, inf q f q,,s j,k f q,,sj,k js f s,,sj,k (4) Combining (3) and (4) one obtains α j,k js f s,,sj,k This concludes the proof of the lemma. Lemma 3: Let Ω = (, ) and let supp ψ j,k = [a, b] such that a< b or a <b. Then ψ j,k (),,Ω. Now since J= J(Ω) is dense in L (Ω) and each J (Ω) is finite dimensional, we construct finite-dimensional subspaces of W, (Ω) from the frame of J (Ω). Theorem 4: For J, let X J (Ω) := spanψ j,k : j<j, k I j }. Then X J (Ω) is a finite-dimensional subspace of W, (Ω) and for any v W, (Ω) W s+, (Ω) inf v Ϝ,,Ω Js v s+,,ω Ϝ X J (Ω) where, s<. Proof: The proof is rather long but it is similar to the one of Theorem 3. in [4], and thus it is omitted. I. APPLICATION TO TIME-OPTIMAL CONTOL POBLEMS In this section we provide numerical solutions to the one-dimensional time-optimal HJB equation using the antiderivatives of wavelets as the underlined epansion functions of the solution space. A. Problem Formulation Consider an optimal control problem whose dynamics are given by the nonlinear differential equation ẋ(t) =f[(t)] + g[(t)] u (5) with boundary conditions (t )= and (t f )=, where (t), f :, g :, and t f is free. The control is constrained by u. Notice that by writing g()u = g() sgn[g()] u := g() v where v we can further assume, without loss of generality, that g() for all. The cost function to be minimized is min u tf dt. () t The minimizing control for this problem is obtained from the solution of the following Hamilton-Jacobi-Bellman (HJB) equation ( ( ) ) + min f()+g()u u [,] =+f() () g() () =, (7) subject to the boundary condition () =. Once is known, the optimal control is given in a feedback form as follows (since g() for all ) u = sgn ( ). (8) In this paper we deal specifically with symmetric (i.e., even) solutions of (7). The following assumptions will ensure that is even. A: f is an odd function, i.e. f() = f( ),. A: g is an even function, i.e., g() =g( ),. B. The Wavelet eflection Algorithm (WA) In [5] it is shown that the half space W,p ( + ) has the following etension property. Theorem 5 ([5]): Let v W,p ( + ) ( p< ) and define v on by v() = v(), >, v( ),. (9)

4 The map v v defines an etension operator from W,p ( + ) onto W,p (). Corollary : Let Ω + =(,) and Ω=(, +). Let v W, (Ω + ) and define v on Ω as in (9). The map v v defines an etension operator from W, (Ω + ) onto W, (Ω). Let now sym W, (Ω + ) be the value function satisfying the HJB equation (7). From (8) we have that sym () >, Ω +. Therefore, the HJB equation (7) for Ω + can be written as +f() sym () g() sym () =, Ω +, with boundary condition sym () = or, equivalently, as +f( ) sym ( ) g( ) sym ( ) =, () with boundary condition sym () =. We now etend the value function sym from W, (Ω + ) to W, (Ω) such that sym () = Now sym W, (Ω) and sym () = sym (), Ω +, sym ( ), Ω \ Ω +. sym (), Ω +, sym ( ), Ω \ Ω +, or, equivalently, sym () = sym ( ), Ω. Therefore, for all Ω equation () can be written as +f( ) sym () g( ) sym () =, () with boundary condition sym () =. C. Solution to the HJB equation We have two cases to consider. Case : Assume that sym >. In this case equation () reduces to +f( ) sym () g( ) sym () =, Ω, () with boundary condition sym () =. Case : Assume that sym <. In this case equation () reduces to f( ) sym ()+g( ) sym () =, Ω, (3) with boundary condition sym () =. Let us now denote by sym the solution to equation () and by sym the solution to equation (3). For convenience, in the following we denote r () := f( ) g( ) and r () := f( )+g( ). Therefore, equations () and (3) can be re-written as HJB mod ( symi,r i ):=+ symi () r i() =, (4) with boundary condition symi () =, for i =,. We seek an approimate solution WAi to equation (4) using the method of weighted residuals []. To this end, we assume a solution WAi X J (Ω), (i =, ) of the form J WAi () = c i j,kψ j,k (), j= k I j i =, (5) and c i j,k are the associated coefficients. Substituting epression (5) into equation HJB mod ( symi,r i )=results in the error J Err i := HJB mod c i j,kψ j,k,r i, j= k I j i =,. () The coefficients c i j,k are determined by setting the projection of the error () on each element that spans the subspace J (Ω) (namely, ψ j,k } where j<jand k I j )to zero. Thus, Err i,ψ j,k L (Ω) :=, i =, (7) This approach will yield two solutions for sym, namely WA and WA, such that WA () > and WA () <, Ω respectively. The solution to the HJB equation (7) for all Ω is then given as follows WA (), Ω +, WA () = (8) WA (), Ω \ Ω +. D. Convergence We show that the solution ( WA ) obtained using the WA algorithm converges to (the unique viscosity solution [7], [8]) as J. Theorem : Let C (Ω) be the unique viscosity solution defined as (), Ω +, () := () = ( ), Ω \ Ω +. where, i W, (Ω) W s+, (Ω) (i =, ) for some s, and WA be the approimate solution (using WA) of the time-optimal HJB equation (7). Then WA,,Ω as J. Proof: The proof is a direct consequence of Theorem 4 in [] therefore we only give a sketch of the proof here. Let Ji be the projection of i on X J for i =,. Therefore

5 TABLE I NUMBE OF SC USING DAUBECHIES WAELETS FO EXAMPLE. E = WA,,Ω = E E = WA,,Ω = p J M w M SC δ E E d/ d.5.5 TABLE II NUMBE OF SC USING SYMLETS FO EXAMPLE p J M w M SC δ E E WA 3 3 (a) alue Function d/d d WA /d (b) Derivative of alue Function Fig.. Eample using Daubechies wavelets (p =,J =3,M SC = 4). using Theorem 4 we have that lim J i J i,,ω =. Then we prove that lim J Ji WAi,,Ω =. Now lim J i WAi,,Ω = lim J i Ji + Ji WAi,,Ω lim J i Ji,,Ω + lim J Ji WAi,,Ω = which completes the proof E = WA,,Ω =.4 WA 3 3 d/ d E E = WA,,Ω =.58 d/d d WA /d NUMEICAL EXAMPLES In this section we provide several eamples to demonstrate the benefits of the proposed method. In all eamples, the solution to the HJB equation is not differentiable at the origin. The problems are also simple enough so that the solution can be obtained analytically. This allows us to make meaningful comparisons on the accuracy of the solution obtained in each case. Eample : Consider system (5) with f() = a, g() =, and a>. The optimal value function is analytically obtained as () = a ln(a +). Using the WA algorithm (for a =) with the Daubechies wavelets of order p =, 4 (for scales J =3, 3 respectively) provides the results shown in Fig. and Fig. respectively. We define the significant coefficients (SC) as the coefficients that are required to calculate the cost function WA using the wavelets of order p at the minimum scale J so that the error E = WA,,Ω, where is the eact solution to the HJB equation. The number of significant coefficients used in the calculation of WA with Daubechies wavelets and symlets, and for different values of p, are given in Tables I and II respectively. In these tables M w,m SC denote the total number of wavelet coefficients for scale J and the number of SC respectively and δ is the threshold for selecting the SC. For reference, the last column also shows the error E = WA,,Ω. Eample : Consider system (5) with f() = 5 3 and g() = +5. The optimal value function is analytically obtained as () = +5. Using the WA algorithm with Daubechies wavelets of order p =, 4 (for scales J =5, 5 respectively) provides the results shown in Fig. 3 and Fig. 4 respectively. The number of significant coefficients used for calculating WA for p =, 4 using Daubechies wavelets and symlets are given (a) alue Function (b) Derivative of alue Function Fig.. Eample using Daubechies Wavelets (p =4,J =3,M SC = 34). in Tables III and I, respectively. I. COMPAISON WITH POLYNOMIAL EXPANSIONS We have also used polynomials to solve the previous eamples. The results for both the eamples using polynomials are shown in Fig. 5 and Fig.. The comparison between the wavelets and polynomials is summarized in Table which shows the superiority of the use of wavelets when compared to polynomial epansions. (In Table, M p are the number of polynomials that were used to solve the problem.) II. CONCLUSIONS In this paper we have proposed a numerical scheme for solving the time-optimal Hamilton-Jacobi-Bellman Equation on the interval. It is well known that the solution is often non-differentiable at the origin. Our approach uses antiderivatives of wavelets as trial functions in order to efficiently capture this behavior. Numerical eamples clearly TABLE III NUMBE OF SC USING DAUBECHIES WAELETS FO EXAMPLE p J M w M SC δ E E TABLE I NUMBE OF SC USING SYMLETS FO EXAMPLE p J M w M SC δ E E

6 3 3.5 E = WA,,Ω =.74 E E = WA,,Ω =.99 demonstrate the advantage of using wavelets in the proposed numerical algorithm as compared to the polynomials. d/ d 4 Acknowledgment: This work was supported by NSF award no. CMS (a) alue Function WA 4 d/d d WA /d 3 3 Fig. 3. E. using Daubechies wavelets (p =,J =5,M SC =)..5.5 E = WA,,Ω =.5835 WA 3 3 (a) alue Function d/ d 4 4 E E = WA,,Ω =.5874 d/d d WA /d 3 3 Fig. 4. E. using Daubechies wavelets (p =4,J =5,M SC =39) E = WA,,Ω =.5587 WA 3 3 (a) alue Function d/ d E E = WA,,Ω =.83 d/d d WA /d Fig. 5. Eample using polynomials (M p =4). E = WA,,Ω =.8 WA 3 3 d/ d 4 4 E E = WA,,Ω =.544 d/d d WA /d 3 3 EFEENCES [] S. Mallat, A Wavelet Tour of Signal Processing. NY: Academic Press, 999. []. Glowinski, W. M. Lawton, M. avachol, and P. Tanenbaum, Wavelet solutions of linear and nonlinear elliptic, parabolic and hyperbolic problems in one space dimension, in Computing Methods in Applied Sciences and Engineering, SIAM, Philadelphia, 99. [3] M. Holmstrom, Solving hyperbolic pdes using interpolating wavelets, Journal of Scientific Computation, vol., no., pp. 45 4, [4] J. C. Xu and W. C. Shann, Galerkin-wavelet methods for two-point boundary value problems, Numeriche Mathematic, vol. 3, pp. 3 44, 99. [5] G. Beylkin, On wavelet-based algorithms for solving differential equations, in Wavelets: Mathematics and Applications, J. Benedetto and M. Frazier, Eds. Boca aton, FL: CDC Press, 994, pp [] W. Dahmen, A. Kurdila, and P. Oswald, Multiscale Wavelet Methods for Partial Differential Equations. San Diego, CA: Academic Press, 997. [7] S. Bertoluzza, G. Naldi, and J. C. avel, Wavelet methods for the numerical solution of boundary value problems on the interval, in Wavelets: Theory, Algorithms and Applications. San Diego, CA: Academic Press, 994, pp [8] C. Park and P. Tsiotras, Sub-optimal feedback control using a successive galerkin-wavelet algorithm, in Proceedings of the American Control Conference, Denver, CO, 3, pp [9], Approimations to optimal feedback control using a successive wavelet collocation algorithm, in Proceedings of the American Control Conference, Denver, CO, 3, pp [] S. Jain and P. Tsiotras, A solution of the time-optimal Hamilton- Jacobi-Bellman equation on the interval using wavelets, in Proceedings of 43rd IEEE Conference on Decision and Control, Paradise Island, Bahamas, 4, pp [] I. Daubechies, Ten Lectures on Wavelets. PA: Society for Industrial and Applied Mathematics, 99. [] C. S. Burrus,. A. Gopinath, and H. Guo, Introduction to Wavelets and Wavelet Transforms. NJ: Prentice Hall, 998. [3]. A. Adams, Sobolev Spaces. NY: Academic Press, 975. [4] S. C. Brenner and L.. Scott, The Mathematical Theory of Finite Element Methods. NY: Springer-erlag, 994. [5] S. Kesavan, Topics in Functional Analysis. India: Wiley Eastern Limited, 989. [] B. A. Finlayson, The Method of Weighted esiduals and ariational Principles. NY: Academic Press, 97. [7] M. G. Crandall and P. L. Lions, iscosity solutions of Hamilton- Jacobi equations, in Transactions of the American Mathematical Society, vol. 77, no., 983, pp. 4. [8] P. Lions, Dynamic programming and the use of viscosity solution, in Proceedings of the 5th IEEE Conference on Decision and Control, Athens, Greece, 98, pp (a) alue Function Fig.. Eample using polynomials (M p =). TABLE WAELETS S. POLYNOMIALS E. M SC E E M p E E