RD-R14i 390 A LIMIT THEOREM ON CHARACTERISTIC FUNCTIONS VIA RN ini EXTREMAL PRINCIPLE(U) TEXAS UNIV AT AUSTIN CENTER FOR CYBERNETIC STUDIES A BEN-TAL DEC 83 CCS-RR-477 UNCLASSIFIED N@884-i-C-236 F/0 2/1 NL
-% r 6 ',.-. INI.---.-*.33 lii -- n2=5 MICROCOPY RESOLUTION TEST CHART NATIONAL BUREAU OF STANDARDS- I963-A IIIi
p - C -%- - 0 %, / %U IVIA Research Report CCS 477 A LIMIT THEOREM ON CHARACTERISTIC FUNCTIONS AN EXTREMAL PRINCIPLE by A. Ben-Tal* CENTER FOR CYBERNETIC STUDIES The University of Texas Austin,Texas 78712 MAY 2 1984 " Tais document hus been npptovcd fox public relezwe ond.zoa. tt di.itribution is unlimited. 84 05 2I 14' i* ',.;,N 'V ",,,,.",:,,',k,:;';':,.,.,.,..--.,-.-..v. -. --,,..... -, ' I " i..- * "' -' " " " ' - " " "" ' ' " " % %
........ -... 57 477 777- K " Research Report CCS 477 A LIMIT THEOREM ON CHARACTERISTIC FUNCTIONS. VIA AN EXTREMAL PRINCIPLE by A. Ben-Tal* I DTIC ~ LECTE December 1983 MAY 2 4 1984 f A *Faculty of Industrial Engineering & Management, Technion-Israel Institute of Technology, Haifa, Israel This paper is supported partly by ONR Contract N00014-81-C-0236 with the Center for Cybernetic Studies, The University of Texas at Austin and NSF Grant No. ECS-8214081 and Technion VPR Fund. Reproduction in whole or in part is permitted for any purpose of the United States Government. CENTER FOR CYBERNETIC STUDIES A. Charnes, Director Graduate School of Business 4.138 The University of Texas at Austin Austin, TexaLs 78712 (512) 471-1821 to" '".". \& ib'
--.. "I Abstract entropy functional.. / -We-provef/ classical limit theorem on characteristic functions by using duality between a pair of optimization problems, one of which is an infinite dimensional minimization involving the relative KEY WORDS: Characteristic Functions, Optimization in infinite dimensional spaces, Duality, Relative Entropy. 42-. 'Tj - -- it.. N, --, + @.,... V %
Introduction Modern optimization theory has been employed successfully in many diverse fields such as Economics, Physics, Statistics, Biology and Engineering. This paper is a small step toward demonstrating the use of optimization theory as proof mechanism in Probability. The result from Probability Theory in question here is a limit theorem on characteristic functions. Let X be a random variable with distribution F, support Ix,X ] and characteristic function x L R itx *(t) Then xr - lim 1 log *(-iy) xl - -lim 1 log *(iy) L 0 y The result is given in Lucas' classical book "Characteristic Functions" [3]; first a weaker result, concerning only analytic characteristic functions, is proved in Chapter 7. The full statement is given in Chapter 11, as part of Theorem 11.1.2. It is derived from the result in Ch. 7 via a chain of lemmas on boundary characteristic functions. Here we prove the above limit theorem by using duality relations between two extremum problems. One of these problems is an infinitedimensional convex program involving the minimization of the relative entropy functional, which is of dundamental importance in Statistical Information Theory, Thermodynamics and Communication Theory. The plan of the paper is as follows: Section 1 gives a formal '*. statement of the limit theorem (Theorem A). Section 2 gives the duality theorem (Theorem B), which is in fact an adaptation of a result in the authors' paper 11]. In Section 3 we prove Theorem A via Theorem B. '1d
-2-1. A limit theorem on characteristic functions Let X be a random variable, and F xw) its distribution function. X Let *(t) denote the characteristic function of FX, i.e. i (t) = I e t X df (x) The ieft extremity of Fx is the number xl with the property YV > 0- Fx (X L-E) = 0, F X(XL+) > 0 and the right extremity of F is the number XR with the property cv > 0 : F x ( x R - ) < 1 F x (x R ) = 1 c.'-.x The interval [xlxr] is the support of Fx, Clearly L jx R i t x *(t) = f e- df x x) XLX F is bounded to the left if x >-- and bounded to the right if XL XR< Theorem A. If F x be bounded to the right, then its right extremity is given by ( m - log ij(-iy) y14 Y If F is bounded from the left, then its left extremity is given by 1 (12) x L -- lim 1 log (y), (2) XL y 2. A duality theorem on relative entropies Let D be the class of generalized densities f = (Radon-NikOdym dt 4' derivatives) of distribution functions F, on a given probability df s space, with support ExLIR In particular Fx E D and L'xR XX f-- dt 4.A... %.%~ :-:'q~. KvV
.,a -3- is the corresponding density. The relative entropy (divergence,.,,' Kullback-Leibler distance) between F E D and F is given by the quantity.1 (lf f x X xr I fltllogl f(t) f ]dt. XL X It is well known that I(";fx) is a nonnegative convex functional and is equal to zero if and only if f - fx (a.e. with respect to dt) see [2]. A special case of a problem studied in [l,ch.3] is the infinite-dimensional convex program x (E) inf{(f,f x ) : I g(t)f(t)dt > a) fed ar X ' / where g(t) is a. given sumable function. It was shown in [1] that a dual problem is given by xry9(t) 1(H) sup~ay-log I e f x(t).dt) yo X K Moreover, from Th. 1 in I1] the following duality relations hold - I between (E) and (H). Theorem B. If (E) is feasible then inf (E) is attained, sup(h) is finite and min (E) - sup (H) 3. Proof of Theorem A via Theorem B First note that the trivial inequality Vy > 0 Zyx < *Yx R
'; - ' -..-' - -. -. - -. - " -.. *,..,.44 implies lir 1 log E e -XR y4 i.e. (3) lim 1_ log h(-iy) < *IVMYy- x R Consider now the problem -~(E ) fed inffl(fif): f xltf(t)dt > x R XR for some fixed c > 0. This is a special case of problem (E) with g(t) t a - xr - C. The dual is X R yt sup(y(xr-c) - log R e f xt)dt} i.e. (D ) sup{y(xr-c) - log *(-iy)} y;>o Problem (E) is clearly feasible for every c > 0, and we infer from Theorem B: s~p(d ) > lim(y(x.-) - log *(-iy)} - lim yixr7-11g i (-iy)] y*4 Vow,.for the limit to be finite, it is necessary that: (4) x R - C < lim 1Y log *(-iy) 0. - y.wq Combining (3) and (4) we obtain equation (1).
*......- -, - -.. " o.-.,.,o.55 ~-5-I To prove equation (2) we note that the inequalit r,-.. is trivial, while the inequality (6) -lim 1 log 4i(iy) x + E V E > 0 y-*a. Y L follows by applying Theorem B (in the above manner) to the dual pair: J%" inf{i(f,fx) X R x J tf(t)dt >xl - c) 4L sup{-y(xl+e) - y>o log *(iy)} Now, (5) and (6) indeed imply (2), and the proof of Theorem A is thereby completed. References [I] Ben-Tal, A., "The entropic penalty approach to stochastic programming", Math. of Operations Research (to appear). [2] Kullback, S., Information Theory and Statistics, Wiley, New York, 1959. [3] Lucas, B., Characteristic Functions (Second Edition), Griffin, -.5 London, 1970..5.. ", / i ; " e '',",,"' '' '' '' '",2- ' ' '? '' ''? ' -i ' - - ".2,; -< ',. ".2 ' " '' '" '' '' ';- " - -7 ''."." " ' ; ; '
Unclassified SECURITY CLASSIFICATION OF THIS PAGE (When De. Entered) REPORT DOCUMENTATION PAGE READ INSTRUCTIONS BEFORE COMPLETING FORM I. REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER CCS 477 Ifb -Jq13? 4. TITLE (and Subtitle) S. TYPE OF REPORT & PERIOD COVERED A Limit Theorem on Characteristic Functions Via an Extremal Principle 6. PERFORMING ORG. REPORT NUMBER 7. AUTHOR(&) S. CONTRACT OR GRANT NUMBER('e) A. Ben-Tal N00014-81-C-0236 9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK Center for Cybernetic Studies The University of Texas at Austin Austin, Texas 78712 AREA A WORK UNIT NUMBERS,1. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE December 1983 Office of Naval Research (Code 434) D3. NBOA w DC.13. n g as~- o, NUMBER OF PAGES Washington, D.C. 7 14. MONITORING AGENCY NAME & ADDRESS(If different fron Controlilng O11co) 15. SECURITY CLASS. (of this report) 1S. DISTRIBUTION STATEMENT (of this Report) Unclassified Isa. DECLASSIFICATION DOWNGRADING SCHEDULE This document has been approved for public release and sale; its distribution is unlimited. 17. DISTRIBUTION STATEMENT (of the abstract entered in Block 20. ii different from Report) IS. SUPPLEMENTARY NOTES 19. KEY WORDS (Continue on reverse alde if nececeary and identify by block number) Characteristic functions, optimization in infinite dimensional spaces, duality, relative entropy 20. ABSTRACT (Continue en reveree ade It neceear and Identify by block number) We prove a classical limit theorem on characteristic functions by using duality between a pair of optimization problems, one of which is an infinite dimensional minimization involving the relative entropy functional. DD,, 1473 aoi-: 2OFIV IS OBSOLETE Unclassified SECURITY CLASSIFICATION OF THIS PAGE (hen Data Entered) 0,0.
LA 7 7- Ar v~# t 4 '~~4L rivt ' 4 4 4 t~v ift~or 41~4 'Aw t A UA,~ of 4A.4 0~, f% vu 04