Sequential Investment, Universal Portfolio Algos and Log-loss

Transcription

1 1/37 Sequential Investment, Universal Portfolio Algos and Log-loss Chaitanya Ryali, ECE UCSD March 3, 2014

2 Table of contents 2/

3 Definitions and Notations 3/37 A market vector x = {x 1,x 2,...,x m } for m assets is a vector of nonnegative real numbers representing price relatives for a given trading period. x i 0 denotes the ratio of closing to opening price of the ith asset for that period. An initial wealth invested in m assets according to the fractions Q 1,Q 2,...,Q m multiplies by a factor of m i=1 x iq i at the end of the period. Market behavior during n trading periods is represented by a sequence of market vectors x n = ( x 1, x 2,..., x n ).

4 Definitions and Notations 4/37 The probability simplex in R m is denoted by m 1. An investment strategy Q for n trading periods is a sequence Q 1,..., Q n of vector valued functions t : R Q t 1 + m 1 ith component Q i,t ( x t 1 )of vector t ( x Q t 1 ) denotes the fraction of the current wealth invested in the i th asset at the beginning of the tth period on the basis of the past market behavior x t 1

5 Wealth Factor 5/37 S n (Q, x n ) = n m ( x i,t Q i,t ( x t 1 )) (1) t=1 denotes the wealth factor of strategy Q after n trading periods. i=1 Q t has nonnegative component summing to one expresses no short sales and no buying on margin.

6 Examples 6/37 Buy-and-Hold: S n (Q, x n ) = m n Q j,1 j=1 max t=1 n x j,t x j,t j=1,...,m t=1

7 Examples 7/37 Constantly Rebalanced Portfolios: Parametrized by a probability vector B = (B 1,B 2,...,B m ) m 1 Q t ( x t 1 ) = regardless of t and B x t 1 n m S n ( B, x n ) = ( x i,t B i ). t=1 i=1 Example: (1, 1 2 ),(1,2),(1, 1 2 ),(1,2),... Buy and Hold No profit, No loss CRP : B = ( 1 2, 1 2 ) (9 8 )n/2, exponentially increasing wealth.

8 Minimax Wealth Ratio 8/37 Given a class Q of investment strategies, the worst case logarithmic wealth ratio of a strategy P is given by W n (P,Q) = sup sup xn Q Q ln S n(q, x n ) S n (P, x n ). Minimax logarithmic wealth ratio is defined as: W n (Q) = inf P W n(p,q). W n (P,Q) = o(n) means strategy P achieves the same exponent of growth as the best reference strategy in class Q for all market behaviors.

9 Prediction under log-loss and Investment 9/37 Any investment strategy Q can be used define a forecaster that predicts elements y t Y{1,...,m} of a sequence y n Y n with probability vectors ˆp t m 1 Kelly Market Vectors: Market vectors x with a single component equal to 1 and all other components equal to zero. If x 1,..., x n are Kelly market vectors, we denote the index of the only non zero component of each vector x t by y t, we may define a forecaster f by f t (y y t 1 ) = Q y,t ( x t 1 ). f is induced by investment strategy Q.

10 Prediction under log-loss and Investment 10/37 When x n is a sequence of Kelly vectors determined by the indices y n, we write S n (Q,y n ) for S n (Q, x n ). Note that S n (Q,y n ) = f n (y n ), where f is the forecaster induced by Q, where f n (y n ) = n t=1 f t(y t y t 1 ), where y n Y n f n(y n ) = 1. Conversely, given a f n (y n ), we may define f t (y t y t 1 ) = f t(y t ) f t 1 (y t 1 ), where f t (y t ) = y n t+1 Yn t f n(y n ).

11 Prediction under log-loss and Investment 11/37 Log-loss: l(f t,y t ) = lnf t (y t y t 1 ) Regret against a reference forecaster f is ˆL n L f,n = ln f n(y n ) ˆp n (y n ) = ln Q(yn ) P(y n ), where Q and P are the investment strategies induced by f and ˆp.

12 12/37 Lemma Let Q be a class of investment strategies, and let F denote the class of forecasters induced by the strategies in Q. Then, the minimax regret satisfies W n (Q) V n (F). V n (F) = infsupsup p n y n f F ln f n(y n ) p n (y n )

13 13/37 Proof. Let P be any investment strategy and let p be it s induced forecaster. Then sup sup ln S n(q, x n ) xn Q Q S n (P, x n ) max sup ln S n(q,y n ) y n Y n Q Q S n (P,y n ) = max sup ln f n(y n ) y n Y n f F p n (y n ) = V n (p,f) V n (F).

14 14/37 Given a prediction p, we define an investment strategy P as follows: P j,t ( x t 1 ) = y t 1 Y t 1p t(j y t 1 )p t 1 (y t 1 )( t 1 s=1 x y s,s) y t 1 Y t 1p t 1(y t 1 )( t 1 s=1 x y s,s) The obtained investment strategy induces p, and so we say p and P induce each other. t 1 s=1 x y s,s may be viewed as the return of the extremal investment strategy that, on each trading period t, invests everything on the y t th asset.

15 15/37 Theorem Let P be an investment strategy induced by a forecaster p, and let Q be an arbitrary class of investment strategies. Then for any market sequence x n, sup ln S n(q, x n ) Q Q S n (P, x n ) max sup ln y n Y n Q Q n t=1 Q y,t( x t 1 ) p n (y n )

16 16/37 Lemma let a 1,...,a n, b 1,...,b n be non negative numbers. Then, where we define 0/0 = 0. n i=1 a i n i=1 b i a j max, j=1,...,n b j

17 17/37 Lemma The wealth factor achieved by an investment strategy Q may be written as S n (Q, x n ) = ( y n Y n t=1 n n x yt,t)( Q yt,t( x t 1 )). If the investment strategy P is induced by a forecaster p n, then S n (P, x n ) = ( y n Y n t=1 t=1 n x yt,t)p n (y n ).

18 18/37 Proof. S n (Q, x n ) = n m ( x j,t Q j,t ( x t 1 )) t=1 j=1 = y n Y n ( = y n Y n ( n x yt,tq yt,t( x t 1 )) t=1 n n x yt,t)( Q yt,t( x t 1 )). t=1 t=1

19 19/37 Proof. S n (P, x n ) = = n m ( x j,t P j,t ( x t 1 )) t=1 n t=1 j=1 m j=1 y t 1 Y t 1p t(y t 1 j)x j,t ( t 1 s=1 x y s,s) y t 1 Y t 1p t(y t 1 j)x j,t ( t 1 n y = t Y t( t s=1 x y s,s)p t (y t ) t=1 y t 1 Y t 1( t 1 s=1 x y s,s)p t 1 (y t 1 ) = n x yt,t)p n (y n ). ( y n Y n t=1 s=1 x y s,s)

20 20/37 Proof. Fix any market sequence x n and choose any reference strategy Q Q. Denote by S n (y n, x n ) = n t=1 x y t,t, then S n (Q, x n ) S n (P, x n ) = y n Y n S n(y n, x n )( n t=1 Q y t,t ( xt 1 )) y n Y n S n(y n, x n )p n (y n ) S n (y n, x n )( n t=1 max Q y t,t( x t 1 )) S n (y n, x n )p n (y n ) n t=1 Q y t,t ( xt 1 ) y n :S n(y n, x n )>0 = max y n Y n max y n Y n sup q Q p n (y n ) n t=1 Q y t,t( x t 1 ) p n (y n. )

21 21/37 Theorem Let Q be a class of static investment strategies, and let F denote the class of forecasters induced by strategies in Q. Then W n (Q) = V n (F). Furthermore, the minimax optimal investment strategy is defined by P j,t( x t 1 ) = y t 1 Y t 1p t (j yt 1 )p t 1 (yt 1 )( t 1 s=1 x y s,s) y t 1 Y t 1p t 1 (yt 1 )( t 1 s=1 x y s,s) where p is the normalized maximum likelihood forecaster p n(y n ) = sup n Q Q t=1 Q y t,t y n Y n sup n Q Q t=1 Q. y t,t

22 Normalized Maximum Likelihood Forecaster 22/37 Definition The normalized maximum likelihood forecaster is defined by the following: pn sup (yn ) = f F f n (y n ) x n Y n sup f F f n (y n )

23 Normalized Maximum Likelihood Forecaster 23/37 Theorem For any class F of experts and integer n > 0, the normalized maximum likelihood forecaster p is the unique forecaster such that sup (ˆL(y n ) inf L f(y n )) = V n (F). y n Y n f F Moreover, p is an equalizer that is, for all y n Y n, ln sup f F f n (y n ) p n (yn ) = ln x n Y n sup f F f n (x n ) = V n (F).

24 24/37 Proof. The normalized maximum likelihood forecaster p is minimax optimal for the class F; that is, max y n Ynln sup Q Q n t=1 Q y t,t p n (yn ) = V n (F). Now, let P be the investment strategy induced by minimax forecaster p for Q. By theorem, we get W n (Q) sup sup ln S n(q, x n ) xn Q Q S n (P, x n ) max sup ln y n Y n Q Q n t=1 Q y t,t p n (yn ) = V n (F)

25 Constantly Rebalanced Portfolios 25/37 W n (Q) = m 1 2 lnn+ln Γ(1/2)m Γ(m/2) +o(1)

26 26/37 We restrict our attention to class Q of all constantly rebalanced portfolios. Each strategy Q in this class is determined by a vector B = {B 1,B 2,...,B m } m 1 The Universal Portfolio strategy P is given by P j,t ( x t 1 ) = m 1 B j S t 1 ( B, x t 1 )µ( B)d B m 1 S t 1 ( B, x t 1 )µ( B)d B, j = 1,2...,m, t = 1,...,n, and µ is a density function on m 1.

27 27/37 The wealth achieved by the universal portfolio is just the average of the wealths achieved by the individual strategies in the class. S n (P, x n ) = = n t=1 j=1 n t=1 m P j,t ( x t 1 )x j,t m m 1 j=1 x j,tb j S t 1 ( B, x t 1 )µ( B)d B m 1 S t 1 ( B, x t 1 )µ( B)d B n = m 1 S t ( B, x t )µ( B)d B t=1 m 1 S t 1 ( B, x t 1 )µ( B)d B = S n ( B, x n )µ( B)d B m 1

28 28/37 It s like a Buy-and-Hold on all Constantly Rebalanced Portfolios (CRP) S n (P, x n ) = S n ( B, x n )µ( B)d B m 1 If it helps, think of it as: (Riemann sum approximation) S n (P, x n ) = i Q i S n ( B i, x n ), where, given the elements i of a fine partition of the simplex m 1, we assume that B i m 1 and Q i = i µ( B)d B.

29 29/37 Theorem If µ is the uniform density on the probability density simplex m 1 R m, then the wealth achieved by the universal portfolio satisfies sup xn sup ln S n( B, x n ) B m 1 S n (P, x n (m 1)ln(n+1). ) If the universal portfolio is defined using the Dirichlet(1/2,...,1/2) density µ, then sup xn sup ln S n( B, x n ) B m 1 S n (P, x n ) m 1 2 lnn+ln Γ(1/2)m Γ(m/2) +m 1 ln2+o(1). 2

30 30/37 Universal Portfolio involves integration over m-dimensional simplex. EG s computational cost is linear in m invests at a time t using the vector P t = (P 1,t,...,P m,t ) where t = (1/m,...,1/m) and P P i,t 1 exp(η(x i,t 1 / P t 1 x t 1 )) P i,t = m j=1 P j,t 1exp(η(x j,t 1 / P t 1 x t 1 )) where i = 1,2,...,m and t = 2,3,...

31 31/37 Special case of gradient-based forecaster: P i,t = P i,t 1 exp(η l t 1 ( P t 1 ) i ) m j=1 P j,t 1exp(η l t 1 ( P t 1 ) j ) when the loss function is set as l t 1 ( P t 1 ) = ln P t 1 x t 1.

32 32/37 Theorem Assume that the price relatives x i,t all fall between two positive constants c < C. Then the worst-case logarithmic wealth ratio of the with η = (c/c) (8lnm)/n is bounded by lnm η + nη C 2 8 c 2 = C n c 2 lnm.

33 Simple proof without transaction costs 33/37 Main Idea: Portfolios that are near each other perform similarly, and there is a large fraction of portfolios near the optimal one. Suppose in hindsight B is the optimal CRP. Let B = (1 α) B +α z, for some m 1.(Meaning, is z B close to B ). For a single period Over n periods, gain of CRP B (1 α)(gain ofcrp B ). wealth of CRP B (1 α) n (wealth of CRP B ).

34 Simple proof without transaction costs wealth of UNIVERSAL wealth of best CRP E B m 1 [(1 α)n ] = = = n = Prob B m 1 [(1 α) n x]dx (1 x 1/n ) m 1 dx 0 y n 1 (1 y) m 1 dy (m 1)!(n 1)! = n(!) n+m 2 1 = ) ( n+m 1 m 1 34/37

35 Commission 35/37 The following assumptions are made: The costs paid changing from distribution B 1 to B 3 is no more than the costs paid changing from B 1 to B 2 and then from B 2 to B 3. The cost, per dollars, of changing from a distribution B to a distribution (1 α) B 1 +α B is no more than αc, because at most an α fraction of the money is being moved. An investment strategy I which invests an initial fraction α of it s money according to investment strategy I 1 and an initial 1 α of it s money according to I 2, will achieve at least α times the wealth of I 1 plus 1 α times the wealth of I 2.

36 Result with Commission 36/37 Theorem In the presence of commission 0 c 1, wealth of UNIVERSAL c wealth of best CRP ( (1+c)n+m 1 m 1 1 ((1+c)n+1) m 1. ) 1

37 Result with Commission 37/37 Proof. Based on the properties we assumed, if B j (1 α)b j, then single-period profit of CRP single-period profit of CRP Over n periods, this gives B B (1 α)(1 cα). wealth of CRP B (1 α) (1+c)n (wealth of CRP B ). The previous proof can be applied and we can replace n by (1+c)nin the final guarantee.