Statistical analysis of time series: Gibbs measures and chains with complete connections

Transcription

1 Statistical analysis of time series: Gibbs measures and chains with complete connections Rui Vilela Mendes (Institute) 1 / 38

2 Statistical analysis of time series data Time series X 2 Y : the state space Y Z : the path space X 2 X 1 X 0 X 1 X 2 (Institute) 2 / 38

3 Statistical analysis of time series data Time series X 2 X 1 X 0 X 1 X 2 X 2 Y : the state space Y Z : the path space Statistical properties: (3 levels) (1) Expectation values of the observables (2) Probability measures on state space Y (3) Probability measures on path space Y Z (Institute) 2 / 38

4 Statistical analysis of time series data Time series X 2 X 1 X 0 X 1 X 2 X 2 Y : the state space Y Z : the path space Statistical properties: (3 levels) (1) Expectation values of the observables (2) Probability measures on state space Y (3) Probability measures on path space Y Z Level 1, 2 and 3- statistical indicators. (Mean partial sums, empirical measures (pdf s) and empirical process) (Institute) 2 / 38

5 Statistical analysis of time series data Time series X 2 X 1 X 0 X 1 X 2 X 2 Y : the state space Y Z : the path space Statistical properties: (3 levels) (1) Expectation values of the observables (2) Probability measures on state space Y (3) Probability measures on path space Y Z Level 1, 2 and 3- statistical indicators. (Mean partial sums, empirical measures (pdf s) and empirical process) Analysis and reconstruction of the process: Purpose: To extract Grammar and measure (Institute) 2 / 38

6 Statistical analysis of time series data Time series X 2 X 1 X 0 X 1 X 2 X 2 Y : the state space Y Z : the path space Statistical properties: (3 levels) (1) Expectation values of the observables (2) Probability measures on state space Y (3) Probability measures on path space Y Z Level 1, 2 and 3- statistical indicators. (Mean partial sums, empirical measures (pdf s) and empirical process) Analysis and reconstruction of the process: Purpose: To extract Grammar and measure Examples: Hydrodynamic turbulence Market uctuations: (there are analogies but the statistical indicators are di erent) (Institute) 2 / 38

7 Statistical analysis of time series data Working hypothesis: statistical methods are an appropriate tool to describe and reconstruct the market uctuation process Related to modern view of the e cient market (expected value of abnormal returns is zero - Fama) Opposite view: behavioral component must always be included However: Behavioral trends not inconsistent with statistical description if the di erent reaction times of the market components as well as secondary reactions are taken into account (Olsen et al.) (Institute) 3 / 38

8 Statistical analysis of time series data Working hypothesis: statistical methods are an appropriate tool to describe and reconstruct the market uctuation process Related to modern view of the e cient market (expected value of abnormal returns is zero - Fama) Opposite view: behavioral component must always be included However: Behavioral trends not inconsistent with statistical description if the di erent reaction times of the market components as well as secondary reactions are taken into account (Olsen et al.) Application of statistical tools requires: (i) Stationary or asymptotically stationary process (ii) Typical samples (Institute) 3 / 38

9 Statistical analysis of time series data Working hypothesis: statistical methods are an appropriate tool to describe and reconstruct the market uctuation process Related to modern view of the e cient market (expected value of abnormal returns is zero - Fama) Opposite view: behavioral component must always be included However: Behavioral trends not inconsistent with statistical description if the di erent reaction times of the market components as well as secondary reactions are taken into account (Olsen et al.) Application of statistical tools requires: (i) Stationary or asymptotically stationary process (ii) Typical samples Stocks as experimental probes revealing the mechanisms of the market process (i) ) preprocessing of the data High-frequency versus low-frequency data Complexity versus statistics trade-o (Institute) 3 / 38

10 Statistical analysis of time series data Daily data p (t) 150 IBM 60 Bayer ( ) 60 BMW ( ) 800 NYSE(composite) ( ) ( ) (Institute) 4 / 38

11 Statistical analysis of time series data Detrending by a polynomial (Institute) 5 / 38

12 Statistical analysis of time series data Detrended data p(t) q(t) 80 IBM(detr.) 30 Bayer(detr.) ( ) 30 BMW(detr.) ( ) 250 NYSE(detr.) ( ) ( ) (Institute) 6 / 38

13 Statistical analysis of time series data Detrended and rescaled data x(t) = (p(t) q(t)) <p(t)> q(t) ( ) 15 IBM(detr.+resc.) BMW(detr.+resc.) Bayer(detr.+resc.) ( ) NYSE(detr.+resc.) ( ) ( ) (Institute) 7 / 38

14 Statistical analysis of time series data Ten-days window volatility and comparison of asymptotic volatility for the rescaled and non-rescaled data (IBM and BMW) 4 Volatility 10 days (IBM) 6 Asympt. volatility (IBM) Volatility 10 days (BMW ) 2.5 Asympt. volatility (BMW) Local non-stationarity versus asymptotic stationarity (Institute) 8 / 38

15 Also direct test of stationarity computing the entropies of multi-symbol words in the rst and second half of the samples (Institute) 9 / 38 Statistical analysis of time series data Ten-days window volatility and comparison of asymptotic volatility for the rescaled and non-rescaled data (Bayer and NYSE) 2 Volatility 10 days (Bayer) 2.5 Asympt. volatility (Bayer) Volatility 10 days (NYSE) 30 Asympt. volatility (NYSE)

16 Statistical indicators n days return r(t, n) = log p (t + n) log p (t) (Institute) 10 / 38

17 Statistical indicators n days return r(t, n) = log p (t + n) log p (t) (i) Maximum (over t) of r(t, n) δ (n) = max fr(t, n)g t (Institute) 10 / 38

18 Statistical indicators n days return r(t, n) = log p (t + n) log p (t) (i) Maximum (over t) of r(t, n) δ (n) = max fr(t, n)g t (ii) Moments of the distribution of jr(t, n)j S q (n) = hjr(t, n)j q i (Institute) 10 / 38

19 Statistical indicators n days return r(t, n) = log p (t + n) log p (t) (i) Maximum (over t) of r(t, n) δ (n) = max fr(t, n)g t (ii) Moments of the distribution of jr(t, n)j S q (n) = hjr(t, n)j q i (iii) If in some range (n = 2 to n = 60) χ(q) is the scaling exponent S q (n) s n χ(q) (Institute) 10 / 38

20 delta(n) delta(n) delta(n) delta(n) Statistical indicators Maximum δ (n) of log-prices di erences 10 0 IBM 10 0 Bayer n BMW n NYSE n n (Institute) 11 / 38

21 S_q(n) S_q(n) S_q(n) S_q(n) Statistical indicators Moments of the jr(t, n)j distribution 10 0 IBM 10 0 Bayer n BMW n NYSE n n (Institute) 12 / 38

22 chi(q) Statistical indicators Scaling exponent χ(q) 3.5 IBM(*) Bayer(+) BMW(o) NYSE(x) q (Institute) 13 / 38

23 Statistical indicators Main conclusions: (a) δ (n) is log-concave and probably asymptotically constant for large r (b) S q (n) is a log-concave function of n with an inertial range (c) The scaling law χ(q) is an increasing concave function of q (d) χ(1) in the scaling region (n = 2 to n = 60) is close to 0.5 (e) Scaling properties of NYSE somewhat di erent from the others (Institute) 14 / 38

24 Statistical indicators Main conclusions: (a) δ (n) is log-concave and probably asymptotically constant for large r (b) S q (n) is a log-concave function of n with an inertial range (c) The scaling law χ(q) is an increasing concave function of q (d) χ(1) in the scaling region (n = 2 to n = 60) is close to 0.5 (e) Scaling properties of NYSE somewhat di erent from the others Properties (a) to (c) are shared by the turbulence data, but with di erent values for the statistical indicators (in turbulence data χ(1) = 1 3, here χ(1) t 0.5 ) essentially uncorrelated signal for n 2) (Institute) 14 / 38

25 Statistical indicators Main conclusions: (a) δ (n) is log-concave and probably asymptotically constant for large r (b) S q (n) is a log-concave function of n with an inertial range (c) The scaling law χ(q) is an increasing concave function of q (d) χ(1) in the scaling region (n = 2 to n = 60) is close to 0.5 (e) Scaling properties of NYSE somewhat di erent from the others Properties (a) to (c) are shared by the turbulence data, but with di erent values for the statistical indicators (in turbulence data χ(1) = 1 3, here χ(1) t 0.5 ) essentially uncorrelated signal for n 2) The behavior of the statistical indicators δ (n), S q (n) and χ(q) =) If the process is a topological Markov chain the transitions allowed by the transition matrix T must lie inside a strictly convex domain around the diagonal of T (Institute) 14 / 38

26 C (r(1),t) C ( r(1),t) Statistical indicators Correlation function of one-day returns (?) and its absolute value (o) C (r(1), T ) = hr (t + T, 1) r (t, 1)i C (jr(1)j, T ) = hjr (t + T, 1)j jr (t, 1)ji 4 x 10 5 IBM T (Institute) 15 / 38

27 r(t+1) Statistical indicators Dynamics of one-day returns r (t, 1)! r (t + 1, 1) r(t) (Institute) 16 / 38

28 Looking for a Gibbs measure Why Gibbs measures are "natural" Events fx i g p i = probability of event X i (Institute) 17 / 38

29 Looking for a Gibbs measure Why Gibbs measures are "natural" Events fx i g p i = probability of event X i Constraints: (i) Normalization, i p i = 1 (ii) Expectation value of known observables, i p i F k (X i ) = C k (Institute) 17 / 38

30 Looking for a Gibbs measure Why Gibbs measures are "natural" Events fx i g p i = probability of event X i Constraints: (i) Normalization, i p i = 1 (ii) Expectation value of known observables, i p i F k (X i ) = C k Maximum entropy principle (to maximize the uncertainty about what is not known) = use the most unbiased estimate S S = i p i log p i + λ 0 i p i + k λ k p i F k (X i ) i p i = 0 =) log p i 1 + λ 0 + k λ k F k (X i ) = 0! p i = exp 1 + λ 0 + λ k F k (X i ) k with λ 0, λ 1, obtained from the constraints (Institute) 17 / 38

31 Looking for a Gibbs measure Coding by a nite alphabet Σ Space Ω of orbits ω = i 1 i 2 i k, i k 2 Σ Dynamical law: a shift σ σω = i 2 i k (Institute) 18 / 38

32 Looking for a Gibbs measure Coding by a nite alphabet Σ Space Ω of orbits ω = i 1 i 2 i k, i k 2 Σ Dynamical law: a shift σ σω = i 2 i k Grammar: set of allowed sequences in Ω (Institute) 18 / 38

33 Looking for a Gibbs measure Coding by a nite alphabet Σ Space Ω of orbits ω = i 1 i 2 i k, i k 2 Σ Dynamical law: a shift σ σω = i 2 i k Grammar: set of allowed sequences in Ω Sequences which coincide on the rst n symbols: n n block) denoted by [i 1 i 2 i n ] Probability measures over the cylinders cylinder (or (Institute) 18 / 38

34 Looking for a Gibbs measure Coding by a nite alphabet Σ Space Ω of orbits ω = i 1 i 2 i k, i k 2 Σ Dynamical law: a shift σ σω = i 2 i k Grammar: set of allowed sequences in Ω Sequences which coincide on the rst n symbols: n cylinder (or n block) denoted by [i 1 i 2 i n ] Probability measures over the cylinders Gibbs measure c 1 µ ([i 1(ω)i 2 (ω) i n (ω)]) c 2 exp ( np + (S n φ) (ω)) (S n φ) (ω) = n 1 k=0 φ σk ω, φ being Hölder continuous function on Ω (the potential) P (φ, G ) : a function depending on potential and grammar (the pressure of φ) (Institute) 18 / 38

35 Looking for a Gibbs measure Relation to the entropy H n h (µ) = lim n! n = lim 1 n! n µ ([i 1 i 2 i n ]) log µ ([i 1 i 2 i n ]) i 1 i n (Institute) 19 / 38

36 Looking for a Gibbs measure Relation to the entropy H n h (µ) = lim n! n = lim 1 n! n µ ([i 1 i 2 i n ]) log µ ([i 1 i 2 i n ]) i 1 i n Variational principle: for each potential and grammar, sup η h (η) + R φdη over all σ invariant measures η is reached only for the Gibbs measure µ and equals the pressure Z P (φ, G ) = h (µ) + φdµ (Institute) 19 / 38

37 Looking for a Gibbs measure Relation to the entropy H n h (µ) = lim n! n = lim 1 n! n µ ([i 1 i 2 i n ]) log µ ([i 1 i 2 i n ]) i 1 i n Variational principle: for each potential and grammar, sup η h (η) + R φdη over all σ invariant measures η is reached only for the Gibbs measure µ and equals the pressure Z P (φ, G ) = h (µ) + φdµ Potential may be chosen such that P = 0 (normalized potential). Then φ (ω) = lim log µ ([i 1(ω) i n (ω)]) n! µ ([i 2 (ω) i n (ω)]) (Practical use hindered by poor statistics of large blocks) (Institute) 19 / 38

38 Looking for a Gibbs measure Gibbs measures for nite range potentials ( nite range potentials approximate uniformly any Hölder continuous potential) (Institute) 20 / 38

39 Looking for a Gibbs measure Gibbs measures for nite range potentials ( nite range potentials approximate uniformly any Hölder continuous potential) Property of range r potentials: for all values i 1 i 2 i n with n r =) µ ([i 1 i n ]) = µ ([i 1 i r ]) µ ([i n r +1 i n ]) µ ([i 2 i r ]) µ ([i n r +1 i n 1 ]) h (µ) = µ ([i 1 i k ]) log µ ([i 1 i k ]) i 1 i k µ ([i 1 i k 1 ]) = H k H k 1 for all k r if r > 1. If r = 1 h (µ) = H 1 H k = i1 i k µ ([i 1 i k ]) log µ ([i 1 i k ]) (1) (Institute) 20 / 38

40 Looking for a Gibbs measure Gibbs measures for nite range potentials ( nite range potentials approximate uniformly any Hölder continuous potential) Property of range r potentials: for all values i 1 i 2 i n with n r =) µ ([i 1 i n ]) = µ ([i 1 i r ]) µ ([i n r +1 i n ]) µ ([i 2 i r ]) µ ([i n r +1 i n 1 ]) h (µ) = µ ([i 1 i k ]) log µ ([i 1 i k ]) i 1 i k µ ([i 1 i k 1 ]) = H k H k 1 for all k r if r > 1. If r = 1 h (µ) = H 1 H k = i1 i k µ ([i 1 i k ]) log µ ([i 1 i k ]) =) criterium to nd the range of the potential: range of the potential found when H k H k 1 tends to a constant. Once the range is found, the potential may be constructed from the empirical weights eµ ([i 1 i k ]). (Institute) 20 / 38 (1)

41 Looking for a Gibbs measure Another consequence of (1) is that for k > r µ ([i 1 i k+1 ]) = µ ([i 1 i k ]) µ ([i 2 i k+1 ]) µ ([i 2 i k ]) (Institute) 21 / 38

42 Looking for a Gibbs measure Another consequence of (1) is that for k > r µ ([i 1 i k+1 ]) = µ ([i 1 i k ]) µ ([i 2 i k+1 ]) µ ([i 2 i k ]) Application to the market uctuations: Five-symbols code Σ = f 2, 1, 0, 1, 2g for r (t) = log p (t + 1) log p (t) r Average r (t) and standard deviation s = r 2 (t) r (t) 2 r (t) r (t) > s () 2 s r (t) r (t) > s 3 () 1 s 3 r (t) r (t) > s 3 () 0 s 3 r σ (t) r (t) > s () 1 s r (t) r (t) () 2 (Institute) 21 / 38

43 H(k) H(k 1) p H(k) H(k 1) p Looking for a Gibbs measure H k H k 1 and the number of occurring blocks of size k (IBM and Bayer) k 1.5 IBM Bayer k k k (Institute) 22 / 38

44 H(k) H(k 1) p H(k) H(k 1) p Looking for a Gibbs measure H k H k 1 and the number of occurring blocks of size k (BMW and NYSE) k 2 BMW NYSE k k k (Institute) 23 / 38

45 Looking for a Gibbs measure Behavior of H k H k 1 quite di erent from hydrodynamic turbulence data (Institute) 24 / 38

46 Looking for a Gibbs measure Behavior of H k H k 1 quite di erent from hydrodynamic turbulence data To check whether a short-range potential is reliable : For successively higher k estimate µ e ([i 1 i k+1 ]) = eµ ([i 1 i k ]) eµ ([i 2 i k+1 ]) eµ ([i 2 i k ]) then compare with the empirically observed eµ ([i 1 i k+1 ]) (Institute) 24 / 38

47 Looking for a Gibbs measure Behavior of H k H k 1 quite di erent from hydrodynamic turbulence data To check whether a short-range potential is reliable : For successively higher k estimate µ e ([i 1 i k+1 ]) = eµ ([i 1 i k ]) eµ ([i 2 i k+1 ]) eµ ([i 2 i k ]) then compare with the empirically observed eµ ([i 1 i k+1 ]) Standard deviation of the relative positive errors! eµ ([i 1 i k+1 ]) µ ε k = max 0, e ([i 1 i k+1 ]) 1 2 (eµ ([i 1 i k+1 ]) + µ e ([i 1 i k+1 ])) (Institute) 24 / 38

48 Looking for a Gibbs measure Underestimation errors one (o) and two (*) standard deviations away from the mean and the total number p (k) of observed blocks p(k) k (Institute) 25 / 38

49 Looking for a Gibbs measure Large number of large deviation errors Correspond to blocks involving 2 and 2 Large deviations misrepresented by empirically constructed measure (Institute) 26 / 38

50 Looking for a Gibbs measure Large number of large deviation errors Correspond to blocks involving 2 and 2 Large deviations misrepresented by empirically constructed measure?? ) non-gibbsian measure or?? ) Gibbsian measure with long-range potential (sharp rise of H k H k 1 at k = 2 followed by a very slow increase) (Institute) 26 / 38

51 Looking for a Gibbs measure Large number of large deviation errors Correspond to blocks involving 2 and 2 Large deviations misrepresented by empirically constructed measure?? ) non-gibbsian measure or?? ) Gibbsian measure with long-range potential (sharp rise of H k H k 1 at k = 2 followed by a very slow increase) Large deviation analysis applied to the calculation of H k consistent with this hypothesis (Institute) 26 / 38

52 Looking for a Gibbs measure Large number of large deviation errors Correspond to blocks involving 2 and 2 Large deviations misrepresented by empirically constructed measure?? ) non-gibbsian measure or?? ) Gibbsian measure with long-range potential (sharp rise of H k H k 1 at k = 2 followed by a very slow increase) Large deviation analysis applied to the calculation of H k consistent with this hypothesis In any case it needs an approach suited to deal with long-memory processes (Institute) 26 / 38

53 Chains with complete connections Chain with complete connections (CCC) (Institute) 27 / 38

54 Chains with complete connections Chain with complete connections (CCC) 1 8a i 2 Σ P (X 1 = a 1,, X n = a n ) > 0 (Institute) 27 / 38

55 Chains with complete connections Chain with complete connections (CCC) 1 8a i 2 Σ 2 The limit P (X 1 = a 1,, X n = a n ) > 0 exists 8a i, j 1 lim P (X 0 = a 0 jx j = a j, m j 1) m! = P (X 0 = a 0 jx j = a j, j 1) (Institute) 27 / 38

56 Chains with complete connections Chain with complete connections (CCC) 1 8a i 2 Σ 2 The limit P (X 1 = a 1,, X n = a n ) > 0 exists 8a i, j 1 lim P (X 0 = a 0 jx j = a j, m j 1) m! = P (X 0 = a 0 jx j = a j, j 1) 3 There is a sequence (γ m ) m1 with lim m! γ m = 0, such that for all fa j, b j 2 Σ, j 1g with a j = b j for m j 1 P (X0 = a 0 jx j = a j, j 1) 1 γ P (X 0 = a 0 jx j = b j, j 1) m (Institute) 27 / 38

57 Looking for a Gibbs measure Chain with complete connections and summable decay (CCCSD) CCC with γ m < (Institute) 28 / 38

58 Looking for a Gibbs measure Chain with complete connections and summable decay (CCCSD) CCC with γ m < Conditions 1. and 2. implicitly assumed for the pre-processed data Decays γ m estimated from a typical sample of the process. From the empirical probabilities P (a 0 ja 1 a m A) = P (a 0a 1 a m A) P (a 1 a m A) A a block of arbitrary length maxa P (a 0 ja 1 a m A) g (a 0 a 1 a m ) = min A P (a 0 ja 1 a m A) γ m = max g (a 0 a 1 a m ) a 0 a 1 a m 1 If the statistics for long blocks is poor ) large uctuations in γ m (Institute) 28 / 38

59 Looking for a Gibbs measure Chain with complete connections and summable decay (CCCSD) CCC with γ m < Conditions 1. and 2. implicitly assumed for the pre-processed data Decays γ m estimated from a typical sample of the process. From the empirical probabilities P (a 0 ja 1 a m A) = P (a 0a 1 a m A) P (a 1 a m A) A a block of arbitrary length maxa P (a 0 ja 1 a m A) g (a 0 a 1 a m ) = min A P (a 0 ja 1 a m A) γ m = max g (a 0 a 1 a m ) a 0 a 1 a m If the statistics for long blocks is poor ) large uctuations in γ m Better estimate of the decay behavior with g(m) = g (a 0 a 1 a m ), the average being taken over all sets a 0 a 1 a m of size m. 1 (Institute) 28 / 38

60 g(m) Chains with complete connections g(m) computed using A blocks of length 5 to 8 (, +, o, ) m (Institute) 29 / 38

61 Chains with complete connections Result compatible with exponential decay ) summability of the γ m s (Institute) 30 / 38

62 Chains with complete connections Result compatible with exponential decay ) summability of the γ m s A CCC-process with summable decays is the d Markov approximations of order k limit of its (Institute) 30 / 38

63 Chains with complete connections Result compatible with exponential decay ) summability of the γ m s A CCC-process with summable decays is the d limit of its Markov approximations of order k d distance Couplingn between o two processes X = fx n g and Y = fy n g is another process X en, ey n over Σ Σ such that the marginal probabilities of ex and ey coincide with those of X and Y n o d (X, Y ) = inf P X e0 6= ey 0 : X en, ey n is a stationary coupling of X and Y (Institute) 30 / 38

64 Chains with complete connections Result compatible with exponential decay ) summability of the γ m s A CCC-process with summable decays is the d limit of its Markov approximations of order k d distance Couplingn between o two processes X = fx n g and Y = fy n g is another process X en, ey n over Σ Σ such that the marginal probabilities of ex and ey coincide with those of X and Y n o d (X, Y ) = inf P X e0 6= ey 0 : X en, ey n is a stationary coupling of X and Y d distance tending to zero does mean that the processes will coincide after a certain time (Institute) 30 / 38

65 Chains with complete connections Result compatible with exponential decay ) summability of the γ m s A CCC-process with summable decays is the d limit of its Markov approximations of order k d distance Couplingn between o two processes X = fx n g and Y = fy n g is another process X en, ey n over Σ Σ such that the marginal probabilities of ex and ey coincide with those of X and Y n o d (X, Y ) = inf P X e0 6= ey 0 : X en, ey n is a stationary coupling of X and Y d distance tending to zero does mean that the processes will coincide after a certain time Perfect simulation always understood in the d distance sense. It does not mean perfect prediction (Means that a process is constructed with the same conditional probabilities of the original one) (Institute) 30 / 38

66 Chains with complete connections Simulation scheme by the sequence of canonical Markov approximations of nite order k (k CMA) k CMA of a process X is a Markov chain Y (k) of order k with conditional probabilities P (k) P (k) (a 0 ja 1 a k ) = P (a 0 ja 1 a k ) = P (a 0 ja 1 a k A) A (Institute) 31 / 38

67 Chains with complete connections Simulation scheme by the sequence of canonical Markov approximations of nite order k (k CMA) k CMA of a process X is a Markov chain Y (k) of order k with conditional probabilities P (k) P (k) (a 0 ja 1 a k ) = P (a 0 ja 1 a k ) = P (a 0 ja 1 a k A) A For a CCC X with summable decays d X, Y (k) C γ k (2) The property of the Markov approximation, essential for the approximation result (2), is inf P (a 0ja 1 a k A) P (k) (a 0 ja 1 a k ) sup P (a 0 ja 1 a k A) A meaning that for Markov approximation schemes, other than the canonical one, Eq.(2) holds provided (3) is satis ed (Institute) 31 / 38 A (3)

68 Looking for a Gibbs measure For the market uctuation data: k Markov approximation: i) Empirical transition probabilities ep (a 0 ja 1 a m ) inferred from the probability of blocks of order m + 1. up to m Max ii) k Simulation: look at the current block (a 1 a k ) and use the k empirical probability to infer the next state a 0. If that block has not appeared in the trainig data, use the k 1 sized block a 2 a k and the k 1 order empirical probabilities (Institute) 32 / 38

69 Looking for a Gibbs measure For the market uctuation data: k Markov approximation: i) Empirical transition probabilities ep (a 0 ja 1 a m ) inferred from the probability of blocks of order m + 1. up to m Max ii) k Simulation: look at the current block (a 1 a k ) and use the k empirical probability to infer the next state a 0. If that block has not appeared in the trainig data, use the k 1 sized block a 2 a k and the k 1 order empirical probabilities Averaged squared error e 2 = D(ea 0 a 0 ) 2E (Institute) 32 / 38

70 e e e e Chains with complete connections The past predicting the future (o) and the future predicting the past (*), compared to random choice 1.32 IBM 1.29 BAYER k k 1.26 BMW 1.5 Random IBM BAYER BMW k k (Institute) 33 / 38

71 e e e e Chains with complete connections The past predicting the future (o) and the future predicting the past (*), compared to random choice IBM BAYER k k 0.39 BMW 0.76 Random IBM BAYER BMW k k (Institute) 34 / 38

72 Chains with complete connections Main conclusions: Average prediction better than random choice. Main improvement results from correct accounting of the two-symbol probabilities (k = 1) Small (but consistent) improvement using past information up k = 4 or 5. No signi cant improvement using higher order approximations Bulk of data represented by a short-memory process. Nevertheless there is evidence for a small long-memory component that is captured by the higher-order Markov approximations There is a maximum k = k m that should be used for the simulation process Inter-companies prediction: improvement coming from the one-symbol probabilities (as compared to random choice) is obtained. For the long-memory component the behavior is company-dependent. (Institute) 35 / 38

73 Statistical analysis of market data General conclusions 1 Bulk of the market uctuation process is a short-memory process. In addition it has a small long-memory component associated with the large uctuations of the returns. 2 Existence of the long-memory component suggests the chains with complete connections and summable decays as a framework 3 Although the decays may be exponentially converging, the lack of accurate data for long blocks prevents an accurate description by a nite range Gibbs potential. 4 The sequence of empirical based k Markov approximations seems the most unbiased simulation of the process. Eventual convergence in the d distance sense, because of summable decays. 5 For the future: high frequency market data. Beware of the possibly multi-scale and multi-component nature of the processes. (Institute) 36 / 38

74 References # J.-R. Chazottes, E. Floriani and R. Lima; Relative entropy and identi cation of Gibbs measures in dynamical systems, J. of Stat. Phys. 90 (1998) # E. F. Fama; Market e ciency, long-term returns and behavioral nance, Journal of Financial Economics 49 (1998) # R. B. Olsen, M. M. Dacorogna, U. A. Müller and O. V. Pictet; Going back to basics - Rethinking market e ciency, Olsen n& Associates discussion paper RBO # D. Ruelle; Thermodynamic formalism, Encyclopedia of Mathematics and its Applications 5, Addison-Wesley # O. Onicescu and G. Mihoc; Le comportment asymptotique des chaines à liaisons complètes; Disq. Math. Phys. 1 (1940) # M. Iosifescu and S. Grigorescu; Dependence with complete connections and its applications, Cambridge U. P., Cambridge # D. S. Ornstein and B. Weiss; How sampling reveals a process, The Annals of Prob. 18 (1990) (Institute) 37 / 38

75 References # X. Bressaud, R. Fernandez and A. Galves; Speed of d convergence for Markov approximations of chains with complete connections. A coupling approach, Stoch. Proc. and Appl. 83 (1999) # F. Comets, R. Fernandez and P. A. Ferrari; Processes with long memory: Regenerative construction and perfect simulation, arxiv:math.pr/ # RVM, R. Lima and T. Araújo; A process-reconstruction analysis of market uctuations, Int. Journal of Theoretical and Applied Finance, 5 (2002) 797 (Institute) 38 / 38