数理統計学から た Thermo-Majorization: 統計モデルの 較と情報スペクトル. National Institute of Informatics Keiji Matsumoto

Transcription

1 数理統計学から た Thermo-Majorization: 統計モデルの 較と情報スペクトル National Institute of Informatics Keiji Matsumoto

2 Thermo Majorization ρ # : (Gibbs) state ρ, ρ # = 0, σ, ρ # = 0 The transition is possible ρ, ρ # σ, ρ # (ρ 11 /ρ #,11 ) 1 majorizes (σ 11 /ρ #,11 ) 1 5 4(ρ 11 /ρ #,11 ) 4(σ 11 /ρ #,11 ), j tr ρ tρ # - tr σ tρ # -, t 0 A - : positive part of the matrix A 5

3 Thermo-Majorization : History The term first appears in Horodecki, M + Oppenheim, J (2014) But had been known since decades ago: Mathematical Physics : mapping btw. state families Uhlmann, Alberiti, Ruch, Schramer, etc Mathematical Statistics : comparison of statistical experments Blackwell, Le Cam, Hajek, Torgersen, etc Why statisticians are bothered with conversion btw state families?

4 Information Spectrum A framework of information theory free of stochastic assumptions such as IID, Markov, etc Classical version : invented by Te-Sun Han ( 韓太舜 ), with Shannon Award Quantum version : Hiroshi Nagaoka (UEC) DK({ρ } σ D ({ρ } σ = inf{c; lim G tr ρ e I σ - = 0} = sup{c; lim tr ρ e I σ - = 1} G (version by Bowen-Datta) c.f. Renner, Datta, Bowen give alternative representation of these by Smooth Renyi entropy Computer science, randomness extraction, privacy amplification

5 Thermo-Majorization tr ρ tρ # - tr σ tρ # -, t 0 Information Spectrum DK({ρ } σ D ({ρ } σ = inf{b; lim G tr ρ e O σ - = 0} = sup{b; lim G tr ρ e O σ - = 1} If you are not xxxxxx, Should notice some relations btw these.

6 Plan of the talk: Comparison of statistical experiments - general theory, historical contexts - 2-states case - relation to information measures - quantum states Information spectrum (mainly hypothesis test) - upper & lower divergence rate (classical, quantum) - hypothesis test (- resolvability) -relation to smooth Renyi entropies Asymptotic 2-states conversions - sufficient condition by information spectrum - characterization of quantum relative entropy (KL-divergence)

7 Comparison of statistical experiments

8 References Blackwell, D. and Girshick, M. A. Theory of Games and Statistical Decisions (1954) LeCam, L. ( many works since 60 s) Asymptotic Methods in Statistical Decision Theory, Springer (1986) Torgersen, E. Comparison of Statistical Experiments, Cambridge University Press(1991) Goel P K (concise and comprehensive review) When is one experiment ʻalways better than another? The Statistician 52, Part 4, pp (2003)

9 Statistical decision problems x X : (array of) data d(x) D : decision x obeys probability distribution p R x, θ Θ unknown g R d R: the gain of the decision d when the data source is p R X 等は各 分かりやすいので考えて ( 理論は極限まで 般化されてるが気にしないで ) E = { p R ; θ Θ} is called an experiment or a statistical model Optimize d to maximize the expectation G R (d, E): = E R g R d(x) = g R d x p R x [ X under various setting min-max/average about θ/constrained

10 Some notes Settings are fairly general X : measurable space D : topological space, with Bair σ-filed, d: measureable P R : prob measure, may not be majorized by a common measure Θ: a set g is usually subject to some reasonable conditions g R : upper-semi continuous/continuous + bounded from above/bouded In general, randomized decision is also allowed ここでは記述が 倒だから避けるが どっか破たんしてたらごめんなさい Usually, loss is minimized, rather than gain is maximized の事情があるので ここではゲイン

11 Example of Statistical Decisions Ex 1 x : past/present data about economics θ : parameters determining the market price d : your strategy of selling/buying g R (d) : how much you earn Ex 2 θ R a d : your estimate of θ g R d = d θ b : how close was your guess Ex 3 θ {0,1} d : your guess about θ 0,1 g R d = 1 if you are right, otherwise =0

12 On arbitrariness of g Ex 2 θ R a d : your estimate about the true value of parameter g R d = d θ b : how close was your guess No overwhelming reason to prefer square-error to other error measures

13 Statistical Decision Problems : Optimization? Life is so hard that no decision is uniformly optimal usually : if G R d, E : = g R (x)p R x [ very good, G R c(d, E) is poor Ex 3 θ 0,1 d : your guess about θ 0,1 g R d = 1 if you are right, otherwise =0 If d x = 0 irrespective of x, G d (d, E) = 1 but G 7 d, E = 0 1 Maximize the average about θ (Bayesian gain): G e d, E 4 π R G R (d, E) R 2 Maximize the worst case about θ (min-max): G(d, E) = inf R h G R(d, E) 3 Maximize e.g. G Ri (d, E), subject to the constrain, e.g. G R (d, E) c, θ θ 7 And so on

14 optimal decision? depends on G depends on min-max or Bayesian gain, or others Life is so hard. You have to solve them one by one. But, there is a hope. Comparison of statistical experiments - asymptotic theory LAN, bounds on cloning etc - experimental design 存在感こっちのほうが圧倒的で でも説明しやすいのはこっちなので

15 Experimental design E k = {F(f R ); θ Θ} Given {f R ; θ Θ}, can move F (experimental design) Nice if one can say {F(f R ); θ Θ} is more informative than the other Ex x = Ff R + ξ, ξ N(0, σi a ) f R : l-dim vector, x: k-dim vector, k l, F: k l real matrix Optimize F obtain most informative p R x = N(Ff R, σi a ) If N(Ff R, σi a ) is better than N(F p f R, σi a ), use F rather than F

16 Comparison of statistical experiments e = e R R h ( 普通は ε を使うが ε ほかのところでも使うので ) DEF E is e- defficinet relative to E, E u E d p d θ Θ G R d, E G R d p, E 1 2 e R holds for any g with 0 g R 1, on any D どんな E 上の decision d に対しても それを上回る E 上の decision d がある (e 程度のマージンで ) しかも それが任意の決定空間と利得で E u E E u E p & E u E p E で最適化考えても E と 体同じ If e R = 0, θ Θ, E E E E is an equivalence relation Distance measure Δ E, E p min {ε ; E u E p, e R = ε} is partial order

17 Randomization criteria (Blackwell, Le Cam) Thm E u E Λ : transition probability p θ Θ, Λ p R p R 7 e R sup } R h π R G R d, E sup R h(π R G R d, E 7 e b R) holds for any π R (non-zero at most finitely many points) for any g with 0 g R 1, on any D } This is the reason why statisticians are interested in thermo-majorization-type conditions

18 Ex Ex. Comparison of linear normal experiments x = Ff R + ξ, ξ N(0, σi a ) f R : l-dim vector, x: k-dim vector, k l, F: k l real matrix Hansen, O. and Torgersen, E. : The Annals of Statistics 1974, Vol. 2, No. 2, N Ff R, σi a ; θ Θ N F f R, σi a ; θ Θ FF ~ F p F p~ 0 ( if σ is unknown, k k p rank FF ~ F p F p~ 0, in addition) Quantum [M 2010] many extensions,e.g., about covariance matrix, also computation of Δ ρ R ; θ Θ ρr : Gaussian state covariance =Σ ρ R ; θ Θ E R trρ R X 7, tr ρ R P 7, ) = Ff R, J:matrix defining CCR C, S, Σˆ C ~ JF = JF, Σ p = S ~ ΣˆS + C ~ ΣC, Σˆ + ij 0

19 Ex Statistical Inference on Unital qubit channels Λ R : a unital channel with the unknown parameter θ F: input state + measurement Many papers, each dealing with each own setting [Fujiwara02] [Fujiwara03][Sacchi05-1][Sacchi05-2][Sacchi05-3] etc Λ R I = I Λ R But all of them say maximally entangled state is optimal Def ρ R R h I ρ R R h M p M {P Ž Œ } R h {P Ž Œ p } R h Fact [M2013] Φ: max-ent state ρ {Λ R I Φ } R h I {Λ R I ρ } R h

20 Local asymptotic normality (LAN) θ R - Once upon a time, A. Fisher insisted that maximally likelihood estimate (MLE, 最尤推定量 ) should be the best estimate, and gave a proof, using,, which would satisfy most of us, but not mathematicians - Counter example found out, even for Gaussian distributions (Hodge estimator etc) - Introduction of regularity conditions, on models and estimators - To simplify the argument, invented was LAN log p R- If θ x log p R x (J R ) 7 l R x h + h b J R 7 p R ( ) has the first derivative ( as a function onto L^2)

21 Local asymptotic normality (LAN) θ R E R : = p R- x ; h R E R : = N h, J R 7 R θ R Thm If θ p R ( ) has the first derivative (as a function on L b ) And some mild conditions on p R Δ E R, E R 0, uniformly θ in arbitray compact subset of Θ Estimation of θ on {p R } reduces to the one on N h, J R 7 R If you believe x ( N h, J R 7 ) is the best estimate of the mean h, should also believe that MLE is the best 反例は全てガウスでもなので ガウスに落としてから排除するよう条件つける Shiryaev, Spokoiny, Statistical Experiments and Decisions: Asymptotic Theory, World Scientific 2000 Ibragimov. Khasminskii, Statistical Estimation: Asymptotic Theory, Springer,1981

22 LAN First reduce to the problem to the estimation of the mean of Gaussian random variable, which is easy 7 Gaussian shift model N h, J R functions as a R standard form of parameter family k - when not iid, other standard forms, such as LAMN etc - non-parametric こういった 問題の帰着 は情報科学だといろんなところに現れる計算量理論 (comparison of hardness by reduction) 量 情報のエンタングルメントの理論 (resource theory)

23 Θ = 2, E = {p d, p 7 } Thm [Torgersen70] To check E u E, only have to check Binary decision problem, d D = {0,1} sup R h π R G R d, E sup } } c R h(π R G R d, E 7 π b R e R ) holds for any π R for any g with 0 g R 1, on D = {0,1} w.l.g., π d = 1, π 7 = t, g 7 d = 1 g d d,

24 Θ = 2, E = {p d, p 7 } Thm [Torgersen70] To check E u E, one only have to check Binary decision problem, d D = {0,1} sup } R h π R G R d, E sup (π R } c R h G R d, E 7 π b R e R ) holds π d = 1, π 7 = t, for any g with 0 g R 1, g 7 d = 1 g d d, sup } 4 π R R h G R d, E = sup } 4 p d x t p 7 x - [ 4 g d d x [p d x t p 7 (x)] [ + t + t = if g d 0 = g 7 1 = 1

25 Θ = 2, E = {p d, p 7 } Thm [Torgersen70] To check E u E, one only have to check Binary decision problem, d D = {0,1} E u E t 0 4 p d x t p 7 x - [ 4 p d x t p 7 x - [ + e d te 7 Thermo-majorization with error term When e R = 0, thermo-majorization had been known since almost half-a-century ago

26 Relation to f-divergence sup }( ) 4 π R R6d,7 G R d, E = sup }( ) 4 π R 4 g R d x p R (x) R6d,7 [ f(z) sup } = sup }( ) 4 4 π R g R d x [ R6d,7 = 4 p 7 x f p d x p 7 x [ π d g d d z + π 7 g 7 (d) p R (x) = 4 sup =: D Ÿ (p d p 7 [ } 4 π R g R d p R (x) R6d,7 f is convex and lower semi-continuous Any such f can be written in above form (use Legendre transform) f(z) = zlogz : KL-divergence (or relative entropy) z : Tsalis-Renyi-like quantity 1 z : L1-norm 1 tz - : Thermo-Majorization

27 Relation to f-divergence Thm For any g and π, there is f s.t. sup }( ) 4 π R G R d, E = D Ÿ (p d p 7 : = 4 p 7 x f(p d x /p 7 x ) R6d,7 And vice versa! Any f-divergence is optimized Bayes gain [ E u E D Ÿ (p d p 7 D Ÿ (p d p 7 holds for any lower semicont. convex function f Lem [CohenKempermanZbaganu98] Any lower-semicontinuous convex f can be : f z = a + bz + z t - dμ(t) Combination of above facts leads to another proof of Thermo-Majorization fact (i) D Ÿ (tp d tp 7 = td Ÿ (p d p 7 (ii) p d, p 7 D Ÿ (p d p 7 Λis convex (iii) D Ÿ (p d p 7 D Ÿ (Λ(p d ) Λ(p 7 ) Λ: transition probability

28 Quantum version I E = ρ R R h G R Λ, E trg R Γ(ρ R ) g R : bounded self-adjoint, on H Γ: Completely positive trace preserving map DEF E u E E is e- defficinet relative to E, Γ p Γ θ Θ G R Γ, E G R Γ p, E 1 2 e R holds for any g with 0 g R I, on any H Thm [M10] sup R h π R E u E Λ : CPTP map θ Θ, G R Γ, E sup Λ p R p R p 7 e R R h (π R G R Γ, E 7 e b R) holds for any π R (non-zero at most finitely many points) for any g with 0 g R I, on any H

29 Quantum version II Def E u I E M p M {P Ž Œ } R h u {P Ž Œ p } R h Fact E u I E Λ : positive (may not be CP) trace preserving map p θ Θ, Λ p R p R 7 e R is not true. Counter example by [M13] No good necessary and sufficient condition for I u E I u E E u E The opposite not true. But if e=0, some good relation. Thm [Buscemi 12] E d E E d I E E d E E

30 Quantum local asymptotic normality (LAN) It had been noted that ρ R and its tangent space can be approximated by Gaussian states, showing the achievable lower bound to asymptotic error bound of asymptotically unbiased estimators [M 98] [HayashiM 2002] [Hayashi 2002] E R : = ρ ; h R a R- E R : Gaussian shift (multi-mode) h R a Thm [Kahn Guta 2005] dim H Δ E R, E R 0, uniformly θ

31 Θ = 2, E = {ρ d, ρ 7 } Thm [AlbertiUhlmann85] Suppose dim H = 2 E E E I E t 0, tr ρ d t ρ 7 - tr ρ d t ρ 7 - [M10] [Jencova10] Suppose ρ d, ρ 7 = 0 E u I E t 0, tr ρ d t ρ 7 - tr ρ d t ρ (e d te 7 )

32 Relation to quantum divergence D ρ d ρ 7 sup π R trg R Γ(ρ R ) R6d,7 (i) D (tρ d tρ 7 = D (tρ d tρ 7 satisfies (ii) ρ d, ρ 7 D (ρ d ρ 7 is convex (iii) D (ρ d ρ 7 D (Λ(ρ d ) Λ(ρ 7 ), Λ: CPTP Also, any function satisfying above three is written in the RHS form If restricted to commutative ops, D p d p 7 = D Ÿ (p d p 7 ), for some f D : quantum version of f-divergence D Ÿ ρ d ρ 7 D ρ d ρ 7 D Ÿ ±² (ρ d ρ 7 ), Fact E E D (ρ d ρ 7 D (ρ d ρ 7 holds for all D with above conditions

33 Classical and Quantum Information Spectrum, Especially on divergence rate

34 Classical Information Spectrum Founded by Te-Sun Han ( 韓太舜 ) Novel frame work of information theory : no need to assume iid, memoryless, Markov, Ergodic etc. What is done: Rewrite the solutions of information theoretic problems into spectrum formulas, which is still abstract. Thus, if more concrete form is necessary, have to evaluate them further. Yet, information theoretic part finishes at this stage: the rest of the job is mathematical. Also, abstract spectrum formulas help understanding relations btw various information theoretic problems. As such, if you want to appreciate its full value, have to learn various aspects of information theory. Here treat only hypothesis test

35 Hypothesis test and divergence rate E = {p d, p 7 } : Guess which from the data x p R P 1 0}: Prob of choosing 1 while 0 is true P {0 1}: Prob of choosing 0 while 1 is true Make P {0 1} small, while keeping P {1 0} reasonably small e O P 0 1}, 1. sup {b ; P 1 0 0} b = limsup G 2. sup {b ; limsup P 1 0 1} 1 n log P {0,1}

36 Divergence rate and hypothesis test Prob (x obeys p d ) D( p d {p 7 }) D( p d {p 7 }) x 1 n log p 7 p d x When iid 1 n 4 log p 7 X 1 p d X 1 4 p d (x) [ log p d x p 7 x = D(p d p 7 ) e O P 0 1}, b = limsup G sup b ; P = D( p d {p 7 }) 1 n log P {0,1} sup b ; limsup P = D( p d {p 7 })

37 Quantum version By Hirohoshi Nagaoka( 岡浩司 )in about 1998 普通 log p/q のオペレーター版を考えたくなるが そこにハマらなかった 仮説検定の最適化の途中経過で何がポイントかを考えた の 可換版 DK({ρ d } ρ 7 D ({ρ d } ρ 7 = inf{b; lim tr ρ d e O ρ 7 G = 0} - = sup{b; lim tr ρ d e O ρ 7 G = 1} - 1. When commutative, coincide with classical version 2. equals error exponent, just as its classical version sup b ; P = D( ρ d {ρ 7 }) sup b ; limsup P = D( ρ d {ρ 7 }) 3. If ρ R = ρ R both DK and D coincide with D(ρ d ρ 7 trρ d logρ d logρ 7 D(ρ d ρ 7 = sup b ; P = sup b; limsup P Had been known [ 合 Petz 80] was not known [OgawaNagaoka 00]

38 Another representation of D: smooth Renyi In fact, spectrum-like quantity have been also used in computer science, for analysis of random number generation, security of cryptography (randomness extractor) Its quantum version by R. Renner D (ρ d ρ 7 = min{b; ρ d e O ρ 7 } D (ρ d ρ 7 = min{d (ρ d ρ 7 ; ρ p d ρ 7 ε} lim d limsup G 7 D (ρ d ρ 7 = D( ρ d {ρ 7 }) [RennarDatta 05]

39 Yet some more representations 7 lim limsup D d G (ρ d ρ 7 = D( ρ d {ρ 7 }) [RennarDatta 05] D (ρ d ρ 7 = min{d (ρ d ρ 7 ; ρ p d ρ 7 ε} 7 lim limsup D d G º (ρ d ρ 7 = D( ρ d {ρ 7 }) D º (ρ d ρ 7 = min{d º (ρ d ρ 7 ; ρ d p ρ 7 ε} D º (ρ d ρ 7 = any CPTP-monotone quantum version of α-renyi relative entropy (α>1) e.g. 7 7 º log trρ d 7 º ρ 7 º

40 on n n is merely an index. Can be any number may not be system size. It could be size of reservoir, for example

41 When DK and D º (ρ d ρ 7 coincide with D? Obviously, if iid. In classical case, when n is the system size, and the system is Markovian etc, (Maybe, when the systems are not globally correlated) Probability D( p d {p 7 }) D( p d {p 7 }) x 1 n log p 7 p d x

42 An application : resolvability [Ogawa M] x ² 7, x ² b x»¼ p 7» ¼ ρ ¼» [½ ¼ 167 will be very close to E ρ [i ¼ if L is large enough. How large is enough large?» ¼ E ² 1 4 ρ ¼ L [½ 167 E ρ [i ¼ 7 0 holds if limsup ² G 7 log L sup D K({ρ [ [ ¼ ¼} {E ρ [i ¼})

43 Putting altogether : Divergence rate and Thermo-Majorization

44 Classical case: Θ = 2, E = {p d, p 7 } By Thermo-Majorization with error E = {p d, p 7 } u¼ E p = {q d, q 7 } lim e,d = 0, e,7 = 0 (Λ p d q d, Λ p 7 = q 7 ) Thm b 4 p d x e O p 7 x 4 q d x e O q 7 x - - [ もしも寝ていなければ以下が える筈 E u¼ E p [ D ({p d } p 7 DK({q d } q 7 D K({p d } p 7 DK({q d } q 7 D({p d } p 7 D({q d } q 7 + e,d DK({ρ d } ρ 7 D ({ρ d } ρ 7 = inf{b; lim tr ρ d e O ρ 7 = 0} G - = sup{b; lim tr ρ d e O ρ 7 = 1} G -

45 Classical case: Θ = 2, E = {p d, p 7 } Fact E = {p d, p 7 } u¼ E p = {q d, q 7 } liminf e,d 2, e,7 = 0 DK({p d } p 7 D({q d } q 7 Λ p d q d 7 2, 完全に区別できるほどには悪くない Λ p 7 = q 7 DK({ρ d } ρ 7 D ({ρ d } ρ 7 = inf{b; lim tr ρ d e O ρ 7 = 0} G - = sup{b; lim tr ρ d e O ρ 7 = 1} G -

46 Standard form: Θ = 2, E = {ρ d, ρ 7 } 古典のときみたいに安直じゃないので頭捻る Standard form of binary experiments (dichotomies) {r d, r 7 } : probability distributions over {0,1} lim r d 0 = 1 lim r 7 0 = 0, and exponentially r 7 0 e O 0 1 hypothesis test = conversion from E = {ρ d, ρ 7 } to {r d, r 7 } Among various conversions, Choose the one maximizing b max b = D( ρ d ρ 7

47 Standard form: Θ = 2, E = {ρ d, ρ 7 } It is natural to consider conversion from {r d, r 7 } to E = {ρ d, ρ 7 } Lem If c = exp( D (ρ d ρ 7 ), There is a states τ d, τ 7 with τ d ρ d 7 ε cτ d + (1 c)τ 7 = ρ 7 Proof : obvious from the def. D (ρ d ρ 7 = min{b; ρ d e O ρ 7 } D (ρ d ρ 7 = min{d (ρ d ρ 7 ; ρ p d ρ 7 ε} strategy By mixing τ d, τ 7 with probability r R (0), r R (1), obtain ρ d, ρ 7 Thm There is a CPTP map Λ with Λ r d ρ d 7 0, Λ r 7 = ρ 7 If r d 0 0, r 7 0 = e O, 1 b > lim limsup d G n D (ρ d ρ 7 = DK({ρ d } ρ 7

48 Θ = 2, E = {ρ d, ρ 7 } E u¼ E p lim e,d = 0, e,7 = 0 (Λ ρ d σ d, Λ ρ 7 = ρ 7 ) E = {ρ {r d, r d, ρ 7 } 7 } E = {σ d, σ 7 } Thm E u¼ E p D ({ρ d } ρ 7 DK({σ d } σ 7 D K({ρ d } ρ 7 DK({σ d } σ 7 D({ρ d } ρ 7 D({σ d } σ 7 If D = DK = D, only have to compare D Can prove uniqueness of quantum extension of relative entropy, which is smooth and CPTP monotone

49 Stability Thm If D (Λ(ρ) Λ σ D (ρ σ 1 D(ρ σ) = clim lim d G n inf { D ρ σ ; ρ p ρ ε } 7 おしまい