Tractability of Multivariate Problems

Transcription

1 Erich Novak University of Jena Chemnitz, Summer School

2 Plan for the talk Example: approximation of C -functions What is tractability? Tractability by smoothness? Tractability by sparsity, finite order weights, or structure Tractability by randomization Integral equations Markov chain Monte Carlo Optimal importance sampling Chemnitz, Summer School

3 Example: approximation of C -functions Approximation of f C k ([0, 1] d ) by linear algorithms S n (f) = n L i (f) g i. i=1 Optimal methods: order of convergence is n k/d, error in L. The order is excellent if k/d is large. Does it mean that the problem is easy? What about k =? We also will allow nonlinear methods S n = φ N with continuous N : C k R n, φ : R n L. Chemnitz, Summer School

4 A class of very smooth functions F d = {f : [0, 1] d R D α f 1 for all α N d 0 }. The class is small, error bounds should be excellent. S n = φ N with continuous N : F d R n, φ : R n L, e(s n ) = sup f F d f S n (f), e(n, d) = inf S n e(s n ), n(ε, d) = inf{n e(n, d) ε}. Well known: For any d and r > 0 e(n, d) = O(n r ), n(ε, d) = O(ε 1/r ). Conventional conclusion: The problem is easy since the order of convergence is excellent. Chemnitz, Summer School

5 Tractability Information Complexity n(ε, d) = inf{n e(n, d) ε}. The problem is strongly polynomially tractable iff n(ε, d) C ε p for all ε (0, 1), d N. The problem is polynomially tractable iff n(ε, d) C d q ε p for all ε (0, 1), d N. The problem is weakly tractable iff lim ε 1 +d ln n(ε, d) ε 1 + d Introduced by Woźniakowski, 2 papers in = 0. Chemnitz, Summer School

6 Result N. & Woźniakowski, 2009 For L -approximation over F d we have e(n, d) = 1 for all n 2 d/2 1 or n(ε, d) 2 d/2 for all ε (0, 1). The problem is intractable. Chemnitz, Summer School

7 Proof Take s = d/2 and consider f : [0, 1] d R, f(x) = i {0,1} s a i (x 1 + x 2 ) i1 (x 3 + x 4 ) i2...(x 2s 1 + x 2s ) is. The space V d of such functions has dimension 2 s and f = sup α D α f for all f V d. For continuous N : V d R 2s 1, there is a f V d with f = 1 such that N(f) = N( f); follows from the Borsuk-Ulam Theorem. Hence S n (f) = φ(n(f)) = S n ( f) and e(s n ) 1 for n = 2 s 1. Chemnitz, Summer School

8 Tractability by smoothness assumptions? Usually, we cannot obtain tractability even by strong smoothness assumptions, see the L approx. problem for C functions. Sometimes: yes. Tractability of star discrepancy Can we compute I d (f) = [0, 1] d f(x) dx for f : [0, 1] d R from F d in polynomial time, i.e., cost(ε, F d ) C ε α d β? Chemnitz, Summer School

9 Star-discrepancy disc ({t 1,...,t n }) of t i [0, 1] d : sup x [0,1] d x 1 x d 1 n n 1 [0,x) (t i ) i=1 Sobolev space (or functions with bounded variation) F 1 = {f : [0, 1] R f(1) = 0, f L 1 }, f = f L1 and F d = F 1 F 1. Hlawka-Zaremba-equality yields where Q n (f) = 1 n n i=1 f(t i). disc ({t 1,...,t n }) = sup I d (f) Q n (f), f 1 Chemnitz, Summer School

10 The star-discrepancy is tractable Heinrich, N., Wasilkowski, Woźniakowski (2001) n(ε, F d ) C d ε 2. The dependence on d is optimal since n(ε, F d ) c d log(ε 1 ). Improved lower bound n(ε, F d ) c d ε 1, Hinrichs (2004). Chemnitz, Summer School

11 Sparsity or partially separable functions A function f : [0, 1] d R of many variables (d large) may be a sum of functions, that only depend on k variables (k small): f(x 1, x 2,...,x d ) = l g l (x i1, x i2,...,x ik ). In optimization such functions are called partially separable. See, e.g., N. & Ritter (1997), Dick, Sloan, Wang, Woźniakowski (2006). Important for applications, Coulomb potential... As a rule: Problems are tractable for such functions (with k fixed and d ), even if the g l are not very smooth. Chemnitz, Summer School

12 Weighted Sobolev Spaces Unit ball of the space H d,γ given by all f : [0, 1] d R with f 2 = u [d] γ 1 d,u [0,1] d ( ) u 2 f(x) dx 1 x u 0 0 = 0, where [d] := {1, 2,...,d} and γ = {γ d,u } are non-negative weights. Results for L 2 approximation for linear (or continuous) information Λ all and for function values Λ std : For equal weights γ d,u = 1 the problem is weakly tractable for Λ all and not weakly tractable for Λ std. For bounded finite order weights (γ d,u = 0 if u > k) the problem is always polynomially tractable, even for Λ std. Werschulz & Woźniakowski (2009) Chemnitz, Summer School

13 Various Weights Product weights: γ d,u = j u γ d,j. Then H(K d,γ ) = H(K 1,γd,1 ) H(K 1,γd,d ) and γ d,j moderates the influence of x j Finite-order weights: γ d,u = 0 for all u > k. Then f = u [d], u k is a sum of functions depending on at most k variables. f u We can model various properties of f by suitable weights. Chemnitz, Summer School

14 Results for Integration For product weights: γ d,u = j u γ d,j Strong Pol. Tract. iff lim sup d d j=1 γ d,j < Pol. Tract. iff lim sup d P d j=1 γ d,j ln d < Weak Tract. iff lim d P d j=1 γ d,j d = 0 For finite-order weights: γ d,u = 0 for all u > k always polynomially tractable for k 1 and γ d,u = 1 for u k: not strongly polynomially tractable. N. & Woźniakowski (2001, 2010), Sloan & Woźniakowski (1998, 2002), Gnewuch & Woźniakowski (2008) Chemnitz, Summer School

15 Tractability by randomization Integral equations Markov chain Monte Carlo Optimal importance sampling Chemnitz, Summer School

16 Solving Integral Equations with Random Bits Compute u(s), integral equation u(x) k(x, y)u(y) dy = f(x) [0,1] d on [0, 1] d with Lipschitz kernel k, k < α < 1 and right hand side. Optimal order with MC (Heinrich & Mathé 1993) e n n 1/2 1/(2d). N. & Pfeiffer (2004): With a discretized version of classical MC and results for summation we get the upper bound cost ε 2 + d (log ε 1 ) 2, only d (log ε 1 ) 2 random bits are needed. Problem is intractable for deterministic algorithms. Chemnitz, Summer School

17 Markov chain Monte Carlo Computation of E π (f) (expectation with respect to π) with a Markov chain Monte Carlo method and burn in n 0, A n,n0 (f) = 1 n n f(x k+n0 ), when it is not possible to simulate π directly. We assume that the Markov chain (X k ) is reversible and has an L 2 -spectral gap, β = P E π L2 L 2 < 1. k=1 Then it is known (ergodic theorem) that A n,n0 (f) E π (f). Error bounds? How should we choose the burn in? Chemnitz, Summer School

18 Result of Rudolf 2009 For f L p (π) with p 4 and ( ) n 0 (1 β) 1 dµ log dπ 1 the error is bounded by sup e µ (A n,n0, f) 2 f p 1 2 n(1 β) + 46 n 2 (1 β) 2. Here µ is the initial distribution, i.e., the distribution of X 1. The cost bound does not depend on d, but β = β(d) might depend on d. One obtains different tractability results depending on the behavior of β = β(d). Explicit error bounds and a recipe for the choice of n 0. Chemnitz, Summer School

19 Optimal importance sampling I(f) = f(x) (x) dx for f H, H a RKHS with R d I 2 = K(x, y) (x) (y) dxdy <. R d R d Randomized error e(a n ) = sup f H 1(E(I(f) A n (f)) 2 ) 1/2. Hinrichs 2010: If K(x, y) 0 then with importance sampling ( π ) 1/2 e(a n ) n 1/2 I. 2 Hence such problems are strongly polynomially tractable. N. and Woźniakowski (2010): under some additional assumptions, the algorithm of Hinrichs is optimal. Chemnitz, Summer School

20 Summary Many problems for functions f : [0, 1] d R are intractable, if considered in the worst case setting for classical function spaces, like C k ([0, 1] d ). Remedies: Weighted spaces, problems with a structure Randomized algorithms Chemnitz, Summer School