Estimation of probability density functions by the Maximum Entropy Method

Transcription

1 Estimation of probability density functions by the Maximum Entropy Method Claudio Bierig, Alexey Chernov Institute for Mathematics Carl von Ossietzky University Oldenburg 3. Workshop of the GAMM AGUQ Dortmund, 4 March 28

2 Forward uncertainty propagation in complex systems: X = S(ξ) ξ = input (random variable, known pdf) X = output (random variable, unknown pdf ρ) S = System s action (deterministic, known) Aim: determine the pdf ρ of X (nonintrusively!) Note: only a finite amount of information can be pushed through S. 2

3 Outline: Methodology Error Analysis Experiments (i.a. rough random obstacle problem) 3

4 Idea : ( Fabio s talk on Monday) Determine point values ρ(x i ), for x < x < < x N Interpolate for x (x i, x i+ ) M. B. Giles, T. Nagapetyan, and K. Ritter, Multilevel Monte Carlo Approximation of Distribution Functions and Densities, SIAM/ASA J. Uncert. Quantif., 3 (25), pp Adv.: More point values better approximation (stability, conv.). Drawback: Many unknown parameters in the algorithm (complex). See also the antiderivative approach : S. Krumscheid and F. Nobile, Multilevel Monte Carlo approximation of functions, MATHICSE technical report Nr

5 Idea 2 (explore in this talk): Determine the (generalized) moments µ,..., µ R of X µ k = E[φ k (X )] Reconstruct η that satisfies the moment constraints: a) µ k = φ k (x)η(x) dx b) η(x) and η(x) dx =. C. Bierig and A. Chernov, Approximation of probability density functions by the Multilevel Monte Carlo Maximum Entropy method, J. Comput. Physics, 34 (26), a a Based on earlier works [Csiszár 75], [Barron, Sheu 9], [Borwein, Lewis 9] Advantage: Only a few parameters (simple) Drawback: More moments better approximation? (stability?). 5

6 Observe: If the moments µ,..., µ R are consistent, the reconstructed density η is usually not uniquely determined! How to select the most appropriate density? The Maximum Entropy (ME) method: ( Find ρ R = argmax η a) µ k = φ k (x)η(x) dx, b) η(x) and η(x) dx =. ) η(x) ln η(x) dx under constraints: k =,..., R It is least biased estimate possible on the given information; i.e., it is maximally noncommittal with regard to missing information. [E.T. Jaynes, 957] 6

7 The Maximum Entropy (ME) method: The solution to this problem can be equiv. characterized as ( R ) ρ R exp λ k φ k (x), λ k R. k= where λ,..., λ R satisty the constraints (moment matching): µ k = φ k (x)ρ R (x) dx, k =,..., R, with µ =, φ (x) =. 7

8 In this sense: Entropy maximization moment matching 8

9 Test example: ρ is the log-normal distribution with µ = and σ =.5 and.2 Estimation of moments µ,..., µ R by MC with 8 samples λ = (λ,..., λ R ) determined by the Newton-Raphson method φ k (x)ρ R (λ, x) dx = µ k, k =,... R. Stopping parameters for the Newton-Raphson Method: λ 9 (convergence) λ 3 (no convergence) #iter (no convergence) 9

10 R=,Entropy=.249, λ=7.384e Monomial Moments: Unstable for R 5

11 R=2,Entropy=.8777, λ=8.3457e Monomial Moments: Unstable for R 5

12 R=3,Entropy=.842, λ=8.64e Monomial Moments: Unstable for R 5

13 R=4,Entropy=.7628, λ=6.9344e Monomial Moments: Unstable for R 5

14 .9 R=5,Entropy=2.4849, λ= Monomial Moments: Unstable for R 5

15 R=6,Entropy=2.4849, λ= Monomial Moments: Unstable for R 5

16 R=,Entropy=.249, λ=9.5939e Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

17 .9 R=2,Entropy=.8777, λ=8.2e Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

18 R=3,Entropy=.842, λ=5.724e Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

19 R=4,Entropy=.7628, λ=6.985e Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

20 R=6,Entropy=.73853, λ=2.3278e Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

21 R=7,Entropy=.7354, λ=7.75e Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

22 R=9,Entropy=.7398, λ= Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

23 R=,Entropy=.7467, λ= Legendre Moments: Stable for R 8 Entropy is may decrease even when the Newton Method does not converge

24 R=,Entropy=2.85,.8 λ=7.2644e Fourier Moments: Stable Entropy is monotonously decreasing 2

25 R=2,Entropy=.8924, λ=7.7399e Fourier Moments: Stable Entropy is monotonously decreasing 2

26 R=3,Entropy=.89239, λ=6.5837e Fourier Moments: Stable Entropy is monotonously decreasing 2

27 R=4,Entropy=.76457, λ=8.9486e Fourier Moments: Stable Entropy is monotonously decreasing 2

28 R=5,Entropy=.7393,.8 λ=5.7226e Fourier Moments: Stable Entropy is monotonously decreasing 2

29 R=6,Entropy=.73924,.8 λ=6.765e Fourier Moments: Stable Entropy is monotonously decreasing 2

30 R=8,Entropy=.736, λ=8.4948e Fourier Moments: Stable Entropy is monotonously decreasing 2

31 R=9,Entropy=.72997, λ=9.555e Fourier Moments: Stable Entropy is monotonously decreasing 2

32 R=3,Entropy=.72736, λ=2.9376e Fourier Moments: Stable Entropy is monotonously decreasing 2

33 R=4,Entropy=.7267, λ=5.292e Fourier Moments: Stable Entropy is monotonously decreasing 2

34 R=5,Entropy=.72665, λ=4.7698e Fourier Moments: Stable Entropy is monotonously decreasing 2

35 R=7,Entropy=.72628, λ=7.75e Fourier Moments: Stable Entropy is monotonously decreasing 2

36 R=8,Entropy=.7263, λ=7.966e Fourier Moments: Stable Entropy is monotonously decreasing 2

37 R=9,Entropy=.7268, λ=6.78e Fourier Moments: Stable Entropy is monotonously decreasing 2

38 .8 R=2,Entropy=.726, λ=8.43e Fourier Moments: Stable Entropy is monotonously decreasing 2

39 Breaking convergence for the Fourier basis by choosing a more concentrated density! e.g. log-normal with µ =, σ =.2 skip 3

40 R=,Entropy=2.295, λ=6.432e Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

41 R=2,Entropy=-.684, λ=6.29e Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

42 R=3,Entropy=-.9, λ=4.9765e Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

43 2.5 R=4,Entropy=-.92, λ=7.8862e Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

44 R=5,Entropy=-.92, λ=8.2299e Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

45 R=6,Entropy=-.948, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

46 2.5 R=8,Entropy=-.954, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

47 R=9,Entropy=-.95, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

48 R=3,Entropy=-.955, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

49 R=4,Entropy=-.955, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

50 R=5,Entropy=-.955, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

51 R=6,Entropy=-.955, λ= Fourier Moments (σ =.2, [a, b] = [, 2]): Remain stable even without convergence! Entropy is still monotonously decreasing! 4

52 Regain stability of the Legendre basis by choosing a smaller approximation interval! e.g. [a, b] = [, 4] 5

53 R=,Entropy=.99553, λ=5.2639e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

54 2.5 2 R=2,Entropy=-.65, λ=6.6e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

55 2.5 2 R=3,Entropy=-.7783, λ=5.9556e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

56 2.5 2 R=4,Entropy=-.9, λ=.527e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

57 2.5 R=5,Entropy=-.934, λ=7.46e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

58 2.5 2 R=6,Entropy=-.954, λ=8.576e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

59 2.5 2 R=8,Entropy=-.955, λ=2.455e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

60 2.5 2 R=9,Entropy=-.955, λ=3.6249e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

61 2.5 R=3,Entropy=-.955, λ=2.6296e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

62 2.5 R=4,Entropy=-.955, λ=4.482e Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

63 2.5 R=5,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

64 2.5 R=6,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

65 2.5 R=7,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

66 2.5 R=9,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

67 2.5 R=2,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4]): Still quite stable even without convergence! Entropy is still monotonously decreasing! 6

68 R=,Entropy=.99553, λ=5.2639e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

69 R=2,Entropy=-.65, λ=6.6e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

70 R=3,Entropy=-.7783, λ=5.9556e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

71 R=4,Entropy=-.9, λ=.527e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

72 R=5,Entropy=-.934, λ=7.46e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

73 R=6,Entropy=-.954, λ=8.576e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

74 R=8,Entropy=-.955, λ=2.455e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

75 R=9,Entropy=-.955, λ=3.6249e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

76 R=3,Entropy=-.955, λ=2.6296e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

77 R=4,Entropy=-.955, λ=4.482e Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

78 R=5,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

79 R=6,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

80 R=7,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

81 R=9,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

82 R=2,Entropy=-.955, λ= Legendre Moments (σ =.2, [a, b] = [, 4], semilog): Oscillations in the negative domain stability of the density 7

83 2.5 2 R=2,Entropy=-.675, λ=5.766e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

84 2.5 2 R=3,Entropy=-.834, λ=9.6224e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

85 2.5 2 R=4,Entropy=-.9, λ=5.68e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

86 2.5 2 R=5,Entropy=-.943, λ=5.9985e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

87 2.5 2 R=6,Entropy=-.954, λ=6.864e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

88 2.5 2 R=8,Entropy=-.955, λ=7.785e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

89 2.5 2 R=9,Entropy=-.955, λ=6.53e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

90 2.5 R=3,Entropy=-.955, λ=8.749e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

91 2.5 R=4,Entropy=-.957, λ=6.3789e Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

92 2.5 R=5,Entropy=-.972, λ= Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

93 2.5 R=6,Entropy=-.979, λ= Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

94 2.5 R=7,Entropy=-.962, λ= Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

95 2.5 2 R=9,Entropy=-.9, λ= Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

96 2.5 R=2,Entropy=-.8995, λ= Legendre Moments (σ =.2, [a, b] = [.3, 3]): Is stable and convergent for a bigger range R 4! Entropy is monoton. decr. (stat. accuracy of MC is reached?) 8

97 Observation: µ k are not known exactly, we only have µ k µ k, k =,..., R (approx. moments µ k may be computed e.g. by MC or MLMC). Moment matching with µ k perturbed density { ( R ) } ρ R E := p θ exp θ k φ k (x), θ k R. k= Question: What is the distance between ρ and ρ R? 9

98 Consider the natural metric D KL (p q) := p(x) ln p(x) q(x) dx ( Kullback-Leibler divergence or relative entropy ) Pythagoras theorem D KL (ρ ρ R ) = D KL (ρ ρ R ) + D }{{} KL (ρ R ρ R ). }{{} trunc. error estim. error ρ is the exact density ρ R E is the ME solution for exact moments µ,..., µ R ρ R E is the ME solution for approx. moments µ,..., µ R Truncation error: D KL (ρ ρ R ) when R Estimation error: D KL (ρ R ρ R ) when µ k µ k. Our aim is the rigorous quantification of these statements. 2

99 From D KL to L p -norms: Relation to L p -norms i) 2 ρ η 2 L D KL (ρ η); ii) C p ρ η p L p D KL (ρ η), ρ, η L. Relation to L 2 -norms of the log-density For two pdf s ρ and η with ln(ρ/η) L D KL (ρ η) 2 e ln(ρ/η) L ln(ρ/η) 2 ρ dx and D KL (ρ η) 2 e ln(ρ/η) c L ln(ρ/η) c 2 ρ dx (#) for any c R. 2

100 From D KL to L p -norms: Relation to L p -norms i) 2 ρ η 2 L D KL (ρ η); ii) C p ρ η p L p D KL (ρ η), ρ, η L. Relation to L 2 -norms of the log-density For two pdf s ρ and η with ln(ρ/η) L D KL (ρ η) 2 e ln(ρ/η) L ln(ρ/η) 2 ρ dx and D KL (ρ η) 2 e ln(ρ/η) c L ln(ρ/η) c 2 ρ dx (#) for any c R. 2

101 Truncation error is driven by the smoothness of ρ: polynomial moments span{, φ,..., φ R } = P R polynomial best approximation Conv. of the truncation error D KL (ρ ρ R ) 2 inf v P R { [ exp { ln ρ v L } ln ρ v 2 L 2 (ρ) R 2s, when ln ρ H s, s exp( br), when ln ρ is analytic. ] Proof: Choose η = e v / e v for v P R in (#). 22

102 Truncation error is driven by the smoothness of ρ: polynomial moments span{, φ,..., φ R } = P R polynomial best approximation Conv. of the truncation error D KL (ρ ρ R ) 2 inf v P R { [ exp { ln ρ v L } ln ρ v 2 L 2 (ρ) R 2s, when ln ρ H s, s exp( br), when ln ρ is analytic. ] Proof: Choose η = e v / e v for v P R in (#). 22

103 Estimation error is driven by the perturbation of moments: Conv. of the estimation error (a.s.-version) Suppose µ µ (2A R C R ) where A R, C R are explicit constants, then ρ R exists and there holds D KL (ρ R ρ R ) C R µ µ 2 23

104 Estimation error is driven by the perturbation of moments: Conv. of the estimation error (a.s.-version) Suppose µ µ (2A R C R ) where A R, C R are explicit constants, then ρ R exists and there holds D KL (ρ R ρ R ) C R µ µ 2 A R = max { v L / v L 2 : v P R } and CR = 2 exp { + ln ρ R }. 23

105 Estimation error is driven by the perturbation of moments: Conv. of the estimation error (a.s.-version) Suppose µ µ (2A R C R ) where A R, C R are explicit constants, then ρ R exists and there holds D KL (ρ R ρ R ) C R µ µ 2 Caution: If µ k is computed by MC ρ R may fail to exist (with some prob.) 23

106 Estimation error is driven by the perturbation of moments: Conv. of the estimation error (probabilistic version, simplified) Suppose E µ µ 2 (2A R C R ) 2 with the same A R, C R, then ρ R exists with prob. p (for p [p, ]) and it holds D KL (ρ R ρ R ) C R p E µ µ 2 ( ) 23

107 Estimation error is driven by the perturbation of moments: Conv. of the estimation error (probabilistic version, simplified) Suppose E µ µ 2 (2A R C R ) 2 with the same A R, C R, then ρ R exists with prob. p (for p [p, ]) and it holds D KL (ρ R ρ R ) C R p E µ µ 2 ( ) Here E µ µ 2 = R E µ k µ k 2 }{{} k= standard MSE( µ k ) can be handled by the standard (ML)MC theory 23

108 Structure of the estimate: D KL (ρ ρ R ) = D KL (ρ ρ R ) + D KL (ρ R ρ R ) assuming ln ρ H D KL (ρ ρ R ) polynomial best approx. + C R p assuming R k= MSE( µ k ) { ln ρ H s, s ln ρ is analytic } { D KL (ρ ρ R ) R 2s exp( br) } + C R p R Bias( µ k ) 2 + Var( µ k ) k= C R is unif. bounded when ln ρ H s, s > Error vs. cost theorems balancing the error contributions 24

109 Structure of the estimate: D KL (ρ ρ R ) = D KL (ρ ρ R ) + D KL (ρ R ρ R ) assuming ln ρ H D KL (ρ ρ R ) polynomial best approx. + C R p assuming R k= MSE( µ k ) { ln ρ H s, s ln ρ is analytic } { D KL (ρ ρ R ) R 2s exp( br) } + C R p R Bias( µ k ) 2 + Var( µ k ) k= C R is unif. bounded when ln ρ H s, s > Error vs. cost theorems balancing the error contributions 24

110 Structure of the estimate: D KL (ρ ρ R ) = D KL (ρ ρ R ) + D KL (ρ R ρ R ) assuming ln ρ H D KL (ρ ρ R ) polynomial best approx. + C R p assuming R k= MSE( µ k ) { ln ρ H s, s ln ρ is analytic } { D KL (ρ ρ R ) R 2s exp( br) } + C R p R Bias( µ k ) 2 + Var( µ k ) k= C R is unif. bounded when ln ρ H s, s > Error vs. cost theorems balancing the error contributions 24

111 Structure of the estimate: D KL (ρ ρ R ) = D KL (ρ ρ R ) + D KL (ρ R ρ R ) assuming ln ρ H D KL (ρ ρ R ) polynomial best approx. + C R p assuming R k= MSE( µ k ) { ln ρ H s, s ln ρ is analytic } { D KL (ρ ρ R ) R 2s exp( br) } + C R p R Bias( µ k ) 2 + Var( µ k ) k= C R is unif. bounded when ln ρ H s, s > Error vs. cost theorems balancing the error contributions 24

112 Theorem (Monte Carlo-ME: Accuracy / Cost, simplified) Assume in addition that a) E[(X X l ) 2 ] N β l, b) Cost(X l ) N γ l Then D KL (ρ ρ R ) < ε with probability p and for the single level approximation of Legendre moments: ε β+γ(δ+) 2sβ if ln(ρ) H s, Cost( ρ R ) (pε) β+γ β ln(ε) β+γ(δ+) β multilevel approximation of Legendre moments: Cost( ρ R ) (pε) max(β,γ) β if ln(ρ) is analytic; (δ+) max(β,γ) ε 2sβ if ln(ρ) H s, (δ+) max(β,γ) ln(ε) β if ln(ρ) is analytic. Here δ is such that E[(φ k (X ) φ k (X l )) 2 ] k δ N β l and β γ. 25

113 Contact with rough surfaces ψ(x) Courtesy: Prof. Udo Nackenhorst, IBNM, Univ. Hannover Unknown parameter: ψ(x) is the road surface profile. (irregular microstructure) 26

114 Model: Contact of an elastic membrane with a rough surface (2d) u f, u ψ, ( u + f )(u ψ) =, } in D, u = on D. ( here D = [, ] 2 ) QoI: Membrane deformation u(x); Contact Area Λ(ω) = {x : u(x) = ψ(x)}. 27

115 Example: Rough obstacle models Power spectrum [Persson et al. 5]: ψ(x) = B q (H) cos(q x + ϕ q ) q q q s where B q(h) = π 5 (2π max( q, q l )) H Many materials in Nature and technics obey this law for amplitudes. B q(h), q =, q l =, q s =26 H= H=.5 2 H= H U(, ) random roughness ϕ q U(, 2π) random phase Forward solver: Own implementation of MMG (TNNM) [Kornhuber 94,... ]

116 Obstacle surfaces of variable/random roughness ψ = ψ(x, ω): 29

117 Estimation of the PDF ρ X of the contact area X = Λ by the Maximum Entropy method 4 ME r = The peak(s) corresponds to ca. 28.2% of the membrane in contact with the surface More experiments and rigorous error analysis in [Bierig/Chernov, JCP, 26]. 3

118 Estimation of the PDF ρ X of the contact area X = Λ by the Maximum Entropy method 4 ME r = 6 KDE r = The peak(s) corresponds to ca. 28.2% of the membrane in contact with the surface More experiments and rigorous error analysis in [Bierig/Chernov, JCP, 26]. 3

119 Estimation of the PDF ρ X of the contact area X = Λ by the Maximum Entropy method 38 L2 Projektion R=2, MLMC L=9 ME R=5, MLMC L=9 kde l=4 37 kde l=5 kde l= The peak(s) corresponds to ca. 28.2% of the membrane in contact with the surface More experiments and rigorous error analysis in [Bierig/Chernov, JCP, 26]. 3

120 Summary: Rigorous convergence analysis for the MLMC-Maximum Entropy Method for compactly suported densities Num. experiments: smoothness assumptions may be relaxed Open issue: how to select the number of moment constraints R in practical computations (in the pre-asymptotic regime)? Adaptivity? With appropriate R the method is able to produce good approximations for a broad class of densities. Thank you for your attention! 3

121 Summary: Rigorous convergence analysis for the MLMC-Maximum Entropy Method for compactly suported densities Num. experiments: smoothness assumptions may be relaxed Open issue: how to select the number of moment constraints R in practical computations (in the pre-asymptotic regime)? Adaptivity? With appropriate R the method is able to produce good approximations for a broad class of densities. Thank you for your attention! 3

122 Summary: Rigorous convergence analysis for the MLMC-Maximum Entropy Method for compactly suported densities Num. experiments: smoothness assumptions may be relaxed Open issue: how to select the number of moment constraints R in practical computations (in the pre-asymptotic regime)? Adaptivity? With appropriate R the method is able to produce good approximations for a broad class of densities. Thank you for your attention! 3

123 Summary: Rigorous convergence analysis for the MLMC-Maximum Entropy Method for compactly suported densities Num. experiments: smoothness assumptions may be relaxed Open issue: how to select the number of moment constraints R in practical computations (in the pre-asymptotic regime)? Adaptivity? With appropriate R the method is able to produce good approximations for a broad class of densities. Thank you for your attention! 3

124 Summary: Rigorous convergence analysis for the MLMC-Maximum Entropy Method for compactly suported densities Num. experiments: smoothness assumptions may be relaxed Open issue: how to select the number of moment constraints R in practical computations (in the pre-asymptotic regime)? Adaptivity? With appropriate R the method is able to produce good approximations for a broad class of densities. Thank you for your attention! 3

125 References: C. Bierig and A. Chernov, Approximation of probability density functions by the Multilevel Monte Carlo Maximum Entropy method, J. Comput. Physics, 34 (26), M. B. Giles, T. Nagapetyan, and K. Ritter, Multilevel Monte Carlo Approximation of Distribution Functions and Densities, SIAM/ASA J. Uncert. Quantif., 3 (25), pp

126 Exact synthetic pdf s ρ,..., ρ 5 33 Numerical examples for synthetic pdf s of different smoothness: 2.5 ρ ρ 2 ρ 3 ρ 4 ρ

127 Numerical examples for synthetic pdf s of different smoothness: Maximum Entropy KDE r = ln ρ is analytic: smoothness assumptions are satisfied Best maximum entropy approximation over R =,..., 6 moments 33

128 Numerical examples for synthetic pdf s of different smoothness: 2 MISE Maximum Entropy KDE r = KDE r = 2 KDE r = 3 KDE r = 4 KDE r = Cost ln ρ is analytic: smoothness assumptions are satisfied Best maximum entropy approximation over R =,..., 6 moments 33

129 Numerical examples for synthetic pdf s of different smoothness: Maximum Entropy KDE r = ln ρ 2 H /2 ɛ (, ): smoothness assumptions are not satisfied Best maximum entropy approximation over R =,..., 6 moments 33

130 Numerical examples for synthetic pdf s of different smoothness: Maximum Entropy KDE r = KDE r = 2 KDE r = 3 KDE r = 4 KDE r = 5 MISE Cost ln ρ 2 H /2 ɛ (, ): smoothness assumptions are not satisfied Best maximum entropy approximation over R =,..., 6 moments 33

131 Numerical examples for synthetic pdf s of different smoothness: 2 Maximum Entropy KDE r = ln ρ 5 is unbounded: smoothness assumptions are not satisfied Best maximum entropy approximation over R =,..., 6 moments 33

132 Numerical examples for synthetic pdf s of different smoothness: MISE 2 3 Maximum Entropy KDE r = KDE r = 2 KDE r = 3 KDE r = 4 KDE r = Cost ln ρ 5 is unbounded: smoothness assumptions are not satisfied Best maximum entropy approximation over R =,..., 6 moments 33

133 Estimation of the PDF ρ X of the contact area X = Λ by the Maximum Entropy method 4 ME r = The peak(s) corresponds to ca. 28.2% of the membrane in contact with the surface More experiments and rigorous error analysis in [Bierig/Chernov, JCP, 26]. 34

134 Estimation of the PDF ρ X of the contact area X = Λ by the Maximum Entropy method 4 ME r = 6 KDE r = The peak(s) corresponds to ca. 28.2% of the membrane in contact with the surface More experiments and rigorous error analysis in [Bierig/Chernov, JCP, 26]. 34