Model reduction of Multiscale Models using Kernel Methods

Size: px

Start display at page:

Download "Model reduction of Multiscale Models using Kernel Methods"

Lester Higgins
6 years ago
Views:

Model reduction of Multiscale Models using Kernel Methods at the WORM 213 Haus der Kirche, Bad Herrenalb Daniel Wirtz August 23, 213 Dipl. Math. Dipl. Inf.

1 Model reduction of Multiscale Models using Kernel Methods at the WORM 213 Haus der Kirche, Bad Herrenalb Daniel Wirtz August 23, 213 Dipl. Math. Dipl. Inf. Daniel Wirtz, IANS, University of Stuttgart Jun.-Prof. Dr. Bernard Haasdonk, IANS, University of Stuttgart 1/52

2 Introduction Contents Introduction Kernel Expansion construction Support Vector Machines Vectorial Greedy Algorithms Experimental results Human spine simulation (N. Karajan) Shock waves in porous media (F. Kissling) 2/52

3 Introduction Up Next Introduction 3/52

4 Introduction Reduction setting C Macro- and D microscale model. Interaction parameters from domains P C, P D One macrolevel step requires many microlevel solves Each simulation of D is actually a function evaluation x P D y start C D y P C Figure: Communication of macro and microscale models f : P D P C x P D (x). Goal: Find a surrogate D, i.e. f f Figure: Example multibody system C with many microscale models D i 4/52

5 Introduction Buzzwords! Response Surface Methodology [Myers et al.(29)myers, Montgomery & Anderson-Cook, Khuri & Mukhopadhyay(21)] Surrogate Modeling Figure: Illustrations of response surfaces 5/52

6 Introduction Building surfaces How to build a surrogate or response surface? Classically via polynomials Interpolation, Regression Multivariate variants Polynomial Chaos Expansions [Wiener(1938), Karajan et al.(213)karajan, Ehlers, Oladyshkin & Otto.] Alternative: Kernels! Symmetric function K : R d R d R Positive definite (analysis!) E.g. Gaussian (for γ > ) K g (x, y) = exp( x y 2 /γ 2 ) 6/52

7 Introduction More famous kernel examples Compactly supported Wendland kernels Kw d,k (x, y) = φ d,k w ( ) x y for d, k N and polynomials γ, φ d,k w (r) = (1 r) l+k + p d,k (r). p d,1 (r) = (l + 1)r + 1, l = d/2 + k + 1 p d,2 (r) = 1 3 (l2 + 4l + 3)r 2 + (l + 2)r + 1, [...] γ is a dilation hyperparameter for the kernel width. d, k determines the smoothness in the sense of K d,k w C 2k (P D ) if P D R s, s d. 7/52

8 Introduction Plots for different kernels Figure: Examples. 1D: K g(γ =.33), Kw 2,, K1,1 w.2d: Kg(γ =.49), K2, w, K3,3 w. γ = 1 for all Kw. 8/52

9 Introduction From kernels to kernel expansions f = Kernel expansion! f(x) = N c i K(x i, x) i=1 with centers x i R d and coefficients c i R. Figure: Illustration for a 1D Gaussian kernel expansion with γ = 1 and x i = [ 1,.5, 1.2], c i = [.5,.2,.8] 9/52

10 Expansion construction Up Next Expansion construction 1/52

11 Expansion construction How to obtain kernel expansions? Various methods available Mostly using only training data {x 1,..., x N },{y 1,..., y N } Support Vector Machines / Regression [Schölkopf & Smola(22)] 1 f = arg min g H N N i=1 V ( y i g(x i) ) + λ g 2 H, V (e) = max{, e ɛ} f = /52

12 Expansion construction ɛ-svr Support Vector Regression with ɛ-loss (Dual problem) For C := 1 2λN and y = (y 1,..., y N ) T R N we solve max α +,α R N W (α +, α ) (1) If α +, α s.t. α +, i C, i = 1... N, = α + i α i, i = 1... N, W (α +, α ) := 1 ( α + α ) T ( K α + α ) 2 N ɛ (α + i + α i ) + y T ( α + α ). i=1 solve (1), then N f(x) := c ik(x, x i), i=1 c i := (α α + ) i Computation via Sequential Minimal Optimization (SMO) 12/52

13 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 1 f(x) and approximation x 13/52 Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

14 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 2 f(x) and approximation x 14/52 Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

15 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 3 f(x) and approximation x 15/52 Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

16 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 4 f(x) and approximation x 16/52 Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

17 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 5 f(x) and approximation x 17/52 Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

18 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 6 f(x) and approximation x 18/52 Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

19 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 7 f(x) and approximation x Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers 19/52

20 Expansion construction ɛ-svr illustration: 1D SMO.5 Iteration 8 f(x) and approximation x Figure: ɛ-svr 1D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers 2/52

21 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 1 f(x) and approximation x 21/52 Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

22 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 2 f(x) and approximation x 22/52 Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

23 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 3 f(x) and approximation x 23/52 Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

24 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 4 f(x) and approximation x 24/52 Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

25 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 5 f(x) and approximation x 25/52 Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

26 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 6 f(x) and approximation x 26/52 Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers

27 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 7 f(x) and approximation x Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers 27/52

28 Expansion construction ɛ-svr illustration: 2D SMO.5 Iteration 8 f(x) and approximation x Figure: ɛ-svr 2D SMO iterations. Red: Target function with margin ɛ =.1, blue: approximation, black: centers 28/52

29 Expansion construction ɛ-svr illustration: 2D SMO full solution.5 It: 3523, #SV=2, eps=.1 f(x) x Figure: ɛ-svr 2D SMO solution. Red: Target function with margin ɛ =.1, blue: approximation, black: centers 29/52

30 Expansion construction The VKOGA Vectorial kernel orthogonal greedy algorithm Let m N and f H m, define X :=, f := and for k >, Ω k := { x P D K(x, ) H X k 1}, the sequences x k := arg max x P D \Ω k X k := X k 1 {x k }, f k := I m X k [f]. f(x) f k 1 (x) 2 2 K(x, ) IXk 1 [K(x, )], H Here, I m X k [f] denotes component-wise interpolation on X k. In/Exclusion of the denominator yields different variants. Details in [Wirtz & Haasdonk(213)] 3/52

31 Expansion construction VKOGA illustration 1.5 Iteration f(x) x 31/52 Figure: VKOGA iterations one to four. Green/Blue: Components, Dashed: Approximations, Red: Total error, Diamonds: Centers, Red circle: Max overall error, Black lines: Max component errors

32 Expansion construction VKOGA illustration 1.5 Iteration f(x) x 32/52 Figure: VKOGA iterations one to four. Green/Blue: Components, Dashed: Approximations, Red: Total error, Diamonds: Centers, Red circle: Max overall error, Black lines: Max component errors

33 Expansion construction VKOGA illustration 1.5 Iteration f(x) x 33/52 Figure: VKOGA iterations one to four. Green/Blue: Components, Dashed: Approximations, Red: Total error, Diamonds: Centers, Red circle: Max overall error, Black lines: Max component errors

34 Expansion construction VKOGA illustration 1.5 Iteration f(x) x 34/52 Figure: VKOGA iterations one to four. Green/Blue: Components, Dashed: Approximations, Red: Total error, Diamonds: Centers, Red circle: Max overall error, Black lines: Max component errors

35 Experimental results Up Next Experimental results 35/52

36 Experimental results Multibody spine simulation: Introduction Cooperation with N. Karajan, D. Otto Linking MBS spine simulation with FE intervertebral model Figure: Illustration from [Karajan et al.(211)karajan, Röhrle, Ehlers & Schmitt] 36/52

37 Experimental results Multibody spine simulation: Animations for bone and disk interactions Figure: Animations for MBS vertebra interaction and ϕ 1, u 2, u 3 disk response cases 37/52

38 Experimental results Multibody spine simulation: Combined load cases for IVDs Figure: Different load case animations for general and geometry-fitted intervertebral disks 38/52

39 Experimental results Multibody spine simulation: Combined load cases for IVDs Computation with quasi-static timesteps from equillibrium state Figure: Joystick like computation of training data 39/52

40 Experimental results Multibody spine simulation: Approximation Plot of M1 against u2 and u3 FEM inputs ϕ 1, u 2, u 3 P D R 3 Disc response M 1, R 2, R 3 P C R 3 Precomputation of 274/1433 disc responses About 3min average simulation time each dot Future work: Actual input is ϕ 1, u 2, u 3, M 1, R 2, R 3 from last (time)step M u3 Plot of M1 against u2 and phi1 x phi1 u2 M1 2 2 u2 3 2 Figure: Plots of training data 4/52

41 Experimental results Multibody spine simulation: Results with Gauss kernels 41/52 Figure: Plots of approximation, u2, u3 M1, ϕ1 = [ 5.7,, 5.3] (top) and u2, ϕ1 M1, u3 = [ 3.1,, 3.1] (bottom)

42 Experimental results Multibody spine simulation: Results with Wendland kernels Figure: Plots of approximation using Wendland kernels and best configuration γ : , k : 1, d : 2 42/52

43 Experimental results Multibody spine simulation: Outlook Simulation using kernel approximations! So far: Using Polynomial Chaos Expansions Figure: Simulation of full multibody spine model using suggorates 43/52

Buckley-Leverett problem S t + (vf(s)) = ɛs xx + ɛ 2 τs xxt On front: Solve

44 Experimental results Two-phase flow in porous media: Intro Cooperation with F. Kissling (IANS) Two-phase flow in porous media with nonclassical shock waves Reg. Buckley-Leverett problem S t + (vf(s)) = ɛs xx + ɛ 2 τs xxt On front: Solve Riemann problems for overshoot Figure: Illustrations from [Kissling & Rhode(212)] 44/52

Experimental results Two-phase flow in porous media: Approximation Riemann problem initial data Left-side saturation s l Scaling parameter τ Result:

45 Experimental results Two-phase flow in porous media: Approximation Riemann problem initial data Left-side saturation s l Scaling parameter τ Result: Plateau value S ɛ Precomputation of 5 Rieman problems Suitable transformation of training data Scaling Logarithmic values Figure: Plots of training data 45/52

46 Experimental results Two-phase flow in porous media: Results 46/52 Figure: Plots of approximations. Top: SVR (Gauss), Bottom: Greedy algorithm (Wendland)

Experimental results Two-phase flow in porous media: Results Time-reduction from 2h to 24min for full simulation N S S SV R L 1 497 5.69 1 2 647 3.

47 Experimental results Two-phase flow in porous media: Results Time-reduction from 2h to 24min for full simulation N S S SV R L Figure: Full simulation plots and error table. Full model (top) and SVR reduced (bottom) Details in [Kissling & Rhode(212)] 47/52

48 Summary What did we see? Setting: Reduction for Microscale models Use of cheap surrogates for micro-scale level Kernel methods can provide such replacements! Support vector regression (SVR) Vectorial greedy algorithms (VKOGA) Examples and results available: Human Spine Simulations Shock-waves in porous media 48/52

49 -THE END-... any questions? Thank you for your attention! 49/52

50 References I [Karajan et al.(213)karajan, Ehlers, Oladyshkin & Otto.] Karajan, N.; Ehlers, W.; Oladyshkin, S. & Otto., D.: Application of the Polynomial Chaos Expansion to Approximate the Homogenised Behaviour of the Intervertebral Disc. Preprint, University of Stuttgart, SRC SimTech (213), in preparation. [Karajan et al.(211)karajan, Röhrle, Ehlers & Schmitt] Karajan, N.; Röhrle, O.; Ehlers, W. & Schmitt, S.: Linking Continuous and Discrete Intervertebral Disc Models through Homogenisation. Preprint, SRC SimTech (211), URL uni-stuttgart.de/publikationen/prints.php?id=657, submitted to Biomechanics and Modeling in Mechanobiology. [Khuri & Mukhopadhyay(21)] Khuri, A. I. & Mukhopadhyay, S.: Response surface methodology. Wiley Interdisciplinary Reviews: Computational Statistics 2 (21), , ISSN /52

51 References II [Kissling & Rhode(212)] Kissling, F. & Rhode, C.: The Computation of Nonclassical Shock Waves in Porous Media with a Heterogeneous Multiscale Method: The Multidimensional Case. Preprint, Stuttgart Research Centre for Simulation Technology (212), URL publikationen/prints.php?id=734, submitted to SIAM Multiscale Modeling and Simulation. [Myers et al.(29)myers, Montgomery & Anderson-Cook] Myers, R. H.; Montgomery, D. C. & Anderson-Cook, C. M.: Response Surface Methodology - Process and Product Optimization Using Designed Experiments. Wiley Series in Probability and Statistics, Wiley 29, 3 edn. [Schölkopf & Smola(22)] Schölkopf, B. & Smola, A. J.: Learning with Kernels. Adaptive Computation and Machine Learning, The MIT Press /52

52 References III [Wiener(1938)] Wiener, N.: The homogeneous chaos. Amer. J. Math. 6 (1938), , ISSN 29327, URL [Wirtz & Haasdonk(213)] Wirtz, D. & Haasdonk, B.: An improved vectorial kernel orthogonal greedy algorithm. Dolomites Research Notes on Approximation 6 (213), /52

CIS 520: Machine Learning Oct 09, Kernel Methods

CIS 520: Machine Learning Oct 09, 207 Kernel Methods Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture They may or may not cover all the material discussed