Hyper-ribbons. Fitting Exponentials. Cell Dynamics. Sloppiness. Coarse-Grained. Models. Model Manifold

Transcription

1 Sloppy Models, Differential geometry, and How science works JPS, Katherine Quinn, Archishman Raju, Mark Transtrum, Ben Machta, Ricky Chachra, Kevin Brown, Ryan Gutenkunst, Josh Waterfall, Fergal Casey, Chris Myers, Cell Dynamics Sloppiness Fitting Exponentials Model Manifold Hyper-ribbons Coarse-Grained Models

2 Fitting Decaying Exponentials Classic ill-posed inverse problem Given Geiger counter measurements from a radioactive pile, can we recover the identity of the elements and/or predict future radioactivity? Good fits with bad decay rates! Ensemble: Ensemble: Interpolation Extrapolation Fit P, S, I 2 y(a,γ,t) = A1 e -γ 1 t +A2 e -γ 2 t +A3 e -γ 3 t 6 Parameter Fit

3 Models: Predictions about Data Scientific model: Predictions about behavior depend on physical constants (parameters) in the model. Sloppiness: the behavior only depends on a few stiff parameter combinations.

4 y(t) A B fit Exercise 1.14a Model manifold in data space 0.5 Two exponentials fit to data θ t A B fit θ 1 C( ) = 2 /2= X i (y i ( ) d i ) 2 /2 i 2 P(θ) exp( C(θ)) [i] C is half the squared distance in data space. What is the metric g ij? [ii] How is the cost related to the log likelihood, if the errors are Gaussian? [iii] View C as a Hamiltonian. What is the temperature?

5 PC12 Differentiation +EGF EGFR NGFR +NGF Tunes down signal (Raf-1) Sos Ras Raf-1 Pumps up signal (Mek) ERK* 10 MEK1/2 ERK* 10 Time ERK1/2 Time Biologists study which proteins talk to which. Modeling?

6 Systems Biology: Cell Protein Reactions 48 Parameter Fit!

7 Ensemble of Models We want to consider not just minimum cost fits, but all parameter sets consistent with the available data. New level of abstraction: statistical mechanics in model space. Don t trust predictions that vary Cost is least-squares fit eigen Boltzmann weights exp(-c/t) bare O is chemical concentration y(t i ), or rate constant θ n

8 48 Parameter Fit to Data Cost is Energy Ensemble of Fits Gives Error Bars eigen Error Bars from Data Uncertainty +EGF bare ERK* 10 ERK* +NGF 10 Time Time

9 Relative parameter fluctuation Parameter (sorted) Parameters Fluctuate over Enormous Range All parameters vary by minimum factor of 50, some by a million Not robust: four or five stiff linear combinations of parameters; 44 sloppy Are predictions possible? log e eigenparameter fluctuation stiff sloppy sorted eigenparameter number

10 Predictions are Possible Model predicts that the left branch isn t important Brown s Experiment Model Prediction Parameters fluctuate orders of magnitude, but still predictive!

11 Parameter Indeterminacy and Sloppiness 10-6 Stiff direction Cost Contours fits of decaying exponentials Note: Horizontal scale shrunk by 1000 times Aspect ratio = Human hair 48 parameter fits are sloppy: Many parameter sets give almost equally good fits Bare Parameter Axes Sloppy direction 10-3 ~5 stiff, ~43 sloppy directions A few stiff constrained directions allow model to remain predictive

12 Exercise 1.14b Model manifold in data space C( ) = 2 /2= X i (y i ( ) d i ) 2 /2 2 i P(d θ) exp( C(θ)) [i] Show P (d ) is a blurred Gaussian sphere in data space. [ii] Make an analogy to the momentum distribution of classical particles with di erent masses.

13 Systems Biology Seventeen models (a) eukaryotic cell cycle (b) Xenopus egg cell cycle (c) eukaryotic mitosis (d) generic circadian rhythm (e) nicotinic acetylcholine intra-receptor dynamics (f) generic kinase cascade (g) Xenopus Wnt signaling (h) Drosophila circadian rhythm (i) rat growth-factor signaling (j) Drosophila segment polarity (k) Drosophila circadian rhythm (l) Arabidopsis circadian rhythm (m)in silico regulatory network (n) human purine metabolism (o) Escherichia coli carbon metabolism (p) budding yeast cell cycle (q) rat growth-factor signaling 3 Enormous Ranges of Eigenvalues (3 48 is a big number) Sloppy Range ~ λ Gutenkunst

14 Which Rate Constants are in the Stiffest Eigenvector? Eigenvector components along the bare parameters reveal which ones are most important for a given eigenvector. Oncogenes * * stiffest * * 2 nd stiffest * Ras * Raf1 * * *

15 Sloppy Universality Outside Bio Enormous range of eigenvalues; Roughly equal density in log; Observed in broad range of systems

16 Some systems aren t sloppy Sloppy Systems Quantum Monte Carlo (best molecule excitations) Monomial sums Not linear regression models Not random matrix theory Systems Bio QMC Radioactivity Many exps GOE Product Fitting plane Eigenvalue Monomials NOT SLOPPY

17 Where is Sloppiness From? Fitting Polynomials to Data Fitting Monomials to Data y = a n x n Functional Forms Same Hessian H ij = 1/(i+j+1) Hilbert matrix: famous Orthogonal Polynomials y = b n L n (x) Functional Forms Distinct Eigen Parameters Hessian H ij = δ ij Sloppiness arises when bare parameters skew in eigenbasis Small Determinant! H = λ n

18 Exploring Parameter Space Rugged? More like Grand Canyon (Waterfall) Glasses: Rugged Landscape Metastable Local Valleys Transition State Passes Optimization Hell: Golf Course Sloppy Models Minima: 5 stiff, N-5 sloppy Search: Flat planes with cliffs

19 Variational Wavefunctions for Quantum Monte Carlo Cyrus Umrigar, Josh Waterfall The most accurate method for solving Schrödinger s equation for molecules rests on a variational wavefunction (followed by diffusion Monte Carlo): φ Ψ(R) = a m exp( u(r ij )) 2 (r 1 ) φ 2 (r 2 )... φ 2 (r N ) m i< j φ 1 (r 1 ) φ 1 (r 2 )... φ 1 (r N ) φ 1 (r 1 ) φ 1 (r 2 )... φ 1 (r N ) φ 2 (r 1 ) φ 2 (r 2 )... φ 2 (r N ) φ N (r 1 ) φ N (r 2 )... φ N (r N ) m φ N (r 1 ) φ N (r 2 )... φ N (r N ) m Often parameters θ are varied to minimize energy (find ground states) Umrigar developed a method which fits the local energies at randomly chosen Rj Data = H Ψ θ (Rj) / Ψ θ (Rj) = E(Rj) then varies parameters to minimize fluctuations in local E s away from eigen-energy...

20 Interatomic Potentials Søren Frederiksen, Karsten Jacobsen, Kevin Brown, JPS Need atomic forces and energies Guess functional form for potential (17 parameters for MEAM, 5 for Finnis-Sinclair) Use electronic ground state energy U(R1,R2, ) (Born-Oppenheimer approximation) Least-squares fit to DFT calculations of energy, forces for a variety of important atomic configurations

21 Universal Scaling Functions Yan-Jiun Chen, Zapperi, Durin, Papanikolaou A00 Text Avalanches through Windows 11 parameter universal scaling form depends on demag length L κ, window width W Size s

22 3 V t S Machine Learning 20 V t S OP Mark Transtrum One OP Neural net trained to predict option price, or recognize handwriting Neural weights sloppy! Output can be viewed as small dimensional representation of Big Data set Is the inverse image of output a hyperribbon in Data space? Alex Alemi Thorsten Joachims CS

23 Colin Clement Ivan Bazarov (Physics) Accelerator Optimization Photoinjector system for Energy Recovery Linac Laser hits flexible mirror, which shapes beam on target Beam of electrons nonlinear distortion through photoinjector Optimize beam profile for emittance Use 3D beam shape (point on model manifold), find stiff knobs to optimally adjust beam

24 Stainforth et al., Uncertainty in predictions of the climate response to rising levels of greenhouse gases, Nature 433, (2005) Yan-Jiun Chen Natalie Mahowald EAS Climate Change? Climate models contain many unknown parameters, fit to data General Circulation Model (air, oceans, clouds), exploring doubling of CO 2 21 total parameters Initial conditions and (only) 6 cloud dynamics parameters varied Heating typically 3.4K, ranged from < 2K to > 11K

25 Ricky Chachra Karel Mertens Econ Macroeconomics? Many structural parameters unknown Dynamic stochastic general equilibrium models (DSGE), such as Smets and Wouters ~25 parameters Larger uncertainties and less predictive than much more primitive models with many more parameters? Parameter sensitivities: is it sloppy? Model reduction: can we simplify it? Can we distill one from more primitive models? Are extensions of the model different?

26 The Universe ΛCDM fit for cosmic microwave background radiation Universe is flat, mostly unknown dark stuff Six parameter ΛCDM model is sloppy fit to CMB; SNe and BAO determine More general models introduce degeneracies Katherine Quinn Michael Niemack

27 The Model Manifold Parameter space Stiff and sloppy directions Canyons, Plateaus Two exponentials θ 1, θ 2 fit to three data points y 1, y 2, y 3 y n = exp(-θ 1 t n ) + exp(-θ 2 t n ) θ 2 θ 1 Data space Manifold of model predictions Parameters as coordinates Model boundaries θ n = ±, θ m cause Plateaus Metric g µν from distance to data

28 Exercise 1.14e Metric for model manifold Cost C( ) = 2 /2= X i (y i ( ) d i ) 2 /2 2 i Cost Hessian C( ) =C( best + ) C( C best +1/2H [i] Write H in terms of first and second derivatives of y i ( ). [ii] If d i = y i ( ), show H = J T J where J i =(1/ i )@y i /@ is the Jacobian. [iii] Show that the squared distance in data space between y( ) and y( + ) is given by a metric tensor g = J T J.

29 Geodesics Straight line in curved space Shortest path between points γ 2 Easy to find cost minimum using polar geodesic coordinates γ 1, γ 2 γ 1 Cost contours in geodesic coordinates nearly concentric circles! Use this for algorithms

30 The Model Manifold is a Hyper-Ribbon Hyper-ribbon: object that is longer than wide, wider than thick, thicker than... Thick directions traversed by stiff eigenparameters, thin as sloppy directions varied. Sum of many exponentials, fit to y(0), y(1) data predictions at y(1/4), y(1/2), y(3/4) Widths along geodesics track eigenvalues almost perfectly! Diffusion equation after three time steps

31 Edges of the model manifold Fitting Exponentials Top: Flat model manifold; articulated edges = plateau Bottom: Stretch to uniform aspect ratio (Isabel Kloumann) θ 2 θ 1

32 Intrinsic curvature R µ ναβ determines geodesic shortest paths independent of embedding, parameters Extrinsic curvature also measures bending in embedding space (i.e., cylinder) independent of parameters Shape operator, geodesic curvature Parameter effects curvature Usually much the largest Defined in analogy to extrinsic curvature (projecting out of surface, rather than into) Curvatures Geodesic Curvature No intrinsic curvature Shape Operator

33 Where is Sloppiness From? Fitting Polynomials to Data Fitting Monomials to Data y = a n x n Functional Forms Same Hessian H ij = 1/(i+j+1) Hilbert matrix: famous Orthogonal Polynomials y = b n L n (x) Functional Forms Distinct Eigen Parameters Hessian H ij = δ ij Sloppiness arises when bare parameters skew in eigenbasis Small Determinant! H = λ n

34 <latexit sha1_base64="gsepejeo4aqt9b/w4htpydoibde=">aaacmxicbvhbbhmxepuutxjuasre+jiii5rijdqtkeciiikgspgkiibppwy6mni9janxxuxzshtln/gw3vgbnmsdtmyddxsojz1zzlwp6siofwfhlavxrt/yunm4dfvo3xvn+w8gztswiz43ytjjmtqhpbz9kqtecwuflmmljsznbxf60xdhntt6c00rmsrxvmtccirpzc2fqtzs50jty89yonj5cg0e961gk6besvjhzws4kl4rjag5opyavcgwxp+dlrr7utnl2ceqdhaqo6balblvfpxlmoa1hhyct4sgidk5q9nxqnub/qkivplgc8xksctrkjkqvhok8gr28dszr/6bcri44f3rsolez81w3i2xbzdbsgyttq7drpkrzq2vsz8iv+jcmikrgs3qkurkzbtp7usf/axpxdbdjavwo9ky2tk88uwohu+mmjpgyf7tmghp3lqc+5s+rim7qc3i/2ndmooxo5nuvu1c89vhra3az7bye+tsck5q6gfyk32vwcdokznfzsohkfwc+tiybhetujt82m7tplvhsce22wpwzgl7znzzjx2ypupbo+bv8d7yczfdn2ev/lc6ggzrz0p2t4wf/wc0e8z1</latexit> <latexit sha1_base64="gsepejeo4aqt9b/w4htpydoibde=">aaacmxicbvhbbhmxepuutxjuasre+jiii5rijdqtkeciiikgspgkiibppwy6mni9janxxuxzshtln/gw3vgbnmsdtmyddxsojz1zzlwp6siofwfhlavxrt/yunm4dfvo3xvn+w8gztswiz43ytjjmtqhpbz9kqtecwuflmmljsznbxf60xdhntt6c00rmsrxvmtccirpzc2fqtzs50jty89yonj5cg0e961gk6besvjhzws4kl4rjag5opyavcgwxp+dlrr7utnl2ceqdhaqo6balblvfpxlmoa1hhyct4sgidk5q9nxqnub/qkivplgc8xksctrkjkqvhok8gr28dszr/6bcri44f3rsolez81w3i2xbzdbsgyttq7drpkrzq2vsz8iv+jcmikrgs3qkurkzbtp7usf/axpxdbdjavwo9ky2tk88uwohu+mmjpgyf7tmghp3lqc+5s+rim7qc3i/2ndmooxo5nuvu1c89vhra3az7bye+tsck5q6gfyk32vwcdokznfzsohkfwc+tiybhetujt82m7tplvhsce22wpwzgl7znzzjx2ypupbo+bv8d7yczfdn2ev/lc6ggzrz0p2t4wf/wc0e8z1</latexit> <latexit sha1_base64="gsepejeo4aqt9b/w4htpydoibde=">aaacmxicbvhbbhmxepuutxjuasre+jiii5rijdqtkeciiikgspgkiibppwy6mni9janxxuxzshtln/gw3vgbnmsdtmyddxsojz1zzlwp6siofwfhlavxrt/yunm4dfvo3xvn+w8gztswiz43ytjjmtqhpbz9kqtecwuflmmljsznbxf60xdhntt6c00rmsrxvmtccirpzc2fqtzs50jty89yonj5cg0e961gk6besvjhzws4kl4rjag5opyavcgwxp+dlrr7utnl2ceqdhaqo6balblvfpxlmoa1hhyct4sgidk5q9nxqnub/qkivplgc8xksctrkjkqvhok8gr28dszr/6bcri44f3rsolez81w3i2xbzdbsgyttq7drpkrzq2vsz8iv+jcmikrgs3qkurkzbtp7usf/axpxdbdjavwo9ky2tk88uwohu+mmjpgyf7tmghp3lqc+5s+rim7qc3i/2ndmooxo5nuvu1c89vhra3az7bye+tsck5q6gfyk32vwcdokznfzsohkfwc+tiybhetujt82m7tplvhsce22wpwzgl7znzzjx2ypupbo+bv8d7yczfdn2ev/lc6ggzrz0p2t4wf/wc0e8z1</latexit> <latexit sha1_base64="gsepejeo4aqt9b/w4htpydoibde=">aaacmxicbvhbbhmxepuutxjuasre+jiii5rijdqtkeciiikgspgkiibppwy6mni9janxxuxzshtln/gw3vgbnmsdtmyddxsojz1zzlwp6siofwfhlavxrt/yunm4dfvo3xvn+w8gztswiz43ytjjmtqhpbz9kqtecwuflmmljsznbxf60xdhntt6c00rmsrxvmtccirpzc2fqtzs50jty89yonj5cg0e961gk6besvjhzws4kl4rjag5opyavcgwxp+dlrr7utnl2ceqdhaqo6balblvfpxlmoa1hhyct4sgidk5q9nxqnub/qkivplgc8xksctrkjkqvhok8gr28dszr/6bcri44f3rsolez81w3i2xbzdbsgyttq7drpkrzq2vsz8iv+jcmikrgs3qkurkzbtp7usf/axpxdbdjavwo9ky2tk88uwohu+mmjpgyf7tmghp3lqc+5s+rim7qc3i/2ndmooxo5nuvu1c89vhra3az7bye+tsck5q6gfyk32vwcdokznfzsohkfwc+tiybhetujt82m7tplvhsce22wpwzgl7znzzjx2ypupbo+bv8d7yczfdn2ev/lc6ggzrz0p2t4wf/wc0e8z1</latexit> <latexit sha1_base64="i6pu+ylpgbdaqif5lymea5ghq4e=">aaadsnicdvjnb9naef27byr5suhizuvc1aio2cksxjaqisfucvqkklakgxmvn/gq9q7zhqorld/hhrm3fgqxdidehbut0ddasgpnvnkz+3y8czejg77/xxe3ni9cvlr12bty9dr1g63tm0ojss34gklm6zmydm+e5amumpgtqnpi44wfx6dp6vzxo66nupivzgo+zmeqxuqwqatf286bucohey7re6o0tyrhmioncsbqqnrzxsjo0jbralouoxd7ns41qxppvc5hvikqsikfox1mwwi0csjozjaivft/zigmhcfamdt0onruqmw8idilda7sl+aawnzoemhmp0jw/g3zklw39w7r3gu0wldmhl21ymdkrp11f6fr2ctpq6bxef3av3j9//d2a1/h7a/zmlsdhfwmz9qa05n+qnx2e35jdd0ilkgblo0oan22nvmz21/cmjbmfpgfjqt6mczjtn9peahsfkz8zemjotfjqlmfob1rkaqesxwjtefxkyrijznlswvavak5n6vbf+vgju4ejsshixk5ziuljmvgudf6r+ziam4wm9kambzwk2upagbot8+zqwjop3k9gpz7gd8lxvbbbw+w49git8kdsksc8packgfkiawicz46x53vzg/3k/vn/en+wlbdz1lzi/xlg5u/aqve/l8=</latexit> <latexit sha1_base64="i6pu+ylpgbdaqif5lymea5ghq4e=">aaadsnicdvjnb9naef27byr5suhizuvc1aio2cksxjaqisfucvqkklakgxmvn/gq9q7zhqorld/hhrm3fgqxdidehbut0ddasgpnvnkz+3y8czejg77/xxe3ni9cvlr12bty9dr1g63tm0ojss34gklm6zmydm+e5amumpgtqnpi44wfx6dp6vzxo66nupivzgo+zmeqxuqwqatf286bucohey7re6o0tyrhmioncsbqqnrzxsjo0jbralouoxd7ns41qxppvc5hvikqsikfox1mwwi0csjozjaivft/zigmhcfamdt0onruqmw8idilda7sl+aawnzoemhmp0jw/g3zklw39w7r3gu0wldmhl21ymdkrp11f6fr2ctpq6bxef3av3j9//d2a1/h7a/zmlsdhfwmz9qa05n+qnx2e35jdd0ilkgblo0oan22nvmz21/cmjbmfpgfjqt6mczjtn9peahsfkz8zemjotfjqlmfob1rkaqesxwjtefxkyrijznlswvavak5n6vbf+vgju4ejsshixk5ziuljmvgudf6r+ziam4wm9kambzwk2upagbot8+zqwjop3k9gpz7gd8lxvbbbw+w49git8kdsksc8packgfkiawicz46x53vzg/3k/vn/en+wlbdz1lzi/xlg5u/aqve/l8=</latexit> <latexit sha1_base64="i6pu+ylpgbdaqif5lymea5ghq4e=">aaadsnicdvjnb9naef27byr5suhizuvc1aio2cksxjaqisfucvqkklakgxmvn/gq9q7zhqorld/hhrm3fgqxdidehbut0ddasgpnvnkz+3y8czejg77/xxe3ni9cvlr12bty9dr1g63tm0ojss34gklm6zmydm+e5amumpgtqnpi44wfx6dp6vzxo66nupivzgo+zmeqxuqwqatf286bucohey7re6o0tyrhmioncsbqqnrzxsjo0jbralouoxd7ns41qxppvc5hvikqsikfox1mwwi0csjozjaivft/zigmhcfamdt0onruqmw8idilda7sl+aawnzoemhmp0jw/g3zklw39w7r3gu0wldmhl21ymdkrp11f6fr2ctpq6bxef3av3j9//d2a1/h7a/zmlsdhfwmz9qa05n+qnx2e35jdd0ilkgblo0oan22nvmz21/cmjbmfpgfjqt6mczjtn9peahsfkz8zemjotfjqlmfob1rkaqesxwjtefxkyrijznlswvavak5n6vbf+vgju4ejsshixk5ziuljmvgudf6r+ziam4wm9kambzwk2upagbot8+zqwjop3k9gpz7gd8lxvbbbw+w49git8kdsksc8packgfkiawicz46x53vzg/3k/vn/en+wlbdz1lzi/xlg5u/aqve/l8=</latexit> <latexit sha1_base64="i6pu+ylpgbdaqif5lymea5ghq4e=">aaadsnicdvjnb9naef27byr5suhizuvc1aio2cksxjaqisfucvqkklakgxmvn/gq9q7zhqorld/hhrm3fgqxdidehbut0ddasgpnvnkz+3y8czejg77/xxe3ni9cvlr12bty9dr1g63tm0ojss34gklm6zmydm+e5amumpgtqnpi44wfx6dp6vzxo66nupivzgo+zmeqxuqwqatf286bucohey7re6o0tyrhmioncsbqqnrzxsjo0jbralouoxd7ns41qxppvc5hvikqsikfox1mwwi0csjozjaivft/zigmhcfamdt0onruqmw8idilda7sl+aawnzoemhmp0jw/g3zklw39w7r3gu0wldmhl21ymdkrp11f6fr2ctpq6bxef3av3j9//d2a1/h7a/zmlsdhfwmz9qa05n+qnx2e35jdd0ilkgblo0oan22nvmz21/cmjbmfpgfjqt6mczjtn9peahsfkz8zemjotfjqlmfob1rkaqesxwjtefxkyrijznlswvavak5n6vbf+vgju4ejsshixk5ziuljmvgudf6r+ziam4wm9kambzwk2upagbot8+zqwjop3k9gpz7gd8lxvbbbw+w49git8kdsksc8packgfkiawicz46x53vzg/3k/vn/en+wlbdz1lzi/xlg5u/aqve/l8=</latexit> <latexit sha1_base64="2gdtihrvnudqyapp+wrzun7ieno=">aaacm3icbvbna9tafhxyvxknaz32mmvseldbdsrtahowghipguicdrkwllfar+wluyux+1rshkb/pvde+kd6kiqcukqv+q9d2zm0tgcwhpn3edut5fjy9p0rr/bg4aphtzy261tpt589b+y8olnzyrgfsexm5ikhlkuh+qafsn6rg05vivl5cnm08m8/cwnfpj/ipocjrsdapijrdflcod6kqkvxalszz3je9zrbqbcvtkohmfajgdtt5qz1orsfikvv86voawlxyphgiswi1yq6jgytctmr4sae3/gxipdjcef2+u/wy2caoikb38nxxgrfntjjrr0gfo6jkhoutpkqhhaw55rd0gkfoqqp4nzuljnxzn8py5jmxj2nzkn+vvfsze1cjw5ykc+uewvxf96wwprwvaqdf8g1wx1kc0kwi4scyvgyzldohahmcpdxwqbuuiau5rorivipfj+cdtub3wloxrtvyiun2ivx0iqa3kif3smjdidbv/gbn/dt++zde7+836vrmne38xl+gxf7b4geqou=</latexit> <latexit sha1_base64="onz3ztpxmfyikfm7vrei/tk6g3u=">aaacm3icbvbbsxtbgj1nraaxalqpvqwjqgi27gah7ueiejfcqcekqja7ze5mzedm7dlzbum67m/pvrf/iadbeqiiv/+dk8sdmj4yelz3fxzzxpqkbsb1b5zsm5w3q2vld5x19xubw9uph3smytrlxzqirj9fxddbfescb8hous2ijatrrxehm7//k2nde3uk05qnjtlxpoaugjxc6vfdwjcexlrmaskmxuhd22s3/v2fkwjdwmonudfpfvznjsnchrhf8ap7mgzaqokngwwgbezv4le+kcjq3w25c+bl4j2reucb/p41ebxjshrljxkasaaacmlmwhntgozea6ecfru/mywl9ikcs4glikhmhvk8c4f3rdlccaltu4dn6vonnehjpjkyk7n85ru3e//ndtkivw5zrtimmkklq3emmcr4viaecc0oikklhgpu/4rpmghcwdzcssv4rymvk1675bkt78s2sy8wkknpqiyayenfuacdowpurrt9qdfoh7p1lp2/zp1zvxgtou872+gfnidh192rhq==</latexit> <latexit sha1_base64="onz3ztpxmfyikfm7vrei/tk6g3u=">aaacm3icbvbbsxtbgj1nraaxalqpvqwjqgi27gah7ueiejfcqcekqja7ze5mzedm7dlzbum67m/pvrf/iadbeqiiv/+dk8sdmj4yelz3fxzzxpqkbsb1b5zsm5w3q2vld5x19xubw9uph3smytrlxzqirj9fxddbfescb8hous2ijatrrxehm7//k2nde3uk05qnjtlxpoaugjxc6vfdwjcexlrmaskmxuhd22s3/v2fkwjdwmonudfpfvznjsnchrhf8ap7mgzaqokngwwgbezv4le+kcjq3w25c+bl4j2reucb/p41ebxjshrljxkasaaacmlmwhntgozea6ecfru/mywl9ikcs4glikhmhvk8c4f3rdlccaltu4dn6vonnehjpjkyk7n85ru3e//ndtkivw5zrtimmkklq3emmcr4viaecc0oikklhgpu/4rpmghcwdzcssv4rymvk1675bkt78s2sy8wkknpqiyayenfuacdowpurrt9qdfoh7p1lp2/zp1zvxgtou872+gfnidh192rhq==</latexit> <latexit sha1_base64="2wmtmq0gvwd0obzg5hizcossbry=">aaacm3icbvbnsysxfm34/kxf9b2lm2arwtcakyjubmhn44ggyfxodizmmrhbjdmkd6rlmp/kxj/iqhaxt8tt+w8vrv34dsbwoodebs6jmykseplotf2ynpmdm1+olc4tr6xw136e2zq3jldzklnzgvplpdc8dqikv8wmpyqw/ck+phr5fzfcwjhqmxhmvkvolrajybscffx/hiwbota3qshsoswp6v5oqxfsbujdreif15p6oled2fxfhtogzxima+hzojhcg1a1whyotncvgjrrtuaazaz8lfgtukmtnetv+6cxslxxduxsazs+yabbuaocsv5wgtzyjljresu7jmqquo0w48wl3nrkdyepcu8dhqvvnwqqrb2q2e2o8tnp3kj8zuvkkox3c6gzhlhmb4esxgji8aha3bogm5bdrygzwv0vsz41ligruejk8d9h/krow02fnp1tujvcndqxj9brbqojh+2hq/qbnaa2yugwpac/6nm78568f+/1bxtkm+z8qh/g/fspte+ocw==</latexit> Exercise Sloppy Monomials Fitting Polynomials to Data Sloppy eigenvalues log-equally spaced; m / M 1. Determinant varies with what power of? <latexit sha1_base64="ujqz8jk9mhpy1jb2qbhtjaro3e0=">aaaca3icbzdpbhmxema9y78soe3paqqcljjkxbrtvpvaqydwogcusewqtli2rlpe2csq1zb2bkjolqupyi034mi74kq5qmtilj594xl//uvwsu9j8jok791/8pdrxupwk6fpnrfa288vvkmdwl4wyrirhdwqqbfpkhrewydq5qov8+v3y/7lfj2xrn+hucvhbwmtsymagjvqf8+0kbpata3pylg75yjhqkegavrcmfeefqtbqtn3fgqwx7ybqbc+gfhfm+umjcozm1qex5upe+mi2+nnsogqquetn4ktydfm0otpjkdcmhk6bsqwzzxvprm1o0kvwrw/k9k16lb1ny/ap7lcilokoyuc7wdpymnygcmpfc5awe0xhl2gmq6c1fchhzyrvgu+g5ycl8afe/kt3l8ngqi8n1d5ufkbtfzt3tl8x29qu3k0bks2naewnw+vtekbzxi8l6rdqqfkiue4gbjymqehisdyrqahvf3lu+jiv5cmvfttfuf0yi1jg71mb9k7lrjddso+shpwz4l9ijajf9hl6he8e7+k39xcjap1za77p+ldpz+muds=</latexit> <latexit sha1_base64="ujqz8jk9mhpy1jb2qbhtjaro3e0=">aaaca3icbzdpbhmxema9y78soe3paqqcljjkxbrtvpvaqydwogcusewqtli2rlpe2csq1zb2bkjolqupyi034mi74kq5qmtilj594xl//uvwsu9j8jok791/8pdrxupwk6fpnrfa288vvkmdwl4wyrirhdwqqbfpkhrewydq5qov8+v3y/7lfj2xrn+hucvhbwmtsymagjvqf8+0kbpata3pylg75yjhqkegavrcmfeefqtbqtn3fgqwx7ybqbc+gfhfm+umjcozm1qex5upe+mi2+nnsogqquetn4ktydfm0otpjkdcmhk6bsqwzzxvprm1o0kvwrw/k9k16lb1ny/ap7lcilokoyuc7wdpymnygcmpfc5awe0xhl2gmq6c1fchhzyrvgu+g5ycl8afe/kt3l8ngqi8n1d5ufkbtfzt3tl8x29qu3k0bks2naewnw+vtekbzxi8l6rdqqfkiue4gbjymqehisdyrqahvf3lu+jiv5cmvfttfuf0yi1jg71mb9k7lrjddso+shpwz4l9ijajf9hl6he8e7+k39xcjap1za77p+ldpz+muds=</latexit> <latexit sha1_base64="ujqz8jk9mhpy1jb2qbhtjaro3e0=">aaaca3icbzdpbhmxema9y78soe3paqqcljjkxbrtvpvaqydwogcusewqtli2rlpe2csq1zb2bkjolqupyi034mi74kq5qmtilj594xl//uvwsu9j8jok791/8pdrxupwk6fpnrfa288vvkmdwl4wyrirhdwqqbfpkhrewydq5qov8+v3y/7lfj2xrn+hucvhbwmtsymagjvqf8+0kbpata3pylg75yjhqkegavrcmfeefqtbqtn3fgqwx7ybqbc+gfhfm+umjcozm1qex5upe+mi2+nnsogqquetn4ktydfm0otpjkdcmhk6bsqwzzxvprm1o0kvwrw/k9k16lb1ny/ap7lcilokoyuc7wdpymnygcmpfc5awe0xhl2gmq6c1fchhzyrvgu+g5ycl8afe/kt3l8ngqi8n1d5ufkbtfzt3tl8x29qu3k0bks2naewnw+vtekbzxi8l6rdqqfkiue4gbjymqehisdyrqahvf3lu+jiv5cmvfttfuf0yi1jg71mb9k7lrjddso+shpwz4l9ijajf9hl6he8e7+k39xcjap1za77p+ldpz+muds=</latexit> <latexit sha1_base64="ujqz8jk9mhpy1jb2qbhtjaro3e0=">aaaca3icbzdpbhmxema9y78soe3paqqcljjkxbrtvpvaqydwogcusewqtli2rlpe2csq1zb2bkjolqupyi034mi74kq5qmtilj594xl//uvwsu9j8jok791/8pdrxupwk6fpnrfa288vvkmdwl4wyrirhdwqqbfpkhrewydq5qov8+v3y/7lfj2xrn+hucvhbwmtsymagjvqf8+0kbpata3pylg75yjhqkegavrcmfeefqtbqtn3fgqwx7ybqbc+gfhfm+umjcozm1qex5upe+mi2+nnsogqquetn4ktydfm0otpjkdcmhk6bsqwzzxvprm1o0kvwrw/k9k16lb1ny/ap7lcilokoyuc7wdpymnygcmpfc5awe0xhl2gmq6c1fchhzyrvgu+g5ycl8afe/kt3l8ngqi8n1d5ufkbtfzt3tl8x29qu3k0bks2naewnw+vtekbzxi8l6rdqqfkiue4gbjymqehisdyrqahvf3lu+jiv5cmvfttfuf0yi1jg71mb9k7lrjddso+shpwz4l9ijajf9hl6he8e7+k39xcjap1za77p+ldpz+muds=</latexit> C poly =(1/2) Z 1 0 (f(x) MX m=0 m x m ) 2 dx For discrete data at (t 1,...,t N ), show J i i )/@ forms the Vandermonde matrix, t 1 t t M 1 J = B 1 t 2 t t M 2 1 t 3 t t M A 3... For M=N (a square matrix), calculate det(vandermonde). (Hint: It is a polynomial, with roots when t i = t j.) If m / M 1, what sets? The Vandermonde matrix is famous for being ill-conditioned.

35 <latexit sha1_base64="z+aetzdsfgxmuvda+brwhip1f5a=">aaadhxicdvjnbxmxepuux2x5suhizuskvfqp2g1icegq1etoqajsvsqmkdc7m1j1x2j7g8jq/wgx/goxdidegqvi3+ckc6upjgtr6c3m8/pywsm4dxh8mwgvxb5y9drg9ejgzvu373q27x5yxrmgq6afnkcztsi4wqhjtubrazdktobhdrk3zb/o0viu1wu3khes6vtxgjpqpdxzdb6nsnovo3lrnn0el7rdcdpq/izqcgmd5gj43nfpexsx4pn29nbecsqpmxlzl1osmi3gft6h0vhbnagvkrjrwauzfnumokbrovvgkwmv/0+e2yjncmpvjw+qldsmgpw/cvjl1cbzsatdwz2w1dhoxxef1pw1f3kqen1hjxlwga8obfwwtjpa7qidpolfkar8zmek04178srgikha0cvt7e8639ncs0r6ktnbrr0lceng9dibe9heawwxpoyetnhkoais7bhevw4ddz2tq6gnx8rbiv27o6bs2oxmfoxy0ny9tyt/lrtvrng2rrkqk4eknr5uvakchuvxgzwbze4spkdmco8v2iz6ytj/osi/hgt9yhfbqb+xxl3kzb+7+6qdxwa5tx6qbzkqp2sxdmg+griwva8+bp+dl+gh8fp4nfx2whogbc89ci7ch78af/p+wq==</latexit> <latexit sha1_base64="z+aetzdsfgxmuvda+brwhip1f5a=">aaadhxicdvjnbxmxepuux2x5suhizuskvfqp2g1icegq1etoqajsvsqmkdc7m1j1x2j7g8jq/wgx/goxdidegqvi3+ckc6upjgtr6c3m8/pywsm4dxh8mwgvxb5y9drg9ejgzvu373q27x5yxrmgq6afnkcztsi4wqhjtubrazdktobhdrk3zb/o0viu1wu3khes6vtxgjpqpdxzdb6nsnovo3lrnn0el7rdcdpq/izqcgmd5gj43nfpexsx4pn29nbecsqpmxlzl1osmi3gft6h0vhbnagvkrjrwauzfnumokbrovvgkwmv/0+e2yjncmpvjw+qldsmgpw/cvjl1cbzsatdwz2w1dhoxxef1pw1f3kqen1hjxlwga8obfwwtjpa7qidpolfkar8zmek04178srgikha0cvt7e8639ncs0r6ktnbrr0lceng9dibe9heawwxpoyetnhkoais7bhevw4ddz2tq6gnx8rbiv27o6bs2oxmfoxy0ny9tyt/lrtvrng2rrkqk4eknr5uvakchuvxgzwbze4spkdmco8v2iz6ytj/osi/hgt9yhfbqb+xxl3kzb+7+6qdxwa5tx6qbzkqp2sxdmg+griwva8+bp+dl+gh8fp4nfx2whogbc89ci7ch78af/p+wq==</latexit> <latexit sha1_base64="z+aetzdsfgxmuvda+brwhip1f5a=">aaadhxicdvjnbxmxepuux2x5suhizuskvfqp2g1icegq1etoqajsvsqmkdc7m1j1x2j7g8jq/wgx/goxdidegqvi3+ckc6upjgtr6c3m8/pywsm4dxh8mwgvxb5y9drg9ejgzvu373q27x5yxrmgq6afnkcztsi4wqhjtubrazdktobhdrk3zb/o0viu1wu3khes6vtxgjpqpdxzdb6nsnovo3lrnn0el7rdcdpq/izqcgmd5gj43nfpexsx4pn29nbecsqpmxlzl1osmi3gft6h0vhbnagvkrjrwauzfnumokbrovvgkwmv/0+e2yjncmpvjw+qldsmgpw/cvjl1cbzsatdwz2w1dhoxxef1pw1f3kqen1hjxlwga8obfwwtjpa7qidpolfkar8zmek04178srgikha0cvt7e8639ncs0r6ktnbrr0lceng9dibe9heawwxpoyetnhkoais7bhevw4ddz2tq6gnx8rbiv27o6bs2oxmfoxy0ny9tyt/lrtvrng2rrkqk4eknr5uvakchuvxgzwbze4spkdmco8v2iz6ytj/osi/hgt9yhfbqb+xxl3kzb+7+6qdxwa5tx6qbzkqp2sxdmg+griwva8+bp+dl+gh8fp4nfx2whogbc89ci7ch78af/p+wq==</latexit> <latexit sha1_base64="z+aetzdsfgxmuvda+brwhip1f5a=">aaadhxicdvjnbxmxepuux2x5suhizuskvfqp2g1icegq1etoqajsvsqmkdc7m1j1x2j7g8jq/wgx/goxdidegqvi3+ckc6upjgtr6c3m8/pywsm4dxh8mwgvxb5y9drg9ejgzvu373q27x5yxrmgq6afnkcztsi4wqhjtubrazdktobhdrk3zb/o0viu1wu3khes6vtxgjpqpdxzdb6nsnovo3lrnn0el7rdcdpq/izqcgmd5gj43nfpexsx4pn29nbecsqpmxlzl1osmi3gft6h0vhbnagvkrjrwauzfnumokbrovvgkwmv/0+e2yjncmpvjw+qldsmgpw/cvjl1cbzsatdwz2w1dhoxxef1pw1f3kqen1hjxlwga8obfwwtjpa7qidpolfkar8zmek04178srgikha0cvt7e8639ncs0r6ktnbrr0lceng9dibe9heawwxpoyetnhkoais7bhevw4ddz2tq6gnx8rbiv27o6bs2oxmfoxy0ny9tyt/lrtvrng2rrkqk4eknr5uvakchuvxgzwbze4spkdmco8v2iz6ytj/osi/hgt9yhfbqb+xxl3kzb+7+6qdxwa5tx6qbzkqp2sxdmg+griwva8+bp+dl+gh8fp4nfx2whogbc89ci7ch78af/p+wq==</latexit> <latexit sha1_base64="2gdtihrvnudqyapp+wrzun7ieno=">aaacm3icbvbna9tafhxyvxknaz32mmvseldbdsrtahowghipguicdrkwllfar+wluyux+1rshkb/pvde+kd6kiqcukqv+q9d2zm0tgcwhpn3edut5fjy9p0rr/bg4aphtzy261tpt589b+y8olnzyrgfsexm5ikhlkuh+qafsn6rg05vivl5cnm08m8/cwnfpj/ipocjrsdapijrdflcod6kqkvxalszz3je9zrbqbcvtkohmfajgdtt5qz1orsfikvv86voawlxyphgiswi1yq6jgytctmr4sae3/gxipdjcef2+u/wy2caoikb38nxxgrfntjjrr0gfo6jkhoutpkqhhaw55rd0gkfoqqp4nzuljnxzn8py5jmxj2nzkn+vvfsze1cjw5ykc+uewvxf96wwprwvaqdf8g1wx1kc0kwi4scyvgyzldohahmcpdxwqbuuiau5rorivipfj+cdtub3wloxrtvyiun2ivx0iqa3kif3smjdidbv/gbn/dt++zde7+836vrmne38xl+gxf7b4geqou=</latexit> <latexit sha1_base64="onz3ztpxmfyikfm7vrei/tk6g3u=">aaacm3icbvbbsxtbgj1nraaxalqpvqwjqgi27gah7ueiejfcqcekqja7ze5mzedm7dlzbum67m/pvrf/iadbeqiiv/+dk8sdmj4yelz3fxzzxpqkbsb1b5zsm5w3q2vld5x19xubw9uph3smytrlxzqirj9fxddbfescb8hous2ijatrrxehm7//k2nde3uk05qnjtlxpoaugjxc6vfdwjcexlrmaskmxuhd22s3/v2fkwjdwmonudfpfvznjsnchrhf8ap7mgzaqokngwwgbezv4le+kcjq3w25c+bl4j2reucb/p41ebxjshrljxkasaaacmlmwhntgozea6ecfru/mywl9ikcs4glikhmhvk8c4f3rdlccaltu4dn6vonnehjpjkyk7n85ru3e//ndtkivw5zrtimmkklq3emmcr4viaecc0oikklhgpu/4rpmghcwdzcssv4rymvk1675bkt78s2sy8wkknpqiyayenfuacdowpurrt9qdfoh7p1lp2/zp1zvxgtou872+gfnidh192rhq==</latexit> <latexit sha1_base64="onz3ztpxmfyikfm7vrei/tk6g3u=">aaacm3icbvbbsxtbgj1nraaxalqpvqwjqgi27gah7ueiejfcqcekqja7ze5mzedm7dlzbum67m/pvrf/iadbeqiiv/+dk8sdmj4yelz3fxzzxpqkbsb1b5zsm5w3q2vld5x19xubw9uph3smytrlxzqirj9fxddbfescb8hous2ijatrrxehm7//k2nde3uk05qnjtlxpoaugjxc6vfdwjcexlrmaskmxuhd22s3/v2fkwjdwmonudfpfvznjsnchrhf8ap7mgzaqokngwwgbezv4le+kcjq3w25c+bl4j2reucb/p41ebxjshrljxkasaaacmlmwhntgozea6ecfru/mywl9ikcs4glikhmhvk8c4f3rdlccaltu4dn6vonnehjpjkyk7n85ru3e//ndtkivw5zrtimmkklq3emmcr4viaecc0oikklhgpu/4rpmghcwdzcssv4rymvk1675bkt78s2sy8wkknpqiyayenfuacdowpurrt9qdfoh7p1lp2/zp1zvxgtou872+gfnidh192rhq==</latexit> <latexit sha1_base64="2wmtmq0gvwd0obzg5hizcossbry=">aaacm3icbvbnsysxfm34/kxf9b2lm2arwtcakyjubmhn44ggyfxodizmmrhbjdmkd6rlmp/kxj/iqhaxt8tt+w8vrv34dsbwoodebs6jmykseplotf2ynpmdm1+olc4tr6xw136e2zq3jldzklnzgvplpdc8dqikv8wmpyqw/ck+phr5fzfcwjhqmxhmvkvolrajybscffx/hiwbota3qshsoswp6v5oqxfsbujdreif15p6oled2fxfhtogzxima+hzojhcg1a1whyotncvgjrrtuaazaz8lfgtukmtnetv+6cxslxxduxsazs+yabbuaocsv5wgtzyjljresu7jmqquo0w48wl3nrkdyepcu8dhqvvnwqqrb2q2e2o8tnp3kj8zuvkkox3c6gzhlhmb4esxgji8aha3bogm5bdrygzwv0vsz41ligruejk8d9h/krow02fnp1tujvcndqxj9brbqojh+2hq/qbnaa2yugwpac/6nm78568f+/1bxtkm+z8qh/g/fspte+ocw==</latexit> Sloppy Monomials (a) Fitting Polynomials to Data C poly =(1/2) Z 1 0 (f(x) MX m=0 m x m ) 2 dx (a) Note that the first derivative of the cost C poly is zero at the best fit. Show that the Hessian second derivative of the cost is H poly mn C n = 1 m + n +1. (1) The Hilbert matrix is famous for being ill-conditioned too. The monomial coefficients in a polynomial fit are remarkably poorly determined.

36 Hierarchy of widths and curvatures Hierarchy of widths 10 7 W 4 W W 3 Cross sections: fixing f at 0, ½, 1 Theorem: interpolation good with many data points Geometrical convergence 10-5 Eigendirection at best fit Multi-decade span of widths, curvatures, eigenvalues Widths ~ λ sloppy eigs Parameter curvature K P = 10 3 K >> extrinsic curvature

37 Why is it so thin and flat? Follows from model analyticity in control parameters P 3 (t) 2Δf 5 Model f(t,θ) analytic: f (n) (t)/n! R -n Polynomial fit P m-1 (t) to f(t 1 ),,f(t m ) Interpolation convergence theorem Δf m+1 = f(t)-p m-1 (t) < (t-t 1 ) (t-t 2 ) (t-t m ) f (m) (ξ)/m! ~ (Δt / R) m More than one data per R

38 <latexit sha1_base64="2gdtihrvnudqyapp+wrzun7ieno=">aaacm3icbvbna9tafhxyvxknaz32mmvseldbdsrtahowghipguicdrkwllfar+wluyux+1rshkb/pvde+kd6kiqcukqv+q9d2zm0tgcwhpn3edut5fjy9p0rr/bg4aphtzy261tpt589b+y8olnzyrgfsexm5ikhlkuh+qafsn6rg05vivl5cnm08m8/cwnfpj/ipocjrsdapijrdflcod6kqkvxalszz3je9zrbqbcvtkohmfajgdtt5qz1orsfikvv86voawlxyphgiswi1yq6jgytctmr4sae3/gxipdjcef2+u/wy2caoikb38nxxgrfntjjrr0gfo6jkhoutpkqhhaw55rd0gkfoqqp4nzuljnxzn8py5jmxj2nzkn+vvfsze1cjw5ykc+uewvxf96wwprwvaqdf8g1wx1kc0kwi4scyvgyzldohahmcpdxwqbuuiau5rorivipfj+cdtub3wloxrtvyiun2ivx0iqa3kif3smjdidbv/gbn/dt++zde7+836vrmne38xl+gxf7b4geqou=</latexit> <latexit sha1_base64="onz3ztpxmfyikfm7vrei/tk6g3u=">aaacm3icbvbbsxtbgj1nraaxalqpvqwjqgi27gah7ueiejfcqcekqja7ze5mzedm7dlzbum67m/pvrf/iadbeqiiv/+dk8sdmj4yelz3fxzzxpqkbsb1b5zsm5w3q2vld5x19xubw9uph3smytrlxzqirj9fxddbfescb8hous2ijatrrxehm7//k2nde3uk05qnjtlxpoaugjxc6vfdwjcexlrmaskmxuhd22s3/v2fkwjdwmonudfpfvznjsnchrhf8ap7mgzaqokngwwgbezv4le+kcjq3w25c+bl4j2reucb/p41ebxjshrljxkasaaacmlmwhntgozea6ecfru/mywl9ikcs4glikhmhvk8c4f3rdlccaltu4dn6vonnehjpjkyk7n85ru3e//ndtkivw5zrtimmkklq3emmcr4viaecc0oikklhgpu/4rpmghcwdzcssv4rymvk1675bkt78s2sy8wkknpqiyayenfuacdowpurrt9qdfoh7p1lp2/zp1zvxgtou872+gfnidh192rhq==</latexit> <latexit sha1_base64="onz3ztpxmfyikfm7vrei/tk6g3u=">aaacm3icbvbbsxtbgj1nraaxalqpvqwjqgi27gah7ueiejfcqcekqja7ze5mzedm7dlzbum67m/pvrf/iadbeqiiv/+dk8sdmj4yelz3fxzzxpqkbsb1b5zsm5w3q2vld5x19xubw9uph3smytrlxzqirj9fxddbfescb8hous2ijatrrxehm7//k2nde3uk05qnjtlxpoaugjxc6vfdwjcexlrmaskmxuhd22s3/v2fkwjdwmonudfpfvznjsnchrhf8ap7mgzaqokngwwgbezv4le+kcjq3w25c+bl4j2reucb/p41ebxjshrljxkasaaacmlmwhntgozea6ecfru/mywl9ikcs4glikhmhvk8c4f3rdlccaltu4dn6vonnehjpjkyk7n85ru3e//ndtkivw5zrtimmkklq3emmcr4viaecc0oikklhgpu/4rpmghcwdzcssv4rymvk1675bkt78s2sy8wkknpqiyayenfuacdowpurrt9qdfoh7p1lp2/zp1zvxgtou872+gfnidh192rhq==</latexit> <latexit sha1_base64="2wmtmq0gvwd0obzg5hizcossbry=">aaacm3icbvbnsysxfm34/kxf9b2lm2arwtcakyjubmhn44ggyfxodizmmrhbjdmkd6rlmp/kxj/iqhaxt8tt+w8vrv34dsbwoodebs6jmykseplotf2ynpmdm1+olc4tr6xw136e2zq3jldzklnzgvplpdc8dqikv8wmpyqw/ck+phr5fzfcwjhqmxhmvkvolrajybscffx/hiwbota3qshsoswp6v5oqxfsbujdreif15p6oled2fxfhtogzxima+hzojhcg1a1whyotncvgjrrtuaazaz8lfgtukmtnetv+6cxslxxduxsazs+yabbuaocsv5wgtzyjljresu7jmqquo0w48wl3nrkdyepcu8dhqvvnwqqrb2q2e2o8tnp3kj8zuvkkox3c6gzhlhmb4esxgji8aha3bogm5bdrygzwv0vsz41ligruejk8d9h/krow02fnp1tujvcndqxj9brbqojh+2hq/qbnaa2yugwpac/6nm78568f+/1bxtkm+z8qh/g/fspte+ocw==</latexit> Sloppy Monomials Fitting Polynomials to Data C poly =(1/2) Z 1 0 (f(x) MX m=0 m x m ) 2 dx The interpolation theorem says that a fit to m data points by a theory with bounded derivative f (m) ( ) < Am! can di er from the m 1 degree polynomial fit P m 1 (t) by at most f m+1 = f(t) P m 1 (t) <A(t t 1 )(t t 2 )...(t t m ) (1) [i] Why can m points be fit by an m 1 degree polynomial? Why is the di erence zero at all the data points? [ii] What does the bound on f (m) ( ) tell us about the maximum value of t m in a polynomial fit to the function? [iii] Given this bound on the derivative, can any fit have bigger range (thicker model manifold cross-section) than polynomials? <latexit sha1_base64="ew7po24pnk7os6f9gihiyoju2lk=">aaaeoxicdvpbbtnaehwbacxcwnheqgmjuijifvdi8abqezwab1bavuljgq3tcblqxsluotsk+s7+gzf+hvemtgkk++lxzsyzm2dmo5hg1rvav1dws+vr12+s3azcun3n7r31jfshvmcmxv1yc22oimzrcix7jjubryodteycd6otd4x/cizgcq32xd7cnmqdxvmem0dx/y3vn12luupqucreeierh2akhxedg6i2kmgy3nipc8ag5q6chpqsqcicgxglowtrtj4fn8n37oyq6yxge0jq8dhbjrfqz+nxpc4b03r3ldfo4dw8bfm4bjftkpa0rqop0bkaifxmsbvwybcphsivlpwjx77w7k/ipa27rs0xdic1dzvuhaoujvgt8/snle4ucqkyuvizcapv4cylhgjcbccmrd01xt9s+m9wo5toz70tg5uuqmqbsnkuv4f34hcye/afgn+fqm+xokx+18w2t+twdzrrhfwm8aonlrphqsxcfwjvlxb2lsku0tjd8gidtwxj4ho4jkzmaqmq52mlo+uykzbmikpqkdtcmdhnctiy0dtniipnvfx0f5kdkxhpuy22hdo5z+b5n8/8uzehqdzjdrltqcqha5q0ywqajpyqxyf0k3wcgrjrcmqrqgy0tu2lvm6j2d21qi7u6k9xw5stf+cqec6naja/7f76lxprnekihwtmbsdsjvxvwozjsudalcziimunbiadmhwtahst/8km8jruekg1lahwdvztysaeswtzgvgkzg5ol33f5b98ncylr3otrkazow2yfuoz4zwn50rlyugdkzpbysojk8rdzlhmz9qwiotllv81dry2w9zm+gwruvnilsda8db4etsdmhgz7aqfgnawh8slr6xd0qfs53k1/lhcln+dha6uzhmebjdoufmh3fl9bg==</latexit>

39 Hyperellipsoid bounds on model manifold Katherine Quinn Consider models f(t) with similar radius of variation X (f (n) /n!) 2 <NR n Triangle = Model manifold, fitting exponentials Ellipsoid = Bound for any theory with R=1 Any prediction must be contained in a hyperellipsoid whose eigenvalues are exponentially separated If R=1, parameter sphere of radius N maps via Vandermonde J to hyperribbon ellipsoid, axes = singular values = λ

40 Physics: Sloppiness and Emergence Ben Machta, Ricky Chachra, Mark Transtrum Emergence of distilled laws from microscopic complexity Ising: long bonds Diffusion: long hops Irrelevant on macroscale Both sloppy at long-wavelengths

41 Sloppiness and Continuum Limits Ben Machta, Ricky Chachra, Mark Transtrum Microscopic hopping gives continuum diffusion equation Sloppy after coarse graining in time Densities! 0.14! 0.07! 10 2! 10! 0! +10! 10! 0! +10! 10! 0! +10! 10! 0! +10! Sites! 10 0! Eigenvalues! 10 2! 10 4! 10 6! 10 8! 3! 2! 1! 0! +1! +2! +3! Sites! 1! 2! 3! 4! 5! 6! 7! Diffusion time steps!

42 Eigenvalue Sloppiness and Critical Points Ben Machta, Ricky Chachra, Mark Transtrum Ising model with longrange bonds Fisher information metric Sloppy after coarse graining in space Configurations! 10 5! 10 3! Eigenvalues! 10 1! 10 1! Tc 10 3! 10 5! 0! 1! 2! 3! 4! 5! 6! Coarsening Steps!

43 Fitting Decaying Exponentials Classic ill-posed inverse problem Given Geiger counter measurements from a radioactive pile, can we recover the identity of the elements and/or predict future radioactivity? Good fits with bad decay rates! Ensemble: Ensemble: Interpolation Extrapolation Fit P, S, I y(a,γ,t) = A1 e -γ 1 t +A2 e -γ 2 t +A3 e -γ 3 t 6 Parameter Fit

44 <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="hp+6lruf2d3tzaldqaqqvekmxyw=">aaab2xicbzdnsgmxfixv1l86vq1rn8eiucozbnqpuhfzwbzco5rm5k4bmskmyr2hdh0bf25efc93vo3pz0jbdwq+zknivscullqubn9ebwd3b/+gfugfnfzjk9nmo2fz0gjsilzl5jnmfpxu2cvjcp8lgzylffbj6f0i77+gstlxtzqrmmr4wmtuck7o6oyaraadlmw2ivxdc9yanb+gss7kddujxa0dhefbucunsafw7g9liwuxuz7ggupnm7rrtrxzzi6dk7a0n+5oykv394ukz9bostjdzdhn7ga2mp/lbiwlt1eldvesarh6kc0vo5wtdmajnchizrxwyasblykjn1yqa8z3hysbg29d77odbu3wmya6nmmfxeein3ahd9cblghi4bxevyn35n2suqp569lo4i+8zx84xio4</latexit> <latexit sha1_base64="twz//b1wptknwtc8ab/3hb+vtyi=">aaadnnicdvjnb9naen0kfbrtooxkzuqdjnbgnhe4vejcqlweucs1auscmvv6hk9qr83umcay8tv4d9z4n2zislqjzgxfvtn5ejthkupdrvu70wzduxvv/s4d5+huo8dp9vz3hyyvtccbynncj0jumjukbyqpxvghkwdhihfh1cel/+ihainzdu7zaicznyozs8hjusf+46evcqkivor0rbc+5rpsbsazaienhsj1ghqbeosiewdulwxqgawlc4glqth32n7gkqnjyj9pkpjieomeuo1dmel+bfnwwiutuse4qwmkvyancmsokqrms+nirtjvphx8ekdsvfi9xpx7eugcbjxp0ylh1muzwzf0+pfn0o/e5h3ysgpobxxvtxy1/t9d35yp3et7xdshfoco67bujzdhxygqwjg+quhmc3+16norh+vr1gpuhhad0sezoyevbfhmvdjy9spp3yxig8bz0tnyaaldunnaxeh4w1y7f+wdud13zbanvdu4ygs7dfz++veuysz+uei5mwpplwhscu1spghnkg0wxfzxky4tvnw2o6lwm7aaf5ajilzletvfghv7m6limthzllqvlxtgnn1l8l++cunx+0klvvesklexissukifl2kikdb1xkercs9sriirrlsgut2nf8dzh3gbdtz3p7xlnltthz9hz1meee8c+sbn2ygzmnf82pzchzwhrteusnarlajbwuj1lt6z19q9pnx/+</latexit> <latexit sha1_base64="twz//b1wptknwtc8ab/3hb+vtyi=">aaadnnicdvjnb9naen0kfbrtooxkzuqdjnbgnhe4vejcqlweucs1auscmvv6hk9qr83umcay8tv4d9z4n2zislqjzgxfvtn5ejthkupdrvu70wzduxvv/s4d5+huo8dp9vz3hyyvtccbynncj0jumjukbyqpxvghkwdhihfh1cel/+ihainzdu7zaicznyozs8hjusf+46evcqkivor0rbc+5rpsbsazaienhsj1ghqbeosiewdulwxqgawlc4glqth32n7gkqnjyj9pkpjieomeuo1dmel+bfnwwiutuse4qwmkvyancmsokqrms+nirtjvphx8ekdsvfi9xpx7eugcbjxp0ylh1muzwzf0+pfn0o/e5h3ysgpobxxvtxy1/t9d35yp3et7xdshfoco67bujzdhxygqwjg+quhmc3+16norh+vr1gpuhhad0sezoyevbfhmvdjy9spp3yxig8bz0tnyaaldunnaxeh4w1y7f+wdud13zbanvdu4ygs7dfz++veuysz+uei5mwpplwhscu1spghnkg0wxfzxky4tvnw2o6lwm7aaf5ajilzletvfghv7m6limthzllqvlxtgnn1l8l++cunx+0klvvesklexissukifl2kikdb1xkercs9sriirrlsgut2nf8dzh3gbdtz3p7xlnltthz9hz1meee8c+sbn2ygzmnf82pzchzwhrteusnarlajbwuj1lt6z19q9pnx/+</latexit> <latexit sha1_base64="xmo7sm462crxyzz8nojyd1fzgt4=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjdrenkkcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvsm4pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxd0dub7a//a29p9t1zhuvpmee50hd957+w6e86+m3je62xrc2vuompvtw/aj+2vtwhrbznz1llmbfeh0nghag==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="/qy9oqktxlqen+yql6lb6tirp9w=">aaadqxicdvlbbtnaehutlsxcwnjkzuqdjbaiu0kcl0qvkfavidrktvtrp2a8hser2muzo6anrhwb/8abf8mmjksbwrzs2toxnzk9uzljw573e63vvnx7zt31e+79bw8fpd7yfhjkikolgokik/rjhiyyqwjekjm6ktvhhmv0hj1/npupf5a2slchpc1pnoneyuqkzeufm2s/a1vifznityt68knqygsdxziw0laflllsbjwsxmgiac8l6yqez1nijem0dttbjpxgsr0cpsq421m5h16ndyytlmwd5euxurigptjgogjpac2jwkqqqiwrfdney+ygeu2kqul7tej39czdc+saszjf67kvyqe6w7ndgpau8y6s2a4mw5qbkfmd/xcy2odd78wecu9h2ov+k9zr8dytb3hnmzcgfv9t7u+uhiikadmawsxn0gkqbixzmvcla9/slrs8fov2rj5ig6dlsvs24pnuwoyg7khxv8f1buhgljfwfgy3gb8ew87s9sonx0fcicq3py4ynobu90oe16hziozstjwhesu5tujuqow22xg9unomxlgmhsskjlg/bqv2akanuthtplkrcwwyvd+c/jfvtolkw7iwqqyylggesqomuic5bcgwulfclffoaxsfkajgwvbcrl2cvzrytxc0pfc9gx+wvbx7brmodeez89zpor7z3tl19px9z+si1svw59aoddr+0z5on7s/nqgttwxou+eatcuf0xghba==</latexit> <latexit sha1_base64="aqfnoakqxlq7/krkbihyamceupo=">aaaco3icbva9axtbej1zpmwrh5at0s0se5akibsqsf0ydglsbsdytkgnhhorownx3ge7c8biomif8x9wkz+rlk2afdembfrsss5ioq8whu+9yxzexghl2fe/eysphj56vlq23nry9nnzjfbmi2obl0bsqoy6n6cxwtiqower1nragmi01nqsn71r/jnzmlbl2rhpchqlomluoisyk6l2p1kupsjtokncoykx1h3uin0r2jknqhb1mcv9v/5cfegftqi5iuqlwyr0uxr6dzxu9qp2tt/35xd3sxbltg/2+pilabxg7w/hojdlshlljdyoa7/guywgldrut8lsuohydcc0ddtdloyomt9ei9dogyskn+5llobqvxmvptbo0tglmzvtstei//ogjse7o0plrcmuycwipnscc9eukcbkkgq9cwslue6vqk7rogrxd8uvecyffj8cv+khfj/46np4cwuswra8gg4esamh8b4oyqasruah/ijr76v307vxfi+ik97tzeu4a+/px+krr3k=</latexit> <latexit sha1_base64="igdgc+i9dkg8ebreq00940xh6pk=">aaaco3icbvc7bhnbfj015iehiygszuqiysls7skkjiulszspuedxq/ka1d3xxxvk2ydm7ky2q/0c/ienp0fhq0mbqrt0znopcmkrrjo651zdusfmldtkul+d2ooho7t7+4/qj58chb41nj4bmdtxavsivakehwbqyqt7jenhknmicahwgc7evp7wgrwraxjfqwwnmcwsgukbzkwg8x4v+dhqpiwk/2qobgwltnix+yapg8ihlc2h65yfirdtr+q+vzfgi3mfl1mrva3rssdohlsddw1+n3i35lh3tp8+llxzmmh88aepygnmscgwzuy5gu0k0csfwrlu5wyzeauy4djsbgi0k2j9e8lfwmxko1tblxbfq/9ofbabs4pdm6zunhe9svyfn84popsumslywkrsfkw54ptyqkg+lrofqzulils0f+vidhoe2brrtgtv7sn3yebvx3m73jvbxmu2wt57wzqsxtx2ynrsgl2yphpshn1jp9hp57pz3fnl/n5ea87tzho2befpxzj5shk=</latexit> <latexit sha1_base64="igdgc+i9dkg8ebreq00940xh6pk=">aaaco3icbvc7bhnbfj015iehiygszuqiysls7skkjiulszspuedxq/ka1d3xxxvk2ydm7ky2q/0c/ienp0fhq0mbqrt0znopcmkrrjo651zdusfmldtkul+d2ooho7t7+4/qj58chb41nj4bmdtxavsivakehwbqyqt7jenhknmicahwgc7evp7wgrwraxjfqwwnmcwsgukbzkwg8x4v+dhqpiwk/2qobgwltnix+yapg8ihlc2h65yfirdtr+q+vzfgi3mfl1mrva3rssdohlsddw1+n3i35lh3tp8+llxzmmh88aepygnmscgwzuy5gu0k0csfwrlu5wyzeauy4djsbgi0k2j9e8lfwmxko1tblxbfq/9ofbabs4pdm6zunhe9svyfn84popsumslywkrsfkw54ptyqkg+lrofqzulils0f+vidhoe2brrtgtv7sn3yebvx3m73jvbxmu2wt57wzqsxtx2ynrsgl2yphpshn1jp9hp57pz3fnl/n5ea87tzho2befpxzj5shk=</latexit> <latexit sha1_base64="h+xl/fpf2spojc9ophd1e9m6zui=">aaaco3icbvbnsynbfoxrd1eju2b16kuxcpfgmbfblwhbiydrmspk4vcm88y09nzq/wbzmmz/8ukf8obfiwdfvhq3j8nbr4kgoqoer1+fmzkgxpfomzqe+fhz1+xcbx7h95/f+t+lu5pmwmbhpcrv5yeyvdlbdklsej5phdhuebze7vx+2t/urqbjcq0z7mvwmchiciarbfxjyedhqimwkvytarkutvrnbe6bpa4kh1q2glzbxhqhg17jfaoiwvjmpv7pmhsfnvpvbfwg23jh4f+jnyennsfhul/1+6niy0xikdcm67kz9qrqjixcsubnbjmqv3cjxustinh0ithtjv+zsp9hqbyvit5s308uebszjeobro40n71k/m7r5htt9aqzzdlhisalolxxsnlvjo9ljylu0biqwtq/cjeadyjs3tvbgvf55k/kdlplus3vyg3sbk3qmgurbju1mce22s7bz4eswws7zvfskt05n86d8+y8jkntzmrmmx2a8/ogfwutzw==</latexit> <latexit sha1_base64="qyx86aunuwf0wsxnaycktphjcfu=">aaachxicbvdjsgnbek1xjxgly81lowh6itmi6euqviinceaftbhroj2mswehu0yiq37ei7/ixymihryif2npoud2ooh1e1xvxs/mldtkuu/oyojy+mrkzao6ptm7n19bwdwzaa65apjupfoircoutestjclxkwmbcajeexh9wprnn0ibmsan1mteo8arreasi1kpqo0cb4x0uwvd7ln95meosajivq0k5obw192nriamhoxvolbm1t0b2f/ifzk1g+uougsarlb79tspz2orefdotmtzm2ox5wyurl/q50zkyk/xsrqsttawpl0mtuuzdat0wjrqexjia/v7r4gxmb04tjuxutf89krxp6+vu7txlmss5sqspnwoyhwjljvrsy7ugppqwyjcs/txxruokzmntazb+73yx3k2xffcundi09ibisqwaquwar7swgecqqoawoew7uernpw758f5dl6gpspoz88s/idz9ghx76mu</latexit> <latexit sha1_base64="brpwwpfghf3vdbxhyvp7gbzznmw=">aaachxicbvdlsgmxfm3ud33vx85nuis6qtns0i0gubfxfawknwg4k2bayozbckcyhv6ig3/fjqtfxlgr/8zmw0grbwin59x7k3ucvaqntv1pvaamz2bn5heqi0vlk6u1tfurnwsk8tzlzkjuatbcipi3uadkn6nieawsxwe3p6v/fcevfkl8ixnkvqh6sqgfazssx2ue+4vwqaz9gnbj6qaguicker19sbf/fxexzxh8uwhvr+3ydxsi+pc4y7jzshko0fjr7243yvney2qsto44dopeuc5mkg+qbqz5cuwwerxjaawr114x3g5ad43spwgizimrdtwfhqvewudrycojwl6e9erxp6+tyxjkfsjom+qxgz0uzpjiqsuoafcozldmhgbtwvyvsj4oyggclunwjlf+s64ogo7dcc5mgk0ywjzzitukthxyse7igwmrnmhknjysz/jipvhp1qv1niqtwooedfil1scxeomkzg==</latexit> <latexit sha1_base64="brpwwpfghf3vdbxhyvp7gbzznmw=">aaachxicbvdlsgmxfm3ud33vx85nuis6qtns0i0gubfxfawknwg4k2bayozbckcyhv6ig3/fjqtfxlgr/8zmw0grbwin59x7k3ucvaqntv1pvaamz2bn5heqi0vlk6u1tfurnwsk8tzlzkjuatbcipi3uadkn6nieawsxwe3p6v/fcevfkl8ixnkvqh6sqgfazssx2ue+4vwqaz9gnbj6qaguicker19sbf/fxexzxh8uwhvr+3ydxsi+pc4y7jzshko0fjr7243yvney2qsto44dopeuc5mkg+qbqz5cuwwerxjaawr114x3g5ad43spwgizimrdtwfhqvewudrycojwl6e9erxp6+tyxjkfsjom+qxgz0uzpjiqsuoafcozldmhgbtwvyvsj4oyggclunwjlf+s64ogo7dcc5mgk0ywjzzitukthxyse7igwmrnmhknjysz/jipvhp1qv1niqtwooedfil1scxeomkzg==</latexit> <latexit sha1_base64="koqm8kloinwmiiudgnatfhpfzki=">aaachxicbvdlssnafj3uv62vqes3g0wom5piqtdcwy24qmbrosnhzjpth04eznwiifrh3pgrblwo4skn+dcmbqvtptbw5px778w9xisfrsv6mgplyyura8x10sbm1vaoubvx0mgsgg+yuiaq7yhmugs8iqilb0ekg+9jfuenlnp/7p4rlclgfpoid30ybkivggamuwbt2k2faziawpheuccchqiktsroiuotn7udq47gtgtlrlm2qtyedjhym1immzrc88pphsz2eybmgtyd24qwm+azmetjkhnrhgebwyb3mhqaz3u3nww3pkez0qp9ugunqdprf3ek4gud+f5w6qmo9byxi/95nrj7591ubfgmpgdth/qxpbjspcrae4ozlelggcmr/zwyishgmawah2dpr7xiwqdv26ran1a5xpvfusqh5jbuie3osj1ckqzpekyeybn5ia/go/fsvbnv09kcmevzj39gfh4d0wohrg==</latexit> Sloppy Minimization (c) Decaying exponentials y(t) A B fit y (t) = NX 1 =0 exp( t). 0.5 J i i )/@ t (c) For our exercise, where the data are perfectly fit by = 0, show that the cost Hessian is a continuous integral H =(J T J) = J t J t = Z 1 0 J(t, )J(t, )dt where the Jacobian is now the 1 N matrix J(t, ) =exp( t)(1 t).

45 Intensive embeddings: Ising, CMB, digits Katherine Quinn ferromagnetic positive field slowly expanding universes our universe (behind) negative field rapidly expanding universes universes with universes with large primordial small primordial density fluctuations density fluctuations anti-ferromagnetic (b) Cosmic Microwave Background (a) Ising Model of Atomic Spins (c) Clasifying Hand-Written Digits Probabilistic models gµ = Fisher Information Metric Isometric embedding Orthogonality catastrophe Replica theory extrapolate to zero data

46 <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> Exercise 1.15 Metrics in Probability Space Fisher information 2 log P (x) g ( x = Z dx P log Natural metric in space of probability distributions. Gives distance between nearby probabilities (d(θ+δ)-d(θ)) 2 = Dα gαβ Dβ Agrees with least-squares distance. Ease of distinguishing along different parameter directions

47 <latexit sha1_base64="nl3rm0rceu9s6nv/n8kzd9fcizu=">aaadunicbvjnb9naelutpoopkmkry4oyyzfi4kricegqvalbvksgjm2lbgknn+t4lfvhd9c0kffvrejc+cfcoabrj5r+mplhed7mvnmj8vpophld72atfuv2nbs796z7uw8epmrspt6wssyihzoej+lub0k5i+lymcxpasoord6nj/7yoorpplehwrkp1dql0wgwmqsyaavt3p4z4jhh8zzgyjqkifvbnmugqeqkkcqyraghi1chipldo8kpsb/kq+lzx4hhivvqtcye3ibmv6ntxjklpiactcccreq61v55rdxrd/sis7aiwgozfqulu5hmpyblmvb5h6fsh1lobqv7bcgxlnxoykrhb6ed4ctjokilx0dv339a9sllmfa0boyx49gosaqqwx7zlaufffazz2qn7cve7a46cpnz3qmkmf1jza9mizswx6dzkapq2qmvzxurqnt1capcmec59ll0l/42jr1totzzebk9rtptufwgm6c3bu1jg0ov8rxpe5jf2jthiowk56zqmpcqhnpcwpmkkzallohewxgikqd5drifeq4z82ohqrirvguvd+qqsbmoff1zrkbe58rk/7hjpolx05zfaazotdzcqcarslb5x2jobcwkrzuaipiefzeqbbclr9dss+hdt3wthpc7pbft+9hv7r/crmphego8mxyjz7wy9o13xtayg8t8yv4wf5m/a99qp+tmvb4prznbnifglajv/gfwfwyj</latexit> <latexit sha1_base64="nl3rm0rceu9s6nv/n8kzd9fcizu=">aaadunicbvjnb9naelutpoopkmkry4oyyzfi4kricegqvalbvksgjm2lbgknn+t4lfvhd9c0kffvrejc+cfcoabrj5r+mplhed7mvnmj8vpophld72atfuv2nbs796z7uw8epmrspt6wssyihzoej+lub0k5i+lymcxpasoord6nj/7yoorpplehwrkp1dql0wgwmqsyaavt3p4z4jhh8zzgyjqkifvbnmugqeqkkcqyraghi1chipldo8kpsb/kq+lzx4hhivvqtcye3ibmv6ntxjklpiactcccreq61v55rdxrd/sis7aiwgozfqulu5hmpyblmvb5h6fsh1lobqv7bcgxlnxoykrhb6ed4ctjokilx0dv339a9sllmfa0boyx49gosaqqwx7zlaufffazz2qn7cve7a46cpnz3qmkmf1jza9mizswx6dzkapq2qmvzxurqnt1capcmec59ll0l/42jr1totzzebk9rtptufwgm6c3bu1jg0ov8rxpe5jf2jthiowk56zqmpcqhnpcwpmkkzallohewxgikqd5drifeq4z82ohqrirvguvd+qqsbmoff1zrkbe58rk/7hjpolx05zfaazotdzcqcarslb5x2jobcwkrzuaipiefzeqbbclr9dss+hdt3wthpc7pbft+9hv7r/crmphego8mxyjz7wy9o13xtayg8t8yv4wf5m/a99qp+tmvb4prznbnifglajv/gfwfwyj</latexit> <latexit sha1_base64="nl3rm0rceu9s6nv/n8kzd9fcizu=">aaadunicbvjnb9naelutpoopkmkry4oyyzfi4kricegqvalbvksgjm2lbgknn+t4lfvhd9c0kffvrejc+cfcoabrj5r+mplhed7mvnmj8vpophld72atfuv2nbs796z7uw8epmrspt6wssyihzoej+lub0k5i+lymcxpasoord6nj/7yoorpplehwrkp1dql0wgwmqsyaavt3p4z4jhh8zzgyjqkifvbnmugqeqkkcqyraghi1chipldo8kpsb/kq+lzx4hhivvqtcye3ibmv6ntxjklpiactcccreq61v55rdxrd/sis7aiwgozfqulu5hmpyblmvb5h6fsh1lobqv7bcgxlnxoykrhb6ed4ctjokilx0dv339a9sllmfa0boyx49gosaqqwx7zlaufffazz2qn7cve7a46cpnz3qmkmf1jza9mizswx6dzkapq2qmvzxurqnt1capcmec59ll0l/42jr1totzzebk9rtptufwgm6c3bu1jg0ov8rxpe5jf2jthiowk56zqmpcqhnpcwpmkkzallohewxgikqd5drifeq4z82ohqrirvguvd+qqsbmoff1zrkbe58rk/7hjpolx05zfaazotdzcqcarslb5x2jobcwkrzuaipiefzeqbbclr9dss+hdt3wthpc7pbft+9hv7r/crmphego8mxyjz7wy9o13xtayg8t8yv4wf5m/a99qp+tmvb4prznbnifglajv/gfwfwyj</latexit> <latexit sha1_base64="nl3rm0rceu9s6nv/n8kzd9fcizu=">aaadunicbvjnb9naelutpoopkmkry4oyyzfi4kricegqvalbvksgjm2lbgknn+t4lfvhd9c0kffvrejc+cfcoabrj5r+mplhed7mvnmj8vpophld72atfuv2nbs796z7uw8epmrspt6wssyihzoej+lub0k5i+lymcxpasoord6nj/7yoorpplehwrkp1dql0wgwmqsyaavt3p4z4jhh8zzgyjqkifvbnmugqeqkkcqyraghi1chipldo8kpsb/kq+lzx4hhivvqtcye3ibmv6ntxjklpiactcccreq61v55rdxrd/sis7aiwgozfqulu5hmpyblmvb5h6fsh1lobqv7bcgxlnxoykrhb6ed4ctjokilx0dv339a9sllmfa0boyx49gosaqqwx7zlaufffazz2qn7cve7a46cpnz3qmkmf1jza9mizswx6dzkapq2qmvzxurqnt1capcmec59ll0l/42jr1totzzebk9rtptufwgm6c3bu1jg0ov8rxpe5jf2jthiowk56zqmpcqhnpcwpmkkzallohewxgikqd5drifeq4z82ohqrirvguvd+qqsbmoff1zrkbe58rk/7hjpolx05zfaazotdzcqcarslb5x2jobcwkrzuaipiefzeqbbclr9dss+hdt3wthpc7pbft+9hv7r/crmphego8mxyjz7wy9o13xtayg8t8yv4wf5m/a99qp+tmvb4prznbnifglajv/gfwfwyj</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> <latexit sha1_base64="kzqfif2rt33s91kj8t/zyv/9vae=">aaadbnicjvjbi9nafj4mxtz4axv8cbehi9avlukr9evywbafk2x3fzo1tkynybctszyzyjaqn3+hb/4gx/wjo0m73e7wbq+enhzn9p0vjyoe18b3f3cc99bto3d37nn3hzx89ljbe3kk81ixmlbc5ookohoelzax3ag4krtqlbjwhj0enphjh6a0z+whwrqwy2giecwznryke52fn7loqweu41xllyrts/ezzymrjfxw8l1s8te1l4qvoajiqq0zwg+itu2juckhafo95+gvfclvmbeqgykotargeivkkljqztgv30yvexmcx+soz/vexa/ll2xdubhttaix02u8hbd8lk54khqi2qhhzxpvghoxbs8vcfl2kgfcuvgvsjdx21zibpoeatnf0dxs9v2h3xredokv00crg4fdx2seszidazigwk8dvzczqhngbnj+pyacsloawns6kmagz1v7ljv+bze5tr/bplanft2sqgim9sklbgazr74ea8b/xaalit/oki6l0obky0fxkbdjcxn7em4vmcmw1qfmccsvs5raqy29um+kefxfeds5gg0dfxh8hfx336/k2eg76buaoab9qpvocxqjcwkdp07pee7son/dz+4l9+uy1emsap6ik+yozgegubqw</latexit> Exercise 1.15b, 1.16a Fisher Information Metric in Probability Space Fisher information 2 log P (x) g ( x = Z dx P log Least-squares cost P LS (x ) =exp( P i (y i( ) x i ) 2 /2 2 i )/ Q i p 2 2 i (b) Calculate the FIM g for the least-squares probability P LS. Show that it is J T J,whereJ i =(@y i /@ )/

48 Exercise 1.16c Hellinger distance p P (x) is a point on the unit sphere. Hellinger defines a dot product between probability distributions P and Q: P Q = P p p x P (x) Q(x) The Hellinger distance between P and Q is the straight-line distance between them on the sphere: d 2 Hel(P, Q) =(P Q) 2 =(P P 2P Q + Q Q) Z = dx( p p P (x) Q(x)) 2 =2 2P Q. [i] Argue that Hellinger is a valid distance. [ii] Show that it gives the correct FIM as its metric, up to a constant factor. [iii] Show that d Hel (P, Q) apple p 2. <latexit sha1_base64="irlduconfrtdowxy5oxevspvvqw=">aaae/nicbvrnb9naehwbamv8tccny4galeatjqebqlspcamva1ii+ixfabrer51v7lxzxaduvit+chcoimsv38gnf8nsnkzjm5vsjwbn7bz3dmwvjbjsjca/ldvs+dr1g2s37vu379y9t75x/1almatsgczrio89oljebtvqxefsojwmxf7ejrzho7n/ngjs8uts67ou9wisch5wsjsm+hulh65iupcz0lbjqs9s5+2kgxm98il8y7g6doarijbikyzegb4wyatxonibk6xu77eim4dmgs8czggqc5ff4ccaupn4gdvj8jg+zuyyhec8hnf9bj7kk9zldbmfttsbinxwos627brdavad2afxzxh/ghisyzlndhapu4oar66hfgztfeq9rxvbe0hzjndcdypyqeryhicdxtpexkvgsxxusohftu2o1jcnugvux9hydj0wcpgtiifi2dj2t1qfdbnnygrcaw91qjzm19mdqlrrnepjc0xuyg9ddvr4pjfjot7tuaquow92zf34+yx+mvswovvb5ltrtbxyrw67tpgz+hfaxhhts6np8h68lwhg0ami5zyezm0i4f7mvdrmx0mxi/ttidktkdhtir+xwnyasmmohvcfpgjrukcgzjgtdauyfhscr1ocgjxvq0cotusvsxcod/zzcm28dynwmneao4vo/vpmo96yllgankfbpjvd7f76x9dpabajitqisnwbjvt3cii1pxfd8zlfukkhjgrddawjmerlk893de8w40oqshxqziq7j8hjrnrz7gglkaau75nksr1upom3vzylnnnm0kjrkexgpvmvwgsxhkfmsyrucuqkdeak2ol/dgnc87lkq8fhq95s1jud1ubuy6kda9yj67fvszrwa2vx2rpa1offs3npw+lh6wf5a/l7+vf5d1g6ujlfplawvvnpfwxamvo=</latexit> <latexit sha1_base64="irlduconfrtdowxy5oxevspvvqw=">aaae/nicbvrnb9naehwbamv8tccny4galeatjqebqlspcamva1ii+ixfabrer51v7lxzxaduvit+chcoimsv38gnf8nsnkzjm5vsjwbn7bz3dmwvjbjsjca/ldvs+dr1g2s37vu379y9t75x/1almatsgczrio89oljebtvqxefsojwmxf7ejrzho7n/ngjs8uts67ou9wisch5wsjsm+hulh65iupcz0lbjqs9s5+2kgxm98il8y7g6doarijbikyzegb4wyatxonibk6xu77eim4dmgs8czggqc5ff4ccaupn4gdvj8jg+zuyyhec8hnf9bj7kk9zldbmfttsbinxwos627brdavad2afxzxh/ghisyzlndhapu4oar66hfgztfeq9rxvbe0hzjndcdypyqeryhicdxtpexkvgsxxusohftu2o1jcnugvux9hydj0wcpgtiifi2dj2t1qfdbnnygrcaw91qjzm19mdqlrrnepjc0xuyg9ddvr4pjfjot7tuaquow92zf34+yx+mvswovvb5ltrtbxyrw67tpgz+hfaxhhts6np8h68lwhg0ami5zyezm0i4f7mvdrmx0mxi/ttidktkdhtir+xwnyasmmohvcfpgjrukcgzjgtdauyfhscr1ocgjxvq0cotusvsxcod/zzcm28dynwmneao4vo/vpmo96yllgankfbpjvd7f76x9dpabajitqisnwbjvt3cii1pxfd8zlfukkhjgrddawjmerlk893de8w40oqshxqziq7j8hjrnrz7gglkaau75nksr1upom3vzylnnnm0kjrkexgpvmvwgsxhkfmsyrucuqkdeak2ol/dgnc87lkq8fhq95s1jud1ubuy6kda9yj67fvszrwa2vx2rpa1offs3npw+lh6wf5a/l7+vf5d1g6ujlfplawvvnpfwxamvo=</latexit> <latexit sha1_base64="irlduconfrtdowxy5oxevspvvqw=">aaae/nicbvrnb9naehwbamv8tccny4galeatjqebqlspcamva1ii+ixfabrer51v7lxzxaduvit+chcoimsv38gnf8nsnkzjm5vsjwbn7bz3dmwvjbjsjca/ldvs+dr1g2s37vu379y9t75x/1almatsgczrio89oljebtvqxefsojwmxf7ejrzho7n/ngjs8uts67ou9wisch5wsjsm+hulh65iupcz0lbjqs9s5+2kgxm98il8y7g6doarijbikyzegb4wyatxonibk6xu77eim4dmgs8czggqc5ff4ccaupn4gdvj8jg+zuyyhec8hnf9bj7kk9zldbmfttsbinxwos627brdavad2afxzxh/ghisyzlndhapu4oar66hfgztfeq9rxvbe0hzjndcdypyqeryhicdxtpexkvgsxxusohftu2o1jcnugvux9hydj0wcpgtiifi2dj2t1qfdbnnygrcaw91qjzm19mdqlrrnepjc0xuyg9ddvr4pjfjot7tuaquow92zf34+yx+mvswovvb5ltrtbxyrw67tpgz+hfaxhhts6np8h68lwhg0ami5zyezm0i4f7mvdrmx0mxi/ttidktkdhtir+xwnyasmmohvcfpgjrukcgzjgtdauyfhscr1ocgjxvq0cotusvsxcod/zzcm28dynwmneao4vo/vpmo96yllgankfbpjvd7f76x9dpabajitqisnwbjvt3cii1pxfd8zlfukkhjgrddawjmerlk893de8w40oqshxqziq7j8hjrnrz7gglkaau75nksr1upom3vzylnnnm0kjrkexgpvmvwgsxhkfmsyrucuqkdeak2ol/dgnc87lkq8fhq95s1jud1ubuy6kda9yj67fvszrwa2vx2rpa1offs3npw+lh6wf5a/l7+vf5d1g6ujlfplawvvnpfwxamvo=</latexit> <latexit sha1_base64="irlduconfrtdowxy5oxevspvvqw=">aaae/nicbvrnb9naehwbamv8tccny4galeatjqebqlspcamva1ii+ixfabrer51v7lxzxaduvit+chcoimsv38gnf8nsnkzjm5vsjwbn7bz3dmwvjbjsjca/ldvs+dr1g2s37vu379y9t75x/1almatsgczrio89oljebtvqxefsojwmxf7ejrzho7n/ngjs8uts67ou9wisch5wsjsm+hulh65iupcz0lbjqs9s5+2kgxm98il8y7g6doarijbikyzegb4wyatxonibk6xu77eim4dmgs8czggqc5ff4ccaupn4gdvj8jg+zuyyhec8hnf9bj7kk9zldbmfttsbinxwos627brdavad2afxzxh/ghisyzlndhapu4oar66hfgztfeq9rxvbe0hzjndcdypyqeryhicdxtpexkvgsxxusohftu2o1jcnugvux9hydj0wcpgtiifi2dj2t1qfdbnnygrcaw91qjzm19mdqlrrnepjc0xuyg9ddvr4pjfjot7tuaquow92zf34+yx+mvswovvb5ltrtbxyrw67tpgz+hfaxhhts6np8h68lwhg0ami5zyezm0i4f7mvdrmx0mxi/ttidktkdhtir+xwnyasmmohvcfpgjrukcgzjgtdauyfhscr1ocgjxvq0cotusvsxcod/zzcm28dynwmneao4vo/vpmo96yllgankfbpjvd7f76x9dpabajitqisnwbjvt3cii1pxfd8zlfukkhjgrddawjmerlk893de8w40oqshxqziq7j8hjrnrz7gglkaau75nksr1upom3vzylnnnm0kjrkexgpvmvwgsxhkfmsyrucuqkdeak2ol/dgnc87lkq8fhq95s1jud1ubuy6kda9yj67fvszrwa2vx2rpa1offs3npw+lh6wf5a/l7+vf5d1g6ujlfplawvvnpfwxamvo=</latexit> Large systems (thermodynamic limit), modest parameter changes all are near maximum distance

49 <latexit sha1_base64="mbvin+wdgbzchnp9iqvreo3h8ww=">aaafq3icdvtpj9ngfdykled+ykfhlk+swyvvchzulv5wc+jcpsp11c5qxceaj8fx7i5nwsx4l5wv/40l/wa3/geuprshxiv1jepdzvkyjc7lezpve+973zhbcw5sgl65dlnx/+llk1ev+v99/c2317du3hxsvk0p26nkkp00i4yjltme5vawpyvnsjuj9iq7eojitw6znlzjp+3xii0qsps84jrydku3en8lgvty2rdbl/lhtz8/m6rjrwypq+yre+tbpjon4ycdmmad/myjzzwfwzl4mxs2iyzcwsib9mrwnc/wa0jmc22behdmg544z+ecfsjkflpaihwxozpwn/pdbfxaqbadajtr3z9hkccigqhaytoaohnyksfkq4xkxhdlmqfvgfycgvlczoucazdm1lwgpechnbaocncyzcmgmodg1gfe4wgd0jrr3clnqfxnhqfizvjnpqoswdhstcjb7uiek4e5hgojpawy97fhjenskeigg11/8mcvvdwcamckcztjoz+ubrdjhsm88odixo+1hmu7wa0hjk0m1ma16ldoyhjdqyjbhyuyvvhjaqklr3bhn2b9ie6np+tvr4ebk+p8d8f4hzs5odafmtee0wasr5dfjt/ep/p7hj2vmwstomvzcrmfrh2v++kuwrzdh6u6ajp7nelwactcndkwdsimthzgckqt4bpprh4f7ha8ls3qj1jd1g86hc1soyqtcq/is9zjc8kffbyjdqy7urgglqkon1c6fcown5/pdlfqqgjudvtkba/qajfd2g7hybvgohf1xrbxrtjdeo2m0bpcuvjbjjlh4cougqitp4lhzagnwxf6qjzsjqykoixf074d1va9evjw3yxcq9h6pzzrkmqy4yrdny4d83hmot8vm9e2+gxrclmqlxk+aspqavabe6egazprk47riftj5abas6ijrr6njyreh7d80xg8gufhojpntu//1nfx1bvl3fygxutd8+57j7zy2/no72xvbe+f3rv+q/7f/ff9fzdbl1/qznznnvv9//4hjewy3w==</latexit> <latexit sha1_base64="mbvin+wdgbzchnp9iqvreo3h8ww=">aaafq3icdvtpj9ngfdykled+ykfhlk+swyvvchzulv5wc+jcpsp11c5qxceaj8fx7i5nwsx4l5wv/40l/wa3/geuprshxiv1jepdzvkyjc7lezpve+973zhbcw5sgl65dlnx/+llk1ev+v99/c2317du3hxsvk0p26nkkp00i4yjltme5vawpyvnsjuj9iq7eojitw6znlzjp+3xii0qsps84jrydku3en8lgvty2rdbl/lhtz8/m6rjrwypq+yre+tbpjon4ycdmmad/myjzzwfwzl4mxs2iyzcwsib9mrwnc/wa0jmc22behdmg544z+ecfsjkflpaihwxozpwn/pdbfxaqbadajtr3z9hkccigqhaytoaohnyksfkq4xkxhdlmqfvgfycgvlczoucazdm1lwgpechnbaocncyzcmgmodg1gfe4wgd0jrr3clnqfxnhqfizvjnpqoswdhstcjb7uiek4e5hgojpawy97fhjenskeigg11/8mcvvdwcamckcztjoz+ubrdjhsm88odixo+1hmu7wa0hjk0m1ma16ldoyhjdqyjbhyuyvvhjaqklr3bhn2b9ie6np+tvr4ebk+p8d8f4hzs5odafmtee0wasr5dfjt/ep/p7hj2vmwstomvzcrmfrh2v++kuwrzdh6u6ajp7nelwactcndkwdsimthzgckqt4bpprh4f7ha8ls3qj1jd1g86hc1soyqtcq/is9zjc8kffbyjdqy7urgglqkon1c6fcown5/pdlfqqgjudvtkba/qajfd2g7hybvgohf1xrbxrtjdeo2m0bpcuvjbjjlh4cougqitp4lhzagnwxf6qjzsjqykoixf074d1va9evjw3yxcq9h6pzzrkmqy4yrdny4d83hmot8vm9e2+gxrclmqlxk+aspqavabe6egazprk47riftj5abas6ijrr6njyreh7d80xg8gufhojpntu//1nfx1bvl3fygxutd8+57j7zy2/no72xvbe+f3rv+q/7f/ff9fzdbl1/qznznnvv9//4hjewy3w==</latexit> <latexit sha1_base64="mbvin+wdgbzchnp9iqvreo3h8ww=">aaafq3icdvtpj9ngfdykled+ykfhlk+swyvvchzulv5wc+jcpsp11c5qxceaj8fx7i5nwsx4l5wv/40l/wa3/geuprshxiv1jepdzvkyjc7lezpve+973zhbcw5sgl65dlnx/+llk1ev+v99/c2317du3hxsvk0p26nkkp00i4yjltme5vawpyvnsjuj9iq7eojitw6znlzjp+3xii0qsps84jrydku3en8lgvty2rdbl/lhtz8/m6rjrwypq+yre+tbpjon4ycdmmad/myjzzwfwzl4mxs2iyzcwsib9mrwnc/wa0jmc22behdmg544z+ecfsjkflpaihwxozpwn/pdbfxaqbadajtr3z9hkccigqhaytoaohnyksfkq4xkxhdlmqfvgfycgvlczoucazdm1lwgpechnbaocncyzcmgmodg1gfe4wgd0jrr3clnqfxnhqfizvjnpqoswdhstcjb7uiek4e5hgojpawy97fhjenskeigg11/8mcvvdwcamckcztjoz+ubrdjhsm88odixo+1hmu7wa0hjk0m1ma16ldoyhjdqyjbhyuyvvhjaqklr3bhn2b9ie6np+tvr4ebk+p8d8f4hzs5odafmtee0wasr5dfjt/ep/p7hj2vmwstomvzcrmfrh2v++kuwrzdh6u6ajp7nelwactcndkwdsimthzgckqt4bpprh4f7ha8ls3qj1jd1g86hc1soyqtcq/is9zjc8kffbyjdqy7urgglqkon1c6fcown5/pdlfqqgjudvtkba/qajfd2g7hybvgohf1xrbxrtjdeo2m0bpcuvjbjjlh4cougqitp4lhzagnwxf6qjzsjqykoixf074d1va9evjw3yxcq9h6pzzrkmqy4yrdny4d83hmot8vm9e2+gxrclmqlxk+aspqavabe6egazprk47riftj5abas6ijrr6njyreh7d80xg8gufhojpntu//1nfx1bvl3fygxutd8+57j7zy2/no72xvbe+f3rv+q/7f/ff9fzdbl1/qznznnvv9//4hjewy3w==</latexit> <latexit sha1_base64="mbvin+wdgbzchnp9iqvreo3h8ww=">aaafq3icdvtpj9ngfdykled+ykfhlk+swyvvchzulv5wc+jcpsp11c5qxceaj8fx7i5nwsx4l5wv/40l/wa3/geuprshxiv1jepdzvkyjc7lezpve+973zhbcw5sgl65dlnx/+llk1ev+v99/c2317du3hxsvk0p26nkkp00i4yjltme5vawpyvnsjuj9iq7eojitw6znlzjp+3xii0qsps84jrydku3en8lgvty2rdbl/lhtz8/m6rjrwypq+yre+tbpjon4ycdmmad/myjzzwfwzl4mxs2iyzcwsib9mrwnc/wa0jmc22behdmg544z+ecfsjkflpaihwxozpwn/pdbfxaqbadajtr3z9hkccigqhaytoaohnyksfkq4xkxhdlmqfvgfycgvlczoucazdm1lwgpechnbaocncyzcmgmodg1gfe4wgd0jrr3clnqfxnhqfizvjnpqoswdhstcjb7uiek4e5hgojpawy97fhjenskeigg11/8mcvvdwcamckcztjoz+ubrdjhsm88odixo+1hmu7wa0hjk0m1ma16ldoyhjdqyjbhyuyvvhjaqklr3bhn2b9ie6np+tvr4ebk+p8d8f4hzs5odafmtee0wasr5dfjt/ep/p7hj2vmwstomvzcrmfrh2v++kuwrzdh6u6ajp7nelwactcndkwdsimthzgckqt4bpprh4f7ha8ls3qj1jd1g86hc1soyqtcq/is9zjc8kffbyjdqy7urgglqkon1c6fcown5/pdlfqqgjudvtkba/qajfd2g7hybvgohf1xrbxrtjdeo2m0bpcuvjbjjlh4cougqitp4lhzagnwxf6qjzsjqykoixf074d1va9evjw3yxcq9h6pzzrkmqy4yrdny4d83hmot8vm9e2+gxrclmqlxk+aspqavabe6egazprk47riftj5abas6ijrr6njyreh7d80xg8gufhojpntu//1nfx1bvl3fygxutd8+57j7zy2/no72xvbe+f3rv+q/7f/ff9fzdbl1/qznznnvv9//4hjewy3w==</latexit> Exercise 1.16d Hellinger distance d 2 Hel(P, Q) =2 Z P Q = 2P Q dx p P (x) p Q(x) [iv] If P = {1/6,...,1/6} are the probabilities of rolling di erent numbers for a fair die, and Q = {1/10,...,1/10, 1/2} are the probabilities for a loaded die, what would the Hellinger distance be between P and Q? (A formula is fine.) [v] Our gambler keeps using the loaded die. Can the casino catch him? Let P N (j) be the probability that rolling the die N times gives the sequence j = {j 1,...,j N }. Show that P N Q N =(P Q) N and hence d 2 Hel(P N,Q n )=2 2(P Q) N. After N = 100 rolls, how close is the Hellinger distance from its maximum? Large systems have Hellinger distances that saturate rapidly as parameters are changed hard to use for visualization

50 Intensive embeddings: Ising and CMB Katherine Quinn CMB: Intensive embedding Hellinger distance useless for visualizing CMB spectra. Thousands of Fourier components, millions of data points all pairs are nearly orthogonal. CMB: Hellinger embedding

51 Exercise 1.16e Bhattacharyya intensive distance (Quinn) Hellinger distance for N replicas (samples) is P N Q N =exp(n log(p Q)); it is extensive in the amount of data. Katherine Quinn takes N! 0 to define our intensive distance d 2 intensive = lim N!0 d 2 Hel(P N,Q N )/N = 2 log(p Q). (f) Derive this equation. (Hint: Z N exp(n log Z) 1+N log Z for small N.) Show that the intensive distance does not satisfy the triangle inequality. (Hint: Find two distributions of loaded dice which do not overlap (so P Q =0), but both overlap with a third.) Show that it does satisfy the other conditions for a distance. Show, for the nonlinear least-squares model of part (b), that the intensive distance equals the distance in data space between the two predictions. <latexit sha1_base64="wsvlkqv329ljnikuouuxv33+1ng=">aaafshicbvrnbxmxef2a8lv8fthygdegjdaumwojbkqebekvkkpwueb028jrnu2seu3f9janovdvuhdkxm/gwggeudhepig0+lcanrnpvpc8dlpkyv0cfz2z0giepxf+wsxw0uurv68txr/xxurkcnzmwmrzlmuwpvc47yst+k40yipu4tt0/5mpvz1ay4vwr92oxn2c9zxibweoxl3rjb1eaaeyvc5cr0ll+mggo8zmcyrcg2httmaggehmqtuyoprooyastdz7g5dwtdvyimsnejws2+ssut/enay6ndytem7n46fdzcubgldgbgis0jvyohpimgnr+jkr1xav2kqeohy2j9rmi3ea4hbqn8pch4k+jbmsqnbxywpmkzbjss/ugd9uncm7kzdbw+0lbru3xxi2y+ibs1h0xr5+pih5lnji0iz6y0ssc38jholag5xve/yimegvzbf7k2k778bzykt43marme2kwhp9e53qpibw+z0sjzwl0yfzgsl7zszdvxfsa9wnygsmpt+bqdmjxw30kbowv6t6whpinomock6l9jyf+bzqgcfux/onhpcubhtbpk4xxalcund1jegrd93645kazzirn2opb4ipwkzx9tunm2qlzymm8zhkrkrfrc4yuafitrvmsoacfpixxmt/8qcbqthp4qxthwo4vpk4auj1ydosefjty6h3+nsllb8vzibezt2kjy6gshy6q+ljlmy4j+2ugj2wlpxjv0tj6+jmr+prbxzsyegvrtdepjporxzdqbkmhhj1fpfikk4xnda6u2mpmk7n3ukxjno8kmh6ugtw7ntj0u2ocarg0g93zzg67rm+7pcpwif2d1w/bbo4q56slixxdk9q7/yomsushrupzfo5tydj3vm/2e7l8ke7dhxkivthr43ysvob6v8vusygd3jebuogjocdhzdduko3jyqruicpnzberebdoopurs49ftcv40jwk7gdtinu8db4gqwhm8f2wbufgt8apxo/m5+b35u/mr+puhfotpfcdp5zzxf+ahhxuye=</latexit> <latexit sha1_base64="wsvlkqv329ljnikuouuxv33+1ng=">aaafshicbvrnbxmxef2a8lv8fthygdegjdaumwojbkqebekvkkpwueb028jrnu2seu3f9janovdvuhdkxm/gwggeudhepig0+lcanrnpvpc8dlpkyv0cfz2z0giepxf+wsxw0uurv68txr/xxurkcnzmwmrzlmuwpvc47yst+k40yipu4tt0/5mpvz1ay4vwr92oxn2c9zxibweoxl3rjb1eaaeyvc5cr0ll+mggo8zmcyrcg2httmaggehmqtuyoprooyastdz7g5dwtdvyimsnejws2+ssut/enay6ndytem7n46fdzcubgldgbgis0jvyohpimgnr+jkr1xav2kqeohy2j9rmi3ea4hbqn8pch4k+jbmsqnbxywpmkzbjss/ugd9uncm7kzdbw+0lbru3xxi2y+ibs1h0xr5+pih5lnji0iz6y0ssc38jholag5xve/yimegvzbf7k2k778bzykt43marme2kwhp9e53qpibw+z0sjzwl0yfzgsl7zszdvxfsa9wnygsmpt+bqdmjxw30kbowv6t6whpinomock6l9jyf+bzqgcfux/onhpcubhtbpk4xxalcund1jegrd93645kazzirn2opb4ipwkzx9tunm2qlzymm8zhkrkrfrc4yuafitrvmsoacfpixxmt/8qcbqthp4qxthwo4vpk4auj1ydosefjty6h3+nsllb8vzibezt2kjy6gshy6q+ljlmy4j+2ugj2wlpxjv0tj6+jmr+prbxzsyegvrtdepjporxzdqbkmhhj1fpfikk4xnda6u2mpmk7n3ukxjno8kmh6ugtw7ntj0u2ocarg0g93zzg67rm+7pcpwif2d1w/bbo4q56slixxdk9q7/yomsushrupzfo5tydj3vm/2e7l8ke7dhxkivthr43ysvob6v8vusygd3jebuogjocdhzdduko3jyqruicpnzberebdoopurs49ftcv40jwk7gdtinu8db4gqwhm8f2wbufgt8apxo/m5+b35u/mr+puhfotpfcdp5zzxf+ahhxuye=</latexit> <latexit sha1_base64="wsvlkqv329ljnikuouuxv33+1ng=">aaafshicbvrnbxmxef2a8lv8fthygdegjdaumwojbkqebekvkkpwueb028jrnu2seu3f9janovdvuhdkxm/gwggeudhepig0+lcanrnpvpc8dlpkyv0cfz2z0giepxf+wsxw0uurv68txr/xxurkcnzmwmrzlmuwpvc47yst+k40yipu4tt0/5mpvz1ay4vwr92oxn2c9zxibweoxl3rjb1eaaeyvc5cr0ll+mggo8zmcyrcg2httmaggehmqtuyoprooyastdz7g5dwtdvyimsnejws2+ssut/enay6ndytem7n46fdzcubgldgbgis0jvyohpimgnr+jkr1xav2kqeohy2j9rmi3ea4hbqn8pch4k+jbmsqnbxywpmkzbjss/ugd9uncm7kzdbw+0lbru3xxi2y+ibs1h0xr5+pih5lnji0iz6y0ssc38jholag5xve/yimegvzbf7k2k778bzykt43marme2kwhp9e53qpibw+z0sjzwl0yfzgsl7zszdvxfsa9wnygsmpt+bqdmjxw30kbowv6t6whpinomock6l9jyf+bzqgcfux/onhpcubhtbpk4xxalcund1jegrd93645kazzirn2opb4ipwkzx9tunm2qlzymm8zhkrkrfrc4yuafitrvmsoacfpixxmt/8qcbqthp4qxthwo4vpk4auj1ydosefjty6h3+nsllb8vzibezt2kjy6gshy6q+ljlmy4j+2ugj2wlpxjv0tj6+jmr+prbxzsyegvrtdepjporxzdqbkmhhj1fpfikk4xnda6u2mpmk7n3ukxjno8kmh6ugtw7ntj0u2ocarg0g93zzg67rm+7pcpwif2d1w/bbo4q56slixxdk9q7/yomsushrupzfo5tydj3vm/2e7l8ke7dhxkivthr43ysvob6v8vusygd3jebuogjocdhzdduko3jyqruicpnzberebdoopurs49ftcv40jwk7gdtinu8db4gqwhm8f2wbufgt8apxo/m5+b35u/mr+puhfotpfcdp5zzxf+ahhxuye=</latexit> <latexit sha1_base64="wsvlkqv329ljnikuouuxv33+1ng=">aaafshicbvrnbxmxef2a8lv8fthygdegjdaumwojbkqebekvkkpwueb028jrnu2seu3f9janovdvuhdkxm/gwggeudhepig0+lcanrnpvpc8dlpkyv0cfz2z0giepxf+wsxw0uurv68txr/xxurkcnzmwmrzlmuwpvc47yst+k40yipu4tt0/5mpvz1ay4vwr92oxn2c9zxibweoxl3rjb1eaaeyvc5cr0ll+mggo8zmcyrcg2httmaggehmqtuyoprooyastdz7g5dwtdvyimsnejws2+ssut/enay6ndytem7n46fdzcubgldgbgis0jvyohpimgnr+jkr1xav2kqeohy2j9rmi3ea4hbqn8pch4k+jbmsqnbxywpmkzbjss/ugd9uncm7kzdbw+0lbru3xxi2y+ibs1h0xr5+pih5lnji0iz6y0ssc38jholag5xve/yimegvzbf7k2k778bzykt43marme2kwhp9e53qpibw+z0sjzwl0yfzgsl7zszdvxfsa9wnygsmpt+bqdmjxw30kbowv6t6whpinomock6l9jyf+bzqgcfux/onhpcubhtbpk4xxalcund1jegrd93645kazzirn2opb4ipwkzx9tunm2qlzymm8zhkrkrfrc4yuafitrvmsoacfpixxmt/8qcbqthp4qxthwo4vpk4auj1ydosefjty6h3+nsllb8vzibezt2kjy6gshy6q+ljlmy4j+2ugj2wlpxjv0tj6+jmr+prbxzsyegvrtdepjporxzdqbkmhhj1fpfikk4xnda6u2mpmk7n3ukxjno8kmh6ugtw7ntj0u2ocarg0g93zzg67rm+7pcpwif2d1w/bbo4q56slixxdk9q7/yomsushrupzfo5tydj3vm/2e7l8ke7dhxkivthr43ysvob6v8vusygd3jebuogjocdhzdduko3jyqruicpnzberebdoopurs49ftcv40jwk7gdtinu8db4gqwhm8f2wbufgt8apxo/m5+b35u/mr+puhfotpfcdp5zzxf+ahhxuye=</latexit> The intensive distance allows for great visualizations. But it s not a metric, and it embeds the manifold in a Minkowski-like space.

52 Intensive embeddings: Ising, CMB, digits Katherine Quinn slowly expanding universes our universe ferromagnetic positive field (behind) negative field rapidly expanding universes universes with universes with large primordial small primordial density fluctuations density fluctuations anti-ferromagnetic (b) Cosmic Microwave Background (a) Ising Model of Atomic Spins (c) Clasifying Hand-Written Digits Probabilistic models gµ = Fisher Information Metric Isometric embedding Replica theory extrapolate to zero data Embedded in Minkowski space

53 Renormalization group and the model manifold Archishman Raju, Ben Machta all spins down critical point all spin states equally likely all spins up RG in parameter space: Relevant t, h grow, irrelevant shrink (a) RG on model manifold: Relevant distances unchanged; irrelevant shrink Emergent variables are those which remain equally distinguishable at long length scales

54 Generation of Reduced Models Mark Transtrum (not me) Can we coarse-grain sloppy models? If most parameter directions are useless, why not remove some? Transtrum has systematic method! (1) Geodesic along sloppiest direction to nearby point on manifold boundary (2) Eigendirection simplifies at model boundary to chemically reasonable simplified model Coarse-graining = boundaries of model manifold. Sloppiest Eigendirection Simplified at Boundary (Unsaturable reaction)

55 Generation of Reduced Models Mark Transtrum (not me) 48 params 29 ODEs

56 Generation of Reduced Models Mark Transtrum (not me) 12 params 6 ODEs Reduced model fits all experimental data Effective renormalized params

57 Control and Carnot Corrections Ben Machta (not me) T 1 P Q 2 P Complex Carnot cycle: Isothermal Expansion, Adiabatic, Change Bath, Isothermal, Adiabatic What is the entropy cost for the control system? T 2 Incorporate control system into Hamiltonian Move P(t), β(t) = 1/kT around controlled loop

58 Feynman Ratchet & Pawl and Molecular Motors Molecular motors Driven by ATP Weak driving random hops back and forth Strong driving, steady walk Pawl Pawl rectifies random thermal motion Hot gas T1 > T2 mass rises But cold gas T1 < T2, pawl hops, mass falls! T1 = T2 Ratchet satisfies detailed balance

59 <latexit sha1_base64="evuxd1ow74ywag5cq79wj4b99go=">aaaczhicdvfda9rafj3ej9a12rtfj0egf0f8wjjdss0ofptbx4huw9isytk5sydojunmtwez8id989gx/o5ositu0qmzhm79mnsma6qwgiy/pf/e/qcpt7yfjr7vphm6g+ztn5q61rzmvja1ps+yaskuzfgghpnga6sycwfz5cc+fnyn2ohafcnva8ukluougjn0uhryjinskatx7ac86uajzkquqjnpijhrr2lsmbzqlew1ql3ljiz6ybjrdt7qku2ewttgqei+u7twsezkoy2dfncdzw3soopk70xkg3e4oe7xjq7j7nir8g0uzmy0moqdxmsdoa1+jhnn2woucsmmwurhg0vlnaouwfvvdtsmx7isfo4qvofz2sgkjr5ysk6lwrujka7q3qrlkmnwveyy+33n37fe/fds0wjxtlrcns2c4utbrssp1rr3nozca0e5corxldxbkb9gmnf0/zjyjvzelp6fne4nutijvkzhj+83dmyt5+qleu0ickhoygcskznh5je35qxennfj7/gh/rn1qu9tag7ih/bf3ajqwld6</latexit> <latexit sha1_base64="evuxd1ow74ywag5cq79wj4b99go=">aaaczhicdvfda9rafj3ej9a12rtfj0egf0f8wjjdss0ofptbx4huw9isytk5sydojunmtwez8id989gx/o5ositu0qmzhm79mnsma6qwgiy/pf/e/qcpt7yfjr7vphm6g+ztn5q61rzmvja1ps+yaskuzfgghpnga6sycwfz5cc+fnyn2ohafcnva8ukluougjn0uhryjinskatx7ac86uajzkquqjnpijhrr2lsmbzqlew1ql3ljiz6ybjrdt7qku2ewttgqei+u7twsezkoy2dfncdzw3soopk70xkg3e4oe7xjq7j7nir8g0uzmy0moqdxmsdoa1+jhnn2woucsmmwurhg0vlnaouwfvvdtsmx7isfo4qvofz2sgkjr5ysk6lwrujka7q3qrlkmnwveyy+33n37fe/fds0wjxtlrcns2c4utbrssp1rr3nozca0e5corxldxbkb9gmnf0/zjyjvzelp6fne4nutijvkzhj+83dmyt5+qleu0ickhoygcskznh5je35qxennfj7/gh/rn1qu9tag7ih/bf3ajqwld6</latexit> <latexit sha1_base64="evuxd1ow74ywag5cq79wj4b99go=">aaaczhicdvfda9rafj3ej9a12rtfj0egf0f8wjjdss0ofptbx4huw9isytk5sydojunmtwez8id989gx/o5ositu0qmzhm79mnsma6qwgiy/pf/e/qcpt7yfjr7vphm6g+ztn5q61rzmvja1ps+yaskuzfgghpnga6sycwfz5cc+fnyn2ohafcnva8ukluougjn0uhryjinskatx7ac86uajzkquqjnpijhrr2lsmbzqlew1ql3ljiz6ybjrdt7qku2ewttgqei+u7twsezkoy2dfncdzw3soopk70xkg3e4oe7xjq7j7nir8g0uzmy0moqdxmsdoa1+jhnn2woucsmmwurhg0vlnaouwfvvdtsmx7isfo4qvofz2sgkjr5ysk6lwrujka7q3qrlkmnwveyy+33n37fe/fds0wjxtlrcns2c4utbrssp1rr3nozca0e5corxldxbkb9gmnf0/zjyjvzelp6fne4nutijvkzhj+83dmyt5+qleu0ickhoygcskznh5je35qxennfj7/gh/rn1qu9tag7ih/bf3ajqwld6</latexit> <latexit sha1_base64="evuxd1ow74ywag5cq79wj4b99go=">aaaczhicdvfda9rafj3ej9a12rtfj0egf0f8wjjdss0ofptbx4huw9isytk5sydojunmtwez8id989gx/o5ositu0qmzhm79mnsma6qwgiy/pf/e/qcpt7yfjr7vphm6g+ztn5q61rzmvja1ps+yaskuzfgghpnga6sycwfz5cc+fnyn2ohafcnva8ukluougjn0uhryjinskatx7ac86uajzkquqjnpijhrr2lsmbzqlew1ql3ljiz6ybjrdt7qku2ewttgqei+u7twsezkoy2dfncdzw3soopk70xkg3e4oe7xjq7j7nir8g0uzmy0moqdxmsdoa1+jhnn2woucsmmwurhg0vlnaouwfvvdtsmx7isfo4qvofz2sgkjr5ysk6lwrujka7q3qrlkmnwveyy+33n37fe/fds0wjxtlrcns2c4utbrssp1rr3nozca0e5corxldxbkb9gmnf0/zjyjvzelp6fne4nutijvkzhj+83dmyt5+qleu0ickhoygcskznh5je35qxennfj7/gh/rn1qu9tag7ih/bf3ajqwld6</latexit> <latexit sha1_base64="uruq4yuej0qrcrjp47kyfik/ags=">aaaco3icdvfra9rafj3ev42pbvwjxy4uhsq4jluilvgoceurczvuw9ijy2t2jhk6mcszibce/df/ht/8n06yq9ridwwczn2duteppda2dh95/o2bt27f2bob3lv/4oh2yofrqslrzxhgs1nq84qzlelhzaor8bzsyipe4lly8baln31hbusptuyqwrhgmrkp4mw6ath4qrpmhgrww90rz9sgwzs0qigquovdeafuymqpzcqtcdtvjde0ytokjr+oxbtm9qjoy2dtanfgbz5qm6nli77/va2br7xickv1etgh0ilzxbfn8fupbubrls8zxqyg4eigwytyk8mbi+hlkjxmibqfpyzkg+li8jmus14xqcyxzjh5ffy2bjobxklrxxusgl9ggc4dvaxaezf9jlvydcos0lk7pyz06uwkhhxgrirezxaezdvyj/4vnq9tuh83qlw1rcxxg9jagi2hoxgshuzu5corxrvwxohnzf3burmgbgl/fgrxk9pxkaph0efx8ojnzh1b5al5svzirf6ti/kotmmmca+8y++tn/v3/q/+f/9knep7m5rh5b/48w8g8mrg</latexit> <latexit sha1_base64="uruq4yuej0qrcrjp47kyfik/ags=">aaaco3icdvfra9rafj3ev42pbvwjxy4uhsq4jluilvgoceurczvuw9ijy2t2jhk6mcszibce/df/ht/8n06yq9ridwwczn2duteppda2dh95/o2bt27f2bob3lv/4oh2yofrqslrzxhgs1nq84qzlelhzaor8bzsyipe4lly8baln31hbusptuyqwrhgmrkp4mw6ath4qrpmhgrww90rz9sgwzs0qigquovdeafuymqpzcqtcdtvjde0ytokjr+oxbtm9qjoy2dtanfgbz5qm6nli77/va2br7xickv1etgh0ilzxbfn8fupbubrls8zxqyg4eigwytyk8mbi+hlkjxmibqfpyzkg+li8jmus14xqcyxzjh5ffy2bjobxklrxxusgl9ggc4dvaxaezf9jlvydcos0lk7pyz06uwkhhxgrirezxaezdvyj/4vnq9tuh83qlw1rcxxg9jagi2hoxgshuzu5corxrvwxohnzf3burmgbgl/fgrxk9pxkaph0efx8ojnzh1b5al5svzirf6ti/kotmmmca+8y++tn/v3/q/+f/9knep7m5rh5b/48w8g8mrg</latexit> <latexit sha1_base64="uruq4yuej0qrcrjp47kyfik/ags=">aaaco3icdvfra9rafj3ev42pbvwjxy4uhsq4jluilvgoceurczvuw9ijy2t2jhk6mcszibce/df/ht/8n06yq9ridwwczn2duteppda2dh95/o2bt27f2bob3lv/4oh2yofrqslrzxhgs1nq84qzlelhzaor8bzsyipe4lly8baln31hbusptuyqwrhgmrkp4mw6ath4qrpmhgrww90rz9sgwzs0qigquovdeafuymqpzcqtcdtvjde0ytokjr+oxbtm9qjoy2dtanfgbz5qm6nli77/va2br7xickv1etgh0ilzxbfn8fupbubrls8zxqyg4eigwytyk8mbi+hlkjxmibqfpyzkg+li8jmus14xqcyxzjh5ffy2bjobxklrxxusgl9ggc4dvaxaezf9jlvydcos0lk7pyz06uwkhhxgrirezxaezdvyj/4vnq9tuh83qlw1rcxxg9jagi2hoxgshuzu5corxrvwxohnzf3burmgbgl/fgrxk9pxkaph0efx8ojnzh1b5al5svzirf6ti/kotmmmca+8y++tn/v3/q/+f/9knep7m5rh5b/48w8g8mrg</latexit> <latexit sha1_base64="uruq4yuej0qrcrjp47kyfik/ags=">aaaco3icdvfra9rafj3ev42pbvwjxy4uhsq4jluilvgoceurczvuw9ijy2t2jhk6mcszibce/df/ht/8n06yq9ridwwczn2duteppda2dh95/o2bt27f2bob3lv/4oh2yofrqslrzxhgs1nq84qzlelhzaor8bzsyipe4lly8baln31hbusptuyqwrhgmrkp4mw6ath4qrpmhgrww90rz9sgwzs0qigquovdeafuymqpzcqtcdtvjde0ytokjr+oxbtm9qjoy2dtanfgbz5qm6nli77/va2br7xickv1etgh0ilzxbfn8fupbubrls8zxqyg4eigwytyk8mbi+hlkjxmibqfpyzkg+li8jmus14xqcyxzjh5ffy2bjobxklrxxusgl9ggc4dvaxaezf9jlvydcos0lk7pyz06uwkhhxgrirezxaezdvyj/4vnq9tuh83qlw1rcxxg9jagi2hoxgshuzu5corxrvwxohnzf3burmgbgl/fgrxk9pxkaph0efx8ojnzh1b5al5svzirf6ti/kotmmmca+8y++tn/v3/q/+f/9knep7m5rh5b/48w8g8mrg</latexit> Control and Carnot Corrections Q 2 T 1 T 2 P P h S control i =2 g µ = Ben Machta (not me) Z Pf P 2 log( p gpp dp =FIM mg Pressure Control Continuously variable transmission Mass falls to drive piston (fast costs S) Pressure fluctuations allow mass to fall (slow costs S) Fisher Information Matrix: Natural metric (distance) between probability distributions

60 <latexit sha1_base64="mgkrjujaqyqntdonziyuiqq9adm=">aaacqhicbvbnbxoxfptstkx0izbhxp7cvqjshxa5tiduquohork1heia0fvzaauvvbk9qrdip/xsn9bbz7nkkcjknacy2emchcnseoannj1jjov1ufqvkd3ze/rseflf5ewr12/evt+9p7u6n5w6xettzhk0jiwijhno0llmcnneujezfv/53xmyvmj1080zgqq4uwisodovdavdh1p9c8k+1jn630z1pxce9feehmjrvf15zbyaydyci5nawo1ca0jbmme32my2q0a1kpjhy1itry1oddglcufqreb7wp3bh2mep6qcl2htl44yn1igcyjlwlb6uaum+qwn1pnuyup2sfgxsispxhnbwbt/lio1+jcxwntaezr4yrtd1g57k/f/xi9346+dhvbz7kjxzajxlsfpwluji2giozn3blkr/q3ap2iq+6zsxzcqb395l5w2g3huie+atdzhuuezfwahrm5i9ow12dfrsw7j7de7yffsovgtxay3we1mtbqumx32cmhdpzneq40=</latexit> <latexit sha1_base64="mgkrjujaqyqntdonziyuiqq9adm=">aaacqhicbvbnbxoxfptstkx0izbhxp7cvqjshxa5tiduquohork1heia0fvzaauvvbk9qrdip/xsn9bbz7nkkcjknacy2emchcnseoannj1jjov1ufqvkd3ze/rseflf5ewr12/evt+9p7u6n5w6xettzhk0jiwijhno0llmcnneujezfv/53xmyvmj1080zgqq4uwisodovdavdh1p9c8k+1jn630z1pxce9feehmjrvf15zbyaydyci5nawo1ca0jbmme32my2q0a1kpjhy1itry1oddglcufqreb7wp3bh2mep6qcl2htl44yn1igcyjlwlb6uaum+qwn1pnuyup2sfgxsispxhnbwbt/lio1+jcxwntaezr4yrtd1g57k/f/xi9346+dhvbz7kjxzajxlsfpwluji2giozn3blkr/q3ap2iq+6zsxzcqb395l5w2g3huie+atdzhuuezfwahrm5i9ow12dfrsw7j7de7yffsovgtxay3we1mtbqumx32cmhdpzneq40=</latexit> <latexit sha1_base64="mgkrjujaqyqntdonziyuiqq9adm=">aaacqhicbvbnbxoxfptstkx0izbhxp7cvqjshxa5tiduquohork1heia0fvzaauvvbk9qrdip/xsn9bbz7nkkcjknacy2emchcnseoannj1jjov1ufqvkd3ze/rseflf5ewr12/evt+9p7u6n5w6xettzhk0jiwijhno0llmcnneujezfv/53xmyvmj1080zgqq4uwisodovdavdh1p9c8k+1jn630z1pxce9feehmjrvf15zbyaydyci5nawo1ca0jbmme32my2q0a1kpjhy1itry1oddglcufqreb7wp3bh2mep6qcl2htl44yn1igcyjlwlb6uaum+qwn1pnuyup2sfgxsispxhnbwbt/lio1+jcxwntaezr4yrtd1g57k/f/xi9346+dhvbz7kjxzajxlsfpwluji2giozn3blkr/q3ap2iq+6zsxzcqb395l5w2g3huie+atdzhuuezfwahrm5i9ow12dfrsw7j7de7yffsovgtxay3we1mtbqumx32cmhdpzneq40=</latexit> <latexit sha1_base64="mgkrjujaqyqntdonziyuiqq9adm=">aaacqhicbvbnbxoxfptstkx0izbhxp7cvqjshxa5tiduquohork1heia0fvzaauvvbk9qrdip/xsn9bbz7nkkcjknacy2emchcnseoannj1jjov1ufqvkd3ze/rseflf5ewr12/evt+9p7u6n5w6xettzhk0jiwijhno0llmcnneujezfv/53xmyvmj1080zgqq4uwisodovdavdh1p9c8k+1jn630z1pxce9feehmjrvf15zbyaydyci5nawo1ca0jbmme32my2q0a1kpjhy1itry1oddglcufqreb7wp3bh2mep6qcl2htl44yn1igcyjlwlb6uaum+qwn1pnuyup2sfgxsispxhnbwbt/lio1+jcxwntaezr4yrtd1g57k/f/xi9346+dhvbz7kjxzajxlsfpwluji2giozn3blkr/q3ap2iq+6zsxzcqb395l5w2g3huie+atdzhuuezfwahrm5i9ow12dfrsw7j7de7yffsovgtxay3we1mtbqumx32cmhdpzneq40=</latexit> <latexit sha1_base64="gdb+6yymspm7ifmrairgazhvsm0=">aaaeqxicdvnlb9naehythsw8wjhygvejjrbculucvukqqerzqkh0jxxtal1e26uu12z3ntay/ne48a+4cefcays4cmhswki0mkdp89pvvpn1mymmhqw+l7xav65eu758w7156/aduyur9/znmmvg91gqu33ou8oluhzpciv5yay5txzjd/yt7sp+mopaiftt2nngjwmnlagfoxzd09xwavh8lkvjqlvqkch7xs0tc8kvytvpqi2l6lggfdbjkzxer+jl6bpepuifxfm3w7uafqm8tjvqqfkss6o9pcfkvfvbjdcglowaykbfccpsdcoacgltjagcb/ego7iylaxxc3nt5xm11w8lmo5fcs+hk/vqct0kj2vyg4xh64gruaswg405hcgr4iujarctiqek/j6vx35cus2k04jqmcv0mrb2fapeuhvjjhpv1kk8xkdfgnwijtnu+cyf/9ifaibzlkykyegcr+ulef23d1hh4dwhorqylwbx3ocgjgby4vifqhejgvfgz+vzdgepcidgwsywwyzxzd6o1y/dzgif3gqvkhbqgd5buoyw8at2eyzlj1arc4o3pdd1srwksxw8mbo3ezrb3e6tlg9e3pcifqk33gbgirq4w40slpoan+gbmjy5owqt63qrcgqdudmvpytrg/6gnrgmhg1ycxobt1c+ksbleyixxsq15mg4yoykam4hn5ybnlf2qin+hfbrpdup6h9qwip0bdwnmfuwau/5ioimxswthzorgzixy5xzx7gj3iyvjovqlskklr4kcwk2heo7qsa03jpkeajknk6kaytpdeb46vwuyxhx5mtgf70/hpshb9fxtjybozadb85dp+mmnefoljnyxs6ew1ofwl9a31rf2x/bx9s/2j8xqa2lpua+85e1f/0g2slfpa==</latexit> <latexit sha1_base64="gdb+6yymspm7ifmrairgazhvsm0=">aaaeqxicdvnlb9naehythsw8wjhygvejjrbculucvukqqerzqkh0jxxtal1e26uu12z3ntay/ne48a+4cefcays4cmhswki0mkdp89pvvpn1mymmhqw+l7xav65eu758w7156/aduyur9/znmmvg91gqu33ou8oluhzpciv5yay5txzjd/yt7sp+mopaiftt2nngjwmnlagfoxzd09xwavh8lkvjqlvqkch7xs0tc8kvytvpqi2l6lggfdbjkzxer+jl6bpepuifxfm3w7uafqm8tjvqqfkss6o9pcfkvfvbjdcglowaykbfccpsdcoacgltjagcb/ego7iylaxxc3nt5xm11w8lmo5fcs+hk/vqct0kj2vyg4xh64gruaswg405hcgr4iujarctiqek/j6vx35cus2k04jqmcv0mrb2fapeuhvjjhpv1kk8xkdfgnwijtnu+cyf/9ifaibzlkykyegcr+ulef23d1hh4dwhorqylwbx3ocgjgby4vifqhejgvfgz+vzdgepcidgwsywwyzxzd6o1y/dzgif3gqvkhbqgd5buoyw8at2eyzlj1arc4o3pdd1srwksxw8mbo3ezrb3e6tlg9e3pcifqk33gbgirq4w40slpoan+gbmjy5owqt63qrcgqdudmvpytrg/6gnrgmhg1ycxobt1c+ksbleyixxsq15mg4yoykam4hn5ybnlf2qin+hfbrpdup6h9qwip0bdwnmfuwau/5ioimxswthzorgzixy5xzx7gj3iyvjovqlskklr4kcwk2heo7qsa03jpkeajknk6kaytpdeb46vwuyxhx5mtgf70/hpshb9fxtjybozadb85dp+mmnefoljnyxs6ew1ofwl9a31rf2x/bx9s/2j8xqa2lpua+85e1f/0g2slfpa==</latexit> <latexit sha1_base64="gdb+6yymspm7ifmrairgazhvsm0=">aaaeqxicdvnlb9naehythsw8wjhygvejjrbculucvukqqerzqkh0jxxtal1e26uu12z3ntay/ne48a+4cefcays4cmhswki0mkdp89pvvpn1mymmhqw+l7xav65eu758w7156/aduyur9/znmmvg91gqu33ou8oluhzpciv5yay5txzjd/yt7sp+mopaiftt2nngjwmnlagfoxzd09xwavh8lkvjqlvqkch7xs0tc8kvytvpqi2l6lggfdbjkzxer+jl6bpepuifxfm3w7uafqm8tjvqqfkss6o9pcfkvfvbjdcglowaykbfccpsdcoacgltjagcb/ego7iylaxxc3nt5xm11w8lmo5fcs+hk/vqct0kj2vyg4xh64gruaswg405hcgr4iujarctiqek/j6vx35cus2k04jqmcv0mrb2fapeuhvjjhpv1kk8xkdfgnwijtnu+cyf/9ifaibzlkykyegcr+ulef23d1hh4dwhorqylwbx3ocgjgby4vifqhejgvfgz+vzdgepcidgwsywwyzxzd6o1y/dzgif3gqvkhbqgd5buoyw8at2eyzlj1arc4o3pdd1srwksxw8mbo3ezrb3e6tlg9e3pcifqk33gbgirq4w40slpoan+gbmjy5owqt63qrcgqdudmvpytrg/6gnrgmhg1ycxobt1c+ksbleyixxsq15mg4yoykam4hn5ybnlf2qin+hfbrpdup6h9qwip0bdwnmfuwau/5ioimxswthzorgzixy5xzx7gj3iyvjovqlskklr4kcwk2heo7qsa03jpkeajknk6kaytpdeb46vwuyxhx5mtgf70/hpshb9fxtjybozadb85dp+mmnefoljnyxs6ew1ofwl9a31rf2x/bx9s/2j8xqa2lpua+85e1f/0g2slfpa==</latexit> <latexit sha1_base64="gdb+6yymspm7ifmrairgazhvsm0=">aaaeqxicdvnlb9naehythsw8wjhygvejjrbculucvukqqerzqkh0jxxtal1e26uu12z3ntay/ne48a+4cefcays4cmhswki0mkdp89pvvpn1mymmhqw+l7xav65eu758w7156/aduyur9/znmmvg91gqu33ou8oluhzpciv5yay5txzjd/yt7sp+mopaiftt2nngjwmnlagfoxzd09xwavh8lkvjqlvqkch7xs0tc8kvytvpqi2l6lggfdbjkzxer+jl6bpepuifxfm3w7uafqm8tjvqqfkss6o9pcfkvfvbjdcglowaykbfccpsdcoacgltjagcb/ego7iylaxxc3nt5xm11w8lmo5fcs+hk/vqct0kj2vyg4xh64gruaswg405hcgr4iujarctiqek/j6vx35cus2k04jqmcv0mrb2fapeuhvjjhpv1kk8xkdfgnwijtnu+cyf/9ifaibzlkykyegcr+ulef23d1hh4dwhorqylwbx3ocgjgby4vifqhejgvfgz+vzdgepcidgwsywwyzxzd6o1y/dzgif3gqvkhbqgd5buoyw8at2eyzlj1arc4o3pdd1srwksxw8mbo3ezrb3e6tlg9e3pcifqk33gbgirq4w40slpoan+gbmjy5owqt63qrcgqdudmvpytrg/6gnrgmhg1ycxobt1c+ksbleyixxsq15mg4yoykam4hn5ybnlf2qin+hfbrpdup6h9qwip0bdwnmfuwau/5ioimxswthzorgzixy5xzx7gj3iyvjovqlskklr4kcwk2heo7qsa03jpkeajknk6kaytpdeb46vwuyxhx5mtgf70/hpshb9fxtjybozadb85dp+mmnefoljnyxs6ew1ofwl9a31rf2x/bx9s/2j8xqa2lpua+85e1f/0g2slfpa==</latexit> <latexit sha1_base64="7twqmwfpajah+ulapqj8wlzpg2m=">aaacvhicbvfdb9mwfhxc1whfbr55saiqwsrkshcgajqjoe0xlwg3qaki27ltrdpozjudysqfhcf+du5acbpxjuth59ypc69pjbixcfw7cg/dvnp33sh96mhdr4+f9j4+m5qy1gwmrbslvqdegoakjpzbarevbikpgho6/tlq51egds/vn7upyc7juvefz8r6ku/9yhr8z6wurbuug1paojdjyleu4nhtrhtyuienxo4yckuuhfzwxcpxvkiqcsizh045pearmtc6sf0upfkyxpv510jl12u0+bmejg+zu+lhdhegp6rboe9rcd7oyks4s6yztppftb6h+bbve1i8hy6idly7vxbyxj8exv3gmydzgt7arzr3fmzfywojyjjbjjklcwxnjmjlmqdvutzqebyms5h5qigem3fd8rv8yjmfxptap2vxx/5b4yg0ziopz2yxmde1lvyfnqvt4njuukpqc4ptby1qgw2j25/ebdfarnh4qjjm3itmk6ijs/6/i3+e5prkn8h0ajteo2r81d/5udvhaxqbxqibsta7dilouiomiaxvgzxybtz8hbbhoptb1ddy1txhexfe/qfvwtkm</latexit> <latexit sha1_base64="7twqmwfpajah+ulapqj8wlzpg2m=">aaacvhicbvfdb9mwfhxc1whfbr55saiqwsrkshcgajqjoe0xlwg3qaki27ltrdpozjudysqfhcf+du5acbpxjuth59ypc69pjbixcfw7cg/dvnp33sh96mhdr4+f9j4+m5qy1gwmrbslvqdegoakjpzbarevbikpgho6/tlq51egds/vn7upyc7juvefz8r6ku/9yhr8z6wurbuug1paojdjyleu4nhtrhtyuienxo4yckuuhfzwxcpxvkiqcsizh045pearmtc6sf0upfkyxpv510jl12u0+bmejg+zu+lhdhegp6rboe9rcd7oyks4s6yztppftb6h+bbve1i8hy6idly7vxbyxj8exv3gmydzgt7arzr3fmzfywojyjjbjjklcwxnjmjlmqdvutzqebyms5h5qigem3fd8rv8yjmfxptap2vxx/5b4yg0ziopz2yxmde1lvyfnqvt4njuukpqc4ptby1qgw2j25/ebdfarnh4qjjm3itmk6ijs/6/i3+e5prkn8h0ajteo2r81d/5udvhaxqbxqibsta7dilouiomiaxvgzxybtz8hbbhoptb1ddy1txhexfe/qfvwtkm</latexit> <latexit sha1_base64="7twqmwfpajah+ulapqj8wlzpg2m=">aaacvhicbvfdb9mwfhxc1whfbr55saiqwsrkshcgajqjoe0xlwg3qaki27ltrdpozjudysqfhcf+du5acbpxjuth59ypc69pjbixcfw7cg/dvnp33sh96mhdr4+f9j4+m5qy1gwmrbslvqdegoakjpzbarevbikpgho6/tlq51egds/vn7upyc7juvefz8r6ku/9yhr8z6wurbuug1paojdjyleu4nhtrhtyuienxo4yckuuhfzwxcpxvkiqcsizh045pearmtc6sf0upfkyxpv510jl12u0+bmejg+zu+lhdhegp6rboe9rcd7oyks4s6yztppftb6h+bbve1i8hy6idly7vxbyxj8exv3gmydzgt7arzr3fmzfywojyjjbjjklcwxnjmjlmqdvutzqebyms5h5qigem3fd8rv8yjmfxptap2vxx/5b4yg0ziopz2yxmde1lvyfnqvt4njuukpqc4ptby1qgw2j25/ebdfarnh4qjjm3itmk6ijs/6/i3+e5prkn8h0ajteo2r81d/5udvhaxqbxqibsta7dilouiomiaxvgzxybtz8hbbhoptb1ddy1txhexfe/qfvwtkm</latexit> <latexit sha1_base64="7twqmwfpajah+ulapqj8wlzpg2m=">aaacvhicbvfdb9mwfhxc1whfbr55saiqwsrkshcgajqjoe0xlwg3qaki27ltrdpozjudysqfhcf+du5acbpxjuth59ypc69pjbixcfw7cg/dvnp33sh96mhdr4+f9j4+m5qy1gwmrbslvqdegoakjpzbarevbikpgho6/tlq51egds/vn7upyc7juvefz8r6ku/9yhr8z6wurbuug1paojdjyleu4nhtrhtyuienxo4yckuuhfzwxcpxvkiqcsizh045pearmtc6sf0upfkyxpv510jl12u0+bmejg+zu+lhdhegp6rboe9rcd7oyks4s6yztppftb6h+bbve1i8hy6idly7vxbyxj8exv3gmydzgt7arzr3fmzfywojyjjbjjklcwxnjmjlmqdvutzqebyms5h5qigem3fd8rv8yjmfxptap2vxx/5b4yg0ziopz2yxmde1lvyfnqvt4njuukpqc4ptby1qgw2j25/ebdfarnh4qjjm3itmk6ijs/6/i3+e5prkn8h0ajteo2r81d/5udvhaxqbxqibsta7dilouiomiaxvgzxybtz8hbbhoptb1ddy1txhexfe/qfvwtkm</latexit> Exercise 6.23 Entropy cost for running engines (Machta, Raju, Quinn, Clement) Gibbs =(1/ )exp( H(P, Q) PV). Show log( ) is G(P, ) plus terms linear in p = P and. Exponential family. <latexit sha1_base64="kstdktcy3zcatrvex6o38bmfgwg=">aaacbhicdzdlsgmxfiyz9vbhw9vln8eiuboyi9tlqiccywq2ftqhznjmg5pkhiqjlqeln76kgxekupuh3pk2phdbrx8i/hznnctnj1lotehowykslc4trxrx3bx1jc2t0vzouyezirrbep6ovoq15uzshmgg01aqkbyrp9fr8gxsv76hsrnexplrskob+5lfjgbjubdudt28qwu8v00tsavhmmmyc8zh3rhbqiavodn0ax8id01ltrwhqix+nftaxpvu6b3ts0gm7d2ey63bpkpnmgnlgof07hyytvnmhrhp29zklkgo8+ksy7hvsq/gibjhgjil3ydyllqeich2cmwg+ndtav+qttmth4c5k2lmqcszh+kmq5passkwxxqlho+swuqx+1diblhhymxurg3ha1p4v2kgno88/zko1e7ncrrbgeyba+cdi1adf6aogocao/aansczc+88oi/o66y14mxndsepow+fvqexgq==</latexit> <latexit sha1_base64="kstdktcy3zcatrvex6o38bmfgwg=">aaacbhicdzdlsgmxfiyz9vbhw9vln8eiuboyi9tlqiccywq2ftqhznjmg5pkhiqjlqeln76kgxekupuh3pk2phdbrx8i/hznnctnj1lotehowykslc4trxrx3bx1jc2t0vzouyezirrbep6ovoq15uzshmgg01aqkbyrp9fr8gxsv76hsrnexplrskob+5lfjgbjubdudt28qwu8v00tsavhmmmyc8zh3rhbqiavodn0ax8id01ltrwhqix+nftaxpvu6b3ts0gm7d2ey63bpkpnmgnlgof07hyytvnmhrhp29zklkgo8+ksy7hvsq/gibjhgjil3ydyllqeich2cmwg+ndtav+qttmth4c5k2lmqcszh+kmq5passkwxxqlho+swuqx+1diblhhymxurg3ha1p4v2kgno88/zko1e7ncrrbgeyba+cdi1adf6aogocao/aansczc+88oi/o66y14mxndsepow+fvqexgq==</latexit> <latexit sha1_base64="kstdktcy3zcatrvex6o38bmfgwg=">aaacbhicdzdlsgmxfiyz9vbhw9vln8eiuboyi9tlqiccywq2ftqhznjmg5pkhiqjlqeln76kgxekupuh3pk2phdbrx8i/hznnctnj1lotehowykslc4trxrx3bx1jc2t0vzouyezirrbep6ovoq15uzshmgg01aqkbyrp9fr8gxsv76hsrnexplrskob+5lfjgbjubdudt28qwu8v00tsavhmmmyc8zh3rhbqiavodn0ax8id01ltrwhqix+nftaxpvu6b3ts0gm7d2ey63bpkpnmgnlgof07hyytvnmhrhp29zklkgo8+ksy7hvsq/gibjhgjil3ydyllqeich2cmwg+ndtav+qttmth4c5k2lmqcszh+kmq5passkwxxqlho+swuqx+1diblhhymxurg3ha1p4v2kgno88/zko1e7ncrrbgeyba+cdi1adf6aogocao/aansczc+88oi/o66y14mxndsepow+fvqexgq==</latexit> <latexit sha1_base64="kstdktcy3zcatrvex6o38bmfgwg=">aaacbhicdzdlsgmxfiyz9vbhw9vln8eiuboyi9tlqiccywq2ftqhznjmg5pkhiqjlqeln76kgxekupuh3pk2phdbrx8i/hznnctnj1lotehowykslc4trxrx3bx1jc2t0vzouyezirrbep6ovoq15uzshmgg01aqkbyrp9fr8gxsv76hsrnexplrskob+5lfjgbjubdudt28qwu8v00tsavhmmmyc8zh3rhbqiavodn0ax8id01ltrwhqix+nftaxpvu6b3ts0gm7d2ey63bpkpnmgnlgof07hyytvnmhrhp29zklkgo8+ksy7hvsq/gibjhgjil3ydyllqeich2cmwg+ndtav+qttmth4c5k2lmqcszh+kmq5passkwxxqlho+swuqx+1diblhhymxurg3ha1p4v2kgno88/zko1e7ncrrbgeyba+cdi1adf6aogocao/aansczc+88oi/o66y14mxndsepow+fvqexgq==</latexit> (e) For a collection of particles interacting with Hamiltonian H, using = (p, )=(P, ), relate the four terms g p = h@2 log( )/@ i in terms of physical quantities given by the second derivatives of G. dg = SdT + VdP + µdn) (d) Compute the 4 4 matrixg p explicitly for the ideal gas. (Hint:

61 <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="wz9npf0ile6brrt4cqqon5c5+i0=">aaadc3icjvjnb9naef27flthk22pxbyiqokq1a5c9fckslx6qkyibau4jdabibvqer3axanglu+99k/0woekceup9ma/yejaqfoqjgt5zcyb99bjtbqu1gxbd89fuhx7zt3fpda9+w8epmovr+zzvdac+jyxutlimaupfpsdcbiotagwjrl2k+o3s/7+rzbw5oqdm2oyzixvyii4c1galxtp4grsouomrapevk30sfypnrdo2pnqvmzzeauiomtvaepugxngnft03wz02knr+k7zniq5+zvg4xvmvtpfxgxq/hm2jmsf/z8+uvyj6mwgqptyr8k0ut7zmp5dnzxtdodlwnxwrgtu7gtdoa56e4qn6jamolh7mh7nvmhaos6ztymw0g5ymumel4dihqxn+dflyybqsqzsskz/zuwfywvmj7nbrzlav3+fkflm7trlkimhpblxe7pin3qdwk22hqvquncg+jxrpjdu5xr2mehygobothewbgselfijzhh3eh1auitw+iffbhu9bhh0w/e9zs52s45f8pg8jeskjk/jdtkleekt7p16595n78i/8z/5x/yvv1tfa2zwyvz4334aiklvua==</latexit> <latexit sha1_base64="wz9npf0ile6brrt4cqqon5c5+i0=">aaadc3icjvjnb9naef27flthk22pxbyiqokq1a5c9fckslx6qkyibau4jdabibvqer3axanglu+99k/0woekceup9ma/yejaqfoqjgt5zcyb99bjtbqu1gxbd89fuhx7zt3fpda9+w8epmovr+zzvdac+jyxutlimaupfpsdcbiotagwjrl2k+o3s/7+rzbw5oqdm2oyzixvyii4c1galxtp4grsouomrapevk30sfypnrdo2pnqvmzzeauiomtvaepugxngnft03wz02knr+k7zniq5+zvg4xvmvtpfxgxq/hm2jmsf/z8+uvyj6mwgqptyr8k0ut7zmp5dnzxtdodlwnxwrgtu7gtdoa56e4qn6jamolh7mh7nvmhaos6ztymw0g5ymumel4dihqxn+dflyybqsqzsskz/zuwfywvmj7nbrzlav3+fkflm7trlkimhpblxe7pin3qdwk22hqvquncg+jxrpjdu5xr2mehygobothewbgselfijzhh3eh1auitw+iffbhu9bhh0w/e9zs52s45f8pg8jeskjk/jdtkleekt7p16595n78i/8z/5x/yvv1tfa2zwyvz4334aiklvua==</latexit> <latexit sha1_base64="wz9npf0ile6brrt4cqqon5c5+i0=">aaadc3icjvjnb9naef27flthk22pxbyiqokq1a5c9fckslx6qkyibau4jdabibvqer3axanglu+99k/0woekceup9ma/yejaqfoqjgt5zcyb99bjtbqu1gxbd89fuhx7zt3fpda9+w8epmovr+zzvdac+jyxutlimaupfpsdcbiotagwjrl2k+o3s/7+rzbw5oqdm2oyzixvyii4c1galxtp4grsouomrapevk30sfypnrdo2pnqvmzzeauiomtvaepugxngnft03wz02knr+k7zniq5+zvg4xvmvtpfxgxq/hm2jmsf/z8+uvyj6mwgqptyr8k0ut7zmp5dnzxtdodlwnxwrgtu7gtdoa56e4qn6jamolh7mh7nvmhaos6ztymw0g5ymumel4dihqxn+dflyybqsqzsskz/zuwfywvmj7nbrzlav3+fkflm7trlkimhpblxe7pin3qdwk22hqvquncg+jxrpjdu5xr2mehygobothewbgselfijzhh3eh1auitw+iffbhu9bhh0w/e9zs52s45f8pg8jeskjk/jdtkleekt7p16595n78i/8z/5x/yvv1tfa2zwyvz4334aiklvua==</latexit> <latexit sha1_base64="wz9npf0ile6brrt4cqqon5c5+i0=">aaadc3icjvjnb9naef27flthk22pxbyiqokq1a5c9fckslx6qkyibau4jdabibvqer3axanglu+99k/0woekceup9ma/yejaqfoqjgt5zcyb99bjtbqu1gxbd89fuhx7zt3fpda9+w8epmovr+zzvdac+jyxutlimaupfpsdcbiotagwjrl2k+o3s/7+rzbw5oqdm2oyzixvyii4c1galxtp4grsouomrapevk30sfypnrdo2pnqvmzzeauiomtvaepugxngnft03wz02knr+k7zniq5+zvg4xvmvtpfxgxq/hm2jmsf/z8+uvyj6mwgqptyr8k0ut7zmp5dnzxtdodlwnxwrgtu7gtdoa56e4qn6jamolh7mh7nvmhaos6ztymw0g5ymumel4dihqxn+dflyybqsqzsskz/zuwfywvmj7nbrzlav3+fkflm7trlkimhpblxe7pin3qdwk22hqvquncg+jxrpjdu5xr2mehygobothewbgselfijzhh3eh1auitw+iffbhu9bhh0w/e9zs52s45f8pg8jeskjk/jdtkleekt7p16595n78i/8z/5x/yvv1tfa2zwyvz4334aiklvua==</latexit> Control and Carnot Corrections Ben Machta (not me) S control i =4 p N log(p a /P b )+2 p 15N log(t 1 /T 2 ) Subextensive: Work ~N T1 log(p2/p1) a Exponential family variables simpler: p = β P = P/kT Zero curvature to metric: Model b manifold flat: (x,y) = (A log(β P), B log(β)) P g (P, ) µ = g (p, ) µ = g (x,y) µ = d N/P 2 N/ P N/ P 5N/2 2 N/p N/ c y=b Log[β] 0.05 c d x=a Log[β P] b a -0.25

62 <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> <latexit sha1_base64="z+2vaf1z0biwzeazhdg6txs2hae=">aaacsnicbvdpsxtbgj2nttr0h7e99jiyckoh7galhloq2omnizioka3lt5mvcxb2zjvzbses+fu89nsbf4sxhizfi5nkbat9mpb v9jrgf4ftaxlz89xvl/ux756/watsf72xjnccuwjo4w9s8ghkhp7jenhww4rslthaxrxdeaf/kdrpnfdmuq4ygcs5ugkic8ldygv6lfchn9drccpkzgdordzkywma9q0tovaf77dy/fdunk45bey481oatudjn3ih3m7sqjdfu92k2i7m7s3kkyzbivz8kckqkitvegkjv/x0igiq01cgxp9kmxpuiilkrro63hhmadxawpse6ohqzco51vm+qevdpniwp808bn6ckkezlljlvrk7e732juj//p6by32bqxueugoxwlrqfccdj/1yofsoia18qself6vxjydbug+/bovixp88lny0m5fyss62mnuf67qwgxv2qbbzbh7xpbzaeuwhhpskl2zg/yn+bn8dv4gt4tolahm3rf/ufu+a1wushw=</latexit> Control and Carnot Corrections Ben Machta (not me) Szilard Engine (Bennett, Feynman) Work done by expanding piston is kt log 2 = T ΔS Szilard argued that information can be exchanged for work. We saw this in the Burning information exercise, where a data tape stored bits by putting atoms on one side or another of a partitioned piston Machta argues that extracting this work will cost entropy 4 k log 2, a net loss! h S control i =4 p N log(p a /P b )+

63 Big Questions Consequences of Sloppiness Efficacy of principal component analysis Least squares fits: efficacy of Levenberg-Marquardt algorithm Biological evolution: sloppy or robust? Neutral spaces, evolvability Does pattern recognition work because of hyperribbons? Does big data work because of hyperribbons? Is science possible because of hyperribbons? Different models can describe the same behavior Why is the world comprehensible?

64 Sloppy Applications Several applications emerge A. Fitting data vs. measuring parameters (Gutenkunst) Fits good: measured bad B. Finding best fits by geodesic acceleration (Transtrum) C. Optimal experimental design (Casey) E. Estimating systematic errors: (Jacobsen et al.) D. Sloppy fitness and evolution (Gutenkunst) F. Sloppiness and Robustness

65 A. Are rate constants useful? Fits vs. measurements eigen params Best Fit Missing one param Measured Fits good: measured bad Fit bare params Monte Carlo (anharmonic) Easy to Fit (14 expts); Measuring huge job (48 params, 25%) One missing parameter measurement = No predictivity Sloppy Directions = Enormous Fluctuations in Parameters Sloppy Directions often do not impinge on predictivity

66 B. Finding best fits: Geodesic acceleration Geodesic Paths nearly circles Follow local geodesic velocity? δθ µ = g µν ν C è Gauss-Newton è Hits manifold boundary Model Graph add weight λ of parameter metric yields Levenberg- Marquardt: Step size now limited by curvature Follow parabola, geodesic acceleration Cheap to calculate; faster; more success

67 B. Finding best fits: Model manifold dynamics (Isabel Kloumann) Dynamics on the model manifold: Searching for the best fit Jeffrey s prior plus noise Big noise concentrates on manifold edges Note scales: flat Top: Levenberg- Marquardt Bottom: Geodesic acceleration Large points: Initial conditions which fail to converge to best fit

68 C. EGFR Trafficking Model Fergal Casey, Cerione lab Cerione lab: testing hypothesis, experimental design (Cool1 β-pix) 41 chemicals, 53 rate constants; only 11 of 41 species can be measured Does Cool-1 triple complex sequester Cbl, delay endocytosis in wild type NIH3T3 cells? recycling internalization Receptor activation of MAPK pathway and other effectors (e.g. Src)

69 C. Trafficking: experimental design Which experiment best reduces prediction uncertainty? Amount of triple complex was not well predicted V-optimal experimental design: single & multiple measurements Total active Cdc42 at 10 min.; Cerione independently concurs Experiment indicates significant sequestering in wild type Predictivity without decreasing parameter uncertainty

70 D. Evolution in Chemotype space Implications of sloppiness? Fitness gain from first successful mutation Culture of identical bacteria, one mutation at a time Mutation changes one or two rate constants (no pleiotropy): orthogonal moves in rate constant (chemotype) space Cusps in first fitness gain (one for each rate constant, big gap) Multiple mutations get stuck on ridge in sloppy landscape

71 E. Bayesian Errors for Atoms Sloppy Model Approach to Error Estimation of Interatomic Potentials Søren Frederiksen, Karsten W. Jacobsen, Kevin Brown, JPS Quantum Electronic Structure (Si) 90 atoms (Mo) (Arias) Interatomic Potentials V(r 1,r 2, ) Fast to compute Limit m e /M 0 justified Guess functional form Pair potential V(r i -r j ) poor Bond angle dependence Coordination dependence Fit to experiment (old) Fit to forces from electronic structure calculations (new) Atomistic potential 820,000 Mo atoms (Jacobsen, Schiøtz) 17 Parameter Fit

72 E. Interatomic Potential Error Bars Ensemble of Acceptable Fits to Data Not transferable Unknown errors 3% elastic constant 10% forces 100% fcc-bcc, dislocation core Best fit is sloppy: ensemble of fits that aren t much worse than best fit. Ensemble in Model Space! T 0 set by equipartition energy = best cost T 0 Green = DFT, Red = Fits Error Bars from quality of best fit

73 Sloppy Molybdenum: Does it Work? Estimating Systematic Errors Bayesian error σ i gives total error if ratio r = error i /σ i distributed as a Gaussian: cumulative distribution P(r)=Erf(r/ 2) Three potentials Force errors Elastic moduli Surfaces Structural Dislocation core 7% < σ i < 200% Sloppy model systematic error most of total ~2 << 200%/7% Note: tails Worst errors underestimated by ~ factor of 2

74 Systematic Error Estimates for DFT GGA-DFT as Multiparameter Fit? J. J. Mortensen, K. Kaasbjerg, S. L. Frederiksen, J. K. Nørskov, JPS, K. W. Jacobsen, (Anja Tuftelund, Vivien Petzold, Thomas Bligaard) Enhancement factor F x (s) in the exchange energy E x Large fluctuations Actual error / predicted error Deviation from experiment well described by ensemble!

75 F. Parameter robustness and sloppiness Do parameters matter at all? Bryan Daniels, Yanjiun Chen, Ryan Gutenkunst, Chris Myers Fruit fly embryo Text development model robust: 1/200 parameter sets in allowed region [± 3000%] fits data. (Naïve green cube: L 48 = 1/200) Segment polarity model is sloppy (eigenvalues, left) and robust (PCA, phenotype-preserving in cube, right). Text Model is sloppy: only four parameter directions vary less than ± 3000%. (Red brick: allowed regions of three stiffest to give 1/200 acceptance).

76 F. Environmental robustness and sloppiness Circadian Rhythms Sloppiness facilitates finding parameter values for robust response to environmental change. Bacteria know the time of day! How do they keep their clocks on time in the cold? All reaction rates exponentially dependent on temperature! Delicate cancellation? Three sloppy directions, 18 rates exponentially dependent on temperature. Three stiff directions at each temperature? 18 3 (25 C) 3 (30 C) 3 (35 C) = 9 dimensional robust parameter space

77 F. Evolvability, robustness, and sloppiness If it s robust, can it evolve? Sloppy signal transduction model Individual evolvabilities ec(f) within phenotype Population evolvability ed within phenotype [Sloppy neutral spaces allow species to explore large ranges of parameters] How can evolution proceed if mutating the parameters doesn t matter? More robust, less evolvable! Evolutionary force F