Physics, Mathematics and Computers

Transcription

1 Physics, Mathematics and Computers Better algorithms + Better tools = Better science! Fernando Pérez Department of Applied Mathematics University of Colorado, Boulder. U. C. Berkeley May 16, 2007

2 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 2 / 45

3 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 2 / 45

4 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 2 / 45

5 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 2 / 45

6 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing 5 Other projects of interest F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 2 / 45

7 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing 5 Other projects of interest F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 3 / 45

8 My background: theoretical physics, numerics Physics Lattice QCD (Quantum Chromo Dynamics): quarks and gluons. Classical chaos: classical 3-body Coulomb system. Quantum chaos: a quantum version of Lyapunov exponents. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 4 / 45

9 My background: theoretical physics, numerics Physics Lattice QCD (Quantum Chromo Dynamics): quarks and gluons. Classical chaos: classical 3-body Coulomb system. Quantum chaos: a quantum version of Lyapunov exponents. Applied Mathematics Algorithm design for computational physics. Strong connection between the physics and the numerics. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 4 / 45

10 My background: theoretical physics, numerics Physics Lattice QCD (Quantum Chromo Dynamics): quarks and gluons. Classical chaos: classical 3-body Coulomb system. Quantum chaos: a quantum version of Lyapunov exponents. Applied Mathematics Algorithm design for computational physics. Strong connection between the physics and the numerics. Computing Tools I picked up as I needed them. I ended up writing many of them, because they didn t exist... or what existed was not good enough. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 4 / 45

11 Why am I here? Evolution of intent Particle physics, quantum chaos: extremely theoretical. Applied Mathematics: ideas that are actually applicable. Computing: shattered my prejudice against practicality. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 5 / 45

12 Why am I here? Evolution of intent Particle physics, quantum chaos: extremely theoretical. Applied Mathematics: ideas that are actually applicable. Computing: shattered my prejudice against practicality. Consistency of background Physics: a solid understanding of nature s basic mechanisms. Math: robust approaches to extract valid, reliable answers. Computing: build practical tools that support the scientific effort. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 5 / 45

13 Why am I here? Evolution of intent Particle physics, quantum chaos: extremely theoretical. Applied Mathematics: ideas that are actually applicable. Computing: shattered my prejudice against practicality. Consistency of background Physics: a solid understanding of nature s basic mechanisms. Math: robust approaches to extract valid, reliable answers. Computing: build practical tools that support the scientific effort. Why neuroscience? The life sciences are now quantitiative. A fascinating multiscale problem. Very well suited to my background (renormalization, wavelets,...) F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 5 / 45

14 Some physical problems of interest Their natural formulation is a (Partial) Differential Equation (PDE): Poisson: gravity, electricity,... 2 u = f Schrödinger: quantum mechanics ( 1 ) V ψ = Eψ Navier-Stokes (modified form): fluids αv µ 2 v + p = f v = 0 A good fraction of the world s (scientific) computing time is spent on these. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 6 / 45

15 What are we after? Immediate Goals Accurate and reliable methods of solution... that work efficiently (time and storage)... in more than one dimension... and for a wide class of problems. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 7 / 45

16 What are we after? Immediate Goals Accurate and reliable methods of solution... that work efficiently (time and storage)... in more than one dimension... and for a wide class of problems. Overall program A robust formulation: apply integral operators instead of inverting differential ones (don t divide by zero!). Build approximations with finite but controlled precision. Develop multiscale, fully adaptive algorithms... That generalize naturally to more than one dimension. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 7 / 45

17 Why do we need fast algorithms? F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

18 Why do we need fast algorithms? Because computers are getting bigger and faster! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

19 Why do we need fast algorithms? Because computers are getting bigger and faster! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

20 Why do we need fast algorithms? Because computers are getting bigger and faster! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

21 Why do we need fast algorithms? Because computers are getting bigger and faster! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

22 Why do we need fast algorithms? Because computers are getting bigger and faster! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

23 Why do we need fast algorithms? Because computers are getting bigger and faster! How do you get a fast algorithm? Compress: factorize, find an alternate representation, approximate, decouple. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 8 / 45

24 Multiresolution algorithms in multiple dimensions Key mathematical ideas: 1 Multiresolution analysis (wavelets): sparse representations (compress). 2 Separated representations: reduction of dimensionality cost (approximate, decouple). F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 9 / 45

25 Multiresolution algorithms in multiple dimensions Key mathematical ideas: 1 Multiresolution analysis (wavelets): sparse representations (compress). 2 Separated representations: reduction of dimensionality cost (approximate, decouple). Group effort over many years (1988-today): 1 Gregory Beylkin, Lucas Monzón, Christopher Kurcz - CU Boulder 2 Martin Mohlenkamp - Ohio University 3 Robert Harrison, George Fann, Takeshi Yanai, Zhengting Gan - ORNL 4 Vani Cheruvu - (now at NCAR) 5 Robert Cramer - (now at Raytheon) F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 9 / 45

26 Outline Physics/Math MRA IPython Parallel Other projects Functions Operators d = 1 recap 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing 5 Other projects of interest F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 10 / 45

27 Functions Operators d = 1 recap A specific example: Poisson s equation The solution to 2 φ(r) = ρ(r) can be written in integral form as Z φ(r) = Z G(r,r )ρ(r )d 3 r = 1 r r ρ(r )d 3 r. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 11 / 45

28 Functions Operators d = 1 recap A specific example: Poisson s equation The solution to 2 φ(r) = ρ(r) can be written in integral form as Z φ(r) = Z G(r,r )ρ(r )d 3 r = 1 r r ρ(r )d 3 r. We need: A way of representing ρ(r ) and G(r,r ) efficiently (very different problems) A way of applying the integral (a generalized multiplication). This is a full 3-dimensional problem. We d like a method that easily generalizes to other G(r,r ) functions. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 11 / 45

29 Functions Operators d = 1 recap Multiresolution analysis, intuitively Imagine a simple signal f(t) you want to study: F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 12 / 45

30 Functions Operators d = 1 recap Multiresolution analysis, intuitively Imagine a simple signal f(t) you want to study: At each scale n, divide the unit interval [0,1] into 2 n binary subintervals: F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 12 / 45

31 Multiresolution basics (2) And compute: Average values of function at level n : space V n. Differences between successive levels: space W n = V n+1 V n.

32 Multiresolution basics (2) And compute: Average values of function at level n : space V n. Differences between successive levels: space W n = V n+1 V n. f(t) can studied (compressed, denoised,...) from {V 0,W 0,W 1,...} : V2 V4 V9-full W2 W4 W Note: the coefficients in W n are small and localized around changes.

33 Functions: adaptive, controlled accuracy decompositions Nnod = 12, ǫ = 1.0e 10, Nblocks = 21

34 Functions: adaptive, controlled accuracy decompositions Nnod = 12, ǫ = 1.0e 10, Nblocks = 21 Nnod = 10, ǫ = 5.0e 11, Nblocks = 634

35 Sparse operators Sparse: having few non-zero entries (compressible fast good!). Instead of applying this:

36 Sparse operators Sparse: having few non-zero entries (compressible fast good!). Instead of applying this: We apply this (blue is zero, not done):

37 Functions Operators d = 1 recap Adaptive natural-scale application A graphical representation F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 16 / 45

38 Functions Operators d = 1 recap For one dimension (d = 1), lessons learned The Good 1 Multiwavelets sparse operators fast algorithms. 2 Accuracy is guaranteed by construction. 3 We efficiently handle multi-scale interactions. 4 Hierarchical, adaptive algorithm. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 17 / 45

39 Functions Operators d = 1 recap For one dimension (d = 1), lessons learned The Good 1 Multiwavelets sparse operators fast algorithms. 2 Accuracy is guaranteed by construction. 3 We efficiently handle multi-scale interactions. 4 Hierarchical, adaptive algorithm. The Bad This approach does not directly extend to d > 1. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 17 / 45

40 Operators(d > 1): Gaussians to the rescue Use Gaussians for approximations M 1 r r w m e τ m r r 2, m=1 with controlled accuracy ε: The problem factorizes partially: a practical solution for d > 1! φ ijk = N i j k =1G ii,jj,kk ρ i j k = M m=1 w m F m i ii j F m jj k F m kk ρ i j k

41 Functions Operators d = 1 recap What is all of this good for? In computational physics Electrostatics, scattering, quantum mechanics, fluids,... Quantum mechanics (many electron atoms): A very nasty multidimensional problem. Collaborations with ORNL and Ohio U: a generic toolkit (MADNESS). F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 19 / 45

42 Functions Operators d = 1 recap What is all of this good for? In computational physics Electrostatics, scattering, quantum mechanics, fluids,... Quantum mechanics (many electron atoms): A very nasty multidimensional problem. Collaborations with ORNL and Ohio U: a generic toolkit (MADNESS). Applications in neuroscience? This is speculative!!! Hierarchical modeling: Multiresolution operator representations effective multiscale theories. Novel approaches for high-dimensional datasets. PDE - based algorithms for image processing. Wavelet-based representations of fmri objects for algorithmic work....? I hope to talk to some of you today about this. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 19 / 45

43 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing 5 Other projects of interest F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 20 / 45

44 Python in scientific computing Free, interactive, highly portable language. Extremely readable syntax ( executable pseudo-code ). Rich built-in types: lists, sets, dictionaries (hash tables), strings,... Very comprehensive standard library (batteries included): Text processing, networking protocols, threading, GUIs,... Standard libraries for Matlab/IDL-like arrays. Easy to wrap existing C, C++ and FORTRAN codes: Great code reuse! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 21 / 45

45 Python is growing in scientific computing Neuroimaging article by J. Millman and M. Brett

46 Data analysis for epilepsy surgery John Hunter, Pediatric Neurology, U. Chicago

47 Correlation analysis of seizure data

48 Final location of epileptic foci for surgery

49 Why do I use Python? Very complex algorithms: rich types and easy object model. Exploratory interactive work: IPython. Visualization: 2D and 3D Numerics: numpy, scipy, in-house Fortran/C codes Access to lots of third-party libraries (GUIs, networking,...) We need these things in scientific computing... but we don t want to write them ourselves! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 26 / 45

50 IPython: better interactive work Why write this? Scientific computing is inherently exploratory. A good interactive environment is a necessity. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 27 / 45

51 IPython: better interactive work Why write this? Scientific computing is inherently exploratory. A good interactive environment is a necessity. 1 A better Python shell: object introspection, system access,... 2 An embeddable interpreter: allow any program to be probed interactively. 3 A flexible component: base layer for other interactive systems. 4 High-level distributed/parallel computing. 5 A component we can plug into GUIs, browser shells, etc. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 27 / 45

52 Matlab-like usage F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 28 / 45

53 Sophisticated 3d visualizations with VTK F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 29 / 45

54 Who uses IPython? IPython is available for all Linux distributions, OS X and Windows. SAGE: open source mathematics, very ambitious project (U. Washington). PyRAF: astronomical image analysis (Space Telescope Science Institute). CASA: Common Astronomy Software Applications (National Radio Astronomy Observatory). Ganga: job control for the LHCb and ATLAS experiments at CERN. PyMAD: a neutron spectrometer at the Institut Laue Langevin (Grenoble). Pymerase: microarray gene expression databases. Many others... F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 30 / 45

55 Why the popularity? Write a useful tool... That is highly customizable. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 31 / 45

56 Why the popularity? Write a useful tool... That is highly customizable. Keep a lively community and grow it: Mailing lists, wiki-based website. Developers in the US and Europe. User-contributed articles and videos... F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 31 / 45

57 Why the popularity? Write a useful tool... That is highly customizable. Keep a lively community and grow it: Mailing lists, wiki-based website. Developers in the US and Europe. User-contributed articles and videos... Listen to the needs of users (esp. major projects): Add functionality they need. Adapt and evolve the design as needed. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 31 / 45

58 Why the popularity? Write a useful tool... That is highly customizable. Keep a lively community and grow it: Mailing lists, wiki-based website. Developers in the US and Europe. User-contributed articles and videos... Listen to the needs of users (esp. major projects): Add functionality they need. Adapt and evolve the design as needed. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 31 / 45

59 Why the popularity? Write a useful tool... That is highly customizable. Keep a lively community and grow it: Mailing lists, wiki-based website. Developers in the US and Europe. User-contributed articles and videos... Listen to the needs of users (esp. major projects): Add functionality they need. Adapt and evolve the design as needed. The lesson? Solve your problem well, with enough generality and flexibility... and you ll solve a lot more than just your problem. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 31 / 45

60 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing 5 Other projects of interest F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 32 / 45

61 Parallel computing: why should we care? F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 33 / 45

62 Parallel computing: why should we care? Because reality looks like this: F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 33 / 45

63 We can t escape thermodynamics

64 We can t escape thermodynamics The vendor s solutions: Multicore chips: even in your laptop. Graphics cards for general computing: > 128 processors per card. High-density clusters: SiCortex (> 5000 processors in a cabinet).

65 Parallel programming? There are plenty of bad news It is in general, extremely difficult. Scientific productivity plummets with enormous up-front efforts. Development, debugging, running is hard and cumbersome. With propietary tools, licensing costs can go through the roof. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 35 / 45

66 Parallel programming? There are plenty of bad news It is in general, extremely difficult. Scientific productivity plummets with enormous up-front efforts. Development, debugging, running is hard and cumbersome. With propietary tools, licensing costs can go through the roof. But not all is doom and gloom Many problems are embarrassingly parallel: uncoupled components. This is common in neuroscience: Analyze many scans in an fmri run. Global parameter search in generative models. SPM has plenty of opportunity for easy massive speedups. Even not-so-embarrassingly parallel problems can be tractable with the right tools. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 35 / 45

67 Network-aware IPython F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 36 / 45

68 Distributed/parallel computing Think of Python as the CPU But these souped-up kernels let you talk to it conveniently. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 37 / 45

69 What does IPython offer here? Easy reuse and distribution of existing serial ( normal ) codes. High-level abstractions for embarrassingly parallel problems. Out-of-the box task farming tools. Task farming system is low-latency can be integrated into more complex codes. Implement any approach to parallelism you want: Task farming. Traditional Message Passing (MPI). Integrate hybrid codes. Actively developed (Colorado, Berkeley). F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 38 / 45

70 Outline Physics/Math MRA IPython Parallel Other projects 1 Physics and Mathematics 2 Numerical Multiresolution Algorithms 3 Computing with Python and IPython 4 The future: IPython and parallel computing 5 Other projects of interest F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 39 / 45

71 FluidLab: a MayaVi based CFD visualization tool With: K. Julien, P. Schmitt and B. Barrow (Applied Math, U. Colorado).

72 FluidLab: an Envisage project. Envisage: plugin-based system for scientific application development.

73 Graphics cards for numerics Why? Graphics cards ( GPUs ) are basically very fast, highly parallel computers... F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 42 / 45

74 Graphics cards for numerics Why? Graphics cards ( GPUs ) are basically very fast, highly parallel computers... Problem: the unequally spaced FFT (Fast Fourier Transform) Very painful, but we got our part working. Recent hardware makes life a lot easier. Much less of a square peg (numerical algorithm) into a round hole (a 3d graphics language). F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 42 / 45

75 Graphics cards for numerics Why? Graphics cards ( GPUs ) are basically very fast, highly parallel computers... Problem: the unequally spaced FFT (Fast Fourier Transform) Very painful, but we got our part working. Recent hardware makes life a lot easier. Much less of a square peg (numerical algorithm) into a round hole (a 3d graphics language). Possibilities for neuroscience? Real-time analysis of EEG/ECoG data (K. Koepsell, R. Canolty). Speed up FFT and linear-algebra based algorithms. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 42 / 45

76 In closing... Better algorithms: From the physics to the mathematics: correct and robust. Multi-scale interactions. Hierarchical models and effective theories. High dimensional problems. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 43 / 45

77 In closing... Better algorithms: From the physics to the mathematics: correct and robust. Multi-scale interactions. Hierarchical models and effective theories. High dimensional problems. Better computational tools: Exploratory computing. Integration with GUIs, the internet, low-level codes,... Improved access to parallel resources. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 43 / 45

78 In closing... Better algorithms: From the physics to the mathematics: correct and robust. Multi-scale interactions. Hierarchical models and effective theories. High dimensional problems. Better computational tools: Exploratory computing. Integration with GUIs, the internet, low-level codes,... Improved access to parallel resources. A culture of open and reproducible computational research: Transparent workflows. Flexible tools with interchangeable components. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 43 / 45

79 In closing... Better algorithms: From the physics to the mathematics: correct and robust. Multi-scale interactions. Hierarchical models and effective theories. High dimensional problems. Better computational tools: Exploratory computing. Integration with GUIs, the internet, low-level codes,... Improved access to parallel resources. A culture of open and reproducible computational research: Transparent workflows. Flexible tools with interchangeable components. F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 43 / 45

80 In closing... Ultimately Better algorithms: From the physics to the mathematics: correct and robust. Multi-scale interactions. Hierarchical models and effective theories. High dimensional problems. Better computational tools: Exploratory computing. Integration with GUIs, the internet, low-level codes,... Improved access to parallel resources. A culture of open and reproducible computational research: Transparent workflows. Flexible tools with interchangeable components. Better algorithms + better tools = better science! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 43 / 45

81 EXTRA SLIDES just in case... F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 44 / 45

82 Sparsity also in the m direction? The Gaussian expansion gave us separation of directions... at the cost of a new internal degree of freedom, the separation index m. Do we really need all these terms? F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 45 / 45

83 Sparsity also in the m direction? The Gaussian expansion gave us separation of directions... at the cost of a new internal degree of freedom, the separation index m. Do we really need all these terms? NO: the 2-scale differences cancel most of them! F. P. (App. Math - CU Boulder) Physics, Mathematics and Computers Berkeley 45 / 45