MPI at MPI. Jens Saak. Max Planck Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory

Transcription

1 MAX PLANCK INSTITUTE November 5, 2010 MPI at MPI Jens Saak Max Planck Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory FOR DYNAMICS OF COMPLEX TECHNICAL SYSTEMS MAGDEBURG Max Planck Institute Magdeburg Jens Saak, MPI at MPI 1/10

2 Parallel Computing Basics Why Parallel Computing? O nce upon a time... Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

3 Parallel Computing Basics Why Parallel Computing? O nce upon a time......the world was simple... Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

4 Parallel Computing Basics Why Parallel Computing? O nce upon a time......the world was simple......and problems were small... Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

5 Parallel Computing Basics Why Parallel Computing? O nce upon a time......the world was simple......and problems were small......all computations could be carried out on a single computer. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

6 Parallel Computing Basics Why Parallel Computing? Then the world grew complicated and problems became large and difficult. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

7 Parallel Computing Basics Why Parallel Computing? Then the world grew complicated and problems became large and difficult. Computations became to challenging for a single computer. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

8 Parallel Computing Basics Why Parallel Computing? Then the world grew complicated and problems became large and difficult. Computations became to challenging for a single computer. Therefore problems were split Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

9 Parallel Computing Basics Why Parallel Computing? Then the world grew complicated and problems became large and difficult. Computations became to challenging for a single computer. Therefore problems were split and distributed to multiple computers Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

10 Parallel Computing Basics Why Parallel Computing? Then the world grew complicated and problems became large and difficult. Computations became to challenging for a single computer. Therefore problems were split and distributed to multiple computers jointly finding their solution. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 2/10

11 Parallel Computing Basics Types of Parallelism Quasi Parallelism, or Multitasking modern operating systems emulate parallelism via time-slicing Max Planck Institute Magdeburg Jens Saak, MPI at MPI 3/10

12 Parallel Computing Basics Types of Parallelism Quasi Parallelism, or Multitasking modern operating systems emulate parallelism via time-slicing Distributed Memory Parallelism computation on separate units with own memory on each unit Max Planck Institute Magdeburg Jens Saak, MPI at MPI 3/10

13 Parallel Computing Basics Types of Parallelism Quasi Parallelism, or Multitasking modern operating systems emulate parallelism via time-slicing Distributed Memory Parallelism computation on separate units with own memory on each unit Shared Memory Parallelism multiple computation units access a common memory Max Planck Institute Magdeburg Jens Saak, MPI at MPI 3/10

14 Parallel Computing Basics Implementations and Standards Quasi Parallelism operating systems scheduler processes process-threads (e.g. POSIX-threads, pthreads) Max Planck Institute Magdeburg Jens Saak, MPI at MPI 4/10

15 Parallel Computing Basics Implementations and Standards Quasi Parallelism operating systems scheduler processes process-threads (e.g. POSIX-threads, pthreads) Distributed Memory Parallelism Data handling via the Message Passing Interface (MPI) (standard for a collection of operations and their semantics, NOT prescribing the actual protocol or implementation.) MVAPICH/MVAPICH2 (implements MPI-1/MPI-2, BSD-License) OpenMPI (implements MPI-1, MPI-2, freely available) MPICH2 (implements MPI-1, MPI-2, MPI-2.1, MPI-2.2, freely available) Max Planck Institute Magdeburg Jens Saak, MPI at MPI 4/10

16 Parallel Computing Basics Implementations and Standards Quasi Parallelism operating systems scheduler processes process-threads (e.g. POSIX-threads, pthreads) Distributed Memory Parallelism Data handling via the Message Passing Interface (MPI) (standard for a collection of operations and their semantics, NOT prescribing the actual protocol or implementation.) MVAPICH/MVAPICH2 (implements MPI-1/MPI-2, BSD-License) OpenMPI (implements MPI-1, MPI-2, freely available) MPICH2 (implements MPI-1, MPI-2, MPI-2.1, MPI-2.2, freely available) Shared Memory Parallelism OpenMP API: compiler and preprocessor directives. (latest version 3.0; May 2008) GCC and later Intel Compilers 10.1 and later Oracle, IBM, Cray,... Max Planck Institute Magdeburg Jens Saak, MPI at MPI 4/10

17 Parallel Computing at MPI Magdeburg otto node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend [40-41] ibr[1-2] [40-41] [40-41] [40-41] [17-32] node[33-48] Typ [33-48] [33-48] [1-11] [33-48] [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

18 Parallel Computing at MPI Magdeburg otto frontend and service nodes ottoapp Megware appliance [1-2] node[1-16] Typ [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend [40-41] ibr[1-2] [40-41] [40-41] [40-41] [17-32] node[33-48] Typ [33-48] [33-48] [1-11] [33-48] [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

19 Parallel Computing at MPI Magdeburg otto frontend and service nodes ottoapp Megware appliance [1-2] node[1-16] Typ [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend Cluster-Management: [40-41] ibr[1-2] [17-32] [40-41] MEGWARE ClusterWare [40-41] Appliance [40-41] [33-48] IPMI capable (Intelligent Platform Management Interface) [1-11] integration to job queueing system [33-48] [33-48] node[33-48] Typ [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

20 Parallel Computing at MPI Magdeburg otto frontend and service nodes ottoapp Megware appliance otto[1-2] Frontend [1-2] frontend nodes: [1-2] [40-41] [1-11] user interaction, binary creation, job queueing,... ibr[1-2] [40-41] [40-41] [40-41] node[1-16] Typ 2 node[17-32] Typ [17-32] [17-32] [17-32] node[33-48] Typ [33-48] [33-48] [33-48] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de [ ] [ ] Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

21 Parallel Computing at MPI Magdeburg otto network and storage node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend [40-41] ibr[1-2] [40-41] [40-41] [40-41] [17-32] node[33-48] Typ [33-48] [33-48] [1-11] [33-48] service network [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

22 Parallel Computing at MPI Magdeburg otto network and storage node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] storage: Panasas PanFS-8 netto capacity: 28TB otto[1-2] Frontend connect: 2x10GBit Ethernet [40-41] [1-11] ibr[1-2] [40-41] [40-41] [40-41] [17-32] [17-32] node[33-48] Typ [33-48] [33-48] [33-48] service network Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de [ ] [ ] Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

23 Parallel Computing at MPI Magdeburg otto network and storage node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend communication network: x Voltaire QDR Infiniband switches 36 ports, 8port inter-connnect transfer rate: 40GBit/s [40-41] [1-11] ibr[1-2] [40-41] [40-41] [40-41] [17-32] node[33-48] Typ [33-48] [33-48] [33-48] service network [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

24 Parallel Computing at MPI Magdeburg otto node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend [40-41] ibr[1-2] [40-41] [40-41] [40-41] [17-32] node[33-48] Typ [33-48] [33-48] [1-11] [33-48] compute nodes [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

25 Parallel Computing at MPI Magdeburg otto node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] [17-32] Institutsnetz dns: otto[1-2] Frontend ntp: ntp.mpi-magdeburg.mpg.de MEGWARE MiriQuid X5600: [40-41] ibr[1-2] [40-41] [40-41] [40-41] [1-11] x Xeon Westmere X GHz 6 cores per CPU peak performance per node: 127,68GFlops RAM: 48GB DDR3 ECC L3 Cache: 12MB [ ] [17-32] node[33-48] Typ [33-48] [33-48] [33-48] compute nodes [ ] Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

26 Parallel Computing at MPI Magdeburg otto node[1-16] Typ ottoapp Megware appliance [1-2] MEGWARE Saxonid C6100: otto[1-2] Frontend L3 Cache: 12MB [1-2] 2x AMD Opteron 6168 Magny-Cours 1,9GHz 12 cores per CPU RAM: 96GB DDR3 ECC hard disc: 650GB SAS (15000rps) [40-41] [1-11] ibr[1-2] [40-41] [40-41] [40-41] node[17-32] Typ [17-32] [17-32] [17-32] node[33-48] Typ [33-48] [33-48] [33-48] compute nodes [ ] [ ] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

27 Parallel Computing at MPI Magdeburg otto node[1-16] Typ ottoapp Megware appliance [1-2] [1-2] node[17-32] Typ [17-32] [17-32] otto[1-2] Frontend [40-41] ibr[1-2] [40-41] [40-41] [40-41] [17-32] node[33-48] Typ [33-48] [33-48] [1-11] [33-48] Institutsnetz dns: ntp: ntp.mpi-magdeburg.mpg.de [ ] quasi parallel: e.g., threaded I/O, distributed: every node has its own RAM, [ ] shared: 12/24 cores per node access the common RAM. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 5/10

28 Parallel Computing at MPI Magdeburg Molecular Simulations and Design From wavefunctions to proteins in networks The Molecular Simulations and Design (MSD) group aims at an explanation of chemical and biological phenomena at molecular level. Complex phenomena in chemistry and biology MSD group develops and applies tools from Quantum mechanics QM/MM hybrid simulations Molecular dynamics Brownian dynamics Bioinformatics Protein structural modeling Contact: Dr. Matthias Stein Max Planck Institute Magdeburg Jens Saak, MPI at MPI 6/10

29 Parallel Computing at MPI Magdeburg Computational Methods in Systems and Control Theory (CSC) Interests: Mathematical methods for application in systems and control theory and their numerical realization, with focus on Numerical Linear Algebra. Parallel Computing: accelerate solvers on modern multi-core processors solve very large matrix equations, model order reduction problems, or optimal control problems on high performance parallel machines. Parallel Computing aware Software developed in CSC: PLiCMR C.M.E.S.S. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 7/10

30 Parallel Computing in CSC PLiCMR Parallel Library in Control and Model Reduction. Distributed memory parallel model order reduction via Balanced Truncation Singular Perturbation Approximation Hankel-Norm Approximation Partners: High Performance Computing and Architectures Group (University Jaume I; Castellón; Spain) Max Planck Institute Magdeburg Jens Saak, MPI at MPI 8/10

31 Parallel Computing in CSC C.M.E.S.S. C-language Matrix Equation Sparse Solver C-Library for solving large sparse: Lyapunov equations FX + XF T = GG T, FXE T + EXF T = GG T, See poster, as well. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 9/10

32 Parallel Computing in CSC C.M.E.S.S. C-language Matrix Equation Sparse Solver C-Library for solving large sparse: Lyapunov equations FX + XF T = GG T, FXE T + EXF T = GG T, Riccati equations C + A T X + XA XBX = 0, C + A T XE + E T XA E T XBXE = 0 See poster, as well. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 9/10

33 Parallel Computing in CSC C.M.E.S.S. C-language Matrix Equation Sparse Solver C-Library for solving large sparse: Lyapunov equations FX + XF T = GG T, FXE T + EXF T = GG T, Riccati equations C + A T X + XA XBX = 0, C + A T XE + E T XA E T XBXE = 0 Model Reduction Balanced Truncation (first and second order systems) H 2-optimal Reduction (IRKA, TSIA) See poster, as well. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 9/10

34 Parallel Computing in CSC C.M.E.S.S. C-language Matrix Equation Sparse Solver C-Library for solving large sparse: Lyapunov equations FX + XF T = GG T, FXE T + EXF T = GG T, Riccati equations C + A T X + XA XBX = 0, C + A T XE + E T XA E T XBXE = 0 Model Reduction Balanced Truncation (first and second order systems) H 2-optimal Reduction (IRKA, TSIA) Linear Quadratic Regulator Problems See poster, as well. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 9/10

35 Parallel Computing in CSC C.M.E.S.S. C-language Matrix Equation Sparse Solver C-Library for solving large sparse: Lyapunov equations FX + XF T = GG T, FXE T + EXF T = GG T, Riccati equations C + A T X + XA XBX = 0, C + A T XE + E T XA E T XBXE = 0 Model Reduction Balanced Truncation (first and second order systems) H 2-optimal Reduction (IRKA, TSIA) Linear Quadratic Regulator Problems Currently focused on multi-core desktop computers, i.e., shared memory. See poster, as well. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 9/10

36 The End Read MPI at MPI as Max Planck Institute Magdeburg Jens Saak, MPI at MPI 10/10

37 The End Read MPI at MPI as Message Passing Interface at Max Planck Institute Max Planck Institute Magdeburg Jens Saak, MPI at MPI 10/10

38 The End Read MPI at MPI as Message Passing Interface at Max Planck Institute Meet otto at MPI this November, Max Planck Institute Magdeburg Jens Saak, MPI at MPI 10/10

39 The End Read MPI at MPI as Message Passing Interface at Max Planck Institute Meet otto at MPI this November, and let us hope that this story ends: Max Planck Institute Magdeburg Jens Saak, MPI at MPI 10/10

40 The End Read MPI at MPI as Message Passing Interface at Max Planck Institute Meet otto at MPI this November, and let us hope that this story ends:...and they all computed happily ever after. Max Planck Institute Magdeburg Jens Saak, MPI at MPI 10/10