One Optimized I/O Configuration per HPC Application
|
|
- Alexander Robertson
- 5 years ago
- Views:
Transcription
1 One Optimized I/O Configuration per HPC Application Leveraging I/O Configurability of Amazon EC2 Cloud Mingliang Liu, Jidong Zhai, Yan Zhai Tsinghua University Xiaosong Ma North Carolina State University Oak Ridge National Laboratory Wenguang Chen Tsinghua University APSys 2011, July 12 1 / 25
2 Introduction Outline 1 Introduction 2 Storage System Options in the Amazon EC2 CCI 3 Preliminary Applications Results 4 Conclusion 2 / 25
3 Introduction Background I/O becomes the bottleneck for many HPC applications Intensive I/O operations and concurrency One-size-fits-all I/O configuration There is a trend to migrate the HPC applications from traditional platforms to cloud Cloud provides tremendous flexibility in configuring I/O system Fully controlled virtual machines Easily deployed user scripts Multiple types of low-level devices Online device acquisition and migration 3 / 25
4 Introduction Motivation The Problem Can we employ I/O configurability of cloud for HPC apps? 4 / 25
5 Introduction The Problem Motivation Can we employ I/O configurability of cloud for HPC apps? Configurability lies in: Set up the specific file system at start up Explore different types of low-level devices Tune the file system inherent parameters 4 / 25
6 Introduction The Problem Motivation Can we employ I/O configurability of cloud for HPC apps? Configurability lies in: Set up the specific file system at start up Explore different types of low-level devices Tune the file system inherent parameters Challenges: The feasibility is highly workload-dependent Tradeoff between efficiency vs. cost-effectiveness 4 / 25
7 Introduction Amazon EC2 CCI CCI: Cluster Computing Instance Amazon s solution to HPC in Cloud Quad-core Intel Xeon X5570 CPU, with 23GB memory Interconnected by 10 Gigabit Ethernet Amazon Linux AMI, RedHat family OS with Intel MPI local block storage (Ephemeral) with 2*800 GB Elastic Block Store (EBS), attached as block storage devices Simple Storage Service (S3), key-value based object storage 5 / 25
8 Outline 1 Introduction 2 Storage System Options in the Amazon EC2 CCI 3 Preliminary Applications Results 4 Conclusion 6 / 25
9 File System Selection Different I/O access pattern and concurrency require different kinds of file system - shared vs. parallel NFS is enough for low I/O demands, simple to deploy A parallel file system (eg. PVFS, Lustre) can be employed to support large and shared file writes scale up well
10 File System Selection Different I/O access pattern and concurrency require different kinds of file system - shared vs. parallel NFS is enough for low I/O demands, simple to deploy A parallel file system (eg. PVFS, Lustre) can be employed to support large and shared file writes scale up well Easy to choose and setup 118 LOC bash for NFS, and 173 LOC bash for PVFS 7 / 25
11 Device Selection Considerations Devices differ in levels of abstraction and access interfaces. Storage Pros. Cons. S3 off-shelf, designed for Internet and database apps no POSIX Ephemeral Free non-persistent, only two disks EBS - persistent beyond instances - more disks available Charged
12 Device Selection Considerations Devices differ in levels of abstraction and access interfaces. Storage Pros. Cons. S3 off-shelf, designed for Internet and database apps no POSIX Ephemeral Free non-persistent, only two disks EBS - persistent beyond instances - more disks available Charged The choice depends on the needs of individual applications 8 / 25
13 File System Internal Parameters Options Description Sync mode NFS sync vs async write modes Device number Combining multiple disks into a software RAID0 I/O server number NFS single-server bottleneck, PVFS can employ many I/O servers Meta-data distribution PVFS can distribute metadata to multiple servers I/O Server Placement Part-time vs. dedicated I/O servers Data Striping striping factor, unit size
14 File System Internal Parameters Options Description Sync mode NFS sync vs async write modes Device number Combining multiple disks into a software RAID0 I/O server number NFS single-server bottleneck, PVFS can employ many I/O servers Meta-data distribution PVFS can distribute metadata to multiple servers I/O Server Placement Part-time vs. dedicated I/O servers Data Striping striping factor, unit size Again The choice depends on the needs of individual applications 9 / 25
15 Dedicated NFS, Ephemeral Disks Computing Intances... 10GB Ethernet NFS Server Ephemeral Disks There is 1 NFS I/O server deployed in one dedicated instance, mounting two ephemeral disks. 10 / 25
16 Parttime NFS Mounting Ephemeral Computing Intances... Ephemeral Disks 10GB Ethernet NFS Server There is 1 NFS I/O server deployed in one parttime instance, mounting two ephemeral disks. 11 / 25
17 Dedicated NFS Mounting EBS Disks Computing Intances... 10GB Ethernet NFS Server EBS Disks There is 1 NFS I/O server deployed in one dedicated instance, mounting 8 EBS disks into RAID0. 12 / 25
18 1 Dedicated PVFS I/O server Computing Intances... 10GB Ethernet PVFS Server Ephemeral Disks There is 1 PVFS I/O server deployed in one dedicated instance. 13 / 25
19 2 Dedicated PVFS I/O Servers Computing Intances... 10GB Ethernet PVFS Server PVFS Server There are 2 PVFS I/O servers deployed in two dedicated instances. 14 / 25
20 4 Dedicated PVFS I/O Servers Computing Intances... 10GB Ethernet PVFS Server PVFS Server PVFS Server PVFS Server There are 4 PVFS I/O servers deployed in the dedicated instances, low untilized. 15 / 25
21 4 Part-time PVFS I/O servers Ephemeral Disks Computing Intances... PVFS PVFS PVFS PVFS 10GB Ethernet There are 4 PVFS I/O servers deployed in the computing instances, working part-time. 16 / 25
22 Options: sync mode and device W rite B a n d w id th (M B /s ) E B S -s y n c E p h e m e ra l-s y n c E B S -a s y n c E p h e m e ra l-a s y n c M M M 1 G 2 G 4 G B lo c k S iz e (B y te ) Test by IOR benchmark in 16 processes at 2 nodes. 17 / 25
23 Options: sync mode and device W rite B a n d w id th (M B /s ) E B S -s y n c E p h e m e ra l-s y n c E B S -a s y n c E p h e m e ra l-a s y n c M M M 1 G 2 G 4 G B lo c k S iz e (B y te ) Test by IOR benchmark in 16 processes at 2 nodes. Observations: No obvious difference between EBS and ephemeral 17 / 25
24 Options: sync mode and device W rite B a n d w id th (M B /s ) E B S -s y n c E p h e m e ra l-s y n c E B S -a s y n c E p h e m e ra l-a s y n c M M M 1 G 2 G 4 G B lo c k S iz e (B y te ) Test by IOR benchmark in 16 processes at 2 nodes. Observations: No obvious difference between EBS and ephemeral Async mode prefers small block sizes 17 / 25
25 NFS Write Bandwidth d is k 2 -d is k s 4 -d is k s 8 -d is k s W rite B a n d w id th (M B /s ) N u m o f P ro c e s s e s Combining 1,2,4,8 EBS disks into a software RAID0 18 / 25
26 NFS Write Bandwidth d is k 2 -d is k s 4 -d is k s 8 -d is k s W rite B a n d w id th (M B /s ) N u m o f P ro c e s s e s Combining 1,2,4,8 EBS disks into a software RAID0 The RAID0 doesn t scale well - possibly because of the virtualized layer 18 / 25
27 Preliminary Applications Results Outline 1 Introduction 2 Storage System Options in the Amazon EC2 CCI 3 Preliminary Applications Results 4 Conclusion 19 / 25
28 Preliminary Applications Results BTIO: CLASS=C, SUBTYPE=full T o ta l E x e c u tio n T im e (s ) N u m o f P ro c e s s e s N F S -D e d ic a te d N F S -P a rttim e P V F S -1 -S e rv e r P V F S -2 -S e rv e r P V F S -4 -S e rv e r P V F S -4 -S e rv e r-p a rt 20 / 25
29 Preliminary Applications Results BTIO: CLASS=C, SUBTYPE=full T o ta l E x e c u tio n T im e (s ) N u m o f P ro c e s s e s N F S -D e d ic a te d N F S -P a rttim e P V F S -1 -S e rv e r P V F S -2 -S e rv e r P V F S -4 -S e rv e r P V F S -4 -S e rv e r-p a rt PVFS outperforms NFS configurations all the time 20 / 25
30 Preliminary Applications Results BTIO: CLASS=C, SUBTYPE=full T o ta l E x e c u tio n T im e (s ) N u m o f P ro c e s s e s N F S -D e d ic a te d N F S -P a rttim e P V F S -1 -S e rv e r P V F S -2 -S e rv e r P V F S -4 -S e rv e r P V F S -4 -S e rv e r-p a rt PVFS outperforms NFS configurations all the time PVFS scales up by adding more I/O servers 20 / 25
31 Preliminary Applications Results BTIO: CLASS=C, SUBTYPE=full T o ta l E x e c u tio n T im e (s ) N u m o f P ro c e s s e s N F S -D e d ic a te d N F S -P a rttim e P V F S -1 -S e rv e r P V F S -2 -S e rv e r P V F S -4 -S e rv e r P V F S -4 -S e rv e r-p a rt PVFS outperforms NFS configurations all the time PVFS scales up by adding more I/O servers Parttime I/O servers provide pretty good performance 20 / 25
32 Preliminary Applications Results BTIO: Total Cost Analysis Cost calculation Cost($) = Num computing instances Run time 1.6/3600 T o ta l C o s t ($ ) N F S -D e d ic a te d N F S -P a rttim e P V F S -1 -S e rv e r P V F S -2 -S e rv e r P V F S -4 -S e rv e r P V F S -4 -S e rv e r-p a rt N u m o f P ro c e s s e s 21 / 25
33 Preliminary Applications Results POP: Parallel Ocean Program Process 0 carries out all I/O tasks via POSIX interface, which is very different from BTIO POP does not scale on EC2, due to its heavy communication with small messages T o ta l E x e c u tio n T im e (s ) N u m o f P ro c e s s e s N F S -p a rttim e P V F S -2 -S e rv e r P V F S -4 -S e rv e r P V F S -4 -S e rv e r-p a rt 22 / 25
34 Conclusion Outline 1 Introduction 2 Storage System Options in the Amazon EC2 CCI 3 Preliminary Applications Results 4 Conclusion 23 / 25
35 Conclusion Future Work The Problem Configure the I/O system for a HPC app automatically HPC Apps... Analyzing IO Patterns I/O Demands Mapping Model Optimized Configurations File System Devices Writing Mode Number of Server Placement I/O Options 24 / 25
36 Conclusion Conclusion Cloud enables users to build per-application I/O systems Our preliminary results hint that HPC app behaves differently with different I/O system configurations in cloud Configuration per app depends on its I/O access pattern and concurrency Tradeoff: cost vs. efficiency 25 / 25
37 Conclusion Conclusion Cloud enables users to build per-application I/O systems Our preliminary results hint that HPC app behaves differently with different I/O system configurations in cloud Configuration per app depends on its I/O access pattern and concurrency Tradeoff: cost vs. efficiency Thank you! 25 / 25
RAID+: Deterministic and Balanced Data Distribution for Large Disk Enclosures
RAID+: Deterministic and Balanced Data Distribution for Large Disk Enclosures Guangyan Zhang, Zican Huang, Xiaosong Ma SonglinYang, Zhufan Wang, Weimin Zheng Tsinghua University Qatar Computing Research
More informationArcGIS Deployment Pattern. Azlina Mahad
ArcGIS Deployment Pattern Azlina Mahad Agenda Deployment Options Cloud Portal ArcGIS Server Data Publication Mobile System Management Desktop Web Device ArcGIS An Integrated Web GIS Platform Portal Providing
More informationSoftware Architecture. CSC 440: Software Engineering Slide #1
Software Architecture CSC 440: Software Engineering Slide #1 Topics 1. What is software architecture? 2. Why do we need software architecture? 3. Architectural principles 4. UML package diagrams 5. Software
More informationNEC PerforCache. Influence on M-Series Disk Array Behavior and Performance. Version 1.0
NEC PerforCache Influence on M-Series Disk Array Behavior and Performance. Version 1.0 Preface This document describes L2 (Level 2) Cache Technology which is a feature of NEC M-Series Disk Array implemented
More informationInformation System Desig
n IT60105 Lecture 7 Unified Modeling Language Lecture #07 Unified Modeling Language Introduction to UML Applications of UML UML Definition Learning UML Things in UML Structural Things Behavioral Things
More informationScalable and Power-Efficient Data Mining Kernels
Scalable and Power-Efficient Data Mining Kernels Alok Choudhary, John G. Searle Professor Dept. of Electrical Engineering and Computer Science and Professor, Kellogg School of Management Director of the
More informationWeather Research and Forecasting (WRF) Performance Benchmark and Profiling. July 2012
Weather Research and Forecasting (WRF) Performance Benchmark and Profiling July 2012 Note The following research was performed under the HPC Advisory Council activities Participating vendors: Intel, Dell,
More informationStochastic Modelling of Electron Transport on different HPC architectures
Stochastic Modelling of Electron Transport on different HPC architectures www.hp-see.eu E. Atanassov, T. Gurov, A. Karaivan ova Institute of Information and Communication Technologies Bulgarian Academy
More informationWRF performance tuning for the Intel Woodcrest Processor
WRF performance tuning for the Intel Woodcrest Processor A. Semenov, T. Kashevarova, P. Mankevich, D. Shkurko, K. Arturov, N. Panov Intel Corp., pr. ak. Lavrentieva 6/1, Novosibirsk, Russia, 630090 {alexander.l.semenov,tamara.p.kashevarova,pavel.v.mankevich,
More informationVMware VMmark V1.1 Results
Vendor and Hardware Platform: IBM System x3950 M2 Virtualization Platform: VMware ESX 3.5.0 U2 Build 110181 Performance VMware VMmark V1.1 Results Tested By: IBM Inc., RTP, NC Test Date: 2008-09-20 Performance
More informationA Data Communication Reliability and Trustability Study for Cluster Computing
A Data Communication Reliability and Trustability Study for Cluster Computing Speaker: Eduardo Colmenares Midwestern State University Wichita Falls, TX HPC Introduction Relevant to a variety of sciences,
More informationGeog 469 GIS Workshop. Managing Enterprise GIS Geodatabases
Geog 469 GIS Workshop Managing Enterprise GIS Geodatabases Outline 1. Why is a geodatabase important for GIS? 2. What is the architecture of a geodatabase? 3. How can we compare and contrast three types
More informationArcGIS Enterprise: What s New. Philip Heede Shannon Kalisky Melanie Summers Sam Williamson
ArcGIS Enterprise: What s New Philip Heede Shannon Kalisky Melanie Summers Sam Williamson ArcGIS Enterprise is the new name for ArcGIS for Server What is ArcGIS Enterprise ArcGIS Enterprise is powerful
More informationArcGIS Enterprise: What s New. Philip Heede Shannon Kalisky Melanie Summers Shreyas Shinde
ArcGIS Enterprise: What s New Philip Heede Shannon Kalisky Melanie Summers Shreyas Shinde ArcGIS Enterprise is the new name for ArcGIS for Server ArcGIS Enterprise Software Components ArcGIS Server Portal
More informationA Tale of Two Erasure Codes in HDFS
A Tale of Two Erasure Codes in HDFS Dynamo Mingyuan Xia *, Mohit Saxena +, Mario Blaum +, and David A. Pease + * McGill University, + IBM Research Almaden FAST 15 何军权 2015-04-30 1 Outline Introduction
More informationQuantum Chemical Calculations by Parallel Computer from Commodity PC Components
Nonlinear Analysis: Modelling and Control, 2007, Vol. 12, No. 4, 461 468 Quantum Chemical Calculations by Parallel Computer from Commodity PC Components S. Bekešienė 1, S. Sėrikovienė 2 1 Institute of
More informationEnabling ENVI. ArcGIS for Server
Enabling ENVI throughh ArcGIS for Server 1 Imagery: A Unique and Valuable Source of Data Imagery is not just a base map, but a layer of rich information that can address problems faced by GIS users. >
More informationHYCOM and Navy ESPC Future High Performance Computing Needs. Alan J. Wallcraft. COAPS Short Seminar November 6, 2017
HYCOM and Navy ESPC Future High Performance Computing Needs Alan J. Wallcraft COAPS Short Seminar November 6, 2017 Forecasting Architectural Trends 3 NAVY OPERATIONAL GLOBAL OCEAN PREDICTION Trend is higher
More informationParallelization of the Molecular Orbital Program MOS-F
Parallelization of the Molecular Orbital Program MOS-F Akira Asato, Satoshi Onodera, Yoshie Inada, Elena Akhmatskaya, Ross Nobes, Azuma Matsuura, Atsuya Takahashi November 2003 Fujitsu Laboratories of
More informationIntroduction to Portal for ArcGIS
Introduction to Portal for ArcGIS Derek Law Product Management March 10 th, 2015 Esri Developer Summit 2015 Agenda Web GIS pattern Product overview Installation and deployment Security and groups Configuration
More informationContinuous Machine Learning
Continuous Machine Learning Kostiantyn Bokhan, PhD Project Lead at Samsung R&D Ukraine Kharkiv, October 2016 Agenda ML dev. workflows ML dev. issues ML dev. solutions Continuous machine learning (CML)
More informationDevelopment of a GIS Interface for WEPP Model Application to Great Lakes Forested Watersheds
Development of a GIS Interface for WEPP Model Application to Great Lakes Forested Watersheds J.R. Frankenberger 1, S. Dun 2, D.C. Flanagan 1, J.Q. Wu 2, W.J. Elliot 3 1 USDA-ARS, West Lafayette, IN 2 Washington
More informationLeveraging Web GIS: An Introduction to the ArcGIS portal
Leveraging Web GIS: An Introduction to the ArcGIS portal Derek Law Product Management DLaw@esri.com Agenda Web GIS pattern Product overview Installation and deployment Configuration options Security options
More informationCluster Computing: Updraft. Charles Reid Scientific Computing Summer Workshop June 29, 2010
Cluster Computing: Updraft Charles Reid Scientific Computing Summer Workshop June 29, 2010 Updraft Cluster: Hardware 256 Dual Quad-Core Nodes 2048 Cores 2.8 GHz Intel Xeon Processors 16 GB memory per
More informationArcGIS GeoAnalytics Server: An Introduction. Sarah Ambrose and Ravi Narayanan
ArcGIS GeoAnalytics Server: An Introduction Sarah Ambrose and Ravi Narayanan Overview Introduction Demos Analysis Concepts using GeoAnalytics Server GeoAnalytics Data Sources GeoAnalytics Server Administration
More informationClass Diagrams. CSC 440/540: Software Engineering Slide #1
Class Diagrams CSC 440/540: Software Engineering Slide # Topics. Design class diagrams (DCDs) 2. DCD development process 3. Associations and Attributes 4. Dependencies 5. Composition and Constraints 6.
More informationIntroduction to Portal for ArcGIS. Hao LEE November 12, 2015
Introduction to Portal for ArcGIS Hao LEE November 12, 2015 Agenda Web GIS pattern Product overview Installation and deployment Security and groups Configuration options Portal for ArcGIS + ArcGIS for
More informationEsri Overview for Mentor Protégé Program:
Agenda Passionate About Helping You Succeed Esri Overview for Mentor Protégé Program: Northrop Grumman CSSS Jeff Dawley 3 September 2010 Esri Overview ArcGIS as a System ArcGIS 10 - Map Production - Mobile
More informationAnalysis of Software Artifacts
Analysis of Software Artifacts System Performance I Shu-Ngai Yeung (with edits by Jeannette Wing) Department of Statistics Carnegie Mellon University Pittsburgh, PA 15213 2001 by Carnegie Mellon University
More informationSome thoughts about energy efficient application execution on NEC LX Series compute clusters
Some thoughts about energy efficient application execution on NEC LX Series compute clusters G. Wellein, G. Hager, J. Treibig, M. Wittmann Erlangen Regional Computing Center & Department of Computer Science
More informationELF products in the ArcGIS platform
ELF products in the ArcGIS platform Presentation to: Author: Date: NMO Summit 2016, Dublin, Ireland Clemens Portele 18 May 2016 The Building Blocks 18 May, 2016 More ELF users through affiliated platforms
More informationDr. Andrea Bocci. Using GPUs to Accelerate Online Event Reconstruction. at the Large Hadron Collider. Applied Physicist
Using GPUs to Accelerate Online Event Reconstruction at the Large Hadron Collider Dr. Andrea Bocci Applied Physicist On behalf of the CMS Collaboration Discover CERN Inside the Large Hadron Collider at
More informationPortal for ArcGIS: An Introduction. Catherine Hynes and Derek Law
Portal for ArcGIS: An Introduction Catherine Hynes and Derek Law Agenda Web GIS pattern Product overview Installation and deployment Configuration options Security options and groups Portal for ArcGIS
More informationQuantum ESPRESSO Performance Benchmark and Profiling. February 2017
Quantum ESPRESSO Performance Benchmark and Profiling February 2017 2 Note The following research was performed under the HPC Advisory Council activities Compute resource - HPC Advisory Council Cluster
More informationHigh-Performance Scientific Computing
High-Performance Scientific Computing Instructor: Randy LeVeque TA: Grady Lemoine Applied Mathematics 483/583, Spring 2011 http://www.amath.washington.edu/~rjl/am583 World s fastest computers http://top500.org
More informationAdvanced Computing Systems for Scientific Research
Undergraduate Review Volume 10 Article 13 2014 Advanced Computing Systems for Scientific Research Jared Buckley Jason Covert Talia Martin Recommended Citation Buckley, Jared; Covert, Jason; and Martin,
More informationThe File Geodatabase API. Craig Gillgrass Lance Shipman
The File Geodatabase API Craig Gillgrass Lance Shipman Schedule Cell phones and pagers Please complete the session survey we take your feedback very seriously! Overview File Geodatabase API - Introduction
More informationEssentials of Large Volume Data Management - from Practical Experience. George Purvis MASS Data Manager Met Office
Essentials of Large Volume Data Management - from Practical Experience George Purvis MASS Data Manager Met Office There lies trouble ahead Once upon a time a Project Manager was tasked to go forth and
More informationAn Overview of HPC at the Met Office
An Overview of HPC at the Met Office Paul Selwood Crown copyright 2006 Page 1 Introduction The Met Office National Weather Service for the UK Climate Prediction (Hadley Centre) Operational and Research
More informationTelecommunication Services Engineering (TSE) Lab. Chapter IX Presence Applications and Services.
Chapter IX Presence Applications and Services http://users.encs.concordia.ca/~glitho/ Outline 1. Basics 2. Interoperability 3. Presence service in clouds Basics 1 - IETF abstract model 2 - An example of
More informationA Spatial Data Infrastructure for Landslides and Floods in Italy
V Convegno Nazionale del Gruppo GIT Grottaminarda 14 16 giugno 2010 A Spatial Data Infrastructure for Landslides and Floods in Italy Ivan Marchesini, Vinicio Balducci, Gabriele Tonelli, Mauro Rossi, Fausto
More informationReliability at Scale
Reliability at Scale Intelligent Storage Workshop 5 James Nunez Los Alamos National lab LA-UR-07-0828 & LA-UR-06-0397 May 15, 2007 A Word about scale Petaflop class machines LLNL Blue Gene 350 Tflops 128k
More informationPerformance Analysis of Lattice QCD Application with APGAS Programming Model
Performance Analysis of Lattice QCD Application with APGAS Programming Model Koichi Shirahata 1, Jun Doi 2, Mikio Takeuchi 2 1: Tokyo Institute of Technology 2: IBM Research - Tokyo Programming Models
More informationSTATISTICAL PERFORMANCE
STATISTICAL PERFORMANCE PROVISIONING AND ENERGY EFFICIENCY IN DISTRIBUTED COMPUTING SYSTEMS Nikzad Babaii Rizvandi 1 Supervisors: Prof.Albert Y.Zomaya Prof. Aruna Seneviratne OUTLINE Introduction Background
More informationDirect Self-Consistent Field Computations on GPU Clusters
Direct Self-Consistent Field Computations on GPU Clusters Guochun Shi, Volodymyr Kindratenko National Center for Supercomputing Applications University of Illinois at UrbanaChampaign Ivan Ufimtsev, Todd
More informationRevenue Maximization in a Cloud Federation
Revenue Maximization in a Cloud Federation Makhlouf Hadji and Djamal Zeghlache September 14th, 2015 IRT SystemX/ Telecom SudParis Makhlouf Hadji Outline of the presentation 01 Introduction 02 03 04 05
More informationVisualizing Big Data on Maps: Emerging Tools and Techniques. Ilir Bejleri, Sanjay Ranka
Visualizing Big Data on Maps: Emerging Tools and Techniques Ilir Bejleri, Sanjay Ranka Topics Web GIS Visualization Big Data GIS Performance Maps in Data Visualization Platforms Next: Web GIS Visualization
More informationEnabling Web GIS. Dal Hunter Jeff Shaner
Enabling Web GIS Dal Hunter Jeff Shaner Enabling Web GIS In Your Infrastructure Agenda Quick Overview Web GIS Deployment Server GIS Deployment Security and Identity Management Web GIS Operations Web GIS
More informationPortal for ArcGIS: An Introduction
Portal for ArcGIS: An Introduction Derek Law Esri Product Management Esri UC 2014 Technical Workshop Agenda Web GIS pattern Product overview Installation and deployment Security and groups Configuration
More informationVirtual Machine Initiated Operations Logic for Resource Management. Master thesis. Nii Apleh Lartey. Oslo University College
UNIVERSITY OF OSLO Department of Informatics Virtual Machine Initiated Operations Logic for Resource Management Oslo University College Master thesis Nii Apleh Lartey May 18, 2009 Virtual Machine Initiated
More informationIntroduction to Google Drive Objectives:
Introduction to Google Drive Objectives: Learn how to access your Google Drive account Learn to create new documents using Google Drive Upload files to store on Google Drive Share files and folders with
More informationPI SERVER 2012 Do. More. Faster. Now! Copyr i g h t 2012 O S Is o f t, L L C. 1
PI SERVER 2012 Do. More. Faster. Now! Copyr i g h t 2012 O S Is o f t, L L C. 1 AUGUST 7, 2007 APRIL 14, 2010 APRIL 24, 2012 Copyr i g h t 2012 O S Is o f t, L L C. 2 PI Data Archive Security PI Asset
More informationETH Beowulf day January 31, Adrian Biland, Zhiling Chen, Derek Feichtinger, Christoph Grab, André Holzner, Urs Langenegger
CMS SM Meeting Nov 28 2005 Analysis and simulation of proton-proton collision data at LHC ETH Beowulf day January 31, 2006 Adrian Biland, Zhiling Chen, Derek Feichtinger, Christoph Grab, André Holzner,
More informationTowards Better Understanding of Black-box Auto-Tuning: A Comparative Analysis for Storage Systems
Towards Better Understanding of Black-box Auto-Tuning: A Comparative Analysis for Storage Systems 2018 USENIX Annual Technical Conference Zhen Cao 1, Vasily Tarasov 2, Sachin Tiwari 1, and Erez Zadok 1
More informationMulti-Approximate-Keyword Routing Query
Bin Yao 1, Mingwang Tang 2, Feifei Li 2 1 Department of Computer Science and Engineering Shanghai Jiao Tong University, P. R. China 2 School of Computing University of Utah, USA Outline 1 Introduction
More informationEnsemble Consistency Testing for CESM: A new form of Quality Assurance
Ensemble Consistency Testing for CESM: A new form of Quality Assurance Dorit Hammerling Institute for Mathematics Applied to Geosciences National Center for Atmospheric Research (NCAR) Joint work with
More informationThe Green Index (TGI): A Metric for Evalua:ng Energy Efficiency in HPC Systems
The Green Index (TGI): A Metric for Evalua:ng Energy Efficiency in HPC Systems Wu Feng and Balaji Subramaniam Metrics for Energy Efficiency Energy- Delay Product (EDP) Used primarily in circuit design
More informationHellenic National Meteorological Service (HNMS) GREECE
WWW TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA- PROCESSING AND FORECASTING SYSTEM (GDPFS), AND THE ANNUAL NUMERICAL WEATHER PREDICTION (NWP) PROGRESS REPORT FOR THE YEAR 2005 Hellenic National Meteorological
More informationDesign and implementation of a new meteorology geographic information system
Design and implementation of a new meteorology geographic information system WeiJiang Zheng, Bing. Luo, Zhengguang. Hu, Zhongliang. Lv National Meteorological Center, China Meteorological Administration,
More informationMSC HPC Infrastructure Update. Alain St-Denis Canadian Meteorological Centre Meteorological Service of Canada
MSC HPC Infrastructure Update Alain St-Denis Canadian Meteorological Centre Meteorological Service of Canada Outline HPC Infrastructure Overview Supercomputer Configuration Scientific Direction 2 IT Infrastructure
More informationRed Sky. Pushing Toward Petascale with Commodity Systems. Matthew Bohnsack. Sandia National Laboratories Albuquerque, New Mexico USA
Red Sky Pushing Toward Petascale with Commodity Systems Matthew Bohnsack Sandia National Laboratories Albuquerque, New Mexico USA mpbohns@sandia.gov Tuesday March 9, 2010 Matthew Bohnsack (Sandia Nat l
More informationThe Geo Web: Enabling GIS on the Internet IT4GIS Keith T. Weber, GISP GIS Director ISU GIS Training and Research Center.
The Geo Web: Enabling GIS on the Internet IT4GIS Keith T. Weber, GISP GIS Director ISU GIS Training and Research Center In the Beginning GIS was independent The GIS analyst or manager was typically a oneperson
More information2.6 Complexity Theory for Map-Reduce. Star Joins 2.6. COMPLEXITY THEORY FOR MAP-REDUCE 51
2.6. COMPLEXITY THEORY FOR MAP-REDUCE 51 Star Joins A common structure for data mining of commercial data is the star join. For example, a chain store like Walmart keeps a fact table whose tuples each
More informationI N T R O D U C T I O N : G R O W I N G I T C O M P L E X I T Y
Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com W H I T E P A P E R I n v a r i a n t A n a l y z e r : A n A u t o m a t e d A p p r o a c h t o
More informationCollaborative WRF-based research and education enabled by software containers
Collaborative WRF-based research and education enabled by software containers J. Hacker, J. Exby, K. Fossell National Center for Atmospheric Research Contributions from Tim See (U. North Dakota) 1 Why
More informationParallel Transposition of Sparse Data Structures
Parallel Transposition of Sparse Data Structures Hao Wang, Weifeng Liu, Kaixi Hou, Wu-chun Feng Department of Computer Science, Virginia Tech Niels Bohr Institute, University of Copenhagen Scientific Computing
More informationIvana Zinno, Francesco Casu, Claudio De Luca, Riccardo Lanari, Michele Manunta. CNR IREA, Napoli, Italy
An Unsupervised Implementation of the P-SBAS DiNSAR Algorithm for Processing Large Data Volumes through Distributed Computing Infrastructures within Operational Environments Ivana Zinno, Francesco Casu,
More informationHow to deal with uncertainties and dynamicity?
How to deal with uncertainties and dynamicity? http://graal.ens-lyon.fr/ lmarchal/scheduling/ 19 novembre 2012 1/ 37 Outline 1 Sensitivity and Robustness 2 Analyzing the sensitivity : the case of Backfilling
More informationMarla Meehl Manager of NCAR/UCAR Networking and Front Range GigaPoP (FRGP)
Big Data at the National Center for Atmospheric Research (NCAR) & expanding network bandwidth to NCAR over Pacific Wave and Western Regional Network (WRN) Marla Meehl Manager of NCAR/UCAR Networking and
More informationSpyMeSat Mobile App. Imaging Satellite Awareness & Access
SpyMeSat Mobile App Imaging Satellite Awareness & Access Imaging & Geospatical Technology Forum ASPRS Annual Conference March 12-16, 2017 Baltimore, MD Ella C. Herz 1 Orbit Logic specializes in software
More informationGPS Mapping with Esri s Collector App. What We ll Cover
GPS Mapping with Esri s Collector App Part 1: Overview What We ll Cover Part 1: Overview and requirements Part 2: Preparing the data in ArcGIS for Desktop Part 3: Build a web map in ArcGIS Online Part
More informationTime Series Analysis with SAR & Optical Satellite Data
Time Series Analysis with SAR & Optical Satellite Data Thomas Bahr ESRI European User Conference Thursday October 2015 harris.com Motivation Changes in land surface characteristics mirror a multitude of
More informationProgress in NWP on Intel HPC architecture at Australian Bureau of Meteorology
Progress in NWP on Intel HPC architecture at Australian Bureau of Meteorology www.cawcr.gov.au Robin Bowen Senior ITO Earth System Modelling Programme 04 October 2012 ECMWF HPC Presentation outline Weather
More informationOverview of xcode Overview of xcode
Ilya Agapov Original motivation: spontaneous radiation Need for diagnostics, power loads and potentially science cases Numerical methods well understood, single particle solver provided by O. Chubar (SRWlib)
More informationMigrating Defense Workflows from ArcMap to ArcGIS Pro. Renee Bernstein and Jared Sellers
Migrating Defense Workflows from ArcMap to ArcGIS Pro Renee Bernstein and Jared Sellers ArcGIS Desktop Desktop Web Device ArcMap ArcCatalog ArcScene ArcGlobe ArcGIS Pro portal Server Online Content and
More informationTowards VM Consolidation Using Idle States
Towards Consolidation Using Idle States Rayman Preet Singh, Tim Brecht, S. Keshav University of Waterloo @AC EE 15 1 Traditional Consolidation Hypervisor Hardware Hypervisor Hardware Hypervisor Hardware
More informationMPI at MPI. Jens Saak. Max Planck Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory
MAX PLANCK INSTITUTE November 5, 2010 MPI at MPI Jens Saak Max Planck Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory FOR DYNAMICS OF COMPLEX TECHNICAL
More informationPeriodic I/O Scheduling for Supercomputers
Periodic I/O Scheduling for Supercomputers Guillaume Aupy 1, Ana Gainaru 2, Valentin Le Fèvre 3 1 Inria & U. of Bordeaux 2 Vanderbilt University 3 ENS Lyon & Inria PMBS Workshop, November 217 Slides available
More informationCactus Tools for Petascale Computing
Cactus Tools for Petascale Computing Erik Schnetter Reno, November 2007 Gamma Ray Bursts ~10 7 km He Protoneutron Star Accretion Collapse to a Black Hole Jet Formation and Sustainment Fe-group nuclei Si
More informationComputationally Efficient Analysis of Large Array FTIR Data In Chemical Reaction Studies Using Distributed Computing Strategy
575f Computationally Efficient Analysis of Large Array FTIR Data In Chemical Reaction Studies Using Distributed Computing Strategy Ms Suyun Ong, Dr. Wee Chew, * Dr. Marc Garland Institute of Chemical and
More informationarxiv:astro-ph/ v1 15 Sep 1999
Baltic Astronomy, vol.8, XXX XXX, 1999. THE ASTEROSEISMOLOGY METACOMPUTER arxiv:astro-ph/9909264v1 15 Sep 1999 T.S. Metcalfe and R.E. Nather Department of Astronomy, University of Texas, Austin, TX 78701
More informationMapReduce in Spark. Krzysztof Dembczyński. Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland
MapReduce in Spark Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, second semester
More informationIntegrated Electricity Demand and Price Forecasting
Integrated Electricity Demand and Price Forecasting Create and Evaluate Forecasting Models The many interrelated factors which influence demand for electricity cannot be directly modeled by closed-form
More informationGeospatial Fire Behavior Modeling App to Manage Wildfire Risk Online. Kenyatta BaRaKa Jackson US Forest Service - Consultant
Geospatial Fire Behavior Modeling App to Manage Wildfire Risk Online Kenyatta BaRaKa Jackson US Forest Service - Consultant Fire Behavior Modeling and Forest Fuel Management Modeling Fire Behavior is an
More informationSchwarz-type methods and their application in geomechanics
Schwarz-type methods and their application in geomechanics R. Blaheta, O. Jakl, K. Krečmer, J. Starý Institute of Geonics AS CR, Ostrava, Czech Republic E-mail: stary@ugn.cas.cz PDEMAMIP, September 7-11,
More informationHistory of the partnership between SMHI and NSC. Per Undén
History of the partnership between SMHI and NSC Per Undén Outline Pre-history and NWP Preparations parallelisation HPD Council Decision and early developments Climate modelling Other applications HPD Project
More informationxlogic SuperRelay is a compact and expandable CPU replacing mini PLCs, multiple timers, relays and counters.
SuperRelay S U P E R R E L A Y, T H E P E R F E C T A L T E R N A T I V E T O L O W C O S T P L C s A N D B A S I C R E L A Y S I N T H I S B R O C H U R E : ELC 18 Standard Models ELC 18 Economy Models
More informationRoad Ahead: Linear Referencing and UPDM
Road Ahead: Linear Referencing and UPDM Esri European Petroleum GIS Conference November 7, 2014 Congress Centre, London Your Work Making a Difference ArcGIS Is Evolving Your GIS Is Becoming Part of an
More informationOptimization strategy for MASNUM surface wave model
Hillsboro, September 27, 2018 Optimization strategy for MASNUM surface wave model Zhenya Song *, + * First Institute of Oceanography (FIO), State Oceanic Administrative (SOA), China + Intel Parallel Computing
More informationWeb GIS & ArcGIS Pro. Zena Pelletier Nick Popovich
Web GIS & ArcGIS Pro Zena Pelletier Nick Popovich Web GIS Transformation of the ArcGIS Platform Desktop Apps GIS Web Maps Web Scenes Layers Evolution of the modern GIS Desktop GIS (standalone GIS) GIS
More informationCOMPUTER SCIENCE TRIPOS
CST.2016.2.1 COMPUTER SCIENCE TRIPOS Part IA Tuesday 31 May 2016 1.30 to 4.30 COMPUTER SCIENCE Paper 2 Answer one question from each of Sections A, B and C, and two questions from Section D. Submit the
More informationIMS4 ARWIS. Airport Runway Weather Information System. Real-time data, forecasts and early warnings
Airport Runway Weather Information System Real-time data, forecasts and early warnings Airport Runway Weather Information System FEATURES: Detection and prediction of runway conditions Alarms on hazardous
More informationAdministrivia. Course Objectives. Overview. Lecture Notes Week markem/cs333/ 2. Staff. 3. Prerequisites. 4. Grading. 1. Theory and application
Administrivia 1. markem/cs333/ 2. Staff 3. Prerequisites 4. Grading Course Objectives 1. Theory and application 2. Benefits 3. Labs TAs Overview 1. What is a computer system? CPU PC ALU System bus Memory
More informationSheffield Computing. Matt Robinson Elena Korolkova Paul Hodgson
Sheffield Computing Matt Robinson Elena Korolkova Paul Hodgson Tier-3 Zero gridpp funding has gone into the Tier-3 cluster. It is entirely funded by the PPPA groups at Sheffield. Currently stands at 130
More informationThe integration of land change modeling framework FUTURES into GRASS GIS 7
The integration of land change modeling framework FUTURES into GRASS GIS 7 Anna Petrasova, Vaclav Petras, Douglas A. Shoemaker, Monica A. Dorning, Ross K. Meentemeyer NCSU OSGeo Research and Education
More informationCurrent Status of Chinese Virtual Observatory
Current Status of Chinese Virtual Observatory Chenzhou Cui, Yongheng Zhao National Astronomical Observatories, Chinese Academy of Science, Beijing 100012, P. R. China Dec. 30, 2002 General Information
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Map-Reduce Denis Helic KTI, TU Graz Oct 24, 2013 Denis Helic (KTI, TU Graz) KDDM1 Oct 24, 2013 1 / 82 Big picture: KDDM Probability Theory Linear Algebra
More informationArcGIS Runtime: Migrating from ArcGIS Engine. Rex Hansen
ArcGIS Runtime: Migrating from ArcGIS Engine Rex Hansen Thank You to Our Sponsors Migrating from ArcGIS Engine to ArcGIS Runtime ArcGIS Runtime API: new and evolved workflows on all platforms Windows Linux
More informationWhat Would John Snow Do (Today)? Part 1
What Would John Snow Do (Today)? Part 1 Tanya Bigos and Derek Law @Tanyabigos @GIS_Bandit Thurs Oct 19 th, 2017 Outline Overview of the ArcGIS Platform Whiteboard discussion Summary Questions A Whole New
More informationParallel PIPS-SBB Multi-level parallelism for 2-stage SMIPS. Lluís-Miquel Munguia, Geoffrey M. Oxberry, Deepak Rajan, Yuji Shinano
Parallel PIPS-SBB Multi-level parallelism for 2-stage SMIPS Lluís-Miquel Munguia, Geoffrey M. Oxberry, Deepak Rajan, Yuji Shinano ... Our contribution PIPS-PSBB*: Multi-level parallelism for Stochastic
More information