Towards Better Understanding of Black-box Auto-Tuning: A Comparative Analysis for Storage Systems

Size: px
Start display at page:

Download "Towards Better Understanding of Black-box Auto-Tuning: A Comparative Analysis for Storage Systems"

Transcription

1 Towards Better Understanding of Black-box Auto-Tuning: A Comparative Analysis for Storage Systems 2018 USENIX Annual Technical Conference Zhen Cao 1, Vasily Tarasov 2, Sachin Tiwari 1, and Erez Zadok 1 1 Stony Brook University; 2 IBM Research Almaden; Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 1

2 Outline l Introduction l Background l Experiment Settings l Evaluations l Related Work l Conclusions & Future Work Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 2

3 Motivation l Why tuning storage systems? uslow storage impacts I/O bound workloads udefault settings are sub-optimal utuning can provide significant gains 9 [FAST 10] l Manual tuning is intractable l Auto-tuning storage systems ublack-box optimization is promising ulack of comparison of techniques ulack of understanding Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 3

4 Contributions l First comparative study on auto-tuning storage systems u 5 techniques l Various aspects u Cumulative & instantaneous throughput u Impacts of hyper-parameters l Explanations on evaluation results u From storage perspective l Future Goal: complete solution to tune storage systems Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 4

5 Outline l Introduction l Background l Experiment Settings l Evaluations l Related Work l Conclusions & Future Work Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 5

6 Concepts l Storage system u File system, underlying storage hardware and any layers between them l Parameters u Configurable options u E.g., file-system block size l Parameter values u E.g., 1K, 2K, 4K (Ext4 block size) l Configuration u Combination of parameter values u E.g., [Ext4, 4K, data=ordered] l Parameter space u All possible configurations l Hyper-parameter Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 6

7 Challenges l Vast parameter space uext4: 59 parameters, configs udevices, Layers udistributed, large-scale l Discrete and non-numeric ulinux I/O scheduler: noop, cfq, deadline l Non-linearity l Sensitivity to environment uhardware & workloads Manual Tuning Inefficient Gradient Unavailable Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 7

8 Inapplicable Methods l Control Theory uunstable in controlling non-linear systems l Supervised Machine Learning ulong training phase uhigh-quality training data l Inapplicable or inefficient to serve as the core auto-tuning algorithm ucould be helpful as a supplement Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 8

9 Black-box Optimization l Successfully applied in auto-tuning system configurations l Examples u Genetic Algorithms (GA) u Simulated Annealing (SA) u Bayesian Optimization (BO) l Obliviousness to system s internals Configuration evaluate select Evaluation Results Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 9

10 Key Factors l Fitness: optimization objective(s) uthroughput, latency, energy, ucomplex cost functions l Exploration usearch the unvisited area (e.g., randomly) l Exploitation uutilize neighborhood or history l History uhow much past data is kept and used for exploration/exploitation Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 10

11 Applied Methods l Simulated Annealing (SA) l Genetic Algorithms (GA) l Deep Q-Network (DQN) l Bayesian Optimization (BO) l Random Search (RS) urandom selection without replacement Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 11

12 Genetic Algorithms l Inspired by natural evolution l Concepts ugene: file system, block size, uallele: Ext4, XFS, Btrfs, uchromosome: configuration upopulation: set of configurations l Selection l Genetic operators ucrossover umutation History Exploitation vs. Exploration Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 12

13 Outline l Introduction l Background l Experiment Settings l Evaluations l Related Work l Conclusions & Future Work Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 13

14 l Hardware Experimental Setup u M1: 2 Intel Xeon single-core 2.8GHz CPU, 2G RAM, 73GB Seagate SCSI drive u M2: 1 Intel Xeon quad-core 2.4GHz CPU, 24G RAM, 4 drives (SAS-HDD 500GB, SAS-HDD 146GB, 1 SATA-HDD, SSD) l Filebench u Macro-workloads: fileserver, mailserver, webserver, dbserver u Default working set size Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 14

15 Experiment Setup (cont.) l Search spaces ustorage V1 File system, inode size, block size, block group, journal options, mount options, special options ustorage V2 V1 + I/O scheduler 6,222 configurations l Methodology uexhaustive Search Storage V2: 4 workloads 4 devices 3+ runs for each configuration Collected over 2+ years usimulate auto-tuning algorithms Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 15

16 Outline l Introduction l Background l Experiment Settings l Evaluations l Related Work l Conclusions & Future Work Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 16

17 Best Throughput Best Throughput (kops/s) M2-Mailserver-HDD3 GA SA BO DQN RS Time (hrs) Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 17

18 Success rate for finding near-optimal configurations Near-optimal configuration: one with throughput higher than 99% of the global optimal value. Percentage of Runs 100% 80% 60% 40% 20% GA SA BO DQN RS M2-Fileserver-HDD Time (hrs) Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 18

19 Instant Throughput Throughput (kops/s) M2-Mailserver-HDD Time (hrs) RS SA GA DQN BO Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 19

20 Genetic Algorithm (GA) Diversity Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 20

21 Correlation Analysis l Correlation analysis u Ordinary Least Squares (OLS) u Example: block size and journal option are the most correlated Ext4 parameter (Fileserver, SSD) l Explanations on evaluation results u GA and BO can identify important parameters through history u SA keeps no history ; thus performs poorly u DQN spends too much time on exploration u Random Search Near-optimal configurations take up 4.5% of the whole search space (M2, Mailserver, HDD). Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 21

22 Outline l Introduction l Background l Experiment Settings l Evaluations l Related Work l Conclusions & Future Work Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 22

23 Related Work l Auto-tuning storage u Storage system design (bin-packing heuristics) [Alvarez et al.] u Data recovery scheduling (GA) [Keeton et al.] u HDF5 optimization (GA) [Behzad et al.] u Lustre optimization (DQN) [Li et al.] l Auto-tuning other systems u Database [Alipourfard et al.] u Cloud VMs [Aken et al.] Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 23

24 Outline l Introduction l Background l Experiment Settings l Evaluations l Related Work l Conclusions & Future Work Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 24

25 Conclusions & Contributions l First comparative analysis on 5 techniques on auto-tuning storage systems u Efficiency on finding near-optimal configurations u Instant throughput l Provide insights from storage perspective u Importance of parameters E.g., impact of mutation rates on convergence l Valuable datasets u Will release to public Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 25

26 Future Work l More complex workloads and search spaces l Hyper-parameter tuning l More sophisticated auto-tuning u E.g., penalty functions to cope with costly parameter changes Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 26

27 Towards Better Understanding of Black-box Auto-Tuning: A Comparative Analysis for Storage Systems Zhen Cao, Vasily Tarasov, Sachin Tiwari, and Erez Zadok Towards Better Understanding of Black-box Auto-Tuning (ATC 18) 27

VMware VMmark V1.1 Results

VMware VMmark V1.1 Results Vendor and Hardware Platform: IBM System x3950 M2 Virtualization Platform: VMware ESX 3.5.0 U2 Build 110181 Performance VMware VMmark V1.1 Results Tested By: IBM Inc., RTP, NC Test Date: 2008-09-20 Performance

More information

One Optimized I/O Configuration per HPC Application

One Optimized I/O Configuration per HPC Application One Optimized I/O Configuration per HPC Application Leveraging I/O Configurability of Amazon EC2 Cloud Mingliang Liu, Jidong Zhai, Yan Zhai Tsinghua University Xiaosong Ma North Carolina State University

More information

A Tale of Two Erasure Codes in HDFS

A Tale of Two Erasure Codes in HDFS A Tale of Two Erasure Codes in HDFS Dynamo Mingyuan Xia *, Mohit Saxena +, Mario Blaum +, and David A. Pease + * McGill University, + IBM Research Almaden FAST 15 何军权 2015-04-30 1 Outline Introduction

More information

NEC PerforCache. Influence on M-Series Disk Array Behavior and Performance. Version 1.0

NEC PerforCache. Influence on M-Series Disk Array Behavior and Performance. Version 1.0 NEC PerforCache Influence on M-Series Disk Array Behavior and Performance. Version 1.0 Preface This document describes L2 (Level 2) Cache Technology which is a feature of NEC M-Series Disk Array implemented

More information

Minimization of Energy Loss using Integrated Evolutionary Approaches

Minimization of Energy Loss using Integrated Evolutionary Approaches Minimization of Energy Loss using Integrated Evolutionary Approaches Attia A. El-Fergany, Member, IEEE, Mahdi El-Arini, Senior Member, IEEE Paper Number: 1569614661 Presentation's Outline Aim of this work,

More information

Weather Research and Forecasting (WRF) Performance Benchmark and Profiling. July 2012

Weather Research and Forecasting (WRF) Performance Benchmark and Profiling. July 2012 Weather Research and Forecasting (WRF) Performance Benchmark and Profiling July 2012 Note The following research was performed under the HPC Advisory Council activities Participating vendors: Intel, Dell,

More information

ArcGIS Deployment Pattern. Azlina Mahad

ArcGIS Deployment Pattern. Azlina Mahad ArcGIS Deployment Pattern Azlina Mahad Agenda Deployment Options Cloud Portal ArcGIS Server Data Publication Mobile System Management Desktop Web Device ArcGIS An Integrated Web GIS Platform Portal Providing

More information

Advanced Machine Learning Practical 4b Solution: Regression (BLR, GPR & Gradient Boosting)

Advanced Machine Learning Practical 4b Solution: Regression (BLR, GPR & Gradient Boosting) Advanced Machine Learning Practical 4b Solution: Regression (BLR, GPR & Gradient Boosting) Professor: Aude Billard Assistants: Nadia Figueroa, Ilaria Lauzana and Brice Platerrier E-mails: aude.billard@epfl.ch,

More information

Oil Field Production using Machine Learning. CS 229 Project Report

Oil Field Production using Machine Learning. CS 229 Project Report Oil Field Production using Machine Learning CS 229 Project Report Sumeet Trehan, Energy Resources Engineering, Stanford University 1 Introduction Effective management of reservoirs motivates oil and gas

More information

A heuristic algorithm for the Aircraft Landing Problem

A heuristic algorithm for the Aircraft Landing Problem 22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3 to 8 December 2017 mssanz.org.au/modsim2017 A heuristic algorithm for the Aircraft Landing Problem Amir Salehipour

More information

CPU Consolidation versus Dynamic Voltage and Frequency Scaling in a Virtualized Multi-Core Server: Which is More Effective and When

CPU Consolidation versus Dynamic Voltage and Frequency Scaling in a Virtualized Multi-Core Server: Which is More Effective and When 1 CPU Consolidation versus Dynamic Voltage and Frequency Scaling in a Virtualized Multi-Core Server: Which is More Effective and When Inkwon Hwang, Student Member and Massoud Pedram, Fellow, IEEE Abstract

More information

WRF performance tuning for the Intel Woodcrest Processor

WRF performance tuning for the Intel Woodcrest Processor WRF performance tuning for the Intel Woodcrest Processor A. Semenov, T. Kashevarova, P. Mankevich, D. Shkurko, K. Arturov, N. Panov Intel Corp., pr. ak. Lavrentieva 6/1, Novosibirsk, Russia, 630090 {alexander.l.semenov,tamara.p.kashevarova,pavel.v.mankevich,

More information

S0214 : GPU Based Stacking Sequence Generation For Composite Skins Using GA

S0214 : GPU Based Stacking Sequence Generation For Composite Skins Using GA S0214 : GPU Based Stacking Sequence Generation For Composite Skins Using GA Date: 16th May 2012 Wed, 3pm to 3.25pm(Adv. Session) Sathyanarayana K., Manish Banga, and Ravi Kumar G. V. V. Engineering Services,

More information

StreamSVM Linear SVMs and Logistic Regression When Data Does Not Fit In Memory

StreamSVM Linear SVMs and Logistic Regression When Data Does Not Fit In Memory StreamSVM Linear SVMs and Logistic Regression When Data Does Not Fit In Memory S.V. N. (vishy) Vishwanathan Purdue University and Microsoft vishy@purdue.edu October 9, 2012 S.V. N. Vishwanathan (Purdue,

More information

Free Search in Multidimensional Space

Free Search in Multidimensional Space Free Search in Multidimensional Space Technology School, Maritime and Technology Faculty Southampton Solent University Content Aims Test Problems Bump test function Michalewicz test function Norwegian

More information

Quantum Artificial Intelligence and Machine Learning: The Path to Enterprise Deployments. Randall Correll. +1 (703) Palo Alto, CA

Quantum Artificial Intelligence and Machine Learning: The Path to Enterprise Deployments. Randall Correll. +1 (703) Palo Alto, CA Quantum Artificial Intelligence and Machine : The Path to Enterprise Deployments Randall Correll randall.correll@qcware.com +1 (703) 867-2395 Palo Alto, CA 1 Bundled software and services Professional

More information

ASTRA BASED SWARM OPTIMIZATIONS OF THE BERLinPro INJECTOR

ASTRA BASED SWARM OPTIMIZATIONS OF THE BERLinPro INJECTOR ASTRA BASED SWARM OPTIMIZATIONS OF THE BERLinPro INJECTOR M. Abo-Bakr FRAAC4 Michael Abo-Bakr 2012 ICAP Warnemünde 1 Content: Introduction ERL s Introduction BERLinPro Motivation Computational Aspects

More information

A parallel metaheuristics for the single machine total weighted tardiness problem with sequence-dependent setup times

A parallel metaheuristics for the single machine total weighted tardiness problem with sequence-dependent setup times A parallel metaheuristics for the single machine total weighted tardiness problem with sequence-dependent setup times Wojciech Bożejko Wroc law University of Technology Institute of Computer Engineering,

More information

Environment (Parallelizing Query Optimization)

Environment (Parallelizing Query Optimization) Advanced d Query Optimization i i Techniques in a Parallel Computing Environment (Parallelizing Query Optimization) Wook-Shin Han*, Wooseong Kwak, Jinsoo Lee Guy M. Lohman, Volker Markl Kyungpook National

More information

Anomaly Detection and Categorization Using Unsupervised Deep Learning

Anomaly Detection and Categorization Using Unsupervised Deep Learning Anomaly Detection and Categorization Using Unsupervised S6340 Thursday 7 th April 2016 GPU Technology Conference A. Stephen McGough, Noura Al Moubayed, Jonathan Cumming, Eduardo Cabrera, Peter Matthews,

More information

Genetic Algorithm. Outline

Genetic Algorithm. Outline Genetic Algorithm 056: 166 Production Systems Shital Shah SPRING 2004 Outline Genetic Algorithm (GA) Applications Search space Step-by-step GA Mechanism Examples GA performance Other GA examples 1 Genetic

More information

Quantitative Estimation of the Performance Delay with Propagation Effects in Disk Power Savings

Quantitative Estimation of the Performance Delay with Propagation Effects in Disk Power Savings Quantitative Estimation of the Performance Delay with Propagation Effects in Disk Power Savings Feng Yan 1, Xenia Mountrouidou 1, Alma Riska 2, and Evgenia Smirni 1 1 College of William and Mary, Williamsburg,

More information

Energy-aware scheduling for GreenIT in large-scale distributed systems

Energy-aware scheduling for GreenIT in large-scale distributed systems Energy-aware scheduling for GreenIT in large-scale distributed systems 1 PASCAL BOUVRY UNIVERSITY OF LUXEMBOURG GreenIT CORE/FNR project Context and Motivation Outline 2 Problem Description Proposed Solution

More information

CONTEMPORARY ANALYTICAL ECOSYSTEM PATRICK HALL, SAS INSTITUTE

CONTEMPORARY ANALYTICAL ECOSYSTEM PATRICK HALL, SAS INSTITUTE CONTEMPORARY ANALYTICAL ECOSYSTEM PATRICK HALL, SAS INSTITUTE Copyright 2013, SAS Institute Inc. All rights reserved. Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data

More information

FPGA Implementation of a HOG-based Pedestrian Recognition System

FPGA Implementation of a HOG-based Pedestrian Recognition System MPC Workshop Karlsruhe 10/7/2009 FPGA Implementation of a HOG-based Pedestrian Recognition System Sebastian Bauer sebastian.bauer@fh-aschaffenburg.de Laboratory for Pattern Recognition and Computational

More information

Integer weight training by differential evolution algorithms

Integer weight training by differential evolution algorithms Integer weight training by differential evolution algorithms V.P. Plagianakos, D.G. Sotiropoulos, and M.N. Vrahatis University of Patras, Department of Mathematics, GR-265 00, Patras, Greece. e-mail: vpp

More information

Sparse and Robust Optimization and Applications

Sparse and Robust Optimization and Applications Sparse and and Statistical Learning Workshop Les Houches, 2013 Robust Laurent El Ghaoui with Mert Pilanci, Anh Pham EECS Dept., UC Berkeley January 7, 2013 1 / 36 Outline Sparse Sparse Sparse Probability

More information

Che-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University

Che-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University Che-Wei Chang chewei@mail.cgu.edu.tw Department of Computer Science and Information Engineering, Chang Gung University } 2017/11/15 Midterm } 2017/11/22 Final Project Announcement 2 1. Introduction 2.

More information

Reinforcement Learning

Reinforcement Learning Reinforcement Learning Model-Based Reinforcement Learning Model-based, PAC-MDP, sample complexity, exploration/exploitation, RMAX, E3, Bayes-optimal, Bayesian RL, model learning Vien Ngo MLR, University

More information

Segment-Fixed Priority Scheduling for Self-Suspending Real-Time Tasks

Segment-Fixed Priority Scheduling for Self-Suspending Real-Time Tasks Segment-Fixed Priority Scheduling for Self-Suspending Real-Time Tasks Junsung Kim, Björn Andersson, Dionisio de Niz, and Raj Rajkumar Carnegie Mellon University 2/31 Motion Planning on Self-driving Parallel

More information

Doing Right By Massive Data: How To Bring Probability Modeling To The Analysis Of Huge Datasets Without Taking Over The Datacenter

Doing Right By Massive Data: How To Bring Probability Modeling To The Analysis Of Huge Datasets Without Taking Over The Datacenter Doing Right By Massive Data: How To Bring Probability Modeling To The Analysis Of Huge Datasets Without Taking Over The Datacenter Alexander W Blocker Pavlos Protopapas Xiao-Li Meng 9 February, 2010 Outline

More information

REINFORCEMENT LEARNING

REINFORCEMENT LEARNING REINFORCEMENT LEARNING Larry Page: Where s Google going next? DeepMind's DQN playing Breakout Contents Introduction to Reinforcement Learning Deep Q-Learning INTRODUCTION TO REINFORCEMENT LEARNING Contents

More information

Multi-Approximate-Keyword Routing Query

Multi-Approximate-Keyword Routing Query Bin Yao 1, Mingwang Tang 2, Feifei Li 2 1 Department of Computer Science and Engineering Shanghai Jiao Tong University, P. R. China 2 School of Computing University of Utah, USA Outline 1 Introduction

More information

ArcGIS GeoAnalytics Server: An Introduction. Sarah Ambrose and Ravi Narayanan

ArcGIS GeoAnalytics Server: An Introduction. Sarah Ambrose and Ravi Narayanan ArcGIS GeoAnalytics Server: An Introduction Sarah Ambrose and Ravi Narayanan Overview Introduction Demos Analysis Concepts using GeoAnalytics Server GeoAnalytics Data Sources GeoAnalytics Server Administration

More information

Parameter estimation using simulated annealing for S- system models of biochemical networks. Orland Gonzalez

Parameter estimation using simulated annealing for S- system models of biochemical networks. Orland Gonzalez Parameter estimation using simulated annealing for S- system models of biochemical networks Orland Gonzalez Outline S-systems quick review Definition of the problem Simulated annealing Perturbation function

More information

Large Scale Semi-supervised Linear SVMs. University of Chicago

Large Scale Semi-supervised Linear SVMs. University of Chicago Large Scale Semi-supervised Linear SVMs Vikas Sindhwani and Sathiya Keerthi University of Chicago SIGIR 2006 Semi-supervised Learning (SSL) Motivation Setting Categorize x-billion documents into commercial/non-commercial.

More information

Deep Learning. Convolutional Neural Networks Applications

Deep Learning. Convolutional Neural Networks Applications Deep Learning Using a Convolutional Neural Network Dr. Ing. Morris Riedel Adjunct Associated Professor School of Engineering and Natural Sciences, University of Iceland Research Group Leader, Juelich Supercomputing

More information

Basics of reinforcement learning

Basics of reinforcement learning Basics of reinforcement learning Lucian Buşoniu TMLSS, 20 July 2018 Main idea of reinforcement learning (RL) Learn a sequential decision policy to optimize the cumulative performance of an unknown system

More information

Local Search and Optimization

Local Search and Optimization Local Search and Optimization Outline Local search techniques and optimization Hill-climbing Gradient methods Simulated annealing Genetic algorithms Issues with local search Local search and optimization

More information

CAEFEM v9.5 Information

CAEFEM v9.5 Information CAEFEM v9.5 Information Concurrent Analysis Corporation, 50 Via Ricardo, Thousand Oaks, CA 91320 USA Tel. (805) 375 1060, Fax (805) 375 1061 email: info@caefem.com or support@caefem.com Web: http://www.caefem.com

More information

JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2006

JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2006 JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2006 [TURKEY/Turkish State Meteorological Service] 1. Summary

More information

Recurrent Latent Variable Networks for Session-Based Recommendation

Recurrent Latent Variable Networks for Session-Based Recommendation Recurrent Latent Variable Networks for Session-Based Recommendation Panayiotis Christodoulou Cyprus University of Technology paa.christodoulou@edu.cut.ac.cy 27/8/2017 Panayiotis Christodoulou (C.U.T.)

More information

Vibro-acoustic Analysis for Noise Reduction of Electric Machines

Vibro-acoustic Analysis for Noise Reduction of Electric Machines Vibro-acoustic Analysis for Noise Reduction of Electric Machines From Flux to OptiStruct Example : Synchronous Machine Flux 2D coupling to OptiStruct Patrick LOMBARD Application Team Manager Patrick.Lombard@cedrat.com

More information

Analysis of Software Artifacts

Analysis of Software Artifacts Analysis of Software Artifacts System Performance I Shu-Ngai Yeung (with edits by Jeannette Wing) Department of Statistics Carnegie Mellon University Pittsburgh, PA 15213 2001 by Carnegie Mellon University

More information

RAID+: Deterministic and Balanced Data Distribution for Large Disk Enclosures

RAID+: Deterministic and Balanced Data Distribution for Large Disk Enclosures RAID+: Deterministic and Balanced Data Distribution for Large Disk Enclosures Guangyan Zhang, Zican Huang, Xiaosong Ma SonglinYang, Zhufan Wang, Weimin Zheng Tsinghua University Qatar Computing Research

More information

MSC HPC Infrastructure Update. Alain St-Denis Canadian Meteorological Centre Meteorological Service of Canada

MSC HPC Infrastructure Update. Alain St-Denis Canadian Meteorological Centre Meteorological Service of Canada MSC HPC Infrastructure Update Alain St-Denis Canadian Meteorological Centre Meteorological Service of Canada Outline HPC Infrastructure Overview Supercomputer Configuration Scientific Direction 2 IT Infrastructure

More information

Energy-efficient Mapping of Big Data Workflows under Deadline Constraints

Energy-efficient Mapping of Big Data Workflows under Deadline Constraints Energy-efficient Mapping of Big Data Workflows under Deadline Constraints Presenter: Tong Shu Authors: Tong Shu and Prof. Chase Q. Wu Big Data Center Department of Computer Science New Jersey Institute

More information

Reducing Noisy-Neighbor Impact with a Fuzzy Affinity- Aware Scheduler

Reducing Noisy-Neighbor Impact with a Fuzzy Affinity- Aware Scheduler Reducing Noisy-Neighbor Impact with a Fuzzy Affinity- Aware Scheduler L U I S T O M Á S A N D J O H A N T O R D S S O N D E PA R T M E N T O F C O M P U T I N G S C I E N C E U M E Å U N I V E R S I T

More information

Zebo Peng Embedded Systems Laboratory IDA, Linköping University

Zebo Peng Embedded Systems Laboratory IDA, Linköping University TDTS 01 Lecture 8 Optimization Heuristics for Synthesis Zebo Peng Embedded Systems Laboratory IDA, Linköping University Lecture 8 Optimization problems Heuristic techniques Simulated annealing Genetic

More information

Stochastic Modelling of Electron Transport on different HPC architectures

Stochastic Modelling of Electron Transport on different HPC architectures Stochastic Modelling of Electron Transport on different HPC architectures www.hp-see.eu E. Atanassov, T. Gurov, A. Karaivan ova Institute of Information and Communication Technologies Bulgarian Academy

More information

Event-based formulations for the RCPSP with production and consumption of resources

Event-based formulations for the RCPSP with production and consumption of resources Event-based formulations for the RCPSP with production and consumption of resources Oumar KONE 1, Christian ARTIGUES 1, Pierre LOPEZ 1, and Marcel MONGEAU 2 May 2010 1: CNRS ; LAAS ; 7 avenue du colonel

More information

A Spatial Data Infrastructure for Landslides and Floods in Italy

A Spatial Data Infrastructure for Landslides and Floods in Italy V Convegno Nazionale del Gruppo GIT Grottaminarda 14 16 giugno 2010 A Spatial Data Infrastructure for Landslides and Floods in Italy Ivan Marchesini, Vinicio Balducci, Gabriele Tonelli, Mauro Rossi, Fausto

More information

arxiv: v2 [cs.ne] 22 Feb 2013

arxiv: v2 [cs.ne] 22 Feb 2013 Sparse Penalty in Deep Belief Networks: Using the Mixed Norm Constraint arxiv:1301.3533v2 [cs.ne] 22 Feb 2013 Xanadu C. Halkias DYNI, LSIS, Universitè du Sud, Avenue de l Université - BP20132, 83957 LA

More information

SOLVING SPARSE REPRESENTATIONS FOR OBJECT CLASSIFICATION USING QUANTUM D-WAVE 2X MACHINE

SOLVING SPARSE REPRESENTATIONS FOR OBJECT CLASSIFICATION USING QUANTUM D-WAVE 2X MACHINE SOLVING SPARSE REPRESENTATIONS FOR OBJECT CLASSIFICATION USING QUANTUM D-WAVE 2X MACHINE! Nga Nguyen, Amy Larson, Carleton Coffrin, John Perry, Gary Salazar, and Garrett Kenyon Los Alamos National Laboratory

More information

Location Prediction of Moving Target

Location Prediction of Moving Target Location of Moving Target Department of Radiation Oncology Stanford University AAPM 2009 Outline of Topics 1 Outline of Topics 1 2 Outline of Topics 1 2 3 Estimation of Stochastic Process Stochastic Regression:

More information

Enumeration and generation of all constitutional alkane isomers of methane to icosane using recursive generation and a modified Morgan s algorithm

Enumeration and generation of all constitutional alkane isomers of methane to icosane using recursive generation and a modified Morgan s algorithm The C n H 2n+2 challenge Zürich, November 21st 2015 Enumeration and generation of all constitutional alkane isomers of methane to icosane using recursive generation and a modified Morgan s algorithm Andreas

More information

Tennis player segmentation for semantic behavior analysis

Tennis player segmentation for semantic behavior analysis Proposta di Tennis player segmentation for semantic behavior analysis Architettura Software per Robot Mobili Vito Renò, Nicola Mosca, Massimiliano Nitti, Tiziana D Orazio, Donato Campagnoli, Andrea Prati,

More information

Sparse solver 64 bit and out-of-core addition

Sparse solver 64 bit and out-of-core addition Sparse solver 64 bit and out-of-core addition Prepared By: Richard Link Brian Yuen Martec Limited 1888 Brunswick Street, Suite 400 Halifax, Nova Scotia B3J 3J8 PWGSC Contract Number: W7707-145679 Contract

More information

Performance, Power & Energy. ELEC8106/ELEC6102 Spring 2010 Hayden Kwok-Hay So

Performance, Power & Energy. ELEC8106/ELEC6102 Spring 2010 Hayden Kwok-Hay So Performance, Power & Energy ELEC8106/ELEC6102 Spring 2010 Hayden Kwok-Hay So Recall: Goal of this class Performance Reconfiguration Power/ Energy H. So, Sp10 Lecture 3 - ELEC8106/6102 2 PERFORMANCE EVALUATION

More information

Exam Machine Learning for the Quantified Self with answers :00-14:45

Exam Machine Learning for the Quantified Self with answers :00-14:45 Exam Machine Learning for the Quantified Self with answers 21. 06. 2017 12:00-14:45 NOTES: 1. YOUR NAME MUST BE WRITTEN ON EACH SHEET IN CAPITALS. 2. Answer the questions in Dutch or English. 3. Points

More information

Member Level Forecasting Using SAS Enterprise Guide and SAS Forecast Studio

Member Level Forecasting Using SAS Enterprise Guide and SAS Forecast Studio Paper 2240 Member Level Forecasting Using SAS Enterprise Guide and SAS Forecast Studio Prudhvidhar R Perati, Leigh McCormack, BlueCross BlueShield of Tennessee, Inc., an Independent Company of BlueCross

More information

I/O Devices. Device. Lecture Notes Week 8

I/O Devices. Device. Lecture Notes Week 8 I/O Devices CPU PC ALU System bus Memory bus Bus interface I/O bridge Main memory USB Graphics adapter I/O bus Disk other devices such as network adapters Mouse Keyboard Disk hello executable stored on

More information

REPORT, RE0813, MIL, ENV, 810G, TEMP, IN-HOUSE, , PASS

REPORT, RE0813, MIL, ENV, 810G, TEMP, IN-HOUSE, , PASS REPORT, RE0813, MIL, ENV, 810G, TEMP, IN-HOUSE, REV. A C r y s t a l G r o u p I n c 8 5 0 K a c e n a R d., H i a w a t h a, I A P h o n e : 8 7 7-2 7 9-7 8 6 3 F a x : 3 1 9-3 9 3-2 3 3 8 8 / 1 5 / 2

More information

Optimization Techniques for Virtual Machine Placement and Migration

Optimization Techniques for Virtual Machine Placement and Migration http://www.iaras.org/iaras/ournals/imcm Optimization Techniques for Virtual Machine Placement and Migration SATORU OHTA Department of Electrical and Computer Engineering, Toyama Prefectural University,

More information

JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2007

JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2007 JOINT WMO TECHNICAL PROGRESS REPORT ON THE GLOBAL DATA PROCESSING AND FORECASTING SYSTEM AND NUMERICAL WEATHER PREDICTION RESEARCH ACTIVITIES FOR 2007 [TURKEY/Turkish State Meteorological Service] 1. Summary

More information

The Green Index (TGI): A Metric for Evalua:ng Energy Efficiency in HPC Systems

The Green Index (TGI): A Metric for Evalua:ng Energy Efficiency in HPC Systems The Green Index (TGI): A Metric for Evalua:ng Energy Efficiency in HPC Systems Wu Feng and Balaji Subramaniam Metrics for Energy Efficiency Energy- Delay Product (EDP) Used primarily in circuit design

More information

Optimization strategy for MASNUM surface wave model

Optimization strategy for MASNUM surface wave model Hillsboro, September 27, 2018 Optimization strategy for MASNUM surface wave model Zhenya Song *, + * First Institute of Oceanography (FIO), State Oceanic Administrative (SOA), China + Intel Parallel Computing

More information

The treatment of uncertainty in uniform workload distribution problems

The treatment of uncertainty in uniform workload distribution problems The treatment of uncertainty in uniform workload distribution problems tefan PE KO, Roman HAJTMANEK University of šilina, Slovakia 34 th International Conference Mathematical Methods in Economics Liberec,

More information

Indicator Constraints in Mixed-Integer Programming

Indicator Constraints in Mixed-Integer Programming Indicator Constraints in Mixed-Integer Programming Andrea Lodi University of Bologna, Italy - andrea.lodi@unibo.it Amaya Nogales-Gómez, Universidad de Sevilla, Spain Pietro Belotti, FICO, UK Matteo Fischetti,

More information

Introduction to ArcGIS GeoAnalytics Server. Sarah Ambrose & Noah Slocum

Introduction to ArcGIS GeoAnalytics Server. Sarah Ambrose & Noah Slocum Introduction to ArcGIS GeoAnalytics Server Sarah Ambrose & Noah Slocum Agenda Overview Analysis Capabilities + Demo Deployment and Configuration Questions ArcGIS GeoAnalytics Server uses the power of distributed

More information

Generating Urban Mobility Data Sets Using Scalable GANs

Generating Urban Mobility Data Sets Using Scalable GANs Generating Urban Mobility Data Sets Using Scalable GANs Abhinav Jauhri & John Paul Shen ECE Department Carnegie Mellon University {ajauhri, jpshen}@cmu.edu Objective Generate city-scale human mobility

More information

Energy-Efficient Real-Time Task Scheduling in Multiprocessor DVS Systems

Energy-Efficient Real-Time Task Scheduling in Multiprocessor DVS Systems Energy-Efficient Real-Time Task Scheduling in Multiprocessor DVS Systems Jian-Jia Chen *, Chuan Yue Yang, Tei-Wei Kuo, and Chi-Sheng Shih Embedded Systems and Wireless Networking Lab. Department of Computer

More information

Application of Optimization Methods and Edge AI

Application of Optimization Methods and Edge AI and Hai-Liang Zhao hliangzhao97@gmail.com November 17, 2018 This slide can be downloaded at Link. Outline 1 How to Design Novel Models with Methods Embedded Naturally? Outline 1 How to Design Novel Models

More information

Development of a Deep Recurrent Neural Network Controller for Flight Applications

Development of a Deep Recurrent Neural Network Controller for Flight Applications Development of a Deep Recurrent Neural Network Controller for Flight Applications American Control Conference (ACC) May 26, 2017 Scott A. Nivison Pramod P. Khargonekar Department of Electrical and Computer

More information

Geog 469 GIS Workshop. Managing Enterprise GIS Geodatabases

Geog 469 GIS Workshop. Managing Enterprise GIS Geodatabases Geog 469 GIS Workshop Managing Enterprise GIS Geodatabases Outline 1. Why is a geodatabase important for GIS? 2. What is the architecture of a geodatabase? 3. How can we compare and contrast three types

More information

Artificial Intelligence for HVAC Systems

Artificial Intelligence for HVAC Systems Artificial Intelligence for HVAC Systems V. Ziebart, ACSS IAMP ZHAW Motivation More than 80% of the energy consumed in Swiss households is used for room heating & cooling and domestic hot water.* The increasing

More information

Parallel PIPS-SBB Multi-level parallelism for 2-stage SMIPS. Lluís-Miquel Munguia, Geoffrey M. Oxberry, Deepak Rajan, Yuji Shinano

Parallel PIPS-SBB Multi-level parallelism for 2-stage SMIPS. Lluís-Miquel Munguia, Geoffrey M. Oxberry, Deepak Rajan, Yuji Shinano Parallel PIPS-SBB Multi-level parallelism for 2-stage SMIPS Lluís-Miquel Munguia, Geoffrey M. Oxberry, Deepak Rajan, Yuji Shinano ... Our contribution PIPS-PSBB*: Multi-level parallelism for Stochastic

More information

Deduction and Induction. Stephan Schulz. The Inference Engine. Machine Learning. A Match Made in Heaven

Deduction and Induction. Stephan Schulz. The Inference Engine. Machine Learning. A Match Made in Heaven Deduction and Induction A Match Made in Heaven Stephan Schulz The Inference Engine Machine Learning Deduction and Induction A Match Made in Heaven Stephan Schulz or a Deal with the Devil? The Inference

More information

An Experimental Evaluation of Passage-Based Process Discovery

An Experimental Evaluation of Passage-Based Process Discovery An Experimental Evaluation of Passage-Based Process Discovery H.M.W. Verbeek and W.M.P. van der Aalst Technische Universiteit Eindhoven Department of Mathematics and Computer Science P.O. Box 513, 5600

More information

UTPlaceF 3.0: A Parallelization Framework for Modern FPGA Global Placement

UTPlaceF 3.0: A Parallelization Framework for Modern FPGA Global Placement UTPlaceF 3.0: A Parallelization Framework for Modern FPGA Global Placement Wuxi Li, Meng Li, Jiajun Wang, and David Z. Pan University of Texas at Austin wuxili@utexas.edu November 14, 2017 UT DA Wuxi Li

More information

Continuous heat flow analysis. Time-variant heat sources. Embedded Systems Laboratory (ESL) Institute of EE, Faculty of Engineering

Continuous heat flow analysis. Time-variant heat sources. Embedded Systems Laboratory (ESL) Institute of EE, Faculty of Engineering Thermal Modeling, Analysis and Management of 2D Multi-Processor System-on-Chip Prof David Atienza Alonso Embedded Systems Laboratory (ESL) Institute of EE, Falty of Engineering Outline MPSoC thermal modeling

More information

FPGA Implementation of a Predictive Controller

FPGA Implementation of a Predictive Controller FPGA Implementation of a Predictive Controller SIAM Conference on Optimization 2011, Darmstadt, Germany Minisymposium on embedded optimization Juan L. Jerez, George A. Constantinides and Eric C. Kerrigan

More information

URP 4273 Section 3058 Survey Of Planning Information Systems (3 Credits) Spring 2017

URP 4273 Section 3058 Survey Of Planning Information Systems (3 Credits) Spring 2017 URP 4273 Section 3058 Survey Of Planning Information Systems (3 Credits) Spring 2017 Instructor: Office Periods: Stanley Latimer 466 ARCH Phone: 352 294-1493 e-mail: latimer@geoplan.ufl.edu Monday Thursday,

More information

Extreme Sparse Multinomial Logistic Regression: A Fast and Robust Framework for Hyperspectral Image Classification

Extreme Sparse Multinomial Logistic Regression: A Fast and Robust Framework for Hyperspectral Image Classification Article Extreme Sparse Multinomial Logistic Regression: A Fast and Robust Framework for Hyperspectral Image Classification Faxian Cao 1, Zhijing Yang 1, *, Jinchang Ren 2, Wing-Kuen Ling 1, Huimin Zhao

More information

Classification of Hand-Written Digits Using Scattering Convolutional Network

Classification of Hand-Written Digits Using Scattering Convolutional Network Mid-year Progress Report Classification of Hand-Written Digits Using Scattering Convolutional Network Dongmian Zou Advisor: Professor Radu Balan Co-Advisor: Dr. Maneesh Singh (SRI) Background Overview

More information

URP 4273 Section 8233 Introduction to Planning Information Systems (3 Credits) Fall 2017

URP 4273 Section 8233 Introduction to Planning Information Systems (3 Credits) Fall 2017 URP 4273 Section 8233 Introduction to Planning Information Systems (3 Credits) Fall 2017 Instructor: Office Periods: Stanley Latimer 466 ARCH Phone: 352 294-1493 e-mail: latimer@geoplan.ufl.edu Monday

More information

Table S1 shows the SOM parameters that were used in the main manuscript. These correspond to the default set of options of the SOM Toolbox.

Table S1 shows the SOM parameters that were used in the main manuscript. These correspond to the default set of options of the SOM Toolbox. Supplemental file 1: SOM training parameters and sensitivity analysis. Table S1 shows the SOM parameters that were used in the main manuscript. These correspond to the default set of options of the SOM

More information

Faster Machine Learning via Low-Precision Communication & Computation. Dan Alistarh (IST Austria & ETH Zurich), Hantian Zhang (ETH Zurich)

Faster Machine Learning via Low-Precision Communication & Computation. Dan Alistarh (IST Austria & ETH Zurich), Hantian Zhang (ETH Zurich) Faster Machine Learning via Low-Precision Communication & Computation Dan Alistarh (IST Austria & ETH Zurich), Hantian Zhang (ETH Zurich) 2 How many bits do you need to represent a single number in machine

More information

Justin Appleby CS 229 Machine Learning Project Report 12/15/17 Kevin Chalhoub Building Electricity Load Forecasting

Justin Appleby CS 229 Machine Learning Project Report 12/15/17 Kevin Chalhoub Building Electricity Load Forecasting Justin Appleby CS 229 Machine Learning Project Report 12/15/17 Kevin Chalhoub Building Electricity Load Forecasting with ARIMA and Sequential Linear Regression Abstract Load forecasting is an essential

More information

LC Commissioning, Operations and Availability

LC Commissioning, Operations and Availability International Technology Recommendation Panel X-Band Linear Collider Path to the Future LC Commissioning, Operations and Availability Tom Himel Stanford Linear Accelerator Center April 26-27, 2004 Integrating

More information

CS162 Operating Systems and Systems Programming Lecture 17. Performance Storage Devices, Queueing Theory

CS162 Operating Systems and Systems Programming Lecture 17. Performance Storage Devices, Queueing Theory CS162 Operating Systems and Systems Programming Lecture 17 Performance Storage Devices, Queueing Theory October 25, 2017 Prof. Ion Stoica http://cs162.eecs.berkeley.edu Review: Basic Performance Concepts

More information

Reducing The Computational Cost of Bayesian Indoor Positioning Systems

Reducing The Computational Cost of Bayesian Indoor Positioning Systems Reducing The Computational Cost of Bayesian Indoor Positioning Systems Konstantinos Kleisouris, Richard P. Martin Computer Science Department Rutgers University WINLAB Research Review May 15 th, 2006 Motivation

More information

A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters

A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters ANTONINO TUMEO, ORESTE VILLA Collaborators: Karol Kowalski, Sriram Krishnamoorthy, Wenjing Ma, Simone Secchi May 15, 2012 1 Outline!

More information

Developing IoT-Based Factory Automation Using F-RAM

Developing IoT-Based Factory Automation Using F-RAM Developing IoT-Based Factory Automation Using F-RAM Douglas Mitchell Flash Memory Summit 2018 Santa Clara, CA 1 INDUSTRIAL SYSTEMS TODAY Industry 4.0 The smart factory, Cyber-physical systems monitor factory

More information

Regression, Curve Fitting and Optimisation

Regression, Curve Fitting and Optimisation Supervised by Elena Zanini STOR-i, University of Lancaster 4 September 2015 1 Introduction Root Finding 2 3 Simulated Annealing 4 5 The Rosenbrock Banana Function 6 7 Given a set of data, what is the optimum

More information

Online Algorithms for a Generalized Parallel Machine Scheduling Problem

Online Algorithms for a Generalized Parallel Machine Scheduling Problem PARALLEL MACHINE SCHEDULING PROBLEM 1 Online Algorithms for a Generalized Parallel Machine Scheduling Problem István SZALKAI 1,2, György DÓSA 1,3 1 Department of Mathematics, University of Pannonia, Veszprém,

More information

先進的計算基盤システムシンポジウム SACSIS2012 Symposium on Advanced Computing Systems and Infrastructures SACSIS /5/18 VM 1 VM VM 0-1 1) 18% 50% 2) 3) Matching

先進的計算基盤システムシンポジウム SACSIS2012 Symposium on Advanced Computing Systems and Infrastructures SACSIS /5/18 VM 1 VM VM 0-1 1) 18% 50% 2) 3) Matching VM 1 VM VM 0-1 1) 18% 50% 2) 3) Matching-based Virtual Machine Packing Algorithm for Lower Power Consumption SATOSHI TAKAHASHI, ATSUKO TAKEFUSA, MAIKO SHIGENO, HIDEMOTO NAKADA, TOMOHIRO KUDOH and AKIKO

More information

Saving Power in the Data Center. Marios Papaefthymiou

Saving Power in the Data Center. Marios Papaefthymiou Saving Power in the Data Center Marios Papaefthymiou Energy Use in Data Centers Several MW for large facilities Responsible for sizable fraction of US electricity consumption 2006: 1.5% or 61 billion kwh

More information

What s Cooking? Predicting Cuisines from Recipe Ingredients

What s Cooking? Predicting Cuisines from Recipe Ingredients What s Cooking? Predicting Cuisines from Recipe Ingredients Kevin K. Do Department of Computer Science Duke University Durham, NC 27708 kevin.kydat.do@gmail.com Abstract Kaggle is an online platform for

More information

Jake Diebolt, GIS Technician/Coordinator

Jake Diebolt, GIS Technician/Coordinator Jake Diebolt, GIS Technician/Coordinator Population: 13 090 (2006 Census) 7 Organized Municipalities, two Incorporated Towns and two unorganized Townships. Population doubles during the summer tourist

More information