Complete Faceting. code4lib, Feb Toke Eskildsen Mikkel Kamstrup Erlandsen
|
|
|
- Charlene Sparks
- 5 years ago
- Views:
Transcription
1 Complete Faceting code4lib, Feb Toke Eskildsen Mikkel Kamstrup Erlandsen
2 Battle Plan Terminology (geek level: 1) History (geek level: 3) Data Structures (geek level: 6) Indexing (geek level: 4) Searching or actually Counting (geek level: 5) Scaling or How we Cheat (geek level: 3) Free Bacon! (geek level: )
3 Wait, What is Summa? Keywords Search Engine Designed for Libraries Open Source (LGPL) Integrated Search Cold Facts 100% Java Lucene Index(es) Developed Since Winter 2005 In Production Since Nov Lightning Talk Later
4 Terminology Documents contain Fields ti: Applied Quantum Mechanics gen_subj: physics subj: quantum mechanics ti: Smooth Manifolds in Physics gen_subj: mathematics gen_subj: physics subj: smooth manifolds
5 Terminology Documents contain Fields ti: Applied Quantum Mechanics gen_subj: physics subj: quantum mechanics ti: Smooth Manifolds in Physics gen_subj: mathematics gen_subj: physics subj: smooth manifolds Facets contain Tags The title facet Applied Quantum Mechanics Smooth Manifolds in Physics The subject facet physics quantum mechanics mathematics smooth manifolds
6 Terminology Documents contain Fields ti: Applied Quantum Mechanics gen_subj: physics subj: quantum mechanics ti: Smooth Manifolds in Physics gen_subj: mathematics gen_subj: physics subj: smooth manifolds Facets contain Tags The title facet Applied Quantum Mechanics Smooth Manifolds in Physics The subject facet physics quantum mechanics The spaghetti is called References mathematics smooth manifolds
7 Diving In Iterate Lucene hits, collect field content Use clean OO facet/tag structure Create cache map in memory Collect tag counts with nice HashMap Logical path onwards? Use field cache or similar BitSets
8 Stop! What Do We Want? Scale Up and Down Iterative Updates Decoupling from Text Search Engine
9 Facet Mapping Index DocID References offset End (4) 5 References Offset FacetID TagID Resolve the tag string for docs 0 and 3 Facet 1 (sorted list of Tags) TagID Offset in tag file Tag file bacon\npuppies\n... 6
10 Persistence All references are arrays Just dump them directly to the file system Two strategies for updating tag files Append tags on the fly Store as a full dump at a point in time Two strategies for resolving tags on search Get them from the file system (SSDs rules) Load them fully into memory (ouch)
11 Persistence All references are arrays Just dump them directly to the file system Two strategies for updating tag files Append tags on the fly Store as a full dump at a point in time Two strategies for resolving tags on search Get them from the file system (SSDs rules) Load them fully into memory (ouch) Put those SSDs to work!
12 Facet structure building Summa Manipulators: Analyze payload Write to a private sub-index Attach additional info and pass the payload on The Facet Manipulator: Receives Document ID and Fields from the Document manipulator Performs an iterative update of the facet structure
13 Searching The Job of a Search Node: Search private sub-index, or other private source Add response to collector The Job of the Facet Node: Receive a BitSet from a previous Document Search Node Generate Facet response, add it to the collector
14 Tag Counting Index DocID References offset End (4) 5 References Offset FacetID TagID Increment doc count for tag 1 TagCounter 1 coupled to Facet 1 TagID Counter (int)
15 Distributed Search Summa is equipped to take full advantage of a distributed setup Each sub search node produces a part of the full answer A special search node aggregates results from a set of sub search nodes
16 Distribution is tricky Merge the top three tags from two nodes: Node 1 Node 2 Result Tag A 2 Tag E 2 Tag A 4 + = Tag B 1 Tag A 2 Tag D 2 Tag C 1 Tag D 2 Tag E 2
17 Distribution is tricky Merge the top three tags from two nodes: Node 1 Node 2 Result Tag A 2 Tag E 2 Tag A 4 + = Tag B 1 Tag A 2 Tag D 2 Tag C 1 Tag D 2 Tag E 2 FAIL
18 Distribution is tricky Merge the top three tags from two nodes: Node 1 Node 2 Result Tag A 2 Tag E 2 Tag A 4 + = Tag B 1 Tag A 2 Tag D 2 Tag C 1 Tag D 2 Tag E 2 Node 1 Node 2 Result Tag A 2 Tag E 2 Tag A 4 Tag B 1 Tag A 2 Tag B 3 + = Tag C 1 Tag D 2 Tag D 3 Tag D 1 Tag B 2 Tag E 1 Tag C 1 Tag F 1 Tag F 1
19 Real Life in Numbers 10M docs, 10M tags, 100M refs, 1 machine A few 1000 hits < 100 ms A few hits < 200 ms 10M hits ~3 sec 100M docs, 1G tags, 1G refs, 3 machines A few 1000 hits ~1 sec (ouch) A few hits ~1 sec 10M hits < 3 sec 100M hits ~15 sec
20 Bonus Level! Persistent sorted Tags Index lookup (alphabetic listings) Localized range queries Sort without warm-up and memory overhead
21 Level Completed! Dead Troll (CC) BY-NC-SA by Kim Smith
22 Questions? wiki.statsbiblioteket.dk/summa Summa, Integrated Search Document/Field, Facet/Tag FieldCache, BitSet Iterative updates Lucene decoupling Structure Persistence, Memory Overhead Indexing WeakHashMap, MergeSort Tag Counting Distributed searching Cheating Scalability numbers Collator Order
Prioritized Garbage Collection Using the Garbage Collector to Support Caching
Prioritized Garbage Collection Using the Garbage Collector to Support Caching Diogenes Nunez, Samuel Z. Guyer, Emery D. Berger Tufts University, University of Massachusetts Amherst November 2, 2016 D.
Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models
Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models Chengjie Qin 1, Martin Torres 2, and Florin Rusu 2 1 GraphSQL, Inc. 2 University of California Merced August 31, 2017 Machine
Problem Decomposition: One Professor s Approach to Coding
Problem Decomposition: One Professor s Approach to Coding zombie[1] zombie[3] Fewer Buuuuugs zombie[4] zombie[2] zombie[5] zombie[0] Fundamentals of Computer Science I Overview Problem Solving Understand
Advanced Implementations of Tables: Balanced Search Trees and Hashing
Advanced Implementations of Tables: Balanced Search Trees and Hashing Balanced Search Trees Binary search tree operations such as insert, delete, retrieve, etc. depend on the length of the path to the
Information Retrieval Using Boolean Model SEEM5680
Information Retrieval Using Boolean Model SEEM5680 1 Unstructured (text) vs. structured (database) data in 1996 2 2 Unstructured (text) vs. structured (database) data in 2009 3 3 The problem of IR Goal
CS425: Algorithms for Web Scale Data
CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets book. These slides have been modified for CS425. The original slides can be accessed at: www.mmds.org Challenges
Section 3.3: Discrete-Event Simulation Examples
Section 33: Discrete-Event Simulation Examples Discrete-Event Simulation: A First Course c 2006 Pearson Ed, Inc 0-13-142917-5 Discrete-Event Simulation: A First Course Section 33: Discrete-Event Simulation
Impulse. Observations
Impulse Observations What is the physics behind breaking an egg? If we look at the situation in terms of force, an egg dropping seems to have the same force (of gravity) as an egg sitting on a counter,
Using R for Iterative and Incremental Processing
Using R for Iterative and Incremental Processing Shivaram Venkataraman, Indrajit Roy, Alvin AuYoung, Robert Schreiber UC Berkeley and HP Labs UC BERKELEY Big Data, Complex Algorithms PageRank (Dominant
Behavioral Simulations in MapReduce
Behavioral Simulations in MapReduce Guozhang Wang, Marcos Vaz Salles, Benjamin Sowell, Xun Wang, Tuan Cao, Alan Demers, Johannes Gehrke, Walker White Cornell University 1 What are Behavioral Simulations?
Time. Today. l Physical clocks l Logical clocks
Time Today l Physical clocks l Logical clocks Events, process states and clocks " A distributed system a collection P of N singlethreaded processes without shared memory Each process p i has a state s
Lecture 14: Nov. 11 & 13
CIS 2168 Data Structures Fall 2014 Lecturer: Anwar Mamat Lecture 14: Nov. 11 & 13 Disclaimer: These notes may be distributed outside this class only with the permission of the Instructor. 14.1 Sorting
Course Announcements. Bacon is due next Monday. Next lab is about drawing UIs. Today s lecture will help thinking about your DB interface.
Course Announcements Bacon is due next Monday. Today s lecture will help thinking about your DB interface. Next lab is about drawing UIs. John Jannotti (cs32) ORMs Mar 9, 2017 1 / 24 ORMs John Jannotti
Dictionary: an abstract data type
2-3 Trees 1 Dictionary: an abstract data type A container that maps keys to values Dictionary operations Insert Search Delete Several possible implementations Balanced search trees Hash tables 2 2-3 trees
Hashing Data Structures. Ananda Gunawardena
Hashing 15-121 Data Structures Ananda Gunawardena Hashing Why do we need hashing? Many applications deal with lots of data Search engines and web pages There are myriad look ups. The look ups are time
Large-Scale Behavioral Targeting
Large-Scale Behavioral Targeting Ye Chen, Dmitry Pavlov, John Canny ebay, Yandex, UC Berkeley (This work was conducted at Yahoo! Labs.) June 30, 2009 Chen et al. (KDD 09) Large-Scale Behavioral Targeting
APPEND AND MERGE. Colorado average snowfall amounts - Aggregated by County and Month
APPEND AND MERGE Climate change affects the environment is many ways. This past winter produced little snow in California which is likely to compound their drought. The lack of snow also negatively impacts
Big Data Infrastructure CS 489/698 Big Data Infrastructure (Winter 2016)
Big Data Infrastructure CS 489/698 Big Data Infrastructure (Winter 2016) Week 12: Real-Time Data Analytics (2/2) March 31, 2016 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
Lecture 13: Sequential Circuits, FSM
Lecture 13: Sequential Circuits, FSM Today s topics: Sequential circuits Finite state machines 1 Clocks A microprocessor is composed of many different circuits that are operating simultaneously if each
ECE 571 Advanced Microprocessor-Based Design Lecture 9
ECE 571 Advanced Microprocessor-Based Design Lecture 9 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 20 February 2018 Announcements HW#4 was posted. About branch predictors Don
DM507 - Algoritmer og Datastrukturer Project 1
DM507 - Algoritmer og Datastrukturer Project 1 Christian Skjøth Mat-Øk 280588 Morten Olsen Mat-Øk 090789 19. marts 2012 Task 1 - Double for-loop So, first we needed to create an instance of the class File
COMP 250 Fall Midterm examination
COMP 250 Fall 2004 - Midterm examination October 18th 2003, 13:35-14:25 1 Running time analysis (20 points) For each algorithm below, indicate the running time using the simplest and most accurate big-oh
Data Structures in Java
Data Structures in Java Lecture 20: Algorithm Design Techniques 12/2/2015 Daniel Bauer 1 Algorithms and Problem Solving Purpose of algorithms: find solutions to problems. Data Structures provide ways of
Dot-Product Join: Scalable In-Database Linear Algebra for Big Model Analytics
Dot-Product Join: Scalable In-Database Linear Algebra for Big Model Analytics Chengjie Qin 1 and Florin Rusu 2 1 GraphSQL, Inc. 2 University of California Merced June 29, 2017 Machine Learning (ML) Is
CSCE 561 Information Retrieval System Models
CSCE 561 Information Retrieval System Models Satya Katragadda 26 August 2015 Agenda Introduction to Information Retrieval Inverted Index IR System Models Boolean Retrieval Model 2 Introduction Information
Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE
General e Image Coder Structure Motion Video x(s 1,s 2,t) or x(s 1,s 2 ) Natural Image Sampling A form of data compression; usually lossless, but can be lossy Redundancy Removal Lossless compression: predictive
THE ZCACHE: DECOUPLING WAYS AND ASSOCIATIVITY. Daniel Sanchez and Christos Kozyrakis Stanford University
THE ZCACHE: DECOUPLING WAYS AND ASSOCIATIVITY Daniel Sanchez and Christos Kozyrakis Stanford University MICRO-43, December 6 th 21 Executive Summary 2 Mitigating the memory wall requires large, highly
Deterministic Finite Automaton (DFA)
1 Lecture Overview Deterministic Finite Automata (DFA) o accepting a string o defining a language Nondeterministic Finite Automata (NFA) o converting to DFA (subset construction) o constructed from a regular
Arup Nanda Starwood Hotels
Arup Nanda Starwood Hotels Why Analyze The Database is Slow! Storage, CPU, memory, runqueues all affect the performance Know what specifically is causing them to be slow To build a profile of the application
An Optimized Interestingness Hotspot Discovery Framework for Large Gridded Spatio-temporal Datasets
IEEE Big Data 2015 Big Data in Geosciences Workshop An Optimized Interestingness Hotspot Discovery Framework for Large Gridded Spatio-temporal Datasets Fatih Akdag and Christoph F. Eick Department of Computer
Lecture 13: Sequential Circuits, FSM
Lecture 13: Sequential Circuits, FSM Today s topics: Sequential circuits Finite state machines Reminder: midterm on Tue 2/28 will cover Chapters 1-3, App A, B if you understand all slides, assignments,
The Boolean Model ~1955
The Boolean Model ~1955 The boolean model is the first, most criticized, and (until a few years ago) commercially more widespread, model of IR. Its functionalities can often be found in the Advanced Search
Chuck Cartledge, PhD. 21 January 2018
Big Data: Data Analysis Boot Camp Non-SQL and R Chuck Cartledge, PhD 21 January 2018 1/19 Table of contents (1 of 1) 1 Intro. 2 Non-SQL DBMS Classic Non-SQL databases 3 Hands-on Airport connections as
Farewell, PipelinePilot Migrating the Exquiron cheminformatics platform to KNIME and the ChemAxon technology
Farewell, PipelinePilot Migrating the Exquiron cheminformatics platform to KNIME and the ChemAxon technology Serge P. Parel, PhD ChemAxon User Group Meeting, Budapest 21 st May, 2014 Outline Exquiron Who
Data-Intensive Distributed Computing
Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 9: Real-Time Data Analytics (2/2) March 29, 2018 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
ST-Links. SpatialKit. Version 3.0.x. For ArcMap. ArcMap Extension for Directly Connecting to Spatial Databases. ST-Links Corporation.
ST-Links SpatialKit For ArcMap Version 3.0.x ArcMap Extension for Directly Connecting to Spatial Databases ST-Links Corporation www.st-links.com 2012 Contents Introduction... 3 Installation... 3 Database
Asymptotic Running Time of Algorithms
Asymptotic Complexity: leading term analysis Asymptotic Running Time of Algorithms Comparing searching and sorting algorithms so far: Count worst-case of comparisons as function of array size. Drop lower-order
IR: Information Retrieval
/ 44 IR: Information Retrieval FIB, Master in Innovation and Research in Informatics Slides by Marta Arias, José Luis Balcázar, Ramon Ferrer-i-Cancho, Ricard Gavaldá Department of Computer Science, UPC
Graphical User Interfaces for Emittance and Correlation Plot. Henrik Loos
Graphical User Interfaces for Emittance and Correlation Plot Common GUI Features Overview Files Configs Measure Data Point Export Common GUI Features Save Saves the present data. Auto-generated file name
Phrase Finding; Stream-and-Sort vs Request-and-Answer. William W. Cohen
Phrase Finding; Stream-and-Sort vs Request-and-Answer William W. Cohen Announcement Two-day extension for all on HW1B: now due Thursday 1/29 Correction Using Large-vocabulary Naïve Bayes For each example
Impression Store: Compressive Sensing-based Storage for. Big Data Analytics
Impression Store: Compressive Sensing-based Storage for Big Data Analytics Jiaxing Zhang, Ying Yan, Liang Jeff Chen, Minjie Wang, Thomas Moscibroda & Zheng Zhang Microsoft Research The Curse of O(N) in
Administering your Enterprise Geodatabase using Python. Jill Penney
Administering your Enterprise Geodatabase using Python Jill Penney Assumptions Basic knowledge of python Basic knowledge enterprise geodatabases and workflows You want code Please turn off or silence cell
COMPUTER SCIENCE TRIPOS
CST.2016.2.1 COMPUTER SCIENCE TRIPOS Part IA Tuesday 31 May 2016 1.30 to 4.30 COMPUTER SCIENCE Paper 2 Answer one question from each of Sections A, B and C, and two questions from Section D. Submit the
Parallel Transposition of Sparse Data Structures
Parallel Transposition of Sparse Data Structures Hao Wang, Weifeng Liu, Kaixi Hou, Wu-chun Feng Department of Computer Science, Virginia Tech Niels Bohr Institute, University of Copenhagen Scientific Computing
High Performance Computing
Master Degree Program in Computer Science and Networking, 2014-15 High Performance Computing 2 nd appello February 11, 2015 Write your name, surname, student identification number (numero di matricola),
EGR 111 Heat Transfer
EGR 111 Heat Transfer The purpose of this lab is to use MATLAB to determine the heat transfer in a 1-dimensional system. New MATLAB commands: (none) 1. Heat Transfer 101 Heat Transfer is thermal energy
Computational Complexity. This lecture. Notes. Lecture 02 - Basic Complexity Analysis. Tom Kelsey & Susmit Sarkar. Notes
Computational Complexity Lecture 02 - Basic Complexity Analysis Tom Kelsey & Susmit Sarkar School of Computer Science University of St Andrews http://www.cs.st-andrews.ac.uk/~tom/ twk@st-andrews.ac.uk
FACULTY OF SCIENCE ACADEMY OF COMPUTER SCIENCE AND SOFTWARE ENGINEERING OBJECT ORIENTED PROGRAMMING DATE 07/2014 SESSION 8:00-10:00
FACULTY OF SCIENCE ACADEMY OF COMPUTER SCIENCE AND SOFTWARE ENGINEERING MODULE CAMPUS CSC2A10 OBJECT ORIENTED PROGRAMMING AUCKLAND PARK CAMPUS (APK) EXAM JULY 2014 DATE 07/2014 SESSION 8:00-10:00 ASSESOR(S)
Theoretical aspects of ERa, the fastest practical suffix tree construction algorithm
Theoretical aspects of ERa, the fastest practical suffix tree construction algorithm Matevž Jekovec University of Ljubljana Faculty of Computer and Information Science Oct 10, 2013 Text indexing problem
Become a Microprobe Power User Part 2: Qualitative & Quantitative Analysis
Become a Microprobe Power User Part 2: Qualitative & Quantitative Analysis Mike Spilde Spring IOM Seminar February 5, 2008 Qualitative Analysis Why use qualitative scans? Elemental ID (especially trace
SOLUTION: SOLUTION: SOLUTION:
Convert R and S into nondeterministic finite automata N1 and N2. Given a string s, if we know the states N1 and N2 may reach when s[1...i] has been read, we are able to derive the states N1 and N2 may
Running Time. Overview. Case Study: Sorting. Sorting problem: Analysis of algorithms: framework for comparing algorithms and predicting performance.
Running Time Analysis of Algorithms As soon as an Analytic Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will arise -
Foundations Of Mathematical Logic (Dover Books On Mathematics) By Haskell B. Curry
Foundations Of Mathematical Logic (Dover Books On Mathematics) By Haskell B. Curry 0486634620 - Foundations of Mathematical Logic - Foundations of Mathematical Logic (Dover Books on Mathematics) by Haskell
Measuring Goodness of an Algorithm. Asymptotic Analysis of Algorithms. Measuring Efficiency of an Algorithm. Algorithm and Data Structure
Measuring Goodness of an Algorithm Asymptotic Analysis of Algorithms EECS2030 B: Advanced Object Oriented Programming Fall 2018 CHEN-WEI WANG 1. Correctness : Does the algorithm produce the expected output?
P Q1 Q2 Q3 Q4 Q5 Tot (60) (20) (20) (20) (60) (20) (200) You are allotted a maximum of 4 hours to complete this exam.
Exam INFO-H-417 Database System Architecture 13 January 2014 Name: ULB Student ID: P Q1 Q2 Q3 Q4 Q5 Tot (60 (20 (20 (20 (60 (20 (200 Exam modalities You are allotted a maximum of 4 hours to complete this
Acceptably inaccurate. Probabilistic data structures
Acceptably inaccurate Probabilistic data structures Hello Today's talk Motivation Bloom filters Count-Min Sketch HyperLogLog Motivation Tape HDD SSD Memory Tape HDD SSD Memory Speed Tape HDD SSD Memory
Huffman Coding. C.M. Liu Perceptual Lab, College of Computer Science National Chiao-Tung University
Huffman Coding C.M. Liu Perceptual Lab, College of Computer Science National Chiao-Tung University http://www.csie.nctu.edu.tw/~cmliu/courses/compression/ Office: EC538 (03)573877 cmliu@cs.nctu.edu.tw
CSE613: Parallel Programming, Spring 2012 Date: May 11. Final Exam. ( 11:15 AM 1:45 PM : 150 Minutes )
CSE613: Parallel Programming, Spring 2012 Date: May 11 Final Exam ( 11:15 AM 1:45 PM : 150 Minutes ) This exam will account for either 10% or 20% of your overall grade depending on your relative performance
CS173 Running Time and Big-O. Tandy Warnow
CS173 Running Time and Big-O Tandy Warnow CS 173 Running Times and Big-O analysis Tandy Warnow Today s material We will cover: Running time analysis Review of running time analysis of Bubblesort Review
Comp 11 Lectures. Mike Shah. June 20, Tufts University. Mike Shah (Tufts University) Comp 11 Lectures June 20, / 38
Comp 11 Lectures Mike Shah Tufts University June 20, 2017 Mike Shah (Tufts University) Comp 11 Lectures June 20, 2017 1 / 38 Please do not distribute or host these slides without prior permission. Mike
Design at the Register Transfer Level
Week-7 Design at the Register Transfer Level Algorithmic State Machines Algorithmic State Machine (ASM) q Our design methodologies do not scale well to real-world problems. q 232 - Logic Design / Algorithmic
Time. To do. q Physical clocks q Logical clocks
Time To do q Physical clocks q Logical clocks Events, process states and clocks A distributed system A collection P of N single-threaded processes (p i, i = 1,, N) without shared memory The processes in
django in the real world
django in the real world yes! it scales!... YAY! Israel Fermin Montilla Software Engineer @ dubizzle December 14, 2017 from iferminm import more data Software Engineer @ dubizzle Venezuelan living in Dubai,
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago Joint work with: Ian Foster: Univ. of
The Fragment Network: A Chemistry Recommendation Engine Built Using a Graph Database
Supporting Information The Fragment Network: A Chemistry Recommendation Engine Built Using a Graph Database Richard J. Hall, Christopher W. Murray and Marcel L. Verdonk. Contents This supporting information
Computation of Large Sparse Aggregated Areas for Analytic Database Queries
Computation of Large Sparse Aggregated Areas for Analytic Database Queries Steffen Wittmer Tobias Lauer Jedox AG Collaborators: Zurab Khadikov Alexander Haberstroh Peter Strohm Business Intelligence and
Factorized Relational Databases Olteanu and Závodný, University of Oxford
November 8, 2013 Database Seminar, U Washington Factorized Relational Databases http://www.cs.ox.ac.uk/projects/fd/ Olteanu and Závodný, University of Oxford Factorized Representations of Relations Cust
ECE 571 Advanced Microprocessor-Based Design Lecture 10
ECE 571 Advanced Microprocessor-Based Design Lecture 10 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 23 February 2017 Announcements HW#5 due HW#6 will be posted 1 Oh No, More
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago Joint work with: Ian Foster: Univ. of
Scripting Languages Fast development, extensible programs
Scripting Languages Fast development, extensible programs Devert Alexandre School of Software Engineering of USTC November 30, 2012 Slide 1/60 Table of Contents 1 Introduction 2 Dynamic languages A Python
A General-Purpose Counting Filter: Making Every Bit Count. Prashant Pandey, Michael A. Bender, Rob Johnson, Rob Patro Stony Brook University, NY
A General-Purpose Counting Filter: Making Every Bit Count Prashant Pandey, Michael A. Bender, Rob Johnson, Rob Patro Stony Brook University, NY Approximate Membership Query (AMQ) insert(x) ismember(x)
Material Covered on the Final
Material Covered on the Final On the final exam, you are responsible for: Anything covered in class, except for stories about my good friend Ken Kennedy All lecture material after the midterm ( below the
Data Structures. Outline. Introduction. Andres Mendez-Vazquez. December 3, Data Manipulation Examples
Data Structures Introduction Andres Mendez-Vazquez December 3, 2015 1 / 53 Outline 1 What the Course is About? Data Manipulation Examples 2 What is a Good Algorithm? Sorting Example A Naive Algorithm Counting
University of New Mexico Department of Computer Science. Midterm Examination. CS 361 Data Structures and Algorithms Spring, 2003
University of New Mexico Department of Computer Science Midterm Examination CS 361 Data Structures and Algorithms Spring, 2003 Name: Email: Print your name and email, neatly in the space provided above;
Finite-State Machines (Automata) lecture 12
Finite-State Machines (Automata) lecture 12 cl a simple form of computation used widely one way to find patterns 1 A current D B A B C D B C D A C next 2 Application Fields Industry real-time control,
13 Concurrent Programming using Tasks and Services
60 Advanced Java for Bioinformatics, WS 17/18, D. Huson, January 18, 2018 13 Concurrent Programming using Tasks and Services Any programs that you write will be multi-threaded. Modern computers have multiple
Lecture: Pipelining Basics
Lecture: Pipelining Basics Topics: Performance equations wrap-up, Basic pipelining implementation Video 1: What is pipelining? Video 2: Clocks and latches Video 3: An example 5-stage pipeline Video 4:
Multi-Approximate-Keyword Routing Query
Bin Yao 1, Mingwang Tang 2, Feifei Li 2 1 Department of Computer Science and Engineering Shanghai Jiao Tong University, P. R. China 2 School of Computing University of Utah, USA Outline 1 Introduction
CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Catie Baker Spring 2015
CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Catie Baker Spring 2015 Today Registration should be done. Homework 1 due 11:59pm next Wednesday, April 8 th. Review math
Maschinelle Sprachverarbeitung
Maschinelle Sprachverarbeitung Retrieval Models and Implementation Ulf Leser Content of this Lecture Information Retrieval Models Boolean Model Vector Space Model Inverted Files Ulf Leser: Maschinelle
Bloom Filters, Minhashes, and Other Random Stuff
Bloom Filters, Minhashes, and Other Random Stuff Brian Brubach University of Maryland, College Park StringBio 2018, University of Central Florida What? Probabilistic Space-efficient Fast Not exact Why?
An Efficient Evolutionary Algorithm for Solving Incrementally Structured Problems
An Efficient Evolutionary Algorithm for Solving Incrementally Structured Problems Jason Ansel Maciej Pacula Saman Amarasinghe Una-May O Reilly MIT - CSAIL July 14, 2011 Jason Ansel (MIT) PetaBricks July
Dictionary: an abstract data type
2-3 Trees 1 Dictionary: an abstract data type A container that maps keys to values Dictionary operations Insert Search Delete Several possible implementations Balanced search trees Hash tables 2 2-3 trees
Harvard Center for Geographic Analysis Geospatial on the MOC
2017 Massachusetts Open Cloud Workshop Boston University Harvard Center for Geographic Analysis Geospatial on the MOC Ben Lewis Harvard Center for Geographic Analysis Aaron Williams MapD Small Team Supporting
Latches. October 13, 2003 Latches 1
Latches The second part of CS231 focuses on sequential circuits, where we add memory to the hardware that we ve already seen. Our schedule will be very similar to before: We first show how primitive memory
Spatial Analytics Workshop
Spatial Analytics Workshop Pete Skomoroch, LinkedIn (@peteskomoroch) Kevin Weil, Twitter (@kevinweil) Sean Gorman, FortiusOne (@seangorman) #spatialanalytics Introduction The Rise of Spatial Analytics
Analytical Modeling of Parallel Systems
Analytical Modeling of Parallel Systems Chieh-Sen (Jason) Huang Department of Applied Mathematics National Sun Yat-sen University Thank Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar for providing
Frequency Estimators
Frequency Estimators Outline for Today Randomized Data Structures Our next approach to improving performance. Count-Min Sketches A simple and powerful data structure for estimating frequencies. Count Sketches
arxiv: v2 [cs.ds] 9 Nov 2017
![arxiv: v2 [cs.ds] 9 Nov 2017](/thumbs/77/74832569.jpg "arxiv: v2 [cs.ds] 9 Nov 2017")
Replace or Retrieve Keywords In Documents At Scale Vikash Singh Belong.co Bangalore, India vikash@belong.co arxiv:1711.00046v2 [cs.ds] 9 Nov 2017 Abstract In this paper we introduce, the FlashText 1 algorithm
Searching, mainly via Hash tables
Data structures and algorithms Part 11 Searching, mainly via Hash tables Petr Felkel 26.1.2007 Topics Searching Hashing Hash function Resolving collisions Hashing with chaining Open addressing Linear Probing
Window-aware Load Shedding for Aggregation Queries over Data Streams
Window-aware Load Shedding for Aggregation Queries over Data Streams Nesime Tatbul Stan Zdonik Talk Outline Background Load shedding in Aurora Windowed aggregation queries Window-aware load shedding Experimental
Quiz 2. Due November 26th, CS525 - Advanced Database Organization Solutions
Name CWID Quiz 2 Due November 26th, 2015 CS525 - Advanced Database Organization s Please leave this empty! 1 2 3 4 5 6 7 Sum Instructions Multiple choice questions are graded in the following way: You
Efficient algorithms for symmetric tensor contractions
Efficient algorithms for symmetric tensor contractions Edgar Solomonik 1 Department of EECS, UC Berkeley Oct 22, 2013 1 / 42 Edgar Solomonik Symmetric tensor contractions 1/ 42 Motivation The goal is to
COMP 355 Advanced Algorithms
COMP 355 Advanced Algorithms Algorithm Design Review: Mathematical Background 1 Polynomial Running Time Brute force. For many non-trivial problems, there is a natural brute force search algorithm that
NEC PerforCache. Influence on M-Series Disk Array Behavior and Performance. Version 1.0
NEC PerforCache Influence on M-Series Disk Array Behavior and Performance. Version 1.0 Preface This document describes L2 (Level 2) Cache Technology which is a feature of NEC M-Series Disk Array implemented
Photoluminescence Spectrometer (FLS980)
Photoluminescence Spectrometer (FLS980) Instrument Summary: The Edinburgh Instruments FLS980 photoluminescence spectrometer offers both steady state and time resolved (lifetime) fluorescence spectroscopy
CS6901: review of Theory of Computation and Algorithms
CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational
CS 347 Parallel and Distributed Data Processing
CS 347 Parallel and Distributed Data Processing Spring 2016 Notes 3: Query Processing Query Processing Decomposition Localization Optimization CS 347 Notes 3 2 Decomposition Same as in centralized system
4th year Project demo presentation
4th year Project demo presentation Colm Ó héigeartaigh CASE4-99387212 coheig-case4@computing.dcu.ie 4th year Project demo presentation p. 1/23 Table of Contents An Introduction to Quantum Computing The
Bloom Filters, general theory and variants
Bloom Filters: general theory and variants G. Caravagna caravagn@cli.di.unipi.it Information Retrieval Wherever a list or set is used, and space is a consideration, a Bloom Filter should be considered.