CS320 Algorithms: Theory and Practice. Course Introduction

Transcription

1 Course Objectives CS320 Algorithms: Theory and Practice Algorithms: Design strategies for algorithmic problem solving Course Introduction "For me, great algorithms are the poetry of computation. Just like verse, they can be terse, allusive, dense, and even mysterious. But once unlocked, they cast a brilliant new light on some aspect of computing." - Francis Sullivan Your algorithmic toolbox: Graph algorithms, greedy, divide and conquer, dynamic programming, reductions 2 Algorithms: Course Objectives Design strategies for algorithmic problem solving Reasoning about algorithm correctness Analysis of time and space complexity The Book Grading Assignments 40% (25% programming 15% written) Quizzes 10% Midterm 20% Final 30% Implementation create an implementation that respects the runtime analysis 1

2 Course staff Implementation Why I love Python Asa Ben-Hur (instructor) Zhisheng Xu (TA) 7 Programs will be written in Python v v v v v v v A concise and powerful interpreted language Powerful data structures v tuples, lists, dictionaries Simple, easy to learn syntax Highly readable, compact code Supports object oriented and functional programming An extensive standard library Strong support for integration with other languages (C,C++,Java) I am more productive! Machine performance vs. programmer performance Makes programming fun! image from: ftp:// LovePython.zip Which version? 2.x or 3.x? Stick with 2.x for now. Python 3 is a non-backward compatible version that removes a few warts from the language. Does anyone use python? One of the three official languages in google. Peter Norvig, Director of Research at Google: "Python has been an important part of Google since the beginning, and remains so as the system grows and evolved. Today dozens of Google engineers use Python, and we're looking for more people with skils in this language" q q How will we learn Python? Course website has links to Python tutorials and other resources Special lab times to help you get up to speed Yahoo groups, Yahoo maps % python

3 HW #1 Play with graphs and adjacency matrices Our first problem Problem: Matching residents to hospitals Each resident has a preference list of hospitals Each hospital has a preference list of residents We would like a method for assigning residents to hospitals that is stable. NRMP. (National Resident Matching Program) Matching system in use since the early 50s. Addressed issues of competition between hospitals. Handles over 38,000 residents yearly. Stable Matching Goal. Given a set of preferences among hospitals and students, design a stable matching process. Unstable pair: resident x and hospital y are an unstable pair if: x prefers y to its assigned hospital. y prefers x to one of its admitted applicants. Note that (x,y) is not part of the current matching Stable assignment. Assignment with no unstable pairs. Natural and desirable condition Simplifying the problem Matching students/companies problem a bit messy: Hospital may look for multiple residents Students looking for a single residency Maybe there are more residencies than students, or fewer residencies than students Formulate a "bare-bones" version of the problem The Stable Matching Problem Problem: Given n men and n women where Each man lists women in order of preference Each woman lists men in order of preference find a stable matching of all men and women Note that the preference lists are a strict total order favorite least favorite 1 st 2 nd 3 rd Xavier Amy Bertha Clare Yancey Bertha Amy Clare Zeus Amy Bertha Clare favorite least favorite 1 st 2 nd 3 rd Amy Yancey Xavier Zeus Bertha Xavier Yancey Zeus Clare Xavier Yancey Zeus Stable Matching Problem Perfect matching: Each man matched with exactly one woman, and each woman matched with exactly one man. Stability: no incentive for some pair to undermine the assignment. A pair (m,w) is unstable if man m and woman w prefer each other to current partners. Unstable pair (m, w) can improve their situation. Stable matching: perfect matching with no unstable pairs. Stable matching problem. Given the preference lists of n men and n women, find a stable matching if one exists. Men s Preference Profile Women s Preference Profile

4 Formulation Men: M={m 1,...,m n } Women: W={w 1,...,w n } The cartesian product MxW is the set of all ordered pairs. A perfect matching S is a set of pairs (subset of MxW) such that each individual occurs in exactly one pair How many perfect matchings are there? Instability Given a perfect matching, eg S = { (m 1,w 1 ), (m 2,w 2 ) } But m 1 prefers w 2 and w 2 prefers m 1 (m 1,w 2 ) is an instability for S Notice (m 1,w 2 ) is not in S! S is a stable matching if: S is perfect and there is no instability wrt S Example 1 m 1 : w 1, w 2 m 2 : w 1, w 2 w 1 : m 1, m 2 w 2 : m 1, m 2 What are the perfect matchings? 20 Example 1 Example 1 Example 2 m 1 : w 1, w 2 m 2 : w 1, w 2 w 1 : m 1, m 2 w 2 : m 1, m 2 1. { (m 1,w 1 ), (m 2,w 2 ) } 2. { (m 1,w 2 ), (m 2,w 1 ) } which is stable/unstable? m 1 : w 1, w 2 m 2 : w 1, w 2 w 1 : m 1, m 2 w 2 : m 1, m 2 1. { (m 1,w 1 ), (m 2,w 2 ) } stable 2. { (m 1,w 2 ), (m 2,w 1 ) } unstable, WHY? m 1 : w 1, w 2 m 2 : w 2, w 1 w 1 : m 2, m 1 w 2 : m 1, m 2 1. { (m 1,w 1 ), (m 2,w 2 ) } 2. { (m 1,w 2 ), (m 2,w 1 ) } which is / are unstable/stable? Conclusion? 24 4

5 A naïve algorithm for S in the set of all perfect matchings : if S is stable : return S return None A naïve algorithm for S in the set of all perfect matchings : if S is stable : return S return None Is this algorithm correct? What is its running time? A couple of questions Given a preference list, does a stable matching exist? Can we efficiently construct a stable matching if there is one? initially: no match Towards an algorithm An unmatched man m proposes to the woman w highest on his list. Will this be part of a stable matching? initially: no match Towards an algorithm An unmatched man m proposes to the woman w highest on his list. Will this be part of a stable matching? Not necessarily: w may like some m better So w and m will be in a temporary state of engagement. w is prepared to change her mind when a man higher on her list proposes While not everyone is matched An unmatched man m proposes to the woman w highest on his list that he hasn't proposed to yet. If w is free, they become engaged If w is engaged to m : If w prefers m over m, m stays free If w prefers m over m 5

6 The Gayle-Shapley algorithm 1, m becomes free 1 D. Gale and L. S. Shapley: "College Admissions and the Stability of Marriage", American Mathematical Monthly 69, 9-14, Victor Bertha Wyatt Diane Bertha Amy Clare Erika Xavier Bertha Erika Clare Diane Amy Yancey Amy Diane Clare Bertha Erika Zeus Bertha Diane Amy Erika Clare Bertha Xavier Men s Preference Profile 0 th 1 st 2 nd 3 rd 4 th Amy Diane Erika Clare Women s Preference Profile 0 th 1 st 2 nd 3 rd 4 th Amy Zeus Victor Wyatt Wyatt Yancey Victor Zeus Clare Wyatt Xavier Yancey Zeus Victor Diane Victor Zeus Yancey Yancey Xavier Xavier Wyatt Erika Yancey Wyatt Zeus Xavier Victor DEMO The Gayle-Shapley algorithm 1, m becomes free A few non-obvious questions: How long does it take? Does the algorithm return a stable matching? Does it even return a perfect matching? 1 D. Gale and L. S. Shapley: "College Admissions and the Stability of Marriage", American Mathematical Monthly 69, 9-14, Observations Observations Observations, m becomes free, m becomes free, m becomes free Each woman remains engaged from the first proposal and the sequence of partners gets better Each man proposes to less and less preferred women and will not propose to the same woman twice Claim. The algorithm terminates after at most n 2 iterations of the while loop. Claim. The algorithm terminates after at most n 2 iterations of the while loop. At each iteration a man proposes (only once) to a woman he has never proposed to, and there are only n 2 possible pairs (m,w)

7 8/20/12 Observations Proof of Correctness: Stability Summary, m becomes free When the loop terminates, the matching is perfect Proof: By contradiction. Assume there is a free man, m. Because the loop terminates, m proposed to all women But then all women are engaged, hence there is no free man àcontradiction 37 Claim. No unstable pairs. Proof. Consider an arbitrary pair (m, w) which is not in the Gale-Shapley matching S*. men propose in decreasing S* order of preference m, w Case 1: m never proposed to w. m, w m prefers his GS partner to w (m, w) is stable.... Case 2: m proposed to w. w rejected m (right away or later) w prefers her GS partner to m. (m, w) is stable. women only trade up In either case (m, w) is not an instability wrt S* 38 Stable matching problem. Given n men and n women and their preferences, find a stable matching if one exists. Gale-Shapley algorithm. Guaranteed to find a stable matching for any problem instance. Q. How to implement the GS algorithm efficiently? Q. If there are multiple stable matchings, which one does GS find? 39 Which solution? Symmetry Non-determinism m 1 : w 1, w 2 m 2 : w 2, w 1 w 1 : m 2, m 1 w 2 : m 1, m 2 The stable matching problem is symmetric wrt to men and women, but the GS algorithm is asymmetric. q Notice the following line in the GS algorithm: Two stable solutions 1: { (m 1,w 1 ), (m 2,w 2 ) } 2: { (m 1,w 2 ), (m 2,w 1 ) } GS will always find one of them (which?) There is a certain unfairness in the algorithm: If all men list different women as their first choice, they will end up with their first choice, regardless of the women's preferences. q The algorithm does not specify WHICH Still, it can be shown that all executions of the algorithm find the same stable matching (see book) When will the other be found?

8 Extensions: Matching Residents to Hospitals In this setting: Men hospitals, Women med school residents. Variant 1. Some participants declare others as unacceptable. Representative Problems Remember the problem solving paradigm q Formulate it with precision (usually using mathematical concepts, such as sets, relations, and graphs) Variant 2. Unequal number of men and women. resident A unwilling to work in Cleveland q Design an algorithm Variant 3. Limited polygamy. hospital X wants to hire 3 residents It has been proven that lying does not get you a better match! q q q Prove its correctness Analyze its complexity Implement respecting the derived complexity Variant 4. The stable roommates problem Interval Scheduling You have a resource (hotel room, printer, lecture room, telescope, manufacturing facility...) There are requests to use the resource in the form of start time s i and finish time f i, such that s i <f i Objective: grant as many requests as possible. Two requests i and j are compatible if they don't overlap, ie f i <=s j or f j <=s i Interval Scheduling Input. Set of jobs with start times and finish times. Goal. Find maximum cardinality subset of compatible jobs. b a c d e f g Algorithmic Approach The interval scheduling problem is amenable to a very simple solution. Now that you know this, can you think of it? Hint: Think how to pick a first interval while preserving the longest possible free time h Time 47 8

9 Weighted Interval Scheduling Input. Set of jobs with start times, finish times, and weights. Goal. Find maximum weight subset of compatible jobs Time Bipartite Matching Stable matching was defined as matching elements of two disjoint sets. We can express this in terms of graphs. A graph is bipartite if its nodes can be partitioned in two sets X and Y, such that the edges go from one x in X to a y in Y Bipartite Matching Input. Bipartite graph. Goal. Find maximum cardinality matching. Matching in bipartite graphs can model assignment problems, e.g., assigning jobs to machines, where an edge between a job j and a machine m indicates that m can do job j, or professors and courses. How is this different from the stable matching problem? A B C D 4 E Independent Set Input. Graph. Goal. Find maximum cardinality independent set. 1 subset of nodes such that no two are joined by an edge 2 v v v Independent set problem There is no known efficient way to solve the independent set problem. What does that mean? The only solution we have so far is trying all sub sets and finding the largest independent one. q q q Representative Problems / Complexities Interval scheduling: n log(n) greedy algorithm. Weighted interval scheduling: n log(n) dynamic programming algorithm. Bipartite matching: polynomial max-flow based algorithm v How many sub sets of a set of n nodes are there? q Independent set: NP-complete (no known polynomial algorithm exists). 6 Can you formulate interval scheduling as an independent set problem? 7 q Competitive facility location (see book): PSPACEcomplete

10 Algorithm Algorithm: effective procedure mapping input to output effective: unambiguous, executable Turing defined it as: "like a Turing machine" program = effective procedure Is there an algorithm for every possible problem? Ulam's problem def f(n) : if (n==1) return 1 elif (odd(n)) return f(3*n+1) return f(n/2) Steps in running f(n) for a few values of n: 1 2, 1, 3, 10, 5, 16, 8, 4, 2, 1 4, 2, 1 5, 16, 8, 4, 2, 1 6, 3, 10, 5, 16, 8, 4, 2, 1 7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1 8, 4, 2, 1 9, 28, 14, 7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1 10, 5, 16, 8, 4, 2, 1 Does f(n) always stop? Ulam's problem def f(n) : if (n==1) return 1 elif (odd(n)) return f(3*n+1) return f(n/2) Nobody has found an n for which f does not stop Nobody has found a proof (so there can be no algorithm deciding this.) A generalization of this problem has been proven to be undecidable. A problem P is undecidable, if there is no algorithm that produces P(x) for every possible input x The Halting Problem is undecidable Given a program P and input x will P stop on x? We can prove (cs420): the halting problem is undecidable i.e. there is no algorithm Halt(P,x) that for any program P and input x decides whether P stops on x. 10