Partition of weighted sets (problems with numbers)

TOPICS IN ALGORITHMS http://eclass.aueb.gr/courses/inf7/ Spring 27 I. ΜILIS Partition of weighted sets (problems with numbers) AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS

Weighted set problems SUBSET SUM I: objects S={,,n}, positive integer weights w i, i=,,n, positive integer W Q: is there A Í S s.t. = W? PARTITION I: objects S={,,n}, positive integer weights w i, i=,,n Q: is there A Í S s.t? - KNAPSACK I: objects S={,,n}, positive integer weights w i, i=,,n, values v i, i=,,n, positive integer W å Q: find A Í S s.t. w W and is maximized. i iîa AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 2

BIN PACKING Weighted set problems I: objects S={,,n}, positive integer weights w i, i=,,n, positive integer W Q: find a partition of S into s.t. and m is minimized å iî A j wi W, j =,2,..., m SCHEDULING (P C max ) I: objects S={,,n}, positive integer weights w i, i=,,n, positive integer m Q: find a partition of S into s.t. is minimized AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 3

Tree of reductions (partial) Cook s Th. " Π Î NP SAT 3-SAT MAX 2-SAT CLIQUE / IS / VC HP / HC 3-DM 3- GC TSP / Δ -TSP ZOE SUBSET SUM PARTITION - KNAPSACK SCHEDULING BIN PACKING Polynomial Γενίκευση του Π Πρόβλημα Π Ειδική περίπτωση του Π NP -complete ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 4

Matching problems Matching in a graph G=(V,E): A subset M ÍE of edges s.t. no two edges in M have a vertex in common. MATCHING INSTANCE: A graph G=(V,E) QUESTION: Find a maximal/maximum/perfect matching in G Maximal: it is not a subset of another matching, i.e. it can not be extended Maximum: a matching of maximum cardinality, M, i.e. a maximum maximal matching Perfect: a matching of cardinality M =n/2 (it is defined only for graphs with even # of vertices) All matching problems are polynomial, even their weighted versions ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 5

Matching problems PERFECT BIPARTITE MATCHING INSTANCE: Bipartite graph B=(B, G, E), with B = G =m, edges E Í B G QUESTION: Is there a subset of edges M Í Ε with M = m such that for each pair (b,g), (b',g') Î M it holds that b¹b', g¹g'? All bipartite matching problems are polynomial ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 6

Matching problems A generalization of Bipartite matching: 3-DIMENSIONAL MATCHING (3-DM) INSTANCE: Sets B, G, P, with B = G = P =m, a set of triples T Í B G P (i.e. a tripartite graph) QUESTION: Is there a subset M Í T with M = m such that for each pair (b,g,p), (b',g',p') Î M it holds that b¹b', g¹g', p¹p'? 3-DM is NP-complete ( 3-SAT p 3-DM ) ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 7

Zero-One Equations (ZOE) η διατύπωση ΕΙΣΟΔΟΣ: Σύστημα m εξισώσεων όπου α ij Î {,}, i m, j n ΕΡΩΤΗΣΗ: Υπάρχει λύση του συστήματος x,x 2,..., x n τ.ω. x i Î {,}, j n ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 8 2 2 2 2 22 2 2 2 = + + + = + + + = + + + n mn m m n n n n x a x a x a x a x a x a x a x a x a!!!!

Zero-One Equations (ZOE) 2 η διατύπωση ΕΙΣΟΔΟΣ: - πίνακας Α διαστάσεων m x n ΕΡΩΤΗΣΗ: Υπάρχει - διάνυσμα x = [x,x 2,..., x n ] τ.ω. Α x= ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 9 û ù ë é = û ù ë é û ù ë é 5 4 3 2 x x x x x

Zero-One Equations (ZOE) 3 η διατύπωση ΕΙΣΟΔΟΣ: - πίνακας Α διαστάσεων m x n ΕΡΩΤΗΣΗ: Υπάρχει υποσύνολο στηλών του Α που αν προστεθούν μεταξύ τους δίνουν σε όλες τις γραμμές ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II û ù ë é

3-DM p ZOE ZOE is NP-complete 3-DM: m αγόρια, m κορίτσια, m κατοικίδια, και n τριάδες Κατασκευάζουμε πίνακα Α με: 3m γραμμές (μία για κάθε αγόρι, κορίτσι, κατοικίδιο) n στήλες (μία για κάθε τριάδα) Κάθε στήλη/τριάδα έχει : - τρία στοιχεία (στις γραμμές αυτά που αντιστοιχούν στο αγόρι, κορίτσι, κατοικίδιο της τριάδας) - όλα τα άλλα στοιχεία Ένα 3-DM αντιστοιχεί σε m στήλες/τριάδες του πίνακα Α που αν προστεθούν μεταξύ τους δίνουν σε όλες τις γραμμές (οι γραμμές του Α είναι 3m και κάθε στήλη έχει τρία, άρα πρέπει να επιλεγούν m στήλες για να πάρουμε από την πρόσθεσή τους σε όλες τις γραμμές) ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II

3-DM p ZOE ZOE is NP-complete - παράδειγμα 5 τριάδες Al Bob Chet Alice Beatrice Carol Armadillo Bobcat Canary To 3-DM έχει απάντηση ΝΑΙ εάνν το ΖΟΕ έχει απάντηση ΝΑΙ ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 2

Tree of reductions (partial) Cook s Th. " Π Î NP SAT 3-SAT MAX 2-SAT CLIQUE / IS / VC HP / HC 3-DM 3- GC TSP / Δ -TSP ZOE SUBSET SUM PARTITION - KNAPSACK SCHEDULING BIN PACKING Polynomial Γενίκευση του Π Πρόβλημα Π Ειδική περίπτωση του Π NP -complete ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 3

Problems with numbers SUBSET SUM I: items S={,,n}, integer weights w i >, i=,,n, integer W> Q: is there A Í S s.t. = W? PARTITION I: items S={,,n}, integer weights w i >, i=,,n Q: is there A Í S s.t? - KNAPSACK I: items S={,,n}, weights w i >, values v i >, i=,,n, W>, all integers å Q: find A Í S s.t. w W and is maximized iîa i ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 4

SUBSET-SUM is NP-complete ZOE SUBSET SUM w, w 2,, w n W Υπάρχει υποσύνολο στηλών που αν προστεθούν μεταξύ τους, κάνουν σε όλες τις γραμμές? Υπάρχει υποσύνολο αριθμών που αν προστεθούν μεταξύ τους, κάνουν W? Αναγωγή? ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 5

MSB SUBSET-SUM is NP-complete ZOE 8 5 4 8 3 Θεωρούμε τις στήλες δυαδικούς ακεραίους (MSB πάνω) Υπάρχει υποσύνολο των ακεραίων 8, 5, 4, 8 που κάνει 2 = 3? NAI! Είναι η αναγωγή σωστή? SUBSET SUM w, w 2,, w n W Υπάρχει υποσύνολο αριθμών που αν προστεθούν μεταξύ τους, κάνουν W? ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 6

SUBSET-SUM is NP-complete Είναι η αναγωγή σωστή? Δυστυχώς ΟΧΙ! Το άθροισμα δυαδικών αριθμών n bits μπορεί να ΕΙΝΑΙ 2 n - ΑΛΛΑ το άθροισμα των αντίστοιχων διανυσμάτων να ΜΗΝ ΕΙΝΑΙ. Π.χ.. γιατί Δυαδική πρόσθεση + + ¹ 5 6 2 = 3! Το SUBSET SUM έχει απάντηση ΝΑΙ αλλά το ΖΟΕ έχει απάντηση ΟΧΙ ΓΙΑΤΙ? ΚΡΑΤΟΥΜΕΝΟ! ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 7

SUBSET-SUM is NP-complete Μπορούμε να σώσουμε την αναγωγή? ΝΑΙ! Σκεφτείτε κάθε στήλη του Α σαν ακέραιο στο σύστημα με βάση το n+! Τότε δεν θα υπάρχει κρατούμενο και η αναγωγή είναι σωστή: Το SUBSET SUM έχει απάντηση ΝΑΙ εάνν το ΖΟΕ έχει απάντηση ΝΑΙ ΑΛΛΑ ΠΡΟΣΟΧΗ!!! m W = n+ = Ο(n m ) και επίσης w i = Ο(n m ) Έστω Ι ένα στιγμιότυπο του ΖΟΕ και Ι το στιγμιότυπο του SUBSET SUM τότε Ι = Ο(exp( I ) Γι αυτό το SUBSET SUM έχει αλγόριθμο πολυπλοκότητας Ο(nW) (ψευτο-πολυωνυμικό) ΑΛΓΟΡΙΘΜΟΙ - ΑΝΟΙΞΗ 26 - Ι. ΜΗΛΗΣ 6 NP-COMPLETENESS II 8

Strong NP-completeness 3-SAT, CLIQUE, VC, IS, 3-GC, HP, HC, TSP: The reductions for all these problems use only small (polynomial) integers (names of nodes, indices, even in TSP we used only and 2) All these problems are strongly NP-complete ZOE p * SUBSET SUM: * This reduction creates exponentially large integers (w i s and W)! Τhe size of the SUBSET SUM s instance constructed is exponential in the size of the ZOE s instance! ( If I={n}, I = log n and I = {2 n }, I =n, then I =n=2 log n = 2 I ) SUBSET SUM is not strongly NP-complete but it is NP-complete. There is an O(nW) algorithm for SUBSET SUM SUBSET SUM is O(poly) if W is O(poly( I ))! AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 9

Subset Sum SUBSET SUM I: a set S={a, a 2,..., a n } of n positive integers and integer B Q: is there a subset A Í S such that å Î a i A i = B? BRUTE FORCE there are 2 n possible combinations of n items Go through all combinations; stop in the first one such that å a B ; otherwise report NO Complexity: O(n2 n i Î A i = ) Can we do better? AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 2

Subproblem - Recursive formula Let S i ={a, a 2,,a i }, Subset Sum - DP Subproblem: M (i, b), i n, b B, M (i, b): is there a subset A Í{a, a 2,..., a i } s.t.? If a i > b, then the the answer for M (i, b) is YES iff å Î there is a subset A Í S i- s.t. a b i A i = If a i b, then the the answer for M (i, b) is YES iff either there is a subset A Í S i- s.t. a or there is a subset A Í S i- s.t. å iîa If i=, then the answer is YES if B= and NO otherwise (Base case for i=) a å b i Î A i = å a i Î A i = b - ai i = b AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 2

Subset Sum - DP Recursive formula M ( i, b) = ìtrue ï ï ïfalse ï í ïm ( i -,b) ï ï ï M ( i -,b) ï î OR if i = and b = if i = and b if i ³ and if i ³ and i a M ( i -,b - a ) a i i > > b b AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 22

There are only nb subproblems Subset Sum - DP Fill a n x B Boolean matrix M[ i, b], i n, b B SubsetSum (S,n,B) M[,]= True; for b = to B do M[,b]=False; for i = to n do for b = to B do if a i > b then M[i,b]=M[i-,b] else M[i,b]= M[i-,b] OR M[i-,b-a i ] Complexity: O(nB) AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 23

Example: (4,,2,5), B=6 Subset Sum - DP b 2 3 4 5 6 i /a i /- T F F F F F F /4 T F F F T F F 2/ T T F F T T F 3/2 T T T T T T T 4/5 T T T T T T T if a i > b then M(i,b)=M(i-,b) else M(i,b)= M(i-,b) OR M(i-,b-a i ) AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 24

Subset Sum Complexity: O(nB) I = log a + log a 2 + + log a n + log B (n+) log B = O(n log B) Hence, O(nB) is O(exp(I)) NOT POLYNOMIAL But, N(I) =B, that is O(nB) is O(poly(N(I)) PSEUDO-POLYNOMIAL AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / - INTRO 25

Strong NP-completeness 3-SAT, CLIQUE, VC, IS, 3-GC, HP, HC, TSP: The reductions for all these problems use only small (polynomial) integers (names of nodes, indices, even in TSP we used only and 2) All these problems are strongly NP-complete ZOE p * SUBSET SUM: * This reduction creates exponentially large integers (w i s and W)! Τhe size of the SUBSET SUM s instance constructed is exponential in the size of the ZOE s instance! ( If I={n}, I = log n and I = {2 n }, I =n, then I =n=2 log n = 2 I ) SUBSET SUM is not strongly NP-complete but it is NP-complete. There is an O(nW) algorithm for SUBSET SUM SUBSET SUM is O(poly) if W is O(poly( I ))! AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 26

Strong NP-completeness and pseudo-polynomial algorithms A problem Π is strongly NP-complete if it remains NP-complete even if any instance of length I is restricted to contain integers at most O(poly( I )) or it remains NP-complete even if its instances are coded in unary Let N(I) be the largest number appearing in an instance of a problem An algorithm is a pseudo-polynomial one if it is polynomial in I and N(I) Unless P=NP, there is no pseudo-polynomial algorithm for strongly NPcomplete problems (next slide) For problems that are NP-complete, but not strongly NP-complete there is a pseudo-polynomial algorithm (usually a dynamic programming one) AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 27

Strong NP-completeness and pseudo-polynomial algorithms Let : I be an instance of a problem Π, of size I N(I) be the largest number in I p(n) be a polynomial Π p(n) be Π restricted to instances for which N(I) p( I ) We say that Π is strongly NP-complete if Π p(n) is NP-complete Th. Unless P=NP, there is no pseudo-polynomial algorithm for a strongly NP-complete problem Π Proof: Suppose that there exists such a pseudo-polynomial algorithm Q for Π Q solves any instance of Π in q( I, N(I) ) time; q: a polynomial Q solves Π p(n) in q( I, p( I ) ) time, that is polynomial in I P=NP! AUEB / DoI / TOPICS IN ALGORITHMS / Spring 27 / I. MILIS / 6 - PARTITIONS 28