Permuting the underlying k variables by a permutation Ï. (ζÏ)i = ζÏ(i). (ζÏ)ij = ζÏ(i)Ï(j). - Multiplying each
On Approximation Resistance of Predicates
Algorithm Hardness
Subhash Khot New York University
zero-sum game
Joint work with Madhur Tulsiani and Pratik Worah
value = 0 ?
Overview of the Talk
- Introduction and Survey - Basics: Algorithmic Side - Basics: Hardness Side - Characterization of Approximation Resistance - Conclusion
Constraint Satisfaction Problems (CSP)
Constraint Satisfaction Problems (CSP) - Boolean ({−1, 1}-valued) variables and constraints.
Constraint Satisfaction Problems (CSP) - Boolean ({−1, 1}-valued) variables and constraints. - All constraints of same type, variable negation allowed.
3-SAT x1 ∨ x22 ∨ x 19 x3 ∨ x 9 ∨ x23 x5 ∨ x 7 ∨ x 9 .. .
Constraint Satisfaction Problems (CSP) - Boolean ({−1, 1}-valued) variables and constraints. - All constraints of same type, variable negation allowed.
3-SAT
CUT x1
x1 ∨ x22 ∨ x 19
x2
x3 ∨ x 9 ∨ x23 x5 ∨ x 7 ∨ x 9 .. .
x4 x5
x7 x6
x3
Constraint Satisfaction Problems (CSP) - Boolean ({−1, 1}-valued) variables and constraints. - All constraints of same type, variable negation allowed.
3-SAT
CUT x1
x1 ∨ x22 ∨ x 19
x2
x3 ∨ x 9 ∨ x23 x5 ∨ x 7 ∨ x 9 .. .
x4 x5
x7 x6
x3
x1 = 6 x2 x2 = 6 x5 x3 = 6 x4 .. .
Constraint Satisfaction Problems (CSP) 3-XOR: Linear equations modulo 2 (in ±1 variables)
Constraint Satisfaction Problems (CSP) 3-XOR: Linear equations modulo 2 (in ±1 variables) x5 · x9 · x16 = 1 x6 · x12 · x22 = −1 x7 · x8 · x15 = −1 .. .
Constraint Satisfaction Problems (CSP) 3-XOR: Linear equations modulo 2 (in ±1 variables) x5 · x9 · x16 = 1
x5 · x9 · x16 = 1
x6 · x12 · x22 = −1
x6 · x 12 · x22 = 1
x7 · x8 · x15 = −1 .. .
−→
x7 · x8 · x 15 = 1 .. .
Constraint Satisfaction Problems (CSP) 3-XOR: Linear equations modulo 2 (in ±1 variables) x5 · x9 · x16 = 1
x5 · x9 · x16 = 1
x6 · x12 · x22 = −1
x6 · x 12 · x22 = 1
x7 · x8 · x15 = −1 .. .
−→
x7 · x8 · x 15 = 1 .. .
CSP(f ) : Given predicate f : {−1, 1}k → {0, 1}. Each constraint is f applied to some k (possibly negated) variables.
Constraint Satisfaction Problems (CSP) 3-XOR: Linear equations modulo 2 (in ±1 variables) x5 · x9 · x16 = 1
x5 · x9 · x16 = 1
x6 · x12 · x22 = −1
x6 · x 12 · x22 = 1
x7 · x8 · x15 = −1 .. .
−→
x7 · x8 · x 15 = 1 .. .
CSP(f ) : Given predicate f : {−1, 1}k → {0, 1}. Each constraint is f applied to some k (possibly negated) variables. C ≡ f xi1 , x i2 , . . . , x ik−1 , xik
Constraint Satisfaction Problems (CSP) 3-XOR: Linear equations modulo 2 (in ±1 variables) x5 · x9 · x16 = 1
x5 · x9 · x16 = 1
x6 · x12 · x22 = −1
x6 · x 12 · x22 = 1
x7 · x8 · x15 = −1 .. .
−→
x7 · x8 · x 15 = 1 .. .
CSP(f ) : Given predicate f : {−1, 1}k → {0, 1}. Each constraint is f applied to some k (possibly negated) variables. C ≡ f xi1 , x i2 , . . . , x ik−1 , xik
k fixed constant. Number of variables and constraints → ∞.
Constraint Satisfaction Problems (CSP) x1 x2
x4
x5
x7
x3
x6 CSP(f ) : Given predicate f : {−1, 1}k → {0, 1}. Each constraint is f applied to some k (possibly negated) variables. C ≡ f xi1 , x i2 , . . . , x ik−1 , xik
k fixed constant. Number of variables and constraints → ∞.
Satisfiability vs Approximability
- [Schaefer 78], Dichotomy Theorem: For every predicate f : {−1, 1}k → {0, 1}, deciding the satisfiability of CSP(f ) is either in P or is NP-hard.
Satisfiability vs Approximability
- [Schaefer 78], Dichotomy Theorem: For every predicate f : {−1, 1}k → {0, 1}, deciding the satisfiability of CSP(f ) is either in P or is NP-hard. - Approximability: given a (1 − o(1))-satisfiable instance, how good an assignment can one find efficiently?
Satisfiability vs Approximability
- [Schaefer 78], Dichotomy Theorem: For every predicate f : {−1, 1}k → {0, 1}, deciding the satisfiability of CSP(f ) is either in P or is NP-hard. - Approximability: given a (1 − o(1))-satisfiable instance, how good an assignment can one find efficiently? - Let ρ(f ) =
|f −1 (1)| 2k
= Ex [f (x )].
Satisfiability vs Approximability
- [Schaefer 78], Dichotomy Theorem: For every predicate f : {−1, 1}k → {0, 1}, deciding the satisfiability of CSP(f ) is either in P or is NP-hard. - Approximability: given a (1 − o(1))-satisfiable instance, how good an assignment can one find efficiently? - Let ρ(f ) =
|f −1 (1)| 2k
- ρ(3-SAT) = 7/8,
= Ex [f (x )]. ρ(3-XOR) = 1/2.
Satisfiability vs Approximability
- [Schaefer 78], Dichotomy Theorem: For every predicate f : {−1, 1}k → {0, 1}, deciding the satisfiability of CSP(f ) is either in P or is NP-hard. - Approximability: given a (1 − o(1))-satisfiable instance, how good an assignment can one find efficiently? - Let ρ(f ) =
|f −1 (1)| 2k
- ρ(3-SAT) = 7/8,
= Ex [f (x )]. ρ(3-XOR) = 1/2.
- Observation: a random assignment to CSP(f ) satisfies ρ(f ) fraction of the constraints.
Approximable and Approximation Resistant Predicates - f : {−1, 1}k → {0, 1}.
ρ(f ) =
|f −1 (1)| . 2k
- Definition: f is called approximable if for some ε > 0, there is an efficient algorithm that given a (1 − ε)-satisfiable instance of CSP(f ), outputs a (ρ(f ) + ε)-satisfying assignment.
Approximable and Approximation Resistant Predicates - f : {−1, 1}k → {0, 1}.
ρ(f ) =
|f −1 (1)| . 2k
- Definition: f is called approximable if for some ε > 0, there is an efficient algorithm that given a (1 − ε)-satisfiable instance of CSP(f ), outputs a (ρ(f ) + ε)-satisfying assignment.
- Otherwise f is called approximation resistant (A.R.).
Approximable and Approximation Resistant Predicates - f : {−1, 1}k → {0, 1}.
ρ(f ) =
|f −1 (1)| . 2k
- Definition: f is called approximable if for some ε > 0, there is an efficient algorithm that given a (1 − ε)-satisfiable instance of CSP(f ), outputs a (ρ(f ) + ε)-satisfying assignment.
- Otherwise f is called approximation resistant (A.R.). - [GW 92]: 2-SAT and 2-XOR are approximable.
Approximable and Approximation Resistant Predicates - f : {−1, 1}k → {0, 1}.
ρ(f ) =
|f −1 (1)| . 2k
- Definition: f is called approximable if for some ε > 0, there is an efficient algorithm that given a (1 − ε)-satisfiable instance of CSP(f ), outputs a (ρ(f ) + ε)-satisfying assignment.
- Otherwise f is called approximation resistant (A.R.). - [GW 92]: 2-SAT and 2-XOR are approximable. - [Håstad 97]: For k ≥ 3, k-SAT and k-XOR are A.R.
Approximation Resistance - To show that f is approximation resistant, one shows that for every ε > 0, it is NP-hard (UG-hard) to distinguish if a CSP(f ) instance is (1 − ε)-satisfiable or is at most (ρ(f ) + ε)-satisfiable.
Approximation Resistance - To show that f is approximation resistant, one shows that for every ε > 0, it is NP-hard (UG-hard) to distinguish if a CSP(f ) instance is (1 − ε)-satisfiable or is at most (ρ(f ) + ε)-satisfiable.
Approximation Resistance - To show that f is approximation resistant, one shows that for every ε > 0, it is NP-hard (UG-hard) to distinguish if a CSP(f ) instance is (1 − ε)-satisfiable or is at most (ρ(f ) + ε)-satisfiable.
- Reduction from a hard problem to the promise gap problem:
≤ ρ(f ) + ε
≥1−ε
Approximation Resistance - To show that f is approximation resistant, one shows that for every ε > 0, it is NP-hard (UG-hard) to distinguish if a CSP(f ) instance is (1 − ε)-satisfiable or is at most (ρ(f ) + ε)-satisfiable.
- Reduction from a hard problem to the promise gap problem:
≤ ρ(f ) + ε
≥1−ε
- Captures the notion that it is hard to do better than a random assignment.
Approximation Resistance: Known Results
[Håstad 97]: k-SAT and k-XOR are A.R. (k ≥ 3).
Approximation Resistance: Known Results
[Håstad 97]: k-SAT and k-XOR are A.R. (k ≥ 3). - Best hardness for 2-SAT and MAX-CUT.
Approximation Resistance: Known Results
[Håstad 97]: k-SAT and k-XOR are A.R. (k ≥ 3). - Best hardness for 2-SAT and MAX-CUT. [ST 00, EH 05]: Graph Linearity Test predicate is A.R.
Approximation Resistance: Known Results
[Håstad 97]: k-SAT and k-XOR are A.R. (k ≥ 3). - Best hardness for 2-SAT and MAX-CUT. [ST 00, EH 05]: Graph Linearity Test predicate is A.R. - Alternate proof of [Håstad 96]: Clique on n-vertex graphs is hard to approximate within n1−ε .
Approximation Resistance: Known Results
[Håstad 97]: k-SAT and k-XOR are A.R. (k ≥ 3). - Best hardness for 2-SAT and MAX-CUT. [ST 00, EH 05]: Graph Linearity Test predicate is A.R. - Alternate proof of [Håstad 96]: Clique on n-vertex graphs is hard to approximate within n1−ε . [GHS 00]: Not-All-Equal(x1 , x2 , x3 , x4 ) is A.R.
Approximation Resistance: Known Results
[Håstad 97]: k-SAT and k-XOR are A.R. (k ≥ 3). - Best hardness for 2-SAT and MAX-CUT. [ST 00, EH 05]: Graph Linearity Test predicate is A.R. - Alternate proof of [Håstad 96]: Clique on n-vertex graphs is hard to approximate within n1−ε . [GHS 00]: Not-All-Equal(x1 , x2 , x3 , x4 ) is A.R. - Hard to color 2-colorable 4-uniform hypergraphs with O(1) colors.
Approximation Resistance: Known Results [ST 06∗ , Chan 12]: Hypergraph Linearity Test predicate is A.R. q
f : {−1, 1}q+(2 −q−1) → {0,1}. x1 , . . . , xq , {yS }S⊆[q], |S|≥2 is a satisfying assignment iff yS =
Y
xi
∀S ⊆ [q], |S| ≥ 2.
i∈S
- k-ary predicate with k + 1 satisfying assignments.
Approximation Resistance: Known Results [ST 06∗ , Chan 12]: Hypergraph Linearity Test predicate is A.R. q
f : {−1, 1}q+(2 −q−1) → {0,1}. x1 , . . . , xq , {yS }S⊆[q], |S|≥2 is a satisfying assignment iff yS =
Y
xi
∀S ⊆ [q], |S| ≥ 2.
i∈S
- k-ary predicate with k + 1 satisfying assignments. - Optimal hardness for Max-k-CSP.
Approximation Resistance: Known Results [ST 06∗ , Chan 12]: Hypergraph Linearity Test predicate is A.R. q
f : {−1, 1}q+(2 −q−1) → {0,1}. x1 , . . . , xq , {yS }S⊆[q], |S|≥2 is a satisfying assignment iff yS =
Y
xi
∀S ⊆ [q], |S| ≥ 2.
i∈S
- k-ary predicate with k + 1 satisfying assignments. - Optimal hardness for Max-k-CSP. - Hard: independent set on degree d graphs within O
d log4 d
.
Approximation Resistance: Known Results [ST 06∗ , Chan 12]: Hypergraph Linearity Test predicate is A.R. q
f : {−1, 1}q+(2 −q−1) → {0,1}. x1 , . . . , xq , {yS }S⊆[q], |S|≥2 is a satisfying assignment iff yS =
Y
xi
∀S ⊆ [q], |S| ≥ 2.
i∈S
- k-ary predicate with k + 1 satisfying assignments. - Optimal hardness for Max-k-CSP. - Hard: independent set on degree d graphs within O - Hard to color k-colorable graphs with 2Ω(k
1/3 )
d log4 d
colors [H 13].
.
Approximation Resistance: Known Results [ST 06∗ , Chan 12]: Hypergraph Linearity Test predicate is A.R. q
f : {−1, 1}q+(2 −q−1) → {0,1}. x1 , . . . , xq , {yS }S⊆[q], |S|≥2 is a satisfying assignment iff yS =
Y
xi
∀S ⊆ [q], |S| ≥ 2.
i∈S
- k-ary predicate with k + 1 satisfying assignments. - Optimal hardness for Max-k-CSP. - Hard: independent set on degree d graphs within O 1/3
d log4 d
- Hard to color k-colorable graphs with 2Ω(k ) colors [H 13]. √ ˜ - O(1/ k) soundness for Label Cover with k labels.
.
Towards Characterization of Approximation Resistance A characterization is a necessary and sufficient condition for a predicate f : {−1, 1}k → {0, 1} to be A.R. - [Håstad 97, Zwick 98] A predicate on 3 variables is A.R. iff it is implied by 3-XOR (up to variable negations).
Towards Characterization of Approximation Resistance A characterization is a necessary and sufficient condition for a predicate f : {−1, 1}k → {0, 1} to be A.R. - [Håstad 97, Zwick 98] A predicate on 3 variables is A.R. iff it is implied by 3-XOR (up to variable negations). - [Hast 05]: Out of 400 predicates on 4 variables, classified 79 to be approximation resistant and 275 to be not so.
Towards Characterization of Approximation Resistance A characterization is a necessary and sufficient condition for a predicate f : {−1, 1}k → {0, 1} to be A.R. - [Håstad 97, Zwick 98] A predicate on 3 variables is A.R. iff it is implied by 3-XOR (up to variable negations). - [Hast 05]: Out of 400 predicates on 4 variables, classified 79 to be approximation resistant and 275 to be not so. - [Chan 12]: Sufficient condition: f −1 (1) is non-degenerate subgroup of {−1, 1}k (every pair of co-ordinates is independent).
Towards Characterization of Approximation Resistance A characterization is a necessary and sufficient condition for a predicate f : {−1, 1}k → {0, 1} to be A.R. - [Håstad 97, Zwick 98] A predicate on 3 variables is A.R. iff it is implied by 3-XOR (up to variable negations). - [Hast 05]: Out of 400 predicates on 4 variables, classified 79 to be approximation resistant and 275 to be not so. - [Chan 12]: Sufficient condition: f −1 (1) is non-degenerate subgroup of {−1, 1}k (every pair of co-ordinates is independent). - [AM 09∗ ]: Sufficient condition: There exists a balanced and pairwise independent distribution on {−1, 1}k supported on f −1 (1).
Towards Characterization of Approximation Resistance A characterization is a necessary and sufficient condition for a predicate f : {−1, 1}k → {0, 1} to be A.R. - [Håstad 97, Zwick 98] A predicate on 3 variables is A.R. iff it is implied by 3-XOR (up to variable negations). - [Hast 05]: Out of 400 predicates on 4 variables, classified 79 to be approximation resistant and 275 to be not so. - [Chan 12]: Sufficient condition: f −1 (1) is non-degenerate subgroup of {−1, 1}k (every pair of co-ordinates is independent). - [AM 09∗ ]: Sufficient condition: There exists a balanced and pairwise independent distribution on {−1, 1}k supported on f −1 (1). - [AK 13∗ ]: Characterization when f is even and instance is k-partite.
Overview of the Talk
- Introduction and Survey - Basics: Algorithmic Side - Basics: Hardness Side - Characterization of Approximation Resistance - Conclusion
Goemans-Williamson’s SDP Relaxation for MAX-CUT MAX-CUT: Find a cut in the graph G([n], E ) that maximizes the fraction of edges cut.
max x1 ,...,xn ∈{−1,1}
1 X 1 − xi xj . |E | (i, j)∈E 2
SDP-Relaxation:
max v1 ,...,vn
Subject to
1 X 1 − vi · vj . |E | (i, j)∈E 2 kvi k = 1
∀ i ∈ [n].
SDP Solution for (Near-)Satisfiable CSP(f ) Instance Given: CSP(f ) instance that is (near-)satisfiable. Variables x1 , . . . , xn and constraints C1 , . . . , Cm . x1 x2
x4
x5
x7 x6
x3
SDP Solution for (Near-)Satisfiable CSP(f ) Instance Given: CSP(f ) instance that is (near-)satisfiable. Variables x1 , . . . , xn and constraints C1 , . . . , Cm . x1 x2
x4
x5
x7 x6
SDP-Solution . . .
x3
SDP Solution for (Near-)Satisfiable CSP(f ) Instance Given: CSP(f ) instance that is (near-)satisfiable. Variables x1 , . . . , xn and constraints C1 , . . . , Cm . x1 x2
x4
x5
x7 x6
SDP-Solution . . . Notation:
x3
SDP Solution for (Near-)Satisfiable CSP(f ) Instance Given: CSP(f ) instance that is (near-)satisfiable. Variables x1 , . . . , xn and constraints C1 , . . . , Cm . x1 x2
x4
x5
x7
x3
x6 SDP-Solution . . . Notation: - µ: distribution supported on f −1 (1) ⊆ {−1, 1}k .
SDP Solution for (Near-)Satisfiable CSP(f ) Instance Given: CSP(f ) instance that is (near-)satisfiable. Variables x1 , . . . , xn and constraints C1 , . . . , Cm . x1 x2
x4
x5
x7
x3
x6 SDP-Solution . . . Notation: - µ: distribution supported on f −1 (1) ⊆ {−1, 1}k . - µij = Ez∼µ [zj zj ] are the
k 2
second moments of µ.
SDP Solution for (Near-)Satisfiable CSP(f ) Instance µ: distribution supported on f −1 (1), µij = Ez∼µ [zj zj ].
SDP Solution for (Near-)Satisfiable CSP(f ) Instance µ: distribution supported on f −1 (1), µij = Ez∼µ [zj zj ]. SDP-Solution: v1
v2
x1
x2
x4
µ v3
x5
x7 x6
x3
SDP Solution for (Near-)Satisfiable CSP(f ) Instance µ: distribution supported on f −1 (1), µij = Ez∼µ [zj zj ]. SDP-Solution: v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 - Variables have unit vectors associated: v1 , . . . , vn .
SDP Solution for (Near-)Satisfiable CSP(f ) Instance µ: distribution supported on f −1 (1), µij = Ez∼µ [zj zj ]. SDP-Solution: v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 - Variables have unit vectors associated: v1 , . . . , vn . - Every constraint has associated local distribution: C → µ.
SDP Solution for (Near-)Satisfiable CSP(f ) Instance µ: distribution supported on f −1 (1), µij = Ez∼µ [zj zj ]. SDP-Solution: v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 - Variables have unit vectors associated: v1 , . . . , vn . - Every constraint has associated local distribution: C → µ. - Consistency: ∀i, j ∈ C ,
vi · vj = µij .
Rounding SDP Solution for CSP(f ) Instance v1
v2
x1
x2
x4
µ v3
x5
x7 x6
Rounding:
x3
Rounding SDP Solution for CSP(f ) Instance v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 Rounding: - Randomly project v1 , . . . , vn onto Rd .
vi → yi ∈ Rd .
Rounding SDP Solution for CSP(f ) Instance v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 Rounding: - Randomly project v1 , . . . , vn onto Rd . - Select a function ψ : Rd → {−1, 1}.
vi → yi ∈ Rd .
Rounding SDP Solution for CSP(f ) Instance v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 Rounding: - Randomly project v1 , . . . , vn onto Rd . - Select a function ψ : Rd → {−1, 1}. - Rounding: xi → vi → yi → ψ(yi ).
vi → yi ∈ Rd .
Rounding SDP Solution for CSP(f ) Instance v1
v2
x1
x2
x4
µ v3
x5
x7
x3
x6 Rounding: - Randomly project v1 , . . . , vn onto Rd .
vi → yi ∈ Rd .
- Select a function ψ : Rd → {−1, 1}. - Rounding: xi → vi → yi → ψ(yi ). [Rag 08∗ ]: For CSPs, this is the most general algorithm!
Overview of the Talk
- Introduction and Survey - Basics: Algorithmic Side - Basics: Hardness Side - Characterization of Approximation Resistance - Conclusion
Proving Hardness Results - Reduction from a known hard problem (Label Cover) or a conjectured hard problem (Unique Game).
Proving Hardness Results - Reduction from a known hard problem (Label Cover) or a conjectured hard problem (Unique Game). - Probabilistically Checkable Proofs (PCPs).
Proving Hardness Results - Reduction from a known hard problem (Label Cover) or a conjectured hard problem (Unique Game). - Probabilistically Checkable Proofs (PCPs). - Constructing “Gadgets" ≡ “Dictatorship Test".
Proving Hardness Results - Reduction from a known hard problem (Label Cover) or a conjectured hard problem (Unique Game). - Probabilistically Checkable Proofs (PCPs). - Constructing “Gadgets" ≡ “Dictatorship Test". - Given a boolean function, test whether it is a dictatorship function.
Proving Hardness Results - Reduction from a known hard problem (Label Cover) or a conjectured hard problem (Unique Game). - Probabilistically Checkable Proofs (PCPs). - Constructing “Gadgets" ≡ “Dictatorship Test". - Given a boolean function, test whether it is a dictatorship function. - Definition: G : {−1, 1}N → {−1, 1} is a dictatorship function if G(z1 , . . . , zN ) = zi
for some fixed
1 ≤ i ≤ N.
Dictatorship Test for CSP(f ) Fix f : {−1, 1}k → {0, 1} and µ supported on f −1 (1).
Dictatorship Test for CSP(f ) Fix f : {−1, 1}k → {0, 1} and µ supported on f −1 (1). To test if G : {−1, 1}N → {−1, 1} is a dictatorship function.
Dictatorship Test for CSP(f ) Fix f : {−1, 1}k → {0, 1} and µ supported on f −1 (1). To test if G : {−1, 1}N → {−1, 1} is a dictatorship function. Pick a random k × N matrix by picking:
Dictatorship Test for CSP(f ) Fix f : {−1, 1}k → {0, 1} and µ supported on f −1 (1). To test if G : {−1, 1}N → {−1, 1} is a dictatorship function. Pick a random k × N matrix by picking: - every column ∼ µ independently and
Dictatorship Test for CSP(f ) Fix f : {−1, 1}k → {0, 1} and µ supported on f −1 (1). To test if G : {−1, 1}N → {−1, 1} is a dictatorship function. Pick a random k × N matrix by picking: - every column ∼ µ independently and - adding small noise.
Dictatorship Test for CSP(f ) Fix f : {−1, 1}k → {0, 1} and µ supported on f −1 (1). To test if G : {−1, 1}N → {−1, 1} is a dictatorship function. Pick a random k × N matrix by picking: - every column ∼ µ independently and - adding small noise. Accept iff (G(row1 ), G(row2 ), . . . , G(rowk )) ∈ f −1 (1).
Figure: Dictatorship Test on G given f , µ.
Figure: Dictatorship Test on G given f , µ.
Completeness and Soundness of Test Completeness: If G is i th dictatorship, the test accepts with prob. 1. (G(row1 ), G(row2 ), . . . , G(rowk )) = columni ∈ f −1 (1).
Completeness and Soundness of Test Completeness: If G is i th dictatorship, the test accepts with prob. 1. (G(row1 ), G(row2 ), . . . , G(rowk )) = columni ∈ f −1 (1).
Soundness: If G is far from a dictatorship,
Completeness and Soundness of Test Completeness: If G is i th dictatorship, the test accepts with prob. 1. (G(row1 ), G(row2 ), . . . , G(rowk )) = columni ∈ f −1 (1).
Soundness: If G is far from a dictatorship, - The acceptance probability depends only on {µij }.
Completeness and Soundness of Test Completeness: If G is i th dictatorship, the test accepts with prob. 1. (G(row1 ), G(row2 ), . . . , G(rowk )) = columni ∈ f −1 (1).
Soundness: If G is far from a dictatorship, - The acceptance probability depends only on {µij }. - Choose µ appropriately so that the test accepts with probability at most ρ(f ) + o(1).
Completeness and Soundness of Test Completeness: If G is i th dictatorship, the test accepts with prob. 1. (G(row1 ), G(row2 ), . . . , G(rowk )) = columni ∈ f −1 (1).
Soundness: If G is far from a dictatorship, - The acceptance probability depends only on {µij }. - Choose µ appropriately so that the test accepts with probability at most ρ(f ) + o(1). - Possible to choose µ itself from a suitable distribution.
Completeness and Soundness of Test Completeness: If G is i th dictatorship, the test accepts with prob. 1. (G(row1 ), G(row2 ), . . . , G(rowk )) = columni ∈ f −1 (1).
Soundness: If G is far from a dictatorship, - The acceptance probability depends only on {µij }. - Choose µ appropriately so that the test accepts with probability at most ρ(f ) + o(1). - Possible to choose µ itself from a suitable distribution. Invariance Principle [Rotar0 75, MOO 05, C 05, Mossel 07].
Overview of the Talk
- Introduction and Survey - Basics: Algorithmic Side - Basics: Hardness Side - Characterization of Approximation Resistance - Conclusion
A (sort-of) Characterization by [Rag 08] and [RS 09]
- [Rag 08∗ ]: f is approximation resistant iff for every ε > 0 there exists a 1 − ε vs. ρ(f ) + ε “integrality gap" instance.
A (sort-of) Characterization by [Rag 08] and [RS 09]
- [Rag 08∗ ]: f is approximation resistant iff for every ε > 0 there exists a 1 − ε vs. ρ(f ) + ε “integrality gap" instance. - [RS 09]: For every ε > 0, this can be determined in time exp(exp(1/ε)), by a brute-force search.
A (sort-of) Characterization by [Rag 08] and [RS 09]
- [Rag 08∗ ]: f is approximation resistant iff for every ε > 0 there exists a 1 − ε vs. ρ(f ) + ε “integrality gap" instance. - [RS 09]: For every ε > 0, this can be determined in time exp(exp(1/ε)), by a brute-force search. - Gives a recursively enumerable condition.
A (sort-of) Characterization by [Rag 08] and [RS 09]
- [Rag 08∗ ]: f is approximation resistant iff for every ε > 0 there exists a 1 − ε vs. ρ(f ) + ε “integrality gap" instance. - [RS 09]: For every ε > 0, this can be determined in time exp(exp(1/ε)), by a brute-force search. - Gives a recursively enumerable condition. - But what properties of f give rise to gap instances?
A (sort-of) Characterization by [Rag 08] and [RS 09]
- [Rag 08∗ ]: f is approximation resistant iff for every ε > 0 there exists a 1 − ε vs. ρ(f ) + ε “integrality gap" instance. - [RS 09]: For every ε > 0, this can be determined in time exp(exp(1/ε)), by a brute-force search. - Gives a recursively enumerable condition. - But what properties of f give rise to gap instances? - Is it just properties of f or is the topology of the instance also important?
A (sort-of) Characterization by [Rag 08] and [RS 09]
- [Rag 08∗ ]: f is approximation resistant iff for every ε > 0 there exists a 1 − ε vs. ρ(f ) + ε “integrality gap" instance. - [RS 09]: For every ε > 0, this can be determined in time exp(exp(1/ε)), by a brute-force search. - Gives a recursively enumerable condition. - But what properties of f give rise to gap instances? - Is it just properties of f or is the topology of the instance also important? (Hint: Just f )
The Austrin-Mossel condition in a new language k
- For a distribution µ on {−1, 1}k , let ζ(µ) ∈ Rk+(2) denote the vector of first and second moments ζi = Ex ∼µ [xi ]
ζij = Ex ∼µ [xi · xj ]
The Austrin-Mossel condition in a new language k
- For a distribution µ on {−1, 1}k , let ζ(µ) ∈ Rk+(2) denote the vector of first and second moments ζi = Ex ∼µ [xi ]
ζij = Ex ∼µ [xi · xj ]
- Let C(f ) be the convex polytope C(f ) =
n
o
ζ(µ) | µ is supported on f −1 (1) .
The Austrin-Mossel condition in a new language k
- For a distribution µ on {−1, 1}k , let ζ(µ) ∈ Rk+(2) denote the vector of first and second moments ζi = Ex ∼µ [xi ]
ζij = Ex ∼µ [xi · xj ]
- Let C(f ) be the convex polytope C(f ) =
n
o
ζ(µ) | µ is supported on f −1 (1) .
- [AM 09∗ ]: f is approximation resistant if 0 ∈ C(f ).
The Austrin-Mossel condition in a new language k
- For a distribution µ on {−1, 1}k , let ζ(µ) ∈ Rk+(2) denote the vector of first and second moments ζi = Ex ∼µ [xi ]
ζij = Ex ∼µ [xi · xj ]
- Let C(f ) be the convex polytope C(f ) =
n
o
ζ(µ) | µ is supported on f −1 (1) .
- [AM 09∗ ]: f is approximation resistant if 0 ∈ C(f ). - Our condition is in terms of existence of a probability measure Λ on C(f ) with certain symmetry properties.
Transformations of a measure Λ on C(f )
- Each ζ ∈ C(f ) can be transformed by:
Transformations of a measure Λ on C(f )
- Each ζ ∈ C(f ) can be transformed by: - Permuting the underlying k variables by a permutation π (ζπ )i = ζπ(i)
(ζπ )ij = ζπ(i)π(j)
Transformations of a measure Λ on C(f )
- Each ζ ∈ C(f ) can be transformed by: - Permuting the underlying k variables by a permutation π (ζπ )i = ζπ(i)
(ζπ )ij = ζπ(i)π(j)
- Multiplying each variable xi by a sign bi ∈ {−1, 1} (ζb )i = bi · ζi
(ζb )ij = bi bj · ζij
Transformations of a measure Λ on C(f )
- Each ζ ∈ C(f ) can be transformed by: - Permuting the underlying k variables by a permutation π (ζπ )i = ζπ(i)
(ζπ )ij = ζπ(i)π(j)
- Multiplying each variable xi by a sign bi ∈ {−1, 1} (ζb )i = bi · ζi
(ζb )ij = bi bj · ζij
- Projecting ζ to coordinates corresponding to a subset S ⊆ [k].
Transformations of a measure Λ on C(f )
- Each ζ ∈ C(f ) can be transformed by: - Permuting the underlying k variables by a permutation π (ζπ )i = ζπ(i)
(ζπ )ij = ζπ(i)π(j)
- Multiplying each variable xi by a sign bi ∈ {−1, 1} (ζb )i = bi · ζi
(ζb )ij = bi bj · ζij
- Projecting ζ to coordinates corresponding to a subset S ⊆ [k]. - For S ⊆ [k], π : S → S, b ∈ {−1, 1}S , let ΛS,π,b denote the measure obtained by transforming each point in support of Λ as above.
Our Characterization - Recall that f : {−1, 1}k → {0, 1} can be written as f (x ) =
X S⊆[k]
b f (S) ·
Y i∈S
xi = ρ(f ) +
k X X t=1 |S|=t
b f (S) ·
Y i∈S
xi
Our Characterization - Recall that f : {−1, 1}k → {0, 1} can be written as f (x ) =
X S⊆[k]
b f (S) ·
Y
xi = ρ(f ) +
k X X t=1 |S|=t
i∈S
b f (S) ·
Y
xi
i∈S
- [KTW 13∗ ]: f is approximation resistant if and only if there exists a probability measure Λ on C(f ) such that for all t = 1, . . . , k ! X X X Y b bi · ΛS,π,b ≡ 0 f (S) · |S|=t π:S→S b∈{−1,1}S
i∈S
Our Characterization - Recall that f : {−1, 1}k → {0, 1} can be written as f (x ) =
X S⊆[k]
b f (S) ·
Y
xi = ρ(f ) +
k X X t=1 |S|=t
i∈S
b f (S) ·
Y
xi
i∈S
- [KTW 13∗ ]: f is approximation resistant if and only if there exists a probability measure Λ on C(f ) such that for all t = 1, . . . , k ! X X X Y b bi · ΛS,π,b ≡ 0 f (S) · |S|=t π:S→S b∈{−1,1}S
i∈S t
- If |S| = t, then ΛS,π,b is a measure on Rt+(2) . For each t, above expression is a linear combination of such measures.
Proof Structure No good Λ exists
Good Λ exists
Proof Structure No good Λ exists
Good Λ exists
Standard PCP ideas
Hardness
Proof Structure No good Λ exists
Good Λ exists
zero-sum game value > 0
Algorithm Hardness
Hardness
Algorithm
zero-sum game value = 0
Standard PCP ideas
Hardness
Proof Structure No good Λ exists
Good Λ exists
zero-sum game value > 0
Algorithm Hardness
Algo
Hardness
Algorithm
zero-sum game value = 0
Standard PCP ideas
Hardness
Proof Structure No good Λ exists
Good Λ exists
zero-sum game value > 0
Algorithm Hardness
Algo
Hardness
Algorithm
zero-sum game value = 0
Standard PCP ideas
Hardness
Proof Structure No good Λ exists
Good Λ exists
zero-sum game value > 0
Algorithm Hardness
Algo
Hardness
Algorithm
zero-sum game value = 0
Standard PCP ideas
Hardness
Figure: The Two-Player Game
The (infinite) two-player game - Similar game used by O’Donnell and Wu for Max-Cut.
The (infinite) two-player game - Similar game used by O’Donnell and Wu for Max-Cut. - Hardness player tries to present a “hard to round" instance. Each constraint has local distribution µ with moments ζ(µ). Plays a probability measure Λ on C(f ) (corresponds to instance).
The (infinite) two-player game - Similar game used by O’Donnell and Wu for Max-Cut. - Hardness player tries to present a “hard to round" instance. Each constraint has local distribution µ with moments ζ(µ). Plays a probability measure Λ on C(f ) (corresponds to instance). - Algorithm player tries to round by first projecting to random d-dimensional subspace. Plays rounding strategy ψ : Rd → {−1, 1} (d = k + 1 suffices).
The (infinite) two-player game - Similar game used by O’Donnell and Wu for Max-Cut. - Hardness player tries to present a “hard to round" instance. Each constraint has local distribution µ with moments ζ(µ). Plays a probability measure Λ on C(f ) (corresponds to instance). - Algorithm player tries to round by first projecting to random d-dimensional subspace. Plays rounding strategy ψ : Rd → {−1, 1} (d = k + 1 suffices). - PayOff = Expected fraction of constraints satisfied by ψ − ρ(f )
The (infinite) two-player game - Similar game used by O’Donnell and Wu for Max-Cut. - Hardness player tries to present a “hard to round" instance. Each constraint has local distribution µ with moments ζ(µ). Plays a probability measure Λ on C(f ) (corresponds to instance). - Algorithm player tries to round by first projecting to random d-dimensional subspace. Plays rounding strategy ψ : Rd → {−1, 1} (d = k + 1 suffices). - PayOff = Expected fraction of constraints satisfied by ψ − ρ(f ) - Value > 0 implies (a distribution over) rounding strategies which show that predicate is approximable. (since every instance corresponds to some Λ)
PayOff (Λ, ψ) of the game - A random constraint in the instance corresponds to ζ ∼ Λ.
PayOff (Λ, ψ) of the game - A random constraint in the instance corresponds to ζ ∼ Λ. - When Algorithm player tries to round SDP solution, she sees vectors with inner products according to ζ.
PayOff (Λ, ψ) of the game - A random constraint in the instance corresponds to ζ ∼ Λ. - When Algorithm player tries to round SDP solution, she sees vectors with inner products according to ζ. - Projecting gives Gaussians y1 , . . . , yk with correlation matrix corresponding to ζ (y1 , . . . , yk ∼ N(ζ)).
PayOff (Λ, ψ) of the game - A random constraint in the instance corresponds to ζ ∼ Λ. - When Algorithm player tries to round SDP solution, she sees vectors with inner products according to ζ. - Projecting gives Gaussians y1 , . . . , yk with correlation matrix corresponding to ζ (y1 , . . . , yk ∼ N(ζ)). - Expected fraction of constraints satisfied Eζ∼Λ Ey1 ,...yk ∼N(ζ) [ f ( ψ(y1 ), . . . , ψ(yk ) ) ] X Y b = ρ(f ) + Eζ∼Λ Ey1 ,...yk ∼N(ζ) f (S) · ψ(yi ) S6=∅
i∈S
PayOff (Λ, ψ) of the game - A random constraint in the instance corresponds to ζ ∼ Λ. - When Algorithm player tries to round SDP solution, she sees vectors with inner products according to ζ. - Projecting gives Gaussians y1 , . . . , yk with correlation matrix corresponding to ζ (y1 , . . . , yk ∼ N(ζ)). - Expected fraction of constraints satisfied Eζ∼Λ Ey1 ,...yk ∼N(ζ) [ f ( ψ(y1 ), . . . , ψ(yk ) ) ] X Y b = ρ(f ) + Eζ∼Λ Ey1 ,...yk ∼N(ζ) f (S) · ψ(yi ) S6=∅
- PayOff(Λ, ψ) = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
S6=∅
i∈S
i Q b f (S) · i∈S ψ(yi ) .
Obtaining conditions on Λ when Value = 0 - PayOff = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
i Q b f (S) · ψ(y ) . i S6=∅ i∈S
Obtaining conditions on Λ when Value = 0 - PayOff = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
i Q b f (S) · ψ(y ) . i S6=∅ i∈S
- There exists Λ which gives PayOff ≤ 0 for all ψ.
Obtaining conditions on Λ when Value = 0 - PayOff = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
i Q b f (S) · ψ(y ) . i S6=∅ i∈S
- There exists Λ which gives PayOff ≤ 0 for all ψ. - PayOff: a multi-linear polynomial in the variables ψ(y ) for y ∈ Rd . The polynomial is upper bounded by zero for all assignments ψ.
Obtaining conditions on Λ when Value = 0 - PayOff = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
i Q b f (S) · ψ(y ) . i S6=∅ i∈S
- There exists Λ which gives PayOff ≤ 0 for all ψ. - PayOff: a multi-linear polynomial in the variables ψ(y ) for y ∈ Rd . The polynomial is upper bounded by zero for all assignments ψ. - All coefficients must be 0. Coefficients are: Z X γ(·) d C (S, π, b) · ΛS,π,b . S,π,b
Obtaining conditions on Λ when Value = 0 - PayOff = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
i Q b f (S) · ψ(y ) . i S6=∅ i∈S
- There exists Λ which gives PayOff ≤ 0 for all ψ. - PayOff: a multi-linear polynomial in the variables ψ(y ) for y ∈ Rd . The polynomial is upper bounded by zero for all assignments ψ. - All coefficients must be 0. Coefficients are: Z X γ(·) d C (S, π, b) · ΛS,π,b . S,π,b
- Conclude that X S,π,b
C (S, π, b) · ΛS,π,b ≡ 0.
Obtaining conditions on Λ when Value = 0 - PayOff = Eζ∼Λ Ey1 ,...yk ∼N(ζ)
hP
i Q b f (S) · ψ(y ) . i S6=∅ i∈S
- There exists Λ which gives PayOff ≤ 0 for all ψ. - PayOff: a multi-linear polynomial in the variables ψ(y ) for y ∈ Rd . The polynomial is upper bounded by zero for all assignments ψ. - All coefficients must be 0. Coefficients are: Z X γ(·) d C (S, π, b) · ΛS,π,b . S,π,b
- Conclude that X S,π,b
- Lots of work . . .
C (S, π, b) · ΛS,π,b ≡ 0.
Overview of the Talk
- Introduction and Survey - Basics: Algorithmic Side - Basics: Hardness Side - The Characterization of Approximation Resistance - Conclusion
Concluding Remarks - We also -
Characterize approximation resistance for k-partite instances. Characterize approximation resistance for Sherali-Adams LP. Can give alternate exposition to [Rag 08∗ ]. Can extend to non-boolean predicates and mix of predicates.
Concluding Remarks - We also -
Characterize approximation resistance for k-partite instances. Characterize approximation resistance for Sherali-Adams LP. Can give alternate exposition to [Rag 08∗ ]. Can extend to non-boolean predicates and mix of predicates.
- Problem: The characterization is recursively enumerable, but is it decidable? Can Λ always be finitely supported?
Concluding Remarks - We also -
Characterize approximation resistance for k-partite instances. Characterize approximation resistance for Sherali-Adams LP. Can give alternate exposition to [Rag 08∗ ]. Can extend to non-boolean predicates and mix of predicates.
- Problem: The characterization is recursively enumerable, but is it decidable? Can Λ always be finitely supported? - Problem: Is there a linear threshold function that is approximation resistant?
Concluding Remarks - We also -
Characterize approximation resistance for k-partite instances. Characterize approximation resistance for Sherali-Adams LP. Can give alternate exposition to [Rag 08∗ ]. Can extend to non-boolean predicates and mix of predicates.
- Problem: The characterization is recursively enumerable, but is it decidable? Can Λ always be finitely supported? - Problem: Is there a linear threshold function that is approximation resistant? - Problem: Approximation resistance on satisfiable instances?
Concluding Remarks - We also -
Characterize approximation resistance for k-partite instances. Characterize approximation resistance for Sherali-Adams LP. Can give alternate exposition to [Rag 08∗ ]. Can extend to non-boolean predicates and mix of predicates.
- Problem: The characterization is recursively enumerable, but is it decidable? Can Λ always be finitely supported? - Problem: Is there a linear threshold function that is approximation resistant? - Problem: Approximation resistance on satisfiable instances? - Problem: Dichotomy conjecture?
Thank You Questions?
