Documentation

Complexity.Computability.GenericRAM.Native.HeavyBranchFold

Heavy-variable branching and streaming common-zero accumulation #

Implements the heavy-variable branching reduction and space-efficient streaming accumulator from §5.5 (Proof of Theorem 2.2, pp. 38–39) of OpenAI, The Exponential Time Hypothesis is False over SparsePolyCompile, bridged to SparseEquations and BooleanFold.

Main definitions #

Main results #

References #

1. Common-Zero Counting, SparseEquations Bridges, and heavyVarList #

Extend a finite assignment σ : BitString n (Fin n → Bool) by false outside [n].

Instances For

    Exact number of common zeros N(P) of sys over BitString n (Fin n → Bool).

    Instances For

      Lift a sparse 𝔽₂ equation eq : List (List ℕ) to Finset (Finset (Fin n)).

      Instances For

        Sorted list of heavy variables v < n with b₀ < varDegree v sys (§5.5 of [OAI26c]).

        Instances For
          theorem Cslib.GenericRAM.HeavyBranchFold.rootCount_eq_commonZeroCount {n : ℕ} {sys : List (List (List ℕ))} (hcan : ∀ eq ∈ sys, Complexity.IsCanonicalEquation eq) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) :

          Specification bridge to SparseEquations.commonZeroCount: for any canonical sparse 𝔽₂ system sys on < n variables, rootCount n sys agrees with Cslib.BitString.commonZeroCount.

          theorem Cslib.GenericRAM.HeavyBranchFold.pairwise_lt_heavyVarList (n b₀ : ℕ) (sys : List (List (List ℕ))) :
          List.Pairwise (fun (x1 x2 : ℕ) => x1 < x2) (heavyVarList n b₀ sys)
          @[simp]
          theorem Cslib.GenericRAM.HeavyBranchFold.lt_of_mem_heavyVarList {n b₀ v : ℕ} {sys : List (List (List ℕ))} (hv : v ∈ heavyVarList n b₀ sys) :
          v < n
          theorem Cslib.GenericRAM.HeavyBranchFold.varDegree_le_of_not_mem_heavyVarList {n b₀ u : ℕ} {sys : List (List (List ℕ))} (hun : u < n) (hu : u ∉ heavyVarList n b₀ sys) :

          Specification bridge to SparseEquations.heavyVars: heavyVarList n b₀ sys enumerates Cslib.BitString.heavyVars in strictly increasing order.

          theorem Cslib.GenericRAM.HeavyBranchFold.length_heavyVarList_le_div {n b₀ : ℕ} {sys : List (List (List ℕ))} (hb₀ : 0 < b₀) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) :
          (heavyVarList n b₀ sys).length ≤ (List.map (fun (eq : List (List ℕ)) => (Complexity.equationVars eq).length) sys).sum / b₀

          Markov heavy-variable bound (§5.5, p. 38): for 0 < b₀ and variables < n, (heavyVarList n b₀ sys).length ≤ (sys.map fun eq => (equationVars eq).length).sum / b₀.

          2. Order-Preserving Dense Free-Variable Relabeling #

          Densely relabel a free variable v : ℕ outside S as v - (S.filter (· < v)).length (defined totally for all v : ℕ, with natural subtraction truncated at 0 if (S.filter (· < v)).length is at least v).

          Instances For
            theorem Cslib.GenericRAM.HeavyBranchFold.relabelFreeVar_strictMono {S : List ℕ} (hS : S.Nodup) {u v : ℕ} (hu : u ∉ S) (huv : u < v) :

            For S.Nodup, relabelFreeVar S is strictly increasing from u ∉ S to any larger v.

            theorem Cslib.GenericRAM.HeavyBranchFold.relabelFreeVar_lt_sub {n : ℕ} {S : List ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) {u : ℕ} (hu : u ∉ S) (hun : u < n) :

            For S.Nodup in < n, every free variable u < n outside S maps below n - S.length.

            theorem Cslib.GenericRAM.HeavyBranchFold.relabelFreeVar_injOn {S : List ℕ} (hS : S.Nodup) {u v : ℕ} (hu : u ∉ S) (hv : v ∉ S) (heq : relabelFreeVar S u = relabelFreeVar S v) :
            u = v

            For S.Nodup, relabelFreeVar S is injective on all variables outside S.

            noncomputable def Cslib.GenericRAM.HeavyBranchFold.freeVarEquiv {n : ℕ} {S : List ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) :
            { i : Fin n // ↑i ∉ S } ≃ Fin (n - S.length)

            Order-preserving bijection [n] \ S ≃ Fin (n - S.length) for S.Nodup in < n.

            Instances For

              3. Multi-Variable Substitution, Canonicalization, and prepareBranch #

              Substitute (S, vals) across eq and cancel duplicate monomials modulo 2 (looking up the first index S.idxOf v if S has duplicates, defaulting missing entries ≥ vals.length to false, and ignoring trailing vals entries past S.length).

              Instances For

                Relabel all free variables in an equation using relabelFreeVar S.

                Instances For

                  Outcome of preparing a single heavy-variable branch (S ↦ vals) (§5.5 of [OAI26c]).

                  • contradiction : BranchOutcome

                    Contradictory constant equation 1 = 0 ([[]]) appeared; 0 satisfying completions.

                  • tautology (freeVars : ℕ) : BranchOutcome

                    All equations simplified to 0 = 0; all 2 ^ freeVars completions satisfy the system.

                  • reduced (freeVars : ℕ) (sys : List (List (List ℕ))) : BranchOutcome

                    Non-empty reduced system on freeVars = n - |S| densely relabeled free variables.

                  Instances For

                    Structural option view of BranchOutcome for native encoding and option elimination.

                    Instances For
                      @[irreducible]

                      Native binary encoding of BranchOutcome (n' in unary, sys' as framed nested lists).

                      Instances For

                        Restrict sys to S ↦ vals, drop 0 = 0 ([]), short-circuit 1 = 0 ([[]]) and empty systems, or densely relabel [n] \ S ≃ [n - S.length] (truncated at 0 if n < |S|).

                        Instances For
                          theorem Cslib.GenericRAM.HeavyBranchFold.mem_substituteEquation_exists {S : List ℕ} {vals : List Bool} {eq : List (List ℕ)} {m' : List ℕ} (hm' : m' ∈ substituteEquation S vals eq) :
                          ∃ m ∈ eq, m' = List.filter (fun (v : ℕ) => decide (v ∉ S)) m
                          theorem Cslib.GenericRAM.HeavyBranchFold.prepareBranch_freeVars_eq {n n' : ℕ} {S : List ℕ} {vals : List Bool} {sys : List (List (List ℕ))} :
                          (prepareBranch n S vals sys = BranchOutcome.tautology n' → n' = n - S.length) ∧ ∀ (sys' : List (List (List ℕ))), prepareBranch n S vals sys = BranchOutcome.reduced n' sys' → n' = n - S.length
                          theorem Cslib.GenericRAM.HeavyBranchFold.prepareBranch_eq_reduced_iff {n n' : ℕ} {S : List ℕ} {vals : List Bool} {sys sys' : List (List (List ℕ))} :
                          prepareBranch n S vals sys = BranchOutcome.reduced n' sys' ↔ have nonZero := List.filter (fun (eq : List (List ℕ)) => !eq.isEmpty) (List.map (substituteEquation S vals) sys); [[]] ∉ nonZero ∧ nonZero ≠ [] ∧ n' = n - S.length ∧ sys' = List.map (relabelEquation S) nonZero
                          theorem Cslib.GenericRAM.HeavyBranchFold.prepareBranch_reduced_canonical {n : ℕ} {S : List ℕ} {vals : List Bool} {sys sys' : List (List (List ℕ))} {n' : ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) (hcan : ∀ eq ∈ sys, Complexity.IsCanonicalEquation eq) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) (hred : prepareBranch n S vals sys = BranchOutcome.reduced n' sys') :
                          n' = n - S.length ∧ sys' ≠ [] ∧ [] ∉ sys' ∧ [[]] ∉ sys' ∧ (∀ eq' ∈ sys', Complexity.IsCanonicalEquation eq') ∧ ∀ eq' ∈ sys', ∀ m' ∈ eq', ∀ v' ∈ m', v' < n'

                          For S.Nodup in < n and canonical sys on < n, any .reduced n' sys' branch has n' = n - S.length, sys' ≠ [], [] ∉ sys', [[]] ∉ sys', and canonical < n' equations.

                          theorem Cslib.GenericRAM.HeavyBranchFold.prepareBranch_reduced_degree_le {n k : ℕ} {S : List ℕ} {vals : List Bool} {sys sys' : List (List (List ℕ))} {n' : ℕ} (hdeg : ∀ eq ∈ sys, ∀ m ∈ eq, m.length ≤ k) (hred : prepareBranch n S vals sys = BranchOutcome.reduced n' sys') (eq' : List (List ℕ)) :
                          eq' ∈ sys' → ∀ m' ∈ eq', m'.length ≤ k

                          Degree preservation: if sys has degree at most k, so does any reduced branch sys'.

                          theorem Cslib.GenericRAM.HeavyBranchFold.prepareBranch_reduced_length_le {n : ℕ} {S : List ℕ} {vals : List Bool} {sys sys' : List (List (List ℕ))} {n' : ℕ} (hred : prepareBranch n S vals sys = BranchOutcome.reduced n' sys') :
                          sys'.length ≤ sys.length ∧ ∀ eq' ∈ sys', ∃ eq ∈ sys, eq'.length ≤ eq.length ∧ (Complexity.equationVars eq').length ≤ (Complexity.equationVars eq).length

                          Monomial-count and scope-width preservation across prepareBranch.

                          theorem Cslib.GenericRAM.HeavyBranchFold.varDegree_relabelFreeVar_le {n : ℕ} {S : List ℕ} {vals : List Bool} {sys sys' : List (List (List ℕ))} {n' : ℕ} (hS : S.Nodup) {u : ℕ} (hu : u ∉ S) (hred : prepareBranch n S vals sys = BranchOutcome.reduced n' sys') :

                          Per-variable incidence preservation: for S.Nodup and u ∉ S, varDegree (relabelFreeVar S u) on any .reduced branch sys' never exceeds varDegree u sys.

                          theorem Cslib.GenericRAM.HeavyBranchFold.prepareBranch_reduced_varDegree_le {n b₀ : ℕ} {S : List ℕ} {vals : List Bool} {sys sys' : List (List (List ℕ))} {n' : ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) (hinc : ∀ u < n, u ∉ S → Complexity.varDegree u sys ≤ b₀) (hred : prepareBranch n S vals sys = BranchOutcome.reduced n' sys') (u' : ℕ) :
                          u' < n' → Complexity.varDegree u' sys' ≤ b₀

                          Uniform free-variable incidence preservation (§5.5 of [OAI26c]): for S.Nodup in < n, if every u < n outside S has varDegree u sys ≤ b₀, every u' < n' in sys' has ≤ b₀.

                          4. Exact Common-Zero Count Preservation #

                          Exact common-zero count represented by a BranchOutcome.

                          Instances For

                            Dispatch a BranchOutcome to an external counting oracle solver on .reduced branches.

                            Instances For
                              theorem Cslib.GenericRAM.HeavyBranchFold.branchRootCount_prepareBranch {n : ℕ} {S : List ℕ} {vals : List Bool} {sys : List (List (List ℕ))} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) (hvals : vals.length = S.length) :

                              Single-branch exactness: for S.Nodup in < n, sys on < n, and vals.length = S.length, branchRootCount (prepareBranch n S vals sys) counts common zeros extending (S, vals).

                              theorem Cslib.GenericRAM.HeavyBranchFold.sum_branch_count_eq_rootCount {n : ℕ} {S : List ℕ} {sys : List (List (List ℕ))} {solver : ℕ → List (List (List ℕ)) → ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) (hsolver : ∀ (vals : List Bool), vals.length = S.length → ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch n S vals sys = BranchOutcome.reduced n' sys' → solver n' sys' = rootCount n' sys') :
                              (List.map (fun (idx : ℕ) => branchCountOracle solver (prepareBranch n S (BitString.lowBits S.length idx) sys)) (List.range (2 ^ S.length))).sum = rootCount n sys

                              Exponential-branching decomposition identity: for S.Nodup in < n and sys on < n, summing branchCountOracle solver over 0 ≤ idx < 2 ^ S.length via lowBits S.length idx equals rootCount n sys whenever solver is exact on .reduced branches.

                              5. Plain Polynomial-Time Certificates for heavyVarList and prepareBranch #

                              6. Space-Efficient Streaming BooleanFold Combinator #

                              @[reducible, inline]

                              Input tuple (n, S, sys) for streaming heavy-variable branching.

                              Instances For

                                Native encoding of BranchInput (n in unary, S and sys as framed lists).

                                Instances For

                                  Encoded-size non-expansion: every .reduced n' sys' branch produced by prepareBranch has encoded solver input (n', sys') no longer than the parent encBranchInput (n, S, sys).

                                  Single streaming step of heavy-variable branching.

                                  Instances For

                                    Stream all 2 ^ |S| heavy-variable assignments via BooleanFold.run without materializing the assignment list, returning (finalIdx, (|S|, (inp, totalCount))).

                                    Instances For

                                      Total common-zero count computed by streaming 2 ^ |S| heavy-variable branches.

                                      Instances For
                                        theorem Cslib.GenericRAM.HeavyBranchFold.runPrefix_branchBody_accumulator (solver : ℕ → List (List (List ℕ)) → ℕ) (inp : BranchInput) (j : ℕ) :
                                        (BooleanFold.runPrefix (branchBody solver) inp.2.1.length j (inp, 0)).2.2 = (inp, (List.map (fun (idx : ℕ) => branchCountOracle solver (prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2)) (List.range j)).sum)

                                        Prefix invariant of BooleanFold.runPrefix: inp is preserved and the accumulator holds the sum of the first j decoded branch counts.

                                        theorem Cslib.GenericRAM.HeavyBranchFold.streamBranches_correct {n : ℕ} {S : List ℕ} {sys : List (List (List ℕ))} {solver : ℕ → List (List (List ℕ)) → ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) (hsolver : ∀ (vals : List Bool), vals.length = S.length → ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch n S vals sys = BranchOutcome.reduced n' sys' → solver n' sys' = rootCount n' sys') :
                                        (streamBranches solver (n, S, sys)).2.2 = ((n, S, sys), rootCount n sys)

                                        Streaming correctness: for S.Nodup in < n and sys on < n, (streamBranches solver (n, S, sys)).2.2 = ((n, S, sys), rootCount n sys) when solver is exact on .reduced branches.

                                        theorem Cslib.GenericRAM.HeavyBranchFold.countByBranching_eq_rootCount {n : ℕ} {S : List ℕ} {sys : List (List (List ℕ))} {solver : ℕ → List (List (List ℕ)) → ℕ} (hS : S.Nodup) (hSn : ∀ s ∈ S, s < n) (hvars : ∀ eq ∈ sys, ∀ m ∈ eq, ∀ v ∈ m, v < n) (hsolver : ∀ (vals : List Bool), vals.length = S.length → ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch n S vals sys = BranchOutcome.reduced n' sys' → solver n' sys' = rootCount n' sys') :
                                        countByBranching solver (n, S, sys) = rootCount n sys

                                        Streaming count correctness: for S.Nodup in < n and sys on < n, countByBranching solver (n, S, sys) = rootCount n sys when solver is exact on .reduced branches.

                                        Selected-branch native certificate for branchCountOracle solver: a fixed csolver runs ONLY on .reduced n' sys' outcomes, while .contradiction emits 0 and .tautology n' emits 2 ^ n' via BooleanFold.clock in linear time and space.

                                        Instances For
                                          theorem Cslib.GenericRAM.HeavyBranchFold.runPrefix_sum_le {solver : ℕ → List (List (List ℕ)) → ℕ} {inp : BranchInput} (hbound : ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → solver n' sys' ≤ 2 ^ n') {j : ℕ} (hj : j ≤ 2 ^ inp.2.1.length) :
                                          (BooleanFold.runPrefix (branchBody solver) inp.2.1.length j (inp, 0)).2.2.2 ≤ j * 2 ^ (inp.1 - inp.2.1.length)

                                          Prefix sum bound: when solver n' sys' ≤ 2 ^ n' on reduced branches of inp, every streaming prefix j ≤ 2 ^ |S| keeps the running sum at most j * 2 ^ (n - |S|).

                                          theorem Cslib.GenericRAM.HeavyBranchFold.runPrefix_sum_le_two_pow {solver : ℕ → List (List (List ℕ)) → ℕ} {inp : BranchInput} (hSn : inp.2.1.length ≤ inp.1) (hbound : ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → solver n' sys' ≤ 2 ^ n') {j : ℕ} (hj : j ≤ 2 ^ inp.2.1.length) :
                                          (BooleanFold.runPrefix (branchBody solver) inp.2.1.length j (inp, 0)).2.2.2 ≤ 2 ^ inp.1

                                          When |S| ≤ n and solver n' sys' ≤ 2 ^ n' on reduced branches, the running sum satisfies Z ≤ 2 ^ n throughout the entire streaming traversal (§5.5 p. 39 of [OAI26c]).

                                          Solver-independent polynomial overhead per streaming round and for initialization.

                                          Instances For
                                            theorem Cslib.GenericRAM.HeavyBranchFold.streamBranches_computableInO {solver : ℕ → List (List (List ℕ)) → ℕ} (csolver : NativeCertificate (fun (p : ℕ × List (List (List ℕ))) => solver p.1 p.2) (Turing.MultiTapeTM.encPair Turing.MultiTapeTM.encUnary (Turing.MultiTapeTM.encList (Turing.MultiTapeTM.encList (Turing.MultiTapeTM.encList Turing.MultiTapeTM.encNat)))) Turing.MultiTapeTM.encNat) (R_solver Q_solver : BranchInput → ℕ) (hbound : ∀ (inp : BranchInput), ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → solver n' sys' ≤ 2 ^ n') (hR : ∀ (inp : BranchInput), ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → csolver.time (n', sys') ≤ R_solver inp) (hQ : ∀ (inp : BranchInput), ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → csolver.space (n', sys') ≤ Q_solver inp) :

                                            Space-efficient streaming ComputableInO bound for streamBranches (§5.5 of [OAI26c]): for fixed solver bounded by 2 ^ n' with time ≤ R_solver and space ≤ Q_solver on .reduced branches, runs in time O(stepOverhead |inp| + (2 ^ |S| + 1) * (stepOverhead |inp| + R_solver inp)) and workspace O(stepOverhead |inp| + Q_solver inp).

                                            theorem Cslib.GenericRAM.HeavyBranchFold.countByBranching_computableInO {solver : ℕ → List (List (List ℕ)) → ℕ} (csolver : NativeCertificate (fun (p : ℕ × List (List (List ℕ))) => solver p.1 p.2) (Turing.MultiTapeTM.encPair Turing.MultiTapeTM.encUnary (Turing.MultiTapeTM.encList (Turing.MultiTapeTM.encList (Turing.MultiTapeTM.encList Turing.MultiTapeTM.encNat)))) Turing.MultiTapeTM.encNat) (R_solver Q_solver : BranchInput → ℕ) (hbound : ∀ (inp : BranchInput), ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → solver n' sys' ≤ 2 ^ n') (hR : ∀ (inp : BranchInput), ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → csolver.time (n', sys') ≤ R_solver inp) (hQ : ∀ (inp : BranchInput), ∀ idx < 2 ^ inp.2.1.length, ∀ (n' : ℕ) (sys' : List (List (List ℕ))), prepareBranch inp.1 inp.2.1 (BitString.lowBits inp.2.1.length idx) inp.2.2 = BranchOutcome.reduced n' sys' → csolver.space (n', sys') ≤ Q_solver inp) :

                                            Space-efficient streaming ComputableInO bound for countByBranching (§5.5 of [OAI26c]): for fixed solver bounded by 2 ^ n' with time ≤ R_solver and space ≤ Q_solver on .reduced branches, returns the total count in time O(stepOverhead |inp| + (2 ^ |S| + 1) * (stepOverhead |inp| + R_solver inp)) and workspace O(stepOverhead |inp| + Q_solver inp) (with no 2 ^ |S| factor in workspace).

                                            Polynomial-workspace preservation: whenever Q_solver is polynomially bounded in the encoded input length, the total streaming workspace stepOverhead |inp| + Q_solver inp is also polynomially bounded (independent of 2 ^ |S|).