Skip to content

Correctness fixes from a brute-force review (1/4) - #23

Open
Autoplectic wants to merge 1 commit into
mainfrom
sofic-correctness-2
Open

Autoplectic wants to merge 1 commit into
mainfrom
sofic-correctness-2

Conversation

@Autoplectic

Copy link
Copy Markdown
Member

First of four PRs from a second correctness review. Every fix was reproduced against a brute-force reference implementation or a closed-form value and has a regression test. CHANGELOG.md (Unreleased, 0.4.0) has the full list.

Breaking changes

  • correction="bonferroni" is the default for every CSSR learner. Uncorrected CSSR found spurious states in about 10–15% of long-sample runs.
  • viterbi returns the n+1 states X_0..X_n, aligned with smooth.
  • "crypticity" means C_mu − E everywhere; C± − E is now "bidirectional_crypticity" / bidirectional_crypticity().
  • The Gács–Körner model is documented as the common-information variable, not a generator of the process.
  • atoms() includes the negative atom, and left_quotients includes the empty residual.
  • Brzozowski minimization returns a trimmed DFA, like Hopcroft and Moore.
  • Examples emit string symbols.
  • There is one golden mean, in the Lind & Marcus form that forbids 11; duplicate examples are removed.
  • butterfly_process now matches its paper (the old construction was i.i.d.).

Notable fixes

  • Shift synchronization methods crashed with AttributeError.
  • Cryptic order was wrong for zero-crypticity processes.
  • Bayesian posterior machines started in the data's final state.
  • Büchi lasso acceptance missed accepting states visited mid-loop.
  • Wheeler order detection, index and minimization were wrong in several cases.
  • Modular VPA minimization crashed on valid inputs.
  • Language operations on empty languages failed.
  • IDFA enumeration undercounted.
  • Stack MLE was biased.
  • Spectral learning hung on sampled data.
  • log_word_probability underflowed on long words.
  • Channel statistical complexity depended on the hash seed.
  • golden_mean_ghmm did not reproduce the golden mean.
  • Topological entropy was inflated on non-right-resolving presentations.
  • YAML dropped unused alphabet symbols and could not store sympy values.

Testing infrastructure

  • New Hypothesis strategies in sofic.testing.
  • Brute-force reference implementations in tests/oracles.py.
  • ci and nightly Hypothesis profiles.

Checks

  • 1352 tests pass.
  • ruff is clean.
  • A fresh Sphinx build passes with -W.

…mples; testing strategies

Every fix was reproduced against a brute-force reference or closed-form value
and has a regression test. See CHANGELOG.md (Unreleased, 0.4.0) for details.

- Shifts: word-counting topological entropy, trim_transient, factor languages,
  Dyck reversal, edge-shift Parry labels, sorted TMC symbols, SoficShift
  synchronization methods.
- Generators: cryptic order for zero-crypticity processes, log_word_probability
  underflow, crypticity key, Gacs-Korner model semantics, hash-seed-dependent
  channel complexity, fast mixed-state explosion.
- Inference: Bayesian start state, exact stack MLE, spectral noise floor and
  pruning, Bonferroni default for CSSR, censored ALERGIA, Viterbi n+1 states,
  per-sequence cross-validation.
- Automata: Buchi lassos, Wheeler family, modular VPA minimize, empty operands,
  minimizer agreement, atoms/quotients, IDFA enumeration, transducer epsilon
  handling, faster VPA operations.
- Serialization/viz: explicit alphabets, sympy/Fraction YAML, deep copy, unique
  node names, TikZ escaping.
- Examples emit string symbols; one golden mean (Lind-Marcus, forbids 11);
  duplicate examples removed; butterfly_process matches its paper.
- New sofic.testing strategies and brute-force oracles; ci/nightly Hypothesis
  profiles.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant