Skip to content

Committed rules documentation: rules.md, decisions.md, mechanisms.md with enforcement tests - #381

Open
derek73 wants to merge 46 commits into
masterfrom
claude/library-rules-documentation-938523
Open

Committed rules documentation: rules.md, decisions.md, mechanisms.md with enforcement tests#381
derek73 wants to merge 46 commits into
masterfrom
claude/library-rules-documentation-938523

Conversation

@derek73

@derek73 derek73 commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Implements the library-rules-documentation design, then absorbs thirteen primary-source review rounds and two harvest sweeps. This PR is the durable residue of every written record the project has: stage docstrings, session memories, the issue tracker, and the gitignored planning specs are all either promoted here or deliberately left in place with a recorded reason.

What this adds

Three contributor docs in docs/design/ (measured at head, not estimated):

Six enforcement tests (tests/v2/): the executable-examples runner (every example line runs against the live parser; deviates: markers asserted strictly), citation integrity (cited IDs exist; code-comment excerpts are verbatim; implemented: lists match the modules actually citing each rule; doc-internal anchors resolve; the legacy §-citation pattern is banned), and a known-bad-spellings denylist (which caught its seed entry live in two published docs on its first run).

Docstring slimming: stage docstrings keep their Consumes/Produces/Reads headers plus quoted rule citations and code-local notes; the 57-line rule-1b block is a 10-line citation. All 51 committed references to gitignored planning docs retired.

AGENTS.md gains the maintenance section: same-PR amendment rule, counting-claims taxonomy, release-log claim sourcing, rule-shaped issue drafting, primary-source review, harvest-landing provenance.

Zero behavior change, proven three ways

  • AST comparison (docstrings stripped): all 26 changed nameparser/ files identical in executable structure to master — re-verified at head.
  • Full suite: 3387 passed, mypy and ruff clean.
  • Differential gate at every ledgered baseline, re-run at head, each unexplained: 0 — 1.4.0 (108 intentional, ledgered), previous minor (1), 2.0.0 (90).

Review process

Every extraction pass ended with an adversarial probe against the live parser; the probes falsified nine drafted statements before landing. Thirteen post-PR topic reviews by primary-source sessions (the four-question protocol now in AGENTS.md) falsified and corrected statements, restored flattened attribution, and contributed the arcs no extraction could reach — each round's claims measured before landing. Six issues were spun out of the review process (#382#385, plus linkage into #316/#368); every commit-SHA and issue citation in the decision record is verified against git and the tracker.

🤖 Generated with Claude Code

derek73 and others added 23 commits August 15, 2026 15:40
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The denylist's first run caught its seed entry live in usage.rst and
release_log.rst -- Policy(segment_scripts=()) fails mypy; both sites
now hand out the frozenset() spelling the warning itself offers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Five rules (H1, P1, O1-O3) with executable examples; decisions.md
gains P1/O1/O2 history; the 57-line rule-1b comment block becomes a
10-line citation. The adversarial probe falsified O2's 'exactly four
words' claim (a suffix-bearing five-word name rotates); the statement
now counts name words with titles, suffixes and nicknames set aside,
and the falsifying input is pinned as an example.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Old-numbering map in decisions.md; drop the gitignored-spec pointer
from rules_doc.py; PR citations on O1/O2 entries; runner rejects an
ungated [ja] annotation (the extra-absent CI split).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…es values

Citation ID grammar now admits bare and underscored mechanism slugs
(SPANS, FOLDED_TAG); the value grammar accepts one level of list
nesting for pieces= assertions, with a unit test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…divisions)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The spec's drafted litmus statement ('recognized by vocabulary, not
by written shape') was falsified by the live parser: the leading-
abbreviation shape heuristic is real ('Xyz. John Smith' takes the
title), and the abugida gap is its LIMIT, recorded as H2's Accepted
consequence. S2 records the bare-ambiguous-acronym behavior
('Jack Wei Ma' → suffix='Ma') as an Accepted consequence pending
#326-adjacent design work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ambiguities assertion form implemented in the runner; plan-deviation
phrases retargeted to decisions.md entries; 27 rules total.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d reality

The batched adversarial probe falsified or sharpened: P2 (join runs
to the end), H2 (unbroken-run definition; comma carve-out), H3
(comma carve-out), C1 (lenient/strict prose), S2 (bare-ambiguous
words-to-spare guard), M1 (S1 wins), M2 (comma and particle
carve-outs), W1 (family-position precision), W2 (once; vocabulary),
W3 (rescoped to the comma doctrine; segmenter clause qualified), W4
(interpunct source-order carve-out), N3 (suffix-count artifact), T1
(wording). Falsifying inputs pinned as examples; all excerpts
updated in lockstep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ase repair)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The denylist caught its own seed spelling being reintroduced in the
D1 decisions entry -- reworded to describe the bad spelling without
containing it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…on shapes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
….0 promotion

Harvest measurements spot-checked at landing; the two flagged
uncertainties resolved against suffixes.py's own vetting block (the
per-90-surnames count became an invariant). The 3.0 shim-shaped list
is promoted from session memory into the 3-0-reevaluations section;
the memory file is now a pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All 35 gitignored-spec references (spec/conventions/amendment section
numbers) now point at mechanisms.md entries, rules, or decisions.md;
three new mechanism entries carry the promoted contracts
(LOCALE-PACKS-PURE-DATA, FACADE-CONTRACT, CONFIG-SHIM-SNAPSHOT).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eview)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@derek73 derek73 added docs Documentation fixes and updates tests labels Aug 16, 2026
@derek73 derek73 self-assigned this Aug 16, 2026
@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.51%. Comparing base (5320a42) to head (1c465b3).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #381   +/-   ##
=======================================
  Coverage   98.51%   98.51%           
=======================================
  Files          44       44           
  Lines        2895     2895           
=======================================
  Hits         2852     2852           
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 3 commits August 15, 2026 18:15
…seams, stale later-task notes

Six branch-introduced defects (orphaned totality-exception reference,
legacy rule-1b pointer in particles.py, garbled shim citation seam,
two mis-targeted LOCALE-PACKS anchors with broken wrapping, wrong
RECORDED-ROSTERS home) plus the improvement items, and the five
pre-2.0.0 'a later task' facade notes the analyzer surfaced. Assign's
stage header now admits its is_suffix_lenient consult (pre-existing
inaccuracy). AST re-verified identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y source)

Attribution: decision dates restored where entries carried release
dates (W1/W4/T3); D1 gains the actual decision (warn-not-raise, the
filterable-inertness rationale, the conditional ja hint); W2's #312
entry regains its provenance (gates inherited, not designed). New:
the codepoint-scope decision and half-flanked-interpunct Declined
under T3, the B7 sanctioned-extra Declined-with-measurement,
SELF-EXPIRING-GUARD and the extra-environment-split field note in
Verification shapes. STATE-OFFSET-CHANNELS contract admits
presence-only consumers (both interpunct readers, measured). New
verified examples: W1's segmentation ambiguity, W2 peeling across an
interpunct-divided name (the review's conjecture, confirmed live),
W4's nakaguro/interpunct order contrast.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(primary source)

The 양/군 Excluded entry had attached the spaced-safety argument to
the glued exclusion — corrected from the vetting block itself, with
김지군 resolving the harvest's flagged uncertainty properly; 氏 gets
its real reason (王氏 historical form); 殿 regains the harm clause.
Four Declined dates corrected to git author dates (2026-08-05); the
_check_tree asymmetry diagnosis restored; D1 gains the autodoc
boundary fact. New: the Vietnamese middle_as_family
Declined-with-measurement, the fullwidth-colon M2 Accepted
consequence (#317 linked), the Japanese spaced-forms asymmetry in W
Background, Vietnamese facts in O Background, and the
CANONICAL-VOCABULARY-AT-THE-BOUNDARY mechanism.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
derek73 and others added 3 commits August 15, 2026 18:34
…ource)

rules.md#M1's statement was FALSIFIED as written: the marker drop is
conditional on clause size (a one-word clause keeps its word -- Nee
is an attested surname), a qualifier decisions.md had and rules.md
lost in transit. Statement corrected, the real guard pinned as the
boundary example ('Jane Smith (Nee)' maiden 'Nee'), clause
independence added as statement + example, and the citation excerpt
updated in lockstep. decisions.md#M1 gains the three separately
argued decisions, the M1/M2 disjointness guarantee with its 7,775-
record verification, and the neighbour-scoping Declined with its
measurement. Verification shapes: the honest-limit preamble now
credits mutation testing with part of the wrong-predicate class
(the #329 survivors), and the skip-reachability trap joins the
field notes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rimary source)

decisions.md#P2 claimed #367 made 'Sir de Mesnil' chain -- the
inverse of the truth (#367 REMOVED a chain; the family reading is
P1's fold), contradicting P1's own correct entry; both now agree and
the misfiled example moved to P1. The pieces= assertion form is
implemented (its first using rule arrived): new P4 states that a
leading particle chains nothing, with [[Sir],[de],[Mesnil]] and the
chained counterpart as structural examples. P1's family-first
boundary becomes the doc's first live deviates: marker (#368
superseded #359's working-as-intended sentence; intended
family='de la Vega', today family='Juan'), and its statement now
names #364 as open rather than pre-answering it. New: S3
(period-joined vocabulary -- the rule whose absence made a docstring
wrong four times), the particle-curation and esq Excluded blocks
(mirroring AGENTS' algebra, counts deliberately omitted), two P1
Declined entries (the strict-xfail back door; the not-a-title
predicate that st/do/freiherr break), the #365 two-orders-disagree
why, the deprecation-bridge template in 3-0-reevaluations,
VOCABULARY-OVERLAP-AS-PRECONDITION, and #372's standing caution on
LEDGER-RULE-SEPARATION.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mary source)

LEDGER-RULE-SEPARATION's contract was falsified by measurement: a
strict fields-subset pair shares the regex tier in the live 1.4
ledger and file order decides between them (a pure reorder
reattributes seven names, caught only by _CROSS_RULE_WINNERS). The
contract now states the between-tiers/within-tier split, the stale
1639-figure is framed as the dated #372-as-filed measurement with
the #375/#376 correction, and 'never actually run' is fixed (run
twice in #375, failed). The whole dormancy arc (#328-#376) lands in
decisions.md: three shipped decisions ([[never]] monotone
exclusions, always-on dormant with three diagnoses, the 2.0.0
baseline gap) and four Declined-with-measurements. New:
CROSS-RULE-OUTCOME-PINS mechanism, the alternation-plus-sync-roster
second half on CURATED-VOCABULARY-ALTERNATION, the
narrowing-relocates-the-bug field note, and the roster list
completed on RECORDED-ROSTERS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
derek73 and others added 4 commits August 15, 2026 18:59
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…source)

H1 gains the empty-family Accepted consequence the reviewer's own
worst error called for ('Sir John' → family='', executable); the H
Background carries the renunciate criterion (retains-a-family-name,
not religious-title — Rabbi Cohen measured correct); S2's ambiguity
flag is stated as firing in both readings with the assertable
example; decisions#H2's issue range narrowed to #343/#344. New
category: contested vocabulary memberships get Open blocks keyed to
the vocabulary set (#342/#346/#343/#344) and the rules.md preamble
names them as the third grep-invisible class. Excluded blocks gain
the non-CJK honorific-tails prohibition (जी, the 殿 criterion in
Devanagari) and ঠাকুর-is-Tagore under TITLES.
LOCALE-PACKS-PURE-DATA's contract states add-only — a pack can
never remove a base entry, which is why #342-class fixes must
change shipped vocabulary. W Background gains the script-member
no-op fact and the abugida divergence limit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Decision dates (not release dates; git over narrative), harvest-
landing provenance discipline (uncertainties resolved from the
source, never by inference), and the primary-source review protocol
-- each practiced and review-corrected during PR #381, none
previously written down.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@derek73 derek73 added this to the v2.2 milestone Aug 16, 2026
derek73 and others added 9 commits August 15, 2026 19:24
…rimary source)

H2's scope was FALSE as landed ('name-opening'): the shape rule
fires at the head of the part carrying the given name, post-comma
included ('Smith, Major. John' → title='Major.') — a correction
PR #315 made to the older docs two weeks before this branch, which
the extraction failed to pick up; H3 had the identical narrowing
('Morse, Det. Insp. Jane' now pins it). H2 also states its
precedence over the suffix vocabulary ('Esq. Smith' → title), with
the matching named exception carved into TWO-LAYER-ASSIGN's
contract. C1's lenient/strict bullet reattributed to the v2 Policy
commits per git author dates. Two new deviates: markers carry the
settled-but-unshipped bundle intent (#296 'Smith, PhD', #291's
family-name loss made visible). New: the comma-suffix-arc decisions
section (milestone moves, two Declined-with-limits), the trailing-
titles Open (#316) with the surname-collision Excluded entry,
VOCABULARY-FEEDS-STRUCTURE, the out-of-vocabulary corpus limit
field note, and the esq singleton-intersection clause.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(primary source)

The comma-suffix arc gains its true origin (#291 filed 2026-07-26)
and the missing earlier Declined -- splitting multi-word entries
into single words, declined on the name-swallowing measurements in
the issue body, which is why the seven entries were removed rather
than split. New Excluded block for the eight dead multi-word
entries (with the chargé/charge split and its precedent); 'born'
recorded as a never-shipped drafting invention; the never-given
ENCODING and freeze rationales (#293) land beside the particle
curation block. S Background states the per-word matching truth the
docs assumed everywhere. FACADE-CONTRACT closes the stays-warning-
free misreading; FOLDED_TAG gains its consumer-side strip
(Parser.revise, with the measured reordering hazard); two new
mechanisms from the session's fixes (WARN-AT-THE-CALLER,
LEGACY-STATE-SIGNATURE); C1's #296 marker gains the title-field
note so a future measurement doesn't misread it as stale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ry source)

Two Background falsifications corrected: the H criterion was
conflating list membership (how the title ADDRESSES -- the Arabic
honorifics qualify while fully retaining family names) with the
separate #346 empty-family question (surname retention); the N
Background still called the typographic pairs caller-opt-in two
rounds after #273 shipped them as defaults. FACADE-CONTRACT's
'identical results' gains the RFC's own release-log-classified
carve-out (Kennedy-nickname measured). New decisions sections from
the arc: N2's cross-pair mis-extraction history with its Declined
offset-filter, ma-do, deviates-registry (option C -- why packs
carry no callables, now cross-linked from LOCALE-PACKS and
FORCE-A-DECISION), normalization-fold (the deliberate lower/casefold
asymmetry), render-default (the declined lossless née template),
and P3's Google Code provenance with the GC-namespace note on
legacy-rule-numbers. M1's canonicalization bullet gains its date,
origin, and the through-patches ruling; C1 gains the i/v blast
radius. Excluded: curly single quotes; the 2026-07-19
transliteration deferrals (Arabic, Greek, Ottoman, Hebrew,
sri/shri). N2 states the dangling-suppression filter A1 depends on;
R Background names the default view.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rce)

Zero falsifications this round; all additive. New rule A2 (content-
free input parses to the empty name; born-empty ambiguities
survive, with the v1 contrast and the review-fix history).
normalization-fold gains the fixed-point requirement and the
_title_key corollary (the shipped-then-fixed silently-inert key).
New decisions: given-name-titles' twice-declined validation (the
'sir and dame' counterexample), N2's why-only-the-apostrophe
(Derek's double-quote distinguishing test), and the two-corpus
provenance (2026-07-24: tracker-harvested corpus because v1 test
banks are structurally blind to 2.0 additions). New mechanism:
AMBIGUITY-AT-THE-DECISION-SITE (every deciding stage carries an
emitter; a branch that changes nothing is not a decision). Two
verification field notes: guard-the-family parametrization, and
growth-guard calibration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rimary source)

P3 falsified and corrected: the Latin-capital initial veto on
single-letter connectives is now stated ('Jose E Maria Santos'
pinned) with the script asymmetry as an Accepted consequence —
Cyrillic blessed by #267's closure, the Latin half never separately
adjudicated (recorded as such). C section gains which commas count
(U+060C and U+FF0C segment, U+3001 does not — #265) with executable
examples. Dates disciplined per the convention: M2 and O2 now
distinguish design date from issue and landing; T1 credits #266 for
bidi existing at all; O4 gains its #270-body provenance and the
free-form-tuples Declined. W1 records the opt-in→default-on delta
from the filed proposal and the JMnedict Declined (staleness, no
frequency data, licensing). The stale Open-on-closed-#270 blocks in
O1/O2 become flagged status notes; nee's silently-resolved risk gets
its contrast entry; no_vowels' removal is recorded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(primary source)

normalization-fold's 'documented 1.4 deviation' inverted the
history: v1.4 has no casefold anywhere (measured against the tag) —
casefold is a classified 2.0 change, and the 1.x line was symmetric
BY DESIGN, which strengthens the do-not-fix-symmetric warning with
a real alternative rather than a hypothetical. The stale
empty_attribute_default 3-0 bullet (promoted from a memory already
three days stale at promotion) is replaced by the actual removal
record under removed-v1-surface (#44 origin, the z-Smith in-band-
signaling bug, substitute-before-format, tombstone-not-deletion,
readers-not-just-writers). New comparison-surface section: the
eq/hash trilemma, the cross-constants asymmetry, and two Declined
entries including the deliberately-issueless lenient-matching
resolved-as-no. The bridge discipline (warn in a released version
first) becomes a standing 3-0 rule; WARN-AT-THE-CALLER gains its
first measured instance; comparison joins rules.md's Not-in-scope
with a pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sweeping session memories against the new docs surfaced one more
falsification from the docs' own author: P2's 'join runs to the end'
is wrong for multi-group names -- a second particle starts a new
group and groups distribute positionally (the v1 Portuguese tests
pin it; 'Vincent van Gogh van Beethoven' → middle='van Gogh'), with
the #132 combined reading living in the surnames view (now
assertable and pinned). Promoted from memories: O1's opt-in
rationale (PR #154's Martin/Franklin/Benjamin finding), the
comma-suffix bundle's ship+repair amendment, the 2026-07-06
api-churn declines and the 3.0 config parking lot (contextvars
scoped override; Parser(constants) with #262's contract framing),
the bytes removal record, the property-key pickle skip's
dies-with-shim note, and the tags-are-ground-truth verification
field note (the twice-recurred 1.2.1-shim mix-up).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gitignored specs' residue sweep landed: the retired #271/#272
slug taboo corrected (the selector was deleted by #333 AFTER the
review round that landed the taboo -- the tree moved under the
claim), the JMnedict licensing simplification fixed per the
amendment's own correction, and nine residue items: the 19-word
audit table with Derek's framing thesis (intent for unshipped v2.2
work whose only record was gitignored), the A6 phrase-peel note,
new rule P5 (bound given names -- closing a known extraction gap)
with its Excluded entries, the R2 all-particles core/facade
divergence recorded as unadjudicated, NFC-for-classification-only,
the initials-repertoire declines (#320), the script-table placement
reversal with the 〆 rider, Turkic orthography (dotless ı), the
declined script-resolved ambiguity kinds, and the bridge-fairness
converse (a warning must wait when the replacement doesn't exist).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
derek73 and others added 4 commits August 16, 2026 13:08
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… memory, again

All cited commit SHAs resolve with matching subjects; four entries
carried reviewer-remembered dates one day off the git author dates
(the round-10 session asserted 07-19 'per git author dates' for
commits git dates 07-18 -- same-day for the N2 arc, which
strengthens its own story), and P3's Google Code provenance is
2013, not 2014. Fourteen delta issue/PR citations all verify.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The N3 contradiction resolved by measurement (decisions.md and the
_assign docstring said 'one non-title piece'; the code counts one
piece IN TOTAL, titles included -- "'Smitty' Dr. Jones" proves it,
and rules.md's Accepted consequence was right all along). W1 gains
the interpunct division-suppression Accepted (vocabulary and
segmenter alike stand down on 安东尼·陈志明, per #298) and W3 stops
implying otherwise. The 19-word audit table recovers its lost 19th
word from the spec (se: drop from TITLES, the PE/SE post-nominal
argument). The ma-do entry credits the right guard for 'Jack Ma';
the Anh Do example gains its contested framing (#385); the A2
citation moves out of mid-sentence; _group's docstring lists P4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant