Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,14 @@ Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py):
- 2026-08 #319 — the wholly-suffix predicate was lifted into the vocabulary layer so the comma decision and the honorific peel's segment test cannot drift apart.
- 2026-08-23 #296/#325 — the listing form's post-comma part is read for what it is. Three commits of the approved #291/#296 bundle (spec 2026-07-30, plan 2026-08-01; see the comma-suffix arc below), in the plan's order. (1) A comma followed only by titles fixed no family boundary, so the pre-comma name keeps its positional read: `John Smith, Mr.` reads given 'John', family 'Smith', title 'Mr.' where 2.0 had merged the family ('Bob Jones, author' moves with it at every baseline). "Nothing but titles" is title-shaped AND not suffix vocabulary — the plan's helper said only the first, and would have read `Smith, PhD Jr.` as the title run 'PhD Jr.' and `Smith, Mr. Jr.` as 'Mr. Jr.' (both master's suffix 'Jr.'; the gate and a measurement against master caught them). (2) The TITLES/suffix overlap audit and the ordering, one commit because each alone is wrong (the plan's F1): reading the credential run first would make `Smith, Dr.` a suffix while 'dr' was SUFFIX_WORDS residue, and the audit alone leaves `Smith, Jr.` a title by the period-abbreviation inference. The audit table's dispositions ship as approved — 'do', 'jr', 'junior', 'phd', 'se' out of TITLES; 'dr' and 'sra' out of the suffix sets; 'ms' and 'sa' into the ambiguous set — with ONE deviation on evidence the table did not have: 'md' keeps dual membership, because bare 'Md' before a name is the Bengali and South Asian Muslim abbreviation of Muhammad (#343/#345's corpus rows `Md Abdul Karim`, `Md. Abdul Karim`; the drop read the first as given 'Md', middle 'Abdul') while 'MD' after the name is the degree — position decides, as for 'sr'. The ordering is wider than the plan's lone piece: a post-comma part that is nothing but suffix pieces is the credential run, whole, which is what #325 needed (`Smith, Ph. D. Jr.` had put the split credential in the given name once a second suffix made it no longer lone — a 1.4.0 regression; v1 read suffix 'Ph. D.', title 'Jr.'). Vocabulary decides which words qualify and position breaks the tie for the genuine duals: `Smith, Sr.` is Senior where `Sr. Garcia` is Señor, `Smith, Esq.` is the postnominal where H2 still reads `Esq. Smith` as a title (H2 carries that as Accepted). `Smith, Dr. Jr.` keeps the 2.0 deviation's other case — a title, then the last piece as the suffix (v1 read first 'Jr.'; 2.0 through master suffix 'Jr.') — and `Smith, John Jr.` its given name. Consequences the audit carries, all measured and pinned: a leading 'PhD' or 'Jr' is a name word; `Do Quang Minh` reads given 'Do' with the particle-or-given fork, and `Dr. Do Van Johnson, MD` given 'Do', family 'Van Johnson' (the #367 rule for that name is replaced; TITLES ∩ ambiguous is {freiherr, st} now, which keeps the chained emitter reachable); a trailing bare 'Dr' is a name word, joining every other title-only word there (#316 is the open question); `田中さん, PhD` reads suffix 'さん, PhD'; the #100 prefix-join tests use 'lt', a genuine dual, as their token. #291 (multi-word credentials) is the bundle's fourth commit and its own PR.
- 2026-08-23 (review of PR #428, four reviewers) — the repair's condition is "no name word", not "all titles": `John Smith, Mr. Jr.` (a title and a postnominal) had kept the merge the rationale argues against, and reads given 'John', family 'Smith', suffix 'Jr.' now. Its guard counts NAME pieces: the positional read peels a trailing suffix first, so `Smith Jr., Mr.` — two pieces, one name — read positionally lost its family (the code review); it keeps family 'Smith', suffix 'Jr.' at every baseline. The positional read records its ORDER, and post_rules' leading-piece scan keys on "assign records no order after a family comma" rather than on the structure, so P1's family-first fold reaches the pre-comma name as it does without the comma — `de Mesnil Juan, Dr.` under a family-first order had read family 'de' (the test review). The credential run does NOT step over a delimiter core the policy names (`Smith, RN - CRNA` reads given 'RN' under `extra_suffix_delimiters` as without it): the code review asked for it, a draft did it, and tests/test_suffixes.py's "the delimiter must not affect parsing at all" outside the suffix-comma form is v1 parity pinned on purpose — C1 carries the limit as Accepted. A second deviation from the audit table: `ms` and `sa` do NOT join the ambiguous set — the gate is position-blind and the collision is not; gated, `John Smith, MS` lost its suffix-comma route and read title 'MS', and `Smith, Ms.` passed the gate on its one period and read as a credential anyway. Both are genuine duals, and position decides: `Smith, Ms.` reads suffix 'Ms.' (title at every baseline; C1 carries it as Accepted, with `Smith, Ms. Jane` keeping the title) and `John Smith, MS` keeps suffix 'MS'. And the gate itself now means what S2 says: an ambiguous acronym counts when written with ITS periods, one after each letter (`M.A.`, `J.D.`), not on any period — `Smith, Ed.` had become a credential on a name's trailing period, and `Jack Ma.` had read suffix 'Ma.' since 2.0 (S2 carries the boundary; 1.4.0 read family 'Ma.'). Two limits recorded rather than fixed: `田中さん, Dr.` keeps the honorific glued, joining master's `田中さん, Mr.` — the honorific peel runs in script_segment on the suffix-comma and no-comma structures, before group or assign can say the comma fixed nothing (C1 Accepted); and group's chain emitter still reports no PARTICLE_OR_GIVEN on the family-comma path, so `Freiherr von Berg Ma, Mr.` reads as its no-comma twin but without the report — group runs before assign decides which read applies, and plumbing a deferred report through the state is more than the shape is worth today.
- 2026-08-23 #429 — one comma segment is one suffix entry, and the two stages disagreed about which segments those are. group asked by segment INDEX (`tail_start`: 1 after a suffix comma, 2 after a family comma), which is right wherever assign reads the segment as suffixes for the same structural reason — but not after a family comma, where segment 1 is a name slot assign re-reads by CONTENT with `_segment_holds_no_name` and routes piece by piece (#296/#325). So the roles were right and the render was not: `Smith, MD PhD` gave suffix 'MD, PhD' with a comma the writer never typed, and `Smith, CBE MC`, `Smith, BSc MBA` and `Smith, PhD Jr.` the same, while the full-name `John Smith, MD PhD` has rendered 'MD PhD' since 1.4.0. The predicate moved down into group beside the piece predicates it already reads — the same direction `_leading_titles` moved in #424, and the only direction available, since assign imports group.
SCOPE, stated as the condition actually reads rather than as the first draft described it: any family comma whose following segment holds no name word, NOT only a one-word family. `John Smith, Jr. III` renders suffix 'Jr. III' (1.4.0's reading), and a title-led segment is in too — `Smith, Dr. MD PhD` renders 'MD PhD'. The differential cannot witness the widening: of 1069 corpus names, 303 carry a structure comma, the predicate is true for 36, only 9 of those have two or more pieces in the segment, and none of the 9 combines a multi-word pre-comma name with two suffix pieces. 0 unexplained is therefore evidence about the corpus, not about the class — the case table carries it instead.
THE REGRESSION THE REVIEW FOUND, and the reason this entry is long: the first draft let ANY piece open an entry, as the tail block always had. That is safe on a tail segment, where assign routes every piece to SUFFIX — which is what `tail` means — and wrong off it, where a title piece routes to TITLE. Two failures, one cause. The `joined` tag is role-BLIND and the facade heals it for every role, so a title opening an entry collapsed `title_list` from ['Rev.', 'Dr.'] to ['Rev. Dr.'] on `Smith, Rev. Dr.`; and where the pre-comma name left a suffix, the tag glued the next suffix BACKWARD across the writer's own comma — `Smith Jr., Mr. Jr.` rendered suffix 'Jr. Jr.', the exact inverse of the bug being fixed. Neither is visible to the differential, which compares strings, nor to the case table as it stood, which asserts the title STRING (space-joined either way). The rule that replaces it separates two joins that had been one: WITHIN a piece the tag renders a merged piece as one unit, whatever role it holds; BETWEEN pieces it continues an entry, and only a piece that renders into the same run may do that. The entry is sticky across a piece that is not in it, so an interleaved title does not split the run it sits in (`Smith, MD Dr. PhD` renders 'MD PhD'); a delimiter core still closes it, which is the one thing that separates two entries.
Scoped deliberately, and the scoping is the other decision: `tail` still gates delimiter-core dropping, which is v1 `expand_suffix_delimiter` parity (#206, commit 021823e — three code comments say #191, which is the German/Dutch vocabulary PR, and are corrected here) and a TAIL rule. The core set is threaded to the block as `seg_cores` so the widened condition cannot drop cores where `tail` says not to. Worth knowing before anyone tries to pin that: for every realistic delimiter the two spellings are indistinguishable by construction, because the core must first pass the very predicate that gates the block — the only reachable class is a policy whose delimiter core is itself suffix or title vocabulary (`Policy(extra_suffix_delimiters={" jr "})`, `Smith, MD jr`), which the case table does not enshrine.
Cost, measured rather than optimised away: `_segment_holds_no_name` now runs TWICE per family-comma parse, once here and once in assign, on the same pieces — verified pure recomputation, not two questions (408 of 506 comma-bearing test literals reach both sites, 0 disagreements, and off-tail `seg_cores` is empty so `kept` never filters, which is why assign sees exactly the pieces group tested). Removing the second call is worth 0.6–1.4 µs, 1.2–2.2% of a family-comma parse and 0% of every other parse (NO_COMMA and SUFFIX_COMMA make no call at all). Caching it needs a derived ParseState field the two sites would otherwise not share, so the number is recorded instead. The order is cheapest-first per 7009fcf: two bool loads and an int compare gate the call.
R1 gained the separator sentence with this change. That the invariant it states — one comma segment is one suffix entry — was written down in no rule is a large part of why two stages disagreed about it for years: C1 governs routing, R1 governed word ORDER and never mentioned separation, and the only statement of the rule was a code comment.
Left alone, measured and pre-existing: the NO-COMMA path comma-joins a space-separated run too (`John Smith MD PhD` renders 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and master alike). The fix requires FAMILY_COMMA and does not reach it; that path has no writer-drawn boundary and is a separate question, recorded here and NOT yet filed. It is load-bearing for one thing: `str()` of a fixed parse is a no-comma string, so re-parsing it re-inserts the comma — round-tripping is not stable on these shapes, and an earlier draft of the release bullet claimed it was.
The name `Smith, MD PhD` entered the corpora as this fix's C1 example — a FIRING one, carrying no `· boundary` annotation, which marks the non-firing example a rule must have — so its ledger entries arrive with it. A fourth written for the 1.4.0 ledger EXPLAINED NOTHING and was dropped: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name there, its fields covering all three that move. That is #373's dormant-rule diagnosis doing its job, and #426 is the precedent for dropping rather than keeping a shadowed rule.

### T1 — separators, not joiners

Expand Down
14 changes: 11 additions & 3 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -623,7 +623,8 @@ C1. Rationale: a credential run after the comma means the name is in
"Smith, PhD" → suffix="PhD"
"Smith, Jr." → suffix="Jr."
"Smith, Sr." → suffix="Sr."
"Smith, Ph. D. Jr." → suffix="Ph. D., Jr."
"Smith, Ph. D. Jr." → suffix="Ph. D. Jr."
"Smith, MD PhD" → suffix="MD PhD"
"Smith, Dr." → title="Dr."
"Smith, Dr. Jr." → suffix="Jr."
"John Smith, Mr." → given="John"
Expand All @@ -648,7 +649,7 @@ C1. Rationale: a credential run after the comma means the name is in
form alone, and that limitation is kept as parity: "Smith, RN -
CRNA" reads given "RN" under the policy as without it.
"John Smith, LEED AP" → family="Smith" deviates: #291 (today: family="John Smith")
history: decisions.md#C1 · interacts: H2, P6 · implemented: nameparser/_pipeline/_segment.py, nameparser/_pipeline/_assign.py
history: decisions.md#C1 · interacts: H2, P6 · implemented: nameparser/_pipeline/_segment.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py

C2. Rationale: text beyond the recognized comma parts should be
taken in without silent guessing.
Expand Down Expand Up @@ -874,10 +875,17 @@ R1. Rationale: a field is a way of reading the parse, not a stored
words — O3's fold and, since #379, P6's attached tussenvoegsel —
which render before the rest of the family wherever they stood in
the string.
Words are separated as the writer separated them. The suffix view
is the one place this is visible, because it is the only field
that can hold parts the writer comma-separated: a run of
post-nominals written with spaces renders with spaces, and one
written with commas keeps them.
"Dr. Juan Q. Xavier de la Vega III" → family="de la Vega"
"Hassan, Mohamad Ahmad Ali" middle_as_family → family="Ahmad Ali Hassan"
"Hassan, Mohamad Ahmad Ali" → family="Hassan" · boundary
implemented: nameparser/_types.py
"Smith, MD PhD" → suffix="MD PhD"
"John Smith, MD, Bart" → suffix="MD, Bart"
history: decisions.md#C1 · implemented: nameparser/_types.py

R2. Rationale: callers need the surname with and without its
particles — sorting wants "Vega", display wants "de la Vega".
Expand Down
Loading