Repository navigation
Remove the structure filter: dead under cost-gap admission after #365 #366
Copy link
Copy link
Open
Description
Summary
After #365 the structure filter measures its threshold from the pre-history #1. Re-measured under that rule, the pre-registered deletion check holds on all three conditions, so the filter looks dead under cost-gap admission and can be removed together with its supporting machinery.
The rule (pre-registered before #365, evaluated right after it)
Delete the filter if all of:
- Page 1 (first 9 candidates, n=20) differs from a filter-off oracle (
structure_cost_filter = 10^9) on ≤ 1 of the corpus readings - Replaying the commit log's rank>0 selections, the owner breakdown of what gets selected differs from the oracle by 0
- No filter-only drop (a dropped path within the cost-gap bound, which admission alone would have kept) lands in page-1 range
Measured (main 450fa55 + #365, frozen commit-log snapshot, all arms at the same time; counts only)
| Condition | Result |
|---|---|
| 1. page 1 vs oracle | differs on 0 / 142 corpus readings |
| 2. replay selections by owner vs oracle | identical, without history and with history |
| 3. filter-only drops in page-1 range | 0 (filter-only drops: 26 of 593 dropped paths, across 67 readings) |
Expectation before measuring was that the filter would stay (most surfaces it lets back in are junk); with the threshold measured from the #1 instead of the population minimum, admission already removes everything the filter used to catch on page 1.
Scope of the removal PR
structure_cost_filterhard filter inrerank, with what exists only for it: the prefix floor (structure_prefix_floor), the per-transition cap (structure_cost_transition_cap), single-segment imputation,StructureFilter/on_drop, and explain'sdropped_by_structure/structure_filter(added in fix(core): measure the structure filter from the #1, not the population minimum (#353) #365 as provisional).- Settings keys
structure_cost_filter/structure_cost_transition_cap: keep accepting them (on-disk settings must keep loading) and warn that they no longer have an effect. - Identity exemption (分節崩れ・接辞付き複合語の変換ミス #263) disappears with the filter; check that the typed-kana path stays selectable through admission (it is exempt there too).
- Re-run both accuracy corpora, the 3-width top-1 check, replay (no history / history) against main, and the cost-gap T rule (
DEFAULT_CANDIDATE_MAX_COST_GAP): removing the filter changes the population admission sees. - Speed ±5% on the 1-best path (removing work, expected neutral or faster).
Order
Before #362 (PR-F, single_char_kanji_penalty), so PR-F is measured on the final candidate pipeline rather than one whose filter is about to go. Then #361 (PR-E2).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity