Repository navigation
Accept dangling pointers and cut parse memory - #16
Merged
Merged
Conversation
loads() no longer raises for a pointer that matches no record; validate() reports each one as a dangling-pointer error. GedcomStructure uses slots, tags are interned, and lines are read one at a time, halving peak memory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Validation currently misses dangling pointers on extension and non-pointer-typed structures.
Review effort: Balanced
Findings: 1
What changed in this PR
Updates parsing to preserve dangling pointers while reducing memory usage.
Changes:
- Defers dangling-pointer detection to validation.
- Streams line splitting, interns tags, and slots structures.
- Adds documentation and regression coverage.
| File | Description |
|---|---|
gedcom7/parser.py |
Reduces parsing allocations and preserves unresolved pointers. |
gedcom7/types.py |
Enables dataclass slots. |
test/test_parser.py |
Tests line-splitting equivalence. |
test/test_conformance.py |
Tests dangling-pointer behavior. |
README.md |
Documents the behavior change. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
validate() checked a pointer only where the specification gives the structure a pointer payload. Now that loads() keeps pointers to missing records, a pointer on an extension, on a value or no-payload structure, or on a record must be reported too. The target's record type is still checked only where the payload names one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Dangling pointers
loads()/load()no longer raiseGedcomParseErrorfor a pointer that matches no record. The pointer is kept as written, andvalidate()reports each one as adangling-pointererror with its path (e.g.@F1@ FAM > WIFE).Until now, one broken pointer stopped the whole parse, so a consumer such as gramps-gedcom7 could neither list all broken pointers nor import the rest of the file. Duplicate cross-reference identifiers still raise, because with duplicates the parser cannot tell which record a pointer means.
Behaviour change: code that relied on
loads()to reject such files must now callvalidate().Memory
GedcomStructureis aslots=Truedataclass.Measured on a synthetic 35 MB file (100,000 individuals, 1.8M lines):
Behaviour change: because of
__slots__,GedcomStructureinstances no longer accept attributes beyond their fields, have no__dict__, and cannot be weakly referenced.Tests
@VOID@parses, and thatvalidate()reports exactly the three.re.splitbehaviour on every string overa,\r,\nup to length 7.🤖 Generated with Claude Code