Repository navigation
fix(types): defer unresolved forward refs in canonical subclasses - #1690
justadreamer wants to merge 1 commit into
Conversation
The legacy-identity removal in CanonicalBoundaryModel.__pydantic_init_subclass__ forces model_rebuild with raise_errors on, so a subclass annotated with a model defined later in its module raised PydanticUndefinedAnnotation at class creation. Plain pydantic defers such a class until model_rebuild(); the removal now does the same.
IPR Policy Agreement Required@justadreamer — thanks for the contribution. Before this PR can be merged, the AgenticAdvertising.Org IPR Policy requires your agreement. To agree, post a new comment on this PR with the exact phrase: Your signature is recorded once and covers all contributions to AAO repositories. See |
|
Thanks @justadreamer, this is a good catch and the fix looks right. Deferring an unresolved forward reference in the removal rebuild matches plain pydantic, and the regression test covers it. I've approved the CI runs. Once you've posted the IPR agreement comment above and CI is green, we'll merge it for the next 9.3.x patch. |
…t 560 MiB RSS Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions realizes the type surface through adcp.types._eager, and on 9.3.0 that peaked at ~495 MiB RSS (~560 MiB for adcp.client) against ~130 MiB on 6.6. Two things built pydantic-core validators at import for models a process may never use: - The import-time patchers (_ergonomic, _forward_compat, canonical_creative, aliases) edited model_fields and then called model_rebuild(force=True) on ~35 request/response roots. Pydantic does not share nested defer_build schemas between parents, so each forced rebuild walked the model's whole subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the import. They now call adcp.types.base.defer_model_rebuild(), which puts the model back into the defer_build placeholder state so the patched fields take effect on first use. Same outcome, later moment. - A generated `class X(RootModel[A | B])` built the union's validator at the RootModel[...] expression, before X or any config it declares existed; adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone cost ~60 MiB. Generated composing roots now parametrize adcp.types.base.AdCPRootModel, which carries defer_build=True, via the new post-generate fixer defer_rootmodel_builds. Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12): adcp.types._eager 495 -> 114 MiB (import 14.5 s -> 4.1 s) adcp.client 561 -> 175 MiB adcp.server 565 -> 183 MiB adcp.server.test_controller 562 -> 183 MiB adcp.decisioning 573 -> 187 MiB adcp.compat.versions 498 -> 118 MiB The cost moves to first use: the first validation of a deferred root builds its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's JSON schema), and a process that has used every one of them sits at ~215 MiB instead of ~520 MiB. build_all_models() still realizes everything eagerly for freezegun callers. tests/test_import_memory.py imports each entry module in a subprocess and fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades the deferral contract, including that a canonical subclass may forward- reference a model defined later in its module (supersedes #1690). Fixes #1694 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t 560 MiB RSS Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions realizes the type surface through adcp.types._eager, and on 9.3.0 that peaked at ~495 MiB RSS (~560 MiB for adcp.client) against ~130 MiB on 6.6. Two things built pydantic-core validators at import for models a process may never use: - The import-time patchers (_ergonomic, _forward_compat, canonical_creative, aliases) edited model_fields and then called model_rebuild(force=True) on ~35 request/response roots. Pydantic does not share nested defer_build schemas between parents, so each forced rebuild walked the model's whole subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the import. They now call adcp.types.base.defer_model_rebuild(), which puts the model back into the defer_build placeholder state so the patched fields take effect on first use. Same outcome, later moment. - A generated `class X(RootModel[A | B])` built the union's validator at the RootModel[...] expression, before X or any config it declares existed; adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone cost ~60 MiB. Generated composing roots now parametrize adcp.types.base.AdCPRootModel, which carries defer_build=True, via the new post-generate fixer defer_rootmodel_builds. Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12): adcp.types._eager 495 -> 114 MiB (import 14.5 s -> 4.1 s) adcp.client 561 -> 175 MiB adcp.server 565 -> 183 MiB adcp.server.test_controller 562 -> 183 MiB adcp.decisioning 573 -> 187 MiB adcp.compat.versions 498 -> 118 MiB The cost moves to first use: the first validation of a deferred root builds its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's JSON schema), and a process that has used every one of them sits at ~215 MiB instead of ~520 MiB. build_all_models() still realizes everything eagerly for freezegun callers. Deferring more models exposed a pydantic limitation that already applied to every defer_build model: pydantic-core reaches a serializer by reading instance.__pydantic_serializer__ when it dumps by inference (an Any-typed field, serialize_as_any, a plain-pydantic parent), and a placeholder there raised TypeError instead of building. Every deferred SDK model now carries placeholders that are also descriptors (_DeferredValSer), installed by AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by defer_model_rebuild, so that read builds the model and returns the real serializer. This also fixes the pre-existing TypeError for an unvalidated or parent-built instance of any generated model dumped through such a parent. tests/test_import_memory.py imports each entry module in a subprocess and fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades the deferral contract, including that a canonical subclass may forward- reference a model defined later in its module (supersedes #1690). Fixes #1694 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t 560 MiB RSS Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions realizes the type surface through adcp.types._eager, and on 10.0.0-beta.2 that peaked at ~517 MiB RSS (~590 MiB for adcp.client) against ~130 MiB on 6.6. Two things built pydantic-core validators at import for models a process may never use: - The import-time patchers (_ergonomic, _forward_compat, canonical_creative, aliases) edited model_fields and then called model_rebuild(force=True) on ~35 request/response roots. Pydantic does not share nested defer_build schemas between parents, so each forced rebuild walked the model's whole subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the import. They now call adcp.types.base.defer_model_rebuild(), which puts the model back into the defer_build placeholder state so the patched fields take effect on first use. Same outcome, later moment. - A generated `class X(AdCPRootModel[A | B])` built the union's validator at the subscript, before X or any config it declares existed, because the #1613 base carried pydantic's default config; adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone cost ~60 MiB. AdCPRootModel now carries defer_build=True, so the parametrized class and X both defer. Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12): adcp.types._eager 517 -> 121 MiB (import 13.9 s -> 3.6 s) adcp.client 588 -> 189 MiB adcp.server.test_controller 610 -> 214 MiB adcp.decisioning -> 224 MiB The cost moves to first use: the first validation of a deferred root builds its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's JSON schema), and a process that has used every one of them sits at ~215 MiB instead of ~520 MiB. build_all_models() still realizes everything eagerly for freezegun callers. Deferring more models exposed a pydantic limitation that already applied to every defer_build model: pydantic-core reaches a serializer by reading instance.__pydantic_serializer__ when it dumps by inference (an Any-typed field, serialize_as_any, a plain-pydantic parent), and a placeholder there raised TypeError instead of building. Every deferred SDK model now carries placeholders that are also descriptors (_DeferredValSer), installed by AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by defer_model_rebuild, so that read builds the model and returns the real serializer. This also fixes the pre-existing TypeError for an unvalidated or parent-built instance of any generated model dumped through such a parent. tests/test_import_memory.py imports each entry module in a subprocess and fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades the deferral contract, including that a canonical subclass may forward- reference a model defined later in its module (the SDK 10 port of #1704; #1690 on main is subsumed). Fixes #1694 for SDK 10; main carries the same fix as #1704. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t 560 MiB RSS (#1704) Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions realizes the type surface through adcp.types._eager, and on 9.3.0 that peaked at ~495 MiB RSS (~560 MiB for adcp.client) against ~130 MiB on 6.6. Two things built pydantic-core validators at import for models a process may never use: - The import-time patchers (_ergonomic, _forward_compat, canonical_creative, aliases) edited model_fields and then called model_rebuild(force=True) on ~35 request/response roots. Pydantic does not share nested defer_build schemas between parents, so each forced rebuild walked the model's whole subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the import. They now call adcp.types.base.defer_model_rebuild(), which puts the model back into the defer_build placeholder state so the patched fields take effect on first use. Same outcome, later moment. - A generated `class X(RootModel[A | B])` built the union's validator at the RootModel[...] expression, before X or any config it declares existed; adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone cost ~60 MiB. Generated composing roots now parametrize adcp.types.base.AdCPRootModel, which carries defer_build=True, via the new post-generate fixer defer_rootmodel_builds. Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12): adcp.types._eager 495 -> 114 MiB (import 14.5 s -> 4.1 s) adcp.client 561 -> 175 MiB adcp.server 565 -> 183 MiB adcp.server.test_controller 562 -> 183 MiB adcp.decisioning 573 -> 187 MiB adcp.compat.versions 498 -> 118 MiB The cost moves to first use: the first validation of a deferred root builds its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's JSON schema), and a process that has used every one of them sits at ~215 MiB instead of ~520 MiB. build_all_models() still realizes everything eagerly for freezegun callers. Deferring more models exposed a pydantic limitation that already applied to every defer_build model: pydantic-core reaches a serializer by reading instance.__pydantic_serializer__ when it dumps by inference (an Any-typed field, serialize_as_any, a plain-pydantic parent), and a placeholder there raised TypeError instead of building. Every deferred SDK model now carries placeholders that are also descriptors (_DeferredValSer), installed by AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by defer_model_rebuild, so that read builds the model and returns the real serializer. This also fixes the pre-existing TypeError for an unvalidated or parent-built instance of any generated model dumped through such a parent. tests/test_import_memory.py imports each entry module in a subprocess and fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades the deferral contract, including that a canonical subclass may forward- reference a model defined later in its module (supersedes #1690). Fixes #1694 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
Heads-up: #1704 (merged to main as aac52ca, for #1694) replaced the forced rebuild in |
|
I have read the IPR Policy |
|
Fixed on main by #1704: the subclass hook now defers the rebuild instead of forcing it, and |
IPR Policy — signedThanks, @justadreamer. Your agreement to the IPR Policy is recorded at |
…t 560 MiB RSS (#1713) Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions realizes the type surface through adcp.types._eager, and on 10.0.0-beta.2 that peaked at ~517 MiB RSS (~590 MiB for adcp.client) against ~130 MiB on 6.6. Two things built pydantic-core validators at import for models a process may never use: - The import-time patchers (_ergonomic, _forward_compat, canonical_creative, aliases) edited model_fields and then called model_rebuild(force=True) on ~35 request/response roots. Pydantic does not share nested defer_build schemas between parents, so each forced rebuild walked the model's whole subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the import. They now call adcp.types.base.defer_model_rebuild(), which puts the model back into the defer_build placeholder state so the patched fields take effect on first use. Same outcome, later moment. - A generated `class X(AdCPRootModel[A | B])` built the union's validator at the subscript, before X or any config it declares existed, because the #1613 base carried pydantic's default config; adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone cost ~60 MiB. AdCPRootModel now carries defer_build=True, so the parametrized class and X both defer. Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12): adcp.types._eager 517 -> 121 MiB (import 13.9 s -> 3.6 s) adcp.client 588 -> 189 MiB adcp.server.test_controller 610 -> 214 MiB adcp.decisioning -> 224 MiB The cost moves to first use: the first validation of a deferred root builds its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's JSON schema), and a process that has used every one of them sits at ~215 MiB instead of ~520 MiB. build_all_models() still realizes everything eagerly for freezegun callers. Deferring more models exposed a pydantic limitation that already applied to every defer_build model: pydantic-core reaches a serializer by reading instance.__pydantic_serializer__ when it dumps by inference (an Any-typed field, serialize_as_any, a plain-pydantic parent), and a placeholder there raised TypeError instead of building. Every deferred SDK model now carries placeholders that are also descriptors (_DeferredValSer), installed by AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by defer_model_rebuild, so that read builds the model and returns the real serializer. This also fixes the pre-existing TypeError for an unvalidated or parent-built instance of any generated model dumped through such a parent. tests/test_import_memory.py imports each entry module in a subprocess and fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades the deferral contract, including that a canonical subclass may forward- reference a model defined later in its module (the SDK 10 port of #1704; #1690 on main is subsumed). Fixes #1694 for SDK 10; main carries the same fix as #1704. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Why
CanonicalBoundaryModel.__pydantic_init_subclass__removes inherited legacy creative identity and then callsmodel_rebuild(force=True), which raises on any annotation it cannot resolve yet. A subclass of a canonical model whose field names a model defined later in the same module therefore fails at class creation:Plain pydantic accepts this and defers the class until
model_rebuild(). Sellers that extend the wire models (e.g. adding their own fields onPackageRequestbeside a locally extended creative model) hit it on adcp 9.x.What changed
model_rebuild(force=True, raise_errors=False)in the removal hook, so an unresolved forward reference leaves the class incomplete instead of raising. The removal itself still runs, and the caller's latermodel_rebuild()completes it.tests/test_canonical_removed_fields.pythat defines the referenced model after the subclass, rebuilds, and checks the removed fields stay absent.Testing
mainwithPydanticUndefinedAnnotationand passes with the change.tests/test_canonical_removed_fields.py,test_canonical_boundary_validation.py,test_canonical_boundary_serialization.py: 114 passed.ruff checkclean on the touched files.