Skip to content

fix(types): defer unresolved forward refs in canonical subclasses - #1690

Closed
justadreamer wants to merge 1 commit into
adcontextprotocol:mainfrom
postindustria-tech:fix/canonical-subclass-forward-ref
Closed

justadreamer wants to merge 1 commit into
adcontextprotocol:mainfrom
postindustria-tech:fix/canonical-subclass-forward-ref

Conversation

@justadreamer

Copy link
Copy Markdown

Why

CanonicalBoundaryModel.__pydantic_init_subclass__ removes inherited legacy creative identity and then calls model_rebuild(force=True), which raises on any annotation it cannot resolve yet. A subclass of a canonical model whose field names a model defined later in the same module therefore fails at class creation:

from adcp.types import PackageRequest

class MyPackageRequest(PackageRequest):
    creatives: list["MyCreative"] | None = None   # PydanticUndefinedAnnotation: name 'MyCreative' is not defined

class MyCreative(...): ...

Plain pydantic accepts this and defers the class until model_rebuild(). Sellers that extend the wire models (e.g. adding their own fields on PackageRequest beside a locally extended creative model) hit it on adcp 9.x.

What changed

  • model_rebuild(force=True, raise_errors=False) in the removal hook, so an unresolved forward reference leaves the class incomplete instead of raising. The removal itself still runs, and the caller's later model_rebuild() completes it.
  • A regression test in tests/test_canonical_removed_fields.py that defines the referenced model after the subclass, rebuilds, and checks the removed fields stay absent.

Testing

  • The new test fails on main with PydanticUndefinedAnnotation and passes with the change.
  • tests/test_canonical_removed_fields.py, test_canonical_boundary_validation.py, test_canonical_boundary_serialization.py: 114 passed.
  • ruff check clean on the touched files.

The legacy-identity removal in CanonicalBoundaryModel.__pydantic_init_subclass__
forces model_rebuild with raise_errors on, so a subclass annotated with a model
defined later in its module raised PydanticUndefinedAnnotation at class
creation. Plain pydantic defers such a class until model_rebuild(); the removal
now does the same.
@github-actions

Copy link
Copy Markdown
Contributor

IPR Policy Agreement Required

@justadreamer — thanks for the contribution. Before this PR can be merged, the AgenticAdvertising.Org IPR Policy requires your agreement.

To agree, post a new comment on this PR with the exact phrase:

I have read the IPR Policy

Your signature is recorded once and covers all contributions to AAO repositories. See signatures/README.md for what gets recorded and why.

Copy link
Copy Markdown
Contributor

Thanks @justadreamer, this is a good catch and the fix looks right. Deferring an unresolved forward reference in the removal rebuild matches plain pydantic, and the regression test covers it. I've approved the CI runs. Once you've posted the IPR agreement comment above and CI is green, we'll merge it for the next 9.3.x patch.

bokelley added a commit that referenced this pull request Oct 29, 2026
…t 560 MiB RSS

Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions
realizes the type surface through adcp.types._eager, and on 9.3.0 that peaked
at ~495 MiB RSS (~560 MiB for adcp.client) against ~130 MiB on 6.6. Two
things built pydantic-core validators at import for models a process may
never use:

- The import-time patchers (_ergonomic, _forward_compat, canonical_creative,
  aliases) edited model_fields and then called model_rebuild(force=True) on
  ~35 request/response roots. Pydantic does not share nested defer_build
  schemas between parents, so each forced rebuild walked the model's whole
  subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the
  import. They now call adcp.types.base.defer_model_rebuild(), which puts the
  model back into the defer_build placeholder state so the patched fields
  take effect on first use. Same outcome, later moment.

- A generated `class X(RootModel[A | B])` built the union's validator at the
  RootModel[...] expression, before X or any config it declares existed;
  adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone
  cost ~60 MiB. Generated composing roots now parametrize
  adcp.types.base.AdCPRootModel, which carries defer_build=True, via the new
  post-generate fixer defer_rootmodel_builds.

Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12):

  adcp.types._eager            495 -> 114 MiB  (import 14.5 s -> 4.1 s)
  adcp.client                  561 -> 175 MiB
  adcp.server                  565 -> 183 MiB
  adcp.server.test_controller  562 -> 183 MiB
  adcp.decisioning             573 -> 187 MiB
  adcp.compat.versions         498 -> 118 MiB

The cost moves to first use: the first validation of a deferred root builds
its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's
JSON schema), and a process that has used every one of them sits at ~215 MiB
instead of ~520 MiB. build_all_models() still realizes everything eagerly for
freezegun callers.

tests/test_import_memory.py imports each entry module in a subprocess and
fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades
the deferral contract, including that a canonical subclass may forward-
reference a model defined later in its module (supersedes #1690).

Fixes #1694

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bokelley added a commit that referenced this pull request Oct 29, 2026
…t 560 MiB RSS

Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions
realizes the type surface through adcp.types._eager, and on 9.3.0 that peaked
at ~495 MiB RSS (~560 MiB for adcp.client) against ~130 MiB on 6.6. Two
things built pydantic-core validators at import for models a process may
never use:

- The import-time patchers (_ergonomic, _forward_compat, canonical_creative,
  aliases) edited model_fields and then called model_rebuild(force=True) on
  ~35 request/response roots. Pydantic does not share nested defer_build
  schemas between parents, so each forced rebuild walked the model's whole
  subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the
  import. They now call adcp.types.base.defer_model_rebuild(), which puts the
  model back into the defer_build placeholder state so the patched fields
  take effect on first use. Same outcome, later moment.

- A generated `class X(RootModel[A | B])` built the union's validator at the
  RootModel[...] expression, before X or any config it declares existed;
  adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone
  cost ~60 MiB. Generated composing roots now parametrize
  adcp.types.base.AdCPRootModel, which carries defer_build=True, via the new
  post-generate fixer defer_rootmodel_builds.

Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12):

  adcp.types._eager            495 -> 114 MiB  (import 14.5 s -> 4.1 s)
  adcp.client                  561 -> 175 MiB
  adcp.server                  565 -> 183 MiB
  adcp.server.test_controller  562 -> 183 MiB
  adcp.decisioning             573 -> 187 MiB
  adcp.compat.versions         498 -> 118 MiB

The cost moves to first use: the first validation of a deferred root builds
its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's
JSON schema), and a process that has used every one of them sits at ~215 MiB
instead of ~520 MiB. build_all_models() still realizes everything eagerly for
freezegun callers.

Deferring more models exposed a pydantic limitation that already applied to
every defer_build model: pydantic-core reaches a serializer by reading
instance.__pydantic_serializer__ when it dumps by inference (an Any-typed
field, serialize_as_any, a plain-pydantic parent), and a placeholder there
raised TypeError instead of building. Every deferred SDK model now carries
placeholders that are also descriptors (_DeferredValSer), installed by
AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by
defer_model_rebuild, so that read builds the model and returns the real
serializer. This also fixes the pre-existing TypeError for an unvalidated or
parent-built instance of any generated model dumped through such a parent.

tests/test_import_memory.py imports each entry module in a subprocess and
fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades
the deferral contract, including that a canonical subclass may forward-
reference a model defined later in its module (supersedes #1690).

Fixes #1694

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bokelley added a commit that referenced this pull request Oct 29, 2026
…t 560 MiB RSS

Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions
realizes the type surface through adcp.types._eager, and on 10.0.0-beta.2 that peaked
at ~517 MiB RSS (~590 MiB for adcp.client) against ~130 MiB on 6.6. Two
things built pydantic-core validators at import for models a process may
never use:

- The import-time patchers (_ergonomic, _forward_compat, canonical_creative,
  aliases) edited model_fields and then called model_rebuild(force=True) on
  ~35 request/response roots. Pydantic does not share nested defer_build
  schemas between parents, so each forced rebuild walked the model's whole
  subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the
  import. They now call adcp.types.base.defer_model_rebuild(), which puts the
  model back into the defer_build placeholder state so the patched fields
  take effect on first use. Same outcome, later moment.

- A generated `class X(AdCPRootModel[A | B])` built the union's validator at
  the subscript, before X or any config it declares existed, because the
  #1613 base carried pydantic's default config; adagents.AdcpAgentsAuthorization
  and core.format.CanonicalParameters alone cost ~60 MiB. AdCPRootModel now
  carries defer_build=True, so the parametrized class and X both defer.

Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12):

  adcp.types._eager            517 -> 121 MiB  (import 13.9 s -> 3.6 s)
  adcp.client                  588 -> 189 MiB
  adcp.server.test_controller  610 -> 214 MiB
  adcp.decisioning                 -> 224 MiB

The cost moves to first use: the first validation of a deferred root builds
its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's
JSON schema), and a process that has used every one of them sits at ~215 MiB
instead of ~520 MiB. build_all_models() still realizes everything eagerly for
freezegun callers.

Deferring more models exposed a pydantic limitation that already applied to
every defer_build model: pydantic-core reaches a serializer by reading
instance.__pydantic_serializer__ when it dumps by inference (an Any-typed
field, serialize_as_any, a plain-pydantic parent), and a placeholder there
raised TypeError instead of building. Every deferred SDK model now carries
placeholders that are also descriptors (_DeferredValSer), installed by
AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by
defer_model_rebuild, so that read builds the model and returns the real
serializer. This also fixes the pre-existing TypeError for an unvalidated or
parent-built instance of any generated model dumped through such a parent.

tests/test_import_memory.py imports each entry module in a subprocess and
fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades
the deferral contract, including that a canonical subclass may forward-
reference a model defined later in its module (the SDK 10 port of #1704; #1690 on main is subsumed).

Fixes #1694 for SDK 10; main carries the same fix as #1704.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bokelley added a commit that referenced this pull request Oct 29, 2026
…t 560 MiB RSS (#1704)

Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions
realizes the type surface through adcp.types._eager, and on 9.3.0 that peaked
at ~495 MiB RSS (~560 MiB for adcp.client) against ~130 MiB on 6.6. Two
things built pydantic-core validators at import for models a process may
never use:

- The import-time patchers (_ergonomic, _forward_compat, canonical_creative,
  aliases) edited model_fields and then called model_rebuild(force=True) on
  ~35 request/response roots. Pydantic does not share nested defer_build
  schemas between parents, so each forced rebuild walked the model's whole
  subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the
  import. They now call adcp.types.base.defer_model_rebuild(), which puts the
  model back into the defer_build placeholder state so the patched fields
  take effect on first use. Same outcome, later moment.

- A generated `class X(RootModel[A | B])` built the union's validator at the
  RootModel[...] expression, before X or any config it declares existed;
  adagents.AdcpAgentsAuthorization and core.format.CanonicalParameters alone
  cost ~60 MiB. Generated composing roots now parametrize
  adcp.types.base.AdCPRootModel, which carries defer_build=True, via the new
  post-generate fixer defer_rootmodel_builds.

Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12):

  adcp.types._eager            495 -> 114 MiB  (import 14.5 s -> 4.1 s)
  adcp.client                  561 -> 175 MiB
  adcp.server                  565 -> 183 MiB
  adcp.server.test_controller  562 -> 183 MiB
  adcp.decisioning             573 -> 187 MiB
  adcp.compat.versions         498 -> 118 MiB

The cost moves to first use: the first validation of a deferred root builds
its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's
JSON schema), and a process that has used every one of them sits at ~215 MiB
instead of ~520 MiB. build_all_models() still realizes everything eagerly for
freezegun callers.

Deferring more models exposed a pydantic limitation that already applied to
every defer_build model: pydantic-core reaches a serializer by reading
instance.__pydantic_serializer__ when it dumps by inference (an Any-typed
field, serialize_as_any, a plain-pydantic parent), and a placeholder there
raised TypeError instead of building. Every deferred SDK model now carries
placeholders that are also descriptors (_DeferredValSer), installed by
AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by
defer_model_rebuild, so that read builds the model and returns the real
serializer. This also fixes the pre-existing TypeError for an unvalidated or
parent-built instance of any generated model dumped through such a parent.

tests/test_import_memory.py imports each entry module in a subprocess and
fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades
the deferral contract, including that a canonical subclass may forward-
reference a model defined later in its module (supersedes #1690).

Fixes #1694

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Heads-up: #1704 (merged to main as aac52ca, for #1694) replaced the forced rebuild in CanonicalBoundaryModel.__pydantic_init_subclass__ with a deferred one (defer_model_rebuild), so a subclass that forward-references a model defined later in its module no longer raises at class creation. It carries the regression scenario from this PR as tests/test_lazy_rebuild.py::test_a_canonical_subclass_may_forward_reference_a_later_model. I think this draft can be closed as superseded, but leaving that to you.

@justadreamer

Copy link
Copy Markdown
Author

I have read the IPR Policy

@justadreamer

Copy link
Copy Markdown
Author

Fixed on main by #1704: the subclass hook now defers the rebuild instead of forcing it, and test_a_canonical_subclass_may_forward_reference_a_later_model in tests/test_lazy_rebuild.py covers this case.

@github-actions

Copy link
Copy Markdown
Contributor

IPR Policy — signed

Thanks, @justadreamer. Your agreement to the IPR Policy is recorded at signatures/ipr-signatures.json and applies to all AAO repositories.

aao-ipr-bot Bot pushed a commit to adcontextprotocol/adcp that referenced this pull request Oct 29, 2026
bokelley added a commit that referenced this pull request Oct 29, 2026
…t 560 MiB RSS (#1713)

Importing adcp.client, adcp.server, adcp.decisioning or adcp.compat.versions
realizes the type surface through adcp.types._eager, and on 10.0.0-beta.2 that peaked
at ~517 MiB RSS (~590 MiB for adcp.client) against ~130 MiB on 6.6. Two
things built pydantic-core validators at import for models a process may
never use:

- The import-time patchers (_ergonomic, _forward_compat, canonical_creative,
  aliases) edited model_fields and then called model_rebuild(force=True) on
  ~35 request/response roots. Pydantic does not share nested defer_build
  schemas between parents, so each forced rebuild walked the model's whole
  subtree into a fresh validator and serializer: ~300 MiB and ~7 s of the
  import. They now call adcp.types.base.defer_model_rebuild(), which puts the
  model back into the defer_build placeholder state so the patched fields
  take effect on first use. Same outcome, later moment.

- A generated `class X(AdCPRootModel[A | B])` built the union's validator at
  the subscript, before X or any config it declares existed, because the
  #1613 base carried pydantic's default config; adagents.AdcpAgentsAuthorization
  and core.format.CanonicalParameters alone cost ~60 MiB. AdCPRootModel now
  carries defer_build=True, so the parametrized class and X both defer.

Peak RSS of `python -I -c "import <module>"` (Linux, CPython 3.12):

  adcp.types._eager            517 -> 121 MiB  (import 13.9 s -> 3.6 s)
  adcp.client                  588 -> 189 MiB
  adcp.server.test_controller  610 -> 214 MiB
  adcp.decisioning                 -> 224 MiB

The cost moves to first use: the first validation of a deferred root builds
its schema (90-330 ms for the request models, ~2.4 s for McpWebhookPayload's
JSON schema), and a process that has used every one of them sits at ~215 MiB
instead of ~520 MiB. build_all_models() still realizes everything eagerly for
freezegun callers.

Deferring more models exposed a pydantic limitation that already applied to
every defer_build model: pydantic-core reaches a serializer by reading
instance.__pydantic_serializer__ when it dumps by inference (an Any-typed
field, serialize_as_any, a plain-pydantic parent), and a placeholder there
raised TypeError instead of building. Every deferred SDK model now carries
placeholders that are also descriptors (_DeferredValSer), installed by
AdCPBaseModel/AdCPRootModel.__pydantic_init_subclass__ and by
defer_model_rebuild, so that read builds the model and returns the real
serializer. This also fixes the pre-existing TypeError for an unvalidated or
parent-built instance of any generated model dumped through such a parent.

tests/test_import_memory.py imports each entry module in a subprocess and
fails when its peak RSS exceeds 320 MiB; tests/test_lazy_rebuild.py grades
the deferral contract, including that a canonical subclass may forward-
reference a model defined later in its module (the SDK 10 port of #1704; #1690 on main is subsumed).

Fixes #1694 for SDK 10; main carries the same fix as #1704.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants