Skip to content

Promote _catalog_objects to a public catalog module and unify the required_*schema.json loaders #1497

Open

Description

@KonstantinMirin

Problem

The row-storage readiness check imports the private _catalog_objects from adcp.reporting.outbox._schema to fingerprint its tables. The same catalog-fingerprint pattern is repeated across the reporting package: ten required_*schema.json manifests, loaded by eight modules, each with its own files(...).joinpath(...).read_text() and comparison loop. Only the row-storage manifest has a generator script, so the other nine are regenerated by hand or not at all. This predates the row-storage series; #1464 added the tenth manifest and the cross-package private import.

Evidence

All references are to main at 47084e83a, under src/adcp/reporting/.

  • ledger/row_storage.py:556, inside _verify_catalog:

    from adcp.reporting.outbox._schema import _catalog_objects

    _catalog_objects is defined at outbox/_schema.py:43 and is not in that module's public names.

  • Manifests (10): feed/required_schema.json, ledger/required_provisional_schema.json, ledger/required_row_storage_schema.json, materializer/required_schema.json, outbox/required_schema.json, outbox/required_status_schema.json, outbox/required_status_selector_schema.json, production/required_schema.json, projection/required_schema.json, and receipts/required_schema.json.

  • Loaders (8 modules): receipts/schema.py:18, outbox/_schema.py:20 and :23, outbox/status_schema.py:13 and :16, ledger/row_storage.py:559, materializer/schema.py:21, feed/schema.py:18, production/schema.py:19, and projection/schema.py:18, each of the form files("adcp.reporting.<pkg>").joinpath("required_<x>schema.json").read_text().

  • Generators: scripts/generate_row_storage_catalog.py only ("Regenerate required_row_storage_schema.json from a clean migration").

Suggested fix

Add one module, for example adcp.reporting.catalog, with catalog_objects(connection, *, table_names, function_identity, name_prefix) and verify_required(connection, manifest_path), and reduce each of the eight loaders to a manifest path. Generalize scripts/generate_row_storage_catalog.py to take a DDL path and a manifest path, so every manifest has the same generator and a drift check in CI can regenerate all ten and diff them.

Introduced in #1464 (the private import); the manifest pattern predates it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpythonPull requests that update python code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions