diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e909cc17..5f5ccc6da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1529,8 +1529,13 @@ until you regenerate. five ports**: written as the canonical string, read from a string or a JSON number exactly. The ports disagreed before (TypeScript refused a number, Python stored a float). A `field.double` inside a value object stays a JSON number, and a fractional number that is - not a declared decimal stays a float in Python (it answered 500). A decimal inside a - `field.map` value object is not covered: it is not part of the rule and is not gated. + not a declared decimal stays a float in Python (it answered 500). The rule also holds + for a value object held by a `field.map` (applied to each map value; the keys are arbitrary + and never matched with member names), gated by the `jsonb-value-object-map-decimals` + scenario in every port. To get there, the generated Kotlin controller now creates, reads + and patches a `field.map` of value objects (it skipped that column before), and the + generated C# `DbContext` maps a value object that holds a map as one converted jsonb + property instead of an owned type (EF Core 8 cannot own a dictionary inside JSON). - **A `field.decimal` array works in C#, Java and Kotlin.** Java binds a Postgres `numeric[]` as exact `BigDecimal` elements, a non-required C# decimal array is nullable, and each element follows the scalar decimal rule (a bad element is `400 validation`). diff --git a/agent-context/skills/metaobjects-codegen/SKILL.md b/agent-context/skills/metaobjects-codegen/SKILL.md index 4332d7fc8..0b3aa831f 100644 --- a/agent-context/skills/metaobjects-codegen/SKILL.md +++ b/agent-context/skills/metaobjects-codegen/SKILL.md @@ -63,8 +63,9 @@ in order and every change that needs action. - A `field.decimal` crosses the API as one canonical string, and a fractional validator `@min` or `@max` is refused (write an integer bound). - **Resync the generators you own.** An ejected copy keeps the logic it was copied with, and a - plain `gen` does not refresh it. `meta eject --list` (`dotnet meta eject`, `metaobjects eject`, - `mvn metaobjects:eject`) reports each copy that differs from the reference; + plain `gen` does not refresh it. `meta eject --list` (TypeScript; `metaobjects gen --list` in Python, which has no + `eject --list`; `dotnet meta gen --list`; `mvn metaobjects:eject -Dlist=true` in Java and Kotlin) + reports each copy that differs from the reference; `eject --force` takes the new reference for a copy you never edited, and a copy you edited is three-way merged instead. Steps for 1.0.x: https://github.com/metaobjectsdev/metaobjects/blob/main/docs/features/migrations/upgrading-within-1.0.md. diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 0d9c36f86..075b0e1c1 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -34,7 +34,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | | [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 105 (31 core + 10 tph + 9 m2m + 3 jsonb + 2 write-through + 12 projection + 27 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (HttpListener + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 106 (31 core + 10 tph + 9 m2m + 4 jsonb + 2 write-through + 12 projection + 27 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (HttpListener + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL | | [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | | [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ | @@ -347,9 +347,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc `report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). -### `fixtures/api-contract-conformance/` (105) +### `fixtures/api-contract-conformance/` (106) -All 105 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 106 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -363,7 +363,7 @@ URL grammar `?filter[][]=` (FR-009). On top of the 31 core scenarios the corpus carries seven sub-corpora — `tph/` (10, single-table inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), -`jsonb/` (3, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), +`jsonb/` (4, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), `write-through/` (2, table-write + view-read entities), `projection/` (12, a read-only view answers reads and refuses writes with 405; a projection with no declared identity has no item route; a projection keyed on a field not named `id` is addressed by @@ -554,7 +554,7 @@ files into a real Cube. ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 410 + yaml 16 + verify 37 -+ render 16 + persistence 59 + api-contract 105 + source-resolution 25 + scope 10 + ++ render 16 + persistence 59 + api-contract 106 + source-resolution 25 + scope 10 + dependency 23) each map to a feature doc, and so do the two requirement corpora and the `cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/field-types.md b/docs/features/field-types.md index 9641eb26d..c5e04a990 100644 --- a/docs/features/field-types.md +++ b/docs/features/field-types.md @@ -74,17 +74,16 @@ Four rows need a footnote: > added a boundary check of your own while this was open, it is now redundant rather than > load-bearing. - **The RUNTIME tier is not there yet, and no conformance corpus covers it.** No - persistence- or api-contract-conformance fixture exercises `field.map` on any port; it is - loader- and codegen-gated only. Of the runtime persistence layers, only Python's - `ObjectManager` encodes a map (its jsonb write codec names `FIELD_SUBTYPE_MAP` - alongside `FIELD_SUBTYPE_OBJECT`). TypeScript's `runtime-ts`, Java's OMDB and the Kotlin - Exposed persistence lane carry no map handling at all — OMDB's jsonb path in particular - is gated on the `@storage` attr, which a map does not have, and serializes through a - per-`MetaObject` Gson adapter that has no map-of-value-object binding. Closing that is a - cross-port runtime workstream, not a codegen change, and it is the prerequisite for the - shared `op: roundtrip` persistence scenario that would gate this subtype the way every - other persistable subtype is gated. + **The RUNTIME tier is partial, and no persistence conformance corpus covers it.** No + persistence-conformance fixture exercises `field.map`; the api-contract + `jsonb/scenarios/jsonb-value-object-map-decimals.yaml` gates a map of value objects through + each port's generated API. Of the runtime persistence layers, Python's `ObjectManager` encodes + a map (its jsonb write codec names `FIELD_SUBTYPE_MAP` alongside `FIELD_SUBTYPE_OBJECT`), and + Java's OMDB maps a `field.map @objectRef` to a jsonb column whose values are the value object + (`GenericSQLDriver`). TypeScript's `runtime-ts` and the Kotlin Exposed persistence lane carry + no map-of-value-object handling. Closing that gap across ports is a cross-port runtime + workstream, not a codegen change, and it is the prerequisite for the shared `op: roundtrip` + persistence scenario that would gate this subtype the way every other persistable subtype is gated. So: for data you intend to READ BACK THROUGH A PORT RUNTIME today, a stable key set is still better declared as an `object.value` behind `field.object`, which every runtime diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md index 4332d7fc8..0b3aa831f 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -63,8 +63,9 @@ in order and every change that needs action. - A `field.decimal` crosses the API as one canonical string, and a fractional validator `@min` or `@max` is refused (write an integer bound). - **Resync the generators you own.** An ejected copy keeps the logic it was copied with, and a - plain `gen` does not refresh it. `meta eject --list` (`dotnet meta eject`, `metaobjects eject`, - `mvn metaobjects:eject`) reports each copy that differs from the reference; + plain `gen` does not refresh it. `meta eject --list` (TypeScript; `metaobjects gen --list` in Python, which has no + `eject --list`; `dotnet meta gen --list`; `mvn metaobjects:eject -Dlist=true` in Java and Kotlin) + reports each copy that differs from the reference; `eject --force` takes the new reference for a copy you never edited, and a copy you edited is three-way merged instead. Steps for 1.0.x: https://github.com/metaobjectsdev/metaobjects/blob/main/docs/features/migrations/upgrading-within-1.0.md. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md index 4332d7fc8..0b3aa831f 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -63,8 +63,9 @@ in order and every change that needs action. - A `field.decimal` crosses the API as one canonical string, and a fractional validator `@min` or `@max` is refused (write an integer bound). - **Resync the generators you own.** An ejected copy keeps the logic it was copied with, and a - plain `gen` does not refresh it. `meta eject --list` (`dotnet meta eject`, `metaobjects eject`, - `mvn metaobjects:eject`) reports each copy that differs from the reference; + plain `gen` does not refresh it. `meta eject --list` (TypeScript; `metaobjects gen --list` in Python, which has no + `eject --list`; `dotnet meta gen --list`; `mvn metaobjects:eject -Dlist=true` in Java and Kotlin) + reports each copy that differs from the reference; `eject --force` takes the new reference for a copy you never edited, and a copy you edited is three-way merged instead. Steps for 1.0.x: https://github.com/metaobjectsdev/metaobjects/blob/main/docs/features/migrations/upgrading-within-1.0.md. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md index 4332d7fc8..0b3aa831f 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -63,8 +63,9 @@ in order and every change that needs action. - A `field.decimal` crosses the API as one canonical string, and a fractional validator `@min` or `@max` is refused (write an integer bound). - **Resync the generators you own.** An ejected copy keeps the logic it was copied with, and a - plain `gen` does not refresh it. `meta eject --list` (`dotnet meta eject`, `metaobjects eject`, - `mvn metaobjects:eject`) reports each copy that differs from the reference; + plain `gen` does not refresh it. `meta eject --list` (TypeScript; `metaobjects gen --list` in Python, which has no + `eject --list`; `dotnet meta gen --list`; `mvn metaobjects:eject -Dlist=true` in Java and Kotlin) + reports each copy that differs from the reference; `eject --force` takes the new reference for a copy you never edited, and a copy you edited is three-way merged instead. Steps for 1.0.x: https://github.com/metaobjectsdev/metaobjects/blob/main/docs/features/migrations/upgrading-within-1.0.md. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md index 4332d7fc8..0b3aa831f 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -63,8 +63,9 @@ in order and every change that needs action. - A `field.decimal` crosses the API as one canonical string, and a fractional validator `@min` or `@max` is refused (write an integer bound). - **Resync the generators you own.** An ejected copy keeps the logic it was copied with, and a - plain `gen` does not refresh it. `meta eject --list` (`dotnet meta eject`, `metaobjects eject`, - `mvn metaobjects:eject`) reports each copy that differs from the reference; + plain `gen` does not refresh it. `meta eject --list` (TypeScript; `metaobjects gen --list` in Python, which has no + `eject --list`; `dotnet meta gen --list`; `mvn metaobjects:eject -Dlist=true` in Java and Kotlin) + reports each copy that differs from the reference; `eject --force` takes the new reference for a copy you never edited, and a copy you edited is three-way merged instead. Steps for 1.0.x: https://github.com/metaobjectsdev/metaobjects/blob/main/docs/features/migrations/upgrading-within-1.0.md. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md index 4332d7fc8..0b3aa831f 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/SKILL.md @@ -63,8 +63,9 @@ in order and every change that needs action. - A `field.decimal` crosses the API as one canonical string, and a fractional validator `@min` or `@max` is refused (write an integer bound). - **Resync the generators you own.** An ejected copy keeps the logic it was copied with, and a - plain `gen` does not refresh it. `meta eject --list` (`dotnet meta eject`, `metaobjects eject`, - `mvn metaobjects:eject`) reports each copy that differs from the reference; + plain `gen` does not refresh it. `meta eject --list` (TypeScript; `metaobjects gen --list` in Python, which has no + `eject --list`; `dotnet meta gen --list`; `mvn metaobjects:eject -Dlist=true` in Java and Kotlin) + reports each copy that differs from the reference; `eject --force` takes the new reference for a copy you never edited, and a copy you edited is three-way merged instead. Steps for 1.0.x: https://github.com/metaobjectsdev/metaobjects/blob/main/docs/features/migrations/upgrading-within-1.0.md. diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index 261035fb3..cb5eee070 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -53,14 +53,14 @@ fixtures/api-contract-conformance/ ``` Beside the 31 core scenarios above sit seven **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (74 scenarios in all, 105 with the +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (75 scenarios in all, 106 with the core): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| | `tph/` | 10 | single-table inheritance | both | | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | -| `jsonb/` | 3 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | +| `jsonb/` | 4 | typed value-object columns, and the numbers inside them (a double stays a JSON number; a decimal is a canonical string, option A of the 1.1.0 review) | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | | `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | | `report/` | 27 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default; a dimension over an array or a map is neither filterable nor sortable. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | diff --git a/fixtures/api-contract-conformance/jsonb/README.md b/fixtures/api-contract-conformance/jsonb/README.md index 7b15c11f3..a108f1586 100644 --- a/fixtures/api-contract-conformance/jsonb/README.md +++ b/fixtures/api-contract-conformance/jsonb/README.md @@ -58,6 +58,7 @@ jsonb/ └── scenarios/ ├── jsonb-open-bag-roundtrip.yaml # open-bag parsed-value contract ├── jsonb-value-object-patch.yaml # Program D — cross-port VO-column PATCH/POST + ├── jsonb-value-object-map-decimals.yaml # decimals in a field.map of value objects, per map value └── jsonb-value-object-numbers.yaml # doubles and decimals inside a value object ``` diff --git a/fixtures/api-contract-conformance/jsonb/meta.json b/fixtures/api-contract-conformance/jsonb/meta.json index cc9e9c61d..7dc9b1352 100644 --- a/fixtures/api-contract-conformance/jsonb/meta.json +++ b/fixtures/api-contract-conformance/jsonb/meta.json @@ -17,6 +17,18 @@ { "field.decimal": { "name": "amount", "@precision": 12, "@scale": 4 } } ] }}, + { "object.value": { + "name": "Tier", + "children": [ + { "field.decimal": { "name": "amount", "@precision": 12, "@scale": 4 } } + ] + }}, + { "object.value": { + "name": "Pricing", + "children": [ + { "field.map": { "name": "tiers", "@objectRef": "Tier" } } + ] + }}, { "object.entity": { "name": "Document", "children": [ @@ -29,6 +41,8 @@ { "field.object": { "name": "markers", "@objectRef": "Marker", "@storage": "jsonb", "isArray": true } }, { "field.object": { "name": "measure", "@objectRef": "Measure", "@storage": "jsonb" } }, { "field.object": { "name": "measures", "@objectRef": "Measure", "@storage": "jsonb", "isArray": true } }, + { "field.map": { "name": "tiers", "@objectRef": "Tier" } }, + { "field.object": { "name": "pricing", "@objectRef": "Pricing", "@storage": "jsonb" } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } ] }} diff --git a/fixtures/api-contract-conformance/jsonb/scenarios/jsonb-value-object-map-decimals.yaml b/fixtures/api-contract-conformance/jsonb/scenarios/jsonb-value-object-map-decimals.yaml new file mode 100644 index 000000000..0a3ad5688 --- /dev/null +++ b/fixtures/api-contract-conformance/jsonb/scenarios/jsonb-value-object-map-decimals.yaml @@ -0,0 +1,77 @@ +name: jsonb-value-object-map-decimals +description: > + A decimal inside a value object that a `field.map` holds follows the top-level decimal rule + (ADR-0019 Amendment 1, W1 to W3) for EACH map value, the way an array applies it per element. + The map keys are arbitrary text and are never compared with the value object's member names: + a key named `amount` is a key, and its value is a Tier. Two shapes: a top-level map + (Document.tiers, a map of Tier) and a map nested in a value object (Document.pricing, a Pricing + whose `tiers` is a map of Tier). Tier is { amount: field.decimal @precision 12 @scale 4 }. A + string or a JSON number is read exactly and emitted as the canonical string ("1.5000" in or 1.5 + in gives "1.5" out). More fraction digits than @scale is `400 {"error": "validation"}` and + persists nothing. The title and primaryMarker are required on every write; implicit-id POSTs + land at 3, 4, ... +requests: + - id: r1 + method: POST + path: /api/documents + body: + title: "map-a" + primaryMarker: { label: "a", score: 1 } + tiers: + amount: { amount: "1.5000" } + gold: { amount: 2.50 } + pricing: + tiers: + amount: { amount: 100 } + silver: { amount: "0.25" } + expect: + status: 201 + body: + hasId: true + row: + tiers: + amount: { amount: "1.5" } + gold: { amount: "2.5" } + pricing: + tiers: + amount: { amount: "100" } + silver: { amount: "0.25" } + - id: r2 + method: GET + path: /api/documents/3 + expect: + status: 200 + body: + row: + tiers: + amount: { amount: "1.5" } + gold: { amount: "2.5" } + pricing: + tiers: + amount: { amount: "100" } + silver: { amount: "0.25" } + - id: r3 + method: POST + path: /api/documents + body: + title: "map-bad-top" + primaryMarker: { label: "b", score: 1 } + tiers: + gold: { amount: "0.12345" } + expect: + status: 400 + body: + error: validation + - id: r4 + method: POST + path: /api/documents + body: + title: "map-bad-nested" + primaryMarker: { label: "c", score: 1 } + pricing: + tiers: + amount: { amount: "0.12345" } + expect: + status: 400 + body: + error: validation diff --git a/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs b/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs index 03fe73fe7..0d83f5a5e 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs @@ -346,15 +346,12 @@ public void WriteThrough_read_model_carries_type_converter_and_jsonb_vo_columns( Assert.Contains( "modelBuilder.Entity().Property(x => x.DocUrl).HasColumnType(\"text\").HasConversion(", dbctx); - // [1] — and the owned-VO OwnsOne config over the read model, so the jsonb VO round-trips. - Assert.Contains("modelBuilder.Entity().OwnsOne(x => x.BillingAddress", dbctx); // The write entity still gets its own copies of both configs (unchanged behavior). Assert.Contains("public Uri? DocUrl { get; set; }", writeEntity); Assert.Contains( "modelBuilder.Entity().Property(x => x.DocUrl).HasColumnType(\"text\").HasConversion(", dbctx); - Assert.Contains("modelBuilder.Entity().OwnsOne(x => x.BillingAddress", dbctx); } // I1 (Important) -- a sourceless CONCRETE object.entity: no source.rdb child diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index 68989d0e7..bbb2ef233 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -816,6 +816,20 @@ private static void EmitMapJsonbHelper(StringBuilder sb) sb.AppendLine(" v => System.Text.Json.JsonSerializer.Deserialize<"); sb.AppendLine(" System.Collections.Generic.Dictionary>(v, Options)!);"); sb.AppendLine(); + // A value object that holds a map is stored as ONE document converted whole: EF Core 8 cannot + // map a Dictionary member inside an owned-JSON type. The same Options serve it, so + // a decimal member keeps its canonical-string wire form (DecimalWire.Declared on the member). + sb.AppendLine(" /// A whole value-object document (or collection of them) as one jsonb value,"); + sb.AppendLine(" /// for a value object that holds a map (EF Core 8 cannot own a dictionary inside JSON)."); + sb.AppendLine(" internal static Microsoft.EntityFrameworkCore.Storage.ValueConversion.ValueConverter ObjectConverter() => new("); + sb.AppendLine(" v => System.Text.Json.JsonSerializer.Serialize(v, Options),"); + sb.AppendLine(" v => System.Text.Json.JsonSerializer.Deserialize(v, Options)!);"); + sb.AppendLine(); + sb.AppendLine(" internal static Microsoft.EntityFrameworkCore.ChangeTracking.ValueComparer ObjectComparer() => new("); + sb.AppendLine(" (a, b) => System.Text.Json.JsonSerializer.Serialize(a, Options) == System.Text.Json.JsonSerializer.Serialize(b, Options),"); + sb.AppendLine(" v => System.Text.Json.JsonSerializer.Serialize(v, Options).GetHashCode(),"); + sb.AppendLine(" v => System.Text.Json.JsonSerializer.Deserialize(System.Text.Json.JsonSerializer.Serialize(v, Options), Options)!);"); + sb.AppendLine(); sb.AppendLine(" internal static Microsoft.EntityFrameworkCore.ChangeTracking.ValueComparer<"); sb.AppendLine(" System.Collections.Generic.Dictionary?> Comparer() => new("); sb.AppendLine(" (a, b) => Eq(a, b), v => Hash(v), v => Snap(v));"); @@ -1066,6 +1080,19 @@ private void EmitFieldTypeConfig( if (DbColumnTypeConfig(className, f) is { } cfg) modelLines.Add(cfg); } + // True when a value object declares a field.map, or holds a value object that does. + private static bool HoldsMap(MetaObject vo, MetaRoot root, HashSet seen) + { + if (!seen.Add(vo)) return false; + foreach (var nf in vo.Fields()) + { + if (nf.SubType == FIELD_SUBTYPE_MAP) return true; + if (nf.SubType == FIELD_SUBTYPE_OBJECT && ValueObjectNames.ResolveFieldRef(nf, root) is { } nested + && HoldsMap(nested, root, seen)) return true; + } + return false; + } + // Owned-type config for an object-typed entity field. @storage flattened maps each // nested scalar to "{parentCol}_{nestedCol}"; every other storage collapses to one // json column (.ToJson) — matching the TS-owned schema DDL. null (with a warning) @@ -1106,6 +1133,21 @@ private void EmitFieldTypeConfig( // every sibling port stores the member SYMBOL — a cross-port wire-contract break, and a // positionally fragile one: reordering @values silently re-maps already-stored data, // which is precisely why @intValueMap was specified as a MAP rather than an array. + // A value object that holds a field.map (directly or through a nested value object) cannot be + // an EF Core 8 owned-JSON type: the dictionary member has no owned mapping and model + // finalization fails. Such a field is one jsonb property converted whole instead, which keeps + // the stored document identical (every member serializes by its [JsonPropertyName], a decimal + // as its canonical string through DecimalWire.Declared). + if (field.Storage != STORAGE_FLATTENED && HoldsMap(vo, ctx.Root, new HashSet())) + { + var modelType = CSharpNaming.IsRequired(entity, field) ? "" : "?"; + var voType = MapValueTypeRef(field, ctx); + var clr = field.ResolvedIsArray() + ? $"System.Collections.Generic.ICollection<{voType}>{modelType}" + : voType + modelType; + return $" modelBuilder.Entity<{owner}>().Property(x => x.{nav}).HasColumnName({parentColRef})" + + $".HasColumnType(\"jsonb\").HasConversion({MapJsonbHelperName}.ObjectConverter<{clr}>(), {MapJsonbHelperName}.ObjectComparer<{clr}>());"; + } var jsonEnums = JsonEnumConversions(vo, ctx); if (field.ResolvedIsArray()) return jsonEnums.Count == 0 diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbFixture.cs b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbFixture.cs index 26513b5b0..ab0c827ac 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbFixture.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbFixture.cs @@ -32,6 +32,8 @@ internal static class JsonbFixture "markers" jsonb, "measure" jsonb, "measures" jsonb, + "tiers" jsonb, + "pricing" jsonb, CONSTRAINT "documents_pkey" PRIMARY KEY ("id") ); """; @@ -72,8 +74,8 @@ public static async Task ApplySeedAsync(string connString, string seedPath) if (rowNode is not JsonObject row) continue; await using var ins = c.CreateCommand(); ins.CommandText = - "INSERT INTO \"documents\" (\"id\", \"title\", \"payload\", \"primaryMarker\", \"optionalMarker\", \"markers\", \"measure\", \"measures\") " + - "VALUES (@id, @title, @payload, @primaryMarker, @optionalMarker, @markers, @measure, @measures)"; + "INSERT INTO \"documents\" (\"id\", \"title\", \"payload\", \"primaryMarker\", \"optionalMarker\", \"markers\", \"measure\", \"measures\", \"tiers\", \"pricing\") " + + "VALUES (@id, @title, @payload, @primaryMarker, @optionalMarker, @markers, @measure, @measures, @tiers, @pricing)"; ins.Parameters.AddWithValue("@id", row["id"]!.GetValue()); ins.Parameters.AddWithValue("@title", row["title"]!.GetValue()); // payload + the VO columns are JSON — serialize each and bind as jsonb (an @@ -85,6 +87,8 @@ public static async Task ApplySeedAsync(string connString, string seedPath) AddJsonbParam(ins, "@markers", row["markers"]); AddJsonbParam(ins, "@measure", row["measure"]); AddJsonbParam(ins, "@measures", row["measures"]); + AddJsonbParam(ins, "@tiers", row["tiers"]); + AddJsonbParam(ins, "@pricing", row["pricing"]); await ins.ExecuteNonQueryAsync(); } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbMapDecimalPatchGeneratedTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbMapDecimalPatchGeneratedTest.cs new file mode 100644 index 000000000..31409a3bc --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbMapDecimalPatchGeneratedTest.cs @@ -0,0 +1,60 @@ +// JsonbMapDecimalPatchGeneratedTest — PATCH over decimals held by a field.map of value objects +// (ADR-0019 Amendment 1, C1), against the REAL generated routes. The shared scenario covers POST and +// GET; this locks the merge path and the key-named-like-a-member case: map keys are arbitrary text and +// are never matched with the value object's member names. + +using System.Net; +using System.Text; +using System.Text.Json.Nodes; +using MetaObjects.IntegrationTests.Runner; +using Xunit; + +namespace MetaObjects.IntegrationTests.Api; + +public sealed class JsonbMapDecimalPatchGeneratedTest +{ + private static async Task<(HttpStatusCode, JsonNode?)> SendAsync(HttpClient c, string method, string path, string json) + { + var res = await c.SendAsync(new HttpRequestMessage(new HttpMethod(method), path) + { + Content = new StringContent(json, Encoding.UTF8, "application/json"), + }); + var text = await res.Content.ReadAsStringAsync(); + return (res.StatusCode, string.IsNullOrEmpty(text) ? null : JsonNode.Parse(text)); + } + + [Fact] + public async Task Patch_applies_the_decimal_rule_to_each_map_value_and_ignores_the_key_name() + { + await using var pg = await PostgresContainer.StartAsync(); + await using var server = await JsonbGeneratedServerFactory.StartAsync(pg); + await server.ApplySeedAsync(); + using var c = new HttpClient { BaseAddress = new Uri(server.BaseUrl) }; + + var (st, created) = await SendAsync(c, "POST", "/api/documents", + """{"title":"t","primaryMarker":{"label":"a","score":1}}"""); + Assert.Equal(HttpStatusCode.Created, st); + var id = created!["id"]!.GetValue(); + + // A key named `amount` is a key; its value is a Tier. Canonical string out, read exactly in. + (st, var patched) = await SendAsync(c, "PATCH", $"/api/documents/{id}", + """{"tiers":{"amount":{"amount":1.5000},"gold":{"amount":"2.50"}},"pricing":{"tiers":{"amount":{"amount":"100.0"}}}}"""); + Assert.Equal(HttpStatusCode.OK, st); + Assert.Equal("1.5", patched!["tiers"]!["amount"]!["amount"]!.GetValue()); + Assert.Equal("2.5", patched["tiers"]!["gold"]!["amount"]!.GetValue()); + Assert.Equal("100", patched["pricing"]!["tiers"]!["amount"]!["amount"]!.GetValue()); + + (st, var read) = await SendAsync(c, "GET", $"/api/documents/{id}", ""); + Assert.Equal(HttpStatusCode.OK, st); + Assert.Equal("100", read!["pricing"]!["tiers"]!["amount"]!["amount"]!.GetValue()); + + // More fraction digits than @scale 4, in either shape: 400 and nothing persisted. + (st, _) = await SendAsync(c, "PATCH", $"/api/documents/{id}", """{"tiers":{"gold":{"amount":"0.12345"}}}"""); + Assert.Equal(HttpStatusCode.BadRequest, st); + (st, _) = await SendAsync(c, "PATCH", $"/api/documents/{id}", """{"pricing":{"tiers":{"amount":{"amount":0.12345}}}}"""); + Assert.Equal(HttpStatusCode.BadRequest, st); + (st, read) = await SendAsync(c, "GET", $"/api/documents/{id}", ""); + Assert.Equal("1.5", read!["tiers"]!["amount"]!["amount"]!.GetValue()); + Assert.Equal("100", read["pricing"]!["tiers"]!["amount"]!["amount"]!.GetValue()); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs index 03cd25a9a..69bd8cac5 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/JsonbReferenceServer.cs @@ -41,7 +41,7 @@ internal sealed class JsonbReferenceServer : IAsyncDisposable private const int MarkerLabelMaxLength = 40; // The jsonb columns carried in a document row's wire projection (id + title are scalar). - private static readonly string[] JsonbColumns = { "payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures" }; + private static readonly string[] JsonbColumns = { "payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures", "tiers", "pricing" }; private JsonbReferenceServer(PostgresContainer pg, HttpListener listener, string baseUrl) { @@ -160,8 +160,8 @@ private async Task CreateDocumentAsync(HttpListenerContext ctx) await using (var ins = c.CreateCommand()) { ins.CommandText = - "INSERT INTO \"documents\" (\"title\", \"payload\", \"primaryMarker\", \"optionalMarker\", \"markers\", \"measure\", \"measures\") " + - "VALUES (@title, @payload, @primaryMarker, @optionalMarker, @markers, @measure, @measures) RETURNING id"; + "INSERT INTO \"documents\" (\"title\", \"payload\", \"primaryMarker\", \"optionalMarker\", \"markers\", \"measure\", \"measures\", \"tiers\", \"pricing\") " + + "VALUES (@title, @payload, @primaryMarker, @optionalMarker, @markers, @measure, @measures, @tiers, @pricing) RETURNING id"; ins.Parameters.AddWithValue("@title", body["title"]!.GetValue()); AddJsonbParam(ins, "@payload", body["payload"]); AddJsonbParam(ins, "@primaryMarker", body["primaryMarker"]); @@ -169,6 +169,8 @@ private async Task CreateDocumentAsync(HttpListenerContext ctx) AddJsonbParam(ins, "@markers", body["markers"]); AddJsonbParam(ins, "@measure", body["measure"]); AddJsonbParam(ins, "@measures", body["measures"]); + AddJsonbParam(ins, "@tiers", body["tiers"]); + AddJsonbParam(ins, "@pricing", body["pricing"]); newId = Convert.ToInt64(await ins.ExecuteScalarAsync()); } await SendJsonAsync(ctx, 201, await SelectRowAsync(c, newId) ?? new Dictionary()); @@ -271,8 +273,71 @@ private static bool ValidMarkerArray(JsonNode? node) return outObj; } + // Tier { amount: decimal(12, 4) }. A map of Tier applies the Tier rule to EACH map value; the map + // keys are arbitrary text and are never matched with Tier's member names (a key `amount` is a key). + private static JsonObject? CanonicalTier(JsonNode? node) + { + if (node is not JsonObject obj) return null; + var outObj = new JsonObject(); + foreach (var kvp in obj) + { + if (kvp.Value is null || kvp.Value.GetValueKind() == JsonValueKind.Null) continue; + if (kvp.Key == "amount") + { + var el = JsonSerializer.SerializeToElement(kvp.Value); + if (!DecimalWire.TryParseElement(el, out var d) || !DecimalWire.Fits(d, MeasurePrecision, MeasureScale)) + return null; + outObj["amount"] = DecimalWire.Format(d); + } + else outObj[kvp.Key] = kvp.Value.DeepClone(); + } + return outObj; + } + + private static JsonObject? CanonicalTierMap(JsonNode? node) + { + if (node is not JsonObject map) return null; + var outMap = new JsonObject(); + foreach (var kvp in map) + { + var canon = CanonicalTier(kvp.Value); + if (canon is null) return null; + outMap[kvp.Key] = canon; + } + return outMap; + } + + private static bool CanonicalizeTiers(JsonObject body) + { + if (PresentNonNull(body, "tiers")) + { + var canon = CanonicalTierMap(body["tiers"]); + if (canon is null) return false; + body["tiers"] = canon; + } + if (PresentNonNull(body, "pricing")) + { + if (body["pricing"] is not JsonObject pricing) return false; + var outP = new JsonObject(); + foreach (var kvp in pricing) + { + if (kvp.Value is null || kvp.Value.GetValueKind() == JsonValueKind.Null) continue; + if (kvp.Key == "tiers") + { + var canon = CanonicalTierMap(kvp.Value); + if (canon is null) return false; + outP["tiers"] = canon; + } + else outP[kvp.Key] = kvp.Value.DeepClone(); + } + body["pricing"] = outP; + } + return true; + } + private static bool CanonicalizeMeasures(JsonObject body) { + if (!CanonicalizeTiers(body)) return false; if (PresentNonNull(body, "measure")) { var canon = CanonicalMeasure(body["measure"]); diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md index b8c9de4c8..03772ab65 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KNOWN_GAPS.md @@ -78,13 +78,16 @@ value → set). Notes: `fixtures/api-contract-conformance/jsonb/scenarios/jsonb-value-object-patch.yaml`. - `field.object @storage flattened` — EXCLUDED: the Exposed table materialises it as per-subfield columns; there is no single `Table.` to bind. - - `field.map` — EXCLUDED (`it !is MapField`): dict-of-VO, staged out. + - `field.map @objectRef` (dict-of-VO) — SETTABLE, created and read like a jsonb value-object + column (`Map`, each value validated, a decimal in a value following the + ADR-0019 Amendment 1 rule per value). A scalar-valued `field.map` (`@valueType`) stays + EXCLUDED (`isVoMapColumn`), staged out. - `field.string @dbColumnType=jsonb` open-bag — EXCLUDED (`isJsonbOpenBag`): create-only (next bullet); its PATCH is the tracked Kotlin follow-up (the kotlinx `parseToJsonElement` bridge) the sibling `codegen-spring` `KNOWN_GAPS.md` lists under "Still staged out". The remaining exclusions are deliberate staging (each for the substrate reason - above), not per-port bugs — save the map one, which is a live divergence from + above), not per-port bugs — save the scalar-valued map one, which is a live divergence from Java and C# (both patch map columns; C#'s G7 records the split). Closing any of them is separately-scoped follow-up work, not a piecemeal port fix. **TPH residual.** The per-subtype path has its own SSOT — diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt index 5283ca0b2..dac7c31bc 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinEntityGenerator.kt @@ -295,9 +295,9 @@ open class KotlinEntityGenerator : MultiFileDirectGeneratorBase() { // Program D (spec §0): a field.object (value-object) member carries @field:Valid so a // parent validator.validate(bean) / @Valid cascades into the nested VO's own // constraints (the create-body POST path). jakarta validateValue does NOT cascade, so - // the PATCH path validates present VO values explicitly in the controller. MapField is - // NOT an ObjectField (dict-of-VO is staged out), so it never gets @Valid here. - if (field is ObjectField) { + // the PATCH path validates present VO values explicitly in the controller. A dict-of-VO + // MapField cascades the same way: @Valid on a Map validates each VALUE. + if (field is ObjectField || (field is MapField && readObjectRef(field) != null)) { propBuilder.addAnnotation(constraint(VALID)) } } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt index 4d3c88f59..78c797eb1 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinSpringControllerGenerator.kt @@ -214,14 +214,17 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase and validated (spec §0). Still EXCLUDED: MapField - // (dict-of-VO, staged out) and the `field.string @dbColumnType=jsonb` open bag (its + // the generated VO record / List and validated (spec §0). A dict-of-VO MapField is + // settable the same way (Map, each value validated). Still EXCLUDED: a + // scalar-valued MapField and the `field.string @dbColumnType=jsonb` open bag (its // in-process type is a kotlinx JsonElement the raw-JsonNode patch path can't bind — a // create-only column, see KNOWN_GAPS). // When this is EMPTY (a PK + only map/open-bag columns) the controller emits no ObjectMapper // ctor param / TypeReference import (they'd be unused → allWarningsAsErrors). val patchSettableFields = entity.metaFields.filter { - it !is MapField && it.name != pkFieldName && + // A dict-of-VO map (field.map @objectRef) IS settable, like a jsonb value-object column; + // a scalar-valued map stays out (unchanged). + (it !is MapField || isVoMapColumn(it)) && it.name != pkFieldName && !KotlinTypeMapper.isJsonbOpenBag(it) && // Program D: only a jsonb value-object column is settable. A flattened object // field (@storage:flattened) is materialised by the Exposed table as per-subfield @@ -452,9 +455,10 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase) and the Exposed Column codec - // encodes it to jsonb. MapField (dict-of-VO) stays skipped (staged out); a flattened - // object field (no single `Table.`) is skipped too. - if (field is MapField || (field is ObjectField && !isJsonbObjectColumn(field))) continue + // encodes it to jsonb. A dict-of-VO MapField is written the same way (Map); a + // scalar-valued MapField stays skipped, and so does a flattened object field (no + // single `Table.`). + if ((field is MapField && !isVoMapColumn(field)) || (field is ObjectField && !isJsonbObjectColumn(field))) continue // Skip the PK column on insert — the table's @generation=increment owns it. // If the entity has no auto-incrementing PK the consumer can override the // generated handler; this is the 95% case. @@ -531,6 +535,8 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase" } else { (KotlinTypeMapper.enumTypeName(field, entity, providedEnums) ?: KotlinTypeMapper.kotlinTypeName(field)).toString() } @@ -552,6 +558,10 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase): String? { if (!field.hasMetaAttr(ObjectField.ATTR_OBJECTREF, true)) return null return runCatching { field.getMetaAttr(ObjectField.ATTR_OBJECTREF, true).valueAsString }.getOrNull() } @@ -1922,6 +1932,16 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase): Boolean = + field is MapField && readObjectRef(field) != null + + /** The fully-qualified Kotlin type of the value object a dict-of-VO map's values reference. */ + private fun voMapValueTypeFqn(field: MapField, loader: MetaDataLoader): String { + val target = readObjectRef(field)?.let { KotlinGenUtil.lookupCanonicalObject(loader, it) } + return target?.let { PackageMapping.toKotlin(it.name) } ?: "kotlin.Any" + } + /** * The fully-qualified Kotlin type of the value object a [field] `field.object` references * (e.g. `acme.store.Marker`) — the Jackson bind target for the PATCH TypeReference. Emitted @@ -2164,17 +2184,17 @@ open class KotlinSpringControllerGenerator : MultiFileDirectGeneratorBase: ResultRow → data class. Program D: a field.object value-object jsonb // column IS read here — the Exposed Column codec (the shared Jackson metaJsonbMapper) // already decodes the jsonb text to the VO record / List, so `row[Table.col]` yields - // the typed value the data-class property expects. MapField (dict-of-VO) stays skipped - // (staged out). The `field.string @dbColumnType=jsonb` open bag is a StringField, so it is + // the typed value the data-class property expects. A dict-of-VO MapField is read the same + // way (Map); a scalar-valued MapField stays skipped. The `field.string @dbColumnType=jsonb` open bag is a StringField, so it is // read here too (its column decodes to a kotlinx JsonElement). // #214: reads route to `readObj` (the view for a write-through entity), which carries the // DERIVED columns — so a derived scalar field (a StringField / etc.) maps here too. append("/** GENERATED — map an Exposed ResultRow to the ${shortName} data class. */\n") append("private fun rowTo${shortName}(row: ResultRow): ${shortName} = ${shortName}(\n") for (field in entity.metaFields) { - // MapField (staged out) and a flattened object field (materialised as per-subfield - // columns, no single `Table.`) are skipped — the data class defaults them. - if (field is MapField || (field is ObjectField && !isJsonbObjectColumn(field))) continue + // A scalar-valued MapField (staged out) and a flattened object field (materialised as + // per-subfield columns, no single `Table.`) are skipped — the data class defaults them. + if ((field is MapField && !isVoMapColumn(field)) || (field is ObjectField && !isJsonbObjectColumn(field))) continue // The column PROPERTY, which the table generator renames when the field name is a // member of Exposed's Table (`source` is declared `sourceColumn`). Identity otherwise. append(" ${field.name} = row[${readObj}.${KotlinNaming.safeColumnProperty(field.name, exposedApi())}],\n") diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt index 4febaabd1..fa900a07d 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinDecimalWireTest.kt @@ -331,6 +331,61 @@ class KotlinDecimalWireTest { assertTrue(e is com.fasterxml.jackson.core.JsonProcessingException, "an excess-scale element must be refused: $e") } + private val mapModel = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.value": { "name": "Tier", "children": [ + { "field.decimal": { "name": "amount", "@precision": 12, "@scale": 4 } } + ] } }, + { "object.value": { "name": "Pricing", "children": [ + { "field.map": { "name": "tiers", "@objectRef": "Tier" } } + ] } }, + { "object.entity": { "name": "Doc", "children": [ + { "field.long": { "name": "id" } }, + { "field.map": { "name": "tiers", "@objectRef": "Tier" } }, + { "field.object": { "name": "pricing", "@objectRef": "Pricing", "@storage": "jsonb" } }, + { "source.rdb": { "@table": "docs" } }, + { "identity.primary": { "@fields": ["id"], "@generation": "increment" } } + ] } } + ] } + }""".trimIndent() + + @Test + fun `a decimal in the value objects of a field map follows the top-level rule per value, and a key named amount is just a key`() { + val files = emit(mapModel, listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator())) + val sources = files.filterKeys { it.endsWith(".kt") }.map { (path, src) -> + com.tschuchort.compiletesting.SourceFile.kotlin(path.substringAfterLast('/'), src) + } + val result = com.tschuchort.compiletesting.KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + assertEquals(com.tschuchort.compiletesting.KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + val mapper = com.fasterxml.jackson.databind.ObjectMapper().registerModule(com.fasterxml.jackson.module.kotlin.KotlinModule.Builder().build()) + val doc = result.classLoader.loadClass("acme.shop.Doc") + fun roundTrip(json: String): com.fasterxml.jackson.databind.JsonNode = + mapper.valueToTree(mapper.readValue(json, doc)) + // Top-level map and a map nested in a value object: a string or a number is read exactly, + // written as the canonical string, for EACH value. The key "amount" is a key, not a member. + val out = roundTrip("""{"id":1, + "tiers":{"amount":{"amount":"1.5000"},"gold":{"amount":2.50}}, + "pricing":{"tiers":{"amount":{"amount":100},"silver":{"amount":"0.25"}}}}""") + assertEquals("1.5", out.get("tiers").get("amount").get("amount").asText()) + assertEquals("2.5", out.get("tiers").get("gold").get("amount").asText()) + assertEquals("100", out.get("pricing").get("tiers").get("amount").get("amount").asText()) + assertEquals("0.25", out.get("pricing").get("tiers").get("silver").get("amount").asText()) + // More fraction digits than @scale (or a non-number) is refused, never rounded. + for (bad in listOf( + """{"id":1,"tiers":{"gold":{"amount":"0.12345"}}}""", + """{"id":1,"tiers":{"amount":{"amount":0.12345}}}""", + """{"id":1,"pricing":{"tiers":{"amount":{"amount":"0.12345"}}}}""", + """{"id":1,"pricing":{"tiers":{"x":{"amount":"NaN"}}}}""", + )) { + val e = runCatching { mapper.readValue(bad, doc) }.exceptionOrNull() + assertTrue(e is com.fasterxml.jackson.core.JsonProcessingException, "must be refused: $bad but got $e") + } + } + @Test fun `a precision with no scale declares scale 0, as the Java port reads it`() { val json = model.replace("\"@precision\": 12, \"@scale\": 2,", "\"@precision\": 12,") diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md index 150db2a8e..8ba9f4e43 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/KNOWN_GAPS.md @@ -140,9 +140,9 @@ walk: the component types as `Map`, `@Valid` cascades into the map's and PATCH validates each value explicitly — vanilla handler; the TPH carve-out is below. That entry used to say a persistence-conformance roundtrip column was needed first; the CODEGEN rung did not in fact depend on it, and the gate -it named is still open and still the right one — for the RUNTIME tier, not this one. OMDB does -not read or write a map (its jsonb path keys off the `@storage` attr a map does not carry), so a -mapped column is generated-code-only until that lands. See `docs/features/field-types.md`. +it named is still open and still the right one — for the RUNTIME tier, not this one. OMDB reads and +writes a `field.map @objectRef` as a jsonb column of value objects (each value follows the +decimal rule); the TPH carve-out below still applies. See `docs/features/field-types.md`. **Still staged out** (tracked follow-ups): the Kotlin `field.string @dbColumnType=jsonb` open-bag PATCH (needs a kotlinx `parseToJsonElement` bridge). diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java index 690ac5630..47d350b77 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringDecimalWireCompileRunTest.java @@ -46,6 +46,12 @@ public class SpringDecimalWireCompileRunTest { { "object.value": { "name": "Price", "children": [ { "field.decimal": { "name": "net", "@precision": 12, "@scale": 2 } } ]}}, + { "object.value": { "name": "Tier", "children": [ + { "field.decimal": { "name": "amount", "@precision": 12, "@scale": 4 } } + ]}}, + { "object.value": { "name": "Pricing", "children": [ + { "field.map": { "name": "tiers", "@objectRef": "Tier" } } + ]}}, { "object.entity": { "name": "Entry", "children": [ { "source.rdb": { "@table": "entries" } }, { "field.long": { "name": "id" } }, @@ -53,6 +59,8 @@ public class SpringDecimalWireCompileRunTest { { "field.string": { "name": "label", "@filterable": true } }, { "field.decimal": { "name": "tiers", "isArray": true, "@precision": 12, "@scale": 2 } }, { "field.object": { "name": "price", "@objectRef": "Price", "@storage": "jsonb" } }, + { "field.map": { "name": "tierMap", "@objectRef": "Tier" } }, + { "field.object": { "name": "pricing", "@objectRef": "Pricing", "@storage": "jsonb" } }, { "field.object": { "name": "prices", "@objectRef": "Price", "isArray": true, "@storage": "jsonb" } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } ]}} @@ -172,6 +180,36 @@ public void generatedDtoPatchAndAllowlistFollowTheDecimalWireRule() throws Excep assertEquals("PatchValidationException", e.getCause().getClass().getSimpleName()); } + // C1 on a field.map of value objects: the rule applies to EACH map value; a key named + // "amount" is just a key (never matched with Tier's member), at top level and nested. + JsonNode mapOut = MAPPER.valueToTree(MAPPER.readValue( + "{\"tierMap\":{\"amount\":{\"amount\":\"1.5000\"},\"gold\":{\"amount\":2.50}}," + + "\"pricing\":{\"tiers\":{\"amount\":{\"amount\":100},\"silver\":{\"amount\":\"0.25\"}}}}", dto)); + assertEquals("1.5", mapOut.get("tierMap").get("amount").get("amount").asText()); + assertEquals("2.5", mapOut.get("tierMap").get("gold").get("amount").asText()); + assertEquals("100", mapOut.get("pricing").get("tiers").get("amount").get("amount").asText()); + assertEquals("0.25", mapOut.get("pricing").get("tiers").get("silver").get("amount").asText()); + for (String bad : new String[]{"{\"tierMap\":{\"gold\":{\"amount\":\"0.12345\"}}}", + "{\"tierMap\":{\"amount\":{\"amount\":0.12345}}}", + "{\"pricing\":{\"tiers\":{\"amount\":{\"amount\":\"0.12345\"}}}}"}) { + try { + MAPPER.readValue(bad, dto); + fail("an excess-scale decimal inside a map value must be refused: " + bad); + } catch (com.fasterxml.jackson.core.JsonProcessingException expected) { + // Spring maps this to 400 validation + } + } + for (String bad : new String[]{"{\"tierMap\":{\"gold\":{\"amount\":\"0.12345\"}}}", + "{\"pricing\":{\"tiers\":{\"amount\":{\"amount\":\"0.12345\"}}}}"}) { + try { + patchClass.getMethod("fromJson", JsonNode.class, ObjectMapper.class) + .invoke(null, MAPPER.readTree(bad), MAPPER); + fail("a PATCH map value with an excess-scale decimal must be refused: " + bad); + } catch (java.lang.reflect.InvocationTargetException e) { + assertEquals("PatchValidationException", e.getCause().getClass().getSimpleName()); + } + } + // The allowlist names its decimal fields. Class allow = cl.loadClass(PKG + ".EntryFilterAllowlist"); assertEquals(java.util.Set.of("amount"), allow.getField("DECIMAL_FIELDS").get(null)); diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt index 89c66217a..bb97528a4 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/DocumentApiServer.kt @@ -82,6 +82,8 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { it[markers] = toJsonElement(r["markers"]) it[measure] = toJsonElement(r["measure"]) it[measures] = toJsonElement(r["measures"]) + it[tiers] = toJsonElement(r["tiers"]) + it[pricing] = toJsonElement(r["pricing"]) } } } @@ -153,6 +155,8 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { it[markers] = toJsonElement(markersVal) it[measure] = toJsonElement(canon["measure"]) it[measures] = toJsonElement(canon["measures"]) + it[tiers] = toJsonElement(canon["tiers"]) + it[pricing] = toJsonElement(canon["pricing"]) }[DocumentTable.id] } val row = transaction(db) { @@ -194,7 +198,7 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { } // Apply only the present keys (absent = untouched). Guard the empty-SET case (Exposed throws). val canon = canonicalizeMeasures(body) ?: return sendJson(exchange, 400, mapOf("error" to "validation")) - val settableKeys = listOf("title", "payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures") + val settableKeys = listOf("title", "payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures", "tiers", "pricing") if (settableKeys.any { body.containsKey(it) }) { transaction(db) { DocumentTable.update({ DocumentTable.id eq id }) { @@ -205,6 +209,8 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { if (body.containsKey("markers")) it[markers] = toJsonElement(body["markers"]) if (body.containsKey("measure")) it[measure] = toJsonElement(canon["measure"]) if (body.containsKey("measures")) it[measures] = toJsonElement(canon["measures"]) + if (body.containsKey("tiers")) it[tiers] = toJsonElement(canon["tiers"]) + if (body.containsKey("pricing")) it[pricing] = toJsonElement(canon["pricing"]) } } } @@ -238,6 +244,28 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { if (many !is List<*>) return null out["measures"] = many.map { canonicalMeasure(it) ?: return null } } + // A field.map of Tier: the decimal rule applies to EACH map value; the keys are arbitrary + // text and never matched with Tier's member names (a key named "amount" is just a key). + val tiersVal = body["tiers"] + if (tiersVal != null) out["tiers"] = canonicalTierMap(tiersVal) ?: return null + // Pricing { tiers: map of Tier } + val pricingVal = body["pricing"] + if (pricingVal != null) { + val pricingMap = pricingVal as? Map<*, *> ?: return null + val outPricing = LinkedHashMap() + for ((k, v) in pricingMap) outPricing[k as String] = v + val inner = outPricing["tiers"] + if (inner != null) outPricing["tiers"] = canonicalTierMap(inner) ?: return null + out["pricing"] = outPricing + } + return out + } + + /** A map whose every value is a Tier; each value's decimal is canonicalised, or null on a violation. */ + private fun canonicalTierMap(value: Any?): Map? { + val map = value as? Map<*, *> ?: return null + val out = LinkedHashMap() + for ((k, v) in map) out[k as String] = canonicalMeasure(v) ?: return null // Tier has the one decimal(12, 4) return out } @@ -282,6 +310,8 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { "markers" to fromJsonElement(row[DocumentTable.markers]), "measure" to fromJsonElement(row[DocumentTable.measure]), "measures" to fromJsonElement(row[DocumentTable.measures]), + "tiers" to fromJsonElement(row[DocumentTable.tiers]), + "pricing" to fromJsonElement(row[DocumentTable.pricing]), ) /** Jackson value (Map/List/scalar/null) → kotlinx `JsonElement` (the column type). */ @@ -326,6 +356,8 @@ class DocumentApiServer(private val pg: PostgresContainer) : AutoCloseable { val markers = jsonb("markers", { it.toString() }, { Json.parseToJsonElement(it) }).nullable() val measure = jsonb("measure", { it.toString() }, { Json.parseToJsonElement(it) }).nullable() val measures = jsonb("measures", { it.toString() }, { Json.parseToJsonElement(it) }).nullable() + val tiers = jsonb("tiers", { it.toString() }, { Json.parseToJsonElement(it) }).nullable() + val pricing = jsonb("pricing", { it.toString() }, { Json.parseToJsonElement(it) }).nullable() override val primaryKey = PrimaryKey(id) } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/generated/GeneratedDocumentControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/generated/GeneratedDocumentControllerHarness.kt index 6c2b917c8..2050ce96d 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/generated/GeneratedDocumentControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/jsonb/generated/GeneratedDocumentControllerHarness.kt @@ -172,7 +172,7 @@ class GeneratedDocumentControllerHarness( // Program D: the seed rows carry the required primaryMarker (+ optional single/array // VO on row 1). Pass every present VO column through the generated controller's OWN // create path — so the create MUST handle value objects for seeding to succeed. - for (k in listOf("primaryMarker", "optionalMarker", "markers", "measure", "measures")) { + for (k in listOf("primaryMarker", "optionalMarker", "markers", "measure", "measures", "tiers", "pricing")) { if (r.containsKey(k)) body[k] = r[k] } val res = exchange("POST", "/api/documents", body) diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java index 32c2ef6eb..e959236ee 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/JsonbReferenceServer.java @@ -81,15 +81,17 @@ private void createSchema() throws SQLException { + "optional_marker JSONB, " // nullable single VO + "markers JSONB, " // nullable array-of-VO + "measure JSONB, " // nullable single Measure VO (decimal amount) - + "measures JSONB)"); // nullable array-of-Measure VO + + "measures JSONB, " // nullable array-of-Measure VO + + "tiers JSONB, " // nullable map of Tier VO (decimal amount per value) + + "pricing JSONB)"); // nullable Pricing VO holding a map of Tier } } private void applySeed(List> seedRows) throws SQLException { try (Connection c = connect()) { try (PreparedStatement ps = c.prepareStatement( - "INSERT INTO \"documents\" (id, title, payload, primary_marker, optional_marker, markers, measure, measures) " - + "VALUES (?, ?, ?, ?, ?, ?, ?, ?)")) { + "INSERT INTO \"documents\" (id, title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing) " + + "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")) { for (Map r : seedRows) { ps.setLong(1, ((Number) r.get("id")).longValue()); ps.setString(2, String.valueOf(r.get("title"))); @@ -99,6 +101,8 @@ private void applySeed(List> seedRows) throws SQLException { bindJsonb(ps, 6, r.get("markers")); bindJsonb(ps, 7, r.get("measure")); bindJsonb(ps, 8, r.get("measures")); + bindJsonb(ps, 9, r.get("tiers")); + bindJsonb(ps, 10, r.get("pricing")); ps.executeUpdate(); } } @@ -163,7 +167,7 @@ private void doHandle(HttpExchange exchange) throws IOException, SQLException { private Map findById(long id) throws SQLException { try (Connection c = connect(); PreparedStatement ps = c.prepareStatement( - "SELECT id, title, payload, primary_marker, optional_marker, markers, measure, measures " + "SELECT id, title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing " + "FROM \"documents\" WHERE id = ?")) { ps.setLong(1, id); try (ResultSet rs = ps.executeQuery()) { @@ -175,9 +179,9 @@ private Map findById(long id) throws SQLException { private Map create(Map body) throws SQLException { try (Connection c = connect(); PreparedStatement ps = c.prepareStatement( - "INSERT INTO \"documents\" (title, payload, primary_marker, optional_marker, markers, measure, measures) " - + "VALUES (?, ?, ?, ?, ?, ?, ?) " - + "RETURNING id, title, payload, primary_marker, optional_marker, markers, measure, measures")) { + "INSERT INTO \"documents\" (title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing) " + + "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) " + + "RETURNING id, title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing")) { ps.setString(1, String.valueOf(body.get("title"))); bindJsonb(ps, 2, body.get("payload")); bindJsonb(ps, 3, body.get("primaryMarker")); @@ -185,6 +189,8 @@ private Map create(Map body) throws SQLException bindJsonb(ps, 5, body.get("markers")); bindJsonb(ps, 6, body.get("measure")); bindJsonb(ps, 7, body.get("measures")); + bindJsonb(ps, 8, body.get("tiers")); + bindJsonb(ps, 9, body.get("pricing")); try (ResultSet rs = ps.executeQuery()) { rs.next(); return rowFrom(rs); @@ -209,6 +215,8 @@ private Map patch(long id, Map body) throws SQLE addAssignment(body, "markers", "markers", true, setClauses, values, jsonb); addAssignment(body, "measure", "measure", true, setClauses, values, jsonb); addAssignment(body, "measures", "measures", true, setClauses, values, jsonb); + addAssignment(body, "tiers", "tiers", true, setClauses, values, jsonb); + addAssignment(body, "pricing", "pricing", true, setClauses, values, jsonb); if (!setClauses.isEmpty()) { String sql = "UPDATE \"documents\" SET " + String.join(", ", setClauses) + " WHERE id = ?"; try (Connection c = connect(); PreparedStatement ps = c.prepareStatement(sql)) { @@ -244,6 +252,8 @@ private Map rowFrom(ResultSet rs) throws SQLException { row.put("markers", parseJson(rs.getString("markers"))); row.put("measure", parseJson(rs.getString("measure"))); row.put("measures", parseJson(rs.getString("measures"))); + row.put("tiers", parseJson(rs.getString("tiers"))); + row.put("pricing", parseJson(rs.getString("pricing"))); return row; } @@ -336,9 +346,42 @@ private static boolean canonicalizeMeasures(Map body) { } body.put("measures", out); } + // A field.map of Tier: the decimal rule applies to EACH map value; keys are never matched + // against Tier's member names (a key named "amount" is just a key). + Object tiers = body.get("tiers"); + if (body.containsKey("tiers") && tiers != null) { + Map canon = canonicalTierMap(tiers); + if (canon == null) return false; + body.put("tiers", canon); + } + // Pricing { tiers: map of Tier } + Object pricing = body.get("pricing"); + if (body.containsKey("pricing") && pricing != null) { + if (!(pricing instanceof Map pm)) return false; + Map outPricing = new LinkedHashMap<>((Map) pm); + Object inner = outPricing.get("tiers"); + if (inner != null) { + Map canon = canonicalTierMap(inner); + if (canon == null) return false; + outPricing.put("tiers", canon); + } + body.put("pricing", outPricing); + } return true; } + /** A map whose every value is a Tier; each value's decimal is canonicalised, or null on a violation. */ + private static Map canonicalTierMap(Object map) { + if (!(map instanceof Map m)) return null; + Map out = new LinkedHashMap<>(); + for (Map.Entry e : m.entrySet()) { + Map canon = canonicalMeasure(e.getValue()); // Tier has the same one decimal(12, 4) + if (canon == null) return null; + out.put(String.valueOf(e.getKey()), canon); + } + return out; + } + /** A Measure with its decimal amount range-checked and written as its canonical string, or null. */ @SuppressWarnings("unchecked") private static Map canonicalMeasure(Object measure) { diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryDocumentRepositorySource.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryDocumentRepositorySource.java index 6b5577b67..fb4e0d08e 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryDocumentRepositorySource.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/generated/InMemoryDocumentRepositorySource.java @@ -80,7 +80,7 @@ public Optional findById(Long id) { @Override public DocumentDto create(DocumentDto dto) { DocumentDto saved = new DocumentDto(nextId.getAndIncrement(), dto.title(), dto.payload(), - dto.primaryMarker(), dto.optionalMarker(), dto.markers(), dto.measure(), dto.measures()); + dto.primaryMarker(), dto.optionalMarker(), dto.markers(), dto.measure(), dto.measures(), dto.tiers(), dto.pricing()); rows.add(saved); return saved; } @@ -98,7 +98,9 @@ public Optional update(Long id, DocumentDto dto) { dto.optionalMarker() != null ? dto.optionalMarker() : cur.optionalMarker(), dto.markers() != null ? dto.markers() : cur.markers(), dto.measure() != null ? dto.measure() : cur.measure(), - dto.measures() != null ? dto.measures() : cur.measures()); + dto.measures() != null ? dto.measures() : cur.measures(), + dto.tiers() != null ? dto.tiers() : cur.tiers(), + dto.pricing() != null ? dto.pricing() : cur.pricing()); rows.set(i, merged); return Optional.of(merged); } @@ -121,7 +123,9 @@ public Optional patch(Long id, DocumentPatch patch) { patch.hasOptionalMarker() ? patch.optionalMarker() : cur.optionalMarker(), patch.hasMarkers() ? patch.markers() : cur.markers(), patch.hasMeasure() ? patch.measure() : cur.measure(), - patch.hasMeasures() ? patch.measures() : cur.measures()); + patch.hasMeasures() ? patch.measures() : cur.measures(), + patch.hasTiers() ? patch.tiers() : cur.tiers(), + patch.hasPricing() ? patch.pricing() : cur.pricing()); rows.set(i, merged); return Optional.of(merged); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/io/object/gson/MetaObjectDeserializer.java b/server/java/metadata/src/main/java/com/metaobjects/io/object/gson/MetaObjectDeserializer.java index 3574b45dd..959fea6cf 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/io/object/gson/MetaObjectDeserializer.java +++ b/server/java/metadata/src/main/java/com/metaobjects/io/object/gson/MetaObjectDeserializer.java @@ -183,6 +183,8 @@ protected void readFieldValue(MetaObject mo, MetaField mf, Object vo, // Check if this is an array field using universal @isArray support if (mf.isArrayType()) { readFieldObjectArray(mo, mf, vo, el, context); + } else if (isObjectRefMap(mf)) { + readFieldObjectMap(mf, vo, el, context); } else { readFieldObject(mo, mf, vo, el, context); } @@ -268,6 +270,35 @@ protected void readFieldObjectArray(MetaObject mo, MetaField mf, Object vo, mf.setObjectArray( vo, objects ); } + /** A {@code field.map} whose values are a referenced value object ({@code @objectRef}). */ + private static boolean isObjectRefMap(MetaField mf) { + return com.metaobjects.field.MapField.SUBTYPE_MAP.equals(mf.getSubType()) + && MetaDataUtil.hasObjectRef(mf); + } + + /** Read a map of value objects: each VALUE is the referenced object (its own field rules, + * decimals included, apply); the keys are arbitrary text and are never matched with the + * object's member names. */ + protected void readFieldObjectMap(MetaField mf, Object vo, JsonElement el, + JsonDeserializationContext context) { + if (!el.isJsonObject()) throw new MetaDataException("Expected JsonObject when reading MetaField "+ + "["+mf+"], but found JsonElement: "+el); + Class clazz = getObjectRefClass(mf); + // Deserialize each value with the REFERENCED object's own deserializer: several value + // objects share one Java class (ValueObject), so a class-keyed context.deserialize would + // read a Tier with the containing object's rules when the JSON carries no "@type". + MetaObjectDeserializer valueReader = new MetaObjectDeserializer(MetaDataUtil.getObjectRef(mf)); + java.util.Map out = new java.util.LinkedHashMap<>(); + for (java.util.Map.Entry e : ((JsonObject) el).entrySet()) { + JsonElement v = e.getValue(); + if (v.isJsonNull()) { out.put(e.getKey(), null); continue; } + if (!v.isJsonObject()) throw new MetaDataException("Expected JsonObject for key ["+e.getKey()+ + "] when reading MetaField ["+mf+"], but found JsonElement: "+v); + out.put(e.getKey(), valueReader.deserialize(v, clazz, context)); + } + mf.setObject(vo, out); + } + protected void readFieldObject(MetaObject mo, MetaField mf, Object vo, JsonElement el, JsonDeserializationContext context) { diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java index 212316f68..f79a4f51f 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/SimpleMappingHandlerDB.java @@ -262,9 +262,12 @@ protected String resolveDbColumnType(MetaField mf) { return null; } - /** Returns true if the field is declared as a typed-owned-object jsonb column via {@code @storage="jsonb"}. */ + /** Returns true if the field is a typed jsonb column: a field.map of a value object, or one declared via {@code @storage="jsonb"}. */ protected boolean isJsonbField(MetaField mf) { try { + if (com.metaobjects.field.MapField.SUBTYPE_MAP.equals(mf.getSubType()) && com.metaobjects.util.MetaDataUtil.hasObjectRef(mf)) { + return true; + } return mf.hasMetaAttr(com.metaobjects.field.ObjectField.ATTR_STORAGE) && CoreDBMetaDataProvider.DB_COLUMN_TYPE_JSONB.equals( mf.getMetaAttr(com.metaobjects.field.ObjectField.ATTR_STORAGE).getValueAsString()); diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/GenericSQLDriver.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/GenericSQLDriver.java index d73080ecc..f9be855f8 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/GenericSQLDriver.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/driver/GenericSQLDriver.java @@ -76,13 +76,16 @@ public class GenericSQLDriver implements DatabaseDriver { private final java.util.Map fallbackGsonCache = new java.util.concurrent.ConcurrentHashMap<>(); /** - * Returns true if the field is declared as a typed-owned-object jsonb column via + * Returns true if the field is a typed jsonb column: a field.map of a value object, or one declared via * {@code @storage="jsonb"} (the cross-port owned-object storage shape; see * {@link com.metaobjects.field.ObjectField#ATTR_STORAGE}). Uses the attr model so * jsonb detection works even without a dedicated {@code JsonbField} class. */ protected boolean isJsonbField(MetaField f) { try { + if (com.metaobjects.field.MapField.SUBTYPE_MAP.equals(f.getSubType()) && MetaDataUtil.hasObjectRef(f)) { + return true; + } return f.hasMetaAttr(com.metaobjects.field.ObjectField.ATTR_STORAGE) && CoreDBMetaDataProvider.DB_COLUMN_TYPE_JSONB.equals( f.getMetaAttr(com.metaobjects.field.ObjectField.ATTR_STORAGE).getValueAsString()); @@ -191,11 +194,16 @@ protected Object deserializeJsonb(MetaField f, String json) { try { // isArray:true → the JSON is an array of the referenced VO; target List instead of VO. boolean isArray = f.isArrayType(); + // field.map @objectRef → the JSON is an object whose VALUES are the referenced VO + // (keys are arbitrary text, never VO member names); target Map so the VO + // deserializer (and its decimal read) applies to each value. + boolean isMap = com.metaobjects.field.MapField.SUBTYPE_MAP.equals(f.getSubType()) + && MetaDataUtil.hasObjectRef(f); // Step 1: registry-bound typed POJO wins, deserialized via Gson reflection. if (MetaDataUtil.hasObjectRef(f)) { Class bound = ObjectClassRegistry.global().resolve(resolveObjectRefFqn(f)); if (bound != null) { - return buildGson(f.getLoader()).fromJson(json, jsonbTargetType(bound, isArray)); + return buildGson(f.getLoader()).fromJson(json, jsonbTargetType(bound, isArray, isMap)); } } // Step 2: fall back to the MetaObject's declared class (e.g. ValueObject). @@ -207,7 +215,7 @@ protected Object deserializeJsonb(MetaField f, String json) { m -> new GsonBuilder() .registerTypeAdapter(refClass, new MetaObjectDeserializer(m)) .create()); - return fallbackGson.fromJson(json, jsonbTargetType(refClass, isArray)); + return fallbackGson.fromJson(json, jsonbTargetType(refClass, isArray, isMap)); } catch (Exception e) { throw new IllegalStateException("jsonb deserialize failed for field [" + f + "]: " + e, e); } @@ -218,7 +226,11 @@ protected Object deserializeJsonb(MetaField f, String json) { * array ({@code isArray:true}), else the bare {@code elementClass}. The parameterized {@code List} * type drives Gson to apply the element deserializer per array element (see {@link #deserializeJsonb}). */ - private static java.lang.reflect.Type jsonbTargetType(Class elementClass, boolean isArray) { + private static java.lang.reflect.Type jsonbTargetType(Class elementClass, boolean isArray, boolean isMap) { + if (isMap) { + return com.google.gson.reflect.TypeToken.getParameterized( + java.util.LinkedHashMap.class, String.class, elementClass).getType(); + } return isArray ? com.google.gson.reflect.TypeToken.getParameterized(java.util.List.class, elementClass).getType() : elementClass; diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/JsonbMapOfValueObjectDBTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/JsonbMapOfValueObjectDBTest.java new file mode 100644 index 000000000..29b67fbb5 --- /dev/null +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/JsonbMapOfValueObjectDBTest.java @@ -0,0 +1,156 @@ +package com.metaobjects.manager.db; + +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.manager.ObjectConnection; +import com.metaobjects.manager.db.driver.DerbyDriver; +import com.metaobjects.object.MetaObject; +import com.metaobjects.object.value.ValueObject; +import com.metaobjects.registry.MetaDataLoaderRegistry; +import com.metaobjects.registry.ServiceRegistryFactory; + +import org.junit.After; +import org.junit.AfterClass; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; + +import javax.sql.DataSource; +import java.io.PrintWriter; +import java.math.BigDecimal; +import java.sql.*; +import java.util.Collection; +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.logging.Logger; + +import static org.junit.Assert.*; + +/** + * ADR-0019 Amendment 1, C1: a {@code field.map} whose values are a value object holds a decimal + * per value, read exactly and written back unchanged. The map keys are arbitrary text and are + * never matched with the value object's member names (a key named {@code amount} is a key). + * Two shapes: a top-level jsonb map column and a map nested inside a value object. + */ +public class JsonbMapOfValueObjectDBTest { + + private static ObjectManagerDB omdb; + private static String dbFile; + private static MetaDataLoader loader; + private static MetaDataLoaderRegistry registry; + private ObjectConnection oc; + + @BeforeClass + public static void setupDB() throws Exception { + registry = new MetaDataLoaderRegistry(ServiceRegistryFactory.getDefault()); + loader = MetaDataLoader.fromResources("test-jsonb-map-db", java.util.List.of("meta.jsonbmap.json")); + registry.registerLoader(loader); + + dbFile = "jsonbmap-testing-" + System.currentTimeMillis(); + Class.forName("org.apache.derby.jdbc.EmbeddedDriver"); + getConnection().close(); + DataSource ds = new DataSource() { + @Override public Connection getConnection() throws SQLException { return JsonbMapOfValueObjectDBTest.getConnection(); } + @Override public Connection getConnection(String u, String p) throws SQLException { return getConnection(); } + @Override public PrintWriter getLogWriter() { return new PrintWriter(System.out); } + @Override public void setLogWriter(PrintWriter out) {} + @Override public void setLoginTimeout(int s) {} + @Override public int getLoginTimeout() { return 100; } + @Override public Logger getParentLogger() throws SQLFeatureNotSupportedException { throw new UnsupportedOperationException(); } + @Override public T unwrap(Class iface) { throw new UnsupportedOperationException(); } + @Override public boolean isWrapperFor(Class iface) { return false; } + }; + omdb = new ObjectManagerDB(); + omdb.setDatabaseDriver(new DerbyDriver()); + omdb.setDataSource(ds); + omdb.init(); + try (Connection c = getConnection(); Statement s = c.createStatement()) { + s.execute("CREATE TABLE JSONMAP_DOC (\n" + + " id BIGINT GENERATED ALWAYS AS IDENTITY CONSTRAINT JSONMAP_DOC_id_PK PRIMARY KEY,\n" + + " tiers VARCHAR(4000),\n" + + " pricing VARCHAR(4000))"); + } + } + + private static Connection getConnection() throws SQLException { + return DriverManager.getConnection("jdbc:derby:memory:" + dbFile + ";create=true"); + } + + @AfterClass + public static void destroyDB() throws Exception { + if (dbFile != null) { + try { DriverManager.getConnection("jdbc:derby:memory:" + dbFile + ";drop=true"); } + catch (SQLNonTransientConnectionException ignored) {} + } + if (loader != null) loader.destroy(); + } + + @Before public void open() throws Exception { oc = omdb.getConnection(); } + + @After + public void close() throws Exception { + try (Connection c = getConnection(); Statement s = c.createStatement()) { s.execute("DELETE FROM JSONMAP_DOC"); } + omdb.releaseConnection(oc); + } + + private ValueObject tier(String amount) { + ValueObject t = (ValueObject) registry.findMetaObjectByName("jsonmap::Tier").newInstance(); + t.setObject("amount", new BigDecimal(amount)); + return t; + } + + private static BigDecimal amountOf(Object tier) { + assertTrue("a map value must be a Tier value object, got " + (tier == null ? "null" : tier.getClass()), + tier instanceof ValueObject); + return (BigDecimal) ((ValueObject) tier).getObject("amount"); + } + + @SuppressWarnings("unchecked") + @Test + public void mapOfValueObjectsRoundTripsEachDecimalExactly() throws Exception { + MetaObject doc = registry.findMetaObjectByName("jsonmap::Doc"); + ValueObject item = (ValueObject) doc.newInstance(); + Map tiers = new LinkedHashMap<>(); + tiers.put("amount", tier("1.5000")); // a KEY named like the member: still a key + tiers.put("gold", tier("2.5")); + item.setObject("tiers", tiers); + + ValueObject pricing = (ValueObject) registry.findMetaObjectByName("jsonmap::Pricing").newInstance(); + Map nested = new LinkedHashMap<>(); + nested.put("amount", tier("100")); + nested.put("silver", tier("0.25")); + pricing.setObject("tiers", nested); + item.setObject("pricing", pricing); + omdb.createObject(oc, item); + + Collection rows = omdb.getObjects(oc, doc); + assertEquals(1, rows.size()); + ValueObject loaded = (ValueObject) rows.iterator().next(); + + Map top = (Map) loaded.getObject("tiers"); + assertEquals(java.util.List.of("amount", "gold"), new java.util.ArrayList<>(top.keySet())); + assertEquals(0, new BigDecimal("1.5").compareTo(amountOf(top.get("amount")))); + assertEquals(0, new BigDecimal("2.5").compareTo(amountOf(top.get("gold")))); + + ValueObject loadedPricing = (ValueObject) loaded.getObject("pricing"); + Map inner = (Map) loadedPricing.getObject("tiers"); + assertEquals(0, new BigDecimal("100").compareTo(amountOf(inner.get("amount")))); + assertEquals(0, new BigDecimal("0.25").compareTo(amountOf(inner.get("silver")))); + } + + @SuppressWarnings("unchecked") + @Test + public void storedJsonNumbersAreReadExactlyNotViaDouble() throws Exception { + try (Connection c = getConnection(); Statement s = c.createStatement()) { + s.execute("INSERT INTO JSONMAP_DOC (tiers, pricing) VALUES (" + + "'{\"amount\":{\"amount\":12345678.1234},\"x\":{\"amount\":\"0.0001\"}}', " + + "'{\"tiers\":{\"amount\":{\"amount\":0.1000}}}')"); + } + MetaObject doc = registry.findMetaObjectByName("jsonmap::Doc"); + ValueObject loaded = (ValueObject) omdb.getObjects(oc, doc).iterator().next(); + Map top = (Map) loaded.getObject("tiers"); + assertEquals(new BigDecimal("12345678.1234"), amountOf(top.get("amount"))); + assertEquals(new BigDecimal("0.0001"), amountOf(top.get("x"))); + Map inner = (Map) ((ValueObject) loaded.getObject("pricing")).getObject("tiers"); + assertEquals(new BigDecimal("0.1000"), amountOf(inner.get("amount"))); + } +} diff --git a/server/java/omdb/src/test/resources/meta.jsonbmap.json b/server/java/omdb/src/test/resources/meta.jsonbmap.json new file mode 100644 index 000000000..97ca5e535 --- /dev/null +++ b/server/java/omdb/src/test/resources/meta.jsonbmap.json @@ -0,0 +1,38 @@ +{ + "metadata.root": { + "package": "jsonmap", + "children": [ + { + "object.entity": { + "name": "Doc", + "@object": "com.metaobjects.object.value.ValueObject", + "children": [ + { "source.rdb": { "@table": "JSONMAP_DOC" } }, + { "field.long": { "name": "id", "@column": "id" } }, + { "field.map": { "name": "tiers", "@column": "tiers", "@objectRef": "jsonmap::Tier" } }, + { "field.object": { "name": "pricing", "@column": "pricing", "@storage": "jsonb", "@objectRef": "jsonmap::Pricing" } }, + { "identity.primary": { "name": "primary", "@fields": ["id"], "@generation": "increment" } } + ] + } + }, + { + "object.value": { + "name": "Tier", + "@object": "com.metaobjects.object.value.ValueObject", + "children": [ + { "field.decimal": { "name": "amount", "@column": "amount", "@precision": 12, "@scale": 4 } } + ] + } + }, + { + "object.value": { + "name": "Pricing", + "@object": "com.metaobjects.object.value.ValueObject", + "children": [ + { "field.map": { "name": "tiers", "@column": "tiers", "@objectRef": "jsonmap::Tier" } } + ] + } + } + ] + } +} diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index 72987e644..865e9a05c 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -229,24 +229,33 @@ def _nested_decimal_spec(vo: MetaObject, seen: frozenset[str]) -> dict[str, Any] for f in vo.fields(): if f.sub_type == fc.FIELD_SUBTYPE_DECIMAL: spec["d"][f.name] = _decimal_limit(f) - elif f.sub_type == fc.FIELD_SUBTYPE_OBJECT: - target = object_ref_target(f) - inner = _nested_decimal_spec(target, seen) if target is not None else None + elif f.sub_type in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): + inner = _field_decimal_spec(f, seen) if inner is not None: spec["o"][f.name] = inner return spec if spec["d"] or spec["o"] else None +def _field_decimal_spec(f: MetaField, seen: frozenset[str]) -> dict[str, Any] | None: + """The spec for a value-object-holding field. A ``field.map`` is wrapped as + ``{"m": }``: the spec applies to EACH map value and the keys are never + compared with member names. A map whose value is not a value object has no spec.""" + target = object_ref_target(f) + inner = _nested_decimal_spec(target, seen) if target is not None else None + if inner is not None and f.sub_type == fc.FIELD_SUBTYPE_MAP: + return {"m": inner} + return inner + + def _nested_decimals(entities: list[MetaObject]) -> dict[str, Any]: """``{object field name: spec}`` for each jsonb value-object field of *entities* that reaches a decimal (the first declaration of a name wins).""" out: dict[str, Any] = {} for entity in entities: for f in entity.fields(): - if f.name in out or f.sub_type != fc.FIELD_SUBTYPE_OBJECT: + if f.name in out or f.sub_type not in (fc.FIELD_SUBTYPE_OBJECT, fc.FIELD_SUBTYPE_MAP): continue - target = object_ref_target(f) - spec = _nested_decimal_spec(target, frozenset()) if target is not None else None + spec = _field_decimal_spec(f, frozenset()) if spec is not None: out[f.name] = spec return out diff --git a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py index d6b52a070..03ac96332 100644 --- a/server/python/src/metaobjects/codegen/runtime/decimal_wire.py +++ b/server/python/src/metaobjects/codegen/runtime/decimal_wire.py @@ -173,12 +173,17 @@ def coerce_decimal_fields( def _nested(value: Any, spec: dict[str, Any], path: str) -> Any: """A jsonb value-object field (*value* a dict, or a list of them) under the top-level decimal rule: each declared decimal inside is parsed under W2, held to its range under W3 - (*spec*: ``{"d": {name: (precision, scale) | None}, "o": {name: spec}}``) and stored as its + (*spec*: ``{"d": {name: (precision, scale) | None}, "o": {name: spec}}``, or ``{"m": spec}`` for a + ``field.map`` of value objects, applied to each value) and stored as its canonical string; every other number is a float. Raises ``ValueError`` naming the path.""" if isinstance(value, list): return [_nested(v, spec, path) for v in value] if not isinstance(value, dict): return _floats(value) + if "m" in spec: + # a field.map of value objects: the spec applies to EACH value; keys are arbitrary + # text and never matched against member names. + return {k: _nested(v, spec["m"], f"{path}.{k}") for k, v in value.items()} out: dict[str, Any] = {} for k, v in value.items(): if k in spec["d"] and v is not None: diff --git a/server/python/tests/integration/api_contract_jsonb_server.py b/server/python/tests/integration/api_contract_jsonb_server.py index 754e73b44..46f87206e 100644 --- a/server/python/tests/integration/api_contract_jsonb_server.py +++ b/server/python/tests/integration/api_contract_jsonb_server.py @@ -60,9 +60,11 @@ "markers": "markers", "measure": "measure", "measures": "measures", + "tiers": "tiers", + "pricing": "pricing", } _JSONB_FIELDS: frozenset[str] = frozenset( - {"payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures"} + {"payload", "primaryMarker", "optionalMarker", "markers", "measure", "measures", "tiers", "pricing"} ) @@ -82,7 +84,9 @@ def create_schema(self) -> None: " optional_marker JSONB," " markers JSONB," " measure JSONB," - " measures JSONB" + " measures JSONB," + " tiers JSONB," + " pricing JSONB" ")" ) @@ -100,8 +104,8 @@ def apply_seed(self, rows: list[dict[str, Any]]) -> None: for r in rows: cur.execute( 'INSERT INTO "documents" ' - "(id, title, payload, primary_marker, optional_marker, markers, measure, measures) " - "VALUES (%s, %s, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb)", + "(id, title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing) " + "VALUES (%s, %s, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb)", ( int(r["id"]), r["title"], @@ -111,6 +115,8 @@ def apply_seed(self, rows: list[dict[str, Any]]) -> None: _dump_payload(r.get("markers")), _dump_payload(r.get("measure")), _dump_payload(r.get("measures")), + _dump_payload(r.get("tiers")), + _dump_payload(r.get("pricing")), ), ) cur.execute( @@ -126,7 +132,7 @@ def find_by_id(self, ident: int) -> dict[str, Any] | None: cur = conn.cursor() try: cur.execute( - "SELECT id, title, payload, primary_marker, optional_marker, markers, measure, measures " + "SELECT id, title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing " 'FROM "documents" WHERE id = %s', (int(ident),), ) @@ -141,8 +147,8 @@ def create(self, dto: dict[str, Any]) -> dict[str, Any]: try: cur.execute( 'INSERT INTO "documents" ' - "(title, payload, primary_marker, optional_marker, markers, measure, measures) " - "VALUES (%s, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb) RETURNING id", + "(title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing) " + "VALUES (%s, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb, %s::jsonb) RETURNING id", ( dto.get("title"), _dump_payload(dto.get("payload")), @@ -151,6 +157,8 @@ def create(self, dto: dict[str, Any]) -> dict[str, Any]: _dump_payload(dto.get("markers")), _dump_payload(dto.get("measure")), _dump_payload(dto.get("measures")), + _dump_payload(dto.get("tiers")), + _dump_payload(dto.get("pricing")), ), ) new_id = int(cur.fetchone()[0]) @@ -233,7 +241,7 @@ def create_document(dto: dict[str, Any]) -> Any: if dto.get(name) is None: return _validation_400() # Present values validated with the create rules (title @maxLength + nested VO). - if _title_violation(dto) or _vo_present_violation(dto) or not _canonicalize_measures(dto): + if _title_violation(dto) or _vo_present_violation(dto) or not _canonicalize_measures(dto) or not _canonicalize_tiers(dto): return _validation_400() return repo.create(dto) @@ -247,7 +255,7 @@ def update_document(document_id: int, dto: dict[str, Any]) -> Any: if name in dto and dto[name] is None: return _validation_400() # Validate PRESENT, non-null values (nested VO recurses into each element). - if _title_violation(dto) or _vo_present_violation(dto) or not _canonicalize_measures(dto): + if _title_violation(dto) or _vo_present_violation(dto) or not _canonicalize_measures(dto) or not _canonicalize_tiers(dto): return _validation_400() saved = repo.update(document_id, dto) if saved is None: @@ -335,6 +343,59 @@ def _canonicalize_measures(dto: dict[str, Any]) -> bool: return True +_TIER_PRECISION, _TIER_SCALE = 12, 4 + + +def _canonical_tier_map(value: Any) -> dict[str, Any] | None: + """A ``field.map`` of ``Tier`` ({amount: decimal(12, 4)}): the decimal rule applies to EACH + map value; the keys are arbitrary and never compared with member names. ``None`` on a violation.""" + if not isinstance(value, dict): + return None + out: dict[str, Any] = {} + for key, tier in value.items(): + canon = _canonical_decimal_member(tier, "amount", _TIER_PRECISION, _TIER_SCALE) + if canon is None: + return None + out[key] = canon + return out + + +def _canonical_decimal_member(obj: Any, member: str, precision: int, scale: int) -> dict[str, Any] | None: + if not isinstance(obj, dict): + return None + out = dict(obj) + if out.get(member) is not None: + try: + d = parse_decimal(out[member]) + check_decimal_range(d, precision, scale) + out[member] = format_decimal(d) + except (ValueError, ArithmeticError): + return None + return out + + +def _canonicalize_tiers(dto: dict[str, Any]) -> bool: + """Validate and canonicalise Document.tiers (a map of Tier) and Document.pricing (a Pricing + whose ``tiers`` is a map of Tier) in place; False on a violation.""" + if dto.get("tiers") is not None: + canon = _canonical_tier_map(dto["tiers"]) + if canon is None: + return False + dto["tiers"] = canon + pricing = dto.get("pricing") + if pricing is not None: + if not isinstance(pricing, dict): + return False + out = dict(pricing) + if out.get("tiers") is not None: + canon = _canonical_tier_map(out["tiers"]) + if canon is None: + return False + out["tiers"] = canon + dto["pricing"] = out + return True + + def _dump_payload(value: Any) -> Any: """jsonb bind: a dict/list is ``json.dumps``-serialized for the ``::jsonb`` cast; ``None`` stays ``None`` (SQL NULL); a pre-serialized string passes @@ -358,7 +419,7 @@ def _decode_jsonb(value: Any) -> Any: def _row_to_dict(row: Any) -> dict[str, Any]: - ident, title, payload, primary_marker, optional_marker, markers, measure, measures = row + ident, title, payload, primary_marker, optional_marker, markers, measure, measures, tiers, pricing = row return { "id": int(ident), "title": title, @@ -368,4 +429,6 @@ def _row_to_dict(row: Any) -> dict[str, Any]: "markers": _decode_jsonb(markers), "measure": _decode_jsonb(measure), "measures": _decode_jsonb(measures), + "tiers": _decode_jsonb(tiers), + "pricing": _decode_jsonb(pricing), } diff --git a/server/python/tests/integration/test_generated_decimal_wire.py b/server/python/tests/integration/test_generated_decimal_wire.py index 98702e134..5b6410bc7 100644 --- a/server/python/tests/integration/test_generated_decimal_wire.py +++ b/server/python/tests/integration/test_generated_decimal_wire.py @@ -201,6 +201,12 @@ def test_a_value_beyond_the_declared_range_is_a_400_and_nothing_is_rounded(clien {"field.double": {"name": "readings", "isArray": True}}, {"field.decimal": {"name": "amount", "@precision": 12, "@scale": 4}}, ]}}, + {"object.value": {"name": "Tier", "children": [ + {"field.decimal": {"name": "amount", "@precision": 12, "@scale": 4}}, + ]}}, + {"object.value": {"name": "Pricing", "children": [ + {"field.map": {"name": "tiers", "@objectRef": "Tier"}}, + ]}}, {"object.entity": {"name": "Sample", "children": [ {"source.rdb": {"@table": "samples"}}, {"field.long": {"name": "id"}}, @@ -208,6 +214,8 @@ def test_a_value_beyond_the_declared_range_is_a_400_and_nothing_is_rounded(clien {"field.decimal": {"name": "series", "isArray": True, "@precision": 9, "@scale": 4}}, {"field.object": {"name": "fix", "@objectRef": "Fix", "@storage": "jsonb"}}, {"field.object": {"name": "fixes", "@objectRef": "Fix", "@storage": "jsonb", "isArray": True}}, + {"field.map": {"name": "tiers", "@objectRef": "Tier"}}, + {"field.object": {"name": "pricing", "@objectRef": "Pricing", "@storage": "jsonb"}}, {"identity.primary": {"name": "pk", "@fields": ["id"], "@generation": "increment"}}, ]}}, ], @@ -224,7 +232,8 @@ def float_client() -> TestClient: pkg_dir = tmp / pkg pkg_dir.mkdir() (pkg_dir / "__init__.py").write_text("") - (pkg_dir / "Fix.py").write_text(render_entity_model(objs["Fix"])) + for vo in ("Fix", "Tier", "Pricing"): + (pkg_dir / f"{vo}.py").write_text(render_entity_model(objs[vo])) sample = objs["Sample"] (pkg_dir / "sample_filter_allowlist.py").write_text(render_filter_allowlist(sample) or "") (pkg_dir / "Sample.py").write_text(render_entity_model(sample)) @@ -235,7 +244,7 @@ def float_client() -> TestClient: finally: sys.path.remove(str(tmp)) db = sqlite3.connect(":memory:", check_same_thread=False) - db.execute("CREATE TABLE samples (id INTEGER PRIMARY KEY AUTOINCREMENT, score REAL, series TEXT, fix TEXT, fixes TEXT)") + db.execute("CREATE TABLE samples (id INTEGER PRIMARY KEY AUTOINCREMENT, score REAL, series TEXT, fix TEXT, fixes TEXT, tiers TEXT, pricing TEXT)") om = ObjectManager(root, SqliteDriver(db)) # type: ignore[arg-type] class Repo(_Repo): @@ -283,7 +292,7 @@ def test_a_decimal_inside_a_jsonb_value_object_follows_the_top_level_rule(float_ ) assert ok.status_code == 201, ok.text body = ok.json() - assert body["fix"] == {"latitude": 1.5, "amount": "100.5", "readings": None} or body["fix"]["amount"] == "100.5" + assert body["fix"] == {"latitude": 1.5, "amount": "100.5"} assert [f["amount"] for f in body["fixes"]] == ["2.5", None] assert float_client.get("/api/samples/1").json()["fix"]["amount"] == "100.5" for bad in ('"0.12345"', '"abc"', '"123456789.5"'): @@ -292,6 +301,33 @@ def test_a_decimal_inside_a_jsonb_value_object_follows_the_top_level_rule(float_ assert len(float_client.get("/api/samples").json()) == 1 +def test_a_decimal_in_a_value_object_map_applies_to_each_value_and_keys_are_never_members(float_client: TestClient) -> None: + # C1: a field.map of a value object applies the nested rule to EACH map value; a key named + # like a member ("amount") is just a key. Top-level map and a map nested in a value object. + hdr = {"content-type": "application/json"} + ok = float_client.post( + "/api/samples", + content='{"tiers": {"amount": {"amount": "1.5000"}, "gold": {"amount": 2.50}},' + ' "pricing": {"tiers": {"amount": {"amount": 100}, "silver": {"amount": "0.25"}}}}', + headers=hdr, + ) + assert ok.status_code == 201, ok.text + expected_tiers = {"amount": {"amount": "1.5"}, "gold": {"amount": "2.5"}} + expected_pricing = {"tiers": {"amount": {"amount": "100"}, "silver": {"amount": "0.25"}}} + body = ok.json() + assert body["tiers"] == expected_tiers and body["pricing"] == expected_pricing + got = float_client.get("/api/samples/1").json() + assert got["tiers"] == expected_tiers and got["pricing"] == expected_pricing + for bad in ( + '{"tiers": {"gold": {"amount": "0.12345"}}}', + '{"pricing": {"tiers": {"amount": {"amount": "0.12345"}}}}', + '{"tiers": {"amount": {"amount": "abc"}}}', + ): + resp = float_client.post("/api/samples", content=bad, headers=hdr) + assert resp.status_code == 400 and resp.json() == {"error": "validation"}, (bad, resp.text) + assert len(float_client.get("/api/samples").json()) == 1 + + def test_a_decimal_array_is_range_checked_element_wise_and_coerced_for_the_driver(float_client: TestClient) -> None: # SQLite has no array column, so the write itself is covered on Postgres by the shared # api-contract decimal corpus; here: the route refuses an excess-scale element (nothing diff --git a/server/python/tests/runtime/test_decimal_wire.py b/server/python/tests/runtime/test_decimal_wire.py index f7cfcd1b9..23a8a23fe 100644 --- a/server/python/tests/runtime/test_decimal_wire.py +++ b/server/python/tests/runtime/test_decimal_wire.py @@ -157,3 +157,31 @@ def test_a_non_decimal_number_in_a_body_is_a_float_not_a_decimal() -> None: assert coerced["weight"] == 2.5 and isinstance(coerced["weight"], float) assert coerced["telemetry"] == {"latitude": 51.5, "tags": [1.25, 3]} json.dumps(coerced["telemetry"]) + + +def test_a_map_of_value_objects_applies_the_nested_rule_to_each_value_and_never_matches_keys() -> None: + # Pricing { tiers: field.map @objectRef Tier }, Tier { amount: decimal(12, 4) }. A map key + # named "amount" is a key, not the member: its value is a Tier. + tier = {"d": {"amount": (12, 4)}, "o": {}} + nested = { + "tiers": {"m": tier}, + "pricing": {"d": {}, "o": {"tiers": {"m": tier}}}, + } + body = parse_json_object( + '{"tiers": {"amount": {"amount": "1.5000"}, "gold": {"amount": 2.50, "w": 1.5}},' + ' "pricing": {"tiers": {"amount": {"amount": 100}}}}' + ) + out = to_wire(coerce_decimal_fields(body, frozenset(), None, nested)) + assert out == { + "tiers": {"amount": {"amount": "1.5"}, "gold": {"amount": "2.5", "w": 1.5}}, + "pricing": {"tiers": {"amount": {"amount": "100"}}}, + } + for bad in ( + '{"tiers": {"gold": {"amount": "0.12345"}}}', + '{"pricing": {"tiers": {"amount": {"amount": "0.12345"}}}}', + ): + with pytest.raises(ValueError, match="tiers"): + coerce_decimal_fields(parse_json_object(bad), frozenset(), None, nested) + # a null map, and a map value that is not an object, fall through + assert coerce_decimal_fields({"tiers": None}, frozenset(), None, nested) == {"tiers": None} + assert coerce_decimal_fields({"tiers": {"a": 1.5}}, frozenset(), None, nested) == {"tiers": {"a": 1.5}} diff --git a/server/typescript/packages/integration-tests/src/api-contract-jsonb-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-jsonb-generated-server.ts index 9ef06aed5..502f122da 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-jsonb-generated-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-jsonb-generated-server.ts @@ -89,7 +89,9 @@ export const db = drizzle(pool); "optional_marker" jsonb, "markers" jsonb, "measure" jsonb, - "measures" jsonb + "measures" jsonb, + "tiers" jsonb, + "pricing" jsonb )`, ); diff --git a/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts b/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts index ed67a0edf..c5a013a9a 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-jsonb-server.ts @@ -44,6 +44,8 @@ export interface DocumentRow { markers?: Marker[] | null; measure?: Measure | null; measures?: Measure[] | null; + tiers?: Record | null; + pricing?: { tiers?: Record | null } | null; } export interface JsonbServerHandle { @@ -79,6 +81,8 @@ export async function startJsonbServer(connectionUri: string, root: MetaRoot): P .addColumn("markers", "jsonb") .addColumn("measure", "jsonb") .addColumn("measures", "jsonb") + .addColumn("tiers", "jsonb") + .addColumn("pricing", "jsonb") .execute(); const driver = kyselyDriver({ db: kysely as never, dialect: "postgres" }); diff --git a/server/typescript/packages/runtime-ts/src/type-coercer.ts b/server/typescript/packages/runtime-ts/src/type-coercer.ts index cee6b6876..12fcc1781 100644 --- a/server/typescript/packages/runtime-ts/src/type-coercer.ts +++ b/server/typescript/packages/runtime-ts/src/type-coercer.ts @@ -23,6 +23,7 @@ import { TYPE_FIELD, FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_DECIMAL, + FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_TIMESTAMP, @@ -294,7 +295,8 @@ function parseJsonTextArrays(entity: MetaData, row: Row): Row { /** The `object.value` a jsonb `field.object` references, or undefined (not a value object, or unresolved). */ function valueObjectOf(field: MetaData): MetaData | undefined { - if (field.type !== TYPE_FIELD || field.subType !== FIELD_SUBTYPE_OBJECT) return undefined; + // A field.map of value objects binds @objectRef the same way (its values are the value objects). + if (field.type !== TYPE_FIELD || (field.subType !== FIELD_SUBTYPE_OBJECT && field.subType !== FIELD_SUBTYPE_MAP)) return undefined; // ADR-0059 R7: the object the loader bound @objectRef to. const target = refTarget(field, FIELD_ATTR_OBJECT_REF); return target?.subType === OBJECT_SUBTYPE_VALUE ? target : undefined; @@ -327,7 +329,18 @@ function mapDecimals( if (vo === undefined) continue; const one = (el: unknown): unknown => el !== null && typeof el === "object" && !Array.isArray(el) ? mapDecimals(vo, el as Row, convert) : el; - if (Array.isArray(v)) { + if (child.subType === FIELD_SUBTYPE_MAP) { + // The rule applies to EACH map value; keys are arbitrary text, never compared with the + // value object's member names (a key named `amount` is just a key). + if (typeof v === "object" && !Array.isArray(v)) { + let mapped: Row | null = null; + for (const [key, el] of Object.entries(v as Row)) { + const m = one(el); + if (m !== el) (mapped ??= { ...(v as Row) })[key] = m; + } + if (mapped !== null) next = mapped; + } + } else if (Array.isArray(v)) { const mapped = v.map(one); if (mapped.some((el, i) => el !== v[i])) next = mapped; } else { diff --git a/server/typescript/packages/runtime-ts/src/validator-runner.ts b/server/typescript/packages/runtime-ts/src/validator-runner.ts index 5b55b5f7c..84d149fa1 100644 --- a/server/typescript/packages/runtime-ts/src/validator-runner.ts +++ b/server/typescript/packages/runtime-ts/src/validator-runner.ts @@ -7,7 +7,7 @@ import { VALIDATOR_SUBTYPE_REQUIRED, VALIDATOR_SUBTYPE_LENGTH, VALIDATOR_SUBTYPE_REGEX, FIELD_SUBTYPE_STRING, FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, FIELD_SUBTYPE_CURRENCY, - FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_DECIMAL, + FIELD_SUBTYPE_BOOLEAN, FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_OBJECT, FIELD_SUBTYPE_MAP, FIELD_SUBTYPE_DECIMAL, FIELD_ATTR_PRECISION, FIELD_ATTR_SCALE, FIELD_ATTR_REQUIRED, FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_DEFAULT, FIELD_ATTR_DB_COLUMN_TYPE, DB_COLUMN_TYPE_JSONB, FIELD_ATTR_OBJECT_REF, @@ -152,6 +152,38 @@ export function runValidators( continue; } + // A map of value objects (`field.map @objectRef`): every map value is a VO validated in full, + // as an array element is. The keys are arbitrary text and are never compared with the VO's + // member names. A map whose value is a scalar (@valueType) is unchanged. + if (field.subType === FIELD_SUBTYPE_MAP) { + const vo = resolveVoRef(field); + if (vo !== undefined) { + if (typeof value !== "object" || Array.isArray(value)) { + errors.push({ + field: field.name, rule: "type", + message: `'${field.name}' must be a map of ${vo.name}`, + expected: "object", received: Array.isArray(value) ? "array" : typeof value, + }); + continue; + } + for (const [key, el] of Object.entries(value as Record)) { + if (typeof el !== "object" || el === null || Array.isArray(el)) { + errors.push({ + field: `${field.name}.${key}`, rule: "type", + message: `'${field.name}.${key}' must be a ${vo.name} object`, + expected: vo.name, received: el === null ? "null" : Array.isArray(el) ? "array" : typeof el, + }); + continue; + } + const sub = runValidators(vo, el as Record); + if (!sub.ok) { + for (const e of sub.errors) errors.push({ ...e, field: `${field.name}.${key}.${e.field}` }); + } + } + continue; + } + } + // A scalar array (`field.string isArray`, …) — stored as a native array or a JSON array. // Each element is checked against the element rules; an element error names its index, // as the value-object branch above does. ADR-0039: resolving `isArray`. diff --git a/server/typescript/packages/runtime-ts/test/type-coercer.test.ts b/server/typescript/packages/runtime-ts/test/type-coercer.test.ts index 0b8de1edb..8e7f6860d 100644 --- a/server/typescript/packages/runtime-ts/test/type-coercer.test.ts +++ b/server/typescript/packages/runtime-ts/test/type-coercer.test.ts @@ -193,3 +193,64 @@ describe("runValidators on a decimal (ADR-0019 Amendment 1)", () => { } }); }); + +describe("a field.decimal inside a value object held by a field.map (ADR-0019 Amendment 1, C1)", () => { + async function doc(): Promise { + const model = { "metadata.root": { package: "ledger", children: [ + { "object.value": { name: "Tier", children: [ + { "field.decimal": { name: "amount", "@precision": 9, "@scale": 4 } }, + ]}}, + { "object.value": { name: "Pricing", children: [ + { "field.map": { name: "tiers", "@objectRef": "Tier" } }, + ]}}, + { "object.entity": { name: "Doc", children: [ + { "field.long": { name: "id" } }, + { "field.map": { name: "tiers", "@objectRef": "Tier" } }, + { "field.object": { name: "pricing", "@objectRef": "Pricing", "@storage": "jsonb" } }, + { "field.map": { name: "labels", "@valueType": "string" } }, + { "identity.primary": { name: "pk", "@fields": "id" } }, + ]}}, + ]}}; + const { root, errors } = await new MetaDataLoader({ strict: true }).load([ + new InMemoryStringSource(JSON.stringify(model), { id: "ledger.json" }), + ]); + expect(errors).toEqual([]); + return root.objects().find((o) => o.name === "Doc") as MetaData; + } + + test("write: each map value is bound canonical, top-level and nested; a key named amount is only a key", async () => { + const e = await doc(); + const out = coerceRowOnWrite(e, { + id: 1, + tiers: { amount: { amount: "1.5000" }, gold: { amount: 2.5 } }, + pricing: { tiers: { amount: { amount: 100 }, silver: { amount: "0.25" } } }, + labels: { amount: "1.5000" }, + }, "postgres"); + expect(out.tiers).toEqual({ amount: { amount: "1.5" }, gold: { amount: "2.5" } }); + expect(JSON.parse(out.pricing as string)).toEqual({ tiers: { amount: { amount: "100" }, silver: { amount: "0.25" } } }); + expect(out.labels).toEqual({ amount: "1.5000" }); + }); + + test("write: excess scale in any map value is refused", async () => { + const e = await doc(); + expect(() => coerceRowOnWrite(e, { id: 1, tiers: { gold: { amount: "0.12345" } } }, "postgres")).toThrow(/field\.decimal 'amount' cannot be written/); + expect(() => coerceRowOnWrite(e, { id: 1, pricing: { tiers: { amount: { amount: "0.12345" } } } }, "postgres")).toThrow(/field\.decimal 'amount' cannot be written/); + }); + + test("read: stored uncanonical map values are answered canonical", async () => { + const e = await doc(); + const out = coerceRowOnRead(e, { id: 1, tiers: { amount: { amount: "1.5000" } }, pricing: { tiers: { x: { amount: 2.5 } } } }, "postgres"); + expect(out.tiers).toEqual({ amount: { amount: "1.5" } }); + expect(out.pricing).toEqual({ tiers: { x: { amount: "2.5" } } }); + }); + + test("runValidators: a map value with an excess scale fails, keyed by the map key", async () => { + const { runValidators } = await import("../src/validator-runner.js"); + const e = await doc(); + expect(runValidators(e, { id: 1, tiers: { amount: { amount: "1.5" } } }).ok).toBe(true); + const bad = runValidators(e, { id: 1, tiers: { amount: { amount: "0.12345" } } }); + expect(bad.ok).toBe(false); + expect(runValidators(e, { id: 1, pricing: { tiers: { amount: { amount: "0.12345" } } } }).ok).toBe(false); + expect(runValidators(e, { id: 1, labels: { amount: "0.12345" } }).ok).toBe(true); + }); +});