feat(rest): add the Ossie REST API spec by jbonofre · Pull Request #573 · apache/ossie

jbonofre · 2026-10-12T16:13:34Z

Summary

Follows the [PROPOSAL] Introduce Ossie REST API spec thread on dev@. Adds rest/openapi.yaml, a draft standard REST API to produce, consume and orchestrate Ossie models, meant to be implemented by catalogs (Polaris, Unity, Collibra, ...) and consumed by query engines, BI tools and converter orchestrators.

Media types: application/vnd.ossie+yaml / application/vnd.ossie+json, with an optional version="0.2" parameter as a negotiation hint (suggested by Sebastien Gandon). The document's version field stays authoritative.
Models: GET/POST /v1/models, GET/PUT/DELETE /v1/models/{name}, with ETag/If-Match and page-token pagination. The document schema references core-spec/ossie-schema.json.
Converters: GET /v1/converters, POST /v1/converters/{converter}/import and /export.
Validation / discovery: POST /v1/validate, GET /v1/config.
Errors, auth, extensions: RFC 9457 problem+json, bearer auth, vendor extensions under /v1/x-{vendor}/.

Open points

Alignment with the Polaris Ossie REST API design doc (models are in a flat namespace here, no catalog/namespace hierarchy).
Conversion warnings: warnings field on import, X-Ossie-Warning headers on export.
Docs (rest/README.md) and a CI lint step for the spec are not included yet.

Test plan

openapi.yaml validates as OpenAPI 3.1 with openapi-spec-validator, including the core-spec/ossie-schema.json reference.

Define a standard REST API to produce, consume and orchestrate Ossie models: model CRUD with ETag concurrency, validation, and converter import/export. Documents are exchanged as application/vnd.ossie+yaml or application/vnd.ossie+json with an optional spec-line version parameter.

ayushtkn

Thanx @jbonofre, dropped few suggestions, rest LGTM

ayushtkn · 2026-10-12T17:22:40Z

+        "200":
+          description: The converted model.
+          content:
+            application/vnd.ossie+yaml: { schema: { $ref: "#/components/schemas/ConversionResult" } }


/import says the result is an Ossie document, then types vnd.ossie+yaml and vnd.ossie+json as ConversionResult ({model, warnings}). Those media types are defined at the top of this file as a core-spec document, and that schema's root has additionalProperties: false, so a client validating vnd.ossie against ossie-schema.json rejects the wrapper outright:

'version' is a required property 'name' is a required property 'datasets' is a required property Additional properties are not allowed ('model', 'warnings' were unexpected)

Return the bare document as vnd.ossie with warnings in headers, as export does, or keep ConversionResult on application/json only.

ayushtkn · 2026-10-12T17:23:22Z

+        Either send an Ossie document as the body, or send `{"model": "<name>"}`
+        as `application/json` to export a stored model. Conversion warnings
+        (constructs the target cannot express) are returned in the
+        `Ossie-Warnings` header as a count and in `X-Ossie-Warning` headers.


The description cites Ossie-Warnings as a count and repeated X-Ossie-Warning headers; the PR text mentions only X-Ossie-Warning. The 200 response at :278 declares neither, so generated clients will drop them. Declare one header shape in the response object — repeated values are awkward to express in OpenAPI, so a single X-Ossie-Warning holding a list, or a JSON body, is easier to implement the same way twice. X- prefixes are also deprecated by RFC 6648.

ayushtkn · 2026-10-12T17:30:34Z

+      schema: { type: string }
+
+  parameters:
+    ModelName:


The path segment and the document's required name can diverge, and nothing says what happens: PUT /v1/models/orders carrying name: customers could rename, 400, or 409. The parameter has no description, and Conflict at line 427 reads "A model with this name already exists" without saying which name it means.

A mismatch needs a declared outcome — 400 on create and replace — so that two catalogs do not pick differently.

ayushtkn · 2026-10-12T17:31:27Z

+        nothing is stored unless `store` is true.
+      parameters:
+        - $ref: "#/components/parameters/ConverterName"
+        - name: store


store=true is a second operation with no contract. Import declares ['200','401','404','415','422'] — no 201, no Location, no ETag, and no 409 when that name already exists.
A caller cannot distinguish "converted" from "stored", or "stored" from "stored over an existing model".

ayushtkn · 2026-10-12T17:32:29Z

+      description: A semantic model, as defined by the Ossie core specification.
+      $ref: "../core-spec/ossie-schema.json"
+
+    ModelSummary:


Two of the three mechanisms you offered Jakub on the thread are not here. You wrote "1. Explicit model versioning (e.g. version_id, updated_at, content hash) … 3. Lightweight HEAD / summary metadata on endpoints so catalogs can poll cheaply without fetching the full model." Conditional requests (2) are implemented — If-None-Match/304 and If-Match/412 — but there is no HEAD on any path, and ModelSummary is [name, description, version, etag] with no updated_at or version_id. version here is documented as the Ossie spec version, not a model revision, which is the confusion a polling catalog will hit first.

ayushtkn · 2026-10-12T17:34:21Z

+  responses:
+    BadRequest: { description: "Malformed request.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }
+    Unauthorized: { description: "Missing or invalid credentials.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }
+    NotFound: { description: "Resource not found.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }


404 is declared on both /import and /export, where it can mean an unknown converter or, for {"model": "<name>"} on export, an unknown model. Those need to be distinguishable by a client — either separate responses, or a machine-readable field naming the missing resource.

ayushtkn · 2026-10-12T17:36:49Z

+          type: array
+          items: { type: string }
+          examples: [["application/vnd.ossie+yaml", "application/vnd.ossie+json"]]
+        features:


A closed enum of models | validate | conversions means a server that adds a feature returns a value this schema forbids. Many generated clients reject the whole payload on an unknown enum value rather than the one field, so an older client breaks against a newer server. For a spec meant to be implemented by several catalogs independently, this should be an open string.

ayushtkn · 2026-10-12T17:37:55Z

+    BadRequest: { description: "Malformed request.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }
+    Unauthorized: { description: "Missing or invalid credentials.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }
+    NotFound: { description: "Resource not found.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }
+    NotAcceptable: { description: "No representation matches Accept, including the version parameter.", content: { application/problem+json: { schema: { $ref: "#/components/schemas/Error" } } } }


406 is declared only on GET /v1/models/{name}. Create and replace have no response for a body whose version names a spec line the server does not implement — today it falls through to a generic 422, which a client cannot distinguish from a malformed model. They need a 406 of their own, with ServerConfig.spec_versions as the advertised set.

ayushtkn · 2026-10-12T17:38:35Z

+  schemas:
+    OssieDocument:
+      description: A semantic model, as defined by the Ossie core specification.
+      $ref: "../core-spec/ossie-schema.json"


This resolves only when the file is read beside core-spec/. Served alone — the normal case for a catalog publishing its API — it cannot be resolved at all, so Swagger UI and codegen from a URL fail. The reference needs a form a consumer holding only this document can resolve: an absolute $id-based reference or a bundled copy. Note the trade-off either way — a relative ref tracks the repo revision, whereas the raw .../main/... URL that ontology/ontology.json uses floats with main.

ayushtkn · 2026-10-12T17:39:25Z

+          type: array
+          items: { type: string, enum: [models, validate, conversions] }
+
+    ValidationResult:


{valid, errors} has nowhere for warnings, but validation/validate.py:114 defines ValidationWarning and reports them separately from errors. A server wrapping the reference validator must either discard them or misreport them, and a document with warnings only would come back valid: true, errors: [].

dimas-b

Posting some comments from the potential implementor's POV (Polaris). I hope you find them reasonable.

dimas-b · 2026-10-12T23:12:35Z

+    ## Extensibility
+
+    Servers MAY add vendor-specific response fields and operations under
+    `/v1/x-{vendor}/`. Clients MUST ignore response fields they do not know.


Since it's a custom path, does it have to be specified at Ossie? Any custom path should work fine as long as it does not overlap with Ossie paths.

Also v1 is probably not relevant to custom paths as they will have to follow their own versioning.

indeed, why v1 ?

That's fair as soon as it's "delegated" to the vendor. I will update accordingly.

dimas-b · 2026-10-12T23:18:00Z

+        - $ref: "#/components/parameters/PageToken"
+        - name: name
+          in: query
+          description: Return only models whose name contains this text.


If this is meant to be a query rather than exact name, would name-contains be clearer?

Otherwise, GET /v1/models?name=foo returning foobar looks strange 🤔

Good point, let me clarify this one.

dimas-b · 2026-10-12T23:18:44Z

+            application/json:
+              schema:
+                $ref: "#/components/schemas/ListModelsResponse"
+        "401": { $ref: "#/components/responses/Unauthorized" }


Should /config have a 401 response too?

What about 403?

dimas-b · 2026-10-12T23:22:55Z

+            Location: { schema: { type: string, format: uri-reference } }
+          content:
+            application/vnd.ossie+yaml: { schema: { $ref: "#/components/schemas/OssieDocument" } }
+            application/vnd.ossie+json: { schema: { $ref: "#/components/schemas/OssieDocument" } }


Would it make sense to add a note that clients SHOULD provide the Accept header?

dimas-b · 2026-10-12T23:37:07Z

+    * `application/vnd.ossie+json`
+
+    Both accept an optional `version` parameter naming a spec line, for example
+    `application/vnd.ossie+yaml; version="0.2"` meaning any `0.2.x` document. The


Should version be a query parameter perhaps? It looks like it's a hard expectation of the client rather than a content "negotiation" option 🤔 A server may also only support one particular format version... Is real negotiation possible?

dimas-b · 2026-10-12T23:45:00Z

+    get:
+      tags: [Conversions]
+      operationId: listConverters
+      summary: List the converters registered on this server


Would it make sense to define pagination from the start? It is usually very hard to add later (due to old clients).

dimas-b · 2026-10-12T23:47:57Z

+      description: ETag the client last saw.
+      schema: { type: string }
+    PageSize:
+      name: page_size


WDYT about stating that page size is only a hint to the server and servers can return less entries than page_size?.. e.g. total content size may be a factor.

dimas-b · 2026-10-12T23:54:45Z

+        media_types:
+          type: array
+          items: { type: string }
+          examples: [["application/vnd.ossie+yaml", "application/vnd.ossie+json"]]


In the top-level description, version is part of the media type, so having it separate here contradicts that. Note: from my POV, handling spec version separately from media types may be preferable.

dimas-b · 2026-10-13T00:05:58Z

+    description: Server capabilities.
+
+paths:
+  /v1/config:


In my experience it is valuable to make the config endpoint stable as the API evolves. This way clients can discover the highest API version supported by the server via a well-known config URL.

In other words, WDYT about /config (without v1)?

If newer API versions return different configuration JSON shapes, they can add /v2/config for the extra information.

dimas-b · 2026-10-13T00:15:57Z

+      type: object
+      required: [spec_versions, media_types]
+      properties:
+        spec_versions:


It could be valuable to also list API versions (of this API) supported by the server.

It may be that a v2 is required sooner than all clients migrate, so server may want to support multiple APIs.

Checking /vN/config may not be conclusive because a 404 response may be due to a client configuration mistake (bad base URL) rather than lack of server capabilities. Conversely, a 200 response may be due to hitting an config endpoint of another API (again due to badly configured URI).

Additionally, the version info may include data about customizations (extra paths) supported by the server.

christianeu-db · 2026-10-13T04:18:53Z

+        valid: { type: boolean }
+        errors:
+          type: array
+          items:
+            type: object
+            required: [path, message]
+            properties:
+              path: { type: string, description: JSON Pointer into the document. }
+              message: { type: string }


With custom dialects / extensions, I think we'll see a fair number of documents that pass ossie validate but that a client still can't fully parse unless it knows the vendor extension. #576 hints at this even at the smallest scale (a single-SUM metric comes back in a vendor-only dialect from several converters) so "valid" and "readable by the next tool" aren't quite the same thing yet.

Building on the warnings idea above, I wonder if the response could also carry information about the parts an implementation didn't validate or doesn't recognize. A caller compiling queries could use that information to understand how likely they are to succeed, rather than discovering it the hard way. Validating specific vendor extensions could stay optional and up to each implementer.

@ekut, you've been doing a lot of the query-experience thinking here, so I'd really value your opinion on what kind of metadata would be useful to you when dealing with models that have lots of vendor dialects / extensions

Thanks for the ping. Two things, both as warnings in the shape errors already has, plus a code:

{"path": "/metrics/0/expression", "code": "no_core_dialect", "message": "only THOUGHTSPOT"} {"path": "/metrics/0/expression/dialects/0", "code": "not_checked", "message": "no parser for THOUGHTSPOT"}

no_core_dialect: a field or metric with no ANSI_SQL or OSSIE_SQL_2026 entry. A client can work this out itself, but it is the first line a compiler draws (ossie-clickhouse refuses such a document), and a shared code puts that line in the same place for every tool.

not_checked: a dialect entry, or a custom_extensions vendor, the server did not validate. Only the server knows this. On current main, validate.py passes a metric whose only entry is THOUGHTSPOT: "this is ((( not a formula" with no warning, since THOUGHTSPOT and six other dialects are in SKIP_SQL_VALIDATION; the same text tagged ANSI_SQL fails. Today a caller cannot tell "checked and fine" from "not looked at".

@ekut, that shape covers what I had in mind. @jbonofre, would a warnings array next to errors (same path and message, plus a code) work for ValidationResult, with ConversionResult.warnings using the same shape instead of plain strings? That would also answer @ayushtkn's point, and here or in a follow-up is your call.

flyrain · 2026-10-23T22:45:52Z

+  license:
+    name: Apache 2.0
+    url: https://www.apache.org/licenses/LICENSE-2.0.html


These seem redundant. Can we remove them?

dimas-b · 2026-10-27T21:23:46Z

+  schemas:
+    OssieDocument:
+      description: A semantic model, as defined by the Ossie core specification.
+      $ref: "../core-spec/ossie-schema.json"


core-spec/ossie-schema.json currently defines source as a dot-delimited string, IIRC.

This may be a problem in catalogs (like Polaris) that permit dots inside table names.

@jbonofre @ayushtkn: Do you think this is worth a dev discussion?

Agreed with @ayushtkn , referring like ../core-spec/ossie-schema.json is problematic in openAPI.

Moreover, I think the Ossie core spec schema should be versioned.

+1 for a dev@ thread.

The gap is wider than the dots: source is only {"type": "string", "minLength": 1} with no
pattern and no delimiter rule. The dotted form lives in prose and examples, not in the schema,
and nothing in the repo parses it. So a.b.c could be a three-part name or a two-part one whose
table is b.c, with no escaping rule to tell them apart. Iceberg REST sidesteps this by
delimiting namespaces with %1F rather than a dot, so any catalog on that model has to guess
when it reads an Ossie source.

One thing that may let the two proceed in parallel: OssieDocument is a bare $ref and doesn't
restate source, so whatever dev@ settles should land here without needing a change to this
spec. #298, #173, #338 and #104 are all open on this, and #637 was closed on 25 Oct after
reviewers asked to take source portability to dev@

The reason I did like this is because we don't have a release yet (so nothing publicly available as an URL).
I mimic what we have in the validate.py but I forgot this one (I did for ontology).

Let me use raw.githubusercontent.com/apache/ossie/main/... as for the other.

I will start a discussion about source on the dev mailing list.

Yeah, that's exactly the Polaris case: apache/polaris#5685

Iceberg table names can have a . in them, but Dataset.source is still just a plain string. The dotted form is only mentioned in the description, with no quoting. That PR tried to guess the split from catalog state, which isn't a great long-term model, so it's on hold for now...

A typical Hive-style sales.public.orders still binds fine, with the last segment as the table. The clash only shows up when the table name itself has a dot. For example ns=a table=b.c vs ns=a.b table=c both end up as the same string a.b.c.

cc: @jbonofre @dimas-b

Thank you for leading the discussion, @jbonofre !

IMHO, since source is defined by Ossie, parsing rules should also be defined by Ossie to ensure interoperability. An Ossie document should be parseable the same way in all catalog implementations, I think.

The relative ../core-spec/ossie-schema.json reference only resolves when the spec is read beside core-spec/, so Swagger UI and codegen fail on a published copy. Point OssieDocument at the raw GitHub URL, as the ontology schemas already do.

christianeu-db

Thanks @jbonofre! Left one note on the schema reference and a reply on the validation thread.

christianeu-db · 2026-11-10T21:01:52Z

+          type: array
+          items: { type: string }
+          description: Ossie spec versions the server can read and write.
+          examples: [["0.2.0.dev0"]]


Since #691 the schema on main requires version: 0.3.0.dev0, so a body that uses the spec_versions example (0.2.0.dev0) now fails OssieDocument with [Schema] version: '0.3.0.dev0' was expected. The version="0.2" media-type example at the top still says 0.2 too. Could both move to 0.3?

christianeu-db · 2026-11-10T21:02:21Z

+        valid: { type: boolean }
+        errors:
+          type: array
+          items:
+            type: object
+            required: [path, message]
+            properties:
+              path: { type: string, description: JSON Pointer into the document. }
+              message: { type: string }


@ekut, that shape covers what I had in mind. @jbonofre, would a warnings array next to errors (same path and message, plus a code) work for ValidationResult, with ConversionResult.warnings using the same shape instead of plain strings? That would also answer @ayushtkn's point, and here or in a follow-up is your call.

ayushtkn reviewed Oct 12, 2026

View reviewed changes

dimas-b reviewed Oct 12, 2026

View reviewed changes

dimas-b reviewed Oct 13, 2026

View reviewed changes

christianeu-db reviewed Oct 13, 2026

View reviewed changes

ekut mentioned this pull request Oct 14, 2026

Add measurements for model portability across converters apache/ossie-converters#24

Open

flyrain reviewed Oct 23, 2026

View reviewed changes

dimas-b reviewed Oct 27, 2026

View reviewed changes

ArnabKarmakar1108 mentioned this pull request Oct 28, 2026

Validate semantic-model documents against the Apache Ossie JSON Schema apache/polaris#5895

Draft

6 tasks done

vigneshio mentioned this pull request Oct 29, 2026

fix(semantic-models): resolve dotted dataset.source unambiguously apache/polaris#5685

Open

jbonofre mentioned this pull request Nov 3, 2026

Guidance on integrating Polaris-hosted Ossie models with Apache Superse apache/polaris#5968

Open

mfournioux mentioned this pull request Nov 4, 2026

Feature request: Will Cube Core support the Apache Ossie REST API as a client (e.g. to load models from Polaris)? cube-js/cube#12610

Open

christianeu-db reviewed Nov 10, 2026

View reviewed changes

benhuds mentioned this pull request Nov 11, 2026

Semantic models (Apache Ossie) as a registry kind atlanai/ai-registry#231

Open

1 task done

Conversation

jbonofre commented Oct 12, 2026

Summary

Open points

Test plan

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

dimas-b Oct 13, 2026 •

edited

Loading

dimas-b Oct 29, 2026 •

edited

Loading