test(compliance): add portability rungs M7 (window ORDER BY) and M8 (ratio-to-total)

compliance/portability/README.md

@@ -17,13 +17,14 @@
   under the License.
 -->
 
-# Portability measurement ladder (M0-M3)
+# Portability measurement ladder (M0-M8)
 
 A small set of portable fixtures, and a grader, for measuring how faithfully a
-model survives a round-trip through each converter. This is issue #576
-(measurements for model portability across converters). The fixtures are the
-inputs; `grader/` scores each returned document against its input, element by
-element, into the outcome vocabulary in `manifest.yaml`. A thin runner that drives
+model survives a round-trip through each converter. This is
+apache/ossie-converters#24 (measurements for model portability across
+converters; formerly #576 here). The fixtures are the inputs; `grader/` scores
+each returned document against its input, element by element, into the outcome
+vocabulary in `manifest.yaml`. A thin runner that drives
 the converters (modelled on `cube/tools/interop_matrix.py` in apache/ossie-converters)
 turns the whole thing into one command; the grader is written to sit behind it.
 
@@ -37,7 +38,12 @@ difference in a converter's output localizes to a single construct:
 | M0 | baseline | one dataset, one field, one `SUM` metric |
 | M1 | a scalar field, no aggregation | `amount * 1.1` calculated column |
 | M2 | the core aggregate set | `SUM`, `COUNT`, `COUNT(DISTINCT)`, `AVG`, `MIN`, `MAX` |
-| M3 | a filtered aggregate | `SUM(CASE WHEN ... THEN x ELSE 0 END)` |
+| M3 | a filtered aggregate | `SUM(CASE WHEN ... THEN x END)` (no `ELSE`, so it equals the `FILTER` form), and the same with `FILTER (WHERE ...)` |
+| M4 | a cross-dataset metric | two datasets, a relationship, a model-scoped metric across the join |
+| M5 | metric composition | a model-scoped metric built from two others, inlined (a metric expression cannot refer to other metrics) |
+| M6 | date and time scalars | `YEAR`, `DATE_TRUNC`, `DATEDIFF(part, start, end)` |
+| M7 | window functions with an `ORDER BY` | `LAG`, a running `SUM` with a `ROWS` frame, `RANK() OVER (ORDER BY ... DESC)` on the default NULL placement (in `OSSIE_SQL_2026`, the dialect that defines it) |
+| M8 | ratio-to-total | `x / SUM(x) OVER ()` and `x / SUM(x) OVER (PARTITION BY ...)` |
 
 `manifest.yaml` lists the fixtures, the element each rung exercises, and the
 per-element outcome vocabulary the grader assigns (PORTABLE / TAG_PRESERVED,
@@ -61,10 +67,6 @@ grades the real fixtures all-PORTABLE. See `grader/README.md`.
 
 ## Scope and placement
 
-Rungs M4-M8 (relationships and cross-dataset metrics, metric-to-metric
-composition, date and time scalars, window and offset functions) are held
-pending #354, #591 and #587, and are not in this set.
-
-Location is a starting point. If these should sit with the interoperability
-harness from #348 or elsewhere once that lands, they can move; the fixtures are
-self-contained and the manifest paths are relative.
+The fixtures measure the specification rather than any one converter, so they
+live in this repository; the fixtures are self-contained and the manifest paths
+are relative.

compliance/portability/grader/README.md

@@ -19,16 +19,21 @@
 
 # Portability fidelity grader
 
-`grade.py` is the scoring half of issue #576. Conformance (`validate.py` passing)
-is a floor and says nothing about whether a model survived a converter round-trip;
-this grader is the fidelity half.
+`grade.py` is the scoring half of apache/ossie-converters#24 (formerly #576).
+Conformance (`validate.py` passing) is a floor and says nothing about whether a
+model survived a converter round-trip; this grader is the fidelity half.
 
 It does not drive the converters. It takes the input document and the document a
 converter returned, and diffs them element by element into the outcome vocabulary
 in `../manifest.yaml`. A thin runner (modelled on `cube/tools/interop_matrix.py` in
 apache/ossie-converters) drives each converter and feeds the
 grader, so the measurement table becomes one command.
 
+Elements are metrics and field expressions. A model-scoped metric is keyed by its
+name, a dataset-scoped metric (`datasets[].metrics`) and a field by `dataset.name`,
+the way the spec references them. A metric a converter moves between the two scopes
+with the same expression is graded REWRITTEN (identity not preserved), not PORTABLE.
+
 ## Usage
 
 ```
@@ -49,10 +54,23 @@ PLACEHOLDER, ADDED, RELABELED, DROPPED_DIAGNOSED, DROPPED_SILENT, REJECTED.
   marked cosmetic, while one it cannot prove equivalent is flagged for review.
   Without `sqlglot` the grader still runs and marks every rewrite as needing
   review.
+- **A qualifier is checked against the model.** Before trees are compared, each
+  column is resolved to the dataset it names: a bare name in a field expression
+  or dataset-scoped metric is the element's own dataset, elsewhere it is the one
+  dataset that declares it. A requalify that names the same dataset is cosmetic;
+  one that names another dataset (`customers.amount` for `sales.amount`) needs
+  review.
 - **Textual by default, semantic where it can be.** PORTABLE means the converter
   returned the same expression it was given (modulo cosmetic whitespace). True
   semantic equivalence of SQL is undecidable in general, so the grader auto-clears
   only the changes a normalized tree proves equal and leaves the rest for a human.
+- **NULL placement in a window `ORDER BY` follows the authored tag.** sqlglot's
+  base dialect sorts NULL as the smallest value, the opposite of the
+  `OSSIE_SQL_2026` default (#587). An `OSSIE_SQL_2026` expression is compared
+  with NULL as the largest value, so making the default explicit (`DESC NULLS
+  FIRST`) is cosmetic and flipping it (`DESC NULLS LAST`) needs review. An
+  `ANSI_SQL` expression leaves the default to the engine, so an omitted and an
+  explicit placement are only cosmetic when they agree under both orderings.
 - **Identity beyond name.** A dropped input element and an added output element
   that carry an equivalent expression are paired as one element whose identity
   (name) was not preserved, rather than scored as a drop plus an unrelated add.
@@ -62,5 +80,7 @@ PLACEHOLDER, ADDED, RELABELED, DROPPED_DIAGNOSED, DROPPED_SILENT, REJECTED.
 ## Tests
 
 `test_grade.py` has one test per outcome plus an identity check that runs the real
-M0-M3 fixtures through a no-op converter and asserts every element grades
-PORTABLE. Run with `PYTHONPATH=. pytest test_grade.py`.
+M0-M8 fixtures through a no-op converter and asserts every element grades
+PORTABLE. Two manifest checks assert that every element the manifest names is found
+by kind and name in each variant, and that each primary variant uses the rung's primary
+reference form. Run with `PYTHONPATH=. pytest test_grade.py`.

compliance/portability/grader/grade.py

@@ -15,7 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
-"""Fidelity grader for the portability measurement ladder (issue #576).
+"""Fidelity grader for the portability measurement ladder (apache/ossie-converters#24, formerly #576).
 
 The grader answers one question per element: when a model is driven through a
 converter and back, what happened to each metric and field expression it started
@@ -58,6 +58,7 @@
 from __future__ import annotations
 
 import argparse
+import functools
 import json
 import re
 import sys
@@ -103,25 +104,92 @@ def _normalize(expression):
     return re.sub(r"\s*([(),])\s*", r"\1", collapsed)
 
 
-def _equivalent_tree(left, right):
-    """Return True if two SQL expressions are equal after stripping table
-    qualifiers and quoting, False if they differ, or None if it cannot be decided
-    (sqlglot missing, or either side does not parse). Used only to tell a cosmetic
-    REWRITTEN from one that needs review."""
+def _field_owners(document):
+    """{field name: the datasets that declare it}, lower-cased, read from the input
+    document (the authored model), so a dataset qualifier can be checked against it."""
+    owners = {}
+    if isinstance(document, dict):
+        for dataset in document.get("datasets") or []:
+            if not isinstance(dataset, dict):
+                continue
+            dataset_name = str(dataset.get("name") or "").lower()
+            for field_def in dataset.get("fields") or []:
+                if isinstance(field_def, dict) and field_def.get("name"):
+                    owners.setdefault(str(field_def["name"]).lower(), set()).add(dataset_name)
+    return {name: frozenset(datasets) for name, datasets in owners.items()}
+
+
+@functools.lru_cache(maxsize=None)
+def _nulls_large_reader():
+    """sqlglot's base dialect, except that NULL sorts as the largest value."""
+    from sqlglot.dialects.dialect import Dialect
+
+    class _NullsLarge(Dialect):
+        NULL_ORDERING = "nulls_are_large"
+
+    return _NullsLarge
+
+
+def _null_orderings(dialect):
+    """The sqlglot readers to compare an expression under, one per NULL placement
+    the expression may mean when a window ORDER BY item omits NULLS FIRST/LAST.
+
+    OSSIE_SQL_2026 fixes the default (expression_language.md, #587): NULL sorts as
+    the largest value, so one reader with that ordering. Any other dialect keeps
+    its own default, which the expression does not say, so an omitted and an
+    explicit placement are only proven equal if they agree under both orderings.
+    """
+    if dialect == "OSSIE_SQL_2026":
+        return (_nulls_large_reader(),)
+    return (None, _nulls_large_reader())  # None is sqlglot's base dialect, NULLs small
+
+
+def _equivalent_tree(left, right, owners=None, home=None, dialect=None):
+    """Return True if two SQL expressions are equal after resolving dataset
+    qualifiers and dropping quoting, False if they differ, or None if it cannot be
+    decided (sqlglot missing, or either side does not parse). Used only to tell a
+    cosmetic REWRITTEN from one that needs review.
+
+    A qualifier is meaning, not spelling, so each column is resolved to the dataset
+    it names before comparing: a bare name in a field expression or dataset-scoped
+    metric names ``home``, the element's own dataset; elsewhere a bare name resolves
+    to the one dataset that declares it (``owners``, from _field_owners) and stays
+    bare when several do. A written qualifier is kept as written, so
+    ``customers.amount`` differs from ``sales.amount`` even if only sales declares
+    ``amount``. A name no dataset declares, and every name without ``owners``, is
+    compared without its qualifier, since there is nothing to check it against.
+
+    ``dialect`` is the authored tag: it decides what an omitted NULLS FIRST/LAST in
+    a window ORDER BY means."""
     try:
         import sqlglot
         from sqlglot import exp
     except ImportError:  # pragma: no cover - environment dependent
         return None
 
-    def canonical(text):
+    home = (home or "").lower() or None
+
+    def resolved(column):
+        if owners is None:
+            return None
+        if home:
+            return column.table.lower() if column.table else home
+        declared_by = owners.get(column.name.lower(), frozenset())
+        if not declared_by:
+            return None  # no dataset declares it, so there is nothing to check against
+        if column.table:
+            return column.table.lower()
+        return next(iter(declared_by)) if len(declared_by) == 1 else None
+
+    def canonical(text, reader):
         try:
-            tree = sqlglot.parse_one(text)
+            tree = sqlglot.parse_one(text, read=reader)
         except Exception:
             return None
         for column in tree.find_all(exp.Column):
-            # Drop the table qualifier so sales.amount and amount compare equal.
-            column.set("table", None)
+            dataset = resolved(column)
+            # sales.amount and amount compare equal when amount resolves to sales.
+            column.set("table", exp.to_identifier(dataset) if dataset else None)
             column.set("db", None)
             column.set("catalog", None)
         for identifier in tree.find_all(exp.Identifier):
@@ -136,11 +204,14 @@ def canonical(text):
             pass
         return tree.sql(normalize=True)
 
-    left_sql = canonical(left)
-    right_sql = canonical(right)
-    if left_sql is None or right_sql is None:
-        return None
-    return left_sql == right_sql
+    equal = True
+    for reader in _null_orderings(dialect):
+        left_sql = canonical(left, reader)
+        right_sql = canonical(right, reader)
+        if left_sql is None or right_sql is None:
+            return None
+        equal = equal and left_sql == right_sql
+    return equal
 
 
 def _dialect_map(expression_obj):
@@ -171,10 +242,16 @@ class Element:
     kind: str  # "metric" or "field"
     name: str
     dialects: dict
+    dataset: str = ""  # the declaring dataset of a field or dataset-scoped metric
 
 
 def _collect(document):
-    """Collect metrics and field expressions, keyed by (kind, name)."""
+    """Collect metrics and field expressions, keyed by (kind, name).
+
+    A dataset-scoped metric (``datasets[].metrics``) is keyed ``dataset.metric``,
+    the name the spec references it by, so it cannot collide with a model-scoped
+    metric of the same name.
+    """
     elements = {}
     if not isinstance(document, dict):
         return elements
@@ -192,10 +269,11 @@ def _collect(document):
         for field_def in dataset.get("fields") or []:
             if isinstance(field_def, dict) and field_def.get("name"):
                 name = f"{dataset_name}.{field_def['name']}"
-                expression = field_def.get("expression")
-                elements[("field", name)] = Element(
-                    "field", name, _dialect_map(expression)
-                )
+                elements[("field", name)] = Element("field", name, _dialect_map(field_def.get("expression")), dataset_name)
+        for metric in dataset.get("metrics") or []:
+            if isinstance(metric, dict) and metric.get("name"):
+                name = f"{dataset_name}.{metric['name']}"
+                elements[("metric", name)] = Element("metric", name, _dialect_map(metric.get("expression")), dataset_name)
     return elements
 
 
@@ -232,9 +310,9 @@ def _diagnosed(name, diagnostics):
     return name in diagnostics or name.split(".")[-1] in diagnostics
 
 
-def _rewritten(input_element, authored_text, returned_text):
+def _rewritten(input_element, authored_text, returned_text, owners=None, dialect=None):
     """A REWRITTEN finding, with the tree comparison that tells cosmetic from review."""
-    equivalent = _equivalent_tree(authored_text, returned_text)
+    equivalent = _equivalent_tree(authored_text, returned_text, owners, input_element.dataset, dialect)
     if equivalent is True:
         note = "cosmetic (normalized trees equal)"
     elif equivalent is False:
@@ -247,7 +325,7 @@ def _rewritten(input_element, authored_text, returned_text):
     )
 
 
-def _classify_present(input_element, output_element):
+def _classify_present(input_element, output_element, owners=None):
     """Classify an input element that still has a counterpart in the output."""
     in_dialects = input_element.dialects
     out_dialects = output_element.dialects
@@ -273,7 +351,7 @@ def _classify_present(input_element, output_element):
         # The changed form is the rewrite to report, ahead of a copy that came back.
         authored_tag = changed[0]
         authored_text = in_dialects[authored_tag]
-        return _rewritten(input_element, authored_text, out_dialects[authored_tag])
+        return _rewritten(input_element, authored_text, out_dialects[authored_tag], owners, authored_tag)
     if kept:
         preserved = "OSSIE_SQL_2026" in kept
         return Finding(
@@ -297,7 +375,7 @@ def _classify_present(input_element, output_element):
     # REWRITTEN: a portable tag is present but the expression text changed.
     if portable_out:
         returned_text = portable_out.get(authored_tag) or next(iter(portable_out.values()))
-        return _rewritten(input_element, authored_text, returned_text)
+        return _rewritten(input_element, authored_text, returned_text, owners, authored_tag)
 
     # No portable tag: either a placeholder stand-in or a real vendor-only return.
     if out_dialects:
@@ -314,7 +392,7 @@ def _classify_present(input_element, output_element):
     return Finding(input_element.kind, input_element.name, SURVIVED_VENDOR_ONLY, detail="no expression returned")
 
 
-def _repair_by_expression(dropped, added):
+def _repair_by_expression(dropped, added, owners=None):
     """Pair a DROPPED input element with an ADDED output element that carries an
     equivalent expression, so a converter that did not preserve an element's name
     (orionbelt writes a field's expression as its name) scores as one element whose
@@ -325,13 +403,14 @@ def _repair_by_expression(dropped, added):
     pairs = []
     used = set()
     for in_el in dropped:
+        in_tag = next(iter(in_el.dialects), None)
         in_text = next(iter(in_el.dialects.values()), "")
         for out_el in added:
             if id(out_el) in used:
                 continue
             out_text = next(iter(out_el.dialects.values()), "")
             if in_text and (_normalize(out_el.name) == in_text or out_text == in_text
-                            or _equivalent_tree(in_text, out_text) is True):
+                            or _equivalent_tree(in_text, out_text, owners, in_el.dataset, in_tag) is True):
                 pairs.append((in_el, out_el))
                 used.add(id(out_el))
                 break
@@ -352,19 +431,21 @@ def grade(input_document, output_document, diagnostics_text="", converter=""):
         return report
 
     diagnostics = diagnostics_text or ""
+    # Qualifiers that carry meaning are read from the input, the authored model.
+    owners = _field_owners(input_document)
 
     present = {}
     dropped = []
     for key, element in input_elements.items():
         if key in output_elements:
-            present[key] = _classify_present(element, output_elements[key])
+            present[key] = _classify_present(element, output_elements[key], owners)
         else:
             dropped.append(element)
 
     added_elements = [el for key, el in output_elements.items() if key not in input_elements]
 
     # Recover identity: an unnamed-preserving rename shows up as drop + add.
-    repaired = _repair_by_expression(dropped, added_elements)
+    repaired = _repair_by_expression(dropped, added_elements, owners)
     repaired_in = {id(a): b for a, b in repaired}
     repaired_out = {id(b) for _, b in repaired}
 
@@ -402,7 +483,7 @@ def _format(report):
 
 
 def main(argv=None):
-    parser = argparse.ArgumentParser(description="Grade a converter round-trip for #576.")
+    parser = argparse.ArgumentParser(description="Grade a converter round-trip (apache/ossie-converters#24).")
     parser.add_argument("--input", required=True, help="the input .ossie.yaml/.json document")
     parser.add_argument("--returned", required=True, help="the document the converter returned")
     parser.add_argument("--diagnostics", help="a text file of the converter's warnings/notices")