[fupelaqu]

The defect

Column.sql renders a column's sub-fields, then its script: c VARCHAR FIELDS (raw KEYWORD) SCRIPT AS (UPPER(k)). The DDL parser refused that rendering with ')' expected but 'S' found, because its column production read a script OR sub-fields, never both (storedScript | optionalMultiFields). So a table holding such a column did not re-parse.

A materialized view renders exactly that column for a string GROUP BY key (UPPER, LOWER, CONCAT, …), which gets a keyword sub-field, and it re-parses its DDL when it deploys. Such a view was refused at creation.

The fix

• Parser.column reads FIELDS (…), then an optional SCRIPT AS (…) [STORED], in the order Column.sql renders them. It builds the same Column: the script through ScriptProcessor.fromScript, with the sub-fields beside it.
• The shapes that already parsed parse exactly as before.
• SCRIPT AS (…) FIELDS (…) is never rendered, and stays refused.
• Parse cost is unchanged: column is still a lazy val PackratParser, and it tries the same two sub-productions as before.
• One production, about 15 lines. No case-class arity change, so no downstream rebuild.

Tests

• New ColumnFieldsAndScriptSpec (sql, 9 tests). Each row parses a table, renders it (Table.sql) and parses the rendering again. It asserts that the rendering is stable and that the column reads the same (script, STORED, sub-fields, DEFAULT, COMMENT). The rows:
  • FIELDS + SCRIPT, plain, STORED, with a COMMENT on the column and on the sub-field, and with a DEFAULT and a COMMENT;
  • the four shapes that already parsed: a script alone, a STORED script alone, sub-fields alone, a script with a DEFAULT;
  • the refusal of SCRIPT AS (…) FIELDS (…), pinned (')' expected but 'F' found).
  • Without the fix, 4 of the 9 fail (the FIELDS + SCRIPT rows), with ')' expected but 'S' found.
• New DDL row in GatewayApiIntegrationSpec: CREATE TABLE … (id INT, k KEYWORD, c VARCHAR FIELDS (raw KEYWORD) SCRIPT AS (UPPER(k))), with one document k = 'ab'. The pipeline computes c = 'AB', and the mapping is c: text with fields.raw: keyword.
  • ES 8.18.3: 1 / 1. Without the fix: 0 / 1, the statement refused with ')' expected but 'S' found.
  • ES 6.8.23: 1 / 1.
• sql/test: 1922 succeeded, 0 failed (1913 + 9). No existing pin moved, and no existing DDL renders or parses differently.
• Independent review, in process. A generated population of 4,032 column shapes: 6 types × 7 sub-field forms × 4 script forms × DEFAULT × NOT NULL × COMMENT × OPTIONS, nested STRUCT fields included.
  • Of the 1,440 shapes that parsed before, every one renders byte-identically and reads the same column after the fix.
  • The other 2,592 are all FIELDS + SCRIPT, and now parse.
  • All 4,032 round-trip equal.
  • ALTER TABLE … ADD COLUMN shapes are unchanged.

Release note

"A column can declare its sub-fields and its script together (c VARCHAR FIELDS (raw KEYWORD) SCRIPT AS (UPPER(k))), as SHOW TABLE renders it."

Recorded, unchanged (pre-existing)

• The DDL parser does not check a script against its column's type: c INT SCRIPT AS (UPPER(k)) and o STRUCT SCRIPT AS (UPPER(k)) already parse. The fix extends that to o STRUCT FIELDS (a KEYWORD) SCRIPT AS (UPPER(k)) (PARSE only).
• The refusal of SCRIPT AS (…) FIELDS (…) is the combinator's message (')' expected but 'F' found, or end of input expected in ALTER TABLE … ADD COLUMN), and does not name the required order.

Verification not run here

• The ES 7.17 and ES 9 legs, and the arrow and JDBC legs.
• core/test and the bridge suites on this branch (the change is in sql only).
• Ran here: CI's lint line (headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck), green. sbt "+ compile" (2.12 and 2.13), green on a tree holding this commit.

The materialized-view fix for string GROUP BY keys in extensions needs this change.

🤖 Generated with Claude Code