[fupelaqu]

fix(sql): a typed query answers one class per column, the type it reports

Summary

On the Painless path (script fields, sort scripts, GROUP BY key scripts, script predicates) a schema-typed query could answer a value whose class was not the type the column reported. It could also fail outright where DuckDB and PostgreSQL answer:

• A conditional answered two classes. For example, CASE WHEN n > 0 THEN CAST(d AS DATE) ELSE CAST('2024-01-01' AS DATE) END reported DATE. The cast arm answered a LocalDate, the DATE literal was widened to a ZonedDateTime, and a bare DATE column stayed its doc value.
  • A consumer called a LocalDate method on the other rows, or the reverse, and failed. Example: DATE_TRUNC(CASE … END, MONTH) called truncatedTo on a LocalDate.
  • COALESCE converted no arm: COALESCE(CAST(n AS DOUBLE), 2) answered the integer 2.
  • NULLIF answered its first argument's doc value.
  • GREATEST / LEAST over dates answered a TIMESTAMP where DuckDB answers a DATE.
• LAST_DAY kept its operand's class.
  • Over a timestamp it kept the time of day: LAST_DAY(ts) answered 2024-02-29T23:30Z where DuckDB answers 2024-02-29.
  • Beside a DATE arm it made the conditional answer two classes.
  • Over a date_nanos column, whose type is unknown to the schema, it answered a ZonedDateTime with its time of day.
• ASIN, ACOS, LOG, LOG10 and SQRT outside their domain answered NaN or -Infinity, which the column read as the text "NaN". DuckDB fails the statement.
• A NULL literal argument was rendered into the script.
  • NULLIF(d, NULL) and LEAST(NULL, dn) produced null != null && … and Math.min(null, …), which Painless refuses to compile.
  • DATE_ADD(COALESCE(CAST(d AS DATE), NULL), …) called .plus on a possibly-null value.
• A date-only string did not cast to TIMESTAMP / DATETIME. CAST(kd AS TIMESTAMP) over a keyword holding 2024-01-03 threw on every document and failed the statement. DuckDB and PostgreSQL read the midnight of that day.
• A DATE cast to BIGINT or TIME answered the date itself, or failed.
  • CAST(CURRENT_DATE AS BIGINT) and CAST(CAST('2024-01-03' AS DATE) AS BIGINT) answered the LocalDate under the BIGINT label.
  • CAST(CAST(ts AS DATE) AS BIGINT), CAST(LAST_DAY(ts) AS BIGINT) and a DATE conditional under a cast to BIGINT or TIME (CAST(COALESCE(CAST(d AS DATE), CAST('2024-01-01' AS DATE)) AS BIGINT)) called toInstant() on a LocalDate and failed the statement.

Root cause

A conditional took its arms' common out and converted each arm from its baseType. A schema-resolved arm's out and baseType are its column's runtime type, whatever function the arm wears. So a CASE of two DATE arms over a date column was a TIMESTAMP that converted nothing. LAST_DAY applied with(lastDayOfMonth()) to whatever its receiver was. NULL literals went through the same rendering as any other argument.

The fix

Nine commits:

1. LAST_DAY answers a DATE whatever its operand.

   • In a query, the receiver is converted to a DATE before the adjuster. The conversion works from what the receiver renders: a date doc value goes through the UTC normalisation that Elasticsearch 6.8 needs (….toInstant().atZone(ZoneId.of('Z')).toLocalDate()), and a receiver that already renders a LocalDate is used as it is.
   • LAST_DAY reports DATE, as DuckDB types last_day(TIMESTAMP).

2. A CASE, COALESCE, NULLIF, GREATEST or LEAST answers one class: the type it reports.

   • There is one rule, SQLTypeUtils.convertArm. The node's type is DuckDB's common type of what its arms report: DATE + DATE = DATE, DATE + TIMESTAMP = TIMESTAMP, INT + DOUBLE = DOUBLE.
   • In a query, each arm is converted to that type once, from what it renders:
     • a CASE converts its results and its ELSE;
     • COALESCE converts every arm;
     • NULLIF converts its answer;
     • GREATEST / LEAST over dates convert the winning instant.
   • Over numbers, GREATEST / LEAST already convert every row to their reported type (typedResult, the parent fix). That is the same rule, so the two halves agree: one common reported type, rendered by a cast.
   • A simple CASE's operand and WHEN values keep their comparison, and NULLIF keeps its comparison arm.
   • A date doc value never gets a bare .toLocalDate(). An arm that is already narrowed is not converted again. A conversion of a compound rendering applies to all of it, not to the last branch of a conditional.
   • Arms with no common type are refused (seventh commit): numbers beside dates, text beside numbers, a TIME beside a date, as DuckDB refuses them. An aggregate arm and an arm whose type is unknown (no schema, date_nanos, ip) keep their bytes.
   • LAST_DAY over a nameless operand that can be NULL (DATE_TRUNC(CASE … ELSE NULL END, MONTH)) binds it once and converts and adjusts it inside a guard on it; appended to the guarded expression, .with(…) ran on its NULL.
   • The type an ingest processor writes is read in the scope it renders in (SQLTypeUtils.processorType, read by ScriptProcessor.fromScript, ALTER COLUMN … SET SCRIPT and the extensions' scripted view fields). An UPDATE's one-off pipeline description is unchanged.

3. A NULL literal argument is folded, not rendered (and a CAST over a CASE converts every branch).

   • In a query, NULLIF(x, NULL) is x and NULLIF(NULL, x) is NULL. COALESCE drops a NULL arm, and GREATEST / LEAST skip a NULL argument (DuckDB and PostgreSQL do the same). With no argument left, each is NULL.
   • The folded node keeps the other argument's type and its NULL guard.
   • A method folded onto a bound expression that can be NULL is guarded.
   • IFNULL and NVL do not parse in core; COALESCE is their spelling.
   • A CAST over a CASE converts the CASE as one value, bound once and under one NULL guard. Appended to p ? a : b, the conversion converted the ELSE branch only, and String.valueOf around it answered the text "null" for its NULL rows. This closes the recorded "a CAST over a CASE converts its last branch only": 24 statements on ES 8 and 28 on ES 6.8 converted the last branch only on the round-1 tip, 0 now.

4. A date-only string casts to TIMESTAMP and DATETIME as its midnight.

   • In a query, the operand is bound once and completed with T00:00:00 when it is 10 characters long, before the same parse. This is one length test per document, with no formatter built per document.
   • A string literal is completed when the script renders.

5. A DATE cast to BIGINT answers its epoch milliseconds, and to TIME its midnight, for every DATE operand (the lead's ruling).

   • CAST(<DATE> AS BIGINT) is the epoch milliseconds of the value's UTC midnight; CAST(<DATE> AS TIME) is 00:00:00.
   • It is one rule in the cast (Conversion.convertsDate, converted by Conversion.dateAtMidnight), for every operand that renders a DATE: a cast to DATE, CURRENT_DATE, a DATE literal, LAST_DAY, and a CASE, COALESCE, NULLIF, GREATEST or LEAST that answers a DATE.
   • What the operand renders is read from its own chain (Conversion.rendersDate): a conditional renders the type it answers, a function that only adjusts a date (DATE_ADD, DATE_TRUNC, an interval) renders its receiver's class, and any other function renders its own type. The operand's out cannot answer, because the cast sets it to the target: CAST(CURRENT_DATE AS BIGINT) read BIGINT -> BIGINT and converted nothing.
   • A conditional counts only once its arms are converted to the type it answers, which is the decision it takes itself (SQLTypeUtils.armsType over the arms it answers). A conditional whose arms are left as they render (no common type, or an arm of a type still unknown: no schema, date_nanos) renders no single class, and the cast keeps its conversion. Without this, the bridge pin TRY_CAST(COALESCE(NULLIF(createdAt, DATE_PARSE(…)), CURRENT_DATE - INTERVAL 2 HOUR) BIGINT) (no schema, SQLQuerySpec) moved. It is unchanged.
   • The operand is converted as one value, bound once and guarded. A DATE CASE answers NULL without being nullable.
   • A DATE column renders its doc value, an instant, and keeps main's answers: CAST(d AS BIGINT) is the stored instant's epoch milliseconds, and CAST(d AS TIME) that instant's UTC time of day.
   • DuckDB 1.5.5 refuses both casts (Unimplemented type for cast (DATE -> BIGINT), (DATE -> TIME)), so the values are the rule's.

6. A DATE operand is its UTC midnight under a time of day or a sub-day interval; whole days over a DATE report DATE.

   • HOUR / MINUTE / SECOND / the sub-second parts / OFFSET_SECONDS and EXTRACT(… FROM …) of an operand that renders a DATE read it at its UTC midnight, as DuckDB reads hour(DATE) (0). This also repairs the pre-existing HOUR(CAST(d AS DATE)), which failed on main.
   • A sub-day interval over an operand that renders a DATE starts from its UTC midnight and is a TIMESTAMP (DATE + INTERVAL 1 HOUR, as DuckDB types it).
   • DATE_ADD / DATE_SUB / DATETIME_ADD / DATETIME_SUB over an operand that reports a DATE report a DATE for whole days (DAY, WEEK, MONTH, QUARTER, YEAR) and a TIMESTAMP sub-day. DuckDB 1.5.5 types DATE + INTERVAL 1 DAY a TIMESTAMP; the lead's date-arithmetic ruling (a date plus whole days is a date) and the gate oracle's CAST(d + INTERVAL 1 DAY AS DATE) win.
   • Every consumer reads what its operand renders through one derivation (SQLTypeUtils.chainRendered / operandRendersDate), the one the cast to BIGINT / TIME reads; that cast now reads each conditional's own arm decision (AnswersArms, SQLTypeUtils.armsAnswer, fifth commit) rather than restating it.

7. A CASE or COALESCE whose arms have no common type is refused, after the types are read, with DuckDB 1.5.5's reason (Cannot mix values of type INT and DATE in CASE expression - an explicit cast is required, … in COALESCE operator …). A BOOLEAN beside a number is accepted (DuckDB: INTEGER); NULL, string literals, unknown and aggregate arms are left out. NULLIF keeps its comparison rule, which already refuses such arguments. Over the 13,289-statement corpus it fires on 1,826 statements: 1,802 are ones DuckDB refuses (ORACLE-REFUSES-TYPE), 24 carry an ip arm (no oracle); none is a statement DuckDB answers.

8. ASIN, ACOS, LOG, LOG10 and SQRT outside their domain fail the statement, as DuckDB does.

   • The script throws an IllegalArgumentException with DuckDB's reason: ASIN is undefined outside [-1,1], cannot take logarithm of zero / of a negative number, cannot take square root of a negative number.
   • Painless has no throw expression, so the throw is Optional.empty().orElseThrow(() -> new IllegalArgumentException('<reason>')).
   • Elasticsearch fails the search with the illegal argument's reason as the caused_by of each shard failure. The status is the server's, not the throw's: ES 8.18.3 answers 400, ES 6.8.23 answers 500 (measured in the testkit row below). The earlier claim that the illegal argument makes 6.8 answer 400, where Objects.requireNonNull answered 500, does not hold: 6.8 answers this script error 500 too.
   • LN does not parse in core; LOG is the natural logarithm.
   • Not in this PR: reading a non-finite DOUBLE back as a Double (NaN / Infinity in a DOUBLE column). It is deferred to the typed-rows item.

9. LAST_DAY over a date_nanos column answers a DATE (ninth commit).

   • date_nanos types Any, so the arm conversion leaves LAST_DAY's receiver as it renders. In a query, LAST_DAY now reads such a receiver's runtime class, once per row (LastDayOfMonth.dateOf): a LocalDate as it is, a LocalDateTime its date, a zoned value its UTC date. A nameless receiver is bound once.
   • The dispatch fires on the arm conversion's own test (SQLTypeUtils.armSourceType unknown), and in LAST_DAY only. A typed receiver renders as before, byte for byte. An ingest processor and a transform keep their bytes.
   • The date_nanos CONDITIONAL arms are not covered by this commit; they keep main's bytes (see "Recorded").

10. On ES 6.8, a comparison of two date columns is scripted, as on the other majors (cd85e6b1, fix(sql): a typed query answers one class per column, the type it reports #443): the hand-maintained ES 6 bridge gets the template's two-operand requiresScript (ts < d failed there with failed to parse date field [d]); DateArithmeticSpec's 600 TIMESTAMP-vs-DATE comparisons now run on every major (the 72 column-against-column ones ran on 7 and up only): ES 6.8.23 600 / 600 equal DuckDB, 0 errors, the whole suite 30 / 30; es6bridge/test 326 / 326; CI lint line green on a fresh clone.

Query context only. An ingest processor and a transform keep their bytes, so a pipeline does not churn on ALTER. A type is context-free, and date arithmetic and GREATEST / LEAST read it. So while a processor or transform script renders, a conditional and LAST_DAY answer the type they answered before. This is scoped at the three render entry points by SQLTypeUtils.renderingIn. Without it, 500 processor renders of a 13,000-statement corpus moved, some to a wrong answer. With it, none moves.

What changes for the user

• On ES 6.8 and later, a CASE, COALESCE, NULLIF, GREATEST or LEAST over dates answers its reported type in every row. A date function over it no longer fails on half the rows.
• LAST_DAY answers a DATE, without the time of day.
• NULLIF(x, NULL), COALESCE(…, NULL), LEAST(NULL, …) and GREATEST(NULL, …) compile and answer DuckDB's value.
• CAST(<keyword> AS TIMESTAMP) reads 2024-01-03 as 2024-01-03T00:00:00Z.
• CAST(<a DATE> AS BIGINT) answers the epoch milliseconds of its UTC midnight, and CAST(<a DATE> AS TIME) 00:00:00. A DATE column answers as before.
• Ingest pipelines and materialized-view computed fields are unchanged.

Dependency

Based on main after #439 to #444 merged; the rebase was clean. Before this push, the date / conditional / client-error area specs ran once on ES 8.18.3 on the rebased tip: DateArithmeticSpec, DateFunctionExecutionSpec, ScrollCompletenessSpec, GroupByCompletenessSpec and ClientErrorStatusSpec, 193 / 193. CI runs every major.

A rejected search page says why its shards failed (core and every client)

Elasticsearch gives why a shard failed (ASIN is undefined outside [-1,1] under a script's runtime error) only under failed_shards[i].reason.caused_by. The ES 8 / ES 9 client appended it; the REST clients did not, so on Elasticsearch 7 a streamed SELECT outside a function's domain answered all shards failed: runtime error only. One function in core, ShardFailures.shardReason, now reads it for every client: the ES 6 / ES 7 REST clients append it to a rejected page's message, and the ES 8 / ES 9 clients use it in place of their own copy (their message is unchanged). The test of a function outside its domain asserts the reason on every major. An existing REST unit row expected the message without it; it now expects Elasticsearch's reason.

The scroll test of a rejected page now uses a query every major refuses with a 400 (value > 'not-a-number' on an integer field). It relied on a script error, and Elasticsearch 6.8 answers a Painless runtime error with a 500.

Tests

• ConditionalArmClassSpec (new, 11 rows):

  • every arm pair: DATE / DATE, DATE / TIMESTAMP, DATE literal / DATE column, CURRENT_DATE, LAST_DAY, INT / DOUBLE;
  • the consumers;
  • the pairs with no common type, unchanged;
  • an unknown arm;
  • an ingest processor's bytes, unchanged.

• NullLiteralFoldSpec (new, 8 rows): each fold, the consumers' NULL guard, no NULL constant over the family, the SQL round trip, and an ingest processor's bytes.

• DateOnlyTimestampCastSpec (new): a keyword to TIMESTAMP and to DATETIME, a date-only and a date-time literal, the completed text against the formatter, and an ingest processor's bytes.

• DateCastMidnightSpec (new, 12 rows):

  • a DATE column, a cast to DATE, CURRENT_DATE, a DATE literal, CASE, COALESCE, NULLIF, GREATEST, LEAST and LAST_DAY, each to BIGINT and to TIME: the rendered Painless and the reported type;
  • the DATE column's two casts pinned as on main;
  • DATE_ADD over a DATE converted, a TIMESTAMP DATE_ADD and a DATETIME_ADD kept on the TIMESTAMP conversion;
  • the conversion's own calls against the JVM (2024-01-03 is 1704240000000 and 00:00);
  • an ingest processor's and a transform's bytes, as before.
  • On bdbb2727 (the four commits' tip) 9 of the 12 fail; the DATE column, the JVM and the processor / transform rows pass.

• DateDocValueEmissionSpec: LAST_DAY over d, ts, a date_nanos operand, and its reported DATE.

• Testkit DateFunctionExecutionSpec, block GATE-024 typed-path values, over a 4-document table and a date_nanos / undeclared-date index. Each row asserts DuckDB 1.5.5's value for every document. On this path a script field comes back with the response parser's class, so a DATE is asserted as its yyyy-MM-dd text and a TIMESTAMP as the instant it denotes: a timestamp under a DATE label fails, and so does a date under a TIMESTAMP label.

  • 11 fix rows:
    • a CASE of a cast to DATE and LAST_DAY;
    • a DATE literal beside a cast to DATE;
    • a bare DATE column beside a cast to DATE;
    • DATE_TRUNC over a CASE of a TIMESTAMP and a DATE;
    • COALESCE(CAST(n AS DOUBLE), 2);
    • NULLIF(d, NULL);
    • DATETIME_ADD(LEAST(NULL, dn), INTERVAL 1 HOUR);
    • ISNOTNULL(CAST(kd AS TIMESTAMP));
    • LAST_DAY(ts);
    • LAST_DAY over an undeclared date column;
    • a DATE cast to BIGINT and TIME (9 statements): COALESCE, CASE and two NULLIF to BIGINT, COALESCE to TIME, CAST(CAST(ts AS DATE) AS BIGINT | TIME), a DATE literal and CURRENT_DATE to BIGINT. It asserts the UTC-midnight epoch milliseconds and 00:00:00, not DuckDB's values (DuckDB refuses the casts).
  • 1 row of 11 statements that were right on main and whose script changes. Their values are asserted; the EXTRACT one runs on ES 7 and later, because it fails on 6.8 before and after this change.

• Round 2 rows:

  • ConditionalArmClassSpec: LAST_DAY's guard over a nameless operand; the type a processor writes (SQLTypeUtils.processorType, fromScript) and an UPDATE's pipeline description.
  • NullLiteralFoldSpec: a cast over a CASE (VARCHAR, TIMESTAMP) under one guard; its processor row names the recorded twin it pins (a script that cannot compile).
  • DateOperandSpec (new): time-of-day parts over each DATE producer, sub-day intervals, the reported types, the COALESCE beside DATE_ADD, an ingest processor's bytes.
  • MixedArmsRefusalSpec (new): each refused arm pair with DuckDB's reason, the accepted pairs.
  • MathDomainSpec (new): each function's domain test and reason, an ingest processor's bytes.
  • Testkit block GATE-024 typed-path values, 6 new rows: a time of day of a DATE (6 statements), LAST_DAY inside its operand's guard (2), a cast over a CASE (2, the NULL row asserted as NULL, not the text), a COALESCE beside DATE_ADD (2), the refusals (2), the domain failures (4 + an in-domain control). All 6 are red at the round-1 tip on ES 8.18.3 and ES 6.8.23 and green at the tip.

• Round 3 rows:

  • MathDomainSpec: the throw is an IllegalArgumentException.
  • DateDocValueEmissionSpec: LAST_DAY over a date_nanos operand reads its runtime class; a nameless operand of unknown type is bound once; a typed receiver has no class read.
  • Testkit row "fail a function outside its domain, as DuckDB does": through GatewayApi.run, each of ASIN, ACOS, SQRT, LOG and LOG10, streamed (no LIMIT: the Community quota streams it, and the failure surfaces when the rows are read) and with LIMIT 10 (a search: run answers an ElasticFailure). It asserts the failure, the server's status where the client carries it (500 on ES 6, 400 on ES 7 and later; ES 7 and 9 not measured locally) and DuckDB's reason where the client carries it (the REST clients, ES 6 and 7; not the Java client, ES 8 and 9), as ES's reason, not the script's own literal. Each gap is alerted in the run (see the ES row below).
  • Testkit row "answer a DATE for LAST_DAY over a date_nanos column": an index, a pattern and a GROUP BY key. The key is asserted as its DATE text; the count is compared by value (BigDecimal), as the GREATEST / LEAST rows do: ES 8 answers it 1.0 (the response parser's class, the recorded typed-rows gap), ES 6.8 1. n2 (no dn) answers NULL, DuckDB 1.5.5's answer (last_day(NULL::TIMESTAMP_NS), run).

Moved pins

Spec	Pin	Why
DateDocValueEmissionSpec	LAST_DAY(d), LAST_DAY(ts), and the receiver-method row	The receiver is made a DATE through the UTC normalisation before the adjuster. The methods called on the doc value are now toInstant and with.
ParameterIdentitySpec	LAST_DAY(d) = d	Still one parameter for d. LAST_DAY's LocalDate is promoted to meet the raw column.
MixedTemporalComparisonSpec	LAST_DAY(d) = d, LAST_DAY(d) > CAST(ts AS DATE)	The first moves to the promote table. The second is now two date-only operands.
PredicateTransformSurvivalSpec	LAST_DAY(d) = ts, LAST_DAY(d) = LAST_DAY(ts), COALESCE(d) = 'x'	LAST_DAY is a DATE: each receiver is converted once. COALESCE(d) answers its DATE.
NullIfOperandSpec	NULLIF(CAST(n AS KEYWORD), 'x')	It reports the KEYWORD it answers (was VARCHAR). Its comparison is now asserted on the rendering.
PainlessTargetSpec	COALESCE(d, d2) > CAST('2024-01-01' AS DATE): 1 → 0	Both arms are converted through the UTC normalisation, so no 6.8 normalisation is added.
ConditionalArmClassSpec	LAST_DAY(CASE WHEN n > 0 THEN ts ELSE d END)	The CASE is bound and LAST_DAY applies inside a guard on it (round 2).
ParameterIdentitySpec	CASE WHEN DATE_ADD(d, INTERVAL 1 DAY) > ts THEN DATE_ADD(d, INTERVAL 1 DAY) ELSE d END (and its swap)	d + 1 day reports a DATE beside the DATE d, so the CASE answers a DATE, each arm converted once; the interval still folds onto its own parameter (round 2).
SQLQuerySpec (both bridges)	TRY_CAST(COALESCE(NULLIF(createdAt, …), CURRENT_DATE - INTERVAL 2 HOUR) AS BIGINT)	The DATE minus two hours starts from its UTC midnight and the TIMESTAMP is converted to its epoch milliseconds. It was LocalDate.minus(2, HOURS), which throws, swallowed into NULL by the TRY_CAST (round 2).
SQLQuerySpec (both bridges)	the SQRT, LOG, LOG10, ASIN, ACOS script fields	They carry the domain test (round 2), and throw an IllegalArgumentException (round 3).
DateDocValueEmissionSpec	LAST_DAY over a date_nanos operand	It reads the receiver's runtime class (ninth commit).

The 5th commit moves no existing pin. No processor or ingest pin moved. ConditionalArmClassSpec, NullLiteralFoldSpec and DateOnlyTimestampCastSpec each pin an ingest processor's bytes as they are on main.

Results (round 3, tip 12019581, base 5dd10fb9)

Check	Result
Render corpus (13,289 statements, 25,121 renders per mode), against the rebased round-2 tip	Query: 97 move. 80 are the domain throw only (requireNonNull → IllegalArgumentException), and 17 are LAST_DAY over a date_nanos operand (LAST_DAY(dn) ×5, over a CASE, DATE_TRUNC or GREATEST of dn ×12). ES 6.8 target: 80, the throw only (dn is a plain date there). Processor and transform: 0. No typed receiver moves.
sql/test	2,082 / 2,082
core/test	1,308 / 1,308
softclient4es8-sql-bridge/test / es6bridge/test	309 / 309, 322 / 322
ES 8.18.3 / ES 6.8.23, DateFunctionExecutionSpec block GATE-024 typed-path values, at 9cb31963 (the two testkit rows amended into their commits; no product byte changed)	ES 8 (Java client): 19 / 19. ES 6.8 (REST client): 19 / 19. Before the row fixes: 17 / 19 and 18 / 19. ES 7, ES 9 and the ES 6 Jest client: not run locally (CI).
— domain failure, ES 8.18.3, Java client	Streamed: the stream fails when read, with the client's own co.elastic.clients.elasticsearch._types.ElasticsearchException ("[es/search] failed: [search_phase_execution_exception] all shards failed"), not an ElasticError: no status, no reason. LIMIT 10: ElasticFailure, status 400, message "Elasticsearch error during singleSearch: search_phase_execution_exception - all shards failed; runtime error": the reason is missing (it is the caused_by of each shard failure, which the client does not read).
— domain failure, ES 6.8.23, REST client	Streamed: the stream fails when read, with the client's own org.elasticsearch.client.ResponseException (status line HTTP/1.1 500), not an ElasticError: no status carried; the message carries ES's response, with the reason ("caused_by":{"type":"illegal_argument_exception","reason":"ASIN is undefined outside [-1,1]"}). LIMIT 10: ElasticFailure, status 500 (the server's: "status":500), the reason carried.
CI lint line on a fresh clone of the committed bytes	green (re-run on 9cb31963)
sbt "+ compile" (2.12 + 2.13), on the same clone	green on 12019581; not re-run on 9cb31963 (testkit main sources changed: CI)

Results (round 2, tip 6f70a933)

Check	Result
All RIGHT-on-main changed statements, against the base control 43b9aa0b (the reviewer's tools, same day)	ES 8: 895 / 895 kept, 0 regressions; ES 6.8: 535 / 535, 0 regressions. The 11 / 8 ids that regressed at round 1 are RIGHT.
The reviewer's 46 targeted shapes, ES 8	0 RIGHT on base → not RIGHT; 9 move, all to DuckDB's answer (the B-1 shapes, HOUR(LAST_DAY(ts)), the cast over a CASE, LAST_DAY(CASE … ELSE NULL END)). The ORDER BY shapes fail on both trees (recorded).
Render census, processor and transform	0 / 25,121 each moved against 43b9aa0b; the UPDATE descriptions and fromScript types of the reviewer's 18 expressions identical to base
Coverage against the round-1 tip (rendered fresh at 6fea466b)	RCV-1 95 / 82 / 1, RCV-2 11 / 11, RCV-3 10 / 10, LS-5 279 / 157, LS-3 310 / 246 (309 / 245)
sql/test	2,060 / 2,060
softclient4es8-sql-bridge/test / es6bridge/test	309 / 309, 322 / 322 (the moved pins above)
ES 8.18.3 / ES 6.8.23, DateFunctionExecutionSpec	34 / 34 each
sbt "+ compile", CI lint line on a fresh clone	green

Results (round 1)

Check	Result
sql/test	2,042 / 2,042
softclient4es8-sql-bridge/test	309 / 309, no pin moved
es6bridge/test	322 / 322, no pin moved
ES 8.18.3, Java client: GATE-024 typed-path values	12 / 12
ES 8.18.3, the 11 earlier rows on the base	0 / 11. The 10 fix rows fail on the class or value they fix. The "right on main" row fails because 5 of its 11 statements answered another class than DuckDB's type on main (the gate compared their values); the other 6 answered as now.
ES 8.18.3, the DATE cast row on bdbb2727 (the four commits' tip)	0 / 1: 9 of its 9 statements wrong. 7 answer the date itself (2024-01-15 for a BIGINT), 2 fail the search (`CAST(CAST(ts AS DATE) AS BIGINT
ES 6.8.23, REST client: GATE-024 typed-path values	12 / 12
ES 6.8.23, the 11 earlier rows on the base	0 / 11
ES 6.8.23, the DATE cast row on bdbb2727	0 / 1: 9 of 9 statements wrong
Extensions community/test with this core's sql classes first on the classpath	223 / 223, as on the base jar; no MV spec moved
sbt "+ compile" (2.12 + 2.13)	green
headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck, caches cleared, on a clone of the committed bytes	green

ES 7, ES 9 and the ES 6 Jest client are left to CI. The extensions run was not repeated for the 5th commit.

Release-gate coverage (render census, 13,289 statements, query context)

Group	ES 8	ES 6.8	ES 7
RCV-1: one class, equal to the reported type	95 / 97 (94 without a wrong receiver)	82 / 83	1 / 1
RCV-2: no NULL constant	11 / 11	11 / 11
RCV-3: no date-only ISO_DATE_TIME parse	10 / 10	10 / 10
LS-5: LAST_DAY renders a LocalDate	279 / 279 (+ 3 LAST_DAY(dn), excluded)	157 / 157
LS-3: one class per CASE	309 / 339	245 / 246
Residual-4 ids: one class	28 / 28	28 / 28	2 / 2 (ES 9: 3 / 3)

• RCV-1 misses.
  • ES 8: three cells with a date_nanos arm under DATE_TRUNC. That arm's type is unknown (date_nanos types Any), so the node keeps its bytes as on main: two render two classes, and one renders one class but calls truncatedTo on a LocalDate.
  • ES 6.8: CAST(DATE_TRUNC(CASE … dn …, MONTH) AS DATE), where the cast over DATE_TRUNC's TEMPORAL answers a ZonedDateTime.
• LS-3 misses.
  • ES 8: 29 cells with a date_nanos arm (kept as on main) and one pre-existing CAST-over-CASE precedence cell. 24 percentile cells are outside these groups.
  • ES 6.8: the same precedence cell.
• Residual 4 is discharged; the canary (CASE of a cast to DATE and a DATE literal) renders one class. On Elasticsearch the same arm pair, as a simple CASE, is the fix row "keep a DATE literal a DATE beside a cast to DATE".
• Ingest processor and transform renders: 0 of 25,121 each changed, at the four commits and again with the 5th.
• The 5th commit's query renders: 56 of 25,121 move on ES 8 and the same 56 on ES 6.8: 14 shapes × 2 indices × SELECT / GROUP BY. Every one is a cast of a DATE conditional to BIGINT or TIME. On the base:
  • 6 shapes failed: the F-7 five (COALESCE, CASE and two NULLIF to BIGINT, COALESCE to TIME: toInstant() on a LocalDate) and CAST(NULLIF(<DATE literal>, CAST(ts AS TIME)) AS TIME) (toLocalTime() on a LocalDate). That last one is converted now, but its comparison of a DATE with a TIME is unchanged; it was not run.
  • 5 shapes answered the epoch milliseconds or the midnight of the instant GREATEST / LEAST chose (rendered), and at the four commits' tip they answered the date itself. These are LEAST / GREATEST over dates to BIGINT (4 shapes) and LEAST(CAST(d AS DATE), NULL) to TIME. The 5th commit gives them the base's values again.
  • 3 shapes answered the LocalDate on the base and at the tip: NULLIF of DATE literals or DATE_PARSE to BIGINT and TIME.
  • The census holds no cast over a cast to DATE, no CURRENT_DATE and no LAST_DAY under these casts. DateCastMidnightSpec pins them.
  • The processor twins of RCV-2 and RCV-3 are recorded unchanged. In an ingest processor, 11 of 13 RCV-2 renders still carry a NULL constant, and the 10 RCV-3 renders still parse a date-only keyword with ISO_DATE_TIME.
• Changed query renders: 3,767 statements on ES 8 (895 right on main) and 3,970 on ES 6.8 (535 right on main). The 11 statements of the "right on main" row are a sample of them.

Open points for review

• The render scope (SQLTypeUtils.renderingIn) is a new mechanism (accepted). It is a DynamicVariable set at the render entry points: Identifier.painless, FunctionN.painless, Case.painless and ScriptProcessor.fromScript. While a processor or transform renders, a conditional's and LAST_DAY's type answer what they answered before, so processor bytes do not move. It is thread-local, and withValue restores the outer value on exit, including on a throw.
• The DATE cast rule (the lead's ruling, the 5th commit). CAST(<DATE> AS BIGINT) answers the epoch milliseconds of the value's UTC midnight, and CAST(<DATE> AS TIME) 00:00:00, for every operand that renders a DATE.
  • There is one rule in the cast. What renders a DATE is read from the operand's chain, never from its out, which the cast sets to the target.
  • A DATE column keeps main's answers. CAST(d AS BIGINT) is the stored instant's epoch milliseconds, as Elasticsearch SQL casts a date to a long. CAST(d AS TIME) is that instant's UTC time of day, 00:00:00 for a date-only value; a tracked testkit row pins it ("a CASE over CAST(d AS TIME) and a TIME literal").
  • Rows: DateCastMidnightSpec (12) and the testkit row "convert a DATE to BIGINT and TIME through its UTC midnight" (9 statements). The row is red on bdbb2727 and green at the 5th commit on ES 8.18.3 and ES 6.8.23.
  • Numbers: 56 query renders move on each major, and 0 processor and 0 transform renders (see the census).
  • The 5 GREATEST / LEAST shapes that answered the date itself at the four commits' tip were a silent change from the base by commit 2. The 5th commit restores their values (rendered; none of them is in the testkit's "keep answering what it answered where it was right" row).
  • Out of scope, recorded only: CAST(d AS INT) and CAST(d AS DOUBLE) answer the raw doc value (pre-existing, silent). CAST(ts AS BIGINT) is unchanged.

Recorded for the lead (not fixed here)

• date_nanos arms keep main's bytes (residual, ruling 8).
  • ES 8, RCV-1c: PDC-DC-32664-P-SEL, PDC-DC-34122-P-SEL, PDC-DC-35094-P-SEL.
  • ES 8, LS-3 (29): PD1-D1-00132-P, 00156-P, 00293-P, 00294-P, 00708-P, 00732-P, 00780-C, 00780-P, 00804-C, 00869-P, 00870-P, 00872-C, 00872-P, 00873-C, 01164-P, 01188-P, 01284-P, 01308-P, 01332-P, 01440-P, 01441-P, 01445-P, 01446-P, 01447-P, 01884-P, 02022-P; PDC-DC-34890-P-GBK, PDC-DC-34890-P-SEL, PDC-DC-38361-P-GBK.
  • ES 6.8, RCV-1c: PDC-DC-34122-P-SEL. There dn is a plain date; the cast over DATE_TRUNC's TEMPORAL answers a ZonedDateTime under DATE.
  • LAST_DAY(dn) on ES 8 (LS-5-DN, 3 cells) answers a DATE at round 3 (ninth commit). The conditional arms above are not covered.
• A non-finite DOUBLE reads back as text (deferred to the typed-rows item). A DOUBLE column whose value is NaN or Infinity reads the text, not a Double. The domain functions no longer produce one; other expressions still can.
• LAST_DAY(GREATEST(NULL, dn)) has no NULL guard (pre-existing). The folded GREATEST leaves dn unguarded, so a document without dn fails the statement, before and after the ninth commit (rendered, not run).
• A CAST over a CASE converts its last branch only (pre-existing, silent). The cast's conversion is appended to an unparenthesised ternary, so only the ELSE branch is converted.
  • Size: 24 of the 328 CAST-over-conditional statements on ES 8, the same on main (6 shapes × P/C × SEL/GBK). 28 on ES 6.8 at this tip (24 on main), the 7th shape being CAST(CASE … dn ELSE DATE_PARSE(kd, '%Y-%m-%d') END AS TIME).
  • Shapes: PDC-DC-33898 (CAST(CASE … CAST(d AS DATETIME) ELSE <DATE literal> END AS DATE)), 33920, 34365, 34404, 35341, 35374; ES 6.8 also 34408.
• Two derivations of a numeric common type (Two tables differing only by qualifier collapse into one alias, silently dropping an alias #292, pre-existing). CASE / COALESCE take theirs through leastCommonSuperType, GREATEST / LEAST through numericType (the PostgreSQL rule). They can part for a NUMERIC beside a whole number.
• DATETIME is two classes (pre-existing). A string cast to DATETIME renders a LocalDateTime, while a DATETIME doc value or arm is a ZonedDateTime.
• Processor twins of RCV-2 / RCV-3 (unchanged by ruling 3). In an ingest processor, 11 of 13 RCV-2 renders still carry a NULL constant, and the 10 RCV-3 renders still parse a date-only keyword with ISO_DATE_TIME.
• A cast to BIGINT over a non-temporal function of a date converts from the column's TIMESTAMP (pre-existing, rendered, not run). CAST(DATE_FORMAT(CAST(d AS DATE), '%Y') AS BIGINT) renders .toInstant().toEpochMilli() on a String, and CAST(YEAR(CAST(ts AS DATE)) AS BIGINT) the same on a number. The 5th commit does not change them.
• IFNULL and NVL do not parse in core. Neither the repository's documentation nor the website lists them as supported; COALESCE is their spelling.

Release notes

• CASE, COALESCE, NULLIF, GREATEST and LEAST answer one class per column: the type they report, DuckDB's common type of their arms. Arms with no common type behave as before.
• LAST_DAY answers a DATE, a date_nanos operand included; it reported its operand's type before.
• A NULL literal argument of NULLIF, COALESCE, LEAST or GREATEST is folded. Such statements compile and answer NULL-skipping values.
• CAST(<string> AS TIMESTAMP | DATETIME) reads a date-only value as its midnight UTC.
• CAST(<DATE> AS BIGINT) answers the epoch milliseconds of its UTC midnight, and CAST(<DATE> AS TIME) 00:00:00, for a cast to DATE, CURRENT_DATE, a DATE literal, LAST_DAY and a DATE conditional. A DATE column answers as before.
• Query context only: ingest pipelines and materialized-view computed fields render as before.
• Reported types are unchanged except:
  • LAST_DAY (now DATE);
  • a conditional over arms whose common type differs from their column's runtime type;
  • the consumers whose type follows such a conditional: DATE_ADD(COALESCE(d, d2), INTERVAL 1 DAY) TIMESTAMP -> DATE;
  • DATE_ADD / DATE_SUB of whole days over a DATE (DATE_ADD(d, INTERVAL 1 DAY): TIMESTAMP -> DATE), and a sub-day DATETIME_ADD over a DATE (DATETIME_ADD(GREATEST(<DATE>, <DATE>), INTERVAL 1 HOUR), DATETIME_ADD(CAST(ts AS DATE), INTERVAL 1 HOUR): DATE -> TIMESTAMP). A materialized view computing one declares the new type; DATE and TIMESTAMP map to the same date field.
• HOUR / MINUTE / SECOND / OFFSET_SECONDS / EXTRACT of a DATE answer 0 (its UTC midnight); HOUR(CAST(d AS DATE)) failed on main. A sub-day interval over a DATE answers its TIMESTAMP.
• A CASE or COALESCE whose arms have no common type is refused with DuckDB's reason; it answered two classes, or failed in Painless.
• ASIN / ACOS outside [-1, 1], and LOG / LOG10 / SQRT out of their domain, fail the statement with status 400 and DuckDB's reason; they answered NaN / -Infinity.
• No case-class arity change: no downstream rebuild is needed for binary compatibility.

🤖 Generated with Claude Code