diff --git a/CN/modules/ROOT/nav.adoc b/CN/modules/ROOT/nav.adoc index 7ed31aff..acd391a2 100644 --- a/CN/modules/ROOT/nav.adoc +++ b/CN/modules/ROOT/nav.adoc @@ -32,6 +32,7 @@ ** xref:master/oracle_compatibility/compat_stragg.adoc[23、STRAGG 函数] ** xref:master/oracle_compatibility/compat_alter_index_unusable.adoc[24、禁用索引] ** xref:master/oracle_compatibility/compat_dbtimezone.adoc[25、dbtimezone] +** xref:master/oracle_compatibility/to_number_scientific_notation.adoc[26、TO_NUMBER 支持科学计数法] * 容器化与云服务 ** 容器化指南 *** xref:master/containerization/k8s_deployment.adoc[K8S部署] @@ -119,6 +120,7 @@ **** xref:master/oracle_builtin_functions/stragg.adoc[stragg] **** xref:master/oracle_builtin_functions/dbtimezone_impl.adoc[dbtimezone] **** xref:master/oracle_builtin_functions/vsize.adoc[vsize] +**** xref:master/oracle_builtin_functions/to_number_scientific_notation.adoc[TO_NUMBER 科学计数法支持] *** xref:master/gb18030.adoc[国标GB18030] * 参考指南 ** xref:master/tools_reference.adoc[工具参考] diff --git a/CN/modules/ROOT/pages/master/oracle_builtin_functions/to_number_scientific_notation.adoc b/CN/modules/ROOT/pages/master/oracle_builtin_functions/to_number_scientific_notation.adoc new file mode 100644 index 00000000..930e348d --- /dev/null +++ b/CN/modules/ROOT/pages/master/oracle_builtin_functions/to_number_scientific_notation.adoc @@ -0,0 +1,84 @@ +:sectnums: +:sectnumlevels: 5 + += TO_NUMBER 科学计数法支持设计 + +== 目标与范围 + +本特性扩展 `sys.to_number(text)` 的字符串解析，使其接受科学计数法及首尾普通空格。结果仍通过数值类型输入函数生成，不经过浮点转换。用户示例见 xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER 支持科学计数法]。 + +改动位于不指定格式模型的路径。双参数调用继续使用原有格式模型处理逻辑，不增加 `EEEE` 等格式元素的输入支持，也不增加新的 SQL 函数签名。 + +== 代码组织与调用关系 + +[cols="3,3",options="header"] +|=== +|源码路径 |职责 +|`contrib/ivorysql_ora/src/builtin_functions/builtin_functions--1.0.sql` |将 `sys.to_number(text)` 和 `sys.to_number(text,text)` 注册到 C 入口 `ora_to_number` +|`contrib/ivorysql_ora/src/builtin_functions/numeric_datatype_functions.c` |根据参数个数取得字符串和可选格式模型，调用 `ora_to_number_internal` +|`src/backend/utils/adt/formatting.c` |解析字符串，计算精度及标度，调用 `numeric_in` +|`contrib/ivorysql_ora/sql/ora_number.sql` |科学计数法、空格及异常输入的回归用例 +|`contrib/ivorysql_ora/expected/ora_number.out` |对应的结果和错误信息 +|=== + +[source,text] +---- +sys.to_number(text) + → ora_to_number(value, fmt = NULL) + → ora_to_number_internal(value, fmt) + → numeric_in(numstr, InvalidOid, typmod) + → NUMBER +---- + +两个字符串入口均声明为 `STRICT`、`IMMUTABLE` 和 `PARALLEL SAFE`。`NULL` 参数由 `STRICT` 在进入 C 函数前处理；Oracle 空字符串转 `NULL` 的语义也在解析器之外生效。 + +== 无格式模型解析 + +`ora_to_number_internal()` 在 `fmt == NULL` 时，将输入转为 C 字符串，并使用指针逐段扫描： + +. 跳过首部 ASCII 空格，读取可选的尾数符号。 +. 扫描尾数数字和至多一个小数点，分别记录整数位数 `int_digits` 与小数位数 `frac_digits`，并记录是否存在数字和非零数字。 +. 若遇到 `e` 或 `E`，读取可选指数符号和全部指数数字；指数必须至少有一位数字。 +. 跳过尾部 ASCII 空格，检查剩余字符及尾数数字是否有效。 +. 根据指数计算数值类型修饰符，并调用 `numeric_in()` 完成数值构造及范围检查。 + +扫描不使用通用空白字符判断，因此制表符和换行不会被当作首尾空格忽略。指数数字全部被扫描，避免将 `'1e0000000010'` 的指数错误截断。 + +== 精度与标度 + +设尾数整数位数为 `I`、小数位数为 `F`、带符号指数为 `E`，初步计算： + +[source,text] +---- +scale = max(0, F - E) +precision = max(I + F + max(0, E - F), scale) +---- + +随后将 `precision` 限制到 `NUMERIC_MAX_PRECISION`（当前为 1000），再保证 `scale` 不超过 `precision`。符号和小数点不计入数字位数。 + +[cols="2,2,2",options="header"] +|=== +|输入 |推导的类型修饰符 |结果 +|`1e3` |`numeric(4,0)` |`1000` +|`-1.5e-2` |`numeric(3,3)` |`-0.015` +|`1.5e+3` |`numeric(4,0)` |`1500` +|`1e0000000010` |`numeric(11,0)` |`10000000000` +|=== + +原字符串与推导的类型修饰符传给 `numeric_in()`，由十进制数值路径处理精度、标度和溢出；这里没有将尾数先转换为 `float8`。 + +== 指数边界与错误处理 + +指数绝对值以 `int64` 累积，以 `PG_INT32_MAX / 2` 为解析上界。超过上界后停止累积，但仍扫描指数数字： + +* 正指数且尾数存在非零数字时，报 `value overflows numeric format`，SQLSTATE 为 `22003`。 +* 超大负指数或全零尾数设置归零标志；通过后续格式检查后，将输入替换为字符串 `"0"` 再调用 `numeric_in()`。 +* 上界内的数值继续走普通数值转换。限制精度后仍放不下的数值由 `numeric_in()` 报错，例如 `'1e2000'`。 + +缺少尾数或指数数字、重复小数点、内部空格以及多余字符等通常报 `invalid number format model`，SQLSTATE 为 `0A000`。解析错误与数值范围错误分别处理，保留各自的诊断信息。 + +== 回归验证 + +`ora_number` 回归用例覆盖大小写指数、正负指数、尾数符号、小数位、指数前导零、首尾空格、零指数、单侧小数点、全零尾数、指数边界、精度限制、下溢归零及无效输入。空字符串和特殊值也有独立用例。 + +在已配置并编译的 IvorySQL 源码目录中，可运行 `make -C contrib/ivorysql_ora oracle-installcheck ORA_REGRESS=ora_number` 验证该测试组；目标实例需已安装相应版本的 `ivorysql_ora` 扩展。 diff --git a/CN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc b/CN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc index 9c3d170d..de79cd65 100644 --- a/CN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc +++ b/CN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc @@ -758,6 +758,9 @@ select * from t_regexp_like where regexp_like(value,'ABC','i'); ``` === `to_number` 函数 + +不指定格式模型时，支持科学计数法及首尾普通空格，详见 xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER 支持科学计数法]。 + 功能：是将一些处理过的按一定格式编排过的字符串变回数值型的格式，支持参数：text/text,text测试用例如下： 将字符串'-34,338,492'转换为数值型格式： diff --git a/CN/modules/ROOT/pages/master/oracle_compatibility/to_number_scientific_notation.adoc b/CN/modules/ROOT/pages/master/oracle_compatibility/to_number_scientific_notation.adoc new file mode 100644 index 00000000..c059000c --- /dev/null +++ b/CN/modules/ROOT/pages/master/oracle_compatibility/to_number_scientific_notation.adoc @@ -0,0 +1,78 @@ +:sectnums: +:sectnumlevels: 5 + += TO_NUMBER 支持科学计数法 + +== 概述 + +在不指定格式模型时，Oracle 兼容函数 `TO_NUMBER` 可以将科学计数法字符串转换为 `NUMBER`，例如将 `'1e3'` 转换为 `1000`，将 `'-1.5e-2'` 转换为 `-0.015`。转换也接受字符串首尾的普通空格。 + +本文介绍单参数字符串转换。带格式模型的调用见 xref:master/oracle_compatibility/builtin_types_functions.adoc[内置数据类型与内置函数]；本特性不扩展格式模型的支持范围。 + +== 调用方式 + +[source,sql] +---- +TO_NUMBER(str) +---- + +`str` 是表示数值的字符串，返回类型为 `NUMBER`。在 Oracle 兼容模式下可直接使用 `TO_NUMBER`；也可显式调用 `sys.to_number(str)`。 + +科学计数法由尾数和指数构成，其数值等于“尾数 × 10 的指数次方”： + +* 尾数可以带一个前置 `+` 或 `-`，并包含至多一个小数点，且必须至少有一位数字。 +* 指数以 `e` 或 `E` 开始，可以带一个 `+` 或 `-`，后面必须有整数数字。 +* 指数允许前导零；`e0`、`e+0` 和 `e-0` 均表示零指数。 +* 字符串首尾可以包含普通空格（ASCII 空格）；内部空格、制表符和换行不属于允许的格式。 + +== 转换示例 + +[cols="3,2",options="header"] +|=== +|调用 |结果 +|`TO_NUMBER('1e3')` |`1000` +|`TO_NUMBER('1E3')` |`1000` +|`TO_NUMBER('-1.5e-2')` |`-0.015` +|`TO_NUMBER('1.5e+3')` |`1500` +|`TO_NUMBER('1E-3')` |`0.001` +|`TO_NUMBER(' 1e3 ')` |`1000` +|`TO_NUMBER('12 ')` |`12` +|`TO_NUMBER('1e0000000010')` |`10000000000` +|`TO_NUMBER('1e-0000000010')` |`0.0000000001` +|`TO_NUMBER('123.')` |`123` +|`TO_NUMBER('.5')` |`0.5` +|=== + +[source,sql] +---- +SELECT TO_NUMBER('1e3') AS integer_value, + TO_NUMBER('-1.5e-2') AS decimal_value; +---- + +结果分别为 `1000` 和 `-0.015`。无需先将字符串转换为浮点类型。 + +== 无效输入与边界 + +缺少指数数字、重复符号或小数点、内部空格以及多余字符会导致 `invalid number format model` 错误。例如： + +[source,sql] +---- +SELECT TO_NUMBER('1e'); -- 指数缺少数字 +SELECT TO_NUMBER('1e+'); -- 指数符号后缺少数字 +SELECT TO_NUMBER('1 e3'); -- 尾数与指数之间有空格 +SELECT TO_NUMBER('1e 3'); -- 指数中有空格 +SELECT TO_NUMBER('1.2e3.4'); -- 指数不是整数 +---- + +仅包含空格、单独的符号或小数点，以及 `NaN`、`Infinity`、`-Infinity` 同样不是此转换路径支持的输入。`NULL` 参数返回 `NULL`；Oracle 空字符串语义下的 `TO_NUMBER(''::varchar2)` 也返回 `NULL`，但全空格字符串会报错。 + +超大指数的结果受 IvorySQL 数值范围约束，不能据此假定与 Oracle 的全部数值边界相同： + +[cols="3,2",options="header"] +|=== +|调用 |行为 +|`TO_NUMBER('1e2000')` |报 `numeric field overflow` 错误 +|`TO_NUMBER('1e2147483648')` |报 `value overflows numeric format` 错误 +|`TO_NUMBER('1e-2147483648')` |下溢为 `0` +|`TO_NUMBER('0e2147483648')` |返回 `0` +|=== diff --git a/EN/modules/ROOT/nav.adoc b/EN/modules/ROOT/nav.adoc index 594bda81..cf1f6ba1 100644 --- a/EN/modules/ROOT/nav.adoc +++ b/EN/modules/ROOT/nav.adoc @@ -31,7 +31,8 @@ ** xref:master/oracle_compatibility/compat_create_index_online.adoc[22、ONLINE Parameter for CREATE INDEX] ** xref:master/oracle_compatibility/compat_stragg.adoc[23、STRAGG function] ** xref:master/oracle_compatibility/compat_alter_index_unusable_en.adoc[24、Alter Index Unusable] -** xref:master/oracle_compatibility/compat_dbtimezone_en.adoc[24、dbtimezone] +** xref:master/oracle_compatibility/compat_dbtimezone_en.adoc[25、dbtimezone] +** xref:master/oracle_compatibility/to_number_scientific_notation.adoc[26、TO_NUMBER Scientific Notation] * Containerization and Cloud Service ** Containerization *** xref:master/containerization/k8s_deployment.adoc[K8S deployment] @@ -119,6 +120,7 @@ *** xref:master/oracle_builtin_functions/stragg.adoc[stragg] *** xref:master/oracle_builtin_functions/dbtimezone_impl_en.adoc[dbtimezone] *** xref:master/oracle_builtin_functions/vsize_en.adoc[vsize] +*** xref:master/oracle_builtin_functions/to_number_scientific_notation.adoc[TO_NUMBER Scientific Notation] ** xref:master/gb18030.adoc[GB18030 Character Set] * Reference ** xref:master/tools_reference.adoc[Tool Reference] diff --git a/EN/modules/ROOT/pages/master/oracle_builtin_functions/to_number_scientific_notation.adoc b/EN/modules/ROOT/pages/master/oracle_builtin_functions/to_number_scientific_notation.adoc new file mode 100644 index 00000000..2094faf5 --- /dev/null +++ b/EN/modules/ROOT/pages/master/oracle_builtin_functions/to_number_scientific_notation.adoc @@ -0,0 +1,84 @@ +:sectnums: +:sectnumlevels: 5 + += Design of TO_NUMBER scientific-notation support + +== Goals and scope + +This feature extends string parsing for `sys.to_number(text)` to accept scientific notation and surrounding ordinary spaces. Results still come from the numeric input function without an intermediate floating-point conversion. For user examples, see xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER Support for Scientific Notation]. + +The change applies to calls without a format model. Two-argument calls retain the existing format-model processing. It adds neither input support for format elements such as `EEEE` nor new SQL function signatures. + +== Code organization and call path + +[cols="3,3",options="header"] +|=== +|Source path |Responsibility +|`contrib/ivorysql_ora/src/builtin_functions/builtin_functions--1.0.sql` |Registers `sys.to_number(text)` and `sys.to_number(text,text)` with the C entry point `ora_to_number` +|`contrib/ivorysql_ora/src/builtin_functions/numeric_datatype_functions.c` |Retrieves the string and optional format model according to the argument count, then calls `ora_to_number_internal` +|`src/backend/utils/adt/formatting.c` |Parses the string, derives precision and scale, and calls `numeric_in` +|`contrib/ivorysql_ora/sql/ora_number.sql` |Regression cases for scientific notation, spaces, and invalid input +|`contrib/ivorysql_ora/expected/ora_number.out` |Expected values and error messages +|=== + +[source,text] +---- +sys.to_number(text) + → ora_to_number(value, fmt = NULL) + → ora_to_number_internal(value, fmt) + → numeric_in(numstr, InvalidOid, typmod) + → NUMBER +---- + +Both string entry points are declared `STRICT`, `IMMUTABLE`, and `PARALLEL SAFE`. `STRICT` handles NULL arguments before the C function runs. Oracle empty-string-to-NULL semantics also take effect outside this parser. + +== Parsing without a format model + +When `fmt == NULL`, `ora_to_number_internal()` converts the input to a C string and scans it with a pointer: + +. Skip leading ASCII spaces and read an optional mantissa sign. +. Scan mantissa digits and at most one decimal point. Count `int_digits` and `frac_digits` separately, and track whether any digit or nonzero digit occurs. +. If `e` or `E` follows, read an optional exponent sign and every exponent digit. At least one exponent digit is required. +. Skip trailing ASCII spaces, then validate remaining characters and the presence of mantissa digits. +. Derive the numeric type modifier from the exponent, then call `numeric_in()` to construct the value and check its range. + +The scan does not use a general whitespace predicate, so tabs and newlines are not ignored at either end. Scanning all exponent digits prevents an input such as `'1e0000000010'` from having its exponent truncated incorrectly. + +== Precision and scale + +Let `I` be the mantissa's integer digit count, `F` its fractional digit count, and `E` the signed exponent. The initial calculation is: + +[source,text] +---- +scale = max(0, F - E) +precision = max(I + F + max(0, E - F), scale) +---- + +The implementation then caps `precision` at `NUMERIC_MAX_PRECISION` (currently 1000) and ensures that `scale` does not exceed `precision`. Signs and decimal points do not count as digits. + +[cols="2,2,2",options="header"] +|=== +|Input |Derived type modifier |Result +|`1e3` |`numeric(4,0)` |`1000` +|`-1.5e-2` |`numeric(3,3)` |`-0.015` +|`1.5e+3` |`numeric(4,0)` |`1500` +|`1e0000000010` |`numeric(11,0)` |`10000000000` +|=== + +The original string and derived type modifier go to `numeric_in()`. This decimal numeric path handles precision, scale, and overflow without first converting the mantissa to `float8`. + +== Exponent boundaries and errors + +The exponent magnitude accumulates in `int64`, with `PG_INT32_MAX / 2` as the parser limit. Once the limit is exceeded, accumulation stops but exponent digits are still scanned: + +* A positive exponent with a nonzero mantissa raises `value overflows numeric format`, with SQLSTATE `22003`. +* An oversized negative exponent or an all-zero mantissa sets a zero-result flag. After subsequent format validation, the input is replaced by the string `"0"` before calling `numeric_in()`. +* Values within the parser limit follow normal numeric conversion. Values that still exceed the capped precision are rejected by `numeric_in()`, as with `'1e2000'`. + +Missing mantissa or exponent digits, repeated decimal points, internal spaces, and extra characters generally raise `invalid number format model`, with SQLSTATE `0A000`. Parsing errors and numeric range errors retain separate diagnostics. + +== Regression verification + +The `ora_number` regression cases cover lowercase and uppercase exponents, positive and negative exponents, mantissa signs, fractional digits, leading exponent zeros, surrounding spaces, zero exponents, one-sided decimal points, all-zero mantissas, exponent boundaries, precision limits, underflow to zero, and invalid input. Empty strings and special values have separate cases. + +From a configured and built IvorySQL source tree, run `make -C contrib/ivorysql_ora oracle-installcheck ORA_REGRESS=ora_number` to verify this test group. The target instance must have the matching version of the `ivorysql_ora` extension installed. diff --git a/EN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc b/EN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc index 8150dedf..bde1e77c 100644 --- a/EN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc +++ b/EN/modules/ROOT/pages/master/oracle_compatibility/builtin_types_functions.adoc @@ -759,6 +759,9 @@ select * from t_regexp_like where regexp_like(value,'ABC','i'); ``` === `to_number` function + +Without a format model, scientific notation and surrounding ordinary spaces are supported. See xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER Support for Scientific Notation]. + function: is to change some processed strings arranged in a certain format back to a numeric format, support parameters: text/text, text test cases are as follows: Convert the string '-34,338,492' to numeric format: diff --git a/EN/modules/ROOT/pages/master/oracle_compatibility/to_number_scientific_notation.adoc b/EN/modules/ROOT/pages/master/oracle_compatibility/to_number_scientific_notation.adoc new file mode 100644 index 00000000..b1a57055 --- /dev/null +++ b/EN/modules/ROOT/pages/master/oracle_compatibility/to_number_scientific_notation.adoc @@ -0,0 +1,78 @@ +:sectnums: +:sectnumlevels: 5 + += TO_NUMBER support for scientific notation + +== Overview + +Without a format model, the Oracle-compatible `TO_NUMBER` function converts scientific-notation strings to `NUMBER`. For example, `'1e3'` becomes `1000`, and `'-1.5e-2'` becomes `-0.015`. The conversion also accepts ordinary spaces around the string. + +This page covers the single-argument string conversion. See xref:master/oracle_compatibility/builtin_types_functions.adoc[Built-in Data Types and Functions] for calls with a format model; this feature does not extend format-model support. + +== Syntax + +[source,sql] +---- +TO_NUMBER(str) +---- + +`str` is a string representing a number. The return type is `NUMBER`. In Oracle compatibility mode, call `TO_NUMBER` directly, or explicitly call `sys.to_number(str)`. + +Scientific notation consists of a mantissa and an exponent. Its value is the mantissa multiplied by 10 raised to the exponent: + +* The mantissa can have one leading `+` or `-` and at most one decimal point. It must contain at least one digit. +* The exponent begins with `e` or `E`, optionally followed by one `+` or `-`, and must contain integer digits. +* Leading zeros are allowed in the exponent. `e0`, `e+0`, and `e-0` all mean a zero exponent. +* Ordinary spaces (ASCII spaces) are allowed at either end of the string. Internal spaces, tabs, and newlines are outside the accepted format. + +== Conversion examples + +[cols="3,2",options="header"] +|=== +|Call |Result +|`TO_NUMBER('1e3')` |`1000` +|`TO_NUMBER('1E3')` |`1000` +|`TO_NUMBER('-1.5e-2')` |`-0.015` +|`TO_NUMBER('1.5e+3')` |`1500` +|`TO_NUMBER('1E-3')` |`0.001` +|`TO_NUMBER(' 1e3 ')` |`1000` +|`TO_NUMBER('12 ')` |`12` +|`TO_NUMBER('1e0000000010')` |`10000000000` +|`TO_NUMBER('1e-0000000010')` |`0.0000000001` +|`TO_NUMBER('123.')` |`123` +|`TO_NUMBER('.5')` |`0.5` +|=== + +[source,sql] +---- +SELECT TO_NUMBER('1e3') AS integer_value, + TO_NUMBER('-1.5e-2') AS decimal_value; +---- + +The results are `1000` and `-0.015`, respectively. No intermediate conversion to a floating-point type is needed. + +== Invalid input and boundaries + +Missing exponent digits, repeated signs or decimal points, internal spaces, and extra characters cause an `invalid number format model` error. For example: + +[source,sql] +---- +SELECT TO_NUMBER('1e'); -- missing exponent digits +SELECT TO_NUMBER('1e+'); -- no digits after the exponent sign +SELECT TO_NUMBER('1 e3'); -- space between mantissa and exponent +SELECT TO_NUMBER('1e 3'); -- space inside the exponent +SELECT TO_NUMBER('1.2e3.4'); -- exponent is not an integer +---- + +All-space strings, a sign or decimal point alone, and `NaN`, `Infinity`, and `-Infinity` are also unsupported in this conversion path. A `NULL` argument returns `NULL`. Under Oracle empty-string semantics, `TO_NUMBER(''::varchar2)` also returns `NULL`, whereas an all-space string raises an error. + +Very large exponents are subject to IvorySQL's numeric limits. These behaviors do not imply that all numeric boundaries match Oracle's: + +[cols="3,2",options="header"] +|=== +|Call |Behavior +|`TO_NUMBER('1e2000')` |Raises `numeric field overflow` +|`TO_NUMBER('1e2147483648')` |Raises `value overflows numeric format` +|`TO_NUMBER('1e-2147483648')` |Underflows to `0` +|`TO_NUMBER('0e2147483648')` |Returns `0` +|===