Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CN/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
** xref:master/oracle_compatibility/compat_stragg.adoc[23、STRAGG 函数]
** xref:master/oracle_compatibility/compat_alter_index_unusable.adoc[24、禁用索引]
** xref:master/oracle_compatibility/compat_dbtimezone.adoc[25、dbtimezone]
** xref:master/oracle_compatibility/to_number_scientific_notation.adoc[26、TO_NUMBER 支持科学计数法]
* 容器化与云服务
** 容器化指南
*** xref:master/containerization/k8s_deployment.adoc[K8S部署]
Expand Down Expand Up @@ -119,6 +120,7 @@
**** xref:master/oracle_builtin_functions/stragg.adoc[stragg]
**** xref:master/oracle_builtin_functions/dbtimezone_impl.adoc[dbtimezone]
**** xref:master/oracle_builtin_functions/vsize.adoc[vsize]
**** xref:master/oracle_builtin_functions/to_number_scientific_notation.adoc[TO_NUMBER 科学计数法支持]
*** xref:master/gb18030.adoc[国标GB18030]
* 参考指南
** xref:master/tools_reference.adoc[工具参考]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
:sectnums:
:sectnumlevels: 5

= TO_NUMBER 科学计数法支持设计

== 目标与范围

本特性扩展 `sys.to_number(text)` 的字符串解析,使其接受科学计数法及首尾普通空格。结果仍通过数值类型输入函数生成,不经过浮点转换。用户示例见 xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER 支持科学计数法]。

改动位于不指定格式模型的路径。双参数调用继续使用原有格式模型处理逻辑,不增加 `EEEE` 等格式元素的输入支持,也不增加新的 SQL 函数签名。

== 代码组织与调用关系

[cols="3,3",options="header"]
|===
|源码路径 |职责
|`contrib/ivorysql_ora/src/builtin_functions/builtin_functions--1.0.sql` |将 `sys.to_number(text)` 和 `sys.to_number(text,text)` 注册到 C 入口 `ora_to_number`
|`contrib/ivorysql_ora/src/builtin_functions/numeric_datatype_functions.c` |根据参数个数取得字符串和可选格式模型,调用 `ora_to_number_internal`
|`src/backend/utils/adt/formatting.c` |解析字符串,计算精度及标度,调用 `numeric_in`
|`contrib/ivorysql_ora/sql/ora_number.sql` |科学计数法、空格及异常输入的回归用例
|`contrib/ivorysql_ora/expected/ora_number.out` |对应的结果和错误信息
|===

[source,text]
----
sys.to_number(text)
→ ora_to_number(value, fmt = NULL)
→ ora_to_number_internal(value, fmt)
→ numeric_in(numstr, InvalidOid, typmod)
→ NUMBER
----

两个字符串入口均声明为 `STRICT`、`IMMUTABLE` 和 `PARALLEL SAFE`。`NULL` 参数由 `STRICT` 在进入 C 函数前处理;Oracle 空字符串转 `NULL` 的语义也在解析器之外生效。

== 无格式模型解析

`ora_to_number_internal()` 在 `fmt == NULL` 时,将输入转为 C 字符串,并使用指针逐段扫描:

. 跳过首部 ASCII 空格,读取可选的尾数符号。
. 扫描尾数数字和至多一个小数点,分别记录整数位数 `int_digits` 与小数位数 `frac_digits`,并记录是否存在数字和非零数字。
. 若遇到 `e` 或 `E`,读取可选指数符号和全部指数数字;指数必须至少有一位数字。
. 跳过尾部 ASCII 空格,检查剩余字符及尾数数字是否有效。
. 根据指数计算数值类型修饰符,并调用 `numeric_in()` 完成数值构造及范围检查。

扫描不使用通用空白字符判断,因此制表符和换行不会被当作首尾空格忽略。指数数字全部被扫描,避免将 `'1e0000000010'` 的指数错误截断。

== 精度与标度

设尾数整数位数为 `I`、小数位数为 `F`、带符号指数为 `E`,初步计算:

[source,text]
----
scale = max(0, F - E)
precision = max(I + F + max(0, E - F), scale)
----

随后将 `precision` 限制到 `NUMERIC_MAX_PRECISION`(当前为 1000),再保证 `scale` 不超过 `precision`。符号和小数点不计入数字位数。

[cols="2,2,2",options="header"]
|===
|输入 |推导的类型修饰符 |结果
|`1e3` |`numeric(4,0)` |`1000`
|`-1.5e-2` |`numeric(3,3)` |`-0.015`
|`1.5e+3` |`numeric(4,0)` |`1500`
|`1e0000000010` |`numeric(11,0)` |`10000000000`
|===

原字符串与推导的类型修饰符传给 `numeric_in()`,由十进制数值路径处理精度、标度和溢出;这里没有将尾数先转换为 `float8`。

== 指数边界与错误处理

指数绝对值以 `int64` 累积,以 `PG_INT32_MAX / 2` 为解析上界。超过上界后停止累积,但仍扫描指数数字:

* 正指数且尾数存在非零数字时,报 `value overflows numeric format`,SQLSTATE 为 `22003`。
* 超大负指数或全零尾数设置归零标志;通过后续格式检查后,将输入替换为字符串 `"0"` 再调用 `numeric_in()`。
* 上界内的数值继续走普通数值转换。限制精度后仍放不下的数值由 `numeric_in()` 报错,例如 `'1e2000'`。

缺少尾数或指数数字、重复小数点、内部空格以及多余字符等通常报 `invalid number format model`,SQLSTATE 为 `0A000`。解析错误与数值范围错误分别处理,保留各自的诊断信息。

== 回归验证

`ora_number` 回归用例覆盖大小写指数、正负指数、尾数符号、小数位、指数前导零、首尾空格、零指数、单侧小数点、全零尾数、指数边界、精度限制、下溢归零及无效输入。空字符串和特殊值也有独立用例。

在已配置并编译的 IvorySQL 源码目录中,可运行 `make -C contrib/ivorysql_ora oracle-installcheck ORA_REGRESS=ora_number` 验证该测试组;目标实例需已安装相应版本的 `ivorysql_ora` 扩展。
Original file line number Diff line number Diff line change
Expand Up @@ -758,6 +758,9 @@ select * from t_regexp_like where regexp_like(value,'ABC','i');
```

=== `to_number` 函数

不指定格式模型时,支持科学计数法及首尾普通空格,详见 xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER 支持科学计数法]。

功能:是将一些处理过的按一定格式编排过的字符串变回数值型的格式,支持参数:text/text,text测试用例如下:
将字符串'-34,338,492'转换为数值型格式:

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
:sectnums:
:sectnumlevels: 5

= TO_NUMBER 支持科学计数法

== 概述

在不指定格式模型时,Oracle 兼容函数 `TO_NUMBER` 可以将科学计数法字符串转换为 `NUMBER`,例如将 `'1e3'` 转换为 `1000`,将 `'-1.5e-2'` 转换为 `-0.015`。转换也接受字符串首尾的普通空格。

本文介绍单参数字符串转换。带格式模型的调用见 xref:master/oracle_compatibility/builtin_types_functions.adoc[内置数据类型与内置函数];本特性不扩展格式模型的支持范围。

== 调用方式

[source,sql]
----
TO_NUMBER(str)
----

`str` 是表示数值的字符串,返回类型为 `NUMBER`。在 Oracle 兼容模式下可直接使用 `TO_NUMBER`;也可显式调用 `sys.to_number(str)`。

科学计数法由尾数和指数构成,其数值等于“尾数 × 10 的指数次方”:

* 尾数可以带一个前置 `+` 或 `-`,并包含至多一个小数点,且必须至少有一位数字。
* 指数以 `e` 或 `E` 开始,可以带一个 `+` 或 `-`,后面必须有整数数字。
* 指数允许前导零;`e0`、`e+0` 和 `e-0` 均表示零指数。
* 字符串首尾可以包含普通空格(ASCII 空格);内部空格、制表符和换行不属于允许的格式。

== 转换示例

[cols="3,2",options="header"]
|===
|调用 |结果
|`TO_NUMBER('1e3')` |`1000`
|`TO_NUMBER('1E3')` |`1000`
|`TO_NUMBER('-1.5e-2')` |`-0.015`
|`TO_NUMBER('1.5e+3')` |`1500`
|`TO_NUMBER('1E-3')` |`0.001`
|`TO_NUMBER(' 1e3 ')` |`1000`
|`TO_NUMBER('12 ')` |`12`
|`TO_NUMBER('1e0000000010')` |`10000000000`
|`TO_NUMBER('1e-0000000010')` |`0.0000000001`
|`TO_NUMBER('123.')` |`123`
|`TO_NUMBER('.5')` |`0.5`
|===

[source,sql]
----
SELECT TO_NUMBER('1e3') AS integer_value,
TO_NUMBER('-1.5e-2') AS decimal_value;
----

结果分别为 `1000` 和 `-0.015`。无需先将字符串转换为浮点类型。

== 无效输入与边界

缺少指数数字、重复符号或小数点、内部空格以及多余字符会导致 `invalid number format model` 错误。例如:

[source,sql]
----
SELECT TO_NUMBER('1e'); -- 指数缺少数字
SELECT TO_NUMBER('1e+'); -- 指数符号后缺少数字
SELECT TO_NUMBER('1 e3'); -- 尾数与指数之间有空格
SELECT TO_NUMBER('1e 3'); -- 指数中有空格
SELECT TO_NUMBER('1.2e3.4'); -- 指数不是整数
----

仅包含空格、单独的符号或小数点,以及 `NaN`、`Infinity`、`-Infinity` 同样不是此转换路径支持的输入。`NULL` 参数返回 `NULL`;Oracle 空字符串语义下的 `TO_NUMBER(''::varchar2)` 也返回 `NULL`,但全空格字符串会报错。

超大指数的结果受 IvorySQL 数值范围约束,不能据此假定与 Oracle 的全部数值边界相同:

[cols="3,2",options="header"]
|===
|调用 |行为
|`TO_NUMBER('1e2000')` |报 `numeric field overflow` 错误
|`TO_NUMBER('1e2147483648')` |报 `value overflows numeric format` 错误
|`TO_NUMBER('1e-2147483648')` |下溢为 `0`
|`TO_NUMBER('0e2147483648')` |返回 `0`
|===
4 changes: 3 additions & 1 deletion EN/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@
** xref:master/oracle_compatibility/compat_create_index_online.adoc[22、ONLINE Parameter for CREATE INDEX]
** xref:master/oracle_compatibility/compat_stragg.adoc[23、STRAGG function]
** xref:master/oracle_compatibility/compat_alter_index_unusable_en.adoc[24、Alter Index Unusable]
** xref:master/oracle_compatibility/compat_dbtimezone_en.adoc[24、dbtimezone]
** xref:master/oracle_compatibility/compat_dbtimezone_en.adoc[25、dbtimezone]
** xref:master/oracle_compatibility/to_number_scientific_notation.adoc[26、TO_NUMBER Scientific Notation]
* Containerization and Cloud Service
** Containerization
*** xref:master/containerization/k8s_deployment.adoc[K8S deployment]
Expand Down Expand Up @@ -119,6 +120,7 @@
*** xref:master/oracle_builtin_functions/stragg.adoc[stragg]
*** xref:master/oracle_builtin_functions/dbtimezone_impl_en.adoc[dbtimezone]
*** xref:master/oracle_builtin_functions/vsize_en.adoc[vsize]
*** xref:master/oracle_builtin_functions/to_number_scientific_notation.adoc[TO_NUMBER Scientific Notation]
** xref:master/gb18030.adoc[GB18030 Character Set]
* Reference
** xref:master/tools_reference.adoc[Tool Reference]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
:sectnums:
:sectnumlevels: 5

= Design of TO_NUMBER scientific-notation support

== Goals and scope

This feature extends string parsing for `sys.to_number(text)` to accept scientific notation and surrounding ordinary spaces. Results still come from the numeric input function without an intermediate floating-point conversion. For user examples, see xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER Support for Scientific Notation].

The change applies to calls without a format model. Two-argument calls retain the existing format-model processing. It adds neither input support for format elements such as `EEEE` nor new SQL function signatures.

== Code organization and call path

[cols="3,3",options="header"]
|===
|Source path |Responsibility
|`contrib/ivorysql_ora/src/builtin_functions/builtin_functions--1.0.sql` |Registers `sys.to_number(text)` and `sys.to_number(text,text)` with the C entry point `ora_to_number`
|`contrib/ivorysql_ora/src/builtin_functions/numeric_datatype_functions.c` |Retrieves the string and optional format model according to the argument count, then calls `ora_to_number_internal`
|`src/backend/utils/adt/formatting.c` |Parses the string, derives precision and scale, and calls `numeric_in`
|`contrib/ivorysql_ora/sql/ora_number.sql` |Regression cases for scientific notation, spaces, and invalid input
|`contrib/ivorysql_ora/expected/ora_number.out` |Expected values and error messages
|===

[source,text]
----
sys.to_number(text)
→ ora_to_number(value, fmt = NULL)
→ ora_to_number_internal(value, fmt)
→ numeric_in(numstr, InvalidOid, typmod)
→ NUMBER
----

Both string entry points are declared `STRICT`, `IMMUTABLE`, and `PARALLEL SAFE`. `STRICT` handles NULL arguments before the C function runs. Oracle empty-string-to-NULL semantics also take effect outside this parser.

== Parsing without a format model

When `fmt == NULL`, `ora_to_number_internal()` converts the input to a C string and scans it with a pointer:

. Skip leading ASCII spaces and read an optional mantissa sign.
. Scan mantissa digits and at most one decimal point. Count `int_digits` and `frac_digits` separately, and track whether any digit or nonzero digit occurs.
. If `e` or `E` follows, read an optional exponent sign and every exponent digit. At least one exponent digit is required.
. Skip trailing ASCII spaces, then validate remaining characters and the presence of mantissa digits.
. Derive the numeric type modifier from the exponent, then call `numeric_in()` to construct the value and check its range.

The scan does not use a general whitespace predicate, so tabs and newlines are not ignored at either end. Scanning all exponent digits prevents an input such as `'1e0000000010'` from having its exponent truncated incorrectly.

== Precision and scale

Let `I` be the mantissa's integer digit count, `F` its fractional digit count, and `E` the signed exponent. The initial calculation is:

[source,text]
----
scale = max(0, F - E)
precision = max(I + F + max(0, E - F), scale)
----

The implementation then caps `precision` at `NUMERIC_MAX_PRECISION` (currently 1000) and ensures that `scale` does not exceed `precision`. Signs and decimal points do not count as digits.

[cols="2,2,2",options="header"]
|===
|Input |Derived type modifier |Result
|`1e3` |`numeric(4,0)` |`1000`
|`-1.5e-2` |`numeric(3,3)` |`-0.015`
|`1.5e+3` |`numeric(4,0)` |`1500`
|`1e0000000010` |`numeric(11,0)` |`10000000000`
|===

The original string and derived type modifier go to `numeric_in()`. This decimal numeric path handles precision, scale, and overflow without first converting the mantissa to `float8`.

== Exponent boundaries and errors

The exponent magnitude accumulates in `int64`, with `PG_INT32_MAX / 2` as the parser limit. Once the limit is exceeded, accumulation stops but exponent digits are still scanned:

* A positive exponent with a nonzero mantissa raises `value overflows numeric format`, with SQLSTATE `22003`.
* An oversized negative exponent or an all-zero mantissa sets a zero-result flag. After subsequent format validation, the input is replaced by the string `"0"` before calling `numeric_in()`.
* Values within the parser limit follow normal numeric conversion. Values that still exceed the capped precision are rejected by `numeric_in()`, as with `'1e2000'`.

Missing mantissa or exponent digits, repeated decimal points, internal spaces, and extra characters generally raise `invalid number format model`, with SQLSTATE `0A000`. Parsing errors and numeric range errors retain separate diagnostics.

== Regression verification

The `ora_number` regression cases cover lowercase and uppercase exponents, positive and negative exponents, mantissa signs, fractional digits, leading exponent zeros, surrounding spaces, zero exponents, one-sided decimal points, all-zero mantissas, exponent boundaries, precision limits, underflow to zero, and invalid input. Empty strings and special values have separate cases.

From a configured and built IvorySQL source tree, run `make -C contrib/ivorysql_ora oracle-installcheck ORA_REGRESS=ora_number` to verify this test group. The target instance must have the matching version of the `ivorysql_ora` extension installed.
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,9 @@ select * from t_regexp_like where regexp_like(value,'ABC','i');
```

=== `to_number` function

Without a format model, scientific notation and surrounding ordinary spaces are supported. See xref:master/oracle_compatibility/to_number_scientific_notation.adoc[TO_NUMBER Support for Scientific Notation].

function: is to change some processed strings arranged in a certain format back to a numeric format, support parameters: text/text, text test cases are as follows:
Convert the string '-34,338,492' to numeric format:

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
:sectnums:
:sectnumlevels: 5

= TO_NUMBER support for scientific notation

== Overview

Without a format model, the Oracle-compatible `TO_NUMBER` function converts scientific-notation strings to `NUMBER`. For example, `'1e3'` becomes `1000`, and `'-1.5e-2'` becomes `-0.015`. The conversion also accepts ordinary spaces around the string.

This page covers the single-argument string conversion. See xref:master/oracle_compatibility/builtin_types_functions.adoc[Built-in Data Types and Functions] for calls with a format model; this feature does not extend format-model support.

== Syntax

[source,sql]
----
TO_NUMBER(str)
----

`str` is a string representing a number. The return type is `NUMBER`. In Oracle compatibility mode, call `TO_NUMBER` directly, or explicitly call `sys.to_number(str)`.

Scientific notation consists of a mantissa and an exponent. Its value is the mantissa multiplied by 10 raised to the exponent:

* The mantissa can have one leading `+` or `-` and at most one decimal point. It must contain at least one digit.
* The exponent begins with `e` or `E`, optionally followed by one `+` or `-`, and must contain integer digits.
* Leading zeros are allowed in the exponent. `e0`, `e+0`, and `e-0` all mean a zero exponent.
* Ordinary spaces (ASCII spaces) are allowed at either end of the string. Internal spaces, tabs, and newlines are outside the accepted format.

== Conversion examples

[cols="3,2",options="header"]
|===
|Call |Result
|`TO_NUMBER('1e3')` |`1000`
|`TO_NUMBER('1E3')` |`1000`
|`TO_NUMBER('-1.5e-2')` |`-0.015`
|`TO_NUMBER('1.5e+3')` |`1500`
|`TO_NUMBER('1E-3')` |`0.001`
|`TO_NUMBER(' 1e3 ')` |`1000`
|`TO_NUMBER('12 ')` |`12`
|`TO_NUMBER('1e0000000010')` |`10000000000`
|`TO_NUMBER('1e-0000000010')` |`0.0000000001`
|`TO_NUMBER('123.')` |`123`
|`TO_NUMBER('.5')` |`0.5`
|===

[source,sql]
----
SELECT TO_NUMBER('1e3') AS integer_value,
TO_NUMBER('-1.5e-2') AS decimal_value;
----

The results are `1000` and `-0.015`, respectively. No intermediate conversion to a floating-point type is needed.

== Invalid input and boundaries

Missing exponent digits, repeated signs or decimal points, internal spaces, and extra characters cause an `invalid number format model` error. For example:

[source,sql]
----
SELECT TO_NUMBER('1e'); -- missing exponent digits
SELECT TO_NUMBER('1e+'); -- no digits after the exponent sign
SELECT TO_NUMBER('1 e3'); -- space between mantissa and exponent
SELECT TO_NUMBER('1e 3'); -- space inside the exponent
SELECT TO_NUMBER('1.2e3.4'); -- exponent is not an integer
----

All-space strings, a sign or decimal point alone, and `NaN`, `Infinity`, and `-Infinity` are also unsupported in this conversion path. A `NULL` argument returns `NULL`. Under Oracle empty-string semantics, `TO_NUMBER(''::varchar2)` also returns `NULL`, whereas an all-space string raises an error.

Very large exponents are subject to IvorySQL's numeric limits. These behaviors do not imply that all numeric boundaries match Oracle's:

[cols="3,2",options="header"]
|===
|Call |Behavior
|`TO_NUMBER('1e2000')` |Raises `numeric field overflow`
|`TO_NUMBER('1e2147483648')` |Raises `value overflows numeric format`
|`TO_NUMBER('1e-2147483648')` |Underflows to `0`
|`TO_NUMBER('0e2147483648')` |Returns `0`
|===
Loading