Skip to content

Preserve declared scalar types when reading SQLite values - #89

Merged
xrotwang merged 1 commit into
cldf:masterfrom
antonsoo:fix/sqlite-declared-scalar-types
Oct 28, 2026
Merged

xrotwang merged 1 commit into
cldf:masterfrom
antonsoo:fix/sqlite-declared-scalar-types

Conversation

@antonsoo

Copy link
Copy Markdown
Contributor

Database.read() applies the write converter a second time, returning integers for boolean columns and floats for decimal columns. For example:

import json
from csvw import TableGroup
from csvw.db import Database

tg = TableGroup.fromvalue({'tables': [{'url': 'data', 'tableSchema': {
    'columns': [{'name': 'flag', 'datatype': 'boolean'}]}}]})
db = Database(tg)
db.write(data=[{'flag': True}, {'flag': False}])
print(json.dumps(db.read()['data']))

Before: [{"flag": 1}, {"flag": 0}]. After: [{"flag": true}, {"flag": false}].

Use the existing DBType.read converter for scalar reads. Add type-sensitive boolean/decimal regressions, including nulls; equality alone misses this because True == 1. Decimal storage keeps SQLite's existing REAL precision.

Validation: full suites pass on Python 3.12 (841 passed, 16 expected failures) and 3.9 (844 passed, 13 expected failures), with 100% configured coverage. Changed source/test Flake8 passes. Shipped Frictionless CSV data and exactly representable decimal values exercise CSV/direct writes, renamed columns and memory/file SQLite: all 12 typed pipelines are repaired, while eight complete non-boolean/table controls and raw SQLite row hashes remain exact. A fresh installed wheel reproduces the results. The Frictionless fixture's unrelated foreign-key metadata is excluded explicitly; its CSV bytes are unchanged.

@xrotwang
xrotwang merged commit 1ba9b5b into cldf:master Oct 28, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants