Skip to content
bmsuissePublic

About

No description, website, or topics provided.

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

arrowsql

Fast SQL Server driver for Python with a Rust core. Drop-in for mssql-python, Arrow-native.

PyPI Python versions CI MIT license

  • 2-7x faster than mssql-python, up to 10x faster than pyodbc on large reads.
  • Arrow out of the box: results go straight to DuckDB, polars, pyarrow and Delta, no pyarrow required.
  • Parallel reads and bulk loads over several connections.
  • No ODBC driver: native TDS in Rust, zero required dependencies, wheels for Linux, macOS and Windows.
  • Drop-in: same DB-API, same connection strings; import mssql_python keeps working.

Install

pip install arrowsql

Extras: arrowsql[arrow] (pyarrow for cursor.arrow()), arrowsql[azure] (Entra ID login). Do not install it next to Microsoft's mssql-python: both provide the mssql_python package.

Quick start

import arrowsql

conn = arrowsql.connect("Server=myserver;Database=sales;Uid=app;Pwd=...;Encrypt=yes")
cur = conn.cursor()
cur.execute("SELECT id, total FROM dbo.orders WHERE customer_id = ?", (42,))
for row in cur.fetchall():
    print(row.id, row.total)

Arrow, DuckDB, polars

import duckdb

t = cur.execute("SELECT * FROM dbo.orders").fetchall_arrow()          # native Arrow table
duckdb.sql("SELECT customer_id, sum(total) FROM t GROUP BY 1").show()

# Large tables: read over 8 connections and stream into DuckDB, deltalake or polars
with arrowsql.read_arrow_stream(conn_str, "SELECT * FROM dbo.orders",
                                partition_on="id", partitions=8) as stream:
    duckdb.sql("SELECT avg(total) FROM stream").show()

String columns travel as UTF-8 on SQL Server 2019+ (about half the bytes). More in examples/.

Bulk loads

cur.bulkcopy_arrow("dbo.orders", arrow_table)                          # Arrow -> TDS bulk load
arrowsql.write_arrow(conn_str, "dbo.orders", arrow_table, workers=8)   # in parallel

Async and agent mode

  • arrowsql.aio: native asyncio with a connection pool; cancelling a task cancels the query (docs/ASYNC.md).
  • arrowsql-agent -c "SELECT ...": bounded, typed output and JSON errors for LLM agents (docs/AGENT_MODE.md).

Benchmarks

NYC taxi data (2.96M rows x 20 columns), SQL Server 2022 on the same host. Compare ratios: the box was shared.

mssql-python arrowsql pyodbc
fetchall(), 100k rows 0.351 s 0.117 s 0.865 s
fetchmany, 3M rows 9.74 s 2.75 s 27.0 s
Arrow, 3M rows over 8 connections 4.25 s 0.59 s –
Bulk load, 3M rows (8 workers) 12.35 s 1.82 s –
executemany, 100k rows 1.26 s 0.65 s 0.76 s
16 threads x point lookups 2.40 s 0.43 s –

Reproduce: scripts/bench_reproduce.sh. Full matrix: benchmarks/RESULTS.md.

Docs

Compatibility · Platforms and connection strings · Features · Architecture · Performance notes · Contributing

License

MIT. Builds on microsoft/mssql-python and mssql-tds (both MIT).

About

No description, website, or topics provided.

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages