Managed tables

architecture/client-architecture.mdx

@@ -27,7 +27,7 @@ The app UI can also use [Live Queries / Watch Queries](/client-sdks/watch-querie
 
 When you implement the PowerSync Client SDK in your application, you need to define a [client-side schema](/intro/setup-guide#define-your-client-side-schema) with tables, columns and indexes that correspond to your [Sync Streams](/sync/streams/overview). You provide this schema when the PowerSync-managed SQLite database is [instantiated](/intro/setup-guide#instantiate-the-powersync-database).
 
-The tables defined in your client-side schema are usable in SQL queries as if they were actual SQLite tables, while in reality they are created as _SQLite views_ based on the schemaless JSON data being synced (see [PowerSync Protocol](/architecture/powersync-protocol)).
+By default, the tables defined in your client-side schema are usable in SQL queries as if they were actual SQLite tables, while in reality they are created as _SQLite views_ based on the schemaless JSON data being synced (see [PowerSync Protocol](/architecture/powersync-protocol)).
 
 The PowerSync Client SDK automatically maintains the following tables in the SQLite database:
 
@@ -36,6 +36,7 @@ The PowerSync Client SDK automatically maintains the following tables in the SQL
 | `ps_data__<table>` | This contains the data for each "table", in JSON format. Since JSON is being used, this table's schema does not change when columns are added, removed or changed in the Sync Streams and client-side schema. |
 | `ps_data_local__<table>` | Same as the previous point, but for [local-only](/client-sdks/advanced/local-only-usage) tables. |
 | `<table>` (`VIEW`) | These are views on the above `ps_data` tables, with each defined column in the client-side schema extracted from the JSON. For example, a `description` text column would be `CAST(data ->> '$.description' as TEXT)`. |
+| `<table>` (`TABLE`) | For [managed tables](/client-sdks/advanced/performance-optimization#managed-tables), PowerSync replaces the `ps_data__` storage table and view with a single table matching your schema. |
 | `ps_untyped` | Any synced table that is not defined in the client-side schema is placed here. If the table is added to the schema at a later point, the data is then migrated to `ps_data__<table>`. |
 | `ps_oplog` | This is the operation history data as received from the [PowerSync Service](/architecture/powersync-service), grouped per bucket. |
 | `ps_crud` | The client-side upload queue (see [Writing Data](#writing-data-via-sqlite-database-and-upload-queue) below) |
@@ -50,7 +51,7 @@ The copy of the row in `ps_oplog` may be newer than the one in `ps_data__<table>
 It is possible for different [buckets](/architecture/powersync-service#bucket-system) to include overlapping data (for example, if multiple buckets contain data from the same table). If rows with the same table and ID have been synced via multiple buckets, it may be present multiple times in `ps_oplog`, but only one will be preserved in the `ps_data__<table>` table (the one with the highest `op_id`).
 
 <Note>
-  **Raw Tables Instead of JSON-Backed SQLite Views**: If you run into limitations with the above JSON-based SQLite view system, check out the [Raw Tables feature](/client-sdks/advanced/raw-tables) which allows you to define and manage raw SQLite tables to work around some of the limitations of PowerSync's default JSON-backed SQLite views system.
+  **Alternatives to JSON-based views:** [Managed tables](/client-sdks/advanced/performance-optimization#managed-tables) (alpha) store data in SQLite columns for faster queries, and PowerSync still creates and migrates them. [Raw tables](/client-sdks/advanced/raw-tables) give you full control over the SQLite tables, for example to use foreign keys, but you have to create and migrate them yourself. To compare the options, see [Choosing a Table Type](/client-sdks/advanced/performance-optimization#choosing-a-table-type).
 </Note>
 
 ## Writing Data (via SQLite Database and Upload Queue)

client-sdks/advanced/performance-optimization.mdx

@@ -1,18 +1,18 @@
 ---
-title: "SQLite Performance Optimization"
+title: "SQLite Performance Optimizations"
 description: "Optimize query performance in PowerSync client SDKs"
 ---
 
 PowerSync client SDKs use SQLite, and apply reasonable defaults to optimize performance:
 
 1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) (WAL) is enabled to support concurrent reads and writes. PowerSync also dispatches queries to multiple threads for parallelism. With the JavaScript web SDK, a WAL-like option is [available on Chromium browsers](/client-sdks/reference/javascript-web#2-opfs-based-alternatives).
 2. SDKs also configure connections for performance by enabling a large page cache of around 50MB and setting `pragma synchronous = normal` to avoid frequent fsync operations.
 
 ## Debugging Query Performance
 
 Slow queries can have two causes:
 
 1. The query itself is slow, e.g. because it reads a lot of rows or uses unoptimized joins not backed by an index. The [Fixing Slow Queries](#fixing-slow-queries) section describes how these queries can be optimized.
 2. The database is blocked. There can only be a single writer at a time, and the number of concurrent readers is bound by an option set when opening the database. So a query that appears slow might actually be fast, having run at an unfortunate time when the database was busy.
 
 To understand the problem, having a trace of query runtimes available helps.
@@ -54,10 +54,11 @@
 
 If a query itself is expensive and takes a long time to run, several options can improve performance. Some of these require a restructuring of your app's schema.
 
-1. For auto-updating watched queries on frequently-changed tables, queries might run very often. Watched queries should typically be cheap to run, as they otherwise risk blocking SQLite connections for too long. For more expensive queries that still need to be watched, consider increasing their throttle to run them less often.
-2. As an alternative to regular watched queries, use [High Performance Diffs](/client-sdks/high-performance-diffs), which use triggers internally to only report changed rows back to your app.
-3. Where possible, filter on the `id` column of tables to reduce the number of rows read, which makes queries more efficient. When filtering on other columns of large tables, make sure these columns are covered by indexes declared in your app's schema.
-4. PowerSync tables are views over JSON data that extract columns by parsing from JSON each time a row is accessed. While SQLite has a cache for parsed JSON, this can still be inefficient for queries computing on columns. [Raw Tables](/client-sdks/advanced/raw-tables) let you use plain SQLite tables instead, which are faster to query but require special consideration for migrations.
+1. Where possible, filter on the `id` column of tables to reduce the number of rows read, which makes queries more efficient. When filtering on other columns of large tables, make sure these columns are covered by indexes declared in your app's schema.
+2. For auto-updating watched queries on frequently-changed tables, queries might run very often. Watched queries should typically be cheap to run, as they otherwise risk blocking SQLite connections for too long. For more expensive queries that still need to be watched, consider increasing their throttle to run them less often.
+3. By default, PowerSync tables are views over JSON data that extract columns by parsing from JSON each time a row is accessed. While SQLite has a cache for parsed JSON, this is still slow for queries that read many rows, for example to filter, sort or aggregate. [Managed tables](#managed-tables) store data in SQLite columns instead, at the cost of slower schema changes.
+4. If your app runs the same quick queries many times, SQLite spends part of each run reading the SQL and planning the query again. In the JavaScript Web and Dart SDKs, a [prepared statement cache](#prepared-statement-cache) keeps recently planned queries so that SQLite can reuse them.
+5. In the JavaScript SDKs, you can replace regular watched queries with [High Performance Diffs](/client-sdks/high-performance-diffs). These use triggers internally to report only changed rows back to your app.
 
 ### Prepared Statement Cache
 
@@ -94,3 +95,113 @@
 In both SDKs, each connection uses its own independent cache and the maximum size applies to those caches. When the cache is full, the least-recently-used statement is evicted.
 
 For more information, see the `preparedStatementsCache` option [for JavaScript](https://powersync-ja.github.io/powersync-js/web-sdk/globals#webspecificopenoptions) or [for Dart](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteOptions/preparedStatementCacheSize.html).
+
+{/* SDK doc comments and changelogs link to #direct-tables, the earlier name of this section. Keep this anchor. */}
+
+<span id="direct-tables" />
+
+## Managed Tables
+
+<Note>
+  **Availability**
+
+  Managed tables are currently in [alpha](/resources/feature-status), so their API may change based on feedback. They were introduced in the following versions of our client SDKs:
+
+  - **JavaScript** (React Native: `2.3.4`, Web: `2.6.0`, Node.js: `1.2.1`, Capacitor: `0.9.5`)
+  - **Dart**: Version `2.6.0`.
+  - **Kotlin**: Version `1.16.0`.
+  - **Swift**: Version `1.17.0`.
+  - **.NET**: Not supported yet.
+  - **Rust**: Version `0.2.0`.
+</Note>
+
+By default, each table in your client-side schema is a **managed view**. PowerSync stores synced rows as [JSON in an internal table](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) and creates a SQLite view that extracts your columns from the JSON. Queries on the view decode the JSON of every row they read.
+
+Queries on a **managed table** run faster than on a managed view, because they read values directly instead of decoding JSON. A managed table stores rows in a SQLite table with a column for each column in your schema. You query and write to it with the same SQL statements as a managed view. PowerSync still creates and migrates the table and adds local writes to the upload queue.
+
+How much faster depends on how many rows a query reads:
+
+- **Most gain:** Queries that filter, sort or aggregate across many rows, such as `SELECT sum(hours) FROM time_entries`.
+- **Some gain:** Queries that return many rows, such as `SELECT * FROM tasks` on a large table.
+- **Little gain:** Queries that look up a few rows by `id` or through an index.
+
+Indexes speed up lookups on managed views, so keep indexes on join keys and on columns you often filter by. It isn't practical to index every column you query, though. An index also doesn't help a query that returns many rows. Managed tables make these queries faster without extra indexes.
+
+<Info>
+  **Benchmarks**
+
+  In our tests on a table with 1 million rows, `SELECT *` queries were about 2x faster on a managed table. Filtering, sorting or aggregating on a column without an index was about 6x faster.
+
+  These numbers come from simulated benchmarks, so results in your app depend on your data and queries. If you try managed tables, let us know how they perform in your app.
+
+  Managed tables can also make the database file slightly smaller, because rows don't repeat the column names that JSON stores. How much depends on your data. For example, rows with large text values save less.
+</Info>
+
+### Choosing a Table Type
+
+Start with managed views. Change a table's type only if you hit a limit: its queries are too slow, or you need SQLite features that the client-side schema can't define. You choose the type for each table, so one schema can mix all three.
+
+|  | Managed views (default) | Managed tables | Raw tables |
+| --- | --- | --- | --- |
+| Created and migrated by | PowerSync | PowerSync | You |
+| Queries read | JSON, through a view | SQLite columns | SQLite columns |
+| Schema changes | Near-instant | Take longer as the table grows | Migrations you write |
+
+- **Managed views:** Use these by default: schema changes are near-instant, because PowerSync only re-creates the views. The trade-off is slower queries that read many rows, such as aggregates and long lists.
+- **Managed tables (alpha):** Switch a table to a managed table if its queries are measurably too slow. This is typically a large table whose queries read many rows, for example to filter, sort, aggregate or list them. The trade-off is that schema changes on a managed table take longer as the table grows.
+- **Raw tables:** Use [raw tables](/client-sdks/advanced/raw-tables) when you want full control over the SQLite table, for example to add foreign keys, custom triggers or expression indexes, or to use a library such as [Room](/client-sdks/orms/kotlin/room) that manages the schema. The trade-off is that you have to create and migrate the table yourself.
+
+### Enabling Managed Tables
+
+Set the `direct` flag on the table definition. The flag name refers to queries reading the table directly, with no view in between.
+
+<CodeGroup>
+
+```typescript TypeScript
+const schema = new Schema({
+  users: new Table({ name: column.text }, { direct: true })
+});
+```
+
+```dart Dart/Flutter
+const schema = Schema([
+  Table.direct('users', [Column.text('name')])
+]);
+```
+
+```kotlin Kotlin
+val schema = Schema(
+  Table("users", listOf(Column.text("name")), direct = true)
+)
+```
+
+```swift Swift
+let schema = Schema(
+  Table(name: "users", columns: [.text("name")], direct: true)
+)
+```
+
+```rust Rust
+let mut schema = Schema::default();
+schema
+  .tables
+  .push(Table::create("users", vec![Column::text("name")], |tbl| {
+    tbl.direct = true
+  }));
+```
+
+</CodeGroup>
+
+Before you switch a table, check how managed tables differ from managed views:
+
+- **Column types:** Managed views convert each value to the column type in your schema. Managed tables store each value with the type it was synced or written with. For example, a value synced as text into an `integer` column stays text. Filters and sorting on that column can then return different results than on a managed view.
+- **View names:** Managed tables aren't views, so you can't set the `viewName` option. If you use `viewName` to [move data from a local-only table to a synced table](/client-sdks/advanced/local-only-usage#moving-data-to-synced-tables), keep that table as a managed view.
+- **High Performance Diffs:** In the JavaScript SDKs, [High Performance Diffs](/client-sdks/high-performance-diffs) don't support managed tables yet. Keep tables that you watch with High Performance Diffs as managed views.
+
+### Migrations
+
+PowerSync migrates managed tables for you, so you don't write migration code. It runs these migrations when your app opens the database with the new schema, before the database is ready for queries. Before you release an app update that changes a managed table, consider these points:
+
+- **Switching to or from a managed table:** Change the `direct` flag. PowerSync copies the table's rows into the new format on the device, without downloading them again. The copy takes longer as the table grows. In our tests, a table with 1 million rows took a few seconds.
+- **Adding or removing a column:** PowerSync runs the `ALTER TABLE` statements. These changes take longer as the table grows. In our tests on a table with 1 million rows, each change took up to about a second. On a managed view, the same changes are near-instant, because PowerSync only re-creates the view.
+- **New columns are empty until the next sync:** If an app update adds a column that is already synced, a managed view shows its values right away, even offline. A managed table shows `NULL` in that column until the next completed sync, so offline devices see `NULL` until they reconnect. PowerSync then fills the column from synced data already on the device, without downloading it again. Note that new columns are usually already synced when the app update ships, because our recommended order for [additive schema changes](/maintenance-ops/deploying-schema-changes) publishes the app last.

client-sdks/advanced/raw-tables.mdx

@@ -4,10 +4,6 @@ description: "Access raw SQLite tables to bypass JSON view limitations for nativ
 sidebarTitle: "Raw SQLite Tables"
 ---
 
-<Warning>
-  Raw tables are not currently compatible with the [High Performance Diffs](/client-sdks/high-performance-diffs) feature available in our JavaScript SDKs. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome!
-</Warning>
-
 By default, PowerSync uses a [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) where data is stored schemalessly in JSON format and then presented through SQLite views based on the client-side schema. Raw tables allow you to define native SQLite tables in the client-side schema, bypassing this.
 
 This eliminates overhead associated with extracting values from the JSON data and provides access to advanced SQLite features like foreign key constraints and custom indexes.
@@ -27,12 +23,13 @@ This eliminates overhead associated with extracting values from the JSON data an
 
 ## When to Use Raw Tables
 
-Consider raw tables when you need:
+Use raw tables when you need control over the SQLite table that the client-side schema doesn't give you:
+
+- **Constraints:** `FOREIGN KEY` and `ON DELETE CASCADE` constraints.
+- **Custom columns and indexes:** `GENERATED` columns, or indexes on expressions. The client-side schema only supports indexes on plain columns.
+- **Custom table setup:** Custom triggers, or a library such as [Room](/client-sdks/orms/kotlin/room) that creates the schema for you.
 
-- **Indexes** - PowerSync's default schema has basic support for indexes on columns, while raw tables give you complete control to create indexes on expressions, use `GENERATED` columns, etc.
-- **Improved performance** for complex queries (e.g., `SELECT SUM(value) FROM transactions`) - raw tables more efficiently get these values directly from the SQLite column, instead of extracting the value from the JSON object on every row.
-- **Reduced storage overhead** - eliminate JSON object overhead for each row in `ps_data__<table>.data` column.
-- **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers.
+Raw tables also make queries that read many rows faster, such as `SELECT SUM(value) FROM transactions`, because SQLite reads values directly from the columns instead of extracting them from JSON. If faster queries are your only reason, consider [managed tables](/client-sdks/advanced/performance-optimization#managed-tables) (alpha) instead. They also store data in SQLite columns, but PowerSync creates and migrates them for you. To compare the options, see [Choosing a Table Type](/client-sdks/advanced/performance-optimization#choosing-a-table-type).
 
 <Warning>
   **Advanced SQLite features** like `FOREIGN KEY` and `ON DELETE CASCADE` constraints need [special consideration](#using-foreign-keys).

client-sdks/cascading-delete.mdx

@@ -7,9 +7,9 @@ keywords:
   - "delete"
 ---
 
-PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available.
+By default, PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available.
 
-There's no built-in support for cascading deletes on the client, but you can achieve this in two ways:
+There's no built-in support for cascading deletes on the client, but you can achieve this in three ways:
 
 1. Manual deletion in a transaction - Delete all related records in a single transaction (recommended for most cases)
    <Note>
@@ -19,6 +19,7 @@ There's no built-in support for cascading deletes on the client, but you can ach
    <Note>
      You create triggers on the internal tables (not the views defined by the client schema), similar to what is done [here](https://github.com/powersync-ja/powersync-js/blob/e77b1abfbed91988de1f4c707c24855cd66b2219/demos/react-supabase-todolist/src/app/utils/fts_setup.ts#L50).
    </Note>
+3. Raw tables - PowerSync can sync into existing tables defined by your app, and those tables can use [foreign key constraints](/client-sdks/advanced/raw-tables#using-foreign-keys).
 
 ## Example: Manual Transaction
 

client-sdks/high-performance-diffs.mdx

@@ -22,6 +22,8 @@ While [basic/incremental watch queries](/client-sdks/watch-queries) enable react
   - API design and developer experience
   - Additional features or optimizations needed
 
+  The API does not support [raw tables](/client-sdks/advanced/raw-tables) or [managed tables](/client-sdks/advanced/performance-optimization#managed-tables) yet. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome!
+
   Join our [Discord community](https://discord.gg/powersync) to share your experience and get help.
 </Warning>
 

resources/feature-status.mdx

@@ -69,6 +69,7 @@ Below is a summary of the current main PowerSync features and their release stat
 | **Client SDKs** |  |
 | High Performance Diffs | Experimental |
 | Checkpoint Requests | Alpha |
+| Managed Tables | Alpha |
 | Tauri SDK | Alpha |
 | TanStack Query | Alpha |
 | Rust SDK | Beta |