@@ -43,8 +43,8 @@ each binding in `WebServiceProvider::register()` with a `bound()` check instead,
Eventing & Messaging
-[EDA](modules/eda.md) · [EDA Brokers](modules/eda-brokers.md) · [Messaging](modules/messaging.md)
-One `EventPublisher` port with in-memory, queue, RabbitMQ, Postgres and Kafka adapters, a genuine same-transaction outbox, and the lower-level raw-bytes broker layer beneath it.
+[EDA](modules/eda.md) · [EDA Brokers](modules/eda-brokers.md) · [EDA Database Outbox](modules/eda-database.md) · [Messaging](modules/messaging.md)
+One `EventPublisher` port with in-memory, queue, RabbitMQ, database (any SQL server), Postgres and Kafka adapters, a genuine same-transaction outbox safe with concurrent consumers on every database, and the lower-level raw-bytes broker layer beneath it.
diff --git a/docs/modules/cqrs.md b/docs/modules/cqrs.md
index e93da570..cdf09cc5 100644
--- a/docs/modules/cqrs.md
+++ b/docs/modules/cqrs.md
@@ -116,8 +116,9 @@ work raises no `DomainEvent` at all (Laravel discards `afterCommit` callbacks on
ever published for it.
**Non-atomic caveat:** the DB commit and the broker publish are two steps with a gap — an honest
-at-least-once-ish, non-atomic guarantee. True exactly-once durability is the SP-4 transactional outbox
-(`firefly/eda-postgres`), which hangs off the same `DB::afterCommit` hook + a Postgres advisory lock.
+at-least-once-ish, non-atomic guarantee. Atomic durability is the same-transaction outbox
+(`firefly.eda.provider=database`, or `postgres`), which writes the event inside the aggregate's transaction
+through `firefly/data`'s `PreCommitEventHook`.
When no firefly/eda `EventPublisher` is bound, the bridge falls back to `NoOpEventPublisher` (drops
silently), so cqrs installed without eda still boots and simply emits no integration events.
@@ -147,8 +148,7 @@ edge): the bridge listens for `DomainEvent`s on the in-process `Context` dispatc
Read-model / projection scaffolding (→ `firefly/eventsourcing`), real authorization (→ M11), real query
cache (→ firefly/cache), CQRS metrics + health (→ M12), attribute-driven command validation, the fluent
-builder / distributed-tracing ergonomics, and exactly-once outbox durability (→ SP-4) are deferred and
-documented honestly. The `HandlerManifest` needs no hand-wiring: like every other
+builder / distributed-tracing ergonomics are deferred and documented honestly. The `HandlerManifest` needs no hand-wiring: like every other
compiled manifest it resolves to the `firefly:cache` artifact if present, otherwise an in-process scan of
`firefly.scan.paths`, otherwise empty — so an uncached app registers the same handlers a cached one does. A
handler outside `firefly.scan.paths` still never registers. Under Octane, `CorrelationContext` is
diff --git a/docs/modules/eda-brokers.md b/docs/modules/eda-brokers.md
index 418266b6..4de6b8b5 100644
--- a/docs/modules/eda-brokers.md
+++ b/docs/modules/eda-brokers.md
@@ -2,8 +2,9 @@
`firefly/eda-rabbitmq`, `firefly/eda-postgres`, and `firefly/eda-kafka` are real message-broker
adapters behind the `firefly/eda` (M9) `EventPublisher` port — the "real brokers" seam M9 deliberately
-left for later (see [Event-Driven Architecture § Known-latent](eda.md#known-latent)). Each is its own
-opt-in package, gated behind `firefly.eda.provider`, and each ships its own `HealthIndicator`. Installing
+left for later. Each is its own opt-in package, gated behind `firefly.eda.provider`, and each ships its own
+`HealthIndicator`. The Postgres adapter is the portable [database outbox](eda-database.md) plus Postgres's two
+optimizations; on any other database select `firefly.eda.provider=database` instead. Installing
any of them without selecting it as the active provider is fully inert: no connection is opened, no
extension is required, and `/actuator/health` is unaffected.
@@ -18,7 +19,7 @@ messages back off the wire — something M9's `start()`/`stop()` never needed.
| Package | Client library | Destination shape | DLQ | Health indicator name |
|---|---|---|---|---|
| `firefly/eda-rabbitmq` | `php-amqplib/php-amqplib` (AMQP 0-9-1) | `"exchange/routingKey"` (or a bare routing key against the default exchange) | Broker-native DLX (dead-letter exchange) | `rabbitmq` |
-| `firefly/eda-postgres` | `ext-pdo_pgsql` (via Laravel's `ConnectionInterface`) | a destination string stored on the outbox row | Outbox row `status='FAILED'` | `postgres` |
+| `firefly/eda-postgres` | `ext-pdo_pgsql`; builds on `firefly/eda-database` | a destination string stored on the outbox row | Outbox row `status='FAILED'` | `postgres` |
| `firefly/eda-kafka` | `ext-rdkafka` (**OPTIONAL**) | a topic name | Dead-letter topic `.DLT`, with `x-dlt-*` provenance headers | `kafka` |
All three implement the same `EventPublisher::publish(string $destination, string $eventType, array
@@ -29,10 +30,11 @@ bool)/stop()` shape.
## The `firefly.eda.provider` switch
-`firefly.eda.provider` now has five values: `memory` (default) and `queue` bind the M9 in-process/queue
-adapters (`EdaAutoConfiguration`, `#[Order(1000)]`); `rabbitmq`, `postgres`, and `kafka` each bind their own
-`EventPublisher` + `EventConsumer` pair from `#[Order(900)]` autoconfigurations (`RabbitMqAutoConfiguration`,
-`PostgresOutboxAutoConfiguration`, `KafkaAutoConfiguration`) — running *before* `EdaAutoConfiguration`'s
+`firefly.eda.provider` now has six values: `memory` (default) and `queue` bind the M9 in-process/queue
+adapters (`EdaAutoConfiguration`, `#[Order(1000)]`); `database`, `rabbitmq`, `postgres`, and `kafka` each bind
+their own `EventPublisher` + `EventConsumer` pair from `#[Order(900)]` autoconfigurations
+(`DatabaseOutboxAutoConfiguration`, `RabbitMqAutoConfiguration`, `PostgresOutboxAutoConfiguration`,
+`KafkaAutoConfiguration`) — running *before* `EdaAutoConfiguration`'s
`#[ConditionalOnMissingBean(EventPublisher::class)]` so it backs off correctly (the
`firefly/scheduling-postgres` win-the-race precedent). Each broker's beans are additionally
`#[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: '')]`-gated, so with any other
@@ -56,12 +58,15 @@ provider selected the whole package resolves nothing and requires nothing.
### Postgres (`firefly/eda-postgres`)
+Under `provider=postgres` the outbox reads [the database outbox's keys](eda-database.md#configuration); the
+`firefly.eda.postgres.*` spellings below remain as aliases.
+
| Key | Default | Meaning |
|---|---|---|
-| `firefly.eda.postgres.connection` | the default DB connection | The named Laravel connection the outbox publisher/consumer/relay use — must be `pgsql` for `pg_notify`/`LISTEN` to activate (any other driver, e.g. `sqlite` in tests, silently skips the NOTIFY optimization and falls back to polling). |
-| `firefly.eda.postgres.channel` | `firefly_eda_events` | The `LISTEN`/`NOTIFY` channel name and the outbox row's `channel` column value. |
-| `firefly.eda.postgres.max_attempts` | `3` | How many `nack()`s (in-process consumer) or failed relay attempts an outbox row tolerates before it is marked `FAILED`. |
-| `firefly.eda.postgres.relay.downstream_provider` | *(unset)* | **OPTIONAL.** Selects the relay's downstream: the shipped aliases `rabbitmq`\|`kafka`, the class-string of any `EventPublisher`, or a bound container id. When set, `firefly:outbox:relay` forwards `PENDING` rows there; when unset the command refuses to run with an error naming this key, and delivery is entirely the terminal in-process consumer's job. An app that binds its own publisher under the container id `firefly.eda.relay.downstream` may leave this key unset — that binding is checked first. Resolution is validated when the relay command runs (not at boot), so a misconfiguration fails before a single row is claimed instead of at the first publish. Whatever it resolves to, a `PostgresEventPublisher` is **refused**: it would re-insert `PENDING` rows into the same outbox. |
+| `firefly.eda.postgres.connection` | the default DB connection | Alias of `firefly.eda.database.connection`. The connection must be `pgsql` for `pg_notify`/`LISTEN` to activate; on any other driver the NOTIFY optimization is skipped and the consumer polls. |
+| `firefly.eda.postgres.channel` | `firefly_eda_events` | Alias of `firefly.eda.database.channel` — also the `LISTEN`/`NOTIFY` channel name. |
+| `firefly.eda.postgres.max_attempts` | `3` | Alias of `firefly.eda.database.max-attempts`. |
+| `firefly.eda.postgres.relay.downstream_provider` | *(unset)* | Alias of `firefly.eda.database.relay.downstream-provider` — see [the optional relay](#b-the-optional-relay-fireflyoutboxrelay). |
### Kafka (`firefly/eda-kafka`)
@@ -87,7 +92,7 @@ trace the row carries instead of a fresh root.
It does nothing unless `firefly.observability.tracing.enabled` and `firefly.observability.tracing.eda.enabled`
are on (with them off the bound `EdaTracing` is the no-op). Set it to `false` to keep in-process spans while
refusing to put trace identifiers on a wire someone else reads. The key is read in exactly one place —
-`Firefly\Eda\Tracing\BrokerTracing` — by the three publisher beans and by `RelayDownstream`, because a gate
+`Firefly\Eda\Tracing\BrokerTracing` — by the publisher beans (the database outbox's too) and by `RelayDownstream`, because a gate
that lives only in a bean fails open wherever a publisher is built by container autowiring. The relay honours it
whether you name a shipped adapter by its alias (`rabbitmq`) or by its own class-string: both spellings take the
same configured path. A downstream of your OWN — a publisher class this package ships no adapter for, or one you
@@ -106,7 +111,9 @@ M9 shipped only at-least-once delivery within a process/queue — publishing a d
atomic with the aggregate's own database write. `firefly/eda-postgres` closes that gap with a **genuine
same-transaction outbox**: the `firefly_eda_outbox` row is written *inside* the aggregate's own open
transaction, on the aggregate's own connection, so it commits or rolls back atomically with the aggregate
-— no dual-write, no "commit then hope the publish succeeds."
+— no dual-write, no "commit then hope the publish succeeds." The outbox itself — schema, writer, pre-commit
+hook, leased consumer and relay — lives in [`firefly/eda-database`](eda-database.md), which runs it on every
+database under `provider=database`; this package adds the in-transaction `pg_notify` and the `LISTEN` wake-up.
@@ -126,7 +133,7 @@ interface PreCommitEventHook
`DomainEventDispatcher::afterCommit()` — which already runs *during* the pre-commit drain, before
`DB::connection($connection)->afterCommit(...)` is even registered — now calls
`$this->preCommitHook?->handle($event, $connection)` first. The hook is `null` by default: with no
-broker package installed (or any provider other than `postgres`), `DomainEventDispatcher` behaves exactly
+outbox package installed (or any provider other than `database` or `postgres`), `DomainEventDispatcher` behaves exactly
as it did in M8 — publish deferred to `DB::afterCommit` only, zero behaviour change. `$connection` is
threaded through unchanged from the dispatcher, so a `#[Transactional(connection: 'x')]` aggregate's
outbox row lands on connection `x`, not some default connection.
@@ -149,14 +156,16 @@ that transaction actually commits — so the consumer's `LISTEN` wakes with low
moment the row becomes visible to other connections, and never wakes for a row that gets rolled back.
Outbox columns (one source of truth, `OutboxSchema`): `id, destination, channel, event_type, payload,
-headers, transaction_id, status, attempts, error_message, created_at, processed_at, failed_at`. `status`
+headers, transaction_id, status, attempts, error_message, created_at, processed_at, failed_at, claimed_by,
+claimed_until` — the last two are the claim's lease. `status`
is one of `PENDING` (just written) / `PUBLISHED` (delivered) / `FAILED` (exhausted `max_attempts`).
### Double-publish avoidance
With `provider=postgres`, both the in-tx `OutboxPreCommitHook` *and* the ordinary M10 after-commit bridge
(`DomainEventBridge` → `CommandEventPublisher` → `EdaCommandEventPublisher`) could in principle write the
-same event to the outbox twice. `PostgresOutboxAutoConfiguration` prevents this by binding, all at
+same event to the outbox twice. `PostgresOutboxAutoConfiguration` (and `DatabaseOutboxAutoConfiguration` does the
+same under `provider=database`) prevents this by binding, all at
`#[Order(900)]` (ahead of `firefly/data`'s and `firefly/cqrs`'s `#[Order(1000)]` defaults):
- its own `DomainEventDispatcher` carrying the `OutboxPreCommitHook` (wins `DataAutoConfiguration`'s
@@ -179,35 +188,33 @@ state. Pick one (or neither, if you're only using RabbitMQ/Kafka directly and ne
This is the default, always-available path: `firefly:eda:consume` drives `PostgresEventConsumer`, which:
1. `subscribe()` issues `LISTEN ` (pgsql only).
-2. `poll($timeoutMs)` first waits (briefly, best-effort) on a `NOTIFY` via `NotificationWaiter` — guarded so
- *any* failure on that wait (including a deprecation-to-exception handler tripping on the
- soon-to-be-deprecated `pgsqlGetNotify()` fallback path) never crashes the poll loop — then, regardless of
- whether a notification arrived, **claims** the oldest `status='PENDING'` row with
- `SELECT ... FOR UPDATE SKIP LOCKED` (pgsql) or a plain `ORDER BY id` (sqlite, for tests). The poll-fallback
- claim also picks up rows that were written while no consumer was listening.
+2. `poll($timeoutMs)` first **claims** one row — a lease written by the strategy the server supports
+ (`SKIP LOCKED` on pgsql) — and only when nothing was claimable waits (briefly, best-effort, guarded against
+ any failure, including a deprecation-to-exception handler tripping on the soon-to-be-deprecated
+ `pgsqlGetNotify()` fallback path) on a `NOTIFY` via `NotificationWaiter`, then claims once more. The claim
+ also picks up rows that were written while no consumer was listening.
3. The claimed row is handed to `ConsumerLoop`, which delivers it to the `SubscriberRegistry` — driving
every matching `#[EventListener]` handler, exactly as the in-memory/queue adapters do.
-4. On success, `ack()` runs a guarded `UPDATE ... SET status='PUBLISHED' WHERE id=? AND status='PENDING'` —
- the `WHERE status='PENDING'` guard makes the ack idempotent. On a handler throw, `nack()` increments
- `attempts` and either resets the row to `PENDING` (retry) or — past `max_attempts` — marks it `FAILED`.
+4. On success, `ack()` runs a guarded `UPDATE ... SET status='PUBLISHED' WHERE id=? AND status='PENDING' AND
+ claimed_by=` — the guard makes the ack idempotent and settles only a row this consumer still
+ holds. On a handler throw, `nack()` increments `attempts` and either frees the row as `PENDING` (retry) or —
+ past `max-attempts` — marks it `FAILED`.
There is **no `EventPublisher::publish()` call anywhere in this path** — `PostgresEventConsumer` reads and
updates the outbox table directly, so there is no re-INSERT and no risk of self-reference.
-This is a **durable `PENDING`→`PUBLISHED` status window**, not an in-memory high-water mark: a restarted
-consumer resumes at the oldest still-`PENDING` row and never replays a `PUBLISHED` row. A **single**
-consumer worker gives exactly-once in-process delivery. Running **multiple** workers concurrently degrades
-to at-least-once — `FOR UPDATE SKIP LOCKED` plus the guarded `ack()` UPDATE prevent two workers from both
-claiming or both acking the same row, but a worker that crashes *after* running the handler but *before*
-its `ack()` commits will have another worker (or its own restart) redeliver that row. **Write your
-`#[EventListener]` handlers idempotently** if you run more than one consumer worker. A single worker
-consuming in `id` order also gives effective FIFO per outbox (not per-partition/per-key ordering like
-Kafka — just the natural insertion order of one table).
+This is a **durable `PENDING`→`PUBLISHED` status window**: a restarted consumer resumes at the oldest
+claimable row and never replays a `PUBLISHED` one. Any number of consumer workers may run at once — each row is
+leased to one of them while it is delivered, and a worker that crashes mid-delivery leaves its row to the next
+worker when the lease ends (`firefly.eda.database.lease-seconds`), so across a crash delivery is at-least-once:
+write your `#[EventListener]` handlers idempotently. Workers share the table, so ordering is the table's
+insertion order only per worker, not across them.
### (b) The optional relay — `firefly:outbox:relay`
`firefly:outbox:relay` is a **distinct, optional** path that fronts a **different downstream broker**. Enable
-it by setting `firefly.eda.postgres.relay.downstream_provider` — to a shipped alias (`rabbitmq`/`kafka`), to
+it by setting `firefly.eda.database.relay.downstream-provider` (or its alias
+`firefly.eda.postgres.relay.downstream_provider`) — to a shipped alias (`rabbitmq`/`kafka`), to
an `EventPublisher` class-string, or to the id of a binding you supply — or by binding your own publisher
under the container id `firefly.eda.relay.downstream` (checked first, so the key may then stay unset). A
shipped adapter named by its OWN class-string is the same downstream as its alias and resolves identically,
@@ -224,13 +231,12 @@ row is claimed. Boot-time validation was considered and rejected: a downstream b
`boot()` may not exist yet when the check would run, so it would fail correctly-configured applications — and
it would fail every web request and every unrelated artisan command over a relay-only concern.
-`OutboxRelay::relayBatch()` claims a batch of `PENDING` rows (`FOR UPDATE SKIP LOCKED` on pgsql, inside a
-short transaction so concurrent relay workers never double-claim; a plain per-row `WHERE id=? AND
-status='PENDING'` guard on drivers without row locking), publishes each through the configured downstream
-`EventPublisher`, and marks it `PUBLISHED` — or, on a publish failure, increments `attempts` and marks it
-`FAILED` past `max_attempts`, exactly like the in-process consumer's `nack()`.
+`OutboxRelay::relayBatch()` leases a batch of claimable rows with the same strategy as the consumer (several
+relay workers never forward the same row), forwards each through the configured downstream `EventPublisher`,
+and marks it `PUBLISHED` — or, on a forward failure, counts the attempt and marks it `FAILED` past
+`max-attempts`.
-Both the command and `OutboxRelay`'s constructor **refuse a `PostgresEventPublisher` as the downstream** —
+Both the command and `OutboxRelay`'s constructor **refuse the outbox writer (`DatabaseEventPublisher` or `PostgresEventPublisher`) as the downstream** —
that would re-INSERT `PENDING` rows into the very same outbox, an infinite loop — throwing a `LogicException`
/ printing a clear error instead. The refusal applies however the downstream was named: alias, class-string
or container binding. The relay is for genuinely bridging to a *different* broker (e.g. you want
@@ -420,18 +426,26 @@ anonymous Pest closure, so PHPStan can see `skipUnlessDocker()` statically):
| Adapter | Required extension | Environment variable |
|---|---|---|
| RabbitMQ | — (`php-amqplib` is a hard Composer dependency) | `FIREFLY_RABBITMQ_DSN` |
-| Postgres | `ext-pdo_pgsql` | `FIREFLY_PG_DSN` (parsed as `host=…;port=…;dbname=…;user=…;password=…`) |
+| Postgres | `ext-pdo_pgsql` | `FIREFLY_PG_DSN` (`host=…;port=…;dbname=…;user=…;password=…` or a URL, parsed by `IntegrationDsn`) |
| Kafka | `ext-rdkafka` | `FIREFLY_KAFKA_BROKERS` |
+| Database outbox, dead letters, locks | `pdo_pgsql` / `pdo_mysql` / `pdo_sqlsrv` | `FIREFLY_PG_DSN`, `FIREFLY_MYSQL_DSN`, `FIREFLY_MARIADB_DSN`, `FIREFLY_SQLSRV_DSN` |
+| Redis (CacheLock) | — (`predis/predis`, dev) | `FIREFLY_REDIS_URL` |
+
+The last two rows cover the database-agnostic suites (the outbox, the dead-letter stores, the lock providers and
+the data layer on real servers) and `CacheLock` over Redis: they need no Docker check, only the variable and the
+client.
Missing extension, missing environment variable, or no Docker daemon reachable — any one of these skips
the suite with a clear message, so `composer check` and ordinary CI runs never depend on a live broker.
Run them explicitly (with the broker/extension/Docker prerequisites satisfied) via:
```bash
-FIREFLY_PG_DSN="host=127.0.0.1;port=5432;dbname=firefly;user=postgres;password=postgres" \
- vendor/bin/pest --group=integration
+FIREFLY_PG_DSN="host=127.0.0.1;port=5432;dbname=firefly;user=firefly;password=firefly" \
+ composer test:integration
```
+CI runs one lane per server and one for the brokers — see [Integration Testing](integration-testing.md#ci-lanes).
+
## Example: switching an app to the Postgres outbox
Set the provider, and uncomment the `postgres` block the reference already ships at its defaults — `relay` is
@@ -454,10 +468,11 @@ optional and only read by `firefly:outbox:relay`, so leave it out for terminal i
```
```bash
-composer require firefly/eda-postgres
php artisan migrate # creates firefly_eda_outbox
php artisan firefly:eda:consume # terminal in-process delivery — run this as a long-lived worker
```
+On any other database, use `provider=database` — see [EDA Database Outbox](eda-database.md).
+
No application code changes: the same `#[EventListener]` handlers and the same `EventPublisher::publish()`
call sites that worked against `memory`/`queue` now run against a durable, same-transaction outbox.
diff --git a/docs/modules/eda-database.md b/docs/modules/eda-database.md
new file mode 100644
index 00000000..18c6e11e
--- /dev/null
+++ b/docs/modules/eda-database.md
@@ -0,0 +1,94 @@
+# EDA Database Outbox
+
+`firefly/eda-database` is the portable same-transaction outbox behind `firefly.eda.provider=database`. An
+`EventPublisher::publish()` — and every domain event, through the pre-commit hook — writes one
+`firefly_eda_outbox` row on the aggregate's own connection inside its open transaction, so the event commits or
+rolls back with the aggregate on PostgreSQL, MySQL, MariaDB, SQL Server or SQLite. `php artisan firefly:eda:consume`
+delivers committed rows to your `#[EventListener]` handlers; the optional `php artisan firefly:outbox:relay`
+forwards them to a second broker. Rows are claimed by **leases**, so any number of consumers and relays may run
+against one outbox. `firefly.eda.provider=postgres` ([EDA Brokers](eda-brokers.md)) is this package plus
+`pg_notify` on write and a `LISTEN` wake-up in the consumer.
+
+## Switching an application to the database outbox
+
+Set the provider, uncomment the block the reference ships at its defaults, migrate, and run a consumer:
+
+
+```php
+ // 'database' => [
+ // 'connection' => 'mysql',
+ // 'channel' => 'firefly_eda_events',
+ // 'max-attempts' => 3,
+ // 'claim-strategy' => env('FIREFLY_EDA_CLAIM_STRATEGY', 'auto'),
+ // 'lease-seconds' => 300,
+ // 'poll-interval-ms' => 500,
+ // 'relay' => [
+ // 'downstream-provider' => 'rabbitmq',
+ // ],
+ // ],
+```
+
+```bash
+php artisan migrate # creates firefly_eda_outbox (and adds the lease columns to an older one)
+php artisan firefly:eda:consume # terminal in-process delivery — run one or more as long-lived workers
+```
+
+## Configuration
+
+| Key | Default | Meaning |
+|---|---|---|
+| `firefly.eda.provider` | `memory` | `database` selects this package; `postgres` selects the Postgres flavour, which reads the same keys. |
+| `firefly.eda.database.connection` | the default connection | The connection the outbox writer (outside a transaction), the consumer and the relay use. A domain event is always written on the aggregate's own connection. Alias: `firefly.eda.postgres.connection`. |
+| `firefly.eda.database.channel` | `firefly_eda_events` | The value of the row's `channel` column (and, under `postgres`, the `LISTEN`/`NOTIFY` channel). Alias: `firefly.eda.postgres.channel`. |
+| `firefly.eda.database.max-attempts` | `3` | Failed deliveries (consumer) or forwards (relay) a row tolerates before it is marked `FAILED`. Alias: `firefly.eda.postgres.max_attempts`. |
+| `firefly.eda.database.claim-strategy` | `auto` | `auto`, `skip-locked`, `read-past` or `optimistic-lease` — see below. |
+| `firefly.eda.database.lease-seconds` | `300` | How long a claimed row belongs to its claimant. Size it above your slowest listener. |
+| `firefly.eda.database.poll-interval-ms` | `500` | How long an idle consumer sleeps between claims (`postgres` waits on `NOTIFY` instead). |
+| `firefly.eda.database.relay.downstream-provider` | *(unset)* | Only read by `firefly:outbox:relay`: `rabbitmq`, `kafka`, an `EventPublisher` class-string, or a container id. Alias: `firefly.eda.postgres.relay.downstream_provider`. |
+
+When a `firefly.eda.database.*` key and its `firefly.eda.postgres.*` alias are both set, the database key wins, so an
+application can move between the two providers without losing its settings.
+
+## Claim strategies
+
+A claim leases rows to one claimant: the strategy writes `claimed_by` and `claimed_until` on the rows it picked and
+commits that before any listener runs or any forward is attempted. A row is claimable while it is `PENDING` and its
+lease is empty or over; acknowledging, failing or relaying a row happens only while the claimant still holds it.
+The strategies differ only in how they pick rows without two claimants picking the same one:
+
+| Strategy | How it picks | Chosen by `auto` on |
+|---|---|---|
+| `skip-locked` | `SELECT … FOR UPDATE SKIP LOCKED` in a short claim transaction (MySQL and MariaDB run it `READ COMMITTED` so publishers never wait on its gap locks) | PostgreSQL; MySQL 8.0.1+; MariaDB 10.6+ |
+| `read-past` | `SELECT TOP n … WITH (UPDLOCK, READPAST, ROWLOCK)` in a short claim transaction | SQL Server |
+| `optimistic-lease` | an unlocked read, then one conditional `UPDATE … WHERE status = 'PENDING' AND (claimed_until IS NULL OR claimed_until < now)` per row; the row is won when exactly one row changed | SQLite; any server the probe does not recognise; MySQL before 8.0.1; MariaDB before 10.6 |
+
+`auto` reads the server's version (`DatabaseCapabilities` in `firefly/data`), never the driver's name: a `mysql`
+connection whose server is MariaDB is recognised as MariaDB. A forced strategy the server cannot run fails at the
+first claim with an error naming the server and its version — it is never downgraded silently. The strategy is
+resolved at the first claim, so booting an application opens no connection to find out.
+
+Lease instants are the application's clock in UTC at millisecond precision; keep your nodes' clocks synchronised
+(NTP) well within `lease-seconds`.
+
+## Delivery and failure
+
+- **Consumer** (`firefly:eda:consume`): claims one row, delivers it to the `SubscriberRegistry`, then marks it
+ `PUBLISHED`. A listener that throws counts an attempt and frees the row for an immediate retry, or marks it
+ `FAILED` past `max-attempts`. A row whose JSON cannot be decoded is dead-lettered at once (`FAILED`, no retry).
+- **Relay** (`firefly:outbox:relay`): claims a batch, forwards each row to the downstream broker inside the trace
+ the row carries, and marks it `PUBLISHED` (or counts the failure). It refuses the outbox writer as its downstream.
+- **Several workers**: each row is delivered once while its worker lives within its lease. A worker that crashes
+ mid-delivery leaves its row leased until the lease ends; then another worker delivers it — across a crash,
+ delivery is at-least-once, so listeners should be idempotent.
+- **Health**: under `provider=database` the actuator reports an `outbox` component — the driver, the strategy in
+ use and the `PENDING` backlog, or `DOWN` with the error.
+
+## Moving from `firefly/eda-postgres`
+
+Nothing to change: `provider=postgres` keeps its beans, its health indicator and its table, and is now this outbox
+plus the two Postgres optimizations. The classes that moved here keep their old names as deprecated subclasses for
+one release (`Firefly\Eda\Postgres\Outbox\OutboxSchema`, `OutboxRow`, `OutboxRelay`, `RelayDownstream`,
+`Firefly\Eda\Postgres\Console\OutboxRelayCommand`). Run `php artisan migrate` once to add the lease columns.
+
+See the [database support matrix](../database-support.md) for what each server does, and [EDA Brokers](eda-brokers.md)
+for the broker adapters.
diff --git a/docs/modules/eda.md b/docs/modules/eda.md
index e4ec7356..8c3833a8 100644
--- a/docs/modules/eda.md
+++ b/docs/modules/eda.md
@@ -194,9 +194,14 @@ interface DeadLetterStore
}
```
-`InMemoryDeadLetterStore` (the `#[ConditionalOnMissingBean]` default) is an in-process, append-only list of
-`DeadLetterEntry` (envelope + exception class + exception message) — nothing is silently lost, but it does
-not survive past the process. A durable adapter is SP-4 territory.
+`firefly.eda.dead-letter.store` selects the store under every provider: `memory` (the default,
+`InMemoryDeadLetterStore`, an in-process, append-only list of `DeadLetterEntry` — envelope + exception class +
+exception message — lost when the process exits) or `database` (`DatabaseDeadLetterStore`, one
+`firefly_eda_dead_letters` row per exhausted envelope on `firefly.eda.dead-letter.connection`; run
+`php artisan migrate`). Both implement `ReplayableDeadLetterStore` — `find(DeadLetterQuery)` and `remove(id)` —
+which `php artisan firefly:eda:dead-letters:replay` uses to republish entries, oldest first, through the bound
+`EventPublisher` to their original destination (`--id`, `--destination`, `--event-type`, `--limit`, `--dry-run`,
+`--keep`). An application-bound `DeadLetterStore` still wins.
## Serializer seam
@@ -239,14 +244,16 @@ fails open wherever a publisher is built by autowiring instead.
| Key | Type | Default | Meaning |
|---|---|---|---|
-| `firefly.eda.provider` | `memory`\|`queue`\|`rabbitmq`\|`postgres`\|`kafka` | `memory` | `EdaAutoConfiguration` selects `InMemoryEventBus` or `QueueEventBus`; the three broker values are honoured by the adapter packages (see [EDA brokers](eda-brokers.md)) and read here as "not queue". |
+| `firefly.eda.provider` | `memory`\|`queue`\|`database`\|`rabbitmq`\|`postgres`\|`kafka` | `memory` | `EdaAutoConfiguration` selects `InMemoryEventBus` or `QueueEventBus`; the other values are honoured by the adapter packages — `database` by [EDA Database Outbox](eda-database.md), `rabbitmq`, `postgres` and `kafka` by [EDA Brokers](eda-brokers.md) — and read here as "not queue". |
| `firefly.eda.serialization_format` | string | `json` | Selects `Serializer`; anything but `json` throws `SerializationException`. |
| `firefly.eda.retries` | int | `0` | Handler-level retry count applied to every `#[EventListener]` by `EventListenerWiringPass`. |
| `firefly.eda.retry_delay` | float (seconds) | `0.0` | Linear-backoff base delay (`retry_delay * attempt`). |
+| `firefly.eda.dead-letter.store` | `memory`\|`database` | `memory` | Where exhausted envelopes go — see [Retry / DLQ model](#retry-dlq-model). |
+| `firefly.eda.dead-letter.connection` | string\|null | `null` | The connection `DatabaseDeadLetterStore` writes on (`null` = the default connection). |
| `firefly.eda.queue.connection` | string\|null | `null` (default connection) | Queue connection `QueueEventBus`/`DispatchEventJob` dispatch onto, when `provider=queue`. |
| `firefly.eda.queue.name` | string\|null | `null` (default queue) | Queue name, when `provider=queue`. |
| `firefly.eda.destinations` | list\ | `[]` | The broker destinations `php artisan firefly:eda:consume` binds when `--destination` is not passed. Read **only** by that command; it must be a list of strings or the command throws a `ConfigurationException`. |
-| `firefly.eda.tracing.brokers.enabled` | bool | `true` | Whether the rabbitmq/kafka/postgres publishers may stamp a `traceparent` on the envelope they hand to the transport. Read in ONE place (`BrokerTracing`), by the adapters' publisher beans and by the relay's downstream overrides. Set it to `false` to keep in-process spans while putting no trace identifier on a third party's wire; it can only ever downgrade the bound `EdaTracing` to the no-op, never switch tracing on. |
+| `firefly.eda.tracing.brokers.enabled` | bool | `true` | Whether the rabbitmq/kafka/database/postgres publishers may stamp a `traceparent` on the envelope they hand to the transport. Read in ONE place (`BrokerTracing`), by the adapters' publisher beans and by the relay's downstream overrides. Set it to `false` to keep in-process spans while putting no trace identifier on a third party's wire; it can only ever downgrade the bound `EdaTracing` to the no-op, never switch tracing on. |
!!! note "A publish `destination` is a call-site argument, not a config key"
`EventPublisher::publish(string $destination, ...)` takes the destination explicitly at the call site —
@@ -308,13 +315,6 @@ final class OrderService
These are carried-forward, documented limitations of the M9 shipment — not bugs:
-- **Real brokers are OUT.** Only the in-memory and Laravel-queue adapters ship. Kafka/RabbitMQ/Postgres/Redis
- land as their own `firefly/eda-` packages at **SP-4**, each gated behind
- `#[ConditionalOnProperty('firefly.eda.provider', '')]`, patterned on `firefly/scheduling-postgres`.
-- **The durable transactional outbox is deferred to SP-4.** M9 gives at-least-once delivery within a
- process/queue, not a DB-transaction-to-broker durability guarantee. The substrate is already built and
- untouched — M8's `DB::afterCommit` hook (`DomainEventDispatcher`) and the in-house `PgAdvisoryLock`
- (`firefly/scheduling-postgres`) — so SP-4's `firefly/eda-postgres` relay is a clean future drop-in.
- **Avro/Protobuf serializers, event `filter`, and an event circuit-breaker are thin/deferred seams.** JSON
is the only shipped serializer; the others throw `SerializationException` until their own opt-in packages
land. Filter and circuit-breaker are not implemented in M9. Publisher actuator-health wiring is **M12**.
diff --git a/docs/modules/integration-testing.md b/docs/modules/integration-testing.md
index 3202b236..13a2b6f8 100644
--- a/docs/modules/integration-testing.md
+++ b/docs/modules/integration-testing.md
@@ -34,6 +34,35 @@ it('reads from a real Postgres container', function () {
});
```
+## Connecting to a real backend: `IntegrationDsn`
+
+Every integration test reads its backend from the environment through one helper,
+`Firefly\Testing\Integration\IntegrationDsn`, which accepts both `host=…;port=…;dbname=…;user=…;password=…` and
+URL-form DSNs (`pgsql://user:pass@host:5432/db`, `amqp://guest:guest@host:5672/%2f`), refuses anything else, and
+treats a blank variable as unset — so a missing backend skips its tests and never fails them:
+
+| Variable | Backend | Lane |
+|---|---|---|
+| `FIREFLY_PG_DSN` | PostgreSQL | `pgsql` |
+| `FIREFLY_MYSQL_DSN` | MySQL | `mysql` |
+| `FIREFLY_MARIADB_DSN` | MariaDB | `mariadb` |
+| `FIREFLY_SQLSRV_DSN` | SQL Server (`encrypt` defaults to `yes`; `trust_server_certificate` defaults to `false`, so a self-signed local or CI server must set `trust_server_certificate=true` in its DSN) | `sqlsrv` |
+| `FIREFLY_RABBITMQ_DSN`, `FIREFLY_KAFKA_BROKERS`, `FIREFLY_REDIS_URL` | the brokers | `brokers` |
+
+`IntegrationDsn::connectionConfig($lane)` returns a Laravel connection config, `IntegrationDsn::connect($lane)` a
+standalone connection (what a second test process uses), `IntegrationDsn::skipReason($lane)` the reason a lane
+cannot run here (or `null`). The `sqlite` lane needs no variable: it runs on a temporary file in WAL mode, so two
+processes can race over it.
+
+## CI lanes
+
+`composer test:integration` runs the `integration` group. CI runs it in an `integration` job per database server
+(PostgreSQL 16, MySQL 8.4, MariaDB 11, SQL Server 2022), each exporting only its own DSN, and in a `brokers` job
+(RabbitMQ, Kafka with `ext-rdkafka`, Redis). Each job also sets `FIREFLY_INTEGRATION_LANE`, and
+`tests/Integration/IntegrationLaneTest.php` then FAILS if that lane's variables are missing or its servers do not
+answer — a lane cannot go green by skipping everything. `docker-compose.integration.yml` at the repository root
+starts the same services locally (SQL Server behind the `sqlsrv` profile).
+
## `RequiresDocker` + `skipUnlessDocker()`
`Firefly\Testing\Integration\RequiresDocker` is a trait (`@mixin PHPUnit\Framework\TestCase`) that marks a test
@@ -180,5 +209,7 @@ available:
vendor/bin/pest --group=integration
```
-Docker-less environments that invoke `vendor/bin/pest --group=integration` still get a clean run: every test
-using `RequiresDocker::skipUnlessDocker()` reports **skipped**, not failed.
+`composer test:integration` is the same run with the memory limit collection needs. Docker-less environments
+that invoke `vendor/bin/pest --group=integration` still get a clean run: every test using
+`RequiresDocker::skipUnlessDocker()` — and every test whose `FIREFLY_*` variable is unset — reports
+**skipped**, not failed.
diff --git a/docs/modules/messaging.md b/docs/modules/messaging.md
index 7f72bd28..eb28a6cd 100644
--- a/docs/modules/messaging.md
+++ b/docs/modules/messaging.md
@@ -163,6 +163,12 @@ This is a **separate class** from eda's `DeadLetterStore` (`Firefly\Messaging\De
`Firefly\Eda\DeadLetter\DeadLetterStore`) — no shared code, no cross-package edge; it wraps a raw `Message`
rather than an `EventEnvelope`, faithful to the two ports' different payload shapes.
+`firefly.messaging.dead-letter.store` selects it: `memory` (default) or `database` (`DatabaseDeadLetterStore`, one
+`firefly_messaging_dead_letters` row per message on `firefly.messaging.dead-letter.connection`, the raw bytes
+base64-encoded so they round-trip on every driver; run `php artisan migrate`).
+`php artisan firefly:messaging:dead-letters:replay` republishes entries to their original topic (`--id`, `--topic`,
+`--limit`, `--dry-run`, `--keep`).
+
## Config keys
| Key | Type | Default | Meaning |
@@ -170,6 +176,8 @@ rather than an `EventEnvelope`, faithful to the two ports' different payload sha
| `firefly.messaging.provider` | `memory`\|`queue` | `memory` | Selects `InMemoryMessageBroker` or `QueueMessageBroker` (`MessagingAutoConfiguration`). |
| `firefly.messaging.queue.connection` | string\|null | `null` (default connection) | Queue connection `QueueMessageBroker`/`DispatchMessageJob` dispatch onto, when `provider=queue`. |
| `firefly.messaging.queue.name` | string\|null | `null` (default queue) | Queue name, when `provider=queue`. |
+| `firefly.messaging.dead-letter.store` | `memory`\|`database` | `memory` | Where a `#[MessageListener(deadLetterTopic: …)]` message goes once its retries are exhausted — see above. |
+| `firefly.messaging.dead-letter.connection` | string\|null | `null` | The connection `DatabaseDeadLetterStore` writes on (`null` = the default connection). |
There is no `firefly.messaging.retries`/`retry_delay` config key — retry/DLQ policy lives entirely on each
`#[MessageListener]` attribute (see above), not in configuration.
diff --git a/docs/modules/scheduling-database.md b/docs/modules/scheduling-database.md
new file mode 100644
index 00000000..bb1581c1
--- /dev/null
+++ b/docs/modules/scheduling-database.md
@@ -0,0 +1,41 @@
+# Scheduling Database Locks
+
+`firefly/scheduling-database` adds two `DistributedLock` backends for `#[Scheduled(lock: …)]` tasks, so a task runs
+on at most one node per tick without Redis or PostgreSQL. See [Scheduling](scheduling.md) for the port, the other
+backends and how a locked task is guarded.
+
+| `firefly.scheduling.lock.provider` | Class | Works on | TTL | Holds a connection while the task runs |
+|---|---|---|---|---|
+| `database` | `DatabaseLeaseLock` | any Laravel database connection | honoured on every driver | no |
+| `mysql` | `MySqlNamedLock` | MySQL, MariaDB | ignored (session-scoped) | yes — released when the session closes |
+
+## `DatabaseLeaseLock`
+
+One row per lock name in `firefly_scheduling_locks` (`name`, `locked_by`, `locked_until`, `locked_at`). Acquiring is
+a conditional `UPDATE` that takes over a lease whose `locked_until` has passed, or — when the row does not exist —
+an `INSERT`; a duplicate key means another node holds a live lease. The `INSERT` runs inside a savepoint when the
+caller already has a transaction open, so a lost race never aborts the caller's unit of work (on PostgreSQL a failed
+statement otherwise poisons the whole transaction). Releasing deletes the row only while it still carries this
+node's acquisition token, so a task that ran past its TTL cannot release the lease another node has taken since. A
+lock name is at most 255 characters; a longer one is refused rather than truncated.
+
+Run `php artisan migrate` after selecting the provider. Lease instants are the application's clock in UTC, so keep
+your nodes' clocks synchronised well within the lock TTLs.
+
+## `MySqlNamedLock`
+
+`GET_LOCK(name, 0)` / `RELEASE_LOCK(name)` — the MySQL-family counterpart of `PgAdvisoryLock`. The lock belongs to
+the database session and is freed when the connection closes; there is no table and no TTL. Names longer than the
+server's 64-character limit are hashed to `firefly:`, identically on every node. The connection must be a
+`mysql` or `mariadb` connection, or the first acquisition fails with a configuration error.
+
+## Configuration
+
+| Key | Default | Meaning |
+|---|---|---|
+| `firefly.scheduling.lock.provider` | `none` | `database` or `mysql` select this package's backends. |
+| `firefly.scheduling.lock.database.connection` | the default connection | The connection `DatabaseLeaseLock` writes its lease rows on. |
+| `firefly.scheduling.lock.mysql.connection` | the default connection | The MySQL or MariaDB connection `MySqlNamedLock` takes its named locks on. |
+
+Both backends are proven on every server they support by the `integration` CI lanes — see the
+[database support matrix](../database-support.md).
diff --git a/docs/modules/scheduling.md b/docs/modules/scheduling.md
index b2245b37..643ff837 100644
--- a/docs/modules/scheduling.md
+++ b/docs/modules/scheduling.md
@@ -33,6 +33,8 @@ always wins):
|---|---|---|---|
| *(unset)* / `none` | `NoneLock` | `firefly/scheduling` | **The default.** Every `tryAcquire()` succeeds and `release()` is a no-op — correct for a single-instance deployment (the only "node" always wins) and for a bare skeleton with no lock infrastructure, but provides **no real mutual exclusion** across nodes. |
| `cache` | `CacheLock` | `firefly/scheduling` | Wraps Laravel's atomic `Cache::lock()`. Works over any lock-capable driver (`database`, `redis`, `memcached`, and `array` for tests). The `null` cache driver grants **no-op** locks (every acquire "succeeds" with no real exclusion, same as `NoneLock`); `array` is **per-process only** — it does not coordinate across separate PHP-FPM workers or hosts. Production multi-node deployments need `database` or `redis`. |
+| `database` | `DatabaseLeaseLock` | `firefly/scheduling-database` | A lease row per lock name on any database connection; the TTL is honoured on every driver. See [Scheduling Database Locks](scheduling-database.md). |
+| `mysql` | `MySqlNamedLock` | `firefly/scheduling-database` | MySQL/MariaDB `GET_LOCK`, session-scoped, no TTL. |
| `postgres` | `PgAdvisoryLock` | `firefly/scheduling-postgres` | Session-scoped Postgres advisory lock (see below). Included in the root library; requires a PostgreSQL connection and `ext-pdo_pgsql`. |
`CacheLock::tryAcquire()` throws a `ConfigurationException` up front if the configured cache store isn't a
@@ -235,7 +237,9 @@ there would abort the whole `schedule:run` for that minute rather than skip one
| Key | Default | Meaning |
|---|---|---|
-| `firefly.scheduling.lock.provider` | `'none'` | The `DistributedLock` backend a locked task uses: `none`, `cache`, or `postgres` (needs `firefly/scheduling-postgres`) — see [The `DistributedLock` port](#the-distributedlock-port). |
+| `firefly.scheduling.lock.provider` | `'none'` | The `DistributedLock` backend a locked task uses: `none`, `cache`, `database`, `mysql` (both need `firefly/scheduling-database`) or `postgres` (needs `firefly/scheduling-postgres`) — see [The `DistributedLock` port](#the-distributedlock-port). |
+| `firefly.scheduling.lock.database.connection` | the default connection | The connection `DatabaseLeaseLock` writes its lease rows on. |
+| `firefly.scheduling.lock.mysql.connection` | the default connection | The MySQL or MariaDB connection `MySqlNamedLock` takes its named locks on. |
| `firefly.scheduling.initial-delay.enabled` | `true` | Apply `#[Scheduled(initialDelay:)]`. `false` **refuses to boot** an application whose manifest carries one, rather than accepting the parameter and ignoring it. |
| `firefly.scheduling.initial-delay.store` | `''` | Cache store the anchor lives in; `''` is the default store. Must be shared across processes (redis/memcached/database) wherever the scheduler is cron-driven. |
| `firefly.scheduling.initial-delay.release` | `''` | Names this deployment, so every release arms its own window. Empty keeps the permanent anchor — one window per cache key, for the life of the key. |
diff --git a/docs/modules/transactional.md b/docs/modules/transactional.md
index 41b448ed..f82fd159 100644
--- a/docs/modules/transactional.md
+++ b/docs/modules/transactional.md
@@ -139,7 +139,9 @@ max_execution_time` (milliseconds, `SELECT`s only) or mariadb `SET SESSION max_s
statement — mariadb has no `max_execution_time` variable, and a `mysql` connection whose server is mariadb is
detected through the server version), each with `innodb_lock_wait_timeout`, every previous value read first
and restored in a `finally` independently of the other, sqlite `PDO::ATTR_TIMEOUT` (the busy timeout — the
-only knob sqlite has) restored to the configured `busy_timeout` — and a wall-clock deadline is taken. When the
+only knob sqlite has) restored to the configured `busy_timeout`, sqlsrv `SET LOCK_TIMEOUT` (milliseconds, a
+lock wait only — error 1222, translated to `CannotAcquireLockException`) with the previous `@@LOCK_TIMEOUT`
+read first and restored — and a wall-clock deadline is taken. When the
method **returns** past that deadline the transaction is rolled back and `TransactionTimedOutException` (504
`TRANSACTION_TIMED_OUT`) is thrown; a method whose own exception ended it keeps that exception. A joined
`REQUIRED` and a `NESTED` savepoint run under the outer budget — Spring semantics.
@@ -386,7 +388,8 @@ $template->execute($work, new TransactionalDescriptor(
limits both. The driver-level statement timeout is best-effort in the same way: sqlite has only a busy
timeout (a merely slow statement is not interruptible, and the wall-clock check catches it on return), and
mysql's `max_execution_time` applies to `SELECT`s only (mariadb's `max_statement_time` interrupts any
- statement).
+ statement). SQL Server has no server-side knob for a statement that is merely slow: `LOCK_TIMEOUT` bounds
+ only the time a statement waits for a lock.
- **`#[TransactionalEventListener]` needs the connection's event dispatcher for `BEFORE_COMMIT`** (every
Laravel-configured connection has one; a hand-built `Connection` without `setEventDispatcher()` gets a
`ConfigurationException` at the first `BEFORE_COMMIT` registration). `order` sorts transactional
diff --git a/mkdocs.yml b/mkdocs.yml
index df54cf58..f1d4de6e 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -132,6 +132,7 @@ nav:
- Resilience & Scheduling:
- Resilience: modules/resilience.md
- Scheduling: modules/scheduling.md
+ - Scheduling Database Locks: modules/scheduling-database.md
- Data & Domain:
- Domain (DDD): modules/domain.md
- Data & Repositories: modules/data.md
@@ -140,6 +141,7 @@ nav:
- Eventing & Messaging:
- Event-Driven Architecture: modules/eda.md
- EDA Brokers: modules/eda-brokers.md
+ - EDA Database Outbox: modules/eda-database.md
- Messaging: modules/messaging.md
- CQRS:
- CQRS (Command/Query): modules/cqrs.md
@@ -163,6 +165,7 @@ nav:
- Installer: modules/installer.md
- Reference:
- CLI Reference: cli.md
+ - Database Support Matrix: database-support.md
- Feature Flag Contract: feature-flags-contract.md
- Versioning: versioning.md
- Contributing:
diff --git a/packages/cli/tests/Support/FoundationFlowPostgresTestCase.php b/packages/cli/tests/Support/FoundationFlowPostgresTestCase.php
index 1255641d..4b15d53f 100644
--- a/packages/cli/tests/Support/FoundationFlowPostgresTestCase.php
+++ b/packages/cli/tests/Support/FoundationFlowPostgresTestCase.php
@@ -4,6 +4,7 @@
namespace Firefly\Cli\Tests\Support;
+use Firefly\Testing\Integration\IntegrationDsn;
use Firefly\Testing\Integration\RequiresDocker;
use Illuminate\Contracts\Config\Repository;
use Illuminate\Foundation\Application;
@@ -14,8 +15,8 @@
* The @group integration variant of the Foundation harness: boots the SAME cached-path persistence slice
* (WidgetRepository derived query + FoundationWriter #[Transactional] commit) but against REAL Postgres instead of
* sqlite :memory:. Docker-gated via RequiresDocker (skips when Docker is absent). The Postgres backend is described
- * by FIREFLY_PG_DSN (host=…;port=…;dbname=…;user=…;password=…) — the exact opt-in mechanism scheduling-postgres'
- * PgAdvisoryLockIntegrationTest uses — wrapped in a duck-typed container and mapped into the Laravel `testing`
+ * by FIREFLY_PG_DSN (host=…;port=…;dbname=…;user=…;password=…) — parsed by firefly/testing's IntegrationDsn, the one
+ * parser every integration suite shares — wrapped in a duck-typed container and mapped into the Laravel `testing`
* connection through firefly/testing's fireflyConfigFor() @ServiceConnection helper (the same shape a
* testcontainers container exposes, so dropping in testcontainers/testcontainers later is a one-line swap). Only
* the DB connection differs from the sqlite pass; every other layer boots identically.
@@ -80,17 +81,11 @@ protected function defineSqliteMemory(Application $app): void
*/
private function resolvePostgres(): object
{
- $dsn = getenv('FIREFLY_PG_DSN');
- if (! is_string($dsn) || $dsn === '') {
+ $parts = IntegrationDsn::parts(IntegrationDsn::PGSQL);
+ if ($parts === null) {
$this->markTestSkipped('Set FIREFLY_PG_DSN (host=…;port=…;dbname=…;user=…;password=…) to run the @group integration Postgres pass.');
}
- $parts = [];
- foreach (explode(';', $dsn) as $kv) {
- [$key, $value] = array_pad(explode('=', $kv, 2), 2, '');
- $parts[$key] = $value;
- }
-
return new class($parts)
{
/** @param array $parts */
diff --git a/packages/data/src/Capability/DatabaseCapabilities.php b/packages/data/src/Capability/DatabaseCapabilities.php
new file mode 100644
index 00000000..d0387639
--- /dev/null
+++ b/packages/data/src/Capability/DatabaseCapabilities.php
@@ -0,0 +1,97 @@
+getConfig('driver');
+ $driver = is_string($configured) ? $configured : '';
+
+ try {
+ $pdo = $connection->getPdo();
+ $version = $pdo->getAttribute(PDO::ATTR_SERVER_VERSION);
+
+ if ($driver === '') {
+ $reported = $pdo->getAttribute(PDO::ATTR_DRIVER_NAME);
+ $driver = is_string($reported) ? $reported : '';
+ }
+ } catch (Throwable) {
+ $version = '';
+ }
+
+ return self::fromVersion($driver, is_string($version) ? $version : '');
+ }
+
+ public static function fromVersion(string $driver, string $serverVersion): self
+ {
+ return new self($driver, $serverVersion, $driver === 'mariadb' || stripos($serverVersion, 'mariadb') !== false);
+ }
+
+ public function version(): string
+ {
+ $raw = $this->serverVersion;
+
+ if ($this->mariaDb && str_starts_with($raw, '5.5.5-')) {
+ $raw = substr($raw, strlen('5.5.5-'));
+ }
+
+ return preg_match('/^\d+(?:\.\d+){0,2}/', $raw, $matches) === 1 ? $matches[0] : '0';
+ }
+
+ public function isMySqlFamily(): bool
+ {
+ return $this->driver === 'mysql' || $this->driver === 'mariadb';
+ }
+
+ public function supportsSkipLocked(): bool
+ {
+ return match (true) {
+ $this->driver === 'pgsql' => true,
+ $this->isMySqlFamily() && $this->mariaDb => version_compare($this->version(), '10.6', '>='),
+ $this->isMySqlFamily() => version_compare($this->version(), '8.0.1', '>='),
+ default => false,
+ };
+ }
+
+ public function supportsReadPast(): bool
+ {
+ return $this->driver === 'sqlsrv';
+ }
+
+ public function describe(): string
+ {
+ return $this->driver.' '.($this->serverVersion === '' ? '(unknown version)' : $this->serverVersion);
+ }
+}
diff --git a/packages/data/src/Transaction/Timeout/StatementTimeoutApplier.php b/packages/data/src/Transaction/Timeout/StatementTimeoutApplier.php
index b08b17fa..8097e4c8 100644
--- a/packages/data/src/Transaction/Timeout/StatementTimeoutApplier.php
+++ b/packages/data/src/Transaction/Timeout/StatementTimeoutApplier.php
@@ -35,6 +35,11 @@
* sqlite PDO::ATTR_TIMEOUT = , the busy timeout — the only knob sqlite has (a statement that is
* merely slow is not interruptible). PDO cannot read the attribute back, so the restore goes
* to the connection's configured `busy_timeout` (ms) or PDO_SQLITE's own 60 s default.
+ * sqlsrv SET LOCK_TIMEOUT — how long a statement waits for a lock before failing with error 1222
+ * (translated to CannotAcquireLockException). SQL Server has no server-side knob that interrupts a
+ * statement that is merely slow, so a lock wait is the part of the budget the database can enforce.
+ * The setting is SESSION-scoped, so the previous value (@@LOCK_TIMEOUT, -1 = wait forever by
+ * default) is read first and put back afterwards, exactly like the mysql variables.
* anything else nothing.
*
* Every statement is best-effort, exactly like the isolation SET: a driver that refuses it does not fail the
@@ -58,6 +63,7 @@ public function apply(Connection $connection, int $seconds): Closure
'mysql' => $this->isMaria($connection) ? $this->mariadb($connection, $seconds) : $this->mysql($connection, $seconds),
'mariadb' => $this->mariadb($connection, $seconds),
'sqlite' => $this->sqlite($connection, $seconds),
+ 'sqlsrv' => $this->sqlsrv($connection, $seconds),
default => $noop,
};
} catch (Throwable) {
@@ -161,6 +167,44 @@ private function sqlite(Connection $connection, int $seconds): Closure
};
}
+ /**
+ * @return Closure(): void
+ */
+ private function sqlsrv(Connection $connection, int $seconds): Closure
+ {
+ $noop = static function (): void {};
+
+ try {
+ $row = $connection->selectOne('select @@lock_timeout as value');
+ // unprepared(), never statement(): pdo_sqlsrv runs a prepared statement through sp_prepexec, and SQL Server
+ // restores a SET made inside it when that call returns, so the session would keep waiting forever.
+ /** @var literal-string $set an integer formatted by %d: nothing but digits reaches the batch */
+ $set = sprintf('set lock_timeout %d', $seconds * 1000);
+ $connection->unprepared($set);
+ } catch (Throwable) {
+ return $noop;
+ }
+
+ $previous = is_object($row) ? (get_object_vars($row)['value'] ?? null) : null;
+
+ if (! is_numeric($previous)) {
+ // Nothing came back (pretend(), a server that answers nothing): there is no value to put back.
+ return $noop;
+ }
+
+ $restore = (int) $previous;
+
+ return static function () use ($connection, $restore): void {
+ try {
+ /** @var literal-string $reset an integer formatted by %d */
+ $reset = sprintf('set lock_timeout %d', $restore);
+ $connection->unprepared($reset);
+ } catch (Throwable) {
+ // Best-effort, see the class docblock.
+ }
+ };
+ }
+
/**
* One step that runs every restore in reverse order of application, skipping the variables that were
* never set.
diff --git a/packages/data/tests/Capability/DatabaseCapabilitiesTest.php b/packages/data/tests/Capability/DatabaseCapabilitiesTest.php
new file mode 100644
index 00000000..47a2590d
--- /dev/null
+++ b/packages/data/tests/Capability/DatabaseCapabilitiesTest.php
@@ -0,0 +1,52 @@
+supportsSkipLocked())->toBe($skipLocked)
+ ->and($capabilities->mariaDb)->toBe($maria)
+ ->and($capabilities->version())->toBe($numeric);
+})->with([
+ 'postgres 16' => ['pgsql', '16.15', true, false, '16.15'],
+ 'postgres, version unreadable' => ['pgsql', '', true, false, '0'],
+ 'mysql 8.4' => ['mysql', '8.4.11', true, false, '8.4.11'],
+ 'mysql 8.0.1, the first with SKIP LOCKED' => ['mysql', '8.0.1', true, false, '8.0.1'],
+ 'mysql 8.0.0' => ['mysql', '8.0.0', false, false, '8.0.0'],
+ 'mysql 5.7' => ['mysql', '5.7.44-log', false, false, '5.7.44'],
+ 'mariadb 11 on the mariadb driver' => ['mariadb', '11.8.9-MariaDB-ubu2404', true, true, '11.8.9'],
+ 'mariadb 10.6' => ['mariadb', '10.6.18-MariaDB', true, true, '10.6.18'],
+ 'mariadb 10.5' => ['mariadb', '10.5.25-MariaDB', false, true, '10.5.25'],
+ 'mariadb behind a mysql driver, replication prefix' => ['mysql', '5.5.5-10.11.6-MariaDB-1:10.11.6+maria~ubu2204', true, true, '10.11.6'],
+ 'mariadb 10.4 behind a mysql driver' => ['mysql', '5.5.5-10.4.34-MariaDB', false, true, '10.4.34'],
+ 'sqlite' => ['sqlite', '3.46.1', false, false, '3.46.1'],
+ 'sqlsrv' => ['sqlsrv', '16.00.4135', false, false, '16.00.4135'],
+ 'an unknown driver' => ['oracle', '23.4', false, false, '23.4'],
+]);
+
+it('knows READPAST is a SQL Server table hint and nothing else', function () {
+ expect(DatabaseCapabilities::fromVersion('sqlsrv', '16.00.4135')->supportsReadPast())->toBeTrue()
+ ->and(DatabaseCapabilities::fromVersion('pgsql', '16.15')->supportsReadPast())->toBeFalse()
+ ->and(DatabaseCapabilities::fromVersion('mariadb', '11.4.2-MariaDB')->isMySqlFamily())->toBeTrue()
+ ->and(DatabaseCapabilities::fromVersion('sqlsrv', '16.00.4135')->isMySqlFamily())->toBeFalse()
+ ->and(DatabaseCapabilities::fromVersion('mysql', '8.4.11')->describe())->toBe('mysql 8.4.11')
+ ->and(DatabaseCapabilities::fromVersion('mysql', '')->describe())->toBe('mysql (unknown version)');
+});
+
+it('probes a live connection without throwing, reading the version PDO reports', function () {
+ $capabilities = DatabaseCapabilities::of(new SQLiteConnection(new PDO('sqlite::memory:'), ':memory:'));
+
+ expect($capabilities->driver)->toBe('sqlite')
+ ->and($capabilities->serverVersion)->toMatch('/^3\.\d+/')
+ ->and($capabilities->supportsSkipLocked())->toBeFalse();
+});
diff --git a/packages/data/tests/Integration/DatabaseCapabilitiesIntegrationTest.php b/packages/data/tests/Integration/DatabaseCapabilitiesIntegrationTest.php
new file mode 100644
index 00000000..23156f3c
--- /dev/null
+++ b/packages/data/tests/Integration/DatabaseCapabilitiesIntegrationTest.php
@@ -0,0 +1,39 @@
+version())->not->toBe('0')
+ ->and($capabilities->supportsSkipLocked())->toBe(in_array($lane, ['pgsql', 'mysql', 'mariadb'], true))
+ ->and($capabilities->supportsReadPast())->toBe($lane === 'sqlsrv')
+ ->and($capabilities->mariaDb)->toBe($lane === 'mariadb');
+
+ if ($capabilities->supportsSkipLocked()) {
+ $schema = $connection->getSchemaBuilder();
+ $schema->dropIfExists('firefly_capability_probe');
+ $schema->create('firefly_capability_probe', static function (Blueprint $table): void {
+ $table->bigIncrements('id');
+ });
+ $connection->table('firefly_capability_probe')->insert(['id' => 1]);
+
+ $rows = $connection->transaction(static fn () => $connection->table('firefly_capability_probe')->lock('for update skip locked')->get());
+
+ expect($rows)->toHaveCount(1);
+ $schema->drop('firefly_capability_probe');
+ }
+
+ $connection->disconnect();
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/data/tests/Integration/PersistenceExceptionTranslationIntegrationTest.php b/packages/data/tests/Integration/PersistenceExceptionTranslationIntegrationTest.php
new file mode 100644
index 00000000..cef091b4
--- /dev/null
+++ b/packages/data/tests/Integration/PersistenceExceptionTranslationIntegrationTest.php
@@ -0,0 +1,143 @@
+getSchemaBuilder();
+ $schema->dropIfExists('firefly_translation_children');
+ $schema->dropIfExists('firefly_translation_parents');
+ $schema->create('firefly_translation_parents', static function (Blueprint $table): void {
+ $table->integer('id')->primary();
+ $table->string('name');
+ });
+ $schema->create('firefly_translation_children', static function (Blueprint $table): void {
+ $table->integer('id')->primary();
+ $table->integer('parent_id');
+ $table->string('email')->unique();
+ $table->string('name');
+ $table->foreign('parent_id')->references('id')->on('firefly_translation_parents');
+ });
+ $connection->table('firefly_translation_parents')->insert(['id' => 1, 'name' => 'p']);
+ $connection->table('firefly_translation_children')->insert(['id' => 1, 'parent_id' => 1, 'email' => 'a@x.test', 'name' => 'c']);
+}
+
+function translationOf(Connection $connection, Closure $statement): ?Throwable
+{
+ try {
+ $statement();
+ } catch (Throwable $e) {
+ return (new PersistenceExceptionTranslator)->translate($e, $connection->getDriverName());
+ }
+
+ return null;
+}
+
+/** Make $connection give up on a lock wait quickly, in the server's own dialect. */
+function translationShortLockWait(Connection $connection, string $lane): void
+{
+ match ($lane) {
+ 'pgsql' => $connection->statement("set lock_timeout = '300ms'"),
+ 'mysql', 'mariadb' => $connection->statement('set session innodb_lock_wait_timeout = 1'),
+ // A prepared SET is undone when sp_prepexec returns on SQL Server; only a direct batch sticks to the session.
+ 'sqlsrv' => $connection->unprepared('set lock_timeout 300'),
+ default => null,
+ };
+}
+
+foreach (IntegrationDsn::lanes() as $lane) {
+ it("translates integrity, grammar and lock-wait failures on {$lane}", function () use ($lane): void {
+ $overrides = $lane === 'sqlite' ? ['database' => IntegrationDsn::sqlitePath('translation'), 'busy_timeout' => 300] : [];
+ $a = IntegrationDsn::connect($lane, 'translation-a', $overrides);
+ $b = IntegrationDsn::connect($lane, 'translation-b', $overrides);
+ translationSchema($a);
+
+ try {
+ $children = static fn (Connection $c) => $c->table('firefly_translation_children');
+
+ expect(translationOf($a, static fn () => $children($a)->insert(['id' => 2, 'parent_id' => 1, 'email' => 'a@x.test', 'name' => 'dup'])))->toBeInstanceOf(DuplicateKeyException::class)
+ ->and(translationOf($a, static fn () => $children($a)->insert(['id' => 1, 'parent_id' => 1, 'email' => 'b@x.test', 'name' => 'pk'])))->toBeInstanceOf(DuplicateKeyException::class)
+ ->and(translationOf($a, static fn () => $children($a)->insert(['id' => 3, 'parent_id' => 1, 'email' => 'c@x.test', 'name' => null])))->toBeInstanceOf(DataIntegrityViolationException::class)
+ ->and(translationOf($a, static fn () => $children($a)->insert(['id' => 4, 'parent_id' => 99, 'email' => 'd@x.test', 'name' => 'fk'])))->toBeInstanceOf(DataIntegrityViolationException::class)
+ ->and(translationOf($a, static fn () => $a->table('firefly_translation_missing')->get()))->toBeInstanceOf(BadSqlGrammarException::class);
+
+ // A lock wait that times out: a holds the row (sqlite: the database) for writing, b gives up.
+ $a->beginTransaction();
+ $a->table('firefly_translation_parents')->where('id', 1)->update(['name' => 'held']);
+ translationShortLockWait($b, $lane);
+ $waited = translationOf($b, static fn () => $b->table('firefly_translation_parents')->where('id', 1)->update(['name' => 'waiter']));
+ $a->rollBack();
+
+ expect($waited)->toBeInstanceOf(CannotAcquireLockException::class);
+ } finally {
+ $schema = $a->getSchemaBuilder();
+ $schema->dropIfExists('firefly_translation_children');
+ $schema->dropIfExists('firefly_translation_parents');
+ $a->disconnect();
+ $b->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
+
+foreach (['pgsql', 'mysql', 'mariadb', 'sqlsrv'] as $lane) {
+ it("translates the deadlock victim on {$lane}, whichever process the server picks", function () use ($lane): void {
+ $connection = IntegrationDsn::connect($lane, 'deadlock-parent');
+ $schema = $connection->getSchemaBuilder();
+ $schema->dropIfExists('firefly_deadlock_probe');
+ $schema->create('firefly_deadlock_probe', static function (Blueprint $table): void {
+ $table->integer('id')->primary();
+ $table->string('name');
+ });
+ $connection->table('firefly_deadlock_probe')->insert([['id' => 1, 'name' => 'a'], ['id' => 2, 'name' => 'b']]);
+
+ $go = sys_get_temp_dir().'/firefly-deadlock-'.bin2hex(random_bytes(6));
+ $racer = new Process([PHP_BINARY, dirname(__DIR__).'/Support/deadlock-racer.php', $lane, $go], timeout: 40);
+
+ try {
+ $connection->beginTransaction();
+ $connection->table('firefly_deadlock_probe')->where('id', 1)->update(['name' => 'parent']);
+ $racer->start();
+ expect($racer->waitUntil(static fn (string $type, string $output): bool => str_contains($output, 'locked')))->toBeTrue($racer->getErrorOutput());
+ touch($go);
+ usleep(300_000); // the racer is now waiting for row 1
+
+ $parent = translationOf($connection, static fn () => $connection->table('firefly_deadlock_probe')->where('id', 2)->update(['name' => 'parent']));
+ if ($parent === null) {
+ $connection->commit();
+ } elseif ($connection->transactionLevel() > 0) {
+ $connection->rollBack();
+ }
+
+ expect($racer->wait())->toBe(0, $racer->getErrorOutput());
+ $racerSaw = trim($racer->getOutput());
+
+ // Exactly one side lost, and it was told so in the kernel's own vocabulary.
+ expect($parent instanceof DeadlockLoserDataAccessException xor str_ends_with($racerSaw, 'deadlock'))->toBeTrue('parent: '.($parent === null ? 'committed' : $parent::class)."; racer: {$racerSaw}")
+ ->and($parent === null || $parent instanceof DeadlockLoserDataAccessException)->toBeTrue();
+ } finally {
+ $racer->stop();
+ @unlink($go);
+ if ($connection->transactionLevel() > 0) {
+ $connection->rollBack();
+ }
+ $schema->dropIfExists('firefly_deadlock_probe');
+ $connection->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/data/tests/Integration/RepositoryLockingIntegrationTest.php b/packages/data/tests/Integration/RepositoryLockingIntegrationTest.php
new file mode 100644
index 00000000..616aa271
--- /dev/null
+++ b/packages/data/tests/Integration/RepositoryLockingIntegrationTest.php
@@ -0,0 +1,130 @@
+scan([
+ 'Firefly\\Data\\Tests\\Fixtures\\Repository\\' => dirname(__DIR__).'/Fixtures/Repository',
+ ]));
+}
+
+function lockingLane(string $lane): Connection
+{
+ $overrides = $lane === 'sqlite' ? ['database' => IntegrationDsn::sqlitePath('locking')] : [];
+ config([
+ 'database.connections.it_owner' => IntegrationDsn::connectionConfig($lane, $overrides),
+ 'database.connections.it_other' => IntegrationDsn::connectionConfig($lane, $overrides),
+ ]);
+ DB::setDefaultConnection('it_owner');
+ Schema::dropIfExists('records');
+ Schema::dropIfExists('versioned_records');
+ Schema::create('records', function (Blueprint $table): void {
+ $table->increments('id');
+ $table->string('status');
+ $table->integer('amount');
+ $table->string('email')->nullable();
+ });
+ Schema::create('versioned_records', function (Blueprint $table): void {
+ $table->increments('id');
+ $table->string('name');
+ $table->integer('version')->default(0);
+ });
+
+ return DB::connection('it_other');
+}
+
+function lockingWriteFromOther(Connection $other, string $lane): ?Throwable
+{
+ match ($lane) {
+ 'pgsql' => $other->statement("set lock_timeout = '300ms'"),
+ 'mysql', 'mariadb' => $other->statement('set session innodb_lock_wait_timeout = 1'),
+ 'sqlsrv' => $other->unprepared('set lock_timeout 300'), // a prepared SET is undone when sp_prepexec returns
+ default => null,
+ };
+
+ try {
+ $other->table('records')->where('id', 1)->update(['status' => 'stolen']);
+ } catch (Throwable $e) {
+ return (new PersistenceExceptionTranslator)->translate($e, $other->getDriverName());
+ }
+
+ return null;
+}
+
+foreach (['pgsql', 'mysql', 'mariadb', 'sqlsrv'] as $lane) {
+ it("holds the row under PESSIMISTIC_WRITE and PESSIMISTIC_READ on {$lane}", function () use ($lane): void {
+ $other = lockingLane($lane);
+ Record::query()->create(['status' => 'open', 'amount' => 50, 'email' => 'a@x.test']);
+ $repository = integrationLockingRepository();
+
+ try {
+ DB::transaction(function () use ($repository, $other, $lane): void {
+ expect($repository->findByIdForUpdate(1)?->email)->toBe('a@x.test')
+ ->and(lockingWriteFromOther($other, $lane))->toBeInstanceOf(CannotAcquireLockException::class);
+ });
+ DB::transaction(function () use ($repository, $other, $lane): void {
+ expect($repository->findByAmountBetween(1, 100))->toHaveCount(1)
+ ->and(lockingWriteFromOther($other, $lane))->toBeInstanceOf(CannotAcquireLockException::class);
+ });
+
+ // Outside the transaction the row is free again.
+ expect(lockingWriteFromOther($other, $lane))->toBeNull();
+ } finally {
+ Schema::dropIfExists('records');
+ Schema::dropIfExists('versioned_records');
+ DB::connection('it_owner')->disconnect();
+ $other->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
+
+foreach (IntegrationDsn::lanes() as $lane) {
+ it("refuses a stale versioned save on {$lane}", function () use ($lane): void {
+ $other = lockingLane($lane);
+
+ try {
+ $repository = new VersionedRecordRepository;
+ $id = $repository->save(new VersionedRecord(['name' => 'orig']))->id;
+ $first = $repository->findById($id);
+ $second = $repository->findById($id);
+ assert($first instanceof VersionedRecord && $second instanceof VersionedRecord);
+
+ $first->name = 'first';
+ $repository->save($first);
+ $second->name = 'second';
+
+ expect(fn () => $repository->save($second))->toThrow(OptimisticLockException::class)
+ ->and($repository->findById($id)?->name)->toBe('first')
+ ->and($repository->findById($id)?->version)->toEqual(1);
+ } finally {
+ Schema::dropIfExists('records');
+ Schema::dropIfExists('versioned_records');
+ DB::connection('it_owner')->disconnect();
+ $other->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/data/tests/Integration/StatementTimeoutApplierIntegrationTest.php b/packages/data/tests/Integration/StatementTimeoutApplierIntegrationTest.php
new file mode 100644
index 00000000..68f88a33
--- /dev/null
+++ b/packages/data/tests/Integration/StatementTimeoutApplierIntegrationTest.php
@@ -0,0 +1,138 @@
+selectOne($sql);
+
+ return is_object($row) ? (get_object_vars($row)['value'] ?? null) : null;
+}
+
+function timeoutTranslated(Closure $statement, string $driver): ?Throwable
+{
+ try {
+ $statement();
+ } catch (Throwable $e) {
+ return (new PersistenceExceptionTranslator)->translate($e, $driver);
+ }
+
+ return null;
+}
+
+it('interrupts a slow statement on pgsql with SET LOCAL statement_timeout', function () {
+ $connection = IntegrationDsn::connect('pgsql', 'timeout-pgsql');
+ $connection->beginTransaction();
+ $restore = (new StatementTimeoutApplier)->apply($connection, 1);
+
+ $started = microtime(true);
+ $error = timeoutTranslated(static fn () => $connection->select('select pg_sleep(3)'), 'pgsql');
+
+ expect($error)->toBeInstanceOf(QueryTimeoutException::class)
+ ->and(microtime(true) - $started)->toBeLessThan(2.5);
+ $connection->rollBack();
+ $restore();
+ $connection->disconnect();
+})->group('integration')->skip(IntegrationDsn::skipReason('pgsql') !== null, (string) IntegrationDsn::skipReason('pgsql'));
+
+it('interrupts any statement past max_statement_time on mariadb, then restores it', function () {
+ $connection = IntegrationDsn::connect('mariadb', 'timeout-mariadb');
+ $before = timeoutValue($connection, 'select @@session.max_statement_time as value');
+ $connection->beginTransaction();
+ $restore = (new StatementTimeoutApplier)->apply($connection, 1);
+
+ $error = timeoutTranslated(static fn () => $connection->select('select sleep(3)'), 'mariadb');
+ $connection->rollBack();
+ $restore();
+
+ expect($error)->toBeInstanceOf(QueryTimeoutException::class)
+ ->and(timeoutValue($connection, 'select @@session.max_statement_time as value'))->toEqual($before);
+ $connection->disconnect();
+})->group('integration')->skip(IntegrationDsn::skipReason('mariadb') !== null, (string) IntegrationDsn::skipReason('mariadb'));
+
+it('bounds a read with max_execution_time on mysql, then restores both session variables', function () {
+ $connection = IntegrationDsn::connect('mysql', 'timeout-mysql');
+ $executionBefore = timeoutValue($connection, 'select @@session.max_execution_time as value');
+ $lockBefore = timeoutValue($connection, 'select @@session.innodb_lock_wait_timeout as value');
+ $connection->beginTransaction();
+ $restore = (new StatementTimeoutApplier)->apply($connection, 1);
+
+ expect(timeoutValue($connection, 'select @@session.max_execution_time as value'))->toEqual(1000)
+ ->and(timeoutValue($connection, 'select @@session.innodb_lock_wait_timeout as value'))->toEqual(1);
+
+ // MySQL interrupts SLEEP() inside a timed-out SELECT either with ER_QUERY_TIMEOUT (3024) or by returning 1; both
+ // end the statement inside the budget, which is the property that matters.
+ $started = microtime(true);
+ $error = timeoutTranslated(static fn () => $connection->select('select sleep(3) as slept'), 'mysql');
+ expect(microtime(true) - $started)->toBeLessThan(2.5)
+ ->and($error === null || $error instanceof QueryTimeoutException)->toBeTrue();
+
+ $connection->rollBack();
+ $restore();
+ expect(timeoutValue($connection, 'select @@session.max_execution_time as value'))->toEqual($executionBefore)
+ ->and(timeoutValue($connection, 'select @@session.innodb_lock_wait_timeout as value'))->toEqual($lockBefore);
+ $connection->disconnect();
+})->group('integration')->skip(IntegrationDsn::skipReason('mysql') !== null, (string) IntegrationDsn::skipReason('mysql'));
+
+it('fails a lock wait past LOCK_TIMEOUT on sqlsrv, then restores the session value', function () {
+ $holder = IntegrationDsn::connect('sqlsrv', 'timeout-holder');
+ $waiter = IntegrationDsn::connect('sqlsrv', 'timeout-waiter');
+ $schema = $holder->getSchemaBuilder();
+ $schema->dropIfExists('firefly_timeout_probe');
+ $schema->create('firefly_timeout_probe', static function (Blueprint $table): void {
+ $table->integer('id')->primary();
+ $table->string('name');
+ });
+ $holder->table('firefly_timeout_probe')->insert(['id' => 1, 'name' => 'a']);
+
+ try {
+ $holder->beginTransaction();
+ $holder->table('firefly_timeout_probe')->where('id', 1)->update(['name' => 'held']);
+
+ $before = timeoutValue($waiter, 'select @@lock_timeout as value');
+ $waiter->beginTransaction();
+ $restore = (new StatementTimeoutApplier)->apply($waiter, 1);
+ $error = timeoutTranslated(static fn () => $waiter->table('firefly_timeout_probe')->where('id', 1)->update(['name' => 'waiter']), 'sqlsrv');
+ $waiter->rollBack();
+ $restore();
+
+ expect($error)->toBeInstanceOf(CannotAcquireLockException::class)
+ ->and(timeoutValue($waiter, 'select @@lock_timeout as value'))->toEqual($before);
+ } finally {
+ $holder->rollBack();
+ $schema->dropIfExists('firefly_timeout_probe');
+ $holder->disconnect();
+ $waiter->disconnect();
+ }
+})->group('integration')->skip(IntegrationDsn::skipReason('sqlsrv') !== null, (string) IntegrationDsn::skipReason('sqlsrv'));
+
+it('sets and restores the busy timeout on a sqlite file without failing the unit of work', function () {
+ $path = IntegrationDsn::sqlitePath('timeout');
+ $connection = IntegrationDsn::connect('sqlite', 'timeout-sqlite', ['database' => $path]);
+ $connection->beginTransaction();
+ $restore = (new StatementTimeoutApplier)->apply($connection, 1);
+
+ expect($connection->selectOne('select 1 as one'))->not->toBeNull();
+ $connection->commit();
+ $restore();
+ $connection->disconnect();
+ foreach ([$path, $path.'-wal', $path.'-shm'] as $file) {
+ if (is_file($file)) {
+ unlink($file);
+ }
+ }
+})->group('integration');
diff --git a/packages/data/tests/Support/deadlock-racer.php b/packages/data/tests/Support/deadlock-racer.php
new file mode 100644
index 00000000..c7338c5f
--- /dev/null
+++ b/packages/data/tests/Support/deadlock-racer.php
@@ -0,0 +1,40 @@
+beginTransaction();
+$connection->table('firefly_deadlock_probe')->where('id', 2)->update(['name' => 'racer']);
+fwrite(STDOUT, "locked\n");
+fflush(STDOUT);
+
+$deadline = microtime(true) + 20;
+while (! is_file($go) && microtime(true) < $deadline) {
+ usleep(1000);
+}
+
+try {
+ $connection->table('firefly_deadlock_probe')->where('id', 1)->update(['name' => 'racer']);
+ $connection->commit();
+ fwrite(STDOUT, "committed\n");
+} catch (Throwable $e) {
+ $connection->rollBack();
+ $translated = (new PersistenceExceptionTranslator)->translate($e, $connection->getDriverName());
+ fwrite(STDOUT, ($translated instanceof DeadlockLoserDataAccessException ? 'deadlock' : 'other '.$translated::class)."\n");
+}
+
+$connection->disconnect();
diff --git a/packages/data/tests/Transaction/StatementTimeoutApplierTest.php b/packages/data/tests/Transaction/StatementTimeoutApplierTest.php
index b6fe1778..2da3178e 100644
--- a/packages/data/tests/Transaction/StatementTimeoutApplierTest.php
+++ b/packages/data/tests/Transaction/StatementTimeoutApplierTest.php
@@ -9,6 +9,7 @@
use Illuminate\Database\MySqlConnection;
use Illuminate\Database\PostgresConnection;
use Illuminate\Database\SQLiteConnection;
+use Illuminate\Database\SqlServerConnection;
/**
* A driver-grammar connection over an in-memory sqlite PDO: under pretend() nothing executes, so the exact
@@ -168,3 +169,72 @@ public function statement($query, $bindings = [])
expect($phases['applied'])->toBe([])
->and($phases['restored'])->toBe([]);
});
+
+it('sets SQL Server\'s session LOCK_TIMEOUT in milliseconds, restoring nothing it did not read', function () {
+ $phases = applyAndRestore(pretendConnection(SqlServerConnection::class, 'sqlsrv'), 5);
+
+ expect($phases['applied'])->toBe(['select @@lock_timeout as value', 'set lock_timeout 5000'])
+ ->and($phases['restored'])->toBe([]);
+});
+
+it('puts SQL Server\'s previous LOCK_TIMEOUT back — -1, wait forever, is the server default', function () {
+ $connection = new class(new PDO('sqlite::memory:'), 'pretend', '', ['driver' => 'sqlsrv', 'name' => 'pretend']) extends SqlServerConnection
+ {
+ /**
+ * @param string $query
+ * @param array $bindings
+ * @param bool $useReadPdo
+ */
+ public function selectOne($query, $bindings = [], $useReadPdo = true)
+ {
+ parent::selectOne($query, $bindings, $useReadPdo);
+
+ return str_contains($query, '@@lock_timeout') ? (object) ['value' => -1] : null;
+ }
+ };
+
+ $phases = applyAndRestore($connection, 2);
+
+ expect($phases['applied'])->toBe(['select @@lock_timeout as value', 'set lock_timeout 2000'])
+ ->and($phases['restored'])->toBe(['set lock_timeout -1']);
+});
+
+it('sets SQL Server\'s LOCK_TIMEOUT as a direct batch, because a prepared SET does not outlive sp_prepexec', function () {
+ $connection = new class(new PDO('sqlite::memory:'), 'pretend', '', ['driver' => 'sqlsrv', 'name' => 'pretend']) extends SqlServerConnection
+ {
+ /** @var list */
+ public array $direct = [];
+
+ /**
+ * @param string $query
+ * @param array $bindings
+ * @param bool $useReadPdo
+ */
+ public function selectOne($query, $bindings = [], $useReadPdo = true)
+ {
+ return (object) ['value' => -1];
+ }
+
+ /**
+ * @param string $query
+ * @param array $bindings
+ */
+ public function statement($query, $bindings = []): bool
+ {
+ throw new LogicException("prepared statement used for [{$query}]");
+ }
+
+ /** @param string $query */
+ public function unprepared($query): bool
+ {
+ $this->direct[] = $query;
+
+ return true;
+ }
+ };
+
+ $restore = (new StatementTimeoutApplier)->apply($connection, 3);
+ $restore();
+
+ expect($connection->direct)->toBe(['set lock_timeout 3000', 'set lock_timeout -1']);
+});
diff --git a/packages/eda-database/.gitattributes b/packages/eda-database/.gitattributes
new file mode 100644
index 00000000..538b69a4
--- /dev/null
+++ b/packages/eda-database/.gitattributes
@@ -0,0 +1,2 @@
+/tests export-ignore
+/.gitattributes export-ignore
diff --git a/packages/eda-database/LICENSE b/packages/eda-database/LICENSE
new file mode 100644
index 00000000..2240005d
--- /dev/null
+++ b/packages/eda-database/LICENSE
@@ -0,0 +1,204 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+Copyright 2026 Firefly Software Solutions Inc.
diff --git a/packages/eda-database/README.md b/packages/eda-database/README.md
new file mode 100644
index 00000000..8d50d0ff
--- /dev/null
+++ b/packages/eda-database/README.md
@@ -0,0 +1,12 @@
+# firefly/eda-database
+
+LaraFly's portable same-transaction outbox. With `firefly.eda.provider=database`, an `EventPublisher::publish()` —
+and every domain event, through the pre-commit hook — writes a `firefly_eda_outbox` row on the aggregate's own
+connection inside its open transaction, on PostgreSQL, MySQL, MariaDB, SQL Server or SQLite. `firefly:eda:consume`
+delivers committed rows to `#[EventListener]` handlers and the optional `firefly:outbox:relay` forwards them to a
+downstream broker. Rows are claimed by leases through an `OutboxClaimStrategy` (SKIP LOCKED, READPAST or a portable
+optimistic lease, chosen from the server's capabilities), so any number of consumers and relays may run at once.
+`firefly/eda-postgres` builds on this package and adds `pg_notify`/`LISTEN`. See
+[docs/modules/eda-database.md](../../docs/modules/eda-database.md).
+
+Apache-2.0 © Firefly Software Solutions Inc.
diff --git a/packages/eda-database/cache/firefly-eda-database-components.php b/packages/eda-database/cache/firefly-eda-database-components.php
new file mode 100644
index 00000000..f1f50859
--- /dev/null
+++ b/packages/eda-database/cache/firefly-eda-database-components.php
@@ -0,0 +1,124 @@
+ [
+ 'class' => 'Firefly\\Eda\\Database\\DatabaseOutboxAutoConfiguration',
+ 'stereotype' => 'configuration',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 900,
+ 'qualifier' => null,
+ 'interfaces' => [
+ ],
+ 'beans' => [
+ 0 => [
+ 'method' => 'subscriberRegistry',
+ 'returns' => 'Firefly\\Eda\\Bus\\SubscriberRegistry',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ ],
+ ],
+ 1 => [
+ 'method' => 'eventPublisher',
+ 'returns' => 'Firefly\\Eda\\EventPublisher',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Firefly\\Config\\Config',
+ 1 => 'Illuminate\\Database\\ConnectionResolverInterface',
+ 2 => 'Firefly\\Eda\\Bus\\SubscriberRegistry',
+ 3 => 'Firefly\\Eda\\Tracing\\EdaTracing',
+ ],
+ ],
+ 2 => [
+ 'method' => 'outboxPreCommitHook',
+ 'returns' => 'Firefly\\Eda\\Database\\Outbox\\OutboxPreCommitHook',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Illuminate\\Database\\ConnectionResolverInterface',
+ 1 => 'Firefly\\Config\\Config',
+ 2 => 'Firefly\\Cqrs\\Handler\\HandlerManifest',
+ 3 => 'Firefly\\Cqrs\\Correlation\\CorrelationContext',
+ 4 => 'Firefly\\Eda\\Bus\\SubscriberRegistry',
+ 5 => 'Firefly\\Eda\\Tracing\\EdaTracing',
+ ],
+ ],
+ 3 => [
+ 'method' => 'domainEventDispatcher',
+ 'returns' => 'Firefly\\Data\\Domain\\DomainEventDispatcher',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Firefly\\Data\\Domain\\AggregateTracker',
+ 1 => 'Firefly\\Context\\Event\\ApplicationEventPublisher',
+ 2 => 'Firefly\\Eda\\Database\\Outbox\\OutboxPreCommitHook',
+ ],
+ ],
+ 4 => [
+ 'method' => 'commandEventPublisher',
+ 'returns' => 'Firefly\\Cqrs\\Event\\CommandEventPublisher',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ ],
+ ],
+ 5 => [
+ 'method' => 'eventConsumer',
+ 'returns' => 'Firefly\\Eda\\Consumer\\EventConsumer',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Firefly\\Config\\Config',
+ 1 => 'Illuminate\\Database\\ConnectionResolverInterface',
+ ],
+ ],
+ ],
+ 'lazy' => false,
+ 'dependencies' => [
+ ],
+ ],
+ 1 => [
+ 'class' => 'Firefly\\Eda\\Database\\OutboxHealthIndicator',
+ 'stereotype' => 'component',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'qualifier' => null,
+ 'interfaces' => [
+ 0 => 'Firefly\\Actuator\\Health\\HealthIndicator',
+ ],
+ 'beans' => [
+ ],
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Illuminate\\Database\\ConnectionResolverInterface',
+ 1 => 'Firefly\\Config\\Config',
+ ],
+ ],
+];
diff --git a/packages/eda-database/cache/firefly-eda-database-context.php b/packages/eda-database/cache/firefly-eda-database-context.php
new file mode 100644
index 00000000..adf9214e
--- /dev/null
+++ b/packages/eda-database/cache/firefly-eda-database-context.php
@@ -0,0 +1,120 @@
+ [
+ 'class' => 'Firefly\\Eda\\Database\\DatabaseOutboxAutoConfiguration',
+ 'postConstruct' => [
+ ],
+ 'preDestroy' => [
+ ],
+ 'listeners' => [
+ ],
+ 'conditions' => [
+ ],
+ 'beanConditions' => [
+ 0 => [
+ 'method' => 'subscriberRegistry',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ 1 => [
+ 'method' => 'eventPublisher',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ 2 => [
+ 'method' => 'outboxPreCommitHook',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ 3 => [
+ 'method' => 'domainEventDispatcher',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ 4 => [
+ 'method' => 'commandEventPublisher',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ 5 => [
+ 'method' => 'eventConsumer',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ ],
+ ],
+ 1 => [
+ 'class' => 'Firefly\\Eda\\Database\\OutboxHealthIndicator',
+ 'postConstruct' => [
+ ],
+ 'preDestroy' => [
+ ],
+ 'listeners' => [
+ ],
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.eda.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ 'beanConditions' => [
+ ],
+ ],
+];
diff --git a/packages/eda-database/composer.json b/packages/eda-database/composer.json
new file mode 100644
index 00000000..3f22da00
--- /dev/null
+++ b/packages/eda-database/composer.json
@@ -0,0 +1,65 @@
+{
+ "name": "firefly/eda-database",
+ "description": "LaraFly eda database adapter: a GENUINE same-transaction firefly_eda_outbox on any Laravel database connection (PostgreSQL, MySQL, MariaDB, SQL Server, SQLite), claimed by leases through an OutboxClaimStrategy port (SKIP LOCKED, READPAST or a portable optimistic lease, chosen from the server's capabilities) so any number of consumers and relays can run concurrently; an in-process consumer for #[EventListener] handlers and an OPTIONAL firefly:outbox:relay to a downstream broker. Auto-configured behind firefly.eda.provider=database.",
+ "type": "library",
+ "license": "Apache-2.0",
+ "homepage": "https://github.com/fireflyframework/fireflyframework-php",
+ "authors": [
+ {
+ "name": "Firefly Software Solutions Inc.",
+ "homepage": "https://github.com/fireflyframework"
+ }
+ ],
+ "keywords": [
+ "firefly",
+ "laravel",
+ "outbox",
+ "database"
+ ],
+ "support": {
+ "issues": "https://github.com/fireflyframework/fireflyframework-php/issues",
+ "source": "https://github.com/fireflyframework/fireflyframework-php/tree/main/packages/eda-database"
+ },
+ "require": {
+ "php": "^8.3",
+ "firefly/actuator": "*@dev",
+ "firefly/autoconfigure": "*@dev",
+ "firefly/config": "*@dev",
+ "firefly/container": "*@dev",
+ "firefly/context": "*@dev",
+ "firefly/cqrs": "*@dev",
+ "firefly/data": "*@dev",
+ "firefly/domain": "*@dev",
+ "firefly/eda": "*@dev",
+ "firefly/kernel": "*@dev",
+ "illuminate/console": "^13.0",
+ "illuminate/contracts": "^13.0",
+ "illuminate/database": "^13.0",
+ "illuminate/support": "^13.0"
+ },
+ "extra": {
+ "laravel": {
+ "providers": [
+ "Firefly\\Eda\\Database\\EdaDatabaseServiceProvider",
+ "Firefly\\Eda\\Database\\EdaDatabaseBootServiceProvider"
+ ]
+ },
+ "branch-alias": {
+ "dev-main": "26.x-dev"
+ }
+ },
+ "autoload": {
+ "psr-4": {
+ "Firefly\\Eda\\Database\\": "src/"
+ }
+ },
+ "autoload-dev": {
+ "psr-4": {
+ "Firefly\\Eda\\Database\\Tests\\": "tests/"
+ }
+ },
+ "minimum-stability": "stable",
+ "config": {
+ "sort-packages": true
+ }
+}
diff --git a/packages/eda-postgres/database/migrations/2026_07_27_000000_create_firefly_eda_outbox_table.php b/packages/eda-database/database/migrations/2026_07_27_000000_create_firefly_eda_outbox_table.php
similarity index 90%
rename from packages/eda-postgres/database/migrations/2026_07_27_000000_create_firefly_eda_outbox_table.php
rename to packages/eda-database/database/migrations/2026_07_27_000000_create_firefly_eda_outbox_table.php
index c11c7cd2..e0e144fe 100644
--- a/packages/eda-postgres/database/migrations/2026_07_27_000000_create_firefly_eda_outbox_table.php
+++ b/packages/eda-database/database/migrations/2026_07_27_000000_create_firefly_eda_outbox_table.php
@@ -2,7 +2,7 @@
declare(strict_types=1);
-use Firefly\Eda\Postgres\Outbox\OutboxSchema;
+use Firefly\Eda\Database\Outbox\OutboxSchema;
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
diff --git a/packages/eda-database/database/migrations/2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php b/packages/eda-database/database/migrations/2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php
new file mode 100644
index 00000000..50e1826d
--- /dev/null
+++ b/packages/eda-database/database/migrations/2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php
@@ -0,0 +1,36 @@
+ OutboxSchema::claimColumns($table));
+ }
+
+ public function down(): void
+ {
+ if (! Schema::hasTable(OutboxSchema::TABLE) || ! Schema::hasColumn(OutboxSchema::TABLE, 'claimed_by')) {
+ return;
+ }
+
+ Schema::table(OutboxSchema::TABLE, function (Blueprint $table): void {
+ $table->dropIndex(OutboxSchema::CLAIM_INDEX);
+ $table->dropColumn(['claimed_by', 'claimed_until']);
+ });
+ }
+};
diff --git a/packages/eda-database/src/Console/OutboxRelayCommand.php b/packages/eda-database/src/Console/OutboxRelayCommand.php
new file mode 100644
index 00000000..d263f3fa
--- /dev/null
+++ b/packages/eda-database/src/Console/OutboxRelayCommand.php
@@ -0,0 +1,103 @@
+ publish -> mark PUBLISHED/FAILED).';
+
+ public function handle(ConnectionResolverInterface $connections, Container $container, Config $config): int
+ {
+ try {
+ $downstream = RelayDownstream::resolve($container, $config);
+ $settings = DatabaseOutboxSettings::from($config);
+
+ $relay = new OutboxRelay(
+ $settings->connectionFrom($connections),
+ $downstream,
+ OutboxClaimStrategies::configured($settings->claimStrategy),
+ (int) $this->option('batch-size'),
+ $settings->maxAttempts,
+ // The UNGATED seam: it CONTINUES the traceparent the claimed row carries; RelayDownstream already gated
+ // the downstream publisher on firefly.eda.tracing.brokers.enabled.
+ BrokerTracing::bound($container),
+ $settings->leaseSeconds,
+ );
+
+ return $this->drain($relay);
+ } catch (ConfigurationException $e) {
+ $this->error($e->getMessage());
+
+ return self::FAILURE;
+ }
+ }
+
+ private function drain(OutboxRelay $relay): int
+ {
+ $maxMessages = $this->option('max-messages');
+ $timeLimit = $this->option('time-limit');
+ $deadline = is_numeric($timeLimit) ? time() + (int) $timeLimit : null;
+ $sleep = max(0, (int) $this->option('sleep'));
+ $published = 0;
+ $stop = false;
+
+ if (function_exists('pcntl_async_signals')) {
+ pcntl_async_signals(true);
+ pcntl_signal(SIGINT, function () use (&$stop): void {
+ $stop = true;
+ });
+ pcntl_signal(SIGTERM, function () use (&$stop): void {
+ $stop = true;
+ });
+ }
+
+ while (true) {
+ $n = $relay->relayBatch();
+ $published += $n;
+
+ if (($maxMessages !== null && is_numeric($maxMessages) && $published >= (int) $maxMessages)
+ || ($deadline !== null && time() >= $deadline)
+ || $stop) {
+ break;
+ }
+ if ($n === 0) {
+ sleep($sleep);
+ }
+ }
+
+ $this->info("firefly:outbox:relay — published {$published} row(s).");
+
+ return self::SUCCESS;
+ }
+}
diff --git a/packages/eda-database/src/DatabaseEventConsumer.php b/packages/eda-database/src/DatabaseEventConsumer.php
new file mode 100644
index 00000000..18f93118
--- /dev/null
+++ b/packages/eda-database/src/DatabaseEventConsumer.php
@@ -0,0 +1,164 @@
+claim = $claim ?? OutboxClaimStrategies::configured(OutboxClaimStrategies::AUTO);
+ $this->claimant = $claimant ?? LeaseClock::claimant();
+ $this->awaitWork = $awaitWork ?? function (int $timeoutMs): void {
+ usleep(max(0, min($timeoutMs, $this->pollIntervalMs)) * 1000);
+ };
+ $this->clock = $clock;
+ }
+
+ public function claimant(): string
+ {
+ return $this->claimant;
+ }
+
+ /** Destinations are irrelevant here: the consumer claims every PENDING row of the outbox. */
+ public function subscribe(array $destinations): void {}
+
+ public function start(): void {}
+
+ public function poll(int $timeoutMs): ?ReceivedEnvelope
+ {
+ $received = $this->claimOne();
+ if ($received !== null) {
+ return $received;
+ }
+
+ try {
+ ($this->awaitWork)($timeoutMs);
+ } catch (Throwable) {
+ // The wait is an optimization; the claim below delivers regardless.
+ }
+
+ return $this->claimOne();
+ }
+
+ public function ack(ReceivedEnvelope $received): void
+ {
+ ClaimableRows::owned($this->connection, OutboxRow::asInt($received->deliveryTag), $this->claimant)->update([
+ 'status' => OutboxSchema::STATUS_PUBLISHED,
+ 'processed_at' => $this->connection->raw('CURRENT_TIMESTAMP'),
+ 'claimed_by' => null,
+ 'claimed_until' => null,
+ ]);
+ }
+
+ /**
+ * A handler failure (requeue: true) counts an attempt and frees the row for an immediate retry, or FAILS it past
+ * maxAttempts; a poison record (requeue: false) FAILS at once — retrying bytes nobody can decode changes nothing.
+ */
+ public function nack(ReceivedEnvelope $received, bool $requeue = true): void
+ {
+ $id = OutboxRow::asInt($received->deliveryTag);
+ $row = ClaimableRows::owned($this->connection, $id, $this->claimant)->first();
+ if ($row === null) {
+ return; // the lease is gone: the row is another claimant's now
+ }
+
+ $attempts = OutboxRow::asInt($row->attempts) + 1;
+ $failed = ! $requeue || $attempts >= $this->maxAttempts;
+
+ ClaimableRows::owned($this->connection, $id, $this->claimant)->update([
+ 'attempts' => $attempts,
+ 'status' => $failed ? OutboxSchema::STATUS_FAILED : OutboxSchema::STATUS_PENDING,
+ 'error_message' => $received->isPoison()
+ ? 'undecodable outbox row: '.($received->failure?->getMessage() ?? 'unknown')
+ : 'in-process handler failed',
+ 'failed_at' => $failed ? $this->connection->raw('CURRENT_TIMESTAMP') : null,
+ 'claimed_by' => null,
+ 'claimed_until' => null,
+ ]);
+ }
+
+ public function stop(): void {}
+
+ private function claimOne(): ?ReceivedEnvelope
+ {
+ $claim = OutboxClaim::at($this->claimant, 1, LeaseClock::now($this->clock), $this->leaseSeconds);
+ $row = ClaimableRows::load($this->connection, $this->claim->claim($this->connection, $claim), $this->claimant)->first();
+
+ if ($row === null) {
+ return null;
+ }
+
+ $id = OutboxRow::asInt($row->id);
+
+ try {
+ return new ReceivedEnvelope(new EventEnvelope(
+ OutboxRow::asString($row->event_type),
+ OutboxRow::asString($row->destination),
+ OutboxRow::asMap($row->payload),
+ OutboxRow::asStringMap($row->headers),
+ ), $id);
+ } catch (JsonException $e) {
+ return ReceivedEnvelope::poison(OutboxRow::asString($row->payload), $id, $e, OutboxRow::asString($row->destination));
+ }
+ }
+}
diff --git a/packages/eda-database/src/DatabaseEventPublisher.php b/packages/eda-database/src/DatabaseEventPublisher.php
new file mode 100644
index 00000000..82bf147f
--- /dev/null
+++ b/packages/eda-database/src/DatabaseEventPublisher.php
@@ -0,0 +1,99 @@
+tracing = $tracing ?? new NoOpEdaTracing;
+ }
+
+ /**
+ * Records the pattern on the SHARED SubscriberRegistry — the same singleton the outbox auto-configuration hands to
+ * this writer and that firefly:eda:consume feeds every claimed envelope into. Delivery is the consumer's job.
+ */
+ public function subscribe(string $eventTypePattern, callable $handler): void
+ {
+ $this->registry->subscribe($eventTypePattern, $handler);
+ }
+
+ /**
+ * @param array $payload
+ * @param array $headers
+ */
+ public function publish(string $destination, string $eventType, array $payload, array $headers = []): void
+ {
+ $this->tracing->tracePublish($destination, $eventType, $headers, function (array $headers) use ($destination, $eventType, $payload): void {
+ $id = $this->connection->table(OutboxSchema::TABLE)->insertGetId([
+ 'destination' => $destination,
+ 'channel' => $this->channel,
+ 'event_type' => $eventType,
+ 'payload' => json_encode($payload, JSON_THROW_ON_ERROR),
+ 'headers' => json_encode($headers === [] ? new stdClass : $headers, JSON_THROW_ON_ERROR),
+ 'transaction_id' => $headers['x-correlation-id'] ?? null,
+ 'status' => OutboxSchema::STATUS_PENDING,
+ 'attempts' => 0,
+ 'created_at' => $this->connection->raw('CURRENT_TIMESTAMP'),
+ ]);
+
+ $this->afterInsert($id);
+ });
+ }
+
+ /**
+ * Runs inside the traced closure, on the same connection — so inside the same transaction — as the INSERT. A no-op
+ * here; PostgresEventPublisher overrides it with the in-transaction pg_notify.
+ */
+ protected function afterInsert(int $id): void {}
+
+ public function start(): void {}
+
+ public function stop(): void {}
+}
diff --git a/packages/eda-database/src/DatabaseOutboxAutoConfiguration.php b/packages/eda-database/src/DatabaseOutboxAutoConfiguration.php
new file mode 100644
index 00000000..43e70722
--- /dev/null
+++ b/packages/eda-database/src/DatabaseOutboxAutoConfiguration.php
@@ -0,0 +1,112 @@
+ DatabaseEventPublisher (the outbox writer, and the object
+ * EventListenerWiringPass subscribes listeners on — into the shared SubscriberRegistry below);
+ * - outboxPreCommitHook() + domainEventDispatcher() bind OUR DomainEventDispatcher carrying the hook, winning
+ * DataAutoConfiguration's #[ConditionalOnMissingBean(DomainEventDispatcher::class)];
+ * - commandEventPublisher() NoOps the AFTER-commit eda leg, winning CqrsAutoConfiguration's
+ * #[ConditionalOnMissingBean(CommandEventPublisher::class)] — the double-publish avoidance: a domain event is
+ * written EXACTLY ONCE, in the transaction, by the hook;
+ * - eventConsumer() binds the leased DatabaseEventConsumer firefly:eda:consume drives.
+ * Every bean is gated on provider=database, so with any other provider this class is inert — and
+ * PostgresOutboxAutoConfiguration does the same for provider=postgres from the same building blocks.
+ *
+ * Nothing here opens a connection: the claim strategy resolves at the first claim (ConfiguredClaimStrategy).
+ */
+#[Configuration]
+#[Order(900)]
+final class DatabaseOutboxAutoConfiguration
+{
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'database')]
+ public function subscriberRegistry(): SubscriberRegistry
+ {
+ return new SubscriberRegistry;
+ }
+
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'database')]
+ public function eventPublisher(Config $config, ConnectionResolverInterface $connections, SubscriberRegistry $registry, ?EdaTracing $tracing = null): EventPublisher
+ {
+ $settings = DatabaseOutboxSettings::from($config);
+
+ return new DatabaseEventPublisher(
+ $connections->connection($settings->connection),
+ $registry,
+ $settings->channel,
+ BrokerTracing::resolve($config, $tracing),
+ );
+ }
+
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'database')]
+ public function outboxPreCommitHook(ConnectionResolverInterface $connections, Config $config, HandlerManifest $manifest, CorrelationContext $correlation, SubscriberRegistry $registry, ?EdaTracing $tracing = null): OutboxPreCommitHook
+ {
+ return new OutboxPreCommitHook(
+ $connections,
+ $registry,
+ DatabaseOutboxSettings::from($config)->channel,
+ $config->string('firefly.cqrs.default_destination', 'cqrs.events'),
+ $manifest->destinations(),
+ $correlation,
+ BrokerTracing::resolve($config, $tracing),
+ );
+ }
+
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'database')]
+ public function domainEventDispatcher(AggregateTracker $tracker, ApplicationEventPublisher $publisher, OutboxPreCommitHook $hook): DomainEventDispatcher
+ {
+ return new DomainEventDispatcher($tracker, $publisher, $hook);
+ }
+
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'database')]
+ public function commandEventPublisher(): CommandEventPublisher
+ {
+ return new NoOpEventPublisher;
+ }
+
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'database')]
+ public function eventConsumer(Config $config, ConnectionResolverInterface $connections): EventConsumer
+ {
+ $settings = DatabaseOutboxSettings::from($config);
+
+ return new DatabaseEventConsumer(
+ $settings->connectionFrom($connections),
+ OutboxClaimStrategies::configured($settings->claimStrategy),
+ $settings->maxAttempts,
+ $settings->leaseSeconds,
+ $settings->pollIntervalMs,
+ );
+ }
+}
diff --git a/packages/eda-database/src/DatabaseOutboxSettings.php b/packages/eda-database/src/DatabaseOutboxSettings.php
new file mode 100644
index 00000000..6083cd56
--- /dev/null
+++ b/packages/eda-database/src/DatabaseOutboxSettings.php
@@ -0,0 +1,90 @@
+string($connectionKey);
+
+ $channelKey = self::firstSetKey($config, 'firefly.eda.database.channel', 'firefly.eda.postgres.channel');
+ $channel = $channelKey === null ? 'firefly_eda_events' : $config->string($channelKey);
+
+ $maxAttemptsKey = self::firstSetKey($config, 'firefly.eda.database.max-attempts', 'firefly.eda.postgres.max_attempts');
+ $maxAttempts = $maxAttemptsKey === null ? 3 : $config->int($maxAttemptsKey);
+
+ return new self(
+ connection: $connection === '' ? null : $connection,
+ channel: $channel,
+ maxAttempts: max(1, $maxAttempts),
+ claimStrategy: $config->string('firefly.eda.database.claim-strategy', 'auto'),
+ leaseSeconds: (float) max(1, $config->int('firefly.eda.database.lease-seconds', 300)),
+ pollIntervalMs: max(10, $config->int('firefly.eda.database.poll-interval-ms', 500)),
+ );
+ }
+
+ /**
+ * The first of $keys whose value is set AND not null, or null when none is: the alias rule every outbox key
+ * follows, shared with RelayDownstream so the relay's downstream key cannot drift from it. A null value is an
+ * unset env() and must fall through to the alias, which Config::has() alone would not do.
+ *
+ * @internal
+ */
+ public static function firstSetKey(Config $config, string ...$keys): ?string
+ {
+ foreach ($keys as $key) {
+ if ($config->get($key) !== null) {
+ return $key;
+ }
+ }
+
+ return null;
+ }
+
+ /** The outbox connection as the concrete Connection the claim needs (driver name, PDO, transactions). */
+ public function connectionFrom(ConnectionResolverInterface $connections): Connection
+ {
+ $connection = $connections->connection($this->connection);
+
+ return $connection instanceof Connection ? $connection : throw self::notConcrete($this->connection, $connection);
+ }
+
+ private static function notConcrete(?string $name, ConnectionInterface $connection): ConfigurationException
+ {
+ return new ConfigurationException(sprintf(
+ 'The outbox connection "%s" resolved to %s; the outbox needs an Illuminate\Database\Connection (it reads the '
+ .'driver and the server version to choose its claim).',
+ $name ?? 'default',
+ get_debug_type($connection),
+ ));
+ }
+}
diff --git a/packages/eda-database/src/EdaDatabaseBootServiceProvider.php b/packages/eda-database/src/EdaDatabaseBootServiceProvider.php
new file mode 100644
index 00000000..d3f07353
--- /dev/null
+++ b/packages/eda-database/src/EdaDatabaseBootServiceProvider.php
@@ -0,0 +1,31 @@
+app->make('config')->get('firefly.eda.provider') !== 'database') {
+ return;
+ }
+
+ $this->loadMigrationsFrom(self::MIGRATIONS);
+
+ if ($this->app->runningInConsole()) {
+ $this->commands([OutboxRelayCommand::class]);
+ }
+ }
+}
diff --git a/packages/eda-database/src/EdaDatabaseServiceProvider.php b/packages/eda-database/src/EdaDatabaseServiceProvider.php
new file mode 100644
index 00000000..405ed7e5
--- /dev/null
+++ b/packages/eda-database/src/EdaDatabaseServiceProvider.php
@@ -0,0 +1,25 @@
+table(OutboxSchema::TABLE)
+ ->where('status', OutboxSchema::STATUS_PENDING)
+ ->where(static function (Builder $query) use ($claim): void {
+ $query->whereNull('claimed_until')->orWhere('claimed_until', '<', $claim->nowString());
+ });
+ }
+
+ public static function candidates(Connection $connection, OutboxClaim $claim): Builder
+ {
+ return self::query($connection, $claim)->select('id')->orderBy('id')->limit($claim->limit);
+ }
+
+ /**
+ * @return list
+ */
+ public static function ids(Builder $query): array
+ {
+ $ids = [];
+ foreach ($query->pluck('id') as $id) {
+ $ids[] = OutboxRow::asInt($id);
+ }
+
+ return $ids;
+ }
+
+ /**
+ * @return array{claimed_by: string, claimed_until: string}
+ */
+ public static function leaseColumns(OutboxClaim $claim): array
+ {
+ return ['claimed_by' => $claim->claimant, 'claimed_until' => $claim->untilString()];
+ }
+
+ /**
+ * @param list $ids
+ */
+ public static function lease(Connection $connection, array $ids, OutboxClaim $claim): void
+ {
+ if ($ids === []) {
+ return;
+ }
+
+ $connection->table(OutboxSchema::TABLE)->whereIn('id', $ids)->update(self::leaseColumns($claim));
+ }
+
+ /**
+ * @param list $ids
+ * @return Collection
+ */
+ public static function load(Connection $connection, array $ids, string $claimant): Collection
+ {
+ if ($ids === []) {
+ /** @var Collection $none */
+ $none = new Collection;
+
+ return $none;
+ }
+
+ /** @var Collection $rows */
+ $rows = $connection->table(OutboxSchema::TABLE)
+ ->whereIn('id', $ids)
+ ->where('claimed_by', $claimant)
+ ->orderBy('id')
+ ->get();
+
+ return $rows;
+ }
+
+ public static function owned(ConnectionInterface $connection, int $id, string $claimant): Builder
+ {
+ return $connection->table(OutboxSchema::TABLE)
+ ->where('id', $id)
+ ->where('status', OutboxSchema::STATUS_PENDING)
+ ->where('claimed_by', $claimant);
+ }
+}
diff --git a/packages/eda-database/src/Outbox/Claim/ConfiguredClaimStrategy.php b/packages/eda-database/src/Outbox/Claim/ConfiguredClaimStrategy.php
new file mode 100644
index 00000000..d61b4755
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/ConfiguredClaimStrategy.php
@@ -0,0 +1,41 @@
+ connection name => resolved strategy */
+ private array $resolved = [];
+
+ private ?OutboxClaimStrategy $last = null;
+
+ public function __construct(private readonly string $configured = OutboxClaimStrategies::AUTO) {}
+
+ public function name(): string
+ {
+ return $this->last?->name() ?? $this->configured;
+ }
+
+ public function claim(Connection $connection, OutboxClaim $claim): array
+ {
+ return $this->strategyFor($connection)->claim($connection, $claim);
+ }
+
+ public function strategyFor(Connection $connection): OutboxClaimStrategy
+ {
+ $key = (string) $connection->getName();
+
+ return $this->last = $this->resolved[$key] ??= OutboxClaimStrategies::resolve($connection, $this->configured);
+ }
+}
diff --git a/packages/eda-database/src/Outbox/Claim/OptimisticLeaseClaim.php b/packages/eda-database/src/Outbox/Claim/OptimisticLeaseClaim.php
new file mode 100644
index 00000000..a0d4588f
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/OptimisticLeaseClaim.php
@@ -0,0 +1,40 @@
+where('id', $id)->update(ClaimableRows::leaseColumns($claim));
+
+ if ($changed === 1) {
+ $won[] = $id;
+ }
+ }
+
+ return $won;
+ }
+}
diff --git a/packages/eda-database/src/Outbox/Claim/OutboxClaim.php b/packages/eda-database/src/Outbox/Claim/OutboxClaim.php
new file mode 100644
index 00000000..be4240f7
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/OutboxClaim.php
@@ -0,0 +1,34 @@
+now);
+ }
+
+ public function untilString(): string
+ {
+ return LeaseClock::format($this->until);
+ }
+}
diff --git a/packages/eda-database/src/Outbox/Claim/OutboxClaimStrategies.php b/packages/eda-database/src/Outbox/Claim/OutboxClaimStrategies.php
new file mode 100644
index 00000000..f099489e
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/OutboxClaimStrategies.php
@@ -0,0 +1,62 @@
+ */
+ public const array NAMES = [self::AUTO, SkipLockedClaim::NAME, ReadPastClaim::NAME, OptimisticLeaseClaim::NAME];
+
+ /** The strategy for a configured name, resolved at the first claim (no connection is opened here). */
+ public static function configured(string $name): OutboxClaimStrategy
+ {
+ return $name === OptimisticLeaseClaim::NAME ? new OptimisticLeaseClaim : new ConfiguredClaimStrategy($name);
+ }
+
+ public static function resolve(Connection $connection, string $name = self::AUTO): OutboxClaimStrategy
+ {
+ return self::forCapabilities(DatabaseCapabilities::of($connection), $name);
+ }
+
+ public static function forCapabilities(DatabaseCapabilities $capabilities, string $name = self::AUTO): OutboxClaimStrategy
+ {
+ return match ($name) {
+ self::AUTO => match (true) {
+ $capabilities->supportsSkipLocked() => new SkipLockedClaim,
+ $capabilities->supportsReadPast() => new ReadPastClaim,
+ default => new OptimisticLeaseClaim,
+ },
+ SkipLockedClaim::NAME => $capabilities->supportsSkipLocked() ? new SkipLockedClaim : throw self::unsupported($name, $capabilities),
+ ReadPastClaim::NAME => $capabilities->supportsReadPast() ? new ReadPastClaim : throw self::unsupported($name, $capabilities),
+ OptimisticLeaseClaim::NAME => new OptimisticLeaseClaim,
+ default => throw new ConfigurationException(sprintf(
+ 'firefly.eda.database.claim-strategy="%s" is not one of [%s].',
+ $name,
+ implode('|', self::NAMES),
+ )),
+ };
+ }
+
+ private static function unsupported(string $name, DatabaseCapabilities $capabilities): ConfigurationException
+ {
+ return new ConfigurationException(sprintf(
+ 'firefly.eda.database.claim-strategy="%s" but %s does not support it (SKIP LOCKED needs PostgreSQL, MySQL '
+ .'8.0.1+ or MariaDB 10.6+; READPAST needs SQL Server). Use "auto" or "optimistic-lease".',
+ $name,
+ $capabilities->describe(),
+ ));
+ }
+}
diff --git a/packages/eda-database/src/Outbox/Claim/OutboxClaimStrategy.php b/packages/eda-database/src/Outbox/Claim/OutboxClaimStrategy.php
new file mode 100644
index 00000000..09b29864
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/OutboxClaimStrategy.php
@@ -0,0 +1,33 @@
+claimant` and `claimed_until = $claim->until`, committed before the caller does any work with
+ * them — and differs only in how it picks the rows without two claimants picking the same one:
+ *
+ * - SkipLockedClaim `FOR UPDATE SKIP LOCKED` inside a short transaction (PostgreSQL, MySQL 8.0.1+, MariaDB 10.6+);
+ * - ReadPastClaim `WITH (UPDLOCK, READPAST, ROWLOCK)` inside a short transaction (SQL Server);
+ * - OptimisticLeaseClaim an unlocked read, then one conditional UPDATE per row; the row is won when exactly one row
+ * changed (any driver — the portable default for SQLite and anything unknown).
+ *
+ * The LEASE is the guarantee, not the row lock: a lock dies with the claim transaction, so it cannot protect a
+ * handler that runs after it. A row is claimable while it is PENDING and its lease is empty or over; a claimant that
+ * crashes leaves its rows to the next claimant once their lease ends.
+ */
+interface OutboxClaimStrategy
+{
+ public function name(): string;
+
+ /**
+ * Lease up to $claim->limit claimable rows to $claim->claimant, in a unit of work of its own.
+ *
+ * @return list the ids won, ascending
+ */
+ public function claim(Connection $connection, OutboxClaim $claim): array;
+}
diff --git a/packages/eda-database/src/Outbox/Claim/ReadPastClaim.php b/packages/eda-database/src/Outbox/Claim/ReadPastClaim.php
new file mode 100644
index 00000000..9954d923
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/ReadPastClaim.php
@@ -0,0 +1,35 @@
+ $ids */
+ $ids = $connection->transaction(static function () use ($connection, $claim): array {
+ $ids = ClaimableRows::ids(ClaimableRows::candidates($connection, $claim)->lock('with (updlock, readpast, rowlock)'));
+ ClaimableRows::lease($connection, $ids, $claim);
+
+ return $ids;
+ });
+
+ return $ids;
+ }
+}
diff --git a/packages/eda-database/src/Outbox/Claim/SkipLockedClaim.php b/packages/eda-database/src/Outbox/Claim/SkipLockedClaim.php
new file mode 100644
index 00000000..2aaa6c67
--- /dev/null
+++ b/packages/eda-database/src/Outbox/Claim/SkipLockedClaim.php
@@ -0,0 +1,41 @@
+transactionLevel() === 0 && in_array($connection->getDriverName(), ['mysql', 'mariadb'], true)) {
+ $connection->statement('set transaction isolation level read committed');
+ }
+
+ /** @var list $ids */
+ $ids = $connection->transaction(static function () use ($connection, $claim): array {
+ $ids = ClaimableRows::ids(ClaimableRows::candidates($connection, $claim)->lock('for update skip locked'));
+ ClaimableRows::lease($connection, $ids, $claim);
+
+ return $ids;
+ });
+
+ return $ids;
+ }
+}
diff --git a/packages/eda-database/src/Outbox/LeaseClock.php b/packages/eda-database/src/Outbox/LeaseClock.php
new file mode 100644
index 00000000..435240dd
--- /dev/null
+++ b/packages/eda-database/src/Outbox/LeaseClock.php
@@ -0,0 +1,52 @@
+setTimezone(new DateTimeZone('UTC'));
+ }
+
+ public static function plus(DateTimeImmutable $instant, float $seconds): DateTimeImmutable
+ {
+ $milliseconds = max(0, (int) round($seconds * 1000));
+ $interval = new DateInterval('PT'.intdiv($milliseconds, 1000).'S');
+ $interval->f = ($milliseconds % 1000) / 1000;
+
+ return $instant->add($interval);
+ }
+
+ public static function format(DateTimeImmutable $instant): string
+ {
+ return $instant->setTimezone(new DateTimeZone('UTC'))->format('Y-m-d H:i:s.v');
+ }
+
+ /** `host:pid:random` — tells two processes on one node, and two claimants in one process, apart. */
+ public static function claimant(): string
+ {
+ $host = gethostname();
+ $pid = getmypid();
+
+ return substr(is_string($host) && $host !== '' ? $host : 'host', 0, 80).':'.($pid === false ? 0 : $pid).':'.bin2hex(random_bytes(4));
+ }
+}
diff --git a/packages/eda-database/src/Outbox/OutboxPreCommitHook.php b/packages/eda-database/src/Outbox/OutboxPreCommitHook.php
new file mode 100644
index 00000000..f14f7bf9
--- /dev/null
+++ b/packages/eda-database/src/Outbox/OutboxPreCommitHook.php
@@ -0,0 +1,62 @@
+ $destinations eventClass => destination */
+ public function __construct(
+ protected readonly ConnectionResolverInterface $connections,
+ protected readonly SubscriberRegistry $registry,
+ protected readonly string $channel,
+ protected readonly string $defaultDestination,
+ protected readonly array $destinations,
+ protected readonly ?CorrelationContext $correlation = null,
+ protected readonly ?EdaTracing $tracing = null,
+ ) {}
+
+ public function handle(object $event, ?string $connection = null): void
+ {
+ if (! $event instanceof DomainEvent) {
+ return;
+ }
+
+ // The aggregate's connection — the one carrying the open transaction.
+ $publisher = $this->writerFor($this->connections->connection($connection));
+
+ (new EdaCommandEventPublisher($publisher, $this->defaultDestination, $this->destinations, $this->correlation))
+ ->publish($event);
+ }
+
+ /** The writer for one event, on the aggregate's own connection. */
+ protected function writerFor(ConnectionInterface $connection): DatabaseEventPublisher
+ {
+ return new DatabaseEventPublisher($connection, $this->registry, $this->channel, $this->tracing);
+ }
+}
diff --git a/packages/eda-database/src/Outbox/OutboxRelay.php b/packages/eda-database/src/Outbox/OutboxRelay.php
new file mode 100644
index 00000000..db0c51a5
--- /dev/null
+++ b/packages/eda-database/src/Outbox/OutboxRelay.php
@@ -0,0 +1,130 @@
+tracing = $tracing ?? new NoOpEdaTracing;
+ $this->claim = $claim ?? OutboxClaimStrategies::configured(OutboxClaimStrategies::AUTO);
+ $this->claimant = $claimant ?? LeaseClock::claimant();
+ $this->clock = $clock;
+ }
+
+ public function claimant(): string
+ {
+ return $this->claimant;
+ }
+
+ public function relayBatch(): int
+ {
+ $claim = OutboxClaim::at($this->claimant, $this->batchSize, LeaseClock::now($this->clock), $this->leaseSeconds);
+ $relayed = 0;
+
+ foreach (ClaimableRows::load($this->connection, $this->claim->claim($this->connection, $claim), $this->claimant) as $row) {
+ $id = OutboxRow::asInt($row->id);
+
+ try {
+ // The envelope is rebuilt ONLY so the seam has a traceparent to continue and a message id to name the
+ // span with — the downstream still receives the row's own four columns, byte for byte.
+ $this->tracing->traceConsume(
+ new EventEnvelope(
+ OutboxRow::asString($row->event_type),
+ OutboxRow::asString($row->destination),
+ OutboxRow::asMap($row->payload),
+ OutboxRow::asStringMap($row->headers),
+ OutboxRow::asString($row->id),
+ ),
+ fn (EventEnvelope $claimed) => $this->downstream->publish(
+ $claimed->destination,
+ $claimed->eventType,
+ $claimed->payload,
+ $claimed->headers,
+ ),
+ );
+
+ $relayed += ClaimableRows::owned($this->connection, $id, $this->claimant)->update([
+ 'status' => OutboxSchema::STATUS_PUBLISHED,
+ 'processed_at' => $this->connection->raw('CURRENT_TIMESTAMP'),
+ 'claimed_by' => null,
+ 'claimed_until' => null,
+ ]);
+ } catch (Throwable $e) {
+ $attempts = OutboxRow::asInt($row->attempts) + 1;
+ $failed = $attempts >= $this->maxAttempts;
+
+ ClaimableRows::owned($this->connection, $id, $this->claimant)->update([
+ 'attempts' => $attempts,
+ 'status' => $failed ? OutboxSchema::STATUS_FAILED : OutboxSchema::STATUS_PENDING,
+ 'error_message' => $e->getMessage(),
+ 'failed_at' => $failed ? $this->connection->raw('CURRENT_TIMESTAMP') : null,
+ 'claimed_by' => null,
+ 'claimed_until' => null,
+ ]);
+ }
+ }
+
+ return $relayed;
+ }
+}
diff --git a/packages/eda-database/src/Outbox/OutboxRow.php b/packages/eda-database/src/Outbox/OutboxRow.php
new file mode 100644
index 00000000..d22565d4
--- /dev/null
+++ b/packages/eda-database/src/Outbox/OutboxRow.php
@@ -0,0 +1,58 @@
+ */
+ public static function asMap(mixed $json): array
+ {
+ if (! is_string($json)) {
+ return [];
+ }
+
+ $decoded = json_decode($json, true, 512, JSON_THROW_ON_ERROR);
+ if (! is_array($decoded)) {
+ return [];
+ }
+
+ $map = [];
+ foreach ($decoded as $key => $value) {
+ $map[(string) $key] = $value;
+ }
+
+ return $map;
+ }
+
+ /** @return array */
+ public static function asStringMap(mixed $json): array
+ {
+ $map = [];
+ foreach (self::asMap($json) as $key => $value) {
+ if (is_scalar($value)) {
+ $map[$key] = (string) $value;
+ }
+ }
+
+ return $map;
+ }
+}
diff --git a/packages/eda-database/src/Outbox/OutboxSchema.php b/packages/eda-database/src/Outbox/OutboxSchema.php
new file mode 100644
index 00000000..41d971ba
--- /dev/null
+++ b/packages/eda-database/src/Outbox/OutboxSchema.php
@@ -0,0 +1,60 @@
+bigIncrements('id');
+ $table->string('destination');
+ $table->string('channel', 63);
+ $table->string('event_type');
+ $table->json('payload');
+ $table->json('headers');
+ $table->string('transaction_id')->nullable();
+ $table->string('status')->default(self::STATUS_PENDING);
+ $table->unsignedInteger('attempts')->default(0);
+ $table->text('error_message')->nullable();
+ $table->timestamp('created_at')->useCurrent();
+ $table->timestamp('processed_at')->nullable();
+ $table->timestamp('failed_at')->nullable();
+
+ $table->index(['status', 'created_at'], 'firefly_eda_outbox_pending_idx');
+ $table->index(['channel', 'status'], 'firefly_eda_outbox_channel_status_idx');
+
+ self::claimColumns($table);
+ }
+
+ /** The lease columns: in blueprint() for a fresh table, and alone for the migration that upgrades an old one. */
+ public static function claimColumns(Blueprint $table): void
+ {
+ $table->string('claimed_by', 100)->nullable();
+ $table->dateTime('claimed_until', 3)->nullable();
+ $table->index(['status', 'claimed_until'], self::CLAIM_INDEX);
+ }
+}
diff --git a/packages/eda-database/src/Outbox/OutboxWriter.php b/packages/eda-database/src/Outbox/OutboxWriter.php
new file mode 100644
index 00000000..e5aa5048
--- /dev/null
+++ b/packages/eda-database/src/Outbox/OutboxWriter.php
@@ -0,0 +1,12 @@
+, which is false here by construction; autowiring would otherwise use constructor
+ * defaults and put traceparents on a third party's wire with the compliance gate off. Validation is at RUN time (the
+ * relay command), not boot: a downstream bound in another provider's boot() may not exist yet when ours runs.
+ *
+ * Whatever it resolves to, an OutboxWriter is refused — the relay would re-INSERT every claimed row as PENDING into
+ * the outbox it is draining. Not `final`: the deprecated Firefly\Eda\Postgres\Outbox\RelayDownstream extends it.
+ */
+class RelayDownstream
+{
+ /** The config key that selects the relay's downstream. */
+ public const string PROVIDER_KEY = 'firefly.eda.database.relay.downstream-provider';
+
+ /** The firefly/eda-postgres spelling of the same key, still honoured when the new one is unset. */
+ public const string LEGACY_PROVIDER_KEY = 'firefly.eda.postgres.relay.downstream_provider';
+
+ /** The container id an app may bind to supply a fully constructed downstream publisher itself. */
+ public const string BINDING = 'firefly.eda.relay.downstream';
+
+ /** @var array */
+ private const ADAPTERS = [
+ 'rabbitmq' => ['class' => 'Firefly\\Eda\\Rabbitmq\\RabbitMqEventPublisher', 'package' => 'firefly/eda-rabbitmq'],
+ 'kafka' => ['class' => 'Firefly\\Eda\\Kafka\\KafkaEventPublisher', 'package' => 'firefly/eda-kafka'],
+ ];
+
+ /**
+ * The configured downstream name (new key first, then the legacy one), or null when neither is set. A key present
+ * but null (an unset env()) counts as unset, so it never hides a legacy value.
+ */
+ public static function configuredProvider(Config $config): ?string
+ {
+ $key = DatabaseOutboxSettings::firstSetKey($config, self::PROVIDER_KEY, self::LEGACY_PROVIDER_KEY);
+ $provider = $key === null ? '' : $config->string($key);
+
+ return trim($provider) === '' ? null : trim($provider);
+ }
+
+ /** Resolve the downstream publisher, or throw a ConfigurationException that names the exact thing to change. */
+ public static function resolve(Container $container, Config $config): EventPublisher
+ {
+ $provider = self::configuredProvider($config);
+
+ if ($container->bound(self::BINDING)) {
+ return self::guard($provider ?? self::BINDING, self::make($container, self::BINDING, [], $provider ?? self::BINDING));
+ }
+
+ if ($provider === null) {
+ throw new ConfigurationException(sprintf(
+ 'Neither %s nor its legacy alias %s is set, so firefly:outbox:relay has no downstream broker to forward to. '
+ .'Set it to one of [%s], to the class-string of an EventPublisher, or bind your own under the container id '
+ .'"%s". If you did not mean to front a second broker at all, do not run the relay: the outbox already '
+ .'delivers committed rows in-process via `php artisan firefly:eda:consume`.',
+ self::PROVIDER_KEY,
+ self::LEGACY_PROVIDER_KEY,
+ implode('|', array_keys(self::ADAPTERS)),
+ self::BINDING,
+ ));
+ }
+
+ return self::guard($provider, self::instantiate($container, $config, $provider));
+ }
+
+ private static function instantiate(Container $container, Config $config, string $provider): mixed
+ {
+ if ($container->bound($provider)) {
+ return self::make($container, $provider, [], $provider);
+ }
+
+ $alias = isset(self::ADAPTERS[$provider]) ? $provider : self::shippedAdapterAlias($provider);
+
+ if ($alias !== null) {
+ $adapter = self::ADAPTERS[$alias];
+
+ if (! class_exists($adapter['class'])) {
+ throw new ConfigurationException(sprintf(
+ '%s="%s" but %s is not installed — %s could not be found. Run `composer require %s`, or bind your own '
+ .'downstream EventPublisher under the container id "%s".',
+ self::PROVIDER_KEY,
+ $provider,
+ $adapter['package'],
+ $adapter['class'],
+ $adapter['package'],
+ self::BINDING,
+ ));
+ }
+
+ return self::make($container, $adapter['class'], self::parameters($container, $config, $alias), $provider);
+ }
+
+ if (class_exists($provider)) {
+ return self::make($container, $provider, [], $provider);
+ }
+
+ throw new ConfigurationException(sprintf(
+ '%s="%s" names neither a shipped adapter [%s], nor a bound container id, nor an existing class. Fix the '
+ .'value, or bind your own downstream EventPublisher under the container id "%s".',
+ self::PROVIDER_KEY,
+ $provider,
+ implode('|', array_keys(self::ADAPTERS)),
+ self::BINDING,
+ ));
+ }
+
+ private static function shippedAdapterAlias(string $provider): ?string
+ {
+ foreach (self::ADAPTERS as $alias => $adapter) {
+ if ($adapter['class'] === $provider) {
+ return $alias;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * The adapter-specific constructor overrides that carry the sibling package's OWN configuration — and the gated
+ * EdaTracing — across the provider gate it is standing behind.
+ *
+ * @return array
+ */
+ private static function parameters(Container $container, Config $config, string $provider): array
+ {
+ if ($provider === 'rabbitmq') {
+ return [
+ 'exchange' => $config->string('firefly.eda.rabbitmq.exchange', 'firefly.events'),
+ 'tracing' => BrokerTracing::forContainer($container, $config),
+ ];
+ }
+
+ if ($provider === 'kafka') {
+ return [
+ 'factory' => self::make(
+ $container,
+ 'Firefly\\Eda\\Kafka\\KafkaProducerFactory',
+ ['brokers' => $config->string('firefly.eda.kafka.brokers', '127.0.0.1:9092')],
+ $provider,
+ ),
+ 'tracing' => BrokerTracing::forContainer($container, $config),
+ ];
+ }
+
+ return [];
+ }
+
+ /**
+ * @param array $parameters
+ */
+ private static function make(Container $container, string $id, array $parameters, string $provider): mixed
+ {
+ try {
+ return $container->make($id, $parameters);
+ } catch (BindingResolutionException $e) {
+ throw new ConfigurationException(sprintf(
+ '%s="%s" resolved to "%s", which could not be constructed: %s. Bind a ready-made downstream '
+ .'EventPublisher under the container id "%s" instead.',
+ self::PROVIDER_KEY,
+ $provider,
+ $id,
+ $e->getMessage(),
+ self::BINDING,
+ ), previous: $e);
+ }
+ }
+
+ private static function guard(string $provider, mixed $downstream): EventPublisher
+ {
+ if ($downstream instanceof OutboxWriter) {
+ throw new ConfigurationException(sprintf(
+ '%s="%s" resolved to the outbox writer itself (%s). The relay would re-INSERT every claimed row into '
+ .'firefly_eda_outbox as PENDING and claim it again forever. Point it at a DISTINCT broker, or bind one '
+ .'under the container id "%s".',
+ self::PROVIDER_KEY,
+ $provider,
+ $downstream::class,
+ self::BINDING,
+ ));
+ }
+
+ if (! $downstream instanceof EventPublisher) {
+ throw new ConfigurationException(sprintf(
+ '%s="%s" resolved to %s, which does not implement %s.',
+ self::PROVIDER_KEY,
+ $provider,
+ get_debug_type($downstream),
+ EventPublisher::class,
+ ));
+ }
+
+ return $downstream;
+ }
+}
diff --git a/packages/eda-database/src/OutboxHealthIndicator.php b/packages/eda-database/src/OutboxHealthIndicator.php
new file mode 100644
index 00000000..3d3cc994
--- /dev/null
+++ b/packages/eda-database/src/OutboxHealthIndicator.php
@@ -0,0 +1,49 @@
+config);
+ $connection = $settings->connectionFrom($this->connections);
+
+ return Health::up([
+ 'broker' => 'database',
+ 'driver' => $connection->getDriverName(),
+ 'claim-strategy' => OutboxClaimStrategies::resolve($connection, $settings->claimStrategy)->name(),
+ 'pending' => $connection->table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PENDING)->count(),
+ ]);
+ } catch (Throwable $e) {
+ return Health::down(['broker' => 'database', 'error' => $e->getMessage()]);
+ }
+ }
+}
diff --git a/packages/eda-database/tests/CapstoneDatabaseOutboxTest.php b/packages/eda-database/tests/CapstoneDatabaseOutboxTest.php
new file mode 100644
index 00000000..e21f8e0d
--- /dev/null
+++ b/packages/eda-database/tests/CapstoneDatabaseOutboxTest.php
@@ -0,0 +1,85 @@
+app();
+
+ DB::transaction(fn () => $app->make(EventPublisher::class)->publish('orders', 'order.placed', ['id' => 7]));
+
+ /** @var SubscriberRegistry $registry */
+ $registry = $app->make(SubscriberRegistry::class);
+ /** @var EventConsumer $consumer */
+ $consumer = $app->make(EventConsumer::class);
+ $processed = (new ConsumerLoop)->run($consumer, fn (EventEnvelope $envelope) => $registry->deliver($envelope), new ConsumerOptions(maxMessages: 1, pollTimeoutMs: 10));
+
+ /** @var ListenerSpy $spy */
+ $spy = $app->make(ListenerSpy::class);
+
+ expect($app->make(EventPublisher::class))->toBeInstanceOf(DatabaseEventPublisher::class)
+ ->and($consumer)->toBeInstanceOf(DatabaseEventConsumer::class)
+ ->and($processed)->toBe(1)
+ ->and($spy->seen)->toBe(['order.placed'])
+ ->and(DB::table(OutboxSchema::TABLE)->value('status'))->toBe(OutboxSchema::STATUS_PUBLISHED);
+});
+
+it('writes a domain event in the aggregate\'s transaction through the bound pre-commit hook, and drops it on rollback', function () {
+ /** @var DatabaseOutboxCapstoneTestCase $this */
+ /** @var OutboxPreCommitHook $hook */
+ $hook = $this->app()->make(OutboxPreCommitHook::class);
+
+ DB::transaction(fn () => $hook->handle(new OrderPlaced(41), null));
+ try {
+ DB::transaction(function () use ($hook): void {
+ $hook->handle(new OrderPlaced(42), null);
+ throw new RuntimeException('rolled back');
+ });
+ } catch (RuntimeException) {
+ // expected
+ }
+
+ expect(DB::table(OutboxSchema::TABLE)->where('event_type', 'OrderPlaced')->count())->toBe(1)
+ ->and(DB::table(OutboxSchema::TABLE)->value('destination'))->toBe('cqrs.events');
+});
+
+it('reports the outbox UP with the driver and the claim it will use', function () {
+ /** @var DatabaseOutboxCapstoneTestCase $this */
+ $indicator = $this->app()->make(OutboxHealthIndicator::class);
+
+ $health = $indicator->health();
+
+ expect($health->status)->toBe(Status::Up)
+ ->and($health->details)->toBe(['broker' => 'database', 'driver' => 'sqlite', 'claim-strategy' => 'optimistic-lease', 'pending' => 0]);
+});
+
+it('reports the outbox DOWN with the error when the forced claim strategy is one this server cannot honour', function () {
+ /** @var DatabaseOutboxCapstoneTestCase $this */
+ config()->set('firefly.eda.database.claim-strategy', 'skip-locked');
+ $indicator = $this->app()->make(OutboxHealthIndicator::class);
+
+ $health = $indicator->health();
+
+ expect($health->status)->toBe(Status::Down)
+ ->and($health->details['broker'] ?? null)->toBe('database')
+ ->and($health->details['error'] ?? '')->toContain('claim-strategy="skip-locked"');
+});
diff --git a/packages/eda-database/tests/CapstoneFixtures/DatabaseCapstoneListener.php b/packages/eda-database/tests/CapstoneFixtures/DatabaseCapstoneListener.php
new file mode 100644
index 00000000..6552b466
--- /dev/null
+++ b/packages/eda-database/tests/CapstoneFixtures/DatabaseCapstoneListener.php
@@ -0,0 +1,24 @@
+spy->record($envelope->eventType);
+ }
+}
diff --git a/packages/eda-database/tests/Claim/ClaimSqlShapeTest.php b/packages/eda-database/tests/Claim/ClaimSqlShapeTest.php
new file mode 100644
index 00000000..12d6131b
--- /dev/null
+++ b/packages/eda-database/tests/Claim/ClaimSqlShapeTest.php
@@ -0,0 +1,65 @@
+
+ */
+function claimSqlShape(string $driver, OutboxClaimStrategy $strategy): array
+{
+ $pdo = new PDO('sqlite::memory:');
+ $config = ['driver' => $driver, 'name' => 'pretend'];
+ $connection = match ($driver) {
+ 'pgsql' => new PostgresConnection($pdo, 'pretend', '', $config),
+ 'mysql' => new MySqlConnection($pdo, 'pretend', '', $config),
+ 'mariadb' => new MariaDbConnection($pdo, 'pretend', '', $config),
+ 'sqlsrv' => new SqlServerConnection($pdo, 'pretend', '', $config),
+ default => throw new InvalidArgumentException('No grammar under test for '.$driver),
+ };
+ $claim = OutboxClaim::at('node-a', 5, new DateTimeImmutable('2026-10-12 10:00:00', new DateTimeZone('UTC')), 300);
+
+ return array_column($connection->pretend(static function () use ($connection, $strategy, $claim): void {
+ $strategy->claim($connection, $claim);
+ }), 'query');
+}
+
+it('claims with FOR UPDATE SKIP LOCKED on pgsql', function () {
+ expect(claimSqlShape('pgsql', new SkipLockedClaim))->toBe([
+ 'select "id" from "firefly_eda_outbox" where "status" = \'PENDING\' and ("claimed_until" is null or "claimed_until" < \'2026-10-12 10:00:00.000\') order by "id" asc limit 5 for update skip locked',
+ ]);
+});
+
+it('claims with FOR UPDATE SKIP LOCKED under READ COMMITTED on mysql and mariadb', function (string $driver) {
+ expect(claimSqlShape($driver, new SkipLockedClaim))->toBe([
+ 'set transaction isolation level read committed',
+ 'select `id` from `firefly_eda_outbox` where `status` = \'PENDING\' and (`claimed_until` is null or `claimed_until` < \'2026-10-12 10:00:00.000\') order by `id` asc limit 5 for update skip locked',
+ ]);
+})->with(['mysql', 'mariadb']);
+
+it('claims with the UPDLOCK, READPAST, ROWLOCK table hint on sqlsrv', function () {
+ expect(claimSqlShape('sqlsrv', new ReadPastClaim))->toBe([
+ 'select top 5 [id] from [firefly_eda_outbox] with (updlock, readpast, rowlock) where [status] = \'PENDING\' and ([claimed_until] is null or [claimed_until] < \'2026-10-12 10:00:00.000\') order by [id] asc',
+ ]);
+});
+
+it('reads candidates WITHOUT a lock for the optimistic lease, on any grammar', function () {
+ expect(claimSqlShape('mysql', new OptimisticLeaseClaim))->toBe([
+ 'select `id` from `firefly_eda_outbox` where `status` = \'PENDING\' and (`claimed_until` is null or `claimed_until` < \'2026-10-12 10:00:00.000\') order by `id` asc limit 5',
+ ]);
+});
diff --git a/packages/eda-database/tests/Claim/OptimisticLeaseClaimTest.php b/packages/eda-database/tests/Claim/OptimisticLeaseClaimTest.php
new file mode 100644
index 00000000..669a4ddb
--- /dev/null
+++ b/packages/eda-database/tests/Claim/OptimisticLeaseClaimTest.php
@@ -0,0 +1,74 @@
+ OutboxSchema::blueprint($table));
+ foreach (range(1, 5) as $n) {
+ DB::table(OutboxSchema::TABLE)->insert([
+ 'destination' => 'orders', 'channel' => 'firefly_eda_events', 'event_type' => 'order.placed',
+ 'payload' => json_encode(['n' => $n], JSON_THROW_ON_ERROR), 'headers' => '{}',
+ 'status' => OutboxSchema::STATUS_PENDING, 'attempts' => 0,
+ ]);
+ }
+});
+
+function optimisticClaimAt(string $claimant, int $limit, string $now): OutboxClaim
+{
+ return OutboxClaim::at($claimant, $limit, new DateTimeImmutable($now, new DateTimeZone('UTC')), 60);
+}
+
+function optimisticConnection(): Connection
+{
+ return DB::connection();
+}
+
+it('leases rows to one claimant at a time, and hands an ended lease to the next', function () {
+ $strategy = new OptimisticLeaseClaim;
+
+ expect($strategy->claim(optimisticConnection(), optimisticClaimAt('a', 3, '2026-10-12 10:00:00')))->toBe([1, 2, 3])
+ ->and($strategy->claim(optimisticConnection(), optimisticClaimAt('b', 3, '2026-10-12 10:00:01')))->toBe([4, 5])
+ ->and($strategy->claim(optimisticConnection(), optimisticClaimAt('c', 3, '2026-10-12 10:00:30')))->toBe([])
+ // a's lease (until 10:01:00.000) is over at 10:01:00.001; b's (until 10:01:01) is not.
+ ->and($strategy->claim(optimisticConnection(), optimisticClaimAt('c', 10, '2026-10-12 10:01:00.001')))->toBe([1, 2, 3])
+ ->and(DB::table(OutboxSchema::TABLE)->where('claimed_by', 'c')->count())->toBe(3)
+ ->and(DB::table(OutboxSchema::TABLE)->where('id', 1)->value('claimed_until'))->toBe('2026-10-12 10:02:00.001');
+});
+
+it('loses a row another claimant leased between its read and its conditional update', function () {
+ $intruded = false;
+ optimisticConnection()->beforeExecuting(static function (string $query) use (&$intruded): void {
+ if (! $intruded && str_starts_with($query, 'update')) {
+ $intruded = true;
+ DB::table(OutboxSchema::TABLE)->where('id', 1)->update(['claimed_by' => 'intruder', 'claimed_until' => '2999-01-01 00:00:00.000']);
+ }
+ });
+
+ expect((new OptimisticLeaseClaim)->claim(optimisticConnection(), optimisticClaimAt('a', 3, '2026-10-12 10:00:00')))->toBe([2, 3])
+ ->and(DB::table(OutboxSchema::TABLE)->where('id', 1)->value('claimed_by'))->toBe('intruder');
+});
+
+it('never offers a row that is no longer PENDING, and settles only the claimant\'s own rows', function () {
+ DB::table(OutboxSchema::TABLE)->where('id', 2)->update(['status' => OutboxSchema::STATUS_PUBLISHED]);
+ $claim = optimisticClaimAt('a', 5, '2026-10-12 10:00:00');
+
+ $ids = (new OptimisticLeaseClaim)->claim(optimisticConnection(), $claim);
+
+ expect($ids)->toBe([1, 3, 4, 5])
+ ->and(ClaimableRows::load(optimisticConnection(), $ids, 'a')->pluck('id')->all())->toBe([1, 3, 4, 5])
+ ->and(ClaimableRows::load(optimisticConnection(), $ids, 'b')->all())->toBe([])
+ ->and(ClaimableRows::owned(optimisticConnection(), 1, 'b')->count())->toBe(0)
+ ->and(ClaimableRows::owned(optimisticConnection(), 1, 'a')->count())->toBe(1);
+});
diff --git a/packages/eda-database/tests/Claim/OutboxClaimStrategiesTest.php b/packages/eda-database/tests/Claim/OutboxClaimStrategiesTest.php
new file mode 100644
index 00000000..6ae5fe60
--- /dev/null
+++ b/packages/eda-database/tests/Claim/OutboxClaimStrategiesTest.php
@@ -0,0 +1,53 @@
+name())->toBe($expected);
+})->with([
+ 'pgsql' => ['pgsql', '16.15', 'skip-locked'],
+ 'mysql 8.4' => ['mysql', '8.4.11', 'skip-locked'],
+ 'mysql 5.7' => ['mysql', '5.7.44', 'optimistic-lease'],
+ 'mariadb 11' => ['mariadb', '11.8.9-MariaDB', 'skip-locked'],
+ 'mariadb 10.5' => ['mariadb', '10.5.25-MariaDB', 'optimistic-lease'],
+ 'sqlsrv' => ['sqlsrv', '16.00.4135', 'read-past'],
+ 'sqlite' => ['sqlite', '3.46.1', 'optimistic-lease'],
+ 'unknown' => ['firebird', '5.0', 'optimistic-lease'],
+]);
+
+it('honours a forced strategy the server supports and refuses one it does not', function () {
+ expect(OutboxClaimStrategies::forCapabilities(DatabaseCapabilities::fromVersion('pgsql', '16.15'), 'optimistic-lease'))->toBeInstanceOf(OptimisticLeaseClaim::class)
+ ->and(OutboxClaimStrategies::forCapabilities(DatabaseCapabilities::fromVersion('mysql', '8.4.11'), 'skip-locked'))->toBeInstanceOf(SkipLockedClaim::class)
+ ->and(OutboxClaimStrategies::forCapabilities(DatabaseCapabilities::fromVersion('sqlsrv', '16.00'), 'read-past'))->toBeInstanceOf(ReadPastClaim::class)
+ ->and(fn () => OutboxClaimStrategies::forCapabilities(DatabaseCapabilities::fromVersion('mysql', '5.7.44'), 'skip-locked'))
+ ->toThrow(ConfigurationException::class, 'mysql 5.7.44')
+ ->and(fn () => OutboxClaimStrategies::forCapabilities(DatabaseCapabilities::fromVersion('pgsql', '16.15'), 'read-past'))
+ ->toThrow(ConfigurationException::class, 'firefly.eda.database.claim-strategy="read-past"')
+ ->and(fn () => OutboxClaimStrategies::forCapabilities(DatabaseCapabilities::fromVersion('pgsql', '16.15'), 'fastest'))
+ ->toThrow(ConfigurationException::class, 'auto|skip-locked|read-past|optimistic-lease');
+});
+
+it('resolves lazily, at the first claim, never at construction', function () {
+ $strategy = OutboxClaimStrategies::configured('auto');
+ expect($strategy)->toBeInstanceOf(ConfiguredClaimStrategy::class)
+ ->and($strategy->name())->toBe('auto');
+
+ $connection = new SQLiteConnection(new PDO('sqlite::memory:'), ':memory:', '', ['name' => 'lazy']);
+ $connection->getSchemaBuilder()->create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table));
+
+ expect($strategy->claim($connection, OutboxClaim::at('a', 1, new DateTimeImmutable('now'), 1)))->toBe([])
+ ->and($strategy->name())->toBe('optimistic-lease')
+ ->and(OutboxClaimStrategies::configured('optimistic-lease'))->toBeInstanceOf(OptimisticLeaseClaim::class);
+});
diff --git a/packages/eda-database/tests/CompiledManifestFreshnessTest.php b/packages/eda-database/tests/CompiledManifestFreshnessTest.php
new file mode 100644
index 00000000..e54194c0
--- /dev/null
+++ b/packages/eda-database/tests/CompiledManifestFreshnessTest.php
@@ -0,0 +1,30 @@
+ dirname(__DIR__).'/src'];
+ $cache = dirname(__DIR__).'/cache';
+
+ $freshComponents = sys_get_temp_dir().'/firefly-eda-database-fresh-c-'.bin2hex(random_bytes(6)).'.php';
+ $freshContext = sys_get_temp_dir().'/firefly-eda-database-fresh-x-'.bin2hex(random_bytes(6)).'.php';
+
+ try {
+ (new AutoConfigManifestCompiler)->write($src, $freshComponents, $freshContext);
+
+ expect(ComponentManifest::load($freshComponents))->toEqual(ComponentManifest::load($cache.'/firefly-eda-database-components.php'))
+ ->and(ContextManifest::load($freshContext))->toEqual(ContextManifest::load($cache.'/firefly-eda-database-context.php'));
+ } finally {
+ @unlink($freshComponents);
+ @unlink($freshContext);
+ }
+});
diff --git a/packages/eda-database/tests/DatabaseEventConsumerTest.php b/packages/eda-database/tests/DatabaseEventConsumerTest.php
new file mode 100644
index 00000000..f38c9295
--- /dev/null
+++ b/packages/eda-database/tests/DatabaseEventConsumerTest.php
@@ -0,0 +1,140 @@
+ Schema::create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table)));
+
+function consumerConnection(): Connection
+{
+ return DB::connection();
+}
+
+/** A consumer named $claimant whose clock reads $now (UTC) — lease expiry without sleeping. */
+function consumerAt(string $claimant, string $now, int $maxAttempts = 3): DatabaseEventConsumer
+{
+ return new DatabaseEventConsumer(
+ consumerConnection(),
+ new OptimisticLeaseClaim,
+ $maxAttempts,
+ leaseSeconds: 60,
+ claimant: $claimant,
+ awaitWork: static function (int $timeoutMs): void {},
+ clock: static fn (): DateTimeImmutable => new DateTimeImmutable($now, new DateTimeZone('UTC')),
+ );
+}
+
+function consumerPublish(int $id): void
+{
+ (new DatabaseEventPublisher(consumerConnection(), new SubscriberRegistry))->publish('users', 'user.created', ['id' => $id]);
+}
+
+it('delivers each committed row exactly once through ConsumerLoop and never grows the outbox', function () {
+ foreach ([1, 2, 3] as $id) {
+ consumerPublish($id);
+ }
+ $registry = new SubscriberRegistry;
+ $seen = [];
+ $registry->subscribe('user.*', function (EventEnvelope $envelope) use (&$seen): void {
+ $seen[] = $envelope->payload['id'];
+ });
+
+ $processed = (new ConsumerLoop)->run(consumerAt('a', '2026-10-12 10:00:00'), fn (EventEnvelope $e) => $registry->deliver($e), new ConsumerOptions(maxMessages: 3, pollTimeoutMs: 10));
+
+ expect($processed)->toBe(3)
+ ->and($seen)->toBe([1, 2, 3])
+ ->and(DB::table(OutboxSchema::TABLE)->count())->toBe(3)
+ ->and(DB::table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PUBLISHED)->whereNull('claimed_by')->count())->toBe(3);
+});
+
+it('keeps a claimed row from a second consumer until the lease ends, then hands it over', function () {
+ consumerPublish(7);
+
+ $first = consumerAt('a', '2026-10-12 10:00:00')->poll(10);
+ expect($first)->toBeInstanceOf(ReceivedEnvelope::class)
+ ->and(consumerAt('b', '2026-10-12 10:00:59')->poll(10))->toBeNull();
+
+ // a "crashed" without acking; at 10:01:00.001 a's lease is over and b takes the row.
+ $consumerB = consumerAt('b', '2026-10-12 10:01:00.001');
+ $second = $consumerB->poll(10);
+ assert($first instanceof ReceivedEnvelope && $second instanceof ReceivedEnvelope);
+
+ // a comes back and acks: its lease is gone, so nothing happens; b's ack settles the row.
+ consumerAt('a', '2026-10-12 10:01:01')->ack($first);
+ expect(DB::table(OutboxSchema::TABLE)->value('status'))->toBe(OutboxSchema::STATUS_PENDING);
+
+ $consumerB->ack($second);
+ expect(DB::table(OutboxSchema::TABLE)->value('status'))->toBe(OutboxSchema::STATUS_PUBLISHED);
+});
+
+it('nacks back to PENDING with the lease cleared, and FAILS past max attempts', function () {
+ consumerPublish(8);
+ $consumer = consumerAt('a', '2026-10-12 10:00:00', maxAttempts: 2);
+
+ $received = $consumer->poll(10);
+ assert($received instanceof ReceivedEnvelope);
+ $consumer->nack($received);
+ $row = DB::table(OutboxSchema::TABLE)->first();
+ expect($row?->status)->toBe(OutboxSchema::STATUS_PENDING)
+ ->and($row?->attempts)->toEqual(1)
+ ->and($row?->claimed_by)->toBeNull();
+
+ // Retried at once (no lease left to wait out), and the second failure is final.
+ $again = $consumer->poll(10);
+ assert($again instanceof ReceivedEnvelope);
+ $consumer->nack($again);
+ $row = DB::table(OutboxSchema::TABLE)->first();
+ expect($row?->status)->toBe(OutboxSchema::STATUS_FAILED)
+ ->and($row?->failed_at)->not->toBeNull()
+ ->and($row?->error_message)->toBe('in-process handler failed');
+});
+
+it('hands an undecodable row over as poison, which the loop dead-letters as FAILED without retry', function () {
+ DB::table(OutboxSchema::TABLE)->insert([
+ 'destination' => 'users', 'channel' => 'firefly_eda_events', 'event_type' => 'user.created',
+ 'payload' => '{not json', 'headers' => '{}', 'status' => OutboxSchema::STATUS_PENDING, 'attempts' => 0,
+ ]);
+
+ $processed = (new ConsumerLoop)->run(consumerAt('a', '2026-10-12 10:00:00'), fn () => null, new ConsumerOptions(maxMessages: 1, pollTimeoutMs: 10));
+ $row = DB::table(OutboxSchema::TABLE)->first();
+
+ expect($processed)->toBe(1)
+ ->and($row?->status)->toBe(OutboxSchema::STATUS_FAILED)
+ ->and($row?->error_message)->toStartWith('undecodable outbox row:');
+});
+
+it('waits for work only when there was nothing to claim', function () {
+ $waits = [];
+ $consumer = new DatabaseEventConsumer(consumerConnection(), new OptimisticLeaseClaim, awaitWork: static function (int $timeoutMs) use (&$waits): void {
+ $waits[] = $timeoutMs;
+ });
+
+ expect($consumer->poll(25))->toBeNull()->and($waits)->toBe([25]);
+
+ consumerPublish(9);
+ expect($consumer->poll(25))->toBeInstanceOf(ReceivedEnvelope::class)->and($waits)->toBe([25]);
+});
+
+it('keeps PostgresEventConsumer\'s constructor and makes it a database consumer', function () {
+ $consumer = new PostgresEventConsumer(consumerConnection(), 'firefly_eda_events', 3, static function (int $timeoutMs): void {});
+
+ expect($consumer)->toBeInstanceOf(DatabaseEventConsumer::class)
+ ->and($consumer->claimant())->toMatch('/:\d+:[0-9a-f]{8}$/');
+});
diff --git a/packages/eda-database/tests/DatabaseEventPublisherTest.php b/packages/eda-database/tests/DatabaseEventPublisherTest.php
new file mode 100644
index 00000000..2ae97ded
--- /dev/null
+++ b/packages/eda-database/tests/DatabaseEventPublisherTest.php
@@ -0,0 +1,70 @@
+ Schema::create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table)));
+
+it('writes one PENDING row that commits and rolls back with the caller\'s transaction', function () {
+ $publisher = new DatabaseEventPublisher(DB::connection(), new SubscriberRegistry);
+
+ DB::transaction(fn () => $publisher->publish('orders', 'order.placed', ['id' => 1], ['x-correlation-id' => 'c-1']));
+
+ try {
+ DB::transaction(function () use ($publisher): void {
+ $publisher->publish('orders', 'order.placed', ['id' => 2]);
+ throw new RuntimeException('business failure after the outbox write');
+ });
+ } catch (RuntimeException) {
+ // expected
+ }
+
+ $row = DB::table(OutboxSchema::TABLE)->first();
+
+ expect(DB::table(OutboxSchema::TABLE)->count())->toBe(1)
+ ->and($row?->status)->toBe(OutboxSchema::STATUS_PENDING)
+ ->and($row?->transaction_id)->toBe('c-1')
+ ->and($row?->channel)->toBe('firefly_eda_events')
+ ->and($row?->claimed_by)->toBeNull()
+ ->and(json_decode(is_string($row?->payload) ? $row->payload : '', true))->toBe(['id' => 1]);
+});
+
+it('records subscriptions on the shared registry the consumer delivers into', function () {
+ $registry = new SubscriberRegistry;
+ $seen = [];
+ (new DatabaseEventPublisher(DB::connection(), $registry))->subscribe('order.*', function (EventEnvelope $envelope) use (&$seen): void {
+ $seen[] = $envelope->eventType;
+ });
+
+ $registry->deliver(new EventEnvelope('order.placed', 'orders'));
+
+ expect($seen)->toBe(['order.placed']);
+});
+
+it('is an outbox writer, and only its Postgres subclass notifies', function () {
+ $statements = [];
+ DB::listen(static function (QueryExecuted $query) use (&$statements): void {
+ $statements[] = $query->sql;
+ });
+
+ $database = new DatabaseEventPublisher(DB::connection(), new SubscriberRegistry, 'chan');
+ $database->publish('orders', 'order.placed', ['id' => 3]);
+
+ expect($database)->toBeInstanceOf(OutboxWriter::class)
+ ->and(new PostgresEventPublisher(DB::connection(), new SubscriberRegistry))->toBeInstanceOf(DatabaseEventPublisher::class)
+ ->and(implode("\n", $statements))->not->toContain('pg_notify')
+ ->and(DB::table(OutboxSchema::TABLE)->value('channel'))->toBe('chan');
+});
diff --git a/packages/eda-database/tests/DatabaseOutboxSettingsTest.php b/packages/eda-database/tests/DatabaseOutboxSettingsTest.php
new file mode 100644
index 00000000..06d8b7c9
--- /dev/null
+++ b/packages/eda-database/tests/DatabaseOutboxSettingsTest.php
@@ -0,0 +1,54 @@
+ $eda */
+function outboxSettings(array $eda): DatabaseOutboxSettings
+{
+ return DatabaseOutboxSettings::from(new Config(new Repository(['firefly' => ['eda' => $eda]])));
+}
+
+it('defaults to the application connection, auto claims, a five-minute lease', function () {
+ expect(outboxSettings([]))->toEqual(new DatabaseOutboxSettings(null, 'firefly_eda_events', 3, 'auto', 300.0, 500));
+});
+
+it('reads the database keys', function () {
+ expect(outboxSettings(['database' => [
+ 'connection' => 'mysql', 'channel' => 'orders', 'max-attempts' => 5, 'claim-strategy' => 'optimistic-lease',
+ 'lease-seconds' => 30, 'poll-interval-ms' => 250,
+ ]]))->toEqual(new DatabaseOutboxSettings('mysql', 'orders', 5, 'optimistic-lease', 30.0, 250));
+});
+
+it('keeps every firefly.eda.postgres key working as an alias, and lets the database key win', function () {
+ expect(outboxSettings(['postgres' => ['connection' => 'pgsql', 'channel' => 'pgchan', 'max_attempts' => 7]]))
+ ->toEqual(new DatabaseOutboxSettings('pgsql', 'pgchan', 7, 'auto', 300.0, 500))
+ ->and(outboxSettings([
+ 'database' => ['connection' => 'mysql', 'max-attempts' => 2],
+ 'postgres' => ['connection' => 'pgsql', 'max_attempts' => 7],
+ ]))->toEqual(new DatabaseOutboxSettings('mysql', 'firefly_eda_events', 2, 'auto', 300.0, 500));
+});
+
+it('treats a blank connection as the default one and floors nonsense values', function () {
+ expect(outboxSettings(['database' => ['connection' => '', 'max-attempts' => 0, 'lease-seconds' => 0, 'poll-interval-ms' => 1]]))
+ ->toEqual(new DatabaseOutboxSettings(null, 'firefly_eda_events', 1, 'auto', 1.0, 10));
+});
+
+it('treats a connection key that is present but null (an unset env()) as the default connection', function () {
+ expect(outboxSettings(['database' => ['connection' => null]])->connection)->toBeNull()
+ ->and(outboxSettings(['postgres' => ['connection' => null]])->connection)->toBeNull();
+});
+
+it('lets a database key that is present but null fall back to its postgres alias, then to the default', function () {
+ expect(outboxSettings([
+ 'database' => ['connection' => null, 'channel' => null, 'max-attempts' => null],
+ 'postgres' => ['connection' => 'pgsql', 'channel' => 'pgchan', 'max_attempts' => 7],
+ ]))->toEqual(new DatabaseOutboxSettings('pgsql', 'pgchan', 7, 'auto', 300.0, 500))
+ ->and(outboxSettings([
+ 'database' => ['connection' => null, 'channel' => null, 'max-attempts' => null],
+ 'postgres' => ['connection' => null, 'channel' => null, 'max_attempts' => null],
+ ]))->toEqual(new DatabaseOutboxSettings(null, 'firefly_eda_events', 3, 'auto', 300.0, 500));
+});
diff --git a/packages/eda-database/tests/EdaDatabaseBootDiscoveryTest.php b/packages/eda-database/tests/EdaDatabaseBootDiscoveryTest.php
new file mode 100644
index 00000000..da04040d
--- /dev/null
+++ b/packages/eda-database/tests/EdaDatabaseBootDiscoveryTest.php
@@ -0,0 +1,36 @@
+instance('config', new Repository(['firefly' => ['eda' => ['provider' => $provider]]]));
+ $app->instance('files', new Filesystem);
+ $app->register(DatabaseServiceProvider::class);
+ $app->register(MigrationServiceProvider::class);
+ (new EdaDatabaseBootServiceProvider($app))->boot();
+ (new EdaPostgresBootServiceProvider($app))->boot();
+ /** @var Migrator $migrator */
+ $migrator = $app->make('migrator');
+
+ $paths = [];
+ foreach ($migrator->paths() as $path) {
+ $paths[] = realpath($path);
+ }
+
+ expect($paths)->toBe($loaded ? [realpath(dirname(__DIR__).'/database/migrations')] : []);
+ $app->flush();
+})->with([
+ 'memory' => ['memory', false],
+ 'database' => ['database', true],
+ 'postgres' => ['postgres', true],
+]);
diff --git a/packages/eda-database/tests/Fixtures/OrderPlaced.php b/packages/eda-database/tests/Fixtures/OrderPlaced.php
new file mode 100644
index 00000000..9a7f7f4c
--- /dev/null
+++ b/packages/eda-database/tests/Fixtures/OrderPlaced.php
@@ -0,0 +1,16 @@
+ */
+ public array $published = [];
+
+ public bool $fail = false;
+
+ public function subscribe(string $eventTypePattern, callable $handler): void {}
+
+ /**
+ * @param array $payload
+ * @param array $headers
+ */
+ public function publish(string $destination, string $eventType, array $payload, array $headers = []): void
+ {
+ if ($this->fail) {
+ throw new RuntimeException('downstream down');
+ }
+
+ $this->published[] = new EventEnvelope($eventType, $destination, $payload, $headers);
+ }
+
+ public function start(): void {}
+
+ public function stop(): void {}
+}
diff --git a/packages/eda-database/tests/Integration/DatabaseOutboxRoundTripIntegrationTest.php b/packages/eda-database/tests/Integration/DatabaseOutboxRoundTripIntegrationTest.php
new file mode 100644
index 00000000..dd4d1179
--- /dev/null
+++ b/packages/eda-database/tests/Integration/DatabaseOutboxRoundTripIntegrationTest.php
@@ -0,0 +1,157 @@
+ IntegrationDsn::sqlitePath('outbox-roundtrip')] : [];
+ config([
+ 'database.connections.it_writer' => IntegrationDsn::connectionConfig($lane, $overrides),
+ 'database.connections.it_consumer' => IntegrationDsn::connectionConfig($lane, $overrides),
+ ]);
+ DB::setDefaultConnection('it_writer');
+
+ return [DB::connection('it_writer'), DB::connection('it_consumer')];
+}
+
+/** Runs one direction of one of the package's anonymous migrations, on the default (writer) connection. */
+function roundTripMigrate(string $file, string $direction = 'up'): void
+{
+ $migration = require dirname(__DIR__, 2).'/database/migrations/'.$file;
+
+ if (! $migration instanceof Migration || ! method_exists($migration, 'up') || ! method_exists($migration, 'down')) {
+ throw new RuntimeException('Expected an executable migration.');
+ }
+
+ $direction === 'down' ? $migration->down() : $migration->up();
+}
+
+function roundTripConsumer(Connection $connection, string $claimant, string $now, int $maxAttempts = 3): DatabaseEventConsumer
+{
+ return new DatabaseEventConsumer(
+ $connection,
+ OutboxClaimStrategies::resolve($connection),
+ $maxAttempts,
+ leaseSeconds: 60,
+ claimant: $claimant,
+ awaitWork: static function (int $timeoutMs): void {},
+ clock: static fn (): DateTimeImmutable => new DateTimeImmutable($now, new DateTimeZone('UTC')),
+ );
+}
+
+foreach (IntegrationDsn::lanes() as $lane) {
+ it("upgrades and creates the outbox with its migrations on {$lane}", function () use ($lane): void {
+ roundTripSessions($lane);
+ Schema::dropIfExists(OutboxSchema::TABLE);
+ Schema::create(OutboxSchema::TABLE, function (Blueprint $table): void {
+ $table->bigIncrements('id');
+ $table->string('destination');
+ $table->string('channel', 63);
+ $table->string('event_type');
+ $table->json('payload');
+ $table->json('headers');
+ $table->string('transaction_id')->nullable();
+ $table->string('status')->default(OutboxSchema::STATUS_PENDING);
+ $table->unsignedInteger('attempts')->default(0);
+ $table->text('error_message')->nullable();
+ $table->timestamp('created_at')->useCurrent();
+ $table->timestamp('processed_at')->nullable();
+ $table->timestamp('failed_at')->nullable();
+ });
+
+ $upgrade = '2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php';
+ roundTripMigrate($upgrade);
+ expect(Schema::hasColumns(OutboxSchema::TABLE, ['claimed_by', 'claimed_until']))->toBeTrue();
+ roundTripMigrate($upgrade, 'down');
+ expect(Schema::hasColumn(OutboxSchema::TABLE, 'claimed_by'))->toBeFalse();
+
+ Schema::drop(OutboxSchema::TABLE);
+ roundTripMigrate('2026_07_27_000000_create_firefly_eda_outbox_table.php');
+ roundTripMigrate($upgrade);
+ expect(Schema::hasColumns(OutboxSchema::TABLE, ['claimed_by', 'claimed_until']))->toBeTrue();
+ Schema::drop(OutboxSchema::TABLE);
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+
+ it("writes same-transaction, leases, hands over on expiry, retries and relays on {$lane}", function () use ($lane): void {
+ [$writer, $reader] = roundTripSessions($lane);
+ $writer->getSchemaBuilder()->dropIfExists(OutboxSchema::TABLE);
+ $writer->getSchemaBuilder()->create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table));
+
+ try {
+ $publisher = new DatabaseEventPublisher($writer, new SubscriberRegistry);
+
+ // (1) invisible to another session until the writer commits
+ $writer->transaction(function () use ($publisher, $reader): void {
+ $publisher->publish('orders', 'order.placed', ['n' => 1]);
+ // SQL Server without READ_COMMITTED_SNAPSHOT makes a plain read WAIT on the uncommitted row (here, forever:
+ // the writer is this process); READPAST reads what is committed, which is the claim being made.
+ $visible = $reader->table(OutboxSchema::TABLE);
+ expect(($reader->getDriverName() === 'sqlsrv' ? $visible->lock('with(readpast)') : $visible)->count())->toBe(0);
+ });
+ expect($reader->table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PENDING)->count())->toBe(1);
+
+ // (2) a lease keeps a second consumer out until it ends; the late first claimant settles nothing
+ $first = roundTripConsumer($reader, 'a', '2026-10-12 10:00:00')->poll(10);
+ expect($first)->toBeInstanceOf(ReceivedEnvelope::class)
+ ->and(roundTripConsumer($reader, 'b', '2026-10-12 10:00:30')->poll(10))->toBeNull();
+ $consumerB = roundTripConsumer($reader, 'b', '2026-10-12 10:01:00.500');
+ $second = $consumerB->poll(10);
+ assert($first instanceof ReceivedEnvelope && $second instanceof ReceivedEnvelope);
+ expect($second->envelope?->payload)->toBe(['n' => 1]);
+ roundTripConsumer($reader, 'a', '2026-10-12 10:01:01')->ack($first);
+ expect($reader->table(OutboxSchema::TABLE)->value('status'))->toBe(OutboxSchema::STATUS_PENDING);
+ $consumerB->ack($second);
+ expect($reader->table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PUBLISHED)->whereNotNull('processed_at')->count())->toBe(1);
+
+ // (3) nack -> PENDING with attempts+1, then FAILED past max attempts
+ $publisher->publish('orders', 'order.updated', ['n' => 2]);
+ $retrying = roundTripConsumer($reader, 'c', '2026-10-12 11:00:00', maxAttempts: 2);
+ $received = $retrying->poll(10);
+ assert($received instanceof ReceivedEnvelope);
+ $retrying->nack($received);
+ expect($reader->table(OutboxSchema::TABLE)->where('event_type', 'order.updated')->value('status'))->toBe(OutboxSchema::STATUS_PENDING);
+ $again = $retrying->poll(10);
+ assert($again instanceof ReceivedEnvelope);
+ $retrying->nack($again);
+ expect($reader->table(OutboxSchema::TABLE)->where('event_type', 'order.updated')->value('status'))->toBe(OutboxSchema::STATUS_FAILED);
+
+ // (4) the relay forwards to a distinct downstream and refuses the writer
+ $publisher->publish('billing', 'invoice.raised', ['n' => 3], ['x-c' => 'd']);
+ $downstream = new RecordingDownstream;
+ expect((new OutboxRelay($reader, $downstream, OutboxClaimStrategies::resolve($reader)))->relayBatch())->toBe(1)
+ ->and($downstream->published[0]->payload)->toBe(['n' => 3])
+ ->and($downstream->published[0]->headers)->toBe(['x-c' => 'd'])
+ ->and(fn () => new OutboxRelay($reader, $publisher))->toThrow(LogicException::class);
+ } finally {
+ $writer->getSchemaBuilder()->dropIfExists(OutboxSchema::TABLE);
+ $writer->disconnect();
+ $reader->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/eda-database/tests/Integration/OutboxConcurrencyIntegrationTest.php b/packages/eda-database/tests/Integration/OutboxConcurrencyIntegrationTest.php
new file mode 100644
index 00000000..2bcc4ff8
--- /dev/null
+++ b/packages/eda-database/tests/Integration/OutboxConcurrencyIntegrationTest.php
@@ -0,0 +1,122 @@
+isRunning();
+ if (str_contains($process->getOutput(), 'ready')) {
+ return true;
+ }
+ if (! $running) {
+ return false;
+ }
+ usleep(1000);
+ }
+
+ return false;
+}
+
+/** Removes the files that exist — an `@unlink` of a missing file is still a warning the test run reports. */
+function outboxRaceRemoveFiles(string ...$paths): void
+{
+ foreach ($paths as $path) {
+ if (is_file($path)) {
+ unlink($path);
+ }
+ }
+}
+foreach (IntegrationDsn::lanes() as $lane) {
+ foreach (['relay', 'consume'] as $mode) {
+ foreach (['auto', 'optimistic-lease'] as $strategy) {
+ if ($lane === 'sqlite' && $strategy === 'auto') {
+ continue;
+ }
+
+ it("delivers every row exactly once with two racing {$mode} processes on {$lane} ({$strategy})", function () use ($lane, $mode, $strategy): void {
+ $path = IntegrationDsn::sqlitePath('outbox-race');
+ if ($lane === 'sqlite') {
+ outboxRaceRemoveFiles($path, $path.'-wal', $path.'-shm');
+ }
+ $overrides = $lane === 'sqlite' ? ['database' => $path] : [];
+ $connection = IntegrationDsn::connect($lane, 'race-parent', $overrides);
+ $schema = $connection->getSchemaBuilder();
+ $schema->dropIfExists(OutboxSchema::TABLE);
+ $schema->create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table));
+
+ $publisher = new DatabaseEventPublisher($connection, new SubscriberRegistry);
+ $connection->transaction(function () use ($publisher): void {
+ foreach (range(1, 200) as $n) {
+ $publisher->publish('orders', 'order.placed', ['n' => $n]);
+ }
+ });
+
+ $go = sys_get_temp_dir().'/firefly-outbox-race-'.bin2hex(random_bytes(6));
+ $racers = [];
+ foreach (['racer-a', 'racer-b'] as $claimant) {
+ $arguments = [PHP_BINARY, '-d', 'memory_limit=512M', dirname(__DIR__).'/Support/outbox-racer.php', $lane, $mode, $strategy, $go, $claimant];
+ if ($lane === 'sqlite') {
+ $arguments[] = $path;
+ }
+ $process = new Process($arguments, timeout: 120);
+ $process->start();
+ $racers[] = $process;
+ }
+
+ try {
+ foreach ($racers as $process) {
+ expect(outboxRaceAwaitReady($process))->toBeTrue($process->getErrorOutput());
+ }
+ touch($go);
+
+ $delivered = [];
+ foreach ($racers as $process) {
+ expect($process->wait())->toBe(0, $process->getErrorOutput());
+ preg_match_all('/^delivered (\d+)$/m', $process->getOutput(), $matches);
+ foreach ($matches[1] as $n) {
+ $delivered[] = (int) $n;
+ }
+ }
+ sort($delivered);
+
+ expect($delivered)->toBe(range(1, 200))
+ ->and($connection->table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PUBLISHED)->count())->toBe(200)
+ ->and($connection->table(OutboxSchema::TABLE)->whereNotNull('claimed_by')->count())->toBe(0);
+ } finally {
+ foreach ($racers as $process) {
+ $process->stop();
+ }
+ outboxRaceRemoveFiles($go);
+ $schema->dropIfExists(OutboxSchema::TABLE);
+ $connection->disconnect();
+ if ($lane === 'sqlite') {
+ outboxRaceRemoveFiles($path, $path.'-wal', $path.'-shm');
+ }
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+ }
+ }
+}
diff --git a/packages/eda-database/tests/LeaseClockTest.php b/packages/eda-database/tests/LeaseClockTest.php
new file mode 100644
index 00000000..2390d761
--- /dev/null
+++ b/packages/eda-database/tests/LeaseClockTest.php
@@ -0,0 +1,27 @@
+ new DateTimeImmutable('2026-10-12 12:00:00.123456', new DateTimeZone('+02:00')));
+
+ expect(LeaseClock::format($now))->toBe('2026-10-12 10:00:00.123')
+ ->and($now->getTimezone()->getName())->toBe('UTC');
+});
+
+it('moves an instant by fractional seconds', function () {
+ $start = new DateTimeImmutable('2026-10-12 10:00:00.900', new DateTimeZone('UTC'));
+
+ expect(LeaseClock::format(LeaseClock::plus($start, 0.25)))->toBe('2026-10-12 10:00:01.150')
+ ->and(LeaseClock::format(LeaseClock::plus($start, 300)))->toBe('2026-10-12 10:05:00.900');
+});
+
+it('names a claimant after the host and the process, distinct per call', function () {
+ $a = LeaseClock::claimant();
+
+ expect($a)->toMatch('/^.+:\d+:[0-9a-f]{8}$/')
+ ->and(strlen($a))->toBeLessThanOrEqual(100)
+ ->and(LeaseClock::claimant())->not->toBe($a);
+});
diff --git a/packages/eda-database/tests/OutboxProviderCoexistenceBootTest.php b/packages/eda-database/tests/OutboxProviderCoexistenceBootTest.php
new file mode 100644
index 00000000..a9a196a5
--- /dev/null
+++ b/packages/eda-database/tests/OutboxProviderCoexistenceBootTest.php
@@ -0,0 +1,97 @@
+connection;
+ }
+
+ public function getDefaultConnection(): string
+ {
+ return 'default';
+ }
+
+ public function setDefaultConnection($name): void {}
+ };
+
+ return fireflyApplication(
+ ['firefly' => ['eda' => ['provider' => $provider]]],
+ [DataServiceProvider::class, EdaDatabaseServiceProvider::class, EdaPostgresServiceProvider::class],
+ bindings: [
+ ConnectionResolverInterface::class => $resolver,
+ HandlerManifest::class => new HandlerManifest([], []),
+ CorrelationContext::class => new CorrelationContext,
+ AggregateTracker::class => new AggregateTracker,
+ ApplicationEventPublisher::class => new class implements ApplicationEventPublisher
+ {
+ public function publish(object $event): void {}
+ },
+ ],
+ );
+}
+
+/** @return list */
+function outboxCoexistenceDispatchers(Application $app): array
+{
+ /** @var FireflyKernel $kernel */
+ $kernel = $app->make(FireflyKernel::class);
+
+ $beans = [];
+ foreach ($kernel->context()->definitions->all() as $definition) {
+ foreach ($definition->descriptor->beans as $bean) {
+ if ($bean->returns === DomainEventDispatcher::class) {
+ $beans[] = $definition->class().'::'.$bean->method;
+ }
+ }
+ }
+ sort($beans);
+
+ return $beans;
+}
+
+it('lets exactly one dispatcher and one outbox health indicator survive per provider', function (string $provider, string $dispatcher, bool $database, bool $postgres) {
+ $app = outboxCoexistenceApp($provider);
+ /** @var ApplicationContext $context */
+ $context = $app->make(ApplicationContext::class);
+
+ expect(outboxCoexistenceDispatchers($app))->toBe([$dispatcher])
+ ->and($context->has(OutboxHealthIndicator::class))->toBe($database)
+ ->and($context->has(PostgresHealthIndicator::class))->toBe($postgres);
+})->with([
+ 'database' => ['database', DatabaseOutboxAutoConfiguration::class.'::domainEventDispatcher', true, false],
+ 'postgres' => ['postgres', PostgresOutboxAutoConfiguration::class.'::domainEventDispatcher', false, true],
+ 'memory' => ['memory', DataAutoConfiguration::class.'::domainEventDispatcher', false, false],
+]);
diff --git a/packages/eda-database/tests/OutboxRelayCommandTest.php b/packages/eda-database/tests/OutboxRelayCommandTest.php
new file mode 100644
index 00000000..b5b398cf
--- /dev/null
+++ b/packages/eda-database/tests/OutboxRelayCommandTest.php
@@ -0,0 +1,42 @@
+insert([
+ 'destination' => 'orders', 'channel' => 'firefly_eda_events', 'event_type' => 'order.created',
+ 'payload' => json_encode(['id' => $id], JSON_THROW_ON_ERROR), 'headers' => '{}',
+ 'status' => OutboxSchema::STATUS_PENDING, 'attempts' => 0,
+ ]);
+}
+
+it('relays committed rows to the bound downstream under provider=database', function () {
+ /** @var DatabaseRelayCommandTestCase $this */
+ $downstream = new RecordingDownstream;
+ $this->app()->instance(RelayDownstream::BINDING, $downstream);
+ relayCommandRow(3);
+
+ expect($this->runRelay())->toBe(0)
+ ->and($downstream->published)->toHaveCount(1)
+ ->and(DB::table(OutboxSchema::TABLE)->value('status'))->toBe(OutboxSchema::STATUS_PUBLISHED);
+});
+
+it('fails before claiming anything when a forced claim strategy is unsupported', function () {
+ /** @var DatabaseRelayCommandTestCase $this */
+ $this->app()->instance(RelayDownstream::BINDING, new RecordingDownstream);
+ config(['firefly.eda.database.claim-strategy' => 'skip-locked']);
+ relayCommandRow(4);
+
+ expect($this->runRelay())->toBe(1)
+ ->and($this->relayOutput())->toContain('sqlite')
+ ->and(DB::table(OutboxSchema::TABLE)->whereNull('claimed_by')->where('status', OutboxSchema::STATUS_PENDING)->count())->toBe(1);
+});
diff --git a/packages/eda-database/tests/OutboxRelayTest.php b/packages/eda-database/tests/OutboxRelayTest.php
new file mode 100644
index 00000000..07d3e87c
--- /dev/null
+++ b/packages/eda-database/tests/OutboxRelayTest.php
@@ -0,0 +1,99 @@
+ OutboxSchema::blueprint($table));
+ $publisher = new DatabaseEventPublisher(DB::connection(), new SubscriberRegistry);
+ foreach ([1, 2, 3] as $n) {
+ $publisher->publish('orders', 'order.placed', ['n' => $n], ['x-n' => (string) $n]);
+ }
+});
+
+function relayConnection(): Connection
+{
+ return DB::connection();
+}
+
+function relayAt(RecordingDownstream $downstream, string $claimant, string $now, int $batch = 50, int $maxAttempts = 3): OutboxRelay
+{
+ return new OutboxRelay(
+ relayConnection(),
+ $downstream,
+ new OptimisticLeaseClaim,
+ $batch,
+ $maxAttempts,
+ leaseSeconds: 60,
+ claimant: $claimant,
+ clock: static fn (): DateTimeImmutable => new DateTimeImmutable($now, new DateTimeZone('UTC')),
+ );
+}
+
+it('forwards each claimed row to the downstream, byte for byte, and marks it PUBLISHED', function () {
+ $downstream = new RecordingDownstream;
+
+ expect(relayAt($downstream, 'r', '2026-10-12 10:00:00')->relayBatch())->toBe(3)
+ ->and(array_map(static fn (EventEnvelope $e): mixed => $e->payload['n'], $downstream->published))->toBe([1, 2, 3])
+ ->and($downstream->published[0]->headers)->toBe(['x-n' => '1'])
+ ->and(DB::table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PUBLISHED)->whereNull('claimed_by')->count())->toBe(3);
+});
+
+it('splits the outbox between two relays without forwarding a row twice', function () {
+ $a = new RecordingDownstream;
+ $b = new RecordingDownstream;
+
+ expect(relayAt($a, 'a', '2026-10-12 10:00:00', batch: 2)->relayBatch())->toBe(2)
+ ->and(relayAt($b, 'b', '2026-10-12 10:00:00', batch: 2)->relayBatch())->toBe(1)
+ ->and(count($a->published) + count($b->published))->toBe(3);
+});
+
+it('counts a failed forward as an attempt and FAILS the row past max attempts', function () {
+ $downstream = new RecordingDownstream;
+ $downstream->fail = true;
+
+ relayAt($downstream, 'r', '2026-10-12 10:00:00', maxAttempts: 2)->relayBatch();
+ expect(DB::table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PENDING)->where('attempts', 1)->whereNull('claimed_by')->count())->toBe(3);
+
+ relayAt($downstream, 'r', '2026-10-12 10:00:01', maxAttempts: 2)->relayBatch();
+ $row = DB::table(OutboxSchema::TABLE)->first();
+ expect(DB::table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_FAILED)->count())->toBe(3)
+ ->and((string) json_encode($row?->error_message))->toContain('downstream down');
+});
+
+it('refuses an outbox writer as its downstream', function () {
+ expect(fn () => new OutboxRelay(relayConnection(), new DatabaseEventPublisher(relayConnection(), new SubscriberRegistry)))
+ ->toThrow(LogicException::class, 'PostgresEventPublisher')
+ ->and(fn () => new OutboxRelay(relayConnection(), new PostgresEventPublisher(relayConnection(), new SubscriberRegistry)))
+ ->toThrow(LogicException::class, 'outbox writer');
+});
+
+it('keeps the deprecated Postgres relay constructor working', function () {
+ $downstream = new RecordingDownstream;
+
+ expect((new Firefly\Eda\Postgres\Outbox\OutboxRelay(relayConnection(), $downstream, 50, 3, true))->relayBatch())->toBe(3);
+});
+
+it('maps the deprecated useSkipLocked=false onto the portable lease, which two relays can share', function () {
+ $a = new RecordingDownstream;
+ $b = new RecordingDownstream;
+
+ expect((new Firefly\Eda\Postgres\Outbox\OutboxRelay(relayConnection(), $a, 2, 3, false))->relayBatch())->toBe(2)
+ ->and((new Firefly\Eda\Postgres\Outbox\OutboxRelay(relayConnection(), $b, 2, 3, false))->relayBatch())->toBe(1)
+ ->and(count($a->published) + count($b->published))->toBe(3);
+});
diff --git a/packages/eda-database/tests/OutboxSchemaTest.php b/packages/eda-database/tests/OutboxSchemaTest.php
new file mode 100644
index 00000000..20a81acf
--- /dev/null
+++ b/packages/eda-database/tests/OutboxSchemaTest.php
@@ -0,0 +1,66 @@
+down() : $migration->up();
+}
+
+it('creates the outbox with the lease columns a claim writes', function () {
+ outboxSchemaMigrate('2026_07_27_000000_create_firefly_eda_outbox_table.php');
+
+ expect(Schema::hasColumns(OutboxSchema::TABLE, ['id', 'destination', 'channel', 'event_type', 'payload', 'headers', 'transaction_id', 'status', 'attempts', 'error_message', 'created_at', 'processed_at', 'failed_at', 'claimed_by', 'claimed_until']))->toBeTrue();
+
+ // The second migration is a no-op on a table that already has the columns — a fresh install runs both.
+ outboxSchemaMigrate('2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php');
+ expect(Schema::hasColumn(OutboxSchema::TABLE, 'claimed_by'))->toBeTrue();
+});
+
+it('adds the lease columns to an outbox created before them, and takes them away again', function () {
+ // The pre-W1 shape, as an application that migrated with 26.10.1 has it.
+ Schema::create(OutboxSchema::TABLE, function (Blueprint $table): void {
+ $table->bigIncrements('id');
+ $table->string('destination');
+ $table->string('channel', 63);
+ $table->string('event_type');
+ $table->json('payload');
+ $table->json('headers');
+ $table->string('transaction_id')->nullable();
+ $table->string('status')->default(OutboxSchema::STATUS_PENDING);
+ $table->unsignedInteger('attempts')->default(0);
+ $table->text('error_message')->nullable();
+ $table->timestamp('created_at')->useCurrent();
+ $table->timestamp('processed_at')->nullable();
+ $table->timestamp('failed_at')->nullable();
+ });
+
+ outboxSchemaMigrate('2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php');
+ expect(Schema::hasColumns(OutboxSchema::TABLE, ['claimed_by', 'claimed_until']))->toBeTrue();
+
+ outboxSchemaMigrate('2026_10_12_000000_add_claim_columns_to_firefly_eda_outbox_table.php', 'down');
+ expect(Schema::hasColumn(OutboxSchema::TABLE, 'claimed_by'))->toBeFalse();
+});
+
+it('keeps the old Postgres FQCNs working as deprecated subclasses', function () {
+ expect(new Firefly\Eda\Postgres\Outbox\OutboxSchema)->toBeInstanceOf(OutboxSchema::class)
+ ->and(Firefly\Eda\Postgres\Outbox\OutboxSchema::TABLE)->toBe('firefly_eda_outbox')
+ ->and(new Firefly\Eda\Postgres\Outbox\OutboxRow)->toBeInstanceOf(OutboxRow::class)
+ ->and(Firefly\Eda\Postgres\Outbox\OutboxRow::asMap('{"a":1}'))->toBe(['a' => 1]);
+});
diff --git a/packages/eda-database/tests/ReflectionFreeEdaDatabaseTest.php b/packages/eda-database/tests/ReflectionFreeEdaDatabaseTest.php
new file mode 100644
index 00000000..dee7dea2
--- /dev/null
+++ b/packages/eda-database/tests/ReflectionFreeEdaDatabaseTest.php
@@ -0,0 +1,26 @@
+ basenames under $dir whose contents reference boot-time reflection */
+function edaDatabaseReflectionHits(string $dir): array
+{
+ $hits = [];
+ $files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir, RecursiveDirectoryIterator::SKIP_DOTS));
+ /** @var SplFileInfo $file */
+ foreach ($files as $file) {
+ if (! $file->isFile() || $file->getExtension() !== 'php') {
+ continue;
+ }
+ if (preg_match('/ReflectionClass|ReflectionMethod|getAttributes/', (string) file_get_contents((string) $file->getRealPath())) === 1) {
+ $hits[] = $file->getBasename();
+ }
+ }
+ sort($hits);
+
+ return $hits;
+}
+
+it('the firefly/eda-database adapter uses no boot-time reflection', function () {
+ expect(edaDatabaseReflectionHits(__DIR__.'/../src'))->toBe([]);
+});
diff --git a/packages/eda-database/tests/RelayDownstreamTest.php b/packages/eda-database/tests/RelayDownstreamTest.php
new file mode 100644
index 00000000..6f14a81e
--- /dev/null
+++ b/packages/eda-database/tests/RelayDownstreamTest.php
@@ -0,0 +1,68 @@
+ $values dotted key => value
+ * @return array{Container, Config}
+ */
+function downstreamEnv(array $values): array
+{
+ $repository = new Repository;
+ foreach ($values as $key => $value) {
+ $repository->set($key, $value);
+ }
+
+ return [new Container, new Config($repository)];
+}
+
+it('resolves the new key, and the legacy Postgres key as its alias', function () {
+ [$container, $config] = downstreamEnv(['firefly.eda.database.relay.downstream-provider' => RecordingDownstream::class]);
+ expect(RelayDownstream::resolve($container, $config))->toBeInstanceOf(RecordingDownstream::class);
+
+ [$container, $config] = downstreamEnv(['firefly.eda.postgres.relay.downstream_provider' => RecordingDownstream::class]);
+ expect(RelayDownstream::configuredProvider($config))->toBe(RecordingDownstream::class)
+ ->and(RelayDownstream::resolve($container, $config))->toBeInstanceOf(RecordingDownstream::class);
+});
+
+it('lets a new key that is present but null (an unset env()) fall back to the legacy key', function () {
+ [$container, $config] = downstreamEnv([
+ 'firefly.eda.database.relay.downstream-provider' => null,
+ 'firefly.eda.postgres.relay.downstream_provider' => RecordingDownstream::class,
+ ]);
+
+ expect(RelayDownstream::configuredProvider($config))->toBe(RecordingDownstream::class)
+ ->and(RelayDownstream::resolve($container, $config))->toBeInstanceOf(RecordingDownstream::class);
+
+ [, $config] = downstreamEnv([
+ 'firefly.eda.database.relay.downstream-provider' => null,
+ 'firefly.eda.postgres.relay.downstream_provider' => null,
+ ]);
+ expect(RelayDownstream::configuredProvider($config))->toBeNull();
+});
+
+it('names both keys and the in-process alternative when neither is set', function () {
+ [$container, $config] = downstreamEnv([]);
+
+ expect(fn () => RelayDownstream::resolve($container, $config))
+ ->toThrow(ConfigurationException::class, 'firefly.eda.database.relay.downstream-provider')
+ ->and(fn () => RelayDownstream::resolve($container, $config))
+ ->toThrow(ConfigurationException::class, 'firefly.eda.postgres.relay.downstream_provider');
+});
+
+it('refuses the database outbox writer, however it was named', function () {
+ [$container, $config] = downstreamEnv(['firefly.eda.database.relay.downstream-provider' => 'rabbitmq']);
+ $container->instance(RelayDownstream::BINDING, new DatabaseEventPublisher(new SQLiteConnection(new PDO('sqlite::memory:'), ':memory:'), new SubscriberRegistry));
+
+ expect(fn () => RelayDownstream::resolve($container, $config))->toThrow(ConfigurationException::class, 'outbox writer');
+});
diff --git a/packages/eda-database/tests/Support/DatabaseOutboxCapstoneTestCase.php b/packages/eda-database/tests/Support/DatabaseOutboxCapstoneTestCase.php
new file mode 100644
index 00000000..0c1835bb
--- /dev/null
+++ b/packages/eda-database/tests/Support/DatabaseOutboxCapstoneTestCase.php
@@ -0,0 +1,69 @@
+ */
+ protected function fireflyProviders(): array
+ {
+ return [EdaServiceProvider::class, EdaWiringProvider::class, EdaDatabaseServiceProvider::class];
+ }
+
+ /** @return array */
+ protected function configOverrides(): array
+ {
+ return ['firefly.eda.provider' => 'database'];
+ }
+
+ protected function defineFireflyEnvironment(Application $app): void
+ {
+ $app->instance(EventListenerManifest::class, new EventListenerManifest(
+ (new EventListenerScanner)->scan([
+ 'Firefly\\Eda\\Database\\Tests\\CapstoneFixtures\\' => dirname(__DIR__).'/CapstoneFixtures',
+ ]),
+ ));
+ $app->singleton(ListenerSpy::class);
+
+ // The bare app registers neither firefly/cqrs nor firefly/data; the outbox beans name their collaborators as
+ // constructor parameters, so the empty real objects are supplied (never doubles).
+ $app->instance(HandlerManifest::class, new HandlerManifest([], []));
+ $app->instance(CorrelationContext::class, new CorrelationContext);
+ $app->instance(AggregateTracker::class, new AggregateTracker);
+ $app->instance(ApplicationEventPublisher::class, new class implements ApplicationEventPublisher
+ {
+ public function publish(object $event): void {}
+ });
+ }
+
+ protected function setUp(): void
+ {
+ parent::setUp();
+
+ Schema::create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table));
+ }
+}
diff --git a/packages/eda-database/tests/Support/DatabaseRelayCommandTestCase.php b/packages/eda-database/tests/Support/DatabaseRelayCommandTestCase.php
new file mode 100644
index 00000000..7e4f131a
--- /dev/null
+++ b/packages/eda-database/tests/Support/DatabaseRelayCommandTestCase.php
@@ -0,0 +1,55 @@
+ */
+ protected function fireflyProviders(): array
+ {
+ return [EdaDatabaseBootServiceProvider::class];
+ }
+
+ /** @return array */
+ protected function configOverrides(): array
+ {
+ return ['firefly.eda.provider' => 'database'];
+ }
+
+ protected function setUp(): void
+ {
+ parent::setUp();
+
+ Schema::create(OutboxSchema::TABLE, fn (Blueprint $table) => OutboxSchema::blueprint($table));
+ }
+
+ public function runRelay(): int
+ {
+ /** @var Kernel $kernel */
+ $kernel = $this->app()->make(Kernel::class);
+
+ return $kernel->call('firefly:outbox:relay', ['--max-messages' => 0]);
+ }
+
+ public function relayOutput(): string
+ {
+ /** @var Kernel $kernel */
+ $kernel = $this->app()->make(Kernel::class);
+
+ return $kernel->output();
+ }
+}
diff --git a/packages/eda-database/tests/Support/outbox-racer.php b/packages/eda-database/tests/Support/outbox-racer.php
new file mode 100644
index 00000000..6d0e96c6
--- /dev/null
+++ b/packages/eda-database/tests/Support/outbox-racer.php
@@ -0,0 +1,92 @@
+" for every row it delivered, then "done". A 2 ms pause per delivery keeps both racers inside the
+ * claim at the same time; without it one racer can finish before the other has started.
+ */
+$lane = $argv[1] ?? '';
+$mode = $argv[2] ?? '';
+$strategy = $argv[3] ?? 'auto';
+$go = $argv[4] ?? '';
+$claimant = $argv[5] ?? 'racer';
+$connection = IntegrationDsn::connect($lane, 'racer', isset($argv[6]) ? ['database' => $argv[6]] : []);
+$claim = OutboxClaimStrategies::configured($strategy);
+
+$report = static function (mixed $n): void {
+ fwrite(STDOUT, 'delivered '.(is_scalar($n) ? (string) $n : '?')."\n");
+};
+
+$downstream = new class($report) implements EventPublisher
+{
+ /** @param Closure(mixed): void $report */
+ public function __construct(private readonly Closure $report) {}
+
+ public function subscribe(string $eventTypePattern, callable $handler): void {}
+
+ /**
+ * @param array $payload
+ * @param array $headers
+ */
+ public function publish(string $destination, string $eventType, array $payload, array $headers = []): void
+ {
+ usleep(2000);
+ ($this->report)($payload['n'] ?? null);
+ }
+
+ public function start(): void {}
+
+ public function stop(): void {}
+};
+
+$pending = static fn (): int => $connection->table(OutboxSchema::TABLE)->where('status', OutboxSchema::STATUS_PENDING)->count();
+
+fwrite(STDOUT, "ready\n");
+fflush(STDOUT);
+
+$deadline = microtime(true) + 90;
+while (! is_file($go) && microtime(true) < $deadline) {
+ usleep(1000);
+}
+
+if ($mode === 'relay') {
+ $relay = new OutboxRelay($connection, $downstream, $claim, batchSize: 5, leaseSeconds: 120, claimant: $claimant);
+ while (microtime(true) < $deadline) {
+ if ($relay->relayBatch() === 0 && $pending() === 0) {
+ break;
+ }
+ }
+} else {
+ $consumer = new DatabaseEventConsumer($connection, $claim, leaseSeconds: 120, claimant: $claimant, awaitWork: static function (int $timeoutMs): void {
+ usleep(1000);
+ });
+ while (microtime(true) < $deadline) {
+ $received = $consumer->poll(10);
+ if ($received === null) {
+ if ($pending() === 0) {
+ break;
+ }
+
+ continue;
+ }
+ usleep(2000);
+ $report($received->envelope?->payload['n'] ?? null);
+ $consumer->ack($received);
+ }
+}
+
+fwrite(STDOUT, "done\n");
+$connection->disconnect();
diff --git a/packages/eda-kafka/tests/Integration/KafkaRoundTripTest.php b/packages/eda-kafka/tests/Integration/KafkaRoundTripTest.php
index 77187d2b..c44682bd 100644
--- a/packages/eda-kafka/tests/Integration/KafkaRoundTripTest.php
+++ b/packages/eda-kafka/tests/Integration/KafkaRoundTripTest.php
@@ -13,6 +13,7 @@
use Firefly\Eda\Kafka\KafkaProducerFactory;
use Firefly\Eda\Kafka\RdKafkaConsumerClient;
use Firefly\Eda\Kafka\Tests\Support\KafkaIntegrationTestCase;
+use Firefly\Testing\Integration\IntegrationDsn;
uses(KafkaIntegrationTestCase::class);
@@ -33,7 +34,7 @@
* real broker can prove, because producev() is the one call in this package that no unit test can reach.
*/
it('round-trips publish -> consume -> handler -> commit against a real Kafka, and commit() prevents redelivery', function () {
- $brokers = (string) getenv('FIREFLY_KAFKA_BROKERS');
+ $brokers = (string) IntegrationDsn::value(IntegrationDsn::KAFKA);
$topic = 'firefly.eda.test.'.bin2hex(random_bytes(4));
$groupId = 'firefly-test-'.bin2hex(random_bytes(4));
@@ -72,12 +73,12 @@
expect($again)->toBeNull();
})->skip(
- ! extension_loaded('rdkafka') || getenv('FIREFLY_KAFKA_BROKERS') === false,
+ ! extension_loaded('rdkafka') || IntegrationDsn::missing(IntegrationDsn::KAFKA),
'requires ext-rdkafka + FIREFLY_KAFKA_BROKERS',
)->group('integration');
it('routes an exhausted retry (nack requeue:false) to the ".DLT" dead-letter topic', function () {
- $brokers = (string) getenv('FIREFLY_KAFKA_BROKERS');
+ $brokers = (string) IntegrationDsn::value(IntegrationDsn::KAFKA);
$topic = 'firefly.eda.test.'.bin2hex(random_bytes(4));
$dltTopic = $topic.'.DLT';
@@ -136,6 +137,6 @@
->and($headers['x-dlt-source-topic'] ?? null)->toBe($topic)
->and($headers['x-dlt-source-offset'] ?? null)->toMatch('/^\d+$/');
})->skip(
- ! extension_loaded('rdkafka') || getenv('FIREFLY_KAFKA_BROKERS') === false,
+ ! extension_loaded('rdkafka') || IntegrationDsn::missing(IntegrationDsn::KAFKA),
'requires ext-rdkafka + FIREFLY_KAFKA_BROKERS',
)->group('integration');
diff --git a/packages/eda-postgres/README.md b/packages/eda-postgres/README.md
index c6be8d0c..fbbbf515 100644
--- a/packages/eda-postgres/README.md
+++ b/packages/eda-postgres/README.md
@@ -1,13 +1,11 @@
# firefly/eda-postgres
-The GENUINE same-transaction outbox adapter for `firefly/eda`: `PostgresEventPublisher` writes the
-`firefly_eda_outbox` row on the aggregate's own connection, inside the aggregate's own open transaction,
-via `OutboxPreCommitHook` — an implementation of `firefly/data`'s `PreCommitEventHook` seam — so the row
-commits or rolls back atomically with the aggregate, no dual-write. An in-process `LISTEN`/`NOTIFY` +
-poll `PostgresEventConsumer` claims committed `PENDING` rows (`SELECT ... FOR UPDATE SKIP LOCKED`),
-drives matching `#[EventListener]` handlers, and marks them `PUBLISHED` (or `FAILED` past
-`max_attempts`). An optional `firefly:outbox:relay` command forwards `PENDING` rows to a distinct
-downstream broker. Auto-configured behind `firefly.eda.provider=postgres`, with a `PostgresHealthIndicator`.
+The Postgres flavour of LaraFly's same-transaction outbox: `firefly/eda-database`'s outbox — the row written on
+the aggregate's own connection inside its open transaction, rows claimed by leases so any number of consumers and
+relays can run — plus two Postgres optimizations: `PostgresEventPublisher` raises `pg_notify` inside that
+transaction, and `PostgresEventConsumer` `LISTEN`s and wakes on it. Auto-configured behind
+`firefly.eda.provider=postgres`, with a `PostgresHealthIndicator`; the `firefly.eda.postgres.*` keys are aliases of
+`firefly.eda.database.*`.
```bash
composer require fireflyframework/larafly
@@ -27,13 +25,14 @@ php artisan firefly:eda:consume # terminal in-process delivery
bound `EventPublisher` — under this provider, `PostgresEventPublisher` — which records them on the **shared**
`SubscriberRegistry` singleton. `firefly:eda:consume` resolves that same registry and feeds every claimed row
into it, acking a row only after delivery returns. A handler that throws leaves the row `PENDING` with
-`attempts + 1` (or `FAILED` past `max_attempts`), so a failed delivery is retried rather than lost.
+`attempts + 1` (or `FAILED` past `max-attempts`), so a failed delivery is retried rather than lost.
## The optional relay
`firefly:outbox:relay` forwards committed `PENDING` rows to a **distinct** downstream broker. It is genuinely
optional: an app whose only consumers are `#[EventListener]` handlers never runs it. When you do, name the
-downstream with `firefly.eda.postgres.relay.downstream_provider`, which accepts:
+downstream with `firefly.eda.database.relay.downstream-provider` (alias
+`firefly.eda.postgres.relay.downstream_provider`), which accepts:
* `rabbitmq` / `kafka` — the shipped adapter, built from that package's own config keys
(`firefly.eda.rabbitmq.exchange`, `firefly.eda.kafka.brokers`);
@@ -49,6 +48,7 @@ claimed row as `PENDING` and claim it again forever. An unset or unresolvable va
(exit 1, naming the remedy) raised before any row is claimed — never a silent no-op that marks rows
`PUBLISHED` without forwarding them.
-See [EDA Brokers](../../docs/modules/eda-brokers.md) for the full outbox reference.
+See [EDA Brokers](../../docs/modules/eda-brokers.md) and [EDA Database Outbox](../../docs/modules/eda-database.md)
+for the full outbox reference.
Apache-2.0 © Firefly Software Solutions Inc.
diff --git a/packages/eda-postgres/composer.json b/packages/eda-postgres/composer.json
index 7604529f..ffa76fd2 100644
--- a/packages/eda-postgres/composer.json
+++ b/packages/eda-postgres/composer.json
@@ -1,6 +1,6 @@
{
"name": "firefly/eda-postgres",
- "description": "LaraFly eda Postgres adapter: a PostgresEventPublisher backed by a GENUINE same-transaction firefly_eda_outbox table (the outbox row commits atomically with the aggregate), an in-process LISTEN/NOTIFY + poll consumer that claims committed PENDING rows (SELECT ... FOR UPDATE SKIP LOCKED), drives #[EventListener] handlers, and marks them PUBLISHED, plus an OPTIONAL firefly:outbox:relay command that forwards PENDING rows to a distinct downstream broker. Auto-configured behind firefly.eda.provider=postgres.",
+ "description": "LaraFly eda Postgres adapter: firefly/eda-database's GENUINE same-transaction firefly_eda_outbox (leased claims, safe with concurrent consumers and relays) plus Postgres's two optimizations — pg_notify on write, inside the aggregate's transaction, and a LISTEN wake-up in the in-process consumer. Auto-configured behind firefly.eda.provider=postgres; the firefly.eda.postgres.* keys are aliases of firefly.eda.database.*.",
"type": "library",
"license": "Apache-2.0",
"homepage": "https://github.com/fireflyframework/fireflyframework-php",
@@ -32,6 +32,7 @@
"firefly/data": "*@dev",
"firefly/domain": "*@dev",
"firefly/eda": "*@dev",
+ "firefly/eda-database": "*@dev",
"firefly/kernel": "*@dev",
"illuminate/console": "^13.0",
"illuminate/contracts": "^13.0",
diff --git a/packages/eda-postgres/src/Console/OutboxRelayCommand.php b/packages/eda-postgres/src/Console/OutboxRelayCommand.php
index 88301549..c6ce73d9 100644
--- a/packages/eda-postgres/src/Console/OutboxRelayCommand.php
+++ b/packages/eda-postgres/src/Console/OutboxRelayCommand.php
@@ -4,105 +4,7 @@
namespace Firefly\Eda\Postgres\Console;
-use Firefly\Config\Config;
-use Firefly\Eda\Postgres\Outbox\OutboxRelay;
-use Firefly\Eda\Postgres\Outbox\RelayDownstream;
-use Firefly\Eda\Tracing\BrokerTracing;
-use Firefly\Kernel\Exception\Framework\ConfigurationException;
-use Illuminate\Console\Command;
-use Illuminate\Contracts\Container\Container;
-use Illuminate\Database\Connection;
-use Illuminate\Database\ConnectionResolverInterface;
+use Firefly\Eda\Database\Console\OutboxRelayCommand as DatabaseOutboxRelayCommand;
-/**
- * OPTIONAL: forwards committed firefly_eda_outbox PENDING rows to a DISTINCT downstream broker selected by
- * firefly.eda.postgres.relay.downstream_provider. Terminal in-process delivery is NOT this command's job —
- * firefly:eda:consume owns that — so an app that only needs #[EventListener] handlers never runs this at all.
- *
- * WHAT CHANGED AND WHY. This command used to treat downstream_provider as a mere on/off flag and then resolve
- * EventPublisher::class from the container. Under firefly.eda.provider=postgres — the only provider that has an
- * outbox table to relay — that binding IS the outbox writer, so the guard below rejected the one thing the command
- * could ever resolve, and the relay could not be made to work under any configuration. Selection now goes through
- * RelayDownstream (see its docblock for the full resolution order and why the sibling adapter packages are
- * referenced as class-strings), which returns a REAL downstream publisher or throws a ConfigurationException
- * naming exactly what to change.
- *
- * Both failure modes are now loud. An UNSET key is no longer an exit-0 "no-op": running a relay with nothing to
- * relay to is a misconfiguration, and exiting 0 while forwarding nothing is precisely the silent behaviour that hid
- * the defect — so it exits FAILURE with a message pointing at firefly:eda:consume for the in-process case. A
- * MISCONFIGURED key (unknown name, package not installed, unconstructible class, wrong type, or the outbox writer
- * itself) is reported before a single row is claimed, so a failed relay can never mark rows PUBLISHED.
- */
-final class OutboxRelayCommand extends Command
-{
- /** @var string */
- protected $signature = 'firefly:outbox:relay {--max-messages= : stop after N published} {--time-limit= : stop after N seconds} {--sleep=1 : seconds between empty batches} {--batch-size=50}';
-
- /** @var string */
- protected $description = 'OPTIONAL: forward committed firefly_eda_outbox rows to the downstream broker named by firefly.eda.postgres.relay.downstream_provider (claim -> publish -> mark PUBLISHED/FAILED).';
-
- public function handle(ConnectionResolverInterface $connections, Container $container, Config $config): int
- {
- try {
- $downstream = RelayDownstream::resolve($container, $config);
- } catch (ConfigurationException $e) {
- // Reported as a clean console error rather than an escaping exception: the operator needs the remedy,
- // not a stack trace through the container.
- $this->error($e->getMessage());
-
- return self::FAILURE;
- }
-
- $name = $config->has('firefly.eda.postgres.connection') ? $config->string('firefly.eda.postgres.connection') : null;
- $conn = $connections->connection($name);
- $useSkipLocked = $conn instanceof Connection && $conn->getDriverName() === 'pgsql'; // narrow ONLY to read the driver (B2)
-
- $relay = new OutboxRelay(
- $conn,
- $downstream,
- (int) $this->option('batch-size'),
- $config->int('firefly.eda.postgres.max_attempts', 3),
- $useSkipLocked,
- // The UNGATED seam: the relay uses it to CONTINUE the traceparent the claimed row already carries, so
- // the downstream producer span is that trace's child rather than a new root. Writing to a third party's
- // wire is the downstream publisher's job, and RelayDownstream has already gated THAT on
- // firefly.eda.tracing.brokers.enabled.
- BrokerTracing::bound($container),
- );
-
- $maxMessages = $this->option('max-messages');
- $timeLimit = $this->option('time-limit');
- $deadline = is_numeric($timeLimit) ? time() + (int) $timeLimit : null;
- $sleep = max(0, (int) $this->option('sleep'));
- $published = 0;
-
- if (function_exists('pcntl_async_signals')) {
- pcntl_async_signals(true);
- $stop = false;
- pcntl_signal(SIGINT, function () use (&$stop): void {
- $stop = true;
- });
- pcntl_signal(SIGTERM, function () use (&$stop): void {
- $stop = true;
- });
- }
-
- while (true) {
- $n = $relay->relayBatch();
- $published += $n;
-
- if (($maxMessages !== null && is_numeric($maxMessages) && $published >= (int) $maxMessages)
- || ($deadline !== null && time() >= $deadline)
- || (isset($stop) && $stop)) {
- break;
- }
- if ($n === 0) {
- sleep($sleep);
- }
- }
-
- $this->info("firefly:outbox:relay — published {$published} row(s).");
-
- return self::SUCCESS;
- }
-}
+/** @deprecated since W1 — use Firefly\Eda\Database\Console\OutboxRelayCommand (same signature, either provider). */
+final class OutboxRelayCommand extends DatabaseOutboxRelayCommand {}
diff --git a/packages/eda-postgres/src/EdaPostgresBootServiceProvider.php b/packages/eda-postgres/src/EdaPostgresBootServiceProvider.php
index 1d6586ea..a27a7268 100644
--- a/packages/eda-postgres/src/EdaPostgresBootServiceProvider.php
+++ b/packages/eda-postgres/src/EdaPostgresBootServiceProvider.php
@@ -4,10 +4,14 @@
namespace Firefly\Eda\Postgres;
-use Firefly\Eda\Postgres\Console\OutboxRelayCommand;
+use Firefly\Eda\Database\Console\OutboxRelayCommand;
+use Firefly\Eda\Database\EdaDatabaseBootServiceProvider;
use Illuminate\Support\ServiceProvider;
-/** Ships the outbox migration + (Task 5) the OPTIONAL firefly:outbox:relay command via auto-discovery — no frozen edit. */
+/**
+ * Ships the outbox migrations (firefly/eda-database's, the one outbox table) and the OPTIONAL firefly:outbox:relay
+ * command under provider=postgres via auto-discovery.
+ */
final class EdaPostgresBootServiceProvider extends ServiceProvider
{
public function boot(): void
@@ -16,7 +20,7 @@ public function boot(): void
return;
}
- $this->loadMigrationsFrom(__DIR__.'/../database/migrations');
+ $this->loadMigrationsFrom(EdaDatabaseBootServiceProvider::MIGRATIONS);
if ($this->app->runningInConsole()) {
$this->commands([OutboxRelayCommand::class]);
diff --git a/packages/eda-postgres/src/Outbox/OutboxPreCommitHook.php b/packages/eda-postgres/src/Outbox/OutboxPreCommitHook.php
index 90e7394a..25a292b0 100644
--- a/packages/eda-postgres/src/Outbox/OutboxPreCommitHook.php
+++ b/packages/eda-postgres/src/Outbox/OutboxPreCommitHook.php
@@ -4,66 +4,23 @@
namespace Firefly\Eda\Postgres\Outbox;
-use Firefly\Cqrs\Correlation\CorrelationContext;
-use Firefly\Cqrs\Event\EdaCommandEventPublisher;
-use Firefly\Data\Domain\PreCommitEventHook;
-use Firefly\Domain\DomainEvent;
-use Firefly\Eda\Bus\SubscriberRegistry;
+use Firefly\Eda\Database\DatabaseEventPublisher;
+use Firefly\Eda\Database\Outbox\OutboxPreCommitHook as DatabaseOutboxPreCommitHook;
use Firefly\Eda\Postgres\PostgresEventPublisher;
-use Firefly\Eda\Tracing\EdaTracing;
use Illuminate\Database\Connection;
-use Illuminate\Database\ConnectionResolverInterface;
+use Illuminate\Database\ConnectionInterface;
/**
- * Bridges M8's in-tx pre-commit drain to the Postgres outbox. handle($event, $connection) runs synchronously inside
- * the aggregate's tx (DomainEventDispatcher calls it during the pre-commit drain); for a DomainEvent it resolves the
- * AGGREGATE'S OWN connection ($connection, threaded from the dispatcher — I1) and builds a PostgresEventPublisher on it
- * (emitNotify enabled for pgsql), then delegates to an EdaCommandEventPublisher carrying HandlerManifest::destinations()
- * + CorrelationContext (M5), so the domain->envelope mapping (eventType/payload/per-event destination/transaction_id)
- * is IDENTICAL to the after-commit bridge — but the resulting INSERT is same-tx on the aggregate's connection.
- * Non-DomainEvent objects are ignored (the generic after-commit dispatch still handles them). Reflection-free.
- *
- * The SubscriberRegistry it carries is the app's SHARED singleton, threaded straight through to the per-event
- * PostgresEventPublisher. The hook itself never subscribes anything — it is a pure write path — but the publisher's
- * subscribe() is what EventListenerWiringPass drives, and making the registry a REQUIRED constructor dependency of
- * PostgresEventPublisher is exactly what makes the silent "listener subscribed into the void" regression
- * unconstructible (see PostgresEventPublisher's docblock). Passing the shared instance rather than a throwaway
- * `new SubscriberRegistry` keeps a single registry per application, so there is never a second, invisible one to
- * reason about when debugging a missing delivery.
- *
- * THE TRACING IT CARRIES IS NOT OPTIONAL DECORATION. Under firefly.eda.provider=postgres,
- * PostgresOutboxAutoConfiguration::commandEventPublisher() deliberately NoOps the after-commit eda leg so a domain
- * event reaches the outbox EXACTLY ONCE — in-tx, here. That makes this hook the ONLY writer of a DomainEvent's
- * outbox row: the bound EventPublisher bean is reached only by application code calling publish() by hand. So the
- * EdaTracing this class threads into the per-event PostgresEventPublisher is what decides whether the FLAGSHIP
- * rows — the ones that commit with the aggregate — carry a `traceparent` at all. It is the gated one
- * (BrokerTracing::resolve(), honouring firefly.eda.tracing.brokers.enabled), identical to the bean's, so the two
- * write paths are never traced differently. Null keeps the constructor usable from a test that does not care.
+ * firefly/eda-database's pre-commit hook with the Postgres writer: on a pgsql connection the per-event
+ * PostgresEventPublisher also fires the in-transaction pg_notify. Same constructor, same mapping, same tracing and
+ * registry rules — see the parent's docblock.
*/
-final class OutboxPreCommitHook implements PreCommitEventHook
+final class OutboxPreCommitHook extends DatabaseOutboxPreCommitHook
{
- /** @param array $destinations eventClass => destination */
- public function __construct(
- private readonly ConnectionResolverInterface $connections,
- private readonly SubscriberRegistry $registry,
- private readonly string $channel,
- private readonly string $defaultDestination,
- private readonly array $destinations,
- private readonly ?CorrelationContext $correlation = null,
- private readonly ?EdaTracing $tracing = null,
- ) {}
-
- public function handle(object $event, ?string $connection = null): void
+ protected function writerFor(ConnectionInterface $connection): DatabaseEventPublisher
{
- if (! $event instanceof DomainEvent) {
- return;
- }
-
- $conn = $this->connections->connection($connection); // the aggregate's connection — carries the open tx
- $emitNotify = $conn instanceof Connection && $conn->getDriverName() === 'pgsql';
- $publisher = new PostgresEventPublisher($conn, $this->registry, $this->channel, $emitNotify, $this->tracing);
+ $emitNotify = $connection instanceof Connection && $connection->getDriverName() === 'pgsql';
- (new EdaCommandEventPublisher($publisher, $this->defaultDestination, $this->destinations, $this->correlation))
- ->publish($event);
+ return new PostgresEventPublisher($connection, $this->registry, $this->channel, $emitNotify, $this->tracing);
}
}
diff --git a/packages/eda-postgres/src/Outbox/OutboxRelay.php b/packages/eda-postgres/src/Outbox/OutboxRelay.php
index 63e4d19c..0af81e36 100644
--- a/packages/eda-postgres/src/Outbox/OutboxRelay.php
+++ b/packages/eda-postgres/src/Outbox/OutboxRelay.php
@@ -4,130 +4,44 @@
namespace Firefly\Eda\Postgres\Outbox;
-use Firefly\Eda\EventEnvelope;
+use Firefly\Eda\Database\Outbox\Claim\OptimisticLeaseClaim;
+use Firefly\Eda\Database\Outbox\Claim\OutboxClaimStrategies;
+use Firefly\Eda\Database\Outbox\OutboxRelay as DatabaseOutboxRelay;
use Firefly\Eda\EventPublisher;
-use Firefly\Eda\Postgres\PostgresEventPublisher;
use Firefly\Eda\Tracing\EdaTracing;
-use Firefly\Eda\Tracing\NoOpEdaTracing;
+use Illuminate\Database\Connection;
use Illuminate\Database\ConnectionInterface;
-use Illuminate\Support\Collection;
-use LogicException;
-use Throwable;
+use InvalidArgumentException;
/**
- * The OPTIONAL store-and-forward relay: claims committed PENDING outbox rows and publishes them to a DISTINCT
- * downstream broker (a Kafka/RabbitMQ EventPublisher), marking each PUBLISHED or (past maxAttempts) FAILED. It NEVER
- * accepts a PostgresEventPublisher downstream — that would re-INSERT PENDING rows into the same outbox (infinite loop),
- * so the ctor guards with a LogicException (B1). Terminal in-process delivery is NOT this class's job — PostgresEventConsumer
- * owns it. On Postgres the claim uses `FOR UPDATE SKIP LOCKED` (via ->lock('for update skip locked') — the installed
- * Laravel Builder has no skipLocked() helper) inside a short tx so concurrent relay workers never double-claim; on
- * sqlite (unit tests) useSkipLocked=false ⇒ a plain SELECT + per-row UPDATE guard (`WHERE id=? AND status='PENDING'`)
- * gives the same at-least-once dedup single-threaded. The claim/mark is the idempotency: a PUBLISHED row is never
- * re-selected. A single relay worker gives effective FIFO (ORDER BY id). The caller passes useSkipLocked (computed from
- * the concrete Connection's driver) so this class needs only ConnectionInterface (table/raw/transaction).
- *
- * THE RELAY HOP IS A CONTINUATION, NOT A NEW TRACE. A claimed row already carries the `traceparent` the producer
- * span stamped on it when it was INSERTed in the aggregate's transaction. Forwarding those headers straight to
- * $downstream->publish() destroyed it: the downstream adapter publishes through EdaTracing::tracePublish(), which
- * starts its span with no explicit parent — and on a relay worker there is no current span, so it started a fresh
- * ROOT — then merges its own injected headers LAST, overwriting the row's traceparent with an unrelated one. A row
- * published under T1 reached the broker under T2 and the consumer on the far side continued the wrong trace: the
- * same "trace of half a system" the producer spans were added to fix, reintroduced one hop later. So the forward
- * runs inside traceConsume() of an envelope rebuilt from the row — the identical rule SubscriberRegistrySink
- * applies on the in-process path — which makes the row's traceparent the CONSUMER span's remote parent and the
- * downstream producer span that span's child. The seam is the UNGATED bound EdaTracing: traceConsume writes
- * nothing on any wire, so firefly.eda.tracing.brokers.enabled has no business silencing it — that key gates the
- * downstream PUBLISHER, which RelayDownstream builds (see its parameters()).
+ * @deprecated since W1 — use Firefly\Eda\Database\Outbox\OutboxRelay. This keeps the old constructor for one release.
+ * `$useSkipLocked` keeps its intent, mapped onto the lease claims: `true` ("lock the rows I take") becomes
+ * the capability-probed `auto` strategy — SKIP LOCKED wherever the server supports it, never an SQL error
+ * where it does not — and `false` ("take no row locks") becomes the portable optimistic lease, which
+ * also takes none. The old `false` was an unlocked read that two relays could double-forward; the
+ * optimistic lease is not.
*/
-final class OutboxRelay
+final class OutboxRelay extends DatabaseOutboxRelay
{
- private readonly EdaTracing $tracing;
-
public function __construct(
- private readonly ConnectionInterface $connection,
- private readonly EventPublisher $downstream,
- private readonly int $batchSize = 50,
- private readonly int $maxAttempts = 3,
- private readonly bool $useSkipLocked = false,
+ ConnectionInterface $connection,
+ EventPublisher $downstream,
+ int $batchSize = 50,
+ int $maxAttempts = 3,
+ bool $useSkipLocked = false,
?EdaTracing $tracing = null,
) {
- $this->tracing = $tracing ?? new NoOpEdaTracing;
-
- if ($downstream instanceof PostgresEventPublisher) {
- throw new LogicException('OutboxRelay downstream must NOT be a PostgresEventPublisher — it would re-insert PENDING rows into the same outbox (infinite loop). Configure firefly.eda.postgres.relay.downstream_provider=rabbitmq|kafka.');
- }
- }
-
- public function relayBatch(): int
- {
- if ($this->useSkipLocked) {
- /** @var int $relayed */
- $relayed = $this->connection->transaction(fn (): int => $this->processBatch($this->claim()));
-
- return $relayed;
- }
-
- return $this->processBatch($this->claim());
- }
-
- /** @return Collection */
- private function claim(): Collection
- {
- $query = $this->connection->table(OutboxSchema::TABLE)
- ->where('status', OutboxSchema::STATUS_PENDING)
- ->orderBy('id')
- ->limit($this->batchSize);
-
- if ($this->useSkipLocked) {
- $query->lock('for update skip locked'); // pgsql/MySQL raw lock — Laravel has no skipLocked() builder method
- }
-
- /** @var Collection $rows */
- $rows = $query->get();
-
- return $rows;
- }
-
- /** @param Collection $rows */
- private function processBatch(Collection $rows): int
- {
- $relayed = 0;
- foreach ($rows as $row) {
- try {
- // The envelope is rebuilt ONLY so the seam has a traceparent to continue and a message id to name
- // the span with — the downstream still receives the row's own four columns, byte for byte.
- $this->tracing->traceConsume(
- new EventEnvelope(
- OutboxRow::asString($row->event_type),
- OutboxRow::asString($row->destination),
- OutboxRow::asMap($row->payload),
- OutboxRow::asStringMap($row->headers),
- OutboxRow::asString($row->id),
- ),
- fn (EventEnvelope $claimed) => $this->downstream->publish(
- $claimed->destination,
- $claimed->eventType,
- $claimed->payload,
- $claimed->headers,
- ),
- );
-
- $this->connection->table(OutboxSchema::TABLE)
- ->where('id', $row->id)->where('status', OutboxSchema::STATUS_PENDING)
- ->update(['status' => OutboxSchema::STATUS_PUBLISHED, 'processed_at' => $this->connection->raw('CURRENT_TIMESTAMP')]);
- $relayed++;
- } catch (Throwable $e) {
- $attempts = OutboxRow::asInt($row->attempts) + 1;
- $failed = $attempts >= $this->maxAttempts;
- $this->connection->table(OutboxSchema::TABLE)->where('id', $row->id)->update([
- 'attempts' => $attempts,
- 'status' => $failed ? OutboxSchema::STATUS_FAILED : OutboxSchema::STATUS_PENDING,
- 'error_message' => $e->getMessage(),
- 'failed_at' => $failed ? $this->connection->raw('CURRENT_TIMESTAMP') : null,
- ]);
- }
+ if (! $connection instanceof Connection) {
+ throw new InvalidArgumentException('OutboxRelay needs a concrete Illuminate\Database\Connection: the claim reads its driver and server version.');
}
- return $relayed;
+ parent::__construct(
+ $connection,
+ $downstream,
+ OutboxClaimStrategies::configured($useSkipLocked ? OutboxClaimStrategies::AUTO : OptimisticLeaseClaim::NAME),
+ $batchSize,
+ $maxAttempts,
+ $tracing,
+ );
}
}
diff --git a/packages/eda-postgres/src/Outbox/OutboxRow.php b/packages/eda-postgres/src/Outbox/OutboxRow.php
index f312bf5d..fc56535f 100644
--- a/packages/eda-postgres/src/Outbox/OutboxRow.php
+++ b/packages/eda-postgres/src/Outbox/OutboxRow.php
@@ -4,54 +4,10 @@
namespace Firefly\Eda\Postgres\Outbox;
+use Firefly\Eda\Database\Outbox\OutboxRow as DatabaseOutboxRow;
+
/**
- * Narrows the mixed-typed columns of a firefly_eda_outbox query-builder row (stdClass properties carry no static
- * type) to the concrete scalar/array shapes the terminal consumer and the relay need — one place, level-max clean,
- * reflection-free. asMap()/asStringMap() keep JSON_THROW_ON_ERROR so a corrupt payload/headers column fails loud
- * (caught by the relay's per-row try/catch, or surfaced by the consumer poll) rather than being silently dropped.
+ * @deprecated since W1 — the outbox moved to firefly/eda-database; use Firefly\Eda\Database\Outbox\OutboxRow.
+ * Kept for one release so code that names the old class keeps compiling.
*/
-final class OutboxRow
-{
- public static function asString(mixed $value): string
- {
- return is_scalar($value) ? (string) $value : '';
- }
-
- public static function asInt(mixed $value): int
- {
- return is_numeric($value) ? (int) $value : 0;
- }
-
- /** @return array */
- public static function asMap(mixed $json): array
- {
- if (! is_string($json)) {
- return [];
- }
-
- $decoded = json_decode($json, true, 512, JSON_THROW_ON_ERROR);
- if (! is_array($decoded)) {
- return [];
- }
-
- $map = [];
- foreach ($decoded as $key => $value) {
- $map[(string) $key] = $value;
- }
-
- return $map;
- }
-
- /** @return array */
- public static function asStringMap(mixed $json): array
- {
- $map = [];
- foreach (self::asMap($json) as $key => $value) {
- if (is_scalar($value)) {
- $map[$key] = (string) $value;
- }
- }
-
- return $map;
- }
-}
+final class OutboxRow extends DatabaseOutboxRow {}
diff --git a/packages/eda-postgres/src/Outbox/OutboxSchema.php b/packages/eda-postgres/src/Outbox/OutboxSchema.php
index 141c2a15..0140a7df 100644
--- a/packages/eda-postgres/src/Outbox/OutboxSchema.php
+++ b/packages/eda-postgres/src/Outbox/OutboxSchema.php
@@ -4,36 +4,10 @@
namespace Firefly\Eda\Postgres\Outbox;
-use Illuminate\Database\Schema\Blueprint;
+use Firefly\Eda\Database\Outbox\OutboxSchema as DatabaseOutboxSchema;
-/** The ONE definition of the outbox table shape — shared by the migration, the publisher, the relay, and the tests. */
-final class OutboxSchema
-{
- public const string TABLE = 'firefly_eda_outbox';
-
- public const string STATUS_PENDING = 'PENDING';
-
- public const string STATUS_PUBLISHED = 'PUBLISHED';
-
- public const string STATUS_FAILED = 'FAILED';
-
- public static function blueprint(Blueprint $table): void
- {
- $table->bigIncrements('id');
- $table->string('destination');
- $table->string('channel', 63);
- $table->string('event_type');
- $table->json('payload');
- $table->json('headers');
- $table->string('transaction_id')->nullable();
- $table->string('status')->default(self::STATUS_PENDING);
- $table->unsignedInteger('attempts')->default(0);
- $table->text('error_message')->nullable();
- $table->timestamp('created_at')->useCurrent();
- $table->timestamp('processed_at')->nullable();
- $table->timestamp('failed_at')->nullable();
-
- $table->index(['status', 'created_at'], 'firefly_eda_outbox_pending_idx');
- $table->index(['channel', 'status'], 'firefly_eda_outbox_channel_status_idx');
- }
-}
+/**
+ * @deprecated since W1 — the outbox moved to firefly/eda-database; use Firefly\Eda\Database\Outbox\OutboxSchema.
+ * Kept for one release so code that names the old class keeps compiling.
+ */
+final class OutboxSchema extends DatabaseOutboxSchema {}
diff --git a/packages/eda-postgres/src/Outbox/RelayDownstream.php b/packages/eda-postgres/src/Outbox/RelayDownstream.php
index 5b10f265..741ddb6f 100644
--- a/packages/eda-postgres/src/Outbox/RelayDownstream.php
+++ b/packages/eda-postgres/src/Outbox/RelayDownstream.php
@@ -4,287 +4,10 @@
namespace Firefly\Eda\Postgres\Outbox;
-use Firefly\Config\Config;
-use Firefly\Eda\EventPublisher;
-use Firefly\Eda\Postgres\PostgresEventPublisher;
-use Firefly\Eda\Tracing\BrokerTracing;
-use Firefly\Kernel\Exception\Framework\ConfigurationException;
-use Illuminate\Contracts\Container\BindingResolutionException;
-use Illuminate\Contracts\Container\Container;
+use Firefly\Eda\Database\Outbox\RelayDownstream as DatabaseRelayDownstream;
/**
- * Turns firefly.eda.postgres.relay.downstream_provider into an ACTUAL downstream EventPublisher for
- * firefly:outbox:relay — the selection step that was missing entirely.
- *
- * WHAT WAS BROKEN. OutboxRelayCommand read the key purely as an on/off flag and then did
- * `$container->make(EventPublisher::class)`. But firefly:outbox:relay only exists under
- * firefly.eda.provider=postgres, and under that provider PostgresOutboxAutoConfiguration binds EventPublisher to
- * the outbox WRITER. So the command's own guard (and OutboxRelay's constructor guard behind it) rejected the only
- * thing it could ever resolve: relaying rows through the writer would INSERT them straight back into
- * firefly_eda_outbox as PENDING — an infinite loop that grows the table forever. The result was a command with two
- * reachable outcomes and no third: "no downstream configured, no-op, exit 0" or "the resolved publisher is the
- * outbox writer, exit 1". Setting the documented config key could not produce a working relay under any
- * configuration; it merely swapped a silent no-op for a loud one.
- *
- * HOW IT RESOLVES NOW, in order, so an app can always win:
- * 1. an explicit container binding under self::BINDING — the escape hatch for a downstream that needs credentials,
- * TLS options or anything else this package has no business knowing about;
- * 2. the configured value read as a container id — an app that has BOUND it has already constructed it, so every
- * constructor argument is already the app's own choice;
- * 3. a SHIPPED adapter, named either by its ALIAS ('rabbitmq' | 'kafka') or by that adapter's own class-string —
- * two spellings of one downstream, both built with the adapter's OWN config keys;
- * 4. any other existing class-string, autowired — a publisher this package ships no adapter entry for and knows
- * no configuration keys of.
- *
- * WHY THE ADAPTER CLASSES ARE REFERENCED AS STRINGS. deptrac's layer graph forbids an EdaPostgres -> EdaRabbitmq /
- * EdaKafka edge (see deptrac.yaml: every broker is a sibling leaf package, depended on by NONE), and rightly so —
- * firefly/eda-postgres must not drag php-amqplib or ext-rdkafka into a Postgres-only install. A class-string plus a
- * class_exists() guard keeps the compile-time graph clean while still producing a real, fully configured publisher
- * when the operator has installed the sibling package. When they have not, self::ADAPTERS names the exact composer
- * package to require.
- *
- * WHY THE ADAPTER'S OWN CONFIG KEYS ARE PASSED EXPLICITLY. Each broker package gates ITS publisher bean behind
- * #[ConditionalOnProperty(firefly.eda.provider=)], which is false here by construction (the provider
- * is postgres). Resolving the class through the container therefore autowires it with the constructor DEFAULTS, so
- * an app that had carefully set firefly.eda.rabbitmq.exchange would have silently relayed to `firefly.events`
- * instead — a fresh silent misconfiguration in place of the one being fixed. self::parameters() passes those values
- * as named constructor overrides so the relay honours exactly the settings the adapter itself would have read.
- *
- * WHY VALIDATION IS RUNTIME AND NOT BOOT-TIME. Throwing from EdaPostgresBootServiceProvider::boot() was considered
- * and rejected: a downstream bound in ANOTHER provider's boot() may not exist yet when ours runs, so a boot-time
- * check would fail applications that are correctly configured, and it would fail every web request and every
- * unrelated artisan command over a relay-only concern. The relay command is the one place the value is load-bearing,
- * so that is where it is validated — immediately, before a single row is claimed, and never at the first publish.
+ * @deprecated since W1 — use Firefly\Eda\Database\Outbox\RelayDownstream. firefly.eda.postgres.relay.downstream_provider
+ * is still honoured as the legacy alias of firefly.eda.database.relay.downstream-provider.
*/
-final class RelayDownstream
-{
- /** The config key that selects the relay's downstream. */
- public const string PROVIDER_KEY = 'firefly.eda.postgres.relay.downstream_provider';
-
- /** The container id an app may bind to supply a fully constructed downstream publisher itself. */
- public const string BINDING = 'firefly.eda.relay.downstream';
-
- /**
- * The shipped broker adapters, by the alias an operator writes in `downstream_provider`.
- *
- * @var array
- */
- private const ADAPTERS = [
- 'rabbitmq' => ['class' => 'Firefly\\Eda\\Rabbitmq\\RabbitMqEventPublisher', 'package' => 'firefly/eda-rabbitmq'],
- 'kafka' => ['class' => 'Firefly\\Eda\\Kafka\\KafkaEventPublisher', 'package' => 'firefly/eda-kafka'],
- ];
-
- /** The configured downstream name, or null when the operator has not opted into the relay at all. */
- public static function configuredProvider(Config $config): ?string
- {
- if (! $config->has(self::PROVIDER_KEY)) {
- return null;
- }
-
- $provider = trim($config->string(self::PROVIDER_KEY, ''));
-
- return $provider === '' ? null : $provider;
- }
-
- /**
- * Resolve the downstream publisher, or throw a ConfigurationException that names the exact thing to change.
- * Never returns the outbox writer.
- */
- public static function resolve(Container $container, Config $config): EventPublisher
- {
- $provider = self::configuredProvider($config);
-
- // Route 1 is checked BEFORE the unset-key failure, not after: an app that has bound a ready-made downstream
- // has already named it, and demanding a redundant downstream_provider alongside the binding would reject a
- // configuration this class's own error message (and the README) tell operators to use.
- if ($container->bound(self::BINDING)) {
- return self::guard($provider ?? self::BINDING, self::make($container, self::BINDING, [], $provider ?? self::BINDING));
- }
-
- if ($provider === null) {
- throw new ConfigurationException(sprintf(
- '%s is not set, so firefly:outbox:relay has no downstream broker to forward to. Set it to one of [%s], '
- .'to the class-string of an EventPublisher, or bind your own under the container id "%s". If you did not '
- .'mean to front a second broker at all, do not run the relay: firefly.eda.provider=postgres already '
- .'delivers committed outbox rows in-process via `php artisan firefly:eda:consume`.',
- self::PROVIDER_KEY,
- implode('|', array_keys(self::ADAPTERS)),
- self::BINDING,
- ));
- }
-
- return self::guard($provider, self::instantiate($container, $config, $provider));
- }
-
- /**
- * Walk the remaining resolution routes in order (route 1, the explicit binding, is handled by resolve() so it
- * can win even with the key unset). Anything the container cannot build is reported with the
- * BindingResolutionException attached, because "which constructor argument could not be satisfied" is the only
- * detail that makes an autowiring failure actionable.
- */
- private static function instantiate(Container $container, Config $config, string $provider): mixed
- {
- // An app that BOUND this id built the publisher itself, so every argument — tracing included — is already
- // its own choice, and passing constructor overrides here would defeat the binding rather than honour it
- // (Illuminate skips a shared instance the moment make() is handed parameters).
- if ($container->bound($provider)) {
- return self::make($container, $provider, [], $provider);
- }
-
- // ONE route for a shipped adapter, whichever way it was spelled. The class-string used to fall straight
- // through to the bare class_exists() branch below, and the publisher was then built by container autowiring
- // with its constructor DEFAULTS: the exchange an operator had set in firefly.eda.rabbitmq.exchange silently
- // became `firefly.events`, and — the reason this was a defect rather than an untidiness —
- // firefly.eda.tracing.brokers.enabled FAILED OPEN, because Illuminate returns a constructor parameter's
- // default only when the type is UNBOUND, and EdaTracing is bound in every app that has firefly/eda
- // installed. Writing RabbitMqEventPublisher::class instead of 'rabbitmq' therefore put traceparents on a
- // third party's wire with the compliance gate switched off: two spellings of one downstream that behaved
- // differently, and the gate was bypassed by the more explicit of the two.
- $alias = isset(self::ADAPTERS[$provider]) ? $provider : self::shippedAdapterAlias($provider);
-
- if ($alias !== null) {
- $adapter = self::ADAPTERS[$alias];
-
- if (! class_exists($adapter['class'])) {
- throw new ConfigurationException(sprintf(
- '%s="%s" but %s is not installed — %s could not be found. Run `composer require %s`, or bind your own '
- .'downstream EventPublisher under the container id "%s".',
- self::PROVIDER_KEY,
- $provider,
- $adapter['package'],
- $adapter['class'],
- $adapter['package'],
- self::BINDING,
- ));
- }
-
- return self::make($container, $adapter['class'], self::parameters($container, $config, $alias), $provider);
- }
-
- // A publisher class this package ships no adapter entry for: it knows none of that class's config keys, so
- // the container autowires it and the application owns every argument, tracing included.
- if (class_exists($provider)) {
- return self::make($container, $provider, [], $provider);
- }
-
- throw new ConfigurationException(sprintf(
- '%s="%s" names neither a shipped adapter [%s], nor a bound container id, nor an existing class. Fix the '
- .'value, or bind your own downstream EventPublisher under the container id "%s".',
- self::PROVIDER_KEY,
- $provider,
- implode('|', array_keys(self::ADAPTERS)),
- self::BINDING,
- ));
- }
-
- /**
- * The alias a SHIPPED adapter's class-string belongs to, or null when the value names a class this package has
- * no adapter entry for. `downstream_provider` accepts both spellings, and the two have to resolve to the same
- * publisher with the same configuration — otherwise the class-string is a quiet way around every key
- * self::parameters() carries across the adapter's own #[ConditionalOnProperty] gate.
- */
- private static function shippedAdapterAlias(string $provider): ?string
- {
- foreach (self::ADAPTERS as $alias => $adapter) {
- if ($adapter['class'] === $provider) {
- return $alias;
- }
- }
-
- return null;
- }
-
- /**
- * The adapter-specific constructor overrides that carry the sibling package's OWN configuration across the
- * provider gate it is standing behind. Named by constructor parameter — which PHP 8 named arguments already
- * make part of those classes' public API — so a rename there surfaces here as a loud BindingResolutionException
- * rather than a quietly-defaulted broker address.
- *
- * `tracing` IS ONE OF THOSE OVERRIDES, and for the same reason as `exchange`. The adapter's own bean method
- * reads firefly.eda.tracing.brokers.enabled before handing the publisher an EdaTracing; that bean is gated off
- * here by construction, so the publisher is built by container autowiring instead — and Illuminate returns a
- * constructor parameter's DEFAULT only when the class is UNBOUND. EdaTracing is bound in every app that has
- * firefly/eda installed, so `?EdaTracing $tracing = null` was being satisfied with the real tracing whatever
- * the key said, and `firefly:outbox:relay` wrote traceparents onto the downstream broker with the key false —
- * the one deployment the key exists for. BrokerTracing::forContainer() makes the same decision the bean
- * methods make, so the gate holds on every construction path.
- *
- * @return array
- */
- private static function parameters(Container $container, Config $config, string $provider): array
- {
- if ($provider === 'rabbitmq') {
- return [
- 'exchange' => $config->string('firefly.eda.rabbitmq.exchange', 'firefly.events'),
- 'tracing' => BrokerTracing::forContainer($container, $config),
- ];
- }
-
- if ($provider === 'kafka') {
- // The broker list lives one level down, on KafkaProducerFactory, so it is built here and injected as the
- // publisher's `factory` argument — parameter overrides do not reach nested dependencies.
- return [
- 'factory' => self::make(
- $container,
- 'Firefly\\Eda\\Kafka\\KafkaProducerFactory',
- ['brokers' => $config->string('firefly.eda.kafka.brokers', '127.0.0.1:9092')],
- $provider,
- ),
- 'tracing' => BrokerTracing::forContainer($container, $config),
- ];
- }
-
- return [];
- }
-
- /**
- * @param array $parameters
- */
- private static function make(Container $container, string $id, array $parameters, string $provider): mixed
- {
- try {
- return $container->make($id, $parameters);
- } catch (BindingResolutionException $e) {
- throw new ConfigurationException(sprintf(
- '%s="%s" resolved to "%s", which could not be constructed: %s. Bind a ready-made downstream '
- .'EventPublisher under the container id "%s" instead.',
- self::PROVIDER_KEY,
- $provider,
- $id,
- $e->getMessage(),
- self::BINDING,
- ), previous: $e);
- }
- }
-
- /**
- * The last line of defence, and the reason OutboxRelay's own constructor guard is not enough on its own: a clear
- * message at CONFIGURATION time beats a LogicException from deep inside the relay. Refusing the outbox writer
- * here is what keeps `downstream_provider` from ever pointing the relay back at the table it is draining.
- */
- private static function guard(string $provider, mixed $downstream): EventPublisher
- {
- if ($downstream instanceof PostgresEventPublisher) {
- throw new ConfigurationException(sprintf(
- '%s="%s" resolved to the Postgres outbox writer itself. The relay would re-INSERT every claimed row '
- .'into firefly_eda_outbox as PENDING and claim it again forever. Point it at a DISTINCT broker, or bind '
- .'one under the container id "%s".',
- self::PROVIDER_KEY,
- $provider,
- self::BINDING,
- ));
- }
-
- if (! $downstream instanceof EventPublisher) {
- throw new ConfigurationException(sprintf(
- '%s="%s" resolved to %s, which does not implement %s.',
- self::PROVIDER_KEY,
- $provider,
- get_debug_type($downstream),
- EventPublisher::class,
- ));
- }
-
- return $downstream;
- }
-}
+final class RelayDownstream extends DatabaseRelayDownstream {}
diff --git a/packages/eda-postgres/src/PostgresEventConsumer.php b/packages/eda-postgres/src/PostgresEventConsumer.php
index aff03374..2b2a9122 100644
--- a/packages/eda-postgres/src/PostgresEventConsumer.php
+++ b/packages/eda-postgres/src/PostgresEventConsumer.php
@@ -4,49 +4,49 @@
namespace Firefly\Eda\Postgres;
-use Firefly\Eda\Consumer\EventConsumer;
-use Firefly\Eda\Consumer\ReceivedEnvelope;
-use Firefly\Eda\EventEnvelope;
-use Firefly\Eda\Postgres\Outbox\OutboxRow;
-use Firefly\Eda\Postgres\Outbox\OutboxSchema;
+use Closure;
+use Firefly\Eda\Database\DatabaseEventConsumer;
+use Firefly\Eda\Database\Outbox\Claim\OutboxClaimStrategy;
use Firefly\Eda\Postgres\Pgsql\NotificationWaiter;
use Illuminate\Database\Connection;
/**
- * The TERMINAL in-process delivery path (B1): drives #[EventListener] handlers from committed outbox rows with NO
- * EventPublisher call, so there is no re-INSERT / self-reference. subscribe() issues LISTEN ; poll() waits on
- * a NOTIFY (NotificationWaiter, pgsql only — the writer's in-tx pg_notify fires on commit) up to $timeoutMs, then
- * CLAIMS the oldest status='PENDING' row (FOR UPDATE SKIP LOCKED on pgsql; plain ORDER BY id on sqlite) and returns it
- * as an envelope (a poll fallback also catches rows missed while offline). ConsumerLoop delivers it to the
- * SubscriberRegistry and then ack()s -> the row is marked PUBLISHED/processed_at (guarded WHERE status='PENDING', so
- * idempotent); on a handler throw ConsumerLoop nack()s -> attempts+1, or FAILED/failed_at past max_attempts. This is a
- * DURABLE PENDING->PUBLISHED status window (M3), NOT an in-memory high-water mark: a restart resumes at the oldest
- * still-PENDING row and never replays PUBLISHED rows. SINGLE consumer worker gives exactly-once in-process delivery;
- * multi-worker degrades to at-least-once (idempotent handlers documented). Requires the concrete Illuminate\Database\
- * Connection because getPdo()/getDriverName() are not on ConnectionInterface (B2).
+ * firefly/eda-database's DatabaseEventConsumer — the same leased claim, the same guarded ack/nack, the same
+ * "no EventPublisher call" terminal path — with Postgres's low-latency wake-up: subscribe() issues `LISTEN `
+ * and the wait between empty claims is a NOTIFY wait (NotificationWaiter) instead of a sleep, so the in-transaction
+ * pg_notify PostgresEventPublisher fires on commit wakes the consumer the moment the row is visible. On any other
+ * driver (sqlite in tests) there is no LISTEN and no wait. The claim itself is chosen by capability like everywhere
+ * else — on pgsql that is SKIP LOCKED inside a short claim transaction, which is what makes several workers safe.
*/
-final class PostgresEventConsumer implements EventConsumer
+final class PostgresEventConsumer extends DatabaseEventConsumer
{
- /** @var \Closure(int): void */
- private readonly \Closure $awaitNotification;
-
/**
- * @param (\Closure(int): void)|null $awaitNotification Test seam: overrides the pgsql NOTIFY wait. Defaults to the
- * real Postgres LISTEN/NOTIFY wait (a pure low-latency optimization — poll() always falls through to the
- * durable PENDING-row claim, and guards this against ANY failure, so a deprecation-to-exception handler on the
- * deprecated pgsqlGetNotify() path can never crash the poll loop).
+ * @param (Closure(int): void)|null $awaitNotification overrides the pgsql NOTIFY wait (a test seam)
+ * @param (Closure(): \DateTimeImmutable)|null $clock
*/
public function __construct(
- private readonly Connection $connection,
+ Connection $connection,
private readonly string $channel = 'firefly_eda_events',
- private readonly int $maxAttempts = 3,
- ?\Closure $awaitNotification = null,
+ int $maxAttempts = 3,
+ ?Closure $awaitNotification = null,
+ ?OutboxClaimStrategy $claim = null,
+ float $leaseSeconds = 300.0,
+ ?string $claimant = null,
+ ?Closure $clock = null,
) {
- $this->awaitNotification = $awaitNotification ?? function (int $timeoutMs): void {
- if ($this->connection->getDriverName() === 'pgsql') {
- NotificationWaiter::wait($this->connection->getPdo(), $timeoutMs);
- }
- };
+ parent::__construct(
+ $connection,
+ $claim,
+ $maxAttempts,
+ $leaseSeconds,
+ claimant: $claimant,
+ awaitWork: $awaitNotification ?? static function (int $timeoutMs) use ($connection): void {
+ if ($connection->getDriverName() === 'pgsql') {
+ NotificationWaiter::wait($connection->getPdo(), $timeoutMs);
+ }
+ },
+ clock: $clock,
+ );
}
public function subscribe(array $destinations): void
@@ -55,65 +55,4 @@ public function subscribe(array $destinations): void
$this->connection->getPdo()->exec('LISTEN '.$this->channel);
}
}
-
- public function start(): void {}
-
- public function poll(int $timeoutMs): ?ReceivedEnvelope
- {
- try {
- ($this->awaitNotification)($timeoutMs);
- } catch (\Throwable) {
- // NOTIFY is a low-latency optimization; on ANY failure (incl. a deprecation-to-exception handler on the
- // deprecated pgsqlGetNotify path) fall through to the poll-fallback claim below, which delivers regardless.
- }
-
- $pgsql = $this->connection->getDriverName() === 'pgsql';
- $query = $this->connection->table(OutboxSchema::TABLE)
- ->where('status', OutboxSchema::STATUS_PENDING)
- ->orderBy('id');
- if ($pgsql) {
- $query->lock('for update skip locked'); // claim (M4) — Laravel has no skipLocked() helper
- }
- $row = $query->first();
-
- if ($row === null) {
- return null;
- }
-
- return new ReceivedEnvelope(new EventEnvelope(
- OutboxRow::asString($row->event_type),
- OutboxRow::asString($row->destination),
- OutboxRow::asMap($row->payload),
- OutboxRow::asStringMap($row->headers),
- ), OutboxRow::asInt($row->id));
- }
-
- public function ack(ReceivedEnvelope $received): void
- {
- $this->connection->table(OutboxSchema::TABLE)
- ->where('id', $received->deliveryTag)
- ->where('status', OutboxSchema::STATUS_PENDING)
- ->update([
- 'status' => OutboxSchema::STATUS_PUBLISHED,
- 'processed_at' => $this->connection->raw('CURRENT_TIMESTAMP'),
- ]);
- }
-
- public function nack(ReceivedEnvelope $received, bool $requeue = true): void
- {
- $row = $this->connection->table(OutboxSchema::TABLE)->where('id', $received->deliveryTag)->first();
- if ($row === null) {
- return;
- }
- $attempts = OutboxRow::asInt($row->attempts) + 1;
- $failed = $attempts >= $this->maxAttempts;
- $this->connection->table(OutboxSchema::TABLE)->where('id', $received->deliveryTag)->update([
- 'attempts' => $attempts,
- 'status' => $failed ? OutboxSchema::STATUS_FAILED : OutboxSchema::STATUS_PENDING,
- 'error_message' => 'in-process handler failed',
- 'failed_at' => $failed ? $this->connection->raw('CURRENT_TIMESTAMP') : null,
- ]);
- }
-
- public function stop(): void {}
}
diff --git a/packages/eda-postgres/src/PostgresEventPublisher.php b/packages/eda-postgres/src/PostgresEventPublisher.php
index c8f04ebb..d1422724 100644
--- a/packages/eda-postgres/src/PostgresEventPublisher.php
+++ b/packages/eda-postgres/src/PostgresEventPublisher.php
@@ -5,125 +5,37 @@
namespace Firefly\Eda\Postgres;
use Firefly\Eda\Bus\SubscriberRegistry;
-use Firefly\Eda\EventPublisher;
-use Firefly\Eda\Postgres\Outbox\OutboxSchema;
+use Firefly\Eda\Database\DatabaseEventPublisher;
use Firefly\Eda\Tracing\EdaTracing;
-use Firefly\Eda\Tracing\NoOpEdaTracing;
use Illuminate\Database\ConnectionInterface;
-use stdClass;
/**
- * The Postgres EventPublisher: publish() INSERTs ONE firefly_eda_outbox row ON THE CURRENT CONNECTION. When called
- * inside a DB transaction (which M8's #[Transactional] and the OutboxPreCommitHook guarantee for domain events), the
- * INSERT enlists in that transaction and commits ATOMICALLY with the aggregate — the genuine same-tx guarantee. When
- * emitNotify is true (pgsql), it also fires `SELECT pg_notify('', id)` — inside the SAME tx, so Postgres
- * queues the NOTIFY and delivers it exactly when the aggregate COMMITS, waking the in-process consumer's LISTEN with
- * low latency (M2). The in-process consumer (PostgresEventConsumer) claims PENDING rows and drives #[EventListener]
- * handlers; the OPTIONAL firefly:outbox:relay only fronts a distinct downstream broker. start()/stop() are no-ops
- * (the relay/consumer own their own connections). It uses only ConnectionInterface methods (table()/statement()/raw())
- * — no getPdo()/getDriverName() — so ConnectionInterface is the correct ctor type; the driver-gated emitNotify flag is
- * computed by the caller from the concrete Connection. Reflection-free.
- *
- * WHY THE REGISTRY IS A REQUIRED CONSTRUCTOR DEPENDENCY (silent-data-loss regression, CapstoneOutboxDeliveryTest):
- * subscribe() USED TO BE AN EMPTY NO-OP while this very docblock claimed it "feeds the shared SubscriberRegistry".
- * That mattered because EventListenerWiringPass — the boot pass that turns the app's compiled #[EventListener]
- * manifest into live subscriptions — resolves the bound EventPublisher and calls subscribe() ON IT. Under
- * firefly.eda.provider=postgres the bound publisher is THIS class, so every listener the app compiled was handed to
- * a method that discarded it, and the SubscriberRegistry the terminal consumer delivers into stayed permanently
- * empty. The failure was silent rather than loud because SubscriberRegistry::deliver() on an empty registry returns
- * normally: ConsumerLoop read that as a successful delivery and called ack(), flipping the outbox row
- * PENDING -> PUBLISHED, never to be claimed again. Domain events were written same-transaction exactly as promised,
- * drained exactly as promised, and thrown away — no exception, no log line, nothing. Making the registry a REQUIRED
- * parameter (mirroring RabbitMqEventPublisher and KafkaEventPublisher, which have always taken one) means an
- * instance of this class that cannot deliver its subscriptions is no longer constructible at all.
- *
- * THE PRODUCER SPAN WRAPS THE INSERT. publish() runs its whole body — the row and the pg_notify — inside
- * EdaTracing::tracePublish(), so the `traceparent` that span produced is written onto the row itself, and the row
- * carrying it is the one that COMMITS WITH THE AGGREGATE. This is the one adapter where the producer span and the
- * durable record are genuinely atomic: a broker adapter's span ends when the socket accepts the message and the
- * two can still disagree, while here either both the aggregate and its traced outbox row exist or neither does.
- *
- * That holds for BOTH writers of an outbox row, which is the only way the sentence above is worth anything: the
- * bound EventPublisher bean (application code calling publish() by hand) and OutboxPreCommitHook, which builds its
- * own instance per event and is — under firefly.eda.provider=postgres, where commandEventPublisher() NoOps the
- * after-commit leg — the ONLY path a DomainEvent takes. Both are handed the same gated EdaTracing by
- * PostgresOutboxAutoConfiguration.
- *
- * Downstream of the row the trace continues two different ways. PostgresEventConsumer, the terminal in-process
- * path, hands the claimed row's headers to SubscriberRegistrySink, whose traceConsume() continues whatever
- * traceparent it finds. The OPTIONAL relay hands them to a downstream EventPublisher instead, so OutboxRelay
- * wraps that forward in traceConsume() itself — otherwise the downstream adapter's own producer span would start
- * a new root and overwrite the row's traceparent. Either way the trace runs unbroken from the request that
- * changed the aggregate to the listener (or broker) that reacted to it, across a commit boundary.
+ * The Postgres outbox writer: firefly/eda-database's DatabaseEventPublisher — the same INSERT on the caller's
+ * connection, the same traced closure, the same required SubscriberRegistry (see its docblock for why) — plus one
+ * statement. When emitNotify is true (the caller computed it from the concrete connection: pgsql), afterInsert() fires
+ * `SELECT pg_notify('', id)` INSIDE THE SAME TRANSACTION: Postgres queues a NOTIFY raised in a transaction and
+ * delivers it exactly when that transaction COMMITS, so a consumer's LISTEN wakes the moment the row becomes visible
+ * and never for a row that rolled back. The NOTIFY is an optimization only — the consumer's claim is the source of
+ * truth and also finds rows written while nobody listened.
*/
-final class PostgresEventPublisher implements EventPublisher
+final class PostgresEventPublisher extends DatabaseEventPublisher
{
- private readonly EdaTracing $tracing;
-
public function __construct(
- private readonly ConnectionInterface $connection,
- private readonly SubscriberRegistry $registry,
- private readonly string $channel = 'firefly_eda_events',
+ ConnectionInterface $connection,
+ SubscriberRegistry $registry,
+ string $channel = 'firefly_eda_events',
private readonly bool $emitNotify = false,
?EdaTracing $tracing = null,
) {
- $this->tracing = $tracing ?? new NoOpEdaTracing;
+ parent::__construct($connection, $registry, $channel, $tracing);
}
- /**
- * Records the pattern on the SHARED SubscriberRegistry — the same singleton PostgresOutboxAutoConfiguration
- * hands to this publisher and that firefly:eda:consume resolves as the ConsumerLoop's sink. The publisher itself
- * never invokes handlers: delivery is the terminal consumer's job, driven off committed rows.
- */
- public function subscribe(string $eventTypePattern, callable $handler): void
+ protected function afterInsert(int $id): void
{
- $this->registry->subscribe($eventTypePattern, $handler);
+ if ($this->emitNotify) {
+ // In-tx pg_notify: Postgres queues it and delivers on COMMIT, so the consumer's LISTEN wakes exactly when
+ // the PENDING row becomes visible. (sqlite and every other driver: emitNotify=false.)
+ $this->connection->statement('SELECT pg_notify(?, ?)', [$this->channel, (string) $id]);
+ }
}
-
- /**
- * Routed through EdaTracing::tracePublish() — the seam InMemoryEventBus and QueueEventBus have always
- * used, and the reason it takes the headers rather than returning them: propagation means WRITING
- * something on the envelope, so the implementation hands the send closure the headers to carry and the
- * adapter builds the row from those. Before this, the three broker adapters built their envelopes
- * themselves and no producer span existed on the wire at all; their CONSUME side was already traced
- * through SubscriberRegistrySink, so a `traceparent` a foreign producer put in the headers was continued
- * while one of our own was never written. A trace that stops at the broker is a trace of half a system.
- *
- * The ENTIRE body is inside the closure, INSERT and pg_notify together, so the span covers the durable
- * write rather than sitting beside it. transaction_id is still derived from the closure's
- * `x-correlation-id` and never from a traceparent: the relay's idempotency key must not change just
- * because a request happened to be traced.
- *
- * With tracing off the bound EdaTracing is the NoOp, which calls $send with the caller's headers
- * unchanged, so this path is byte-for-byte what it was.
- *
- * @param array $payload
- * @param array $headers
- */
- public function publish(string $destination, string $eventType, array $payload, array $headers = []): void
- {
- $this->tracing->tracePublish($destination, $eventType, $headers, function (array $headers) use ($destination, $eventType, $payload): void {
- $id = $this->connection->table(OutboxSchema::TABLE)->insertGetId([
- 'destination' => $destination,
- 'channel' => $this->channel,
- 'event_type' => $eventType,
- 'payload' => json_encode($payload, JSON_THROW_ON_ERROR),
- 'headers' => json_encode($headers === [] ? new stdClass : $headers, JSON_THROW_ON_ERROR),
- 'transaction_id' => $headers['x-correlation-id'] ?? null,
- 'status' => OutboxSchema::STATUS_PENDING,
- 'attempts' => 0,
- 'created_at' => $this->connection->raw('CURRENT_TIMESTAMP'),
- ]);
-
- if ($this->emitNotify) {
- // In-tx pg_notify: Postgres queues it and delivers on COMMIT, so the consumer's LISTEN wakes exactly
- // when the PENDING row becomes visible. statement() is on ConnectionInterface. (sqlite: emitNotify=false.)
- $this->connection->statement('SELECT pg_notify(?, ?)', [$this->channel, (string) $id]);
- }
- });
- }
-
- public function start(): void {}
-
- public function stop(): void {}
}
diff --git a/packages/eda-postgres/src/PostgresOutboxAutoConfiguration.php b/packages/eda-postgres/src/PostgresOutboxAutoConfiguration.php
index 5ca4208a..a10f3682 100644
--- a/packages/eda-postgres/src/PostgresOutboxAutoConfiguration.php
+++ b/packages/eda-postgres/src/PostgresOutboxAutoConfiguration.php
@@ -18,6 +18,8 @@
use Firefly\Data\Domain\DomainEventDispatcher;
use Firefly\Eda\Bus\SubscriberRegistry;
use Firefly\Eda\Consumer\EventConsumer;
+use Firefly\Eda\Database\DatabaseOutboxSettings;
+use Firefly\Eda\Database\Outbox\Claim\OutboxClaimStrategies;
use Firefly\Eda\EventPublisher;
use Firefly\Eda\Postgres\Outbox\OutboxPreCommitHook;
use Firefly\Eda\Tracing\BrokerTracing;
@@ -26,8 +28,15 @@
use Illuminate\Database\ConnectionResolverInterface;
/**
- * Wires the GENUINE same-transaction Postgres outbox when firefly.eda.provider=postgres. #[Order(900)] (below
- * DataAutoConfiguration's / CqrsAutoConfiguration's 1000) makes this register FIRST, so:
+ * Wires the GENUINE same-transaction Postgres outbox when firefly.eda.provider=postgres.
+ *
+ * FROM W1 ON this is firefly/eda-database's outbox — the same leased claim, the same settings object
+ * (DatabaseOutboxSettings: the firefly.eda.database.* keys, with the firefly.eda.postgres.* keys as aliases) — plus
+ * the two Postgres optimizations: PostgresEventPublisher's in-transaction pg_notify and PostgresEventConsumer's LISTEN
+ * wake-up. DatabaseOutboxAutoConfiguration wires the same building blocks for provider=database; the two never both
+ * apply.
+ *
+ * #[Order(900)] (below DataAutoConfiguration's / CqrsAutoConfiguration's 1000) makes this register FIRST, so:
* - eventPublisher() binds EventPublisher -> PostgresEventPublisher (the outbox writer),
* - outboxPreCommitHook() binds the in-tx hook (T3's PreCommitEventHook),
* - domainEventDispatcher() binds OUR DomainEventDispatcher carrying that hook, winning
@@ -63,15 +72,15 @@ public function subscriberRegistry(): SubscriberRegistry
#[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'postgres')]
public function eventPublisher(Config $config, ConnectionResolverInterface $connections, SubscriberRegistry $registry, ?EdaTracing $tracing = null): EventPublisher
{
- $name = $config->has('firefly.eda.postgres.connection') ? $config->string('firefly.eda.postgres.connection') : null;
- $conn = $connections->connection($name);
+ $settings = DatabaseOutboxSettings::from($config);
+ $conn = $connections->connection($settings->connection);
// Narrow to the concrete Connection ONLY to read the driver (getDriverName() is not on ConnectionInterface — B2).
$emitNotify = $conn instanceof Connection && $conn->getDriverName() === 'pgsql';
return new PostgresEventPublisher(
$conn,
$registry,
- $config->string('firefly.eda.postgres.channel', 'firefly_eda_events'),
+ $settings->channel,
$emitNotify,
BrokerTracing::resolve($config, $tracing),
);
@@ -96,7 +105,7 @@ public function outboxPreCommitHook(ConnectionResolverInterface $connections, Co
return new OutboxPreCommitHook(
$connections,
$registry,
- $config->string('firefly.eda.postgres.channel', 'firefly_eda_events'),
+ DatabaseOutboxSettings::from($config)->channel,
$config->string('firefly.cqrs.default_destination', 'cqrs.events'),
$manifest->destinations(),
$correlation,
@@ -122,23 +131,25 @@ public function commandEventPublisher(): CommandEventPublisher
/**
* The TERMINAL in-process consumer firefly:eda:consume drives in postgres mode: it claims committed PENDING
- * outbox rows (LISTEN/NOTIFY wake + FOR UPDATE SKIP LOCKED), delivers each to the #[EventListener] handlers via
+ * outbox rows (LISTEN/NOTIFY wake + a leased claim chosen by server capability — SKIP LOCKED on pgsql), delivers each to the #[EventListener] handlers via
* the SubscriberRegistry, and marks them PUBLISHED — with NO EventPublisher call, so there is no re-INSERT (B1).
- * Narrows the resolved connection to the concrete type because PostgresEventConsumer needs getPdo()/getDriverName()
- * (not on ConnectionInterface — B2).
+ * DatabaseOutboxSettings::connectionFrom() narrows the resolved connection to the concrete type because
+ * PostgresEventConsumer needs getPdo()/getDriverName() (not on ConnectionInterface — B2), and refuses anything else
+ * with a ConfigurationException instead of an assert that production builds compile away.
*/
#[Bean]
#[ConditionalOnProperty(name: 'firefly.eda.provider', havingValue: 'postgres')]
public function eventConsumer(Config $config, ConnectionResolverInterface $connections): EventConsumer
{
- $name = $config->has('firefly.eda.postgres.connection') ? $config->string('firefly.eda.postgres.connection') : null;
- $conn = $connections->connection($name);
- assert($conn instanceof Connection); // PostgresEventConsumer needs getPdo()/getDriverName() (B2)
+ $settings = DatabaseOutboxSettings::from($config);
return new PostgresEventConsumer(
- $conn,
- $config->string('firefly.eda.postgres.channel', 'firefly_eda_events'),
- $config->int('firefly.eda.postgres.max_attempts', 3),
+ $settings->connectionFrom($connections),
+ $settings->channel,
+ $settings->maxAttempts,
+ null,
+ OutboxClaimStrategies::configured($settings->claimStrategy),
+ $settings->leaseSeconds,
);
}
}
diff --git a/packages/eda-postgres/tests/Integration/PostgresOutboxRoundTripTest.php b/packages/eda-postgres/tests/Integration/PostgresOutboxRoundTripTest.php
index d6c594db..302afc42 100644
--- a/packages/eda-postgres/tests/Integration/PostgresOutboxRoundTripTest.php
+++ b/packages/eda-postgres/tests/Integration/PostgresOutboxRoundTripTest.php
@@ -12,6 +12,7 @@
use Firefly\Eda\Postgres\PostgresEventConsumer;
use Firefly\Eda\Postgres\PostgresEventPublisher;
use Firefly\Eda\Postgres\Tests\Fixtures\SpyDownstreamPublisher;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;
@@ -32,24 +33,11 @@
* Every payload/headers assertion decodes the jsonb/json column and compares the MAP (never a raw-string equality,
* which pgsql key/whitespace normalisation would false-fail).
*
- * @return array a Laravel pgsql connection config parsed from FIREFLY_PG_DSN
+ * @return array
*/
function outbox_pg_config(): array
{
- $parts = [];
- foreach (explode(';', (string) getenv('FIREFLY_PG_DSN')) as $kv) {
- [$key, $value] = array_pad(explode('=', $kv, 2), 2, '');
- $parts[$key] = $value;
- }
-
- return [
- 'driver' => 'pgsql',
- 'host' => $parts['host'] ?? '127.0.0.1',
- 'port' => $parts['port'] ?? '5432',
- 'database' => $parts['dbname'] ?? 'postgres',
- 'username' => $parts['user'] ?? 'postgres',
- 'password' => $parts['password'] ?? '',
- ];
+ return IntegrationDsn::connectionConfig('pgsql');
}
/** Decode a json/jsonb column value to a comparable map (jsonb-aware — never a raw-string compare). */
@@ -171,6 +159,6 @@ function outbox_decode(mixed $column): mixed
$writer->disconnect();
$consumerConn->disconnect();
})->skip(
- ! extension_loaded('pdo_pgsql') || getenv('FIREFLY_PG_DSN') === false,
- 'Set FIREFLY_PG_DSN (and pdo_pgsql) to run the @group integration Postgres outbox round-trip.',
+ IntegrationDsn::skipReason('pgsql') !== null,
+ (string) IntegrationDsn::skipReason('pgsql'),
)->group('integration');
diff --git a/packages/eda-postgres/tests/PostgresReadsDatabaseSettingsTest.php b/packages/eda-postgres/tests/PostgresReadsDatabaseSettingsTest.php
new file mode 100644
index 00000000..0d4cfffb
--- /dev/null
+++ b/packages/eda-postgres/tests/PostgresReadsDatabaseSettingsTest.php
@@ -0,0 +1,35 @@
+app();
+ $app->make(EventPublisher::class)->publish('users', 'user.created', ['id' => 1]);
+
+ /** @var EventConsumer $consumer */
+ $consumer = $app->make(EventConsumer::class);
+ (new ConsumerLoop)->run($consumer, function (): void {
+ throw new RuntimeException('handler boom');
+ }, new ConsumerOptions(maxMessages: 1, pollTimeoutMs: 10));
+
+ $row = DB::table(OutboxSchema::TABLE)->first();
+
+ // max-attempts=1: the first failure is final; the row carries the database channel.
+ expect($consumer)->toBeInstanceOf(PostgresEventConsumer::class)
+ ->and($consumer)->toBeInstanceOf(DatabaseEventConsumer::class)
+ ->and($row?->status)->toBe(OutboxSchema::STATUS_FAILED)
+ ->and($row?->channel)->toBe('orders_channel');
+});
diff --git a/packages/eda-postgres/tests/Support/PostgresSettingsCapstoneTestCase.php b/packages/eda-postgres/tests/Support/PostgresSettingsCapstoneTestCase.php
new file mode 100644
index 00000000..0310fceb
--- /dev/null
+++ b/packages/eda-postgres/tests/Support/PostgresSettingsCapstoneTestCase.php
@@ -0,0 +1,22 @@
+ */
+ protected function configOverrides(): array
+ {
+ return [
+ ...parent::configOverrides(),
+ 'firefly.eda.database.max-attempts' => 1,
+ 'firefly.eda.database.channel' => 'orders_channel',
+ ];
+ }
+}
diff --git a/packages/eda-rabbitmq/tests/Integration/RabbitMqRoundTripTest.php b/packages/eda-rabbitmq/tests/Integration/RabbitMqRoundTripTest.php
index b7e0fdec..98c433b4 100644
--- a/packages/eda-rabbitmq/tests/Integration/RabbitMqRoundTripTest.php
+++ b/packages/eda-rabbitmq/tests/Integration/RabbitMqRoundTripTest.php
@@ -12,33 +12,16 @@
use Firefly\Eda\Rabbitmq\RabbitMqEventConsumer;
use Firefly\Eda\Rabbitmq\RabbitMqEventPublisher;
use Firefly\Eda\Rabbitmq\Tests\Support\RabbitMqIntegrationTestCase;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Config\Repository;
use PhpAmqpLib\Message\AMQPMessage;
uses(RabbitMqIntegrationTestCase::class);
-/** @return Config a Config over firefly.eda.rabbitmq.* parsed from FIREFLY_RABBITMQ_DSN */
+/** @return Config a Config over firefly.eda.rabbitmq.* from FIREFLY_RABBITMQ_DSN (key=value or amqp:// URL form) */
function rabbitTestConfig(): Config
{
- $parts = [];
- foreach (explode(';', (string) getenv('FIREFLY_RABBITMQ_DSN')) as $kv) {
- [$key, $value] = array_pad(explode('=', $kv, 2), 2, '');
- $parts[$key] = $value;
- }
-
- return new Config(new Repository([
- 'firefly' => [
- 'eda' => [
- 'rabbitmq' => [
- 'host' => $parts['host'] ?? '127.0.0.1',
- 'port' => (int) ($parts['port'] ?? 5672),
- 'user' => $parts['user'] ?? 'guest',
- 'password' => $parts['password'] ?? 'guest',
- 'vhost' => '/',
- ],
- ],
- ],
- ]));
+ return new Config(new Repository(['firefly' => ['eda' => ['rabbitmq' => IntegrationDsn::rabbitmq()]]]));
}
it('round-trips publish -> consume -> handler -> ack against a real RabbitMQ', function () {
@@ -71,7 +54,7 @@ function rabbitTestConfig(): Config
);
expect($processed)->toBe(1)->and($received)->toBe([42]);
-})->skip(getenv('FIREFLY_RABBITMQ_DSN') === false, 'Set FIREFLY_RABBITMQ_DSN to run the @group integration RabbitMQ round-trip.')->group('integration');
+})->skip(IntegrationDsn::missing(IntegrationDsn::RABBITMQ), 'Set FIREFLY_RABBITMQ_DSN to run the @group integration RabbitMQ round-trip.')->group('integration');
it('routes an exhausted retry to the DLX queue', function () {
$config = rabbitTestConfig();
@@ -141,4 +124,4 @@ function rabbitTestConfig(): Config
}
expect($decoded['payload']['id'] ?? null)->toBe(99);
-})->skip(getenv('FIREFLY_RABBITMQ_DSN') === false, 'Set FIREFLY_RABBITMQ_DSN to run the @group integration RabbitMQ round-trip.')->group('integration');
+})->skip(IntegrationDsn::missing(IntegrationDsn::RABBITMQ), 'Set FIREFLY_RABBITMQ_DSN to run the @group integration RabbitMQ round-trip.')->group('integration');
diff --git a/packages/eda/README.md b/packages/eda/README.md
index 6a4df783..affceb1f 100644
--- a/packages/eda/README.md
+++ b/packages/eda/README.md
@@ -3,7 +3,8 @@
LaraFly's event-driven-architecture transport: an `EventPublisher` broker-bus port with an in-memory
default adapter and a Laravel-queue async adapter, an `EventEnvelope` + JSON `Serializer` seam, a
`#[EventListener]` attribute compiled into an `EventListenerManifest` and wired onto the bus at boot,
-and a linear-backoff retry / dead-letter helper. Real brokers (Kafka/RabbitMQ/Postgres) and the durable
-transactional outbox ship as their own packages at SP-4.
+and a linear-backoff retry / dead-letter helper. Durable dead letters (`firefly.eda.dead-letter.store=database`)
+and `firefly:eda:dead-letters:replay` ship here; the same-transaction outbox is `firefly/eda-database` (and
+`firefly/eda-postgres`), the brokers `firefly/eda-rabbitmq` and `firefly/eda-kafka`.
Apache-2.0 © Firefly Software Solutions Inc.
diff --git a/packages/eda/cache/firefly-eda-components.php b/packages/eda/cache/firefly-eda-components.php
index c79da697..3e6b6322 100644
--- a/packages/eda/cache/firefly-eda-components.php
+++ b/packages/eda/cache/firefly-eda-components.php
@@ -51,6 +51,7 @@
'order' => 0,
'lazy' => false,
'dependencies' => [
+ 0 => 'Illuminate\\Container\\Container',
],
],
3 => [
diff --git a/packages/eda/composer.json b/packages/eda/composer.json
index 2a49fc82..f936c1bc 100644
--- a/packages/eda/composer.json
+++ b/packages/eda/composer.json
@@ -30,6 +30,7 @@
"illuminate/console": "^13.0",
"illuminate/container": "^13.0",
"illuminate/contracts": "^13.0",
+ "illuminate/database": "^13.0",
"illuminate/queue": "^13.0",
"illuminate/support": "^13.0"
},
diff --git a/packages/eda/database/migrations/2026_10_12_000200_create_firefly_eda_dead_letters_table.php b/packages/eda/database/migrations/2026_10_12_000200_create_firefly_eda_dead_letters_table.php
new file mode 100644
index 00000000..f1ce7fb7
--- /dev/null
+++ b/packages/eda/database/migrations/2026_10_12_000200_create_firefly_eda_dead_letters_table.php
@@ -0,0 +1,21 @@
+ DeadLetterSchema::blueprint($table));
+ }
+
+ public function down(): void
+ {
+ Schema::dropIfExists(DeadLetterSchema::TABLE);
+ }
+};
diff --git a/packages/eda/src/Console/ReplayDeadLettersCommand.php b/packages/eda/src/Console/ReplayDeadLettersCommand.php
new file mode 100644
index 00000000..77425b71
--- /dev/null
+++ b/packages/eda/src/Console/ReplayDeadLettersCommand.php
@@ -0,0 +1,92 @@
+` is added, so a listener can tell a replay from a first delivery. Each entry that
+ * republished is removed unless --keep; one that fails is reported and kept, and the command exits FAILURE. Replay is
+ * at-least-once: a crash between the publish and the removal replays that entry again next time.
+ */
+final class ReplayDeadLettersCommand extends Command
+{
+ /** @var string */
+ protected $signature = 'firefly:eda:dead-letters:replay {--id=* : replay only these entries} {--destination= : only entries dead-lettered from this destination} {--event-type= : only entries of this event type} {--limit=100 : at most this many entries} {--dry-run : list what would be replayed and change nothing} {--keep : keep each entry after it is republished}';
+
+ /** @var string */
+ protected $description = 'Republish dead-lettered eda envelopes through the bound EventPublisher, removing each one that republished.';
+
+ public function handle(DeadLetterStore $store, EventPublisher $publisher): int
+ {
+ if (! $store instanceof ReplayableDeadLetterStore) {
+ $this->error(sprintf('The bound DeadLetterStore (%s) cannot select or remove entries. Bind a ReplayableDeadLetterStore, or drop the binding and set firefly.eda.dead-letter.store=database.', get_debug_type($store)));
+
+ return self::FAILURE;
+ }
+
+ $entries = $store->find($this->query());
+
+ $replayed = 0;
+ $failed = 0;
+ foreach ($entries as $entry) {
+ $envelope = $entry->envelope;
+ $destination = $envelope->headers['x-original-topic'] ?? $envelope->destination;
+ $headers = array_diff_key($envelope->headers, ['x-original-topic' => true, 'x-exception' => true]);
+ $headers['x-replayed-from'] = (string) $entry->id;
+
+ if ($this->option('dry-run') === true) {
+ $this->line(sprintf('would replay %s %s -> %s', $entry->id, $envelope->eventType, $destination));
+
+ continue;
+ }
+
+ try {
+ $publisher->publish($destination, $envelope->eventType, $envelope->payload, $headers);
+ if ($this->option('keep') !== true && $entry->id !== null) {
+ $store->remove($entry->id);
+ }
+ $replayed++;
+ $this->line(sprintf('replayed %s %s -> %s', $entry->id, $envelope->eventType, $destination));
+ } catch (Throwable $e) {
+ $failed++;
+ $this->error(sprintf('could not replay %s %s: %s', $entry->id, $envelope->eventType, $e->getMessage()));
+ }
+ }
+
+ $this->info(sprintf('firefly:eda:dead-letters:replay — %d selected, %d replayed, %d failed.', count($entries), $replayed, $failed));
+
+ return $failed === 0 ? self::SUCCESS : self::FAILURE;
+ }
+
+ private function query(): DeadLetterQuery
+ {
+ $ids = [];
+ foreach ((array) $this->option('id') as $id) {
+ if (is_string($id) && $id !== '') {
+ $ids[] = $id;
+ }
+ }
+
+ $destination = $this->option('destination');
+ $eventType = $this->option('event-type');
+ $limit = $this->option('limit');
+
+ return new DeadLetterQuery(
+ $ids,
+ is_string($destination) && $destination !== '' ? $destination : null,
+ is_string($eventType) && $eventType !== '' ? $eventType : null,
+ is_numeric($limit) ? max(1, (int) $limit) : 100,
+ );
+ }
+}
diff --git a/packages/eda/src/DeadLetter/DatabaseDeadLetterStore.php b/packages/eda/src/DeadLetter/DatabaseDeadLetterStore.php
new file mode 100644
index 00000000..7ca28dc9
--- /dev/null
+++ b/packages/eda/src/DeadLetter/DatabaseDeadLetterStore.php
@@ -0,0 +1,146 @@
+clock === null ? new DateTimeImmutable : ($this->clock)();
+
+ $this->db()->table(DeadLetterSchema::TABLE)->insert([
+ 'id' => (string) Str::uuid7(),
+ 'event_id' => $envelope->eventId,
+ 'event_type' => $envelope->eventType,
+ 'destination' => $envelope->destination,
+ 'payload' => json_encode($envelope->payload === [] ? new stdClass : $envelope->payload, JSON_THROW_ON_ERROR),
+ 'headers' => json_encode($envelope->headers === [] ? new stdClass : $envelope->headers, JSON_THROW_ON_ERROR),
+ 'exception_class' => $cause::class,
+ 'exception_message' => $cause->getMessage(),
+ 'dead_lettered_at' => $now->setTimezone(new DateTimeZone('UTC'))->format('Y-m-d H:i:s.v'),
+ ]);
+ }
+
+ /**
+ * @return list
+ */
+ public function all(): array
+ {
+ return $this->find(new DeadLetterQuery(limit: null));
+ }
+
+ public function find(DeadLetterQuery $query): array
+ {
+ $select = $this->db()->table(DeadLetterSchema::TABLE)->orderBy('dead_lettered_at')->orderBy('id');
+
+ if ($query->ids !== []) {
+ $select->whereIn('id', $query->ids);
+ }
+ if ($query->destination !== null) {
+ $select->where('destination', $query->destination);
+ }
+ if ($query->eventType !== null) {
+ $select->where('event_type', $query->eventType);
+ }
+ if ($query->limit !== null) {
+ $select->limit(max(1, $query->limit));
+ }
+
+ $entries = [];
+ foreach ($select->get() as $row) {
+ $entries[] = self::entry($row);
+ }
+
+ return $entries;
+ }
+
+ public function remove(string $id): bool
+ {
+ return $this->db()->table(DeadLetterSchema::TABLE)->where('id', $id)->delete() === 1;
+ }
+
+ private function db(): ConnectionInterface
+ {
+ return $this->connections->connection($this->connection);
+ }
+
+ private static function entry(object $row): DeadLetterEntry
+ {
+ $columns = get_object_vars($row);
+ $eventId = self::text($columns['event_id'] ?? null);
+ $at = self::text($columns['dead_lettered_at'] ?? null);
+
+ $headers = [];
+ foreach (self::map($columns['headers'] ?? null) as $key => $value) {
+ if (is_scalar($value)) {
+ $headers[$key] = (string) $value;
+ }
+ }
+
+ return new DeadLetterEntry(
+ new EventEnvelope(
+ self::text($columns['event_type'] ?? null),
+ self::text($columns['destination'] ?? null),
+ self::map($columns['payload'] ?? null),
+ $headers,
+ $eventId === '' ? null : $eventId,
+ ),
+ self::text($columns['exception_class'] ?? null),
+ self::text($columns['exception_message'] ?? null),
+ self::text($columns['id'] ?? null),
+ $at === '' ? null : new DateTimeImmutable($at, new DateTimeZone('UTC')),
+ );
+ }
+
+ private static function text(mixed $value): string
+ {
+ return is_scalar($value) ? (string) $value : '';
+ }
+
+ /**
+ * @return array
+ */
+ private static function map(mixed $json): array
+ {
+ $decoded = is_string($json) ? json_decode($json, true, 512, JSON_THROW_ON_ERROR) : null;
+
+ $map = [];
+ foreach (is_array($decoded) ? $decoded : [] as $key => $value) {
+ $map[(string) $key] = $value;
+ }
+
+ return $map;
+ }
+}
diff --git a/packages/eda/src/DeadLetter/DeadLetterEntry.php b/packages/eda/src/DeadLetter/DeadLetterEntry.php
index a131421d..1bad0c36 100644
--- a/packages/eda/src/DeadLetter/DeadLetterEntry.php
+++ b/packages/eda/src/DeadLetter/DeadLetterEntry.php
@@ -4,11 +4,13 @@
namespace Firefly\Eda\DeadLetter;
+use DateTimeImmutable;
use Firefly\Eda\EventEnvelope;
/**
* One dead-lettered envelope plus the exception class + message that caused it. Immutable record; the envelope
- * already carries the x-original-topic / x-exception headers added by RetryingEventHandler.
+ * already carries the x-original-topic / x-exception headers added by RetryingEventHandler. A replayable store fills
+ * `id` (the handle replay and removal use) and `deadLetteredAt`; both stay null for an entry built by hand.
*/
final readonly class DeadLetterEntry
{
@@ -16,5 +18,7 @@ public function __construct(
public EventEnvelope $envelope,
public string $exceptionClass,
public string $exceptionMessage,
+ public ?string $id = null,
+ public ?DateTimeImmutable $deadLetteredAt = null,
) {}
}
diff --git a/packages/eda/src/DeadLetter/DeadLetterQuery.php b/packages/eda/src/DeadLetter/DeadLetterQuery.php
new file mode 100644
index 00000000..7c049516
--- /dev/null
+++ b/packages/eda/src/DeadLetter/DeadLetterQuery.php
@@ -0,0 +1,27 @@
+ $ids empty = any
+ * @param int|null $limit null = all
+ */
+ public function __construct(
+ public array $ids = [],
+ public ?string $destination = null,
+ public ?string $eventType = null,
+ public ?int $limit = 100,
+ ) {}
+
+ public function matches(DeadLetterEntry $entry): bool
+ {
+ return ($this->ids === [] || in_array($entry->id, $this->ids, true))
+ && ($this->destination === null || $entry->envelope->destination === $this->destination)
+ && ($this->eventType === null || $entry->envelope->eventType === $this->eventType);
+ }
+}
diff --git a/packages/eda/src/DeadLetter/DeadLetterSchema.php b/packages/eda/src/DeadLetter/DeadLetterSchema.php
new file mode 100644
index 00000000..836d4d97
--- /dev/null
+++ b/packages/eda/src/DeadLetter/DeadLetterSchema.php
@@ -0,0 +1,29 @@
+string('id', 36)->primary();
+ $table->string('event_id', 64)->nullable();
+ $table->string('event_type');
+ $table->string('destination');
+ $table->json('payload');
+ $table->json('headers');
+ $table->string('exception_class');
+ $table->text('exception_message');
+ $table->dateTime('dead_lettered_at', 3);
+
+ $table->index(['destination', 'event_type'], 'firefly_eda_dead_letters_route_idx');
+ $table->index(['dead_lettered_at'], 'firefly_eda_dead_letters_at_idx');
+ }
+}
diff --git a/packages/eda/src/DeadLetter/DeadLetterStore.php b/packages/eda/src/DeadLetter/DeadLetterStore.php
index 4029fc27..9d9c4ede 100644
--- a/packages/eda/src/DeadLetter/DeadLetterStore.php
+++ b/packages/eda/src/DeadLetter/DeadLetterStore.php
@@ -10,7 +10,8 @@
/**
* The eda dead-letter sink (pyfly eda/dlq.py EdaDeadLetterStore parity). A handler that exhausts its retries has
* its envelope stored here (enriched with x-original-topic / x-exception headers), so no event is silently lost.
- * InMemoryDeadLetterStore is the default; a durable adapter can land at SP-4.
+ * InMemoryDeadLetterStore is the default; DatabaseDeadLetterStore (firefly.eda.dead-letter.store=database) keeps
+ * them durably. Both implement ReplayableDeadLetterStore, which firefly:eda:dead-letters:replay needs.
*/
interface DeadLetterStore
{
diff --git a/packages/eda/src/DeadLetter/InMemoryDeadLetterStore.php b/packages/eda/src/DeadLetter/InMemoryDeadLetterStore.php
index d43541df..b2e8cedb 100644
--- a/packages/eda/src/DeadLetter/InMemoryDeadLetterStore.php
+++ b/packages/eda/src/DeadLetter/InMemoryDeadLetterStore.php
@@ -4,20 +4,23 @@
namespace Firefly\Eda\DeadLetter;
+use DateTimeImmutable;
use Firefly\Eda\EventEnvelope;
+use Illuminate\Support\Str;
use Throwable;
/**
- * The default DeadLetterStore: an in-process append-only list. Zero external services (skeleton default).
+ * The default DeadLetterStore: an in-process append-only list. Zero external services (skeleton default) — and lost
+ * when the process ends; firefly.eda.dead-letter.store=database keeps dead letters durably.
*/
-final class InMemoryDeadLetterStore implements DeadLetterStore
+final class InMemoryDeadLetterStore implements ReplayableDeadLetterStore
{
/** @var list */
private array $entries = [];
public function store(EventEnvelope $envelope, Throwable $cause): void
{
- $this->entries[] = new DeadLetterEntry($envelope, $cause::class, $cause->getMessage());
+ $this->entries[] = new DeadLetterEntry($envelope, $cause::class, $cause->getMessage(), (string) Str::uuid(), new DateTimeImmutable);
}
/**
@@ -27,4 +30,24 @@ public function all(): array
{
return $this->entries;
}
+
+ public function find(DeadLetterQuery $query): array
+ {
+ $found = array_values(array_filter($this->entries, $query->matches(...)));
+
+ return $query->limit === null ? $found : array_slice($found, 0, max(1, $query->limit));
+ }
+
+ public function remove(string $id): bool
+ {
+ foreach ($this->entries as $index => $entry) {
+ if ($entry->id === $id) {
+ array_splice($this->entries, $index, 1);
+
+ return true;
+ }
+ }
+
+ return false;
+ }
}
diff --git a/packages/eda/src/DeadLetter/ReplayableDeadLetterStore.php b/packages/eda/src/DeadLetter/ReplayableDeadLetterStore.php
new file mode 100644
index 00000000..3fc5388e
--- /dev/null
+++ b/packages/eda/src/DeadLetter/ReplayableDeadLetterStore.php
@@ -0,0 +1,20 @@
+ oldest first
+ */
+ public function find(DeadLetterQuery $query): array;
+
+ public function remove(string $id): bool;
+}
diff --git a/packages/eda/src/EdaAutoConfiguration.php b/packages/eda/src/EdaAutoConfiguration.php
index f491a8d9..b21e5769 100644
--- a/packages/eda/src/EdaAutoConfiguration.php
+++ b/packages/eda/src/EdaAutoConfiguration.php
@@ -12,18 +12,22 @@
use Firefly\Eda\Bus\InMemoryEventBus;
use Firefly\Eda\Bus\QueueEventBus;
use Firefly\Eda\Bus\SubscriberRegistry;
+use Firefly\Eda\DeadLetter\DatabaseDeadLetterStore;
use Firefly\Eda\DeadLetter\DeadLetterStore;
use Firefly\Eda\DeadLetter\InMemoryDeadLetterStore;
use Firefly\Eda\Exception\SerializationException;
use Firefly\Eda\Tracing\EdaTracing;
use Firefly\Eda\Tracing\NoOpEdaTracing;
+use Firefly\Kernel\Exception\Framework\ConfigurationException;
use Illuminate\Container\Container;
+use Illuminate\Contracts\Container\BindingResolutionException;
+use Illuminate\Database\ConnectionResolverInterface;
/**
* Always-on eda transport wiring. Selects the EventPublisher adapter by firefly.eda.provider (`memory`, the
* default zero-services bus | `queue`, the async illuminate/queue adapter), the Serializer by
* firefly.eda.serialization_format (`json` only in M9; avro/protobuf are rejected seams), and the DeadLetterStore
- * default. #[Order(1000)] places it after user definitions; each #[ConditionalOnMissingBean] lets an app bind its
+ * (`memory` by default, `database` by firefly.eda.dead-letter.store). #[Order(1000)] places it after user definitions; each #[ConditionalOnMissingBean] lets an app bind its
* own and win. Both adapters are singletons (the #[Bean] default), so the wiring pass and the queue worker share
* ONE SubscriberRegistry. Mirrors SchedulingAutoConfiguration / DataAutoConfiguration.
*/
@@ -59,11 +63,45 @@ public function serializer(Config $config): Serializer
};
}
+ /**
+ * firefly.eda.dead-letter.store: `memory` (the default — in-process, lost on exit) or `database` (durable,
+ * replayable, on firefly.eda.dead-letter.connection). It takes the container, not Config and the connection
+ * resolver, because neither is safe to inject here: Config is autowired, never bound, so Container::call() hands a
+ * nullable `?Config $config = null` its null default; and Laravel aliases ConnectionResolverInterface to `db`
+ * even in an app that never registered DatabaseServiceProvider, so injecting it would break every boot without a
+ * database. The resolver is made only when `database` is chosen. A direct call with no container (the unit tests)
+ * yields the default.
+ */
#[Bean]
#[ConditionalOnMissingBean(DeadLetterStore::class)]
- public function deadLetterStore(): DeadLetterStore
+ public function deadLetterStore(?Container $container = null): DeadLetterStore
{
- return new InMemoryDeadLetterStore;
+ if ($container === null) {
+ return new InMemoryDeadLetterStore;
+ }
+
+ /** @var Config $config */
+ $config = $container->make(Config::class);
+ $store = $config->string('firefly.eda.dead-letter.store', 'memory');
+
+ if ($store === 'memory') {
+ return new InMemoryDeadLetterStore;
+ }
+
+ if ($store !== 'database') {
+ throw new ConfigurationException("firefly.eda.dead-letter.store='{$store}' is not one of [memory|database].");
+ }
+
+ try {
+ $connections = $container->make(ConnectionResolverInterface::class);
+ } catch (BindingResolutionException $e) {
+ throw new ConfigurationException('firefly.eda.dead-letter.store=database needs a database connection resolver; register Laravel\'s DatabaseServiceProvider (every application has it) or bind your own DeadLetterStore.', previous: $e);
+ }
+
+ return new DatabaseDeadLetterStore(
+ $connections,
+ $config->has('firefly.eda.dead-letter.connection') ? $config->string('firefly.eda.dead-letter.connection') : null,
+ );
}
/**
diff --git a/packages/eda/src/EdaConsumerServiceProvider.php b/packages/eda/src/EdaConsumerServiceProvider.php
index cccb88db..a94fd4ec 100644
--- a/packages/eda/src/EdaConsumerServiceProvider.php
+++ b/packages/eda/src/EdaConsumerServiceProvider.php
@@ -5,18 +5,26 @@
namespace Firefly\Eda;
use Firefly\Eda\Console\ConsumeEventsCommand;
+use Firefly\Eda\Console\ReplayDeadLettersCommand;
use Illuminate\Support\ServiceProvider;
/**
- * Registers the firefly:eda:consume Artisan command via package auto-discovery. Console-guarded so nothing runs in an
- * HTTP request. The command is inert until a broker package binds an EventConsumer.
+ * Registers firefly:eda:consume and firefly:eda:dead-letters:replay via package auto-discovery (console only, so
+ * nothing runs in an HTTP request), and ships the firefly_eda_dead_letters migration when
+ * firefly.eda.dead-letter.store=database. The consume command is inert until a broker package binds an EventConsumer.
*/
final class EdaConsumerServiceProvider extends ServiceProvider
{
+ public const string MIGRATIONS = __DIR__.'/../database/migrations';
+
public function boot(): void
{
+ if ($this->app->make('config')->get('firefly.eda.dead-letter.store') === 'database') {
+ $this->loadMigrationsFrom(self::MIGRATIONS);
+ }
+
if ($this->app->runningInConsole()) {
- $this->commands([ConsumeEventsCommand::class]);
+ $this->commands([ConsumeEventsCommand::class, ReplayDeadLettersCommand::class]);
}
}
}
diff --git a/packages/eda/tests/Console/ReplayDeadLettersCommandTest.php b/packages/eda/tests/Console/ReplayDeadLettersCommandTest.php
new file mode 100644
index 00000000..d7dbdab3
--- /dev/null
+++ b/packages/eda/tests/Console/ReplayDeadLettersCommandTest.php
@@ -0,0 +1,91 @@
+app();
+ $app->make(EventPublisher::class)->publish('orders', 'order.placed', ['id' => 1]);
+ $app->make(EventPublisher::class)->publish('orders', 'order.paid', ['id' => 1]);
+
+ expect($app->make(DeadLetterStore::class))->toBeInstanceOf(DatabaseDeadLetterStore::class)
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+
+ // Dry run: lists, changes nothing.
+ expect($this->replay(['--dry-run' => true]))->toBe(0)
+ ->and($this->replayOutput())->toContain('would replay')
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+
+ // The outage is over: replay only order.placed, then the rest.
+ FlakyOrderListener::$failing = false;
+ expect($this->replay(['--event-type' => 'order.placed']))->toBe(0)
+ ->and(DB::table(DeadLetterSchema::TABLE)->pluck('event_type')->all())->toBe(['order.paid'])
+ ->and($this->replay())->toBe(0)
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(0);
+
+ /** @var ListenerSpy $spy */
+ $spy = $app->make(ListenerSpy::class);
+ expect($spy->seen)->toHaveCount(2)
+ ->and($spy->seen[0])->toStartWith('order.placed#')
+ ->and($spy->seen[0])->not->toEndWith('#first');
+});
+
+it('dead-letters a replay that fails again, and --keep leaves the original in place', function () {
+ /** @var DeadLetterCapstoneTestCase $this */
+ $this->app()->make(EventPublisher::class)->publish('orders', 'order.placed', ['id' => 2]);
+
+ // Still failing: the republished envelope is dead-lettered again by the memory bus (the publish itself succeeds),
+ // and --keep leaves the original entry in place — two entries, nothing lost.
+ expect($this->replay(['--keep' => true]))->toBe(0)
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+});
+
+it('keeps an entry whose republish throws, and exits 1', function () {
+ /** @var DeadLetterCapstoneTestCase $this */
+ $this->app()->make(EventPublisher::class)->publish('orders', 'order.placed', ['id' => 3]);
+ $this->app()->instance(EventPublisher::class, new class implements EventPublisher
+ {
+ public function subscribe(string $eventTypePattern, callable $handler): void {}
+
+ public function publish(string $destination, string $eventType, array $payload, array $headers = []): void
+ {
+ throw new RuntimeException('broker down');
+ }
+
+ public function start(): void {}
+
+ public function stop(): void {}
+ });
+
+ expect($this->replay())->toBe(1)
+ ->and($this->replayOutput())->toContain('could not replay')
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(1);
+});
+
+it('refuses a bound DeadLetterStore that cannot select or remove entries, and exits 1', function () {
+ /** @var DeadLetterCapstoneTestCase $this */
+ $this->app()->instance(DeadLetterStore::class, new class implements DeadLetterStore
+ {
+ public function store(EventEnvelope $envelope, Throwable $cause): void {}
+
+ public function all(): array
+ {
+ return [];
+ }
+ });
+
+ expect($this->replay())->toBe(1)
+ ->and($this->replayOutput())->toContain('cannot select or remove entries');
+});
diff --git a/packages/eda/tests/DeadLetter/DatabaseDeadLetterStoreTest.php b/packages/eda/tests/DeadLetter/DatabaseDeadLetterStoreTest.php
new file mode 100644
index 00000000..8e3bb365
--- /dev/null
+++ b/packages/eda/tests/DeadLetter/DatabaseDeadLetterStoreTest.php
@@ -0,0 +1,82 @@
+up();
+});
+
+function deadLetterDatabaseStore(string $now = '2026-10-12 10:00:00.250'): DatabaseDeadLetterStore
+{
+ /** @var ConnectionResolverInterface $connections */
+ $connections = App::make(ConnectionResolverInterface::class);
+
+ return new DatabaseDeadLetterStore($connections, null, static fn (): DateTimeImmutable => new DateTimeImmutable($now, new DateTimeZone('UTC')));
+}
+
+it('keeps a dead letter durably, envelope, cause and instant included', function () {
+ $envelope = new EventEnvelope('order.placed', 'orders', ['id' => 7, 'lines' => [['sku' => 'A']]], ['x-original-topic' => 'orders', 'x-exception' => 'boom'], 'evt-1');
+ deadLetterDatabaseStore()->store($envelope, new LogicException('boom'));
+
+ $entries = deadLetterDatabaseStore()->all();
+
+ expect(Schema::hasTable(DeadLetterSchema::TABLE))->toBeTrue()
+ ->and($entries)->toHaveCount(1)
+ ->and($entries[0]->id)->toMatch('/^[0-9a-f-]{36}$/')
+ ->and($entries[0]->envelope->eventType)->toBe('order.placed')
+ ->and($entries[0]->envelope->destination)->toBe('orders')
+ ->and($entries[0]->envelope->eventId)->toBe('evt-1')
+ ->and($entries[0]->envelope->payload)->toBe(['id' => 7, 'lines' => [['sku' => 'A']]])
+ ->and($entries[0]->envelope->headers)->toBe(['x-original-topic' => 'orders', 'x-exception' => 'boom'])
+ ->and($entries[0]->exceptionClass)->toBe(LogicException::class)
+ ->and($entries[0]->exceptionMessage)->toBe('boom')
+ ->and($entries[0]->deadLetteredAt?->format('Y-m-d H:i:s.v'))->toBe('2026-10-12 10:00:00.250');
+});
+
+it('selects by id, destination and event type, oldest first, and removes one entry', function () {
+ deadLetterDatabaseStore('2026-10-12 10:00:02')->store(new EventEnvelope('order.paid', 'orders'), new RuntimeException('b'));
+ deadLetterDatabaseStore('2026-10-12 10:00:01')->store(new EventEnvelope('order.placed', 'orders'), new RuntimeException('a'));
+ deadLetterDatabaseStore('2026-10-12 10:00:03')->store(new EventEnvelope('user.created', 'users'), new RuntimeException('c'));
+ $store = deadLetterDatabaseStore();
+
+ $orders = $store->find(new DeadLetterQuery(destination: 'orders'));
+
+ expect(array_map(static fn (DeadLetterEntry $e): string => $e->envelope->eventType, $orders))->toBe(['order.placed', 'order.paid'])
+ ->and($store->find(new DeadLetterQuery(eventType: 'user.created')))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(limit: 2)))->toHaveCount(2)
+ ->and($store->find(new DeadLetterQuery(ids: [(string) $orders[1]->id]))[0]->envelope->eventType)->toBe('order.paid')
+ ->and($store->remove((string) $orders[0]->id))->toBeTrue()
+ ->and($store->remove((string) $orders[0]->id))->toBeFalse()
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+});
+
+it('keeps insertion order for dead letters written in the same millisecond, so --limit replays the oldest', function () {
+ $store = deadLetterDatabaseStore('2026-10-12 10:00:00.250');
+ $types = array_map(static fn (int $i): string => "order.same-instant-{$i}", range(1, 40));
+ foreach ($types as $type) {
+ $store->store(new EventEnvelope($type, 'orders'), new RuntimeException('tie'));
+ }
+
+ expect(array_map(static fn (DeadLetterEntry $e): string => $e->envelope->eventType, $store->all()))->toBe($types)
+ ->and($store->find(new DeadLetterQuery(destination: 'orders', limit: 1))[0]->envelope->eventType)->toBe('order.same-instant-1');
+});
diff --git a/packages/eda/tests/DeadLetter/DeadLetterQueryTest.php b/packages/eda/tests/DeadLetter/DeadLetterQueryTest.php
new file mode 100644
index 00000000..d740f822
--- /dev/null
+++ b/packages/eda/tests/DeadLetter/DeadLetterQueryTest.php
@@ -0,0 +1,27 @@
+store(new EventEnvelope('order.placed', 'orders'), new RuntimeException('a'));
+ $store->store(new EventEnvelope('order.paid', 'orders'), new RuntimeException('b'));
+ $store->store(new EventEnvelope('user.created', 'users'), new RuntimeException('c'));
+
+ [$first] = $store->all();
+
+ expect($first->id)->toMatch('/^[0-9a-f-]{36}$/')
+ ->and($first->deadLetteredAt)->toBeInstanceOf(DateTimeImmutable::class)
+ ->and(array_map(static fn (DeadLetterEntry $e): string => $e->envelope->eventType, $store->find(new DeadLetterQuery(destination: 'orders'))))->toBe(['order.placed', 'order.paid'])
+ ->and($store->find(new DeadLetterQuery(eventType: 'user.created')))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(limit: 1)))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(ids: [(string) $first->id])))->toHaveCount(1)
+ ->and($store->remove((string) $first->id))->toBeTrue()
+ ->and($store->remove((string) $first->id))->toBeFalse()
+ ->and($store->all())->toHaveCount(2);
+});
diff --git a/packages/eda/tests/DeadLetter/DeadLetterStoreSelectionTest.php b/packages/eda/tests/DeadLetter/DeadLetterStoreSelectionTest.php
new file mode 100644
index 00000000..3fc24b1e
--- /dev/null
+++ b/packages/eda/tests/DeadLetter/DeadLetterStoreSelectionTest.php
@@ -0,0 +1,68 @@
+ ['eda' => $store === null ? [] : ['dead-letter' => ['store' => $store]]]]));
+}
+
+function deadLetterResolver(): ConnectionResolverInterface
+{
+ return new class(new SQLiteConnection(new PDO('sqlite::memory:'), ':memory:')) implements ConnectionResolverInterface
+ {
+ public function __construct(private readonly SQLiteConnection $connection) {}
+
+ public function connection($name = null): SQLiteConnection
+ {
+ return $this->connection;
+ }
+
+ public function getDefaultConnection(): string
+ {
+ return 'default';
+ }
+
+ public function setDefaultConnection($name): void {}
+ };
+}
+
+/** A container holding what a booted app holds: the Config, and the connection resolver unless $withDatabase is false. */
+function deadLetterContainer(?string $store, bool $withDatabase = true): Container
+{
+ $container = new Container;
+ $container->instance(Config::class, deadLetterConfig($store));
+ if ($withDatabase) {
+ $container->instance(ConnectionResolverInterface::class, deadLetterResolver());
+ }
+
+ return $container;
+}
+
+it('selects the dead-letter store by firefly.eda.dead-letter.store', function () {
+ expect((new EdaAutoConfiguration)->deadLetterStore())->toBeInstanceOf(InMemoryDeadLetterStore::class)
+ ->and((new EdaAutoConfiguration)->deadLetterStore(deadLetterContainer(null)))->toBeInstanceOf(InMemoryDeadLetterStore::class)
+ ->and((new EdaAutoConfiguration)->deadLetterStore(deadLetterContainer('memory')))->toBeInstanceOf(InMemoryDeadLetterStore::class)
+ ->and((new EdaAutoConfiguration)->deadLetterStore(deadLetterContainer('database')))->toBeInstanceOf(DatabaseDeadLetterStore::class)
+ ->and(fn () => (new EdaAutoConfiguration)->deadLetterStore(deadLetterContainer('database', withDatabase: false)))->toThrow(ConfigurationException::class, 'firefly.eda.dead-letter.store=database')
+ ->and(fn () => (new EdaAutoConfiguration)->deadLetterStore(deadLetterContainer('redis')))->toThrow(ConfigurationException::class, 'memory|database');
+});
+
+it('boots an app without a database on the memory store: the resolver is made only when database is chosen', function () {
+ $context = bootFireflyApp(['firefly' => ['eda' => []]], [EdaServiceProvider::class, EdaWiringProvider::class]);
+
+ expect($context->get(DeadLetterStore::class))->toBeInstanceOf(InMemoryDeadLetterStore::class);
+});
diff --git a/packages/eda/tests/DeadLetterFixtures/FlakyOrderListener.php b/packages/eda/tests/DeadLetterFixtures/FlakyOrderListener.php
new file mode 100644
index 00000000..f4c1d1d3
--- /dev/null
+++ b/packages/eda/tests/DeadLetterFixtures/FlakyOrderListener.php
@@ -0,0 +1,28 @@
+spy->record($envelope->eventType.'#'.($envelope->headers['x-replayed-from'] ?? 'first'));
+ }
+}
diff --git a/packages/eda/tests/Integration/DatabaseDeadLetterStoreIntegrationTest.php b/packages/eda/tests/Integration/DatabaseDeadLetterStoreIntegrationTest.php
new file mode 100644
index 00000000..eab650f9
--- /dev/null
+++ b/packages/eda/tests/Integration/DatabaseDeadLetterStoreIntegrationTest.php
@@ -0,0 +1,78 @@
+down() : $migration->up();
+}
+
+foreach (IntegrationDsn::lanes() as $lane) {
+ it("keeps, selects and removes eda dead letters on {$lane}", function () use ($lane): void {
+ config(['database.connections.it_dlq' => IntegrationDsn::connectionConfig($lane, $lane === 'sqlite' ? ['database' => IntegrationDsn::sqlitePath('eda-dlq')] : [])]);
+ DB::setDefaultConnection('it_dlq');
+ edaDeadLetterMigrate('down');
+ edaDeadLetterMigrate('up');
+
+ try {
+ $connections = App::make(ConnectionResolverInterface::class);
+ $at = static fn (string $instant): DatabaseDeadLetterStore => new DatabaseDeadLetterStore($connections, 'it_dlq', static fn (): DateTimeImmutable => new DateTimeImmutable($instant, new DateTimeZone('UTC')));
+
+ $payload = ['customer' => 'Ñandú Café — 東京', 'lines' => [['sku' => 'A', 'qty' => 2]], 'total' => 12.5];
+ $at('2026-10-12 10:00:00.200')->store(new EventEnvelope('order.placed', 'orders', $payload, ['x-original-topic' => 'orders', 'x-exception' => 'boom'], 'evt-1'), new RuntimeException('boom'));
+ $at('2026-10-12 10:00:00.100')->store(new EventEnvelope('order.paid', 'orders', ['id' => 1]), new LogicException('first'));
+ $at('2026-10-12 10:00:00.300')->store(new EventEnvelope('user.created', 'users'), new RuntimeException('third'));
+
+ $store = $at('2026-10-12 10:00:01');
+ $all = $store->all();
+
+ expect(array_map(static fn (DeadLetterEntry $entry): string => $entry->envelope->eventType, $all))->toBe(['order.paid', 'order.placed', 'user.created'])
+ // toEqual, not toBe: MySQL's JSON type re-orders object keys, and order is not part of the contract.
+ ->and($all[1]->envelope->payload)->toEqual($payload)
+ ->and($all[1]->envelope->headers)->toEqual(['x-original-topic' => 'orders', 'x-exception' => 'boom'])
+ ->and($all[1]->envelope->eventId)->toBe('evt-1')
+ ->and($all[1]->deadLetteredAt?->format('Y-m-d H:i:s.v'))->toBe('2026-10-12 10:00:00.200')
+ ->and($store->find(new DeadLetterQuery(destination: 'orders', eventType: 'order.placed')))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(limit: 2)))->toHaveCount(2)
+ ->and($store->remove((string) $all[0]->id))->toBeTrue()
+ ->and($store->find(new DeadLetterQuery(ids: [(string) $all[0]->id])))->toBe([])
+ ->and($store->all())->toHaveCount(2);
+
+ // A tie on the instant: the time-ordered id keeps insertion order under this server's collation.
+ $tied = $at('2026-10-12 10:00:02.000');
+ $types = array_map(static fn (int $i): string => "tie.{$i}", range(1, 20));
+ foreach ($types as $type) {
+ $tied->store(new EventEnvelope($type, 'ties'), new RuntimeException('tie'));
+ }
+ expect(array_map(static fn (DeadLetterEntry $entry): string => $entry->envelope->eventType, $store->find(new DeadLetterQuery(destination: 'ties'))))->toBe($types);
+ } finally {
+ edaDeadLetterMigrate('down');
+ DB::connection('it_dlq')->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/eda/tests/Support/DeadLetterCapstoneTestCase.php b/packages/eda/tests/Support/DeadLetterCapstoneTestCase.php
new file mode 100644
index 00000000..0a1b9a79
--- /dev/null
+++ b/packages/eda/tests/Support/DeadLetterCapstoneTestCase.php
@@ -0,0 +1,73 @@
+ */
+ protected function fireflyProviders(): array
+ {
+ return [EdaServiceProvider::class, EdaWiringProvider::class, EdaConsumerServiceProvider::class];
+ }
+
+ /** @return array */
+ protected function configOverrides(): array
+ {
+ return ['firefly.eda.provider' => 'memory', 'firefly.eda.dead-letter.store' => 'database'];
+ }
+
+ protected function defineFireflyEnvironment(Application $app): void
+ {
+ FlakyOrderListener::$failing = true;
+ $app->instance(EventListenerManifest::class, new EventListenerManifest(
+ (new EventListenerScanner)->scan(['Firefly\\Eda\\Tests\\DeadLetterFixtures\\' => dirname(__DIR__).'/DeadLetterFixtures']),
+ ));
+ $app->singleton(ListenerSpy::class);
+ }
+
+ protected function setUp(): void
+ {
+ parent::setUp();
+
+ Schema::create(DeadLetterSchema::TABLE, fn (Blueprint $table) => DeadLetterSchema::blueprint($table));
+ }
+
+ /** @param array $options */
+ public function replay(array $options = []): int
+ {
+ /** @var Kernel $kernel */
+ $kernel = $this->app()->make(Kernel::class);
+
+ return $kernel->call('firefly:eda:dead-letters:replay', $options);
+ }
+
+ public function replayOutput(): string
+ {
+ /** @var Kernel $kernel */
+ $kernel = $this->app()->make(Kernel::class);
+
+ return $kernel->output();
+ }
+}
diff --git a/packages/feature-flags/src/Store/DatabaseFlagStore.php b/packages/feature-flags/src/Store/DatabaseFlagStore.php
index 0e2f3ab0..9226303d 100644
--- a/packages/feature-flags/src/Store/DatabaseFlagStore.php
+++ b/packages/feature-flags/src/Store/DatabaseFlagStore.php
@@ -60,6 +60,9 @@ public function all(): array
$flag = self::storedFlag($row);
$flags[$flag->key] = $flag;
}
+ // Byte order, like MemoryFlagStore: ORDER BY follows the column collation, which differs per server and locale
+ // (PostgreSQL under en_US.UTF-8 sorts 'a' before 'A'; MySQL's case-insensitive collations tie them).
+ ksort($flags, SORT_STRING);
return $flags;
}
diff --git a/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreAbsentRaceTest.php b/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreAbsentRaceTest.php
index f7486b6f..c6f365f6 100644
--- a/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreAbsentRaceTest.php
+++ b/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreAbsentRaceTest.php
@@ -5,6 +5,7 @@
use Firefly\FeatureFlags\Store\DatabaseFlagStore;
use Firefly\FeatureFlags\Store\FeatureFlagSchema;
use Firefly\FeatureFlags\Tests\Support\DatabaseStoreIntegration;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Database\Schema\Blueprint;
use Symfony\Component\Process\InputStream;
use Symfony\Component\Process\Process;
@@ -98,7 +99,7 @@
$connection->getSchemaBuilder()->drop('recovery_markers');
FeatureFlagSchema::drop($connection->getSchemaBuilder());
}
- })->with($ownership === 'root' ? ['direct'] : ['direct', 'callback'])->group('integration')->skip(getenv($variable) === false, "Set {$variable} to an isolated server.");
+ })->with($ownership === 'root' ? ['direct'] : ['direct', 'callback'])->group('integration')->skip(IntegrationDsn::missing($variable), "Set {$variable} to an isolated server.");
}
}
}
diff --git a/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreIntegrationTest.php b/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreIntegrationTest.php
index 6294eac6..00ef78f3 100644
--- a/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreIntegrationTest.php
+++ b/packages/feature-flags/tests/Store/Integration/DatabaseFlagStoreIntegrationTest.php
@@ -8,6 +8,7 @@
use Firefly\FeatureFlags\Store\FlagStoreConflict;
use Firefly\FeatureFlags\Tests\Support\DatabaseStoreIntegration;
use Firefly\FeatureFlags\Tests\Support\FlagStores;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Database\QueryException;
use Symfony\Component\Process\Process;
@@ -32,7 +33,7 @@
expect($store->get('A'))->toBeNull()->and($store->get('a')?->version)->toBe(1)
->and($store->history('A')[0]->previous)->toBe($store->get('a')?->definition);
FeatureFlagSchema::drop($connection->getSchemaBuilder());
- })->group('integration')->skip(getenv($variable) === false, "Set {$variable} to an isolated server.");
+ })->group('integration')->skip(IntegrationDsn::missing($variable), "Set {$variable} to an isolated server.");
it("rolls back failed audit writes inside an outer transaction on {$backend}", function () use ($driver, $variable): void {
$connection = DatabaseStoreIntegration::connection($driver, $variable);
@@ -51,7 +52,7 @@
expect($store->get('new'))->toBeNull()->and($store->get('keep')?->version)->toBe(1)
->and(count($store->history('keep')))->toBe(1);
FeatureFlagSchema::drop($connection->getSchemaBuilder());
- })->group('integration')->skip(getenv($variable) === false, "Set {$variable} to an isolated server.");
+ })->group('integration')->skip(IntegrationDsn::missing($variable), "Set {$variable} to an isolated server.");
it("serializes competing process updates and emits only surviving callbacks on {$backend}", function () use ($driver, $variable): void {
$connection = DatabaseStoreIntegration::connection($driver, $variable);
FeatureFlagSchema::drop($connection->getSchemaBuilder());
@@ -91,7 +92,7 @@
$process->stop();
FeatureFlagSchema::drop($connection->getSchemaBuilder());
}
- })->group('integration')->skip(getenv($variable) === false, "Set {$variable} to an isolated server.");
+ })->group('integration')->skip(IntegrationDsn::missing($variable), "Set {$variable} to an isolated server.");
it("reads PyFly microsecond UTC rows independently of the SQL session zone on {$backend}", function () use ($driver, $variable): void {
$connection = DatabaseStoreIntegration::connection($driver, $variable);
@@ -106,6 +107,6 @@
expect($store->get('pyfly')?->updatedAt->format('Y-m-d H:i:s.uP'))->toBe('2026-10-01 12:00:00.123456+00:00')
->and($store->history('pyfly')[0]->changedAt->format('Y-m-d H:i:s.uP'))->toBe('2026-10-01 12:00:00.123456+00:00');
FeatureFlagSchema::drop($connection->getSchemaBuilder());
- })->group('integration')->skip(getenv($variable) === false, "Set {$variable} to an isolated server.");
+ })->group('integration')->skip(IntegrationDsn::missing($variable), "Set {$variable} to an isolated server.");
}
diff --git a/packages/feature-flags/tests/Support/DatabaseStoreIntegration.php b/packages/feature-flags/tests/Support/DatabaseStoreIntegration.php
index 7e0131dc..97230eab 100644
--- a/packages/feature-flags/tests/Support/DatabaseStoreIntegration.php
+++ b/packages/feature-flags/tests/Support/DatabaseStoreIntegration.php
@@ -4,7 +4,7 @@
namespace Firefly\FeatureFlags\Tests\Support;
-use Illuminate\Database\Capsule\Manager;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Database\Connection;
use RuntimeException;
@@ -12,22 +12,10 @@ final class DatabaseStoreIntegration
{
public static function connection(string $driver, string $variable): Connection
{
- $dsn = getenv($variable);
- if ($dsn === false || $dsn === '') {
+ if (IntegrationDsn::missing($variable)) {
throw new RuntimeException('An isolated integration DSN is required.');
}
- $parts = [];
- foreach (explode(';', $dsn) as $pair) {
- [$key, $value] = array_pad(explode('=', $pair, 2), 2, '');
- $parts[$key] = $value;
- }
- $capsule = new Manager;
- $capsule->addConnection([
- 'driver' => $driver, 'host' => $parts['host'], 'port' => (int) $parts['port'],
- 'database' => $parts['dbname'], 'username' => $parts['user'], 'password' => $parts['password'],
- 'charset' => $driver === 'pgsql' ? 'utf8' : 'utf8mb4', 'prefix' => '', 'schema' => 'public',
- ]);
- return $capsule->getConnection();
+ return IntegrationDsn::connectFor($driver, $variable);
}
}
diff --git a/packages/firefly/composer.json b/packages/firefly/composer.json
index faee7fec..5eb1f6c3 100644
--- a/packages/firefly/composer.json
+++ b/packages/firefly/composer.json
@@ -33,6 +33,7 @@
"firefly/data": "*@dev",
"firefly/domain": "*@dev",
"firefly/eda": "*@dev",
+ "firefly/eda-database": "*@dev",
"firefly/feature-flags": "*@dev",
"firefly/kernel": "*@dev",
"firefly/messaging": "*@dev",
@@ -40,6 +41,7 @@
"firefly/openapi": "*@dev",
"firefly/resilience": "*@dev",
"firefly/scheduling": "*@dev",
+ "firefly/scheduling-database": "*@dev",
"firefly/scheduling-postgres": "*@dev",
"firefly/security": "*@dev",
"firefly/security-oauth2-client": "*@dev",
diff --git a/packages/installer/README.md b/packages/installer/README.md
index f61700df..73926c7a 100644
--- a/packages/installer/README.md
+++ b/packages/installer/README.md
@@ -31,7 +31,8 @@ A capability records a requirement in the generated `composer.json`, satisfied b
`replace` metadata. All component code is included; configuration selects optional features.
`--with=testing` also adds the external test harness to the application's development dependencies.
-Adapters (`eda-kafka`, `eda-rabbitmq`, `eda-postgres`, `scheduling-postgres`) pull their port in with them
+Adapters (`eda-kafka`, `eda-rabbitmq`, `eda-database`, `eda-postgres`, `scheduling-database`,
+`scheduling-postgres`) pull their port in with them
and are deliberately **excluded from `--full`**: which broker or engine an application talks to is not
something an archetype can guess. Their code is already bundled, but configuring PostgreSQL or Kafka still
requires the corresponding PHP extension. `security-oauth2-client` pulls `security` in the same way — a login with no
diff --git a/packages/installer/src/CapabilityCatalog.php b/packages/installer/src/CapabilityCatalog.php
index 395f3719..bb0c0e9b 100644
--- a/packages/installer/src/CapabilityCatalog.php
+++ b/packages/installer/src/CapabilityCatalog.php
@@ -47,7 +47,9 @@ public static function all(): array
new Capability('eda-kafka', 'firefly/eda-kafka', 'Kafka publisher/consumer over ext-rdkafka', requires: ['eda'], adapter: true),
new Capability('eda-rabbitmq', 'firefly/eda-rabbitmq', 'RabbitMQ publisher/consumer over php-amqplib', requires: ['eda'], adapter: true),
- new Capability('eda-postgres', 'firefly/eda-postgres', 'Postgres same-transaction outbox publisher', requires: ['eda'], adapter: true),
+ new Capability('eda-database', 'firefly/eda-database', 'Portable same-transaction outbox on any database connection, safe with concurrent consumers', requires: ['eda'], adapter: true),
+ new Capability('eda-postgres', 'firefly/eda-postgres', 'Postgres same-transaction outbox: the database outbox plus pg_notify/LISTEN', requires: ['eda-database'], adapter: true),
+ new Capability('scheduling-database', 'firefly/scheduling-database', 'Database lease-table and MySQL named-lock DistributedLock backends', requires: ['scheduling'], adapter: true),
new Capability('scheduling-postgres', 'firefly/scheduling-postgres', 'Postgres advisory-lock DistributedLock backend', requires: ['scheduling'], adapter: true),
new Capability('testing', 'firefly/testing', 'The first-party test kit: boot harness, sqlite fixtures, assertions', dev: true),
diff --git a/packages/messaging/README.md b/packages/messaging/README.md
index 0c15018f..2c94efc9 100644
--- a/packages/messaging/README.md
+++ b/packages/messaging/README.md
@@ -3,7 +3,7 @@
LaraFly's lower-level raw-bytes messaging: a `MessageBrokerPort` (topic + bytes value + key + consumer group)
with an in-memory default adapter and a Laravel-queue async adapter, a broker-agnostic `#[MessageListener]`
attribute (carrying `retries`/`retryDelay`/`deadLetterTopic`) compiled into a `MessageListenerManifest` and wired
-onto the broker at boot, and a bytes-aware retry / dead-letter helper. Real brokers (Kafka/RabbitMQ) ship as their
-own `firefly/messaging-` packages at SP-4.
+onto the broker at boot, and a bytes-aware retry / dead-letter helper. Durable dead letters
+(`firefly.messaging.dead-letter.store=database`) and `firefly:messaging:dead-letters:replay` ship here.
Apache-2.0 © Firefly Software Solutions Inc.
diff --git a/packages/messaging/cache/firefly-messaging-components.php b/packages/messaging/cache/firefly-messaging-components.php
index e3ca7799..632f76ee 100644
--- a/packages/messaging/cache/firefly-messaging-components.php
+++ b/packages/messaging/cache/firefly-messaging-components.php
@@ -38,6 +38,7 @@
'order' => 0,
'lazy' => false,
'dependencies' => [
+ 0 => 'Illuminate\\Container\\Container',
],
],
],
diff --git a/packages/messaging/composer.json b/packages/messaging/composer.json
index 01f47eb6..11aa6cac 100644
--- a/packages/messaging/composer.json
+++ b/packages/messaging/composer.json
@@ -27,8 +27,10 @@
"firefly/context": "*@dev",
"firefly/kernel": "*@dev",
"illuminate/bus": "^13.0",
+ "illuminate/console": "^13.0",
"illuminate/container": "^13.0",
"illuminate/contracts": "^13.0",
+ "illuminate/database": "^13.0",
"illuminate/queue": "^13.0",
"illuminate/support": "^13.0"
},
@@ -36,7 +38,8 @@
"laravel": {
"providers": [
"Firefly\\Messaging\\MessagingServiceProvider",
- "Firefly\\Messaging\\MessagingWiringProvider"
+ "Firefly\\Messaging\\MessagingWiringProvider",
+ "Firefly\\Messaging\\MessagingConsoleServiceProvider"
]
},
"branch-alias": {
diff --git a/packages/messaging/database/migrations/2026_10_12_000300_create_firefly_messaging_dead_letters_table.php b/packages/messaging/database/migrations/2026_10_12_000300_create_firefly_messaging_dead_letters_table.php
new file mode 100644
index 00000000..4eb1beca
--- /dev/null
+++ b/packages/messaging/database/migrations/2026_10_12_000300_create_firefly_messaging_dead_letters_table.php
@@ -0,0 +1,21 @@
+ DeadLetterSchema::blueprint($table));
+ }
+
+ public function down(): void
+ {
+ Schema::dropIfExists(DeadLetterSchema::TABLE);
+ }
+};
diff --git a/packages/messaging/src/Console/ReplayDeadLettersCommand.php b/packages/messaging/src/Console/ReplayDeadLettersCommand.php
new file mode 100644
index 00000000..cc927c1f
--- /dev/null
+++ b/packages/messaging/src/Console/ReplayDeadLettersCommand.php
@@ -0,0 +1,90 @@
+` is added, so
+ * a listener can tell a replay from a first delivery. Each entry that republished is removed unless --keep; one that
+ * fails is reported and kept, and the command exits FAILURE. Replay is at-least-once, like firefly:eda:dead-letters:
+ * replay: a crash between the publish and the removal replays that entry again next time.
+ */
+final class ReplayDeadLettersCommand extends Command
+{
+ /** @var string */
+ protected $signature = 'firefly:messaging:dead-letters:replay {--id=* : replay only these entries} {--topic= : only entries of this topic (the dead-letter topic or the original one)} {--limit=100 : at most this many entries} {--dry-run : list what would be replayed and change nothing} {--keep : keep each entry after it is republished}';
+
+ /** @var string */
+ protected $description = 'Republish dead-lettered messages through the bound MessageBrokerPort, removing each one that republished.';
+
+ public function handle(DeadLetterStore $store, MessageBrokerPort $broker): int
+ {
+ if (! $store instanceof ReplayableDeadLetterStore) {
+ $this->error(sprintf('The bound messaging DeadLetterStore (%s) cannot select or remove entries. Bind a ReplayableDeadLetterStore, or drop the binding and set firefly.messaging.dead-letter.store=database.', get_debug_type($store)));
+
+ return self::FAILURE;
+ }
+
+ $entries = $store->find($this->query());
+
+ $replayed = 0;
+ $failed = 0;
+ foreach ($entries as $entry) {
+ $message = $entry->message;
+ $target = $message->headers['x-original-topic'] ?? $message->topic;
+ $headers = array_diff_key($message->headers, ['x-original-topic' => true, 'x-exception' => true]);
+ $headers['x-replayed-from'] = (string) $entry->id;
+
+ if ($this->option('dry-run') === true) {
+ $this->line(sprintf('would replay %s %s -> %s', $entry->id, $message->topic, $target));
+
+ continue;
+ }
+
+ try {
+ $broker->publish($target, $message->value, $message->key, $headers);
+ if ($this->option('keep') !== true && $entry->id !== null) {
+ $store->remove($entry->id);
+ }
+ $replayed++;
+ $this->line(sprintf('replayed %s %s -> %s', $entry->id, $message->topic, $target));
+ } catch (Throwable $e) {
+ $failed++;
+ $this->error(sprintf('could not replay %s %s: %s', $entry->id, $message->topic, $e->getMessage()));
+ }
+ }
+
+ $this->info(sprintf('firefly:messaging:dead-letters:replay — %d selected, %d replayed, %d failed.', count($entries), $replayed, $failed));
+
+ return $failed === 0 ? self::SUCCESS : self::FAILURE;
+ }
+
+ private function query(): DeadLetterQuery
+ {
+ $ids = [];
+ foreach ((array) $this->option('id') as $id) {
+ if (is_string($id) && $id !== '') {
+ $ids[] = $id;
+ }
+ }
+
+ $topic = $this->option('topic');
+ $limit = $this->option('limit');
+
+ return new DeadLetterQuery(
+ $ids,
+ is_string($topic) && $topic !== '' ? $topic : null,
+ is_numeric($limit) ? max(1, (int) $limit) : 100,
+ );
+ }
+}
diff --git a/packages/messaging/src/DeadLetter/DatabaseDeadLetterStore.php b/packages/messaging/src/DeadLetter/DatabaseDeadLetterStore.php
new file mode 100644
index 00000000..7ccd0fcd
--- /dev/null
+++ b/packages/messaging/src/DeadLetter/DatabaseDeadLetterStore.php
@@ -0,0 +1,133 @@
+clock === null ? new DateTimeImmutable : ($this->clock)();
+
+ $this->db()->table(DeadLetterSchema::TABLE)->insert([
+ 'id' => (string) Str::uuid7(),
+ 'topic' => $message->topic,
+ 'original_topic' => $message->headers['x-original-topic'] ?? null,
+ 'message_key' => $message->key === null ? null : base64_encode($message->key),
+ 'value' => base64_encode($message->value),
+ 'headers' => json_encode($message->headers === [] ? new stdClass : $message->headers, JSON_THROW_ON_ERROR | JSON_INVALID_UTF8_SUBSTITUTE),
+ 'exception_class' => $cause::class,
+ 'exception_message' => mb_scrub($cause->getMessage(), 'UTF-8'),
+ 'dead_lettered_at' => $now->setTimezone(new DateTimeZone('UTC'))->format('Y-m-d H:i:s.v'),
+ ]);
+ }
+
+ /**
+ * @return list
+ */
+ public function all(): array
+ {
+ return $this->find(new DeadLetterQuery(limit: null));
+ }
+
+ public function find(DeadLetterQuery $query): array
+ {
+ $select = $this->db()->table(DeadLetterSchema::TABLE)->orderBy('dead_lettered_at')->orderBy('id');
+
+ if ($query->ids !== []) {
+ $select->whereIn('id', $query->ids);
+ }
+ if ($query->topic !== null) {
+ $topic = $query->topic;
+ $select->where(static function (Builder $where) use ($topic): void {
+ $where->where('topic', $topic)->orWhere('original_topic', $topic);
+ });
+ }
+ if ($query->limit !== null) {
+ $select->limit(max(1, $query->limit));
+ }
+
+ $entries = [];
+ foreach ($select->get() as $row) {
+ $entries[] = self::entry($row);
+ }
+
+ return $entries;
+ }
+
+ public function remove(string $id): bool
+ {
+ return $this->db()->table(DeadLetterSchema::TABLE)->where('id', $id)->delete() === 1;
+ }
+
+ private function db(): ConnectionInterface
+ {
+ return $this->connections->connection($this->connection);
+ }
+
+ private static function entry(object $row): DeadLetterEntry
+ {
+ $columns = get_object_vars($row);
+ $value = base64_decode(self::text($columns['value'] ?? null), true);
+ $key = $columns['message_key'] ?? null;
+ $key = $key === null ? false : base64_decode(self::text($key), true);
+ $at = self::text($columns['dead_lettered_at'] ?? null);
+
+ $headers = [];
+ $json = self::text($columns['headers'] ?? null);
+ $map = $json === '' ? [] : json_decode($json, true, 512, JSON_THROW_ON_ERROR);
+ foreach (is_array($map) ? $map : [] as $name => $header) {
+ if (is_scalar($header)) {
+ $headers[(string) $name] = (string) $header;
+ }
+ }
+
+ return new DeadLetterEntry(
+ new Message(self::text($columns['topic'] ?? null), $value === false ? '' : $value, $key === false ? null : $key, $headers),
+ self::text($columns['exception_class'] ?? null),
+ self::text($columns['exception_message'] ?? null),
+ self::text($columns['id'] ?? null),
+ $at === '' ? null : new DateTimeImmutable($at, new DateTimeZone('UTC')),
+ );
+ }
+
+ private static function text(mixed $value): string
+ {
+ return is_scalar($value) ? (string) $value : '';
+ }
+}
diff --git a/packages/messaging/src/DeadLetter/DeadLetterEntry.php b/packages/messaging/src/DeadLetter/DeadLetterEntry.php
index 26c4e95e..b9b17822 100644
--- a/packages/messaging/src/DeadLetter/DeadLetterEntry.php
+++ b/packages/messaging/src/DeadLetter/DeadLetterEntry.php
@@ -4,11 +4,14 @@
namespace Firefly\Messaging\DeadLetter;
+use DateTimeImmutable;
use Firefly\Messaging\Message;
/**
* One dead-lettered message plus the exception class + message that caused it. The message is already re-keyed to
- * the dead-letter topic and carries the x-original-topic / x-exception headers added by RetryingMessageHandler.
+ * the dead-letter topic and carries the x-original-topic / x-exception headers added by RetryingMessageHandler. A
+ * replayable store fills `id` (what firefly:messaging:dead-letters:replay selects and removes by) and
+ * `deadLetteredAt`.
*/
final readonly class DeadLetterEntry
{
@@ -16,5 +19,7 @@ public function __construct(
public Message $message,
public string $exceptionClass,
public string $exceptionMessage,
+ public ?string $id = null,
+ public ?DateTimeImmutable $deadLetteredAt = null,
) {}
}
diff --git a/packages/messaging/src/DeadLetter/DeadLetterQuery.php b/packages/messaging/src/DeadLetter/DeadLetterQuery.php
new file mode 100644
index 00000000..a108d15b
--- /dev/null
+++ b/packages/messaging/src/DeadLetter/DeadLetterQuery.php
@@ -0,0 +1,30 @@
+ $ids empty = any
+ * @param int|null $limit null = all
+ */
+ public function __construct(
+ public array $ids = [],
+ public ?string $topic = null,
+ public ?int $limit = 100,
+ ) {}
+
+ public function matches(DeadLetterEntry $entry): bool
+ {
+ return ($this->ids === [] || in_array($entry->id, $this->ids, true))
+ && ($this->topic === null
+ || $entry->message->topic === $this->topic
+ || ($entry->message->headers['x-original-topic'] ?? null) === $this->topic);
+ }
+}
diff --git a/packages/messaging/src/DeadLetter/DeadLetterSchema.php b/packages/messaging/src/DeadLetter/DeadLetterSchema.php
new file mode 100644
index 00000000..0cf08f4b
--- /dev/null
+++ b/packages/messaging/src/DeadLetter/DeadLetterSchema.php
@@ -0,0 +1,38 @@
+string('id', 36)->primary();
+ $table->string('topic');
+ $table->string('original_topic')->nullable();
+ $table->text('message_key')->nullable();
+ $table->longText('value');
+ $table->json('headers');
+ $table->string('exception_class');
+ $table->text('exception_message');
+ $table->dateTime('dead_lettered_at', 3);
+
+ $table->index(['topic'], 'firefly_messaging_dead_letters_topic_idx');
+ $table->index(['dead_lettered_at'], 'firefly_messaging_dead_letters_at_idx');
+ }
+}
diff --git a/packages/messaging/src/DeadLetter/DeadLetterStore.php b/packages/messaging/src/DeadLetter/DeadLetterStore.php
index 89316717..ad1f5dce 100644
--- a/packages/messaging/src/DeadLetter/DeadLetterStore.php
+++ b/packages/messaging/src/DeadLetter/DeadLetterStore.php
@@ -10,7 +10,9 @@
/**
* The messaging dead-letter sink (pyfly messaging/error_handling.py parity, bytes flavour). A consumer that
* exhausts its retries has its message stored here re-keyed to its deadLetterTopic. Distinct from eda's
- * DeadLetterStore (envelope vs raw-bytes) — no shared class, no cross-package edge.
+ * DeadLetterStore (envelope vs raw-bytes) — no shared class, no cross-package edge. InMemoryDeadLetterStore is the
+ * default; DatabaseDeadLetterStore (firefly.messaging.dead-letter.store=database) keeps them durably. Both implement
+ * ReplayableDeadLetterStore, which firefly:messaging:dead-letters:replay needs.
*/
interface DeadLetterStore
{
diff --git a/packages/messaging/src/DeadLetter/InMemoryDeadLetterStore.php b/packages/messaging/src/DeadLetter/InMemoryDeadLetterStore.php
index b5694756..f2bb113f 100644
--- a/packages/messaging/src/DeadLetter/InMemoryDeadLetterStore.php
+++ b/packages/messaging/src/DeadLetter/InMemoryDeadLetterStore.php
@@ -4,20 +4,23 @@
namespace Firefly\Messaging\DeadLetter;
+use DateTimeImmutable;
use Firefly\Messaging\Message;
+use Illuminate\Support\Str;
use Throwable;
/**
- * The default messaging DeadLetterStore: an in-process append-only list. Zero external services.
+ * The default messaging DeadLetterStore: an in-process append-only list. Zero external services — and lost when the
+ * process ends; firefly.messaging.dead-letter.store=database keeps dead letters durably.
*/
-final class InMemoryDeadLetterStore implements DeadLetterStore
+final class InMemoryDeadLetterStore implements ReplayableDeadLetterStore
{
/** @var list */
private array $entries = [];
public function store(Message $message, Throwable $cause): void
{
- $this->entries[] = new DeadLetterEntry($message, $cause::class, $cause->getMessage());
+ $this->entries[] = new DeadLetterEntry($message, $cause::class, $cause->getMessage(), (string) Str::uuid(), new DateTimeImmutable);
}
/**
@@ -27,4 +30,24 @@ public function all(): array
{
return $this->entries;
}
+
+ public function find(DeadLetterQuery $query): array
+ {
+ $found = array_values(array_filter($this->entries, $query->matches(...)));
+
+ return $query->limit === null ? $found : array_slice($found, 0, max(1, $query->limit));
+ }
+
+ public function remove(string $id): bool
+ {
+ foreach ($this->entries as $index => $entry) {
+ if ($entry->id === $id) {
+ array_splice($this->entries, $index, 1);
+
+ return true;
+ }
+ }
+
+ return false;
+ }
}
diff --git a/packages/messaging/src/DeadLetter/ReplayableDeadLetterStore.php b/packages/messaging/src/DeadLetter/ReplayableDeadLetterStore.php
new file mode 100644
index 00000000..2ed2395f
--- /dev/null
+++ b/packages/messaging/src/DeadLetter/ReplayableDeadLetterStore.php
@@ -0,0 +1,20 @@
+ oldest first
+ */
+ public function find(DeadLetterQuery $query): array;
+
+ public function remove(string $id): bool;
+}
diff --git a/packages/messaging/src/MessagingAutoConfiguration.php b/packages/messaging/src/MessagingAutoConfiguration.php
index 9436e2a6..5bb62acb 100644
--- a/packages/messaging/src/MessagingAutoConfiguration.php
+++ b/packages/messaging/src/MessagingAutoConfiguration.php
@@ -9,15 +9,20 @@
use Firefly\Container\Attributes\Configuration;
use Firefly\Container\Attributes\Order;
use Firefly\Context\Condition\Attributes\ConditionalOnMissingBean;
+use Firefly\Kernel\Exception\Framework\ConfigurationException;
use Firefly\Messaging\Broker\InMemoryMessageBroker;
use Firefly\Messaging\Broker\QueueMessageBroker;
+use Firefly\Messaging\DeadLetter\DatabaseDeadLetterStore;
use Firefly\Messaging\DeadLetter\DeadLetterStore;
use Firefly\Messaging\DeadLetter\InMemoryDeadLetterStore;
use Illuminate\Container\Container;
+use Illuminate\Contracts\Container\BindingResolutionException;
+use Illuminate\Database\ConnectionResolverInterface;
/**
* Always-on messaging transport wiring. Selects the MessageBrokerPort adapter by firefly.messaging.provider
- * (`memory` default | `queue`) and the DeadLetterStore default. #[Order(1000)] places it after user definitions;
+ * (`memory` default | `queue`) and the DeadLetterStore by firefly.messaging.dead-letter.store (`memory` default |
+ * `database`). #[Order(1000)] places it after user definitions;
* each #[ConditionalOnMissingBean] lets an app bind its own and win. Both adapters are singletons (the #[Bean]
* default), so the wiring pass and the queue worker share ONE subscriber set. Mirrors EdaAutoConfiguration.
*/
@@ -39,10 +44,44 @@ public function messageBroker(Config $config, Container $container): MessageBrok
return new InMemoryMessageBroker;
}
+ /**
+ * firefly.messaging.dead-letter.store: `memory` (the default — in-process, lost on exit) or `database` (durable,
+ * replayable, on firefly.messaging.dead-letter.connection). It takes the container, not Config and the connection
+ * resolver, for the reason EdaAutoConfiguration::deadLetterStore() does: Config is autowired, never bound, so
+ * Container::call() hands a nullable `?Config $config = null` its null default; and Laravel aliases
+ * ConnectionResolverInterface to `db` even in an app that never registered DatabaseServiceProvider, so injecting
+ * it would break every boot without a database. The resolver is made only when `database` is chosen. A direct
+ * call with no container (the unit tests) yields the default.
+ */
#[Bean]
#[ConditionalOnMissingBean(DeadLetterStore::class)]
- public function deadLetterStore(): DeadLetterStore
+ public function deadLetterStore(?Container $container = null): DeadLetterStore
{
- return new InMemoryDeadLetterStore;
+ if ($container === null) {
+ return new InMemoryDeadLetterStore;
+ }
+
+ /** @var Config $config */
+ $config = $container->make(Config::class);
+ $store = $config->string('firefly.messaging.dead-letter.store', 'memory');
+
+ if ($store === 'memory') {
+ return new InMemoryDeadLetterStore;
+ }
+
+ if ($store !== 'database') {
+ throw new ConfigurationException("firefly.messaging.dead-letter.store='{$store}' is not one of [memory|database].");
+ }
+
+ try {
+ $connections = $container->make(ConnectionResolverInterface::class);
+ } catch (BindingResolutionException $e) {
+ throw new ConfigurationException('firefly.messaging.dead-letter.store=database needs a database connection resolver; register Laravel\'s DatabaseServiceProvider (every application has it) or bind your own DeadLetterStore.', previous: $e);
+ }
+
+ return new DatabaseDeadLetterStore(
+ $connections,
+ $config->has('firefly.messaging.dead-letter.connection') ? $config->string('firefly.messaging.dead-letter.connection') : null,
+ );
}
}
diff --git a/packages/messaging/src/MessagingConsoleServiceProvider.php b/packages/messaging/src/MessagingConsoleServiceProvider.php
new file mode 100644
index 00000000..1c65852c
--- /dev/null
+++ b/packages/messaging/src/MessagingConsoleServiceProvider.php
@@ -0,0 +1,31 @@
+app->make('config')->get('firefly.messaging.dead-letter.store') === 'database') {
+ $this->loadMigrationsFrom(self::MIGRATIONS);
+ }
+
+ if ($this->app->runningInConsole()) {
+ $this->commands([ReplayDeadLettersCommand::class]);
+ }
+ }
+}
diff --git a/packages/messaging/tests/Console/ReplayDeadLettersCommandTest.php b/packages/messaging/tests/Console/ReplayDeadLettersCommandTest.php
new file mode 100644
index 00000000..7d475f77
--- /dev/null
+++ b/packages/messaging/tests/Console/ReplayDeadLettersCommandTest.php
@@ -0,0 +1,114 @@
+app();
+ $app->make(MessageBrokerPort::class)->publish('ledger', "entry-\x00-1", 'k');
+ $app->make(MessageBrokerPort::class)->publish('ledger', "entry-\xFF-2", 'k');
+
+ expect($app->make(DeadLetterStore::class))->toBeInstanceOf(DatabaseDeadLetterStore::class)
+ ->and(DB::table(DeadLetterSchema::TABLE)->where('topic', 'ledger.DLT')->where('original_topic', 'ledger')->count())->toBe(2);
+
+ // Dry run: lists, changes nothing.
+ expect($this->replay(['--dry-run' => true]))->toBe(0)
+ ->and($this->replayOutput())->toContain('would replay')
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+
+ // The outage is over: replay one, then the rest.
+ FlakyLedgerListener::$failing = false;
+ expect($this->replay(['--topic' => 'ledger', '--limit' => 1]))->toBe(0)
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(1)
+ ->and($this->replay(['--topic' => 'ledger.DLT']))->toBe(0)
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(0);
+
+ /** @var ListenerSpy $spy */
+ $spy = $app->make(ListenerSpy::class);
+ expect($spy->seen)->toHaveCount(2)
+ ->and($spy->seen[0])->toStartWith("entry-\x00-1#k#")
+ ->and($spy->seen[0])->not->toEndWith('#first')
+ ->and($spy->seen[1])->toStartWith("entry-\xFF-2#k#");
+});
+
+it('drops the retry headers and marks the replay with x-replayed-from', function () {
+ /** @var MessagingDeadLetterCapstoneTestCase $this */
+ $app = $this->app();
+ $app->make(MessageBrokerPort::class)->publish('ledger', 'v', null, ['trace' => 't-1']);
+ $id = DB::table(DeadLetterSchema::TABLE)->value('id');
+
+ /** @var list $seen */
+ $seen = [];
+ $app->make(MessageBrokerPort::class)->subscribe('ledger', static function (Message $message) use (&$seen): void {
+ $seen[] = $message;
+ });
+ FlakyLedgerListener::$failing = false;
+
+ expect($this->replay(['--id' => [$id]]))->toBe(0)
+ ->and($seen)->toHaveCount(1)
+ ->and($seen[0]->headers)->toBe(['trace' => 't-1', 'x-replayed-from' => $id]);
+});
+
+it('dead-letters a replay that fails again, and --keep leaves the original in place', function () {
+ /** @var MessagingDeadLetterCapstoneTestCase $this */
+ $this->app()->make(MessageBrokerPort::class)->publish('ledger', 'still-failing');
+
+ expect($this->replay(['--keep' => true]))->toBe(0)
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+});
+
+it('keeps an entry whose republish throws, and exits 1', function () {
+ /** @var MessagingDeadLetterCapstoneTestCase $this */
+ $this->app()->make(MessageBrokerPort::class)->publish('ledger', 'v');
+ $this->app()->instance(MessageBrokerPort::class, new class implements MessageBrokerPort
+ {
+ public function publish(string $topic, string $value, ?string $key = null, array $headers = []): void
+ {
+ throw new RuntimeException('broker down');
+ }
+
+ public function subscribe(string $topic, callable $handler, ?string $group = null): void {}
+
+ public function start(): void {}
+
+ public function stop(): void {}
+ });
+
+ expect($this->replay())->toBe(1)
+ ->and($this->replayOutput())->toContain('could not replay')
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(1);
+});
+
+it('refuses a bound DeadLetterStore that cannot select or remove entries, and exits 1', function () {
+ /** @var MessagingDeadLetterCapstoneTestCase $this */
+ $this->app()->instance(DeadLetterStore::class, new class implements DeadLetterStore
+ {
+ public function store(Message $message, Throwable $cause): void {}
+
+ public function all(): array
+ {
+ return [];
+ }
+ });
+
+ expect($this->replay())->toBe(1)
+ ->and($this->replayOutput())->toContain('cannot select or remove entries');
+});
+
+it('ships its migration when the store is database', function () {
+ /** @var MessagingDeadLetterCapstoneTestCase $this */
+ expect($this->app()->make('migrator')->paths())->toContain(MessagingConsoleServiceProvider::MIGRATIONS);
+});
diff --git a/packages/messaging/tests/DeadLetter/DatabaseDeadLetterStoreTest.php b/packages/messaging/tests/DeadLetter/DatabaseDeadLetterStoreTest.php
new file mode 100644
index 00000000..ad1251a0
--- /dev/null
+++ b/packages/messaging/tests/DeadLetter/DatabaseDeadLetterStoreTest.php
@@ -0,0 +1,122 @@
+up();
+});
+
+function messagingDeadLetterStore(string $now = '2026-10-12 10:00:00.500'): DatabaseDeadLetterStore
+{
+ /** @var ConnectionResolverInterface $connections */
+ $connections = App::make(ConnectionResolverInterface::class);
+
+ return new DatabaseDeadLetterStore($connections, null, static fn (): DateTimeImmutable => new DateTimeImmutable($now, new DateTimeZone('UTC')));
+}
+
+it('keeps raw bytes byte for byte — NUL, high bytes and all — with key, headers and cause', function () {
+ $bytes = "\x00\x01\xFF\xFEpayload\n\x80";
+ messagingDeadLetterStore()->store(new Message('ledger.DLT', $bytes, 'k-1', ['x-original-topic' => 'ledger', 'x-exception' => 'boom']), new LogicException('boom'));
+
+ $entry = messagingDeadLetterStore()->all()[0];
+
+ expect($entry->id)->toMatch('/^[0-9a-f-]{36}$/')
+ ->and($entry->message->value)->toBe($bytes)
+ ->and($entry->message->topic)->toBe('ledger.DLT')
+ ->and($entry->message->key)->toBe('k-1')
+ ->and($entry->message->headers)->toBe(['x-original-topic' => 'ledger', 'x-exception' => 'boom'])
+ ->and($entry->exceptionClass)->toBe(LogicException::class)
+ ->and($entry->exceptionMessage)->toBe('boom')
+ ->and($entry->deadLetteredAt?->format('Y-m-d H:i:s.v'))->toBe('2026-10-12 10:00:00.500')
+ ->and(DB::table(DeadLetterSchema::TABLE)->value('original_topic'))->toBe('ledger');
+
+ $stored = DB::table(DeadLetterSchema::TABLE)->value('value');
+ expect(is_string($stored) ? base64_decode($stored, true) : null)->toBe($bytes);
+});
+
+it('keeps a message with no key and no headers', function () {
+ messagingDeadLetterStore()->store(new Message('plain.DLT', ''), new RuntimeException('empty'));
+
+ $entry = messagingDeadLetterStore()->all()[0];
+
+ expect($entry->message->value)->toBe('')
+ ->and($entry->message->key)->toBeNull()
+ ->and($entry->message->headers)->toBe([])
+ ->and(DB::table(DeadLetterSchema::TABLE)->value('original_topic'))->toBeNull();
+});
+
+it('selects by id and by either topic, oldest first, and removes one entry', function () {
+ messagingDeadLetterStore('2026-10-12 10:00:02')->store(new Message('ledger.DLT', 'second', null, ['x-original-topic' => 'ledger']), new RuntimeException('b'));
+ messagingDeadLetterStore('2026-10-12 10:00:01')->store(new Message('ledger.DLT', 'first', null, ['x-original-topic' => 'ledger']), new RuntimeException('a'));
+ messagingDeadLetterStore('2026-10-12 10:00:03')->store(new Message('audit.DLT', 'third', null, ['x-original-topic' => 'audit']), new RuntimeException('c'));
+ $store = messagingDeadLetterStore();
+
+ $ledger = $store->find(new DeadLetterQuery(topic: 'ledger'));
+
+ expect(array_map(static fn (DeadLetterEntry $e): string => $e->message->value, $ledger))->toBe(['first', 'second'])
+ ->and($store->find(new DeadLetterQuery(topic: 'ledger.DLT')))->toHaveCount(2)
+ ->and($store->find(new DeadLetterQuery(topic: 'audit.DLT')))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(limit: 2)))->toHaveCount(2)
+ ->and($store->find(new DeadLetterQuery(ids: [(string) $ledger[1]->id]))[0]->message->value)->toBe('second')
+ ->and($store->remove((string) $ledger[0]->id))->toBeTrue()
+ ->and($store->remove((string) $ledger[0]->id))->toBeFalse()
+ ->and(DB::table(DeadLetterSchema::TABLE)->count())->toBe(2);
+});
+
+it('keeps the message when the cause carries raw bytes: the description is made valid UTF-8, the value is not touched', function () {
+ $bytes = "\xC3\x28raw";
+ messagingDeadLetterStore()->store(new Message('ledger.DLT', $bytes, null, ['x-original-topic' => 'ledger', 'x-exception' => "bad \xC3\x28 input"]), new RuntimeException("bad \xC3\x28 input"));
+
+ $entry = messagingDeadLetterStore()->all()[0];
+
+ expect($entry->message->value)->toBe($bytes)
+ ->and(mb_check_encoding($entry->exceptionMessage, 'UTF-8'))->toBeTrue()
+ ->and(mb_check_encoding($entry->message->headers['x-exception'] ?? '', 'UTF-8'))->toBeTrue()
+ ->and($entry->message->headers['x-original-topic'] ?? null)->toBe('ledger');
+});
+
+it('keeps an opaque key byte for byte — longer than a varchar, and not valid UTF-8 — stored as base64 text', function (string $key) {
+ messagingDeadLetterStore()->store(new Message('ledger.DLT', 'v', $key, ['x-original-topic' => 'ledger']), new RuntimeException('boom'));
+
+ $entry = messagingDeadLetterStore()->all()[0];
+ $stored = DB::table(DeadLetterSchema::TABLE)->value('message_key');
+
+ expect($entry->message->key)->toBe($key)
+ ->and(is_string($stored) ? base64_decode($stored, true) : null)->toBe($key)
+ ->and(is_string($stored) && mb_check_encoding($stored, 'ASCII'))->toBeTrue();
+})->with([
+ 'a 300-byte key' => [str_repeat('k', 300)],
+ 'a non-UTF-8 key' => ["\xFF\x00k"],
+ 'a raw 16-byte UUID' => [hex2bin('0192f5a1c3d47e8f9a0b1c2d3e4f5061')],
+ 'an empty key' => [''],
+]);
+
+it('keeps insertion order for dead letters written in the same millisecond, so --limit replays the oldest', function () {
+ $store = messagingDeadLetterStore('2026-10-12 10:00:00.500');
+ $values = array_map(static fn (int $i): string => "same-instant-{$i}", range(1, 40));
+ foreach ($values as $value) {
+ $store->store(new Message('ledger.DLT', $value, null, ['x-original-topic' => 'ledger']), new RuntimeException('tie'));
+ }
+
+ expect(array_map(static fn (DeadLetterEntry $e): string => $e->message->value, $store->all()))->toBe($values)
+ ->and($store->find(new DeadLetterQuery(topic: 'ledger', limit: 1))[0]->message->value)->toBe('same-instant-1');
+});
diff --git a/packages/messaging/tests/DeadLetter/DeadLetterQueryTest.php b/packages/messaging/tests/DeadLetter/DeadLetterQueryTest.php
new file mode 100644
index 00000000..a6b3e4d0
--- /dev/null
+++ b/packages/messaging/tests/DeadLetter/DeadLetterQueryTest.php
@@ -0,0 +1,26 @@
+store(new Message('ledger.DLT', 'a', null, ['x-original-topic' => 'ledger']), new RuntimeException('a'));
+ $store->store(new Message('audit.DLT', 'b', null, ['x-original-topic' => 'audit']), new RuntimeException('b'));
+
+ [$first] = $store->all();
+
+ expect($first->id)->toMatch('/^[0-9a-f-]{36}$/')
+ ->and($first->deadLetteredAt)->toBeInstanceOf(DateTimeImmutable::class)
+ ->and($store->find(new DeadLetterQuery(topic: 'ledger')))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(topic: 'audit.DLT')))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(topic: 'nowhere')))->toBe([])
+ ->and($store->find(new DeadLetterQuery(limit: 1)))->toHaveCount(1)
+ ->and($store->find(new DeadLetterQuery(ids: [(string) $first->id])))->toHaveCount(1)
+ ->and($store->remove((string) $first->id))->toBeTrue()
+ ->and($store->remove((string) $first->id))->toBeFalse()
+ ->and($store->all())->toHaveCount(1);
+});
diff --git a/packages/messaging/tests/DeadLetter/DeadLetterStoreSelectionTest.php b/packages/messaging/tests/DeadLetter/DeadLetterStoreSelectionTest.php
new file mode 100644
index 00000000..dff380fd
--- /dev/null
+++ b/packages/messaging/tests/DeadLetter/DeadLetterStoreSelectionTest.php
@@ -0,0 +1,63 @@
+connection;
+ }
+
+ public function getDefaultConnection(): string
+ {
+ return 'default';
+ }
+
+ public function setDefaultConnection($name): void {}
+ };
+}
+
+/** A container holding what a booted app holds: the Config, and the connection resolver unless $withDatabase is false. */
+function messagingDeadLetterContainer(?string $store, bool $withDatabase = true): Container
+{
+ $container = new Container;
+ $container->instance(Config::class, new Config(new Repository(['firefly' => ['messaging' => $store === null ? [] : ['dead-letter' => ['store' => $store]]]])));
+ if ($withDatabase) {
+ $container->instance(ConnectionResolverInterface::class, messagingDeadLetterResolver());
+ }
+
+ return $container;
+}
+
+it('selects the messaging dead-letter store by firefly.messaging.dead-letter.store', function () {
+ expect((new MessagingAutoConfiguration)->deadLetterStore())->toBeInstanceOf(InMemoryDeadLetterStore::class)
+ ->and((new MessagingAutoConfiguration)->deadLetterStore(messagingDeadLetterContainer(null)))->toBeInstanceOf(InMemoryDeadLetterStore::class)
+ ->and((new MessagingAutoConfiguration)->deadLetterStore(messagingDeadLetterContainer('memory')))->toBeInstanceOf(InMemoryDeadLetterStore::class)
+ ->and((new MessagingAutoConfiguration)->deadLetterStore(messagingDeadLetterContainer('database')))->toBeInstanceOf(DatabaseDeadLetterStore::class)
+ ->and(fn () => (new MessagingAutoConfiguration)->deadLetterStore(messagingDeadLetterContainer('database', withDatabase: false)))->toThrow(ConfigurationException::class, 'firefly.messaging.dead-letter.store=database')
+ ->and(fn () => (new MessagingAutoConfiguration)->deadLetterStore(messagingDeadLetterContainer('kafka')))->toThrow(ConfigurationException::class, 'memory|database');
+});
+
+it('boots an app without a database on the memory store: the resolver is made only when database is chosen', function () {
+ $context = bootFireflyApp(['firefly' => ['messaging' => []]], [MessagingServiceProvider::class, MessagingWiringProvider::class]);
+
+ expect($context->get(DeadLetterStore::class))->toBeInstanceOf(InMemoryDeadLetterStore::class);
+});
diff --git a/packages/messaging/tests/DeadLetterFixtures/FlakyLedgerListener.php b/packages/messaging/tests/DeadLetterFixtures/FlakyLedgerListener.php
new file mode 100644
index 00000000..d520438b
--- /dev/null
+++ b/packages/messaging/tests/DeadLetterFixtures/FlakyLedgerListener.php
@@ -0,0 +1,28 @@
+spy->record($message->value.'#'.($message->key ?? '-').'#'.($message->headers['x-replayed-from'] ?? 'first'));
+ }
+}
diff --git a/packages/messaging/tests/Integration/DatabaseDeadLetterStoreIntegrationTest.php b/packages/messaging/tests/Integration/DatabaseDeadLetterStoreIntegrationTest.php
new file mode 100644
index 00000000..3d1bf196
--- /dev/null
+++ b/packages/messaging/tests/Integration/DatabaseDeadLetterStoreIntegrationTest.php
@@ -0,0 +1,75 @@
+down() : $migration->up();
+}
+
+foreach (IntegrationDsn::lanes() as $lane) {
+ it("keeps raw bytes and selects messaging dead letters on {$lane}", function () use ($lane): void {
+ config(['database.connections.it_dlq' => IntegrationDsn::connectionConfig($lane, $lane === 'sqlite' ? ['database' => IntegrationDsn::sqlitePath('messaging-dlq')] : [])]);
+ DB::setDefaultConnection('it_dlq');
+ messagingDeadLetterMigrate('down');
+ messagingDeadLetterMigrate('up');
+
+ try {
+ $connections = App::make(ConnectionResolverInterface::class);
+ $store = new DatabaseDeadLetterStore($connections, 'it_dlq');
+ $bytes = random_bytes(512)."\x00\xFF\xC3\x28";
+
+ // An opaque key: longer than any varchar, and not valid UTF-8 (a raw UUID, a hash, an Avro key).
+ $key = str_repeat('k', 300)."\xFF\x00k";
+
+ $store->store(new Message('ledger.DLT', $bytes, $key, ['x-original-topic' => 'ledger', 'x-exception' => 'boom']), new RuntimeException('boom'));
+ $store->store(new Message('audit.DLT', 'plain', null, ['x-original-topic' => 'audit']), new RuntimeException('x'));
+
+ $ledger = $store->find(new DeadLetterQuery(topic: 'ledger'));
+
+ expect($ledger)->toHaveCount(1)
+ ->and($ledger[0]->message->value)->toBe($bytes)
+ ->and($ledger[0]->message->key)->toBe($key)
+ ->and($ledger[0]->message->topic)->toBe('ledger.DLT')
+ ->and($store->find(new DeadLetterQuery(topic: 'audit.DLT'))[0]->message->key ?? 'missing')->toBe('missing')
+ ->and($store->remove((string) $ledger[0]->id))->toBeTrue()
+ ->and($store->all())->toHaveCount(1);
+
+ // A tie on the instant: the time-ordered id keeps insertion order under this server's collation.
+ $tied = new DatabaseDeadLetterStore($connections, 'it_dlq', static fn (): DateTimeImmutable => new DateTimeImmutable('2026-10-12 10:00:00.000', new DateTimeZone('UTC')));
+ $values = array_map(static fn (int $i): string => "tie-{$i}", range(1, 20));
+ foreach ($values as $value) {
+ $tied->store(new Message('ties.DLT', $value, null, ['x-original-topic' => 'ties']), new RuntimeException('tie'));
+ }
+ expect(array_map(static fn (DeadLetterEntry $entry): string => $entry->message->value, $store->find(new DeadLetterQuery(topic: 'ties'))))->toBe($values);
+ } finally {
+ messagingDeadLetterMigrate('down');
+ DB::connection('it_dlq')->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/messaging/tests/Support/MessagingDeadLetterCapstoneTestCase.php b/packages/messaging/tests/Support/MessagingDeadLetterCapstoneTestCase.php
new file mode 100644
index 00000000..29a5d46d
--- /dev/null
+++ b/packages/messaging/tests/Support/MessagingDeadLetterCapstoneTestCase.php
@@ -0,0 +1,73 @@
+ */
+ protected function fireflyProviders(): array
+ {
+ return [MessagingServiceProvider::class, MessagingWiringProvider::class, MessagingConsoleServiceProvider::class];
+ }
+
+ /** @return array */
+ protected function configOverrides(): array
+ {
+ return ['firefly.messaging.provider' => 'memory', 'firefly.messaging.dead-letter.store' => 'database'];
+ }
+
+ protected function defineFireflyEnvironment(Application $app): void
+ {
+ FlakyLedgerListener::$failing = true;
+ $app->instance(MessageListenerManifest::class, new MessageListenerManifest(
+ (new MessageListenerScanner)->scan(['Firefly\\Messaging\\Tests\\DeadLetterFixtures\\' => dirname(__DIR__).'/DeadLetterFixtures']),
+ ));
+ $app->singleton(ListenerSpy::class);
+ }
+
+ protected function setUp(): void
+ {
+ parent::setUp();
+
+ Schema::create(DeadLetterSchema::TABLE, fn (Blueprint $table) => DeadLetterSchema::blueprint($table));
+ }
+
+ /** @param array $options */
+ public function replay(array $options = []): int
+ {
+ /** @var Kernel $kernel */
+ $kernel = $this->app()->make(Kernel::class);
+
+ return $kernel->call('firefly:messaging:dead-letters:replay', $options);
+ }
+
+ public function replayOutput(): string
+ {
+ /** @var Kernel $kernel */
+ $kernel = $this->app()->make(Kernel::class);
+
+ return $kernel->output();
+ }
+}
diff --git a/packages/scheduling-database/.gitattributes b/packages/scheduling-database/.gitattributes
new file mode 100644
index 00000000..538b69a4
--- /dev/null
+++ b/packages/scheduling-database/.gitattributes
@@ -0,0 +1,2 @@
+/tests export-ignore
+/.gitattributes export-ignore
diff --git a/packages/scheduling-database/LICENSE b/packages/scheduling-database/LICENSE
new file mode 100644
index 00000000..2240005d
--- /dev/null
+++ b/packages/scheduling-database/LICENSE
@@ -0,0 +1,204 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+Copyright 2026 Firefly Software Solutions Inc.
diff --git a/packages/scheduling-database/README.md b/packages/scheduling-database/README.md
new file mode 100644
index 00000000..43876568
--- /dev/null
+++ b/packages/scheduling-database/README.md
@@ -0,0 +1,10 @@
+# firefly/scheduling-database
+
+LaraFly's database-backed `DistributedLock` providers for `#[Scheduled(lock: …)]`: `DatabaseLeaseLock`
+(`firefly.scheduling.lock.provider=database`), a lease row per lock name in `firefly_scheduling_locks` on any Laravel
+database connection, honouring the TTL on every driver; and `MySqlNamedLock` (`firefly.scheduling.lock.provider=mysql`),
+MySQL and MariaDB `GET_LOCK`/`RELEASE_LOCK` named locks held for the connection's session. Installing the package is
+inert until one of the two providers is selected. See
+[docs/modules/scheduling-database.md](../../docs/modules/scheduling-database.md).
+
+Apache-2.0 © Firefly Software Solutions Inc.
diff --git a/packages/scheduling-database/cache/firefly-scheduling-database-components.php b/packages/scheduling-database/cache/firefly-scheduling-database-components.php
new file mode 100644
index 00000000..35cf26cc
--- /dev/null
+++ b/packages/scheduling-database/cache/firefly-scheduling-database-components.php
@@ -0,0 +1,50 @@
+ [
+ 'class' => 'Firefly\\Scheduling\\Database\\SchedulingDatabaseAutoConfiguration',
+ 'stereotype' => 'configuration',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 900,
+ 'qualifier' => null,
+ 'interfaces' => [
+ ],
+ 'beans' => [
+ 0 => [
+ 'method' => 'databaseLeaseLock',
+ 'returns' => 'Firefly\\Scheduling\\Lock\\DistributedLock',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Firefly\\Config\\Config',
+ 1 => 'Illuminate\\Database\\ConnectionResolverInterface',
+ ],
+ ],
+ 1 => [
+ 'method' => 'mySqlNamedLock',
+ 'returns' => 'Firefly\\Scheduling\\Lock\\DistributedLock',
+ 'name' => null,
+ 'scope' => 'Singleton',
+ 'primary' => false,
+ 'order' => 0,
+ 'lazy' => false,
+ 'dependencies' => [
+ 0 => 'Firefly\\Config\\Config',
+ 1 => 'Illuminate\\Database\\ConnectionResolverInterface',
+ ],
+ ],
+ ],
+ 'lazy' => false,
+ 'dependencies' => [
+ ],
+ ],
+];
diff --git a/packages/scheduling-database/cache/firefly-scheduling-database-context.php b/packages/scheduling-database/cache/firefly-scheduling-database-context.php
new file mode 100644
index 00000000..b793ff87
--- /dev/null
+++ b/packages/scheduling-database/cache/firefly-scheduling-database-context.php
@@ -0,0 +1,47 @@
+ [
+ 'class' => 'Firefly\\Scheduling\\Database\\SchedulingDatabaseAutoConfiguration',
+ 'postConstruct' => [
+ ],
+ 'preDestroy' => [
+ ],
+ 'listeners' => [
+ ],
+ 'conditions' => [
+ ],
+ 'beanConditions' => [
+ 0 => [
+ 'method' => 'databaseLeaseLock',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.scheduling.lock.provider',
+ 1 => 'database',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ 1 => [
+ 'method' => 'mySqlNamedLock',
+ 'conditions' => [
+ 0 => [
+ 'type' => 'Firefly\\Context\\Condition\\Attributes\\ConditionalOnProperty',
+ 'args' => [
+ 0 => 'firefly.scheduling.lock.provider',
+ 1 => 'mysql',
+ 2 => false,
+ ],
+ ],
+ ],
+ ],
+ ],
+ ],
+];
diff --git a/packages/scheduling-database/composer.json b/packages/scheduling-database/composer.json
new file mode 100644
index 00000000..029607a3
--- /dev/null
+++ b/packages/scheduling-database/composer.json
@@ -0,0 +1,60 @@
+{
+ "name": "firefly/scheduling-database",
+ "description": "LaraFly scheduling database adapters: DatabaseLeaseLock, a DistributedLock over a portable firefly_scheduling_locks lease table on any Laravel database connection (firefly.scheduling.lock.provider=database), and MySqlNamedLock over GET_LOCK/RELEASE_LOCK for MySQL and MariaDB (firefly.scheduling.lock.provider=mysql).",
+ "type": "library",
+ "license": "Apache-2.0",
+ "homepage": "https://github.com/fireflyframework/fireflyframework-php",
+ "authors": [
+ {
+ "name": "Firefly Software Solutions Inc.",
+ "homepage": "https://github.com/fireflyframework"
+ }
+ ],
+ "keywords": [
+ "firefly",
+ "laravel",
+ "scheduling",
+ "database"
+ ],
+ "support": {
+ "issues": "https://github.com/fireflyframework/fireflyframework-php/issues",
+ "source": "https://github.com/fireflyframework/fireflyframework-php/tree/main/packages/scheduling-database"
+ },
+ "require": {
+ "php": "^8.3",
+ "firefly/autoconfigure": "*@dev",
+ "firefly/config": "*@dev",
+ "firefly/container": "*@dev",
+ "firefly/context": "*@dev",
+ "firefly/kernel": "*@dev",
+ "firefly/scheduling": "*@dev",
+ "illuminate/contracts": "^13.0",
+ "illuminate/database": "^13.0",
+ "illuminate/support": "^13.0"
+ },
+ "extra": {
+ "laravel": {
+ "providers": [
+ "Firefly\\Scheduling\\Database\\SchedulingDatabaseServiceProvider",
+ "Firefly\\Scheduling\\Database\\SchedulingDatabaseBootServiceProvider"
+ ]
+ },
+ "branch-alias": {
+ "dev-main": "26.x-dev"
+ }
+ },
+ "autoload": {
+ "psr-4": {
+ "Firefly\\Scheduling\\Database\\": "src/"
+ }
+ },
+ "autoload-dev": {
+ "psr-4": {
+ "Firefly\\Scheduling\\Database\\Tests\\": "tests/"
+ }
+ },
+ "minimum-stability": "stable",
+ "config": {
+ "sort-packages": true
+ }
+}
diff --git a/packages/scheduling-database/database/migrations/2026_10_12_000100_create_firefly_scheduling_locks_table.php b/packages/scheduling-database/database/migrations/2026_10_12_000100_create_firefly_scheduling_locks_table.php
new file mode 100644
index 00000000..ceec46d7
--- /dev/null
+++ b/packages/scheduling-database/database/migrations/2026_10_12_000100_create_firefly_scheduling_locks_table.php
@@ -0,0 +1,21 @@
+ LockSchema::blueprint($table));
+ }
+
+ public function down(): void
+ {
+ Schema::dropIfExists(LockSchema::TABLE);
+ }
+};
diff --git a/packages/scheduling-database/src/DatabaseLeaseLock.php b/packages/scheduling-database/src/DatabaseLeaseLock.php
new file mode 100644
index 00000000..c18843e1
--- /dev/null
+++ b/packages/scheduling-database/src/DatabaseLeaseLock.php
@@ -0,0 +1,122 @@
+ name => the locked_by token of this instance's live acquisition */
+ private array $held = [];
+
+ /**
+ * @param (Closure(): DateTimeImmutable)|null $clock
+ */
+ public function __construct(
+ private readonly ConnectionResolverInterface $connections,
+ private readonly ?string $connection = null,
+ ?string $owner = null,
+ private readonly ?Closure $clock = null,
+ ) {
+ $host = gethostname();
+ $pid = getmypid();
+ $this->owner = substr($owner ?? ((is_string($host) && $host !== '' ? $host : 'host').':'.($pid === false ? 0 : $pid)), 0, 200);
+ }
+
+ public function owner(): string
+ {
+ return $this->owner;
+ }
+
+ public function tryAcquire(string $name, float $ttlSeconds): bool
+ {
+ if (strlen($name) > self::MAX_NAME_LENGTH) {
+ throw new InvalidArgumentException(sprintf('A lock name is at most %d characters; got %d.', self::MAX_NAME_LENGTH, strlen($name)));
+ }
+
+ $db = $this->db();
+ $now = ($this->clock === null ? new DateTimeImmutable : ($this->clock)())->setTimezone(new DateTimeZone('UTC'));
+ $token = $this->owner.'/'.bin2hex(random_bytes(4));
+ $lease = [
+ 'locked_by' => $token,
+ 'locked_until' => self::format(self::plus($now, $ttlSeconds)),
+ 'locked_at' => self::format($now),
+ ];
+
+ $taken = $db->table(LockSchema::TABLE)
+ ->where('name', $name)
+ ->where('locked_until', '<=', self::format($now))
+ ->update($lease);
+
+ if ($taken !== 1) {
+ try {
+ $db->transaction(static fn () => $db->table(LockSchema::TABLE)->insert(['name' => $name, ...$lease]));
+ } catch (UniqueConstraintViolationException) {
+ return false;
+ }
+ }
+
+ $this->held[$name] = $token;
+
+ return true;
+ }
+
+ public function release(string $name): void
+ {
+ $token = $this->held[$name] ?? null;
+ if ($token === null) {
+ return;
+ }
+
+ unset($this->held[$name]);
+ $this->db()->table(LockSchema::TABLE)->where('name', $name)->where('locked_by', $token)->delete();
+ }
+
+ private function db(): ConnectionInterface
+ {
+ return $this->connections->connection($this->connection);
+ }
+
+ private static function plus(DateTimeImmutable $instant, float $seconds): DateTimeImmutable
+ {
+ $milliseconds = max(1, (int) round($seconds * 1000));
+ $interval = new DateInterval('PT'.intdiv($milliseconds, 1000).'S');
+ $interval->f = ($milliseconds % 1000) / 1000;
+
+ return $instant->add($interval);
+ }
+
+ private static function format(DateTimeImmutable $instant): string
+ {
+ return $instant->format('Y-m-d H:i:s.v');
+ }
+}
diff --git a/packages/scheduling-database/src/LockSchema.php b/packages/scheduling-database/src/LockSchema.php
new file mode 100644
index 00000000..e6507652
--- /dev/null
+++ b/packages/scheduling-database/src/LockSchema.php
@@ -0,0 +1,24 @@
+string('name')->primary();
+ $table->string('locked_by');
+ $table->dateTime('locked_until', 3);
+ $table->dateTime('locked_at', 3);
+ }
+}
diff --git a/packages/scheduling-database/src/MySqlNamedLock.php b/packages/scheduling-database/src/MySqlNamedLock.php
new file mode 100644
index 00000000..25327b3d
--- /dev/null
+++ b/packages/scheduling-database/src/MySqlNamedLock.php
@@ -0,0 +1,63 @@
+` (48 characters), deterministic across nodes.
+ * GET_LOCK answers 1 (acquired), 0 (held elsewhere) or NULL (an error): only 1 is an acquisition.
+ */
+final class MySqlNamedLock implements DistributedLock
+{
+ public const int MAX_NAME_LENGTH = 64;
+
+ public function __construct(
+ private readonly ConnectionResolverInterface $connections,
+ private readonly ?string $connection = null,
+ ) {}
+
+ public static function key(string $name): string
+ {
+ return strlen($name) <= self::MAX_NAME_LENGTH ? $name : 'firefly:'.sha1($name);
+ }
+
+ public function tryAcquire(string $name, float $ttlSeconds): bool
+ {
+ $row = $this->db()->selectOne('select get_lock(?, 0) as locked', [self::key($name)]);
+ $locked = is_object($row) ? (get_object_vars($row)['locked'] ?? null) : null;
+
+ return is_numeric($locked) && (int) $locked === 1;
+ }
+
+ public function release(string $name): void
+ {
+ $this->db()->selectOne('select release_lock(?) as released', [self::key($name)]);
+ }
+
+ private function db(): Connection
+ {
+ $connection = $this->connections->connection($this->connection);
+
+ if (! $connection instanceof Connection || ! in_array($connection->getDriverName(), ['mysql', 'mariadb'], true)) {
+ throw new ConfigurationException(sprintf(
+ 'firefly.scheduling.lock.provider=mysql needs a mysql or mariadb connection; "%s" is %s. Set '
+ .'firefly.scheduling.lock.mysql.connection, or use provider=database, which works on every driver.',
+ $this->connection ?? 'the default connection',
+ $connection instanceof Connection ? $connection->getDriverName() : get_debug_type($connection),
+ ));
+ }
+
+ return $connection;
+ }
+}
diff --git a/packages/scheduling-database/src/SchedulingDatabaseAutoConfiguration.php b/packages/scheduling-database/src/SchedulingDatabaseAutoConfiguration.php
new file mode 100644
index 00000000..086a4db5
--- /dev/null
+++ b/packages/scheduling-database/src/SchedulingDatabaseAutoConfiguration.php
@@ -0,0 +1,49 @@
+get('firefly.scheduling.lock.database.connection') === null
+ ? null
+ : $config->string('firefly.scheduling.lock.database.connection');
+
+ return new DatabaseLeaseLock($connections, $connection === '' ? null : $connection);
+ }
+
+ #[Bean]
+ #[ConditionalOnProperty(name: 'firefly.scheduling.lock.provider', havingValue: 'mysql')]
+ public function mySqlNamedLock(Config $config, ConnectionResolverInterface $connections): DistributedLock
+ {
+ // The same null-or-blank-means-default rule as databaseLeaseLock(): an unset env() must not fail the boot.
+ $connection = $config->get('firefly.scheduling.lock.mysql.connection') === null
+ ? null
+ : $config->string('firefly.scheduling.lock.mysql.connection');
+
+ return new MySqlNamedLock($connections, $connection === '' ? null : $connection);
+ }
+}
diff --git a/packages/scheduling-database/src/SchedulingDatabaseBootServiceProvider.php b/packages/scheduling-database/src/SchedulingDatabaseBootServiceProvider.php
new file mode 100644
index 00000000..dd4c7b92
--- /dev/null
+++ b/packages/scheduling-database/src/SchedulingDatabaseBootServiceProvider.php
@@ -0,0 +1,20 @@
+app->make('config')->get('firefly.scheduling.lock.provider') === 'database') {
+ $this->loadMigrationsFrom(self::MIGRATIONS);
+ }
+ }
+}
diff --git a/packages/scheduling-database/src/SchedulingDatabaseServiceProvider.php b/packages/scheduling-database/src/SchedulingDatabaseServiceProvider.php
new file mode 100644
index 00000000..b08f8b09
--- /dev/null
+++ b/packages/scheduling-database/src/SchedulingDatabaseServiceProvider.php
@@ -0,0 +1,26 @@
+ dirname(__DIR__).'/src'];
+ $cache = dirname(__DIR__).'/cache';
+
+ $freshComponents = sys_get_temp_dir().'/firefly-scheduling-database-fresh-c-'.bin2hex(random_bytes(6)).'.php';
+ $freshContext = sys_get_temp_dir().'/firefly-scheduling-database-fresh-x-'.bin2hex(random_bytes(6)).'.php';
+
+ try {
+ (new AutoConfigManifestCompiler)->write($src, $freshComponents, $freshContext);
+
+ expect(ComponentManifest::load($freshComponents))->toEqual(ComponentManifest::load($cache.'/firefly-scheduling-database-components.php'))
+ ->and(ContextManifest::load($freshContext))->toEqual(ContextManifest::load($cache.'/firefly-scheduling-database-context.php'));
+ } finally {
+ @unlink($freshComponents);
+ @unlink($freshContext);
+ }
+});
diff --git a/packages/scheduling-database/tests/DatabaseLeaseLockTest.php b/packages/scheduling-database/tests/DatabaseLeaseLockTest.php
new file mode 100644
index 00000000..ccef8a2c
--- /dev/null
+++ b/packages/scheduling-database/tests/DatabaseLeaseLockTest.php
@@ -0,0 +1,114 @@
+up();
+});
+
+/** A lock instance ("node") named $owner whose clock reads $now. */
+function leaseLockAt(string $owner, string $now): DatabaseLeaseLock
+{
+ /** @var ConnectionResolverInterface $connections */
+ $connections = App::make(ConnectionResolverInterface::class);
+
+ return new DatabaseLeaseLock($connections, null, $owner, static fn (): DateTimeImmutable => new DateTimeImmutable($now, new DateTimeZone('UTC')));
+}
+
+it('grants a name to one node at a time and frees it on release', function () {
+ $a = leaseLockAt('node-a', '2026-10-12 10:00:00');
+ $b = leaseLockAt('node-b', '2026-10-12 10:00:01');
+
+ expect($a->tryAcquire('reconcile', 30))->toBeTrue()
+ ->and($b->tryAcquire('reconcile', 30))->toBeFalse()
+ ->and($b->tryAcquire('other-job', 30))->toBeTrue()
+ ->and(DB::table(LockSchema::TABLE)->where('name', 'reconcile')->value('locked_by'))->toStartWith('node-a/');
+
+ $a->release('reconcile');
+
+ expect(DB::table(LockSchema::TABLE)->where('name', 'reconcile')->exists())->toBeFalse()
+ ->and($b->tryAcquire('reconcile', 30))->toBeTrue();
+});
+
+it('hands an ended lease to the next node, and the late holder cannot release it', function () {
+ $a = leaseLockAt('node-a', '2026-10-12 10:00:00');
+ expect($a->tryAcquire('reconcile', 1.5))->toBeTrue();
+
+ expect(leaseLockAt('node-b', '2026-10-12 10:00:01.499')->tryAcquire('reconcile', 30))->toBeFalse();
+
+ $b = leaseLockAt('node-b', '2026-10-12 10:00:01.500');
+ expect($b->tryAcquire('reconcile', 30))->toBeTrue()
+ ->and(DB::table(LockSchema::TABLE)->where('name', 'reconcile')->value('locked_until'))->toBe('2026-10-12 10:00:31.500');
+
+ // a's job finally ends: its release must not drop b's lease.
+ $a->release('reconcile');
+ expect(DB::table(LockSchema::TABLE)->where('name', 'reconcile')->value('locked_by'))->toStartWith('node-b/');
+});
+
+it('refuses a name longer than the column instead of truncating it', function () {
+ expect(fn () => leaseLockAt('a', '2026-10-12 10:00:00')->tryAcquire(str_repeat('x', 256), 1))->toThrow(InvalidArgumentException::class, '255');
+});
+
+it('keeps a caller transaction usable when the lock is already held', function () {
+ leaseLockAt('node-a', '2026-10-12 10:00:00')->tryAcquire('reconcile', 30);
+
+ DB::transaction(function (): void {
+ expect(leaseLockAt('node-b', '2026-10-12 10:00:01')->tryAcquire('reconcile', 30))->toBeFalse()
+ ->and(DB::selectOne('select 1 as one'))->not->toBeNull();
+ });
+});
+
+it('releasing a name it does not hold is a no-op', function () {
+ leaseLockAt('node-a', '2026-10-12 10:00:00')->release('never-acquired');
+
+ expect(DB::table(LockSchema::TABLE)->count())->toBe(0);
+});
+
+it('the bean uses the default connection when the connection key is unset, null or blank', function (array $lock) {
+ /** @var ConnectionResolverInterface $connections */
+ $connections = App::make(ConnectionResolverInterface::class);
+ $config = new Config(new Repository(['firefly' => ['scheduling' => ['lock' => ['provider' => 'database', ...$lock]]]]));
+
+ $bean = (new SchedulingDatabaseAutoConfiguration)->databaseLeaseLock($config, $connections);
+
+ expect($bean)->toBeInstanceOf(DatabaseLeaseLock::class)
+ ->and($bean->tryAcquire('reconcile', 30))->toBeTrue()
+ ->and(DB::table(LockSchema::TABLE)->where('name', 'reconcile')->exists())->toBeTrue();
+})->with([
+ 'unset' => [[]],
+ 'null' => [['database' => ['connection' => null]]],
+ 'blank' => [['database' => ['connection' => '']]],
+]);
+
+it('the bean takes the lease on the connection the key names', function () {
+ config()->set('database.connections.locks', ['driver' => 'sqlite', 'database' => ':memory:', 'prefix' => '']);
+ Schema::connection('locks')->create(LockSchema::TABLE, fn (Blueprint $table) => LockSchema::blueprint($table));
+ /** @var ConnectionResolverInterface $connections */
+ $connections = App::make(ConnectionResolverInterface::class);
+ $config = new Config(new Repository(['firefly' => ['scheduling' => ['lock' => ['provider' => 'database', 'database' => ['connection' => 'locks']]]]]));
+
+ expect((new SchedulingDatabaseAutoConfiguration)->databaseLeaseLock($config, $connections)->tryAcquire('reconcile', 30))->toBeTrue()
+ ->and(DB::connection('locks')->table(LockSchema::TABLE)->count())->toBe(1)
+ ->and(DB::table(LockSchema::TABLE)->count())->toBe(0);
+});
diff --git a/packages/scheduling-database/tests/Integration/DatabaseLeaseLockIntegrationTest.php b/packages/scheduling-database/tests/Integration/DatabaseLeaseLockIntegrationTest.php
new file mode 100644
index 00000000..7056622e
--- /dev/null
+++ b/packages/scheduling-database/tests/Integration/DatabaseLeaseLockIntegrationTest.php
@@ -0,0 +1,76 @@
+ new DateTimeImmutable($now, new DateTimeZone('UTC')));
+}
+
+/** Runs one direction of the package's anonymous lease-table migration, on the default connection. */
+function leaseIntegrationMigrate(string $direction): void
+{
+ $migration = require dirname(__DIR__, 2).'/database/migrations/2026_10_12_000100_create_firefly_scheduling_locks_table.php';
+
+ if (! $migration instanceof Migration || ! method_exists($migration, 'up') || ! method_exists($migration, 'down')) {
+ throw new RuntimeException('Expected an executable migration.');
+ }
+
+ $direction === 'down' ? $migration->down() : $migration->up();
+}
+
+foreach (IntegrationDsn::lanes() as $lane) {
+ it("excludes, expires and hands over leases on {$lane}", function () use ($lane): void {
+ $overrides = $lane === 'sqlite' ? ['database' => IntegrationDsn::sqlitePath('lease-lock')] : [];
+ config([
+ 'database.connections.node_a' => IntegrationDsn::connectionConfig($lane, $overrides),
+ 'database.connections.node_b' => IntegrationDsn::connectionConfig($lane, $overrides),
+ ]);
+ DB::setDefaultConnection('node_a');
+ leaseIntegrationMigrate('down');
+ leaseIntegrationMigrate('up');
+
+ try {
+ $a = leaseIntegrationLock('node_a', 'node-a', '2026-10-12 10:00:00');
+ expect($a->tryAcquire('reconcile', 2))->toBeTrue()
+ ->and(leaseIntegrationLock('node_b', 'node-b', '2026-10-12 10:00:01.999')->tryAcquire('reconcile', 30))->toBeFalse();
+
+ DB::connection('node_b')->transaction(function (): void {
+ expect(leaseIntegrationLock('node_b', 'node-b', '2026-10-12 10:00:01')->tryAcquire('reconcile', 30))->toBeFalse()
+ ->and(DB::connection('node_b')->selectOne('select 1 as one'))->not->toBeNull();
+ });
+
+ $b = leaseIntegrationLock('node_b', 'node-b', '2026-10-12 10:00:02');
+ expect($b->tryAcquire('reconcile', 30))->toBeTrue();
+ $a->release('reconcile');
+ expect(DB::connection('node_b')->table(LockSchema::TABLE)->where('name', 'reconcile')->value('locked_by'))->toStartWith('node-b/');
+
+ $b->release('reconcile');
+ expect(DB::connection('node_a')->table(LockSchema::TABLE)->count())->toBe(0)
+ ->and(leaseIntegrationLock('node_a', 'node-a', '2026-10-12 10:00:03')->tryAcquire('reconcile', 30))->toBeTrue();
+ } finally {
+ leaseIntegrationMigrate('down');
+ DB::connection('node_a')->disconnect();
+ DB::connection('node_b')->disconnect();
+ }
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/scheduling-database/tests/Integration/MySqlNamedLockIntegrationTest.php b/packages/scheduling-database/tests/Integration/MySqlNamedLockIntegrationTest.php
new file mode 100644
index 00000000..71a39960
--- /dev/null
+++ b/packages/scheduling-database/tests/Integration/MySqlNamedLockIntegrationTest.php
@@ -0,0 +1,52 @@
+ IntegrationDsn::connectionConfig($lane),
+ 'database.connections.session_b' => IntegrationDsn::connectionConfig($lane),
+ ]);
+ /** @var ConnectionResolverInterface $connections */
+ $connections = App::make(ConnectionResolverInterface::class);
+ $a = new MySqlNamedLock($connections, 'session_a');
+ $b = new MySqlNamedLock($connections, 'session_b');
+ $name = 'firefly:test:'.bin2hex(random_bytes(4));
+ $long = $name.':'.str_repeat('LongJobName', 10);
+
+ expect($a->tryAcquire($name, 30))->toBeTrue()
+ ->and($b->tryAcquire($name, 30))->toBeFalse()
+ ->and($a->tryAcquire($long, 30))->toBeTrue()
+ ->and($b->tryAcquire($long, 30))->toBeFalse();
+
+ $a->release($name);
+ expect($b->tryAcquire($name, 30))->toBeTrue();
+ $b->release($name);
+
+ // The session ends: the server frees every named lock it held (as soon as it notices the closed socket).
+ DB::connection('session_a')->disconnect();
+ DB::purge('session_a');
+ $freed = false;
+ $deadline = microtime(true) + 3;
+ while (! $freed && microtime(true) < $deadline) {
+ $freed = $b->tryAcquire($long, 30);
+ if (! $freed) {
+ usleep(100_000);
+ }
+ }
+ expect($freed)->toBeTrue();
+ $b->release($long);
+ DB::connection('session_b')->disconnect();
+ })->group('integration')->skip(IntegrationDsn::skipReason($lane) !== null, (string) IntegrationDsn::skipReason($lane));
+}
diff --git a/packages/scheduling-database/tests/LockProviderSelectionBootTest.php b/packages/scheduling-database/tests/LockProviderSelectionBootTest.php
new file mode 100644
index 00000000..876dfec9
--- /dev/null
+++ b/packages/scheduling-database/tests/LockProviderSelectionBootTest.php
@@ -0,0 +1,101 @@
+ ['scheduling' => $provider === null ? [] : ['lock' => ['provider' => $provider]]]],
+ [SchedulingServiceProvider::class, SchedulingPostgresServiceProvider::class, SchedulingDatabaseServiceProvider::class],
+ bindings: [
+ CacheRepositoryContract::class => new CacheRepository(new ArrayStore),
+ CacheFactoryContract::class => new class implements CacheFactoryContract
+ {
+ public function store($name = null)
+ {
+ return new CacheRepository(new ArrayStore);
+ }
+ },
+ ConnectionResolverInterface::class => new class($connection) implements ConnectionResolverInterface
+ {
+ public function __construct(private readonly SQLiteConnection $connection) {}
+
+ public function connection($name = null): SQLiteConnection
+ {
+ return $this->connection;
+ }
+
+ public function getDefaultConnection(): string
+ {
+ return 'default';
+ }
+
+ public function setDefaultConnection($name): void {}
+ },
+ ],
+ );
+}
+
+/** @return list */
+function lockSelectionBeans(Application $app): array
+{
+ /** @var FireflyKernel $kernel */
+ $kernel = $app->make(FireflyKernel::class);
+
+ $beans = [];
+ foreach ($kernel->context()->definitions->all() as $definition) {
+ foreach ($definition->descriptor->beans as $bean) {
+ if ($bean->returns === DistributedLock::class) {
+ $beans[] = $definition->class().'::'.$bean->method;
+ }
+ }
+ }
+ sort($beans);
+
+ return $beans;
+}
+
+it('resolves exactly the lock the provider names', function (?string $provider, string $class, string $bean) {
+ $app = lockSelectionApp($provider);
+ /** @var ApplicationContext $context */
+ $context = $app->make(ApplicationContext::class);
+
+ // The exact class, not instanceof: every lock is final, and a dataset string is not a class-string to PHPStan.
+ expect(get_debug_type($context->get(DistributedLock::class)))->toBe($class)
+ ->and(lockSelectionBeans($app))->toBe([$bean]);
+})->with([
+ 'database' => ['database', DatabaseLeaseLock::class, SchedulingDatabaseAutoConfiguration::class.'::databaseLeaseLock'],
+ 'mysql' => ['mysql', MySqlNamedLock::class, SchedulingDatabaseAutoConfiguration::class.'::mySqlNamedLock'],
+ 'postgres' => ['postgres', PgAdvisoryLock::class, PgAdvisoryLockAutoConfiguration::class.'::distributedLock'],
+ 'cache' => ['cache', CacheLock::class, SchedulingAutoConfiguration::class.'::distributedLock'],
+ 'none' => [null, NoneLock::class, SchedulingAutoConfiguration::class.'::distributedLock'],
+]);
diff --git a/packages/scheduling-database/tests/MySqlNamedLockTest.php b/packages/scheduling-database/tests/MySqlNamedLockTest.php
new file mode 100644
index 00000000..9296b623
--- /dev/null
+++ b/packages/scheduling-database/tests/MySqlNamedLockTest.php
@@ -0,0 +1,62 @@
+connection;
+ }
+
+ public function getDefaultConnection(): string
+ {
+ return 'default';
+ }
+
+ public function setDefaultConnection($name): void {}
+ };
+}
+
+it('asks GET_LOCK with a zero timeout and releases with RELEASE_LOCK', function () {
+ $connection = new MySqlConnection(new PDO('sqlite::memory:'), 'pretend', '', ['driver' => 'mysql', 'name' => 'pretend']);
+ $lock = new MySqlNamedLock(namedLockResolver($connection));
+
+ $log = $connection->pretend(function () use ($lock): void {
+ // pretend() answers nothing, which GET_LOCK reports as an error (NULL): no lock.
+ expect($lock->tryAcquire('reconcile', 30))->toBeFalse();
+ $lock->release('reconcile');
+ });
+
+ // Under pretend() Laravel logs the SQL with its bindings substituted; the bindings array proves the name travelled
+ // as a bound parameter, never concatenated into the statement.
+ expect(array_column($log, 'query'))->toBe(["select get_lock('reconcile', 0) as locked", "select release_lock('reconcile') as released"])
+ ->and(array_column($log, 'bindings'))->toBe([['reconcile'], ['reconcile']]);
+});
+
+it('keeps a name up to 64 bytes and hashes a longer one into the MySQL limit', function () {
+ $long = str_repeat('App\\Jobs\\VeryLongNamespace\\', 4).'Reconcile::run';
+
+ expect(MySqlNamedLock::key('reconcile'))->toBe('reconcile')
+ ->and(MySqlNamedLock::key(str_repeat('x', 64)))->toBe(str_repeat('x', 64))
+ ->and(MySqlNamedLock::key($long))->toBe('firefly:'.sha1($long))
+ ->and(strlen(MySqlNamedLock::key($long)))->toBe(48);
+});
+
+it('refuses a connection that is not MySQL or MariaDB', function () {
+ $lock = new MySqlNamedLock(namedLockResolver(new SQLiteConnection(new PDO('sqlite::memory:'), ':memory:')));
+
+ expect(fn () => $lock->tryAcquire('reconcile', 30))->toThrow(ConfigurationException::class, 'firefly.scheduling.lock.provider=mysql');
+});
diff --git a/packages/scheduling-database/tests/ReflectionFreeSchedulingDatabaseTest.php b/packages/scheduling-database/tests/ReflectionFreeSchedulingDatabaseTest.php
new file mode 100644
index 00000000..c29f0691
--- /dev/null
+++ b/packages/scheduling-database/tests/ReflectionFreeSchedulingDatabaseTest.php
@@ -0,0 +1,26 @@
+ basenames under $dir whose contents reference boot-time reflection */
+function schedulingDatabaseReflectionHits(string $dir): array
+{
+ $hits = [];
+ $files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir, RecursiveDirectoryIterator::SKIP_DOTS));
+ /** @var SplFileInfo $file */
+ foreach ($files as $file) {
+ if (! $file->isFile() || $file->getExtension() !== 'php') {
+ continue;
+ }
+ if (preg_match('/ReflectionClass|ReflectionMethod|getAttributes/', (string) file_get_contents((string) $file->getRealPath())) === 1) {
+ $hits[] = $file->getBasename();
+ }
+ }
+ sort($hits);
+
+ return $hits;
+}
+
+it('the firefly/scheduling-database adapter uses no boot-time reflection', function () {
+ expect(schedulingDatabaseReflectionHits(__DIR__.'/../src'))->toBe([]);
+});
diff --git a/packages/scheduling-database/tests/SchedulingDatabaseBootDiscoveryTest.php b/packages/scheduling-database/tests/SchedulingDatabaseBootDiscoveryTest.php
new file mode 100644
index 00000000..3c90cd0f
--- /dev/null
+++ b/packages/scheduling-database/tests/SchedulingDatabaseBootDiscoveryTest.php
@@ -0,0 +1,39 @@
+instance('config', new Repository(['firefly' => ['scheduling' => $provider === null ? [] : ['lock' => ['provider' => $provider]]]]));
+ $app->instance('files', new Filesystem);
+ $app->register(DatabaseServiceProvider::class);
+ $app->register(MigrationServiceProvider::class);
+ (new SchedulingDatabaseBootServiceProvider($app))->boot();
+ /** @var Migrator $migrator */
+ $migrator = $app->make('migrator');
+
+ $paths = [];
+ foreach ($migrator->paths() as $path) {
+ $paths[] = realpath($path);
+ }
+
+ expect($paths)->toBe($loaded ? [realpath(dirname(__DIR__).'/database/migrations')] : []);
+ $app->flush();
+})->with([
+ 'unset' => [null, false],
+ 'cache' => ['cache', false],
+ 'postgres' => ['postgres', false],
+ 'database' => ['database', true],
+]);
diff --git a/packages/scheduling-postgres/tests/PgAdvisoryLockIntegrationTest.php b/packages/scheduling-postgres/tests/PgAdvisoryLockIntegrationTest.php
index 8ca146ce..9b011e22 100644
--- a/packages/scheduling-postgres/tests/PgAdvisoryLockIntegrationTest.php
+++ b/packages/scheduling-postgres/tests/PgAdvisoryLockIntegrationTest.php
@@ -3,6 +3,7 @@
declare(strict_types=1);
use Firefly\Scheduling\Postgres\PgAdvisoryLock;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Support\Facades\DB;
use Orchestra\Testbench\TestCase;
@@ -30,25 +31,12 @@
$b->release($name);
DB::connection('firefly_pg_b')->disconnect();
})->skip(
- ! extension_loaded('pdo_pgsql') || getenv('FIREFLY_PG_DSN') === false,
- 'requires pdo_pgsql + FIREFLY_PG_DSN',
-);
+ IntegrationDsn::skipReason('pgsql') !== null,
+ (string) IntegrationDsn::skipReason('pgsql'),
+)->group('integration');
-/** @return array a Laravel pgsql connection config parsed from FIREFLY_PG_DSN */
+/** @return array a Laravel pgsql connection config from FIREFLY_PG_DSN */
function firefly_pg_config(): array
{
- $parts = [];
- foreach (explode(';', (string) getenv('FIREFLY_PG_DSN')) as $kv) {
- [$k, $v] = array_pad(explode('=', $kv, 2), 2, '');
- $parts[$k] = $v;
- }
-
- return [
- 'driver' => 'pgsql',
- 'host' => $parts['host'] ?? '127.0.0.1',
- 'port' => $parts['port'] ?? '5432',
- 'database' => $parts['dbname'] ?? 'postgres',
- 'username' => $parts['user'] ?? 'postgres',
- 'password' => $parts['password'] ?? '',
- ];
+ return IntegrationDsn::connectionConfig('pgsql');
}
diff --git a/packages/scheduling/README.md b/packages/scheduling/README.md
index 2b845c68..cc1c4c27 100644
--- a/packages/scheduling/README.md
+++ b/packages/scheduling/README.md
@@ -5,6 +5,7 @@ zero-coordination default) and `CacheLock` (over Laravel's atomic `Cache::lock`)
attribute discovered by the package's SOLE reflection site (`ScheduledScanner`) into a compiled, require-loaded
`ScheduledManifest`; and a `ScheduleWiringPass` that lazily registers native Laravel scheduled tasks — each
task body wrapped in a lock guard (acquire → run → release) and its exceptions logged, never propagated. The
-`postgres` lock backend (`PgAdvisoryLock`) ships in the separate `firefly/scheduling-postgres` adapter package.
+`postgres` lock backend (`PgAdvisoryLock`) ships in `firefly/scheduling-postgres`; the `database`
+(`DatabaseLeaseLock`) and `mysql` (`MySqlNamedLock`) backends ship in `firefly/scheduling-database`.
Apache-2.0 © Firefly Software Solutions Inc.
diff --git a/packages/scheduling/src/Lock/DistributedLock.php b/packages/scheduling/src/Lock/DistributedLock.php
index 8e5b429a..d7214915 100644
--- a/packages/scheduling/src/Lock/DistributedLock.php
+++ b/packages/scheduling/src/Lock/DistributedLock.php
@@ -7,7 +7,8 @@
/**
* The distributed-lock port (a ShedLock analog): the ScheduleWiringPass acquires a named lock before running a
* scheduled task body and releases it after, so a task marked #[Scheduled(lock: ...)] runs on at most one node
- * per tick. NoneLock (default) and CacheLock ship here; PgAdvisoryLock ships in firefly/scheduling-postgres.
+ * per tick. NoneLock (default) and CacheLock ship here; PgAdvisoryLock ships in firefly/scheduling-postgres,
+ * DatabaseLeaseLock and MySqlNamedLock in firefly/scheduling-database.
*/
interface DistributedLock
{
diff --git a/packages/scheduling/tests/Integration/CacheLockRedisIntegrationTest.php b/packages/scheduling/tests/Integration/CacheLockRedisIntegrationTest.php
new file mode 100644
index 00000000..6ed2d128
--- /dev/null
+++ b/packages/scheduling/tests/Integration/CacheLockRedisIntegrationTest.php
@@ -0,0 +1,52 @@
+ 'predis',
+ 'database.redis.default' => ['url' => IntegrationDsn::value(IntegrationDsn::REDIS)],
+ 'cache.stores.it_redis' => ['driver' => 'redis', 'connection' => 'default', 'lock_connection' => 'default'],
+ ]);
+ /** @var Repository $store */
+ $store = Cache::store('it_redis');
+ $a = new CacheLock($store, 'owner-a');
+ $b = new CacheLock($store, 'owner-b');
+ $name = 'firefly:test:'.bin2hex(random_bytes(4));
+
+ expect($a->tryAcquire($name, 30))->toBeTrue()
+ ->and($b->tryAcquire($name, 30))->toBeFalse();
+ $b->release($name);
+ $redis = $store->getStore();
+ if (! $redis instanceof LockProvider) {
+ throw new RuntimeException('The Redis cache store is expected to provide atomic locks.');
+ }
+ // Owner tokens are enforced by Redis itself, not only by CacheLock's bookkeeping: a foreign token releases nothing.
+ expect($b->tryAcquire($name, 30))->toBeFalse()
+ ->and($redis->lock($name, 30, 'owner-b')->release())->toBeFalse()
+ ->and($b->tryAcquire($name, 30))->toBeFalse();
+ $a->release($name);
+ expect($b->tryAcquire($name, 1))->toBeTrue();
+
+ usleep(1_300_000);
+ expect($a->tryAcquire($name, 30))->toBeTrue();
+ $a->release($name);
+})->group('integration')->skip(
+ IntegrationDsn::missing(IntegrationDsn::REDIS) || ! class_exists(Client::class),
+ 'Set FIREFLY_REDIS_URL (and install predis/predis) to run the Redis CacheLock integration test.',
+);
diff --git a/packages/testing/README.md b/packages/testing/README.md
index 72f9b390..61856f3c 100644
--- a/packages/testing/README.md
+++ b/packages/testing/README.md
@@ -8,8 +8,9 @@ and `FireflyDatabaseTestCase`/`UsesSqliteMemory` for database-backed tests; web/
doubles for every one of Firefly's own ports (`RecordingEventPublisher`, `RecordingCommandBus`,
`RecordingMessageBroker`, `RecordingTracer`, `FakeHealthIndicator`, and more); Firefly-flavored Pest
expectations (`toHavePublished`/`toHaveHandledCommand`/`toBeUp`/`toHaveRecordedMetric`/
-`toBeProblemDetails`); a fixture layer (`FixtureRegistry`/`AggregateSeeder`/`ListenerSpy`); and a
-`RequiresDocker` testcontainers hook for `@group('integration')` tests. `require-dev` only, no runtime
+`toBeProblemDetails`); a fixture layer (`FixtureRegistry`/`AggregateSeeder`/`ListenerSpy`); a
+`RequiresDocker` testcontainers hook; and `IntegrationDsn`, the one parser every `@group('integration')` test
+reads its `FIREFLY_*` backend from. `require-dev` only, no runtime
service provider.
See [Testing](../../docs/modules/testing.md) for the full reference.
diff --git a/packages/testing/composer.json b/packages/testing/composer.json
index 4752b8fc..f62e6ccb 100644
--- a/packages/testing/composer.json
+++ b/packages/testing/composer.json
@@ -46,6 +46,7 @@
"illuminate/config": "^13.0",
"illuminate/container": "^13.0",
"illuminate/contracts": "^13.0",
+ "illuminate/database": "^13.0",
"illuminate/http": "^13.0",
"illuminate/routing": "^13.0",
"illuminate/support": "^13.0",
diff --git a/packages/testing/src/Integration/IntegrationDsn.php b/packages/testing/src/Integration/IntegrationDsn.php
new file mode 100644
index 00000000..d34cd9bb
--- /dev/null
+++ b/packages/testing/src/Integration/IntegrationDsn.php
@@ -0,0 +1,317 @@
+ */
+ public const array DATABASES = [
+ 'pgsql' => ['driver' => 'pgsql', 'variable' => self::PGSQL, 'extension' => 'pdo_pgsql'],
+ 'mysql' => ['driver' => 'mysql', 'variable' => self::MYSQL, 'extension' => 'pdo_mysql'],
+ 'mariadb' => ['driver' => 'mariadb', 'variable' => self::MARIADB, 'extension' => 'pdo_mysql'],
+ 'sqlsrv' => ['driver' => 'sqlsrv', 'variable' => self::SQLSRV, 'extension' => 'pdo_sqlsrv'],
+ 'sqlite' => ['driver' => 'sqlite', 'variable' => null, 'extension' => 'pdo_sqlite'],
+ ];
+
+ /** @var array */
+ private const array DEFAULT_PORTS = ['pgsql' => 5432, 'mysql' => 3306, 'mariadb' => 3306, 'sqlsrv' => 1433];
+
+ /** @var list the only keys a URL DSN's query string may set; host, port, user and password come from the authority */
+ private const array URL_QUERY_EXTRAS = ['search_path', 'encrypt', 'trust_server_certificate', 'sslmode', 'charset', 'vhost'];
+
+ private static ?string $sqliteDirectory = null;
+
+ /** @return list */
+ public static function lanes(): array
+ {
+ return array_keys(self::DATABASES);
+ }
+
+ public static function value(string $variable): ?string
+ {
+ $value = getenv($variable);
+
+ return is_string($value) && trim($value) !== '' ? trim($value) : null;
+ }
+
+ public static function missing(string $variable): bool
+ {
+ return self::value($variable) === null;
+ }
+
+ /**
+ * @return array
+ */
+ public static function parse(string $dsn): array
+ {
+ $dsn = trim($dsn);
+
+ if (preg_match('#^[a-z][a-z0-9+.\-]*://#i', $dsn) === 1) {
+ return self::parseUrl($dsn);
+ }
+
+ $parts = [];
+ foreach (explode(';', $dsn) as $position => $pair) {
+ if (trim($pair) === '') {
+ continue;
+ }
+ if (! str_contains($pair, '=')) {
+ // Never echo the DSN or the segment: either may be (part of) a password.
+ throw new InvalidArgumentException(sprintf('Cannot read the DSN: segment %d is not a key=value pair (expected host=…;port=…;dbname=…;user=…;password=… or scheme://user:pass@host:port/db).', $position + 1));
+ }
+ [$key, $value] = explode('=', $pair, 2);
+ $parts[strtolower(trim($key))] = trim($value);
+ }
+
+ if (($parts['host'] ?? '') === '') {
+ throw new InvalidArgumentException('Cannot read the DSN: it names no host (expected host=…;port=…;dbname=…;user=…;password=…).');
+ }
+
+ return $parts;
+ }
+
+ /**
+ * @return array|null
+ */
+ public static function parts(string $variable): ?array
+ {
+ $value = self::value($variable);
+
+ return $value === null ? null : self::parse($value);
+ }
+
+ /** Why $lane cannot run here, or null when it can. */
+ public static function skipReason(string $lane): ?string
+ {
+ $database = self::DATABASES[$lane] ?? throw new InvalidArgumentException(sprintf('Unknown integration lane [%s]; known: %s.', $lane, implode(', ', self::lanes())));
+
+ if (! extension_loaded($database['extension'])) {
+ return sprintf('requires ext-%s for the %s integration lane', $database['extension'], $lane);
+ }
+
+ if ($database['variable'] !== null && self::missing($database['variable'])) {
+ return sprintf('Set %s (host=…;port=…;dbname=…;user=…;password=…) to run the %s integration lane.', $database['variable'], $lane);
+ }
+
+ return null;
+ }
+
+ /**
+ * @param array $overrides
+ * @return array
+ */
+ public static function connectionConfig(string $lane, array $overrides = []): array
+ {
+ $database = self::DATABASES[$lane] ?? throw new InvalidArgumentException(sprintf('Unknown integration lane [%s].', $lane));
+
+ if ($database['variable'] === null) {
+ $path = $overrides['database'] ?? self::sqlitePath();
+
+ return array_replace(self::sqlite(is_string($path) ? $path : self::sqlitePath()), $overrides);
+ }
+
+ return self::connectionConfigFor($database['driver'], $database['variable'], $overrides);
+ }
+
+ /**
+ * A config for $driver from the DSN in $variable — the form a suite uses when it deliberately speaks one driver to
+ * another lane's server (feature-flags runs its `mysql` driver against the MariaDB server).
+ *
+ * @param array $overrides
+ * @return array
+ */
+ public static function connectionConfigFor(string $driver, string $variable, array $overrides = []): array
+ {
+ $parts = self::parts($variable) ?? throw new InvalidArgumentException(sprintf('%s is not set.', $variable));
+
+ $config = [
+ 'driver' => $driver,
+ 'host' => $parts['host'],
+ 'port' => (int) ($parts['port'] ?? self::DEFAULT_PORTS[$driver] ?? 0),
+ 'database' => $parts['dbname'] ?? $parts['database'] ?? 'firefly',
+ 'username' => $parts['user'] ?? $parts['username'] ?? '',
+ 'password' => $parts['password'] ?? '',
+ 'prefix' => '',
+ ];
+
+ $config = match ($driver) {
+ 'pgsql' => [...$config, 'charset' => 'utf8', 'search_path' => $parts['search_path'] ?? 'public'],
+ 'mysql', 'mariadb' => [...$config, 'charset' => 'utf8mb4', 'collation' => 'utf8mb4_unicode_ci'],
+ 'sqlsrv' => [...$config, 'charset' => 'utf8', 'encrypt' => $parts['encrypt'] ?? 'yes', 'trust_server_certificate' => $parts['trust_server_certificate'] ?? 'false'],
+ default => $config,
+ };
+
+ return array_replace($config, $overrides);
+ }
+
+ /**
+ * A sqlite connection config over a FILE, created when missing (Laravel refuses a sqlite path that does not
+ * exist), in WAL mode with a busy timeout, so two OS processes can write it concurrently without SQLITE_BUSY.
+ * The file is created exclusively (O_EXCL, mode 0600) and a symlink at $path is refused, so a pre-planted link
+ * can never redirect the test database elsewhere.
+ *
+ * @return array
+ */
+ public static function sqlite(string $path): array
+ {
+ if (is_link($path)) {
+ throw new InvalidArgumentException(sprintf('Refusing the sqlite path [%s]: it is a symbolic link.', $path));
+ }
+
+ if (! file_exists($path)) {
+ $handle = @fopen($path, 'x');
+ if ($handle === false) {
+ throw new InvalidArgumentException(sprintf('Cannot create the sqlite file [%s] exclusively.', $path));
+ }
+ fclose($handle);
+ chmod($path, 0600);
+ } elseif (! is_file($path)) {
+ throw new InvalidArgumentException(sprintf('Refusing the sqlite path [%s]: it is not a regular file.', $path));
+ }
+
+ return [
+ 'driver' => 'sqlite',
+ 'database' => $path,
+ 'prefix' => '',
+ 'foreign_key_constraints' => true,
+ 'busy_timeout' => 10000,
+ 'journal_mode' => 'wal',
+ 'synchronous' => 'normal',
+ ];
+ }
+
+ /**
+ * A per-process temporary sqlite file inside a private (0700), unpredictably named directory created once per
+ * process; a test that spawns a second process hands it this same path. Stable for a given $name within the
+ * process, so a test can remove and recreate it.
+ */
+ public static function sqlitePath(string $name = 'integration'): string
+ {
+ if (self::$sqliteDirectory === null || ! is_dir(self::$sqliteDirectory)) {
+ $directory = sys_get_temp_dir().'/firefly-it-'.bin2hex(random_bytes(8));
+ if (! mkdir($directory, 0700) && ! is_dir($directory)) {
+ throw new InvalidArgumentException(sprintf('Cannot create the private sqlite directory [%s].', $directory));
+ }
+ self::$sqliteDirectory = $directory;
+ }
+
+ return self::$sqliteDirectory.'/'.preg_replace('/[^A-Za-z0-9_-]/', '_', $name).'.sqlite';
+ }
+
+ /**
+ * A standalone connection (no application): what a racer process or an app-less test uses.
+ *
+ * @param array $overrides
+ */
+ public static function connect(string $lane, string $name = 'integration', array $overrides = []): Connection
+ {
+ $capsule = new Manager;
+ $capsule->addConnection(self::connectionConfig($lane, $overrides), $name);
+
+ return $capsule->getConnection($name);
+ }
+
+ public static function connectFor(string $driver, string $variable, string $name = 'integration'): Connection
+ {
+ $capsule = new Manager;
+ $capsule->addConnection(self::connectionConfigFor($driver, $variable), $name);
+
+ return $capsule->getConnection($name);
+ }
+
+ /**
+ * @return array{host: string, port: int, user: string, password: string, vhost: string}
+ */
+ public static function rabbitmq(): array
+ {
+ $parts = self::parts(self::RABBITMQ) ?? throw new InvalidArgumentException(self::RABBITMQ.' is not set.');
+ $vhost = $parts['vhost'] ?? '';
+
+ return [
+ 'host' => $parts['host'],
+ 'port' => (int) ($parts['port'] ?? 5672),
+ 'user' => $parts['user'] ?? 'guest',
+ 'password' => $parts['password'] ?? 'guest',
+ 'vhost' => $vhost === '' ? '/' : $vhost,
+ ];
+ }
+
+ /**
+ * @return array
+ */
+ private static function parseUrl(string $dsn): array
+ {
+ $url = parse_url($dsn);
+
+ if (! is_array($url) || ($url['host'] ?? '') === '') {
+ throw new InvalidArgumentException('Cannot read the DSN: a URL DSN needs a host (scheme://user:pass@host:port/db).');
+ }
+
+ $parts = ['scheme' => strtolower($url['scheme'] ?? ''), 'host' => $url['host']];
+
+ if (isset($url['port'])) {
+ $parts['port'] = (string) $url['port'];
+ }
+ if (isset($url['user'])) {
+ $parts['user'] = rawurldecode($url['user']);
+ }
+ if (isset($url['pass'])) {
+ $parts['password'] = rawurldecode($url['pass']);
+ }
+
+ $path = rawurldecode(ltrim($url['path'] ?? '', '/'));
+ if ($path !== '') {
+ $parts['dbname'] = $path;
+ $parts['vhost'] = $path;
+ }
+
+ if (isset($url['query'])) {
+ parse_str($url['query'], $query);
+ foreach ($query as $key => $value) {
+ $key = strtolower((string) $key);
+ // The authority is authoritative: the query string only carries connection EXTRAS, never identity.
+ if (is_string($value) && in_array($key, self::URL_QUERY_EXTRAS, true)) {
+ $parts[$key] = $value;
+ }
+ }
+ }
+
+ return $parts;
+ }
+}
diff --git a/packages/testing/tests/Integration/IntegrationDsnTest.php b/packages/testing/tests/Integration/IntegrationDsnTest.php
new file mode 100644
index 00000000..f5d9fba8
--- /dev/null
+++ b/packages/testing/tests/Integration/IntegrationDsnTest.php
@@ -0,0 +1,184 @@
+toBe([
+ 'host' => '127.0.0.1',
+ 'port' => '55432',
+ 'dbname' => 'firefly_w1',
+ 'user' => 'firefly',
+ 'password' => 'p=ss',
+ ]);
+});
+
+it('parses the URL form, decoding credentials, the path as database and vhost, and the query as extras', function () {
+ expect(IntegrationDsn::parse('amqp://guest:g%40st@127.0.0.1:55672/%2f'))->toBe([
+ 'scheme' => 'amqp',
+ 'host' => '127.0.0.1',
+ 'port' => '55672',
+ 'user' => 'guest',
+ 'password' => 'g@st',
+ 'dbname' => '/',
+ 'vhost' => '/',
+ ])->and(IntegrationDsn::parse('sqlsrv://sa:Secret-1@db:1433/firefly?encrypt=no&trust_server_certificate=true'))->toBe([
+ 'scheme' => 'sqlsrv',
+ 'host' => 'db',
+ 'port' => '1433',
+ 'user' => 'sa',
+ 'password' => 'Secret-1',
+ 'dbname' => 'firefly',
+ 'vhost' => 'firefly',
+ 'encrypt' => 'no',
+ 'trust_server_certificate' => 'true',
+ ]);
+});
+
+it('refuses a DSN it cannot read instead of falling back to localhost defaults', function (string $dsn) {
+ expect(fn () => IntegrationDsn::parse($dsn))->toThrow(InvalidArgumentException::class);
+})->with([
+ 'no equals sign' => ['127.0.0.1:5432'],
+ 'no host' => ['port=5432;dbname=x'],
+ 'url without host' => ['pgsql:///firefly'],
+]);
+
+it('treats an unset or blank variable as missing', function () {
+ putenv('FIREFLY_TEST_DSN= ');
+ expect(IntegrationDsn::value('FIREFLY_TEST_DSN'))->toBeNull()
+ ->and(IntegrationDsn::missing('FIREFLY_TEST_DSN'))->toBeTrue()
+ ->and(IntegrationDsn::parts('FIREFLY_TEST_DSN'))->toBeNull();
+
+ putenv('FIREFLY_TEST_DSN=host=db;port=1');
+ expect(IntegrationDsn::missing('FIREFLY_TEST_DSN'))->toBeFalse()
+ ->and(IntegrationDsn::parts('FIREFLY_TEST_DSN'))->toBe(['host' => 'db', 'port' => '1']);
+});
+
+it('builds a Laravel connection config per driver', function () {
+ putenv('FIREFLY_TEST_DSN=host=db;port=1433;dbname=firefly;user=sa;password=pw');
+
+ expect(IntegrationDsn::connectionConfigFor('pgsql', 'FIREFLY_TEST_DSN'))->toBe([
+ 'driver' => 'pgsql', 'host' => 'db', 'port' => 1433, 'database' => 'firefly', 'username' => 'sa',
+ 'password' => 'pw', 'prefix' => '', 'charset' => 'utf8', 'search_path' => 'public',
+ ])->and(IntegrationDsn::connectionConfigFor('mariadb', 'FIREFLY_TEST_DSN')['charset'])->toBe('utf8mb4')
+ ->and(IntegrationDsn::connectionConfigFor('sqlsrv', 'FIREFLY_TEST_DSN'))->toMatchArray([
+ 'driver' => 'sqlsrv', 'encrypt' => 'yes', 'trust_server_certificate' => 'false',
+ ])->and(IntegrationDsn::connectionConfigFor('mysql', 'FIREFLY_TEST_DSN', ['database' => 'other'])['database'])->toBe('other');
+});
+
+it('defaults the port per driver when the DSN names none', function () {
+ putenv('FIREFLY_TEST_DSN=host=db;dbname=f;user=u;password=p');
+
+ expect(IntegrationDsn::connectionConfigFor('pgsql', 'FIREFLY_TEST_DSN')['port'])->toBe(5432)
+ ->and(IntegrationDsn::connectionConfigFor('mysql', 'FIREFLY_TEST_DSN')['port'])->toBe(3306)
+ ->and(IntegrationDsn::connectionConfigFor('sqlsrv', 'FIREFLY_TEST_DSN')['port'])->toBe(1433);
+});
+
+it('gives the sqlite lane a real file with WAL and a busy timeout, so two processes can share it', function () {
+ $path = IntegrationDsn::sqlitePath('dsn-test');
+ integrationDsnRemoveSqlite($path);
+
+ $config = IntegrationDsn::sqlite($path);
+
+ expect(is_file($path))->toBeTrue()
+ ->and($config)->toMatchArray(['driver' => 'sqlite', 'database' => $path, 'journal_mode' => 'wal', 'busy_timeout' => 10000, 'foreign_key_constraints' => true])
+ ->and(IntegrationDsn::skipReason('sqlite'))->toBeNull()
+ ->and(IntegrationDsn::connect('sqlite', 'dsn-test', ['database' => $path])->selectOne('select 1 as one'))->toEqual((object) ['one' => 1]);
+
+ integrationDsnRemoveSqlite($path);
+});
+
+it('names the variable to set when a server lane cannot run', function () {
+ $reason = IntegrationDsn::skipReason('sqlsrv');
+
+ expect($reason)->toBeString()
+ ->and((string) $reason)->toMatch('/FIREFLY_SQLSRV_DSN|pdo_sqlsrv/')
+ ->and(IntegrationDsn::lanes())->toBe(['pgsql', 'mysql', 'mariadb', 'sqlsrv', 'sqlite'])
+ ->and(fn () => IntegrationDsn::skipReason('oracle'))->toThrow(InvalidArgumentException::class);
+});
+
+it('maps the RabbitMQ DSN in both forms to the five AMQP connection arguments', function () {
+ $previous = getenv(IntegrationDsn::RABBITMQ);
+
+ try {
+ putenv(IntegrationDsn::RABBITMQ.'=host=127.0.0.1;port=55672;user=guest;password=pw');
+ expect(IntegrationDsn::rabbitmq())->toBe(['host' => '127.0.0.1', 'port' => 55672, 'user' => 'guest', 'password' => 'pw', 'vhost' => '/']);
+
+ putenv(IntegrationDsn::RABBITMQ.'=amqp://guest:pw@127.0.0.1:55672/%2f');
+ expect(IntegrationDsn::rabbitmq())->toBe(['host' => '127.0.0.1', 'port' => 55672, 'user' => 'guest', 'password' => 'pw', 'vhost' => '/']);
+ } finally {
+ putenv($previous === false ? IntegrationDsn::RABBITMQ : IntegrationDsn::RABBITMQ.'='.$previous);
+ }
+});
+
+it('never echoes the DSN, which carries the password, in a parse error', function (string $dsn) {
+ $message = null;
+ try {
+ IntegrationDsn::parse($dsn);
+ } catch (InvalidArgumentException $error) {
+ $message = $error->getMessage();
+ }
+
+ expect($message)->toBeString()->not->toContain('S3cret-pw');
+})->with([
+ 'segment without equals' => ['host=db;password=S3cret-pw;S3cret-pw'],
+ 'no host' => ['port=5432;password=S3cret-pw'],
+ 'url without host' => ['pgsql://u:S3cret-pw@/firefly'],
+]);
+
+it('validates the SQL Server certificate unless the DSN opts out explicitly', function () {
+ putenv('FIREFLY_TEST_DSN=host=db;user=sa;password=pw');
+ expect(IntegrationDsn::connectionConfigFor('sqlsrv', 'FIREFLY_TEST_DSN')['trust_server_certificate'])->toBe('false');
+
+ putenv('FIREFLY_TEST_DSN=host=db;user=sa;password=pw;trust_server_certificate=true');
+ expect(IntegrationDsn::connectionConfigFor('sqlsrv', 'FIREFLY_TEST_DSN')['trust_server_certificate'])->toBe('true');
+});
+
+it('lets a URL query string add extras but never override the authority', function () {
+ expect(IntegrationDsn::parse('pgsql://u:pw@db:5432/firefly?host=evil&port=1&user=root&password=x&search_path=app'))->toBe([
+ 'scheme' => 'pgsql', 'host' => 'db', 'port' => '5432', 'user' => 'u', 'password' => 'pw',
+ 'dbname' => 'firefly', 'vhost' => 'firefly', 'search_path' => 'app',
+ ]);
+});
+
+it('keeps sqlite files in a private directory and refuses a planted symlink', function () {
+ $path = IntegrationDsn::sqlitePath('private-check');
+
+ expect(IntegrationDsn::sqlitePath('private-check'))->toBe($path)
+ ->and(dirname($path))->not->toBe(sys_get_temp_dir())
+ ->and(fileperms(dirname($path)) & 0777)->toBe(0700);
+
+ integrationDsnRemoveSqlite($path);
+ symlink(__FILE__, $path);
+ try {
+ expect(fn () => IntegrationDsn::sqlite($path))->toThrow(InvalidArgumentException::class, 'symbolic link');
+ } finally {
+ unlink($path);
+ }
+
+ IntegrationDsn::sqlite($path);
+ expect(fileperms($path) & 0777)->toBe(0600);
+ integrationDsnRemoveSqlite($path);
+});
diff --git a/samples/lumen/composer.json b/samples/lumen/composer.json
index 45514f68..96a036a0 100644
--- a/samples/lumen/composer.json
+++ b/samples/lumen/composer.json
@@ -38,6 +38,7 @@
"illuminate/support": "^13.0"
},
"require-dev": {
+ "firefly/eda-database": "*@dev",
"firefly/eda-postgres": "*@dev",
"firefly/testing": "*@dev",
"orchestra/testbench": "^11.1",
diff --git a/samples/lumen/tests/Integration/OutboxPostgresIntegrationTestCase.php b/samples/lumen/tests/Integration/OutboxPostgresIntegrationTestCase.php
index d2d8ee32..751ec256 100644
--- a/samples/lumen/tests/Integration/OutboxPostgresIntegrationTestCase.php
+++ b/samples/lumen/tests/Integration/OutboxPostgresIntegrationTestCase.php
@@ -4,8 +4,10 @@
namespace Lumen\Tests\Integration;
+use Firefly\Eda\EdaConsumerServiceProvider;
use Firefly\Eda\Postgres\EdaPostgresServiceProvider;
use Firefly\Eda\Postgres\Outbox\OutboxSchema;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Contracts\Config\Repository;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Artisan;
@@ -29,13 +31,18 @@ abstract class OutboxPostgresIntegrationTestCase extends LumenTestCase
/** True only when a real Postgres target is reachable (the pdo_pgsql extension is loaded and FIREFLY_PG_DSN is set). */
public static function postgresAvailable(): bool
{
- return extension_loaded('pdo_pgsql') && getenv('FIREFLY_PG_DSN') !== false;
+ return IntegrationDsn::skipReason('pgsql') === null;
}
- /** @return list> */
+ /**
+ * EdaConsumerServiceProvider is what registers `firefly:eda:consume` — the parent's list leaves it out because no
+ * unit test runs a consumer, and that is exactly how this test came to call a command that did not exist.
+ *
+ * @return list>
+ */
protected function fireflyProviders(): array
{
- return [...parent::fireflyProviders(), EdaPostgresServiceProvider::class];
+ return [...parent::fireflyProviders(), EdaPostgresServiceProvider::class, EdaConsumerServiceProvider::class];
}
/** @return array */
@@ -87,27 +94,14 @@ protected function migrate(): void
}
/**
- * A Laravel pgsql connection config parsed from FIREFLY_PG_DSN (host=…;port=…;dbname=…;user=…;password=…) —
- * mirrors packages/eda-postgres/tests/Integration/PostgresOutboxRoundTripTest::outbox_pg_config().
+ * A Laravel pgsql connection config from FIREFLY_PG_DSN (host=…;port=…;dbname=…;user=…;password=…) — parsed by
+ * firefly/testing's IntegrationDsn.
*
* @return array
*/
private static function postgresConfig(): array
{
- $parts = [];
- foreach (explode(';', (string) getenv('FIREFLY_PG_DSN')) as $kv) {
- [$key, $value] = array_pad(explode('=', $kv, 2), 2, '');
- $parts[$key] = $value;
- }
-
- return [
- 'driver' => 'pgsql',
- 'host' => $parts['host'] ?? '127.0.0.1',
- 'port' => $parts['port'] ?? '5432',
- 'database' => $parts['dbname'] ?? 'postgres',
- 'username' => $parts['user'] ?? 'postgres',
- 'password' => $parts['password'] ?? '',
- ];
+ return IntegrationDsn::connectionConfig('pgsql');
}
protected function tearDown(): void
diff --git a/samples/lumen/tests/Integration/OutboxPostgresRoundTripTest.php b/samples/lumen/tests/Integration/OutboxPostgresRoundTripTest.php
index 602c77f8..a4eaeb1f 100644
--- a/samples/lumen/tests/Integration/OutboxPostgresRoundTripTest.php
+++ b/samples/lumen/tests/Integration/OutboxPostgresRoundTripTest.php
@@ -7,6 +7,7 @@
use Firefly\Cqrs\Command\CommandBus;
use Firefly\Cqrs\Query\QueryBus;
use Firefly\Eda\Postgres\Outbox\OutboxSchema;
+use Firefly\Testing\Integration\IntegrationDsn;
use Illuminate\Support\Facades\Artisan;
use Illuminate\Support\Facades\DB;
use Lumen\Application\Command\Deposit;
@@ -63,6 +64,6 @@
->count())->toBe(1)
->and(DB::table(OutboxSchema::TABLE)->count())->toBe($total); // never re-inserted
})->skip(
- ! extension_loaded('pdo_pgsql') || getenv('FIREFLY_PG_DSN') === false,
- 'Set FIREFLY_PG_DSN (and pdo_pgsql) to run the @group integration lumen Postgres outbox round-trip.',
+ ! OutboxPostgresIntegrationTestCase::postgresAvailable(),
+ (string) IntegrationDsn::skipReason('pgsql'),
)->group('integration');
diff --git a/skeleton/config/firefly.php b/skeleton/config/firefly.php
index d312c1c7..f686cf3d 100644
--- a/skeleton/config/firefly.php
+++ b/skeleton/config/firefly.php
@@ -2231,12 +2231,36 @@
| 'none' — no coordination (correct for a single instance). The default.
| 'cache' — the app's atomic cache lock.
| 'postgres' — Postgres advisory locks; requires firefly/scheduling-postgres.
+ | 'database' — a lease table on any database connection; requires firefly/scheduling-database.
+ | 'mysql' — MySQL/MariaDB GET_LOCK named locks; requires firefly/scheduling-database.
|
*/
'scheduling' => [
'lock' => [
'provider' => env('FIREFLY_SCHEDULING_LOCK', 'none'),
+
+ /*
+ | provider=database — the lease table firefly_scheduling_locks (run `php artisan migrate`) on
+ | `connection` (UNSET, null or blank = the default connection). The TTL is honoured on every
+ | driver: a hung task blocks its schedule for at most its #[Scheduled(lockTtl:)].
+ |
+ | Default: connection unset.
+ */
+ // 'database' => [
+ // 'connection' => 'mysql',
+ // ],
+
+ /*
+ | provider=mysql — GET_LOCK named locks, held for the connection's session and released when it
+ | closes; no table, no TTL (lock-ttl is ignored). `connection` must be a mysql or mariadb
+ | connection (UNSET, null or blank = the default connection).
+ |
+ | Default: connection unset.
+ */
+ // 'mysql' => [
+ // 'connection' => 'mysql',
+ // ],
],
/*
@@ -2327,14 +2351,15 @@
/*
|--------------------------------------------------------------------------
- | Events — firefly/eda (+ eda-rabbitmq / eda-postgres / eda-kafka)
+ | Events — firefly/eda (+ eda-database / eda-rabbitmq / eda-postgres / eda-kafka)
|--------------------------------------------------------------------------
|
| `provider` selects the EventPublisher adapter:
| 'memory' — in-process bus. The default; listeners run synchronously.
| 'queue' — Laravel queue; listeners run in `queue:work`.
+ | 'database' — requires firefly/eda-database; the same-transaction outbox on any database.
| 'rabbitmq' — requires firefly/eda-rabbitmq.
- | 'postgres' — requires firefly/eda-postgres; enables the same-transaction outbox.
+ | 'postgres' — requires firefly/eda-postgres; the database outbox plus pg_notify/LISTEN.
| 'kafka' — requires firefly/eda-kafka and ext-rdkafka.
|
| The broker providers are consumed by `php artisan firefly:eda:consume`; 'queue' uses
@@ -2363,6 +2388,19 @@
'retries' => 0,
'retry_delay' => 0.0,
+ /*
+ | Where a listener's envelope goes once its retries are exhausted, under EVERY provider. 'memory'
+ | keeps it in the process (lost when it exits); 'database' writes it to firefly_eda_dead_letters
+ | on `connection` (UNSET = the default connection) — run `php artisan migrate` — and
+ | `php artisan firefly:eda:dead-letters:replay` republishes it.
+ |
+ | Defaults: store 'memory', connection unset.
+ */
+ 'dead-letter' => [
+ 'store' => env('FIREFLY_EDA_DEAD_LETTER_STORE', 'memory'),
+ // 'connection' => 'mysql',
+ ],
+
/*
| Broker destinations `firefly:eda:consume` binds when `--destination` is not passed. Must be a
| list of strings.
@@ -2413,9 +2451,38 @@
// ],
/*
- | provider=postgres — the same-transaction outbox. `connection` UNSET means the default database
- | connection. `channel` is the LISTEN/NOTIFY channel; `max_attempts` bounds relay retries before
- | a row is marked FAILED.
+ | provider=database — and provider=postgres, which reads the same block — the same-transaction
+ | outbox (firefly/eda-database). `connection` UNSET means the default database connection.
+ | `claim-strategy` decides how a consumer or relay takes rows: 'auto' asks the server (SKIP LOCKED
+ | on PostgreSQL, MySQL 8.0.1+ and MariaDB 10.6+; READPAST on SQL Server; a conditional-UPDATE lease
+ | everywhere else), or force 'skip-locked' | 'read-past' | 'optimistic-lease'. A claimed row
+ | belongs to its claimant for `lease-seconds` — size it above your slowest listener. An idle consumer
+ | sleeps `poll-interval-ms` between claims (provider=postgres waits on NOTIFY instead).
+ | `relay.downstream-provider` is only read by `php artisan firefly:outbox:relay`.
+ |
+ | Every key falls back to its firefly.eda.postgres.* spelling below when it is unset or null (an
+ | unset env() counts as unset).
+ |
+ | Defaults: connection unset, channel 'firefly_eda_events', max-attempts 3, claim-strategy 'auto',
+ | lease-seconds 300, poll-interval-ms 500, relay.downstream-provider unset.
+ */
+ // 'database' => [
+ // 'connection' => 'mysql',
+ // 'channel' => 'firefly_eda_events',
+ // 'max-attempts' => 3,
+ // 'claim-strategy' => env('FIREFLY_EDA_CLAIM_STRATEGY', 'auto'),
+ // 'lease-seconds' => 300,
+ // 'poll-interval-ms' => 500,
+ // 'relay' => [
+ // 'downstream-provider' => 'rabbitmq',
+ // ],
+ // ],
+
+ /*
+ | provider=postgres — the legacy spelling of the block above (each key here is an alias of its
+ | firefly.eda.database.* twin); `channel` is also the LISTEN/NOTIFY channel. `connection` UNSET
+ | means the default database connection; `max_attempts` bounds relay retries before a row is
+ | marked FAILED.
|
| `relay.downstream_provider` is OPTIONAL and only used by `php artisan firefly:outbox:relay`,
| which forwards committed outbox rows to a SECOND broker. It takes 'rabbitmq', 'kafka', an
@@ -2444,13 +2511,13 @@
// ],
/*
- | Broker publishers and the trace. The rabbitmq, kafka and postgres adapters route publish()
+ | Broker publishers and the trace. The rabbitmq, kafka, database and postgres adapters route publish()
| through the same EdaTracing seam the in-memory and queue buses use, so the envelope that reaches
| the wire carries a `traceparent` (and `tracestate`) and the consumer on the far side — ours or a
| third party's — continues the trace instead of starting a new one. The consume side was already
- | traced; this is the other half. On Postgres it covers BOTH writers of an outbox row: the
- | EventPublisher bean and the in-transaction OutboxPreCommitHook, which is the only path a domain
- | event takes under provider=postgres — so the traced row is the one that commits with the
+ | traced; this is the other half. On the outbox (provider=database or postgres) it covers BOTH writers
+ | of a row: the EventPublisher bean and the in-transaction OutboxPreCommitHook, which is the only path
+ | a domain event takes under those providers — so the traced row is the one that commits with the
| aggregate. `firefly:outbox:relay` forwards a claimed row under that row's own trace rather than
| starting a new one.
|
@@ -2458,7 +2525,7 @@
| `firefly.observability.tracing.eda.enabled` are on, because with them off the bound EdaTracing is
| the no-op. Set this to false to keep in-process spans while refusing to put trace identifiers on
| a wire someone else reads: it is read in ONE place (Firefly\Eda\Tracing\BrokerTracing), by the
- | three publisher beans AND by the relay, so a shipped adapter cannot slip past it — the relay
+ | four publisher beans AND by the relay, so a shipped adapter cannot slip past it — the relay
| applies it whether you name the adapter by its alias (`rabbitmq`) or by its own class-string.
| A downstream of your OWN, bound under `firefly.eda.relay.downstream` or named by a publisher
| class this framework ships no adapter for, is yours to construct and therefore yours to gate:
@@ -2487,6 +2554,20 @@
'messaging' => [
'provider' => env('FIREFLY_MESSAGING_PROVIDER', 'memory'),
+ /*
+ | Where a #[MessageListener(deadLetterTopic: …)] message goes once its retries are exhausted.
+ | 'memory' keeps it in the process (lost when it exits); 'database' writes it to
+ | firefly_messaging_dead_letters on `connection` (UNSET = the default connection), the raw value
+ | base64-encoded — run `php artisan migrate` — and
+ | `php artisan firefly:messaging:dead-letters:replay` republishes it to its original topic.
+ |
+ | Defaults: store 'memory', connection unset.
+ */
+ 'dead-letter' => [
+ 'store' => env('FIREFLY_MESSAGING_DEAD_LETTER_STORE', 'memory'),
+ // 'connection' => 'mysql',
+ ],
+
// 'queue' => [
// 'connection' => 'redis',
// 'name' => 'messages',
diff --git a/tests/DocsDiagramsTest.php b/tests/DocsDiagramsTest.php
index abe327a5..36c4192a 100644
--- a/tests/DocsDiagramsTest.php
+++ b/tests/DocsDiagramsTest.php
@@ -19,7 +19,9 @@
use Firefly\Eda\Bus\InMemoryEventBus;
use Firefly\Eda\Bus\QueueEventBus;
use Firefly\Eda\Consumer\SubscriberRegistrySink;
+use Firefly\Eda\Database\DatabaseOutboxAutoConfiguration;
use Firefly\Eda\EventEnvelope;
+use Firefly\Eda\Postgres\PostgresOutboxAutoConfiguration;
use Firefly\Eda\Tracing\EdaTracing;
use Firefly\Observability\Cqrs\TracerCqrsTracing;
use Firefly\Observability\Eda\TracerEdaTracing;
@@ -1495,3 +1497,55 @@
}
}
});
+
+it('scopes the CQRS/EDA bridge figure\'s atomicity caption to every provider that registers the same-tx outbox', function () {
+ // The bridge figure ends on a caption that sends the reader elsewhere for same-transaction atomicity. It was
+ // written when only firefly/eda-postgres wrote the outbox row inside the aggregate's transaction, and said
+ // so: "true same-tx atomicity is the postgres outbox adapter". firefly/eda-database now registers the same
+ // pre-commit hook for provider=database on every supported database, and provider=postgres is that outbox
+ // plus pg_notify/LISTEN — so the caption named one of two providers and implied the other did not exist.
+ // outbox-flow.svg was corrected in the same change and this figure was not. The providers are DERIVED here
+ // from the #[ConditionalOnProperty] on each auto-configuration's OutboxPreCommitHook bean, so a third outbox
+ // provider fails this test until the caption and the adapter row name it too.
+ $root = dirname(__DIR__);
+ $docs = $root.'/docs/assets/diagrams/cqrs-eda-bridge.svg';
+ $book = $root.'/book/art/figures/cqrs-eda-bridge.svg';
+
+ expect(file_get_contents($book))->toBe(file_get_contents($docs), 'book/art/figures/cqrs-eda-bridge.svg must stay byte-identical to the docs copy');
+
+ $providers = [];
+ foreach ([DatabaseOutboxAutoConfiguration::class, PostgresOutboxAutoConfiguration::class] as $configuration) {
+ foreach ((new ReflectionMethod($configuration, 'outboxPreCommitHook'))->getAttributes(ConditionalOnProperty::class) as $attribute) {
+ $condition = $attribute->newInstance();
+ if ($condition->name === 'firefly.eda.provider' && $condition->havingValue !== null) {
+ $providers[] = $condition->havingValue;
+ }
+ }
+ }
+ sort($providers);
+ expect($providers)->toBe(['database', 'postgres']);
+
+ $svg = simplexml_load_file($docs);
+ if ($svg === false) {
+ throw new RuntimeException('docs/assets/diagrams/cqrs-eda-bridge.svg is not well-formed XML');
+ }
+ $svg->registerXPathNamespace('s', 'http://www.w3.org/2000/svg');
+ $texts = array_map(static fn (SimpleXMLElement $node): string => trim((string) $node), $svg->xpath('//s:text') ?: []);
+
+ $captions = array_values(array_filter($texts, static fn (string $text): bool => str_contains($text, 'same-tx atomicity')));
+ expect($captions)->toHaveCount(1, 'the bridge figure must still say where same-transaction atomicity lives');
+ [$caption] = $captions;
+
+ expect(str_contains($caption, 'postgres outbox adapter'))->toBeFalse('the caption still credits only the postgres adapter with the same-tx outbox')
+ ->and(str_contains($caption, 'outbox-flow.svg'))->toBeTrue('the caption must keep pointing at the outbox figure');
+ foreach ($providers as $provider) {
+ expect(preg_match('/\b'.preg_quote($provider, '/').'\b/', $caption))->toBe(1, "the caption does not name provider '{$provider}', which registers the same-tx outbox");
+ }
+
+ // The adapter row under the EventPublisher port draws one box per provider; each outbox provider must be
+ // drawn, and the "(same-tx outbox)" note must sit under the outbox providers, not only under postgres.
+ foreach ($providers as $provider) {
+ expect(in_array($provider, $texts, true))->toBeTrue("the adapter row draws no box for provider '{$provider}'");
+ }
+ expect(count(array_keys($texts, '(same-tx outbox)', true)))->toBe(count($providers), 'every outbox provider box carries the (same-tx outbox) note');
+});
diff --git a/tests/Integration/IntegrationLaneTest.php b/tests/Integration/IntegrationLaneTest.php
new file mode 100644
index 00000000..d73e47a5
--- /dev/null
+++ b/tests/Integration/IntegrationLaneTest.php
@@ -0,0 +1,45 @@
+toBeFalse("the brokers lane runs without {$variable}");
+ }
+ expect(extension_loaded('rdkafka'))->toBeTrue('the brokers lane runs without ext-rdkafka');
+
+ $rabbit = IntegrationDsn::rabbitmq();
+ $kafka = explode(':', (string) IntegrationDsn::value(IntegrationDsn::KAFKA));
+ $redis = parse_url((string) IntegrationDsn::value(IntegrationDsn::REDIS));
+ $endpoints = [
+ 'rabbitmq' => [$rabbit['host'], $rabbit['port']],
+ 'kafka' => [$kafka[0], (int) ($kafka[1] ?? 9092)],
+ 'redis' => [is_array($redis) ? (string) ($redis['host'] ?? '127.0.0.1') : '127.0.0.1', is_array($redis) ? (int) ($redis['port'] ?? 6379) : 6379],
+ ];
+ foreach ($endpoints as $name => [$host, $port]) {
+ $socket = @fsockopen($host, $port, $errno, $error, 5.0);
+ expect($socket)->not->toBeFalse("{$name} at {$host}:{$port} is unreachable: {$error}");
+ if (is_resource($socket)) {
+ fclose($socket);
+ }
+ }
+
+ return;
+ }
+
+ expect(IntegrationDsn::DATABASES)->toHaveKey($lane);
+ expect(IntegrationDsn::skipReason($lane))->toBeNull();
+ expect(IntegrationDsn::connect($lane, 'lane-sanity')->selectOne('select 1 as one'))->not->toBeNull();
+})->skip(IntegrationDsn::missing(IntegrationDsn::LANE), 'FIREFLY_INTEGRATION_LANE is only set by the CI integration jobs.')->group('integration');
diff --git a/tests/IntegrationWorkflowTest.php b/tests/IntegrationWorkflowTest.php
new file mode 100644
index 00000000..45077f91
--- /dev/null
+++ b/tests/IntegrationWorkflowTest.php
@@ -0,0 +1,154 @@
+
+ */
+function integrationYamlMap(mixed $value, string $what): array
+{
+ if (! is_array($value) || array_is_list($value) && $value !== []) {
+ throw new RuntimeException("{$what} is not a YAML mapping.");
+ }
+
+ /** @var array $value */
+ return $value;
+}
+
+/** @return list */
+function integrationYamlList(mixed $value, string $what): array
+{
+ if (! is_array($value) || ! array_is_list($value)) {
+ throw new RuntimeException("{$what} is not a YAML sequence.");
+ }
+
+ return $value;
+}
+
+/** @return array */
+function integrationCiJobs(): array
+{
+ $ci = integrationYamlMap(Yaml::parseFile(dirname(__DIR__).'/.github/workflows/ci.yml'), 'ci.yml');
+
+ return integrationYamlMap($ci['jobs'] ?? null, 'ci.yml jobs');
+}
+
+function integrationYamlString(mixed $value): string
+{
+ return is_scalar($value) ? (string) $value : '';
+}
+
+/**
+ * The `run:` text of every step of a job, joined.
+ *
+ * @param array $job
+ */
+function integrationRunText(array $job): string
+{
+ $runs = '';
+ foreach (integrationYamlList($job['steps'] ?? null, 'steps') as $step) {
+ $step = integrationYamlMap($step, 'step');
+ $runs .= is_string($step['run'] ?? null) ? $step['run']."\n" : '';
+ }
+
+ return $runs;
+}
+
+it('runs the integration group through one composer script with enough memory to collect it', function () {
+ /** @var array{scripts: array>} $composer */
+ $composer = json_decode((string) file_get_contents(dirname(__DIR__).'/composer.json'), true, flags: JSON_THROW_ON_ERROR);
+
+ expect($composer['scripts']['test:integration'] ?? null)->toBe('@php -d memory_limit=1G vendor/bin/pest --group integration');
+});
+
+it('runs one integration lane per database server, each with a healthy service and only its own DSN', function () {
+ $job = integrationYamlMap(integrationCiJobs()['integration'] ?? null, 'jobs.integration');
+ $matrix = integrationYamlMap(integrationYamlMap($job['strategy'] ?? null, 'strategy')['matrix'] ?? null, 'matrix');
+
+ $lanes = [];
+ foreach (integrationYamlList($matrix['include'] ?? null, 'matrix.include') as $entry) {
+ $entry = integrationYamlMap($entry, 'matrix entry');
+ $lanes[integrationYamlString($entry['lane'] ?? null)] = [integrationYamlString($entry['image'] ?? null), integrationYamlString($entry['dsn-variable'] ?? null)];
+ expect(integrationYamlString($entry['health'] ?? null))->not->toBe('');
+ }
+
+ expect($lanes)->toBe([
+ 'pgsql' => ['postgres:16', 'FIREFLY_PG_DSN'],
+ 'mysql' => ['mysql:8.4', 'FIREFLY_MYSQL_DSN'],
+ 'mariadb' => ['mariadb:11', 'FIREFLY_MARIADB_DSN'],
+ 'sqlsrv' => ['mcr.microsoft.com/mssql/server:2022-latest', 'FIREFLY_SQLSRV_DSN'],
+ ]);
+
+ $service = integrationYamlMap(integrationYamlMap($job['services'] ?? null, 'services')['database'] ?? null, 'services.database');
+ expect($service['image'] ?? null)->toBe('${{ matrix.image }}')
+ ->and(integrationYamlString($service['options'] ?? null))->toContain('--health-cmd');
+
+ $run = integrationRunText($job);
+ expect($run)->toContain("export \${{ matrix.dsn-variable }}='\${{ matrix.dsn }}'")
+ ->and($run)->toContain('composer test:integration')
+ ->and(substr_count($run, 'export '))->toBe(1)
+ ->and(json_encode($job, JSON_THROW_ON_ERROR))->not->toContain('FIREFLY_RABBITMQ_DSN');
+});
+
+it('runs the brokers lane against RabbitMQ, Kafka (with ext-rdkafka) and Redis, each health-checked', function () {
+ $job = integrationYamlMap(integrationCiJobs()['brokers'] ?? null, 'jobs.brokers');
+ $services = integrationYamlMap($job['services'] ?? null, 'services');
+
+ expect(array_keys($services))->toBe(['rabbitmq', 'kafka', 'redis']);
+ foreach ($services as $name => $service) {
+ expect(integrationYamlString(integrationYamlMap($service, $name)['options'] ?? null))->toContain('--health-cmd');
+ }
+
+ $env = [];
+ foreach (integrationYamlList($job['steps'] ?? null, 'steps') as $step) {
+ $step = integrationYamlMap($step, 'step');
+ if (str_contains(integrationYamlString($step['run'] ?? null), 'composer test:integration')) {
+ $env = integrationYamlMap($step['env'] ?? null, 'env');
+ }
+ }
+
+ expect(array_keys($env))->toBe(['FIREFLY_INTEGRATION_LANE', 'FIREFLY_RABBITMQ_DSN', 'FIREFLY_KAFKA_BROKERS', 'FIREFLY_REDIS_URL'])
+ ->and(json_encode($job, JSON_THROW_ON_ERROR))->toContain('rdkafka');
+});
+
+it('mirrors every CI image in docker-compose.integration.yml', function () {
+ $compose = integrationYamlMap(Yaml::parseFile(dirname(__DIR__).'/docker-compose.integration.yml'), 'compose');
+ $images = [];
+ foreach (integrationYamlMap($compose['services'] ?? null, 'services') as $name => $service) {
+ $images[$name] = integrationYamlString(integrationYamlMap($service, $name)['image'] ?? null);
+ }
+
+ $jobs = integrationCiJobs();
+ $strategy = integrationYamlMap(integrationYamlMap($jobs['integration'] ?? null, 'integration')['strategy'] ?? null, 'strategy');
+ $ci = [];
+ foreach (integrationYamlList(integrationYamlMap($strategy['matrix'] ?? null, 'matrix')['include'] ?? null, 'include') as $entry) {
+ $ci[] = integrationYamlString(integrationYamlMap($entry, 'entry')['image'] ?? null);
+ }
+ foreach (integrationYamlMap(integrationYamlMap($jobs['brokers'] ?? null, 'brokers')['services'] ?? null, 'services') as $name => $service) {
+ $ci[] = integrationYamlString(integrationYamlMap($service, $name)['image'] ?? null);
+ }
+
+ sort($ci);
+ $local = array_values($images);
+ sort($local);
+
+ expect($local)->toBe($ci);
+});
+
+it('keeps the compose mirror out of the library archive', function () {
+ // The compose file is a contributor tool like phpunit.xml.dist and testbench.yaml; a consumer installing
+ // fireflyframework/larafly must not find a set of database containers in its vendor directory.
+ $attributes = (string) file_get_contents(dirname(__DIR__).'/.gitattributes');
+
+ expect($attributes)->toMatch('#^/docker-compose\.integration\.yml\s+export-ignore$#m');
+});
![The genuine same-transaction outbox: an aggregate's transaction begins, the domain write and the outbox INSERT + pg_notify happen on the same connection inside the same transaction, and only on commit does Postgres deliver the NOTIFY to a LISTEN-ing consumer that drains the row to subscribed #[EventListener] handlers.](docs/assets/diagrams/outbox-flow.svg)