Skip to content

Add firefly/eventsourcing: event store, snapshots, projections and event-sourced orchestration - #18

Merged
ancongui merged 34 commits into
mainfrom
feat/eventsourcing
Oct 13, 2026
Merged

ancongui merged 34 commits into
mainfrom
feat/eventsourcing

Conversation

@ancongui

Copy link
Copy Markdown
Contributor

Adds firefly/eventsourcing, at parity with PyFly's eventsourcing module and modeled on jafly's fireflyframework-eventsourcing.

What ships

  • Event store, set by firefly.eventsourcing.store.provider (memory or database).
    • Appends use optimistic concurrency. A version clash on UNIQUE(aggregate_id, version) is translated into a ConcurrencyException.
    • Streams can be read per aggregate (from/to a version) or globally in commit order from a position, filtered by event type, aggregate type or time range.
    • The database store numbers events in commit order on every server, so a reader of a global position never skips an event that commits late.
    • It is proven on PostgreSQL, MySQL, MariaDB, SQL Server and SQLite, including racing appenders: on the same aggregate exactly one wins, and on different aggregates none are lost.
  • Events. The event type registry uses #[EventType(type, version)] and is compiled by firefly:cache. Each event's envelope carries metadata (correlation and causation ids, tenant, user, timestamps), filled by metadata contributors. An upcaster chain is applied on read.
  • Aggregates. An event-sourced aggregate base, plus a snapshot-aware EventSourcedRepository.
  • Snapshots (memory|database): taken every firefly.eventsourcing.snapshot.threshold events, with keep-count retention, and loaded as the latest at or before a version.
  • Projections. Function projections and a ProjectionRunner.
    • Checkpoints are stored in a CheckpointStore (memory|database).
    • A lease on the DistributedLock port ensures only one node advances a projection.
    • Runs in catch-up and live modes, and supports reset/rebuild.
    • Lag is reported in an actuator health component.
    • firefly:eventsourcing:projections, :projections:run and :projections:rebuild.
  • Publishing. Appended events go to firefly/eda in the same transaction through the database outbox when firefly.eda.provider=database|postgres, otherwise after commit, and never twice (firefly.eventsourcing.publish.enabled, destination).
  • Orchestration. firefly.orchestration.persistence.provider=event-sourced stores executions as an aggregate with a snapshot and a projection, and passes W2's provider contract and capstones.
  • Tenancy. A TenantScopedEventStore decorator, enabled with firefly.eventsourcing.tenancy.enabled. Tenant ids are compared exactly (binary collation), so tenants never leak across case on MySQL, MariaDB or SQL Server.

Verification

Includes origin/main (PR #17's docs wave).

Run Result
composer check exit 0, 6720 passed
Integration on PostgreSQL 16, MySQL 8.4, MariaDB 11 and SQLite, plus the broker lanes (RabbitMQ, Kafka, Redis) 372 passed, 0 failed
SQL Server lane (local container) 145 passed, 0 failed
bash scripts/build-docs.sh exit 0

Andres Contreras added 30 commits October 13, 2026 07:42
…ith its metadata and the eventsourcing exceptions
… order and translating racing appends into ConcurrencyException
…so concurrent appenders wait instead of failing with SQLITE_BUSY
… committers on PostgreSQL, MySQL, MariaDB, SQL Server and SQLite
…map them to stored events without reflection
…r parameters and public properties differ, so nothing is stored that cannot be read back
… at scan time a convention-mapped parameter type the mapper cannot rebuild
…h a class member or an intersection, which the mapper would hand the raw stored value
…tion handlers, versions and history replay
…t and the migrations, proven through the real pipeline
…er-commit publishing through the real pipeline and on every server
…d fenced memory and database checkpoint stores
…th fenced checkpoints, catch-up, live mode and rebuild
…jection lag in an eventsourcing health component and add the projection commands
…rced aggregate that replays its checkpoints exactly
…ore with an inline, rebuildable read model
…r=event-sourced, proven by W2's capstones and contract on every server
…nants and with none when tenancy is enabled
…d the event-sourced orchestration provider
@ancongui
ancongui merged commit 6910b4a into main Oct 13, 2026
12 checks passed
@ancongui ancongui mentioned this pull request Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant