Skip to content

feat: add persistent priority scheduling - #5

Merged
achraf059 merged 1 commit into
mainfrom
feat/priority-scheduling
Sep 14, 2026
Merged

achraf059 merged 1 commit into
mainfrom
feat/priority-scheduling

Conversation

@achraf059

Copy link
Copy Markdown
Owner

Summary

Adds persistent priority-aware scheduling to the distributed task processor.

  • Adds HIGH, NORMAL, and LOW task priorities
  • Defaults omitted and legacy priorities to NORMAL
  • Preserves deterministic FIFO ordering with persistent submission sequences
  • Prevents starvation through configurable age-based promotion
  • Preserves priority and sequence across retries and coordinator restarts
  • Adds priority support to the client CLI and job-status output
  • Maintains backward compatibility with existing protocol messages and databases

Scheduling policy

Queued jobs are ordered by:

  1. Effective priority, highest first
  2. Submission sequence, oldest first
  3. Creation time and job ID as deterministic legacy fallbacks

A waiting job rises one effective priority level every configured aging interval, which defaults to 60 seconds. Effective priority is capped at HIGH.

Only the original priority and submission sequence are persisted. Effective priority is calculated when scheduling and is not stored.

Scope boundaries

Priority affects queued-job ordering only.

This change does not add:

  • Task preemption
  • Priority-based admission control
  • Worker-side priority logic
  • Multiple queues
  • Coordinator replication or consensus

Worker selection, overload control, deadlines, cancellation, retries, stale-result rejection, and idempotency retain their existing behavior.

Priority is part of idempotent submission identity. Reusing an idempotency key with a different priority produces a conflict, while omitted priority and explicit NORMAL are equivalent.

Persistence and compatibility

SQLite migration adds:

  • priority, defaulting to NORMAL
  • submission_sequence, defaulting to 0 for legacy records

New submissions receive unique, monotonically increasing sequences that survive coordinator restarts.

Legacy records remain readable. Because historical submission order was not previously stored, ordering among legacy records sharing sequence 0 is best-effort using creation time and job ID.

Legacy SubmitJob and JobSnapshot messages without priority remain compatible.

Testing

Command:

./mvnw clean test failsafe:integration-test failsafe:verify

Results:

  • 119 unit tests
  • 53 integration tests
  • 172 tests total
  • 0 failures
  • 0 errors
  • 0 skipped
  • Build successful

Coverage includes priority ordering, deterministic FIFO behavior, starvation prevention with a controlled clock, retry retention, restart recovery, sequence continuity, SQLite migration, protocol compatibility, CLI validation, and idempotency conflicts.

@achraf059
achraf059 merged commit b2cbd4e into main Sep 14, 2026
1 check passed
@achraf059
achraf059 deleted the feat/priority-scheduling branch September 14, 2026 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant