Skip to content

Replace the tmux integration suite with parallel mini.test child tests - #25

Merged
CoreyKaylor merged 4 commits into
mainfrom
test-mini-parallel
Nov 5, 2026
Merged

CoreyKaylor merged 4 commits into
mainfrom
test-mini-parallel

Conversation

@CoreyKaylor

@CoreyKaylor CoreyKaylor commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

Summary

The full test run drops from ~9 minutes to ~9 seconds (36s with one worker).

Before After
Unit specs 12s ~9s (2 parallel shards)
tmux integration ~540s, serial, ~180 fixed sleeps ~3s per file, run in parallel
Full suite, 8 workers ~552s ~9s

How it works

  • No terminal needed. Integration tests drive headless child Neovim processes with mini.test (pinned to v0.17.0; tests/deps.sh fetches it into the gitignored deps/). The child draws its full grid without a UI. tests/helpers/child.lua reads each cell's text and RGB colors through nvim__inspect_cell and produces plain and ANSI rows identical to tmux capture-pane [-e] -p. The existing visual checks therefore run unchanged; verify.lua moves to tests/helpers/verify.lua as a module.
  • Real waits instead of sleeps. The new lua/diffbandit/util/pending.lua tracks armed timers and in-flight async processes. It is hooked into ui.schedule_once, the folder, queue-prefetch, merge and panel timers, and process.run_async/run_raw_async. child:settle() waits until nothing is outstanding. This is the only plugin-side change, and the plugin itself never reads the tracker.
  • Parallel runner: nvim -l tests/run_all.lua runs two unit shards and one job per tests/integration/test_*.lua, each with its own artifacts and XDG/git sandbox.
    • Filters: DIFFBANDIT_TESTS=scroll,git and DIFFBANDIT_JOBS=N.
    • A failing job prints its output and the path to its captures.
    • A job also fails if any collected case never reported a result. mini.test exits 0 in that situation, which is how a host-side vim.wait can silently lose a case.
  • Real-terminal smoke test: tests/smoke/tmux.sh checks one screen in a real TUI under tmux. CI runs it as its own step.

Verification

  • Parity: all 85 static and scroll captures are byte-identical to the tmux captures, last row excluded. Every run.sh assertion is ported with its message.
  • Negative check: breaking a connector glyph fails 4 jobs with specific messages.
  • Flake check: 10 consecutive full runs passed, plus one run with 1 worker and one with 16 workers.

Also in this PR

  • The scroll-perf test can now fail. It could not before, in bash either: it counted Session:render, which debounced scroll repaints bypass, and its scrolls reached DiffBandit as one event per burst. It now counts every paint and delivers each <C-e> as its own scroll event (1,800 events give ~56 paints). Removing the debounce makes it fail with 80 renders for 80 events, and a new check fails it if the scrolls ever arrive coalesced again.

  • tests/run.lua now exits non-zero on a failing spec. Before, headless nvim sat idle on the startup error.

  • tests/run.lua gains DIFFBANDIT_SPECS and DIFFBANDIT_SPECS_SKIP. The docs described a spec filter, but it didn't exist.

  • The git fixture helpers several specs use move from 20_git_merge into 00_harness, so any subset of specs runs on its own.

  • README, the help file, AGENTS.md and the spec documents are updated. tests/integration/run.sh is removed.

Known gaps (not addressed here)

  • The kitty timers aren't tracked. The timers in kitty/detect.lua and kitty/tmux.lua take no pending token. No integration test uses kitty.
  • The merge LSP case skips when lua-language-server isn't installed, as it did before. CI doesn't install it.

The tmux suite took ~9 minutes: ~180 fixed sleeps, a fresh tmux session
and nvim per scroll phase, and every scenario run serially. The unit
specs took 12s.

Integration tests now drive headless child Neovim instances through
mini.test (pinned, fetched into deps/ by tests/deps.sh). The child reads
its own grid with nvim__inspect_cell, and tests/helpers/child.lua renders
plain and ANSI rows byte-identical to tmux capture-pane, so the existing
visual checks (tests/helpers/verify.lua, formerly the verify.lua script)
run unchanged. All 85 static and scroll captures match the tmux ones
byte for byte.

Fixed sleeps are replaced by a real idle signal: util/pending tracks
armed timers and in-flight async processes at the plugin's timer and
process sites, and child:settle() waits for it.

tests/run_all.lua runs two unit shards plus one job per test file across
all cores: the full suite takes ~9s (36s serial). It also fails a job
when a collected mini.test case never reports, which mini.test alone
does not catch.

Also:
- tests/run.lua exits non-zero on a failing spec instead of idling, and
  gains DIFFBANDIT_SPECS / DIFFBANDIT_SPECS_SKIP. Git fixture helpers
  shared by several specs move into 00_harness so subsets run alone.
- tests/smoke/tmux.sh keeps one real-TUI check under tmux.
- CI runs the parallel suite and the smoke test.
…her timers

mini.test marks a case that left notes (MiniTest.skip) with an uppercase
O/X, which the dropped-case guard did not count: CI, without
lua-language-server, reported 2 of 4 merge cases. The pending unit check
asserted the global outstanding count, which a timer armed by an earlier
spec can change mid-wait; it now tracks its own tokens by label.
The test could not fail, in bash or after the port. It counted
Session:render, but debounced scroll repaints go through
rerender_for_viewport, which paints via session.render.render directly,
so every burst recorded 0 renders. DBRapidScroll also fed every <C-e>
inside one command, so DiffBandit saw one scroll event per burst.

The counter now wraps session.render.render, which every paint goes
through. Each <C-e> is typed separately and processed before the next,
so each one is its own scroll event (80 events now give 3 paints, 1800
give 56). A new check fails the test if the scrolls arrive coalesced,
which would hide the debounce again. DBRapidScroll is gone.

Removing the debounce now fails the test (80 events, 80 renders).
GitHub deprecated Node 20 for actions; checkout@v4 and cache@v4 were
being forced onto Node 24 with a warning. checkout v7 and cache v6 run on
Node 24 natively. Their other changes (fork-PR checkout refused under
pull_request_target/workflow_run, credential file location, ESM) do not
affect this workflow.
@CoreyKaylor
CoreyKaylor merged commit 263f4fd into main Nov 5, 2026
1 check passed
@CoreyKaylor
CoreyKaylor deleted the test-mini-parallel branch November 5, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant