Repository navigation
test: characterize CLI behaviour - #103
Open
shayyz-code wants to merge 2 commits into
Open
shayyz-code wants to merge 2 commits into
shayyz-code wants to merge 2 commits into
Conversation
Add spec_cli_* specs that run the poo binary and pin down its v0.1.11 exit codes, stdout, and stderr: Result output for top-level return, parse, runtime, and Io errors exiting 1, the usage message, --help and --version read as file paths, the --speed timing line, and ignored extra arguments. Error paths assert exit 1 rather than the panic code 101. Recursion is recorded as an ignored known failure for #45. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Assert only the stable prefix of the missing-file Io error, check the lexer case for its exact message, give each argument case a real stderr needle, and include stderr in every contains assertion. Record the non-UTF-8 argument panic as an ignored unix-only known failure for #110. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shayyz-code
force-pushed
the
test/90-characterize-cli
branch
from
October 14, 2026 08:29
a30efa7 to
32c7984
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR records how the
poobinary behaves in v0.1.11, so the upcoming CLI refactor has to change it on purpose. It adds aclimodule to thelanguage_specstest binary (tests/language_specs/cli.rs) with 18spec_cli_*specs. Each spec runsenv!("CARGO_BIN_EXE_poo")in a fresh temp directory that is deleted afterwards. The interpreter and the conformance harness are unchanged.Behaviour the specs record:
returnaddsResult: <Debug>to stdout, such asResult: Int(3)orResult: Vector([Int(1), Int(2)]). Without areturn, and for an empty file, nothing extra is printed.<Kind> error: in '<path>': <message>(Io:failed to read '<path>': ...) and exit 1. Every one of these specs also asserts that the exit code is not 101 (a panic).Usage: <program> <file_path>to stderr and exits 1.--helpand--versionare read as file paths, which gives an Io error and exit 1. So is--speedwhen it comes before the file path.--speedafter the path adds\nInterpreted within: <Duration Debug>to stdout. The spec checks the shape of that line, not the number. On an error the line is not printed.spec_cli_runs_recursive_functionis#[ignore = "#45: ..."]. Today the binary panics with exit 101 instead of printingResult: Int(120).spec_cli_reports_non_utf8_argument_without_panickingis unix-only and#[ignore = "#110: ..."]. Today a non-UTF-8 argument such asb"\xff.poo"panics instd::env::args()with exit 101; the spec expects exit 1 with an Io or usage error.Assertions check stable substrings. They do not depend on the absolute temp path or on timing values.
Issue
Closes #90
Verification
cargo test --test language_specs cli:::cargo test --test language_specs cli::spec_cli_reports_non_utf8 -- --ignoredfails as expected: exit 101 from theenv.rsunwrap.cargo test --test language_specs cli::spec_cli_runs_recursive_function -- --ignoredfails as expected. The binary exits 101, and stderr shows the panic atsrc/type_inference.rs:117:36: Undefined function: fact.just checkpasses (fmt, check, clippy, tests including the conformance corpus, mdbook build and test,dist generate --check,cargo deny check).Each commit is a self-contained Conventional Commit and passes
just checkDocumentation and Visual Impact
None. Only tests changed. The module list in
tests/language_specs/main.rsnow includescli.🤖 Generated with Claude Code