AI ships code in minutes — verifying it hasn't. testsprite opens your live app, uses it like a real user, and shows your coding agent exactly what broke — so it fixes its own work before a bug ever reaches you.
Proof, in public: verification beats model size. With this CLI in the loop, the cheapest model in the field shipped the most correct app on an open leaderboard — 89%, at half the cost of the priciest one. See the leaderboard →
⭐ Help us reach more developers and grow the TestSprite community. Star this repo!
testsprite-cli-demo.mp4
▶ Watch the launch video — the three hard limits every coding agent hits, and the loop that breaks them (4 min).
TestSprite is the AI testing platform 100,000+ teams use to test their software, frontend and backend — in the cloud, against the live product, not mocks. This repo is its official CLI.
It puts that platform in your coding agent's hands: the agent verifies every behavior it ships, and what broke comes back as one result — the failed step, its cause and a fix — it can act on, with no dashboard scraping. Humans drive the same surface from a terminal or CI.
If you find testsprite useful, a GitHub Star ⭐️ would be greatly appreciated — it helps other builders (and their agents) find the project, and stars notify you about new releases.
You need Node.js 20.19+ on the 20.x line, 22.13+ on 22.x, or 24 or later. Using 0.16? Read its command reference.
From your app's repository, install the CLI and run setup. It prompts for your workspace API key and installs the skills for the coding agents it detects.
npm install -g @testsprite/testsprite-cli
testsprite --version
testsprite setup
testsprite auth statusChoose your project and target. Run testsprite project list to find an existing project. For a new frontend project, create it and add the URL where your app is running. Replace the URL below with yours and set PROJECT_ID to the ID printed by project create.
testsprite project create --type frontend --name "Checkout"
PROJECT_ID=your-project-id
testsprite project env create "$PROJECT_ID" --name staging --url https://staging.example.comGet your first test. Ask the project's agent to propose coverage. If it asks a question in a text message, answer with testsprite reply "your answer". Review the proposed steps in the terminal and apply the plan when they match what you want to verify. For a backend project, attach the API specification with --file ./openapi.yaml.
testsprite plan "Verify that a guest can place an order with a test card" --project "$PROJECT_ID" --stepstestsprite apply
testsprite test list --project "$PROJECT_ID"Run and inspect. A project run waits for the suite when you pass --wait. Exit 0 means it passed. On exit 1, look for a failed or blocked row in results[] and copy its resultId to inspect the error and available evidence. If no row failed, check runFailures for an execution-level error such as cancellation. Without --output json, each run also prints a Dashboard: link that opens it in the Portal.
testsprite run --project "$PROJECT_ID" --wait --output jsonRESULT_ID=your-result-id
testsprite result get "$RESULT_ID" --out ".testsprite/results/$RESULT_ID"After deploying a fix, run the affected test again with testsprite run "$TEST_ID" --wait, using its ID from test list. To resume waiting for a run you already started, use its run ID with testsprite run wait "$RUN_ID". The scripting example shows how to extract both IDs and preserve the verdict in CI.
Start your frontend app on port 3000. To run the project's saved tests against the app on this machine, supply the port. The CLI opens a tunnel and waits for the cloud runner.
testsprite run --project "$PROJECT_ID" --local 3000For a local-only project, you can save the port as its first environment with testsprite project env create "$PROJECT_ID" --name local --local 3000. Give the project's agent a Git diff or a spec so it can propose tests without browsing localhost. Review and apply the test plan, then run with --env local. An ordinary Ctrl-C detaches a public run. With a tunnel this command opened, Ctrl-C cancels unfinished work by default and closes the tunnel. Local frontend testing and tunnels
| Task | Command |
|---|---|
| Check credentials or install coding-agent skills | auth status, setup, agent status |
| Manage a project and its targets | project list, project create, project env |
| Propose tests, review them and save them | plan, status, reply, reject, apply |
| Find the ID or steps of a saved test | test list --project <id>, test get <test-id> |
| Launch a project, named tests or a saved list | run --project <id>, run <test-id...>, run --testlist <list-id> |
| Follow or cancel an execution | run wait <run-id...>, run cancel <run-id...> |
| Read results and download evidence | result list --run <run-id>, result list --test <test-id>, result get <result-id> |
| Automate scheduled tests or a pull-request gate | schedule, ci init github, ci run |
For test lists, tunnel management, usage, diagnostics, profiles and exit codes, see the command reference. To migrate shell scripts or previously installed agent skills from 0.16, see Upgrading from 0.16.
- 🧪 Tests like a real user. Runs against a live browser or API in the cloud — real clicks, real navigation, real assertions. Not a mock.
- 🤖 Agent-shaped output.
runnames the run (runId) and each test's result in it (resultId).result get <result-id>returns that one result — the verdict, the error, its cause and fix, the failed step with its screenshot, a backend test's traceback and output, and every step — and--out <dir>downloads its video and each step's screenshot and page. The agent reads the run it just started, never an older one. - Whole-run results. Every selector and
run waituse the same JSON:runs[]describes executions,results[]lists individual results, andsummarycounts outcomes when waiting. A complete run can fail. Only a timeout or interruption that leaves work unfinished addspartial: true. A cancelled execution fails even if its existing results passed. Read detailed evidence withresult get <result-id>. - ♻️ A loop, not a one-shot.
plan → apply → run → result get → fix → run— every pass is banked, not thrown away. - 📐 Scriptable & deterministic. Stable
--output jsoncontract, predictable exit codes, and a--dry-runfor previewing commands. A credentialed run preview can read environment metadata to determine whether it would open a tunnel. - 🚦 CI-native.
testsprite ci init githubwrites a PR gate;testsprite ci run --project "$PROJECT_ID" --env <name> --target-url "$PREVIEW_URL"tests each pull request's preview deployment as one run and exits with the verdict. The job summary is the execution report — the same content as the GitHub App's PR comment — plus::error::annotations on the checks tab, step outputs,--summary-filefor a machine summary and--report junitfor a JUnit sidecar. In any other CI,testsprite ci run --project <id> --env <name> --target-url <url>still runs the suite as one run and makes the exit code the verdict — the pull-request context, draft skip and job summary are GitHub Actions only. Details → - 🔌 One command to onboard your agent.
testsprite setupdrops ready-made skill files into your repo — for the agents it detects you using — so your coding agent knows how to drive the loop on its own.
Every time your agent changes code, it asks one question: is this behavior already covered by the suite?
- Not yet covered →
testsprite plan— the project's agent writes a test for the new behavior;testsprite applysaves it, then run it. - Already covered →
testsprite run --project <project-id> --wait— run the existing tests, so nothing that used to work breaks silently. - Something fails →
testsprite result get <result-id>— the failed step, its cause and a fix; the agent fixes the code and runs again.
Every pass is banked into a durable suite, so coverage compounds as the project grows — a lasting record of every requirement it has ever gotten right, far bigger than any context window.
flowchart TD
A["🤖 Your coding agent<br/>Claude Code · Codex · Antigravity · Kimi · Cursor · Trae …"]
D{"behavior already<br/>covered by the suite?"}
B["<b>testsprite plan → apply</b><br/>the project's agent writes a test<br/>for the new behavior"]
R["<b>testsprite run --project <project-id> --wait</b><br/>run the existing tests"]
C{{"☁️ TestSprite testing agent<br/>runs the test like a real user against<br/>real browsers & real APIs on Cloud"}}
F["<b>testsprite result get</b><br/>one test's result in the run:<br/>failed step · screenshots · video ·<br/>cause · fix"]
S[("📚 Durable integration suite<br/>grows with every pass")]
A -->|"writes / changes code"| D
D -->|"no — new behavior"| B
D -->|"yes"| R
B --> R
R --> C
C -->|"pass ✅"| S
C -->|"fail ❌"| F
F -->|"agent reads the result<br/>& fixes the code"| A
S -.->|"defines what's covered"| D
The cloud is a black box on purpose: your agent describes intent and reads results. It never has to know how the test was driven — only what a real user experienced.
On CoderCup — an open leaderboard where frontier coding agents build the same app under the same rules, with TestSprite as the referee — the cheapest model in the field shipped the most correct app on the board: 89%, at half the cost of the priciest one.
That's the point of all of this: you no longer need the biggest, most expensive model to ship software you can trust — top-tier quality, without paying top-tier prices, within reach of every team.
- 📚 CLI reference — DOCUMENTATION.md
- 🌐 Platform docs — docs.testsprite.com
- 🐛 Issues & feature requests — GitHub issues
- 💬 Quick questions — Discord, or
testsprite --help/testsprite run --helpright in your terminal - 📝 Changelog — CHANGELOG.md
Contributions are welcome — the CLI is plain TypeScript/Node (20.19+, 22.13+, or 24+), tested with Vitest, built with tsc. Getting a dev loop running takes a minute:
git clone https://github.com/TestSprite/testsprite-cli.git
cd testsprite-cli && npm install
npm run build # tsc → dist/ ; try it: node dist/index.js --help
npm test # unit suite — no network or credentials needed
npm run lint:fix # ESLint
npm run typecheck # tsc --noEmitPull requests target the main branch. The full guide — build from source, test tiers, CI checks, branches, project layout — is in CONTRIBUTING.md.
Support — if you need any help, we're responsive on Discord, and feel free to email us at contact@testsprite.com too.
Apache-2.0 © TestSprite