> ## Documentation Index
> Fetch the complete documentation index at: https://jetify-dependabot-npm-and-yarn-npm-and-yarn-387f502f5a.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# testpilot test

> Test the scenarios in the provided pilot plans

```bash theme={null}
testpilot test [path/to/plan.pilot.yaml | path/to/directory/with/pilot/plans] [flags]
```

## Examples[​](#examples "Direct link to Examples")

```bash theme={null}
# Run all tests in the current directory
testpilot test

# Run tests from a specific pilot plan file
testpilot test ./my-test.pilot.yaml

# Run tests from a specific directory
testpilot test ./tests/

# Run only tests with names containing "login"
testpilot test --filter login

# Run tests and save reports to a custom directory
testpilot test --outdir ./test-results

# Run tests in shard 1 of 3 total shards
testpilot test --shard 1/3
```

## Options[​](#options "Direct link to Options")

| Option                       | Description                                                                                                                        |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `-f, --filter string`        | only run tests with names containing the given string                                                                              |
| `-h, --help`                 | help for test                                                                                                                      |
| `--concurrency int`          | number of tests to run concurrently (default 10)                                                                                   |
| `--driver string`            | driver to use for the test (playwright or appium) (default "playwright")                                                           |
| --launch-vars stringToString | Environment variables passed when launching the application under test. Currently only applies to iOS apps and tests (default \[]) |
| `--os string`                | operating system to run tests on (local, linux, macos, windows) (default "local")                                                  |
| `--os-version string`        | operating system version to run tests on (e.g., "13", "18.3")                                                                      |
| `--outdir string`            | Directory in which to save reports (default "testpilot-out")                                                                       |
| `--runtime string`           | runtime environment to execute the test in (chromium, firefox, webkit, native) (default "chromium")                                |
| `--shard value`              | Split tests into n shards and only run tests in shard x                                                                            |
| `--udid value`               | specify the UDID of the iOS device you want to test. Note: this cannot be used along with `--os-version`                           |

## Advanced Options[​](#advanced-options "Direct link to Advanced Options")

The following options are available but not commonly used:

| Option            | Description                                                            |
| ----------------- | ---------------------------------------------------------------------- |
| `--show-browser`  | show the browser                                                       |
| `--viewport ints` | screen dimensions to use for your web browser (playwright driver only) |

## Environment Variables[​](#environment-variables "Direct link to Environment Variables")

The following environment variables are required or affect the behavior of the test command:

| Variable            | Required | Description                         |
| ------------------- | -------- | ----------------------------------- |
| `OPENAI_API_KEY`    | Yes      | OpenAI API key for LLM interactions |
| `ANTHROPIC_API_KEY` | No       | Anthropic API key (optional)        |
| `OPENAI_BASE_URL`   | No       | Custom OpenAI API base URL          |

## SEE ALSO[​](#see-also "Direct link to SEE ALSO")

* [testpilot](/docs/testpilot/reference/cli/testpilot) - Testpilot CLI