Pilotfile Reference

name: "My Test Suite"
login_url: "https://example.com/login"
context:
  - text: "Additional context for all test cases"
cases:
  - id: "test-001"
    name: "Login Test"
    description: "Test user login functionality"
    # ... additional test case fields

Root Level Fields

name

An optional string that provides a human-readable name for the test suite. If not specified, Testpilot will derive the name from the filename.

name: "E-commerce Checkout Tests"

An optional URL that specifies the login page for the application under test. When specified, Testpilot will perform authentication before running test cases.

login_url: "https://myapp.com/auth/login"

context

An optional array of context objects that provide additional information to all test cases in the plan. This context is inherited by all test cases and can be used to provide background information or setup details.

context:
  - text: "This application requires users to be logged in"
  - text: "All tests assume a clean database state"

cases

An array of test case objects that define the individual tests to be executed. Each test case represents a specific scenario or user journey to validate.

cases:
  - id: "checkout-001"
    name: "Complete Purchase"
    # ... test case configuration

Test Case Fields

Each test case in the cases array supports the following fields:

id (required)

A unique identifier for the test case. This ID is used internally by Testpilot to track and reference test cases.

id: "login-success-001"

name (required)

A human-readable name for the test case that describes what the test validates.

name: "Successful User Login"

description (required)

A detailed description of what the test case does and what it validates. This helps with test documentation and debugging.

description: "Verifies that a user can successfully log in with valid credentials"

steps (required)

An array of strings that describe the steps to be performed during the test. Each step represents an action or validation that should be performed.

steps:
  - "Navigate to the login page"
  - "Enter valid username and password"
  - "Click the login button"
  - "Verify successful login redirect"

url

The URL where the test case should begin execution. This is the starting point for the test.

url: "https://myapp.com/dashboard"

viewports

An array of viewport configurations that define the screen sizes at which the test should be executed. This is useful for responsive testing across different device sizes.

viewports:
  - name: "Desktop"
    size: [1920, 1080]
  - name: "Tablet"
    size: [768, 1024]
  - name: "Mobile"
    size: [375, 667]

Each viewport object contains:

name: A descriptive name for the viewport
size: An array of two integers representing width and height in pixels

context

Test case-specific context that provides additional information for this particular test. This is merged with any root-level context.

context:
  - text: "This test requires a user with admin privileges"
  - text: "Database should contain sample product data"

headers

Custom HTTP headers to include in all browser requests for this test case. The browser sends these headers with every request made during test execution.

headers:
  X-Custom-Header: "custom-value"
  Token: "Bearer ${MY_TOKEN}"
  User-Agent: "TestPilot/1.0 Custom Agent"

Important: custom headers are only supported for web testing with Playwright. They’re currently not available when testing mobile applications on Android or iOS platforms.

platform_config

Configuration for testing on different platforms (web, mobile, etc.). This allows you to specify different entry points for different platforms.

platform_config:
  url: "https://web.myapp.com"
  android_pkg: "com.mycompany.myapp"
  ios_bundle: "com.mycompany.MyApp"

The platform configuration includes:

url: The web URL for browser-based testing
android_pkg: The Android package name for mobile app testing
ios_bundle: The iOS bundle identifier for mobile app testing

Context Objects

Context objects provide additional information that can be used by Testpilot during test execution. Currently, only text-based context is supported.

context:
  - text: "User should have completed onboarding"

Each context object contains:

text: A string containing contextual information

File Formats

Testpilot supports multiple file formats for pilot files:

YAML Format (Recommended)

name: "My Tests"
cases:
  - id: "test-001"
    name: "Sample Test"
    description: "A sample test case"
    steps:
      - "Step 1"
      - "Step 2"

TOML Format

name = "My Tests"

[[cases]]
id = "test-001"
name = "Sample Test"
description = "A sample test case"
steps = ["Step 1", "Step 2"]

JSON Format

{
  "name": "My Tests",
  "cases": [
    {
      "id": "test-001",
      "name": "Sample Test",
      "description": "A sample test case",
      "steps": ["Step 1", "Step 2"]
    }
  ]
}

File Naming Conventions

Testpilot recognizes pilot files with the following naming patterns:

*.pilot - Basic pilot file
*.pilot.yaml - YAML format pilot file
*.pilot.yml - YAML format pilot file (alternative extension)
*.pilot.toml - TOML format pilot file
*.pilot.json - JSON format pilot file

Remote Files

Testpilot can load pilot files from remote URLs, including GitHub repositories. When using GitHub URLs, you can optionally set the GITHUB_TOKEN environment variable for authentication.

# Load from a direct URL
testpilot run https://example.com/tests/pilot.yaml

# Load from GitHub
testpilot run https://github.com/user/repo/raw/main/tests/pilot.yaml

Example: E-commerce Testing Suite

Here’s a complete example of a pilot file for testing an e-commerce application:

name: "E-commerce Application Tests"
login_url: "https://shop.example.com/login"
context:
  - text: "Tests assume a clean database with sample products"
  - text: "Payment gateway is in test mode"
cases:
  - id: "product-search-001"
    name: "Product Search"
    description: "Verify users can search for products and view results"
    url: "https://shop.example.com"
    viewports:
      - name: "Desktop"
        size: [1920, 1080]
      - name: "Mobile"
        size: [375, 667]
    steps:
      - "Navigate to the homepage"
      - "Enter 'laptop' in the search box"
      - "Click the search button"
      - "Verify search results are displayed"
      - "Verify at least 3 products are shown"
    context:
      - text: "Search should return products from the electronics category"
  - id: "checkout-001"
    name: "Complete Checkout Process"
    description: "Test the full checkout flow from cart to payment"
    url: "https://shop.example.com/cart"
    steps:
      - "Add a product to cart"
      - "Navigate to checkout"
      - "Fill in shipping information"
      - "Select payment method"
      - "Complete purchase"
      - "Verify order confirmation"
    context:
      - text: "User must be logged in to complete checkout"
      - text: "Use test credit card: 4111111111111111"
  - id: "mobile-app-001"
    name: "Mobile App Login"
    description: "Test login functionality in mobile application"
    platform_config:
      url: "https://shop.example.com/mobile"
      android_pkg: "com.example.shop"
      ios_bundle: "com.example.Shop"
    steps:
      - "Launch the mobile application"
      - "Tap on login button"
      - "Enter valid credentials"
      - "Verify successful login"
    viewports:
      - name: "Mobile"
        size: [375, 667]
  - id: "api-integration-001"
    name: "Test API Integration"
    description: "Verify custom headers are sent correctly with requests"
    url: "https://shop.example.com/api-test"
    headers:
      X-Shop-Client: "testpilot"
      X-Test-Environment: "staging"
      Authorization: "Bearer test-token-123"
    steps:
      - "Navigate to the API integration test page"
      - "Verify custom headers are displayed in the response"
      - "Check that authentication header was processed correctly"

This example demonstrates:

Root-level configuration with name, login URL, and context
Multiple test cases with different purposes
Cross-platform testing configuration
Responsive testing with multiple viewports
Test-specific context and configuration
Custom HTTP headers for API integration testing

Overview

Getting Started

Guides

Integrations

Advanced

Evaluation

Reference

Best Practices

Changelog

Root Level Fields

name

context

cases

Test Case Fields

id (required)

name (required)

description (required)

steps (required)

url

viewports

context

headers

platform_config

Context Objects

File Formats

YAML Format (Recommended)

TOML Format

JSON Format

File Naming Conventions

Remote Files

Example: E-commerce Testing Suite

​Root Level Fields​

​name​

​login_url​

​context​

​cases​

​Test Case Fields​

​id (required)​

​name (required)​

​description (required)​

​steps (required)​

​url​

​viewports​

​context​

​headers​

​platform_config​

​Context Objects​

​File Formats​

​YAML Format (Recommended)​

​TOML Format​

​JSON Format​

​File Naming Conventions​

​Remote Files​

​Example: E-commerce Testing Suite​

Root Level Fields

name

login_url

context

cases

Test Case Fields

id (required)

name (required)

description (required)

steps (required)

url

viewports

context

headers

platform_config

Context Objects

File Formats

YAML Format (Recommended)

TOML Format

JSON Format

File Naming Conventions

Remote Files

Example: E-commerce Testing Suite