Skip to main content
This feature is experimental. It may generate erroneous or incorrect tests, and you should review any tests before executing or checking them in.

​
Why Generate Tests?​

Manually writing test cases can be time-consuming and may miss important scenarios. Testpilot’s test generation capability helps by:
  • Creating test plans much faster than manual authoring
  • Discovering edge cases you might not have considered
  • Providing comprehensive coverage of your application
  • Standardizing test structure and approach
  • Reducing the expertise required to create quality tests

​
Generating Your First Test​

To generate tests, use the testpilot generate command followed by a prompt that describes what you want to test:
This command:
  1. Processes your prompt using AI
  2. Generates a comprehensive test plan in YAML format
  3. Saves the test plan to a file in the current directory
You should see something like the following output in a generated.pilot.yaml file:
For best results, include:
  • The URL of the application you want to test
  • Specific features or flows you want to test
  • Any edge cases or scenarios to include

​
Adding Context for Better Tests​

You can provide additional context to improve test generation quality using the --context flag:
The context files can include:
  • API documentation
  • User flow descriptions
  • Existing test examples
  • Application requirements
  • Known edge cases
You can specify multiple context files to provide more comprehensive information.

​
Customizing Output Location​

By default, generated tests are saved to the current directory. You can specify a different output directory:

​
Example: Generated Test Plan​

Here’s an example of a test plan generated from a prompt:

​
Modifying Generated Tests​

Testpilot generates a starting point for your tests, but you should:
  1. Review the tests: Carefully review all generated test cases
  2. Edit as needed: Adjust steps to match your application’s actual behavior
  3. Add specificity: Make vague steps more specific when necessary
  4. Add assertions: Ensure there are clear verification steps
  5. Remove irrelevant tests: Delete test cases that don’t apply to your application

​
Running Generated Tests​

After reviewing and adjusting the generated tests, run them with:

​
Effective Test Generation Prompts​

For the best results, create detailed, specific prompts: Basic Prompt:
Better Prompt:

​
Best Practices for Generated Tests​

  1. Be specific in your prompts: The more detail you provide, the better the generated tests
  2. Include application URLs: Always specify the URL to test in your prompt
  3. Provide context files: Add relevant documentation to improve generation quality
  4. Review before running: Always review generated tests before execution
  5. Customize as needed: Treat generated tests as a starting point, not the final product
  6. Version control: Save and version your refined test files
By using Testpilot’s test generation capabilities, you can quickly create a solid foundation for your test suite, saving time while ensuring good coverage of your application functionality.