CI/CD Integration

Run TestBro tests automatically in your CI/CD pipeline. This guide covers GitHub Actions, but the same principles apply to any CI system.

Prerequisites

• A TestBro API token (see Authentication)
• A project with test cases (see Test Cases)
• Node.js 18+ available in your CI environment

GitHub Actions

Basic Workflow

Create .github/workflows/testbro.yml:

name: TestBro E2E Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install TestBro CLI
        run: npm install -g @testbro/cli

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run tests
        env:
          TEST_BRO_TOKEN: ${{ secrets.TESTBRO_TOKEN }}
          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
        run: |
          testbro run \
            --url ${{ vars.APP_URL }} \
            --project ${{ vars.TESTBRO_PROJECT_ID }} \
            --mode hybrid \
            --format json

Environment Variables

Add these as repository secrets in GitHub:

Add these as repository variables:

Running Against a Preview Deployment

For pull request workflows, you often want to test against a preview deployment. Here is an example using Vercel:

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Wait for Vercel deployment
        uses: patrickedqvist/wait-for-vercel-preview@v1.3.2
        id: vercel
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          max_timeout: 300

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install TestBro CLI
        run: npm install -g @testbro/cli

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run tests against preview
        env:
          TEST_BRO_TOKEN: ${{ secrets.TESTBRO_TOKEN }}
          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
        run: |
          testbro run \
            --url ${{ steps.vercel.outputs.url }} \
            --project ${{ vars.TESTBRO_PROJECT_ID }} \
            --mode hybrid

JSON Output

Use --format json in CI to get machine-readable output:

testbro run --url $APP_URL --project $PROJECT_ID --mode hybrid --format json

The CLI exits with code 1 when any test fails, which causes the CI job to fail.

Generating Tests from PRDs

You can also generate tests from files in your repository during CI:

- name: Generate and run tests from spec
  env:
    TEST_BRO_TOKEN: ${{ secrets.TESTBRO_TOKEN }}
    OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
  run: |
    testbro run \
      --url ${{ vars.APP_URL }} \
      --file docs/acceptance-criteria.md \
      --project ${{ vars.TESTBRO_PROJECT_ID }} \
      --mode ai \
      --yes

Tips

• Use --mode selector for fastest CI runs when you have stable selectors
• Use --mode hybrid for resilient CI that can adapt to UI changes
• Set --timeout to a higher value in CI (e.g., 60000) to account for slower environments
• Use the --yes flag to auto-confirm any prompts in non-interactive mode
• Store your test-bro.config.json in the repository so the CLI picks up project settings automatically

Next Steps

• CLI Reference -- All command options
• Execution Modes -- Choosing the right mode for CI
• Projects -- Managing projects and test cases