Test CLI Reference

Relay provides a unified relay test command for running the project's Python test suites locally or in automation. The command wraps pytest with sensible defaults so developers and CI systems can produce consistent reports.

Minimum supported CLI

The workflows on this page require Relay CLI v1.10.0 or later for multi-suite execution and JUnit XML exports. Update any pinned installer references when upgrading the CLI so CI jobs continue to receive a compatible version, and verify installer checksums (or pin exact release artefacts) when curling scripts to guard against supply-chain tampering.

Command Overview

relay test --suite unit --suite integration --results-dir dev_process/results/test_runs

• --suite selects one or more pytest markers (unit, integration, manifest).
• --results-dir writes timestamped JUnit XML files so failures can be reviewed in CI.
• --with-sandbox starts the Docker sandbox before running tests when integration fixtures need live services.

By default, results are stored under dev_process/results/test_runs. You can override the path to point at a CI-friendly artifacts directory.

Supported suites

relay test currently recognises the following suites:

• unit
• integration
• manifest

Select one or more suites that align with your pipeline goals. Supplying any other value causes the command to exit before invoking pytest.

CI Examples

Use the following examples to plug relay test into popular CI providers. Each workflow runs the CLI and uploads generated test artifacts so reviewers can inspect failures. Set the RELAY_API_KEY secret (or equivalent) in your CI platform before running these jobs.

GitHub Actions

name: relay-tests
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      RELAY_API_KEY: ${{ secrets.RELAY_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Cache pip dependencies
        uses: actions/cache@v4
        with:
          path: ~/.cache/pip
          key: pip-${{ runner.os }}-${{ hashFiles('requirements.txt') }}
          restore-keys: |
            pip-${{ runner.os }}-
      - name: Install project dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - name: Install Relay CLI
        run: curl -L https://deployrelay.com/cli/install.sh | bash
      - name: Run relay test suites
        run: relay test --suite unit --suite integration --results-dir artifacts/relay-tests
      - name: Upload pytest artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: relay-pytest-results
          path: artifacts/relay-tests/

🔐 Security tip: When using curl to download the Relay CLI installer, prefer pinning to a specific release (for example, via a tagged tarball URL) or verifying the published SHA256 checksum before executing the script.

GitLab CI/CD

stages:
  - test

relay-tests:
  stage: test
  image: python:3.11
  variables:
    RELAY_API_KEY: $RELAY_API_KEY
  before_script:
    - python -m pip install --upgrade pip
    - pip install -r requirements.txt
    - curl -L https://deployrelay.com/cli/install.sh | bash
  script:
    - relay test --suite unit --suite integration --results-dir artifacts/relay-tests
  artifacts:
    when: always
    paths:
      - artifacts/relay-tests/

CircleCI

version: 2.1

jobs:
  relay-tests:
    docker:
      - image: cimg/python:3.11
    environment:
      RELAY_API_KEY: << pipeline.parameters.relay_api_key >>
    steps:
      - checkout
      - run:
          name: Install project dependencies
          command: |
            python -m pip install --upgrade pip
            pip install -r requirements.txt
      - run:
          name: Install Relay CLI
          command: curl -L https://deployrelay.com/cli/install.sh | bash
      - run:
          name: Run relay test suites
          command: relay test --suite unit --suite manifest --results-dir artifacts/relay-tests
      - store_artifacts:
          path: artifacts/relay-tests

workflows:
  relay:
    jobs:
      - relay-tests:
          context: relay-secrets

These snippets assume a shared artifacts/relay-tests directory for output. Adjust suite selections or contexts to match your project's needs, and ensure authentication variables are provided through each platform's secret management.

💡 Tip: If your repository relies on helper scripts such as make install to prepare virtual environments, swap the explicit pip install commands in these examples for those project-specific bootstrapping commands.