GitHub

For a short workflow, start with testing a GitHub pull request preview.

Connect a GitHub repository and choose how its preview reaches Verify. Verify maintains a results comment and merge check for every pull request. Select PR #45, tagged with its GitHub repository, to open the PR's run history, recordings, findings, and feedback.

Use composalai/verify@1 in GitHub Actions or com verify pull-requests in your own pipeline. For GitLab merge requests, use the GitLab configuration guide.

Configure PR testing in your project

Follow the pull request configuration guide for a walkthrough of the dashboard, screenshots of each preview source, run triggers, environment templates, and a first manual run. You need organization administrator access, a connected GitHub repository, and saved project Scenarios.

This page covers GitHub-specific workflow setup, CLI commands, results, and merge checks.

Have an agent configure your GitHub workflow

On the project's PR Testing page, choose My CI and then Copy CI Setup. Give the step to your repository agent and ask it to add the step after your deployment, create an expiring token, store it with gh, and check a real preview.

To prepare the token yourself, run these commands in the target GitHub checkout:

com login
com whoami
gh auth status
gh repo view --json nameWithOwner --jq .nameWithOwner
set +x
set -o pipefail
com pat create --name verify-ci --expires-in 90d | gh secret set COMPOSAL_TOKEN
gh variable set COMPOSAL_ORG --body acme
gh variable set COMPOSAL_PROJECT --body storefront
gh secret list

Replace acme and storefront with your organization and project. The token creator needs administrator access. The pipe sends the token directly to GitHub; keep its value out of logs, prompts, and committed files. Test credentials stay in the project's credential storage.

Choose where the tested preview comes from

Configure one preview source on the project's PR Testing page:

• GitHub Deployment: read a ready deployment link from GitHub.
• Composal Preview: have Composal build and deploy the PR.
• Your CI: send a ready preview from your own workflow.
• PR Comments: read the preview link from an allowed deployment bot's comment.

See the configuration guide for the settings, screenshots, and expected behavior of each mode. The CLI and GitHub Action use that saved configuration.

Understand what gets tested

PR verification compares changed paths and available hunks with existing project Scenario descriptions, steps, and outcomes. It pins the PR head and base and records why each Scenario was selected or omitted. The relevance classifier selects Scenarios at confidence 0.7 or higher; uncertain coverage and Scenarios without usable actors remain untested. Verify creates one grouped Run for the selected journeys and runs independent work in parallel within account and organization limits. A pass covers those selected journeys on that exact preview.

Configure Skip PRs That Only Change for explicit exclusions, such as docs/** and README.md, one per line or separated by commas. Verify skips only when every changed path matches, including a renamed file's previous path. Otherwise the product-selection rules apply. Leave this field empty if you want all changes assessed.

Start and inspect a PR run with com

Install the CLI. For local use, sign in and select your organization:

com login
com whoami
com context switch acme
com verify pull-requests --help

The examples use organization acme, Composal repository shop, and GitHub PR 45. Replace these with your own values. --repo accepts a Composal repository slug or public ID, or the GitHub owner/repo name, such as --org wordware --repo wordwareai/sauna. --org selects the Composal organization; the GitHub name resolves only within that organization. If the same GitHub repository is imported into multiple Composal repositories, use one of the Composal slugs listed in the error.

These legacy CLI commands identify a source repository and require it to have only one PR Testing configuration. For project-based CI, use the action below with project; it also supports branch-only previews. Pass --org and --repo explicitly when using these commands; local commands can infer them from a configured Composal checkout.

Start an existing PR using its saved preview source:

com verify pull-requests run 45 \
  --org acme --repo shop \
  --idempotency-key pr-45-review-1 --json

The response includes a PR id, web_path, and requested_run_id. A draft or closed PR has no new request. Reuse the key when retrying an uncertain response for the same revision; choose a new key for an intentional rerun.

Inspect the history using the returned PR ID:

com verify pull-requests get <pr-id> --org acme --json

A history item has a request id, pinned SHAs, display_state, recorded scenario selection and, once prepared, run_id and execution_path. Follow the link for the grouped run's recordings and results, or read it from the CLI:

com verify runs get <run-id> --org acme --project shop --json

PR numbers select GitHub requests; the returned PR id selects pull-requests get. requested_run_id identifies the commit verification request, while run_id identifies the grouped scenario run. A request may be skipped or blocked before creating a run.

Supply a ready preview from your CI pipeline

Store a Composal administrator API token in your CI secret store and expose it as COMPOSAL_API_TOKEN to CLI steps. The com CLI reads this environment variable; the secret itself can be named COMPOSAL_TOKEN. CI does not need a browser login. See CLI configuration for endpoint and profile precedence.

Deploy the full PR head SHA and wait for readiness before handing it to Verify. On GitHub, use github.event.pull_request.head.sha; github.sha on a PR event can identify a synthetic merge commit.

If your pipeline already provides a ready Verify preview environment, use this sequence:

com verify pull-requests run 45 \
  --org acme --repo shop \
  --head-sha <full-deployed-pr-head-sha> \
  --idempotency-key pr-45-ci-attempt-1 --json

# Run after deployment succeeds and the Verify environment is ready.
com verify pull-requests preview 45 \
  --org acme --repo shop \
  --head-sha <full-deployed-pr-head-sha> \
  --environment <verify-preview-slug> --json

Use --head-sha on run in CI to refuse a stale workflow before it can request verification for a newer commit. Verify refuses a stale head or an environment from another repository. Retrying the same accepted handoff is safe; switching its target requires a fresh request. Keep the preview alive until the browser run and cleanup finish.

For external URLs, CI attests that the target serves the submitted head; Verify cannot independently discover which Git commit a generic application serves. Use immutable per-commit URLs or an appropriate deployment health check. Composal-managed previews also validate the serving deployment’s commit.

Register an external URL for anonymous coverage

For anonymous journeys, an already-deployed URL can become a ready Verify preview environment through the CLI:

printf '%s\n' '{"source_revision":"<full-deployed-pr-head-sha>"}' > preview-policy.json

com verify environments create --org acme --repo shop \
  --slug pr-45-attempt-1 --name 'PR 45 preview' \
  --target-url https://pr-45.preview.example.com \
  --target-kind preview \
  --policy preview-policy.json \
  --idempotency-key preview-pr-45-attempt-1 --json

com verify pull-requests preview 45 \
  --org acme --repo shop \
  --head-sha <full-deployed-pr-head-sha> \
  --environment pr-45-attempt-1 --json

Create the verification request with pull-requests run first, or use the automatic request already created for that revision. External environments supplied by slug must record the full deployed head SHA in source_revision when created. Verify rejects missing or different revisions at handoff, launch, and browser dispatch. Use a unique environment slug and key for each deployed target so earlier runs keep their pinned revisions. The action's preview-url input records the supplied SHA automatically.

URL registration starts with an anonymous persona. For authenticated journeys, supply a ready environment with the required personas and credential references, or use the action's preview-url path, which inherits your configured template. See environments and scenario setup.

Add the handoff to GitHub Actions

After a deployment step named deploy has produced a ready Verify environment slug in verify_environment, add these steps to the same job. Install a CLI release that supports pull-requests:

- name: Install Composal CLI
  shell: bash
  run: |
    curl -fsSL https://dl.composal.com/install.sh | sh
    echo "$HOME/.composal/bin" >> "$GITHUB_PATH"

- name: Hand the PR preview to Verify
  shell: bash
  env:
    COMPOSAL_API_TOKEN: ${{ secrets.COMPOSAL_TOKEN }}
    COMPOSAL_ORG: ${{ vars.COMPOSAL_ORG }}
    COMPOSAL_REPO: ${{ vars.COMPOSAL_REPO }}
    PR_NUMBER: ${{ github.event.pull_request.number }}
    PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
    VERIFY_PREVIEW_ENV: ${{ steps.deploy.outputs.verify_environment }}
  run: |
    set -euo pipefail
    com verify pull-requests run "$PR_NUMBER" \
      --org "$COMPOSAL_ORG" --repo "$COMPOSAL_REPO" \
      --head-sha "$PR_HEAD_SHA" \
      --idempotency-key "github:${GITHUB_RUN_ID}:${GITHUB_RUN_ATTEMPT}" --json
    com verify pull-requests preview "$PR_NUMBER" \
      --org "$COMPOSAL_ORG" --repo "$COMPOSAL_REPO" \
      --head-sha "$PR_HEAD_SHA" --environment "$VERIFY_PREVIEW_ENV" --json

This step submits the target; success means accepted, not passed. Read the PR history until display_state is terminal, then inspect its run_id and per-scenario results. A request may be skipped or blocked before any Run is prepared. The action below handles this lifecycle for you.

com verify runs get <run-id> --org acme --project shop --json \
  | jq -e '.run.state == "terminal" and .run.outcome == "passed"'

Cleanup remains a separate reported state; it does not overwrite the product verdict.

Use the GitHub Action helper

Use the composalai/verify@1 action after your preview deployment is ready. Trigger from GitHub Actions appears for every preview source in PR Testing with a copyable step. Your workflow controls when the Action runs, independently of the automatic event switches.

For a CI-owned deployment, select Your CI and an external template in Verify, then add the action after your ready deployment:

- uses: composalai/verify@1
  id: verify
  with:
    token: ${{ secrets.COMPOSAL_TOKEN }}
    org: ${{ vars.COMPOSAL_ORG }}
    project: ${{ vars.COMPOSAL_PROJECT }}
    preview-url: ${{ steps.deploy.outputs.url }}

Set the required COMPOSAL_PROJECT variable to the Composal project slug or public ID. URL handoffs automatically register preview environments in that project’s Environments page with the PR number and source branch. Repeated handoffs reuse their environment; new verification requests retain separate preview records. After the connected GitHub App observes a merge, Composal archives the generated previews once active verification finishes. Show Archived displays their retained history. Archival does not delete the deployment at your provider.

Supply pr or branch when needed; neither is required when GitHub supplies the context. PR events infer the number, branch, and exact head SHA. Branch pushes infer the branch and deployed push SHA. workflow_run infers a single PR or its source branch. The required project resolves the source repository from its saved PR Testing configuration; the action has no repository input. It requires no checkout, installed CLI, or separate GitHub token; the connected GitHub App maintains the results comment.

For a branch deployment with no open PR, supply the preview URL and project. The action registers the environment and succeeds with status: preview_registered and an environment-id; it does not report a browser test pass. When a PR later opens for that branch, Verify links the branch previews to it and archives them after merge. If a branch has multiple open PRs, supply pr explicitly. Branches registered in multiple projects need an explicit project selection before they can be linked automatically.

# After a deployment in a branch push workflow; branch and SHA are auto-detected.
- uses: composalai/verify@1
  with:
    token: ${{ secrets.COMPOSAL_TOKEN }}
    org: ${{ vars.COMPOSAL_ORG }}
    project: ${{ vars.COMPOSAL_PROJECT }}
    preview-url: ${{ steps.deploy.outputs.url }}
    # For another event type, set branch OR pr and the deployed sha explicitly:
    # branch: feature/checkout
    # sha: <full deployed commit SHA>

With Composal Preview or GitHub Deployment configured, omit preview-url. This complete workflow can request and wait for verification without a deployment step:

name: Verify PR
on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
permissions:
  contents: read
concurrency:
  group: verify-${{ github.event.pull_request.number }}
  cancel-in-progress: true
jobs:
  verify:
    if: ${{ !github.event.pull_request.draft && github.event.pull_request.head.repo.full_name == github.repository }}
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - uses: composalai/verify@1
        id: verify
        with:
          token: ${{ secrets.COMPOSAL_TOKEN }}
          org: ${{ vars.COMPOSAL_ORG }}
          project: ${{ vars.COMPOSAL_PROJECT }}

Set COMPOSAL_TOKEN as an Actions secret containing a Composal administrator API token, COMPOSAL_ORG as a variable containing the organization slug, and COMPOSAL_PROJECT as the project slug or public ID. GitHub withholds repository secrets from fork-triggered workflows; the example skips those jobs. Use your existing trusted deployment process for fork previews rather than exposing secrets with pull_request_target.

The action waits by default. Passed and skipped requests succeed; failed, blocked, superseded, cancelled, cleanup-blocked, and timed-out outcomes fail the step. A wait timeout also fails the step, but the server run continues and its history link remains available. Cancelling a workflow job likewise does not cancel a server run already launched.

Outputs are url (verification run or branch environment), pull-request-id, run-id (verification request), test-run-id (verification Run, including while waiting for a preview), environment-id (branch registration), and status. The job summary links to results. With wait: 'false', a successful step means the request or handoff was accepted. Retries within the same workflow attempt reuse its request; rerunning the workflow creates a fresh one. The action refuses a stale workflow head before requesting a newer-head run.

Read the GitHub comment and full history

Composal Verify appears in the PR’s merge checks for the exact head commit. It shows queued, waiting for preview, and running states, then succeeds for passed coverage, skips unnecessary testing, or reports a failure or setup blocker. New revisions and reruns replace the active check; earlier requests are cancelled. You can require this check through GitHub branch protection. Existing GitHub App installations may need to approve the checks write permission.

To keep Verify out of merge checks, turn off GitHub Check on the PR Testing page. New commits then get no check. GitHub cannot delete a check, so Verify closes an open check as neutral and it stops holding up merges. The results comment and run history continue. Remove the check from branch protection first if it is required.

Verify maintains one results comment per PR. It shows the commit, tested preview, selected areas, selection reasons, current state, and problems to address. Critical and high-severity findings link to issue recordings. These are authenticated links into Verify; private videos are not embedded as public GitHub attachments.

In Verify → Runs, select a run labelled PR #45 to review that verification.

PR verification opens directly as a run. Its header links to the pull request and shows the branch and commit; each scenario shows why it was selected. The actions menu includes Rerun Verification, Run tests manually, and a link to the results comment.

On a PR verification run, open the actions menu and choose Run tests manually, then use Choose tests to search all available saved tests in its project, including tests outside the automatic selection. Select one or several, then choose Run selected. This requests only those tests against the current commit’s preview and saves the results as a new run. Rerun Verification continues to use automatic selection. Manual selection is available while the run’s commit is still the PR’s current commit. From an older run, use Rerun Verification to verify the latest commit. Only administrators can request tests; archived tests and tests belonging to another project are unavailable.

A run exists while it waits for a preview. Skipped runs and setup blockers show their reason in the same run view, even when no scenarios ran. Earlier results and findings remain available in the runs list with their PR number. Requests superseded before their preview was ready are omitted from that list. Older requests that never created a run are not backfilled.

Resolve setup and CI problems

Continue with the Source Control overview, CLI reference, scenario and MCP reference, or preview hosting guide.