Pull request configuration

Set up Verify to test your app when a GitHub pull request changes. Choose when a test run starts and where Verify finds the version of your app to test.

A preview is a running copy of your app with the pull request's changes. Verify needs a working preview link before it can open a browser and test those changes.

This guide covers GitHub pull requests. For GitLab merge requests, use the GitLab setup guide.

Before you start

You need:

• Administrator access to your Composal organization, with Verify enabled.
• A GitHub repository connected to your organization. On PR Testing, use Connect GitHub Repository if it is missing.
• Saved Scenarios, or Write Scenarios enabled to create focused checks from the change.
• A Verify environment in the same project. This holds the test setup, including people to test as, login credentials, and safety settings.

Open Verify → your project → PR Testing. If testing is not set up, administrators see Set Up PR Testing. It asks one question at a time, lists the steps beside it, and has Back and Next buttons:

1. Under Repositories, click Add repositories, select one or more connected repositories, and click Add. Click anywhere on a repository row to open its settings. Each has a readable URL ending in /pr-testing/<source-org>/<repository>. Use Back to repositories to return to the list. If the project has only one repository, PR Testing opens its settings directly; Add repositories is available there too. They share the project’s scenarios, personas, and credentials. If your repository is missing, choose Connect GitHub Repository in the picker. Select its GitHub organization, then use Configure Repositories on GitHub to grant Composal access. For GitLab, choose Connect GitLab Repository to connect an account and import its repository; see the GitLab setup guide. Return to Composal, refresh the repository list, and select the repository to add it to this project. You can also open Integrations → GitHub and choose Add GitHub Repository beside a project. This opens the same connection flow, even when the project has no repositories. Organization settings’ Integrations link opens Verify’s integrations page.
2. Choose Repository type: Auto, MFE / browser app, or API / backend. For browser repositories, choose who builds previews; see Choose who builds previews. API repositories need no preview setup.
3. Add Composal to Your Workflow, only when your CI builds previews.
4. Which Test Accounts Work on Your PR Previews?, only when more than one environment fits. See Choose the test setup.
5. Turn On PR Testing, where you review the settings and turn testing on.

Members see the saved settings but cannot change them.

The screenshots show the real settings page in a local example project. Replace the example names with your own project, environment, and deployment settings.

Write scenarios for each change

Organization owners and admins can turn on Automatic PR/MR scenarios in Organization Settings → Overview, then click Save changes. The setting defaults to off and applies to GitHub and GitLab. Once enabled, organization administrators can turn on Write Scenarios in a repository's PR Testing settings to create focused checks for new behavior or adapt saved scenarios that the change makes outdated. This works with GitHub pull requests and GitLab merge requests.

Verify reads the diff and identifies relevant saved scenarios before waiting for a preview. API repositories and backend-only changes skip automatic browser execution. The comment explains the diff assessment and recommends relevant scenarios. Use Run to run selected existing scenarios against the configured test environment. These results are labeled separately and do not attest that the PR commit is deployed there. If CI reports no_preview, Verify still finishes the diff assessment so backend-only changes can offer those manual runs.

For browser changes, Verify waits for the commit's ready preview and drafts up to five scenarios. It runs each draft, corrects test mistakes from the results, and requires two fresh passing trials. A separate review checks that the draft covers the change and that revisions keep its intended assertions. Product bugs, missing access, incomplete coverage, and exhausted browser budgets keep the verification blocked. The PR comment and Composal show trial results and reasons.

Expand Scenario changes in the GitHub or GitLab comment to see the change assessment, supporting changed files, and a before/after comparison of added or updated steps, expected outcomes, setup, permissions, and cleanup. The comment distinguishes proposed scenarios from changes applied to the shared suite. Long comparisons are shortened. Links to each draft's trial open its tested definition; drafts that have not run yet have no trial link.

Drafts remain scoped to the change. They join its regression run once validated. After the change merges with passing regression results and completed cleanup, new scenarios join the shared suite and validated replacements update their saved scenarios. Generated navigation uses {{base_url}} for the selected environment; the next run resolves it to that environment, including in downloaded YAML files. A human edit made since drafting takes precedence and requires review. Closing an unmerged request never changes the shared suite.

This first version covers browser scenarios for one persona: the first available persona with a configured preview account, or the signed-out visitor. It requires complete patches. Large saved suites are screened against the diff in bounded batches, including scenarios whose existing behavior needs updating. The writer keeps complete definitions and versions for the relevant scenarios; it does not take the first 100 scenarios from the suite. Its context still holds at most 100 relevant scenarios and 100 KB of definitions. An individually oversized scenario, an oversized relevant set, or uncertain relevance pauses writing for attention. If credential sanitization removes patch or saved scenario details, including URL queries or fragments, writing pauses for attention. Explicit scenario triggers still run when the review finds no changed browser behavior. Desktop, messaging, and changes requiring additional personas need authored scenarios. Trials use the environment's action permissions and the configured time limit; organization usage limits still apply. Each candidate gets at most two revisions. Descriptions provide supporting context with credentials and access-bearing URL parts removed. Writing stays blocked if those missing details are needed to establish the feature's requirements or coverage.

Set up with your coding agent

Your coding agent can answer the setup questions from your repository. On Set Up PR Testing, choose Copy Setup Prompt under Set Up With Your Agent and paste it into your agent from the repository's checkout. Once PR testing is set up, the same prompt is behind Set Up With Your Agent at the top of the page.

The agent reads your deploy configuration and recent PRs to find who builds previews, saves the setup, adds the workflow step and token when your CI builds previews, and requests a run on a recent PR to confirm.

From the CLI

com verify pr-testing takes the same answers as the page:

com verify pr-testing show --project shop
com verify pr-testing detect --project shop
com verify pr-testing setup --project shop --previews hosting --deployment Preview --on

If your CLI does not recognize pr-testing, run com update once, or use the Composal MCP setup tools. When a project tests several repositories, pass --repository <repository> to detect and setup, and --repo <repository> when requesting a PR run.

A frontend repository can use Cloudflare Pages comment previews while a backend repository uses CI previews in the same project. Configure each separately. The backend CI workflow supplies the ready UI preview URL for its exact PR head; both repositories use the project’s scenarios and access. The URL may point to staging or a per-PR UI environment; it need not be served by the source repository. Copy the GitHub Actions step from the selected repository’s PR Testing page; it uses an action revision that sends both the project and the workflow repository.

Answers you leave out keep their saved values. A new setup stays off until you pass --on. If an answer is missing, the command names it and lists the choices. Add --json for output an agent can read.

From the MCP server

The Composal MCP server has the same setup as tools: verify_read_pr_testing reads the setup and its choices, verify_detect_pr_previews looks for previews on recent PRs, and verify_save_pr_testing saves the answers. verify_check_pr then requests a run on a PR.

How a pull request is tested

1. A GitHub event, a manual request, or your workflow starts verification.
2. Verify checks the changes. General exclusions can skip changes such as documentation-only edits; an explicit matching Scenario path or event rule overrides those exclusions.
3. Verify waits for a preview from the source you choose below.
4. Verify selects saved Scenarios from explicit event rules or relevance, includes their dependencies, and runs them using the chosen environment's test setup.
5. A results comment appears on the PR with links to the run, findings, and recordings. The optional GitHub check reports the outcome.
6. With Demo Videos on, the comment also gets a short recorded walkthrough of the change.

The preview must contain the changes being tested. A working link to an older version of your app is not enough.

For deterministic coverage, configure PR File Triggers or event rules in the Scenario itself. triggers: ['./fe'] adds matching changes alongside relevance selection. Explicit on.pull_request instead selects supported events with path, target branch and optional metadata conditions. github.actor is the event sender; the PR author is github.event.pull_request.user.login. Neither establishes who authored a particular file change. Selected roots include their connected journeys, and explicit matches force fresh evidence on each commit.

Scenario selection considers the saved steps and expected outcomes. An edit to one documentation paragraph should not select unrelated navigation checks just because they visit the same article. Changes to shared code, routes or page loading can require regression checks for behavior that is meant to stay the same. A selected scenario verifies only its authored outcomes; nearby coverage does not prove a new feature works.

Choose when to run

The Trigger row controls which GitHub events can start a run. Choose Change to open the trigger switches. PR Testing turns them all on or off. Under it, Runs on Its Own lists the PR events and Runs When Asked lists the comment command. You can turn on several triggers.

PR Testing is on for new setups, with the push and ready-for-review triggers. Label and comment-command triggers are off by default. Draft PRs wait until they are ready for review. When only the comment command is on, these settings start a run only when someone posts it, and the panel says so. Scenarios with their own on.pull_request rules still run on the events they list.

After setup, the switch at the top of the summary card turns automatic testing on or off. When it is off, Trigger shows Paused. You can still test PRs by hand or from your workflow.

These general event switches govern relevance selection. A Scenario's explicit on.pull_request rule can select its own supported event types while PR Testing is enabled for that repository. Turning PR Testing off also stops those automatic PR rules. Scenario cron and programmatic dispatch are separate routes.

A comment command requests a run. Your preview setup below supplies the link that a requested run will test.

Choose who builds previews

Who Builds Your PR Previews? has three answers. Pick the one that matches how your team deploys previews:

• A Hosting Service: a host such as Vercel or Netlify deploys each PR. Composal finds the preview from a GitHub deployment or a bot comment.
• My CI: your workflow deploys the PR and sends Composal the link.
• Composal: Composal builds and deploys the PR from a hosted-app preview profile.

Each guide covers the setup and what to check when a preview is not found.

Choose the test setup

A Verify environment holds the setup the browser needs: who to test as, login credentials, and safety settings.

For a hosting service or Composal, an external environment template in the same project fits; Verify uses its setup for each new preview URL. For your CI, any environment in the project fits. When only one fits, Verify uses it without asking. When more than one fits, Which Test Accounts Work on Your PR Previews? lists each environment with its test account names. The Sign-in row shows the choice.

If no environment fits, Turn On PR Testing stays disabled and the Sign-in row links to Add Test Accounts. Save test logins in Credentials, and check that the browser can reach your preview. Private apps may also need private-network access.

Choose Change on the Limits row to set Skip PRs That Only Change and Time Limit (Minutes). A PR is skipped for path exclusions only when every changed path matches. Browser parallelism follows your organization's limit.

Add a demo video to the PR comment

Turn on Demo Videos in the PR Testing summary to record a short walkthrough of each PR. It is off by default.

Once the PR's preview is ready, one browser agent reads the PR description and changes, performs the change's main flow on the preview with your test account, and checks the result on screen. It does not need a matching saved Scenario. The PR comment's Demo section then shows one sentence describing what the recording confirmed and a play card using the demo's last retained screen capture. Demos without a retained capture show a generic play card. The link opens the recording in Composal; anyone with the link can watch it without signing in, and it stops working when the recording's retention ends.

The demo player has a single seek bar and standard playback controls. Scenario recordings also show Events and Phases timelines for investigating test results.

If the walkthrough cannot confirm the change, the comment says the demo is unavailable and why. Demo results never change scenario results or the PR check. PRs that only change skipped paths get no demo.

A project admin can revoke every demo link the PR has published, including earlier commits' demos, with DELETE /api/v1/organizations/{organization}/verify/pull_requests/{id}/demo_share, or replace the current demo's link with POST to the same path. A revoked link returns not found and the comment drops the play card.

Turn on testing and try a PR

1. On the last step, review the rows and choose Turn On PR Testing. If you switched off Trigger → PR Testing, the button reads Set Up PR Testing and saves the setup paused. The page then shows a summary card headed by the repository name.
2. In Test a Pull Request Now, enter the number of an open, ready-for-review PR.
3. Choose Verify PR and follow the run to see whether it is waiting, testing, or finished.
4. Open the results comment on GitHub to review the outcome and links to findings and recordings.

To change a setting later, choose Change on its row in the summary card: Previews, Sign-in, Trigger, or Limits. The controls open in place, with Cancel and Save.

Keep the PR Check switch on if you want Verify to report a merge check. To make that check block merging, require Composal Verify in GitHub branch protection. See merge protection setup. If you turn the check off, the results comment and run history remain available.

If a run is waiting for a preview

Open your preview source guide above for the checks specific to your deployment setup.

If no matching preview arrives within one hour, the run is blocked. Fix the preview setup, then request a new run. An accepted workflow step or a reachable preview URL alone does not mean the tests passed; read the run's final outcome.

Continue with GitHub Actions and CLI instructions, writing Scenarios, or setting up test credentials.