Test your iMessage agent before your users do

Until now the only way to test an iMessage agent was to text it from a phone and read what came back. That is fine for a demo. It doesn’t cover an agent that changes with every pull request.

Verify can now text your agent and check the replies, the same way it already tests your website.

How it works

Composal owns a set of iMessage numbers. When a scenario runs, a testing agent texts your agent from one of them, follows the steps in the scenario, and checks each reply against the outcomes you wrote. Your agent sees an ordinary conversation from a number it doesn’t know. The testing agent gives each reply up to two minutes before it fails the step.

The testing agent uses the same personas as your browser scenarios, so Maya still types in lowercase and still brings a vegan guest. When your agent sends a link, the testing agent opens it in a browser, signs in if the scenario says to, and then goes back to the conversation. It can send and receive photos and files.

When a reply is wrong

The run fails on the outcome your agent didn’t meet. The finding quotes the reply and keeps the full conversation with the run, plus the browser recording if the testing agent opened anything. Choose Send to Agent on the finding to pass it to your coding agent with the conversation included.

Scenarios are files

A Messages scenario is a version 2 scenario file with messaging: true on its actor. This is the booking scenario from the video:

version: 2
name: Book a table for four
primary_actor: maya
actors:
  maya:
    persona: Maya
    messaging: true
steps:
  - actor: maya
    instruction: Text "hey! table for 4 tmrw at 7?" and take the earliest time offered.
  - actor: maya
    instruction: Mention that one guest is vegan. If offered a reminder, say yes.
outcomes:
  - actors: [maya]
    instruction: Offers at least two real times for a party of 4.
  - actors: [maya]
    instruction: Handles the vegan request without dropping the booking.
  - actors: [maya]
    instruction: Confirms the day, time and party size in one message.
  - actors: [maya]
    instruction: Schedules a reminder when asked.

It runs from the Scenarios page, from the CLI, from your coding agent, or on a schedule against production.

On pull requests

Set the Environment’s iMessage number, with or without a website URL. For a PR preview, your CI passes the number it deployed through the Verify action’s imessage-number input, or posts it in a PR comment with the commit:

iMessage Number: +14155550142
Commit: 7c2e9b41d0a3f58c6e1b2d94a7f03c5e8b16d2a9

The PR testing guide covers both.

Get a test number

Messages is enabled per organization. Send us the number your agent answers on and we’ll set up a Composal test number for you. Then open the Environment in Verify, enter both numbers under Messages, and run a scenario. The Messages guide covers sign-in links and preview numbers.