Test SMS OTP sign-up end to end with a real phone number

The helper both tests use

Three functions do the API work: buy a number, note the newest text already on it, and wait for a text newer than that. They call the REST API with fetch and nothing else. Put an API key that can buy in VERIFYWITH_API_KEY.

// verifywith.ts: the API calls a sign-up test needs (Node 18 or later, for fetch)
const API = "https://api.verifywith.io/v1";

export interface TestNumber {
  id: string;
  phoneNumber: string; // "+1" and ten digits
}

interface NumberObject {
  id: string;
  status: string;
  phoneNumber: string | null;
}

interface Message {
  id: string;
  body: string;
  code: string | null;
}

async function api<T>(path: string, init: RequestInit = {}): Promise<T> {
  const res = await fetch(API + path, {
    ...init,
    headers: {
      Authorization: `Bearer ${process.env.VERIFYWITH_API_KEY}`,
      "Content-Type": "application/json",
      ...init.headers,
    },
  });
  const body = await res.json();
  if (!res.ok) throw new Error(`${res.status} ${body.error}: ${body.message} ${body.url ?? ""}`);
  return body as T;
}

/** Buys a one-day number. Sending the same key again returns the same number. */
export async function buyNumber(idempotencyKey: string): Promise<TestNumber> {
  const r = await api<{ order: { nextAction: { url: string } | null }; number: NumberObject }>("/numbers", {
    method: "POST",
    headers: { "Idempotency-Key": idempotencyKey },
    body: JSON.stringify({ duration: "day", nickname: "e2e" }),
  });
  if (r.order.nextAction) throw new Error(`The card holder must confirm the payment: ${r.order.nextAction.url}`);
  let n = r.number;
  for (let i = 0; n.status === "provisioning" && i < 60; i++) {
    await new Promise((ok) => setTimeout(ok, 1000));
    n = await api<NumberObject>(`/numbers/${n.id}`);
  }
  if (n.status !== "active" || !n.phoneNumber) throw new Error(`Number ${n.id} is ${n.status}`);
  return { id: n.id, phoneNumber: n.phoneNumber };
}

/** The newest text on the number, or null. Take it before your app sends the code. */
export async function latestMessageId(numberId: string): Promise<string | null> {
  const page = await api<{ data: Message[] }>(`/numbers/${numberId}/messages?limit=1`);
  return page.data[0]?.id ?? null;
}

/** Waits for a text newer than `after` and returns its code. */
export async function waitForCode(numberId: string, after: string | null, timeoutMs = 120_000): Promise<string> {
  const deadline = Date.now() + timeoutMs;
  for (;;) {
    let message: Message | null = null;
    if (!after) {
      // the number had no texts before: any text on it now is this test's
      const page = await api<{ data: Message[] }>(`/numbers/${numberId}/messages?limit=1`);
      message = page.data[0] ?? null;
    }
    const left = Math.floor((deadline - Date.now()) / 1000);
    if (!message && left > 0) {
      const q = new URLSearchParams({ timeoutSeconds: String(Math.min(left, 30)) });
      if (after) q.set("afterMessageId", after);
      const r = await api<{ message: Message | null }>(`/numbers/${numberId}/messages/next?${q}`);
      message = r.message;
    }
    if (message) {
      if (!message.code) throw new Error(`A text arrived with no code in it: ${message.body}`);
      return message.code;
    }
    if (left <= 0) throw new Error(`No text on number ${numberId} within ${timeoutMs / 1000} seconds`);
  }
}

This drives your own application’s sign-up form, on a build you control. It is not a way to create accounts on someone else’s service; our acceptable use policy forbids bulk sign-ups.

A Playwright test

A worker-scoped fixture buys one number per worker and every test in that worker reuses it. Each test notes the newest text before it asks for a code, so a code from an earlier test is never read by mistake.

// signup.spec.ts
import { test as base, expect } from "@playwright/test";
import { buyNumber, latestMessageId, waitForCode, type TestNumber } from "./verifywith";

// One number per worker. A worker that restarts after a failed test sends
// the same Idempotency-Key, so it gets the same number back instead of a new one.
const test = base.extend<{}, { phone: TestNumber }>({
  phone: [
    async ({}, use, workerInfo) => {
      const run = process.env.TEST_RUN_ID ?? "local";
      await use(await buyNumber(`e2e-${run}-worker-${workerInfo.parallelIndex}`));
    },
    { scope: "worker" },
  ],
});

test("sign-up by text message", async ({ page, phone }) => {
  test.setTimeout(180_000);
  const before = await latestMessageId(phone.id); // earlier tests' texts don't count

  await page.goto("/sign-up");
  // your form's own labels; use phone.phoneNumber.slice(2) if the form adds +1 itself
  await page.getByLabel("Phone number").fill(phone.phoneNumber);
  await page.getByRole("button", { name: "Send code" }).click();

  const code = await waitForCode(phone.id, before, 120_000);
  await page.getByLabel("Code").fill(code);
  await page.getByRole("button", { name: "Verify" }).click();
  await expect(page.getByText("Welcome")).toBeVisible();
});

Set TEST_RUN_ID to your CI’s run number. Without it the key is e2e-local-worker-0 and so on, so local runs within the same day get the same numbers back rather than buying new ones.

The same test in Cypress, through cy.task

Cypress runs tests in the browser, so the API calls go in setupNodeEvents and the test reaches them with cy.task. The Node process lives for the whole cypress run, so one number serves every spec.

// cypress.config.ts: the API calls run in Node, through cy.task
import { defineConfig } from "cypress";
import { buyNumber, latestMessageId, waitForCode, type TestNumber } from "./verifywith";

let phone: Promise<TestNumber> | undefined; // one number for the whole cypress run

export default defineConfig({
  e2e: {
    baseUrl: "http://localhost:3000",
    setupNodeEvents(on) {
      on("task", {
        "sms:number": () => (phone ??= buyNumber(`e2e-${process.env.TEST_RUN_ID ?? "local"}`)),
        // null when the number has no texts yet (a task may return null, never undefined)
        "sms:mark": (numberId: string) => latestMessageId(numberId),
        "sms:code": ({ numberId, after }: { numberId: string; after: string | null }) =>
          waitForCode(numberId, after, 120_000),
      });
    },
  },
});

// cypress/e2e/signup.cy.ts
import type { TestNumber } from "../../verifywith";

describe("sign-up by text message", () => {
  it("accepts the code it sends", () => {
    cy.task<TestNumber>("sms:number", null, { timeout: 90_000 }).then((phone) => {
      cy.task<string | null>("sms:mark", phone.id).then((after) => {
        cy.visit("/sign-up");
        cy.get("input[name=phone]").type(phone.phoneNumber);
        cy.contains("button", "Send code").click();
        cy.task<string>("sms:code", { numberId: phone.id, after }, { timeout: 150_000 }).then((code) => {
          cy.get("input[name=code]").type(code);
          cy.contains("button", "Verify").click();
          cy.contains("Welcome").should("be.visible");
        });
      });
    });
  });
});

A task’s default timeout can be shorter than the wait, so the buying and waiting tasks get their own timeout. If you split specs across several machines, add the machine’s index to the key, the way the Playwright fixture adds the worker’s.

Old texts and timeouts

The test ignores older texts by passing the id of the newest one as afterMessageId: the wait then answers only for a text received after it. An id works where a clock doesn’t, because your CI machine’s time and ours never have to agree. If you would rather compare times, every message has receivedAt in UTC.

Take that id before the click that sends the code, not after. A text can arrive before your wait starts, and a wait with no afterMessageId counts only texts that arrive after the call. For a number with no texts yet, the helper reads the list on every pass for the same reason.

One wait request is held open for at most 90 seconds. The helper asks in 30-second slices until its own deadline, 120 seconds in the examples, and the test timeout sits above that so the failure you see is “no text within 120 seconds” with the number’s id, not a bare test timeout. A text with no code in it fails with the text, which usually means your message template changed.

Parallel workers

Give each parallel worker its own number. Two workers on one number both receive both codes, and neither can tell which is its own. One number per worker keeps each test’s wait to its own texts, and tests inside a worker run one at a time, so they can share.

Three limits shape how wide you can go:

• At most 10 waits open at once per account. An eleventh answers 429 with a Retry-After.
• A key may make 20 buying calls an hour.
• A buying key’s default daily limit, $40.00, covers 10 one-day numbers. Raise it for the CI key on the Developers page, or keep numbers longer.

Use a separate key for CI, so you can change its limits or revoke it without touching anything else.

What a run costs

One number per run is the usual answer. A number per test is the expensive mistake, and a number you keep is cheapest for a suite that runs every night.

A day number lasts 24 hours, so it can serve a whole run and that day’s reruns. For a kept number, store its id as a CI variable and skip the purchase; runs that overlap still need a number each. The price is held on the card until the first text arrives or 24 hours pass, and a number that received nothing (the run broke before sending) can be cancelled from 5 minutes to 24 hours after purchase.

When a mock is the better tool

For almost every test. Stub the SMS sender in your test environment, record the code it would have sent, and read it from there. That costs nothing, takes milliseconds, and never fails because a network was slow.

A real number earns its cost in a few smoke tests, nightly or before a release. It proves the parts a mock skips: your SMS provider account and its sender, the message template, the code in it, and delivery to a US number. If your sign-up checks numbers before sending, the real run also shows what that check does.

It is the wrong tool for load tests, for codes sent by voice call (the number is receive-only), and for numbers outside the US.

Questions people ask

Can I receive an OTP in GitHub Actions or another CI?

Yes. Store the API key as a secret named VERIFYWITH_API_KEY; the runner needs only outbound HTTPS to call the API. Nothing on your side needs a phone.

Can one number receive codes for many test accounts?

Yes. A number receives unlimited texts from any sender while you have it. Whether your app lets one number sit on many accounts is your app’s own rule, so a test may need to delete its account at the end.

Does a failed test run still cost money?

The price is held on the card until the first text arrives or 24 hours pass. If the run broke before your app sent anything, cancel the number from 5 minutes to 24 hours after purchase; when our supplier accepts, the money comes back.

Can I test a code sent by voice call?

No. The number cannot make or receive calls, so a code read out by a call never arrives. Test the text path with a real number and the call path with a mock.