The AI Playwright Test Planner Agent: Transforming User Stories into Testable Invariants
How to use LLMs to extract state invariants, boundary conditions, and equivalence partitions before a single locator is written.
01. The Problem
02. The Concrete Engineering Scenario
A fintech team delivers an updated 'Instant Cross-Border Transfer' feature with live foreign exchange rate quotes, dynamic fees, and KYC tier limits. The SDET needs an exhaustive test plan covering happy paths, negative validations, currency roundoff limits, and state assertions.
Acts as a Senior Test Architect agent that accepts product user stories, OpenAPI specifications, and database entity schemas to generate a structured test matrix with exact preconditions, actions, and post-state assertions.
03. Context & Inputs Provided to Agent
- •Feature User Story: 'As an authenticated user, I can transfer EUR to USD with live locked rate for 60 seconds.'
- •OpenAPI Specification for POST /api/v1/transfers and GET /api/v1/exchange-rates.
- •PostgreSQL schema for wallets (balance numeric(18,4)) and transactions (status enum).
04. Agent Reasoning & Planning Steps
Playwright Interaction
Does not touch browser directly; instead, generates test specification files with describe/it blocks, test.info().annotations, and placeholder steps using Playwright's test runner syntax.
Evidence Collection
Generates a Markdown traceability matrix linking each requirement to specific Playwright test titles and expected HTTP status codes.
05. Concrete Code Excerpt
Generated Playwright Test Specification Blueprint// Generated Blueprint from AI Test Planner Agent
import { test, expect } from '@playwright/test';
test.describe('Feature: Instant Transfer - State Invariant Verification', () => {
// Invariant 1: Source wallet balance deduction must exactly equal sendAmount + fee
test('Transfer initiation strictly debits source wallet including flat fee', async ({ request }) => {
test.info().annotations.push({ type: 'invariant', description: 'sourceBalance(t1) === sourceBalance(t0) - (amount + 25)' });
// Test implementation generated downstream
});
// Invariant 2: Expired quote token must be rejected before debit occurs
test('Expired quote token (age > 60s) returns HTTP 410 Gone without altering wallet balance', async ({ request }) => {
// Assert 410 status and verify DB balance unmodified
});
});“You are an expert SDET Architect. Analyze the provided user story and OpenAPI spec. Output a JSON array of test cases strictly classifying each as 'happy-path', 'boundary-negative', or 'concurrency-hazard'. Extract exact state invariants for PostgreSQL and HTTP status expectations.”
Failure & Edge Case Handling
If the input OpenAPI spec is ambiguous (e.g., undeclared error payload for 409 Conflict), the agent flags a 'Specification Ambiguity Warning' and prompts for clarification rather than assuming a schema.
Measurable Result
Produces an actionable 14-scenario testing matrix in under 15 seconds, uncovering 3 non-obvious failure modes (e.g., negative balance race condition, locked rate expiration mid-checkout).
Human Review & Approval Gate
The SDET reviews the generated matrix, prunes unnecessary duplicate tests, and validates that database invariants align with actual TypeORM entity constraints.
Key Engineering Lessons Learned
LLMs excel at combinatorial boundary analysis when provided with strict schema constraints.
Never allow the planning agent to invent API error structures — feed it the actual Swagger or OpenAPI schema.
Structured output formats (JSON Schema) are essential to pipe test plans into downstream test generation tools.