ankit.pandey
Back to SDET Engineering Journal
AGENT-01EXPERIMENTAL•8 min read•Intermediate

The AI Playwright Test Planner Agent: Transforming User Stories into Testable Invariants

How to use LLMs to extract state invariants, boundary conditions, and equivalence partitions before a single locator is written.

#AI Test Planning#Playwright#Prompt Engineering#Requirements Analysis

01. The Problem

Engineers frequently dive straight into writing Playwright script locators without formalizing state invariants, resulting in fragile tests that assert happy paths while missing critical edge cases like zero-balance debits or concurrent submissions.

02. The Concrete Engineering Scenario

A fintech team delivers an updated 'Instant Cross-Border Transfer' feature with live foreign exchange rate quotes, dynamic fees, and KYC tier limits. The SDET needs an exhaustive test plan covering happy paths, negative validations, currency roundoff limits, and state assertions.

Agent Role Definition:

Acts as a Senior Test Architect agent that accepts product user stories, OpenAPI specifications, and database entity schemas to generate a structured test matrix with exact preconditions, actions, and post-state assertions.

03. Context & Inputs Provided to Agent

  • •Feature User Story: 'As an authenticated user, I can transfer EUR to USD with live locked rate for 60 seconds.'
  • •OpenAPI Specification for POST /api/v1/transfers and GET /api/v1/exchange-rates.
  • •PostgreSQL schema for wallets (balance numeric(18,4)) and transactions (status enum).

04. Agent Reasoning & Planning Steps

Step 1: Domain Boundary Extraction — Identifies numerical boundary limits (min 1.00 EUR, max tier limits, flat 25.00 EUR fee).
Step 2: State Machine Mapping — Models state transitions: INITIATED -> DEBITED -> SETTLING -> COMPLETED / FAILED.
Step 3: Equivalence Partitioning — Splits test cases into Valid Funded, Insufficient Funds, Expired Rate Quote, Malformed UUID Recipient, and Duplicate Submission.
Step 4: Output Generation — Outputs a structured JSON test matrix ready for automated generation.

Playwright Interaction

Does not touch browser directly; instead, generates test specification files with describe/it blocks, test.info().annotations, and placeholder steps using Playwright's test runner syntax.

Evidence Collection

Generates a Markdown traceability matrix linking each requirement to specific Playwright test titles and expected HTTP status codes.

05. Concrete Code Excerpt

Generated Playwright Test Specification Blueprint
// Generated Blueprint from AI Test Planner Agent
import { test, expect } from '@playwright/test';

test.describe('Feature: Instant Transfer - State Invariant Verification', () => {
  // Invariant 1: Source wallet balance deduction must exactly equal sendAmount + fee
  test('Transfer initiation strictly debits source wallet including flat fee', async ({ request }) => {
    test.info().annotations.push({ type: 'invariant', description: 'sourceBalance(t1) === sourceBalance(t0) - (amount + 25)' });
    // Test implementation generated downstream
  });

  // Invariant 2: Expired quote token must be rejected before debit occurs
  test('Expired quote token (age > 60s) returns HTTP 410 Gone without altering wallet balance', async ({ request }) => {
    // Assert 410 status and verify DB balance unmodified
  });
});
Example Agent System Prompt:

“You are an expert SDET Architect. Analyze the provided user story and OpenAPI spec. Output a JSON array of test cases strictly classifying each as 'happy-path', 'boundary-negative', or 'concurrency-hazard'. Extract exact state invariants for PostgreSQL and HTTP status expectations.”

Failure & Edge Case Handling

If the input OpenAPI spec is ambiguous (e.g., undeclared error payload for 409 Conflict), the agent flags a 'Specification Ambiguity Warning' and prompts for clarification rather than assuming a schema.

Measurable Result

Produces an actionable 14-scenario testing matrix in under 15 seconds, uncovering 3 non-obvious failure modes (e.g., negative balance race condition, locked rate expiration mid-checkout).

Human Review & Approval Gate

The SDET reviews the generated matrix, prunes unnecessary duplicate tests, and validates that database invariants align with actual TypeORM entity constraints.

Key Engineering Lessons Learned

Lesson 01

LLMs excel at combinatorial boundary analysis when provided with strict schema constraints.

Lesson 02

Never allow the planning agent to invent API error structures — feed it the actual Swagger or OpenAPI schema.

Lesson 03

Structured output formats (JSON Schema) are essential to pipe test plans into downstream test generation tools.