AGENT-05bEXPERIMENTAL•9 min read•Intermediate

AI Regression Selection Agent: Intelligent Test Impact Analysis on Every Pull Request

Using Git diff AST analysis and dependency graphs to execute only the tests affected by a code change.

#Test Impact Analysis#CI/CD Optimization#Git AST#Regression Engineering

01. The Problem

As test suites grow to hundreds or thousands of tests, running the entire suite on every tiny pull request creates CI bottlenecks, delaying developer feedback loops and increasing cloud computing costs.

02. The Concrete Engineering Scenario

A developer modifies wallet balance calculation logic in backend/src/modules/wallets/wallets.service.ts. Do we need to run all 608 tests, or only the subset affected by wallet balance changes?

Agent Role Definition:

Acts as a CI Test Selector agent that computes code coverage dependency graphs between modified source files and Playwright/Jest test files, dynamically assembling an optimal test execution list.

03. Context & Inputs Provided to Agent

•Git commit diff (git diff origin/main...HEAD).
•TypeScript import dependency tree generated via madge or ts-morph.
•Historical test execution results and defect density records.

04. Agent Reasoning & Planning Steps

Step 1: Impact Radius Calculation — Maps modified TypeScript classes and methods to directly importing controllers and services.

Step 2: Test Mapping — Identifies that wallets.service.ts affects WalletsController, TransfersProcessor, and 3 Playwright specs.

Step 3: Risk Weighting — Adds high-risk financial regression suites (WP-QA-001, WP-QA-007) to the execution plan.

Step 4: Execution Command Assembly — Emits scoped test command (npx playwright test tests/api/wallets tests/defects/wp-qa-001).

Playwright Interaction

Passes filtered grep patterns or specific file path arguments to npx playwright test, executing 28 targeted tests instead of 608.

Evidence Collection

Outputs a CI summary comment on the GitHub PR showing: 'Impact Analysis: 2 files modified -> 28 tests selected (4.6% of suite). Execution time: 4.2s (vs 34s baseline).'

05. Concrete Code Excerpt

Selective CI Execution Script

# Generated dynamic execution command based on PR git diff
npx jest src/modules/wallets/wallets.service.spec.ts \
  && npx playwright test \
      tests/api/wallets.spec.ts \
      tests/defects/wp-qa-001-settlement-refund.spec.ts \
      tests/ui/transfer-flow.spec.ts

Failure & Edge Case Handling

If a change touches shared root infrastructure (e.g., main.ts, database migration, or global env config), the agent automatically escalates to a full regression run.

Measurable Result

Reduces PR check turnaround time from 35 seconds to under 5 seconds while ensuring zero missed regressions.

Human Review & Approval Gate

Developers appreciate immediate PR feedback; nightly builds still run the 100% full regression baseline for comprehensive safety.

Key Engineering Lessons Learned

Lesson 01

Test impact analysis is essential for scaling test automation beyond 1,000 tests.

Lesson 02

Always maintain a full nightly regression sweep to catch indirect side-effects that static analysis misses.

Lesson 03

Clear visual indicators showing WHY a test was selected build developer trust in selective CI.

All Articles Inspect WrightPay Case Study →