Flaky Test Hunter

Goal

Remove flaky tests

How to Run

Run this loop in your Node.js project to automatically detect and eliminate flaky tests by repeatedly executing your test suite and making targeted fixes.

1. 01

   Project Setup

   Ensure your Node.js project has npm test configured in package.json and all necessary dependencies installed.

2. 02

   Launch Agent

   Start the Flaky Test Hunter loop in your preferred coding agent environment (Cursor, Claude Code, etc.).

3. 03

   Monitor Progress

   The agent will run npm test up to 20 times consecutively. If tests pass consistently, the loop exits. If failures occur intermittently, the agent works to fix them.

4. 04

   Review Changes

   After completion, review any test modifications, deletions, or configuration changes made by the agent to ensure correctness.

Workflow Steps

1. 01

   Run npm test once and capture the full output including exit code

2. 02

   Analyze test output to identify any failed tests

3. 03

   If tests pass, increment consecutive pass counter and run again until reaching 20 passes

4. 04

   If tests fail, check if the same test failed in previous runs - if yes, it's consistently failing (not flaky)

5. 05

   If different tests fail across runs or the same test sometimes passes/sometimes fails, mark as flaky

6. 06

   Investigate flaky tests by examining their implementation and dependencies

7. 07

   Fix or remove flaky tests using best practices: stabilize async operations, remove shared state dependencies, add proper assertions

8. 08

   Re-run tests to verify the fix eliminated the flakiness without introducing new issues

9. 09

   Repeat the process until 20 consecutive successful test runs are achieved or max iterations reached

Start the "Flaky Test Hunter" loop.

Goal: Remove flaky tests
Max iterations: 10
Between iterations run: npm test
Exit when: 20 consecutive runs pass


You are an autonomous agent specialized in eliminating flaky tests. Begin by running 'npm test' to establish a baseline. Then execute the test suite repeatedly to identify tests that pass and fail inconsistently. For each flaky test detected, analyze its code to find the root cause (race conditions, shared state, timing issues, etc.) and implement a proper fix that maintains test coverage while ensuring consistent behavior. Continue this process until you achieve 20 consecutive successful test runs or reach the maximum iteration limit.

Self-pace this loop. After each iteration, run `npm test` and evaluate the output, and only continue if the exit condition is not met (20 consecutive runs pass). Stop when the exit condition passes or 10 iterations are reached. Give a short status update each pass.