bmod-tea This folder is the Test Architect module's record, not something to run. Invoke the skill with ; it sets the module up if it never was, and otherwise reports its state. If there is no skill, say so and offer .

|| echo \"\")\n echo \"changed_specs=${CHANGED_SPECS}\" >> $GITHUB_OUTPUT\n echo \"Changed specs: ${CHANGED_SPECS}\"\n\n - name: Run burn-in on changed specs (10 iterations)\n if: steps.changed-tests.outputs.changed_specs != ''\n run: |\n SPECS=\"${{ steps.changed-tests.outputs.changed_specs }}\"\n echo \"Running burn-in: 10 iterations on changed specs\"\n for i in {1..10}; do\n echo \"Burn-in iteration $i/10\"\n npm run test -- $SPECS || {\n echo \"❌ Burn-in failed on iteration $i\"\n exit 1\n }\n done\n echo \"✅ Burn-in passed - 10/10 successful runs\"\n\n - name: Upload artifacts on failure\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: burn-in-failure-artifacts\n path: |\n test-results/\n playwright-report/\n screenshots/\n retention-days: 7\n\n test-e2e-sharded:\n name: E2E Tests (Shard ${{ matrix.shard }}/${{ strategy.job-total }})\n needs: [install-dependencies, test-changed-specs]\n runs-on: ubuntu-latest\n timeout-minutes: 30\n strategy:\n fail-fast: false # Run all shards even if one fails\n matrix:\n shard: [1, 2, 3, 4]\n steps:\n - name: Checkout code\n uses: actions/checkout@v4\n\n - name: Setup Node.js\n uses: actions/setup-node@v4\n with:\n node-version-file: ${{ env.NODE_VERSION_FILE }}\n cache: 'npm'\n\n - name: Restore dependencies\n uses: actions/cache@v4\n with:\n path: |\n ~/.npm\n node_modules\n ~/.cache/ms-playwright\n key: ${{ env.CACHE_KEY }}\n\n - name: Run E2E tests (shard ${{ matrix.shard }})\n run: npm run test:e2e -- --shard=${{ matrix.shard }}/4\n env:\n TEST_ENV: staging\n CI: true\n\n - name: Upload test results\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: test-results-shard-${{ matrix.shard }}\n path: |\n test-results/\n playwright-report/\n retention-days: 30\n\n - name: Upload JUnit report\n if: always()\n uses: actions/upload-artifact@v4\n with:\n name: junit-results-shard-${{ matrix.shard }}\n path: test-results/junit.xml\n retention-days: 30\n\n merge-test-results:\n name: Merge Test Results & Generate Report\n needs: test-e2e-sharded\n runs-on: ubuntu-latest\n if: always()\n steps:\n - name: Download all shard results\n uses: actions/download-artifact@v4\n with:\n pattern: test-results-shard-*\n path: all-results/\n\n - name: Merge HTML reports\n run: |\n npx playwright merge-reports --reporter=html all-results/\n echo \"Merged report available in playwright-report/\"\n\n - name: Upload merged report\n uses: actions/upload-artifact@v4\n with:\n name: merged-playwright-report\n path: playwright-report/\n retention-days: 30\n\n - name: Comment PR with results\n if: github.event_name == 'pull_request'\n uses: daun/playwright-report-comment@v3\n with:\n report-path: playwright-report/\n```\n\n**Key Points**:\n\n- **Install once, reuse everywhere**: Dependencies cached across all jobs\n- **Burn-in first**: Changed specs run 10x before full suite\n- **Fail-fast disabled**: All shards run to completion for full evidence\n- **Parallel execution**: 4 shards cut execution time by ~75%\n- **Artifact retention**: 30 days for reports, 7 days for failure debugging\n\n---\n\n### Example 2: Burn-In Loop Pattern (Standalone Script)\n\n**Context**: Reusable bash script for burn-in testing changed specs locally or in CI.\n\n**Implementation**:\n\n```bash\n#!/bin/bash\n# scripts/burn-in-changed.sh\n# Usage: ./scripts/burn-in-changed.sh [iterations] [base-branch]\n\nset -e # Exit on error\n\n# Configuration\nITERATIONS=${1:-10}\nBASE_BRANCH=${2:-main}\nSPEC_PATTERN='\\.(spec|test)\\.(ts|js|tsx|jsx)

bmod-tea This folder is the Test Architect module's record, not something to run. Invoke the skill with ; it sets the module up if it never was, and otherwise reports its state. If there is no skill, say so and offer .

\n\necho \"🔥 Burn-In Test Runner\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"Iterations: $ITERATIONS\"\necho \"Base branch: $BASE_BRANCH\"\necho \"\"\n\n# Detect changed test files\necho \"📋 Detecting changed test files...\"\nCHANGED_SPECS=$(git diff --name-only $BASE_BRANCH...HEAD | grep -E \"$SPEC_PATTERN\" || echo \"\")\n\nif [ -z \"$CHANGED_SPECS\" ]; then\n echo \"✅ No test files changed. Skipping burn-in.\"\n exit 0\nfi\n\necho \"Changed test files:\"\necho \"$CHANGED_SPECS\" | sed 's/^/ - /'\necho \"\"\n\n# Count specs\nSPEC_COUNT=$(echo \"$CHANGED_SPECS\" | wc -l | xargs)\necho \"Running burn-in on $SPEC_COUNT test file(s)...\"\necho \"\"\n\n# Burn-in loop\nFAILURES=()\nfor i in $(seq 1 $ITERATIONS); do\n echo \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\n echo \"🔄 Iteration $i/$ITERATIONS\"\n echo \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\n\n # Run tests with explicit file list\n if npm run test -- $CHANGED_SPECS 2>&1 | tee \"burn-in-log-$i.txt\"; then\n echo \"✅ Iteration $i passed\"\n else\n echo \"❌ Iteration $i failed\"\n FAILURES+=($i)\n\n # Save failure artifacts\n mkdir -p burn-in-failures/iteration-$i\n cp -r test-results/ burn-in-failures/iteration-$i/ 2>/dev/null || true\n cp -r screenshots/ burn-in-failures/iteration-$i/ 2>/dev/null || true\n\n echo \"\"\n echo \"🛑 BURN-IN FAILED on iteration $i\"\n echo \"Failure artifacts saved to: burn-in-failures/iteration-$i/\"\n echo \"Logs saved to: burn-in-log-$i.txt\"\n echo \"\"\n exit 1\n fi\n\n echo \"\"\ndone\n\n# Success summary\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"🎉 BURN-IN PASSED\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"All $ITERATIONS iterations passed for $SPEC_COUNT test file(s)\"\necho \"Changed specs are stable and ready to merge.\"\necho \"\"\n\n# Cleanup logs\nrm -f burn-in-log-*.txt\n\nexit 0\n```\n\n**Usage**:\n\n```bash\n# Run locally with default settings (10 iterations, compare to main)\n./scripts/burn-in-changed.sh\n\n# Custom iterations and base branch\n./scripts/burn-in-changed.sh 20 develop\n\n# Add to package.json\n{\n \"scripts\": {\n \"test:burn-in\": \"bash scripts/burn-in-changed.sh\",\n \"test:burn-in:strict\": \"bash scripts/burn-in-changed.sh 20\"\n }\n}\n```\n\n**Key Points**:\n\n- **Exit on first failure**: Flaky tests caught immediately\n- **Failure artifacts**: Saved per-iteration for debugging\n- **Flexible configuration**: Iterations and base branch customizable\n- **CI/local parity**: Same script runs in both environments\n- **Clear output**: Visual feedback on progress and results\n\n---\n\n### Example 3: Shard Orchestration with Result Aggregation\n\n**Context**: Advanced sharding strategy for large test suites with intelligent result merging.\n\n**Implementation**:\n\n```javascript\n// scripts/run-sharded-tests.js\nconst { spawn } = require('child_process');\nconst fs = require('fs');\nconst path = require('path');\n\n/**\n * Run tests across multiple shards and aggregate results\n * Usage: node scripts/run-sharded-tests.js --shards=4 --env=staging\n */\n\nconst SHARD_COUNT = parseInt(process.env.SHARD_COUNT || '4');\nconst TEST_ENV = process.env.TEST_ENV || 'local';\nconst RESULTS_DIR = path.join(__dirname, '../test-results');\n\nconsole.log(`🚀 Running tests across ${SHARD_COUNT} shards`);\nconsole.log(`Environment: ${TEST_ENV}`);\nconsole.log('━'.repeat(50));\n\n// Ensure results directory exists\nif (!fs.existsSync(RESULTS_DIR)) {\n fs.mkdirSync(RESULTS_DIR, { recursive: true });\n}\n\n/**\n * Run a single shard\n */\nfunction runShard(shardIndex) {\n return new Promise((resolve, reject) => {\n const shardId = `${shardIndex}/${SHARD_COUNT}`;\n console.log(`\\n📦 Starting shard ${shardId}...`);\n\n const child = spawn('npx', ['playwright', 'test', `--shard=${shardId}`, '--reporter=json'], {\n env: { ...process.env, TEST_ENV, SHARD_INDEX: shardIndex },\n stdio: 'pipe',\n });\n\n let stdout = '';\n let stderr = '';\n\n child.stdout.on('data', (data) => {\n stdout += data.toString();\n process.stdout.write(data);\n });\n\n child.stderr.on('data', (data) => {\n stderr += data.toString();\n process.stderr.write(data);\n });\n\n child.on('close', (code) => {\n // Save shard results\n const resultFile = path.join(RESULTS_DIR, `shard-${shardIndex}.json`);\n try {\n const result = JSON.parse(stdout);\n fs.writeFileSync(resultFile, JSON.stringify(result, null, 2));\n console.log(`✅ Shard ${shardId} completed (exit code: ${code})`);\n resolve({ shardIndex, code, result });\n } catch (error) {\n console.error(`❌ Shard ${shardId} failed to parse results:`, error.message);\n reject({ shardIndex, code, error });\n }\n });\n\n child.on('error', (error) => {\n console.error(`❌ Shard ${shardId} process error:`, error.message);\n reject({ shardIndex, error });\n });\n });\n}\n\n/**\n * Aggregate results from all shards\n */\nfunction aggregateResults() {\n console.log('\\n📊 Aggregating results from all shards...');\n\n const shardResults = [];\n let totalTests = 0;\n let totalPassed = 0;\n let totalFailed = 0;\n let totalSkipped = 0;\n let totalFlaky = 0;\n\n for (let i = 1; i \u003c= SHARD_COUNT; i++) {\n const resultFile = path.join(RESULTS_DIR, `shard-${i}.json`);\n if (fs.existsSync(resultFile)) {\n const result = JSON.parse(fs.readFileSync(resultFile, 'utf8'));\n shardResults.push(result);\n\n // Aggregate stats\n totalTests += result.stats?.expected || 0;\n totalPassed += result.stats?.expected || 0;\n totalFailed += result.stats?.unexpected || 0;\n totalSkipped += result.stats?.skipped || 0;\n totalFlaky += result.stats?.flaky || 0;\n }\n }\n\n const summary = {\n totalShards: SHARD_COUNT,\n environment: TEST_ENV,\n totalTests,\n passed: totalPassed,\n failed: totalFailed,\n skipped: totalSkipped,\n flaky: totalFlaky,\n duration: shardResults.reduce((acc, r) => acc + (r.duration || 0), 0),\n timestamp: new Date().toISOString(),\n };\n\n // Save aggregated summary\n fs.writeFileSync(path.join(RESULTS_DIR, 'summary.json'), JSON.stringify(summary, null, 2));\n\n console.log('\\n━'.repeat(50));\n console.log('📈 Test Results Summary');\n console.log('━'.repeat(50));\n console.log(`Total tests: ${totalTests}`);\n console.log(`✅ Passed: ${totalPassed}`);\n console.log(`❌ Failed: ${totalFailed}`);\n console.log(`⏭️ Skipped: ${totalSkipped}`);\n console.log(`⚠️ Flaky: ${totalFlaky}`);\n console.log(`⏱️ Duration: ${(summary.duration / 1000).toFixed(2)}s`);\n console.log('━'.repeat(50));\n\n return summary;\n}\n\n/**\n * Main execution\n */\nasync function main() {\n const startTime = Date.now();\n const shardPromises = [];\n\n // Run all shards in parallel\n for (let i = 1; i \u003c= SHARD_COUNT; i++) {\n shardPromises.push(runShard(i));\n }\n\n try {\n await Promise.allSettled(shardPromises);\n } catch (error) {\n console.error('❌ One or more shards failed:', error);\n }\n\n // Aggregate results\n const summary = aggregateResults();\n\n const totalTime = ((Date.now() - startTime) / 1000).toFixed(2);\n console.log(`\\n⏱️ Total execution time: ${totalTime}s`);\n\n // Exit with failure if any tests failed\n if (summary.failed > 0) {\n console.error('\\n❌ Test suite failed');\n process.exit(1);\n }\n\n console.log('\\n✅ All tests passed');\n process.exit(0);\n}\n\nmain().catch((error) => {\n console.error('Fatal error:', error);\n process.exit(1);\n});\n```\n\n**package.json integration**:\n\n```json\n{\n \"scripts\": {\n \"test:sharded\": \"node scripts/run-sharded-tests.js\",\n \"test:sharded:ci\": \"SHARD_COUNT=8 TEST_ENV=staging node scripts/run-sharded-tests.js\"\n }\n}\n```\n\n**Key Points**:\n\n- **Parallel shard execution**: All shards run simultaneously\n- **Result aggregation**: Unified summary across shards\n- **Failure detection**: Exit code reflects overall test status\n- **Artifact preservation**: Individual shard results saved for debugging\n- **CI/local compatibility**: Same script works in both environments\n\n---\n\n### Example 4: Selective Test Execution (Changed Files + Tags)\n\n**Context**: Optimize CI by running only relevant tests based on file changes and tags.\n\n**Implementation**:\n\n```bash\n#!/bin/bash\n# scripts/selective-test-runner.sh\n# Intelligent test selection based on changed files and test tags\n\nset -e\n\nBASE_BRANCH=${BASE_BRANCH:-main}\nTEST_ENV=${TEST_ENV:-local}\n\necho \"🎯 Selective Test Runner\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"Base branch: $BASE_BRANCH\"\necho \"Environment: $TEST_ENV\"\necho \"\"\n\n# Detect changed files (all types, not just tests)\nCHANGED_FILES=$(git diff --name-only $BASE_BRANCH...HEAD)\n\nif [ -z \"$CHANGED_FILES\" ]; then\n echo \"✅ No files changed. Skipping tests.\"\n exit 0\nfi\n\necho \"Changed files:\"\necho \"$CHANGED_FILES\" | sed 's/^/ - /'\necho \"\"\n\n# Determine test strategy based on changes\nrun_smoke_only=false\nrun_all_tests=false\naffected_specs=\"\"\n\n# Critical files = run all tests\nif echo \"$CHANGED_FILES\" | grep -qE '(package\\.json|package-lock\\.json|playwright\\.config|cypress\\.config|\\.github/workflows)'; then\n echo \"⚠️ Critical configuration files changed. Running ALL tests.\"\n run_all_tests=true\n\n# Auth/security changes = run all auth + smoke tests\nelif echo \"$CHANGED_FILES\" | grep -qE '(auth|login|signup|security)'; then\n echo \"🔒 Auth/security files changed. Running auth + smoke tests.\"\n npm run test -- --grep \"@auth|@smoke\"\n exit $?\n\n# API changes = run integration + smoke tests\nelif echo \"$CHANGED_FILES\" | grep -qE '(api|service|controller)'; then\n echo \"🔌 API files changed. Running integration + smoke tests.\"\n npm run test -- --grep \"@integration|@smoke\"\n exit $?\n\n# UI component changes = run related component tests\nelif echo \"$CHANGED_FILES\" | grep -qE '\\.(tsx|jsx|vue)

bmod-tea This folder is the Test Architect module's record, not something to run. Invoke the skill with ; it sets the module up if it never was, and otherwise reports its state. If there is no skill, say so and offer .

; then\n echo \"🎨 UI components changed. Running component + smoke tests.\"\n\n # Extract component names and find related tests\n components=$(echo \"$CHANGED_FILES\" | grep -E '\\.(tsx|jsx|vue)

bmod-tea This folder is the Test Architect module's record, not something to run. Invoke the skill with ; it sets the module up if it never was, and otherwise reports its state. If there is no skill, say so and offer .

| xargs -I {} basename {} | sed 's/\\.[^.]*$//')\n for component in $components; do\n # Find tests matching component name\n affected_specs+=$(find tests -name \"*${component}*\" -type f) || true\n done\n\n if [ -n \"$affected_specs\" ]; then\n echo \"Running tests for: $affected_specs\"\n npm run test -- $affected_specs --grep \"@smoke\"\n else\n echo \"No specific tests found. Running smoke tests only.\"\n npm run test -- --grep \"@smoke\"\n fi\n exit $?\n\n# Documentation/config only = run smoke tests\nelif echo \"$CHANGED_FILES\" | grep -qE '\\.(md|txt|json|yml|yaml)

bmod-tea This folder is the Test Architect module's record, not something to run. Invoke the skill with ; it sets the module up if it never was, and otherwise reports its state. If there is no skill, say so and offer .

; then\n echo \"📝 Documentation/config files changed. Running smoke tests only.\"\n run_smoke_only=true\nelse\n echo \"⚙️ Other files changed. Running smoke tests.\"\n run_smoke_only=true\nfi\n\n# Execute selected strategy\nif [ \"$run_all_tests\" = true ]; then\n echo \"\"\n echo \"Running full test suite...\"\n npm run test\nelif [ \"$run_smoke_only\" = true ]; then\n echo \"\"\n echo \"Running smoke tests...\"\n npm run test -- --grep \"@smoke\"\nfi\n```\n\n**Usage in GitHub Actions**:\n\n```yaml\n# .github/workflows/selective-tests.yml\nname: Selective Tests\non: pull_request\n\njobs:\n selective-tests:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n with:\n fetch-depth: 0\n\n - name: Run selective tests\n run: bash scripts/selective-test-runner.sh\n env:\n BASE_BRANCH: ${{ github.base_ref }}\n TEST_ENV: staging\n```\n\n**Key Points**:\n\n- **Intelligent routing**: Tests selected based on changed file types\n- **Tag-based filtering**: Use @smoke, @auth, @integration tags\n- **Fast feedback**: Only relevant tests run on most PRs\n- **Safety net**: Critical changes trigger full suite\n- **Component mapping**: UI changes run related component tests\n\n---\n\n## CI Configuration Checklist\n\nBefore deploying your CI pipeline, verify:\n\n- [ ] **Caching strategy**: node_modules, npm cache, browser binaries cached\n- [ ] **Timeout budgets**: Each job has reasonable timeout (10-30 min)\n- [ ] **Artifact retention**: 30 days for reports, 7 days for failure artifacts\n- [ ] **Parallelization**: Matrix strategy uses fail-fast: false\n- [ ] **Burn-in enabled**: Changed specs run 5-10x before merge\n- [ ] **wait-on app startup**: CI waits for app (wait-on: '\u003chttp://localhost:3000>')\n- [ ] **Secrets documented**: README lists required secrets (API keys, tokens)\n- [ ] **Local parity**: CI scripts runnable locally (npm run test:ci)\n\n## Integration Points\n\n- Used in workflows: `*ci` (CI/CD pipeline setup)\n- Related fragments: `selective-testing.md`, `playwright-config.md`, `test-quality.md`\n- CI tools: GitHub Actions, GitLab CI, CircleCI, Jenkins\n\n_Source: Murat CI/CD strategy blog, Playwright/Cypress workflow examples, enterprise production pipelines._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":22487,"content_sha256":"224795cba3b6c5cfe6f4c85e6f99ade801d84a9f6edbff164a8b27fcc52370b1"},{"filename":"knowledge/component-tdd.md","content":"# Component Test-Driven Development Loop\n\n## Principle\n\nStart every UI change with a failing component test (`cy.mount`, Playwright component test, or RTL `render`). Follow the Red-Green-Refactor cycle: write a failing test (red), make it pass with minimal code (green), then improve the implementation (refactor). Ship only after the cycle completes. Keep component tests under 100 lines, isolated with fresh providers per test, and validate accessibility alongside functionality.\n\n## Rationale\n\nComponent TDD provides immediate feedback during development. Failing tests (red) clarify requirements before writing code. Minimal implementations (green) prevent over-engineering. Refactoring with passing tests ensures changes don't break functionality. Isolated tests with fresh providers prevent state bleed in parallel runs. Accessibility assertions catch usability issues early. Visual debugging (Cypress runner, Storybook, Playwright trace viewer) accelerates diagnosis when tests fail.\n\n## Pattern Examples\n\n### Example 1: Red-Green-Refactor Loop\n\n**Context**: When building a new component, start with a failing test that describes the desired behavior. Implement just enough to pass, then refactor for quality.\n\n**Implementation**:\n\n```typescript\n// Step 1: RED - Write failing test\n// Button.cy.tsx (Cypress Component Test)\nimport { Button } from './Button';\n\ndescribe('Button Component', () => {\n it('should render with label', () => {\n cy.mount(\u003cButton label=\"Click Me\" />);\n cy.contains('Click Me').should('be.visible');\n });\n\n it('should call onClick when clicked', () => {\n const onClickSpy = cy.stub().as('onClick');\n cy.mount(\u003cButton label=\"Submit\" onClick={onClickSpy} />);\n\n cy.get('button').click();\n cy.get('@onClick').should('have.been.calledOnce');\n });\n});\n\n// Run test: FAILS - Button component doesn't exist yet\n// Error: \"Cannot find module './Button'\"\n\n// Step 2: GREEN - Minimal implementation\n// Button.tsx\ntype ButtonProps = {\n label: string;\n onClick?: () => void;\n};\n\nexport const Button = ({ label, onClick }: ButtonProps) => {\n return \u003cbutton onClick={onClick}>{label}\u003c/button>;\n};\n\n// Run test: PASSES - Component renders and handles clicks\n\n// Step 3: REFACTOR - Improve implementation\n// Add disabled state, loading state, variants\ntype ButtonProps = {\n label: string;\n onClick?: () => void;\n disabled?: boolean;\n loading?: boolean;\n variant?: 'primary' | 'secondary' | 'danger';\n};\n\nexport const Button = ({\n label,\n onClick,\n disabled = false,\n loading = false,\n variant = 'primary'\n}: ButtonProps) => {\n return (\n \u003cbutton\n onClick={onClick}\n disabled={disabled || loading}\n className={`btn btn-${variant}`}\n data-testid=\"button\"\n >\n {loading ? \u003cSpinner /> : label}\n \u003c/button>\n );\n};\n\n// Step 4: Expand tests for new features\ndescribe('Button Component', () => {\n it('should render with label', () => {\n cy.mount(\u003cButton label=\"Click Me\" />);\n cy.contains('Click Me').should('be.visible');\n });\n\n it('should call onClick when clicked', () => {\n const onClickSpy = cy.stub().as('onClick');\n cy.mount(\u003cButton label=\"Submit\" onClick={onClickSpy} />);\n\n cy.get('button').click();\n cy.get('@onClick').should('have.been.calledOnce');\n });\n\n it('should be disabled when disabled prop is true', () => {\n cy.mount(\u003cButton label=\"Submit\" disabled={true} />);\n cy.get('button').should('be.disabled');\n });\n\n it('should show spinner when loading', () => {\n cy.mount(\u003cButton label=\"Submit\" loading={true} />);\n cy.get('[data-testid=\"spinner\"]').should('be.visible');\n cy.get('button').should('be.disabled');\n });\n\n it('should apply variant styles', () => {\n cy.mount(\u003cButton label=\"Delete\" variant=\"danger\" />);\n cy.get('button').should('have.class', 'btn-danger');\n });\n});\n\n// Run tests: ALL PASS - Refactored component still works\n\n// Playwright Component Test equivalent\nimport { test, expect } from '@playwright/experimental-ct-react';\nimport { Button } from './Button';\n\ntest.describe('Button Component', () => {\n test('should call onClick when clicked', async ({ mount }) => {\n let clicked = false;\n const component = await mount(\n \u003cButton label=\"Submit\" onClick={() => { clicked = true; }} />\n );\n\n await component.getByRole('button').click();\n expect(clicked).toBe(true);\n });\n\n test('should be disabled when loading', async ({ mount }) => {\n const component = await mount(\u003cButton label=\"Submit\" loading={true} />);\n await expect(component.getByRole('button')).toBeDisabled();\n await expect(component.getByTestId('spinner')).toBeVisible();\n });\n});\n```\n\n**Key Points**:\n\n- Red: Write failing test first - clarifies requirements before coding\n- Green: Implement minimal code to pass - prevents over-engineering\n- Refactor: Improve code quality while keeping tests green\n- Expand: Add tests for new features after refactoring\n- Cycle repeats: Each new feature starts with a failing test\n\n### Example 2: Provider Isolation Pattern\n\n**Context**: When testing components that depend on context providers (React Query, Auth, Router), wrap them with required providers in each test to prevent state bleed between tests.\n\n**Implementation**:\n\n```typescript\n// test-utils/AllTheProviders.tsx\nimport { FC, ReactNode } from 'react';\nimport { QueryClient, QueryClientProvider } from '@tanstack/react-query';\nimport { BrowserRouter } from 'react-router-dom';\nimport { AuthProvider } from '../contexts/AuthContext';\n\ntype Props = {\n children: ReactNode;\n initialAuth?: { user: User | null; token: string | null };\n};\n\nexport const AllTheProviders: FC\u003cProps> = ({ children, initialAuth }) => {\n // Create NEW QueryClient per test (prevent state bleed)\n const queryClient = new QueryClient({\n defaultOptions: {\n queries: { retry: false },\n mutations: { retry: false }\n }\n });\n\n return (\n \u003cQueryClientProvider client={queryClient}>\n \u003cBrowserRouter>\n \u003cAuthProvider initialAuth={initialAuth}>\n {children}\n \u003c/AuthProvider>\n \u003c/BrowserRouter>\n \u003c/QueryClientProvider>\n );\n};\n\n// Cypress custom mount command\n// cypress/support/component.tsx\nimport { mount } from 'cypress/react18';\nimport { AllTheProviders } from '../../test-utils/AllTheProviders';\n\nCypress.Commands.add('wrappedMount', (component, options = {}) => {\n const { initialAuth, ...mountOptions } = options;\n\n return mount(\n \u003cAllTheProviders initialAuth={initialAuth}>\n {component}\n \u003c/AllTheProviders>,\n mountOptions\n );\n});\n\n// Usage in tests\n// UserProfile.cy.tsx\nimport { UserProfile } from './UserProfile';\n\ndescribe('UserProfile Component', () => {\n it('should display user when authenticated', () => {\n const user = { id: 1, name: 'John Doe', email: '[email protected]' };\n\n cy.wrappedMount(\u003cUserProfile />, {\n initialAuth: { user, token: 'fake-token' }\n });\n\n cy.contains('John Doe').should('be.visible');\n cy.contains('[email protected]').should('be.visible');\n });\n\n it('should show login prompt when not authenticated', () => {\n cy.wrappedMount(\u003cUserProfile />, {\n initialAuth: { user: null, token: null }\n });\n\n cy.contains('Please log in').should('be.visible');\n });\n});\n\n// Playwright Component Test with providers\nimport { test, expect } from '@playwright/experimental-ct-react';\nimport { QueryClient, QueryClientProvider } from '@tanstack/react-query';\nimport { UserProfile } from './UserProfile';\nimport { AuthProvider } from '../contexts/AuthContext';\n\ntest.describe('UserProfile Component', () => {\n test('should display user when authenticated', async ({ mount }) => {\n const user = { id: 1, name: 'John Doe', email: '[email protected]' };\n const queryClient = new QueryClient();\n\n const component = await mount(\n \u003cQueryClientProvider client={queryClient}>\n \u003cAuthProvider initialAuth={{ user, token: 'fake-token' }}>\n \u003cUserProfile />\n \u003c/AuthProvider>\n \u003c/QueryClientProvider>\n );\n\n await expect(component.getByText('John Doe')).toBeVisible();\n await expect(component.getByText('[email protected]')).toBeVisible();\n });\n});\n```\n\n**Key Points**:\n\n- Create NEW providers per test (QueryClient, Router, Auth)\n- Prevents state pollution between tests\n- `initialAuth` prop allows testing different auth states\n- Custom mount command (`wrappedMount`) reduces boilerplate\n- Providers wrap component, not the entire test suite\n\n### Example 3: Accessibility Assertions\n\n**Context**: When testing components, validate accessibility alongside functionality using axe-core, ARIA roles, labels, and keyboard navigation.\n\n**Implementation**:\n\n```typescript\n// Cypress with axe-core\n// cypress/support/component.tsx\nimport 'cypress-axe';\n\n// Form.cy.tsx\nimport { Form } from './Form';\n\ndescribe('Form Component Accessibility', () => {\n beforeEach(() => {\n cy.wrappedMount(\u003cForm />);\n cy.injectAxe(); // Inject axe-core\n });\n\n it('should have no accessibility violations', () => {\n cy.checkA11y(); // Run axe scan\n });\n\n it('should have proper ARIA labels', () => {\n cy.get('input[name=\"email\"]').should('have.attr', 'aria-label', 'Email address');\n cy.get('input[name=\"password\"]').should('have.attr', 'aria-label', 'Password');\n cy.get('button[type=\"submit\"]').should('have.attr', 'aria-label', 'Submit form');\n });\n\n it('should support keyboard navigation', () => {\n // Tab through form fields\n cy.get('input[name=\"email\"]').focus().type('[email protected]');\n cy.realPress('Tab'); // cypress-real-events plugin\n cy.focused().should('have.attr', 'name', 'password');\n\n cy.focused().type('password123');\n cy.realPress('Tab');\n cy.focused().should('have.attr', 'type', 'submit');\n\n cy.realPress('Enter'); // Submit via keyboard\n cy.contains('Form submitted').should('be.visible');\n });\n\n it('should announce errors to screen readers', () => {\n cy.get('button[type=\"submit\"]').click(); // Submit without data\n\n // Error has role=\"alert\" and aria-live=\"polite\"\n cy.get('[role=\"alert\"]')\n .should('be.visible')\n .and('have.attr', 'aria-live', 'polite')\n .and('contain', 'Email is required');\n });\n\n it('should have sufficient color contrast', () => {\n cy.checkA11y(null, {\n rules: {\n 'color-contrast': { enabled: true }\n }\n });\n });\n});\n\n// Playwright with axe-playwright\nimport { test, expect } from '@playwright/experimental-ct-react';\nimport AxeBuilder from '@axe-core/playwright';\nimport { Form } from './Form';\n\ntest.describe('Form Component Accessibility', () => {\n test('should have no accessibility violations', async ({ mount, page }) => {\n await mount(\u003cForm />);\n\n const accessibilityScanResults = await new AxeBuilder({ page })\n .analyze();\n\n expect(accessibilityScanResults.violations).toEqual([]);\n });\n\n test('should support keyboard navigation', async ({ mount, page }) => {\n const component = await mount(\u003cForm />);\n\n await component.getByLabel('Email address').fill('[email protected]');\n await page.keyboard.press('Tab');\n\n await expect(component.getByLabel('Password')).toBeFocused();\n\n await component.getByLabel('Password').fill('password123');\n await page.keyboard.press('Tab');\n\n await expect(component.getByRole('button', { name: 'Submit form' })).toBeFocused();\n\n await page.keyboard.press('Enter');\n await expect(component.getByText('Form submitted')).toBeVisible();\n });\n});\n```\n\n**Key Points**:\n\n- Use `cy.checkA11y()` (Cypress) or `AxeBuilder` (Playwright) for automated accessibility scanning\n- Validate ARIA roles, labels, and live regions\n- Test keyboard navigation (Tab, Enter, Escape)\n- Ensure errors are announced to screen readers (`role=\"alert\"`, `aria-live`)\n- Check color contrast meets WCAG standards\n\n### Example 4: Visual Regression Test\n\n**Context**: When testing components, capture screenshots to detect unintended visual changes. Use Playwright visual comparison or Cypress snapshot plugins.\n\n**Implementation**:\n\n```typescript\n// Playwright visual regression\nimport { test, expect } from '@playwright/experimental-ct-react';\nimport { Button } from './Button';\n\ntest.describe('Button Visual Regression', () => {\n test('should match primary button snapshot', async ({ mount }) => {\n const component = await mount(\u003cButton label=\"Primary\" variant=\"primary\" />);\n\n // Capture and compare screenshot\n await expect(component).toHaveScreenshot('button-primary.png');\n });\n\n test('should match secondary button snapshot', async ({ mount }) => {\n const component = await mount(\u003cButton label=\"Secondary\" variant=\"secondary\" />);\n await expect(component).toHaveScreenshot('button-secondary.png');\n });\n\n test('should match disabled button snapshot', async ({ mount }) => {\n const component = await mount(\u003cButton label=\"Disabled\" disabled={true} />);\n await expect(component).toHaveScreenshot('button-disabled.png');\n });\n\n test('should match loading button snapshot', async ({ mount }) => {\n const component = await mount(\u003cButton label=\"Loading\" loading={true} />);\n await expect(component).toHaveScreenshot('button-loading.png');\n });\n});\n\n// Cypress visual regression with percy or snapshot plugins\nimport { Button } from './Button';\n\ndescribe('Button Visual Regression', () => {\n it('should match primary button snapshot', () => {\n cy.wrappedMount(\u003cButton label=\"Primary\" variant=\"primary\" />);\n\n // Option 1: Percy (cloud-based visual testing)\n cy.percySnapshot('Button - Primary');\n\n // Option 2: cypress-plugin-snapshots (local snapshots)\n cy.get('button').toMatchImageSnapshot({\n name: 'button-primary',\n threshold: 0.01 // 1% threshold for pixel differences\n });\n });\n\n it('should match hover state', () => {\n cy.wrappedMount(\u003cButton label=\"Hover Me\" />);\n cy.get('button').realHover(); // cypress-real-events\n cy.percySnapshot('Button - Hover State');\n });\n\n it('should match focus state', () => {\n cy.wrappedMount(\u003cButton label=\"Focus Me\" />);\n cy.get('button').focus();\n cy.percySnapshot('Button - Focus State');\n });\n});\n\n// Playwright configuration for visual regression\n// playwright.config.ts\nexport default defineConfig({\n expect: {\n toHaveScreenshot: {\n maxDiffPixels: 100, // Allow 100 pixels difference\n threshold: 0.2 // 20% threshold\n }\n },\n use: {\n screenshot: 'only-on-failure'\n }\n});\n\n// Update snapshots when intentional changes are made\n// npx playwright test --update-snapshots\n```\n\n**Key Points**:\n\n- Playwright: Use `toHaveScreenshot()` for built-in visual comparison\n- Cypress: Use Percy (cloud) or snapshot plugins (local) for visual testing\n- Capture different states: default, hover, focus, disabled, loading\n- Set threshold for acceptable pixel differences (avoid false positives)\n- Update snapshots when visual changes are intentional\n- Visual tests catch unintended CSS/layout regressions\n\n### Example 5: User-Level Interaction, Not Raw Event Dispatch\n\n**Context**: A component test exists to prove the component behaves the way a person driving it would experience. `fireEvent` dispatches one synthetic event straight at the node. A real interaction is a sequence: pointer down, focus, key events, input events, pointer up, blur. Dispatching only the middle one skips everything the component might legitimately depend on, so the test can pass against a component no user can operate: a button that never receives focus, a field whose `onKeyDown` handler is never exercised, an input that ignores paste.\n\nThis is gated on the project already depending on a user-level API. Where `userEvent` (or its equivalent) is a project dependency, it is the interaction API and `fireEvent` is the deviation. Where it is not installed, `fireEvent` is what the project has and this row does not fire. Adding a dependency is a project decision, not a review finding.\n\n`fireEvent` remains the right tool for the events a user cannot produce directly: a synthetic `error` on an image, a `transitionend`, a scroll event from an observer.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: dispatches change directly. Focus, key events, and input events never\n// happen, so a component that validates on keystroke is never exercised.\nfireEvent.change(screen.getByLabelText('Email'), { target: { value: '[email protected]' } });\nfireEvent.click(screen.getByRole('button', { name: 'Submit' }));\nexpect(onSubmit).toHaveBeenCalled();\n\n// ✅ GOOD: the full interaction sequence a person produces\nconst user = userEvent.setup();\nawait user.type(screen.getByLabelText('Email'), '[email protected]');\nawait user.click(screen.getByRole('button', { name: 'Submit' }));\nexpect(onSubmit).toHaveBeenCalledWith({ email: '[email protected]' });\n\n// ✅ ACCEPTABLE: an event no user can dispatch by hand\nfireEvent.error(screen.getByRole('img', { name: 'Avatar' }));\nexpect(screen.getByTestId('avatar-fallback')).toBeVisible();\n```\n\n**Key Points**:\n\n- Where a user-level API is already a project dependency, it is the interaction API\n- The failure mode is a passing test for a component a real user cannot operate\n- `userEvent` is asynchronous: its calls are awaited, which also removes a class of unawaited-promise flake\n- `fireEvent` stays correct for events users cannot produce (`error`, `transitionend`, observer-driven scroll)\n- If the project has no user-level API installed, this is not a violation\n\n## Integration Points\n\n- **Used in workflows**: `*atdd` (component test generation), `*automate` (component test expansion), `*framework` (component testing setup)\n- **Related fragments**:\n - `test-quality.md` - Keep component tests \u003c100 lines, isolated, focused\n - `fixture-architecture.md` - Provider wrapping patterns, custom mount commands\n - `data-factories.md` - Factory functions for component props\n - `test-levels-framework.md` - When to use component tests vs E2E tests\n\n## TDD Workflow Summary\n\n**Red-Green-Refactor Cycle**:\n\n1. **Red**: Write failing test describing desired behavior\n2. **Green**: Implement minimal code to make test pass\n3. **Refactor**: Improve code quality, tests stay green\n4. **Repeat**: Each new feature starts with failing test\n\n**Component Test Checklist**:\n\n- [ ] Test renders with required props\n- [ ] Test user interactions (click, type, submit)\n- [ ] Test different states (loading, error, disabled)\n- [ ] Test accessibility (ARIA, keyboard navigation)\n- [ ] Test visual regression (snapshots)\n- [ ] Isolate with fresh providers (no state bleed)\n- [ ] Keep tests \u003c100 lines (split by intent)\n\n_Source: CCTDD repository, Murat component testing talks, Playwright/Cypress component testing docs._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":18624,"content_sha256":"265907160c95c8d7b2e9c05304665b68f3267b817920fff8afb131ec04184edb"},{"filename":"knowledge/confidence-gate.md","content":"# Confidence Gate\n\n## Principle\n\nWhen generating tests, scaffolding fixtures, classifying risk, or proposing any non-trivial test artifact, emit a confidence assessment before writing code. If confidence is below the threshold, stop and ask the user instead of generating plausible-looking output built on guesses.\n\n## Rationale\n\nThe failure mode of LLM-generated tests is rarely \"refused to try\" — it is \"generated something plausible that passes locally and breaks silently in CI.\" Hallucinated selectors, invented endpoint paths, fabricated risk scores, and reverse-engineered schemas all produce code that looks correct and tests nothing real. A confidence gate makes that failure mode loud by forcing the agent to declare its evidence and its unknowns before any artifact is committed.\n\n## Required output shape\n\nEvery non-trivial test artifact proposal must include:\n\n```text\nConfidence: \u003c1-10>\nRationale: \u003cone or two sentences citing concrete evidence from the repo or contract>\nUnknowns: \u003cbulleted list of things the agent does not know>\n```\n\nThe Rationale must cite a file path, a contract document, an existing pattern, or a captured observation. Vague rationale (\"based on standard patterns\", \"looks similar to other tests\") is not evidence and forces the score down.\n\n## Threshold rule\n\n- **Confidence ≥ 7** — proceed with generation.\n- **Confidence 5–6** — proceed but surface the assumptions to the user in the output so they can correct mid-flight.\n- **Confidence \u003c 5** — STOP. Do not generate. Ask the user to resolve the most-blocking Unknown first.\n\n## When to apply\n\nApply the gate when generating or proposing:\n\n- **Selectors and page objects.** Must have explored the live application via `playwright-cli` or read existing page object patterns. Confidence \u003c 5 if neither.\n- **Endpoint paths and request shapes.** Must have read the OpenAPI / Swagger contract or existing endpoint enums. Confidence \u003c 5 if the endpoint is being invented.\n- **Risk classification (test-design, NFR).** Must cite probability and impact evidence. Confidence \u003c 5 if scoring is vibes-based.\n- **Fixture composition.** Must understand existing `mergeTests` patterns and fixture boundaries in the repo. Confidence \u003c 5 if composing blindly.\n- **Schema authoring (Zod, Ajv, JSON Schema).** Must have a documented contract source (OpenAPI, JSON schema, existing schema file). Confidence \u003c 5 if reverse-engineering from a single sample response.\n- **Data factories.** Must understand the production data shape and constraints. Confidence \u003c 5 if guessing field validity rules.\n\n## When NOT to apply\n\n- Mechanical refactors with clear scope (rename a variable, add a tag, update an import).\n- Reading or summarizing existing artifacts.\n- Producing reports from already-gathered data.\n- Trivial test additions that copy an existing pattern exactly.\n\nThe gate exists to prevent fabrication, not to bureaucratize obvious work.\n\n## Anti-patterns\n\n❌ **Vanity scores.** `Confidence: 9` with no Rationale, or Rationale that does not cite evidence. Score the evidence, not the optimism.\n\n❌ **Listing then ignoring Unknowns.** Listing unknowns and then proceeding anyway when Confidence is below threshold. If the gate is below threshold, the only valid next action is to ask the user.\n\n❌ **Asking generically.** Asking \"should I proceed?\" instead of resolving the most-blocking Unknown with a concrete one-sentence question.\n\n❌ **Inflating to clear the bar.** Adjusting Confidence upward to avoid the stop rule. If the evidence is weak, the score is weak; resolve the evidence, not the number.\n\n## Patterns that work\n\n✅ **Cite the source.** \"Confidence: 8 — Rationale: read `src/openapi/users.yaml` line 142-167 and existing schema at `tests/api/users.schema.ts`.\"\n\n✅ **One concrete Unknown.** When below threshold, ask one specific question: \"Is `POST /users/{id}/role` documented anywhere? I can't find it in the OpenAPI spec and there are no existing tests for it.\"\n\n✅ **Promote evidence.** When the user answers the Unknown, the Rationale gets stronger and Confidence rises legitimately. The gate is a feedback loop, not a checkpoint.\n\n## Related fragments\n\n- `test-quality.md` — Definition of Done for tests; the gate protects DoD compliance.\n- `risk-governance.md` — risk scoring discipline that informs Rationale for risk-related gates.\n- `probability-impact.md` — scoring scales used in risk-related Rationale.\n- `selector-resilience.md` — selector confidence specifically.\n- `playwright-cli.md` — the sanctioned exploration tool that promotes selector Confidence.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4593,"content_sha256":"99df7e3ff38b6dd69f1c08de85b0d586aa9ac2dc1a33fcd4db682a78af4b4b67"},{"filename":"knowledge/contract-testing.md","content":"# Contract Testing Essentials (Pact)\n\n## Principle\n\nContract testing validates API contracts between consumer and provider services without requiring integrated end-to-end tests. Store consumer contracts alongside integration specs, version contracts semantically, and publish on every CI run. Provider verification before merge surfaces breaking changes immediately, while explicit fallback behavior (timeouts, retries, error payloads) captures resilience guarantees in contracts.\n\n> **Pact.js Utils Note**: When `tea_use_pactjs_utils` is enabled, prefer the patterns in the `pactjs-utils-*.md` fragments over the raw Pact.js patterns shown below. The pactjs-utils library eliminates boilerplate for provider states, verifier configuration, and request filters. See `pactjs-utils-overview.md` for the decision tree.\n\n## Rationale\n\nTraditional integration testing requires running both consumer and provider simultaneously, creating slow, flaky tests with complex setup. Contract testing decouples services: consumers define expectations (pact files), providers verify against those expectations independently. This enables parallel development, catches breaking changes early, and documents API behavior as executable specifications. Pair contract tests with API smoke tests to validate data mapping and UI rendering in tandem.\n\n> **Recommended**: When `tea_use_pactjs_utils` is enabled, use `@seontechnologies/pactjs-utils` utilities instead of the manual patterns below. The library handles JsonMap conversion, verifier configuration, and request filter assembly automatically. See the `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, and `pactjs-utils-request-filter.md` fragments for the simplified approach.\n\n## Pattern Examples\n\n### Example 1: Pact Consumer Test (Frontend → Backend API)\n\n**Context**: React application consuming a user management API, defining expected interactions.\n\n**Implementation**:\n\n```typescript\n// tests/contract/user-api.pact.spec.ts\nimport { PactV3, MatchersV3 } from '@pact-foundation/pact';\nimport { getUserById, createUser, User } from '@/api/user-service';\n\nconst { like, eachLike, string, integer } = MatchersV3;\n\n/**\n * Consumer-Driven Contract Test\n * - Consumer (React app) defines expected API behavior\n * - Generates pact file for provider to verify\n * - Runs in isolation (no real backend required)\n */\n\nconst provider = new PactV3({\n consumer: 'user-management-web',\n provider: 'user-api-service',\n dir: './pacts', // Output directory for pact files\n logLevel: 'warn',\n});\n\ndescribe('User API Contract', () => {\n describe('GET /users/:id', () => {\n it('should return user when user exists', async () => {\n // Arrange: Define expected interaction\n await provider\n .given('user with id 1 exists') // Provider state\n .uponReceiving('a request for user 1')\n .withRequest({\n method: 'GET',\n path: '/users/1',\n headers: {\n Accept: 'application/json',\n Authorization: like('Bearer token123'), // Matcher: any string\n },\n })\n .willRespondWith({\n status: 200,\n headers: {\n 'Content-Type': 'application/json',\n },\n body: like({\n id: integer(1),\n name: string('John Doe'),\n email: string('[email protected]'),\n role: string('user'),\n createdAt: string('2025-01-15T10:00:00Z'),\n }),\n })\n .executeTest(async (mockServer) => {\n // Act: Call consumer code against mock server\n const user = await getUserById(1, {\n baseURL: mockServer.url,\n headers: { Authorization: 'Bearer token123' },\n });\n\n // Assert: Validate consumer behavior\n expect(user).toEqual(\n expect.objectContaining({\n id: 1,\n name: 'John Doe',\n email: '[email protected]',\n role: 'user',\n }),\n );\n });\n });\n\n it('should handle 404 when user does not exist', async () => {\n await provider\n .given('user with id 999 does not exist')\n .uponReceiving('a request for non-existent user')\n .withRequest({\n method: 'GET',\n path: '/users/999',\n headers: { Accept: 'application/json' },\n })\n .willRespondWith({\n status: 404,\n headers: { 'Content-Type': 'application/json' },\n body: {\n error: 'User not found',\n code: 'USER_NOT_FOUND',\n },\n })\n .executeTest(async (mockServer) => {\n // Act & Assert: Consumer handles 404 gracefully\n await expect(getUserById(999, { baseURL: mockServer.url })).rejects.toThrow('User not found');\n });\n });\n });\n\n describe('POST /users', () => {\n it('should create user and return 201', async () => {\n const newUser: Omit\u003cUser, 'id' | 'createdAt'> = {\n name: 'Jane Smith',\n email: '[email protected]',\n role: 'admin',\n };\n\n await provider\n .given('no users exist')\n .uponReceiving('a request to create a user')\n .withRequest({\n method: 'POST',\n path: '/users',\n headers: {\n 'Content-Type': 'application/json',\n Accept: 'application/json',\n },\n body: newUser,\n })\n .willRespondWith({\n status: 201,\n headers: { 'Content-Type': 'application/json' },\n body: like({\n id: integer(2),\n name: string('Jane Smith'),\n email: string('[email protected]'),\n role: string('admin'),\n createdAt: string('2025-01-15T11:00:00Z'),\n }),\n })\n .executeTest(async (mockServer) => {\n const createdUser = await createUser(newUser, {\n baseURL: mockServer.url,\n });\n\n expect(createdUser).toEqual(\n expect.objectContaining({\n id: expect.any(Number),\n name: 'Jane Smith',\n email: '[email protected]',\n role: 'admin',\n }),\n );\n });\n });\n });\n});\n```\n\n**package.json scripts** (when using pactjs-utils conventions, prefer `test:pact:consumer` naming — see `pact-consumer-framework-setup.md`):\n\n```json\n{\n \"scripts\": {\n \"test:pact:consumer\": \"vitest run --config vitest.config.pact.ts\",\n \"publish:pact\": \". ./scripts/env-setup.sh && ./scripts/publish-pact.sh\"\n }\n}\n```\n\n**Key Points**:\n\n- **Consumer-driven**: Frontend defines expectations, not backend\n- **Matchers (Postel's Law)**: Use `like`, `string`, `integer` matchers in `willRespondWith` (responses) for flexible matching. Do NOT use `like()` on request bodies in `withRequest` — the consumer controls what it sends, so request bodies should use exact values. This follows Postel's Law: be strict in what you send (requests), be lenient in what you accept (responses).\n- **Provider states**: given() sets up test preconditions\n- **Isolation**: No real backend needed, runs fast\n- **Pact generation**: Automatically creates JSON pact files\n\n---\n\n### Example 2: Pact Provider Verification (Backend validates contracts)\n\n**Context**: Node.js/Express API verifying pacts published by consumers.\n\n**Implementation**:\n\n```typescript\n// tests/contract/user-api.provider.spec.ts\nimport { Verifier, VerifierOptions } from '@pact-foundation/pact';\nimport { server } from '../../src/server'; // Your Express/Fastify app\nimport { seedDatabase, resetDatabase } from '../support/db-helpers';\n\n/**\n * Provider Verification Test\n * - Provider (backend API) verifies against published pacts\n * - State handlers setup test data for each interaction\n * - Runs before merge to catch breaking changes\n */\n\ndescribe('Pact Provider Verification', () => {\n let serverInstance;\n const PORT = 3001;\n\n beforeAll(async () => {\n // Start provider server\n serverInstance = server.listen(PORT);\n console.log(`Provider server running on port ${PORT}`);\n });\n\n afterAll(async () => {\n // Cleanup\n await serverInstance.close();\n });\n\n it('should verify pacts from all consumers', async () => {\n const opts: VerifierOptions = {\n // Provider details\n provider: 'user-api-service',\n providerBaseUrl: `http://localhost:${PORT}`,\n\n // Pact Broker configuration\n pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,\n pactBrokerToken: process.env.PACT_BROKER_TOKEN,\n publishVerificationResult: process.env.CI === 'true',\n providerVersion: process.env.GITHUB_SHA || 'dev',\n\n // State handlers: Setup provider state for each interaction\n stateHandlers: {\n 'user with id 1 exists': async () => {\n await seedDatabase({\n users: [\n {\n id: 1,\n name: 'John Doe',\n email: '[email protected]',\n role: 'user',\n createdAt: '2025-01-15T10:00:00Z',\n },\n ],\n });\n return 'User seeded successfully';\n },\n\n 'user with id 999 does not exist': async () => {\n // Ensure user doesn't exist\n await resetDatabase();\n return 'Database reset';\n },\n\n 'no users exist': async () => {\n await resetDatabase();\n return 'Database empty';\n },\n },\n\n // Request filters: Add auth headers to all requests\n requestFilter: (req, res, next) => {\n // Mock authentication for verification\n req.headers['x-user-id'] = 'test-user';\n req.headers['authorization'] = 'Bearer valid-test-token';\n next();\n },\n\n // Timeout for verification\n timeout: 30000,\n };\n\n // Run verification\n await new Verifier(opts).verifyProvider();\n });\n});\n```\n\n**CI integration**:\n\n```yaml\n# .github/workflows/contract-test-provider.yml\n# NOTE: Canonical naming is contract-test-provider.yml per pactjs-utils conventions\nname: Pact Provider Verification\non:\n pull_request:\n push:\n branches: [main]\n\njobs:\n verify-contracts:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - name: Setup Node.js\n uses: actions/setup-node@v4\n with:\n node-version-file: '.nvmrc'\n\n - name: Install dependencies\n run: npm ci\n\n - name: Start database\n run: docker-compose up -d postgres\n\n - name: Run migrations\n run: npm run db:migrate\n\n - name: Verify pacts\n run: npm run test:pact:provider:remote:contract\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n GITHUB_SHA: ${{ github.sha }}\n GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }}\n\n - name: Can I Deploy?\n if: github.ref == 'refs/heads/main'\n run: npm run can:i:deploy:provider\n```\n\n**Key Points**:\n\n- **State handlers**: Setup provider data for each given() state\n- **Request filters**: Add auth/headers for verification requests\n- **CI publishing**: Verification results sent to broker\n- **can-i-deploy**: Safety check before production deployment\n- **Database isolation**: Reset between state handlers\n\n---\n\n### Example 3: Contract CI Integration (Consumer & Provider Workflow)\n\n**Context**: Simplified overview of consumer and provider CI coordination. For the complete consumer CI workflow with env blocks, concurrency, and breaking-change detection, see `pact-consumer-framework-setup.md` Example 5.\n\n**Implementation**:\n\n```yaml\n# .github/workflows/contract-test-consumer.yml (Consumer side)\n# NOTE: Canonical naming is contract-test-consumer.yml per pactjs-utils conventions\nname: Pact Consumer Tests\non:\n pull_request:\n push:\n branches: [main]\n\njobs:\n consumer-tests:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - name: Setup Node.js\n uses: actions/setup-node@v4\n with:\n node-version-file: '.nvmrc'\n\n - name: Install dependencies\n run: npm ci\n\n - name: Run consumer contract tests\n run: npm run test:pact:consumer\n\n - name: Publish pacts to broker\n run: npm run publish:pact\n\n - name: Can I deploy consumer? (main only)\n if: github.ref == 'refs/heads/main' && env.PACT_BREAKING_CHANGE != 'true'\n run: npm run can:i:deploy:consumer\n\n - name: Record consumer deployment (main only)\n if: github.ref == 'refs/heads/main'\n run: npm run record:consumer:deployment --env=dev\n```\n\n```yaml\n# .github/workflows/contract-test-provider.yml (Provider side)\n# NOTE: Canonical naming is contract-test-provider.yml per pactjs-utils conventions\nname: Pact Provider Verification\non:\n pull_request:\n push:\n branches: [main]\n repository_dispatch:\n types: [pact_changed] # Webhook from Pact Broker\n\njobs:\n verify-contracts:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - name: Setup Node.js\n uses: actions/setup-node@v4\n with:\n node-version-file: '.nvmrc'\n\n - name: Install dependencies\n run: npm ci\n\n - name: Start dependencies\n run: docker-compose up -d\n\n - name: Run provider verification\n run: npm run test:pact:provider:remote:contract\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n GITHUB_SHA: ${{ github.sha }}\n GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }}\n\n - name: Can I deploy provider? (main only)\n if: github.ref == 'refs/heads/main' && env.PACT_BREAKING_CHANGE != 'true'\n run: npm run can:i:deploy:provider\n\n - name: Record provider deployment (main only)\n if: github.ref == 'refs/heads/main'\n run: npm run record:provider:deployment --env=dev\n```\n\n**Pact Broker Webhook Configuration**:\n\n```json\n{\n \"events\": [\n {\n \"name\": \"contract_content_changed\"\n }\n ],\n \"request\": {\n \"method\": \"POST\",\n \"url\": \"https://api.github.com/repos/your-org/user-api/dispatches\",\n \"headers\": {\n \"Authorization\": \"Bearer ${user.githubToken}\",\n \"Content-Type\": \"application/json\",\n \"Accept\": \"application/vnd.github.v3+json\"\n },\n \"body\": {\n \"event_type\": \"pact_changed\",\n \"client_payload\": {\n \"pact_url\": \"${pactbroker.pactUrl}\",\n \"consumer\": \"${pactbroker.consumerName}\",\n \"provider\": \"${pactbroker.providerName}\"\n }\n }\n }\n}\n```\n\n**Key Points**:\n\n- **Automatic trigger**: Consumer pact changes trigger provider verification via webhook\n- **Branch tracking**: Pacts published per branch for feature testing\n- **can-i-deploy**: Safety gate before production deployment\n- **Record deployment**: Track which version is in each environment\n- **Parallel dev**: Consumer and provider teams work independently\n\n---\n\n### Coordinating Different Short-Lived Branch Names\n\n`matchingBranch: true` works only when consumer and provider branches share a\nname. Release trains often break that assumption: a consumer feature branch may\nneed a provider at `release/week-32`.\n\nTreat this as two separate checks:\n\n1. **Provider selects the consumer branch.** Pass a scoped `consumer` plus\n `consumerBranch` to `buildVerifierOptions` or\n `buildMessageVerifierOptions`. `PACT_CONSUMER_BRANCH` is the default input.\n The explicit `{ consumer, branch }` selector is additive to matching, main,\n and deployed selectors.\n2. **Consumer selects the provider branch.** On PRs only, parse `Pact provider\nbranch: \u003cname>` into `PACT_PROVIDER_BRANCH`. Keep the environment-wide\n `can-i-deploy` call, use `--ignore \u003cprovider>` for that one in-flight\n pacticipant, then run a second `can-i-deploy` with the provider's\n `--branch`. Both calls must fail hard.\n\nThe second check is weaker than `--to-environment`: it proves compatibility\nwith a branch tip, not with deployed software. Its safety comes from narrow\nscope and short lifetime. Never read the override from the merged PR on push,\nand never replace the environment check globally.\n\nKeep manual PR branch selection separate from PactFlow's\n`contract_requiring_verification_published` webhook. That event names the exact\nprovider version missing a verification result. Check out its\n`providerVersionNumber`, confirm that commit belongs to\n`providerVersionBranch`, and publish with the same version and branch. Stop the\njob when the target cannot be checked out. A substitute revision produces no\nevidence for the Broker's requested target.\n\nSee `pactjs-utils-provider-verifier.md`,\n`pact-consumer-framework-setup.md`, and `pact-broker-webhooks.md`.\n\n---\n\n### Example 4: Resilience Coverage (Testing Fallback Behavior)\n\n**Context**: Capture timeout, retry, and error handling behavior explicitly in contracts.\n\n**Implementation**:\n\n```typescript\n// tests/contract/user-api-resilience.pact.spec.ts\nimport { PactV3, MatchersV3 } from '@pact-foundation/pact';\nimport { getUserById, ApiError } from '@/api/user-service';\n\nconst { like, string } = MatchersV3;\n\nconst provider = new PactV3({\n consumer: 'user-management-web',\n provider: 'user-api-service',\n dir: './pacts',\n});\n\ndescribe('User API Resilience Contract', () => {\n /**\n * Test 500 error handling\n * Verifies consumer handles server errors gracefully\n */\n it('should handle 500 errors with retry logic', async () => {\n await provider\n .given('server is experiencing errors')\n .uponReceiving('a request that returns 500')\n .withRequest({\n method: 'GET',\n path: '/users/1',\n headers: { Accept: 'application/json' },\n })\n .willRespondWith({\n status: 500,\n headers: { 'Content-Type': 'application/json' },\n body: {\n error: 'Internal server error',\n code: 'INTERNAL_ERROR',\n retryable: true,\n },\n })\n .executeTest(async (mockServer) => {\n // Consumer should retry on 500\n try {\n await getUserById(1, {\n baseURL: mockServer.url,\n retries: 3,\n retryDelay: 100,\n });\n fail('Should have thrown error after retries');\n } catch (error) {\n expect(error).toBeInstanceOf(ApiError);\n expect((error as ApiError).code).toBe('INTERNAL_ERROR');\n expect((error as ApiError).retryable).toBe(true);\n }\n });\n });\n\n /**\n * Test 429 rate limiting\n * Verifies consumer respects rate limits\n */\n it('should handle 429 rate limit with backoff', async () => {\n await provider\n .given('rate limit exceeded for user')\n .uponReceiving('a request that is rate limited')\n .withRequest({\n method: 'GET',\n path: '/users/1',\n })\n .willRespondWith({\n status: 429,\n headers: {\n 'Content-Type': 'application/json',\n 'Retry-After': '60', // Retry after 60 seconds\n },\n body: {\n error: 'Too many requests',\n code: 'RATE_LIMIT_EXCEEDED',\n },\n })\n .executeTest(async (mockServer) => {\n try {\n await getUserById(1, {\n baseURL: mockServer.url,\n respectRateLimit: true,\n });\n fail('Should have thrown rate limit error');\n } catch (error) {\n expect(error).toBeInstanceOf(ApiError);\n expect((error as ApiError).code).toBe('RATE_LIMIT_EXCEEDED');\n expect((error as ApiError).retryAfter).toBe(60);\n }\n });\n });\n\n /**\n * Test timeout handling\n * Verifies consumer has appropriate timeout configuration\n */\n it('should timeout after 10 seconds', async () => {\n await provider\n .given('server is slow to respond')\n .uponReceiving('a request that times out')\n .withRequest({\n method: 'GET',\n path: '/users/1',\n })\n .willRespondWith({\n status: 200,\n headers: { 'Content-Type': 'application/json' },\n body: like({ id: 1, name: 'John' }),\n })\n .withDelay(15000) // Simulate 15 second delay\n .executeTest(async (mockServer) => {\n try {\n await getUserById(1, {\n baseURL: mockServer.url,\n timeout: 10000, // 10 second timeout\n });\n fail('Should have timed out');\n } catch (error) {\n expect(error).toBeInstanceOf(ApiError);\n expect((error as ApiError).code).toBe('TIMEOUT');\n }\n });\n });\n\n /**\n * Test partial response (optional fields)\n * Verifies consumer handles missing optional data\n */\n it('should handle response with missing optional fields', async () => {\n await provider\n .given('user exists with minimal data')\n .uponReceiving('a request for user with partial data')\n .withRequest({\n method: 'GET',\n path: '/users/1',\n })\n .willRespondWith({\n status: 200,\n headers: { 'Content-Type': 'application/json' },\n body: {\n id: integer(1),\n name: string('John Doe'),\n email: string('[email protected]'),\n // role, createdAt, etc. omitted (optional fields)\n },\n })\n .executeTest(async (mockServer) => {\n const user = await getUserById(1, { baseURL: mockServer.url });\n\n // Consumer handles missing optional fields gracefully\n expect(user.id).toBe(1);\n expect(user.name).toBe('John Doe');\n expect(user.role).toBeUndefined(); // Optional field\n expect(user.createdAt).toBeUndefined(); // Optional field\n });\n });\n});\n```\n\n**API client with retry logic**:\n\n```typescript\n// src/api/user-service.ts\nimport axios, { AxiosInstance, AxiosRequestConfig } from 'axios';\n\nexport class ApiError extends Error {\n constructor(\n message: string,\n public code: string,\n public retryable: boolean = false,\n public retryAfter?: number,\n ) {\n super(message);\n }\n}\n\n/**\n * User API client with retry and error handling\n */\nexport async function getUserById(\n id: number,\n config?: AxiosRequestConfig & { retries?: number; retryDelay?: number; respectRateLimit?: boolean },\n): Promise\u003cUser> {\n const { retries = 3, retryDelay = 1000, respectRateLimit = true, ...axiosConfig } = config || {};\n\n let lastError: Error;\n\n for (let attempt = 1; attempt \u003c= retries; attempt++) {\n try {\n const response = await axios.get(`/users/${id}`, axiosConfig);\n return response.data;\n } catch (error: any) {\n lastError = error;\n\n // Handle rate limiting\n if (error.response?.status === 429) {\n const retryAfter = parseInt(error.response.headers['retry-after'] || '60');\n throw new ApiError('Too many requests', 'RATE_LIMIT_EXCEEDED', false, retryAfter);\n }\n\n // Retry on 500 errors\n if (error.response?.status === 500 && attempt \u003c retries) {\n await new Promise((resolve) => setTimeout(resolve, retryDelay * attempt));\n continue;\n }\n\n // Handle 404\n if (error.response?.status === 404) {\n throw new ApiError('User not found', 'USER_NOT_FOUND', false);\n }\n\n // Handle timeout\n if (error.code === 'ECONNABORTED') {\n throw new ApiError('Request timeout', 'TIMEOUT', true);\n }\n\n break;\n }\n }\n\n throw new ApiError('Request failed after retries', 'INTERNAL_ERROR', true);\n}\n```\n\n**Key Points**:\n\n- **Resilience contracts**: Timeouts, retries, errors explicitly tested\n- **State handlers**: Provider sets up each test scenario\n- **Error handling**: Consumer validates graceful degradation\n- **Retry logic**: Exponential backoff tested\n- **Optional fields**: Consumer handles partial responses\n\n---\n\n### Example 5: Pact Broker Housekeeping & Lifecycle Management\n\n**Context**: Automated broker maintenance to prevent contract sprawl and noise.\n\n**Implementation**:\n\n```typescript\n// scripts/pact-broker-housekeeping.ts\n/**\n * Pact Broker Housekeeping Script\n * - Archive superseded contracts\n * - Expire unused pacts\n * - Tag releases for environment tracking\n */\n\nimport { execFileSync } from 'node:child_process';\n\nconst PACT_BROKER_BASE_URL = process.env.PACT_BROKER_BASE_URL!;\nconst PACT_BROKER_TOKEN = process.env.PACT_BROKER_TOKEN!;\nconst PACTICIPANT = 'user-api-service';\n\n/**\n * Tag release with environment\n */\nfunction tagRelease(version: string, environment: 'staging' | 'production') {\n console.log(`🏷️ Tagging ${PACTICIPANT} v${version} as ${environment}`);\n\n execFileSync(\n 'pact-broker',\n [\n 'create-version-tag',\n '--pacticipant',\n PACTICIPANT,\n '--version',\n version,\n '--tag',\n environment,\n '--broker-base-url',\n PACT_BROKER_BASE_URL,\n '--broker-token',\n PACT_BROKER_TOKEN,\n ],\n { stdio: 'inherit' },\n );\n}\n\n/**\n * Record deployment to environment\n */\nfunction recordDeployment(version: string, environment: 'staging' | 'production') {\n console.log(`📝 Recording deployment of ${PACTICIPANT} v${version} to ${environment}`);\n\n execFileSync(\n 'pact-broker',\n [\n 'record-deployment',\n '--pacticipant',\n PACTICIPANT,\n '--version',\n version,\n '--environment',\n environment,\n '--broker-base-url',\n PACT_BROKER_BASE_URL,\n '--broker-token',\n PACT_BROKER_TOKEN,\n ],\n { stdio: 'inherit' },\n );\n}\n\n/**\n * Clean up old pact versions (retention policy)\n * Keep: last 30 days, all production tags, latest from each branch\n */\nfunction cleanupOldPacts() {\n console.log(`🧹 Cleaning up old pacts for ${PACTICIPANT}`);\n\n execFileSync(\n 'pact-broker',\n [\n 'clean',\n '--pacticipant',\n PACTICIPANT,\n '--broker-base-url',\n PACT_BROKER_BASE_URL,\n '--broker-token',\n PACT_BROKER_TOKEN,\n '--keep-latest-for-branch',\n '1',\n '--keep-min-age',\n '30',\n ],\n { stdio: 'inherit' },\n );\n}\n\n/**\n * Check deployment compatibility\n */\nfunction canIDeploy(version: string, toEnvironment: string): boolean {\n console.log(`🔍 Checking if ${PACTICIPANT} v${version} can deploy to ${toEnvironment}`);\n\n try {\n execFileSync(\n 'pact-broker',\n [\n 'can-i-deploy',\n '--pacticipant',\n PACTICIPANT,\n '--version',\n version,\n '--to-environment',\n toEnvironment,\n '--broker-base-url',\n PACT_BROKER_BASE_URL,\n '--broker-token',\n PACT_BROKER_TOKEN,\n '--retry-while-unknown',\n '10',\n '--retry-interval',\n '30',\n ],\n { stdio: 'inherit' },\n );\n return true;\n } catch (error) {\n console.error(`❌ Cannot deploy to ${toEnvironment}`);\n return false;\n }\n}\n\n/**\n * Main housekeeping workflow\n */\nasync function main() {\n const command = process.argv[2];\n const version = process.argv[3];\n const environment = process.argv[4] as 'staging' | 'production';\n\n switch (command) {\n case 'tag-release':\n tagRelease(version, environment);\n break;\n\n case 'record-deployment':\n recordDeployment(version, environment);\n break;\n\n case 'can-i-deploy':\n const canDeploy = canIDeploy(version, environment);\n process.exit(canDeploy ? 0 : 1);\n\n case 'cleanup':\n cleanupOldPacts();\n break;\n\n default:\n console.error('Unknown command. Use: tag-release | record-deployment | can-i-deploy | cleanup');\n process.exit(1);\n }\n}\n\nmain();\n```\n\n**package.json scripts**:\n\n```json\n{\n \"scripts\": {\n \"pact:tag\": \"ts-node scripts/pact-broker-housekeeping.ts tag-release\",\n \"pact:record\": \"ts-node scripts/pact-broker-housekeeping.ts record-deployment\",\n \"pact:can-deploy\": \"ts-node scripts/pact-broker-housekeeping.ts can-i-deploy\",\n \"pact:cleanup\": \"ts-node scripts/pact-broker-housekeeping.ts cleanup\"\n }\n}\n```\n\n**Deployment workflow integration**:\n\n```yaml\n# .github/workflows/deploy-production.yml\nname: Deploy to Production\non:\n push:\n tags:\n - 'v*'\n\njobs:\n verify-contracts:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - name: Check pact compatibility\n run: npm run pact:can-deploy ${{ github.ref_name }} production\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n\n deploy:\n needs: verify-contracts\n runs-on: ubuntu-latest\n steps:\n - name: Deploy to production\n run: ./scripts/deploy.sh production\n\n - name: Record deployment in Pact Broker\n run: npm run pact:record ${{ github.ref_name }} production\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n```\n\n**Scheduled cleanup**:\n\n```yaml\n# .github/workflows/pact-housekeeping.yml\nname: Pact Broker Housekeeping\non:\n schedule:\n - cron: '0 2 * * 0' # Weekly on Sunday at 2 AM\n\njobs:\n cleanup:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n\n - name: Cleanup old pacts\n run: npm run pact:cleanup\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n```\n\n**Key Points**:\n\n- **Automated tagging**: Releases tagged with environment\n- **Deployment tracking**: Broker knows which version is where\n- **Safety gate**: can-i-deploy blocks incompatible deployments\n- **Retention policy**: Keep recent, production, and branch-latest pacts\n- **Webhook triggers**: Provider verification runs on consumer changes\n\n---\n\n## Provider Scrutiny Protocol\n\nWhen generating consumer contract tests, the agent **MUST** analyze provider source code — or the provider's OpenAPI/Swagger spec — before writing any Pact interaction. Generating contracts from consumer-side assumptions alone leads to mismatches that only surface during provider verification — wrong response shapes, wrong status codes, wrong field names, wrong types, missing required fields, and wrong enum values.\n\n**Source priority**: Provider source code is the most authoritative reference. When an OpenAPI/Swagger spec exists (`openapi.yaml`, `openapi.json`, `swagger.json`), use it as a complementary or alternative source — it documents the provider's contract explicitly and can be faster to parse than tracing through handler code. When both exist, cross-reference them; if they disagree, the source code wins.\n\n### Provider Endpoint Comment\n\nEvery Pact interaction MUST include a provider endpoint comment immediately above the `.given()` call:\n\n```typescript\n// Provider endpoint: server/src/routes/userRouteHandlers.ts -> GET /api/v2/users/:userId\nawait provider.given('user with id 1 exists').uponReceiving('a request for user 1');\n```\n\n**Format**: `// Provider endpoint: \u003crelative-path-to-handler> -> \u003cMETHOD> \u003croute-pattern>`\n\nIf the provider source is not accessible, use: `// Provider endpoint: TODO — provider source not accessible, verify manually`\n\n### Seven-Point Scrutiny Checklist\n\nBefore generating each Pact interaction, read the provider route handler and/or OpenAPI spec and verify:\n\n| # | Check | What to Read (source code / OpenAPI spec) | Common Mismatch |\n| --- | --------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------- |\n| 1 | **Response shape** | Handler's `res.json()` calls / OpenAPI `responses.content.schema` | Nested object vs flat; array wrapper vs direct |\n| 2 | **Status codes** | Handler's `res.status()` calls / OpenAPI `responses` keys | 200 vs 201 for creation; 204 vs 200 for delete |\n| 3 | **Field names** | Response type/DTO definitions / OpenAPI `schema.properties` | `transaction_id` vs `transactionId`; `fraud_score` vs `score` |\n| 4 | **Enum values** | Validation schemas, constants / OpenAPI `schema.enum` | `\"active\"` vs `\"ACTIVE\"`; `\"pending\"` vs `\"in_progress\"` |\n| 5 | **Required fields** | Request validation (Joi, Zod) / OpenAPI `schema.required` | Missing required header; optional field assumed required |\n| 6 | **Data types** | TypeScript types, DB models / OpenAPI `schema.type` + `format` | `string` ID vs `number` ID; ISO date vs Unix timestamp |\n| 7 | **Nested structures** | Response builder, serializer / OpenAPI `$ref` + `allOf`/`oneOf` | `{ data: { items: [] } }` vs `{ items: [] }` |\n\n### Scrutiny Evidence Block\n\nDocument what was found from provider source and/or OpenAPI spec as a block comment in the test file:\n\n```typescript\n/*\n * Provider Scrutiny Evidence:\n * - Handler: server/src/routes/userRouteHandlers.ts:45\n * - OpenAPI: server/openapi.yaml paths./api/v2/users/{userId}.get (if available)\n * - Response type: UserResponseDto (server/src/types/user.ts:12)\n * - Status: 200 (line 52), 404 (line 48)\n * - Fields: { id: number, name: string, email: string, role: \"user\" | \"admin\", createdAt: string }\n * - Required request headers: Authorization (Bearer token)\n * - Validation: Zod schema at server/src/validation/user.ts:8\n */\n```\n\n### Graceful Degradation\n\nWhen provider source code is not accessible (different repo, no access, closed source):\n\n1. **OpenAPI/Swagger spec available**: Use the spec as the source of truth for response shapes, status codes, and field names\n2. **Pact Broker has existing contracts**: Use `pact_mcp` tools to fetch existing provider states and verified interactions as reference\n3. **Neither available**: Generate contracts from consumer-side types but use the TODO form of the mandatory comment: `// Provider endpoint: TODO — provider source not accessible, verify manually` and add a `provider_scrutiny: \"pending\"` field to the output JSON\n4. **Never silently guess**: If you cannot verify, document what you assumed and why\n\n---\n\n## Contract Testing Checklist\n\nBefore implementing contract testing, verify:\n\n- [ ] **Pact Broker setup**: Hosted (Pactflow) or self-hosted broker configured\n- [ ] **Consumer tests**: Generate pacts in CI, publish to broker on merge\n- [ ] **Provider verification**: Runs on PR, verifies all consumer pacts\n- [ ] **State handlers**: Provider implements all given() states\n- [ ] **can-i-deploy**: Blocks deployment if contracts incompatible\n- [ ] **Short-lived branch overrides**: PR-only, scoped to one pacticipant, and\n additive to the environment gate\n- [ ] **Webhooks configured**: Consumer changes trigger provider verification\n- [ ] **Retention policy**: Old pacts archived (keep 30 days, all production tags)\n- [ ] **Resilience tested**: Timeouts, retries, error codes in contracts\n- [ ] **Provider endpoint comments**: Every Pact interaction has `// Provider endpoint:` comment\n- [ ] **Provider scrutiny completed**: Seven-point checklist verified for each interaction\n- [ ] **Scrutiny evidence documented**: Block comment with handler, types, status codes, and fields\n\n## Integration Points\n\n- Used in workflows: `*automate` (integration test generation), `*ci` (contract CI setup)\n- Related fragments: `test-levels-framework.md`, `ci-burn-in.md`, `pact-consumer-framework-setup.md` (consumer vitest `fileParallelism: false` + `pool: 'forks'` + `singleFork: true`), `pactjs-utils-consumer-helpers.md` (PactV4 one-interaction-per-`it()` rule), `pactjs-utils-provider-verifier.md` (provider vitest `pool: 'forks'` + `singleFork: true` — same rule as consumer), `pact-broker-webhooks.md` (PactFlow → GitHub webhook auth, PAT rotation, staleness monitoring)\n- Tools: Pact.js, Pact Broker (Pactflow or self-hosted), Pact CLI\n\n---\n\n## Pact.js Utils Accelerator\n\nWhen `tea_use_pactjs_utils` is enabled, the following utilities replace manual boilerplate:\n\n| Manual Pattern (raw Pact.js) | Pact.js Utils Equivalent | Benefit |\n| -------------------------------------------------------- | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |\n| Manual `JsonMap` casting for `.given()` params | `createProviderState({ name, params })` | Type-safe, auto-conversion of Date/null/nested objects |\n| Repeated builder callbacks for query/header/body | `setJsonContent({ query, headers, body })` | Reusable callback for `.withRequest(...)` and `.willRespondWith(...)` |\n| Inline body lambda `(builder) => builder.jsonBody(body)` | `setJsonBody(body)` | Body-only shorthand for cleaner response builders |\n| 30+ lines of `VerifierOptions` assembly | `buildVerifierOptions({ provider, port, includeMainAndDeployed, stateHandlers })` | One-call setup, env-aware, flow auto-detection |\n| Manual broker URL + selector logic from env vars | `handlePactBrokerUrlAndSelectors({ ..., consumerBranch, options })` | Handles standard selectors and a scoped named consumer branch |\n| DIY Express middleware for auth injection | `createRequestFilter({ tokenGenerator })` | Bearer prefix contract prevents double-prefix bugs |\n| Manual CI branch/tag extraction | `getProviderVersionTags()` | CI-aware (GitHub Actions, GitLab CI, etc.) |\n| Repeated main/master/release branch classification | `isBreakingChangeTolerantBranch(branch)` | One exact boundary for an explicit breaking-change tolerance policy |\n| Message verifier config assembly | `buildMessageVerifierOptions({ provider, messageProviders })` | Same one-call pattern for Kafka/async contracts |\n| Inline no-op filter `(req, res, next) => next()` | `noOpRequestFilter` | Pre-built pass-through for no-auth providers |\n| Hand-written matcher helper duplicating a Zod/TS type | `zodToPactMatchers(ConsumerMovieSchema, example)` | Single source of truth for response shape; consumer-curated scope keeps contracts lean and consumer-driven |\n\nSee the `pactjs-utils-*.md` knowledge fragments for complete examples and anti-patterns (`pactjs-utils-zod-to-pact.md` covers the consumer-curated schema pattern).\n\nFor differently named in-flight branches, use pactjs-utils 1.2.0 or newer.\nThat release adds `consumerBranch` to both verifier builders and ships the\nPR-only provider-branch detection, additive `can-i-deploy` templates, and\n`isBreakingChangeTolerantBranch`.\n\n### PactV4 Determinism & FFI Safety (Mandatory)\n\nFour rules that together prevent both (a) non-deterministic pact generation failures that cause `Cannot change pact content for already published pact` errors at PactFlow publish, and (b) \"request was expected but not received\" flakes observed on Linux CI once a consumer+provider pair has more than one `.pacttest.ts` file:\n\n1. **Consumer Vitest `fileParallelism: false`** in `vitest.config.pact.ts` — prevents parallel workers from racing on the shared pact JSON. See `pact-consumer-framework-setup.md` Example 2.\n2. **Consumer Vitest `pool: 'forks'` + `poolOptions.forks.singleFork: true`** in `vitest.config.pact.ts` — same config as the provider side (`pactjs-utils-provider-verifier.md` Example 8). Best current understanding: the `@pact-foundation/pact` napi-rs binding is not robust across Vitest worker threads sharing a process; serialization alone (via `fileParallelism: false`) is insufficient on the default threads pool in Vitest v1. Forks + `singleFork: true` runs every pact file in one subprocess with a coherent FFI handle and eliminated a reproducible Linux-CI flake across multiple repos. Single-file consumer suites have not been observed to flake; this rule is still recommended as a future-proof. See `pact-consumer-framework-setup.md` Example 2.\n3. **One `addInteraction()` per `it()` block** — see `pactjs-utils-consumer-helpers.md` Example 6.\n4. **`publish-pact.sh` jq normalization** sorts interactions before publish — ensures byte-stable payload to PactFlow regardless of generator ordering quirks. See `pact-consumer-framework-setup.md` Example 4.\n\nProvider suites require the same `pool: 'forks'` + `singleFork: true` combination — see `pactjs-utils-provider-verifier.md` Example 8.\n\n### Webhook Auth & Staleness\n\nWhen `can-i-deploy` in a consumer repo times out with `There is no verified pact between \u003cconsumer> and the version of \u003cprovider> currently in \u003cenv>` — check the provider's PactFlow webhook. Silent failures from an expired/revoked GitHub PAT are the most common non-code cause of this symptom. See `pact-broker-webhooks.md` for the dedicated-machine-user pattern, classic-PAT-with-`repo`-scope rationale, rotation runbook, and staleness monitoring options.\n\n_Source: Pact consumer/provider sample repos, Murat contract testing blog, Pact official documentation, @seontechnologies/pactjs-utils library._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":42012,"content_sha256":"da8ceb53fab22cd80270e3317a9fc7683b901c6303ac32a40022d7105f871f80"},{"filename":"knowledge/data-factories.md","content":"# Data Factories and API-First Setup\n\n## Principle\n\nPrefer factory functions that accept overrides and return complete objects (`createUser(overrides)`). Seed test state through APIs, tasks, or direct DB helpers before visiting the UI—never via slow UI interactions. UI is for validation only, not setup.\n\n## Rationale\n\nStatic fixtures (JSON files, hardcoded objects) create brittle tests that:\n\n- Fail when schemas evolve (missing new required fields)\n- Cause collisions in parallel execution (same user IDs)\n- Hide test intent (what matters for _this_ test?)\n\nDynamic factories with overrides provide:\n\n- **Parallel safety**: UUIDs and timestamps prevent collisions\n- **Schema evolution**: Defaults adapt to schema changes automatically\n- **Explicit intent**: Overrides show what matters for each test\n- **Speed**: API setup is 10-50x faster than UI\n\n## Pattern Examples\n\n### Example 1: Factory Function with Overrides\n\n**Context**: When creating test data, build factory functions with sensible defaults and explicit overrides. Use `faker` for dynamic values that prevent collisions.\n\n**Implementation**:\n\n```typescript\n// test-utils/factories/user-factory.ts\nimport { faker } from '@faker-js/faker';\n\ntype User = {\n id: string;\n email: string;\n name: string;\n role: 'user' | 'admin' | 'moderator';\n createdAt: Date;\n isActive: boolean;\n};\n\nexport const createUser = (overrides: Partial\u003cUser> = {}): User => ({\n id: faker.string.uuid(),\n email: faker.internet.email(),\n name: faker.person.fullName(),\n role: 'user',\n createdAt: new Date(),\n isActive: true,\n ...overrides,\n});\n\n// test-utils/factories/product-factory.ts\ntype Product = {\n id: string;\n name: string;\n price: number;\n stock: number;\n category: string;\n};\n\nexport const createProduct = (overrides: Partial\u003cProduct> = {}): Product => ({\n id: faker.string.uuid(),\n name: faker.commerce.productName(),\n price: parseFloat(faker.commerce.price()),\n stock: faker.number.int({ min: 0, max: 100 }),\n category: faker.commerce.department(),\n ...overrides,\n});\n\n// Usage in tests:\ntest('admin can delete users', async ({ page, apiRequest }) => {\n // Default user\n const user = createUser();\n\n // Admin user (explicit override shows intent)\n const admin = createUser({ role: 'admin' });\n\n // Seed via API (fast!)\n await apiRequest({ method: 'POST', url: '/api/users', data: user });\n await apiRequest({ method: 'POST', url: '/api/users', data: admin });\n\n // Now test UI behavior\n await page.goto('/admin/users');\n await page.click(`[data-testid=\"delete-user-${user.id}\"]`);\n await expect(page.getByText(`User ${user.name} deleted`)).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- `Partial\u003cUser>` allows overriding any field without breaking type safety\n- Faker generates unique values—no collisions in parallel tests\n- Override shows test intent: `createUser({ role: 'admin' })` is explicit\n- Factory lives in `test-utils/factories/` for easy reuse\n\n### Example 2: Nested Factory Pattern\n\n**Context**: When testing relationships (orders with users and products), nest factories to create complete object graphs. Control relationship data explicitly.\n\n**Implementation**:\n\n```typescript\n// test-utils/factories/order-factory.ts\nimport { createUser } from './user-factory';\nimport { createProduct } from './product-factory';\n\ntype OrderItem = {\n product: Product;\n quantity: number;\n price: number;\n};\n\ntype Order = {\n id: string;\n user: User;\n items: OrderItem[];\n total: number;\n status: 'pending' | 'paid' | 'shipped' | 'delivered';\n createdAt: Date;\n};\n\nexport const createOrderItem = (overrides: Partial\u003cOrderItem> = {}): OrderItem => {\n const product = overrides.product || createProduct();\n const quantity = overrides.quantity || faker.number.int({ min: 1, max: 5 });\n\n return {\n product,\n quantity,\n price: product.price * quantity,\n ...overrides,\n };\n};\n\nexport const createOrder = (overrides: Partial\u003cOrder> = {}): Order => {\n const items = overrides.items || [createOrderItem(), createOrderItem()];\n const total = items.reduce((sum, item) => sum + item.price, 0);\n\n return {\n id: faker.string.uuid(),\n user: overrides.user || createUser(),\n items,\n total,\n status: 'pending',\n createdAt: new Date(),\n ...overrides,\n };\n};\n\n// Usage in tests:\ntest('user can view order details', async ({ page, apiRequest }) => {\n const user = createUser({ email: '[email protected]' });\n const product1 = createProduct({ name: 'Widget A', price: 10.0 });\n const product2 = createProduct({ name: 'Widget B', price: 15.0 });\n\n // Explicit relationships\n const order = createOrder({\n user,\n items: [\n createOrderItem({ product: product1, quantity: 2 }), // $20\n createOrderItem({ product: product2, quantity: 1 }), // $15\n ],\n });\n\n // Seed via API\n await apiRequest({ method: 'POST', url: '/api/users', data: user });\n await apiRequest({ method: 'POST', url: '/api/products', data: product1 });\n await apiRequest({ method: 'POST', url: '/api/products', data: product2 });\n await apiRequest({ method: 'POST', url: '/api/orders', data: order });\n\n // Test UI\n await page.goto(`/orders/${order.id}`);\n await expect(page.getByText('Widget A x 2')).toBeVisible();\n await expect(page.getByText('Widget B x 1')).toBeVisible();\n await expect(page.getByText('Total: $35.00')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- Nested factories handle relationships (order → user, order → products)\n- Overrides cascade: provide custom user/products or use defaults\n- Calculated fields (total) derived automatically from nested data\n- Explicit relationships make test data clear and maintainable\n\n### Example 3: Factory with API Seeding\n\n**Context**: When tests need data setup, always use API calls or database tasks—never UI navigation. Wrap factory usage with seeding utilities for clean test setup.\n\n**Implementation**:\n\n```typescript\n// playwright/support/helpers/seed-helpers.ts\nimport { APIRequestContext } from '@playwright/test';\nimport { User, createUser } from '../../test-utils/factories/user-factory';\nimport { Product, createProduct } from '../../test-utils/factories/product-factory';\n\nexport async function seedUser(request: APIRequestContext, overrides: Partial\u003cUser> = {}): Promise\u003cUser> {\n const user = createUser(overrides);\n\n const response = await request.post('/api/users', {\n data: user,\n });\n\n if (!response.ok()) {\n throw new Error(`Failed to seed user: ${response.status()}`);\n }\n\n return user;\n}\n\nexport async function seedProduct(request: APIRequestContext, overrides: Partial\u003cProduct> = {}): Promise\u003cProduct> {\n const product = createProduct(overrides);\n\n const response = await request.post('/api/products', {\n data: product,\n });\n\n if (!response.ok()) {\n throw new Error(`Failed to seed product: ${response.status()}`);\n }\n\n return product;\n}\n\n// Playwright globalSetup for shared data\n// playwright/support/global-setup.ts\nimport { chromium, FullConfig } from '@playwright/test';\nimport { seedUser } from './helpers/seed-helpers';\n\nasync function globalSetup(config: FullConfig) {\n const browser = await chromium.launch();\n const page = await browser.newPage();\n const context = page.context();\n\n // Seed admin user for all tests\n const admin = await seedUser(context.request, {\n email: '[email protected]',\n role: 'admin',\n });\n\n // Save auth state for reuse\n await context.storageState({ path: 'playwright/.auth/admin.json' });\n\n await browser.close();\n}\n\nexport default globalSetup;\n\n// Cypress equivalent with cy.task\n// cypress/support/tasks.ts\nexport const seedDatabase = async (entity: string, data: unknown) => {\n // Direct database insert or API call\n if (entity === 'users') {\n await db.users.create(data);\n }\n return null;\n};\n\n// Usage in Cypress tests:\nbeforeEach(() => {\n const user = createUser({ email: '[email protected]' });\n cy.task('db:seed', { entity: 'users', data: user });\n});\n```\n\n**Key Points**:\n\n- API seeding is 10-50x faster than UI-based setup\n- `globalSetup` seeds shared data once (e.g., admin user)\n- Per-test seeding uses `seedUser()` helpers for isolation\n- Cypress `cy.task` allows direct database access for speed\n\n### Example 4: Anti-Pattern - Hardcoded Test Data\n\n**Problem**:\n\n```typescript\n// ❌ BAD: Hardcoded test data\ntest('user can login', async ({ page }) => {\n await page.goto('/login');\n await page.fill('[data-testid=\"email\"]', '[email protected]'); // Hardcoded\n await page.fill('[data-testid=\"password\"]', 'password123'); // Hardcoded\n await page.click('[data-testid=\"submit\"]');\n\n // What if this user already exists? Test fails in parallel runs.\n // What if schema adds required fields? Test breaks.\n});\n\n// ❌ BAD: Static JSON fixtures\n// fixtures/users.json\n{\n \"users\": [\n { \"id\": 1, \"email\": \"[email protected]\", \"name\": \"User 1\" },\n { \"id\": 2, \"email\": \"[email protected]\", \"name\": \"User 2\" }\n ]\n}\n\ntest('admin can delete user', async ({ page }) => {\n const users = require('../fixtures/users.json');\n // Brittle: IDs collide in parallel, schema drift breaks tests\n});\n```\n\n**Why It Fails**:\n\n- **Parallel collisions**: Hardcoded IDs (`id: 1`, `email: '[email protected]'`) cause failures when tests run concurrently\n- **Schema drift**: Adding required fields (`phoneNumber`, `address`) breaks all tests using fixtures\n- **Hidden intent**: Does this test need `email: '[email protected]'` specifically, or any email?\n- **Slow setup**: UI-based data creation is 10-50x slower than API\n\n**Better Approach**: Use factories\n\n```typescript\n// ✅ GOOD: Factory-based data\ntest('user can login', async ({ page, apiRequest }) => {\n const user = createUser({ email: '[email protected]', password: 'secure123' });\n\n // Seed via API (fast, parallel-safe)\n await apiRequest({ method: 'POST', url: '/api/users', data: user });\n\n // Test UI\n await page.goto('/login');\n await page.fill('[data-testid=\"email\"]', user.email);\n await page.fill('[data-testid=\"password\"]', user.password);\n await page.click('[data-testid=\"submit\"]');\n\n await expect(page).toHaveURL('/dashboard');\n});\n\n// ✅ GOOD: Factories adapt to schema changes automatically\n// When `phoneNumber` becomes required, update factory once:\nexport const createUser = (overrides: Partial\u003cUser> = {}): User => ({\n id: faker.string.uuid(),\n email: faker.internet.email(),\n name: faker.person.fullName(),\n phoneNumber: faker.phone.number(), // NEW field, all tests get it automatically\n role: 'user',\n ...overrides,\n});\n```\n\n**Key Points**:\n\n- Factories generate unique, parallel-safe data\n- Schema evolution handled in one place (factory), not every test\n- Test intent explicit via overrides\n- API seeding is fast and reliable\n\n### Example 5: Factory Composition\n\n**Context**: When building specialized factories, compose simpler factories instead of duplicating logic. Layer overrides for specific test scenarios.\n\n**Implementation**:\n\n```typescript\n// test-utils/factories/user-factory.ts (base)\nexport const createUser = (overrides: Partial\u003cUser> = {}): User => ({\n id: faker.string.uuid(),\n email: faker.internet.email(),\n name: faker.person.fullName(),\n role: 'user',\n createdAt: new Date(),\n isActive: true,\n ...overrides,\n});\n\n// Compose specialized factories\nexport const createAdminUser = (overrides: Partial\u003cUser> = {}): User => createUser({ role: 'admin', ...overrides });\n\nexport const createModeratorUser = (overrides: Partial\u003cUser> = {}): User => createUser({ role: 'moderator', ...overrides });\n\nexport const createInactiveUser = (overrides: Partial\u003cUser> = {}): User => createUser({ isActive: false, ...overrides });\n\n// Account-level factories with feature flags\ntype Account = {\n id: string;\n owner: User;\n plan: 'free' | 'pro' | 'enterprise';\n features: string[];\n maxUsers: number;\n};\n\nexport const createAccount = (overrides: Partial\u003cAccount> = {}): Account => ({\n id: faker.string.uuid(),\n owner: overrides.owner || createUser(),\n plan: 'free',\n features: [],\n maxUsers: 1,\n ...overrides,\n});\n\nexport const createProAccount = (overrides: Partial\u003cAccount> = {}): Account =>\n createAccount({\n plan: 'pro',\n features: ['advanced-analytics', 'priority-support'],\n maxUsers: 10,\n ...overrides,\n });\n\nexport const createEnterpriseAccount = (overrides: Partial\u003cAccount> = {}): Account =>\n createAccount({\n plan: 'enterprise',\n features: ['advanced-analytics', 'priority-support', 'sso', 'audit-logs'],\n maxUsers: 100,\n ...overrides,\n });\n\n// Usage in tests:\ntest('pro accounts can access analytics', async ({ page, apiRequest }) => {\n const admin = createAdminUser({ email: '[email protected]' });\n const account = createProAccount({ owner: admin });\n\n await apiRequest({ method: 'POST', url: '/api/users', data: admin });\n await apiRequest({ method: 'POST', url: '/api/accounts', data: account });\n\n await page.goto('/analytics');\n await expect(page.getByText('Advanced Analytics')).toBeVisible();\n});\n\ntest('free accounts cannot access analytics', async ({ page, apiRequest }) => {\n const user = createUser({ email: '[email protected]' });\n const account = createAccount({ owner: user }); // Defaults to free plan\n\n await apiRequest({ method: 'POST', url: '/api/users', data: user });\n await apiRequest({ method: 'POST', url: '/api/accounts', data: account });\n\n await page.goto('/analytics');\n await expect(page.getByText('Upgrade to Pro')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- Compose specialized factories from base factories (`createAdminUser` → `createUser`)\n- Defaults cascade: `createProAccount` sets plan + features automatically\n- Still allow overrides: `createProAccount({ maxUsers: 50 })` works\n- Test intent clear: `createProAccount()` vs `createAccount({ plan: 'pro', features: [...] })`\n\n### Example 6: Naming the Literals You Do Hardcode\n\n**Context**: Everything above is about generating data so tests stay parallel-safe and unique. This is the other half, and the factories do not cover it: the literals a test writes on purpose because the assertion is about that exact value. A boundary, a rate, a limit, a status code, a currency scale.\n\nThose are correct to hardcode. Leaving them anonymous is what costs. `expect(fee).toBe(2.9)` tells the next reader that the fee is 2.9 and nothing about why, so when the number changes nobody can tell whether the test encodes a requirement or someone's old guess. The reader has to go find the pricing document, and usually does not.\n\nThe fix is a name, not a comment, and not a constant file. A named constant beside the test, or a factory override that reads as the domain fact, puts the meaning at the point of use.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: two unexplained literals. Which is a requirement, which is arbitrary?\ntest('applies the processing fee', async () => {\n const order = createOrder({ subtotal: 100 });\n expect(feeFor(order)).toBe(2.9);\n});\n\n// ✅ GOOD: each number states what it is\nconst STRIPE_PERCENT_FEE = 0.029; // per the payments contract, section 4\n\ntest('applies the processing fee', async () => {\n const subtotal = 100;\n const order = createOrder({ subtotal });\n expect(feeFor(order)).toBe(subtotal * STRIPE_PERCENT_FEE);\n});\n\n// ✅ ALSO GOOD: the factory carries the domain fact, so the test reads as\n// behavior and the limit needs no name at the call site at all\nconst order = createOrderAtItemLimit();\nawait expect(addItem(order)).rejects.toThrow('order is full');\n```\n\n**Key Points**:\n\n- Generate data that only needs to be unique; name data that carries meaning\n- The test is where the requirement gets encoded, so the number needs to say which requirement\n- Name at the point of use; a shared constants file moves the meaning away from the reader again\n- A value used once, whose meaning the test name already states, does not need a second name; this is about unexplained literals, not about every number\n\n## Integration Points\n\n- **Used in workflows**: `*atdd` (test generation), `*automate` (test expansion), `*framework` (factory setup)\n- **Related fragments**:\n - `fixture-architecture.md` - Pure functions and fixtures for factory integration\n - `network-first.md` - API-first setup patterns\n - `test-quality.md` - Parallel-safe, deterministic test design\n\n## Cleanup Strategy\n\nEnsure factories work with cleanup patterns:\n\n```typescript\n// Track created IDs for cleanup\nconst createdUsers: string[] = [];\n\nafterEach(async ({ apiRequest }) => {\n // Clean up all users created during test\n for (const userId of createdUsers) {\n await apiRequest({ method: 'DELETE', url: `/api/users/${userId}` });\n }\n createdUsers.length = 0;\n});\n\ntest('user registration flow', async ({ page, apiRequest }) => {\n const user = createUser();\n createdUsers.push(user.id);\n\n await apiRequest({ method: 'POST', url: '/api/users', data: user });\n // ... test logic\n});\n```\n\n## Feature Flag Integration\n\nWhen working with feature flags, layer them into factories:\n\n```typescript\nexport const createUserWithFlags = (\n overrides: Partial\u003cUser> = {},\n flags: Record\u003cstring, boolean> = {},\n): User & { flags: Record\u003cstring, boolean> } => ({\n ...createUser(overrides),\n flags: {\n 'new-dashboard': false,\n 'beta-features': false,\n ...flags,\n },\n});\n\n// Usage:\nconst user = createUserWithFlags(\n { email: '[email protected]' },\n {\n 'new-dashboard': true,\n 'beta-features': true,\n },\n);\n```\n\n_Source: Murat Testing Philosophy (lines 94-120), API-first testing patterns, faker.js documentation._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":17449,"content_sha256":"036572ce6222b98829c681c9c2cebc54fa89d670356e4bc7d50e402af91f587d"},{"filename":"knowledge/email-auth.md","content":"# Email-Based Authentication Testing\n\n## Principle\n\nEmail-based authentication (magic links, one-time codes, passwordless login) requires specialized testing with email capture services like Mailosaur or Ethereal. Extract magic links via HTML parsing or use built-in link extraction, preserve browser storage (local/session/cookies) when processing links, cache email payloads to avoid exhausting inbox quotas, and cover negative cases (expired links, reused links, multiple rapid requests). Log email IDs and links for troubleshooting, but scrub PII before committing artifacts.\n\n## Rationale\n\nEmail authentication introduces unique challenges: asynchronous email delivery, quota limits (AWS Cognito: 50/day), cost per email, and complex state management (session preservation across link clicks). Without proper patterns, tests become slow (wait for email each time), expensive (quota exhaustion), and brittle (timing issues, missing state). Using email capture services + session caching + state preservation patterns makes email auth tests fast, reliable, and cost-effective.\n\n## Pattern Examples\n\n### Example 1: Magic Link Extraction with Mailosaur\n\n**Context**: Passwordless login flow where user receives magic link via email, clicks it, and is authenticated.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/magic-link-auth.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Magic Link Authentication Flow\n * 1. User enters email\n * 2. Backend sends magic link\n * 3. Test retrieves email via Mailosaur\n * 4. Extract and visit magic link\n * 5. Verify user is authenticated\n */\n\n// Mailosaur configuration\nconst MAILOSAUR_API_KEY = process.env.MAILOSAUR_API_KEY!;\nconst MAILOSAUR_SERVER_ID = process.env.MAILOSAUR_SERVER_ID!;\n\n/**\n * Extract href from HTML email body\n * DOMParser provides XML/HTML parsing in Node.js\n */\nfunction extractMagicLink(htmlString: string): string | null {\n const { JSDOM } = require('jsdom');\n const dom = new JSDOM(htmlString);\n const link = dom.window.document.querySelector('#magic-link-button');\n return link ? (link as HTMLAnchorElement).href : null;\n}\n\n/**\n * Alternative: Use Mailosaur's built-in link extraction\n * Mailosaur automatically parses links - no regex needed!\n */\nasync function getMagicLinkFromEmail(email: string): Promise\u003cstring> {\n const MailosaurClient = require('mailosaur');\n const mailosaur = new MailosaurClient(MAILOSAUR_API_KEY);\n\n // Wait for email (timeout: 30 seconds)\n const message = await mailosaur.messages.get(\n MAILOSAUR_SERVER_ID,\n {\n sentTo: email,\n },\n {\n timeout: 30000, // 30 seconds\n },\n );\n\n // Mailosaur extracts links automatically - no parsing needed!\n const magicLink = message.html?.links?.[0]?.href;\n\n if (!magicLink) {\n throw new Error(`Magic link not found in email to ${email}`);\n }\n\n console.log(`📧 Email received. Magic link extracted: ${magicLink}`);\n return magicLink;\n}\n\ntest.describe('Magic Link Authentication', () => {\n test('should authenticate user via magic link', async ({ page, context }) => {\n // Arrange: Generate unique test email\n const randomId = Math.floor(Math.random() * 1000000);\n const testEmail = `user-${randomId}@${MAILOSAUR_SERVER_ID}.mailosaur.net`;\n\n // Act: Request magic link\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n\n // Assert: Success message\n await expect(page.getByTestId('check-email-message')).toBeVisible();\n await expect(page.getByTestId('check-email-message')).toContainText('Check your email');\n\n // Retrieve magic link from email\n const magicLink = await getMagicLinkFromEmail(testEmail);\n\n // Visit magic link\n await page.goto(magicLink);\n\n // Assert: User is authenticated\n await expect(page.getByTestId('user-menu')).toBeVisible();\n await expect(page.getByTestId('user-email')).toContainText(testEmail);\n\n // Verify session storage preserved\n const localStorage = await page.evaluate(() => JSON.stringify(window.localStorage));\n expect(localStorage).toContain('authToken');\n });\n\n test('should handle expired magic link', async ({ page }) => {\n // Use pre-expired link (older than 15 minutes)\n const expiredLink = 'http://localhost:3000/auth/verify?token=expired-token-123';\n\n await page.goto(expiredLink);\n\n // Assert: Error message displayed\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText('link has expired');\n\n // Assert: User NOT authenticated\n await expect(page.getByTestId('user-menu')).not.toBeVisible();\n });\n\n test('should prevent reusing magic link', async ({ page }) => {\n const randomId = Math.floor(Math.random() * 1000000);\n const testEmail = `user-${randomId}@${MAILOSAUR_SERVER_ID}.mailosaur.net`;\n\n // Request magic link\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n\n const magicLink = await getMagicLinkFromEmail(testEmail);\n\n // Visit link first time (success)\n await page.goto(magicLink);\n await expect(page.getByTestId('user-menu')).toBeVisible();\n\n // Sign out\n await page.getByTestId('sign-out').click();\n\n // Try to reuse same link (should fail)\n await page.goto(magicLink);\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText('link has already been used');\n });\n});\n```\n\n**Cypress equivalent with Mailosaur plugin**:\n\n```javascript\n// cypress/e2e/magic-link-auth.cy.ts\ndescribe('Magic Link Authentication', () => {\n it('should authenticate user via magic link', () => {\n const serverId = Cypress.env('MAILOSAUR_SERVERID');\n const randomId = Cypress._.random(1e6);\n const testEmail = `user-${randomId}@${serverId}.mailosaur.net`;\n\n // Request magic link\n cy.visit('/login');\n cy.get('[data-cy=\"email-input\"]').type(testEmail);\n cy.get('[data-cy=\"send-magic-link\"]').click();\n cy.get('[data-cy=\"check-email-message\"]').should('be.visible');\n\n // Retrieve and visit magic link\n cy.mailosaurGetMessage(serverId, { sentTo: testEmail })\n .its('html.links.0.href') // Mailosaur extracts links automatically!\n .should('exist')\n .then((magicLink) => {\n cy.log(`Magic link: ${magicLink}`);\n cy.visit(magicLink);\n });\n\n // Verify authenticated\n cy.get('[data-cy=\"user-menu\"]').should('be.visible');\n cy.get('[data-cy=\"user-email\"]').should('contain', testEmail);\n });\n});\n```\n\n**Key Points**:\n\n- **Mailosaur auto-extraction**: `html.links[0].href` or `html.codes[0].value`\n- **Unique emails**: Random ID prevents collisions\n- **Negative testing**: Expired and reused links tested\n- **State verification**: localStorage/session checked\n- **Fast email retrieval**: 30 second timeout typical\n\n---\n\n### Example 2: State Preservation Pattern with cy.session / Playwright storageState\n\n**Context**: Cache authenticated session to avoid requesting magic link on every test.\n\n**Implementation**:\n\n```typescript\n// playwright/fixtures/email-auth-fixture.ts\nimport { test as base } from '@playwright/test';\nimport { getMagicLinkFromEmail } from '../support/mailosaur-helpers';\n\ntype EmailAuthFixture = {\n authenticatedUser: { email: string; token: string };\n};\n\nexport const test = base.extend\u003cEmailAuthFixture>({\n authenticatedUser: async ({ page, context }, use) => {\n const randomId = Math.floor(Math.random() * 1000000);\n const testEmail = `user-${randomId}@${process.env.MAILOSAUR_SERVER_ID}.mailosaur.net`;\n\n // Check if we have cached auth state for this email\n const storageStatePath = `./test-results/auth-state-${testEmail}.json`;\n\n try {\n // Try to reuse existing session\n await context.storageState({ path: storageStatePath });\n await page.goto('/dashboard');\n\n // Validate session is still valid\n const isAuthenticated = await page.getByTestId('user-menu').isVisible({ timeout: 2000 });\n\n if (isAuthenticated) {\n console.log(`✅ Reusing cached session for ${testEmail}`);\n await use({ email: testEmail, token: 'cached' });\n return;\n }\n } catch (error) {\n console.log(`📧 No cached session, requesting magic link for ${testEmail}`);\n }\n\n // Request new magic link\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n\n // Get magic link from email\n const magicLink = await getMagicLinkFromEmail(testEmail);\n\n // Visit link and authenticate\n await page.goto(magicLink);\n await expect(page.getByTestId('user-menu')).toBeVisible();\n\n // Extract auth token from localStorage\n const authToken = await page.evaluate(() => localStorage.getItem('authToken'));\n\n // Save session state for reuse\n await context.storageState({ path: storageStatePath });\n\n console.log(`💾 Cached session for ${testEmail}`);\n\n await use({ email: testEmail, token: authToken || '' });\n },\n});\n```\n\n**Cypress equivalent with cy.session + data-session**:\n\n```javascript\n// cypress/support/commands/email-auth.js\nimport { dataSession } from 'cypress-data-session';\n\n/**\n * Authenticate via magic link with session caching\n * - First run: Requests email, extracts link, authenticates\n * - Subsequent runs: Reuses cached session (no email)\n */\nCypress.Commands.add('authViaMagicLink', (email) => {\n return dataSession({\n name: `magic-link-${email}`,\n\n // First-time setup: Request and process magic link\n setup: () => {\n cy.visit('/login');\n cy.get('[data-cy=\"email-input\"]').type(email);\n cy.get('[data-cy=\"send-magic-link\"]').click();\n\n // Get magic link from Mailosaur\n cy.mailosaurGetMessage(Cypress.env('MAILOSAUR_SERVERID'), {\n sentTo: email,\n })\n .its('html.links.0.href')\n .should('exist')\n .then((magicLink) => {\n cy.visit(magicLink);\n });\n\n // Wait for authentication\n cy.get('[data-cy=\"user-menu\"]', { timeout: 10000 }).should('be.visible');\n\n // Preserve authentication state\n return cy.getAllLocalStorage().then((storage) => {\n return { storage, email };\n });\n },\n\n // Validate cached session is still valid\n validate: (cached) => {\n return cy.wrap(Boolean(cached?.storage));\n },\n\n // Recreate session from cache (no email needed)\n recreate: (cached) => {\n // Restore localStorage\n cy.setLocalStorage(cached.storage);\n cy.visit('/dashboard');\n cy.get('[data-cy=\"user-menu\"]', { timeout: 5000 }).should('be.visible');\n },\n\n shareAcrossSpecs: true, // Share session across all tests\n });\n});\n```\n\n**Usage in tests**:\n\n```javascript\n// cypress/e2e/dashboard.cy.ts\ndescribe('Dashboard', () => {\n const serverId = Cypress.env('MAILOSAUR_SERVERID');\n const testEmail = `test-user@${serverId}.mailosaur.net`;\n\n beforeEach(() => {\n // First test: Requests magic link\n // Subsequent tests: Reuses cached session (no email!)\n cy.authViaMagicLink(testEmail);\n });\n\n it('should display user dashboard', () => {\n cy.get('[data-cy=\"dashboard-content\"]').should('be.visible');\n });\n\n it('should show user profile', () => {\n cy.get('[data-cy=\"user-email\"]').should('contain', testEmail);\n });\n\n // Both tests share same session - only 1 email consumed!\n});\n```\n\n**Key Points**:\n\n- **Session caching**: First test requests email, rest reuse session\n- **State preservation**: localStorage/cookies saved and restored\n- **Validation**: Check cached session is still valid\n- **Quota optimization**: Massive reduction in email consumption\n- **Fast tests**: Cached auth takes seconds vs. minutes\n\n---\n\n### Example 3: Negative Flow Tests (Expired, Invalid, Reused Links)\n\n**Context**: Comprehensive negative testing for email authentication edge cases.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/email-auth-negative.spec.ts\nimport { test, expect } from '@playwright/test';\nimport { getMagicLinkFromEmail } from '../support/mailosaur-helpers';\n\nconst MAILOSAUR_SERVER_ID = process.env.MAILOSAUR_SERVER_ID!;\n\ntest.describe('Email Auth Negative Flows', () => {\n test('should reject expired magic link', async ({ page }) => {\n // Generate expired link (simulate 24 hours ago)\n const expiredToken = Buffer.from(\n JSON.stringify({\n email: '[email protected]',\n exp: Date.now() - 24 * 60 * 60 * 1000, // 24 hours ago\n }),\n ).toString('base64');\n\n const expiredLink = `http://localhost:3000/auth/verify?token=${expiredToken}`;\n\n // Visit expired link\n await page.goto(expiredLink);\n\n // Assert: Error displayed\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText(/link.*expired|expired.*link/i);\n\n // Assert: Link to request new one\n await expect(page.getByTestId('request-new-link')).toBeVisible();\n\n // Assert: User NOT authenticated\n await expect(page.getByTestId('user-menu')).not.toBeVisible();\n });\n\n test('should reject invalid magic link token', async ({ page }) => {\n const invalidLink = 'http://localhost:3000/auth/verify?token=invalid-garbage';\n\n await page.goto(invalidLink);\n\n // Assert: Error displayed\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText(/invalid.*link|link.*invalid/i);\n\n // Assert: User not authenticated\n await expect(page.getByTestId('user-menu')).not.toBeVisible();\n });\n\n test('should reject already-used magic link', async ({ page, context }) => {\n const randomId = Math.floor(Math.random() * 1000000);\n const testEmail = `user-${randomId}@${MAILOSAUR_SERVER_ID}.mailosaur.net`;\n\n // Request magic link\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n\n const magicLink = await getMagicLinkFromEmail(testEmail);\n\n // Visit link FIRST time (success)\n await page.goto(magicLink);\n await expect(page.getByTestId('user-menu')).toBeVisible();\n\n // Sign out\n await page.getByTestId('user-menu').click();\n await page.getByTestId('sign-out').click();\n await expect(page.getByTestId('user-menu')).not.toBeVisible();\n\n // Try to reuse SAME link (should fail)\n await page.goto(magicLink);\n\n // Assert: Link already used error\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText(/already.*used|link.*used/i);\n\n // Assert: User not authenticated\n await expect(page.getByTestId('user-menu')).not.toBeVisible();\n });\n\n test('should handle rapid successive link requests', async ({ page }) => {\n const randomId = Math.floor(Math.random() * 1000000);\n const testEmail = `user-${randomId}@${MAILOSAUR_SERVER_ID}.mailosaur.net`;\n\n // Request magic link 3 times rapidly\n for (let i = 0; i \u003c 3; i++) {\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n await expect(page.getByTestId('check-email-message')).toBeVisible();\n }\n\n // Only the LATEST link should work\n const MailosaurClient = require('mailosaur');\n const mailosaur = new MailosaurClient(process.env.MAILOSAUR_API_KEY);\n\n const messages = await mailosaur.messages.list(MAILOSAUR_SERVER_ID, {\n sentTo: testEmail,\n });\n\n // Should receive 3 emails\n expect(messages.items.length).toBeGreaterThanOrEqual(3);\n\n // Get the LATEST magic link\n const latestMessage = messages.items[0]; // Most recent first\n const latestLink = latestMessage.html.links[0].href;\n\n // Latest link works\n await page.goto(latestLink);\n await expect(page.getByTestId('user-menu')).toBeVisible();\n\n // Older links should NOT work (if backend invalidates previous)\n await page.getByTestId('sign-out').click();\n const olderLink = messages.items[1].html.links[0].href;\n\n await page.goto(olderLink);\n await expect(page.getByTestId('error-message')).toBeVisible();\n });\n\n test('should rate-limit excessive magic link requests', async ({ page }) => {\n const randomId = Math.floor(Math.random() * 1000000);\n const testEmail = `user-${randomId}@${MAILOSAUR_SERVER_ID}.mailosaur.net`;\n\n // Request magic link 10 times rapidly (should hit rate limit)\n for (let i = 0; i \u003c 10; i++) {\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n\n // After N requests, should show rate limit error\n const errorVisible = await page\n .getByTestId('rate-limit-error')\n .isVisible({ timeout: 1000 })\n .catch(() => false);\n\n if (errorVisible) {\n console.log(`Rate limit hit after ${i + 1} requests`);\n await expect(page.getByTestId('rate-limit-error')).toContainText(/too many.*requests|rate.*limit/i);\n return;\n }\n }\n\n // If no rate limit after 10 requests, log warning\n console.warn('⚠️ No rate limit detected after 10 requests');\n });\n});\n```\n\n**Key Points**:\n\n- **Expired links**: Test 24+ hour old tokens\n- **Invalid tokens**: Malformed or garbage tokens rejected\n- **Reuse prevention**: Same link can't be used twice\n- **Rapid requests**: Multiple requests handled gracefully\n- **Rate limiting**: Excessive requests blocked\n\n---\n\n### Example 4: Caching Strategy with cypress-data-session / Playwright Projects\n\n**Context**: Minimize email consumption by sharing authentication state across tests and specs.\n\n**Implementation**:\n\n```javascript\n// cypress/support/commands/register-and-sign-in.js\nimport { dataSession } from 'cypress-data-session';\n\n/**\n * Email Authentication Caching Strategy\n * - One email per test run (not per spec, not per test)\n * - First spec: Full registration flow (form → email → code → sign in)\n * - Subsequent specs: Only sign in (reuse user)\n * - Subsequent tests in same spec: Session already active (no sign in)\n */\n\n// Helper: Fill registration form\nfunction fillRegistrationForm({ fullName, userName, email, password }) {\n cy.intercept('POST', 'https://cognito-idp*').as('cognito');\n cy.contains('Register').click();\n cy.get('#reg-dialog-form').should('be.visible');\n cy.get('#first-name').type(fullName, { delay: 0 });\n cy.get('#last-name').type(lastName, { delay: 0 });\n cy.get('#email').type(email, { delay: 0 });\n cy.get('#username').type(userName, { delay: 0 });\n cy.get('#password').type(password, { delay: 0 });\n cy.contains('button', 'Create an account').click();\n cy.wait('@cognito').its('response.statusCode').should('equal', 200);\n}\n\n// Helper: Confirm registration with email code\nfunction confirmRegistration(email) {\n return cy\n .mailosaurGetMessage(Cypress.env('MAILOSAUR_SERVERID'), { sentTo: email })\n .its('html.codes.0.value') // Mailosaur auto-extracts codes!\n .then((code) => {\n cy.intercept('POST', 'https://cognito-idp*').as('cognito');\n cy.get('#verification-code').type(code, { delay: 0 });\n cy.contains('button', 'Confirm registration').click();\n cy.wait('@cognito');\n cy.contains('You are now registered!').should('be.visible');\n cy.contains('button', /ok/i).click();\n return cy.wrap(code); // Return code for reference\n });\n}\n\n// Helper: Full registration (form + email)\nfunction register({ fullName, userName, email, password }) {\n fillRegistrationForm({ fullName, userName, email, password });\n return confirmRegistration(email);\n}\n\n// Helper: Sign in\nfunction signIn({ userName, password }) {\n cy.intercept('POST', 'https://cognito-idp*').as('cognito');\n cy.contains('Sign in').click();\n cy.get('#sign-in-username').type(userName, { delay: 0 });\n cy.get('#sign-in-password').type(password, { delay: 0 });\n cy.contains('button', 'Sign in').click();\n cy.wait('@cognito');\n cy.contains('Sign out').should('be.visible');\n}\n\n/**\n * Register and sign in with email caching\n * ONE EMAIL PER MACHINE (cypress run or cypress open)\n */\nCypress.Commands.add('registerAndSignIn', ({ fullName, userName, email, password }) => {\n return dataSession({\n name: email, // Unique session per email\n\n // First time: Full registration (form → email → code)\n init: () => register({ fullName, userName, email, password }),\n\n // Subsequent specs: Just check email exists (code already used)\n setup: () => confirmRegistration(email),\n\n // Always runs after init/setup: Sign in\n recreate: () => signIn({ userName, password }),\n\n // Share across ALL specs (one email for entire test run)\n shareAcrossSpecs: true,\n });\n});\n```\n\n**Usage across multiple specs**:\n\n```javascript\n// cypress/e2e/place-order.cy.ts\ndescribe('Place Order', () => {\n beforeEach(() => {\n cy.visit('/');\n cy.registerAndSignIn({\n fullName: Cypress.env('fullName'), // From cypress.config\n userName: Cypress.env('userName'),\n email: Cypress.env('email'), // SAME email across all specs\n password: Cypress.env('password'),\n });\n });\n\n it('should place order', () => {\n /* ... */\n });\n it('should view order history', () => {\n /* ... */\n });\n});\n\n// cypress/e2e/profile.cy.ts\ndescribe('User Profile', () => {\n beforeEach(() => {\n cy.visit('/');\n cy.registerAndSignIn({\n fullName: Cypress.env('fullName'),\n userName: Cypress.env('userName'),\n email: Cypress.env('email'), // SAME email - no new email sent!\n password: Cypress.env('password'),\n });\n });\n\n it('should update profile', () => {\n /* ... */\n });\n});\n```\n\n**Playwright equivalent with storageState**:\n\n```typescript\n// playwright.config.ts\nimport { defineConfig } from '@playwright/test';\n\nexport default defineConfig({\n projects: [\n {\n name: 'setup',\n testMatch: /global-setup\\.ts/,\n },\n {\n name: 'authenticated',\n testMatch: /.*\\.spec\\.ts/,\n dependencies: ['setup'],\n use: {\n storageState: '.auth/user-session.json', // Reuse auth state\n },\n },\n ],\n});\n```\n\n```typescript\n// tests/global-setup.ts (runs once)\nimport { test as setup } from '@playwright/test';\nimport { getMagicLinkFromEmail } from './support/mailosaur-helpers';\n\nconst authFile = '.auth/user-session.json';\n\nsetup('authenticate via magic link', async ({ page }) => {\n const testEmail = process.env.TEST_USER_EMAIL!;\n\n // Request magic link\n await page.goto('/login');\n await page.getByTestId('email-input').fill(testEmail);\n await page.getByTestId('send-magic-link').click();\n\n // Get and visit magic link\n const magicLink = await getMagicLinkFromEmail(testEmail);\n await page.goto(magicLink);\n\n // Verify authenticated\n await expect(page.getByTestId('user-menu')).toBeVisible();\n\n // Save authenticated state (ONE TIME for all tests)\n await page.context().storageState({ path: authFile });\n\n console.log('✅ Authentication state saved to', authFile);\n});\n```\n\n**Key Points**:\n\n- **One email per run**: Global setup authenticates once\n- **State reuse**: All tests use cached storageState\n- **cypress-data-session**: Intelligently manages cache lifecycle\n- **shareAcrossSpecs**: Session shared across all spec files\n- **Massive savings**: 500 tests = 1 email (not 500!)\n\n---\n\n## Email Authentication Testing Checklist\n\nBefore implementing email auth tests, verify:\n\n- [ ] **Email service**: Mailosaur/Ethereal/MailHog configured with API keys\n- [ ] **Link extraction**: Use built-in parsing (html.links[0].href) over regex\n- [ ] **State preservation**: localStorage/session/cookies saved and restored\n- [ ] **Session caching**: cypress-data-session or storageState prevents redundant emails\n- [ ] **Negative flows**: Expired, invalid, reused, rapid requests tested\n- [ ] **Quota awareness**: One email per run (not per test)\n- [ ] **PII scrubbing**: Email IDs logged for debug, but scrubbed from artifacts\n- [ ] **Timeout handling**: 30 second email retrieval timeout configured\n\n## Integration Points\n\n- Used in workflows: `*framework` (email auth setup), `*automate` (email auth test generation)\n- Related fragments: `fixture-architecture.md`, `test-quality.md`\n- Email services: Mailosaur (recommended), Ethereal (free), MailHog (self-hosted)\n- Plugins: cypress-mailosaur, cypress-data-session\n\n_Source: Email authentication blog, Murat testing toolkit, Mailosaur documentation._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":24554,"content_sha256":"157996b68a3232f62a6911dd2cd336ca159098cf2210f991ae3e961f27eabc2c"},{"filename":"knowledge/error-handling.md","content":"# Error Handling and Resilience Checks\n\n## Principle\n\nTreat expected failures explicitly: intercept network errors, assert UI fallbacks (error messages visible, retries triggered), and use scoped exception handling to ignore known errors while catching regressions. Test retry/backoff logic by forcing sequential failures (500 → timeout → success) and validate telemetry logging. Log captured errors with context (request payload, user/session) but redact secrets to keep artifacts safe for sharing.\n\n## Rationale\n\nTests fail for two reasons: genuine bugs or poor error handling in the test itself. Without explicit error handling patterns, tests become noisy (uncaught exceptions cause false failures) or silent (swallowing all errors hides real bugs). Scoped exception handling (Cypress.on('uncaught:exception'), page.on('pageerror')) allows tests to ignore documented, expected errors while surfacing unexpected ones. Resilience testing (retry logic, graceful degradation) ensures applications handle failures gracefully in production.\n\n## Pattern Examples\n\n### Example 1: Scoped Exception Handling (Expected Errors Only)\n\n**Context**: Handle known errors (Network failures, expected 500s) without masking unexpected bugs.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/error-handling.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Scoped Error Handling Pattern\n * - Only ignore specific, documented errors\n * - Rethrow everything else to catch regressions\n * - Validate error UI and user experience\n */\n\ntest.describe('API Error Handling', () => {\n test('should display error message when API returns 500', async ({ page }) => {\n // Scope error handling to THIS test only\n const consoleErrors: string[] = [];\n page.on('pageerror', (error) => {\n // Only swallow documented NetworkError\n if (error.message.includes('NetworkError: Failed to fetch')) {\n consoleErrors.push(error.message);\n return; // Swallow this specific error\n }\n // Rethrow all other errors (catch regressions!)\n throw error;\n });\n\n // Arrange: Mock 500 error response\n await page.route('**/api/users', (route) =>\n route.fulfill({\n status: 500,\n contentType: 'application/json',\n body: JSON.stringify({\n error: 'Internal server error',\n code: 'INTERNAL_ERROR',\n }),\n }),\n );\n\n // Act: Navigate to page that fetches users\n await page.goto('/dashboard');\n\n // Assert: Error UI displayed\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText(/error.*loading|failed.*load/i);\n\n // Assert: Retry button visible\n await expect(page.getByTestId('retry-button')).toBeVisible();\n\n // Assert: NetworkError was thrown and caught\n expect(consoleErrors).toContainEqual(expect.stringContaining('NetworkError'));\n });\n\n test('should NOT swallow unexpected errors', async ({ page }) => {\n let unexpectedError: Error | null = null;\n\n page.on('pageerror', (error) => {\n // Capture but don't swallow - test should fail\n unexpectedError = error;\n throw error;\n });\n\n // Arrange: App has JavaScript error (bug)\n await page.addInitScript(() => {\n // Simulate bug in app code\n (window as any).buggyFunction = () => {\n throw new Error('UNEXPECTED BUG: undefined is not a function');\n };\n });\n\n await page.goto('/dashboard');\n\n // Trigger buggy function\n await page.evaluate(() => (window as any).buggyFunction());\n\n // Assert: Test fails because unexpected error was NOT swallowed\n expect(unexpectedError).not.toBeNull();\n expect(unexpectedError?.message).toContain('UNEXPECTED BUG');\n });\n});\n```\n\n**Cypress equivalent**:\n\n```javascript\n// cypress/e2e/error-handling.cy.ts\ndescribe('API Error Handling', () => {\n it('should display error message when API returns 500', () => {\n // Scoped to this test only\n cy.on('uncaught:exception', (err) => {\n // Only swallow documented NetworkError\n if (err.message.includes('NetworkError')) {\n return false; // Prevent test failure\n }\n // All other errors fail the test\n return true;\n });\n\n // Arrange: Mock 500 error\n cy.intercept('GET', '**/api/users', {\n statusCode: 500,\n body: {\n error: 'Internal server error',\n code: 'INTERNAL_ERROR',\n },\n }).as('getUsers');\n\n // Act\n cy.visit('/dashboard');\n cy.wait('@getUsers');\n\n // Assert: Error UI\n cy.get('[data-cy=\"error-message\"]').should('be.visible');\n cy.get('[data-cy=\"error-message\"]').should('contain', 'error loading');\n cy.get('[data-cy=\"retry-button\"]').should('be.visible');\n });\n\n it('should NOT swallow unexpected errors', () => {\n // No exception handler - test should fail on unexpected errors\n\n cy.visit('/dashboard');\n\n // Trigger unexpected error\n cy.window().then((win) => {\n // This should fail the test\n win.eval('throw new Error(\"UNEXPECTED BUG\")');\n });\n\n // Test fails (as expected) - validates error detection works\n });\n});\n```\n\n**Key Points**:\n\n- **Scoped handling**: page.on() / cy.on() scoped to specific tests\n- **Explicit allow-list**: Only ignore documented errors\n- **Rethrow unexpected**: Catch regressions by failing on unknown errors\n- **Error UI validation**: Assert user sees error message\n- **Logging**: Capture errors for debugging, don't swallow silently\n\n---\n\n### Example 2: Retry Validation Pattern (Network Resilience)\n\n**Context**: Test that retry/backoff logic works correctly for transient failures.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/retry-resilience.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Retry Validation Pattern\n * - Force sequential failures (500 → 500 → 200)\n * - Validate retry attempts and backoff timing\n * - Assert telemetry captures retry events\n */\n\ntest.describe('Network Retry Logic', () => {\n test('should retry on 500 error and succeed', async ({ page }) => {\n let attemptCount = 0;\n const attemptTimestamps: number[] = [];\n\n // Mock API: Fail twice, succeed on third attempt\n await page.route('**/api/products', (route) => {\n attemptCount++;\n attemptTimestamps.push(Date.now());\n\n if (attemptCount \u003c= 2) {\n // First 2 attempts: 500 error\n route.fulfill({\n status: 500,\n body: JSON.stringify({ error: 'Server error' }),\n });\n } else {\n // 3rd attempt: Success\n route.fulfill({\n status: 200,\n contentType: 'application/json',\n body: JSON.stringify({ products: [{ id: 1, name: 'Product 1' }] }),\n });\n }\n });\n\n // Act: Navigate (should retry automatically)\n await page.goto('/products');\n\n // Assert: Data eventually loads after retries\n await expect(page.getByTestId('product-list')).toBeVisible();\n await expect(page.getByTestId('product-item')).toHaveCount(1);\n\n // Assert: Exactly 3 attempts made\n expect(attemptCount).toBe(3);\n\n // Assert: Exponential backoff timing (1s → 2s between attempts)\n if (attemptTimestamps.length === 3) {\n const delay1 = attemptTimestamps[1] - attemptTimestamps[0];\n const delay2 = attemptTimestamps[2] - attemptTimestamps[1];\n\n expect(delay1).toBeGreaterThanOrEqual(900); // ~1 second\n expect(delay1).toBeLessThan(1200);\n expect(delay2).toBeGreaterThanOrEqual(1900); // ~2 seconds\n expect(delay2).toBeLessThan(2200);\n }\n\n // Assert: Telemetry logged retry events\n const telemetryEvents = await page.evaluate(() => (window as any).__TELEMETRY_EVENTS__ || []);\n expect(telemetryEvents).toContainEqual(\n expect.objectContaining({\n event: 'api_retry',\n attempt: 1,\n endpoint: '/api/products',\n }),\n );\n expect(telemetryEvents).toContainEqual(\n expect.objectContaining({\n event: 'api_retry',\n attempt: 2,\n }),\n );\n });\n\n test('should give up after max retries and show error', async ({ page }) => {\n let attemptCount = 0;\n\n // Mock API: Always fail (test retry limit)\n await page.route('**/api/products', (route) => {\n attemptCount++;\n route.fulfill({\n status: 500,\n body: JSON.stringify({ error: 'Persistent server error' }),\n });\n });\n\n // Act\n await page.goto('/products');\n\n // Assert: Max retries reached (3 attempts typical)\n expect(attemptCount).toBe(3);\n\n // Assert: Error UI displayed after exhausting retries\n await expect(page.getByTestId('error-message')).toBeVisible();\n await expect(page.getByTestId('error-message')).toContainText(/unable.*load|failed.*after.*retries/i);\n\n // Assert: Data not displayed\n await expect(page.getByTestId('product-list')).not.toBeVisible();\n });\n\n test('should NOT retry on 404 (non-retryable error)', async ({ page }) => {\n let attemptCount = 0;\n\n // Mock API: 404 error (should NOT retry)\n await page.route('**/api/products/999', (route) => {\n attemptCount++;\n route.fulfill({\n status: 404,\n body: JSON.stringify({ error: 'Product not found' }),\n });\n });\n\n await page.goto('/products/999');\n\n // Assert: Only 1 attempt (no retries on 404)\n expect(attemptCount).toBe(1);\n\n // Assert: 404 error displayed immediately\n await expect(page.getByTestId('not-found-message')).toBeVisible();\n });\n});\n```\n\n**Cypress with retry interception**:\n\n```javascript\n// cypress/e2e/retry-resilience.cy.ts\ndescribe('Network Retry Logic', () => {\n it('should retry on 500 and succeed on 3rd attempt', () => {\n let attemptCount = 0;\n\n cy.intercept('GET', '**/api/products', (req) => {\n attemptCount++;\n\n if (attemptCount \u003c= 2) {\n req.reply({ statusCode: 500, body: { error: 'Server error' } });\n } else {\n req.reply({ statusCode: 200, body: { products: [{ id: 1, name: 'Product 1' }] } });\n }\n }).as('getProducts');\n\n cy.visit('/products');\n\n // Wait for final successful request\n cy.wait('@getProducts').its('response.statusCode').should('eq', 200);\n\n // Assert: Data loaded\n cy.get('[data-cy=\"product-list\"]').should('be.visible');\n cy.get('[data-cy=\"product-item\"]').should('have.length', 1);\n\n // Validate retry count\n cy.wrap(attemptCount).should('eq', 3);\n });\n});\n```\n\n**Key Points**:\n\n- **Sequential failures**: Test retry logic with 500 → 500 → 200\n- **Backoff timing**: Validate exponential backoff delays\n- **Retry limits**: Max attempts enforced (typically 3)\n- **Non-retryable errors**: 404s don't trigger retries\n- **Telemetry**: Log retry attempts for monitoring\n\n---\n\n### Example 3: Telemetry Logging with Context (Sentry Integration)\n\n**Context**: Capture errors with full context for production debugging without exposing secrets.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/telemetry-logging.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Telemetry Logging Pattern\n * - Log errors with request context\n * - Redact sensitive data (tokens, passwords, PII)\n * - Integrate with monitoring (Sentry, Datadog)\n * - Validate error logging without exposing secrets\n */\n\ntype ErrorLog = {\n level: 'error' | 'warn' | 'info';\n message: string;\n context?: {\n endpoint?: string;\n method?: string;\n statusCode?: number;\n userId?: string;\n sessionId?: string;\n };\n timestamp: string;\n};\n\ntest.describe('Error Telemetry', () => {\n test('should log API errors with context', async ({ page }) => {\n const errorLogs: ErrorLog[] = [];\n\n // Capture console errors\n page.on('console', (msg) => {\n if (msg.type() === 'error') {\n try {\n const log = JSON.parse(msg.text());\n errorLogs.push(log);\n } catch {\n // Not a structured log, ignore\n }\n }\n });\n\n // Mock failing API\n await page.route('**/api/orders', (route) =>\n route.fulfill({\n status: 500,\n body: JSON.stringify({ error: 'Payment processor unavailable' }),\n }),\n );\n\n // Act: Trigger error\n await page.goto('/checkout');\n await page.getByTestId('place-order').click();\n\n // Wait for error UI\n await expect(page.getByTestId('error-message')).toBeVisible();\n\n // Assert: Error logged with context\n expect(errorLogs).toContainEqual(\n expect.objectContaining({\n level: 'error',\n message: expect.stringContaining('API request failed'),\n context: expect.objectContaining({\n endpoint: '/api/orders',\n method: 'POST',\n statusCode: 500,\n userId: expect.any(String),\n }),\n }),\n );\n\n // Assert: Sensitive data NOT logged\n const logString = JSON.stringify(errorLogs);\n expect(logString).not.toContain('password');\n expect(logString).not.toContain('token');\n expect(logString).not.toContain('creditCard');\n });\n\n test('should send errors to Sentry with breadcrumbs', async ({ page }) => {\n const sentryEvents: any[] = [];\n\n // Mock Sentry SDK\n await page.addInitScript(() => {\n (window as any).Sentry = {\n captureException: (error: Error, context?: any) => {\n (window as any).__SENTRY_EVENTS__ = (window as any).__SENTRY_EVENTS__ || [];\n (window as any).__SENTRY_EVENTS__.push({\n error: error.message,\n context,\n timestamp: Date.now(),\n });\n },\n addBreadcrumb: (breadcrumb: any) => {\n (window as any).__SENTRY_BREADCRUMBS__ = (window as any).__SENTRY_BREADCRUMBS__ || [];\n (window as any).__SENTRY_BREADCRUMBS__.push(breadcrumb);\n },\n };\n });\n\n // Mock failing API\n await page.route('**/api/users', (route) => route.fulfill({ status: 403, body: { error: 'Forbidden' } }));\n\n // Act\n await page.goto('/users');\n\n // Assert: Sentry captured error\n const events = await page.evaluate(() => (window as any).__SENTRY_EVENTS__);\n expect(events).toHaveLength(1);\n expect(events[0]).toMatchObject({\n error: expect.stringContaining('403'),\n context: expect.objectContaining({\n endpoint: '/api/users',\n statusCode: 403,\n }),\n });\n\n // Assert: Breadcrumbs include user actions\n const breadcrumbs = await page.evaluate(() => (window as any).__SENTRY_BREADCRUMBS__);\n expect(breadcrumbs).toContainEqual(\n expect.objectContaining({\n category: 'navigation',\n message: '/users',\n }),\n );\n });\n});\n```\n\n**Cypress with Sentry**:\n\n```javascript\n// cypress/e2e/telemetry-logging.cy.ts\ndescribe('Error Telemetry', () => {\n it('should log API errors with redacted sensitive data', () => {\n const errorLogs = [];\n\n // Capture console errors\n cy.on('window:before:load', (win) => {\n cy.stub(win.console, 'error').callsFake((msg) => {\n errorLogs.push(msg);\n });\n });\n\n // Mock failing API\n cy.intercept('POST', '**/api/orders', {\n statusCode: 500,\n body: { error: 'Payment failed' },\n });\n\n // Act\n cy.visit('/checkout');\n cy.get('[data-cy=\"place-order\"]').click();\n\n // Assert: Error logged\n cy.wrap(errorLogs).should('have.length.greaterThan', 0);\n\n // Assert: Context included\n cy.wrap(errorLogs[0]).should('include', '/api/orders');\n\n // Assert: Secrets redacted\n cy.wrap(JSON.stringify(errorLogs)).should('not.contain', 'password');\n cy.wrap(JSON.stringify(errorLogs)).should('not.contain', 'creditCard');\n });\n});\n```\n\n**Error logger utility with redaction**:\n\n```typescript\n// src/utils/error-logger.ts\ntype ErrorContext = {\n endpoint?: string;\n method?: string;\n statusCode?: number;\n userId?: string;\n sessionId?: string;\n requestPayload?: any;\n};\n\nconst SENSITIVE_KEYS = ['password', 'token', 'creditCard', 'ssn', 'apiKey'];\n\n/**\n * Redact sensitive data from objects\n */\nfunction redactSensitiveData(obj: any): any {\n if (typeof obj !== 'object' || obj === null) return obj;\n\n const redacted = { ...obj };\n\n for (const key of Object.keys(redacted)) {\n if (SENSITIVE_KEYS.some((sensitive) => key.toLowerCase().includes(sensitive))) {\n redacted[key] = '[REDACTED]';\n } else if (typeof redacted[key] === 'object') {\n redacted[key] = redactSensitiveData(redacted[key]);\n }\n }\n\n return redacted;\n}\n\n/**\n * Log error with context (Sentry integration)\n */\nexport function logError(error: Error, context?: ErrorContext) {\n const safeContext = context ? redactSensitiveData(context) : {};\n\n const errorLog = {\n level: 'error' as const,\n message: error.message,\n stack: error.stack,\n context: safeContext,\n timestamp: new Date().toISOString(),\n };\n\n // Console (development)\n console.error(JSON.stringify(errorLog));\n\n // Sentry (production)\n if (typeof window !== 'undefined' && (window as any).Sentry) {\n (window as any).Sentry.captureException(error, {\n contexts: { custom: safeContext },\n });\n }\n}\n```\n\n**Key Points**:\n\n- **Context-rich logging**: Endpoint, method, status, user ID\n- **Secret redaction**: Passwords, tokens, PII removed before logging\n- **Sentry integration**: Production monitoring with breadcrumbs\n- **Structured logs**: JSON format for easy parsing\n- **Test validation**: Assert logs contain context but not secrets\n\n---\n\n### Example 4: Graceful Degradation Tests (Fallback Behavior)\n\n**Context**: Validate application continues functioning when services are unavailable.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/graceful-degradation.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Graceful Degradation Pattern\n * - Simulate service unavailability\n * - Validate fallback behavior\n * - Ensure user experience degrades gracefully\n * - Verify telemetry captures degradation events\n */\n\ntest.describe('Service Unavailability', () => {\n test('should display cached data when API is down', async ({ page }) => {\n // Arrange: Seed localStorage with cached data\n await page.addInitScript(() => {\n localStorage.setItem(\n 'products_cache',\n JSON.stringify({\n data: [\n { id: 1, name: 'Cached Product 1' },\n { id: 2, name: 'Cached Product 2' },\n ],\n timestamp: Date.now(),\n }),\n );\n });\n\n // Mock API unavailable\n await page.route(\n '**/api/products',\n (route) => route.abort('connectionrefused'), // Simulate server down\n );\n\n // Act\n await page.goto('/products');\n\n // Assert: Cached data displayed\n await expect(page.getByTestId('product-list')).toBeVisible();\n await expect(page.getByText('Cached Product 1')).toBeVisible();\n\n // Assert: Stale data warning shown\n await expect(page.getByTestId('cache-warning')).toBeVisible();\n await expect(page.getByTestId('cache-warning')).toContainText(/showing.*cached|offline.*mode/i);\n\n // Assert: Retry button available\n await expect(page.getByTestId('refresh-button')).toBeVisible();\n });\n\n test('should show fallback UI when analytics service fails', async ({ page }) => {\n // Mock analytics service down (non-critical)\n await page.route('**/analytics/track', (route) => route.fulfill({ status: 503, body: 'Service unavailable' }));\n\n // Act: Navigate normally\n await page.goto('/dashboard');\n\n // Assert: Page loads successfully (analytics failure doesn't block)\n await expect(page.getByTestId('dashboard-content')).toBeVisible();\n\n // Assert: Analytics error logged but not shown to user\n const consoleErrors = [];\n page.on('console', (msg) => {\n if (msg.type() === 'error') consoleErrors.push(msg.text());\n });\n\n // Trigger analytics event\n await page.getByTestId('track-action-button').click();\n\n // Analytics error logged\n expect(consoleErrors).toContainEqual(expect.stringContaining('Analytics service unavailable'));\n\n // But user doesn't see error\n await expect(page.getByTestId('error-message')).not.toBeVisible();\n });\n\n test('should fallback to local validation when API is slow', async ({ page }) => {\n // Mock slow API (> 5 seconds)\n await page.route('**/api/validate-email', async (route) => {\n await new Promise((resolve) => setTimeout(resolve, 6000)); // 6 second delay\n route.fulfill({\n status: 200,\n body: JSON.stringify({ valid: true }),\n });\n });\n\n // Act: Fill form\n await page.goto('/signup');\n await page.getByTestId('email-input').fill('[email protected]');\n await page.getByTestId('email-input').blur();\n\n // Assert: Client-side validation triggers immediately (doesn't wait for API)\n await expect(page.getByTestId('email-valid-icon')).toBeVisible({ timeout: 1000 });\n\n // Assert: Eventually API validates too (but doesn't block UX)\n await expect(page.getByTestId('email-validated-badge')).toBeVisible({ timeout: 7000 });\n });\n\n test('should maintain functionality with third-party script failure', async ({ page }) => {\n // Block third-party scripts (Google Analytics, Intercom, etc.)\n await page.route('**/*.google-analytics.com/**', (route) => route.abort());\n await page.route('**/*.intercom.io/**', (route) => route.abort());\n\n // Act\n await page.goto('/');\n\n // Assert: App works without third-party scripts\n await expect(page.getByTestId('main-content')).toBeVisible();\n await expect(page.getByTestId('nav-menu')).toBeVisible();\n\n // Assert: Core functionality intact\n await page.getByTestId('nav-products').click();\n await expect(page).toHaveURL(/.*\\/products/);\n });\n});\n```\n\n**Key Points**:\n\n- **Cached fallbacks**: Display stale data when API unavailable\n- **Non-critical degradation**: Analytics failures don't block app\n- **Client-side fallbacks**: Local validation when API slow\n- **Third-party resilience**: App works without external scripts\n- **User transparency**: Stale data warnings displayed\n\n---\n\n## Error Handling Testing Checklist\n\nBefore shipping error handling code, verify:\n\n- [ ] **Scoped exception handling**: Only ignore documented errors (NetworkError, specific codes)\n- [ ] **Rethrow unexpected**: Unknown errors fail tests (catch regressions)\n- [ ] **Error UI tested**: User sees error messages for all error states\n- [ ] **Retry logic validated**: Sequential failures test backoff and max attempts\n- [ ] **Telemetry verified**: Errors logged with context (endpoint, status, user)\n- [ ] **Secret redaction**: Logs don't contain passwords, tokens, PII\n- [ ] **Graceful degradation**: Critical services down, app shows fallback UI\n- [ ] **Non-critical failures**: Analytics/tracking failures don't block app\n\n## Integration Points\n\n- Used in workflows: `*automate` (error handling test generation), `*test-review` (error pattern detection)\n- Related fragments: `network-first.md`, `test-quality.md`, `contract-testing.md`\n- Monitoring tools: Sentry, Datadog, LogRocket\n\n_Source: Murat error-handling patterns, Pact resilience guidance, enterprise production error handling._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":22960,"content_sha256":"c941031c7535eb111d0afe53385b68b9f12d1478dea96c44c2b66d0267b8f1ce"},{"filename":"knowledge/evidence-integrity.md","content":"# Evidence Integrity\n\n## Principle\n\nA suite lies in two ways. A test that **cannot fail** reports coverage it does not have, and a diagnostic that **could not measure** reports a verdict it did not earn. Both produce green that means nothing, and the second one is worse, because a false negative sends the investigation somewhere wrong and every conclusion downstream inherits the error. Every check needs a way to fail. Every probe needs three states (pass, fail, and could-not-measure) and has to observe the thing it reports on rather than a proxy that correlates with it.\n\n## Rationale\n\n**The Problem**: Suites are scored by their result, so pressure runs one direction. An assertion that never fires, a step marked `continue-on-error`, a runner manifest that names three of eighteen files, a probe that reports \"unreachable\" when the tool it needed was missing: each converts an unknown into a green. Nothing in a CI summary distinguishes a passing check from a check that had no way to fail, and nothing distinguishes \"the device could not reach the host\" from \"the command I used to ask was not installed.\"\n\n**The Solution**: Treat falsifiability as a property to verify, the same as any other. For every check, name the input that would turn it red; if you cannot, the check is decoration. For every diagnostic, separate the measurement from the verdict, make the absence of a measurement its own reported state rather than a silent fail verdict, and name what else could have made it pass.\n\n**Why This Matters**:\n\n- Green means the behavior works instead of meaning the harness ran\n- A failing diagnostic points at the real fault instead of at whichever tool was missing\n- Root-cause work stops compounding on unearned verdicts\n- Gate decisions (PASS / CONCERNS / FAIL) rest on evidence that would have shown the defect\n\n## Pattern Examples\n\n### Example 1: The Check That Cannot Fail\n\n**Context**: Five shapes found live in one suite that reported success on every run.\n\n**Implementation**:\n\n```yaml\n# ❌ Shape 1: an optional assertion downstream of a command that is a no-op here.\n# `back` is Android and Web only; on iOS it does nothing and still reports COMPLETED.\n# The assertion then cannot fail either, because `optional: true` swallows the miss.\n# Neither half can go red, and this pair asserted a Home-screen label from a screen\n# the flow never visits.\n- back\n- assertVisible:\n text: 'Home'\n optional: true\n\n# ✅ Falsifiable: branch the platform, then assert without an escape hatch\n- runFlow:\n when:\n platform: Android\n commands:\n - back\n- runFlow:\n when:\n platform: iOS\n commands:\n - tapOn:\n id: 'nav_back_button'\n- assertVisible:\n id: 'home_screen_root'\n```\n\n```yaml\n# ❌ Shape 2: the job that runs the tests cannot turn the build red\n- name: E2E flows\n continue-on-error: true\n run: maestro test maestro/\n\n# ✅ continue-on-error belongs on artifact collection, never on the test step\n- name: E2E flows\n run: maestro test maestro/\n- name: Upload artifacts\n if: always()\n uses: actions/upload-artifact@v4\n```\n\n```yaml\n# ❌ Shape 3: the manifest names 3 of the 18 flows in the directory.\n# The suite is green because fifteen files never ran.\nflows:\n - login.yaml\n - checkout.yaml\n - profile.yaml\n\n# ✅ Include by pattern, exclude by exception, and assert the executed count\nflows:\n - '*.yaml'\n```\n\n**Shape 4** has no snippet, because the step looks correct: an assertion passes on iOS because the element is still in the hierarchy behind a presented modal, and fails on Android where the modal replaces the hierarchy. Same assertion, different meaning per platform. Any assertion whose truth depends on how a platform composes its view tree needs its own per-platform expectation, not one shared line.\n\n**Shape 5: the assertion is about something that was already true before the action.** A flow opened a deep link and then asserted that a container belonging to the screen it was already on was visible. The container predated the link, so the check held whether or not the link did anything, and on one platform it did nothing. The suite reported all flows green with this one included, and the green was stable rather than intermittent.\n\nTwo tells for it, both cheap:\n\n- **The name promises an effect the assertions never mention.** \"Widget Deep Link Hydration\" asserted the presence of a container, not that anything had hydrated. Read the flow's name as a claim and check that some assertion carries it. A name is the only place many suites record what a test was for, which makes disagreement between name and assertion a reliable smell.\n- **The result differs across environments for reasons unrelated to what it asserts.** The identical vacuous flow was green in CI and red locally, because on one API level the unresolvable link errored and on another it resolved somewhere and the open step completed. When a flow's outcome tracks an environment difference that its assertions never mention, suspect that the assertions are not what is deciding the result.\n\nThe fix generalizes past this shape: **assert the transition rather than the state.** Where only a single state is available, choose an input whose expected value differs from the application's default, so agreement with the default cannot carry the pass. Asserting \"the morning option is selected\" after an action that selects morning proves nothing in an app that starts on morning; asserting that a second action **moved** the selection, and that the first option is no longer selected, cannot pass without the action working.\n\n**Key points**:\n\n- Name the input that would turn each check red. If none exists, the check is decoration.\n- `optional: true`, `continue-on-error`, a partial manifest, and a soft assertion are four common ways a result stops being falsifiable.\n- A fifth, and the hardest to see in review: the assertion is true before the action runs. Nothing about the step looks wrong, and the green is stable.\n- **When you make a hollow check falsifiable and it goes red, the red is the finding.** It is a defect that was always there and is now visible. Reporting it as a regression you introduced is the wrong read and usually gets the fix reverted.\n\n### Example 2: Diagnostics Need a Could-Not-Measure State\n\n**Context**: A probe checking whether a device can reach a host. The tool it invokes is not installed on that device.\n\n**Implementation**:\n\n```bash\n# ❌ Two states only. A missing binary is indistinguishable from a real failure,\n# and this reported \"device cannot reach the network\" three times in one session\n# while the network was fine.\nif adb shell \"wget -q -O - http://10.0.2.2:8081/status\"; then\n echo \"PASS: device reached the dev server\"\nelse\n echo \"FAIL: device cannot reach the dev server\"\nfi\n\n# ✅ Three states. Establish the instrument before trusting the reading.\nif ! adb shell 'command -v curl >/dev/null 2>&1'; then\n echo \"COULD-NOT-MEASURE: no HTTP client on the device; reachability unknown\"\n exit 77\nfi\nif adb shell \"curl -fsS http://127.0.0.1:8081/status >/dev/null 2>&1\"; then\n echo \"PASS: device reached the dev server\"\nelse\n echo \"FAIL: device reached the network stack and the request did not succeed\"\nfi\n```\n\n**A probe fails in two directions, and both produce a confident wrong answer.** The block above is the strict failure: a missing instrument read as a false condition. The permissive failure is subtler and harder to catch, because it produces a green. The same investigation later probed reachability by having the device open a TCP connection to a port that `adb reverse` had mapped. With a reverse mapping in place the device always has a local listener on that port, so the connect succeeds whether or not anything on the host is behind it. The probe measured that a mapping existed and reported that a service was reachable.\n\n**Rule**: a probe must observe the thing it claims to observe, never a proxy that merely correlates with it. Ask what else could make this check pass. Here the fix is the same as Example 4: open a throwaway listener inside the test process and assert that the process itself accepted a socket.\n\n**A probe must also send the same request the real client sends.** A harness health-checked its development server's manifest endpoint and reported \"manifest served, HTTP 200, multipart/mixed\" through five consecutive red runs. It omitted one request header the app under test always sends, and that header selects a different branch through the server's middleware. Both results were correct at the same time: the endpoint the probe asked for was healthy, and the endpoint the app asked for was failing. Copy the client's method, headers, and body shape into the probe, or derive the probe from the client's own code path, and log which request was actually sent so the next reader can check the correspondence instead of assuming it.\n\n**Key points**:\n\n- A non-zero exit means \"the command failed,\" which is not the same claim as \"the condition is false\"\n- A zero exit means \"the command succeeded,\" which is not the same claim as \"the condition is true\"\n- Reserve a distinct exit code and a distinct log word for could-not-measure so it never reads as a fail\n- Apply this to every derived verdict, including the ones a harness prints as a convenience line. A convenience line is quoted later as evidence.\n\n### Example 3: Verify the Property Exists, Then Verify It Behaves\n\n**Context**: Two separate failures, one about existence and one about behavior.\n\n- **Existence**: a config key was invented from a plausible name and committed. The runner rejected the file on a parse error and 18 flows never executed. Nothing in the suite name suggested the cause.\n- **Behavior**: a comment claimed a command was \"a left-edge swipe on iOS.\" The implementation is an empty method that reports success. The comment propagated into other files and into a second session's reasoning before anyone opened the source.\n\n**Rule**: before using a framework property, confirm it exists in the version you pin, from the docs or the shipped artifact. Before writing a comment that asserts **why** something works, confirm the mechanism from the docs or the source. A comment stating a mechanism is a claim with the same evidentiary standing as an assertion, and it is more dangerous, because nothing tests it.\n\nCorollary for reviewers: \"X is not supported / is platform-specific / only works on Y\" needs a citation. One session asserted a flag was GNU-only when the platform's own manual documents it.\n\n### Example 4: Take the Verdict on the Side That Can Prove It\n\n**Context**: A host-side reachability check used to argue that a device could reach a service.\n\nThe host and the device are different network namespaces. A host that resolves and connects proves the host's route and nothing about the guest's. Move the assertion to the side whose route is in question: open a throwaway listener on the host, have the device connect to it, and let the verdict be \"this process observed the socket.\" Structure every environment claim so the proving party is the one that emits the result.\n\n### Example 5: State the Environment Asymmetry Before Arguing From Local to CI\n\n**Context**: A local pass used as evidence about a CI failure.\n\nWrite the asymmetry down whenever a local result enters a CI argument:\n\n| Axis | Local | CI |\n| --------------- | ------------------------------------------ | ---------------------------------- |\n| OS / arch | macOS, arm64 | Linux, x86_64 |\n| Platform ver. | newest API level | two levels older |\n| Image variant | vendor image with store services signed in | plain AOSP-style image, no account |\n| Acceleration | native hypervisor | KVM, may be unavailable |\n| Provisioning | long-lived machine | fresh runner every job |\n| Screen geometry | 914dp tall (1080x2400 at 420dpi) | 807dp tall (1080x2220 at 440dpi) |\n| Credentials | a developer session already on disk | whatever the secret store supplies |\n\nA local pass proves the application path. It proves nothing about acceleration, snapshot restore, `PATH` handling, or an image variant the local machine never runs. Naming the axes converts \"it works on my machine\" from an argument into a scoped fact.\n\nTwo of those axes are worth calling out because they read as trivia and are not:\n\n- **Screen geometry decides what is on screen, and \"visible\" usually means on screen.** The row above is a real pair: a 12% shorter viewport pushed content below the fold and failed four unrelated UI assertions on the runner while every developer machine stayed green. Compare the density-independent height, not the pixel resolution; the two profiles in that row share `1080x` and are different screens.\n- **A developer machine accumulates credentials that a fresh runner has never had.** A cached session file or a fetched certificate sitting in a home directory makes a whole code path invisible locally. When a failure exists only in CI and nothing about the code explains it, ask what the local machine has lying around that the runner does not.\n\n### Example 6: Resolve Environment-Dependent Values Before Anything Derives From Them\n\n**Context**: A harness that probed which host address the device could reach, and ran the probe after that address had already been baked into the built artifact.\n\nThe probe would have reported the right answer and changed nothing, and the app would have loaded over one route while its API calls went over another. Order of operations is part of correctness in a harness: resolve every environment-dependent value first, then derive. If a value is discovered after its consumers are built, the discovery is telemetry rather than configuration.\n\n### Example 7: Verifying the Act Is Not Verifying the Outcome\n\n**Context**: A setting written to a running device, read back, and reported as being in effect.\n\nA harness set `hide_error_dialogs=1` on an emulator, read the value back, confirmed it matched, and logged \"system crash dialogs suppressed\". The write had happened. The dialog appeared anyway: that setting is latched into the framework at boot and on configuration change, so writing it to an already-running device does not necessarily take effect. Reading a value back proves the write, and the write was never the claim.\n\n**Rule**: name the observable the claim is about, and check that one. For a suppressed dialog it is the absence of the dialog in the view hierarchy, not the presence of the flag in the settings store. This is the same substitution as a proxy probe, moved one step earlier: the act stands in for the outcome instead of a correlate standing in for the condition. Configuration that a platform reads once, at a moment you do not control, is where it hides.\n\n### Example 8: Record What a Change Did, Not What It Was For\n\n**Context**: A fix introduced for one failure and kept after it turned out not to fix it.\n\nOne investigation removed a live feature-flag service from an end-to-end run, so a per-user flag that had been evaluated remotely fell back to a seeded database row. **It did not fix the flow it was introduced for.** It was kept anyway, because removing a remote dependency from an E2E run is correct on its own terms, and the write-up says all three things: what it was for, that it did not do that, and why it stayed.\n\nA change described by its intent after its effect is known is a landmine for the next investigation, because the next reader takes the commit message as evidence that the cause was found and stops looking. State the outcome separately from the intent. \"Kept for a different reason, and labelled as such\" costs one sentence and saves someone a re-derivation.\n\n### Example 9: Order Hypotheses by the Cost of the Measurement That Kills Them\n\n**Context**: A UI assertion failing on \"not visible\". Three plausible mechanisms, all wrong.\n\nOne investigation built three separate explanations for the same failure, each with a real code path behind it: a remote flag service answering false, an unresolved dynamic import leaving state null, and a stuck initialization call. Each was written up with its reasoning. Two measurements ended all three at once: a device log line showing the module had loaded, and a direct API query showing the flag was true. The actual cause was that the element sat below the fold, which one lookup in the captured view hierarchy would have shown at the start.\n\n**Rule**: before elaborating a mechanism, list the measurements available and what each would eliminate, then take the cheapest one that kills a whole class of hypothesis. A plausible mechanism is not evidence, and producing more of them feels like progress while the discriminating observation goes untaken. The tell is a session holding several explanations and no new measurements.\n\nThe corollary for a suite: that cheap discriminating measurement should already be sitting in the artifacts. This is why capturing state at failure earns its storage cost, and why \"the artifacts could not tell us\" is a finding about the harness rather than an inconvenience.\n\n## Anti-Patterns\n\n| Anti-pattern | Why it fails | Fix |\n| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |\n| Assertion with a soft or optional modifier as default | Cannot go red; reports coverage that does not exist | Reserve softness for genuinely optional UI, and assert the outcome hard |\n| Assertion that was already true before the action | Cannot distinguish the action working from the action doing nothing | Assert the transition, or an input whose expected state differs from the default |\n| Flow name promising an effect no assertion mentions | The name records the intent and nothing checks it | Read the name as a claim; make some assertion carry it |\n| Outcome tracking an environment difference the assertions never mention | Something other than the assertions is deciding the result | Suspect a vacuous check before investigating the environment |\n| A surface declared untestable without a test of the claim | Coverage is dropped on an assumption, and the assumption is often wrong | Try it; then name precisely which part is out of reach and why |\n| `continue-on-error` on the test step | The suite cannot fail the build | Put it on artifact collection only; use `if: always()` for uploads |\n| Runner manifest listing a subset of the suite | Files silently never run; the count is the only clue | Include by pattern; assert the executed count against the file count |\n| Missing tool reported as a failed condition | Sends the investigation at the wrong subsystem | Three-state probes; distinct exit code for could-not-measure |\n| Probe observing a proxy that correlates with the target | Passes for a reason unrelated to the claim, and a green is never re-examined | Ask what else could make this pass; observe the thing itself |\n| Probe sending a different request than the client | Takes a different branch through the server: healthy probe, failing app, both correct | Copy the client's method, headers, and body into the probe, and log what was sent |\n| Written setting read back and reported as effect | Proves the write; some configuration latches at boot and never applies live | Assert the observable the claim is about, not the act that was supposed to produce it |\n| Change described by its intent once its effect is known | Reads as a found root cause and stops the next investigation looking | Record what it actually did, and why it was kept |\n| Mechanisms elaborated while the cheap measurement goes untaken | Plausibility feels like progress; several explanations, no new evidence | Rank hypotheses by the cost of the observation that would kill them |\n| Verdict emitted by the side that cannot observe it | Proves the wrong namespace | Move the assertion to the party whose route or state is in question |\n| Comment asserting a mechanism with no source read | Propagates into other files and into other people's reasoning | Cite the doc or source line, or omit the mechanism |\n| Local result used as a CI argument, asymmetry unstated | Hides the axes that actually differ | Tabulate the differing axes with the claim |\n| Environment probe running after its consumers | The answer arrives too late to configure anything | Resolve environment-dependent values first, then derive |\n| Reverting a newly-red check as a regression | Restores the hollow green and loses the finding | Treat the red as the pre-existing defect it exposed |\n\n## Evidence Integrity Checklist\n\n- [ ] **Every check is falsifiable**: for each assertion, the input that turns it red is nameable\n- [ ] **No soft assertion by default**: optional modifiers only on genuinely optional UI, with a comment\n- [ ] **The assertion carrying the outcome post-dates its action**: a precondition assertion is allowed when it is labelled as one, and nothing already true before the step is presented as proof of it\n- [ ] **Names reconciled with assertions**: what a test is called is carried by something that can fail\n- [ ] **Untestable claims tested**: a surface is dropped from coverage only after the claim itself has been checked\n- [ ] **No `continue-on-error` on a test step**: only on artifact collection\n- [ ] **Executed count reconciled**: the number of tests that ran matches the number of test files discovered\n- [ ] **Platform-divergent assertions split**: no single assertion whose meaning depends on how a platform composes its view tree\n- [ ] **Probes are three-state**: pass, fail, and could-not-measure, with distinct exit codes\n- [ ] **Probes observe their own claim**: no proxy that merely correlates with the condition being reported\n- [ ] **Probes issue the client's request**: same method, headers, and body shape as the code path they stand in for\n- [ ] **Outcomes verified, not acts**: a write, a set flag, or a dispatched action is not evidence that behavior changed\n- [ ] **Instruments verified before readings**: the probe confirms its tool exists before interpreting its result\n- [ ] **Framework properties verified**: every key, flag, and command confirmed against the pinned version's docs or artifact\n- [ ] **Mechanism comments cited**: any comment claiming why something works names its source\n- [ ] **Verdicts emitted by the proving party**: cross-boundary claims asserted on the side that can observe them\n- [ ] **Environment asymmetry stated**: local-versus-CI arguments list the differing axes\n- [ ] **Resolution precedes derivation**: environment-dependent values resolved before any consumer is built\n- [ ] **Effects recorded separately from intent**: a change kept for a reason other than the one it was made for says so\n- [ ] **Cheapest discriminating measurement taken first**: no mechanism elaborated while an available observation would eliminate a class of hypothesis\n\n## Integration Points\n\n- **Used in workflows**: `*test-review` (the CRITICAL rows exist to catch checks that cannot fail), `*ci` (gate wiring and artifact steps), `*nfr-assess` (a measurement that could not be taken is CONCERNS, never PASS), `*trace` (coverage claims), `*automate` and `*atdd` (generated checks must be falsifiable)\n- **Related fragments**: `confidence-gate.md` (do not fabricate the artifact in the first place), `test-quality.md` (determinism and isolation), `risk-governance.md` (what a gate decision may rest on), `mobile-ci-device-lab.md` (where these failures concentrate on mobile)\n- **Tools**: any CI summary, the runner's own executed-test count, exit codes\n\n_Source: TEA quality-gate standards; hollow-green and false-negative diagnostic patterns observed in a live mobile CI investigation._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":25547,"content_sha256":"864c16573a53b459b8a26c8cae26fc131d9de5056522157a65cf823348a46920"},{"filename":"knowledge/feature-flags.md","content":"# Feature Flag Governance\n\n## Principle\n\nFeature flags enable controlled rollouts and A/B testing, but require disciplined testing governance. Centralize flag definitions in a frozen enum, test both enabled and disabled states, clean up targeting after each spec, and maintain a comprehensive flag lifecycle checklist. For LaunchDarkly-style systems, script API helpers to seed variations programmatically rather than manual UI mutations.\n\n## Rationale\n\nPoorly managed feature flags become technical debt: untested variations ship broken code, forgotten flags clutter the codebase, and shared environments become unstable from leftover targeting rules. Structured governance ensures flags are testable, traceable, temporary, and safe. Testing both states prevents surprises when flags flip in production.\n\n## Pattern Examples\n\n### Example 1: Feature Flag Enum Pattern with Type Safety\n\n**Context**: Centralized flag management with TypeScript type safety and runtime validation.\n\n**Implementation**:\n\n```typescript\n// src/utils/feature-flags.ts\n/**\n * Centralized feature flag definitions\n * - Object.freeze prevents runtime modifications\n * - TypeScript ensures compile-time type safety\n * - Single source of truth for all flag keys\n */\nexport const FLAGS = Object.freeze({\n // User-facing features\n NEW_CHECKOUT_FLOW: 'new-checkout-flow',\n DARK_MODE: 'dark-mode',\n ENHANCED_SEARCH: 'enhanced-search',\n\n // Experiments\n PRICING_EXPERIMENT_A: 'pricing-experiment-a',\n HOMEPAGE_VARIANT_B: 'homepage-variant-b',\n\n // Infrastructure\n USE_NEW_API_ENDPOINT: 'use-new-api-endpoint',\n ENABLE_ANALYTICS_V2: 'enable-analytics-v2',\n\n // Killswitches (emergency disables)\n DISABLE_PAYMENT_PROCESSING: 'disable-payment-processing',\n DISABLE_EMAIL_NOTIFICATIONS: 'disable-email-notifications',\n} as const);\n\n/**\n * Type-safe flag keys\n * Prevents typos and ensures autocomplete in IDEs\n */\nexport type FlagKey = (typeof FLAGS)[keyof typeof FLAGS];\n\n/**\n * Flag metadata for governance\n */\ntype FlagMetadata = {\n key: FlagKey;\n name: string;\n owner: string;\n createdDate: string;\n expiryDate?: string;\n defaultState: boolean;\n requiresCleanup: boolean;\n dependencies?: FlagKey[];\n telemetryEvents?: string[];\n};\n\n/**\n * Flag registry with governance metadata\n * Used for flag lifecycle tracking and cleanup alerts\n */\nexport const FLAG_REGISTRY: Record\u003cFlagKey, FlagMetadata> = {\n [FLAGS.NEW_CHECKOUT_FLOW]: {\n key: FLAGS.NEW_CHECKOUT_FLOW,\n name: 'New Checkout Flow',\n owner: 'payments-team',\n createdDate: '2025-01-15',\n expiryDate: '2025-03-15',\n defaultState: false,\n requiresCleanup: true,\n dependencies: [FLAGS.USE_NEW_API_ENDPOINT],\n telemetryEvents: ['checkout_started', 'checkout_completed'],\n },\n [FLAGS.DARK_MODE]: {\n key: FLAGS.DARK_MODE,\n name: 'Dark Mode UI',\n owner: 'frontend-team',\n createdDate: '2025-01-10',\n defaultState: false,\n requiresCleanup: false, // Permanent feature toggle\n },\n // ... rest of registry\n};\n\n/**\n * Validate flag exists in registry\n * Throws at runtime if flag is unregistered\n */\nexport function validateFlag(flag: string): asserts flag is FlagKey {\n if (!Object.values(FLAGS).includes(flag as FlagKey)) {\n throw new Error(`Unregistered feature flag: ${flag}`);\n }\n}\n\n/**\n * Check if flag is expired (needs removal)\n */\nexport function isFlagExpired(flag: FlagKey): boolean {\n const metadata = FLAG_REGISTRY[flag];\n if (!metadata.expiryDate) return false;\n\n const expiry = new Date(metadata.expiryDate);\n return Date.now() > expiry.getTime();\n}\n\n/**\n * Get all expired flags requiring cleanup\n */\nexport function getExpiredFlags(): FlagMetadata[] {\n return Object.values(FLAG_REGISTRY).filter((meta) => isFlagExpired(meta.key));\n}\n```\n\n**Usage in application code**:\n\n```typescript\n// components/Checkout.tsx\nimport { FLAGS } from '@/utils/feature-flags';\nimport { useFeatureFlag } from '@/hooks/useFeatureFlag';\n\nexport function Checkout() {\n const isNewFlow = useFeatureFlag(FLAGS.NEW_CHECKOUT_FLOW);\n\n return isNewFlow ? \u003cNewCheckoutFlow /> : \u003cLegacyCheckoutFlow />;\n}\n```\n\n**Key Points**:\n\n- **Type safety**: TypeScript catches typos at compile time\n- **Runtime validation**: validateFlag ensures only registered flags used\n- **Metadata tracking**: Owner, dates, dependencies documented\n- **Expiry alerts**: Automated detection of stale flags\n- **Single source of truth**: All flags defined in one place\n\n---\n\n### Example 2: Feature Flag Testing Pattern (Both States)\n\n**Context**: Comprehensive testing of feature flag variations with proper cleanup.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/checkout-feature-flag.spec.ts\nimport { test, expect } from '@playwright/test';\nimport { FLAGS } from '@/utils/feature-flags';\n\n/**\n * Feature Flag Testing Strategy:\n * 1. Test BOTH enabled and disabled states\n * 2. Clean up targeting after each test\n * 3. Use dedicated test users (not production data)\n * 4. Verify telemetry events fire correctly\n */\n\ntest.describe('Checkout Flow - Feature Flag Variations', () => {\n let testUserId: string;\n\n test.beforeEach(async () => {\n // Generate unique test user ID\n testUserId = `test-user-${Date.now()}`;\n });\n\n test.afterEach(async ({ request }) => {\n // CRITICAL: Clean up flag targeting to prevent shared env pollution\n await request.post('/api/feature-flags/cleanup', {\n data: {\n flagKey: FLAGS.NEW_CHECKOUT_FLOW,\n userId: testUserId,\n },\n });\n });\n\n test('should use NEW checkout flow when flag is ENABLED', async ({ page, request }) => {\n // Arrange: Enable flag for test user\n await request.post('/api/feature-flags/target', {\n data: {\n flagKey: FLAGS.NEW_CHECKOUT_FLOW,\n userId: testUserId,\n variation: true, // ENABLED\n },\n });\n\n // Act: Navigate as targeted user\n await page.goto('/checkout', {\n extraHTTPHeaders: {\n 'X-Test-User-ID': testUserId,\n },\n });\n\n // Assert: New flow UI elements visible\n await expect(page.getByTestId('checkout-v2-container')).toBeVisible();\n await expect(page.getByTestId('express-payment-options')).toBeVisible();\n await expect(page.getByTestId('saved-addresses-dropdown')).toBeVisible();\n\n // Assert: Legacy flow NOT visible\n await expect(page.getByTestId('checkout-v1-container')).not.toBeVisible();\n\n // Assert: Telemetry event fired\n const analyticsEvents = await page.evaluate(() => (window as any).__ANALYTICS_EVENTS__ || []);\n expect(analyticsEvents).toContainEqual(\n expect.objectContaining({\n event: 'checkout_started',\n properties: expect.objectContaining({\n variant: 'new_flow',\n }),\n }),\n );\n });\n\n test('should use LEGACY checkout flow when flag is DISABLED', async ({ page, request }) => {\n // Arrange: Disable flag for test user (or don't target at all)\n await request.post('/api/feature-flags/target', {\n data: {\n flagKey: FLAGS.NEW_CHECKOUT_FLOW,\n userId: testUserId,\n variation: false, // DISABLED\n },\n });\n\n // Act: Navigate as targeted user\n await page.goto('/checkout', {\n extraHTTPHeaders: {\n 'X-Test-User-ID': testUserId,\n },\n });\n\n // Assert: Legacy flow UI elements visible\n await expect(page.getByTestId('checkout-v1-container')).toBeVisible();\n await expect(page.getByTestId('legacy-payment-form')).toBeVisible();\n\n // Assert: New flow NOT visible\n await expect(page.getByTestId('checkout-v2-container')).not.toBeVisible();\n await expect(page.getByTestId('express-payment-options')).not.toBeVisible();\n\n // Assert: Telemetry event fired with correct variant\n const analyticsEvents = await page.evaluate(() => (window as any).__ANALYTICS_EVENTS__ || []);\n expect(analyticsEvents).toContainEqual(\n expect.objectContaining({\n event: 'checkout_started',\n properties: expect.objectContaining({\n variant: 'legacy_flow',\n }),\n }),\n );\n });\n\n test('should handle flag evaluation errors gracefully', async ({ page, request }) => {\n // Arrange: Simulate flag service unavailable\n await page.route('**/api/feature-flags/evaluate', (route) => route.fulfill({ status: 500, body: 'Service Unavailable' }));\n\n // Act: Navigate (should fallback to default state)\n await page.goto('/checkout', {\n extraHTTPHeaders: {\n 'X-Test-User-ID': testUserId,\n },\n });\n\n // Assert: Fallback to safe default (legacy flow)\n await expect(page.getByTestId('checkout-v1-container')).toBeVisible();\n\n // Assert: Error logged but no user-facing error\n const consoleErrors = [];\n page.on('console', (msg) => {\n if (msg.type() === 'error') consoleErrors.push(msg.text());\n });\n expect(consoleErrors).toContain(expect.stringContaining('Feature flag evaluation failed'));\n });\n});\n```\n\n**Cypress equivalent**:\n\n```javascript\n// cypress/e2e/checkout-feature-flag.cy.ts\nimport { FLAGS } from '@/utils/feature-flags';\n\ndescribe('Checkout Flow - Feature Flag Variations', () => {\n let testUserId;\n\n beforeEach(() => {\n testUserId = `test-user-${Date.now()}`;\n });\n\n afterEach(() => {\n // Clean up targeting\n cy.task('removeFeatureFlagTarget', {\n flagKey: FLAGS.NEW_CHECKOUT_FLOW,\n userId: testUserId,\n });\n });\n\n it('should use NEW checkout flow when flag is ENABLED', () => {\n // Arrange: Enable flag via Cypress task\n cy.task('setFeatureFlagVariation', {\n flagKey: FLAGS.NEW_CHECKOUT_FLOW,\n userId: testUserId,\n variation: true,\n });\n\n // Act\n cy.visit('/checkout', {\n headers: { 'X-Test-User-ID': testUserId },\n });\n\n // Assert\n cy.get('[data-testid=\"checkout-v2-container\"]').should('be.visible');\n cy.get('[data-testid=\"checkout-v1-container\"]').should('not.exist');\n });\n\n it('should use LEGACY checkout flow when flag is DISABLED', () => {\n // Arrange: Disable flag\n cy.task('setFeatureFlagVariation', {\n flagKey: FLAGS.NEW_CHECKOUT_FLOW,\n userId: testUserId,\n variation: false,\n });\n\n // Act\n cy.visit('/checkout', {\n headers: { 'X-Test-User-ID': testUserId },\n });\n\n // Assert\n cy.get('[data-testid=\"checkout-v1-container\"]').should('be.visible');\n cy.get('[data-testid=\"checkout-v2-container\"]').should('not.exist');\n });\n});\n```\n\n**Key Points**:\n\n- **Test both states**: Enabled AND disabled variations\n- **Automatic cleanup**: afterEach removes targeting (prevent pollution)\n- **Unique test users**: Avoid conflicts with real user data\n- **Telemetry validation**: Verify analytics events fire correctly\n- **Graceful degradation**: Test fallback behavior on errors\n\n---\n\n### Example 3: Feature Flag Targeting Helper Pattern\n\n**Context**: Reusable helpers for programmatic flag control via LaunchDarkly/Split.io API.\n\n**Implementation**:\n\n```typescript\n// tests/support/feature-flag-helpers.ts\nimport { request as playwrightRequest } from '@playwright/test';\nimport { FLAGS, FlagKey } from '@/utils/feature-flags';\n\n/**\n * LaunchDarkly API client configuration\n * Use test project SDK key (NOT production)\n */\nconst LD_SDK_KEY = process.env.LD_SDK_KEY_TEST;\nconst LD_API_BASE = 'https://app.launchdarkly.com/api/v2';\n\ntype FlagVariation = boolean | string | number | object;\n\n/**\n * Set flag variation for specific user\n * Uses LaunchDarkly API to create user target\n */\nexport async function setFlagForUser(flagKey: FlagKey, userId: string, variation: FlagVariation): Promise\u003cvoid> {\n const response = await playwrightRequest.newContext().then((ctx) =>\n ctx.post(`${LD_API_BASE}/flags/${flagKey}/targeting`, {\n headers: {\n Authorization: LD_SDK_KEY!,\n 'Content-Type': 'application/json',\n },\n data: {\n targets: [\n {\n values: [userId],\n variation: variation ? 1 : 0, // 0 = off, 1 = on\n },\n ],\n },\n }),\n );\n\n if (!response.ok()) {\n throw new Error(`Failed to set flag ${flagKey} for user ${userId}: ${response.status()}`);\n }\n}\n\n/**\n * Remove user from flag targeting\n * CRITICAL for test cleanup\n */\nexport async function removeFlagTarget(flagKey: FlagKey, userId: string): Promise\u003cvoid> {\n const response = await playwrightRequest.newContext().then((ctx) =>\n ctx.delete(`${LD_API_BASE}/flags/${flagKey}/targeting/users/${userId}`, {\n headers: {\n Authorization: LD_SDK_KEY!,\n },\n }),\n );\n\n if (!response.ok() && response.status() !== 404) {\n // 404 is acceptable (user wasn't targeted)\n throw new Error(`Failed to remove flag ${flagKey} target for user ${userId}: ${response.status()}`);\n }\n}\n\n/**\n * Percentage rollout helper\n * Enable flag for N% of users\n */\nexport async function setFlagRolloutPercentage(flagKey: FlagKey, percentage: number): Promise\u003cvoid> {\n if (percentage \u003c 0 || percentage > 100) {\n throw new Error('Percentage must be between 0 and 100');\n }\n\n const response = await playwrightRequest.newContext().then((ctx) =>\n ctx.patch(`${LD_API_BASE}/flags/${flagKey}`, {\n headers: {\n Authorization: LD_SDK_KEY!,\n 'Content-Type': 'application/json',\n },\n data: {\n rollout: {\n variations: [\n { variation: 0, weight: 100 - percentage }, // off\n { variation: 1, weight: percentage }, // on\n ],\n },\n },\n }),\n );\n\n if (!response.ok()) {\n throw new Error(`Failed to set rollout for flag ${flagKey}: ${response.status()}`);\n }\n}\n\n/**\n * Enable flag globally (100% rollout)\n */\nexport async function enableFlagGlobally(flagKey: FlagKey): Promise\u003cvoid> {\n await setFlagRolloutPercentage(flagKey, 100);\n}\n\n/**\n * Disable flag globally (0% rollout)\n */\nexport async function disableFlagGlobally(flagKey: FlagKey): Promise\u003cvoid> {\n await setFlagRolloutPercentage(flagKey, 0);\n}\n\n/**\n * Stub feature flags in local/test environments\n * Bypasses LaunchDarkly entirely\n */\nexport function stubFeatureFlags(flags: Record\u003cFlagKey, FlagVariation>): void {\n // Set flags in localStorage or inject into window\n if (typeof window !== 'undefined') {\n (window as any).__STUBBED_FLAGS__ = flags;\n }\n}\n```\n\n**Usage in Playwright fixture**:\n\n```typescript\n// playwright/fixtures/feature-flag-fixture.ts\nimport { test as base } from '@playwright/test';\nimport { setFlagForUser, removeFlagTarget } from '../support/feature-flag-helpers';\nimport { FlagKey } from '@/utils/feature-flags';\n\ntype FeatureFlagFixture = {\n featureFlags: {\n enable: (flag: FlagKey, userId: string) => Promise\u003cvoid>;\n disable: (flag: FlagKey, userId: string) => Promise\u003cvoid>;\n cleanup: (flag: FlagKey, userId: string) => Promise\u003cvoid>;\n };\n};\n\nexport const test = base.extend\u003cFeatureFlagFixture>({\n featureFlags: async ({}, use) => {\n const cleanupQueue: Array\u003c{ flag: FlagKey; userId: string }> = [];\n\n await use({\n enable: async (flag, userId) => {\n await setFlagForUser(flag, userId, true);\n cleanupQueue.push({ flag, userId });\n },\n disable: async (flag, userId) => {\n await setFlagForUser(flag, userId, false);\n cleanupQueue.push({ flag, userId });\n },\n cleanup: async (flag, userId) => {\n await removeFlagTarget(flag, userId);\n },\n });\n\n // Auto-cleanup after test\n for (const { flag, userId } of cleanupQueue) {\n await removeFlagTarget(flag, userId);\n }\n },\n});\n```\n\n**Key Points**:\n\n- **API-driven control**: No manual UI clicks required\n- **Auto-cleanup**: Fixture tracks and removes targeting\n- **Percentage rollouts**: Test gradual feature releases\n- **Stubbing option**: Local development without LaunchDarkly\n- **Type-safe**: FlagKey prevents typos\n\n---\n\n### Example 4: Feature Flag Lifecycle Checklist & Cleanup Strategy\n\n**Context**: Governance checklist and automated cleanup detection for stale flags.\n\n**Implementation**:\n\n```typescript\n// scripts/feature-flag-audit.ts\n/**\n * Feature Flag Lifecycle Audit Script\n * Run weekly to detect stale flags requiring cleanup\n */\n\nimport { FLAG_REGISTRY, FLAGS, getExpiredFlags, FlagKey } from '../src/utils/feature-flags';\nimport * as fs from 'fs';\nimport * as path from 'path';\n\ntype AuditResult = {\n totalFlags: number;\n expiredFlags: FlagKey[];\n missingOwners: FlagKey[];\n missingDates: FlagKey[];\n permanentFlags: FlagKey[];\n flagsNearingExpiry: FlagKey[];\n};\n\n/**\n * Audit all feature flags for governance compliance\n */\nfunction auditFeatureFlags(): AuditResult {\n const allFlags = Object.keys(FLAG_REGISTRY) as FlagKey[];\n const expiredFlags = getExpiredFlags().map((meta) => meta.key);\n\n // Flags expiring in next 30 days\n const thirtyDaysFromNow = Date.now() + 30 * 24 * 60 * 60 * 1000;\n const flagsNearingExpiry = allFlags.filter((flag) => {\n const meta = FLAG_REGISTRY[flag];\n if (!meta.expiryDate) return false;\n const expiry = new Date(meta.expiryDate).getTime();\n return expiry > Date.now() && expiry \u003c thirtyDaysFromNow;\n });\n\n // Missing metadata\n const missingOwners = allFlags.filter((flag) => !FLAG_REGISTRY[flag].owner);\n const missingDates = allFlags.filter((flag) => !FLAG_REGISTRY[flag].createdDate);\n\n // Permanent flags (no expiry, requiresCleanup = false)\n const permanentFlags = allFlags.filter((flag) => {\n const meta = FLAG_REGISTRY[flag];\n return !meta.expiryDate && !meta.requiresCleanup;\n });\n\n return {\n totalFlags: allFlags.length,\n expiredFlags,\n missingOwners,\n missingDates,\n permanentFlags,\n flagsNearingExpiry,\n };\n}\n\n/**\n * Generate markdown report\n */\nfunction generateReport(audit: AuditResult): string {\n let report = `# Feature Flag Audit Report\\n\\n`;\n report += `**Date**: ${new Date().toISOString()}\\n`;\n report += `**Total Flags**: ${audit.totalFlags}\\n\\n`;\n\n if (audit.expiredFlags.length > 0) {\n report += `## ⚠️ EXPIRED FLAGS - IMMEDIATE CLEANUP REQUIRED\\n\\n`;\n audit.expiredFlags.forEach((flag) => {\n const meta = FLAG_REGISTRY[flag];\n report += `- **${meta.name}** (\\`${flag}\\`)\\n`;\n report += ` - Owner: ${meta.owner}\\n`;\n report += ` - Expired: ${meta.expiryDate}\\n`;\n report += ` - Action: Remove flag code, update tests, deploy\\n\\n`;\n });\n }\n\n if (audit.flagsNearingExpiry.length > 0) {\n report += `## ⏰ FLAGS EXPIRING SOON (Next 30 Days)\\n\\n`;\n audit.flagsNearingExpiry.forEach((flag) => {\n const meta = FLAG_REGISTRY[flag];\n report += `- **${meta.name}** (\\`${flag}\\`)\\n`;\n report += ` - Owner: ${meta.owner}\\n`;\n report += ` - Expires: ${meta.expiryDate}\\n`;\n report += ` - Action: Plan cleanup or extend expiry\\n\\n`;\n });\n }\n\n if (audit.permanentFlags.length > 0) {\n report += `## 🔄 PERMANENT FLAGS (No Expiry)\\n\\n`;\n audit.permanentFlags.forEach((flag) => {\n const meta = FLAG_REGISTRY[flag];\n report += `- **${meta.name}** (\\`${flag}\\`) - Owner: ${meta.owner}\\n`;\n });\n report += `\\n`;\n }\n\n if (audit.missingOwners.length > 0 || audit.missingDates.length > 0) {\n report += `## ❌ GOVERNANCE ISSUES\\n\\n`;\n if (audit.missingOwners.length > 0) {\n report += `**Missing Owners**: ${audit.missingOwners.join(', ')}\\n`;\n }\n if (audit.missingDates.length > 0) {\n report += `**Missing Created Dates**: ${audit.missingDates.join(', ')}\\n`;\n }\n report += `\\n`;\n }\n\n return report;\n}\n\n/**\n * Feature Flag Lifecycle Checklist\n */\nconst FLAG_LIFECYCLE_CHECKLIST = `\n# Feature Flag Lifecycle Checklist\n\n## Before Creating a New Flag\n\n- [ ] **Name**: Follow naming convention (kebab-case, descriptive)\n- [ ] **Owner**: Assign team/individual responsible\n- [ ] **Default State**: Determine safe default (usually false)\n- [ ] **Expiry Date**: Set removal date (30-90 days typical)\n- [ ] **Dependencies**: Document related flags\n- [ ] **Telemetry**: Plan analytics events to track\n- [ ] **Rollback Plan**: Define how to disable quickly\n\n## During Development\n\n- [ ] **Code Paths**: Both enabled/disabled states implemented\n- [ ] **Tests**: Both variations tested in CI\n- [ ] **Documentation**: Flag purpose documented in code/PR\n- [ ] **Telemetry**: Analytics events instrumented\n- [ ] **Error Handling**: Graceful degradation on flag service failure\n\n## Before Launch\n\n- [ ] **QA**: Both states tested in staging\n- [ ] **Rollout Plan**: Gradual rollout percentage defined\n- [ ] **Monitoring**: Dashboards/alerts for flag-related metrics\n- [ ] **Stakeholder Communication**: Product/design aligned\n\n## After Launch (Monitoring)\n\n- [ ] **Metrics**: Success criteria tracked\n- [ ] **Error Rates**: No increase in errors\n- [ ] **Performance**: No degradation\n- [ ] **User Feedback**: Qualitative data collected\n\n## Cleanup (Post-Launch)\n\n- [ ] **Remove Flag Code**: Delete if/else branches\n- [ ] **Update Tests**: Remove flag-specific tests\n- [ ] **Remove Targeting**: Clear all user targets\n- [ ] **Delete Flag Config**: Remove from LaunchDarkly/registry\n- [ ] **Update Documentation**: Remove references\n- [ ] **Deploy**: Ship cleanup changes\n`;\n\n// Run audit\nconst audit = auditFeatureFlags();\nconst report = generateReport(audit);\n\n// Save report\nconst outputPath = path.join(__dirname, '../feature-flag-audit-report.md');\nfs.writeFileSync(outputPath, report);\nfs.writeFileSync(path.join(__dirname, '../FEATURE-FLAG-CHECKLIST.md'), FLAG_LIFECYCLE_CHECKLIST);\n\nconsole.log(`✅ Audit complete. Report saved to: ${outputPath}`);\nconsole.log(`Total flags: ${audit.totalFlags}`);\nconsole.log(`Expired flags: ${audit.expiredFlags.length}`);\nconsole.log(`Flags expiring soon: ${audit.flagsNearingExpiry.length}`);\n\n// Exit with error if expired flags exist\nif (audit.expiredFlags.length > 0) {\n console.error(`\\n❌ EXPIRED FLAGS DETECTED - CLEANUP REQUIRED`);\n process.exit(1);\n}\n```\n\n**package.json scripts**:\n\n```json\n{\n \"scripts\": {\n \"feature-flags:audit\": \"ts-node scripts/feature-flag-audit.ts\",\n \"feature-flags:audit:ci\": \"npm run feature-flags:audit || true\"\n }\n}\n```\n\n**Key Points**:\n\n- **Automated detection**: Weekly audit catches stale flags\n- **Lifecycle checklist**: Comprehensive governance guide\n- **Expiry tracking**: Flags auto-expire after defined date\n- **CI integration**: Audit runs in pipeline, warns on expiry\n- **Ownership clarity**: Every flag has assigned owner\n\n---\n\n## Feature Flag Testing Checklist\n\nBefore merging flag-related code, verify:\n\n- [ ] **Both states tested**: Enabled AND disabled variations covered\n- [ ] **Cleanup automated**: afterEach removes targeting (no manual cleanup)\n- [ ] **Unique test data**: Test users don't collide with production\n- [ ] **Telemetry validated**: Analytics events fire for both variations\n- [ ] **Error handling**: Graceful fallback when flag service unavailable\n- [ ] **Flag metadata**: Owner, dates, dependencies documented in registry\n- [ ] **Rollback plan**: Clear steps to disable flag in production\n- [ ] **Expiry date set**: Removal date defined (or marked permanent)\n\n## Integration Points\n\n- Used in workflows: `*automate` (test generation), `*framework` (flag setup)\n- Related fragments: `test-quality.md`, `selective-testing.md`\n- Flag services: LaunchDarkly, Split.io, Unleash, custom implementations\n\n_Source: LaunchDarkly strategy blog, Murat test architecture notes, enterprise feature flag governance._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":23215,"content_sha256":"7e46668d5049933dbb5d60096dd2c3800605582dd4198523b10b4d02586c97dc"},{"filename":"knowledge/file-utils.md","content":"# File Utilities\n\n## Principle\n\nRead and validate files (CSV, XLSX, PDF, ZIP) with automatic parsing, type-safe results, and download handling. Simplify file operations in Playwright tests with built-in format support and validation helpers.\n\n## Rationale\n\nTesting file operations in Playwright requires boilerplate:\n\n- Manual download handling\n- External parsing libraries for each format\n- No validation helpers\n- Type-unsafe results\n- Repetitive path handling\n\nThe `file-utils` module provides:\n\n- **Auto-parsing**: CSV, XLSX, PDF, ZIP automatically parsed\n- **Download handling**: Single function for UI or API-triggered downloads\n- **Type-safe**: TypeScript interfaces for parsed results\n- **Validation helpers**: Row count, header checks, content validation\n- **Format support**: Multiple sheet support (XLSX), text extraction (PDF), archive extraction (ZIP)\n\n## Why Use This Instead of Vanilla Playwright?\n\n| Vanilla Playwright | File Utils |\n| ------------------------------------------- | ------------------------------------------------ |\n| ~80 lines per CSV flow (download + parse) | ~10 lines end-to-end |\n| Manual event orchestration for downloads | Encapsulated in `handleDownload()` |\n| Manual path handling and `saveAs` | Returns a ready-to-use file path |\n| Manual existence checks and error handling | Centralized in one place via utility patterns |\n| Manual CSV parsing config (headers, typing) | `readCSV()` returns `{ data, headers }` directly |\n\n## Pattern Examples\n\n### Example 1: UI-Triggered CSV Download\n\n**Context**: User clicks button, CSV downloads, validate contents.\n\n**Implementation**:\n\n```typescript\nimport { handleDownload, readCSV } from '@seontechnologies/playwright-utils/file-utils';\nimport path from 'node:path';\n\nconst DOWNLOAD_DIR = path.join(__dirname, '../downloads');\n\ntest('should download and validate CSV', async ({ page }) => {\n const downloadPath = await handleDownload({\n page,\n downloadDir: DOWNLOAD_DIR,\n trigger: () => page.getByTestId('download-button-text/csv').click(),\n });\n\n const csvResult = await readCSV({ filePath: downloadPath });\n\n // Access parsed data and headers\n const { data, headers } = csvResult.content;\n expect(headers).toEqual(['ID', 'Name', 'Email']);\n expect(data[0]).toMatchObject({\n ID: expect.any(String),\n Name: expect.any(String),\n Email: expect.any(String),\n });\n});\n```\n\n**Key Points**:\n\n- `handleDownload` waits for download, returns file path\n- `readCSV` auto-parses to `{ headers, data }`\n- Type-safe access to parsed content\n- Clean up downloads in `afterEach`\n\n### Example 2: XLSX with Multiple Sheets\n\n**Context**: Excel file with multiple sheets (e.g., Summary, Details, Errors).\n\n**Implementation**:\n\n```typescript\nimport { readXLSX } from '@seontechnologies/playwright-utils/file-utils';\n\ntest('should read multi-sheet XLSX', async () => {\n const downloadPath = await handleDownload({\n page,\n downloadDir: DOWNLOAD_DIR,\n trigger: () => page.click('[data-testid=\"export-xlsx\"]'),\n });\n\n const xlsxResult = await readXLSX({ filePath: downloadPath });\n\n // Verify worksheet structure\n expect(xlsxResult.content.worksheets.length).toBeGreaterThan(0);\n const worksheet = xlsxResult.content.worksheets[0];\n expect(worksheet).toBeDefined();\n expect(worksheet).toHaveProperty('name');\n\n // Access sheet data\n const sheetData = worksheet?.data;\n expect(Array.isArray(sheetData)).toBe(true);\n\n // Use type assertion for type safety\n const firstRow = sheetData![0] as Record\u003cstring, unknown>;\n expect(firstRow).toHaveProperty('id');\n});\n```\n\n**Key Points**:\n\n- `worksheets` array with `name` and `data` properties\n- Access sheets by name\n- Each sheet has its own headers and data\n- Type-safe sheet iteration\n\n### Example 3: PDF Text Extraction\n\n**Context**: Validate PDF report contains expected content.\n\n**Implementation**:\n\n```typescript\nimport { readPDF } from '@seontechnologies/playwright-utils/file-utils';\n\ntest('should validate PDF report', async () => {\n const downloadPath = await handleDownload({\n page,\n downloadDir: DOWNLOAD_DIR,\n trigger: () => page.getByTestId('download-button-Text-based PDF Document').click(),\n });\n\n const pdfResult = await readPDF({ filePath: downloadPath });\n\n // content is extracted text from all pages\n expect(pdfResult.pagesCount).toBe(1);\n expect(pdfResult.fileName).toContain('.pdf');\n expect(pdfResult.content).toContain('All you need is the free Adobe Acrobat Reader');\n});\n```\n\n**PDF Reader Options:**\n\n```typescript\nconst result = await readPDF({\n filePath: '/path/to/document.pdf',\n mergePages: false, // Keep pages separate (default: true)\n debug: true, // Enable debug logging\n maxPages: 10, // Limit processing to first 10 pages\n});\n```\n\n**Important Limitation - Vector-based PDFs:**\n\nText extraction may fail for PDFs that store text as vector graphics (e.g., those generated by jsPDF):\n\n```typescript\n// Vector-based PDF example (extraction fails gracefully)\nconst pdfResult = await readPDF({ filePath: downloadPath });\n\nexpect(pdfResult.pagesCount).toBe(1);\nexpect(pdfResult.info.extractionNotes).toContain('Text extraction from vector-based PDFs is not supported.');\n```\n\nSuch PDFs will have:\n\n- `textExtractionSuccess: false`\n- `isVectorBased: true`\n- Explanatory message in `extractionNotes`\n\n### Example 4: ZIP Archive Validation\n\n**Context**: Validate ZIP contains expected files and extract specific file.\n\n**Implementation**:\n\n```typescript\nimport { readZIP } from '@seontechnologies/playwright-utils/file-utils';\n\ntest('should validate ZIP archive', async () => {\n const downloadPath = await handleDownload({\n page,\n downloadDir: DOWNLOAD_DIR,\n trigger: () => page.click('[data-testid=\"download-backup\"]'),\n });\n\n const zipResult = await readZIP({ filePath: downloadPath });\n\n // Check file list\n expect(Array.isArray(zipResult.content.entries)).toBe(true);\n expect(zipResult.content.entries).toContain('Case_53125_10-19-22_AM/Case_53125_10-19-22_AM_case_data.csv');\n\n // Extract specific file\n const targetFile = 'Case_53125_10-19-22_AM/Case_53125_10-19-22_AM_case_data.csv';\n const zipWithExtraction = await readZIP({\n filePath: downloadPath,\n fileToExtract: targetFile,\n });\n\n // Access extracted file buffer\n const extractedFiles = zipWithExtraction.content.extractedFiles || {};\n const fileBuffer = extractedFiles[targetFile];\n expect(fileBuffer).toBeInstanceOf(Buffer);\n expect(fileBuffer?.length).toBeGreaterThan(0);\n});\n```\n\n**Key Points**:\n\n- `content.entries` lists all files in archive\n- `fileToExtract` extracts specific files to Buffer\n- Validate archive structure\n- Read and parse individual files from ZIP\n\n### Example 5: API-Triggered Download\n\n**Context**: API endpoint returns file download (not UI click).\n\n**Implementation**:\n\n```typescript\ntest('should download via API', async ({ page, request }) => {\n const downloadPath = await handleDownload({\n page, // Still need page for download events\n downloadDir: DOWNLOAD_DIR,\n trigger: async () => {\n const response = await request.get('/api/export/csv', {\n headers: { Authorization: 'Bearer token' },\n });\n\n if (!response.ok()) {\n throw new Error(`Export failed: ${response.status()}`);\n }\n },\n });\n\n const { content } = await readCSV({ filePath: downloadPath });\n\n expect(content.data).toHaveLength(100);\n});\n```\n\n**Key Points**:\n\n- `trigger` can be async API call\n- API must return `Content-Disposition` header\n- Still need `page` for download events\n- Works with authenticated endpoints\n\n### Example 6: Reading CSV from Buffer (ZIP extraction)\n\n**Context**: Read CSV content directly from a Buffer (e.g., extracted from ZIP).\n\n**Implementation**:\n\n```typescript\n// Read from a Buffer (e.g., extracted from a ZIP)\nconst zipResult = await readZIP({\n filePath: 'archive.zip',\n fileToExtract: 'data.csv',\n});\nconst fileBuffer = zipResult.content.extractedFiles?.['data.csv'];\nconst csvFromBuffer = await readCSV({ content: fileBuffer });\n\n// Read from a string\nconst csvString = 'name,age\\nJohn,30\\nJane,25';\nconst csvFromString = await readCSV({ content: csvString });\n\nconst { data, headers } = csvFromString.content;\nexpect(headers).toContain('name');\nexpect(headers).toContain('age');\n```\n\n## API Reference\n\n### CSV Reader Options\n\n| Option | Type | Default | Description |\n| -------------- | ------------------ | -------- | -------------------------------------- |\n| `filePath` | `string` | - | Path to CSV file (mutually exclusive) |\n| `content` | `string \\| Buffer` | - | Direct content (mutually exclusive) |\n| `delimiter` | `string \\| 'auto'` | `','` | Value separator, auto-detect if 'auto' |\n| `encoding` | `string` | `'utf8'` | File encoding |\n| `parseHeaders` | `boolean` | `true` | Use first row as headers |\n| `trim` | `boolean` | `true` | Trim whitespace from values |\n\n### XLSX Reader Options\n\n| Option | Type | Description |\n| ----------- | -------- | ------------------------------ |\n| `filePath` | `string` | Path to XLSX file |\n| `sheetName` | `string` | Name of sheet to set as active |\n\n### PDF Reader Options\n\n| Option | Type | Default | Description |\n| ------------ | --------- | ------- | --------------------------- |\n| `filePath` | `string` | - | Path to PDF file (required) |\n| `mergePages` | `boolean` | `true` | Merge text from all pages |\n| `maxPages` | `number` | - | Maximum pages to extract |\n| `debug` | `boolean` | `false` | Enable debug logging |\n\n### ZIP Reader Options\n\n| Option | Type | Description |\n| --------------- | -------- | ---------------------------------- |\n| `filePath` | `string` | Path to ZIP file |\n| `fileToExtract` | `string` | Specific file to extract to Buffer |\n\n### Return Values\n\n#### CSV Reader Return Value\n\n```typescript\n{\n content: {\n data: Array\u003cArray\u003cstring | number>>, // Parsed rows (excludes header row if parseHeaders: true)\n headers: string[] | null // Column headers (null if parseHeaders: false)\n }\n}\n```\n\n#### XLSX Reader Return Value\n\n```typescript\n{\n content: {\n worksheets: Array\u003c{\n name: string; // Sheet name\n rows: Array\u003cArray\u003cany>>; // All rows including headers\n headers?: string[]; // First row as headers (if present)\n }>;\n }\n}\n```\n\n#### PDF Reader Return Value\n\n```typescript\n{\n content: string, // Extracted text (merged or per-page based on mergePages)\n pagesCount: number, // Total pages in PDF\n fileName?: string, // Original filename if available\n info?: Record\u003cstring, any> // PDF metadata (author, title, etc.)\n}\n```\n\n> **Note**: When `mergePages: false`, `content` is an array of strings (one per page). When `maxPages` is set, only that many pages are extracted.\n\n#### ZIP Reader Return Value\n\n```typescript\n{\n content: {\n entries: Array\u003c{\n name: string, // File/directory path within ZIP\n size: number, // Uncompressed size in bytes\n isDirectory: boolean // True for directories\n }>,\n extractedFiles: Record\u003cstring, Buffer | string> // Extracted file contents by path\n }\n}\n```\n\n> **Note**: When `fileToExtract` is specified, only that file appears in `extractedFiles`.\n\n## Download Cleanup Pattern\n\n```typescript\ntest.afterEach(async () => {\n // Clean up downloaded files\n await fs.remove(DOWNLOAD_DIR);\n});\n```\n\n## Comparison with Vanilla Playwright\n\nVanilla Playwright (real test) snippet:\n\n```typescript\n// ~80 lines of boilerplate!\nconst [download] = await Promise.all([page.waitForEvent('download'), page.getByTestId('download-button-CSV Export').click()]);\n\nconst failure = await download.failure();\nexpect(failure).toBeNull();\n\nconst filePath = testInfo.outputPath(download.suggestedFilename());\nawait download.saveAs(filePath);\n\nawait expect\n .poll(\n async () => {\n try {\n await fs.access(filePath);\n return true;\n } catch {\n return false;\n }\n },\n { timeout: 5000, intervals: [100, 200, 500] },\n )\n .toBe(true);\n\nconst csvContent = await fs.readFile(filePath, 'utf-8');\n\nconst parseResult = parse(csvContent, {\n header: true,\n skipEmptyLines: true,\n dynamicTyping: true,\n transformHeader: (header: string) => header.trim(),\n});\n\nif (parseResult.errors.length > 0) {\n throw new Error(`CSV parsing errors: ${JSON.stringify(parseResult.errors)}`);\n}\n\nconst data = parseResult.data as Array\u003cRecord\u003cstring, unknown>>;\nconst headers = parseResult.meta.fields || [];\n```\n\nWith File Utils, the same flow becomes:\n\n```typescript\nconst downloadPath = await handleDownload({\n page,\n downloadDir: DOWNLOAD_DIR,\n trigger: () => page.getByTestId('download-button-text/csv').click(),\n});\n\nconst { data, headers } = (await readCSV({ filePath: downloadPath })).content;\n```\n\n## Related Fragments\n\n- `overview.md` - Installation and imports\n- `api-request.md` - API-triggered downloads\n- `recurse.md` - Poll for file generation completion\n\n## Anti-Patterns\n\n**DON'T leave downloads in place:**\n\n```typescript\ntest('creates file', async () => {\n await handleDownload({ ... })\n // File left in downloads folder\n})\n```\n\n**DO clean up after tests:**\n\n```typescript\ntest.afterEach(async () => {\n await fs.remove(DOWNLOAD_DIR);\n});\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13745,"content_sha256":"7d1092930118fb160b9c0a4a48c398d9b08bb909d1e2432662d8e81e1e3b0087"},{"filename":"knowledge/fixture-architecture.md","content":"# Fixture Architecture Playbook\n\n## Principle\n\nBuild test helpers as pure functions first, then wrap them in framework-specific fixtures. Compose capabilities using `mergeTests` (Playwright) or layered commands (Cypress) instead of inheritance. Each fixture should solve one isolated concern (auth, API, logs, network).\n\n## Rationale\n\nTraditional Page Object Models create tight coupling through inheritance chains (`BasePage → LoginPage → AdminPage`). When base classes change, all descendants break. Pure functions with fixture wrappers provide:\n\n- **Testability**: Pure functions run in unit tests without framework overhead\n- **Composability**: Mix capabilities freely via `mergeTests`, no inheritance constraints\n- **Reusability**: Export fixtures via package subpaths for cross-project sharing\n- **Maintainability**: One concern per fixture = clear responsibility boundaries\n\n## Pattern Examples\n\n### Example 1: Pure Function → Fixture Pattern\n\n**Context**: When building any test helper, always start with a pure function that accepts all dependencies explicitly. Then wrap it in a Playwright fixture or Cypress command.\n\n**Implementation**:\n\n```typescript\n// playwright/support/helpers/api-request.ts\n// Step 1: Pure function (ALWAYS FIRST!)\ntype ApiRequestParams = {\n request: APIRequestContext;\n method: 'GET' | 'POST' | 'PUT' | 'DELETE';\n url: string;\n data?: unknown;\n headers?: Record\u003cstring, string>;\n};\n\nexport async function apiRequest({\n request,\n method,\n url,\n data,\n headers = {}\n}: ApiRequestParams) {\n const response = await request.fetch(url, {\n method,\n data,\n headers: {\n 'Content-Type': 'application/json',\n ...headers\n }\n });\n\n if (!response.ok()) {\n throw new Error(`API request failed: ${response.status()} ${await response.text()}`);\n }\n\n return response.json();\n}\n\n// Step 2: Fixture wrapper\n// playwright/support/fixtures/api-request-fixture.ts\nimport { test as base } from '@playwright/test';\nimport { apiRequest } from '../helpers/api-request';\n\nexport const test = base.extend\u003c{ apiRequest: typeof apiRequest }>({\n apiRequest: async ({ request }, use) => {\n // Inject framework dependency, expose pure function\n await use((params) => apiRequest({ request, ...params }));\n }\n});\n\n// Step 3: Package exports for reusability\n// package.json\n{\n \"exports\": {\n \"./api-request\": \"./playwright/support/helpers/api-request.ts\",\n \"./api-request/fixtures\": \"./playwright/support/fixtures/api-request-fixture.ts\"\n }\n}\n```\n\n**Key Points**:\n\n- Pure function is unit-testable without Playwright running\n- Framework dependency (`request`) injected at fixture boundary\n- Fixture exposes the pure function to test context\n- Package subpath exports enable `import { apiRequest } from 'my-fixtures/api-request'`\n\n### Example 2: Composable Fixture System with mergeTests\n\n**Context**: When building comprehensive test capabilities, compose multiple focused fixtures instead of creating monolithic helper classes. Each fixture provides one capability.\n\n**Implementation**:\n\n```typescript\n// playwright/support/fixtures/merged-fixtures.ts\nimport { test as base, mergeTests } from '@playwright/test';\nimport { test as apiRequestFixture } from './api-request-fixture';\nimport { test as networkFixture } from './network-fixture';\nimport { test as authFixture } from './auth-fixture';\nimport { test as logFixture } from './log-fixture';\n\n// Compose all fixtures for comprehensive capabilities\nexport const test = mergeTests(base, apiRequestFixture, networkFixture, authFixture, logFixture);\n\nexport { expect } from '@playwright/test';\n\n// Example usage in tests:\n// import { test, expect } from './support/fixtures/merged-fixtures';\n//\n// test('user can create order', async ({ page, apiRequest, auth, network }) => {\n// await auth.loginAs('[email protected]');\n// await network.interceptRoute('POST', '**/api/orders', { id: 123 });\n// await page.goto('/checkout');\n// await page.click('[data-testid=\"submit-order\"]');\n// await expect(page.getByText('Order #123')).toBeVisible();\n// });\n```\n\n**Individual Fixture Examples**:\n\n```typescript\n// network-fixture.ts\nexport const test = base.extend({\n network: async ({ page }, use) => {\n const interceptedRoutes = new Map();\n\n const interceptRoute = async (method: string, url: string, response: unknown) => {\n await page.route(url, (route) => {\n if (route.request().method() === method) {\n route.fulfill({ body: JSON.stringify(response) });\n }\n });\n interceptedRoutes.set(`${method}:${url}`, response);\n };\n\n await use({ interceptRoute });\n\n // Cleanup\n interceptedRoutes.clear();\n },\n});\n\n// auth-fixture.ts\nexport const test = base.extend({\n auth: async ({ page, context }, use) => {\n const loginAs = async (email: string) => {\n // Use API to setup auth (fast!)\n const token = await getAuthToken(email);\n await context.addCookies([\n {\n name: 'auth_token',\n value: token,\n domain: 'localhost',\n path: '/',\n },\n ]);\n };\n\n await use({ loginAs });\n },\n});\n```\n\n**Key Points**:\n\n- `mergeTests` combines fixtures without inheritance\n- Each fixture has single responsibility (network, auth, logs)\n- Tests import merged fixture and access all capabilities\n- No coupling between fixtures—add/remove freely\n\n### Example 3: Framework-Agnostic HTTP Helper\n\n**Context**: When building HTTP helpers, keep them framework-agnostic. Accept all params explicitly so they work in unit tests, Playwright, Cypress, or any context.\n\n**Implementation**:\n\n```typescript\n// shared/helpers/http-helper.ts\n// Pure, framework-agnostic function\ntype HttpHelperParams = {\n baseUrl: string;\n endpoint: string;\n method: 'GET' | 'POST' | 'PUT' | 'DELETE';\n body?: unknown;\n headers?: Record\u003cstring, string>;\n token?: string;\n};\n\nexport async function makeHttpRequest({ baseUrl, endpoint, method, body, headers = {}, token }: HttpHelperParams): Promise\u003cunknown> {\n const url = `${baseUrl}${endpoint}`;\n const requestHeaders = {\n 'Content-Type': 'application/json',\n ...(token && { Authorization: `Bearer ${token}` }),\n ...headers,\n };\n\n const response = await fetch(url, {\n method,\n headers: requestHeaders,\n body: body ? JSON.stringify(body) : undefined,\n });\n\n if (!response.ok) {\n const errorText = await response.text();\n throw new Error(`HTTP ${method} ${url} failed: ${response.status} ${errorText}`);\n }\n\n return response.json();\n}\n\n// Playwright fixture wrapper\n// playwright/support/fixtures/http-fixture.ts\nimport { test as base } from '@playwright/test';\nimport { makeHttpRequest } from '../../shared/helpers/http-helper';\n\nexport const test = base.extend({\n httpHelper: async ({}, use) => {\n const baseUrl = process.env.API_BASE_URL || 'http://localhost:3000';\n\n await use((params) => makeHttpRequest({ baseUrl, ...params }));\n },\n});\n\n// Cypress command wrapper\n// cypress/support/commands.ts\nimport { makeHttpRequest } from '../../shared/helpers/http-helper';\n\nCypress.Commands.add('apiRequest', (params) => {\n const baseUrl = Cypress.env('API_BASE_URL') || 'http://localhost:3000';\n return cy.wrap(makeHttpRequest({ baseUrl, ...params }));\n});\n```\n\n**Key Points**:\n\n- Pure function uses only standard `fetch`, no framework dependencies\n- Unit tests call `makeHttpRequest` directly with all params\n- Playwright and Cypress wrappers inject framework-specific config\n- Same logic runs everywhere—zero duplication\n\n### Example 4: Fixture Cleanup Pattern\n\n**Context**: When fixtures create resources (data, files, connections), ensure automatic cleanup in fixture teardown. Tests must not leak state.\n\n**Implementation**:\n\n```typescript\n// playwright/support/fixtures/database-fixture.ts\nimport { test as base } from '@playwright/test';\nimport { seedDatabase, deleteRecord } from '../helpers/db-helpers';\n\ntype DatabaseFixture = {\n seedUser: (userData: Partial\u003cUser>) => Promise\u003cUser>;\n seedOrder: (orderData: Partial\u003cOrder>) => Promise\u003cOrder>;\n};\n\nexport const test = base.extend\u003cDatabaseFixture>({\n seedUser: async ({}, use) => {\n const createdUsers: string[] = [];\n\n const seedUser = async (userData: Partial\u003cUser>) => {\n const user = await seedDatabase('users', userData);\n createdUsers.push(user.id);\n return user;\n };\n\n await use(seedUser);\n\n // Auto-cleanup: Delete all users created during test\n for (const userId of createdUsers) {\n await deleteRecord('users', userId);\n }\n createdUsers.length = 0;\n },\n\n seedOrder: async ({}, use) => {\n const createdOrders: string[] = [];\n\n const seedOrder = async (orderData: Partial\u003cOrder>) => {\n const order = await seedDatabase('orders', orderData);\n createdOrders.push(order.id);\n return order;\n };\n\n await use(seedOrder);\n\n // Auto-cleanup: Delete all orders\n for (const orderId of createdOrders) {\n await deleteRecord('orders', orderId);\n }\n createdOrders.length = 0;\n },\n});\n\n// Example usage:\n// test('user can place order', async ({ seedUser, seedOrder, page }) => {\n// const user = await seedUser({ email: '[email protected]' });\n// const order = await seedOrder({ userId: user.id, total: 100 });\n//\n// await page.goto(`/orders/${order.id}`);\n// await expect(page.getByText('Order Total: $100')).toBeVisible();\n//\n// // No manual cleanup needed—fixture handles it automatically\n// });\n```\n\n**Key Points**:\n\n- Track all created resources in array during test execution\n- Teardown (after `use()`) deletes all tracked resources\n- Tests don't manually clean up—happens automatically\n- Prevents test pollution and flakiness from shared state\n\n### Anti-Pattern: Inheritance-Based Page Objects\n\n**Problem**:\n\n```typescript\n// ❌ BAD: Page Object Model with inheritance\nclass BasePage {\n constructor(public page: Page) {}\n\n async navigate(url: string) {\n await this.page.goto(url);\n }\n\n async clickButton(selector: string) {\n await this.page.click(selector);\n }\n}\n\nclass LoginPage extends BasePage {\n async login(email: string, password: string) {\n await this.navigate('/login');\n await this.page.fill('#email', email);\n await this.page.fill('#password', password);\n await this.clickButton('#submit');\n }\n}\n\nclass AdminPage extends LoginPage {\n async accessAdminPanel() {\n await this.login('[email protected]', 'admin123');\n await this.navigate('/admin');\n }\n}\n```\n\n**Why It Fails**:\n\n- Changes to `BasePage` break all descendants (`LoginPage`, `AdminPage`)\n- `AdminPage` inherits unnecessary `login` details—tight coupling\n- Cannot compose capabilities (e.g., admin + reporting features require multiple inheritance)\n- Hard to test `BasePage` methods in isolation\n- Hidden state in class instances leads to unpredictable behavior\n\n**Better Approach**: Use pure functions + fixtures\n\n```typescript\n// ✅ GOOD: Pure functions with fixture composition\n// helpers/navigation.ts\nexport async function navigate(page: Page, url: string) {\n await page.goto(url);\n}\n\n// helpers/auth.ts\nexport async function login(page: Page, email: string, password: string) {\n await page.fill('[data-testid=\"email\"]', email);\n await page.fill('[data-testid=\"password\"]', password);\n await page.click('[data-testid=\"submit\"]');\n}\n\n// fixtures/admin-fixture.ts\nexport const test = base.extend({\n adminPage: async ({ page }, use) => {\n await login(page, '[email protected]', 'admin123');\n await navigate(page, '/admin');\n await use(page);\n },\n});\n\n// Tests import exactly what they need—no inheritance\n```\n\n## Integration Points\n\n- **Used in workflows**: `*atdd` (test generation), `*automate` (test expansion), `*framework` (initial setup)\n- **Related fragments**:\n - `data-factories.md` - Factory functions for test data\n - `network-first.md` - Network interception patterns\n - `test-quality.md` - Deterministic test design principles\n\n## Helper Function Reuse Guidelines\n\nWhen deciding whether to create a fixture, follow these rules:\n\n- **3+ uses** → Create fixture with subpath export (shared across tests/projects)\n- **2-3 uses** → Create utility module (shared within project)\n- **1 use** → Keep inline (avoid premature abstraction)\n- **Complex logic** → Factory function pattern (dynamic data generation)\n\n_Source: Murat Testing Philosophy (lines 74-122), enterprise production patterns, Playwright fixture docs._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12414,"content_sha256":"c0c62381e7a0385fe0d8b59861a3756994d3bbc673e4605ded9607d5217c7e55"},{"filename":"knowledge/fixtures-composition.md","content":"# Fixtures Composition with mergeTests\n\n## Principle\n\nCombine multiple Playwright fixtures using `mergeTests` to create a unified test object with all capabilities. Build composable test infrastructure by merging playwright-utils fixtures with custom project fixtures.\n\n## Rationale\n\nUsing fixtures from multiple sources requires combining them:\n\n- Importing from multiple fixture files is verbose\n- Name conflicts between fixtures\n- Duplicate fixture definitions\n- No clear single test object\n\nPlaywright's `mergeTests` provides:\n\n- **Single test object**: All fixtures in one import\n- **Conflict resolution**: Handles name collisions automatically\n- **Composition pattern**: Mix utilities, custom fixtures, third-party fixtures\n- **Type safety**: Full TypeScript support for merged fixtures\n- **Maintainability**: One place to manage all fixtures\n\n## Pattern Examples\n\n### Example 1: Basic Fixture Merging\n\n**Context**: Combine multiple playwright-utils fixtures into single test object.\n\n**Implementation**:\n\n```typescript\n// playwright/support/merged-fixtures.ts\nimport { mergeTests } from '@playwright/test';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\n// Auth fixture built in your project (setAuthProvider + createAuthFixtures)\nimport { test as authFixture } from './auth-fixture';\n\n// Merge all fixtures\nexport const test = mergeTests(apiRequestFixture, authFixture, recurseFixture);\n\nexport { expect } from '@playwright/test';\n```\n\n```typescript\n// In your tests - import from merged fixtures\nimport { test, expect } from '../support/merged-fixtures';\n\ntest('all utilities available', async ({\n apiRequest, // From api-request fixture\n authToken, // From auth fixture\n recurse, // From recurse fixture\n}) => {\n // All fixtures available in single test signature\n const { body } = await apiRequest({\n method: 'GET',\n path: '/api/protected',\n headers: { Authorization: `Bearer ${authToken}` },\n });\n\n await recurse(\n () => apiRequest({ method: 'GET', path: `/status/${body.id}` }),\n (res) => res.body.ready === true,\n );\n});\n```\n\n**Key Points**:\n\n- Create one `merged-fixtures.ts` per project\n- Import test object from merged fixtures in all test files\n- All utilities available without multiple imports\n- Type-safe access to all fixtures\n\n### Example 2: Combining with Custom Fixtures\n\n**Context**: Add project-specific fixtures alongside playwright-utils.\n\n**Implementation**:\n\n```typescript\n// playwright/support/custom-fixtures.ts - Your project fixtures\nimport { test as base } from '@playwright/test';\nimport { createUser } from './factories/user-factory';\nimport { seedDatabase } from './helpers/db-seeder';\n\nexport const test = base.extend({\n // Custom fixture 1: Auto-seeded user\n testUser: async ({ request }, use) => {\n const user = await createUser({ role: 'admin' });\n await seedDatabase('users', [user]);\n await use(user);\n // Cleanup happens automatically\n },\n\n // Custom fixture 2: Database helpers\n db: async ({}, use) => {\n await use({\n seed: seedDatabase,\n clear: () => seedDatabase.truncate(),\n });\n },\n});\n\n// playwright/support/merged-fixtures.ts - Combine everything\nimport { mergeTests } from '@playwright/test';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\n// Auth fixture built in your project (setAuthProvider + createAuthFixtures)\nimport { test as authFixture } from './auth-fixture';\nimport { test as customFixtures } from './custom-fixtures';\n\nexport const test = mergeTests(\n apiRequestFixture,\n authFixture,\n customFixtures, // Your project fixtures\n);\n\nexport { expect } from '@playwright/test';\n```\n\n```typescript\n// In tests - all fixtures available\nimport { test, expect } from '../support/merged-fixtures';\n\ntest('using mixed fixtures', async ({\n apiRequest, // playwright-utils\n authToken, // playwright-utils\n testUser, // custom\n db, // custom\n}) => {\n // Use playwright-utils\n const { body } = await apiRequest({\n method: 'GET',\n path: `/api/users/${testUser.id}`,\n headers: { Authorization: `Bearer ${authToken}` },\n });\n\n // Use custom fixture\n await db.clear();\n});\n```\n\n**Key Points**:\n\n- Custom fixtures extend `base` test\n- Merge custom with playwright-utils fixtures\n- All available in one test signature\n- Maintainable separation of concerns\n\n### Example 3: Full Utility Suite Integration\n\n**Context**: Production setup with all core playwright-utils and custom fixtures.\n\n**Implementation**:\n\n```typescript\n// playwright/support/merged-fixtures.ts\nimport { mergeTests } from '@playwright/test';\n\n// Playwright utils fixtures\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as interceptFixture } from '@seontechnologies/playwright-utils/intercept-network-call/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\nimport { test as networkRecorderFixture } from '@seontechnologies/playwright-utils/network-recorder/fixtures';\n\n// Custom project fixtures\n// Auth fixture built in your project (setAuthProvider + createAuthFixtures)\nimport { test as authFixture } from './auth-fixture';\nimport { test as customFixtures } from './custom-fixtures';\n\n// Merge everything\nexport const test = mergeTests(apiRequestFixture, authFixture, interceptFixture, recurseFixture, networkRecorderFixture, customFixtures);\n\nexport { expect } from '@playwright/test';\n```\n\n```typescript\n// In tests\nimport { test, expect } from '../support/merged-fixtures';\n\ntest('full integration', async ({\n page,\n context,\n apiRequest,\n authToken,\n interceptNetworkCall,\n recurse,\n networkRecorder,\n testUser, // custom\n}) => {\n // All utilities + custom fixtures available\n await networkRecorder.setup(context);\n\n const usersCall = interceptNetworkCall({ url: '**/api/users' });\n\n await page.goto('/users');\n const { responseJson } = await usersCall;\n\n expect(responseJson).toContainEqual(expect.objectContaining({ id: testUser.id }));\n});\n```\n\n**Key Points**:\n\n- One merged-fixtures.ts for entire project\n- Combine all playwright-utils you use\n- Add custom project fixtures\n- Single import in all test files\n\n### Example 4: Fixture Override Pattern\n\n**Context**: Override default options for specific test files or describes.\n\n**Implementation**:\n\n```typescript\nimport { test, expect } from '../support/merged-fixtures';\n\n// Override auth options for entire file\ntest.use({\n authOptions: {\n userIdentifier: 'admin',\n environment: 'staging',\n },\n});\n\ntest('uses admin on staging', async ({ authToken }) => {\n // Token is for admin user on staging environment\n});\n\n// Override for specific describe block\ntest.describe('manager tests', () => {\n test.use({\n authOptions: {\n userIdentifier: 'manager',\n },\n });\n\n test('manager can access reports', async ({ page }) => {\n // Uses manager token\n await page.goto('/reports');\n });\n});\n```\n\n**Key Points**:\n\n- `test.use()` overrides fixture options\n- Can override at file or describe level\n- Options merge with defaults\n- Type-safe overrides\n\n### Example 5: Avoiding Fixture Conflicts\n\n**Context**: Handle name collisions when merging fixtures with same names.\n\n**Implementation**:\n\n```typescript\n// If two fixtures have same name, last one wins\nimport { test as fixture1 } from './fixture1'; // has 'user' fixture\nimport { test as fixture2 } from './fixture2'; // also has 'user' fixture\n\nconst test = mergeTests(fixture1, fixture2);\n// fixture2's 'user' overrides fixture1's 'user'\n\n// Better: Rename fixtures before merging\nimport { test as base } from '@playwright/test';\nimport { test as fixture1 } from './fixture1';\n\nconst fixture1Renamed = base.extend({\n user1: fixture1._extend.user, // Rename to avoid conflict\n});\n\nconst test = mergeTests(fixture1Renamed, fixture2);\n// Now both 'user1' and 'user' available\n\n// Best: Design fixtures without conflicts\n// - Prefix custom fixtures: 'myAppUser', 'myAppDb'\n// - Playwright-utils uses descriptive names: 'apiRequest', 'authToken'\n```\n\n**Key Points**:\n\n- Last fixture wins in conflicts\n- Rename fixtures to avoid collisions\n- Design fixtures with unique names\n- Playwright-utils uses descriptive names (no conflicts)\n\n## Recommended Project Structure\n\n```text\nplaywright/\n├── support/\n│ ├── merged-fixtures.ts # ⭐ Single test object for project\n│ ├── custom-fixtures.ts # Your project-specific fixtures\n│ ├── auth/\n│ │ ├── auth-fixture.ts # Auth wrapper (if needed)\n│ │ └── custom-auth-provider.ts\n│ ├── fixtures/\n│ │ ├── user-fixture.ts\n│ │ ├── db-fixture.ts\n│ │ └── api-fixture.ts\n│ └── utils/\n│ └── factories/\n└── tests/\n ├── api/\n │ └── users.spec.ts # import { test } from '../../support/merged-fixtures'\n ├── e2e/\n │ └── login.spec.ts # import { test } from '../../support/merged-fixtures'\n └── component/\n └── button.spec.ts # import { test } from '../../support/merged-fixtures'\n```\n\n## Benefits of Fixture Composition\n\n**Compared to direct imports:**\n\n```typescript\n// ❌ Without mergeTests (verbose)\nimport { test as base } from '@playwright/test';\nimport { apiRequest } from '@seontechnologies/playwright-utils/api-request';\nimport { getAuthToken } from './auth';\nimport { createUser } from './factories';\n\ntest('verbose', async ({ request }) => {\n const token = await getAuthToken();\n const user = await createUser();\n const response = await apiRequest({ request, method: 'GET', path: '/api/users' });\n // Manual wiring everywhere\n});\n\n// ✅ With mergeTests (clean)\nimport { test } from '../support/merged-fixtures';\n\ntest('clean', async ({ apiRequest, authToken, testUser }) => {\n const { body } = await apiRequest({ method: 'GET', path: '/api/users' });\n // All fixtures auto-wired\n});\n```\n\n**Reduction:** ~10 lines per test → ~2 lines\n\n## Related Fragments\n\n- `overview.md` - Installation and design principles\n- `api-request.md`, `auth-session.md`, `recurse.md` - Utilities to merge\n- `network-recorder.md`, `intercept-network-call.md`, `log.md` - Additional utilities\n\n## Anti-Patterns\n\n**❌ Importing test from multiple fixture files:**\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/api-request/fixtures';\n// Also need auth... (fixture built in your project: setAuthProvider + createAuthFixtures)\nimport { test as authTest } from './support/auth/auth-fixture';\n// Name conflict! Which test to use?\n```\n\n**✅ Use merged fixtures:**\n\n```typescript\nimport { test } from '../support/merged-fixtures';\n// All utilities available, no conflicts\n```\n\n**❌ Merging too many fixtures (kitchen sink):**\n\n```typescript\n// Merging 20+ fixtures makes test signature huge\nconst test = mergeTests(...20 different fixtures)\n\ntest('my test', async ({ fixture1, fixture2, ..., fixture20 }) => {\n // Cognitive overload\n})\n```\n\n**✅ Merge only what you actually use:**\n\n```typescript\n// Merge the 4-6 fixtures your project actually needs\nconst test = mergeTests(apiRequestFixture, authFixture, recurseFixture, customFixtures);\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":11399,"content_sha256":"cd62666b34894c4130bc0da73503fc24b820094e8a4c3ec2a70a16700d77af5a"},{"filename":"knowledge/intercept-network-call.md","content":"# Intercept Network Call Utility\n\n## Principle\n\nIntercept network requests with a single declarative call that returns a Promise. Automatically parse JSON responses, support both spy (observe) and stub (mock) patterns, and use powerful glob pattern matching for URL filtering.\n\n## Rationale\n\nVanilla Playwright's network interception requires multiple steps:\n\n- `page.route()` to setup, `page.waitForResponse()` to capture\n- Manual JSON parsing\n- Verbose syntax for conditional handling\n- Complex filter predicates\n\nThe `interceptNetworkCall` utility provides:\n\n- **Single declarative call**: Setup and wait in one statement\n- **Automatic JSON parsing**: Response pre-parsed, strongly typed\n- **Flexible URL patterns**: Glob matching with picomatch\n- **Spy or stub modes**: Observe real traffic or mock responses\n- **Concise API**: Reduces boilerplate by 60-70%\n\n## Pattern Examples\n\n### Example 1: Spy on Network (Observe Real Traffic)\n\n**Context**: Capture and inspect real API responses for validation.\n\n**Implementation**:\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/intercept-network-call/fixtures';\n\ntest('should spy on users API', async ({ page, interceptNetworkCall }) => {\n // Setup interception BEFORE navigation\n const usersCall = interceptNetworkCall({\n url: '**/api/users', // Glob pattern\n });\n\n await page.goto('/dashboard');\n\n // Wait for response and access parsed data\n const { responseJson, status } = await usersCall;\n\n expect(status).toBe(200);\n expect(responseJson).toHaveLength(10);\n expect(responseJson[0]).toHaveProperty('name');\n});\n```\n\n**Key Points**:\n\n- Intercept before navigation (critical for race-free tests)\n- Returns Promise with `{ responseJson, status, requestBody }`\n- Glob patterns (`**` matches any path segment)\n- JSON automatically parsed\n\n### Example 2: Stub Network (Mock Response)\n\n**Context**: Mock API responses for testing UI behavior without backend.\n\n**Implementation**:\n\n```typescript\ntest('should stub users API', async ({ page, interceptNetworkCall }) => {\n const mockUsers = [\n { id: 1, name: 'Test User 1' },\n { id: 2, name: 'Test User 2' },\n ];\n\n const usersCall = interceptNetworkCall({\n url: '**/api/users',\n fulfillResponse: {\n status: 200,\n body: mockUsers,\n },\n });\n\n await page.goto('/dashboard');\n await usersCall;\n\n // UI shows mocked data\n await expect(page.getByText('Test User 1')).toBeVisible();\n await expect(page.getByText('Test User 2')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- `fulfillResponse` mocks the API\n- No backend needed\n- Test UI logic in isolation\n- Status code and body fully controllable\n\n### Example 3: Conditional Response Handling\n\n**Context**: Different responses based on request method or parameters.\n\n**Implementation**:\n\n```typescript\ntest('conditional mocking', async ({ page, interceptNetworkCall }) => {\n await interceptNetworkCall({\n url: '**/api/data',\n handler: async (route, request) => {\n if (request.method() === 'POST') {\n // Mock POST success\n await route.fulfill({\n status: 201,\n body: JSON.stringify({ id: 'new-id', success: true }),\n });\n } else if (request.method() === 'GET') {\n // Mock GET with data\n await route.fulfill({\n status: 200,\n body: JSON.stringify([{ id: 1, name: 'Item' }]),\n });\n } else {\n // Let other methods through\n await route.continue();\n }\n },\n });\n\n await page.goto('/data-page');\n});\n```\n\n**Key Points**:\n\n- `handler` function for complex logic\n- Access full `route` and `request` objects\n- Can mock, continue, or abort\n- Flexible for advanced scenarios\n\n### Example 4: Error Simulation\n\n**Context**: Testing error handling in UI when API fails.\n\n**Implementation**:\n\n```typescript\ntest('should handle API errors gracefully', async ({ page, interceptNetworkCall }) => {\n // Simulate 500 error\n const errorCall = interceptNetworkCall({\n url: '**/api/users',\n fulfillResponse: {\n status: 500,\n body: { error: 'Internal Server Error' },\n },\n });\n\n await page.goto('/dashboard');\n await errorCall;\n\n // Verify UI shows error state\n await expect(page.getByText('Failed to load users')).toBeVisible();\n await expect(page.getByTestId('retry-button')).toBeVisible();\n});\n\n// Simulate network timeout\ntest('should handle timeout', async ({ page, interceptNetworkCall }) => {\n await interceptNetworkCall({\n url: '**/api/slow',\n handler: async (route) => {\n // Never respond - simulates timeout\n await new Promise(() => {});\n },\n });\n\n await page.goto('/slow-page');\n\n // UI should show timeout error\n await expect(page.getByText('Request timed out')).toBeVisible({ timeout: 10000 });\n});\n```\n\n**Key Points**:\n\n- Mock error statuses (4xx, 5xx)\n- Test timeout scenarios\n- Validate error UI states\n- No real failures needed\n\n### Example 5: Order Matters - Intercept Before Navigate\n\n**Context**: The interceptor must be set up before the network request occurs.\n\n**Implementation**:\n\n```typescript\n// INCORRECT - interceptor set up too late\nawait page.goto('https://example.com'); // Request already happened\nconst networkCall = interceptNetworkCall({ url: '**/api/data' });\nawait networkCall; // Will hang indefinitely!\n\n// CORRECT - Set up interception first\nconst networkCall = interceptNetworkCall({ url: '**/api/data' });\nawait page.goto('https://example.com');\nconst result = await networkCall;\n```\n\nThis pattern follows the classic test spy/stub pattern:\n\n1. Define the spy/stub (set up interception)\n2. Perform the action (trigger the network request)\n3. Assert on the spy/stub (await and verify the response)\n\n### Example 6: Multiple Intercepts\n\n**Context**: Intercepting different endpoints in same test - setup order is critical.\n\n**Implementation**:\n\n```typescript\ntest('multiple intercepts', async ({ page, interceptNetworkCall }) => {\n // Setup all intercepts BEFORE navigation\n const usersCall = interceptNetworkCall({ url: '**/api/users' });\n const productsCall = interceptNetworkCall({ url: '**/api/products' });\n const ordersCall = interceptNetworkCall({ url: '**/api/orders' });\n\n // THEN navigate\n await page.goto('/dashboard');\n\n // Wait for all (or specific ones)\n const [users, products] = await Promise.all([usersCall, productsCall]);\n\n expect(users.responseJson).toHaveLength(10);\n expect(products.responseJson).toHaveLength(50);\n});\n```\n\n**Key Points**:\n\n- Setup all intercepts before triggering actions\n- Use `Promise.all()` to wait for multiple calls\n- Order: intercept -> navigate -> await\n- Prevents race conditions\n\n### Example 7: Capturing Multiple Requests to the Same Endpoint\n\n**Context**: Each `interceptNetworkCall` captures only the first matching request.\n\n**Implementation**:\n\n```typescript\n// Capturing a known number of requests\nconst firstRequest = interceptNetworkCall({ url: '/api/data' });\nconst secondRequest = interceptNetworkCall({ url: '/api/data' });\n\nawait page.click('#load-data-button');\n\nconst firstResponse = await firstRequest;\nconst secondResponse = await secondRequest;\n\nexpect(firstResponse.status).toBe(200);\nexpect(secondResponse.status).toBe(200);\n\n// Handling an unknown number of requests\nconst getDataRequestInterceptor = () =>\n interceptNetworkCall({\n url: '/api/data',\n timeout: 1000, // Short timeout to detect when no more requests are coming\n });\n\nlet currentInterceptor = getDataRequestInterceptor();\nconst allResponses = [];\n\nawait page.click('#load-multiple-data-button');\n\nwhile (true) {\n try {\n const response = await currentInterceptor;\n allResponses.push(response);\n currentInterceptor = getDataRequestInterceptor();\n } catch (error) {\n // No more requests (timeout)\n break;\n }\n}\n\nconsole.log(`Captured ${allResponses.length} requests to /api/data`);\n```\n\n### Example 8: Using Timeout\n\n**Context**: Set a timeout for waiting on a network request.\n\n**Implementation**:\n\n```typescript\nconst dataCall = interceptNetworkCall({\n method: 'GET',\n url: '/api/data-that-might-be-slow',\n timeout: 5000, // 5 seconds timeout\n});\n\nawait page.goto('/data-page');\n\ntry {\n const { responseJson } = await dataCall;\n console.log('Data loaded successfully:', responseJson);\n} catch (error) {\n if (error.message.includes('timeout')) {\n console.log('Request timed out as expected');\n } else {\n throw error;\n }\n}\n```\n\n## URL Pattern Matching\n\nThe utility uses [picomatch](https://github.com/micromatch/picomatch) for powerful glob pattern matching, dramatically simplifying URL targeting:\n\n**Supported glob patterns:**\n\n```typescript\n'**/api/users'; // Any path ending with /api/users\n'/api/users'; // Exact match\n'**/users/*'; // Any users sub-path\n'**/api/{users,products}'; // Either users or products\n'**/api/users?id=*'; // With query params\n```\n\n**Comparison with vanilla Playwright:**\n\n```typescript\n// Vanilla Playwright - complex predicate\nconst predicate = (response) => {\n const url = response.url();\n return url.endsWith('/api/users') || url.match(/\\/api\\/users\\/\\d+/) || (url.includes('/api/users/') && url.includes('/profile'));\n};\npage.waitForResponse(predicate);\n\n// With interceptNetworkCall - simple glob patterns\ninterceptNetworkCall({ url: '/api/users' }); // Exact endpoint\ninterceptNetworkCall({ url: '/api/users/*' }); // User by ID pattern\ninterceptNetworkCall({ url: '/api/users/*/profile' }); // Specific sub-paths\ninterceptNetworkCall({ url: '/api/users/**' }); // Match all\n```\n\n## API Reference\n\n### `interceptNetworkCall(options)`\n\n| Parameter | Type | Description |\n| ----------------- | ---------- | --------------------------------------------------------------------- |\n| `page` | `Page` | Required when using direct import (not needed with fixture) |\n| `method` | `string` | Optional: HTTP method to match (e.g., 'GET', 'POST') |\n| `url` | `string` | Optional: URL pattern to match (supports glob patterns via picomatch) |\n| `fulfillResponse` | `object` | Optional: Response to use when mocking |\n| `handler` | `function` | Optional: Custom handler function for the route |\n| `timeout` | `number` | Optional: Timeout in milliseconds for the network request |\n\n### `fulfillResponse` Object\n\n| Property | Type | Description |\n| --------- | ------------------------ | ----------------------------------------------------- |\n| `status` | `number` | HTTP status code (default: 200) |\n| `headers` | `Record\u003cstring, string>` | Response headers |\n| `body` | `any` | Response body (will be JSON.stringified if an object) |\n\n### Return Value\n\nReturns a `Promise\u003cNetworkCallResult>` with:\n\n| Property | Type | Description |\n| -------------- | ---------- | --------------------------------------- |\n| `request` | `Request` | The intercepted request |\n| `response` | `Response` | The response (null if mocked) |\n| `responseJson` | `any` | Parsed JSON response (if available) |\n| `status` | `number` | HTTP status code |\n| `requestJson` | `any` | Parsed JSON request body (if available) |\n\n## Comparison with Vanilla Playwright\n\n| Vanilla Playwright | intercept-network-call |\n| ----------------------------------------------------------- | ------------------------------------------------------------ |\n| `await page.route('/api/users', route => route.continue())` | `const call = interceptNetworkCall({ url: '**/api/users' })` |\n| `const resp = await page.waitForResponse('/api/users')` | (Combined in single statement) |\n| `const json = await resp.json()` | `const { responseJson } = await call` |\n| `const status = resp.status()` | `const { status } = await call` |\n| Complex filter predicates | Simple glob patterns |\n\n**Reduction:** ~5-7 lines -> ~2-3 lines per interception\n\n## Related Fragments\n\n- `network-first.md` - Core pattern: intercept before navigate\n- `network-recorder.md` - HAR-based offline testing\n- `overview.md` - Fixture composition basics\n\n## Anti-Patterns\n\n**DON'T intercept after navigation:**\n\n```typescript\nawait page.goto('/dashboard'); // Navigation starts\nconst usersCall = interceptNetworkCall({ url: '**/api/users' }); // Too late!\n```\n\n**DO intercept before navigate:**\n\n```typescript\nconst usersCall = interceptNetworkCall({ url: '**/api/users' }); // First\nawait page.goto('/dashboard'); // Then navigate\nconst { responseJson } = await usersCall; // Then await\n```\n\n**DON'T ignore the returned Promise:**\n\n```typescript\ninterceptNetworkCall({ url: '**/api/users' }); // Not awaited!\nawait page.goto('/dashboard');\n// No deterministic wait - race condition\n```\n\n**DO always await the intercept:**\n\n```typescript\nconst usersCall = interceptNetworkCall({ url: '**/api/users' });\nawait page.goto('/dashboard');\nawait usersCall; // Deterministic wait\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13489,"content_sha256":"ac8213cc28a9f9c452a6fb419356dd1d66ce495d7f29d188fcb1bb51456ba869"},{"filename":"knowledge/library-integration-mandate.md","content":"# Library Integration Mandate\n\n## Principle\n\nA TEA config flag that enables an integration library is an **instruction to write the suite that way**, not a note that the library exists. When the flag is `true` and the package is installed, that library is the default implementation for every capability it covers. The vanilla or hand-rolled equivalent becomes a documented deviation.\n\nThis fragment is the general contract. Each library has its own mandate fragment carrying the substitution table for that package. Load this one plus the per-library one; this fragment decides _how_ a mandate binds, the per-library one decides _what_ it swaps.\n\nThe failure this exists to prevent: a flag that only changes which fragments load, while the code templates the agent copies from stay vanilla. The user then has to name a utility by hand to get it, which makes the flag decorative.\n\n## The Two Gates\n\nA mandate binds only when **both** hold:\n\n1. **The flag is `true`** under `[modules.tea]`, read as `config.\u003cflag>`.\n2. **The package is a dependency** in the project's manifest (`package.json` for the Node libraries).\n\nA flag with no install is an intention, not a capability. Generation must not scaffold imports against a package the project does not have, and review must not deduct per file for not using one. In that state: say so once, recommend the `framework` workflow, and generate the vanilla path.\n\n## Enforcement Levels\n\nEvery substitution in a per-library mandate carries one of two levels.\n\n| Level | Meaning |\n| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |\n| **REQUIRED** | Drop-in. Nothing beyond the import is needed. Emitting the vanilla equivalent instead is a defect, not a style preference. |\n| **RECOMMENDED** | Needs project-side wiring, or an input the project may not have. Propose it, scaffold the wiring when the workflow's scope covers setup. |\n\nFor a RECOMMENDED item, when the wiring is missing and the active workflow cannot create it: say in the output that the utility is the intended pattern, and name the wiring the project still needs. Never fall back silently. A run that quietly hand-rolled the thing and said nothing has hidden the one item the next person needs to fix.\n\n## Deviation Protocol\n\nA hand-rolled implementation is allowed when the library genuinely does not cover the case. When it happens:\n\n1. Put a one-line comment above the code: `// \u003clibrary> deviation: \u003creason>`\n2. List it in the workflow's output summary under a `\u003cLibrary> deviations` heading, with file, line, and reason.\n\nAn unexplained hand-rolled implementation is a finding. A stated one is a decision.\n\n## Self-Check Before Emitting\n\nBefore writing any generated file, run the per-library mandate's self-check list against it. A surviving vanilla call is either fixed or converted into a stated deviation. Emitting it unresolved is the defect.\n\n## Scope Discipline\n\nEvery mandate names the runner and language it binds, and names what it does not touch. A mandate that does not state its scope will be over-applied — into a Maestro flow that has no DOM, into a pytest suite that cannot import a Node package, into a Cypress spec with a different fixture model.\n\nScope is decided by **the runner the file executes under**, never by what the code under test is written in. A Node/TypeScript backend service tested through the Playwright runner is in Playwright Utils scope; the same service tested with Jest is not.\n\n## Registry\n\n| Config flag | Mandate fragment | Package | Binds |\n| -------------------------- | ----------------------------- | -------------------------------------------------- | ------------------------------------------------------------ |\n| `tea_use_playwright_utils` | `playwright-utils-mandate.md` | `@seontechnologies/playwright-utils` | JS/TS suites on the Playwright test runner, browser and API |\n| `tea_use_pactjs_utils` | `pactjs-utils-mandate.md` | `@seontechnologies/pactjs-utils` | JS/TS Pact consumer and provider suites, HTTP and message |\n| `tea_pact_mcp` | `pact-mcp.md` | `@smartbear/mcp` (an MCP server, not a dependency) | Broker-aware steps in test-design, automate, test-review, ci |\n\n`tea_pact_mcp` is the odd one: an MCP server is a runtime capability rather than a project dependency, so its second gate is \"the MCP tools are actually reachable in this session\" rather than a manifest entry. When they are not, degrade to the documented non-broker path and say the broker was unreachable. Never block a workflow on it, and never invent broker data.\n\nNo other config flag carries a mandate. `tea_browser_automation`, `tea_execution_mode`, `tea_capability_probe`, `test_stack_type`, `ci_platform`, and `test_framework` select behavior, not an implementation library.\n\n## Adding a New Library\n\nWhen TEA takes on another integration library, these are the places it has to land. A library wired into fewer than all of them produces the decorative-flag failure again.\n\n1. **Config flag** as a `[[bmod.config_questions]]` entry in `skills/bmod-tea/bmod.toml` with its default, and its install command and prerequisites in `skills/bmod-tea/help/integrations.md`.\n2. **CLI defaults** in `cli/lib/resolve-tea-config.js` (`MODULE_DEFAULTS`), which the test suite asserts stays equal to `bmod.toml`.\n3. **Knowledge fragments**: the per-utility reference fragments, plus a `\u003clibrary>-mandate.md` following the shape of the two that exist. Put them in `skills/bmod-tea/knowledge/` and index every one of them in its `tea-index.csv`.\n4. **A row in the registry table above.**\n5. **Loading**: each consuming workflow's context step loads the mandate FIRST, before the per-utility fragments, and states that it binds the run.\n6. **Generation**: every worker step that emits code in that library's scope carries the mandated template as the primary shape, the vanilla template as the flag-off branch, and the substitutions in its success and failure metrics.\n7. **Aggregation**: whatever shared file the mandated style requires (a merged-fixtures module, a support directory) is created by the aggregation step, and the deviation roll-up reaches the summary.\n8. **Review**: a `criteria-registry.md` row for \"configured utility bypassed\", gated on flag plus install, plus a published criterion row in `test-review-template.md`. Where partial migration is expected, a convention key in `step-02-discover-tests.md` and `cli/lib/convention-baseline.js` so adoption reads as a ratio rather than a pass or fail.\n9. **Docs**: the TEA documentation's configuration reference (what `true` actually means), its knowledge-base reference (the fragment rows and the used-in line), and a how-to under its customization section.\n10. **Changelog** under `[Unreleased]`.\n\n## Relationship to Principle Fragments\n\nA mandate never overrides a principle; it chooses the mechanism that expresses it.\n\n- `network-first.md` says intercept before you navigate. Under the Playwright Utils mandate the interception is `interceptNetworkCall`.\n- `fixture-architecture.md` says pure function, then fixture, then compose once. Under the mandate the composition is `mergeTests`.\n- `contract-testing.md` says the consumer's expectations must reflect what the provider actually returns. Under the Pact.js Utils mandate the matchers come from `zodToPactMatchers` or from provider-scrutinized values, not from hand-written helpers.\n\nWhen a flag is `false`, its principle fragments govern mechanism as well.\n\n## Related Fragments\n\n- `playwright-utils-mandate.md`, `pactjs-utils-mandate.md` — the per-library instances\n- `overview.md`, `pactjs-utils-overview.md` — installation and the utility inventories\n- `pact-mcp.md` — the broker capability and its degradation path\n- `confidence-gate.md` — stop and ask rather than invent an endpoint, selector, schema, or provider state\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":8258,"content_sha256":"c9f4f598fd2d7cdaa390cdbce4121416eccc03aab020d16c84e5075a0bbd0a2e"},{"filename":"knowledge/log.md","content":"# Log Utility\n\n## Principle\n\nUse structured logging that integrates with Playwright's test reports. Support object logging, test step decoration, and multiple log levels (info, step, success, warning, error, debug).\n\n## Rationale\n\nConsole.log in Playwright tests has limitations:\n\n- Not visible in HTML reports\n- No test step integration\n- No structured output\n- Lost in terminal noise during CI\n\nThe `log` utility provides:\n\n- **Report integration**: Logs appear in Playwright HTML reports\n- **Test step decoration**: `log.step()` creates collapsible steps in UI\n- **Object logging**: Automatically formats objects/arrays\n- **Multiple levels**: info, step, success, warning, error, debug\n- **Optional console**: Can disable console output but keep report logs\n\n## Quick Start\n\n```typescript\nimport { log } from '@seontechnologies/playwright-utils';\n\n// Basic logging\nawait log.info('Starting test');\nawait log.step('Test step shown in Playwright UI');\nawait log.success('Operation completed');\nawait log.warning('Something to note');\nawait log.error('Something went wrong');\nawait log.debug('Debug information');\n```\n\n## Pattern Examples\n\n### Example 1: Basic Logging Levels\n\n**Context**: Log different types of messages throughout test execution.\n\n**Implementation**:\n\n```typescript\nimport { log } from '@seontechnologies/playwright-utils';\n\ntest('logging demo', async ({ page }) => {\n await log.step('Navigate to login page');\n await page.goto('/login');\n\n await log.info('Entering credentials');\n await page.fill('#username', 'testuser');\n\n await log.success('Login successful');\n\n await log.warning('Rate limit approaching');\n\n await log.debug({ userId: '123', sessionId: 'abc' });\n\n // Errors still throw but get logged first\n try {\n await page.click('#nonexistent');\n } catch (error) {\n await log.error('Click failed', { console: false }); // suppress console output\n throw error;\n }\n});\n```\n\n**Key Points**:\n\n- `step()` creates collapsible steps in Playwright UI\n- `info()`, `success()`, `warning()` for different message types\n- `debug()` for detailed data (objects/arrays)\n- `error()` with optional console suppression\n- All logs appear in test reports\n\n### Example 2: Object and Array Logging\n\n**Context**: Log structured data for debugging without cluttering console.\n\n**Implementation**:\n\n```typescript\ntest('object logging', async ({ apiRequest }) => {\n const { body } = await apiRequest({\n method: 'GET',\n path: '/api/users',\n });\n\n // Log array of objects\n await log.debug(body); // Formatted as JSON in report\n\n // Log specific object\n await log.info({\n totalUsers: body.length,\n firstUser: body[0]?.name,\n timestamp: new Date().toISOString(),\n });\n\n // Complex nested structures\n await log.debug({\n request: {\n method: 'GET',\n path: '/api/users',\n timestamp: Date.now(),\n },\n response: {\n status: 200,\n body: body.slice(0, 3), // First 3 items\n },\n });\n});\n```\n\n**Key Points**:\n\n- Objects auto-formatted as pretty JSON\n- Arrays handled gracefully\n- Nested structures supported\n- All visible in Playwright report attachments\n\n### Example 3: Test Step Organization\n\n**Context**: Organize test execution into collapsible steps for better readability in reports.\n\n**Implementation**:\n\n```typescript\ntest('organized with steps', async ({ page, apiRequest }) => {\n await log.step('ARRANGE: Setup test data');\n const { body: user } = await apiRequest({\n method: 'POST',\n path: '/api/users',\n body: { name: 'Test User' },\n });\n\n await log.step('ACT: Perform user action');\n await page.goto(`/users/${user.id}`);\n await page.click('#edit');\n await page.fill('#name', 'Updated Name');\n await page.click('#save');\n\n await log.step('ASSERT: Verify changes');\n await expect(page.getByText('Updated Name')).toBeVisible();\n\n // In Playwright UI, each step is collapsible\n});\n```\n\n**Key Points**:\n\n- `log.step()` creates collapsible sections\n- Organize by Arrange-Act-Assert\n- Steps visible in Playwright trace viewer\n- Better debugging when tests fail\n\n### Example 4: Test Step Decorators\n\n**Context**: Create collapsible test steps in Playwright UI using decorators.\n\n**Page Object Methods with @methodTestStep:**\n\n```typescript\nimport { methodTestStep } from '@seontechnologies/playwright-utils';\n\nclass TodoPage {\n constructor(private page: Page) {\n this.name = 'TodoPage';\n }\n\n readonly name: string;\n\n @methodTestStep('Add todo item')\n async addTodo(text: string) {\n await log.info(`Adding todo: ${text}`);\n const newTodo = this.page.getByPlaceholder('What needs to be done?');\n await newTodo.fill(text);\n await newTodo.press('Enter');\n await log.step('step within a decorator');\n await log.success(`Added todo: ${text}`);\n }\n\n @methodTestStep('Get all todos')\n async getTodos() {\n await log.info('Getting all todos');\n return this.page.getByTestId('todo-title');\n }\n}\n```\n\n**Function Helpers with functionTestStep:**\n\n```typescript\nimport { functionTestStep } from '@seontechnologies/playwright-utils';\n\n// Define todo items for the test\nconst TODO_ITEMS = ['buy groceries', 'pay bills', 'schedule meeting'];\n\nconst createDefaultTodos = functionTestStep('Create default todos', async (page: Page) => {\n await log.info('Creating default todos');\n await log.step('step within a functionWrapper');\n const todoPage = new TodoPage(page);\n\n for (const item of TODO_ITEMS) {\n await todoPage.addTodo(item);\n }\n\n await log.success('Created all default todos');\n});\n\nconst checkNumberOfTodosInLocalStorage = functionTestStep('Check total todos count fn-step', async (page: Page, expected: number) => {\n await log.info(`Verifying todo count: ${expected}`);\n const result = await page.waitForFunction((e) => JSON.parse(localStorage['react-todos']).length === e, expected);\n await log.success(`Verified todo count: ${expected}`);\n return result;\n});\n```\n\n### Example 5: File Logging\n\n**Context**: Enable file logging for persistent logs.\n\n**Implementation**:\n\n```typescript\n// playwright/support/fixtures.ts\nimport { test as base } from '@playwright/test';\nimport { log, captureTestContext } from '@seontechnologies/playwright-utils';\n\n// Configure file logging globally\nlog.configure({\n fileLogging: {\n enabled: true,\n outputDir: 'playwright-logs/organized-logs',\n forceConsolidated: false, // One file per test\n },\n});\n\n// Extend base test with file logging context capture\nexport const test = base.extend({\n // Auto-capture test context for file logging\n autoTestContext: [\n async ({}, use, testInfo) => {\n captureTestContext(testInfo);\n await use(undefined);\n },\n { auto: true },\n ],\n});\n```\n\n### Example 6: Integration with Auth and API\n\n**Context**: Log authenticated API requests with tokens (safely).\n\n**Implementation**:\n\n```typescript\nimport { mergeTests } from '@playwright/test';\nimport { log } from '@seontechnologies/playwright-utils';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\n// Auth fixture built in your project (see auth-session.md: setAuthProvider + createAuthFixtures)\nimport { test as authFixture } from './support/auth/auth-fixture';\n\nconst test = mergeTests(authFixture, apiRequestFixture);\n\n// Helper to create safe token preview\nfunction createTokenPreview(token: string): string {\n if (!token || token.length \u003c 10) return '[invalid]';\n return `${token.slice(0, 6)}...${token.slice(-4)}`;\n}\n\ntest('should log auth flow', async ({ authToken, apiRequest }) => {\n await log.info(`Using token: ${createTokenPreview(authToken)}`);\n\n await log.step('Fetch protected resource');\n const { status, body } = await apiRequest({\n method: 'GET',\n path: '/api/protected',\n headers: { Authorization: `Bearer ${authToken}` },\n });\n\n await log.debug({\n status,\n bodyPreview: {\n id: body.id,\n recordCount: body.data?.length,\n },\n });\n\n await log.success('Protected resource accessed successfully');\n});\n```\n\n**Key Points**:\n\n- Never log full tokens (security risk)\n- Use preview functions for sensitive data\n- Combine with auth and API utilities\n- Log at appropriate detail level\n\n## Configuration\n\n**Defaults:** console logging enabled, file logging disabled.\n\n```typescript\n// Enable file logging in config\nlog.configure({\n console: true, // default\n fileLogging: {\n enabled: true,\n outputDir: 'playwright-logs',\n forceConsolidated: false, // One file per test\n },\n});\n\n// Per-test override\nawait log.info('Message', {\n console: { enabled: false },\n fileLogging: { enabled: true },\n});\n```\n\n### Environment Variables\n\n```bash\n# Disable all logging\nSILENT=true\n\n# Disable only file logging\nDISABLE_FILE_LOGS=true\n\n# Disable only console logging\nDISABLE_CONSOLE_LOGS=true\n```\n\n### Level Filtering\n\n```typescript\nlog.configure({\n level: 'warning', // Only warning, error levels will show\n});\n\n// Available levels (in priority order):\n// debug \u003c info \u003c step \u003c success \u003c warning \u003c error\n```\n\n### Sync Methods\n\nFor non-test contexts (global setup, utility functions):\n\n```typescript\n// Use sync methods when async/await isn't available\nlog.infoSync('Initializing configuration');\nlog.successSync('Environment configured');\nlog.errorSync('Setup failed');\n```\n\n## Log Levels Guide\n\n| Level | When to Use | Shows in Report | Shows in Console |\n| --------- | ----------------------------------- | ----------------- | ---------------- |\n| `step` | Test organization, major actions | Collapsible steps | Yes |\n| `info` | General information, state changes | Yes | Yes |\n| `success` | Successful operations | Yes | Yes |\n| `warning` | Non-critical issues, skipped checks | Yes | Yes |\n| `error` | Failures, exceptions | Yes | Configurable |\n| `debug` | Detailed data, objects | Yes (attached) | Configurable |\n\n## Comparison with console.log\n\n| console.log | log Utility |\n| ----------------------- | ------------------------- |\n| Not in reports | Appears in reports |\n| No test steps | Creates collapsible steps |\n| Manual JSON.stringify() | Auto-formats objects |\n| No log levels | 6 log levels |\n| Lost in CI output | Preserved in artifacts |\n\n## Related Fragments\n\n- `overview.md` - Basic usage and imports\n- `api-request.md` - Log API requests\n- `auth-session.md` - Log auth flow (safely)\n- `recurse.md` - Log polling progress\n\n## Anti-Patterns\n\n**DON'T log objects in steps:**\n\n```typescript\nawait log.step({ user: 'test', action: 'create' }); // Shows empty in UI\n```\n\n**DO use strings for steps, objects for debug:**\n\n```typescript\nawait log.step('Creating user: test'); // Readable in UI\nawait log.debug({ user: 'test', action: 'create' }); // Detailed data\n```\n\n**DON'T log sensitive data:**\n\n```typescript\nawait log.info(`Password: ${password}`); // Security risk!\nawait log.info(`Token: ${authToken}`); // Full token exposed!\n```\n\n**DO use previews or omit sensitive data:**\n\n```typescript\nawait log.info('User authenticated successfully'); // No sensitive data\nawait log.debug({ tokenPreview: token.slice(0, 6) + '...' });\n```\n\n**DON'T log excessively in loops:**\n\n```typescript\nfor (const item of items) {\n await log.info(`Processing ${item.id}`); // 100 log entries!\n}\n```\n\n**DO log summary or use debug level:**\n\n```typescript\nawait log.step(`Processing ${items.length} items`);\nawait log.debug({ itemIds: items.map((i) => i.id) }); // One log entry\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":11653,"content_sha256":"ef73615ce4535d4571567191ee9dc382cb87fd1eb75d3ee7e62f8f5dc9ca4c46"},{"filename":"knowledge/maestro-flows.md","content":"# Maestro Flow Patterns\n\n## Principle\n\nA Maestro flow is a declarative YAML sequence run against a real app on a simulator, emulator, or device. Flows must be **self-contained** (each starts from a known app state via `clearState`), **selector-resilient** (accessibility identifiers before visible text, visible text before index), and **assertion-bearing** (a flow that only taps and never asserts proves nothing).\n\n## Rationale\n\n**The Problem**: Mobile UI automation fails differently from browser automation. There is no DOM to query, no network layer to intercept from inside the test, and no single \"page loaded\" event. The common failure modes are index-based taps that break when a list reorders, hardcoded sleeps standing in for real synchronization, and flows that navigate through five screens without asserting anything, so they pass while the feature is broken.\n\n**The Solution**: Maestro already retries and waits on element lookup, so explicit sleeps are almost always a workaround for a missing assertion. Anchor every step on a stable identifier, assert the state you navigated to reach, and reset app state at the start of each flow rather than relying on the flow that ran before it.\n\n**Why This Matters**:\n\n- Flows survive UI reordering and copy changes (identifier-first selection)\n- Flows fail for the real reason instead of timing out three screens later\n- Flows can run in any order and in parallel (state isolation)\n- A green run means the behavior works, not that the taps landed somewhere\n\n## Pattern Examples\n\n### Example 1: Flow Structure and State Isolation\n\n**Context**: Every flow declares its app and resets state before the first interaction.\n\n**Implementation**:\n\n```yaml\n# maestro/login-success.yaml\nappId: com.example.app\nname: Login with valid credentials\ntags:\n - P0\n - auth\n---\n- clearState # isolation: no leftover session from a prior flow\n- clearKeychain\n- launchApp\n\n- assertVisible:\n id: 'login_screen_title'\n\n- tapOn:\n id: 'email_input'\n- inputText: '[email protected]'\n\n- tapOn:\n id: 'password_input'\n- inputText: '${MAESTRO_TEST_PASSWORD}' # never hardcode a credential\n\n- tapOn:\n id: 'login_submit_button'\n\n# assert the outcome, not just that the tap happened\n- assertVisible:\n id: 'home_dashboard'\n- assertVisible:\n text: 'Welcome back'\n```\n\n**Key points**:\n\n- `clearState` before `launchApp` makes the flow independent of execution order\n- `id` refers to the accessibility identifier (`testID` in React Native, `accessibilityIdentifier` on iOS, `resource-id` on Android)\n- The flow ends on an assertion about the destination state\n\n### Example 2: Selector Hierarchy\n\n**Context**: Choosing the most resilient way to address an element.\n\n**Implementation**:\n\n```yaml\n# ✅ Level 1: accessibility identifier (survives copy and layout changes)\n- tapOn:\n id: 'checkout_submit_button'\n\n# ✅ Level 2: visible text, when no identifier exists and the copy is stable\n- tapOn:\n text: 'Place order'\n\n# ✅ Level 3: text with a scoping container, for repeated labels\n- tapOn:\n text: 'Remove'\n below:\n text: 'Blue running shoes'\n\n# ⚠️ Level 4: regex, for dynamic content\n- assertVisible:\n text: 'Order #\\d+ confirmed'\n\n# ❌ Avoid: positional index breaks the moment the list reorders\n- tapOn:\n index: 2\n text: 'Item'\n\n# ❌ Avoid: absolute coordinates break on every other screen size\n- tapOn:\n point: '50%,73%'\n```\n\n**Rule**: `id` > `text` > scoped `text` (`below`/`above`/`leftOf`/`rightOf`/`containsChild`) > regex. Index and point coordinates are last resorts and must carry a comment explaining why nothing better exists.\n\n### Example 3: Synchronization Without Sleeps\n\n**Context**: Waiting for an async result (network call, animation, background job).\n\n**Implementation**:\n\n```yaml\n# ❌ Wrong: a fixed sleep is either flaky or slow, and usually both\n- tapOn:\n id: 'sync_button'\n- sleep: 5000\n- assertVisible:\n id: 'sync_complete_badge'\n\n# ✅ Right: wait for the condition that actually matters\n- tapOn:\n id: 'sync_button'\n- extendedWaitUntil:\n visible:\n id: 'sync_complete_badge'\n timeout: 30000\n\n# ✅ Right: assert the negative case explicitly\n- extendedWaitUntil:\n notVisible:\n id: 'loading_spinner'\n timeout: 10000\n- assertVisible:\n id: 'results_list'\n```\n\n**Key points**:\n\n- `extendedWaitUntil` with an explicit `timeout` states the real service-level expectation\n- Maestro's default element lookup already retries; a bare `sleep` on top of that hides the actual wait condition\n- A long timeout on a specific condition is honest. A long `sleep` is not.\n\n### Example 4: Composition and Reuse\n\n**Context**: Login is a precondition for a dozen flows and must not be copy-pasted.\n\n**Implementation**:\n\n```yaml\n# maestro/subflows/login.yaml\nappId: com.example.app\n---\n- tapOn:\n id: 'email_input'\n- inputText: ${EMAIL}\n- tapOn:\n id: 'password_input'\n- inputText: ${PASSWORD}\n- tapOn:\n id: 'login_submit_button'\n- assertVisible:\n id: 'home_dashboard'\n```\n\n```yaml\n# maestro/checkout-happy-path.yaml\nappId: com.example.app\nname: Checkout with a saved card\ntags:\n - P0\n - checkout\n---\n- clearState\n- launchApp\n- runFlow:\n file: subflows/login.yaml\n env:\n EMAIL: '[email protected]'\n PASSWORD: ${MAESTRO_TEST_PASSWORD}\n\n- tapOn:\n id: 'product_card_0'\n- tapOn:\n id: 'add_to_cart_button'\n- assertVisible:\n text: 'Added to cart'\n```\n\n**Key points**:\n\n- Subflows are the mobile equivalent of a fixture: one owner, many consumers\n- Pass data through `env` rather than baking values into the subflow\n- Keep subflows in a dedicated directory so a flow-count metric does not treat them as tests\n\n### Example 5: Conditional and Platform-Specific Steps\n\n**Context**: A permission dialog appears on a first run, and only on one platform.\n\n**Implementation**:\n\n```yaml\n- launchApp\n\n# Handle an optional dialog without failing when it is absent\n- runFlow:\n when:\n visible:\n id: 'com.android.permissioncontroller:id/permission_allow_button'\n commands:\n - tapOn:\n id: 'com.android.permissioncontroller:id/permission_allow_button'\n\n# Platform-specific branch\n- runFlow:\n when:\n platform: iOS\n commands:\n - tapOn: 'Allow While Using App'\n```\n\n**Key points**:\n\n- `runFlow: when:` is the supported way to express \"only if present\"\n- Do not wrap a genuinely required assertion in a `when:` guard; that converts a real failure into a silent skip\n\n### Example 6: Commands Whose Behavior Is Not What the Name Suggests\n\n**Context**: Four commands that read as cross-platform and are not, each of which has produced a flow that passed while proving nothing.\n\n**Implementation**:\n\n```yaml\n# ❌ `back` is documented for Android and Web only. The iOS driver's back\n# implementation is an empty method: it does nothing and still reports COMPLETED,\n# so a flow that pops a screen this way silently no-ops on one platform.\n- back\n- assertVisible:\n id: 'previous_screen_root'\n\n# ✅ Split it. Android gets the system gesture; iOS gets the app's own control.\n- runFlow:\n when:\n platform: Android\n commands:\n - back\n- runFlow:\n when:\n platform: iOS\n commands:\n - tapOn:\n id: 'nav_back_button'\n- assertVisible:\n id: 'previous_screen_root'\n```\n\n```yaml\n# ❌ On Android `hideKeyboard` is documented as identical to `back`, implemented\n# as the system back key event. In React Native that also fires a Modal's\n# onRequestClose, so this closes the dialog the flow is trying to fill in.\n- inputText: '[email protected]'\n- hideKeyboard\n- tapOn:\n id: 'dialog_submit_button' # gone; the modal was dismissed\n\n# ✅ The documented workaround, and it happens to be cross-platform: dismiss the\n# keyboard by tapping something that does not respond to taps.\n- inputText: '[email protected]'\n- tapOn:\n id: 'dialog_title' # non-interactive element\n- tapOn:\n id: 'dialog_submit_button'\n```\n\n```yaml\n# ❌ `index` counts CURRENTLY-RENDERED matches, not items in the underlying list.\n# Under React Native list virtualization the mapping drifts with scroll position\n# and viewport, so this taps a different row on a different device.\n- tapOn:\n text: 'Order .*'\n index: 3\n\n# ✅ Address the item by what makes it that item\n- tapOn:\n id: 'order_row_${ORDER_ID}'\n# or scope relationally when no id exists\n- tapOn:\n text: 'View'\n below:\n text: 'Order #10432'\n```\n\n```yaml\n# `point` is a sanctioned escape hatch for an element that is genuinely not in the\n# accessibility tree (a canvas, a map overlay, a video surface). The docs warn it\n# is device-dependent, and percentage coordinates rot as soon as the device\n# profile changes. Use it only with the reason written down.\n- tapOn:\n point: '50%,73%' # map canvas: no accessibility node exists for the pin\n```\n\n```yaml\n# `scrollUntilVisible` travels in ONE direction (default `DOWN`, 20s timeout, and a\n# 100% visibility threshold) and stops where it stopped. A later search for an\n# element ABOVE the current position keeps scrolling down and times out, which\n# reads as \"element missing\" rather than \"wrong direction\".\n- scrollUntilVisible:\n element:\n id: 'garment_row_9'\n- scrollUntilVisible:\n element:\n id: 'garment_row_1' # already scrolled past; this fails as if the row were gone\n\n# ✅ Name the direction, or return to a known position before searching again\n- scrollUntilVisible:\n element:\n id: 'garment_row_1'\n direction: UP\n```\n\n**Waiting, precisely**:\n\n- Assertions carry a **default timeout of about 7 seconds**. `extendedWaitUntil` is the sanctioned way to ask for longer, and it names the condition while doing so.\n- **There is no \"wait until the app is ready\" command.** The documented idiom is to wait on the first piece of the app's own chrome that paints. A flow that waits on a launch marker the app never renders will wait for the full timeout and then fail somewhere unrelated.\n- `waitForAnimationToEnd` defaults to a 15-second cap and **succeeds when that cap is reached**, so a bare use of it cannot fail the flow. That makes it safe as a settle step and useless as an assertion.\n- Wrapping a large part of a flow in `retry` is an anti-pattern in the documentation's own words, and the retry count is capped at a small number. Retry a genuinely nondeterministic step, never a journey.\n\n**Key points**:\n\n- Before relying on a command across platforms, check its documented platform support; \"runs without error\" is not the same claim as \"did something\"\n- A command that reports COMPLETED while doing nothing turns every assertion downstream of it into decoration\n- Prefer commands whose failure is observable over commands whose success is unconditional\n\n### Example 7: `text:` Selectors Are Regular Expressions\n\n**Context**: An assertion that reads exactly like the label on screen and could never have matched it.\n\n**Implementation**:\n\n```yaml\n# ❌ Every `text:` value is a regex, and it must match the element's ENTIRE text.\n# The parentheses here are a capture group, so this pattern demands the literal\n# string `Garments 2 of \u003canything> selected`, while the label on screen reads\n# `Garments (2 of 10 selected)`. It could not have matched at any selection count.\n- assertVisible:\n text: 'Garments (2 of .* selected)'\n\n# ✅ Escape the characters that are literal\n- assertVisible:\n text: 'Garments \\(2 of .* selected\\)'\n\n# ✅ Or match on the stable part, padded to cover the whole element\n- assertVisible:\n text: '.*2 of 10 selected.*'\n```\n\n**Key points**:\n\n- `(`, `)`, `[`, `]`, `.`, `*`, `+`, `?`, `|`, `{`, `}`, `^`, ` bmod-tea — Skillopedia are all pattern syntax inside a `text:` value. Any label containing one needs escaping.\n- Matching is against the **entire** element text, so a substring that is plainly on screen still fails without `.*` on both sides.\n- The defect is invisible in review, because the YAML reads as the sentence a human sees on screen. Add the pattern to whatever lint the repository can host, because human review is demonstrably not the control that catches it.\n- **Which direction it breaks depends on the command.** In `assertVisible` the impossible pattern fails after the default timeout and reads as a missing element, sending the diagnosis at the app rather than at the selector. In `assertNotVisible` it passes unconditionally, which is a check that cannot fail: see `evidence-integrity.md`.\n\n### Example 8: A COMPLETED Tap Is Not a Handled Tap\n\n**Context**: A checkbox in a virtualized list. Maestro reported the tap COMPLETED and the app never saw it.\n\nMeasured in isolation: the tap completed in 2.4 seconds, the target was present at `[45,1807][1035,1924]` with `clickable=true`, nothing sat above it in the hierarchy at the point tapped, and the count label still read `0 of 10` ten seconds later. `tapOn` reports COMPLETED once it has resolved the element and dispatched a touch, so its status describes the driver's action and not the app's response. One passing run recorded five taps for two selections, which makes the loss frequent rather than exotic.\n\n**The mechanism is not established.** Those measurements rule out a missing element and an occluding overlay; they do not say where between the driver and the app's handler the touch went. Gesture-responder interaction with a list that has visually stopped moving is a plausible candidate and has not been tested. Treat that as an open question rather than a cause, and treat the pattern below as what it is: a countermeasure that keeps the flow honest while the cause is unknown, per the `evidence-integrity.md` rule that a stated mechanism is a claim needing a source.\n\n**Implementation**:\n\n```yaml\n# ❌ The tap's status is not evidence that the state changed, and a single\n# assertion at the end cannot say WHICH tap was lost.\n- tapOn:\n id: 'garment_checkbox_0'\n- tapOn:\n id: 'garment_checkbox_1'\n- assertVisible:\n text: '.*2 of 10 selected.*'\n\n# ✅ Pair each tap with the assertion that proves it landed, inside a small retry.\n# The assertion is a real one, so a genuinely broken selection still fails.\n- retry:\n maxRetries: 3\n commands:\n - tapOn:\n id: 'garment_checkbox_0'\n - assertVisible:\n text: '.*1 of 10 selected.*'\n- retry:\n maxRetries: 3\n commands:\n - tapOn:\n id: 'garment_checkbox_1'\n - assertVisible:\n text: '.*2 of 10 selected.*'\n```\n\n**Key points**:\n\n- `retry` takes `maxRetries` between `0` and `3`, defaulting to `1`. Retrying one nondeterministic step is the sanctioned use; the documentation calls wrapping a large part of a flow an anti-pattern, and wrapping the entire flow explicitly unpredictable.\n- **Assert per action, not once at the end.** An end-of-sequence assertion cannot name which tap was lost, and that ambiguity is what produces a confident wrong first hypothesis.\n- The retry does not weaken the check. The assertion inside it has to pass on its own, so what the retry absorbs is a lost touch, which is a property of the driver rather than of the app.\n- If a step routinely needs its retry to land, that is a finding about the driver or the list, not an ordinary step. Record it rather than letting the retry hide it.\n\n### Example 9: Visible Means Inside the Viewport, Not Present in the Hierarchy\n\n**Context**: An element that is rendered, correct, and below the fold. `assertVisible` fails on it.\n\n`assertVisible` and `extendedWaitUntil: visible` require the element to be **on screen**, not merely present in the view tree. An element in a scrolling section that has not been scrolled to is present, correct, and not visible, and the failure reads as \"the feature is broken\" rather than \"the flow never scrolled\".\n\n**Implementation**:\n\n```yaml\n# ❌ Latent screen-height dependency: passes on a tall device, fails on a short one\n- extendedWaitUntil:\n visible:\n id: 'premium_unavailable'\n timeout: 20000\n\n# ✅ Scroll to what you assert on\n- scrollUntilVisible:\n element:\n id: 'premium_unavailable'\n- assertVisible:\n id: 'premium_unavailable'\n```\n\n**Key points**:\n\n- **The diagnostic tell is cheap and decisive.** Dump the failing step's `screen-hierarchy` entry and look for the id. Present with bounds outside the screen is a scroll problem; absent is a different bug entirely. Checking this first is worth more than any hypothesis about the feature, and one investigation built three separate plausible mechanisms for such a failure (a remote flag answer, an unresolved dynamic import, a stuck initialization call) before two measurements killed all three and left the viewport.\n- **Any `assertVisible` on an element inside a scrolling section is a screen-height dependency** until the flow scrolls to it. It will pass wherever it was written and fail on the first shorter device.\n- This is the failure mode most likely to reproduce only in CI, because the runner's device profile is rarely the one the flow was written against. `mobile-ci-device-lab.md` carries the profile-parity rule.\n\n### Example 10: Assert the Change, Not a State That May Already Hold\n\n**Context**: A deep-link flow named \"Widget Deep Link Hydration\" that opened a link and asserted the home screen's own container was visible. The container was there before the link, so the assertion held whether or not the link did anything. On one platform it did nothing, and the suite reported 18 of 18.\n\nTwo shapes are stacked here, and the second one catches people who have already fixed the first.\n\n**Shape one: the assertion targets something that predates the action.** A container, a screen root, or a nav bar that was on screen before the step cannot be evidence that the step worked. The tell is that the flow's name describes an effect the assertions never mention.\n\n**Shape two: the assertion targets the right thing, in a state the app might already be in.** Asserting \"morning is selected\" after a link that selects morning passes whenever morning is the default. The check is about the action but still cannot fail.\n\n**Implementation**:\n\n```yaml\n# ❌ Both shapes. The container predates the link, and even a corrected\n# assertion on the default scenario would pass without the link doing anything.\n- openLink: ${WIDGET_URL_AM}\n- assertVisible:\n id: 'scenario-toggles' # already on screen before the link\n\n# ✅ Assert the transition. State the precondition, act, then assert both that\n# the new state holds and that the old one no longer does. Every link in the\n# flow has to earn its own pass this way, including the first.\n- assertVisible:\n id: 'scenario-toggle-morning'\n selected: true # PRECONDITION, not the outcome: names the state moved FROM\n\n- openLink: ${WIDGET_URL_EVENING}\n- assertVisible:\n id: 'scenario-toggle-evening'\n selected: true\n- assertNotVisible:\n id: 'scenario-toggle-morning'\n selected: true # the move is what proves this link was applied\n\n- openLink: ${WIDGET_URL_AM}\n- assertVisible:\n id: 'scenario-toggle-morning'\n selected: true\n- assertNotVisible:\n id: 'scenario-toggle-evening'\n selected: true # and the second link proves itself the same way\n```\n\n**Key points**:\n\n- **Prefer asserting a change over asserting a state.** Where a single state is all you have, pick an input whose expected result differs from the app's default, so agreement with the default cannot carry the pass.\n- **A precondition assertion is legitimate, and it is not the outcome.** Naming the state being moved from is what makes the following change meaningful, so mark it as a precondition and never let it stand in for proof that the action worked. Ordering the links so the first one also moves the selection is what stops the first step passing on the default, which is the trap the corrected block above is arranged to avoid.\n- **Choose a deterministic input.** The same flow had link variants resolving from the current time and the day's forecast, which cannot be asserted without freezing the clock. Two other variants mapped fixed values straight through the same parse-route-apply path. Reach for the deterministic input rather than reaching for a clock stub: it exercises the identical code path and needs no test-only seam.\n- `selected`, `checked`, `enabled`, and `focused` are documented state selectors and compose with `id` and `text` on `tapOn`, `assertVisible`, and `assertNotVisible`. `assertNotVisible` with a state selector is how \"no longer selected\" is expressed.\n- **Check the syntax before the run.** `maestro check-syntax` validates flow files without a device, which is the cheap way to confirm a selector or field exists on the version you pin rather than discovering it in a red run.\n\n## Anti-Patterns\n\n| Anti-pattern | Why it fails | Fix |\n| ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |\n| Flow with no `assertVisible`/`assertTrue` | Passes as long as taps land; proves nothing about behavior | Assert the destination state of every flow |\n| `sleep` used as synchronization | Flaky under load, slow when not | `extendedWaitUntil` on the real condition |\n| `tapOn: index:` on a list | Breaks when the list reorders or the backend returns a different order | Scope by `text` with `below`/`containsChild` |\n| `tapOn: point:` coordinates | Breaks on a different screen size or density | Address the element by `id` |\n| No `clearState` | Flow depends on whatever ran before it; unreproducible in isolation | `clearState` before `launchApp` |\n| Hardcoded credential or PII | Leaks in the repo and in CI logs | `${ENV_VAR}` sourced from the CI secret store |\n| One flow covering six user journeys | A failure names the flow, not the behavior; slow to diagnose | One journey per flow, composed from subflows |\n| Required assertion inside `when:` | Turns a real failure into a silent pass | Guard only genuinely optional UI (permission dialogs, upsells) |\n| `back` used as a cross-platform step | Documented for Android and Web only; on iOS it does nothing and reports COMPLETED | Split with `runFlow: when: platform:`; tap the app's own control on iOS |\n| `hideKeyboard` with a modal open | The Android implementation is the system back key, which dismisses a React Native modal | Tap a non-interactive element to drop the keyboard |\n| `optional: true` on the assertion that carries the outcome | The step cannot fail, so the flow reports coverage it does not have | Assert hard; reserve `optional` for genuinely optional UI |\n| `waitForAnimationToEnd` used as a wait-for-content | It succeeds when its cap is reached, so it cannot fail | `extendedWaitUntil` on the content that must appear |\n| Unescaped regex characters in a `text:` selector | The value is a regex matched against the element's entire text, so it can never pass | Escape literal `(`, `)`, `[`, `]`, `.`; pad partial matches with `.*` |\n| Tap status treated as proof the app handled the tap | `tapOn` reports COMPLETED once the touch is dispatched, and touches do get lost | Pair each state-changing tap with its own assertion inside a `retry` |\n| One assertion at the end of a tap sequence | Cannot name which tap was lost, so the first hypothesis is a guess | Assert after every action that changes state |\n| `scrollUntilVisible` reused after an earlier search moved the list | It travels only in the direction given, so the second search scrolls away from the target | Name the opposite `direction`, or return to a known position first |\n| `assertVisible` on an element inside a scrolling section, with no scroll | Visible means inside the viewport; the element is present, correct, and below the fold | `scrollUntilVisible` first, then assert |\n| Assertion targets a container that predates the action | It was on screen before the step, so it cannot be evidence the step worked | Assert the action's own effect; the flow's name should name what it asserts |\n| Assertion on a state the app may already be in | Passes whenever the expected value is the default, so the action is not under test | Assert the transition, or pick an input whose expected state differs from the default |\n| Time- or forecast-dependent input in a flow assertion | The expected value cannot be stated without freezing the clock | Pick a deterministic input through the same code path |\n\n## Maestro Flow Checklist\n\nBefore merging a flow:\n\n- [ ] **Isolated**: starts with `clearState` (or documents why it must not)\n- [ ] **Asserts an outcome**: at least one `assertVisible`/`assertNotVisible`/`assertTrue` about the destination state\n- [ ] **Identifier-first selectors**: `id` used wherever an accessibility identifier exists\n- [ ] **No positional selection**: no bare `index:` or `point:` without a comment justifying it\n- [ ] **No `sleep` as synchronization**: waits are `extendedWaitUntil` on a named condition with an explicit timeout\n- [ ] **No secrets in the file**: credentials and tokens come from `${ENV}`\n- [ ] **Single journey**: one user-visible outcome per flow, shared setup extracted to a subflow\n- [ ] **Tagged by priority**: `P0`-`P3` tag present so CI can run the risk-appropriate subset\n- [ ] **Runs on both target platforms**, or declares its platform branch explicitly\n- [ ] **Cross-platform commands verified**: every command used on both platforms is documented for both, or split by `runFlow: when: platform:`\n- [ ] **Every assertion can fail**: no `optional: true` on the assertion that carries the flow's outcome, and no assertion sitting downstream of a command that no-ops on that platform\n- [ ] **`text:` selectors read as regex**: literal `(`, `)`, `[`, `]`, `.` escaped, and whole-element matching accounted for\n- [ ] **Every state-changing tap has its own assertion**, rather than one assertion covering a sequence\n- [ ] **`retry` scoped to a single step**, with `maxRetries` inside the documented 0-3 range\n- [ ] **Nothing asserted below the fold**: every assertion on an element inside a scrolling section is preceded by a scroll to it\n- [ ] **The assertion carrying the outcome post-dates its action**: a precondition assertion is allowed when it is labelled as one, and nothing already true before the step is presented as proof of it\n- [ ] **Transitions asserted where possible**: a single-state assertion is justified only when the expected value differs from the default\n- [ ] **Syntax checked**: `maestro check-syntax` run against the pinned version before the flow reaches a device\n\n## Integration Points\n\n- **Used in workflows**: `*framework` (scaffold a Maestro suite), `*automate` (generate flows), `*atdd` (red-phase mobile acceptance flows), `*test-review` (score flow quality), `*ci` (device pipeline)\n- **Related fragments**: `mobile-test-strategy.md` (what belongs in a flow at all), `mobile-ci-device-lab.md` (the build artifact the flows run against, and the CI mechanics around them), `evidence-integrity.md` (why a step that cannot fail is the most expensive defect in a suite), `test-priorities-matrix.md` (P0-P3 tagging), `test-quality.md` (determinism and isolation standards), `selector-resilience.md` (the browser analogue of the selector hierarchy)\n- **Tools**: `maestro test`, `maestro studio` (interactive flow authoring and element inspection), `maestro record`\n\n_Source: Maestro 2.8.0 flow syntax and command reference (selectors, `retry`, `scrollUntilVisible`), mobile test-isolation practice, TEA test-quality standards applied to declarative flows, and defects measured in a live Maestro suite_\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":29447,"content_sha256":"02b1fff0e463708db474310792fd88a98dc425c621bac9cb3b7b7f1fbd47c3d3"},{"filename":"knowledge/mobile-ci-device-lab.md","content":"# Mobile Device Lab in CI\n\n## Principle\n\nBefore a single flow is written, decide **which artifact the flows run against**. That one decision fixes the failure surface of the whole suite: a compiled build is a file the runner installs, while a development-client shell served by a live dev server adds a metro process, a manifest HTTP exchange, and a bundle download to every launch. Everything else in a device lab (emulator snapshots, version pinning, artifact layout, sharding) is mechanical once the artifact is right, and unfixable while it is wrong.\n\n## Rationale\n\n**The Problem**: Mobile CI failures are usually attributed to \"flaky emulators.\" Most of them are not. They are a launch path that only exists in CI, a snapshot that silently never restores, a runner version eight releases from local, or a diagnosis read off the wrong artifact. Teams then add retries, which converts a reproducible configuration defect into an intermittent one.\n\n**The Solution**: Ship the app as a build artifact so the launch path in CI is the launch path users get. Make the emulator boot from a cached snapshot and prove that it did. Pin the runner version and assert the resolved version. Read the failure out of the hierarchy dump instead of the screenshot.\n\n**Why This Matters**:\n\n- Flows exercise the shipped app instead of a development shell\n- Emulator boot drops from tens of seconds to a few, and the saving is verifiable\n- Failures name the step that broke and what was on screen when it broke\n- CI-only failure modes stop being written into flow files as workarounds\n\n## The Build Artifact Decision\n\n| Artifact | What it proves | What it costs | Use for |\n| ------------------------------------------------------ | ---------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------ |\n| **Release-shaped build** (unsigned APK, simulator IPA) | The binary users get, including all native modules | A build step per change, or a cached build keyed on native inputs | The default for every CI suite |\n| **Development build / dev client** | The app with dev tooling, all native modules present | A build step, plus a dev server when the JS bundle is served rather than embedded | Local iteration, debug flows |\n| **Prebuilt development shell (for example Expo Go)** | That the JS runs inside someone else's container | A live dev server, a manifest exchange, and a launch through a third-party app | Manual smoke work only |\n\n**Rule**: the prebuilt shell is the wrong artifact for E2E. It cannot load your native modules, so for any flow touching notifications, OAuth, maps, in-app purchases, or any feature that hands an API key to native code, **a pass in the shell does not prove the native implementation ran**. It proves that whatever the app did in the shell's absence of that module did not throw. Where the module has a fallback path, the flow is exercising the fallback, and the native behavior stays unverified no matter how green the run is. Require evidence that the native path itself was exercised before counting such a flow as coverage, which in practice means running it against a real build.\n\n**Be precise about which part is unreachable, because the coarse version of this rule is wrong and gets flows abandoned that would pass.** Deep links are the case worth stating carefully. A shell cannot register the app's custom scheme, so a `myapp://` URL fails there. It can still route its own URL form into the app with the path and query intact, so the app's link parsing, routing, and resulting state changes are all testable. Only the OS-level handoff is out of reach: a cold start from a real widget or notification tap. One suite recorded a deep-link flow as impossible under the shell for exactly this reason, and the flow passed on the first attempt once the URL was built the shell's way. \"Untestable here\" is a claim that needs the same evidence as any other.\n\n**\"Absent\" is not always the shape it takes, and the other shape is worse.** Some SDKs detect the shell and degrade instead of failing. One in-app-purchase SDK logs `Expo Go app detected. Using RevenueCat in Browser Mode.` and keeps working through a different code path. Nothing errors, the flow proceeds, and what the suite proves is that the fallback path works in a container users never run. A hard `undefined` at least announces itself; a silent degradation gives you a green flow covering the wrong implementation. When a flow touches a native module in a shell, check the device log for what the module decided to do rather than assuming it did nothing.\n\nExpo's own CI tutorial builds a dedicated EAS profile for this (`e2e-test`, with `withoutCredentials: true`, Android `buildType: \"apk\"`, and iOS `simulator: true`) and runs Maestro against those builds. It never runs the flows through Expo Go. See \u003chttps://docs.expo.dev/tutorial/cicd/e2e-tests/> and \u003chttps://expo.dev/blog/expo-go-vs-development-builds>.\n\nThe cost of getting this wrong is measurable in flow source. In one audit, about 120 of 195 lines in a single launch subflow existed solely to fight the development shell (dev-server readiness, manifest retries, a third-party app's own UI), and most of the defects fixed that week would not have existed against a compiled build. Workarounds for a wrong artifact do not stay in the harness; they migrate into the flows and become the suite.\n\n### \"Development Build\" Is Not Automatically the Fix\n\nA development build is the usual proposal once the shell is ruled out, and on Android it frequently changes nothing. In EAS, `developmentClient: true` sets the Gradle task to `:app:assembleDebug`, and a **debug variant does not embed the JS bundle**. The app still needs a live packager and a manifest exchange at launch, which is the same CI-only network surface the shell had. Only a **release** variant embeds the bundle. SDK 54's `debugOptimized` is the near miss worth naming: it optimizes the C++ layer and remains a debug variant, so the bundle is still served rather than embedded.\n\nGetting to a release-variant APK without an account or a build service:\n\n- **`eas build --local` composes badly with CI caching.** Expo documents \"Caching is not supported\" for local builds, and they still require `eas login` or an `EXPO_TOKEN`. `npx expo prebuild` followed by `./gradlew :app:assembleRelease` is the path that caches. See \u003chttps://docs.expo.dev/build-reference/local-builds/>.\n- **A locally prebuilt release APK is debug-signed.** The generated `android/app/build.gradle` sets `release { signingConfig signingConfigs.debug }`, so the artifact installs on an emulator with no credentials, which is exactly what a device lab needs and not something to \"fix\".\n- **`__DEV__` is `false` in a release build.** Any E2E affordance gated behind it silently disappears in the one build the suite is meant to run against. Move the switch to an `EXPO_PUBLIC_`-prefixed variable, which is inlined into the bundle at build time. Expo documents these as \"visible in plain-text in your compiled application\", so whatever the switch gates must be safe to ship: a throwaway credential against a disposable environment, never a real one.\n- **Custom-scheme deep links need a different URL in the shell, and that is a constraint rather than a hole.** `scheme` is documented as \"a build-time configuration, it has no effect in Expo Go\", so the shell never registers the app's own scheme and a `myapp://` link fails there with `Activity not started, unable to resolve Intent`. What works is the shell's routed form, `exp://\u003chost>/--/\u003cpath>?\u003cquery>`, which delivers the path and query string into the app. The parsing, routing, and application logic behind a deep link is therefore fully reachable in a shell. What is not reachable is anything that needs the production scheme registered with the OS: a cold start from a real widget or notification tap, or another app handing the link over. Build the URL from one helper used by every flow, so no single platform branch quietly hardcodes the unregistered scheme.\n\n Universal Links and Android App Links are a different mechanism and get no such reprieve. They are HTTPS links resolved through a domain association (`apple-app-site-association`, `assetlinks.json`) that the OS fetches and verifies, so the shell's routed form exercises the in-app routing and says nothing about whether the handoff would have happened. Verify the association against a real build, and treat routing coverage and handoff coverage as two separate claims.\n\n## If the Suite Must Run Against a Dev Server\n\nSometimes the compiled build is not ready yet and the dev-server path has to work for one release. Treat it as a temporary configuration with these constraints:\n\n- **Reach the host over the debug bridge, not the guest NIC.** `adb reverse tcp:8081 tcp:8081` tunnels over the adb transport, so it survives emulator network breakage that would kill a `10.0.2.2` route. Pin the device with `adb -s \u003cserial>` when more than one is attached.\n- **Do not verify the forward by connecting to it from the device.** A reverse mapping gives the device a local listener on that port unconditionally, so the connect succeeds whether or not anything on the host is behind it. That probe measures that the mapping exists and reports that the server is reachable. Prove it from the host process instead: accept a socket and assert that the accept happened.\n- **Do not assume the toolchain set it up.** Expo CLI issues `adb reverse` from the path where the CLI itself opens the app. Start the server without that flag and the forward silently never happens.\n- **The shell may be asking for a SIGNED manifest, and signing needs an account.** Expo Go sends `expo-expect-signature` with `keyid=\"expo-root\"`. When the app config carries `extra.eas.projectId`, `@expo/cli` answers by fetching a development code-signing certificate from Expo's API and caching it under `~/.expo/codesigning/\u003cprojectId>`. That fetch resolves the current user, and with no session it prompts; under `EXPO_NO_INTERACTIVE=1` the prompt cannot be answered and the request dies with `CommandError: Input is required, but 'npx expo' is in non-interactive mode.` A developer machine never sees this, because `~/.expo/state.json` holds a session and the certificate is already cached. A fresh runner has neither, which is why the failure is CI-only and survives every emulator, image, and network change tried against it. **The trigger is `extra.eas.projectId`, not `owner`**; removing `owner` changes nothing. Two fixes work: an `EXPO_TOKEN` secret (Expo's documented CI authentication), or starting the server `--offline`, which skips the network requests behind the signing path and serves an unsigned manifest, which the shell accepts. Apply whichever you choose on every path, local and CI, so the two do not diverge on the one axis that only breaks in CI.\n- **Health-check the manifest the way the client asks for it.** A bare `GET /` with no headers returns `200` and a browser interstitial, so a harness can log \"dev server reachable\" while every client request fails. Send **every** header the client sends (the platform header, `accept: multipart/mixed`, and the signature-expectation header above), and log the response body: the CLI serializes manifest-path errors as a JSON `error` payload with status `500`. One harness omitted only the signature header and reported \"manifest served, HTTP 200, multipart/mixed\" through five consecutive red runs, because that one header selects a different branch through the middleware than the app under test takes. See `evidence-integrity.md`: a probe must issue the request it stands in for.\n- **Read the discriminating log line.** `Remote update request not successful` is emitted at exactly one place in `expo-updates`, guarded by the HTTP client's 200-299 check. If it appears, an HTTP response arrived with an error status, which makes it a manifest or HTTP problem and rules out connectivity. The surrounding generic lines (`Failed to download remote update`, `Failed to launch embedded or launchable update`) appear for any failure including connection-refused, so only the specific line carries information. Source: `packages/expo-updates/android/src/main/java/expo/modules/updates/loader/FileDownloader.kt` in \u003chttps://github.com/expo/expo>.\n- **Expect the app config to be evaluated per request.** The manifest handler re-reads the project config on every manifest request, so config plugins run per request. Anything environment-sensitive in that config is a live macOS-versus-Linux divergence axis.\n- **Do not build on undocumented packager host variables.** They carry a \"drop the undocumented env variables\" note upstream, and setting one can break a working `adb reverse` plus loopback setup by advertising a different host back to the client.\n\n## Local and CI Run the Same Device Profile\n\nPin one device profile and use it on both sides. A different profile is a different layout, and a different layout is a class of failure that reproduces nowhere but CI.\n\nMeasured: CI booted a `pixel_3a` (1080x2220 at 440dpi, roughly 807dp tall) while every local run used a `medium_phone` (1080x2400 at 420dpi, roughly 914dp). A screen about 12% shorter pushes more of each scrolling section below the fold, and four unrelated flows failed on the runner with \"not visible\" while staying green on every developer machine. Each one read like a product defect. None was.\n\nThe height that matters is **density-independent pixels**, not the pixel resolution, because the layout is laid out in dp. Two profiles with the same `1080x` resolution and different densities are different screens, and comparing the resolutions alone will tell you they match.\n\n- Name the profile in the harness, not in a person's local setup, so both sides read the same value.\n- When the matrix genuinely needs more than one profile, keep the PR-gate profile identical to the local one and let the extra profiles run nightly, where a difference is information rather than noise.\n- A suite running two profiles cannot distinguish a real regression from a screen-height artifact, and the artifact is far more common. That ambiguity costs more than the coverage the second profile adds at the gate.\n\n## Local Emulators Need Repair After Creation\n\n`avdmanager create avd` does not hand back a device a UI driver can use. Three defects, all measured on an Apple Silicon host, all needing a post-creation edit:\n\n- **`target=android-0` in the AVD's `.ini` pointer file.** `avdmanager` cannot parse a dotted API level, so a system image such as `system-images;android-36.1;google_apis;arm64-v8a` writes `target=android-0`. The emulator cannot resolve the platform, silently drops hardware acceleration (`hvf is not enabled on this aarch64 host`, then `qemu_mprotect__osdep: mprotect failed: Permission denied`) and software-emulates ARM64 on an ARM64 host. The device never leaves `offline` in `adb devices`, with nothing in the log naming the cause. The same parse failure leaves `avd.id` and `avd.name` as the literal string `\u003cbuild>`. Correcting `target` makes the identical AVD boot with no acceleration warnings.\n- **`hw.gpu.enabled=no`**, which leaves gfxstream logging `Failed to make display surface context current` and boot never completing.\n- **`hw.keyboard=no`**, wrong for any suite that types through `adb`.\n\nTwo launch flags worth pinning while you are there:\n\n- **`-gpu auto`, not `-gpu swiftshader_indirect`.** Software rendering is fine for a single emulator and does not survive several at once on the same host.\n- **`-no-snapshot-save`, not `-no-snapshot`.** The latter also refuses to LOAD a snapshot, which makes every boot cold and quietly undoes the caching work below.\n\n**Read the values back and fail on a mismatch.** A creation script that writes the right lines and never checks them produces exactly the failure this section describes: an AVD that looks configured, boots to `offline`, and costs a full run to diagnose.\n\n## Android Emulator on Hosted Runners\n\nUsing `reactivecircus/android-emulator-runner`:\n\n- **The `script:` input is not a shell script.** It is trimmed, split on newlines, and each surviving line is executed as its own `sh -c` invocation. `set -euo pipefail` therefore dies on line one under `dash` and, more importantly, applies to nothing after it. Variables, `cd`, functions, multi-line `if`/`for`, and heredocs do not survive between lines. The working pattern is a single line: `script: bash ./scripts/ci-e2e.sh`.\n- **Snapshot caching is a four-step recipe.** Restore the cache, run the action once with a no-op `script:` to create the AVD and save a boot snapshot, save the cache, then run the real test step with `-no-snapshot-save`. Use the split `actions/cache/restore` plus `actions/cache/save` form: a combined `actions/cache` step saves in a post-step gated on success, so a red run never saves what it just built.\n- **Pass hardware inputs on the creation step only.** The action appends `hw.ramSize`, `disk.dataPartition.size`, `hw.cpu.ncore`, and friends to `config.ini` with `>>` on **every** invocation, outside the guard that decides whether to create the AVD. The emulator normalizes those values when it writes the snapshot, so a re-appended literal no longer matches what the snapshot recorded and the snapshot is rejected at boot with `cannot load snapshot: default_boot` and `Reason: different AVD configuration`. Passing identical inputs to both steps does not fix it, because the mismatch is normalized-versus-literal, not step-versus-step. Verified effect of the fix: snapshot restore in single-digit seconds against a roughly 40-second cold boot.\n- **Put an image version component in the cache key.** Key on API level, target, arch, and the system-image or runner-image version. Without it a runner-image bump invalidates the snapshot while the key still hits, producing permanent cold boots with no signal that anything changed.\n- **Leave hardware acceleration on.** The KVM udev rule plus `disable-linux-hw-accel: auto` is the single largest lever on boot time (seconds versus minutes). Check it before optimizing anything else.\n- **Do not use ATD images for UI-driver suites.** The automated-test-device variants strip SystemUI, the launcher, and the IME, and disable hardware rendering. A UI driver needs exactly those. The gain is roughly a fifth of runtime and it is not worth a suite that cannot see the system UI.\n- **Treat a known-bad base image as a hypothesis to falsify, never as a diagnosis.** Specific API levels do go bad on hosted runners for months at a time, with open reports of no network connectivity or a system-UI ANR that holds window focus, so the tracker is worth reading before pinning an older level. It is not worth believing on a symptom match. One investigation adopted a reported no-network defect as its root cause on the strength of a false-negative probe, bumped the API level on that basis, and reproduced the identical failure on the new level. Changing the image is a test of the hypothesis, and a green run is the only thing that confirms it.\n\n## Per-Device Identity for Sharded Runs\n\nWhen flows create and delete data for the signed-in user, each parallel device needs its own fixture account, and the app has to know which account is its own. The usual mechanism is a device-name-to-credential map in the bundle, with the app selecting its entry by reading its own device name.\n\nOn iOS the simulator name is a device property, so the app reads it. **On Android there is nothing to read.** `expo-device`'s `deviceName` resolves `Settings.Global.DEVICE_NAME` on API 32 and above, and the `bluetooth_name` secure setting below that; on an emulator both default to the product model. Measured: an AVD named `Medium_Phone_API_36.1` reports `sdk_gphone64_arm64`. **Four differently-named AVDs produce four identical map keys.** Every shard then signs in as the same user, the shards delete each other's data mid-flow, and every flow still passes. This is the most expensive shape of hollow green in this fragment, because the green is stable and what it hides is a data race.\n\nThe identity has to be **written** per device before the run, then proven:\n\n```bash\n# The API level picks the namespace. Writing the wrong one succeeds and changes\n# nothing the app can read, which is a silent version of the same defect.\napi=$(adb -s \"$serial\" shell getprop ro.build.version.sdk | tr -d '\\r')\nif [ \"$api\" -ge 32 ]; then\n adb -s \"$serial\" shell settings put global device_name \"$name\"\n read_back=$(adb -s \"$serial\" shell settings get global device_name | tr -d '\\r')\nelse\n adb -s \"$serial\" shell settings put secure bluetooth_name \"$name\"\n read_back=$(adb -s \"$serial\" shell settings get secure bluetooth_name | tr -d '\\r')\nfi\n[ \"$read_back\" = \"$name\" ] || { echo \"FAIL: $serial reports '$read_back'\"; exit 1; }\n```\n\nThen reconcile across the booted set: **a duplicate key is a hard error, never a warning.** A warning here becomes a shared account, and a shared account becomes a suite that cannot fail for the right reason.\n\n## Version Drift and Artifact Layout\n\n- **Pin the runner and assert what resolved.** Package-manager and `curl | bash` installers both float. Set an explicit version variable, and assert the reported version in CI. Checking that the binary exists does not catch drift; one project ran eight releases apart between local and CI without noticing.\n- **Do not hardcode the artifact layout.** Older Maestro versions wrote a flat run directory: `commands-(Flow Name).json` and `screenshot-\u003cstatus>-\u003cepoch>-(Flow Name).png` side by side. Newer versions write a directory per flow: `\u003ctimestamp>/\u003cFlow Name>/commands.json`, plus `screen-hierarchy/step-NNN-\u003ccommand>-\u003ctarget>.json`, `screenshots/`, and `logs/`. The change landed somewhere between those, so pin nothing to a version and glob nothing flat. Resolve the newest run directory and walk it.\n- **The per-step hierarchy files are the upgrade worth having.** `screen-hierarchy/step-NNN-*.json` makes each step's view tree separately addressable, which is a strictly better diagnostic surface than one blob per flow: you can read what was on screen at the step before the failure, not only at the failure.\n- **Or take the layout out of the equation.** `maestro test --test-output-dir \u003cdir>` writes `manifest.json`, `commands.json`, and `logs/` directly into a directory you name, and `--flatten-debug-output` writes without per-run subfolders or timestamps. Both are in `maestro test --help` on 2.8.0. Naming the directory beats globbing for the newest one, and it survives the next layout change.\n- **Upload the hierarchy dump, always.** It is the artifact people forget and the one that identifies a selector break.\n\n## Diagnosing a Failed Run\n\nRead, in this order:\n\n1. **`commands.json`** for the run: each step carries its own status, so it names the exact step that failed rather than the flow.\n2. **The hierarchy dump captured at failure** (`screen-hierarchy/`, and the error's embedded hierarchy root): this is what was actually on screen, which is the single highest-value artifact in the run.\n3. **Device logs** for the app's own errors.\n4. **The screenshot, last and with suspicion.** It is captured after teardown, so it frequently shows the launcher rather than the failing screen. Diagnosing from it produces confident wrong answers.\n\n**Check present-but-off-screen before anything else.** When a step fails on \"not visible\", find the id in that step's `screen-hierarchy` entry. Present with bounds outside the screen is a scroll problem in the flow; absent is a defect in the app. This one lookup separates the two most common causes of a red run and costs seconds. Skipping it is how an investigation spends hours building plausible mechanisms for a feature that was working the whole time.\n\n**Count root causes, not red flows.** One serial run failed with `Maestro Android driver did not start up in time`, and three further flows then failed in one to two seconds each with no artifacts written. Four red flows, one defect. A flow that failed in seconds and wrote nothing did not run; treat it as could-not-measure and diagnose the first failure, because a defect count inflated by a cascade sends the investigation at four subsystems instead of one.\n\n## Parallelism\n\n- `--shard-split N` divides the suite across N already-booted devices. `--shard-all N` runs the whole suite on each. Boot the devices first; neither flag provisions them.\n- `--udid` (aliased `--device`) takes a comma-separated list on **both** platforms, so a single `--shard-split` invocation drives Android emulator serials and iOS simulator UDIDs through the same code path. One sharding implementation covers both.\n- **Run one Maestro process per machine.** Two concurrent single-shard processes on one host have been observed to collide on the driver connection, failing with `Failed to connect to /127.0.0.1:7001` and `only one gesture can be performed at a time`. Drive every attached device from a single process with `--shard-split`. **There is no per-process driver port to escape with**: `--driver-host-port` is absent from both `maestro --help` and `maestro test --help` on 2.8.0. A handover note claiming an earlier release added it did not survive the check, which is `evidence-integrity.md`'s verify-the-property rule applied to a flag someone else told you about.\n- **Choose the shard count on measured wall clock, not on per-flow duration.** Same suite, same 14-core / 48 GB host, same day: **four emulators finished in 21.1 minutes, two in 27.3.** Four does oversubscribe the host, with load average around 20 and per-flow times stretching from roughly 40 seconds to several minutes, and the total is still shorter. Wall clock is what gates a pull request, so the per-flow number tempts you to the wrong conclusion. Serial for comparison was roughly 1.5 to 2 hours.\n\n## Anti-Patterns\n\n| Anti-pattern | Why it fails | Fix |\n| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |\n| E2E against a prebuilt development shell | Native modules absent; adds a CI-only launch path; flows fill with workarounds | Build a release-shaped artifact and install it |\n| Development build adopted as the fix for the dev-server dependency | `developmentClient: true` builds a debug variant, which does not embed the JS bundle, so a packager is still required | Build a release variant; only it embeds the bundle |\n| E2E affordance gated behind `__DEV__` | `__DEV__` is false in a release build, so the affordance is absent from the build under test | Gate on an `EXPO_PUBLIC_` variable and keep what it gates safe to ship in plain text |\n| Signed-manifest path left intact in CI | The certificate fetch needs an account session; non-interactive CI cannot answer the prompt | Supply `EXPO_TOKEN`, or serve the manifest `--offline`, on every path |\n| Local and CI on different device profiles | A shorter screen pushes content below the fold, producing \"not visible\" failures that reproduce only in CI and read as product defects | Pin one profile for both; extra profiles run nightly, not at the gate |\n| Native module assumed absent in a development shell | Some SDKs detect the shell and degrade silently, so the flow covers a fallback path users never run | Read the device log for what the module decided; prefer a real build |\n| \"Not visible\" diagnosed before the hierarchy is read | Present-but-off-screen and genuinely-absent are different bugs behind the same message | Look for the id and its bounds in that step's `screen-hierarchy` first |\n| AVD used as the creation tool produced it | `target=android-0`, GPU off, keyboard off: acceleration silently drops or boot never completes | Repair the `.ini` and `config.ini`, then read the values back |\n| Android device identity read rather than written | Every emulator reports the product model, so parallel shards share one map key and one account | Write `device_name` / `bluetooth_name` per device and hard-fail duplicates |\n| Shard count chosen from per-flow duration | Oversubscription stretches each flow while still shortening the run | Choose on measured wall clock, which is what gates the PR |\n| Every red flow counted as its own defect | A driver timeout cascades into fast, artifact-less failures | Diagnose the first failure; artifact-less seconds-long failures are could-not-measure |\n| Multi-line `script:` in the emulator action | Each line is a separate `sh -c`; `set -e` and every variable are lost | One line invoking a real script file |\n| Hardware inputs on both the create and the test step | `config.ini` is re-appended every run; the snapshot is rejected at boot | Pass them on the creation step only, or not at all |\n| Cache key without an image version component | Runner-image bump silently invalidates the snapshot; permanent cold boots | Key on API level, target, arch, and image version |\n| Combined cache step for the AVD | Saves only on success, so the run that built the snapshot never stores it | Split `cache/restore` and `cache/save` |\n| ATD image under a UI driver | SystemUI, launcher, and IME are stripped; hardware rendering is off | Use a standard system image |\n| Floating runner install | Local and CI drift apart silently; behavior differs with no version in the logs | Pin the version and assert the resolved version |\n| Flat artifact glob | Breaks on the run-directory layout change | Resolve the newest run directory and walk it |\n| Diagnosing from the failure screenshot | Taken after teardown; usually shows the launcher | Read the per-step status and the hierarchy dump |\n| Host-side reachability check standing in for the device | Different network namespace; proves nothing about the guest | Prove it from the device, or forward the port over the debug bridge |\n| Device-side connect used to verify an `adb reverse` forward | The mapping itself answers, so the check passes with nothing behind it | Accept a socket in the host process and assert the accept happened |\n| Retries added over a configuration defect | Converts a reproducible failure into an intermittent one | Fix the configuration; keep retries for genuinely nondeterministic steps |\n\n## Device Lab Checklist\n\n- [ ] **Artifact decided first**: flows run against a release-shaped or development build, never a prebuilt development shell\n- [ ] **Release variant confirmed**: the installed artifact embeds the JS bundle and launches with no packager running\n- [ ] **No `__DEV__`-gated test affordance**: E2E switches ride an `EXPO_PUBLIC_` variable and gate nothing that must stay secret\n- [ ] **Runner version pinned and asserted**: CI fails if the resolved version is not the pinned one\n- [ ] **Emulator script is one line**: any real logic lives in a checked-in script file\n- [ ] **Snapshot restore proven**: boot time recorded, and a rejected snapshot fails the job rather than passing slowly\n- [ ] **Hardware inputs on the creation step only**\n- [ ] **Cache key carries an image version component**\n- [ ] **Hardware acceleration verified on**, not left to chance\n- [ ] **Standard system image**, not an ATD variant\n- [ ] **One device profile across local and CI**, compared on density-independent height rather than pixel resolution\n- [ ] **Locally created AVDs asserted after creation**: `target`, GPU, and keyboard read back rather than assumed written\n- [ ] **Per-device identity written and proven unique** before any sharded run, with a duplicate failing the job\n- [ ] **Artifacts uploaded**: per-step statuses, hierarchy dumps, screenshots, and device logs, resolved by run directory or written to a named output directory\n- [ ] **Dev-server path, if used, is explicitly temporary**: port forwarded over the debug bridge, manifest health-checked with every header the client sends, signing resolved for a non-interactive session, and error bodies logged\n- [ ] **Sharding matches the booted device count**, driven by one runner process per machine\n- [ ] **Shard count justified by measured wall clock**, not by per-flow duration\n\n## Integration Points\n\n- **Used in workflows**: `*ci` (pipeline shape, caching, artifacts), `*framework` (scaffolding the device suite and its scripts), `*automate` (flows must not encode harness workarounds), `*nfr-assess` (boot and run duration as evidence)\n- **Related fragments**: `mobile-test-strategy.md` (what belongs on a device at all), `maestro-flows.md` (flow-level quality and command semantics), `evidence-integrity.md` (three-state diagnostics and hollow green, which is where most of these defects hide), `ci-burn-in.md` (burn-in and sharding mechanics)\n- **Tools**: `maestro test`, `adb`, `avdmanager`, `emulator`, `reactivecircus/android-emulator-runner`, EAS or the platform build toolchain\n\n_Source: Maestro 2.8.0 CLI help and documentation; Expo app-config, local-build, and environment-variable documentation; `@expo/cli` code-signing source; `reactivecircus/android-emulator-runner` source and issue tracker; defects, measurements, and timings from live Maestro device-lab investigations on hosted runners and an Apple Silicon host_\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":36206,"content_sha256":"0d992898bd8e1b1d047285097047bc5078528f2d3f6e6c2f61e6339cca71454e"},{"filename":"knowledge/mobile-test-strategy.md","content":"# Mobile Test Strategy\n\n## Principle\n\nMobile applies the same level discipline as any other stack: push verification to the cheapest level that can carry it, and reserve device-level flows for what genuinely requires a device. The mobile-specific part is that the expensive level is **much** more expensive (emulator boot, app install, real network) and the risk surface includes conditions web apps do not have: permissions, backgrounding, offline, deep links, and OS version fragmentation.\n\n## Rationale\n\n**The Problem**: Mobile suites tend to collapse into one level. Teams write end-to-end device flows for everything because that is the only tool they set up, then watch a 40-flow suite take 50 minutes and fail for reasons unrelated to the change. The opposite failure is equally common: unit-testing every reducer and shipping an app nobody ever launched in CI.\n\n**The Solution**: Split by what actually needs the device. Business logic, formatting, state reduction, and API clients are unit and integration concerns and do not need a simulator. Component or screen tests cover rendering and interaction without a full app launch. Device flows exist for the journeys where the integration of app, OS, and backend is itself the risk.\n\n**Why This Matters**:\n\n- Suite runtime stays inside a PR-gate budget\n- Failures point at the layer that broke\n- Device capacity is spent on the risks only a device can prove\n- The same P0-P3 prioritization used elsewhere in TEA transfers unchanged\n\n## The Mobile Test Level Framework\n\n| Level | Runs on | Covers | Typical share (indicative) |\n| ------------------------- | -------------------------- | ----------------------------------------------------------------------- | -------------------------- |\n| **Unit** | Node / JVM / Swift runtime | Pure logic, reducers, formatters, validation, mappers | 60-70% |\n| **Component / screen** | Test renderer, no device | Rendering, props, interaction handlers, conditional UI | 20-25% |\n| **Contract** | No device | The app's HTTP boundary against its backend (Pact or schema validation) | small, high value |\n| **Device flow (Maestro)** | Simulator/emulator/device | Cross-screen journeys, permissions, deep links, background/foreground | 5-15% |\n| **Manual / exploratory** | Real device | Gestures, haptics, biometrics, accessibility, real-network degradation | remainder |\n\n**Duplicate coverage guard** (same rule as every other stack): before adding a device flow, check whether a component test or a contract test already proves the behavior. A device flow that only verifies a label renders is a unit test wearing a 90-second costume.\n\n## What Belongs in a Device Flow\n\nPromote to a Maestro flow when the risk is in the **integration**, not the logic:\n\n- P0 revenue or access journeys end to end (sign in, purchase, submit claim)\n- OS permission grants and denials, including the denied path\n- Deep link entry into a specific screen. The app's own parsing and routing is testable even in a development shell, through the shell's routed URL form; the OS-level handoff needs the app's custom scheme registered, which a shell does not do.\n- Universal Links and Android App Links are a **separate** surface from custom-scheme deep links, and the same shell trick does not cover them. A verified HTTPS link depends on the platform's domain association (`apple-app-site-association`, `assetlinks.json`) being served and accepted, so a shell-routed URL exercises the in-app routing while proving nothing about whether the OS would have handed the link over at all. Score and cover the association separately from the routing.\n- Background, foreground, and process-death restoration\n- Offline and reconnect behavior\n- Push notification tap-through\n- App upgrade with existing local data (explicit exception to clearState isolation: seeds previous-build state, installs the new build without clearState, asserts migration, and cleans up in teardown)\n\nKeep out of device flows:\n\n- Field-level input validation (component level)\n- Every variant of a list cell (component level)\n- Error message copy (component level)\n- API error mapping (contract or unit level)\n\n## Mobile Risk Categories\n\nExtends the standard TEA risk categories with the conditions unique to the platform. Score each with the usual probability × impact 1-9 scale.\n\n| Category | Example risk | Typical level |\n| -------------------- | ---------------------------------------------------- | ------------------------- |\n| **Permissions** | Camera denied leaves the user on a dead screen | Device flow |\n| **Lifecycle** | State lost when the OS kills a backgrounded app | Device flow |\n| **Connectivity** | Offline write is silently discarded on reconnect | Device flow + unit |\n| **Fragmentation** | Layout breaks on a small screen or an older OS | Device matrix |\n| **Upgrade** | Local database migration corrupts existing user data | Device flow |\n| **Store compliance** | Missing privacy declaration blocks release | Release checklist |\n| **Performance** | Cold start over budget, frame drops on scroll | Instrumented, not Maestro |\n| **Binary size** | Growth pushes past a cellular download threshold | CI metric |\n\nCold start, frame rate, memory, and binary size are NFR evidence. Collect them with platform instrumentation and audit them in the NFR workflow; do not try to assert them from a Maestro flow.\n\n## Device Matrix\n\nPick the matrix from risk, not from availability. A defensible minimum:\n\n- **Primary**: newest OS on the highest-usage device for each platform\n- **Floor**: the oldest OS version the app still supports\n- **Form factor**: one small screen, and a tablet only if the app ships a tablet layout\n\nRun the full matrix nightly and on release candidates. Run the primary target only on PRs, because a PR gate that boots six emulators stops being a gate people wait for.\n\n**The PR-gate profile and the local profile must be the same one.** A different screen height moves content below the fold, and `assertVisible` means inside the viewport, so a shorter runner device fails UI assertions that pass on every developer machine and reads as a product defect. Compare on density-independent height rather than pixel resolution: two profiles can share `1080x` and differ by 100dp. The matrix exists to find fragmentation defects, and it can only do that from a gate that is not producing them by accident. See `mobile-ci-device-lab.md`.\n\n## CI Shape\n\n- **Build artifact first**: decide what the flows run against before writing any of them. A release-shaped build (unsigned APK, simulator IPA) is the default; a prebuilt development shell such as Expo Go is not a CI artifact, because the native modules the flows need are absent and the launch path exists only in CI. This decision sets the failure surface of the whole suite; see `mobile-ci-device-lab.md`.\n- **PR gate**: unit + component + contract on every push. P0-tagged Maestro flows on the primary target only.\n- **Nightly**: full Maestro suite across the device matrix.\n- **Release candidate**: full suite plus the upgrade-path flow from the previous production build.\n- **Artifacts**: Maestro writes per-step statuses, a hierarchy dump, screenshots, a video, and device logs per run. Upload all of them, and diagnose from the per-step status and the hierarchy dump captured at failure. The failure screenshot is taken after teardown and often shows the launcher rather than the failing screen, so leading with it produces confident wrong answers.\n- **No live third-party evaluation in the run path**: a flow asserting behavior behind a remotely evaluated feature flag hands the outcome to a third-party service, for a user the run created seconds earlier. Start the environment with the remote provider unconfigured, so the service falls back to its seeded local value, and seed that value as test data. The same applies to remote personalization and experiment services. See `feature-flags.md` for the flag-testing patterns this reuses.\n- **Burn-in**: applies to mobile the same way it applies to browser E2E. New or changed flows run repeatedly before merge, because device flows are the most flake-prone level in the suite.\n- **Sharding**: shard by flow file across parallel emulators, and pick the shard count from measured wall clock rather than from per-flow duration. Oversubscribing the host stretches every flow while still finishing the run sooner, and wall clock is what gates the PR. Each parallel device also needs its own fixture account and a way to know which one is its own; on Android that identity has to be written to the device rather than read from it (`mobile-ci-device-lab.md`).\n\n## Anti-Patterns\n\n| Anti-pattern | Why it fails | Fix |\n| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |\n| Every acceptance criterion becomes a device flow | Suite runtime explodes; failures are slow to diagnose | Apply the level framework and the duplicate-coverage guard |\n| Full device matrix on every PR | Gate becomes too slow to block on, so people bypass it | Primary target on PRs, matrix nightly |\n| Performance asserted inside a Maestro flow | Measures the harness, not the app | Platform instrumentation as NFR evidence |\n| Flows depend on a shared logged-in account | Parallel runs collide; failures are not reproducible | Per-run accounts/data or explicit backend reset when server state changes |\n| No offline or permission-denied coverage | The paths users actually hit in the wild are the untested ones | Score them as risks; they are usually P0 or P1 |\n| Testing against a production backend | Non-deterministic data, and a test order can mutate real state | Dedicated environment or a stubbed backend |\n| Device flows run through a prebuilt development shell | Native modules are absent, so notifications and payments can only be asserted missing; the launch path is CI-only | Build a release-shaped artifact and install it (`mobile-ci-device-lab.md`) |\n| A surface written off as untestable without a check | The coarse claim is often wrong: a shell routes its own deep-link URL form into the app, so link handling is reachable | Test the claim before dropping coverage; name the part that is genuinely out of reach |\n| Flow asserts behavior behind a remotely evaluated flag | A third-party service decides the outcome, for a user created seconds earlier | Unconfigure the remote provider so the seeded local value wins, and seed it as test data |\n\n## Mobile Strategy Checklist\n\n- [ ] **Build artifact decided**: flows run against a release-shaped or development build, never a prebuilt development shell\n- [ ] **Levels assigned**: each acceptance criterion mapped to unit, component, contract, or device flow\n- [ ] **Duplicate coverage checked**: no device flow proving something a cheaper level already proves\n- [ ] **Mobile risk categories scored**: permissions, lifecycle, connectivity, fragmentation, upgrade\n- [ ] **Device matrix justified**: primary, floor, and form factor chosen from usage data\n- [ ] **PR-gate profile matches the local profile**, compared on density-independent height\n- [ ] **PR gate bounded**: P0 flows on the primary target only\n- [ ] **NFR evidence separated**: cold start, frame rate, memory, binary size instrumented rather than asserted in flows\n- [ ] **No live third-party flag or experiment service in the run path**: remote evaluation disabled, values seeded as test data\n- [ ] **Artifacts uploaded**: video, screenshot, and hierarchy dump on failure\n- [ ] **Burn-in enabled** for new and changed flows\n- [ ] **Parallel devices carry distinct fixture accounts**, keyed on an identity the run wrote and verified\n\n## Integration Points\n\n- **Used in workflows**: `*test-design` (risk and level assignment for mobile), `*framework` (scaffold the suite), `*automate` (generate flows at the right level), `*ci` (device pipeline shape), `*nfr-assess` (mobile NFR evidence), `*trace` (map criteria to flows)\n- **Related fragments**: `maestro-flows.md` (flow syntax and quality), `mobile-ci-device-lab.md` (build artifact selection, emulator caching, version pinning, per-device identity, failure diagnosis), `test-levels-framework.md` (the general level model this specializes), `probability-impact.md` (the scoring scale), `test-priorities-matrix.md` (P0-P3), `ci-burn-in.md` (burn-in and sharding mechanics), `feature-flags.md` (testing both flag states without a live provider)\n- **Tools**: `maestro test`, `maestro studio`, platform instrumentation (Xcode Instruments, Android Profiler, Firebase Performance)\n\n_Source: TEA test-levels framework applied to mobile constraints, Maestro CI practice, mobile risk categories from permissions/lifecycle/connectivity/fragmentation failure modes._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":14431,"content_sha256":"d16495f50c2832acae9b5b0cb3c3b3443d2b7625dc3205076367834c66c7072c"},{"filename":"knowledge/network-error-monitor.md","content":"# Network Error Monitor\n\n## Principle\n\nAutomatically detect and fail tests when HTTP 4xx/5xx errors occur during execution. Act like Sentry for tests - catch silent backend failures even when UI passes assertions.\n\n## Rationale\n\nTraditional Playwright tests focus on UI:\n\n- Backend 500 errors ignored if UI looks correct\n- Silent failures slip through\n- No visibility into background API health\n- Tests pass while features are broken\n\nThe `network-error-monitor` provides:\n\n- **Automatic detection**: All HTTP 4xx/5xx responses tracked\n- **Test failures**: Fail tests with backend errors (even if UI passes)\n- **Structured artifacts**: JSON reports with error details\n- **Smart opt-out**: Disable for validation tests expecting errors\n- **Deduplication**: Group repeated errors by pattern\n- **Domino effect prevention**: Limit test failures per error pattern\n- **Respects test status**: Won't suppress actual test failures\n\n## Quick Start\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\n// That's it! Network monitoring is automatically enabled\ntest('my test', async ({ page }) => {\n await page.goto('/dashboard');\n // If any HTTP 4xx/5xx errors occur, the test will fail\n});\n```\n\n## Pattern Examples\n\n### Example 1: Basic Auto-Monitoring\n\n**Context**: Automatically fail tests when backend errors occur.\n\n**Implementation**:\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\n// Monitoring automatically enabled\ntest('should load dashboard', async ({ page }) => {\n await page.goto('/dashboard');\n await expect(page.locator('h1')).toContainText('Dashboard');\n\n // Passes if no HTTP errors\n // Fails if any 4xx/5xx errors detected with clear message:\n // \"Network errors detected: 2 request(s) failed\"\n // Failed requests:\n // GET 500 https://api.example.com/users\n // POST 503 https://api.example.com/metrics\n});\n```\n\n**Key Points**:\n\n- Zero setup - auto-enabled for all tests\n- Fails on any 4xx/5xx response\n- Structured error message with URLs and status codes\n- JSON artifact attached to test report\n\n### Example 2: Opt-Out for Validation Tests\n\n**Context**: Some tests expect errors (validation, error handling, edge cases).\n\n**Implementation**:\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\n// Opt-out with annotation\ntest('should show error on invalid input', { annotation: [{ type: 'skipNetworkMonitoring' }] }, async ({ page }) => {\n await page.goto('/form');\n await page.click('#submit'); // Triggers 400 error\n\n // Monitoring disabled - test won't fail on 400\n await expect(page.getByText('Invalid input')).toBeVisible();\n});\n\n// Or opt-out entire describe block\ntest.describe('error handling', { annotation: [{ type: 'skipNetworkMonitoring' }] }, () => {\n test('handles 404', async ({ page }) => {\n // All tests in this block skip monitoring\n });\n\n test('handles 500', async ({ page }) => {\n // Monitoring disabled\n });\n});\n```\n\n**Key Points**:\n\n- Use annotation `{ type: 'skipNetworkMonitoring' }`\n- Can opt-out single test or entire describe block\n- Monitoring still active for other tests\n- Perfect for intentional error scenarios\n\n### Example 3: Respects Test Status\n\n**Context**: The monitor respects final test statuses to avoid suppressing important test outcomes.\n\n**Behavior by test status:**\n\n- **`failed`**: Network errors logged as additional context, not thrown\n- **`timedOut`**: Network errors logged as additional context\n- **`skipped`**: Network errors logged, skip status preserved\n- **`interrupted`**: Network errors logged, interrupted status preserved\n- **`passed`**: Network errors throw and fail the test\n\n**Example with test.skip():**\n\n```typescript\ntest('feature gated test', async ({ page }) => {\n const featureEnabled = await checkFeatureFlag();\n test.skip(!featureEnabled, 'Feature not enabled');\n // If skipped, network errors won't turn this into a failure\n await page.goto('/new-feature');\n});\n```\n\n### Example 4: Excluding Legitimate Errors\n\n**Context**: Some endpoints legitimately return 4xx/5xx responses.\n\n**Implementation**:\n\n```typescript\nimport { test as base } from '@playwright/test';\nimport { createNetworkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\nexport const test = base.extend(\n createNetworkErrorMonitorFixture({\n excludePatterns: [\n /email-cluster\\/ml-app\\/has-active-run/, // ML service returns 404 when no active run\n /idv\\/session-templates\\/list/, // IDV service returns 404 when not configured\n /sentry\\.io\\/api/, // External Sentry errors should not fail tests\n ],\n }),\n);\n```\n\n**For merged fixtures:**\n\n```typescript\nimport { test as base, mergeTests } from '@playwright/test';\nimport { createNetworkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\nconst networkErrorMonitor = base.extend(\n createNetworkErrorMonitorFixture({\n excludePatterns: [/analytics\\.google\\.com/, /cdn\\.example\\.com/],\n }),\n);\n\nexport const test = mergeTests(authFixture, networkErrorMonitor);\n```\n\n### Example 5: Preventing Domino Effect\n\n**Context**: One failing endpoint shouldn't fail all tests.\n\n**Implementation**:\n\n```typescript\nimport { test as base } from '@playwright/test';\nimport { createNetworkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\nconst networkErrorMonitor = base.extend(\n createNetworkErrorMonitorFixture({\n excludePatterns: [], // Required when using maxTestsPerError\n maxTestsPerError: 1, // Only first test fails per error pattern, rest just log\n }),\n);\n```\n\n**How it works:**\n\nWhen `/api/v2/case-management/cases` returns 500:\n\n- **First test** encountering this error: **FAILS** with clear error message\n- **Subsequent tests** encountering same error: **PASSES** but logs warning\n\nError patterns are grouped by `method + status + base path`:\n\n- `GET /api/v2/case-management/cases/123` -> Pattern: `GET:500:/api/v2/case-management`\n- `GET /api/v2/case-management/quota` -> Pattern: `GET:500:/api/v2/case-management` (same group!)\n- `POST /api/v2/case-management/cases` -> Pattern: `POST:500:/api/v2/case-management` (different group!)\n\n**Why include HTTP method?** A GET 404 vs POST 404 might represent different issues:\n\n- `GET 404 /api/users/123` -> User not found (expected in some tests)\n- `POST 404 /api/users` -> Endpoint doesn't exist (critical error)\n\n**Output for subsequent tests:**\n\n```text\nWarning: Network errors detected but not failing test (maxTestsPerError limit reached):\n GET 500 https://api.example.com/api/v2/case-management/cases\n```\n\n**Recommended configuration:**\n\n```typescript\ncreateNetworkErrorMonitorFixture({\n excludePatterns: [...], // Required - known broken endpoints (can be empty [])\n maxTestsPerError: 1 // Stop domino effect (requires excludePatterns)\n})\n```\n\n**Understanding worker-level state:**\n\nError pattern counts are stored in worker-level global state:\n\n```typescript\n// test-file-1.spec.ts (runs in Worker 1)\ntest('test A', () => {\n /* triggers GET:500:/api/v2/cases */\n}); // FAILS\n\n// test-file-2.spec.ts (runs later in Worker 1)\ntest('test B', () => {\n /* triggers GET:500:/api/v2/cases */\n}); // PASSES (limit reached)\n\n// test-file-3.spec.ts (runs in Worker 2 - different worker)\ntest('test C', () => {\n /* triggers GET:500:/api/v2/cases */\n}); // FAILS (fresh worker)\n```\n\n### Example 6: Integration with Merged Fixtures\n\n**Context**: Combine network-error-monitor with other utilities.\n\n**Implementation**:\n\n```typescript\n// playwright/support/merged-fixtures.ts\nimport { mergeTests } from '@playwright/test';\nimport { test as networkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n// Auth fixture built in your project (setAuthProvider + createAuthFixtures)\nimport { test as authFixture } from './auth-fixture';\n\nexport const test = mergeTests(\n authFixture,\n networkErrorMonitorFixture,\n // Add other fixtures\n);\n\n// In tests\nimport { test, expect } from '../support/merged-fixtures';\n\ntest('authenticated with monitoring', async ({ page, authToken }) => {\n // Both auth and network monitoring active\n await page.goto('/protected');\n\n // Fails if backend returns errors during auth flow\n});\n```\n\n**Key Points**:\n\n- Combine with `mergeTests`\n- Works alongside all other utilities\n- Monitoring active automatically\n- No extra setup needed\n\n### Example 7: Artifact Structure\n\n**Context**: Debugging failed tests with network error artifacts.\n\nWhen test fails due to network errors, artifact attached:\n\n```json\n[\n {\n \"url\": \"https://api.example.com/users\",\n \"status\": 500,\n \"method\": \"GET\",\n \"timestamp\": \"2025-11-10T12:34:56.789Z\"\n },\n {\n \"url\": \"https://api.example.com/metrics\",\n \"status\": 503,\n \"method\": \"POST\",\n \"timestamp\": \"2025-11-10T12:34:57.123Z\"\n }\n]\n```\n\n## Implementation Details\n\n### How It Works\n\n1. **Fixture Extension**: Uses Playwright's `base.extend()` with `auto: true`\n2. **Response Listener**: Attaches `page.on('response')` listener at test start\n3. **Multi-Page Monitoring**: Automatically monitors popups and new tabs via `context.on('page')`\n4. **Error Collection**: Captures 4xx/5xx responses, checking exclusion patterns\n5. **Try/Finally**: Ensures error processing runs even if test fails early\n6. **Status Check**: Only throws errors if test hasn't already reached final status\n7. **Artifact**: Attaches JSON file to test report for debugging\n\n### Performance\n\nThe monitor has minimal performance impact:\n\n- Event listener overhead: ~0.1ms per response\n- Memory: ~200 bytes per unique error\n- No network delay (observes responses, doesn't intercept them)\n\n## Comparison with Alternatives\n\n| Approach | Network Error Monitor | Manual afterEach |\n| --------------------------- | --------------------- | --------------------- |\n| **Setup Required** | Zero (auto-enabled) | Every test file |\n| **Catches Silent Failures** | Yes | Yes (if configured) |\n| **Structured Artifacts** | JSON attached | Custom impl |\n| **Test Failure Safety** | Try/finally | afterEach may not run |\n| **Opt-Out Mechanism** | Annotation | Custom logic |\n| **Status Aware** | Respects skip/failed | No |\n\n## When to Use\n\n**Auto-enabled for:**\n\n- All E2E tests\n- Integration tests\n- Any test hitting real APIs\n\n**Opt-out for:**\n\n- Validation tests (expecting 4xx)\n- Error handling tests (expecting 5xx)\n- Offline tests (network-recorder playback)\n\n## Troubleshooting\n\n### Test fails with network errors but I don't see them in my app\n\nThe errors might be happening during page load or in background polling. Check the `network-errors.json` artifact in your test report for full details including timestamps.\n\n### False positives from external services\n\nConfigure exclusion patterns as shown in the \"Excluding Legitimate Errors\" section above.\n\n### Network errors not being caught\n\nEnsure you're importing the test from the correct fixture:\n\n```typescript\n// Correct\nimport { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\n\n// Wrong - this won't have network monitoring\nimport { test } from '@playwright/test';\n```\n\n## Related Fragments\n\n- `overview.md` - Installation and fixtures\n- `fixtures-composition.md` - Merging with other utilities\n- `error-handling.md` - Traditional error handling patterns\n\n## Anti-Patterns\n\n**DON'T opt out of monitoring globally:**\n\n```typescript\n// Every test skips monitoring\ntest.use({ annotation: [{ type: 'skipNetworkMonitoring' }] });\n```\n\n**DO opt-out only for specific error tests:**\n\n```typescript\ntest.describe('error scenarios', { annotation: [{ type: 'skipNetworkMonitoring' }] }, () => {\n // Only these tests skip monitoring\n});\n```\n\n**DON'T ignore network error artifacts:**\n\n```typescript\n// Test fails, artifact shows 500 errors\n// Developer: \"Works on my machine\" ¯\\_(ツ)_/¯\n```\n\n**DO check artifacts for root cause:**\n\n```typescript\n// Read network-errors.json artifact\n// Identify failing endpoint: GET /api/users -> 500\n// Fix backend issue before merging\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12316,"content_sha256":"e450d5db937dbc709f0ff7ba292f813cd8c0e7d8a0126b8580521210cf38e548"},{"filename":"knowledge/network-first.md","content":"# Network-First Safeguards\n\n## Principle\n\nRegister network interceptions **before** any navigation or user action. Store the interception promise and await it immediately after the triggering step. Replace implicit waits with deterministic signals based on network responses, spinner disappearance, or event hooks.\n\n## Rationale\n\nThe most common source of flaky E2E tests is **race conditions** between navigation and network interception:\n\n- Navigate then intercept = missed requests (too late)\n- No explicit wait = assertion runs before response arrives\n- Hard waits (`waitForTimeout(3000)`) = slow, unreliable, brittle\n\nNetwork-first patterns provide:\n\n- **Zero race conditions**: Intercept is active before triggering action\n- **Deterministic waits**: Wait for actual response, not arbitrary timeouts\n- **Actionable failures**: Assert on response status/body, not generic \"element not found\"\n- **Speed**: No padding with extra wait time\n\n## Pattern Examples\n\n### Example 1: Intercept Before Navigate Pattern\n\n**Context**: The foundational pattern for all E2E tests. Always register route interception **before** the action that triggers the request (navigation, click, form submit).\n\n**Implementation**:\n\n```typescript\n// ✅ CORRECT: Intercept BEFORE navigate\ntest('user can view dashboard data', async ({ page }) => {\n // Step 1: Register interception FIRST\n const usersPromise = page.waitForResponse((resp) => resp.url().includes('/api/users') && resp.status() === 200);\n\n // Step 2: THEN trigger the request\n await page.goto('/dashboard');\n\n // Step 3: THEN await the response\n const usersResponse = await usersPromise;\n const users = await usersResponse.json();\n\n // Step 4: Assert on structured data\n expect(users).toHaveLength(10);\n await expect(page.getByText(users[0].name)).toBeVisible();\n});\n\n// Cypress equivalent\ndescribe('Dashboard', () => {\n it('should display users', () => {\n // Step 1: Register interception FIRST\n cy.intercept('GET', '**/api/users').as('getUsers');\n\n // Step 2: THEN trigger\n cy.visit('/dashboard');\n\n // Step 3: THEN await\n cy.wait('@getUsers').then((interception) => {\n // Step 4: Assert on structured data\n expect(interception.response.statusCode).to.equal(200);\n expect(interception.response.body).to.have.length(10);\n cy.contains(interception.response.body[0].name).should('be.visible');\n });\n });\n});\n\n// ❌ WRONG: Navigate BEFORE intercept (race condition!)\ntest('flaky test example', async ({ page }) => {\n await page.goto('/dashboard'); // Request fires immediately\n\n const usersPromise = page.waitForResponse('/api/users'); // TOO LATE - might miss it\n const response = await usersPromise; // May timeout randomly\n});\n```\n\n**Key Points**:\n\n- Playwright: Use `page.waitForResponse()` with URL pattern or predicate **before** `page.goto()` or `page.click()`\n- Cypress: Use `cy.intercept().as()` **before** `cy.visit()` or `cy.click()`\n- Store promise/alias, trigger action, **then** await response\n- This prevents 95% of race-condition flakiness in E2E tests\n\n### Example 2: HAR Capture for Debugging\n\n**Context**: When debugging flaky tests or building deterministic mocks, capture real network traffic with HAR files. Replay them in tests for consistent, offline-capable test runs.\n\n**Implementation**:\n\n```typescript\n// playwright.config.ts - Enable HAR recording\nexport default defineConfig({\n use: {\n // Record HAR on first run\n recordHar: { path: './hars/', mode: 'minimal' },\n // Or replay HAR in tests\n // serviceWorkers: 'block',\n },\n});\n\n// Capture HAR for specific test\ntest('capture network for order flow', async ({ page, context }) => {\n // Start recording\n await context.routeFromHAR('./hars/order-flow.har', {\n url: '**/api/**',\n update: true, // Update HAR with new requests\n });\n\n await page.goto('/checkout');\n await page.fill('[data-testid=\"credit-card\"]', '4111111111111111');\n await page.click('[data-testid=\"submit-order\"]');\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n\n // HAR saved to ./hars/order-flow.har\n});\n\n// Replay HAR for deterministic tests (no real API needed)\ntest('replay order flow from HAR', async ({ page, context }) => {\n // Replay captured HAR\n await context.routeFromHAR('./hars/order-flow.har', {\n url: '**/api/**',\n update: false, // Read-only mode\n });\n\n // Test runs with exact recorded responses - fully deterministic\n await page.goto('/checkout');\n await page.fill('[data-testid=\"credit-card\"]', '4111111111111111');\n await page.click('[data-testid=\"submit-order\"]');\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n});\n\n// Custom mock based on HAR insights\ntest('mock order response based on HAR', async ({ page }) => {\n // After analyzing HAR, create focused mock\n await page.route('**/api/orders', (route) =>\n route.fulfill({\n status: 200,\n contentType: 'application/json',\n body: JSON.stringify({\n orderId: '12345',\n status: 'confirmed',\n total: 99.99,\n }),\n }),\n );\n\n await page.goto('/checkout');\n await page.click('[data-testid=\"submit-order\"]');\n await expect(page.getByText('Order #12345')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- HAR files capture real request/response pairs for analysis\n- `update: true` records new traffic; `update: false` replays existing\n- Replay mode makes tests fully deterministic (no upstream API needed)\n- Use HAR to understand API contracts, then create focused mocks\n\n### Example 3: Network Stub with Edge Cases\n\n**Context**: When testing error handling, timeouts, and edge cases, stub network responses to simulate failures. Test both happy path and error scenarios.\n\n**Implementation**:\n\n```typescript\n// Test happy path\ntest('order succeeds with valid data', async ({ page }) => {\n await page.route('**/api/orders', (route) =>\n route.fulfill({\n status: 200,\n contentType: 'application/json',\n body: JSON.stringify({ orderId: '123', status: 'confirmed' }),\n }),\n );\n\n await page.goto('/checkout');\n await page.click('[data-testid=\"submit-order\"]');\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n});\n\n// Test 500 error\ntest('order fails with server error', async ({ page }) => {\n // Listen for console errors (app should log gracefully)\n const consoleErrors: string[] = [];\n page.on('console', (msg) => {\n if (msg.type() === 'error') consoleErrors.push(msg.text());\n });\n\n // Stub 500 error\n await page.route('**/api/orders', (route) =>\n route.fulfill({\n status: 500,\n contentType: 'application/json',\n body: JSON.stringify({ error: 'Internal Server Error' }),\n }),\n );\n\n await page.goto('/checkout');\n await page.click('[data-testid=\"submit-order\"]');\n\n // Assert UI shows error gracefully\n await expect(page.getByText('Something went wrong')).toBeVisible();\n await expect(page.getByText('Please try again')).toBeVisible();\n\n // Verify error logged (not thrown)\n expect(consoleErrors.some((e) => e.includes('Order failed'))).toBeTruthy();\n});\n\n// Test network timeout\ntest('order times out after 10 seconds', async ({ page }) => {\n // Stub delayed response (never resolves within timeout)\n await page.route(\n '**/api/orders',\n (route) => new Promise(() => {}), // Never resolves - simulates timeout\n );\n\n await page.goto('/checkout');\n await page.click('[data-testid=\"submit-order\"]');\n\n // App should show timeout message after configured timeout\n await expect(page.getByText('Request timed out')).toBeVisible({ timeout: 15000 });\n});\n\n// Test partial data response\ntest('order handles missing optional fields', async ({ page }) => {\n await page.route('**/api/orders', (route) =>\n route.fulfill({\n status: 200,\n contentType: 'application/json',\n // Missing optional fields like 'trackingNumber', 'estimatedDelivery'\n body: JSON.stringify({ orderId: '123', status: 'confirmed' }),\n }),\n );\n\n await page.goto('/checkout');\n await page.click('[data-testid=\"submit-order\"]');\n\n // App should handle gracefully - no crash, shows what's available\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n await expect(page.getByText('Tracking information pending')).toBeVisible();\n});\n\n// Cypress equivalents\ndescribe('Order Edge Cases', () => {\n it('should handle 500 error', () => {\n cy.intercept('POST', '**/api/orders', {\n statusCode: 500,\n body: { error: 'Internal Server Error' },\n }).as('orderFailed');\n\n cy.visit('/checkout');\n cy.get('[data-testid=\"submit-order\"]').click();\n cy.wait('@orderFailed');\n cy.contains('Something went wrong').should('be.visible');\n });\n\n it('should handle timeout', () => {\n cy.intercept('POST', '**/api/orders', (req) => {\n req.reply({ delay: 20000 }); // Delay beyond app timeout\n }).as('orderTimeout');\n\n cy.visit('/checkout');\n cy.get('[data-testid=\"submit-order\"]').click();\n cy.contains('Request timed out', { timeout: 15000 }).should('be.visible');\n });\n});\n```\n\n**Key Points**:\n\n- Stub different HTTP status codes (200, 400, 500, 503)\n- Simulate timeouts with `delay` or non-resolving promises\n- Test partial/incomplete data responses\n- Verify app handles errors gracefully (no crashes, user-friendly messages)\n\n### Example 4: Deterministic Waiting\n\n**Context**: Never use hard waits (`waitForTimeout(3000)`). Always wait for explicit signals: network responses, element state changes, or custom events.\n\n**Implementation**:\n\n```typescript\n// ✅ GOOD: Wait for response with predicate\ntest('wait for specific response', async ({ page }) => {\n const responsePromise = page.waitForResponse((resp) => resp.url().includes('/api/users') && resp.status() === 200);\n\n await page.goto('/dashboard');\n const response = await responsePromise;\n\n expect(response.status()).toBe(200);\n await expect(page.getByText('Dashboard')).toBeVisible();\n});\n\n// ✅ GOOD: Wait for multiple responses\ntest('wait for all required data', async ({ page }) => {\n const usersPromise = page.waitForResponse('**/api/users');\n const productsPromise = page.waitForResponse('**/api/products');\n const ordersPromise = page.waitForResponse('**/api/orders');\n\n await page.goto('/dashboard');\n\n // Wait for all in parallel\n const [users, products, orders] = await Promise.all([usersPromise, productsPromise, ordersPromise]);\n\n expect(users.status()).toBe(200);\n expect(products.status()).toBe(200);\n expect(orders.status()).toBe(200);\n});\n\n// ✅ GOOD: Wait for spinner to disappear\ntest('wait for loading indicator', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Wait for spinner to disappear (signals data loaded)\n await expect(page.getByTestId('loading-spinner')).not.toBeVisible();\n await expect(page.getByText('Dashboard')).toBeVisible();\n});\n\n// ✅ GOOD: Wait for custom event (advanced)\ntest('wait for custom ready event', async ({ page }) => {\n let appReady = false;\n page.on('console', (msg) => {\n if (msg.text() === 'App ready') appReady = true;\n });\n\n await page.goto('/dashboard');\n\n // Poll until custom condition met\n await page.waitForFunction(() => appReady, { timeout: 10000 });\n\n await expect(page.getByText('Dashboard')).toBeVisible();\n});\n\n// ❌ BAD: Hard wait (arbitrary timeout)\ntest('flaky hard wait example', async ({ page }) => {\n await page.goto('/dashboard');\n await page.waitForTimeout(3000); // WHY 3 seconds? What if slower? What if faster?\n await expect(page.getByText('Dashboard')).toBeVisible(); // May fail if >3s\n});\n\n// Cypress equivalents\ndescribe('Deterministic Waiting', () => {\n it('should wait for response', () => {\n cy.intercept('GET', '**/api/users').as('getUsers');\n cy.visit('/dashboard');\n cy.wait('@getUsers').its('response.statusCode').should('eq', 200);\n cy.contains('Dashboard').should('be.visible');\n });\n\n it('should wait for spinner to disappear', () => {\n cy.visit('/dashboard');\n cy.get('[data-testid=\"loading-spinner\"]').should('not.exist');\n cy.contains('Dashboard').should('be.visible');\n });\n\n // ❌ BAD: Hard wait\n it('flaky hard wait', () => {\n cy.visit('/dashboard');\n cy.wait(3000); // NEVER DO THIS\n cy.contains('Dashboard').should('be.visible');\n });\n});\n```\n\n**Key Points**:\n\n- `waitForResponse()` with URL pattern or predicate = deterministic\n- `waitForLoadState('networkidle')` = wait for all network activity to finish\n- Wait for element state changes (spinner disappears, button enabled)\n- **NEVER** use `waitForTimeout()` or `cy.wait(ms)` - always non-deterministic\n\n### Example 5: Anti-Pattern - Navigate Then Mock\n\n**Problem**:\n\n```typescript\n// ❌ BAD: Race condition - mock registered AFTER navigation starts\ntest('flaky test - navigate then mock', async ({ page }) => {\n // Navigation starts immediately\n await page.goto('/dashboard'); // Request to /api/users fires NOW\n\n // Mock registered too late - request already sent\n await page.route('**/api/users', (route) =>\n route.fulfill({\n status: 200,\n body: JSON.stringify([{ id: 1, name: 'Test User' }]),\n }),\n );\n\n // Test randomly passes/fails depending on timing\n await expect(page.getByText('Test User')).toBeVisible(); // Flaky!\n});\n\n// ❌ BAD: No wait for response\ntest('flaky test - no explicit wait', async ({ page }) => {\n await page.route('**/api/users', (route) => route.fulfill({ status: 200, body: JSON.stringify([]) }));\n\n await page.goto('/dashboard');\n\n // Assertion runs immediately - may fail if response slow\n await expect(page.getByText('No users found')).toBeVisible(); // Flaky!\n});\n\n// ❌ BAD: Generic timeout\ntest('flaky test - hard wait', async ({ page }) => {\n await page.goto('/dashboard');\n await page.waitForTimeout(2000); // Arbitrary wait - brittle\n\n await expect(page.getByText('Dashboard')).toBeVisible();\n});\n```\n\n**Why It Fails**:\n\n- **Mock after navigate**: Request fires during navigation, mock isn't active yet (race condition)\n- **No explicit wait**: Assertion runs before response arrives (timing-dependent)\n- **Hard waits**: Slow tests, brittle (fails if \u003c timeout, wastes time if > timeout)\n- **Non-deterministic**: Passes locally, fails in CI (different speeds)\n\n**Better Approach**: Always intercept → trigger → await\n\n```typescript\n// ✅ GOOD: Intercept BEFORE navigate\ntest('deterministic test', async ({ page }) => {\n // Step 1: Register mock FIRST\n await page.route('**/api/users', (route) =>\n route.fulfill({\n status: 200,\n contentType: 'application/json',\n body: JSON.stringify([{ id: 1, name: 'Test User' }]),\n }),\n );\n\n // Step 2: Store response promise BEFORE trigger\n const responsePromise = page.waitForResponse('**/api/users');\n\n // Step 3: THEN trigger\n await page.goto('/dashboard');\n\n // Step 4: THEN await response\n await responsePromise;\n\n // Step 5: THEN assert (data is guaranteed loaded)\n await expect(page.getByText('Test User')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- Order matters: Mock → Promise → Trigger → Await → Assert\n- No race conditions: Mock is active before request fires\n- Explicit wait: Response promise ensures data loaded\n- Deterministic: Always passes if app works correctly\n\n## Integration Points\n\n- **Used in workflows**: `*atdd` (test generation), `*automate` (test expansion), `*framework` (network setup)\n- **Related fragments**:\n - `fixture-architecture.md` - Network fixture patterns\n - `data-factories.md` - API-first setup with network\n - `test-quality.md` - Deterministic test principles\n\n## Debugging Network Issues\n\nWhen network tests fail, check:\n\n1. **Timing**: Is interception registered **before** action?\n2. **URL pattern**: Does pattern match actual request URL?\n3. **Response format**: Is mocked response valid JSON/format?\n4. **Status code**: Is app checking for 200 vs 201 vs 204?\n5. **HAR file**: Capture real traffic to understand actual API contract\n\n```typescript\n// Debug network issues with logging\ntest('debug network', async ({ page }) => {\n // Log all requests\n page.on('request', (req) => console.log('→', req.method(), req.url()));\n\n // Log all responses\n page.on('response', (resp) => console.log('←', resp.status(), resp.url()));\n\n await page.goto('/dashboard');\n});\n```\n\n_Source: Murat Testing Philosophy (lines 94-137), Playwright network patterns, Cypress intercept best practices._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":16327,"content_sha256":"2920e58e145626f5505bcb75e263dbd0e6ac79a8c4c2ec138f5329e06a6ac014"},{"filename":"knowledge/network-recorder.md","content":"# Network Recorder Utility\n\n## Principle\n\nRecord network traffic to HAR files during test execution, then play back from disk for offline testing. Enables frontend tests to run in complete isolation from backend services with intelligent stateful CRUD detection for realistic API behavior.\n\n## Rationale\n\nTraditional E2E tests require live backend services:\n\n- Slow (real network latency)\n- Flaky (backend instability affects tests)\n- Expensive (full stack running for UI tests)\n- Coupled (UI tests break when API changes)\n\nHAR-based recording/playback provides:\n\n- **True offline testing**: UI tests run without backend\n- **Deterministic behavior**: Same responses every time\n- **Fast execution**: No network latency\n- **Stateful mocking**: CRUD operations work naturally (not just read-only)\n- **Environment flexibility**: Map URLs for any environment\n\n## Quick Start\n\n### 1. Record Network Traffic\n\n```typescript\n// Set mode to 'record' to capture network traffic\nprocess.env.PW_NET_MODE = 'record';\n\ntest('should add, edit and delete a movie', async ({ page, context, networkRecorder }) => {\n // Setup network recorder - it will record all network traffic\n await networkRecorder.setup(context);\n\n // Your normal test code\n await page.goto('/');\n await page.fill('#movie-name', 'Inception');\n await page.click('#add-movie');\n\n // Network traffic is automatically saved to HAR file\n});\n```\n\n### 2. Playback Network Traffic\n\n```typescript\n// Set mode to 'playback' to use recorded traffic\nprocess.env.PW_NET_MODE = 'playback';\n\ntest('should add, edit and delete a movie', async ({ page, context, networkRecorder }) => {\n // Setup network recorder - it will replay from HAR file\n await networkRecorder.setup(context);\n\n // Same test code runs without hitting real backend!\n await page.goto('/');\n await page.fill('#movie-name', 'Inception');\n await page.click('#add-movie');\n});\n```\n\nThat's it! Your tests now run completely offline using recorded network traffic.\n\n## Pattern Examples\n\n### Example 1: Basic Record and Playback\n\n**Context**: The fundamental pattern - record traffic once, play back for all subsequent runs.\n\n**Implementation**:\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/network-recorder/fixtures';\n\n// Set mode in test file (recommended)\nprocess.env.PW_NET_MODE = 'playback'; // or 'record'\n\ntest('CRUD operations work offline', async ({ page, context, networkRecorder }) => {\n // Setup recorder (records or plays back based on PW_NET_MODE)\n await networkRecorder.setup(context);\n\n await page.goto('/');\n\n // First time (record mode): Records all network traffic to HAR\n // Subsequent runs (playback mode): Plays back from HAR (no backend!)\n await page.fill('#movie-name', 'Inception');\n await page.click('#add-movie');\n\n // Intelligent CRUD detection makes this work offline!\n await expect(page.getByText('Inception')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- `PW_NET_MODE=record` captures traffic to HAR files\n- `PW_NET_MODE=playback` replays from HAR files\n- Set mode in test file or via environment variable\n- HAR files auto-organized by test name\n- Stateful mocking detects CRUD operations\n\n### Example 2: Complete CRUD Flow with HAR\n\n**Context**: Full create-read-update-delete flow that works completely offline.\n\n**Implementation**:\n\n```typescript\nprocess.env.PW_NET_MODE = 'playback';\n\ntest.describe('Movie CRUD - offline with network recorder', () => {\n test.beforeEach(async ({ page, networkRecorder, context }) => {\n await networkRecorder.setup(context);\n await page.goto('/');\n });\n\n test('should add, edit, delete movie browser-only', async ({ page, interceptNetworkCall }) => {\n // Create\n await page.fill('#movie-name', 'Inception');\n await page.fill('#year', '2010');\n await page.click('#add-movie');\n\n // Verify create (reads from stateful HAR)\n await expect(page.getByText('Inception')).toBeVisible();\n\n // Update\n await page.getByText('Inception').click();\n await page.fill('#movie-name', \"Inception Director's Cut\");\n\n const updateCall = interceptNetworkCall({\n method: 'PUT',\n url: '/movies/*',\n });\n\n await page.click('#save');\n await updateCall; // Wait for update\n\n // Verify update (HAR reflects state change!)\n await page.click('#back');\n await expect(page.getByText(\"Inception Director's Cut\")).toBeVisible();\n\n // Delete\n await page.click(`[data-testid=\"delete-Inception Director's Cut\"]`);\n\n // Verify delete (HAR reflects removal!)\n await expect(page.getByText(\"Inception Director's Cut\")).not.toBeVisible();\n });\n});\n```\n\n**Key Points**:\n\n- Full CRUD operations work offline\n- Stateful HAR mocking tracks creates/updates/deletes\n- Combine with `interceptNetworkCall` for deterministic waits\n- First run records, subsequent runs replay\n\n### Example 3: Common Patterns\n\n**Recording Only API Calls**:\n\n```typescript\nawait networkRecorder.setup(context, {\n recording: {\n urlFilter: /\\/api\\//, // Only record API calls, ignore static assets\n },\n});\n```\n\n**Playback with Fallback**:\n\n```typescript\nawait networkRecorder.setup(context, {\n playback: {\n fallback: true, // Fall back to live requests if HAR entry missing\n },\n});\n```\n\n**Custom HAR File Location**:\n\n```typescript\nawait networkRecorder.setup(context, {\n harFile: {\n harDir: 'recordings/api-calls',\n baseName: 'user-journey',\n organizeByTestFile: false, // Optional: flatten directory structure\n },\n});\n```\n\n**Directory Organization:**\n\n- `organizeByTestFile: true` (default): `har-files/test-file-name/baseName-test-title.har`\n- `organizeByTestFile: false`: `har-files/baseName-test-title.har`\n\n### Example 4: Response Content Storage - Embed vs Attach\n\n**Context**: Choose how response content is stored in HAR files.\n\n**`embed` (Default - Recommended):**\n\n```typescript\nawait networkRecorder.setup(context, {\n recording: {\n content: 'embed', // Store content inline (default)\n },\n});\n```\n\n**Pros:**\n\n- Single self-contained file - Easy to share, version control\n- Better for small-medium responses (API JSON, HTML pages)\n- HAR specification compliant\n\n**Cons:**\n\n- Larger HAR files\n- Not ideal for large binary content (images, videos)\n\n**`attach` (Alternative):**\n\n```typescript\nawait networkRecorder.setup(context, {\n recording: {\n content: 'attach', // Store content separately\n },\n});\n```\n\n**Pros:**\n\n- Smaller HAR files\n- Better for large responses (images, videos, documents)\n\n**Cons:**\n\n- Multiple files to manage\n- Harder to share\n\n**When to Use Each:**\n\n| Use `embed` (default) when | Use `attach` when |\n| ----------------------------------- | ------------------------------- |\n| Recording API responses (JSON, XML) | Recording large images, videos |\n| Small to medium HTML pages | HAR file size >50MB |\n| You want a single, portable file | Maximum disk efficiency needed |\n| Sharing HAR files with team | Working with ZIP archive output |\n\n### Example 5: Cross-Environment Compatibility (URL Mapping)\n\n**Context**: Record in dev environment, play back in CI with different base URLs.\n\n**The Problem**: HAR files contain URLs for the recording environment (e.g., `dev.example.com`). Playing back on a different environment fails.\n\n**Simple Hostname Mapping:**\n\n```typescript\nawait networkRecorder.setup(context, {\n playback: {\n urlMapping: {\n hostMapping: {\n 'preview.example.com': 'dev.example.com',\n 'staging.example.com': 'dev.example.com',\n 'localhost:3000': 'dev.example.com',\n },\n },\n },\n});\n```\n\n**Pattern-Based Mapping (Recommended):**\n\n```typescript\nawait networkRecorder.setup(context, {\n playback: {\n urlMapping: {\n patterns: [\n // Map any preview-XXXX subdomain to dev\n { match: /preview-\\d+\\.example\\.com/, replace: 'dev.example.com' },\n ],\n },\n },\n});\n```\n\n**Custom Function:**\n\n```typescript\nawait networkRecorder.setup(context, {\n playback: {\n urlMapping: {\n mapUrl: (url) => url.replace('staging.example.com', 'dev.example.com'),\n },\n },\n});\n```\n\n**Complex Multi-Environment Example:**\n\n```typescript\nawait networkRecorder.setup(context, {\n playback: {\n urlMapping: {\n hostMapping: {\n 'localhost:3000': 'admin.example.com',\n 'admin-staging.example.com': 'admin.example.com',\n 'admin.example.com': 'admin.example.com',\n },\n patterns: [\n { match: /admin-\\d+\\.example\\.com/, replace: 'admin.example.com' },\n { match: /admin-staging-pr-\\w+-\\d\\.example\\.com/, replace: 'admin.example.com' },\n ],\n },\n },\n});\n```\n\n**Benefits:**\n\n- Record once on dev, all environments map back to recordings\n- CORS headers automatically updated based on request origin\n- Debug with: `LOG_LEVEL=debug npm run test`\n\n## Why Use This Instead of Native Playwright?\n\n| Native Playwright (`routeFromHAR`) | network-recorder Utility |\n| ---------------------------------- | ------------------------------ |\n| ~80 lines setup boilerplate | ~5 lines total |\n| Manual HAR file management | Automatic file organization |\n| Complex setup/teardown | Automatic cleanup via fixtures |\n| **Read-only tests only** | **Full CRUD support** |\n| **Stateless** | **Stateful mocking** |\n| Manual URL mapping | Automatic environment mapping |\n\n### The game-changer: Stateful CRUD detection\n\nNative Playwright HAR playback is stateless - a POST create followed by GET list won't show the created item. This utility intelligently tracks CRUD operations in memory to reflect state changes, making offline tests behave like real APIs.\n\n## How Stateful CRUD Detection Works\n\nWhen in playback mode, the Network Recorder automatically analyzes your HAR file to detect CRUD patterns. If it finds:\n\n- Multiple GET requests to the same resource endpoint (e.g., `/movies`)\n- Mutation operations (POST, PUT, DELETE) to those resources\n- Evidence of state changes between identical requests\n\nIt automatically switches from static HAR playback to an intelligent stateful mock that:\n\n- Maintains state across requests\n- Auto-generates IDs for new resources\n- Returns proper 404s for deleted resources\n- Supports polling scenarios where state changes over time\n\n**This happens automatically - no configuration needed!**\n\n## API Reference\n\n### NetworkRecorder Methods\n\n| Method | Return Type | Description |\n| -------------------- | ------------------------ | --------------------------------------------- |\n| `setup(context)` | `Promise\u003cvoid>` | Sets up recording/playback on browser context |\n| `cleanup()` | `Promise\u003cvoid>` | Flushes data to disk and cleans up memory |\n| `getContext()` | `NetworkRecorderContext` | Gets current recorder context information |\n| `getStatusMessage()` | `string` | Gets human-readable status message |\n| `getHarStats()` | `Promise\u003cHarFileStats>` | Gets HAR file statistics and metadata |\n\n### Understanding `cleanup()`\n\nThe `cleanup()` method performs memory and resource cleanup - **it does NOT delete HAR files**:\n\n**What it does:**\n\n- Flushes recorded data to disk (writes HAR file in recording mode)\n- Releases file locks\n- Clears in-memory data\n- Resets internal state\n\n**What it does NOT do:**\n\n- Delete HAR files from disk\n- Remove recorded network traffic\n- Clear browser context or cookies\n\n### Configuration Options\n\n```typescript\ntype NetworkRecorderConfig = {\n harFile?: {\n harDir?: string; // Directory for HAR files (default: 'har-files')\n baseName?: string; // Base name for HAR files (default: 'network-traffic')\n organizeByTestFile?: boolean; // Organize by test file (default: true)\n };\n\n recording?: {\n content?: 'embed' | 'attach'; // Response content handling (default: 'embed')\n urlFilter?: string | RegExp; // URL filter for recording\n update?: boolean; // Update existing HAR files (default: false)\n };\n\n playback?: {\n fallback?: boolean; // Fall back to live requests (default: false)\n urlFilter?: string | RegExp; // URL filter for playback\n updateMode?: boolean; // Update mode during playback (default: false)\n };\n\n forceMode?: 'record' | 'playback' | 'disabled';\n};\n```\n\n## Environment Configuration\n\nControl the recording mode using the `PW_NET_MODE` environment variable:\n\n```bash\n# Record mode - captures network traffic to HAR files\nPW_NET_MODE=record npm run test:pw\n\n# Playback mode - replays network traffic from HAR files\nPW_NET_MODE=playback npm run test:pw\n\n# Disabled mode - no network recording/playback\nPW_NET_MODE=disabled npm run test:pw\n\n# Default behavior (when PW_NET_MODE is empty/unset) - same as disabled\nnpm run test:pw\n```\n\n**Tip**: We recommend setting `process.env.PW_NET_MODE` directly in your test file for better control.\n\n## Troubleshooting\n\n### HAR File Not Found\n\nIf you see \"HAR file not found\" errors during playback:\n\n1. Ensure you've recorded the test first with `PW_NET_MODE=record`\n2. Check the HAR file exists in the expected location (usually `har-files/`)\n3. Enable fallback mode: `playback: { fallback: true }`\n\n### Authentication and Network Recording\n\nThe network recorder works seamlessly with authentication:\n\n```typescript\ntest('Authenticated recording', async ({ page, context, authSession, networkRecorder }) => {\n // First authenticate\n await authSession.login('testuser', 'password');\n\n // Then setup network recording with authenticated context\n await networkRecorder.setup(context);\n\n // Test authenticated flows\n await page.goto('/dashboard');\n});\n```\n\n### Concurrent Test Issues\n\nThe recorder includes built-in file locking for safe parallel execution. Each test gets its own HAR file based on the test name.\n\n## Integration with Other Utilities\n\n**With interceptNetworkCall (deterministic waits):**\n\n```typescript\ntest('use both utilities', async ({ page, context, networkRecorder, interceptNetworkCall }) => {\n await networkRecorder.setup(context);\n\n const createCall = interceptNetworkCall({\n method: 'POST',\n url: '/api/movies',\n });\n\n await page.click('#add-movie');\n await createCall; // Wait for create (works with HAR!)\n\n // Network recorder provides playback, intercept provides determinism\n});\n```\n\n## Related Fragments\n\n- `overview.md` - Installation and fixture patterns\n- `intercept-network-call.md` - Combine for deterministic offline tests\n- `auth-session.md` - Record authenticated traffic\n- `network-first.md` - Core pattern for intercept-before-navigate\n\n## Anti-Patterns\n\n**DON'T mix record and playback in same test:**\n\n```typescript\nprocess.env.PW_NET_MODE = 'record';\n// ... some test code ...\nprocess.env.PW_NET_MODE = 'playback'; // Don't switch mid-test\n```\n\n**DO use one mode per test:**\n\n```typescript\nprocess.env.PW_NET_MODE = 'playback'; // Set once at top\n\ntest('my test', async ({ page, context, networkRecorder }) => {\n await networkRecorder.setup(context);\n // Entire test uses playback mode\n});\n```\n\n**DON'T forget to call setup:**\n\n```typescript\ntest('broken', async ({ page, networkRecorder }) => {\n await page.goto('/'); // HAR not active!\n});\n```\n\n**DO always call setup before navigation:**\n\n```typescript\ntest('correct', async ({ page, context, networkRecorder }) => {\n await networkRecorder.setup(context); // Must setup first\n await page.goto('/'); // Now HAR is active\n});\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":15499,"content_sha256":"f86bd52defea901fcf2c3bc2ba78dcb97047e30a49e552a5935bc8452851d33c"},{"filename":"knowledge/nfr-criteria.md","content":"# Non-Functional Requirements (NFR) Criteria\n\n## Principle\n\nNon-functional requirements (security, performance, reliability, maintainability) are **validated through automated tests**, not checklists. NFR assessment uses objective pass/fail criteria tied to measurable thresholds. Ambiguous requirements default to CONCERNS until clarified.\n\n## Rationale\n\n**The Problem**: Teams ship features that \"work\" functionally but fail under load, expose security vulnerabilities, or lack error recovery. NFRs are treated as optional \"nice-to-haves\" instead of release blockers.\n\n**The Solution**: Define explicit NFR criteria with automated validation. Security tests verify auth/authz and secret handling. Performance tests enforce SLO/SLA thresholds with profiling evidence. Reliability tests validate error handling, retries, and health checks. Maintainability is measured by test coverage, code duplication, and observability.\n\n**Why This Matters**:\n\n- Prevents production incidents (security breaches, performance degradation, cascading failures)\n- Provides objective release criteria (no subjective \"feels fast enough\")\n- Automates compliance validation (audit trail for regulated environments)\n- Forces clarity on ambiguous requirements (default to CONCERNS)\n\n## Pattern Examples\n\n### Example 1: Security NFR Validation (Auth, Secrets, OWASP)\n\n**Context**: Automated security tests enforcing authentication, authorization, and secret handling\n\n**Implementation**:\n\n```typescript\n// tests/nfr/security.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Security NFR: Authentication & Authorization', () => {\n test('unauthenticated users cannot access protected routes', async ({ page }) => {\n // Attempt to access dashboard without auth\n await page.goto('/dashboard');\n\n // Should redirect to login (not expose data)\n await expect(page).toHaveURL(/\\/login/);\n await expect(page.getByText('Please sign in')).toBeVisible();\n\n // Verify no sensitive data leaked in response\n const pageContent = await page.content();\n expect(pageContent).not.toContain('user_id');\n expect(pageContent).not.toContain('api_key');\n });\n\n test('JWT tokens expire after 15 minutes', async ({ page, request }) => {\n // Login and capture token\n await page.goto('/login');\n await page.getByLabel('Email').fill('[email protected]');\n await page.getByLabel('Password').fill('ValidPass123!');\n await page.getByRole('button', { name: 'Sign In' }).click();\n\n const token = await page.evaluate(() => localStorage.getItem('auth_token'));\n expect(token).toBeTruthy();\n\n // Wait 16 minutes (use mock clock in real tests)\n await page.clock.fastForward('00:16:00');\n\n // Token should be expired, API call should fail\n const response = await request.get('/api/user/profile', {\n headers: { Authorization: `Bearer ${token}` },\n });\n\n expect(response.status()).toBe(401);\n const body = await response.json();\n expect(body.error).toContain('expired');\n });\n\n test('passwords are never logged or exposed in errors', async ({ page }) => {\n // Trigger login error\n await page.goto('/login');\n await page.getByLabel('Email').fill('[email protected]');\n await page.getByLabel('Password').fill('WrongPassword123!');\n\n // Monitor console for password leaks\n const consoleLogs: string[] = [];\n page.on('console', (msg) => consoleLogs.push(msg.text()));\n\n await page.getByRole('button', { name: 'Sign In' }).click();\n\n // Error shown to user (generic message)\n await expect(page.getByText('Invalid credentials')).toBeVisible();\n\n // Verify password NEVER appears in console, DOM, or network\n const pageContent = await page.content();\n expect(pageContent).not.toContain('WrongPassword123!');\n expect(consoleLogs.join('\\n')).not.toContain('WrongPassword123!');\n });\n\n test('RBAC: users can only access resources they own', async ({ page, request }) => {\n // Login as User A\n const userAToken = await login(request, '[email protected]', 'password');\n\n // Try to access User B's order\n const response = await request.get('/api/orders/user-b-order-id', {\n headers: { Authorization: `Bearer ${userAToken}` },\n });\n\n expect(response.status()).toBe(403); // Forbidden\n const body = await response.json();\n expect(body.error).toContain('insufficient permissions');\n });\n\n test('SQL injection attempts are blocked', async ({ page }) => {\n await page.goto('/search');\n\n // Attempt SQL injection\n await page.getByPlaceholder('Search products').fill(\"'; DROP TABLE users; --\");\n await page.getByRole('button', { name: 'Search' }).click();\n\n // Should return empty results, NOT crash or expose error\n await expect(page.getByText('No results found')).toBeVisible();\n\n // Verify app still works (table not dropped)\n await page.goto('/dashboard');\n await expect(page.getByText('Welcome')).toBeVisible();\n });\n\n test('XSS attempts are sanitized', async ({ page }) => {\n await page.goto('/profile/edit');\n\n // Attempt XSS injection\n const xssPayload = '\u003cscript>alert(\"XSS\")\u003c/script>';\n await page.getByLabel('Bio').fill(xssPayload);\n await page.getByRole('button', { name: 'Save' }).click();\n\n // Reload and verify XSS is escaped (not executed)\n await page.reload();\n const bio = await page.getByTestId('user-bio').textContent();\n\n // Text should be escaped, script should NOT execute\n expect(bio).toContain('<script>');\n expect(bio).not.toContain('\u003cscript>');\n });\n});\n\n// Helper\nasync function login(request: any, email: string, password: string): Promise\u003cstring> {\n const response = await request.post('/api/auth/login', {\n data: { email, password },\n });\n const body = await response.json();\n return body.token;\n}\n```\n\n**Key Points**:\n\n- Authentication: Unauthenticated access redirected (not exposed)\n- Authorization: RBAC enforced (403 for insufficient permissions)\n- Token expiry: JWT expires after 15 minutes (automated validation)\n- Secret handling: Passwords never logged or exposed in errors\n- OWASP Top 10: SQL injection and XSS blocked (input sanitization)\n\n**Security NFR Criteria**:\n\n- ✅ PASS: All 6 tests green (auth, authz, token expiry, secret handling, SQL injection, XSS)\n- ⚠️ CONCERNS: 1-2 tests failing with mitigation plan and owner assigned\n- ❌ FAIL: Critical exposure (unauthenticated access, password leak, SQL injection succeeds)\n\n---\n\n### Example 2: Performance NFR Validation (k6 Load Testing for SLO/SLA)\n\n**Context**: Use k6 for load testing, stress testing, and SLO/SLA enforcement (NOT Playwright)\n\n**Implementation**:\n\n```javascript\n// tests/nfr/performance.k6.js\nimport http from 'k6/http';\nimport { check, sleep } from 'k6';\nimport { Rate, Trend } from 'k6/metrics';\n\n// Custom metrics\nconst errorRate = new Rate('errors');\nconst apiDuration = new Trend('api_duration');\n\n// Performance thresholds (SLO/SLA)\nexport const options = {\n stages: [\n { duration: '1m', target: 50 }, // Ramp up to 50 users\n { duration: '3m', target: 50 }, // Stay at 50 users for 3 minutes\n { duration: '1m', target: 100 }, // Spike to 100 users\n { duration: '3m', target: 100 }, // Stay at 100 users\n { duration: '1m', target: 0 }, // Ramp down\n ],\n thresholds: {\n // SLO: 95% of requests must complete in \u003c500ms\n http_req_duration: ['p(95)\u003c500'],\n // SLO: Error rate must be \u003c1%\n errors: ['rate\u003c0.01'],\n // SLA: API endpoints must respond in \u003c1s (99th percentile)\n api_duration: ['p(99)\u003c1000'],\n },\n};\n\nexport default function () {\n // Test 1: Homepage load performance\n const homepageResponse = http.get(`${__ENV.BASE_URL}/`);\n check(homepageResponse, {\n 'homepage status is 200': (r) => r.status === 200,\n 'homepage loads in \u003c2s': (r) => r.timings.duration \u003c 2000,\n });\n errorRate.add(homepageResponse.status !== 200);\n\n // Test 2: API endpoint performance\n const apiResponse = http.get(`${__ENV.BASE_URL}/api/products?limit=10`, {\n headers: { Authorization: `Bearer ${__ENV.API_TOKEN}` },\n });\n check(apiResponse, {\n 'API status is 200': (r) => r.status === 200,\n 'API responds in \u003c500ms': (r) => r.timings.duration \u003c 500,\n });\n apiDuration.add(apiResponse.timings.duration);\n errorRate.add(apiResponse.status !== 200);\n\n // Test 3: Search endpoint under load\n const searchResponse = http.get(`${__ENV.BASE_URL}/api/search?q=laptop&limit=100`);\n check(searchResponse, {\n 'search status is 200': (r) => r.status === 200,\n 'search responds in \u003c1s': (r) => r.timings.duration \u003c 1000,\n 'search returns results': (r) => JSON.parse(r.body).results.length > 0,\n });\n errorRate.add(searchResponse.status !== 200);\n\n sleep(1); // Realistic user think time\n}\n\n// Threshold validation (run after test)\nexport function handleSummary(data) {\n const p95Duration = data.metrics.http_req_duration.values['p(95)'];\n const p99ApiDuration = data.metrics.api_duration.values['p(99)'];\n const errorRateValue = data.metrics.errors.values.rate;\n\n console.log(`P95 request duration: ${p95Duration.toFixed(2)}ms`);\n console.log(`P99 API duration: ${p99ApiDuration.toFixed(2)}ms`);\n console.log(`Error rate: ${(errorRateValue * 100).toFixed(2)}%`);\n\n return {\n 'summary.json': JSON.stringify(data),\n stdout: `\nPerformance NFR Results:\n- P95 request duration: ${p95Duration \u003c 500 ? '✅ PASS' : '❌ FAIL'} (${p95Duration.toFixed(2)}ms / 500ms threshold)\n- P99 API duration: ${p99ApiDuration \u003c 1000 ? '✅ PASS' : '❌ FAIL'} (${p99ApiDuration.toFixed(2)}ms / 1000ms threshold)\n- Error rate: ${errorRateValue \u003c 0.01 ? '✅ PASS' : '❌ FAIL'} (${(errorRateValue * 100).toFixed(2)}% / 1% threshold)\n `,\n };\n}\n```\n\n**Run k6 tests:**\n\n```bash\n# Local smoke test (10 VUs, 30s)\nk6 run --vus 10 --duration 30s tests/nfr/performance.k6.js\n\n# Full load test (stages defined in script)\nk6 run tests/nfr/performance.k6.js\n\n# CI integration with thresholds\nk6 run --out json=performance-results.json tests/nfr/performance.k6.js\n```\n\n**Key Points**:\n\n- **k6 is the right tool** for load testing (NOT Playwright)\n- SLO/SLA thresholds enforced automatically (`p(95)\u003c500`, `rate\u003c0.01`)\n- Realistic load simulation (ramp up, sustained load, spike testing)\n- Comprehensive metrics (p50, p95, p99, error rate, throughput)\n- CI-friendly (JSON output, exit codes based on thresholds)\n\n**Performance NFR Criteria**:\n\n- ✅ PASS: All SLO/SLA targets met with k6 profiling evidence (p95 \u003c 500ms, error rate \u003c 1%)\n- ⚠️ CONCERNS: Trending toward limits (e.g., p95 = 480ms approaching 500ms) or missing baselines\n- ❌ FAIL: SLO/SLA breached (e.g., p95 > 500ms) or error rate > 1%\n\n**Performance Testing Levels (from Test Architect course):**\n\n- **Load testing**: System behavior under expected load\n- **Stress testing**: System behavior under extreme load (breaking point)\n- **Spike testing**: Sudden load increases (traffic spikes)\n- **Endurance/Soak testing**: System behavior under sustained load (memory leaks, resource exhaustion)\n- **Benchmarking**: Baseline measurements for comparison\n\n**Note**: Playwright can validate **perceived performance** (Core Web Vitals via Lighthouse), but k6 validates **system performance** (throughput, latency, resource limits under load)\n\n---\n\n### Example 3: Reliability NFR Validation (Playwright for UI Resilience)\n\n**Context**: Automated reliability tests validating graceful degradation and recovery paths\n\n**Implementation**:\n\n```typescript\n// tests/nfr/reliability.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Reliability NFR: Error Handling & Recovery', () => {\n test('app remains functional when API returns 500 error', async ({ page, context }) => {\n // Mock API failure\n await context.route('**/api/products', (route) => {\n route.fulfill({ status: 500, body: JSON.stringify({ error: 'Internal Server Error' }) });\n });\n\n await page.goto('/products');\n\n // User sees error message (not blank page or crash)\n await expect(page.getByText('Unable to load products. Please try again.')).toBeVisible();\n await expect(page.getByRole('button', { name: 'Retry' })).toBeVisible();\n\n // App navigation still works (graceful degradation)\n await page.getByRole('link', { name: 'Home' }).click();\n await expect(page).toHaveURL('/');\n });\n\n test('API client retries on transient failures (3 attempts)', async ({ page, context }) => {\n let attemptCount = 0;\n\n await context.route('**/api/checkout', (route) => {\n attemptCount++;\n\n // Fail first 2 attempts, succeed on 3rd\n if (attemptCount \u003c 3) {\n route.fulfill({ status: 503, body: JSON.stringify({ error: 'Service Unavailable' }) });\n } else {\n route.fulfill({ status: 200, body: JSON.stringify({ orderId: '12345' }) });\n }\n });\n\n await page.goto('/checkout');\n await page.getByRole('button', { name: 'Place Order' }).click();\n\n // Should succeed after 3 attempts\n await expect(page.getByText('Order placed successfully')).toBeVisible();\n expect(attemptCount).toBe(3);\n });\n\n test('app handles network disconnection gracefully', async ({ page, context }) => {\n await page.goto('/dashboard');\n\n // Simulate offline mode\n await context.setOffline(true);\n\n // Trigger action requiring network\n await page.getByRole('button', { name: 'Refresh Data' }).click();\n\n // User sees offline indicator (not crash)\n await expect(page.getByText('You are offline. Changes will sync when reconnected.')).toBeVisible();\n\n // Reconnect\n await context.setOffline(false);\n await page.getByRole('button', { name: 'Refresh Data' }).click();\n\n // Data loads successfully\n await expect(page.getByText('Data updated')).toBeVisible();\n });\n\n test('health check endpoint returns service status', async ({ request }) => {\n const response = await request.get('/api/health');\n\n expect(response.status()).toBe(200);\n\n const health = await response.json();\n expect(health).toHaveProperty('status', 'healthy');\n expect(health).toHaveProperty('timestamp');\n expect(health).toHaveProperty('services');\n\n // Verify critical services are monitored\n expect(health.services).toHaveProperty('database');\n expect(health.services).toHaveProperty('cache');\n expect(health.services).toHaveProperty('queue');\n\n // All services should be UP\n expect(health.services.database.status).toBe('UP');\n expect(health.services.cache.status).toBe('UP');\n expect(health.services.queue.status).toBe('UP');\n });\n\n test('circuit breaker opens after 5 consecutive failures', async ({ page, context }) => {\n let failureCount = 0;\n\n await context.route('**/api/recommendations', (route) => {\n failureCount++;\n route.fulfill({ status: 500, body: JSON.stringify({ error: 'Service Error' }) });\n });\n\n await page.goto('/product/123');\n\n // Wait for circuit breaker to open (fallback UI appears)\n await expect(page.getByText('Recommendations temporarily unavailable')).toBeVisible({ timeout: 10000 });\n\n // Verify circuit breaker stopped making requests after threshold (should be ≤5)\n expect(failureCount).toBeLessThanOrEqual(5);\n });\n\n test('rate limiting gracefully handles 429 responses', async ({ page, context }) => {\n let requestCount = 0;\n\n await context.route('**/api/search', (route) => {\n requestCount++;\n\n if (requestCount > 10) {\n // Rate limit exceeded\n route.fulfill({\n status: 429,\n headers: { 'Retry-After': '5' },\n body: JSON.stringify({ error: 'Rate limit exceeded' }),\n });\n } else {\n route.fulfill({ status: 200, body: JSON.stringify({ results: [] }) });\n }\n });\n\n await page.goto('/search');\n\n // Make 15 search requests rapidly\n for (let i = 0; i \u003c 15; i++) {\n await page.getByPlaceholder('Search').fill(`query-${i}`);\n await page.getByRole('button', { name: 'Search' }).click();\n }\n\n // User sees rate limit message (not crash)\n await expect(page.getByText('Too many requests. Please wait a moment.')).toBeVisible();\n });\n});\n```\n\n**Key Points**:\n\n- Error handling: Graceful degradation (500 error → user-friendly message + retry button)\n- Retries: 3 attempts on transient failures (503 → eventual success)\n- Offline handling: Network disconnection detected (sync when reconnected)\n- Health checks: `/api/health` monitors database, cache, queue\n- Circuit breaker: Opens after 5 failures (fallback UI, stop retries)\n- Rate limiting: 429 response handled (Retry-After header respected)\n\n**Reliability NFR Criteria**:\n\n- ✅ PASS: Error handling, retries, health checks verified (all 6 tests green)\n- ⚠️ CONCERNS: Partial coverage (e.g., missing circuit breaker) or no telemetry\n- ❌ FAIL: No recovery path (500 error crashes app) or unresolved crash scenarios\n\n---\n\n### Example 4: Maintainability NFR Validation (CI Tools, Not Playwright)\n\n**Context**: Use proper CI tools for code quality validation (coverage, duplication, vulnerabilities)\n\n**Implementation**:\n\n```yaml\n# .github/workflows/nfr-maintainability.yml\nname: NFR - Maintainability\n\non: [push, pull_request]\n\njobs:\n test-coverage:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n\n - name: Install dependencies\n run: npm ci\n\n - name: Run tests with coverage\n run: npm run test:coverage\n\n - name: Check coverage threshold (80% minimum)\n run: |\n COVERAGE=$(jq '.total.lines.pct' coverage/coverage-summary.json)\n echo \"Coverage: $COVERAGE%\"\n if (( $(echo \"$COVERAGE \u003c 80\" | bc -l) )); then\n echo \"❌ FAIL: Coverage $COVERAGE% below 80% threshold\"\n exit 1\n else\n echo \"✅ PASS: Coverage $COVERAGE% meets 80% threshold\"\n fi\n\n code-duplication:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n\n - name: Check code duplication (\u003c5% allowed)\n run: |\n npx jscpd src/ --threshold 5 --format json --output duplication.json\n DUPLICATION=$(jq '.statistics.total.percentage' duplication.json)\n echo \"Duplication: $DUPLICATION%\"\n if (( $(echo \"$DUPLICATION >= 5\" | bc -l) )); then\n echo \"❌ FAIL: Duplication $DUPLICATION% exceeds 5% threshold\"\n exit 1\n else\n echo \"✅ PASS: Duplication $DUPLICATION% below 5% threshold\"\n fi\n\n vulnerability-scan:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n\n - name: Install dependencies\n run: npm ci\n\n - name: Run npm audit (no critical/high vulnerabilities)\n run: |\n npm audit --json > audit.json || true\n CRITICAL=$(jq '.metadata.vulnerabilities.critical' audit.json)\n HIGH=$(jq '.metadata.vulnerabilities.high' audit.json)\n echo \"Critical: $CRITICAL, High: $HIGH\"\n if [ \"$CRITICAL\" -gt 0 ] || [ \"$HIGH\" -gt 0 ]; then\n echo \"❌ FAIL: Found $CRITICAL critical and $HIGH high vulnerabilities\"\n npm audit\n exit 1\n else\n echo \"✅ PASS: No critical/high vulnerabilities\"\n fi\n```\n\n**Playwright Tests for Observability (E2E Validation):**\n\n```typescript\n// tests/nfr/observability.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Maintainability NFR: Observability Validation', () => {\n test('critical errors are reported to monitoring service', async ({ page, context }) => {\n const sentryEvents: any[] = [];\n\n // Mock Sentry SDK to verify error tracking\n await context.addInitScript(() => {\n (window as any).Sentry = {\n captureException: (error: Error) => {\n console.log('SENTRY_CAPTURE:', JSON.stringify({ message: error.message, stack: error.stack }));\n },\n };\n });\n\n page.on('console', (msg) => {\n if (msg.text().includes('SENTRY_CAPTURE:')) {\n sentryEvents.push(JSON.parse(msg.text().replace('SENTRY_CAPTURE:', '')));\n }\n });\n\n // Trigger error by mocking API failure\n await context.route('**/api/products', (route) => {\n route.fulfill({ status: 500, body: JSON.stringify({ error: 'Database Error' }) });\n });\n\n await page.goto('/products');\n\n // Wait for error UI and Sentry capture\n await expect(page.getByText('Unable to load products')).toBeVisible();\n\n // Verify error was captured by monitoring\n expect(sentryEvents.length).toBeGreaterThan(0);\n expect(sentryEvents[0]).toHaveProperty('message');\n expect(sentryEvents[0]).toHaveProperty('stack');\n });\n\n test('API response times are tracked in telemetry', async ({ request }) => {\n const response = await request.get('/api/products?limit=10');\n\n expect(response.ok()).toBeTruthy();\n\n // Verify Server-Timing header for APM (Application Performance Monitoring)\n const serverTiming = response.headers()['server-timing'];\n\n expect(serverTiming).toBeTruthy();\n expect(serverTiming).toContain('db'); // Database query time\n expect(serverTiming).toContain('total'); // Total processing time\n });\n\n test('structured logging present in application', async ({ request }) => {\n // Make API call that generates logs\n const response = await request.post('/api/orders', {\n data: { productId: '123', quantity: 2 },\n });\n\n expect(response.ok()).toBeTruthy();\n\n // Note: In real scenarios, validate logs in monitoring system (Datadog, CloudWatch)\n // This test validates the logging contract exists (Server-Timing, trace IDs in headers)\n const traceId = response.headers()['x-trace-id'];\n expect(traceId).toBeTruthy(); // Confirms structured logging with correlation IDs\n });\n});\n```\n\n**Key Points**:\n\n- **Coverage/duplication**: CI jobs (GitHub Actions), not Playwright tests\n- **Vulnerability scanning**: npm audit in CI, not Playwright tests\n- **Observability**: Playwright validates error tracking (Sentry) and telemetry headers\n- **Structured logging**: Validate logging contract (trace IDs, Server-Timing headers)\n- **Separation of concerns**: Build-time checks (coverage, audit) vs runtime checks (error tracking, telemetry)\n\n**Maintainability NFR Criteria**:\n\n- ✅ PASS: Clean code (80%+ coverage from CI, \u003c5% duplication from CI), observability validated in E2E, no critical vulnerabilities from npm audit\n- ⚠️ CONCERNS: Duplication >5%, coverage 60-79%, or unclear ownership\n- ❌ FAIL: Absent tests (\u003c60%), tangled implementations (>10% duplication), or no observability\n\n---\n\n## NFR Assessment Checklist\n\nBefore release gate:\n\n- [ ] **Security** (Playwright E2E + Security Tools):\n - [ ] Auth/authz tests green (unauthenticated redirect, RBAC enforced)\n - [ ] Secrets never logged or exposed in errors\n - [ ] OWASP Top 10 validated (SQL injection blocked, XSS sanitized)\n - [ ] Security audit completed (vulnerability scan, penetration test if applicable)\n\n- [ ] **Performance** (k6 Load Testing):\n - [ ] SLO/SLA targets met with k6 evidence (p95 \u003c500ms, error rate \u003c1%)\n - [ ] Load testing completed (expected load)\n - [ ] Stress testing completed (breaking point identified)\n - [ ] Spike testing completed (handles traffic spikes)\n - [ ] Endurance testing completed (no memory leaks under sustained load)\n\n- [ ] **Reliability** (Playwright E2E + API Tests):\n - [ ] Error handling graceful (500 → user-friendly message + retry)\n - [ ] Retries implemented (3 attempts on transient failures)\n - [ ] Health checks monitored (/api/health endpoint)\n - [ ] Circuit breaker tested (opens after failure threshold)\n - [ ] Offline handling validated (network disconnection graceful)\n\n- [ ] **Maintainability** (CI Tools):\n - [ ] Test coverage ≥80% (from CI coverage report)\n - [ ] Code duplication \u003c5% (from jscpd CI job)\n - [ ] No critical/high vulnerabilities (from npm audit CI job)\n - [ ] Structured logging validated (Playwright validates telemetry headers)\n - [ ] Error tracking configured (Sentry/monitoring integration validated)\n\n- [ ] **Ambiguous requirements**: Default to CONCERNS (force team to clarify thresholds and evidence)\n- [ ] **NFR criteria documented**: Measurable thresholds defined (not subjective \"fast enough\")\n- [ ] **Automated validation**: NFR tests run in CI pipeline (not manual checklists)\n- [ ] **Tool selection**: Right tool for each NFR (k6 for performance, Playwright for security/reliability E2E, CI tools for maintainability)\n\n## NFR Gate Decision Matrix\n\n| Category | PASS Criteria | CONCERNS Criteria | FAIL Criteria |\n| ------------------- | -------------------------------------------- | -------------------------------------------- | ---------------------------------------------- |\n| **Security** | Auth/authz, secret handling, OWASP verified | Minor gaps with clear owners | Critical exposure or missing controls |\n| **Performance** | Metrics meet SLO/SLA with profiling evidence | Trending toward limits or missing baselines | SLO/SLA breached or resource leaks detected |\n| **Reliability** | Error handling, retries, health checks OK | Partial coverage or missing telemetry | No recovery path or unresolved crash scenarios |\n| **Maintainability** | Clean code, tests, docs shipped together | Duplication, low coverage, unclear ownership | Absent tests, tangled code, no observability |\n\n**Default**: If targets or evidence are undefined → **CONCERNS** (force team to clarify before sign-off)\n\n## Integration Points\n\n- **Used in workflows**: `*nfr-assess` (automated NFR validation), `*trace` (gate decision Phase 2), `*test-design` (NFR risk assessment via Utility Tree)\n- **Related fragments**: `risk-governance.md` (NFR risk scoring), `probability-impact.md` (NFR impact assessment), `test-quality.md` (maintainability standards), `test-levels-framework.md` (system-level testing for NFRs)\n- **Tools by NFR Category**:\n - **Security**: Playwright (E2E auth/authz), OWASP ZAP, Burp Suite, npm audit, Snyk\n - **Performance**: k6 (load/stress/spike/endurance), Lighthouse (Core Web Vitals), Artillery\n - **Reliability**: Playwright (E2E error handling), API tests (retries, health checks), Chaos Engineering tools\n - **Maintainability**: GitHub Actions (coverage, duplication, audit), jscpd, Playwright (observability validation)\n\n_Source: Test Architect course (NFR testing approaches, Utility Tree, Quality Scenarios), ISO/IEC 25010 Software Quality Characteristics, OWASP Top 10, k6 documentation, SRE practices._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":26671,"content_sha256":"77a3a4711aea7a1efb3d4fe82a6842f53d761eeb3a909fd94e97182f781c8ee9"},{"filename":"knowledge/overview.md","content":"# Playwright Utils Overview\n\n## Principle\n\nUse production-ready, fixture-based utilities from `@seontechnologies/playwright-utils` for common Playwright testing patterns. Build test helpers as pure functions first, then wrap in framework-specific fixtures for composability and reuse. **Works equally well for pure API testing (no browser) and UI testing.**\n\n## Rationale\n\nWriting Playwright utilities from scratch for every project leads to:\n\n- Duplicated code across test suites\n- Inconsistent patterns and quality\n- Maintenance burden when Playwright APIs change\n- Missing advanced features (schema validation, HAR recording, auth persistence)\n\n`@seontechnologies/playwright-utils` provides:\n\n- **Production-tested**: Used in enterprise production environments\n- **Functional-first design**: Core logic as pure functions, fixtures for convenience\n- **Composable fixtures**: Use `mergeTests` to combine utilities\n- **TypeScript support**: Full type safety with generic types\n- **Comprehensive coverage**: API requests, auth, network, logging, file handling, burn-in\n- **Backend-first mentality**: Most utilities work without a browser - pure API/service testing is a first-class use case\n\n## Installation\n\n```bash\nnpm install -D @seontechnologies/playwright-utils\n```\n\n**Peer Dependencies:**\n\n- `@playwright/test` >= 1.54.1 (required)\n- `ajv` >= 8.0.0 (optional - for JSON Schema validation)\n- `zod` >= 3.0.0 (optional - for Zod schema validation)\n\n## Available Utilities\n\n### Core Testing Utilities\n\n| Utility | Purpose | Test Context |\n| -------------------------- | ----------------------------------------------------------------------------- | ------------------ |\n| **api-request** | Typed HTTP client with schema validation, retry, and operation-based overload | **API/Backend** |\n| **recurse** | Polling for async operations, background jobs | **API/Backend** |\n| **auth-session** | Token persistence, multi-user, service-to-service | **API/Backend/UI** |\n| **log** | Playwright report-integrated logging | **API/Backend/UI** |\n| **file-utils** | CSV/XLSX/PDF/ZIP reading & validation | **API/Backend/UI** |\n| **burn-in** | Smart test selection with git diff | **CI/CD** |\n| **network-recorder** | HAR record/playback for offline testing | UI only |\n| **intercept-network-call** | Network spy/stub with auto JSON parsing | UI only |\n| **network-error-monitor** | Automatic HTTP 4xx/5xx detection | UI only |\n\n**Note**: 7 of 10 utilities work without a browser. Only 3 are UI-specific (network-recorder, intercept-network-call, network-error-monitor).\n\n## Design Patterns\n\n### Pattern 1: Functional Core, Fixture Shell\n\n**Context**: All utilities follow the same architectural pattern - pure function as core, fixture as wrapper.\n\n**Implementation**:\n\n```typescript\n// Direct import (pass Playwright context explicitly)\nimport { apiRequest } from '@seontechnologies/playwright-utils';\n\ntest('direct usage', async ({ request }) => {\n const { status, body } = await apiRequest({\n request, // Must pass request context\n method: 'GET',\n path: '/api/users',\n });\n});\n\n// Fixture import (context injected automatically)\nimport { test } from '@seontechnologies/playwright-utils/api-request/fixtures';\n\ntest('fixture usage', async ({ apiRequest }) => {\n const { status, body } = await apiRequest({\n // No need to pass request context\n method: 'GET',\n path: '/api/users',\n });\n});\n```\n\n**Key Points**:\n\n- Pure functions testable without Playwright running\n- Fixtures inject framework dependencies automatically\n- Choose direct import (more control) or fixture (convenience)\n\n### Pattern 2: Subpath Imports for Tree-Shaking\n\n**Context**: Import only what you need to keep bundle sizes small.\n\n**Implementation**:\n\n```typescript\n// Import specific utility\nimport { apiRequest } from '@seontechnologies/playwright-utils/api-request';\n\n// Import specific fixture\nimport { test } from '@seontechnologies/playwright-utils/api-request/fixtures';\n\n// Import everything (use sparingly)\nimport { apiRequest, recurse, log } from '@seontechnologies/playwright-utils';\n```\n\n**Key Points**:\n\n- Subpath imports enable tree-shaking\n- Keep bundle sizes minimal\n- Import from specific paths for production builds\n\n### Pattern 3: Fixture Composition with mergeTests\n\n**Context**: Combine multiple playwright-utils fixtures with your own custom fixtures.\n\n**Implementation**:\n\n```typescript\n// playwright/support/merged-fixtures.ts\nimport { mergeTests } from '@playwright/test';\nimport { log } from '@seontechnologies/playwright-utils';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\n// Auth fixture built in your project (setAuthProvider + createAuthFixtures)\nimport { test as authFixture } from './auth-fixture';\n\n// Merge all fixtures into one test object\nexport const test = mergeTests(apiRequestFixture, authFixture, recurseFixture);\n\nexport { expect } from '@playwright/test';\nexport { log };\n```\n\n```typescript\n// In your tests\nimport { test, expect, log } from '../support/merged-fixtures';\n\ntest('all utilities available', async ({ apiRequest, authToken, recurse }) => {\n await log.step('Making authenticated API request');\n\n const { body } = await apiRequest({\n method: 'GET',\n path: '/api/protected',\n headers: { Authorization: `Bearer ${authToken}` },\n });\n\n await recurse(\n () => apiRequest({ method: 'GET', path: `/status/${body.id}` }),\n (res) => res.body.ready === true,\n );\n});\n```\n\n**Key Points**:\n\n- `mergeTests` combines multiple fixtures without conflicts\n- Create one merged-fixtures.ts file per project\n- Import test object from your merged fixtures in all tests\n- All utilities available in single test signature\n\n## Integration with Existing Tests\n\n### Gradual Adoption Strategy\n\n**1. Start with logging** (zero breaking changes):\n\n```typescript\nimport { log } from '@seontechnologies/playwright-utils';\n\ntest('existing test', async ({ page }) => {\n await log.step('Navigate to page'); // Just add logging\n await page.goto('/dashboard');\n // Rest of test unchanged\n});\n```\n\n**2. Add API utilities** (for API tests):\n\n```typescript\nimport { test } from '@seontechnologies/playwright-utils/api-request/fixtures';\n\ntest('API test', async ({ apiRequest }) => {\n const { status, body } = await apiRequest({\n method: 'GET',\n path: '/api/users',\n });\n\n expect(status).toBe(200);\n});\n```\n\n**3. Expand to network utilities** (for UI tests):\n\n```typescript\nimport { expect } from '@playwright/test';\nimport { test } from '@seontechnologies/playwright-utils/intercept-network-call/fixtures';\n\ntest('UI with network control', async ({ page, interceptNetworkCall }) => {\n const usersCall = interceptNetworkCall({\n url: '**/api/users',\n });\n\n await page.goto('/dashboard');\n const { responseJson } = await usersCall;\n\n expect(responseJson).toHaveLength(10);\n});\n```\n\n**4. Full integration** (merged fixtures):\n\nCreate merged-fixtures.ts and use across all tests.\n\n## Related Fragments\n\n- `api-request.md` - HTTP client with schema validation\n- `network-recorder.md` - HAR-based offline testing\n- `auth-session.md` - Token management\n- `intercept-network-call.md` - Network interception\n- `recurse.md` - Polling patterns\n- `log.md` - Logging utility\n- `file-utils.md` - File operations\n- `fixtures-composition.md` - Advanced mergeTests patterns\n\n## Anti-Patterns\n\n**❌ Don't mix direct and fixture imports in same test:**\n\n```typescript\nimport { apiRequest } from '@seontechnologies/playwright-utils';\n// Auth fixture built in your project (setAuthProvider + createAuthFixtures)\nimport { test } from './support/auth/auth-fixture';\n\ntest('bad', async ({ request, authToken }) => {\n // Confusing - mixing direct (needs request) and fixture (has authToken)\n await apiRequest({ request, method: 'GET', path: '/api/users' });\n});\n```\n\n**✅ Use consistent import style:**\n\n```typescript\nimport { test } from '../support/merged-fixtures';\n\ntest('good', async ({ apiRequest, authToken }) => {\n // Clean - all from fixtures\n await apiRequest({ method: 'GET', path: '/api/users' });\n});\n```\n\n**❌ Don't import everything when you need one utility:**\n\n```typescript\nimport * as utils from '@seontechnologies/playwright-utils'; // Large bundle\n```\n\n**✅ Use subpath imports:**\n\n```typescript\nimport { apiRequest } from '@seontechnologies/playwright-utils/api-request'; // Small bundle\n```\n\n## Reference Implementation\n\nThe official `@seontechnologies/playwright-utils` repository provides working examples of all patterns described in these fragments.\n\n**Repository:** \u003chttps://github.com/seontechnologies/playwright-utils>\n\n**Key resources:**\n\n- **Test examples:** `playwright/tests` - All utilities in action\n- **Framework setup:** `playwright.config.ts`, `playwright/support/merged-fixtures.ts`\n- **CI patterns:** `.github/workflows/` - GitHub Actions with sharding, parallelization\n\n**Quick start:**\n\n```bash\ngit clone https://github.com/seontechnologies/playwright-utils.git\ncd playwright-utils\nnvm use\nnpm install\nnpm run test:pw-ui # Explore tests with Playwright UI\nnpm run test:pw\n```\n\nAll patterns in TEA fragments are production-tested in this repository.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":9801,"content_sha256":"eb9415d849698671b6fb763740d7b58ef9886911da895110da1ac73e2122c7e4"},{"filename":"knowledge/pact-broker-webhooks.md","content":"# Pact Broker Webhooks (PactFlow → GitHub)\n\n## Principle\n\nConfigure PactFlow webhooks to trigger provider verification in GitHub Actions via a dedicated GitHub machine user, a long-lived classic Personal Access Token (PAT), and a PactFlow-stored secret. Monitor for silent webhook failures so an expired/revoked token does not quietly block deployments for days.\n\n## Rationale\n\n### Why webhooks matter\n\n- PactFlow's `contract_requiring_verification_published` webhook is the mechanism that notifies a provider repo (via `repository_dispatch`) that a consumer has published a contract needing verification.\n- The webhook carries `${pactbroker.providerVersionNumber}` and\n `${pactbroker.providerVersionBranch}` for the provider version missing a\n result. The provider workflow checks out that exact registered revision\n before publishing verification.\n- Without a working webhook, `can-i-deploy` in the consumer CI **times out** (900s) and eventually fails with `There is no verified pact between \u003cconsumer-version> and the version of \u003cprovider> currently in \u003cenv>` — even though nothing is wrong in either codebase.\n- Webhook failures are **silent by default**: PactFlow keeps emitting requests, GitHub keeps returning `401 Unauthorized`, but nothing alerts the team until a PR is blocked.\n\n### Why a dedicated GitHub machine user (not a personal PAT)\n\n- Personal PATs die when the person leaves the company, rotates laptops, or revokes credentials during a security review. The contract test pipeline then breaks for reasons unrelated to any code change.\n- A dedicated machine user (e.g., `pactflow-\u003corg>`) is owned by the org, has only the repos it needs, and the PAT lifecycle is controlled by the security/platform team.\n- GitHub **billing does not count** machine users added as outside collaborators to the specific repos they need — confirm with the org owner before assuming it's free.\n\n### Why classic PAT with `repo` scope and no expiration\n\n- PactFlow's webhook calls the GitHub REST API's `repository_dispatch` endpoint. This endpoint requires the **`repo` scope** on a classic PAT (fine-grained PATs work for many flows but have edge cases with `repository_dispatch` that are not universally supported at time of writing — verify with current GitHub docs).\n- Classic PATs support \"No expiration\" — required to avoid the silent-failure trap every 90 days. GitHub warns against this for human users; for a locked-down machine-user PAT stored in PactFlow's secret vault, the security trade-off is documented and accepted.\n- The alternative — rotating a PAT every 30/60/90 days — requires tooling and coordination most teams don't yet have. Long-lived + monitored + machine-user-owned is the pragmatic default.\n\n## Pattern Examples\n\n### Example 1: Webhook URL, Headers, and Body\n\n```json\n{\n \"description\": \"Notify \u003cprovider-repo> when a consumer contract requires verification\",\n \"events\": [{ \"name\": \"contract_requiring_verification_published\" }],\n \"provider\": { \"name\": \"\u003cprovider-pacticipant-name>\" },\n \"request\": {\n \"method\": \"POST\",\n \"url\": \"https://api.github.com/repos/\u003corg>/\u003cprovider-repo>/dispatches\",\n \"headers\": {\n \"Accept\": \"application/vnd.github+json\",\n \"Authorization\": \"Bearer ${user.githubToken}\",\n \"Content-Type\": \"application/json\",\n \"User-Agent\": \"PactFlow\",\n \"X-GitHub-Api-Version\": \"2022-11-28\"\n },\n \"body\": {\n \"event_type\": \"contract_requiring_verification_published\",\n \"client_payload\": {\n \"pact_url\": \"${pactbroker.pactUrl}\",\n \"sha\": \"${pactbroker.providerVersionNumber}\",\n \"branch\": \"${pactbroker.providerVersionBranch}\",\n \"consumer_name\": \"${pactbroker.consumerName}\",\n \"consumer_version_number\": \"${pactbroker.consumerVersionNumber}\",\n \"consumer_version_tags\": \"${pactbroker.consumerVersionTags}\",\n \"consumer_version_branch\": \"${pactbroker.consumerVersionBranch}\"\n }\n }\n }\n}\n```\n\n**Key Points**:\n\n- `${user.githubToken}` references a PactFlow **secret** stored in `Settings → Secrets` (web UI: `/settings/secrets`). The secret holds the classic PAT — never inline the token in the webhook body.\n- `${pactbroker.*}` are PactFlow-injected template variables; the provider workflow reads them from `github.event.client_payload`.\n- Use the `contract_requiring_verification_published` event (not `contract_published`) — the former fires only when a new pact _content_ change needs verification; the latter fires on every publish, including no-op republishes.\n\n### Example 2: Provider GitHub Actions Workflow (Triggered by Webhook)\n\n```yaml\n# .github/workflows/contract-test-provider.yml\nname: contract-test-provider\n\non:\n repository_dispatch:\n types: [contract_requiring_verification_published]\n push:\n branches: [main]\n\njobs:\n verify:\n runs-on: ubuntu-latest\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n # Pulled from webhook client_payload when triggered by PactFlow:\n PACT_PAYLOAD_URL: ${{ github.event.client_payload.pact_url }}\n PACT_PROVIDER_VERSION: ${{ github.event.client_payload.sha }}\n PACT_PROVIDER_BRANCH: ${{ github.event.client_payload.branch }}\n steps:\n - uses: actions/checkout@v4\n with:\n fetch-depth: 0\n - name: Select provider revision\n id: provider-revision\n run: |\n if [ -z \"$PACT_PROVIDER_BRANCH\" ] || [ -z \"$PACT_PROVIDER_VERSION\" ]; then\n echo \"Webhook payload is missing the provider branch or version.\"\n exit 1\n fi\n\n git fetch origin -- \"$PACT_PROVIDER_BRANCH\"\n\n if ! git rev-parse --verify --quiet \"$PACT_PROVIDER_VERSION^{commit}\" >/dev/null; then\n echo \"Provider version $PACT_PROVIDER_VERSION is unavailable.\"\n exit 1\n fi\n\n if ! git merge-base --is-ancestor \"$PACT_PROVIDER_VERSION\" FETCH_HEAD; then\n echo \"Provider version $PACT_PROVIDER_VERSION is not on $PACT_PROVIDER_BRANCH.\"\n exit 1\n fi\n\n git checkout --detach \"$PACT_PROVIDER_VERSION\"\n SELECTED_VERSION=\"$(git rev-parse HEAD)\"\n\n echo \"PACT_PROVIDER_VERSION=$SELECTED_VERSION\" >> \"$GITHUB_ENV\"\n echo \"PACT_PROVIDER_BRANCH=$PACT_PROVIDER_BRANCH\" >> \"$GITHUB_ENV\"\n echo \"GITHUB_BRANCH=$PACT_PROVIDER_BRANCH\" >> \"$GITHUB_ENV\"\n - uses: actions/setup-node@v4\n with:\n node-version: 20\n - run: npm ci\n - name: Run provider verification\n run: npm run test:pact:provider\n - name: Can I deploy provider?\n if: github.event_name == 'push'\n run: npm run can:i:deploy:provider\n```\n\n**Key Points**:\n\n- `repository_dispatch` is the event type emitted by GitHub when the webhook's REST call hits `/repos/\u003corg>/\u003crepo>/dispatches`.\n- The `types` filter must match the webhook's `event_type` (`contract_requiring_verification_published` here).\n- Check out the exact `providerVersionNumber` on `providerVersionBranch`.\n PactFlow emits this event for each main, deployed, or released provider\n version missing a result.\n- Verify that the commit exists and belongs to the registered branch before\n detaching to it. A force-pushed or deleted revision fails the job and leaves\n the requested result unknown.\n- After checkout, overwrite `PACT_PROVIDER_VERSION` with `git rev-parse HEAD`\n and publish the result against `PACT_PROVIDER_BRANCH`.\n- `PACT_PAYLOAD_URL` makes `buildVerifierOptions` verify only the triggering pact (see `pactjs-utils-provider-verifier.md` Example 1).\n\n### Example 3: Secret Rotation Runbook\n\n**Trigger**: `can-i-deploy` in a consumer repo times out with `There is no verified pact between \u003cconsumer-version> and the version of \u003cprovider> currently in \u003cenv>` — AND the provider's `contract-test-provider` workflow shows no recent `repository_dispatch` runs.\n\n**Diagnosis**:\n\n1. In PactFlow UI: `Settings → Webhooks → \u003cwebhook-id> → Test`. A `401 Unauthorized` from GitHub confirms the token is dead.\n2. In PactFlow UI: the webhook's \"Last executed at\" is hours/days stale while consumer pacts are actively being published.\n\n**Rotation**:\n\n1. Log in to GitHub as the dedicated machine user (e.g., `pactflow-\u003corg>`). **Do not use a personal account** — the whole point of the machine user is that the token outlives any individual.\n2. `Settings → Developer settings → Personal access tokens → Tokens (classic) → Generate new token (classic)`.\n3. Configure the token:\n - Name: `pactflow-webhook-\u003cyyyy-mm-dd>`\n - Expiration: **No expiration** (accepted trade-off for a locked-down machine-user token stored in PactFlow's secret vault)\n - Scopes: **`repo`** (full repo scope is required by `repository_dispatch`; `public_repo` alone is insufficient for private repos)\n4. Copy the new token value (shown only once).\n5. In PactFlow UI: `Settings → Secrets → \u003csecret-name>` (e.g., `githubToken`). Paste the new token into the **value** field and save. The webhook does not need to be edited — it references the secret by name via `${user.\u003csecret-name>}`.\n6. Re-test the webhook: `Settings → Webhooks → \u003cwebhook-id> → Test`. Expect `HTTP/1.1 204 No Content` (GitHub's success response for `repository_dispatch`).\n7. In the provider repo: watch `Actions → contract-test-provider` for the newly dispatched run. Re-run the original consumer CI to confirm `can-i-deploy` now passes.\n8. Revoke the old token: in the machine user's GitHub settings, delete the previous `pactflow-webhook-*` token so a leaked copy can't be reused.\n\n**Why no expiration**: A token with a 90-day expiry rotates 4× per year. Each rotation is a silent-failure window if the runbook isn't executed proactively. With monitoring (Example 4) + a locked-down machine-user-owned PAT that is only stored in PactFlow, long-lived is safer than short-lived-but-forgotten.\n\n### Example 4: Staleness Monitoring (Detect Silent Webhook Failures)\n\n**Goal**: Alert the team if verification results haven't been published for a pacticipant pair in the last N hours, so an expired PAT or network issue doesn't silently block `can-i-deploy` for days.\n\nPick one of these (in increasing order of investment):\n\n**Option A — Daily sanity CI job (cheapest)**:\n\n```yaml\n# .github/workflows/pact-staleness-check.yml\nname: pact-staleness-check\non:\n schedule:\n - cron: '0 9 * * 1-5' # weekdays 09:00 UTC\n workflow_dispatch:\njobs:\n check:\n runs-on: ubuntu-latest\n steps:\n - name: Fail if latest verification for \u003cpair> is older than 24h\n env:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n run: |\n # Query broker matrix for newest verification timestamp for consumer/provider pair.\n # Exit 1 if > 24h old; team gets an email on the failed scheduled run.\n ./scripts/assert-recent-verification.sh \u003cconsumer> \u003cprovider> 86400\n```\n\n**Option B — PactFlow metrics endpoint**: Use the SmartBear MCP `get_metrics` / `get_team_metrics` tool (see `pact-mcp.md`) to surface verification freshness in a dashboard or Slack digest.\n\n**Option C — Webhook delivery log**: PactFlow logs every webhook execution. Ship those logs to your SIEM / observability stack and alert on sustained 4xx responses from `api.github.com`.\n\n**Key Points**:\n\n- The point is not \"which option you pick\" — it's that **you pick at least one**. Without monitoring, the first time you learn the webhook is dead is when a release is blocked.\n- Alert threshold should match your consumer-publish cadence: if consumers publish daily, alert after 24–48h of silence; if hourly, after 3–6h.\n- Keep the alert noise-free: page only on sustained staleness, not a single missed run.\n\n## Key Points\n\n- **Dedicated machine user owns the PAT** — never a personal PAT. Name it `pactflow-\u003corg>` or similar; give it outside-collaborator access only to the specific provider repos.\n- **Classic PAT, `repo` scope, no expiration** — required for `repository_dispatch`. The \"no expiration\" trade-off is accepted in exchange for machine-user ownership + PactFlow-secret storage + staleness monitoring.\n- **Store the PAT as a PactFlow secret** at `/settings/secrets`, reference it from the webhook via `${user.\u003csecret-name>}`. Never inline the token.\n- **Monitor for silence** — at minimum, a daily scheduled CI job that asserts a recent verification timestamp exists for each critical consumer/provider pair.\n- **Provider targets need broker history** — publish provider verification\n results with branch metadata. Record deployments and releases so PactFlow can\n identify every provider version that this event must verify.\n- **Rotation is a runbook, not an emergency** — document it (see Example 3), keep it in the repo, and do a practice rotation once a year so it stays fresh.\n- **Symptom to remember**: \"consumer `can-i-deploy` timeout after 900s with `There is no verified pact...`\" + \"provider's `contract-test-provider` workflow has no recent runs\" = expired/revoked PAT. Start with Example 3.\n\n## Related Fragments\n\n- `pactjs-utils-provider-verifier.md` — how `PACT_PAYLOAD_URL` from the webhook's `client_payload.pact_url` is consumed by `buildVerifierOptions`\n- `pact-consumer-framework-setup.md` — consumer CI flow that issues `can-i-deploy` and silently times out when the webhook is dead\n- `pact-mcp.md` — SmartBear MCP tools (`Matrix`, `Metrics - All`) useful for staleness monitoring dashboards\n- `contract-testing.md` — foundational CDC patterns and resilience coverage\n\n## Anti-Patterns\n\n### Wrong: Using a human's personal PAT\n\n```text\n# ❌ PactFlow secret githubToken stores the lead engineer's personal classic PAT\n# When they leave / rotate / revoke → all provider verifications stop silently\n```\n\n### Right: Dedicated machine user owns the PAT\n\n```text\n# ✅ Machine user `pactflow-\u003corg>` generates the PAT; secret is owned by the org\n# PAT lifecycle is decoupled from any individual's employment or laptop state\n```\n\n### Wrong: No staleness monitoring\n\n```text\n# ❌ No scheduled check for verification recency\n# First signal that the webhook is dead: a blocked release PR, several days later\n```\n\n### Right: Daily scheduled sanity check\n\n```text\n# ✅ Scheduled workflow fails if latest verification > 24h old\n# Team gets email alert on failed scheduled run → rotate PAT before anyone is blocked\n```\n\n### Wrong: Short-expiration PAT with no rotation tooling\n\n```text\n# ❌ 90-day expiry PAT, no calendar reminder, no runbook\n# Breaks every 90 days for a day or two until someone notices\n```\n\n### Right: No-expiration PAT on machine user + monitoring + documented runbook\n\n```text\n# ✅ Long-lived PAT, scoped narrowly, stored in PactFlow, monitored for staleness\n# Rotation is intentional (security review, suspected leak) not calendar-driven\n```\n\n_Source: PactFlow webhook documentation, GitHub `repository_dispatch` REST API, an internal production incident (April 2026)_\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":15040,"content_sha256":"7a54a5d1976e899508467468ed848beee66367f218470811528e9742f1d8daff"},{"filename":"knowledge/pact-consumer-di.md","content":"# Pact Consumer DI Pattern\n\n## Principle\n\nInject the Pact mock server URL into consumer code via an optional `baseUrl` field on the API context type instead of using raw `fetch()` inside `executeTest()`. This ensures contract tests exercise the real consumer HTTP client — including retry logic, header assembly, timeout configuration, error handling, and metrics — rather than testing Pact itself.\n\nThe base URL is typically a module-level constant evaluated at import time (`export const API_BASE_URL = env.API_BASE_URL`), but `mockServer.url` is only available at runtime inside `executeTest()`. Dependency injection solves this timing mismatch cleanly: add one optional field to the context type, use nullish coalescing in the HTTP client factory, and inject the mock server URL in tests.\n\n## Rationale\n\n### The Problem\n\nRaw `fetch()` in `executeTest()` only proves that Pact returns what you told it to return. The real consumer HTTP client has retry logic, header assembly, timeout configuration, error handling, and metrics collection — none of which are exercised when you hand-craft fetch calls. Contracts written with raw fetch are hand-maintained guesses about what the consumer actually sends.\n\n### Why NOT vi.mock\n\n`vi.mock` with ESM (`module: Node16`) has hoisting quirks that make it unreliable for overriding module-level constants. A getter-based mock is non-obvious and fragile — it works until the next bundler or TypeScript config change breaks it. DI is a standard pattern that requires zero mock magic and works across all module systems.\n\n### Comparison\n\n| Approach | Production code change | Mock complexity | Exercises real client | Contract accuracy |\n| ------------ | ---------------------- | -------------------------- | --------------------- | --------------------------- |\n| Raw fetch | None | None | No | Low — hand-crafted requests |\n| vi.mock | None | High — ESM hoisting issues | Yes | Medium — fragile setup |\n| DI (baseUrl) | 2 lines | None | Yes | High — real requests |\n\n## Pattern Examples\n\n### Example 1: Production Code Change (2 Lines Total)\n\n**Context**: Add an optional `baseUrl` field to the API context type and use nullish coalescing in the HTTP client factory. This is the entire production code change required.\n\n**Implementation**:\n\n```typescript\n// src/types.ts\nexport type ApiContext = {\n jwtToken: string;\n customerId: number;\n adminUserId?: number;\n correlationId?: string;\n baseUrl?: string; // Override for testing (Pact mock server)\n};\n```\n\n```typescript\n// src/http-client.ts\nimport axios from 'axios';\nimport type { AxiosInstance } from 'axios';\nimport type { ApiContext } from './types.js';\nimport { API_BASE_URL, REQUEST_TIMEOUT } from './constants.js';\n\nfunction createAxiosInstanceWithContext(context: ApiContext): AxiosInstance {\n return axios.create({\n baseURL: context.baseUrl ?? API_BASE_URL,\n timeout: REQUEST_TIMEOUT,\n headers: {\n 'Content-Type': 'application/json',\n Accept: 'application/json',\n Authorization: `Bearer ${context.jwtToken}`,\n ...(context.correlationId && { 'X-Request-Id': context.correlationId }),\n },\n });\n}\n```\n\n**Key Points**:\n\n- `baseUrl` is optional — existing production code never sets it\n- `??` (nullish coalescing) falls back to `API_BASE_URL` when `baseUrl` is undefined\n- Zero production behavior change — only test code provides the override\n- Two lines added total: one type field, one `??` fallback\n\n### Example 2: Shared Test Context Helper\n\n**Context**: Create a reusable helper that builds an `ApiContext` with the mock server URL injected. One helper shared across all consumer test files.\n\n**Implementation**:\n\n```typescript\n// pact/support/test-context.ts\nimport type { ApiContext } from '../../src/types.js';\n\nexport function createTestContext(mockServerUrl: string): ApiContext {\n return {\n jwtToken: 'test-jwt-token',\n customerId: 1,\n baseUrl: `${mockServerUrl}/api/v2`,\n };\n}\n```\n\n**Key Points**:\n\n- `baseUrl` should include the API version prefix when consumer methods use versionless relative paths (e.g., `/transactions`) or endpoint paths are defined without the version segment\n- Single helper shared across all consumer test files — no repetition\n- Returns a plain object — follows pure-function-first pattern from `fixture-architecture.md`\n- Add fields as needed (e.g., `adminUserId`, `correlationId`) for specific test scenarios\n\n### Example 3: Before/After for a Simple Test\n\n**Context**: Migrating an existing raw-fetch test to call real consumer code.\n\n**Before** (raw fetch — tests Pact mock, not consumer code):\n\n```typescript\n.executeTest(async (mockServer: V3MockServer) => {\n const response = await fetch(\n `${mockServer.url}/api/v2/common/fields?ruleType=!&ignoreFeatureFlags=true`,\n {\n headers: {\n Authorization: \"Bearer test-jwt-token\",\n \"Content-Type\": \"application/json\",\n },\n },\n );\n expect(response.status).toBe(200);\n const body = (await response.json()) as Record\u003cstring, unknown>[];\n expect(body).toEqual(expect.arrayContaining([...]));\n});\n```\n\n**After** (real consumer code):\n\n```typescript\n.executeTest(async (mockServer: V3MockServer) => {\n const api = createApiClient(createTestContext(mockServer.url));\n const result = await api.getFilterFields();\n expect(result).toEqual(\n expect.arrayContaining([\n expect.objectContaining({\n id: expect.any(String),\n readable: expect.any(String),\n filterType: expect.any(String),\n }),\n ]),\n );\n});\n```\n\n**Key Points**:\n\n- No HTTP status assertion — the consumer method throws on non-2xx, so reaching the expect proves success\n- Assertions validate the return value shape, not transport details\n- The real client's headers, timeout, and retry logic are exercised transparently\n- Less code, more coverage — the test is shorter and tests more\n\n### Example 4: Contract Accuracy Fix\n\n**Context**: Using real consumer code revealed a contract mismatch that raw fetch silently hid. This is the strongest argument for the pattern.\n\nThe real `getCustomerActivityCount(transactionId, dateRange)` sends:\n\n```json\n{ \"transactionId\": \"txn-123\", \"filters\": { \"dateRange\": \"last_30_days\" } }\n```\n\nThe old test with raw fetch sent:\n\n```json\n{ \"transactionId\": \"txn-123\", \"filters\": {} }\n```\n\nThis was wrong but passed because raw fetch let you hand-craft any body. When switched to real code, Pact immediately returned a 500 Request-Mismatch because the body shape did not match the interaction.\n\n**Implementation** — fix the contract to match reality:\n\n```typescript\n// WRONG — old contract with empty filters\n.withRequest({\n method: \"POST\",\n path: \"/api/v2/customers/activity/count\",\n body: { transactionId: \"txn-123\", filters: {} },\n})\n\n// CORRECT — matches what real code actually sends\n.withRequest({\n method: \"POST\",\n path: \"/api/v2/customers/activity/count\",\n body: {\n transactionId: \"txn-123\",\n filters: { dateRange: \"last_30_days\" },\n },\n})\n```\n\n**Key Points**:\n\n- Contracts become discoverable truth, not hand-maintained guesses\n- Raw fetch silently hid the mismatch — the mock accepted whatever you sent\n- The 500 Request-Mismatch from Pact was immediate and clear\n- Fix the contract when real code reveals a mismatch — that mismatch is a bug the old tests were hiding\n\n### Example 5: Parallel-Endpoint Methods\n\n**Context**: Facade methods that call multiple endpoints via `Promise.all` (e.g., `getTransactionStats` calls count + score + amount in parallel). Keep separate `it` blocks per endpoint and use the lower-level request function directly.\n\n**Implementation**:\n\n```typescript\nimport { describe, it, expect } from 'vitest';\nimport type { V3MockServer } from '@pact-foundation/pact';\nimport { makeApiRequestWithContext } from '../../src/http-client.js';\nimport type { CountStatistics } from '../../src/types.js';\nimport { createTestContext } from '../support/test-context.js';\n\ndescribe('Transaction Statistics - Count Endpoint', () => {\n // ... provider setup ...\n\n it('should return count statistics', async () => {\n const statsRequest = { transactionId: 'txn-123', period: 'daily' };\n\n await provider\n .given('transaction statistics exist')\n .uponReceiving('a request for transaction count statistics')\n .withRequest({\n method: 'POST',\n path: '/api/v2/transactions/statistics/count',\n body: statsRequest,\n })\n .willRespondWith({\n status: 200,\n body: { count: 42, period: 'daily' },\n })\n .executeTest(async (mockServer: V3MockServer) => {\n const context = createTestContext(mockServer.url);\n const result = await makeApiRequestWithContext\u003cCountStatistics>(context, '/transactions/statistics/count', 'POST', statsRequest);\n expect(result.count).toBeDefined();\n });\n });\n});\n```\n\n**Key Points**:\n\n- Each Pact interaction verifies one endpoint contract\n- The `Promise.all` orchestration is internal logic, not a contract concern\n- Use `makeApiRequestWithContext` (lower-level) when the facade method bundles multiple calls\n- Separate `it` blocks keep contracts independent and debuggable\n\n## Anti-Patterns\n\n### Wrong: Raw fetch — tests Pact mock, not consumer code\n\n```typescript\n// BAD: Raw fetch duplicates headers and URL assembly\nconst response = await fetch(`${mockServer.url}/api/v2/transactions`, {\n method: 'GET',\n headers: {\n Authorization: 'Bearer test-jwt-token',\n 'Content-Type': 'application/json',\n },\n});\nexpect(response.status).toBe(200);\n```\n\n### Wrong: vi.mock with getter — fragile ESM hoisting\n\n```typescript\n// BAD: ESM hoisting makes this non-obvious and brittle\nvi.mock('../../src/constants.js', async (importOriginal) => ({\n ...(await importOriginal()),\n get API_BASE_URL() {\n return mockBaseUrl;\n },\n}));\n```\n\n### Wrong: Asserting HTTP status instead of return value\n\n```typescript\n// BAD: Status 200 tells you nothing about the consumer's parsing logic\nexpect(response.status).toBe(200);\n```\n\n### Right: Call real consumer code, assert return values\n\n```typescript\n// GOOD: Exercises real client, validates parsed return value\nconst api = createApiClient(createTestContext(mockServer.url));\nconst result = await api.searchTransactions(request);\nexpect(result.transactions).toBeDefined();\n```\n\n## Rules\n\n1. `baseUrl` field MUST be optional with fallback via `??` (nullish coalescing)\n2. Zero production behavior change — existing code never sets `baseUrl`\n3. Assertions validate return values from consumer methods, not HTTP status codes\n4. For parallel-endpoint facade methods, keep separate `it` blocks per endpoint\n5. Include the API version prefix in `baseUrl` when endpoint paths/consumer methods are versionless (for example, methods call `/transactions` instead of `/api/v2/transactions`)\n6. Create a single shared test context helper — no repetition across test files\n7. If real code reveals a contract mismatch, fix the contract — that mismatch is a bug the old tests were hiding\n\n## Integration Points\n\n- `contract-testing.md` — Foundational Pact.js patterns and provider verification\n- `pactjs-utils-consumer-helpers.md` — `createProviderState()`, `setJsonContent()`, and `setJsonBody()` helpers used alongside this pattern\n- `pactjs-utils-provider-verifier.md` — Provider-side verification configuration\n- `fixture-architecture.md` — Composable fixture patterns (`createTestContext` follows pure-function-first)\n- `api-testing-patterns.md` — API testing best practices\n\nUsed in workflows:\n\n- `automate` — Consumer contract test generation\n- `test-review` — Contract test quality checks\n\n## Source\n\nPattern derived from my-consumer-app Pact consumer test refactor (March 2026). Implements dependency injection for testability as described in Pact.js best practices.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12002,"content_sha256":"8da93042ab11ad2e59ae8e37d44567d7bfec77760bb72467bde0d03987ebadce"},{"filename":"knowledge/pact-consumer-framework-setup.md","content":"# Pact Consumer CDC — Framework Setup\n\n## Principle\n\nWhen scaffolding a Pact.js consumer contract testing framework, align every artifact — directory layout, vitest config, package.json scripts, shell scripts, CI workflow, and test files — with the canonical `@seontechnologies/pactjs-utils` conventions. Consistency across repositories eliminates onboarding friction and ensures CI pipelines are copy-paste portable.\n\n## Rationale\n\nThe TEA framework workflow generates scaffolding for consumer-driven contract (CDC) testing. Without opinionated, battle-tested conventions, each project invents its own structure — different script names, different env var patterns, different CI step ordering — making cross-repo maintenance expensive. This fragment codifies the production-proven patterns from the pactjs-utils reference implementation so that every new project starts correctly.\n\n## Pattern Examples\n\n### Example 1: Directory Structure & File Naming\n\n**Context**: Consumer contract test project layout using pactjs-utils conventions.\n\n**Implementation**:\n\n```text\ntests/contract/\n├── consumer/\n│ ├── get-filter-fields.pacttest.ts # Consumer test (one per endpoint group)\n│ ├── filter-transactions.pacttest.ts\n│ └── get-transaction-stats.pacttest.ts\n└── support/\n ├── pact-config.ts # PactV4 factory (consumer/provider names, output dir)\n ├── provider-states.ts # Provider state factory functions\n └── consumer-helpers.ts # Local shim (until pactjs-utils is published)\n\nscripts/\n├── env-setup.sh # Shared env loader (sourced by all broker scripts)\n├── publish-pact.sh # Publish pact files to broker\n├── can-i-deploy.sh # Deployment safety check\n└── record-deployment.sh # Record deployment after merge\n\n.github/\n├── actions/\n│ ├── detect-breaking-change/\n│ │ └── action.yml # PR checkbox-driven breaking change detection\n│ └── detect-provider-branch/\n│ └── action.yml # PR-only provider branch coordination\n└── workflows/\n └── contract-test-consumer.yml # Consumer CDC CI workflow\n```\n\n**Key Points**:\n\n- Consumer tests use `.pacttest.ts` extension (not `.pact.spec.ts` or `.contract.ts`)\n- Support files live in `tests/contract/support/`, not mixed with consumer tests\n- Shell scripts live in `scripts/` at project root, not nested inside test directories\n- CI workflow named `contract-test-consumer.yml` (not `pact-consumer.yml` or other variants)\n\n---\n\n### Example 2: Vitest Configuration for Pact\n\n**Context**: Minimal vitest config dedicated to contract tests — do NOT copy settings from the project's main `vitest.config.ts`.\n\n**Implementation**:\n\n```typescript\n// vitest.config.pact.ts\n// See pact-consumer-framework-setup.md Example 2 \"Key Points\" for rationale on\n// fileParallelism + pool:forks + singleFork. Do not remove those three settings.\nimport { defineConfig } from 'vitest/config';\n\nexport default defineConfig({\n test: {\n environment: 'node',\n include: ['tests/contract/**/*.pacttest.ts'],\n testTimeout: 30000,\n fileParallelism: false,\n pool: 'forks',\n poolOptions: { forks: { singleFork: true } },\n },\n});\n```\n\n**Key Points**:\n\n- **`fileParallelism: false` is required** — primary defense against non-deterministic pact generation. Without it, parallel workers race on the shared pact JSON file and corrupt interactions. Symptom: local runs pass, CI randomly fails with `Cannot change pact content for already published pact`. The `publish-pact.sh` `jq` sort (Example 4) provides byte-stability at publish time.\n- **`pool: 'forks'` + `singleFork: true` is required for multi-file consumer suites** — same config the provider side uses (`pactjs-utils-provider-verifier.md` Example 8). Best current understanding: the `@pact-foundation/pact` napi-rs binding is not robust across Vitest worker threads sharing a process; with the default threads pool (Vitest v1) and multiple `.pacttest.ts` files on the same consumer+provider pair, we observed reproducible \"request was expected but not received\" flakes on Linux CI only. `singleFork: true` serializes every pact file into one forked subprocess and eliminated the flake across multiple repos. Vitest v2+ defaults to `forks`, but set the pool explicitly so the contract does not drift with Vitest version bumps.\n- **One `.pacttest.ts` per consumer+provider pair is the canonical pattern** — not just an observation. Two files for the same pair in one process (which `singleFork: true` guarantees) cause an FFI handle collision: the second file's `new PactV4(...)` call re-enters the FFI handle still holding stale state from the first file → \"request was expected but not received\" sporadically on Linux CI. The fix is structural — merge the files, not the config. `pool: 'forks'` is still required for pact JSON write safety but does NOT prevent same-pair file splits from colliding. Multiple files for **different** pairs (different consumer or provider name) are correct and safe. See Example 11 for the ✅/❌ pattern.\n- **Interacting settings**: leave `isolate` at its default (`true`). Do NOT set `sequence.concurrent: true`, `maxConcurrency > 1`, or `maxWorkers > 1` in this config — they defeat the serialization this rule relies on. `hookTimeout` may be raised if mock-server startup is slow, but keep `testTimeout` ≥ `hookTimeout`.\n- Do NOT add `setupFiles`, `coverage`, or other settings from the unit test config\n- Keep it minimal — Pact tests run in Node environment with extended timeout\n- 30 second timeout accommodates Pact mock server startup and interaction verification\n- Use a dedicated config file (`vitest.config.pact.ts`), not the main vitest config\n\n---\n\n### Example 3: Package.json Script Naming\n\n**Context**: Colon-separated naming matching pactjs-utils exactly. Scripts source `env-setup.sh` inline.\n\n**Implementation**:\n\n```json\n{\n \"scripts\": {\n \"test:pact:consumer\": \"vitest run --config vitest.config.pact.ts\",\n \"publish:pact\": \". ./scripts/env-setup.sh && ./scripts/publish-pact.sh\",\n \"can:i:deploy:consumer\": \". ./scripts/env-setup.sh && PACTICIPANT=\u003cconsumer-name> PROVIDER_PACTICIPANT=\u003cprovider-name> ./scripts/can-i-deploy.sh\",\n \"record:consumer:deployment\": \". ./scripts/env-setup.sh && PACTICIPANT=\u003cservice-name> ./scripts/record-deployment.sh\"\n }\n}\n```\n\nReplace `\u003cconsumer-name>` and `\u003cprovider-name>` with the Pact Broker\npacticipant names. `PROVIDER_PACTICIPANT` scopes the optional short-lived\nprovider branch override; the plain environment gate still works when no\noverride is set.\n\n**Key Points**:\n\n- Use colon-separated naming: `test:pact:consumer`, NOT `test:contract` or `test:contract:consumer`\n- Broker scripts source `env-setup.sh` inline in package.json (`. ./scripts/env-setup.sh && ...`)\n- `PACTICIPANT` is set per-script invocation, not globally\n- Do NOT use `npx pact-broker` — use `pact-broker` directly (installed as a dependency)\n\n---\n\n### Example 4: Shell Scripts\n\n**Context**: Reusable bash scripts aligned with pactjs-utils conventions.\n\n#### `scripts/env-setup.sh` — Shared Environment Loader\n\n```bash\n#!/bin/bash\n# -e: exit on error -u: error on undefined vars (catches typos/missing env vars in CI)\nset -eu\n\nif [ -f .env ]; then\n set -a\n source .env\n set +a\nfi\n\nexport GITHUB_SHA=\"${GITHUB_SHA:-$(git rev-parse --short HEAD)}\"\nexport GITHUB_BRANCH=\"${GITHUB_BRANCH:-$(git rev-parse --abbrev-ref HEAD)}\"\n```\n\n#### `scripts/publish-pact.sh` — Publish Pacts to Broker\n\n```bash\n#!/bin/bash\n# Publish generated pact files to PactFlow/Pact Broker.\n#\n# Before publish, normalize each pact JSON: sort interactions by (description, provider state name,\n# method, path) and sort object keys via `jq -S`. This gives byte-stable output to the broker even\n# if the PactV4 generator produces ordering drift between runs. Ensures \"Cannot change pact content\"\n# from PactFlow never fires on ordering-only changes.\n#\n# Requires: PACT_BROKER_BASE_URL, PACT_BROKER_TOKEN, GITHUB_SHA, GITHUB_BRANCH, jq\n# -e: exit on error -u: error on undefined vars -o pipefail: fail if any pipe segment fails\nset -euo pipefail\n\n. ./scripts/env-setup.sh\n\nPACT_DIR=\"./pacts\"\n\n# Defense-in-depth: normalize interaction order for byte-stable publishes.\nfor f in \"$PACT_DIR\"/*.json; do\n tmp=\"$(mktemp)\"\n jq -S '.interactions |= sort_by(.description, (.providerStates[0].name // \"\"), .request.method, .request.path)' \\\n \"$f\" > \"$tmp\"\n mv \"$tmp\" \"$f\"\ndone\n\npact-broker publish \"$PACT_DIR\" \\\n --consumer-app-version=\"$GITHUB_SHA\" \\\n --branch=\"$GITHUB_BRANCH\" \\\n --broker-base-url=\"$PACT_BROKER_BASE_URL\" \\\n --broker-token=\"$PACT_BROKER_TOKEN\"\n```\n\n#### `scripts/can-i-deploy.sh` — Deployment Safety Check\n\n```bash\n#!/bin/bash\n# Check if a pacticipant version can be safely deployed\n#\n# Requires: PACTICIPANT (set by caller), PACT_BROKER_BASE_URL, PACT_BROKER_TOKEN, GITHUB_SHA\n# -e: exit on error -u: error on undefined vars -o pipefail: fail if any pipe segment fails\nset -euo pipefail\n\n. ./scripts/env-setup.sh\n\nPACTICIPANT=\"${PACTICIPANT:?PACTICIPANT env var is required}\"\nENVIRONMENT=\"${ENVIRONMENT:-dev}\"\n\n# PR-only override set by detect-provider-branch. Preserve the environment-wide\n# gate for every other dependency, and check the in-flight provider branch\n# separately. Both calls fail hard under set -e.\nif [ -n \"${PACT_PROVIDER_BRANCH:-}\" ] && [ -n \"${PROVIDER_PACTICIPANT:-}\" ]; then\n pact-broker can-i-deploy \\\n --pacticipant \"$PACTICIPANT\" \\\n --version=\"$GITHUB_SHA\" \\\n --to-environment \"$ENVIRONMENT\" \\\n --ignore \"$PROVIDER_PACTICIPANT\" \\\n --retry-while-unknown=10 \\\n --retry-interval=30\n\n pact-broker can-i-deploy \\\n --pacticipant \"$PACTICIPANT\" \\\n --version=\"$GITHUB_SHA\" \\\n --pacticipant \"$PROVIDER_PACTICIPANT\" \\\n --branch=\"$PACT_PROVIDER_BRANCH\" \\\n --retry-while-unknown=10 \\\n --retry-interval=30\nelse\n pact-broker can-i-deploy \\\n --pacticipant \"$PACTICIPANT\" \\\n --version=\"$GITHUB_SHA\" \\\n --to-environment \"$ENVIRONMENT\" \\\n --retry-while-unknown=10 \\\n --retry-interval=30\nfi\n```\n\n#### `scripts/record-deployment.sh` — Record Deployment\n\n```bash\n#!/bin/bash\n# Record a deployment to an environment in Pact Broker\n# Only records on main/master branch (skips feature branches)\n#\n# Requires: PACTICIPANT, PACT_BROKER_BASE_URL, PACT_BROKER_TOKEN, GITHUB_SHA, GITHUB_BRANCH\n# -e: exit on error -u: error on undefined vars -o pipefail: fail if any pipe segment fails\nset -euo pipefail\n\n. ./scripts/env-setup.sh\n\nPACTICIPANT=\"${PACTICIPANT:?PACTICIPANT env var is required}\"\n\nif [ \"$GITHUB_BRANCH\" = \"main\" ] || [ \"$GITHUB_BRANCH\" = \"master\" ]; then\n pact-broker record-deployment \\\n --pacticipant \"$PACTICIPANT\" \\\n --version \"$GITHUB_SHA\" \\\n --environment \"${npm_config_env:-dev}\"\nelse\n echo \"Skipping record-deployment: not on main branch (current: $GITHUB_BRANCH)\"\nfi\n```\n\n**Key Points**:\n\n- `env-setup.sh` uses `set -eu` (no pipefail — it only sources `.env`, no pipes); broker scripts use `set -euo pipefail`\n- Use `pact-broker` directly, NOT `npx pact-broker`\n- Use `PACTICIPANT` env var (required via `${PACTICIPANT:?...}`), not hardcoded service names\n- `can-i-deploy` includes `--retry-while-unknown=10 --retry-interval=30` (waits for provider verification)\n- A PR-only `PACT_PROVIDER_BRANCH` override is additive: the environment check\n ignores only `PROVIDER_PACTICIPANT`, then a second check targets that\n pacticipant's branch. `--branch` never silently replaces `--to-environment`.\n- `record-deployment` has branch guard (only records on main/master)\n- **`publish-pact.sh` normalizes interactions with `jq -S` + `sort_by(...)` before publishing** — ensures byte-stable payload to the broker regardless of generator ordering quirks.\n- Do NOT invent custom env vars like `PACT_CONSUMER_VERSION` or `PACT_BREAKING_CHANGE` in scripts — those are handled by `env-setup.sh` and the CI detect-breaking-change action respectively\n\n---\n\n### Example 5: CI Workflow (`contract-test-consumer.yml`)\n\n**Context**: GitHub Actions workflow for consumer CDC, matching pactjs-utils structure exactly.\n\n**Implementation**:\n\n```yaml\nname: Contract Test - Consumer\non:\n pull_request:\n types: [opened, synchronize, reopened, edited]\n push:\n branches: [main]\n\nenv:\n PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}\n PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}\n GITHUB_SHA: ${{ github.sha }}\n GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }}\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}\n cancel-in-progress: true\n\njobs:\n consumer-contract-test:\n if: github.actor != 'dependabot[bot]'\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6\n\n - uses: actions/setup-node@v6\n with:\n node-version-file: '.nvmrc'\n cache: 'npm'\n\n - name: Detect Pact breaking change\n uses: ./.github/actions/detect-breaking-change\n\n - name: Detect Pact provider branch\n uses: ./.github/actions/detect-provider-branch\n\n - name: Install dependencies\n run: npm ci\n\n # (1) Generate pact files\n - name: Run consumer contract tests\n run: npm run test:pact:consumer\n\n # (2) Publish pacts to broker (publish-pact.sh also normalizes interaction order as defense-in-depth)\n - name: Publish pacts to PactFlow\n run: npm run publish:pact\n\n # After publish, PactFlow fires a webhook that triggers\n # the provider's contract-test-provider.yml workflow.\n # can-i-deploy retries while waiting for provider verification.\n\n # (4) Check deployment safety.\n # NOTE: First-time bootstrap: if no verified contract exists on the broker yet,\n # gate this to main only: if: github.ref == 'refs/heads/main' && env.PACT_BREAKING_CHANGE != 'true'\n # Once the first contract is published and verified on main, remove the main-only condition.\n - name: Can I deploy consumer?\n if: env.PACT_BREAKING_CHANGE != 'true'\n run: npm run can:i:deploy:consumer\n env:\n PACT_PROVIDER_BRANCH: ${{ env.PACT_PROVIDER_BRANCH }}\n\n # (5) Record deployment (main only)\n - name: Record consumer deployment (main only)\n if: github.ref == 'refs/heads/main'\n run: npm run record:consumer:deployment --env=dev\n```\n\n**Key Points**:\n\n- **1:1 local/CI parity is a hard rule**: every CI step is `npm run \u003csame-name-a-dev-uses>`. Never let CI invoke `vitest` or `pact-broker` directly — that divergence is how \"works on my machine\" slips in. Consumer tests, publish, can-i-deploy, and record-deployment are all the same commands a developer runs locally.\n- **Workflow-level `env` block** for broker secrets and git vars — not per-step\n- **`detect-breaking-change` step** runs before install to set `PACT_BREAKING_CHANGE` env var\n- **`detect-provider-branch` step** runs on PR events before install and exports\n a short-lived `PACT_PROVIDER_BRANCH` hint from the PR description\n- **Step numbering skips (3)** — step 3 is the webhook-triggered provider verification (happens externally)\n- **can-i-deploy condition**: `env.PACT_BREAKING_CHANGE != 'true'` after the\n first main contract has been published and verified. During one-time broker\n bootstrap, gate it to main until that verification exists.\n- **Comment on (4)**: \"on PRs, local verification is the gate\"\n- **No upload-artifact step** — the broker is the source of truth for pact files\n- **`dependabot[bot]` skip** on the job (contract tests don't run for dependency updates)\n- **PR types include `edited`** — needed for breaking change checkbox detection in PR body\n- **`GITHUB_BRANCH`** uses `${{ github.head_ref || github.ref_name }}` — `head_ref` for PRs, `ref_name` for pushes\n\n---\n\n### Example 6: Detect Breaking Change Composite Action\n\n**Context**: GitHub composite action that reads a `[x] Pact breaking change` checkbox from the PR body.\n\n**Implementation**:\n\nCreate `.github/actions/detect-breaking-change/action.yml`:\n\n```yaml\nname: 'Detect Pact Breaking Change'\ndescription: 'Reads the PR template checkbox to determine if this change is a Pact breaking change. Sets PACT_BREAKING_CHANGE env var.'\n\noutputs:\n is_breaking_change:\n description: 'Whether the change is a breaking change (true/false)'\n value: ${{ steps.result.outputs.is_breaking_change }}\n\nruns:\n using: 'composite'\n steps:\n # PR event path: read checkbox directly from current PR body.\n - name: Set PACT_BREAKING_CHANGE from PR description (PR only)\n if: github.event_name == 'pull_request'\n uses: actions/github-script@v7\n with:\n script: |\n const prBody = context.payload.pull_request.body || '';\n const breakingChangePattern = /\\[\\s*[xX]\\s*\\]\\s*Pact breaking change/i;\n const isBreakingChange = breakingChangePattern.test(prBody);\n core.exportVariable('PACT_BREAKING_CHANGE', isBreakingChange ? 'true' : 'false');\n console.log(`PACT_BREAKING_CHANGE=${isBreakingChange ? 'true' : 'false'} (from PR description checkbox).`);\n\n # Push-to-main path: resolve the merged PR and read the same checkbox.\n - name: Set PACT_BREAKING_CHANGE from merged PR (push to main)\n if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n uses: actions/github-script@v7\n with:\n script: |\n const { data: prs } = await github.rest.repos.listPullRequestsAssociatedWithCommit({\n owner: context.repo.owner,\n repo: context.repo.repo,\n commit_sha: context.sha,\n });\n const merged = prs.find(pr => pr.merged_at);\n const mergedBody = merged?.body || '';\n const breakingChangePattern = /\\[\\s*[xX]\\s*\\]\\s*Pact breaking change/i;\n const isBreakingChange = breakingChangePattern.test(mergedBody);\n core.exportVariable('PACT_BREAKING_CHANGE', isBreakingChange ? 'true' : 'false');\n console.log(`PACT_BREAKING_CHANGE=${isBreakingChange ? 'true' : 'false'} (from merged PR lookup).`);\n\n - name: Export result\n id: result\n shell: bash\n run: echo \"is_breaking_change=${PACT_BREAKING_CHANGE:-false}\" >> \"$GITHUB_OUTPUT\"\n```\n\n**Key Points**:\n\n- Two separate conditional steps (better CI log readability than single if/else)\n- PR path: reads checkbox directly from PR body\n- Push-to-main path: resolves merged PR via GitHub API, reads same checkbox\n- Exports `PACT_BREAKING_CHANGE` env var for downstream steps\n- `outputs.is_breaking_change` available for consuming workflows\n- Uses a case-insensitive checkbox regex (`/\\[\\s*[xX]\\s*\\]\\s*Pact breaking change/i`) to detect checked states robustly\n\n---\n\n### Example 7: Detect Provider Branch Composite Action\n\n**Context**: A consumer PR needs verification against a provider branch that\nhas not merged or deployed yet. Add `Pact provider branch: \u003cname>` to the PR\ntemplate and parse it only for `pull_request` events.\n\n```yaml\nname: 'Detect Pact Provider Branch'\ndescription: 'Exports a PR-only provider branch override'\n\nruns:\n using: 'composite'\n steps:\n - name: Set PACT_PROVIDER_BRANCH from PR description\n if: github.event_name == 'pull_request'\n uses: actions/github-script@v7\n with:\n script: |\n const body = context.payload.pull_request.body || '';\n const match = body.match(\n /^[^\\S\\r\\n]*Pact provider branch:[^\\S\\r\\n]*(\\S+)[^\\S\\r\\n]*$/im\n );\n core.exportVariable('PACT_PROVIDER_BRANCH', match?.[1] || '');\n```\n\nDo not read this field from the merged PR on push. It is a coordination hint for\none open PR, not durable deployment metadata. After merge, the normal\n`--to-environment` gate must be authoritative again.\n\n---\n\n### Example 8: Consumer Test Using PactV4 Builder\n\n**Context**: Consumer pact test using PactV4 `addInteraction()` builder pattern. The test MUST call **real consumer code** (your actual API client/service functions) against the mock server — not raw `fetch()`. Using `fetch()` directly defeats the purpose of CDC testing because it doesn't verify your actual consumer code works with the contract.\n\n**Implementation**:\n\nThe consumer code must expose a way to inject the base URL (e.g., `setApiUrl()`, constructor parameter, or environment variable). This is a prerequisite for contract testing.\n\n```typescript\n// src/api/movie-client.ts — The REAL consumer code (already exists in your project)\nimport axios from 'axios';\n\nconst axiosInstance = axios.create({\n baseURL: process.env.API_URL || 'http://localhost:3001',\n});\n\n// Expose a way to override the base URL for Pact testing\nexport const setApiUrl = (url: string) => {\n axiosInstance.defaults.baseURL = url;\n};\n\nexport const getMovies = async () => {\n const res = await axiosInstance.get('/movies');\n return res.data;\n};\n\nexport const getMovieById = async (id: number) => {\n const res = await axiosInstance.get(`/movies/${id}`);\n return res.data;\n};\n```\n\n```typescript\n// tests/contract/consumer/get-movies.pacttest.ts\nimport { MatchersV3 } from '@pact-foundation/pact';\nimport type { V3MockServer } from '@pact-foundation/pact';\nimport { createProviderState, setJsonBody, setJsonContent } from '../support/consumer-helpers';\nimport { movieExists } from '../support/provider-states';\nimport { createPact } from '../support/pact-config';\n// Import REAL consumer code — this is what we're actually testing\nimport { getMovies, getMovieById, setApiUrl } from '../../../src/api/movie-client';\n\nconst { like, integer, string } = MatchersV3;\n\nconst pact = createPact();\n\ndescribe('Movies API Consumer Contract', () => {\n const movieWithId = { id: 1, name: 'The Matrix', year: 1999, rating: 8.7, director: 'Wachowskis' };\n\n it('should get a movie by ID', async () => {\n const [stateName, stateParams] = createProviderState(movieExists(movieWithId));\n\n await pact\n .addInteraction()\n .given(stateName, stateParams)\n .uponReceiving('a request to get movie by ID')\n .withRequest(\n 'GET',\n '/movies/1',\n setJsonContent({\n headers: { Accept: 'application/json' },\n }),\n )\n .willRespondWith(\n 200,\n setJsonBody(\n like({\n id: integer(1),\n name: string('The Matrix'),\n year: integer(1999),\n rating: like(8.7),\n director: string('Wachowskis'),\n }),\n ),\n )\n .executeTest(async (mockServer: V3MockServer) => {\n // Inject mock server URL into the REAL consumer code\n setApiUrl(mockServer.url);\n\n // Call the REAL consumer function — this is what CDC testing validates\n const movie = await getMovieById(1);\n\n expect(movie.id).toBe(1);\n expect(movie.name).toBe('The Matrix');\n });\n });\n\n it('should handle movie not found', async () => {\n await pact\n .addInteraction()\n .given('No movies exist')\n .uponReceiving('a request for a non-existent movie')\n .withRequest('GET', '/movies/999')\n .willRespondWith(404, setJsonBody({ error: 'Movie not found' }))\n .executeTest(async (mockServer: V3MockServer) => {\n setApiUrl(mockServer.url);\n\n await expect(getMovieById(999)).rejects.toThrow();\n });\n });\n});\n```\n\n**Key Points**:\n\n- **CRITICAL**: Always test your REAL consumer code — import and call actual API client functions, never raw `fetch()`\n- Using `fetch()` directly only tests that Pact's mock server works, which is meaningless\n- Consumer code MUST expose a URL injection mechanism: `setApiUrl()`, env var override, or constructor parameter\n- If the consumer code doesn't support URL injection, add it — this is a design prerequisite for CDC testing\n- Use PactV4 `addInteraction()` builder (not PactV3 fluent API with `withRequest({...})` object)\n- **Interaction naming convention**: Use the pattern `\"a request to \u003caction> \u003cresource> [\u003ccondition>]\"` for `uponReceiving()`. Examples: `\"a request to get a movie by ID\"`, `\"a request to delete a non-existing movie\"`, `\"a request to create a movie that already exists\"`. These names appear in Pact Broker UI and verification logs — keep them descriptive and unique within the consumer-provider pair.\n- Use `setJsonContent` for request/response builder callbacks with query/header/body concerns; use `setJsonBody` for body-only response callbacks\n- Provider state factory functions (`movieExists`) return `ProviderStateInput` objects\n- `createProviderState` converts to `[stateName, stateParams]` tuple for `.given()`\n\n**Common URL injection patterns** (pick whichever fits your consumer architecture):\n\n| Pattern | Example | Best For |\n| -------------------- | -------------------------------------------- | --------------------- |\n| `setApiUrl(url)` | Mutates axios instance `baseURL` | Singleton HTTP client |\n| Constructor param | `new ApiClient({ baseUrl: mockServer.url })` | Class-based clients |\n| Environment variable | `process.env.API_URL = mockServer.url` | Config-driven apps |\n| Factory function | `createApi({ baseUrl: mockServer.url })` | Functional patterns |\n\n---\n\n### Example 9: Support Files\n\n#### Pact Config Factory\n\n```typescript\n// tests/contract/support/pact-config.ts\nimport path from 'node:path';\nimport { PactV4 } from '@pact-foundation/pact';\n\nexport const createPact = (overrides?: { consumer?: string; provider?: string }) =>\n new PactV4({\n dir: path.resolve(process.cwd(), 'pacts'),\n consumer: overrides?.consumer ?? 'MyConsumerApp',\n provider: overrides?.provider ?? 'MyProviderAPI',\n logLevel: 'warn',\n });\n```\n\n#### Provider State Factories\n\n```typescript\n// tests/contract/support/provider-states.ts\nimport type { ProviderStateInput } from './consumer-helpers';\n\nexport const movieExists = (movie: { id: number; name: string; year: number; rating: number; director: string }): ProviderStateInput => ({\n name: 'An existing movie exists',\n params: movie,\n});\n\nexport const hasMovieWithId = (id: number): ProviderStateInput => ({\n name: 'Has a movie with a specific ID',\n params: { id },\n});\n```\n\n#### Local Consumer Helpers Shim\n\n```typescript\n// tests/contract/support/consumer-helpers.ts\n// TODO(temporary scaffolding): Replace local TemplateHeaders/TemplateQuery types\n// with '@seontechnologies/pactjs-utils' exports when available.\n\ntype TemplateHeaders = Record\u003cstring, string | number | boolean>;\ntype TemplateQueryValue = string | number | boolean | Array\u003cstring | number | boolean>;\ntype TemplateQuery = Record\u003cstring, TemplateQueryValue>;\n\nexport type ProviderStateInput = {\n name: string;\n params: Record\u003cstring, unknown>;\n};\n\ntype JsonMap = { [key: string]: boolean | number | string | null | JsonMap | Array\u003cunknown> };\ntype JsonContentBuilder = {\n headers: (headers: TemplateHeaders) => unknown;\n jsonBody: (body: unknown) => unknown;\n query?: (query: TemplateQuery) => unknown;\n};\n\nexport type JsonContentInput = {\n body?: unknown;\n headers?: TemplateHeaders;\n query?: TemplateQuery;\n};\n\nexport const toJsonMap = (obj: Record\u003cstring, unknown>): JsonMap =>\n Object.fromEntries(\n Object.entries(obj).map(([key, value]) => {\n if (value === null || value === undefined) return [key, 'null'];\n if (typeof value === 'object' && !(value instanceof Date) && !Array.isArray(value)) return [key, JSON.stringify(value)];\n if (typeof value === 'number' || typeof value === 'boolean') return [key, value];\n if (value instanceof Date) return [key, value.toISOString()];\n return [key, String(value)];\n }),\n );\n\nexport const createProviderState = ({ name, params }: ProviderStateInput): [string, JsonMap] => [name, toJsonMap(params)];\n\nexport const setJsonContent =\n ({ body, headers, query }: JsonContentInput) =>\n (builder: JsonContentBuilder): void => {\n if (query && builder.query) {\n builder.query(query);\n }\n\n if (headers) {\n builder.headers(headers);\n }\n\n if (body !== undefined) {\n builder.jsonBody(body);\n }\n };\n\nexport const setJsonBody = (body: unknown) => setJsonContent({ body });\n```\n\n**Key Points**:\n\n- If `@seontechnologies/pactjs-utils` is not yet installed, create a local shim that mirrors the API\n- Add a TODO comment noting to swap for the published package when available\n- The shim exports `createProviderState`, `toJsonMap`, `setJsonContent`, `setJsonBody`, and helper input types\n- Keep shim types local (or sourced from public exports only); do not import from internal Pact paths like `@pact-foundation/pact/src/*`\n\n---\n\n### Example 10: .gitignore Entries\n\n**Context**: Pact-specific entries to add to `.gitignore`.\n\n```text\n# Pact contract testing artifacts\n/pacts/\npact-logs/\n```\n\n---\n\n### Example 11: Test File Organization — One File Per Consumer+Provider Pair\n\n**Context**: Avoiding Pact Rust FFI handle collisions when structuring consumer test files.\n\n**Rule**: Every consumer+provider pair maps to exactly one `.pacttest.ts` file. Never split interactions for the same pair across multiple files.\n\n**Root cause**: The Pact Rust FFI maintains one handle per consumer+provider pair per process. With `singleFork: true` (all files run sequentially in one forked process), two files for the same pair access the same FFI handle back-to-back. The second file's `new PactV4({ consumer, provider })` call re-enters the handle still holding stale interaction state from the first file. The first test in the second file starts the mock server in this corrupted state — \"request was expected but not received\" results, sporadic and Linux-CI-only (execution order differs between environments).\n\n**Evidence**: In `pactjs-utils`, `movies-read.pacttest.ts` and `movies-write.pacttest.ts` both used `consumer: 'SampleAppConsumer', provider: 'SampleMoviesAPI'`. The vitest config and CI workflow were correct throughout. The fix was merging the two files into `movies.pacttest.ts`. The config was not changed.\n\n```typescript\n// ❌ WRONG — same consumer+provider pair split across two files\n// movies-read.pacttest.ts\nconst pact = new PactV4({ consumer: 'SampleAppConsumer', provider: 'SampleMoviesAPI', ... })\ndescribe('Read Operations', () => { /* 4 tests: GET /movies, GET /movies/:id */ })\n\n// movies-write.pacttest.ts ← second PactV4 for the SAME pair = FFI handle collision\nconst pact = new PactV4({ consumer: 'SampleAppConsumer', provider: 'SampleMoviesAPI', ... })\ndescribe('Write Operations', () => { /* 5 tests: POST, PUT, DELETE */ })\n\n// ✅ RIGHT — one file per consumer+provider pair, describe blocks for organization\n// movies.pacttest.ts\nconst pact = new PactV4({ consumer: 'SampleAppConsumer', provider: 'SampleMoviesAPI', ... })\ndescribe('Movies API', () => {\n describe('Read Operations', () => { /* 4 tests */ })\n describe('Write Operations', () => { /* 5 tests */ })\n})\n```\n\n**Key Points**:\n\n- **File = contract**: A `.pacttest.ts` file represents one consumer+provider contract. One contract = one file.\n- **Describe blocks, not files**: Organize by operation type (`Read Operations`, `Write Operations`), resource, or feature — always within one file per pair.\n- **Different pairs = different files**: `ServiceA / BackendAPI` and `ServiceA / AuthAPI` are two contracts and correctly use two separate files. This rule only forbids splitting ONE pair.\n- **`singleFork: true` is not a fix for this**: It ensures correct pact JSON write semantics across files, but when two files share a pair it actually guarantees the FFI collision (both land in the same process). Without it you'd get file-write races instead. Neither is safe. The fix is one file per pair.\n- **Naming convention**: `{domain}.pacttest.ts` when one domain maps to one pair. `{consumer-kebab}-{provider-kebab}.pacttest.ts` when the filename must be self-describing about which pair it covers.\n\n---\n\n## Validation Checklist\n\nBefore presenting the consumer CDC framework to the user, verify:\n\n- [ ] `vitest.config.pact.ts` is minimal **and sets `fileParallelism: false` AND `pool: 'forks'` with `poolOptions.forks.singleFork: true`** (`fileParallelism: false` prevents shared pact JSON corruption from parallel workers; forks + `singleFork: true` is required for pact JSON write safety across files — see Example 2 Key Points for mechanism and evidence)\n- [ ] Each consumer+provider pair is covered by exactly ONE `.pacttest.ts` file — never split interactions for the same pair across multiple files (two `PactV4` instances for the same pair in one process cause FFI handle collision → \"request was expected but not received\" on Linux CI; `singleFork: true` does NOT prevent this — it ensures both files share one process, which guarantees the collision; see Example 11)\n- [ ] `vitest.config.pact.ts` does NOT set `sequence.concurrent: true`, `maxConcurrency > 1`, `maxWorkers > 1`, or `isolate: false` — all four defeat the serialization the rule relies on\n- [ ] `scripts/publish-pact.sh` normalizes interactions with `jq -S '.interactions |= sort_by(.description, (.providerStates[0].name // \"\"), .request.method, .request.path)'` before the `pact-broker publish` call (ensures byte-stable payload to PactFlow regardless of generator ordering)\n- [ ] Script names match pactjs-utils (`test:pact:consumer`, `publish:pact`, `can:i:deploy:consumer`, `record:consumer:deployment`)\n- [ ] Scripts source `env-setup.sh` inline in package.json\n- [ ] Shell scripts use `pact-broker` not `npx pact-broker`\n- [ ] Shell scripts use `PACTICIPANT` env var pattern\n- [ ] `can-i-deploy.sh` has `--retry-while-unknown=10 --retry-interval=30`\n- [ ] `can-i-deploy.sh` keeps the environment-wide check and, only when both\n `PACT_PROVIDER_BRANCH` and `PROVIDER_PACTICIPANT` exist, ignores that one\n provider there and checks its branch in a second failing command\n- [ ] `record-deployment.sh` has branch guard\n- [ ] `env-setup.sh` uses `set -eu`; broker scripts use `set -euo pipefail` — each with explanatory comment\n- [ ] CI workflow named `contract-test-consumer.yml`\n- [ ] CI has workflow-level env block (not per-step)\n- [ ] CI has `detect-breaking-change` step before install\n- [ ] CI step (1) generates pact files (calls `npm run test:pact:consumer`) — its own visible step, not folded into publish\n- [ ] CI steps are 1:1 with developer commands — every CI step calls `npm run \u003csame-name>` a dev would run locally (no direct `vitest` or `pact-broker` invocation)\n- [ ] CI step numbering skips (3) — webhook-triggered provider verification\n- [ ] CI can-i-deploy has `PACT_BREAKING_CHANGE != 'true'` condition\n- [ ] CI has NO upload-artifact step\n- [ ] `.github/actions/detect-breaking-change/action.yml` exists\n- [ ] `.github/actions/detect-provider-branch/action.yml` reads `Pact provider\nbranch: \u003cname>` on PR events only\n- [ ] Consumer tests use `.pacttest.ts` extension\n- [ ] Consumer tests use PactV4 `addInteraction()` builder\n- [ ] `uponReceiving()` names follow `\"a request to \u003caction> \u003cresource> [\u003ccondition>]\"` pattern and are unique within the consumer-provider pair\n- [ ] Interaction callbacks use `setJsonContent` for query/header/body and `setJsonBody` for body-only responses\n- [ ] Request bodies use exact values (no `like()` wrapper) — Postel's Law: be strict in what you send\n- [ ] `like()`, `eachLike()`, `string()`, `integer()` matchers are only used in `willRespondWith` (responses), not in `withRequest` (requests) — matchers check type/shape, not exact values\n- [ ] Consumer tests call REAL consumer code (actual API client functions), NOT raw `fetch()`\n- [ ] Consumer code exposes URL injection mechanism (`setApiUrl()`, env var, or constructor param)\n- [ ] Local consumer-helpers shim present if pactjs-utils not installed\n- [ ] `.gitignore` includes `/pacts/` and `pact-logs/`\n\n## Related Fragments\n\n- `pactjs-utils-overview.md` — Library decision tree and installation\n- `pactjs-utils-consumer-helpers.md` — `createProviderState`, `toJsonMap`, `setJsonContent`, `setJsonBody`, **one-interaction-per-`it()` rule**\n- `pactjs-utils-provider-verifier.md` — Provider-side verification patterns; consumer and provider BOTH require `pool: 'forks'` + `singleFork: true` — same FFI-safety rule applies on both sides\n- `pactjs-utils-request-filter.md` — Auth injection for provider verification\n- `pact-broker-webhooks.md` — PactFlow → GitHub webhook auth pattern (dedicated user, classic PAT, PactFlow secret) and staleness monitoring\n- `contract-testing.md` — Foundational CDC patterns and resilience coverage\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":36757,"content_sha256":"d514f2ee87f6ba64da407881bcb251bc41371794b2e0cfc488d179f380d21d28"},{"filename":"knowledge/pact-mcp.md","content":"# Pact MCP Server (SmartBear)\n\n## Principle\n\nUse the SmartBear MCP server to enable AI agent interaction with PactFlow/Pact Broker during contract testing workflows. The MCP server provides tools for generating pact tests, fetching provider states, reviewing test quality, and checking deployment safety — all accessible through the Model Context Protocol.\n\n## Rationale\n\n### Why MCP for contract testing?\n\n- **Live broker queries**: AI agents can fetch existing provider states, verification results, and deployment status directly from PactFlow\n- **Test generation assistance**: MCP tools generate consumer and provider tests based on existing contracts, OpenAPI specs, or templates\n- **Automated review**: MCP-powered review checks tests against best practices without manual inspection\n- **Deployment safety**: `can-i-deploy` checks integrated into agent workflows for real-time compatibility verification\n\n### When TEA uses it\n\n- **test-design workflow**: Fetch existing provider states to understand current contract landscape\n- **automate workflow**: Generate pact tests using broker knowledge and existing contracts\n- **test-review workflow**: Review pact tests against best practices with automated feedback\n- **ci workflow**: Reference can-i-deploy and matrix tools for pipeline guidance\n\n## Available Tools\n\n| # | Tool | Description | When Used |\n| --- | ------------------------- | ----------------------------------------------------------------------- | --------------------- |\n| 1 | **Generate Pact Tests** | Create consumer/provider tests from code, OpenAPI, or templates | automate workflow |\n| 2 | **Fetch Provider States** | List all provider states from broker for a given consumer-provider pair | test-design, automate |\n| 3 | **Review Pact Tests** | Analyze tests against contract testing best practices | test-review |\n| 4 | **Can I Deploy** | Check deployment safety via broker verification matrix | ci workflow |\n| 5 | **Matrix** | Query consumer-provider verification matrix | ci, test-design |\n| 6 | **PactFlow AI Status** | Check AI credits and permissions (PactFlow Cloud only) | diagnostics |\n| 7 | **Metrics - All** | Workspace-wide contract testing metrics | reporting |\n| 8 | **Metrics - Team** | Team-level adoption statistics (PactFlow Cloud only) | reporting |\n\n## Installation\n\n### Config file locations\n\n| Tool | Global Config File | Format |\n| ----------------- | ------------------------------------- | ---------------------- |\n| Claude Code | `~/.claude.json` | JSON (`mcpServers`) |\n| Codex | `~/.codex/config.toml` | TOML (`[mcp_servers]`) |\n| Gemini CLI | `~/.gemini/settings.json` | JSON (`mcpServers`) |\n| Cursor | `~/.cursor/mcp.json` | JSON (`mcpServers`) |\n| Windsurf | `~/.codeium/windsurf/mcp_config.json` | JSON (`mcpServers`) |\n| VS Code (Copilot) | `.vscode/mcp.json` | JSON (`servers`) |\n\n> **Claude Code tip**: Prefer the `claude mcp add` CLI over manual JSON editing. Use `-s user` for global (all projects) or omit for per-project (default).\n\n### CLI shortcuts (Claude Code and Codex)\n\n```bash\n# Claude Code — use add-json for servers with env vars (-s user = global)\nclaude mcp add-json -s user smartbear \\\n '{\"type\":\"stdio\",\"command\":\"npx\",\"args\":[\"-y\",\"@smartbear/mcp@latest\"],\"env\":{\"PACT_BROKER_BASE_URL\":\"https://{tenant}.pactflow.io\",\"PACT_BROKER_TOKEN\":\"\u003cyour-token>\"}}'\n\n# Codex\ncodex mcp add smartbear -- npx -y @smartbear/mcp@latest\n```\n\n### JSON config (Gemini CLI, Cursor, Windsurf)\n\nAdd a `\"smartbear\"` entry to the `mcpServers` object in the config file for your tool:\n\n```json\n{\n \"mcpServers\": {\n \"smartbear\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@smartbear/mcp@latest\"],\n \"env\": {\n \"PACT_BROKER_BASE_URL\": \"https://{tenant}.pactflow.io\",\n \"PACT_BROKER_TOKEN\": \"\u003cyour-api-token>\"\n }\n }\n }\n}\n```\n\n### Codex TOML config\n\nCodex uses TOML instead of JSON. Add to `~/.codex/config.toml`:\n\n```toml\n[mcp_servers.smartbear]\ncommand = \"npx\"\nargs = [\"-y\", \"@smartbear/mcp@latest\"]\n\n[mcp_servers.smartbear.env]\nPACT_BROKER_BASE_URL = \"https://{tenant}.pactflow.io\"\nPACT_BROKER_TOKEN = \"\u003cyour-api-token>\"\n```\n\nNote the key is `mcp_servers` (underscored), not `mcpServers`.\n\n### VS Code (GitHub Copilot)\n\nAdd to `.vscode/mcp.json` (note: uses `servers` key, not `mcpServers`):\n\n```json\n{\n \"servers\": {\n \"smartbear\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@smartbear/mcp@latest\"],\n \"env\": {\n \"PACT_BROKER_BASE_URL\": \"https://{tenant}.pactflow.io\",\n \"PACT_BROKER_TOKEN\": \"${input:pactToken}\"\n }\n }\n }\n}\n```\n\n> **Note**: Set either `PACT_BROKER_TOKEN` (for PactFlow) or `PACT_BROKER_USERNAME`+`PACT_BROKER_PASSWORD` (for self-hosted). Leave unused vars empty.\n\n## Required Environment Variables\n\n| Variable | Required | Description |\n| ---------------------- | ---------------------------- | --------------------------------------- |\n| `PACT_BROKER_BASE_URL` | Yes (for Pact features) | PactFlow or self-hosted Pact Broker URL |\n| `PACT_BROKER_TOKEN` | For PactFlow / token auth | API token for broker authentication |\n| `PACT_BROKER_USERNAME` | For basic auth (self-hosted) | Username for basic authentication |\n| `PACT_BROKER_PASSWORD` | For basic auth (self-hosted) | Password for basic authentication |\n\n**Authentication**: Use token auth (`PACT_BROKER_TOKEN`) for PactFlow. Use basic auth (`PACT_BROKER_USERNAME` + `PACT_BROKER_PASSWORD`) for self-hosted Pact Broker instances. Only one auth method is needed.\n\n**Requirements**: Node.js 20+\n\n## Pattern Examples\n\n### Example 1: Fetching Provider States During Test Design\n\nWhen designing contract tests, use MCP to query existing provider states:\n\n```text\n# Agent queries SmartBear MCP during test-design workflow:\n# → Fetch Provider States for consumer=\"movie-web\", provider=\"SampleMoviesAPI\"\n# ← Returns: [\"movie with id 1 exists\", \"no movies exist\", \"user is authenticated\"]\n#\n# Agent uses this to generate comprehensive consumer tests covering all states\n```\n\n### Example 2: Reviewing Pact Tests\n\nDuring test-review workflow, use MCP to evaluate test quality:\n\n```text\n# Agent submits test file to SmartBear MCP Review tool:\n# → Review Pact Tests with test file content\n# ← Returns: feedback on matcher usage, state coverage, interaction naming\n#\n# Agent incorporates feedback into review report\n```\n\n### Example 3: Can I Deploy Check in CI\n\nDuring CI workflow design, reference the can-i-deploy tool:\n\n```text\n# Agent generates CI pipeline with can-i-deploy gate:\n# → Can I Deploy: pacticipant=\"SampleMoviesAPI\", version=\"${GITHUB_SHA}\", to=\"production\"\n# ← Returns: { ok: true/false, reason: \"...\" }\n#\n# Agent designs pipeline to block deployment if can-i-deploy fails\n```\n\n## When the Tools Are Not Reachable\n\n`tea_pact_mcp` defaults to `\"mcp\"`, so every contract-testing step reaches this fragment on a normal install. Most installs have no broker. That is expected, and it must cost nothing.\n\n### The probe is a tool-list check, never a broker call\n\nReachability means **the SmartBear MCP tools are present in this session's tool list**. Nothing else. Do not call `Can I Deploy`, `Matrix`, or any other tool to discover whether the broker answers: a missing server is instant and free to detect, while a misconfigured or unreachable broker is a network round trip that can hang the step. The two failure modes have very different costs and only the free one is worth probing for.\n\nCredentials are likewise not probed. `PACT_BROKER_BASE_URL` and `PACT_BROKER_TOKEN` being unset is a reason the first real tool call will fail, not a reason to make one.\n\n### Probe once per run\n\nRecord the result in run state as `pact_mcp_reachable` the first time any step needs it, and have every later step read that value instead of probing again. A run that touches test design, generation, and review must not ask three times, and \"probe once\" is only true if the answer is stored.\n\n### Fallback order, in this order\n\n1. **Provider source.** Route handlers, types, DTOs, validation schemas. Most authoritative.\n2. **An OpenAPI or Swagger spec** (`openapi.yaml`, `openapi.json`, `swagger.json`) when provider source is not readable.\n3. **Neither available:** apply `confidence-gate.md`. Stop and ask. Do not invent a provider state, a status code, or a response shape.\n\nThere is no fourth branch, and inference is not one of the three.\n\n### What the output must say\n\nState it once per run, in the workflow summary, in this form:\n\n`Pact broker: unreachable (SmartBear MCP tools not available). Provider states derived from \u003cprovider source | OpenAPI spec>.`\n\nThen continue. Specifically:\n\n- **Never block a workflow** on the broker. A missing broker is not a HALT condition.\n- **Never retry.** One probe, one answer, no loop and no backoff.\n- **Never present inferred data as broker data.** A provider state read from a handler is provider-sourced; saying or implying it came from the broker misrepresents its authority, and a contract built on that is worse than one built on an acknowledged guess.\n- **`tea_pact_mcp: \"none\"`** skips the probe entirely. Treat it as \"unreachable\" without the check and without the report line: the user already said not to look.\n\nEverything above applies to the tools in this fragment only. It has no bearing on whether a Pact suite belongs in the project at all — that is the relevance gate in `pactjs-utils-mandate.md`.\n\n## Key Points\n\n- **Per-project install recommended**: Different projects may target different PactFlow tenants — match TEA's per-project config philosophy\n- **Env vars are project-specific**: `PACT_BROKER_BASE_URL` and `PACT_BROKER_TOKEN` vary by project/team\n- **Node.js 20+ required**: SmartBear MCP server requires Node.js 20 or higher\n- **PactFlow Cloud features**: Some tools (AI Status, Team Metrics) are only available with PactFlow Cloud, not self-hosted Pact Broker\n- **Complements pactjs-utils**: MCP provides broker interaction during design/review; pactjs-utils provides runtime utilities for test code\n\n## Related Fragments\n\n- `pactjs-utils-overview.md` — runtime utilities that pact tests import\n- `pactjs-utils-provider-verifier.md` — verifier options that reference broker config\n- `pact-broker-webhooks.md` — PactFlow → GitHub webhook auth pattern and staleness monitoring; `Metrics - All` / `Matrix` MCP tools are useful here for dashboards\n- `contract-testing.md` — foundational contract testing patterns\n\n## Anti-Patterns\n\n### Wrong: Using MCP for runtime test execution\n\n```text\n# ❌ Don't use MCP to run pact tests — use npm scripts and CI pipelines\n# MCP is for agent-assisted design, generation, and review\n```\n\n### Right: Use MCP for design-time assistance\n\n```text\n# ✅ Use MCP during planning and review:\n# - Fetch provider states to inform test design\n# - Generate test scaffolds from existing contracts\n# - Review tests for best practice compliance\n# - Check can-i-deploy during CI pipeline design\n```\n\n_Source: SmartBear MCP documentation, PactFlow developer docs._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":11671,"content_sha256":"77f7a9fa2ee04f6cdd2af2f654602179f30008505c22e8ba9711e423ff61fc4e"},{"filename":"knowledge/pactjs-utils-consumer-helpers.md","content":"# Pact.js Utils Consumer Helpers\n\n## Principle\n\nUse `createProviderState`, `toJsonMap`, `setJsonContent`, and `setJsonBody` from `@seontechnologies/pactjs-utils` to build type-safe provider state tuples and reusable PactV4 JSON callbacks for consumer contract tests. These helpers eliminate manual `JsonMap` casting and repetitive inline builder lambdas.\n\n## Rationale\n\n### Problems with raw consumer helper handling\n\n- **JsonMap requirement**: Pact's `.given(stateName, params)` requires `params` to be `JsonMap` — a flat object where every value must be `string | number | boolean | null`\n- **Type gymnastics**: Complex params (Date objects, nested objects, null values) require manual casting that TypeScript can't verify\n- **Inconsistent serialization**: Different developers serialize the same data differently (e.g., dates as ISO strings vs timestamps)\n- **Verbose `.given()` calls**: Repeating state name and params inline makes consumer tests harder to read\n- **Repeated interaction callbacks**: PactV4 interactions duplicate inline `(builder) => { ... }` blocks for body/query/header setup\n\n### Solutions\n\n- **`createProviderState`**: Returns a `[string, JsonMap]` tuple that spreads directly into `.given()` — one function handles name and params\n- **`toJsonMap`**: Explicit coercion rules documented and tested — Date→ISO string, null→\"null\" string, nested objects→JSON string\n- **`setJsonContent`**: Curried callback helper for request/response builders — set `query`, `headers`, and/or `body` from one reusable function\n- **`setJsonBody`**: Body-only shorthand for `setJsonContent({ body })` — ideal for concise `.willRespondWith(...)` bodies\n\n## Pattern Examples\n\n### Example 1: Basic Provider State Creation\n\n```typescript\nimport { PactV3, MatchersV3 } from '@pact-foundation/pact';\nimport { createProviderState } from '@seontechnologies/pactjs-utils';\n\nconst provider = new PactV3({\n consumer: 'movie-web',\n provider: 'SampleMoviesAPI',\n dir: './pacts',\n});\n\ndescribe('Movie API Contract', () => {\n it('should return movie by id', async () => {\n // createProviderState returns [stateName, JsonMap] tuple\n const providerState = createProviderState({\n name: 'movie with id 1 exists',\n params: { id: 1, name: 'Inception', year: 2010 },\n });\n\n await provider\n .given(...providerState) // Spread tuple into .given(name, params)\n .uponReceiving('a request for movie 1')\n .withRequest({ method: 'GET', path: '/movies/1' })\n .willRespondWith({\n status: 200,\n body: MatchersV3.like({ id: 1, name: 'Inception', year: 2010 }),\n })\n .executeTest(async (mockServer) => {\n const res = await fetch(`${mockServer.url}/movies/1`);\n const movie = await res.json();\n expect(movie.name).toBe('Inception');\n });\n });\n});\n```\n\n**Key Points**:\n\n- `createProviderState` accepts `{ name: string, params: Record\u003cstring, unknown> }`\n- Both `name` and `params` are required (pass `params: {}` for states without parameters)\n- Returns `[string, JsonMap]` — spread with `...` into `.given()`\n- `params` values are automatically converted to JsonMap-compatible types\n- Works identically with HTTP (`PactV3`) and message (`MessageConsumerPact`) pacts\n\n### Example 2: Complex Parameters with toJsonMap\n\n```typescript\nimport { toJsonMap } from '@seontechnologies/pactjs-utils';\n\n// toJsonMap conversion rules:\n// - string, number, boolean → passed through\n// - null → \"null\" (string)\n// - undefined → \"null\" (string, same as null)\n// - Date → ISO string (e.g., \"2025-01-15T10:00:00.000Z\")\n// - nested object → JSON string\n// - array → comma-separated string via String() (e.g., [1,2,3] → \"1,2,3\")\n\nconst params = toJsonMap({\n id: 42,\n name: 'John Doe',\n active: true,\n score: null,\n createdAt: new Date('2025-01-15T10:00:00Z'),\n metadata: { role: 'admin', permissions: ['read', 'write'] },\n});\n\n// Result:\n// {\n// id: 42,\n// name: \"John Doe\",\n// active: true,\n// score: \"null\",\n// createdAt: \"2025-01-15T10:00:00.000Z\",\n// metadata: '{\"role\":\"admin\",\"permissions\":[\"read\",\"write\"]}'\n// }\n```\n\n**Key Points**:\n\n- `toJsonMap` is called internally by `createProviderState` — you rarely need it directly\n- Use it when you need explicit control over parameter conversion outside of provider states\n- Conversion rules are deterministic: same input always produces same output\n\n### Example 3: Provider State Without Parameters\n\n```typescript\nimport { createProviderState } from '@seontechnologies/pactjs-utils';\n\n// State without params — second tuple element is empty object\nconst emptyState = createProviderState({ name: 'no movies exist', params: {} });\n// Returns: ['no movies exist', {}]\n\nawait provider\n .given(...emptyState)\n .uponReceiving('a request when no movies exist')\n .withRequest({ method: 'GET', path: '/movies' })\n .willRespondWith({ status: 200, body: [] })\n .executeTest(async (mockServer) => {\n const res = await fetch(`${mockServer.url}/movies`);\n const movies = await res.json();\n expect(movies).toEqual([]);\n });\n```\n\n### Example 4: Multiple Provider States\n\n```typescript\nimport { createProviderState } from '@seontechnologies/pactjs-utils';\n\n// Some interactions require multiple provider states\n// Call .given() multiple times with different states\nawait provider\n .given(...createProviderState({ name: 'user is authenticated', params: { userId: 1 } }))\n .given(...createProviderState({ name: 'movie with id 5 exists', params: { id: 5 } }))\n .uponReceiving('an authenticated request for movie 5')\n .withRequest({\n method: 'GET',\n path: '/movies/5',\n headers: { Authorization: MatchersV3.like('Bearer token') },\n })\n .willRespondWith({ status: 200, body: MatchersV3.like({ id: 5 }) })\n .executeTest(async (mockServer) => {\n // test implementation\n });\n```\n\n### Example 5: When to Use setJsonBody vs setJsonContent\n\n```typescript\nimport { MatchersV3 } from '@pact-foundation/pact';\nimport { setJsonBody, setJsonContent } from '@seontechnologies/pactjs-utils';\n\nconst { integer, string } = MatchersV3;\n\nawait pact\n .addInteraction()\n .given('movie exists')\n .uponReceiving('a request to get movie by name')\n .withRequest(\n 'GET',\n '/movies',\n setJsonContent({\n query: { name: 'Inception' },\n headers: { Accept: 'application/json' },\n }),\n )\n .willRespondWith(\n 200,\n setJsonBody({\n status: 200,\n data: { id: integer(1), name: string('Inception') },\n }),\n );\n```\n\n**Key Points**:\n\n- Use `setJsonContent` when the interaction needs `query`, `headers`, and/or `body` in one callback (most request builders)\n- Use `setJsonBody` when you only need `jsonBody` and want the shorter `.willRespondWith(status, setJsonBody(...))` form\n- `setJsonBody` is equivalent to `setJsonContent({ body: ... })`\n\n### Example 6: One `addInteraction()` per `it()` Block (PactV4 Determinism Rule)\n\n**Context**: PactV4's `pact.addInteraction()` feeds the Rust FFI layer that writes interactions to the pact JSON. Chaining multiple `.addInteraction()...executeTest()` blocks inside a single `it()` — or otherwise registering multiple interactions before a single `executeTest` — causes the FFI to **non-deterministically drop whole interactions** (not individual fields) in roughly 1 out of N runs. The pattern passes locally, then fails intermittently in CI or at publish time with `Cannot change pact content for already published pact` once the dropped interaction reappears on a re-run.\n\n**Rule**: Exactly one `pact.addInteraction()` per `it()` block. For N interactions, write N `it()` blocks, or use `it.each(...)`.\n\n```typescript\n// ❌ WRONG — two addInteraction() inside one it() — FFI non-deterministically drops one\nit('handles movie lookup scenarios', async () => {\n await pact\n .addInteraction()\n .given('movie exists')\n .uponReceiving('a request to get movie by id')\n .withRequest('GET', '/movies/1')\n .willRespondWith(200, setJsonBody({ id: integer(1), name: string('The Matrix') }))\n .executeTest(async (mockServer) => {\n /* ... */\n });\n\n // Sometimes this second interaction never makes it to the pact JSON:\n await pact\n .addInteraction()\n .given('no movies exist')\n .uponReceiving('a request for an empty list')\n .withRequest('GET', '/movies')\n .willRespondWith(200, setJsonBody([]))\n .executeTest(async (mockServer) => {\n /* ... */\n });\n});\n\n// ✅ RIGHT — one addInteraction() per it()\nit('gets a movie by id', async () => {\n await pact\n .addInteraction()\n .given('movie exists')\n .uponReceiving('a request to get movie by id')\n .withRequest('GET', '/movies/1')\n .willRespondWith(200, setJsonBody({ id: integer(1), name: string('The Matrix') }))\n .executeTest(async (mockServer) => {\n /* ... */\n });\n});\n\nit('returns empty list when no movies exist', async () => {\n await pact\n .addInteraction()\n .given('no movies exist')\n .uponReceiving('a request for an empty list')\n .withRequest('GET', '/movies')\n .willRespondWith(200, setJsonBody([]))\n .executeTest(async (mockServer) => {\n /* ... */\n });\n});\n\n// ✅ RIGHT — parameterized via it.each for data-driven coverage\nit.each([\n { id: 1, name: 'The Matrix' },\n { id: 2, name: 'Inception' },\n])('gets movie $id', async ({ id, name }) => {\n await pact\n .addInteraction()\n .given('movie exists', { id, name })\n .uponReceiving(`a request to get movie ${id}`)\n .withRequest('GET', `/movies/${id}`)\n .willRespondWith(200, setJsonBody({ id: integer(id), name: string(name) }))\n .executeTest(async (mockServer) => {\n /* ... */\n });\n});\n```\n\n**Key Points**:\n\n- **This rule stacks with two MANDATORY vitest settings and one file-organization rule. All four address different failure modes; none substitutes for the others**: (1) `fileParallelism: false` — prevents parallel workers racing on the shared pact JSON file; (2) `pool: 'forks'` with `singleFork: true` — required for pact JSON write safety across multiple files; (3) **one `.pacttest.ts` per consumer+provider pair** — `singleFork: true` keeps all files in one process, so two files for the same pair produce an FFI handle collision (\"request was expected but not received\" on Linux CI, sporadic); (4) one-interaction-per-`it()` (this rule) — prevents the FFI from dropping interactions within a single test body. See `pact-consumer-framework-setup.md` Example 10 for the file-organization ✅/❌ pattern.\n- Symptom of violating this rule: the pact file is byte-different between otherwise-identical runs; PactFlow rejects a republish with `Cannot change pact content`.\n- The rule applies to both HTTP consumer pacts (`PactV4`) and message consumer pacts (`MessageConsumerPact`).\n\n## Key Points\n\n- **Spread pattern**: Always use `...createProviderState()` — the tuple spreads into `.given(stateName, params)`\n- **Type safety**: TypeScript enforces `{ name: string, params: Record\u003cstring, unknown> }` input (both fields required)\n- **Null handling**: `null` becomes `\"null\"` string in JsonMap (Pact requirement)\n- **Date handling**: Date objects become ISO 8601 strings\n- **No nested objects in JsonMap**: Nested objects are JSON-stringified — provider state handlers must parse them\n- **Array serialization is lossy**: Arrays are converted via `String()` (e.g., `[1,2,3]` → `\"1,2,3\"`) — prefer passing arrays as JSON-stringified objects for round-trip safety\n- **Message pacts**: Works identically with `MessageConsumerPact` — same `.given()` API\n- **Builder reuse**: `setJsonContent` works for both `.withRequest(...)` and `.willRespondWith(...)` callbacks (query is ignored on response builders)\n- **Body shorthand**: `setJsonBody` keeps body-only responses concise and readable\n- **Matchers check type, not value**: `string('My movie')` means \"any string\", `integer(1)` means \"any integer\". The example values are arbitrary — the provider can return different values and verification still passes as long as the type matches. Use matchers only in `.willRespondWith()` (responses), never in `.withRequest()` (requests) — Postel's Law applies.\n- **Reuse test values across files**: Interactions are uniquely identified by `uponReceiving` + `.given()`, not by placeholder values. Two test files can both use `testId: 100` without conflicting. On the provider side, shared values simplify state handlers — idempotent handlers (check if exists, create if not) only need to ensure one record exists. Use different values only when testing different states of the same entity type (e.g., `movieExists(100)` for happy paths vs. `movieNotFound(999)` for error paths).\n- **One `addInteraction()` per `it()` block (MANDATORY for PactV4)**: Multiple interactions inside one `it()` cause the Rust FFI to non-deterministically drop interactions. Use one `it()` per interaction or `it.each(...)` for parameterized cases. See Example 6.\n\n## Related Fragments\n\n- `pactjs-utils-overview.md` — installation, decision tree, design philosophy\n- `pactjs-utils-provider-verifier.md` — provider-side state handler implementation; same `pool: 'forks'` + `singleFork: true` rule as consumer\n- `pact-consumer-framework-setup.md` — Vitest `fileParallelism: false` + `pool: 'forks'` + `singleFork: true` config and CI wiring\n- `contract-testing.md` — foundational patterns with raw Pact.js\n\n## Anti-Patterns\n\n### Wrong: Manual JsonMap assembly\n\n```typescript\n// ❌ Manual casting — verbose, error-prone, no type safety\nprovider.given('user exists', {\n id: 1 as unknown as string,\n createdAt: new Date().toISOString(),\n metadata: JSON.stringify({ role: 'admin' }),\n} as JsonMap);\n```\n\n### Right: Use createProviderState\n\n```typescript\n// ✅ Automatic conversion with type safety\nprovider.given(\n ...createProviderState({\n name: 'user exists',\n params: { id: 1, createdAt: new Date(), metadata: { role: 'admin' } },\n }),\n);\n```\n\n### Wrong: Inline state names without helper\n\n```typescript\n// ❌ Duplicated state names between consumer and provider — easy to mismatch\nprovider.given('a user with id 1 exists', { id: '1' });\n// Later in provider: 'user with id 1 exists' — different string!\n```\n\n### Right: Share state constants\n\n```typescript\n// ✅ Define state names as constants shared between consumer and provider\nconst STATES = {\n USER_EXISTS: 'user with id exists',\n NO_USERS: 'no users exist',\n} as const;\n\nprovider.given(...createProviderState({ name: STATES.USER_EXISTS, params: { id: 1 } }));\n```\n\n### Wrong: Repeating inline builder lambdas everywhere\n\n```typescript\n// ❌ Repetitive callback boilerplate in every interaction\n.willRespondWith(200, (builder) => {\n builder.jsonBody({ status: 200 });\n});\n```\n\n### Right: Use setJsonBody / setJsonContent\n\n```typescript\n// ✅ Reusable callbacks with less boilerplate\n.withRequest('GET', '/movies', setJsonContent({ query: { name: 'Inception' } }))\n.willRespondWith(200, setJsonBody({ status: 200 }));\n```\n\n### Wrong: Multiple `addInteraction()` in a single `it()`\n\n```typescript\n// ❌ PactV4 FFI non-deterministically drops one of these interactions ~1/N runs\nit('handles both success and empty list', async () => {\n await pact.addInteraction().uponReceiving('get movie').withRequest(/* ... */).executeTest(/* ... */);\n await pact.addInteraction().uponReceiving('empty list').withRequest(/* ... */).executeTest(/* ... */);\n});\n```\n\n### Right: One `addInteraction()` per `it()` (or use `it.each`)\n\n```typescript\n// ✅ Deterministic pact JSON — FFI receives one interaction per test\nit('gets a movie', async () => {\n await pact\n .addInteraction() /* ... */\n .executeTest(/* ... */);\n});\nit('returns empty list', async () => {\n await pact\n .addInteraction() /* ... */\n .executeTest(/* ... */);\n});\n```\n\nSee Example 6 above for the full rationale.\n\n_Source: @seontechnologies/pactjs-utils consumer-helpers module, pactjs-utils sample-app consumer tests._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":15935,"content_sha256":"de0efb6edd92e9e8f731132d02280c4edee882c3b58a9a1b37a027e95986b2ca"},{"filename":"knowledge/pactjs-utils-mandate.md","content":"# Pact.js Utils Mandate\n\n## Principle\n\nWhen `tea_use_pactjs_utils` is `true` and `@seontechnologies/pactjs-utils` is installed, that package is the **default implementation** for every capability it covers in a Pact suite. Raw `@pact-foundation/pact` boilerplate is a documented deviation, never a default. The flag is not a hint that the library exists; it is an instruction to write consumer and provider suites in that style without being asked.\n\nThis fragment instantiates `library-integration-mandate.md`. Read that one for the two gates, the enforcement levels, and the deviation protocol; this one carries the substitutions. The per-utility fragments (`pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`, `pactjs-utils-zod-to-pact.md`) are the reference for how each function is called.\n\n## Scope\n\n**Applies when all of these hold:**\n\n- `tea_use_pactjs_utils` is `true` under `[modules.tea]` (`config.tea_use_pactjs_utils`)\n- `@seontechnologies/pactjs-utils` is a dependency in the project's `package.json`\n- The file is a JavaScript or TypeScript Pact artifact: a consumer test (`.pacttest.ts`), a provider verification test, a message consumer or provider test, or their support files\n\n**Does not apply to** — nothing here overrides these:\n\n- Playwright specs, which follow `playwright-utils-mandate.md`\n- Pact suites in other languages (pact-jvm, pact-python, pact-go)\n- Projects with no contract-testing relevance at all. The flag being `true` is not a reason to introduce Pact into a repo that has no consumer/provider boundary; see \"Relevance Before Scaffolding\" below.\n\n## Relevance Before Scaffolding\n\n`tea_use_pactjs_utils` defaults to `true`, which means \"use these utilities when you write contract tests\", not \"write contract tests everywhere\".\n\nThis is the one gate for the decision. Every workflow defers to it rather than restating its own list.\n\n**Sufficient on its own.** Any one of these settles it:\n\n- An existing `pact/` or `tests/contract/` directory\n- `@pact-foundation/pact` already in `package.json`\n- `PACT_BROKER_BASE_URL` or another `PACT_BROKER_*` variable in the environment or `.env.example`\n- A microservices layout: two or more independently deployable services in this repo that call each other\n- The user asked for contract testing\n\n**Not sufficient on its own.** An outbound HTTP call, a generated API client, or a service URL in `.env.example` is weak evidence: most frontends have all three and call a backend that ships in the same deploy. Treat these as evidence only when **both** hold:\n\n1. The called service has **no source in this repo**, and is not started by this repo's compose file, dev script, or CI, and\n2. A second signal is present — another item from this list, or one of the sufficient signals above.\n\nThe disqualifier in (1) is the whole test, so check it rather than assume it. A repo whose frontend calls its own backend has no consumer-provider boundary in the Pact sense: both sides deploy together, so a contract adds ceremony and no safety.\n\n**With none of that, create no Pact artifacts.** Say in the summary that contract scaffolding was skipped because no consumer-provider boundary was found, and that the `framework` workflow can add it later. When the evidence is ambiguous, apply `confidence-gate.md` and ask rather than scaffolding on a guess.\n\nGetting this wrong is worse than a missing utility: it leaves a dead contract suite that fails CI for a boundary the project does not have.\n\n## What Never Relaxes\n\nThese are correctness rules from the per-utility fragments, not style preferences, and the mandate does not soften them. They apply whether or not pactjs-utils is in use:\n\n- **One `pact.addInteraction()` per `it()` block.** PactV4's Rust FFI drops interactions non-deterministically when several are chained in one test. Use `it.each` for parameterized cases.\n- **Consumer Vitest config** carries `fileParallelism: false` AND `pool: 'forks'` with `poolOptions.forks.singleFork: true`.\n- **Provider Vitest config** carries `pool: 'forks'` with `singleFork: true`.\n- **Provider scrutiny before matchers.** Response matchers come from provider source, an OpenAPI spec, or broker data — never from consumer-side types alone. See the Seven-Point Scrutiny Checklist in `contract-testing.md`.\n- **Postel's Law for matchers.** Matchers belong in `willRespondWith` only. Request bodies in `withRequest` use exact values; the consumer controls what it sends.\n- **A `// Provider endpoint: \u003cpath> -> \u003cMETHOD> \u003croute>` comment** on every interaction.\n\n## Substitution Table\n\n| Need | Raw Pact — do not emit | pactjs-utils — emit this | Level | Fragment |\n| -------------------------------------------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- | ----------- | ----------------------------------- |\n| Provider state on an interaction | `.given('name', { id: 1 } as JsonMap)`, hand-cast params | `.given(...createProviderState({ name, params }))` | REQUIRED | `pactjs-utils-consumer-helpers.md` |\n| Coercing params to Pact's `JsonMap` | Manual casts, `String(date)`, `null` handling per call site | `toJsonMap(value)` | REQUIRED | `pactjs-utils-consumer-helpers.md` |\n| PactV4 request/response builder callbacks | Repeated inline `(b) => { b.query(...); b.headers(...); b.jsonBody(...) }` lambdas | `setJsonContent({ query?, headers?, body? })`, or `setJsonBody(body)` for body-only | REQUIRED | `pactjs-utils-consumer-helpers.md` |\n| HTTP provider verification options | A hand-assembled 30-line `VerifierOptions` object | `buildVerifierOptions({ provider, port, includeMainAndDeployed, stateHandlers })` | REQUIRED | `pactjs-utils-provider-verifier.md` |\n| Message/Kafka provider verification options | A second hand-assembled options object | `buildMessageVerifierOptions({ ... })` | REQUIRED | `pactjs-utils-provider-verifier.md` |\n| Broker URL and consumer version selectors | Hand-written env branching for local, remote, breaking-change, or named-branch flows | `handlePactBrokerUrlAndSelectors(...)` (or let `buildVerifierOptions` read the env) | REQUIRED | `pactjs-utils-provider-verifier.md` |\n| Provider version tags in CI | Hand-written branch/tag extraction per CI platform | `getProviderVersionTags()` | REQUIRED | `pactjs-utils-provider-verifier.md` |\n| Breaking-change tolerant branch check | Repeated `main` / `master` / `release/` string logic | `isBreakingChangeTolerantBranch(branch)` | REQUIRED | `pactjs-utils-provider-verifier.md` |\n| Auth injection during provider verification | A bespoke Express middleware, with its recurring `Bearer Bearer` bug | `createRequestFilter({ tokenGenerator })` | REQUIRED | `pactjs-utils-request-filter.md` |\n| A provider that needs no auth injection | Omitting `requestFilter`, or an empty inline function | `noOpRequestFilter` | REQUIRED | `pactjs-utils-request-filter.md` |\n| Response matchers where a Zod schema exists | Hand-written `MatchersV3` trees duplicating the schema | `zodToPactMatchers(schema, examples?)` | RECOMMENDED | `pactjs-utils-zod-to-pact.md` |\n| Exercising the consumer inside `executeTest` | Raw `fetch(`${mockServer.url}/...`)` | Inject `mockServer.url` as `baseUrl` and call the real client | RECOMMENDED | `pact-consumer-di.md` |\n\n`zodToPactMatchers` is RECOMMENDED because it needs a Zod schema to exist. Where the project has one, derive the matchers from it rather than maintaining a parallel matcher tree. Where it does not, write matchers from provider scrutiny and say so.\n\nThe DI pattern is RECOMMENDED because it needs a two-line change in production code (an optional `baseUrl` on the API context type). Propose it and name the change. Falling back to raw `fetch` without saying so ships a contract that is a hand-crafted guess at what the consumer sends, which is the failure `pact-consumer-di.md` exists to prevent.\n\n## Banned Patterns\n\nWhen this mandate is active, these are defects in generated or reviewed code:\n\n- `.given('state name', someObject as JsonMap)` — a hand-cast provider state where `createProviderState` applies.\n- A literal `VerifierOptions` object passed to `new Verifier(...)`, where `buildVerifierOptions` applies.\n- Hand-built `{ branch: process.env.PACT_CONSUMER_BRANCH }` selectors where the\n builder's scoped `consumer` + `consumerBranch` inputs apply.\n- Hand-written `branch === 'main' || branch === 'master' ||\nbranch.startsWith('release/')` checks where\n `isBreakingChangeTolerantBranch` applies.\n- A bespoke `requestFilter` middleware that prefixes a bearer token by hand.\n- Repeated inline PactV4 builder lambdas that `setJsonContent` or `setJsonBody` would replace.\n- Raw `fetch` inside `executeTest` in a project whose consumer client is importable, with no note saying why.\n- Importing a **pactjs-utils symbol** from anywhere other than `@seontechnologies/pactjs-utils`. `@pact-foundation/pact` remains the correct and required import for Pact's own API — `PactV3`, `PactV4`, `MatchersV3`, `Verifier`, `V3MockServer`, and the types. The two packages are used side by side in every example here; only the helper layer is mandated.\n\n### Legitimate exceptions\n\nThese are not violations and need no deviation note:\n\n- `MatchersV3` used directly for a matcher `zodToPactMatchers` cannot express, or where no schema exists.\n- A raw `VerifierOptions` field passed through `buildVerifierOptions`'s own escape hatch for something it does not model.\n- Raw `fetch` in a consumer test where the consumer client genuinely cannot be imported (a different package, a build artifact, a language boundary) — state it once in the summary rather than per line.\n\n## Canonical Shapes\n\n### Consumer test\n\nPactV4 `addInteraction()`, one per `it()`, with `setJsonContent` on the request and\n`setJsonBody` on the response. That is four of the REQUIRED substitutions in one\nshape, which is the point: a worker copies this, so it has to demonstrate the rules\nrather than describe them.\n\n```typescript\nimport { PactV4, MatchersV3 } from '@pact-foundation/pact';\nimport { createProviderState, setJsonBody, setJsonContent } from '@seontechnologies/pactjs-utils';\nimport { getMovieById } from '../../src/api/movies-client';\n\nconst { integer, string } = MatchersV3;\n\nconst pact = new PactV4({ consumer: 'movie-web', provider: 'SampleMoviesAPI', dir: './pacts' });\n\ndescribe('Movie API Contract', () => {\n it('returns a movie by id', async () => {\n // Provider endpoint: server/src/routes/movies.ts -> GET /movies/:id\n await pact\n .addInteraction()\n .given(...createProviderState({ name: 'movie with id 1 exists', params: { id: 1 } }))\n .uponReceiving('a request for movie 1')\n .withRequest('GET', '/movies/1', setJsonContent({ headers: { Accept: 'application/json' } }))\n .willRespondWith(200, setJsonBody({ id: integer(1), name: string('Inception') }))\n .executeTest(async (mockServer) => {\n // DI: the real client, pointed at the mock server\n const movie = await getMovieById(1, { baseUrl: mockServer.url });\n expect(movie.name).toBe('Inception');\n });\n });\n});\n```\n\nExactly one `addInteraction()` in that `it()`. A second scenario is a second `it()`,\nor `it.each` — never a second chain in the same block. See\n`pactjs-utils-consumer-helpers.md` Example 6 for what the FFI does otherwise.\n\nWhere the project already has a Zod schema for the response, replace the inline\n`MatchersV3` tree with `zodToPactMatchers(MovieSchema)` and keep the schema as the\nsingle source of the shape. `MatchersV3` written by hand is correct only where no\nschema exists.\n\n### Provider verification\n\n```typescript\nimport { Verifier } from '@pact-foundation/pact';\nimport { buildVerifierOptions, createRequestFilter } from '@seontechnologies/pactjs-utils';\nimport type { StateHandlers } from '@seontechnologies/pactjs-utils';\n\nconst stateHandlers: StateHandlers = {\n 'movie with id 1 exists': {\n setup: async (params) => db.seed({ movies: [{ id: params?.id ?? 1 }] }),\n teardown: async () => db.clean('movies'),\n },\n};\n\nawait new Verifier(\n buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: '3001',\n includeMainAndDeployed: process.env.PACT_BREAKING_CHANGE !== 'true',\n consumer: 'movie-web',\n consumerBranch: process.env.PACT_CONSUMER_BRANCH,\n stateHandlers,\n requestFilter: createRequestFilter({ tokenGenerator: () => process.env.TEST_AUTH_TOKEN ?? 'test-token' }),\n }),\n).verifyProvider();\n```\n\nState handler names and their `params` must match the consumer's `createProviderState` exactly. That pairing is the contract's own contract; a mismatch fails verification with a message that points at neither side.\n\n## Self-Check Before Emitting a Pact File\n\nAny `yes` is a blocker.\n\n1. Does an interaction call `.given()` with hand-cast params instead of `createProviderState`?\n2. Does a verification file build `VerifierOptions` by hand?\n3. Does a request filter assemble an `Authorization` header itself?\n4. Does an `it()` block contain more than one `addInteraction`?\n5. Does the consumer Vitest config omit `fileParallelism: false`, `pool: 'forks'`, or `singleFork: true`?\n6. Does `executeTest` call raw `fetch` while the consumer client is importable, with no stated reason?\n7. Is any response matcher derived from consumer-side types rather than provider source, OpenAPI, or broker data?\n8. Is any interaction missing its `// Provider endpoint:` comment?\n9. Does provider verification hand-build an explicit consumer branch selector,\n or set `consumerBranch` without a scoped `consumer`?\n10. Does a breaking-change catch hand-roll tolerant branch classification\n instead of `isBreakingChangeTolerantBranch`, or check breaking-change\n tolerance before rejecting a \"no pacts found\" result that occurred with\n an explicit `PACT_CONSUMER_BRANCH` set (letting a typo masquerade as a\n tolerated breaking change)?\n\nFix, or record a deviation. Do not emit unresolved.\n\n## Broker Interaction\n\nWhen `tea_pact_mcp` is `\"mcp\"` and the SmartBear MCP tools are reachable, use them for what they are authoritative about: existing provider states, the verification matrix, and `can-i-deploy`. Prefer real broker data over a guess at what states exist.\n\nWhen the tools are not reachable — no broker configured, no credentials, a headless run without the server — degrade per `pact-mcp.md`: fall back to provider source or an OpenAPI spec, say in the output that the broker was unreachable, and continue. Never block the workflow on it, and never present inferred state names as if they came from the broker.\n\n## Review Behavior\n\nUnder `test-review`, with the flag `true` and the package installed, each Banned Pattern above is a **maintainability** finding on the file where it appears (registry row `M10`), with the substitution named in the recommendation. The determinism and FFI rules keep their own existing rows (`H6`, `H7`, `H8`, `L4`) and outrank this one: a suite that flakes matters more than a suite that is verbose.\n\n## Related Fragments\n\n- `library-integration-mandate.md` — the general contract this instantiates\n- `pactjs-utils-overview.md` — installation, the full utility table, flow decision tree\n- `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`, `pactjs-utils-zod-to-pact.md`\n- `pact-consumer-framework-setup.md` — directory structure, Vitest configs, scripts, CI workflow\n- `pact-consumer-di.md` — injecting the mock server URL into the real client\n- `contract-testing.md` — provider scrutiny, publishing, determinism\n- `pact-mcp.md` — broker tools and their degradation path\n- `confidence-gate.md` — stop and ask rather than invent a provider state or a response shape\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":16891,"content_sha256":"734eeb427b054949372b4454a007e95f0c5e14e2602f3ce95199c8657c64b27c"},{"filename":"knowledge/pactjs-utils-overview.md","content":"# Pact.js Utils Overview\n\n## Principle\n\nUse production-ready utilities from `@seontechnologies/pactjs-utils` to eliminate boilerplate in consumer-driven contract testing. The library wraps `@pact-foundation/pact` with type-safe helpers for provider state creation, PactV4 JSON interaction builders, verifier configuration, and request filter injection — working equally well for HTTP and message (async/Kafka) contracts.\n\n## Rationale\n\n### Problems with raw @pact-foundation/pact\n\n- **JsonMap casting**: Provider state parameters require `JsonMap` type — manually casting every value is error-prone and verbose\n- **Repeated builder lambdas**: PactV4 interactions often repeat inline callbacks with `builder.query(...)`, `builder.headers(...)`, and `builder.jsonBody(...)`\n- **Verifier configuration sprawl**: `VerifierOptions` requires 30+ lines of scattered configuration (broker URL, selectors, state handlers, request filters, version tags)\n- **Environment variable juggling**: Different env vars for local vs remote flows, breaking change coordination, payload URL matching\n- **Mismatched short-lived branches**: `matchingBranch` cannot coordinate a consumer PR and provider release branch with different names\n- **Express middleware types**: Request filter requires Express types that aren't re-exported from Pact\n- **Bearer prefix bugs**: Easy to double-prefix tokens as `Bearer Bearer ...` in request filters\n- **CI version tagging**: Manual logic to extract branch/tag info from CI environment\n\n### Solutions from pactjs-utils\n\n- **`createProviderState`**: One-call tuple builder for `.given()` — handles all JsonMap conversion automatically\n- **`toJsonMap`**: Explicit type coercion (null→\"null\", Date→ISO string, nested objects flattened)\n- **`setJsonContent`**: Curried callback helper for PactV4 `.withRequest(...)` / `.willRespondWith(...)` builders (query/headers/body)\n- **`setJsonBody`**: Body-only shorthand alias of `setJsonContent({ body })`\n- **`buildVerifierOptions`**: Single function assembles complete VerifierOptions from minimal inputs — handles local/remote/BDCT flows\n- **`buildMessageVerifierOptions`**: Same as above but for message/Kafka provider verification\n- **`handlePactBrokerUrlAndSelectors`**: Resolves broker URL and consumer version selectors from env vars with breaking change awareness\n- **`getProviderVersionTags`**: CI-aware version tagging (extracts branch/tag from GitHub Actions, GitLab CI, etc.)\n- **`isBreakingChangeTolerantBranch`**: Recognizes `main`, `master`, and\n `release/**` when a repository explicitly tolerates coordinated\n breaking-change verification failures\n- **Named branch coordination**: `consumerBranch` / `PACT_CONSUMER_BRANCH` lets\n provider verification add one scoped consumer branch; the CI templates pair\n `PACT_PROVIDER_BRANCH` with an additive branch-aware `can-i-deploy` check\n- **`createRequestFilter`**: Pluggable token generator pattern — prevents double-Bearer bugs by contract\n- **`noOpRequestFilter`**: Pass-through for providers that don't require auth injection\n- **`zodToPactMatchers`**: Converts a Zod schema (+ optional example values or `.openapi({ example })` metadata) into Pact V3 matchers — single source of truth for response shape, no hand-written matcher helpers\n\n## Installation\n\n```bash\nnpm install -D @seontechnologies/pactjs-utils\n\n# Peer dependency\nnpm install -D @pact-foundation/pact\n```\n\n**Requirements**: `@pact-foundation/pact` >= 16.2.0, Node.js >= 18\n\n## Available Utilities\n\n| Category | Function | Description | Use Case |\n| ----------------- | --------------------------------- | ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |\n| Consumer Helpers | `createProviderState` | Builds `[stateName, JsonMap]` tuple from typed input | Consumer tests: `.given(...createProviderState(input))` |\n| Consumer Helpers | `toJsonMap` | Converts any object to Pact-compatible `JsonMap` | Explicit type coercion for provider state params |\n| Consumer Helpers | `setJsonContent` | Curried request/response JSON callback helper | PactV4 `.withRequest(...)` and `.willRespondWith(...)` builders |\n| Consumer Helpers | `setJsonBody` | Body-only alias of `setJsonContent` | Body-only `.willRespondWith(...)` responses |\n| Provider Verifier | `buildVerifierOptions` | Assembles complete HTTP `VerifierOptions` | Provider verification, including a scoped `consumerBranch` selector |\n| Provider Verifier | `buildMessageVerifierOptions` | Assembles message `VerifierOptions` | Kafka/async provider verification, including a scoped `consumerBranch` selector |\n| Provider Verifier | `handlePactBrokerUrlAndSelectors` | Resolves broker URL + selectors from env vars | Env-aware broker configuration |\n| Provider Verifier | `getProviderVersionTags` | CI-aware version tag extraction | Provider version tagging in CI |\n| Provider Verifier | `isBreakingChangeTolerantBranch` | Classifies main/master/release branches | Guarding an explicit `PACT_BREAKING_CHANGE` tolerance policy |\n| Request Filter | `createRequestFilter` | Express middleware with pluggable token generator | Auth injection for provider verification |\n| Request Filter | `noOpRequestFilter` | Pass-through filter (no-op) | Providers without auth requirements |\n| Schema → Matchers | `zodToPactMatchers` | Derives Pact V3 matchers from a Zod schema | Consumer tests: response body matchers from a consumer-curated Zod schema instead of hand-written helpers |\n\n## Decision Tree: Which Flow?\n\n```text\nIs this a monorepo (consumer + provider in same repo)?\n├── YES → Local Flow\n│ - Consumer generates pact files to ./pacts/\n│ - Provider reads pact files from ./pacts/ (no broker needed)\n│ - Use buildVerifierOptions with pactUrls option\n│\n└── NO → Do you have a Pact Broker / PactFlow?\n ├── YES → Remote (CDCT) Flow\n │ - Consumer publishes pacts to broker\n │ - Provider verifies from broker\n │ - Use buildVerifierOptions with broker config\n │ - Set PACT_BROKER_BASE_URL + PACT_BROKER_TOKEN\n │\n └── Do you have an OpenAPI spec?\n ├── YES → BDCT Flow (PactFlow only)\n │ - Provider publishes OpenAPI spec to PactFlow\n │ - PactFlow cross-validates consumer pacts against spec\n │ - No provider verification test needed\n │\n └── NO → Start with Local Flow, migrate to Remote later\n```\n\n## Design Philosophy\n\n1. **One-call setup**: Each utility does one thing completely — no multi-step assembly required\n2. **Environment-aware**: Utilities read env vars for CI/CD integration without manual wiring\n3. **Type-safe**: Full TypeScript types for all inputs and outputs, exported for consumer use\n4. **Fail-safe defaults**: Sensible defaults that work locally; env vars override for CI\n5. **Composable**: Utilities work independently — use only what you need\n\n## Pattern Examples\n\n### Example 1: Minimal Consumer Test\n\n```typescript\nimport { PactV3 } from '@pact-foundation/pact';\nimport { createProviderState } from '@seontechnologies/pactjs-utils';\n\nconst provider = new PactV3({\n consumer: 'my-frontend',\n provider: 'my-api',\n dir: './pacts',\n});\n\nit('should get user by id', async () => {\n await provider\n .given(...createProviderState({ name: 'user exists', params: { id: 1 } }))\n .uponReceiving('a request for user 1')\n .withRequest({ method: 'GET', path: '/users/1' })\n .willRespondWith({ status: 200, body: { id: 1, name: 'John' } })\n .executeTest(async (mockServer) => {\n const res = await fetch(`${mockServer.url}/users/1`);\n expect(res.status).toBe(200);\n });\n});\n```\n\n### Example 2: Minimal Provider Verification\n\n```typescript\nimport { Verifier } from '@pact-foundation/pact';\nimport { buildVerifierOptions, createRequestFilter } from '@seontechnologies/pactjs-utils';\n\nconst opts = buildVerifierOptions({\n provider: 'my-api',\n port: '3001',\n includeMainAndDeployed: true,\n stateHandlers: {\n 'user exists': async (params) => {\n await db.seed({ users: [{ id: params?.id }] });\n },\n },\n requestFilter: createRequestFilter({\n tokenGenerator: () => 'test-token-123',\n }),\n});\n\nawait new Verifier(opts).verifyProvider();\n```\n\n## Key Points\n\n- **Import path**: Always use `@seontechnologies/pactjs-utils` (no subpath exports)\n- **Peer dependency**: `@pact-foundation/pact` must be installed separately\n- **Local flow**: No broker needed — set `pactUrls` in verifier options pointing to local pact files\n- **Remote flow**: Set `PACT_BROKER_BASE_URL` and `PACT_BROKER_TOKEN` env vars\n- **Breaking changes**: Set `includeMainAndDeployed: false` to omit main and\n deployed selectors. `matchingBranch` remains, along with a configured\n `consumerBranch`.\n- **Different branch names**: Set `consumer` plus `consumerBranch` (or\n `PACT_CONSUMER_BRANCH`) on the provider side. On a consumer PR, use the\n template's PR-only `PACT_PROVIDER_BRANCH` flow so `can-i-deploy` preserves\n the environment gate and checks the named provider branch separately.\n- **Builder helpers**: Use `setJsonContent` when you need query/headers/body together; use `setJsonBody` for body-only callbacks\n- **Type exports**: Library exports `StateHandlers`, `RequestFilter`, `JsonMap`, `JsonContentInput`, `ConsumerVersionSelector` types\n\n## Related Fragments\n\n- `pactjs-utils-consumer-helpers.md` — detailed createProviderState, toJsonMap, setJsonContent, and setJsonBody usage\n- `pactjs-utils-provider-verifier.md` — detailed buildVerifierOptions and broker configuration\n- `pactjs-utils-request-filter.md` — detailed createRequestFilter and auth patterns\n- `pactjs-utils-zod-to-pact.md` — detailed zodToPactMatchers usage, consumer-curated schema pattern, and anti-patterns\n- `contract-testing.md` — foundational contract testing patterns (raw Pact.js approach)\n- `test-levels-framework.md` — where contract tests fit in the testing pyramid\n\n## Anti-Patterns\n\n### Wrong: Manual VerifierOptions assembly when pactjs-utils is available\n\n```typescript\n// ❌ Don't assemble VerifierOptions manually\nconst opts: VerifierOptions = {\n provider: 'my-api',\n providerBaseUrl: 'http://localhost:3001',\n pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,\n pactBrokerToken: process.env.PACT_BROKER_TOKEN,\n publishVerificationResult: process.env.CI === 'true',\n providerVersion: process.env.GIT_SHA || 'dev',\n consumerVersionSelectors: [{ mainBranch: true }, { deployedOrReleased: true }],\n stateHandlers: {\n /* ... */\n },\n requestFilter: (req, res, next) => {\n /* ... */\n },\n // ... 20 more lines\n};\n```\n\n### Right: Use buildVerifierOptions\n\n```typescript\n// ✅ Single call handles all configuration\nconst opts = buildVerifierOptions({\n provider: 'my-api',\n port: '3001',\n includeMainAndDeployed: true,\n stateHandlers: {\n /* ... */\n },\n requestFilter: createRequestFilter({ tokenGenerator: () => 'token' }),\n});\n```\n\n### Wrong: Importing raw Pact types for JsonMap conversion\n\n```typescript\n// ❌ Manual JsonMap casting\nimport type { JsonMap } from '@pact-foundation/pact';\n\nprovider.given('user exists', { id: 1 as unknown as JsonMap['id'] });\n```\n\n### Right: Use createProviderState\n\n```typescript\n// ✅ Automatic type conversion\nimport { createProviderState } from '@seontechnologies/pactjs-utils';\n\nprovider.given(...createProviderState({ name: 'user exists', params: { id: 1 } }));\n```\n\n_Source: @seontechnologies/pactjs-utils library, pactjs-utils README, pact-js-example-provider workflows._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12644,"content_sha256":"7403189c1388e3a4186b90d998b6042561ab73f80b5130cb4c787ca07b5d2ef9"},{"filename":"knowledge/pactjs-utils-provider-verifier.md","content":"# Pact.js Utils Provider Verifier\n\n## Principle\n\nUse `buildVerifierOptions`, `buildMessageVerifierOptions`, `handlePactBrokerUrlAndSelectors`, `getProviderVersionTags`, and `isBreakingChangeTolerantBranch` from `@seontechnologies/pactjs-utils` to assemble provider verification and classify coordinated breaking-change branches. These utilities handle local/remote flow detection, broker URL resolution, consumer version selector strategy, named consumer branches, and CI-aware version tagging. The caller controls breaking change behavior via the required `includeMainAndDeployed` parameter.\n\n## Rationale\n\n### Problems with manual VerifierOptions\n\n- **30+ lines of scattered config**: Assembling `VerifierOptions` manually requires broker URL, token, selectors, state handlers, request filters, version info, publish flags — all in one object\n- **Environment variable logic**: Different env vars for local vs remote, CI vs local dev, breaking change vs normal flow\n- **Consumer version selector complexity**: Choosing between `mainBranch`, `deployedOrReleased`, `matchingBranch`, and `includeMainAndDeployed` requires understanding Pact Broker semantics\n- **Breaking change coordination**: When a provider intentionally breaks a contract, manual selector switching is error-prone\n- **Short-lived branch mismatch**: `matchingBranch` cannot find a consumer pact when the provider and consumer branch names differ\n- **Cross-execution protection**: `PACT_PAYLOAD_URL` webhook payloads need special handling to verify only the triggering pact\n\n### Solutions\n\n- **`buildVerifierOptions`**: Single function that reads env vars, selects the right flow, and returns complete `VerifierOptions`\n- **`buildMessageVerifierOptions`**: Same as above for message/Kafka provider verification\n- **`handlePactBrokerUrlAndSelectors`**: Pure function for broker URL + selector resolution (used internally, also exported for advanced use)\n- **`getProviderVersionTags`**: Extracts CI branch/tag info from environment for provider version tagging\n- **`consumerBranch`**: Adds one explicitly named consumer branch without removing matching, main, or deployed selectors\n- **`isBreakingChangeTolerantBranch`**: Classifies only `main`, `master`, and\n `release/**` for an explicitly enabled breaking-change tolerance policy\n\n## Pattern Examples\n\n### Example 1: HTTP Provider Verification (Remote Flow)\n\n```typescript\nimport { Verifier } from '@pact-foundation/pact';\nimport { buildVerifierOptions, createRequestFilter } from '@seontechnologies/pactjs-utils';\nimport type { StateHandlers } from '@seontechnologies/pactjs-utils';\n\nconst stateHandlers: StateHandlers = {\n 'movie with id 1 exists': {\n setup: async (params) => {\n await db.seed({ movies: [{ id: params?.id ?? 1, name: 'Inception' }] });\n },\n teardown: async () => {\n await db.clean('movies');\n },\n },\n 'no movies exist': async () => {\n await db.clean('movies');\n },\n};\n\n// buildVerifierOptions reads these env vars automatically:\n// - PACT_BROKER_BASE_URL (broker URL)\n// - PACT_BROKER_TOKEN (broker auth)\n// - PACT_PAYLOAD_URL (webhook trigger — cross-execution protection)\n// - PACT_CONSUMER_BRANCH (optional named consumer branch for mismatched PR branches)\n// - PACT_PROVIDER_VERSION / PACT_PROVIDER_BRANCH (webhook-selected provider revision)\n// - PACT_BREAKING_CHANGE (if \"true\", uses includeMainAndDeployed selectors)\n// - GITHUB_SHA (provider version)\n// - CI (publish verification results if \"true\")\n\nconst opts = buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: '3001',\n includeMainAndDeployed: process.env.PACT_BREAKING_CHANGE !== 'true',\n stateHandlers,\n requestFilter: createRequestFilter({\n tokenGenerator: () => process.env.TEST_AUTH_TOKEN ?? 'test-token',\n }),\n});\n\nawait new Verifier(opts).verifyProvider();\n```\n\n**Key Points**:\n\n- Set `PACT_BROKER_BASE_URL` and `PACT_BROKER_TOKEN` as env vars — `buildVerifierOptions` reads them automatically\n- `port` is a string (e.g., `'3001'`) — the function builds `providerBaseUrl: http://localhost:${port}` internally\n- `includeMainAndDeployed` is **required** — set `true` for normal flow, `false` for breaking changes\n- State handlers support both simple functions and `{ setup, teardown }` objects\n- `params` in state handlers correspond to the `JsonMap` from consumer's `createProviderState`\n- Verification results are published by default (`publishVerificationResult` defaults to `true`)\n\n### Example 2: Verify a Named Consumer Branch\n\nWhen the provider and consumer PR branch names differ, set both `consumer` and\n`consumerBranch`. The builder adds the explicit branch selector to the normal\nselector set; it does not replace the safety selectors.\n\n```typescript\nconst opts = buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: '3001',\n includeMainAndDeployed: true,\n consumer: 'SampleAppConsumer',\n consumerBranch: process.env.PACT_CONSUMER_BRANCH,\n stateHandlers,\n});\n```\n\nWith `PACT_CONSUMER_BRANCH=feature/new-movie-client`, the selectors are:\n\n```typescript\n[\n { consumer: 'SampleAppConsumer', matchingBranch: true },\n { consumer: 'SampleAppConsumer', branch: 'feature/new-movie-client' },\n { consumer: 'SampleAppConsumer', mainBranch: true },\n { consumer: 'SampleAppConsumer', deployedOrReleased: true },\n];\n```\n\n`consumerBranch` requires `consumer`. The utility throws when an explicit branch\nis unscoped because `{ branch: name }` could select that branch from every\nconsumer of the provider. `buildMessageVerifierOptions` has the same parameter,\ndefault, and guard.\n\n### Example 3: Local Flow (Monorepo, No Broker)\n\n```typescript\nimport { Verifier } from '@pact-foundation/pact';\nimport { buildVerifierOptions } from '@seontechnologies/pactjs-utils';\n\n// When PACT_BROKER_BASE_URL is NOT set, buildVerifierOptions\n// falls back to local pact file verification\nconst opts = buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: '3001',\n includeMainAndDeployed: true,\n // Specify local pact files directly — skips broker entirely\n pactUrls: ['./pacts/movie-web-SampleMoviesAPI.json'],\n stateHandlers: {\n 'movie exists': async (params) => {\n await db.seed({ movies: [{ id: params?.id }] });\n },\n },\n});\n\nawait new Verifier(opts).verifyProvider();\n```\n\n### Example 4: Message Provider Verification (Kafka/Async)\n\n```typescript\nimport { Verifier } from '@pact-foundation/pact';\nimport { buildMessageVerifierOptions } from '@seontechnologies/pactjs-utils';\n\nconst opts = buildMessageVerifierOptions({\n provider: 'OrderEventsProducer',\n includeMainAndDeployed: process.env.PACT_BREAKING_CHANGE !== 'true',\n // Message handlers return the message content that the provider would produce\n messageProviders: {\n 'an order created event': async () => ({\n orderId: 'order-123',\n userId: 'user-456',\n items: [{ productId: 'prod-789', quantity: 2 }],\n createdAt: new Date().toISOString(),\n }),\n 'an order cancelled event': async () => ({\n orderId: 'order-123',\n reason: 'customer_request',\n cancelledAt: new Date().toISOString(),\n }),\n },\n stateHandlers: {\n 'order exists': async (params) => {\n await db.seed({ orders: [{ id: params?.orderId }] });\n },\n },\n});\n\nawait new Verifier(opts).verifyProvider();\n```\n\n**Key Points**:\n\n- `buildMessageVerifierOptions` adds `messageProviders` to the verifier config\n- Each message provider function returns the expected message payload\n- State handlers work the same as HTTP verification\n- Broker integration works identically (same env vars)\n\n### Example 5: Breaking Change Coordination\n\n```typescript\n// When a provider intentionally introduces a breaking change:\n//\n// 1. Set PACT_BREAKING_CHANGE=true in CI environment\n// 2. Your test reads the env var and passes includeMainAndDeployed: false\n// to buildVerifierOptions — this verifies ONLY against the matching\n// branch, skipping main/deployed consumers that would fail\n// 3. Coordinate with consumer team to update their pact on a matching branch\n// 4. Remove PACT_BREAKING_CHANGE flag after consumer updates\n\n// In CI environment (.github/workflows/provider-verify.yml):\n// env:\n// PACT_BREAKING_CHANGE: 'true'\n\n// Your provider test code reads the env var:\nconst isBreakingChange = process.env.PACT_BREAKING_CHANGE === 'true';\n\nconst opts = buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: '3001',\n includeMainAndDeployed: !isBreakingChange, // false during breaking changes\n stateHandlers: {\n /* ... */\n },\n});\n// When includeMainAndDeployed is false and consumerBranch is unset:\n// selectors = [{ matchingBranch: true }]\n// When includeMainAndDeployed is true (normal):\n// selectors = [{ matchingBranch: true }, { mainBranch: true }, { deployedOrReleased: true }]\n```\n\n### Example 6: handlePactBrokerUrlAndSelectors (Advanced)\n\n```typescript\nimport { handlePactBrokerUrlAndSelectors } from '@seontechnologies/pactjs-utils';\nimport type { VerifierOptions } from '@pact-foundation/pact';\n\n// For advanced use cases — mutates the options object in-place (returns void)\nconst options: VerifierOptions = {\n provider: 'SampleMoviesAPI',\n providerBaseUrl: 'http://localhost:3001',\n};\n\nhandlePactBrokerUrlAndSelectors({\n pactPayloadUrl: process.env.PACT_PAYLOAD_URL,\n pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,\n consumer: undefined, // or specific consumer name\n includeMainAndDeployed: true,\n consumerBranch: undefined, // requires consumer when set\n options, // mutated in-place: sets pactBrokerUrl, consumerVersionSelectors, or pactUrls\n});\n\n// After call, options has been mutated with:\n// - options.pactBrokerUrl (from pactBrokerUrl param)\n// - options.consumerVersionSelectors (based on includeMainAndDeployed)\n// OR if pactPayloadUrl matches: options.pactUrls = [pactPayloadUrl]\n```\n\n**Note**: `handlePactBrokerUrlAndSelectors` is called internally by `buildVerifierOptions`. You rarely need it directly — use it only for advanced custom verifier assembly.\n\n### Example 7: getProviderVersionTags\n\n```typescript\nimport { getProviderVersionTags } from '@seontechnologies/pactjs-utils';\n\n// Extracts version tags from CI environment\nconst tags = getProviderVersionTags();\n\n// In GitHub Actions on branch \"feature/add-movies\" (non-breaking):\n// tags = ['feature/add-movies']\n//\n// In GitHub Actions on main branch (non-breaking):\n// tags = ['dev', 'main']\n//\n// In GitHub Actions with PACT_BREAKING_CHANGE=true:\n// tags = ['feature/add-movies'] (no 'dev' tag)\n//\n// Locally (no CI):\n// tags = ['local']\n```\n\nOnly `main` and `master` receive the legacy `dev` tag. Feature and `release/**`\nbranches receive only their branch tag, so a PR verification cannot masquerade\nas the version deployed in `dev`.\n\n### Breaking-Change Tolerant Branch Classification\n\n`isBreakingChangeTolerantBranch(branch)` returns `true` for `main`, `master`,\nand names starting with `release/`. It returns `false` for feature branches and\nfor lookalikes such as `releases/week-32`.\n\nUse it only when repository policy deliberately tolerates a provider\nverification failure while `PACT_BREAKING_CHANGE=true`:\n\n```typescript\nimport { isBreakingChangeTolerantBranch } from '@seontechnologies/pactjs-utils';\n\ntry {\n await verifier.verifyProvider();\n} catch (error) {\n const message = error instanceof Error ? error.message.toLowerCase() : String(error).toLowerCase();\n const noPactsFound = message.includes('no pacts found') || message.includes('no pacts were found');\n\n // A hand-typed consumer branch that selects nothing is always a failed check.\n if (noPactsFound && process.env.PACT_CONSUMER_BRANCH) throw error;\n\n const tolerated = process.env.PACT_BREAKING_CHANGE === 'true' && isBreakingChangeTolerantBranch(process.env.GITHUB_BRANCH ?? '');\n\n if (!tolerated) throw error;\n}\n```\n\nThe explicit-consumer-branch guard must run first. Otherwise a typo in\n`PACT_CONSUMER_BRANCH` becomes a false green whenever breaking-change tolerance\nis active. This tolerance is a visible coordination policy, not a default:\nwithout `PACT_BREAKING_CHANGE=true`, every verification failure still fails the\nbuild.\n\n### Example 8: Provider Vitest Configuration (Required for Multi-File Verification)\n\n**Context**: The Pact Rust FFI that powers the JS `Verifier` holds process-wide state (native handles for messages, matchers, mocks). Vitest's default parallel file workers each spin up their own FFI instance and quickly corrupt that state — causing `MessagePact`/`Verifier` errors like `\"Unable to get the MessageHandle\"`, or non-deterministic verification passes/fails — as soon as you have more than one provider `.spec.ts` file.\n\n**Rule**: Provider verification suites **must** run in a single fork. Use Vitest's `forks` pool with `singleFork: true` in `vitest.config.contract.ts` (or equivalent).\n\n```typescript\n// vitest.config.contract.ts — provider verification config\nimport { defineConfig } from 'vitest/config';\n\nexport default defineConfig({\n test: {\n environment: 'node',\n include: ['tests/contract/**/*.spec.ts'],\n testTimeout: 60000,\n // MANDATORY for multi-file provider verification.\n // The Pact Rust FFI backing the Verifier holds process-wide state; parallel workers corrupt it\n // and produce flaky verification results / \"Unable to get the MessageHandle\" errors.\n // This is especially important for message providers (Kafka/async) where verifier construction\n // allocates native handles per file — singleFork keeps them in one process so state is coherent.\n pool: 'forks',\n poolOptions: {\n forks: {\n singleFork: true,\n },\n },\n },\n});\n```\n\n**Key Points**:\n\n- **Required for message providers** (`buildMessageVerifierOptions`) — the message-handle FFI state is almost guaranteed to corrupt under parallel workers.\n- **Required for HTTP providers with multiple contract test files** — even if each file works in isolation, running them together in parallel produces intermittent failures.\n- `pool: 'forks'` (rather than `threads`) + `singleFork: true` is the exact combo that keeps all verifier runs in a single child process with a single FFI instance.\n- Treat `pool: 'forks'` + `singleFork: true` as the required baseline for all provider suites, including single-file HTTP-only ones. A suite that works today with one file will flake the moment a second file is added, and removing the setting later introduces a regression window.\n- **The same `pool: 'forks'` + `singleFork: true` rule applies on the consumer side.** Consumer `vitest.config.pact.ts` sets it alongside `fileParallelism: false` — see `pact-consumer-framework-setup.md` Example 2. The rule is needed on either side wherever more than one pact test file exists per consumer+provider pair.\n- Use a dedicated `vitest.config.contract.ts` so unit tests still get full parallelism — only contract tests pay the serialization cost.\n- Related `package.json` entry:\n\n ```json\n {\n \"scripts\": {\n \"test:pact:provider\": \"vitest run --config vitest.config.contract.ts\"\n }\n }\n ```\n\n## Environment Variables Reference\n\n| Variable | Required | Description | Default |\n| ----------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ----------- |\n| `PACT_BROKER_BASE_URL` | For remote flow | Pact Broker / PactFlow URL | — |\n| `PACT_BROKER_TOKEN` | For remote flow | API token for broker authentication | — |\n| `PACT_PROVIDER_VERSION` | Webhook flow | Exact provider revision selected by webhook checkout; takes precedence over `GITHUB_SHA` | — |\n| `PACT_PROVIDER_BRANCH` | Branch override | Provider branch selected by webhook checkout; takes precedence over `GITHUB_BRANCH` | — |\n| `PACT_CONSUMER_BRANCH` | Optional | Named consumer branch to add to selectors; requires a specific `consumer` | — |\n| `GITHUB_SHA` | Recommended | Provider version for verification result publishing (auto-set by GitHub Actions) | `'unknown'` |\n| `GITHUB_BRANCH` | Recommended | Branch name for provider version branch and version tags (**not auto-set** — define as `${{ github.head_ref \\|\\| github.ref_name }}`) | `'main'` |\n| `PACT_PAYLOAD_URL` | Optional | Webhook payload URL — triggers verification of specific pact only | — |\n| `PACT_BREAKING_CHANGE` | Optional | Set to `\"true\"` to use breaking change selector strategy | `'false'` |\n| `CI` | Auto-detected | When `\"true\"`, enables verification result publishing | — |\n\n## Key Points\n\n- **Flow auto-detection**: If `PACT_BROKER_BASE_URL` is set → remote flow; otherwise → local flow (requires `pactUrls`)\n- **`port` is a string**: Pass port number as string (e.g., `'3001'`); function builds `http://localhost:${port}` internally\n- **`includeMainAndDeployed` is required**: `true` includes mainBranch + deployedOrReleased; `false` removes those two for breaking changes. `matchingBranch` always remains, and a configured `consumerBranch` remains additive in either mode.\n- **Selector strategy**: Normal flow (`includeMainAndDeployed: true`) includes all selectors; breaking change flow (`false`) includes only `matchingBranch`\n- **Named consumer branch**: `consumerBranch` adds `{ consumer, branch }`; it requires `consumer` and remains additive to the other selectors\n- **Webhook support**: `PACT_PAYLOAD_URL` takes precedence — verifies only the specific pact that triggered the webhook\n- **State handler types**: Both `async (params) => void` and `{ setup: async (params) => void, teardown: async () => void }` are supported\n- **Version publishing**: Verification results are published by default (`publishVerificationResult` defaults to `true`)\n- **Provider Vitest config is MANDATORY for multi-file suites**: Set `pool: 'forks'` + `poolOptions.forks.singleFork: true` in `vitest.config.contract.ts`. Without this the Rust FFI corrupts under parallel workers (see Example 8).\n\n## Related Fragments\n\n- `pactjs-utils-overview.md` — installation, decision tree, design philosophy\n- `pactjs-utils-consumer-helpers.md` — consumer-side state parameter creation, **one-interaction-per-`it()` rule**\n- `pactjs-utils-request-filter.md` — auth injection for provider verification\n- `pact-consumer-framework-setup.md` — consumer-side framework setup, Vitest `fileParallelism: false`, CI wiring\n- `pact-broker-webhooks.md` — PactFlow → GitHub webhook auth/staleness for webhook-triggered provider verification (`contract_requiring_verification_published`)\n- `contract-testing.md` — foundational patterns with raw Pact.js\n\n## Anti-Patterns\n\n### Wrong: Manual broker URL and selector assembly\n\n```typescript\n// ❌ Manual environment variable handling\nconst opts: VerifierOptions = {\n provider: 'my-api',\n providerBaseUrl: 'http://localhost:3001',\n pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,\n pactBrokerToken: process.env.PACT_BROKER_TOKEN,\n publishVerificationResult: process.env.CI === 'true',\n providerVersion: process.env.GIT_SHA || process.env.GITHUB_SHA || 'dev',\n providerVersionBranch: process.env.GITHUB_HEAD_REF || process.env.GITHUB_REF_NAME,\n consumerVersionSelectors:\n process.env.PACT_BREAKING_CHANGE === 'true'\n ? [{ matchingBranch: true }]\n : [{ matchingBranch: true }, { mainBranch: true }, { deployedOrReleased: true }],\n pactUrls: process.env.PACT_PAYLOAD_URL ? [process.env.PACT_PAYLOAD_URL] : undefined,\n stateHandlers: {\n /* ... */\n },\n requestFilter: (req, res, next) => {\n req.headers['authorization'] = `Bearer ${process.env.TEST_TOKEN}`;\n next();\n },\n};\n```\n\n### Right: Use buildVerifierOptions\n\n```typescript\n// ✅ All env var logic handled internally\nconst opts = buildVerifierOptions({\n provider: 'my-api',\n port: '3001',\n includeMainAndDeployed: process.env.PACT_BREAKING_CHANGE !== 'true',\n stateHandlers: {\n /* ... */\n },\n requestFilter: createRequestFilter({\n tokenGenerator: () => process.env.TEST_TOKEN ?? 'test-token',\n }),\n});\n```\n\n### Wrong: Hardcoding consumer version selectors\n\n```typescript\n// ❌ Hardcoded selectors — breaks when flow changes\nconsumerVersionSelectors: [{ mainBranch: true }, { deployedOrReleased: true }],\n```\n\n### Right: Let buildVerifierOptions choose selectors\n\n```typescript\n// ✅ Selector strategy adapts to PACT_BREAKING_CHANGE env var\nconst opts = buildVerifierOptions({\n /* ... */\n});\n// Selectors chosen automatically based on environment\n```\n\n### Wrong: Unscoped Explicit Consumer Branch\n\n```typescript\n// ❌ A branch name can exist on several consumers\nhandlePactBrokerUrlAndSelectors({\n consumerBranch: 'release/week-32',\n consumer: undefined,\n /* ... */\n});\n```\n\n### Right: Pair Consumer and Branch\n\n```typescript\n// ✅ The explicit branch applies to one pacticipant\nbuildVerifierOptions({\n provider: 'my-api',\n port: '3001',\n includeMainAndDeployed: true,\n consumer: 'my-web',\n consumerBranch: process.env.PACT_CONSUMER_BRANCH,\n});\n```\n\n### Wrong: Parallel Vitest workers for provider verification\n\n```typescript\n// ❌ vitest.config.contract.ts — uses default parallel workers\nimport { defineConfig } from 'vitest/config';\nexport default defineConfig({\n test: {\n environment: 'node',\n include: ['tests/contract/**/*.spec.ts'],\n // NO pool/singleFork config — defaults to parallel file workers\n },\n});\n// Symptoms: \"Unable to get the MessageHandle\", non-deterministic verification pass/fail,\n// green locally on single-file run but red in CI with multiple files\n```\n\n### Right: Single fork for provider verification\n\n```typescript\n// ✅ vitest.config.contract.ts — serializes provider verification files\nimport { defineConfig } from 'vitest/config';\nexport default defineConfig({\n test: {\n environment: 'node',\n include: ['tests/contract/**/*.spec.ts'],\n pool: 'forks',\n poolOptions: { forks: { singleFork: true } },\n },\n});\n```\n\n_Source: @seontechnologies/pactjs-utils provider-verifier module, pact-js-example-provider CI workflows._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":22882,"content_sha256":"1c882c7ce145d7e9d12e6c39306947f7ab9897ad27633b638975502ea51431fd"},{"filename":"knowledge/pactjs-utils-request-filter.md","content":"# Pact.js Utils Request Filter\n\n## Principle\n\nUse `createRequestFilter` and `noOpRequestFilter` from `@seontechnologies/pactjs-utils` to inject authentication headers during provider verification. The pluggable token generator pattern prevents double-Bearer bugs and separates auth concerns from verification logic.\n\n## Rationale\n\n### Problems with manual request filters\n\n- **Express type gymnastics**: Pact's `requestFilter` expects `(req, res, next) => void` with Express-compatible types — but Pact doesn't re-export these types\n- **Double-Bearer bug**: Easy to write `Authorization: Bearer Bearer ${token}` when the token generator already includes the prefix\n- **Inline complexity**: Auth logic mixed with verifier config makes tests harder to read\n- **No-op boilerplate**: Providers without auth still need a pass-through function or `undefined`\n\n### Solutions\n\n- **`createRequestFilter`**: Accepts `{ tokenGenerator: () => string }` — generator returns raw token value synchronously, filter adds `\"Bearer \"` prefix\n- **`noOpRequestFilter`**: Pre-built pass-through for providers without auth requirements\n- **Bearer prefix contract**: `tokenGenerator` returns raw value (e.g., `\"abc123\"`), filter always adds `\"Bearer \"` — impossible to double-prefix\n\n## Pattern Examples\n\n### Example 1: Basic Auth Injection\n\n```typescript\nimport { buildVerifierOptions, createRequestFilter } from '@seontechnologies/pactjs-utils';\n\nconst opts = buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: '3001',\n includeMainAndDeployed: true,\n stateHandlers: {\n /* ... */\n },\n requestFilter: createRequestFilter({\n // tokenGenerator returns raw token — filter adds \"Bearer \" prefix\n tokenGenerator: () => 'test-auth-token-123',\n }),\n});\n\n// Every request during verification will have:\n// Authorization: Bearer test-auth-token-123\n```\n\n**Key Points**:\n\n- `tokenGenerator` is **synchronous** (`() => string`) — if you need async token fetching, resolve the token before creating the filter\n- Return the raw token value, NOT `\"Bearer ...\"` — the filter adds the prefix\n- Filter sets `Authorization` header on every request during verification\n\n### Example 2: Dynamic Token (Pre-resolved)\n\n```typescript\nimport { createRequestFilter } from '@seontechnologies/pactjs-utils';\n\n// Since tokenGenerator is synchronous, fetch the token before creating the filter\nlet cachedToken: string;\n\nasync function setupRequestFilter() {\n const response = await fetch('http://localhost:8080/auth/token', {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n clientId: process.env.TEST_CLIENT_ID,\n clientSecret: process.env.TEST_CLIENT_SECRET,\n }),\n });\n const { access_token } = await response.json();\n cachedToken = access_token;\n}\n\nconst requestFilter = createRequestFilter({\n tokenGenerator: () => cachedToken, // Synchronous — returns pre-fetched token\n});\n\nconst opts = buildVerifierOptions({\n provider: 'SecureAPI',\n port: '3001',\n includeMainAndDeployed: true,\n stateHandlers: {\n /* ... */\n },\n requestFilter,\n});\n```\n\n### Example 3: No-Auth Provider\n\n```typescript\nimport { buildVerifierOptions, noOpRequestFilter } from '@seontechnologies/pactjs-utils';\n\n// For providers that don't require authentication\nconst opts = buildVerifierOptions({\n provider: 'PublicAPI',\n port: '3001',\n includeMainAndDeployed: true,\n stateHandlers: {\n /* ... */\n },\n requestFilter: noOpRequestFilter,\n});\n\n// noOpRequestFilter is equivalent to: (req, res, next) => next()\n```\n\n### Example 4: Integration with buildVerifierOptions\n\n```typescript\nimport { buildVerifierOptions, createRequestFilter } from '@seontechnologies/pactjs-utils';\nimport type { StateHandlers } from '@seontechnologies/pactjs-utils';\n\n// Complete provider verification setup\nconst stateHandlers: StateHandlers = {\n 'user is authenticated': async () => {\n // Auth state is handled by the request filter, not state handler\n },\n 'movie exists': {\n setup: async (params) => {\n await db.seed({ movies: [{ id: params?.id }] });\n },\n teardown: async () => {\n await db.clean('movies');\n },\n },\n};\n\nconst requestFilter = createRequestFilter({\n tokenGenerator: () => process.env.TEST_AUTH_TOKEN ?? 'fallback-token',\n});\n\nconst opts = buildVerifierOptions({\n provider: 'SampleMoviesAPI',\n port: process.env.PORT ?? '3001',\n includeMainAndDeployed: process.env.PACT_BREAKING_CHANGE !== 'true',\n stateHandlers,\n requestFilter,\n});\n\n// Run verification\nawait new Verifier(opts).verifyProvider();\n```\n\n## Key Points\n\n- **Bearer prefix contract**: `tokenGenerator` returns raw value → filter adds `\"Bearer \"` → impossible to double-prefix\n- **Synchronous only**: `tokenGenerator` must return `string` (not `Promise\u003cstring>`) — pre-resolve async tokens before creating the filter\n- **Separation of concerns**: Auth logic in `createRequestFilter`, verification logic in `buildVerifierOptions`\n- **noOpRequestFilter**: Use for providers without auth — cleaner than `undefined` or inline no-op\n- **Express compatible**: The returned filter matches Pact's expected `(req, res, next) => void` signature\n\n## Related Fragments\n\n- `pactjs-utils-overview.md` — installation, utility table, decision tree\n- `pactjs-utils-provider-verifier.md` — buildVerifierOptions integration\n- `contract-testing.md` — foundational patterns with raw Pact.js\n\n## Anti-Patterns\n\n### Wrong: Manual Bearer prefix with double-prefix risk\n\n```typescript\n// ❌ Risk of double-prefix: \"Bearer Bearer token\"\nrequestFilter: (req, res, next) => {\n const token = getToken(); // What if getToken() returns \"Bearer abc123\"?\n req.headers['authorization'] = `Bearer ${token}`;\n next();\n};\n```\n\n### Right: Use createRequestFilter with raw token\n\n```typescript\n// ✅ tokenGenerator returns raw value — filter handles prefix\nrequestFilter: createRequestFilter({\n tokenGenerator: () => getToken(), // Returns \"abc123\", not \"Bearer abc123\"\n});\n```\n\n### Wrong: Inline auth logic in verifier config\n\n```typescript\n// ❌ Auth logic mixed with verifier config\nconst opts: VerifierOptions = {\n provider: 'my-api',\n providerBaseUrl: 'http://localhost:3001',\n requestFilter: (req, res, next) => {\n const clientId = process.env.CLIENT_ID;\n const clientSecret = process.env.CLIENT_SECRET;\n // 10 lines of token fetching logic...\n req.headers['authorization'] = `Bearer ${token}`;\n next();\n },\n // ... rest of config\n};\n```\n\n### Right: Separate auth into createRequestFilter\n\n```typescript\n// ✅ Clean separation — async setup wraps token fetch (CommonJS-safe)\nasync function setupVerifierOptions() {\n const token = await fetchAuthToken(); // Resolve async token BEFORE creating filter\n\n const requestFilter = createRequestFilter({\n tokenGenerator: () => token, // Synchronous — returns pre-fetched value\n });\n\n return buildVerifierOptions({\n provider: 'my-api',\n port: '3001',\n includeMainAndDeployed: true,\n requestFilter,\n stateHandlers: {\n /* ... */\n },\n });\n}\n\n// In tests/hooks, callers can await setupVerifierOptions():\n// const opts = await setupVerifierOptions();\n```\n\n_Source: @seontechnologies/pactjs-utils request-filter module, pact-js-example-provider verification tests._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":7278,"content_sha256":"ca3aa1b6e6b4caa7aa63882b3803bf252f359af32374082c7148095468b1c020"},{"filename":"knowledge/pactjs-utils-zod-to-pact.md","content":"# Pact.js Utils Zod to Pact\n\n## Principle\n\nUse `zodToPactMatchers` from `@seontechnologies/pactjs-utils` to derive Pact V3 matchers directly from a Zod schema so you never maintain two representations of the same response shape. The schema is the source of truth for types; plain example values (or `.openapi({ example })` metadata) supply the concrete example data.\n\n## Rationale\n\n### Problems with hand-written matcher helpers\n\n- **Duplication**: Teams that already define response shapes in Zod (or generate OpenAPI from Zod) then redefine the same shape again as hand-written `{ id: integer(...), name: string(...) }` matcher objects.\n- **Silent drift**: Every schema change must be applied in both places; miss one and the contract drifts silently from the real response shape.\n- **Boilerplate helpers per test file**: Consumer tests end up with local `propMatcherFoo(x) => ({ ... })` helpers that mirror the type exactly.\n- **Over-specification**: Importing the provider's full 20-field schema produces a contract that forces the provider to return every field — breaking consumer-driven testing's core benefit (consumer only asserts what it reads).\n\n### Solutions\n\n- **`zodToPactMatchers(schema, example)`** — walks a Zod schema and emits the right `MatchersV3.*` call per field (`string()`, `integer()`, `decimal()`, `boolean()`, `nullValue()`, `eachLike(...)` for arrays, recursive objects, first option for unions, first value for enums, literal-typed matchers for literals).\n- **Three-step example resolution**: (1) the `example` arg wins, (2) `.openapi({ example })` metadata (if `@asteasolutions/zod-to-openapi` is installed), (3) a type-appropriate default (`'string'`, `1.0`, `true`, no-arg `integer()`).\n- **Consumer-curated schemas**: You choose which schema to pass, so you can include only the fields the consumer actually reads — keeping contracts lean and consumer-driven.\n\n## Pattern Examples\n\n### Example 1: Consumer-curated schema (mandatory pattern)\n\n```typescript\n// pact/http/helpers/consumer-schemas.ts\nimport { z } from 'zod';\n\n// Only the fields this consumer actually reads — NOT the shared full-response schema\nexport const ConsumerMovieSchema = z.object({\n id: z.number().int(),\n name: z.string(),\n year: z.number().int(),\n rating: z.number(),\n director: z.string(),\n});\n```\n\n### Example 2: Replacing hand-written matcher helpers\n\n```typescript\n// ❌ Before — hand-written helper duplicates the shape defined in Movie type\nconst propMatcherNoId = (movie: Omit\u003cMovie, 'id'>) => ({\n name: string(movie.name),\n year: integer(movie.year),\n rating: decimal(movie.rating),\n director: string(movie.director),\n});\n\nawait pact\n .addInteraction()\n .given('No movies exist')\n .uponReceiving('a request to add a new movie')\n .withRequest('POST', '/movies', setJsonContent({ body: movieWithoutId }))\n .willRespondWith(\n 200,\n setJsonContent({\n body: {\n status: 200,\n data: { id: integer(), ...propMatcherNoId(movieWithoutId) },\n },\n }),\n );\n```\n\n```typescript\n// ✅ After — schema defines types, plain object provides examples\nimport { zodToPactMatchers, setJsonContent } from '@seontechnologies/pactjs-utils';\nimport { ConsumerMovieSchema } from '../helpers/consumer-schemas';\n\nawait pact\n .addInteraction()\n .given('No movies exist')\n .uponReceiving('a request to add a new movie')\n .withRequest('POST', '/movies', setJsonContent({ body: movieWithoutId }))\n .willRespondWith(\n 200,\n setJsonContent({\n body: {\n status: 200,\n data: zodToPactMatchers(ConsumerMovieSchema, { id: 1, ...movieWithoutId }),\n },\n }),\n );\n```\n\n### Example 3: Array responses with `eachLike`\n\n```typescript\nimport { PactV4, MatchersV3 } from '@pact-foundation/pact';\nimport { zodToPactMatchers, setJsonContent } from '@seontechnologies/pactjs-utils';\nimport { ConsumerMovieSchema } from '../helpers/consumer-schemas';\n\nconst { eachLike } = MatchersV3;\nconst pact = new PactV4({ consumer: 'Movies Web', provider: 'Movies API' });\nconst movie = { id: 1, name: 'My movie', year: 1999, rating: 8.5, director: 'John Doe' };\n\nawait pact\n .addInteraction()\n .given('Movies exist')\n .uponReceiving('a request for all movies')\n .withRequest('GET', '/movies')\n .willRespondWith(\n 200,\n setJsonContent({\n body: {\n status: 200,\n data: eachLike(zodToPactMatchers(ConsumerMovieSchema, movie) as Parameters\u003ctypeof eachLike>[0]),\n },\n }),\n );\n// data expands to: eachLike({ id: integer(1), name: string('My movie'), year: integer(1999), rating: decimal(8.5), director: string('John Doe') })\n```\n\n### Example 4: Message Pact tests (Kafka / async)\n\n```typescript\nimport { PactV4, MatchersV3 } from '@pact-foundation/pact';\nimport { zodToPactMatchers } from '@seontechnologies/pactjs-utils';\nimport { ConsumerMovieSchema } from '../../http/helpers/consumer-schemas';\n\nconst { string } = MatchersV3;\n\n// Schema-derived matchers — no manual matcher construction, no outer like() wrapper\nconst movieValue = zodToPactMatchers(ConsumerMovieSchema, {\n id: 1,\n name: 'Inception',\n year: 2010,\n rating: 8.8,\n director: 'Christopher Nolan',\n});\n\nawait messagePact\n .addAsynchronousInteraction()\n .given('An existing movie exists')\n .expectsToReceive('a movie-created event', (builder) => {\n builder.withJSONContent({\n topic: string('movie-created'),\n messages: [{ key: string('1'), value: movieValue }],\n });\n });\n```\n\nNote: `zodToPactMatchers` on an object schema already wraps each field in the right matcher, so the extra `like()` wrapper from hand-written versions is not needed — each field carries its own type constraint.\n\n### Example 5: OpenAPI example metadata (optional peer)\n\n```typescript\nimport { z } from 'zod';\nimport { extendZodWithOpenApi } from '@asteasolutions/zod-to-openapi';\n\nextendZodWithOpenApi(z);\n\nconst MovieSchema = z.object({\n name: z.string().openapi({ example: 'Inception' }),\n year: z.number().int().openapi({ example: 2010 }),\n});\n\n// No second argument needed — examples come from the schema itself\nzodToPactMatchers(MovieSchema);\n// → { name: string('Inception'), year: integer(2010) }\n```\n\n## Zod to Pact V3 Mapping\n\n| Zod type | Pact V3 matcher |\n| --------------------------------------------- | ----------------------------------------- |\n| `z.string()` | `string(example ?? 'string')` |\n| `z.number().int()` | `integer(example)` (no-arg if no example) |\n| `z.number()` | `decimal(example ?? 1.0)` |\n| `z.boolean()` | `boolean(example ?? true)` |\n| `z.null()` | `nullValue()` |\n| `z.object({...})` | recursive object of field matchers |\n| `z.array(...)` | `eachLike(itemMatchers)` |\n| `z.union([...])` | first option's matcher |\n| `z.literal('x')` / number / bool | typed matcher with literal value |\n| `z.enum([...])` | `string(firstValue)` |\n| `z.optional()` / `.nullable()` / `.default()` | unwraps to the inner schema |\n| anything else | `like(example ?? null)` fallback |\n\n## Key Points\n\n- **Consumer-curated schema is mandatory**: Define schemas that describe only what the consumer actually reads. Do **not** pass the shared full-response schema, and do **not** `import` the provider-side schema — that turns contract tests into schema tests and blocks the provider from deprecating unused fields.\n- **Example precedence**: `example` argument > `.openapi({ example })` metadata > type default. The example only sets the placeholder value; Pact matchers check type/shape, not exact values.\n- **Optional peer**: `@asteasolutions/zod-to-openapi` is an optional peer dependency. If it's not installed, openapi-example extraction silently becomes a no-op and only the `example` argument / defaults are used.\n- **Optional peer (zod)**: `zod` itself is declared as an optional peer of `@seontechnologies/pactjs-utils` so consumers who don't use `zodToPactMatchers` don't need it; consumers who do use it must have zod installed.\n- **Object wrapping**: When passing an object result into `eachLike(...)`, cast to `Parameters\u003ctypeof eachLike>[0]` — `zodToPactMatchers` returns `unknown` by design to stay compatible with both primitive and composite matcher shapes.\n- **Arrays without examples**: If the example array is empty, the first item's field matchers are derived from the schema (and `.openapi({ example })` metadata, if present).\n- **No extra `like()` wrapper**: For objects returned from `zodToPactMatchers`, do not wrap the whole object in `like()`; each field is already a matcher.\n- **Works for HTTP and message pacts**: The same function produces matchers for request/response bodies and for Kafka / async message payloads.\n- **TypeScript**: Import `z` as a runtime value when defining schemas (`import { z } from 'zod'`). If you need a schema type in helper signatures, import it separately (for example, `import type { ZodTypeAny } from 'zod'`).\n\n## Related Fragments\n\n- `pactjs-utils-overview.md` — installation, utility table, decision tree\n- `pactjs-utils-consumer-helpers.md` — `createProviderState`, `setJsonContent`, `setJsonBody`\n- `pactjs-utils-provider-verifier.md` — `buildVerifierOptions` integration\n- `contract-testing.md` — foundational patterns with raw Pact.js, Provider Scrutiny Protocol (required fields / enums / data types / nested structures)\n\n## Anti-Patterns\n\n### Wrong: Passing the provider's full response schema\n\n```typescript\n// ❌ Importing the shared server-side schema forces the provider to return every field\nimport { FullMovieSchema } from '@shared/schemas/movie'; // 20 fields\n\ndata: zodToPactMatchers(FullMovieSchema, movie);\n```\n\nThis creates a contract that requires the provider to return all 20 fields, even the ones this consumer never reads — breaking consumer-driven testing and blocking future field deprecation.\n\n### Right: Consumer-curated schema beside the pact tests\n\n```typescript\n// ✅ pact/http/helpers/consumer-schemas.ts — only the fields this consumer reads\nexport const ConsumerMovieSchema = z.object({\n id: z.number().int(),\n name: z.string(),\n year: z.number().int(),\n rating: z.number(),\n director: z.string(),\n});\n\ndata: zodToPactMatchers(ConsumerMovieSchema, movie);\n```\n\n### Wrong: Hand-written matcher helper duplicating the schema\n\n```typescript\n// ❌ Local helper that mirrors the TS type — drifts silently on every schema change\nconst propMatcherNoId = (movie: Omit\u003cMovie, 'id'>) => ({\n name: string(movie.name),\n year: integer(movie.year),\n rating: decimal(movie.rating),\n director: string(movie.director),\n});\n```\n\n### Right: `zodToPactMatchers` with a consumer-curated schema\n\n```typescript\n// ✅ Schema is the single source of truth; plain object supplies examples\ndata: zodToPactMatchers(ConsumerMovieSchema, { id: 1, ...movieWithoutId });\n```\n\n### Wrong: Wrapping the whole object result in `like()`\n\n```typescript\n// ❌ Redundant — each field is already a matcher\nvalue: like(zodToPactMatchers(ConsumerMovieSchema, movie));\n```\n\n### Right: Use the object directly\n\n```typescript\n// ✅ Each field carries its own type constraint\nvalue: zodToPactMatchers(ConsumerMovieSchema, movie);\n```\n\n_Source: @seontechnologies/pactjs-utils library, pactjs-utils docs (`docs/zod-to-pact/`), pact-js consumer sample repos, Pact docs on consumer-driven contracts_\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":11803,"content_sha256":"a02b0859122ef4f720818ea78a149799202d0baf7852936f05a2f5d0c49935a8"},{"filename":"knowledge/playwright-cli.md","content":"# Playwright CLI — Browser Automation for Coding Agents\n\n## Principle\n\nWhen an AI agent needs to look at a webpage — take a snapshot, grab selectors, capture a screenshot — it shouldn't have to load thousands of tokens of DOM trees and tool schemas into its context window just to do that. Playwright CLI gives the agent a lightweight way to talk to a browser through simple shell commands, keeping the context window free for reasoning and code generation.\n\n## Rationale\n\nPlaywright MCP is powerful, but it's heavy. Every interaction loads full accessibility trees and tool definitions into the LLM context. That's fine for complex, stateful flows where you need rich introspection. But for the common case — \"open this page, tell me what's on it, take a screenshot\" — it's overkill.\n\nPlaywright CLI solves this by returning concise **element references** (`e15`, `e21`) instead of full DOM dumps. The result: ~93% fewer tokens per interaction, which means the agent can run longer sessions, reason more deeply, and still have context left for your actual code.\n\n**The trade-off is simple:**\n\n- **CLI** = fast, lightweight, stateless — great for quick looks at pages\n- **MCP** = rich, stateful, full-featured — great for complex multi-step automation\n\nTEA uses both where each shines (see `tea_browser_automation: \"auto\"`).\n\n## Prerequisites\n\n```bash\nnpm install -g @playwright/cli@latest # Install globally (Node.js 18+)\nplaywright-cli install --skills # Register as an agent skill\n```\n\nThe global npm install is one-time. Run `playwright-cli install --skills` from your project root to register skills in `.claude/skills/` (works with Claude Code, GitHub Copilot, and other coding agents). Agents without skills support can use the CLI directly via `playwright-cli --help`. TEA documents this during installation but does not run it for you.\n\n## How It Works\n\nThe agent interacts with the browser through shell commands. Each command is a single, focused action:\n\n```bash\n# 1. Open a page\nplaywright-cli -s=tea-test-design-\u003crun_key> open https://app.com/login\n\n# 2. Take a snapshot — returns element references, not DOM trees\nplaywright-cli -s=tea-test-design-\u003crun_key> snapshot\n# Output: [{ref: \"e15\", role: \"textbox\", name: \"Email\"},\n# {ref: \"e21\", role: \"textbox\", name: \"Password\"},\n# {ref: \"e33\", role: \"button\", name: \"Sign In\"}]\n\n# 3. Interact using those references\nplaywright-cli -s=tea-test-design-\u003crun_key> fill e15 \"[email protected]\"\nplaywright-cli -s=tea-test-design-\u003crun_key> fill e21 \"password123\"\nplaywright-cli -s=tea-test-design-\u003crun_key> click e33\n\n# 4. Capture evidence\nplaywright-cli -s=tea-test-design-\u003crun_key> screenshot --filename=login-flow.png\n\n# 5. Clean up\nplaywright-cli -s=tea-test-design-\u003crun_key> close\n```\n\nThe `-s=tea-test-design-\u003crun_key>` flag scopes every command to a named session that belongs to one workflow run. `\u003crun_key>` is the scope key the workflow resolves in its first step (such as `story-1-2-user-authentication`, `epic-3`, or `system`), so a second run on the same machine opens its own browser. See [Session Isolation](#session-isolation).\n\n## What TEA Uses It For\n\n**Selector verification** — Before generating test code, TEA can snapshot a page to see the actual labels, roles, and names of elements. Instead of guessing that a button says \"Login\", it knows it says \"Sign In\":\n\n```text\nsnapshot ref {role: \"button\", name: \"Sign In\"}\n → generates: page.getByRole('button', { name: 'Sign In' })\n```\n\n**Page discovery** — During `test-design` exploratory mode, TEA snapshots pages to understand what's actually there, rather than relying only on documentation.\n\n**Evidence collection** — During `test-review`, TEA can capture screenshots, traces, and network logs as evidence without the overhead of a full MCP session.\n\n**Agent-side test debugging** — For existing failing Playwright tests, TEA should prefer Playwright's agent-facing debug loop over ad hoc manual reproduction: `npx playwright test --debug=cli` to step through the test in CLI mode (no GUI Inspector — designed for coding agents), then `npx playwright trace ...` to inspect the resulting trace artifact from the command line. The `--debug=cli` flag (Playwright 1.59+) lets agents attach, step through execution, and inspect page state without ever opening a browser window.\n\n## How CLI Relates to Playwright Utils and API Testing\n\nCLI and playwright-utils are **complementary tools that work at different layers**:\n\n| | Playwright CLI | Playwright Utils |\n| ------------ | -------------------------------------------- | ------------------------------------------------ |\n| **When** | During test _generation_ (the agent uses it) | During test _execution_ (your test code uses it) |\n| **What** | Shell commands to observe your app | Fixtures and helpers imported in test files |\n| **Examples** | `snapshot`, `screenshot`, `network` | `apiRequest`, `auth-session`, `network-recorder` |\n\nThey work together naturally. The agent uses CLI to _understand_ your app, then generates test code that _imports_ playwright-utils:\n\n```bash\n# Agent uses CLI to observe network traffic on the dashboard page\nplaywright-cli -s=tea-automate-\u003crun_key> open https://app.com/dashboard\nplaywright-cli -s=tea-automate-\u003crun_key> network\n# Output: GET /api/users → 200, POST /api/audit → 201, GET /api/settings → 200\nplaywright-cli -s=tea-automate-\u003crun_key> close\n```\n\n```typescript\n// Agent generates API tests using what it discovered, with playwright-utils\nimport { test } from '@seontechnologies/playwright-utils/api-request/fixtures';\n\ntest('GET /api/users returns user list', async ({ apiRequest }) => {\n const { status, body } = await apiRequest\u003cUser[]>({\n method: 'GET',\n path: '/api/users',\n });\n expect(status).toBe(200);\n expect(body.length).toBeGreaterThan(0);\n});\n```\n\n**For pure API testing** (no UI involved), `playwright-cli` browser commands (snapshot, screenshot, click) don't apply — there's no page. But **trace analysis is highly valuable**. Playwright captures full network traces for API tests (requests, responses, headers, timing), and the trace CLI lets the agent inspect them programmatically:\n\n```bash\n# API test fails in CI → open the trace artifact\nnpx playwright trace open test-results/api-users/trace.zip\n\n# What HTTP call failed?\nnpx playwright trace requests --failed\n# Output: #3 POST /api/users → 422 12ms\n\n# Full request/response details (headers, body, timing)\nnpx playwright trace request 3\n\n# What assertion failed and why?\nnpx playwright trace errors\n\n# Done\nnpx playwright trace close\n```\n\nThis gives the agent the full HTTP conversation — wrong payload, expired auth token, schema mismatch, upstream 5xx — without a human opening UI mode. The agent generates API tests directly from documentation, specs, or code analysis using `apiRequest` and `recurse` from playwright-utils, and uses trace analysis to diagnose failures.\n\n**For E2E testing**, CLI shines at both ends — browser commands (snapshot, screenshot) during test generation, and trace analysis (actions, snapshots, requests) during debugging.\n\n**Bottom line:** CLI helps the agent _write better tests_. Playwright-utils helps those tests _run reliably_. Trace analysis helps the agent _fix them when they break_.\n\n## Session Isolation\n\nEvery CLI command targets a named session. Session names are global to the machine: two runs that use the same name drive one browser, and either run's `close` shuts it for both. Name the session `tea-\u003cworkflow>-\u003crun_key>`, where `\u003crun_key>` is the scope key the workflow resolved in its first step, and use that exact name on every `open`, command, and `close` of the run. Workflows that resolve no `run_key` (`ci` and `framework`) name the session `tea-\u003cworkflow>-\u003ctimestamp>`.\n\n```bash\n# test-design run for epic 3\nplaywright-cli -s=tea-test-design-epic-3 open https://app.com\n\n# NFR run for story 1.2 on the same machine, in parallel\nplaywright-cli -s=tea-nfr-story-1-2-user-authentication open https://app.com/admin\n```\n\nWhen one run opens several sessions at once (for example, parallel subagents), append a unique suffix to the run's session name. `atdd` adds a timestamp suffix to every session it opens, and `automate` adds one to the sessions its E2E subagent opens:\n\n```bash\nplaywright-cli -s=tea-atdd-story-1-2-user-authentication-\u003ctimestamp> open https://app.com\n```\n\nClose sessions by name with `close`. `close-all` kills every session on the machine, including the browsers of other runs.\n\n## Autonomous Trace Investigation (Playwright 1.59+)\n\nFor generated tests that already exist and are failing, Playwright 1.59 introduced CLI-native debugging and trace analysis designed specifically for AI agents. Instead of downloading traces and opening the GUI Trace Viewer, agents can now consume the entire trace context directly from the command line.\n\n### Debug a Failing Test (CLI Mode)\n\n```bash\n# Start the test in CLI debug mode — no GUI Inspector, agent-friendly output\nnpx playwright test --debug=cli\nplaywright-cli attach \u003csession-id>\nplaywright-cli --session \u003csession-id> step-over\n```\n\nWith `--debug=cli`, the agent can:\n\n- Step through test execution in real-time\n- Inspect the page's HTML source at each step\n- Review network calls and console logs at the moment of failure\n- Capture before/after snapshots without opening a browser\n\n### Investigate a Trace Artifact\n\n```bash\n# Open a trace from CI or local runs — this starts a session\nnpx playwright trace open test-results/\u003crun>/trace.zip\n\n# List all actions as a numbered tree (# column = 1-based ordinal)\nnpx playwright trace actions\n# Output: # Time Action Duration\n# 1 0:00.00 navigate(...) 120ms\n# 2 0:00.12 fill(#email, ...) 45ms\n# ...\n# 9 0:01.50 expect(toBeVisible) ✗ 30s\n\n# Filter to failing assertions\nnpx playwright trace actions --grep=\"expect\"\n\n# Drill into action #9 (the ordinal from the list above)\nnpx playwright trace action 9\n\n# See the page snapshot after that action (valid: before | input | after)\nnpx playwright trace snapshot 9 --name after\n\n# Other useful subcommands\nnpx playwright trace errors # errors with stack traces\nnpx playwright trace requests --failed # failed network requests\nnpx playwright trace console --errors-only # console errors\n\n# Close when done (removes extracted data)\nnpx playwright trace close\n```\n\n### Autonomous Diagnostic Loop\n\nWhen TEA encounters a failing test in healing/review mode, the recommended investigation flow is:\n\n1. **Run with `--debug=cli`** to step through the failure and identify the failing action\n2. **Get a trace artifact** — configure `trace: 'retain-on-failure'` in `playwright.config.ts` (recommended), add `--trace=retain-on-failure` to the test run, or use an existing CI trace artifact. For `playwright-cli` sessions (not `--debug=cli`), use `tracing-start` / `tracing-stop` instead.\n3. **Filter to assertions** (`trace actions --grep=\"expect\"`) to find the failure point\n4. **Inspect the snapshot** (`trace snapshot \u003cn> --name after`) to see exact page state at failure\n5. **Analyze network/console** to rule out backend issues or timing problems\n6. **Propose a fix** — updated locator, added wait, or flagged flake for human review\n\nThis reduces Mean Time to Repair (MTTR) by giving the agent full failure context rather than just an error message.\n\n### When to Use Each Tool\n\n- `playwright-cli` session commands remain the best lightweight tool for page exploration and selector verification.\n- `npx playwright test --debug=cli` is better for stepping through an already-written failing test (agent-native, no GUI).\n- `npx playwright trace ...` is better for understanding flakes and assertion failures from saved artifacts.\n\nIf your environment exposes the Playwright dashboard or bound-browser flow, it can help humans inspect what an agent is doing in the background, but TEA should treat that as optional observability rather than a hard dependency.\n\n### Binding a Browser for Agent Inspection (`browser.bind()`)\n\nPlaywright 1.59 added `browser.bind()` — a programmatic API that makes a running browser instance available to `playwright-cli` and MCP clients. This is the bridge between \"a test is running\" and \"an agent can see what the test sees.\"\n\n```typescript\n// In a test or fixture: bind the browser so playwright-cli can attach\nconst { endpoint } = await browser.bind('my-debug-session', {\n workspaceDir: process.cwd(),\n});\n// Now: playwright-cli attach my-debug-session\n```\n\n**When TEA uses this:**\n\n- **Debugging a complex E2E failure** — A test fixture calls `browser.bind()` before the failing scenario, then TEA runs `playwright-cli attach` to inspect live page state, network, and console without re-running the test from scratch.\n- **Bridging CLI and MCP** — A bound browser is accessible to both `playwright-cli` and `@playwright/mcp`. TEA's `auto` mode can start with lightweight CLI inspection and escalate to MCP if richer introspection is needed, all against the same browser instance.\n- **CI artifact enhancement** — A CI helper can bind the browser during test runs, letting a post-failure agent attach and investigate before the process exits.\n\nCall `await browser.unbind()` when done to release the session (async — must be awaited).\n\n## Command Quick Reference\n\n| What you want to do | Command |\n| ------------------------- | ------------------------------------------------ |\n| Open a page | `open \u003curl>` |\n| See what's on the page | `snapshot` |\n| Take a screenshot | `screenshot [--filename=path]` |\n| Click something | `click \u003cref>` |\n| Type into a field | `fill \u003cref> \u003ctext>` |\n| Navigate | `goto \u003curl>`, `go-back`, `reload` |\n| Mock a network request | `route \u003cpattern> --status=200 --body='...'` |\n| Start recording a trace | `tracing-start` |\n| Stop and save the trace | `tracing-stop` |\n| Save auth state for reuse | `state-save auth.json` |\n| Load saved auth state | `state-load auth.json` |\n| See network requests | `network` |\n| Manage tabs | `tab-list`, `tab-new`, `tab-close`, `tab-select` |\n| Close the session | `close` |\n\n## When CLI vs MCP (Auto Mode Decision)\n\n| Situation | Tool | Why |\n| ------------------------------------- | ---- | ---------------------------------- |\n| \"What's on this page?\" | CLI | One-shot snapshot, no state needed |\n| \"Verify this selector exists\" | CLI | Single check, minimal tokens |\n| \"Capture a screenshot for evidence\" | CLI | Stateless capture |\n| \"Walk through a multi-step wizard\" | MCP | State carries across steps |\n| \"Debug why this test fails\" (healing) | CLI | `--debug=cli` + trace analysis |\n| \"Record a drag-and-drop flow\" | MCP | Complex interaction semantics |\n\n## Related Fragments\n\n- `overview.md` — Playwright Utils installation and fixture patterns (the test code layer that CLI complements)\n- `api-request.md` — Typed HTTP client for API tests (CLI discovers endpoints, apiRequest tests them)\n- `api-testing-patterns.md` — Pure API test patterns (when CLI isn't needed)\n- `auth-session.md` — Token management (CLI `state-save` informs auth-session usage)\n- `selector-resilience.md` — Robust selector strategies (CLI verifies them against real DOM)\n- `visual-debugging.md` — Trace viewer usage (CLI captures traces)\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":16102,"content_sha256":"f8993bf5c649ff99a534e287f1d99aeef5ad224a8dfd76eb3db775040799f5cd"},{"filename":"knowledge/playwright-config.md","content":"# Playwright Configuration Guardrails\n\n## Principle\n\nLoad environment configs via a central map (`envConfigMap`), standardize timeouts (action 15s, navigation 30s, expect 10s, test 60s), emit HTML + JUnit reporters, and store artifacts under `test-results/` for CI upload. Keep `.env.example`, `.nvmrc`, and browser dependencies versioned so local and CI runs stay aligned.\n\n## Rationale\n\nEnvironment-specific configuration prevents hardcoded URLs, timeouts, and credentials from leaking into tests. A central config map with fail-fast validation catches missing environments early. Standardized timeouts reduce flakiness while remaining long enough for real-world network conditions. Consistent artifact storage (`test-results/`, `playwright-report/`) enables CI pipelines to upload failure evidence automatically. Versioned dependencies (`.nvmrc`, `package.json` browser versions) eliminate \"works on my machine\" issues between local and CI environments.\n\n## Pattern Examples\n\n### Example 1: Environment-Based Configuration\n\n**Context**: When testing against multiple environments (local, staging, production), use a central config map that loads environment-specific settings and fails fast if `TEST_ENV` is invalid.\n\n**Implementation**:\n\n```typescript\n// playwright.config.ts - Central config loader\nimport { config as dotenvConfig } from 'dotenv';\nimport path from 'path';\n\n// Load .env from project root\ndotenvConfig({\n path: path.resolve(__dirname, '../../.env'),\n});\n\n// Central environment config map\nconst envConfigMap = {\n local: require('./playwright/config/local.config').default,\n staging: require('./playwright/config/staging.config').default,\n production: require('./playwright/config/production.config').default,\n};\n\nconst environment = process.env.TEST_ENV || 'local';\n\n// Fail fast if environment not supported\nif (!Object.keys(envConfigMap).includes(environment)) {\n console.error(`❌ No configuration found for environment: ${environment}`);\n console.error(` Available environments: ${Object.keys(envConfigMap).join(', ')}`);\n process.exit(1);\n}\n\nconsole.log(`✅ Running tests against: ${environment.toUpperCase()}`);\n\nexport default envConfigMap[environment as keyof typeof envConfigMap];\n```\n\n```typescript\n// playwright/config/base.config.ts - Shared base configuration\nimport { defineConfig } from '@playwright/test';\nimport path from 'path';\n\nexport const baseConfig = defineConfig({\n testDir: path.resolve(__dirname, '../tests'),\n outputDir: path.resolve(__dirname, '../../test-results'),\n fullyParallel: true,\n forbidOnly: !!process.env.CI,\n retries: process.env.CI ? 2 : 0,\n workers: process.env.CI ? 1 : undefined,\n reporter: [\n ['html', { outputFolder: 'playwright-report', open: 'never' }],\n ['junit', { outputFile: 'test-results/results.xml' }],\n ['list'],\n ],\n use: {\n actionTimeout: 15000,\n navigationTimeout: 30000,\n trace: 'retain-on-failure-and-retries',\n screenshot: 'only-on-failure',\n video: 'retain-on-failure',\n },\n globalSetup: path.resolve(__dirname, '../support/global-setup.ts'),\n timeout: 60000,\n expect: { timeout: 10000 },\n});\n```\n\n```typescript\n// playwright/config/local.config.ts - Local environment\nimport { defineConfig } from '@playwright/test';\nimport { baseConfig } from './base.config';\n\nexport default defineConfig({\n ...baseConfig,\n use: {\n ...baseConfig.use,\n baseURL: 'http://localhost:3000',\n video: 'off', // No video locally for speed\n },\n webServer: {\n command: 'npm run dev',\n url: 'http://localhost:3000',\n wait: {\n stdout: /ready|listening|localhost:/i,\n },\n reuseExistingServer: !process.env.CI,\n timeout: 120000,\n },\n});\n```\n\n```typescript\n// playwright/config/staging.config.ts - Staging environment\nimport { defineConfig } from '@playwright/test';\nimport { baseConfig } from './base.config';\n\nexport default defineConfig({\n ...baseConfig,\n use: {\n ...baseConfig.use,\n baseURL: 'https://staging.example.com',\n ignoreHTTPSErrors: true, // Allow self-signed certs in staging\n },\n});\n```\n\n```typescript\n// playwright/config/production.config.ts - Production environment\nimport { defineConfig } from '@playwright/test';\nimport { baseConfig } from './base.config';\n\nexport default defineConfig({\n ...baseConfig,\n retries: 3, // More retries in production\n use: {\n ...baseConfig.use,\n baseURL: 'https://example.com',\n video: 'on', // Always record production failures\n },\n});\n```\n\n```bash\n# .env.example - Template for developers\nTEST_ENV=local\nAPI_KEY=your_api_key_here\nDATABASE_URL=postgresql://localhost:5432/test_db\n```\n\n**Key Points**:\n\n- Central `envConfigMap` prevents environment misconfiguration\n- Fail-fast validation with clear error message (available envs listed)\n- Base config defines shared settings, environment configs override\n- `.env.example` provides template for required secrets\n- `TEST_ENV=local` as default for local development\n- Production config increases retries and enables video recording\n\n### Example 2: Timeout Standards\n\n**Context**: When tests fail due to inconsistent timeout settings, standardize timeouts across all tests: action 15s, navigation 30s, expect 10s, test 60s. Expose overrides through fixtures rather than inline literals.\n\n**Implementation**:\n\n```typescript\n// playwright/config/base.config.ts - Standardized timeouts\nimport { defineConfig } from '@playwright/test';\n\nexport default defineConfig({\n // Global test timeout: 60 seconds\n timeout: 60000,\n\n use: {\n // Action timeout: 15 seconds (click, fill, etc.)\n actionTimeout: 15000,\n\n // Navigation timeout: 30 seconds (page.goto, page.reload)\n navigationTimeout: 30000,\n },\n\n // Expect timeout: 10 seconds (all assertions)\n expect: {\n timeout: 10000,\n },\n});\n```\n\n```typescript\n// playwright/support/fixtures/timeout-fixture.ts - Timeout override fixture\nimport { test as base } from '@playwright/test';\n\ntype TimeoutOptions = {\n extendedTimeout: (timeoutMs: number) => Promise\u003cvoid>;\n};\n\nexport const test = base.extend\u003cTimeoutOptions>({\n extendedTimeout: async ({}, use, testInfo) => {\n const originalTimeout = testInfo.timeout;\n\n await use(async (timeoutMs: number) => {\n testInfo.setTimeout(timeoutMs);\n });\n\n // Restore original timeout after test\n testInfo.setTimeout(originalTimeout);\n },\n});\n\nexport { expect } from '@playwright/test';\n```\n\n```typescript\n// Usage in tests - Standard timeouts (implicit)\nimport { test, expect } from '@playwright/test';\n\ntest('user can log in', async ({ page }) => {\n await page.goto('/login'); // Uses 30s navigation timeout\n await page.fill('[data-testid=\"email\"]', '[email protected]'); // Uses 15s action timeout\n await page.click('[data-testid=\"login-button\"]'); // Uses 15s action timeout\n\n await expect(page.getByText('Welcome')).toBeVisible(); // Uses 10s expect timeout\n});\n```\n\n```typescript\n// Usage in tests - Per-test timeout override\nimport { test, expect } from '../support/fixtures/timeout-fixture';\n\ntest('slow data processing operation', async ({ page, extendedTimeout }) => {\n // Override default 60s timeout for this slow test\n await extendedTimeout(180000); // 3 minutes\n\n await page.goto('/data-processing');\n await page.click('[data-testid=\"process-large-file\"]');\n\n // Wait for long-running operation\n await expect(page.getByText('Processing complete')).toBeVisible({\n timeout: 120000, // 2 minutes for assertion\n });\n});\n```\n\n```typescript\n// Per-assertion timeout override (inline)\ntest('API returns quickly', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Override expect timeout for fast API (reduce flakiness detection)\n await expect(page.getByTestId('user-name')).toBeVisible({ timeout: 5000 }); // 5s instead of 10s\n\n // Override expect timeout for slow external API\n await expect(page.getByTestId('weather-widget')).toBeVisible({ timeout: 20000 }); // 20s instead of 10s\n});\n```\n\n**Key Points**:\n\n- **Standardized timeouts**: action 15s, navigation 30s, expect 10s, test 60s (global defaults)\n- Fixture-based override (`extendedTimeout`) for slow tests (preferred over inline)\n- Per-assertion timeout override via `{ timeout: X }` option (use sparingly)\n- Avoid hard waits (`page.waitForTimeout(3000)`) - use event-based waits instead\n- CI environments may need longer timeouts (handle in environment-specific config)\n\n### Example 3: Artifact Output Configuration\n\n**Context**: When debugging failures in CI, configure artifacts (screenshots, videos, traces, HTML reports) to be captured on failure and stored in consistent locations for upload.\n\n**Implementation**:\n\n```typescript\n// playwright.config.ts - Artifact configuration\nimport { defineConfig } from '@playwright/test';\nimport path from 'path';\n\nexport default defineConfig({\n // Output directory for test artifacts\n outputDir: path.resolve(__dirname, './test-results'),\n\n use: {\n // Screenshot on failure only (saves space)\n screenshot: 'only-on-failure',\n\n // Video recording on failure + retry\n video: 'retain-on-failure',\n\n // Keep failed attempts and retries for flake analysis\n trace: 'retain-on-failure-and-retries',\n },\n\n reporter: [\n // HTML report (visual, interactive)\n [\n 'html',\n {\n outputFolder: 'playwright-report',\n open: 'never', // Don't auto-open in CI\n },\n ],\n\n // JUnit XML (CI integration)\n [\n 'junit',\n {\n outputFile: 'test-results/results.xml',\n },\n ],\n\n // List reporter (console output)\n ['list'],\n ],\n});\n```\n\n```typescript\n// playwright/support/fixtures/artifact-fixture.ts - Custom artifact capture\nimport { test as base } from '@playwright/test';\nimport fs from 'fs';\nimport path from 'path';\n\nexport const test = base.extend({\n // Auto-capture console logs on failure\n page: async ({ page }, use, testInfo) => {\n const logs: string[] = [];\n\n page.on('console', (msg) => {\n logs.push(`[${msg.type()}] ${msg.text()}`);\n });\n\n await use(page);\n\n // Save logs on failure\n if (testInfo.status !== testInfo.expectedStatus) {\n const logsPath = path.join(testInfo.outputDir, 'console-logs.txt');\n fs.writeFileSync(logsPath, logs.join('\\n'));\n testInfo.attachments.push({\n name: 'console-logs',\n contentType: 'text/plain',\n path: logsPath,\n });\n }\n },\n});\n```\n\n```yaml\n# .github/workflows/e2e.yml - CI artifact upload\nname: E2E Tests\non: [push, pull_request]\n\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n with:\n node-version-file: '.nvmrc'\n\n - name: Install dependencies\n run: npm ci\n\n - name: Install Playwright browsers\n run: npx playwright install --with-deps\n\n - name: Run tests\n run: npm run test\n env:\n TEST_ENV: staging\n\n # Upload test artifacts on failure\n - name: Upload test results\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: test-results\n path: test-results/\n retention-days: 30\n\n - name: Upload Playwright report\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: playwright-report\n path: playwright-report/\n retention-days: 30\n```\n\n```typescript\n// Example: Custom screenshot on specific condition\ntest('capture screenshot on specific error', async ({ page }) => {\n await page.goto('/checkout');\n\n try {\n await page.click('[data-testid=\"submit-payment\"]');\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n } catch (error) {\n // Capture custom screenshot with timestamp\n await page.screenshot({\n path: `test-results/payment-error-${Date.now()}.png`,\n fullPage: true,\n });\n throw error;\n }\n});\n```\n\n**Key Points**:\n\n- `screenshot: 'only-on-failure'` saves space (not every test)\n- `video: 'retain-on-failure'` captures full flow on failures\n- `trace: 'retain-on-failure-and-retries'` keeps enough history to compare failing retries against passing runs\n- `webServer.wait` is better than startup sleeps when local servers print readiness to stdout/stderr\n- HTML report at `playwright-report/` (visual debugging)\n- JUnit XML at `test-results/results.xml` (CI integration)\n- CI uploads artifacts on failure with 30-day retention\n- Custom fixture can capture console logs, network logs, etc.\n\n### Example 4: Parallelization Configuration\n\n**Context**: When tests run slowly in CI, configure parallelization with worker count, sharding, and fully parallel execution to maximize speed while maintaining stability.\n\n**Implementation**:\n\n```typescript\n// playwright.config.ts - Parallelization settings\nimport { defineConfig } from '@playwright/test';\nimport os from 'os';\n\nexport default defineConfig({\n // Run tests in parallel within single file\n fullyParallel: true,\n\n // Worker configuration\n workers: process.env.CI\n ? 1 // Serial in CI for stability (or 2 for faster CI)\n : os.cpus().length - 1, // Parallel locally (leave 1 CPU for OS)\n\n // Prevent accidentally committed .only() from blocking CI\n forbidOnly: !!process.env.CI,\n\n // Retry failed tests in CI\n retries: process.env.CI ? 2 : 0,\n\n // Shard configuration (split tests across multiple machines)\n shard:\n process.env.SHARD_INDEX && process.env.SHARD_TOTAL\n ? {\n current: parseInt(process.env.SHARD_INDEX, 10),\n total: parseInt(process.env.SHARD_TOTAL, 10),\n }\n : undefined,\n});\n```\n\n```yaml\n# .github/workflows/e2e-parallel.yml - Sharded CI execution\nname: E2E Tests (Parallel)\non: [push, pull_request]\n\njobs:\n test:\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n shard: [1, 2, 3, 4] # Split tests across 4 machines\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n with:\n node-version-file: '.nvmrc'\n\n - name: Install dependencies\n run: npm ci\n\n - name: Install Playwright browsers\n run: npx playwright install --with-deps\n\n - name: Run tests (shard ${{ matrix.shard }})\n run: npm run test\n env:\n SHARD_INDEX: ${{ matrix.shard }}\n SHARD_TOTAL: 4\n TEST_ENV: staging\n\n - name: Upload test results\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: test-results-shard-${{ matrix.shard }}\n path: test-results/\n```\n\n```typescript\n// playwright/config/serial.config.ts - Serial execution for flaky tests\nimport { defineConfig } from '@playwright/test';\nimport { baseConfig } from './base.config';\n\nexport default defineConfig({\n ...baseConfig,\n\n // Disable parallel execution\n fullyParallel: false,\n workers: 1,\n\n // Used for: authentication flows, database-dependent tests, feature flag tests\n});\n```\n\n```typescript\n// Usage: Force serial execution for specific tests\nimport { test } from '@playwright/test';\n\n// Serial execution for auth tests (shared session state)\ntest.describe.configure({ mode: 'serial' });\n\ntest.describe('Authentication Flow', () => {\n test('user can log in', async ({ page }) => {\n // First test in serial block\n });\n\n test('user can access dashboard', async ({ page }) => {\n // Depends on previous test (serial)\n });\n});\n```\n\n```typescript\n// Usage: Parallel execution for independent tests (default)\nimport { test } from '@playwright/test';\n\ntest.describe('Product Catalog', () => {\n test('can view product 1', async ({ page }) => {\n // Runs in parallel with other tests\n });\n\n test('can view product 2', async ({ page }) => {\n // Runs in parallel with other tests\n });\n});\n```\n\n**Key Points**:\n\n- `fullyParallel: true` enables parallel execution within single test file\n- Workers: 1 in CI (stability), N-1 CPUs locally (speed)\n- Sharding splits tests across multiple CI machines (4x faster with 4 shards)\n- `test.describe.configure({ mode: 'serial' })` for dependent tests\n- `forbidOnly: true` in CI prevents `.only()` from blocking pipeline\n- Matrix strategy in CI runs shards concurrently\n\n### Example 5: Project Configuration\n\n**Context**: When testing across multiple browsers, devices, or configurations, use Playwright projects to run the same tests against different environments (chromium, firefox, webkit, mobile).\n\n**Implementation**:\n\n```typescript\n// playwright.config.ts - Multiple browser projects\nimport { defineConfig, devices } from '@playwright/test';\n\nexport default defineConfig({\n projects: [\n // Desktop browsers\n {\n name: 'chromium',\n use: { ...devices['Desktop Chrome'] },\n },\n {\n name: 'firefox',\n use: { ...devices['Desktop Firefox'] },\n },\n {\n name: 'webkit',\n use: { ...devices['Desktop Safari'] },\n },\n\n // Mobile browsers\n {\n name: 'mobile-chrome',\n use: { ...devices['Pixel 5'] },\n },\n {\n name: 'mobile-safari',\n use: { ...devices['iPhone 13'] },\n },\n\n // Tablet\n {\n name: 'tablet',\n use: { ...devices['iPad Pro'] },\n },\n ],\n});\n```\n\n```typescript\n// playwright.config.ts - Authenticated vs. unauthenticated projects\nimport { defineConfig } from '@playwright/test';\nimport path from 'path';\n\nexport default defineConfig({\n projects: [\n // Setup project (runs first, creates auth state)\n {\n name: 'setup',\n testMatch: /global-setup\\.ts/,\n },\n\n // Authenticated tests (reuse auth state)\n {\n name: 'authenticated',\n dependencies: ['setup'],\n use: {\n storageState: path.resolve(__dirname, './playwright/.auth/user.json'),\n },\n testMatch: /.*authenticated\\.spec\\.ts/,\n },\n\n // Unauthenticated tests (public pages)\n {\n name: 'unauthenticated',\n testMatch: /.*unauthenticated\\.spec\\.ts/,\n },\n ],\n});\n```\n\n```typescript\n// playwright/support/global-setup.ts - Setup project for auth\nimport { chromium, FullConfig } from '@playwright/test';\nimport path from 'path';\n\nasync function globalSetup(config: FullConfig) {\n const browser = await chromium.launch();\n const page = await browser.newPage();\n\n // Perform authentication\n await page.goto('http://localhost:3000/login');\n await page.fill('[data-testid=\"email\"]', '[email protected]');\n await page.fill('[data-testid=\"password\"]', 'password123');\n await page.click('[data-testid=\"login-button\"]');\n\n // Wait for authentication to complete\n await page.waitForURL('**/dashboard');\n\n // Save authentication state\n await page.context().storageState({\n path: path.resolve(__dirname, '../.auth/user.json'),\n });\n\n await browser.close();\n}\n\nexport default globalSetup;\n```\n\n```bash\n# Run specific project\nnpx playwright test --project=chromium\nnpx playwright test --project=mobile-chrome\nnpx playwright test --project=authenticated\n\n# Run multiple projects\nnpx playwright test --project=chromium --project=firefox\n\n# Run all projects (default)\nnpx playwright test\n```\n\n```typescript\n// Usage: Project-specific test\nimport { test, expect } from '@playwright/test';\n\ntest('mobile navigation works', async ({ page, isMobile }) => {\n await page.goto('/');\n\n if (isMobile) {\n // Open mobile menu\n await page.click('[data-testid=\"hamburger-menu\"]');\n }\n\n await page.click('[data-testid=\"products-link\"]');\n await expect(page).toHaveURL(/.*products/);\n});\n```\n\n```yaml\n# .github/workflows/e2e-cross-browser.yml - CI cross-browser testing\nname: E2E Tests (Cross-Browser)\non: [push, pull_request]\n\njobs:\n test:\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n project: [chromium, firefox, webkit, mobile-chrome]\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n - run: npm ci\n - run: npx playwright install --with-deps\n\n - name: Run tests (${{ matrix.project }})\n run: npx playwright test --project=${{ matrix.project }}\n```\n\n**Key Points**:\n\n- Projects enable testing across browsers, devices, and configurations\n- `devices` from `@playwright/test` provide preset configurations (Pixel 5, iPhone 13, etc.)\n- `dependencies` ensures setup project runs first (auth, data seeding)\n- `storageState` shares authentication across tests (0 seconds auth per test)\n- `testMatch` filters which tests run in which project\n- CI matrix strategy runs projects in parallel (4x faster with 4 projects)\n- `isMobile` context property for conditional logic in tests\n\n## Integration Points\n\n- **Used in workflows**: `*framework` (config setup), `*ci` (parallelization, artifact upload)\n- **Related fragments**:\n - `fixture-architecture.md` - Fixture-based timeout overrides\n - `ci-burn-in.md` - CI pipeline artifact upload\n - `test-quality.md` - Timeout standards (no hard waits)\n - `data-factories.md` - Per-test isolation (no shared global state)\n\n## Configuration Checklist\n\n**Before deploying tests, verify**:\n\n- [ ] Environment config map with fail-fast validation\n- [ ] Standardized timeouts (action 15s, navigation 30s, expect 10s, test 60s)\n- [ ] Artifact storage at `test-results/` and `playwright-report/`\n- [ ] HTML + JUnit reporters configured\n- [ ] `.env.example`, `.nvmrc`, browser versions committed\n- [ ] Parallelization configured (workers, sharding)\n- [ ] Projects defined for cross-browser/device testing (if needed)\n- [ ] CI uploads artifacts on failure with 30-day retention\n\n_Source: Playwright book repo, enterprise configuration example, Murat testing philosophy (lines 216-271)._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":21457,"content_sha256":"6f7cbf42be005165f68d50090bf0e86227cd401b8e0119ad9e65e7cb2a05644e"},{"filename":"knowledge/playwright-utils-mandate.md","content":"# Playwright Utils Mandate\n\n## Principle\n\nWhen `tea_use_playwright_utils` is `true`, `@seontechnologies/playwright-utils` is the **default implementation** for every capability it covers. Vanilla Playwright equivalents are a documented deviation, never a default. The flag is not a hint that the library exists; it is an instruction to write the suite in that style without being asked.\n\nThis fragment instantiates `library-integration-mandate.md`. Read that one for the two gates, the enforcement levels, and the deviation protocol; this one carries the substitutions. The per-utility fragments (`api-request.md`, `intercept-network-call.md`, `auth-session.md`, and the rest) are the reference for how each utility is called.\n\n## Scope\n\n**Applies when all of these hold:**\n\n- `tea_use_playwright_utils` is `true` under `[modules.tea]` (`config.tea_use_playwright_utils`)\n- `@seontechnologies/playwright-utils` is a dependency in the project's `package.json`\n- The suite runs on the Playwright test runner (`@playwright/test`)\n- The language is JavaScript or TypeScript\n\n**Does not apply to** — nothing in this fragment overrides these:\n\n- Cypress suites\n- Backend suites in pytest, JUnit, Go test, xUnit, or RSpec\n- Maestro mobile flows (no DOM, no request interceptor)\n- Pact consumer/provider suites running under Vitest (see `pactjs-utils-mandate.md`)\n\nA Node.js/TypeScript backend service tested through the Playwright runner **is** in scope: seven of the ten utilities work without a browser.\n\n## Enforcement Levels\n\n| Level | Meaning |\n| --------------- | --------------------------------------------------------------------------------------------------------------------------------- |\n| **REQUIRED** | Drop-in. Nothing beyond the import is needed. Emitting the vanilla equivalent instead is a defect, not a style preference. |\n| **RECOMMENDED** | Needs project-side wiring (auth provider, webhook provider, HAR directory, CI script). Propose it, and scaffold it when in scope. |\n\nFor a RECOMMENDED utility: generate the wiring when the active workflow's scope includes setup (`framework`, `ci`), otherwise state in the output that the utility is the intended pattern and name the wiring the project still needs. Do not silently fall back to the vanilla approach and say nothing.\n\nSchema validation sits at RECOMMENDED for the same reason: it needs a schema to exist. Where the project already has one (a Zod model, an OpenAPI spec, a JSON Schema file), pass it to `apiRequest` rather than hand-writing the shape assertions. Where none exists, assert the fields the test is about **and say so in the output**: \"no response schema found for `\u003cendpoint>`; assertions cover the fields under test only\". A silent fallback reads as a deliberate choice to assert less, and the reader cannot tell it from an oversight.\n\n## Substitution Table\n\n| Need | Vanilla Playwright — do not emit | playwright-utils — emit this | Level | Fragment |\n| ---------------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ----------- | --------------------------- |\n| Observe or stub an HTTP call in a UI test | `page.route()`, `page.waitForResponse()`, manual `JSON.parse` | `interceptNetworkCall({ url, fulfillResponse? })` | REQUIRED | `intercept-network-call.md` |\n| HTTP call from a test (API or setup) | `request.get/post/put/delete`, `await response.json()`, hand-rolled retry | `apiRequest({ method, path, body?, headers? })` | REQUIRED | `api-request.md` |\n| Response body validation | Hand-written field-by-field `expect` chains for shape | `apiRequest` schema validation (JSON Schema, Zod, OpenAPI) | RECOMMENDED | `api-request.md` |\n| Wait for an async or eventually consistent condition | `page.waitForTimeout()`, `while` + `sleep`, bare `expect.poll` | `recurse(fn, predicate, { timeout })` | REQUIRED | `recurse.md` |\n| Test-visible logging | `console.log` | `log.info` / `log.step` / `log.success` / `log.warning` / `log.error` | REQUIRED | `log.md` |\n| Combining fixtures | Ad hoc `base.extend` chains per spec file | `mergeTests` in one `support/merged-fixtures.ts` | REQUIRED | `fixtures-composition.md` |\n| Reading a downloaded CSV/XLSX/PDF/ZIP | `page.waitForEvent('download')` + `saveAs` + a parser per format | `handleDownload()` plus `readCSV` / `readXLSX` / `readPDF` / `readZIP` | REQUIRED | `file-utils.md` |\n| Catching backend 4xx/5xx a green UI hides | `page.on('response')` handlers written per spec | `network-error-monitor` fixture (auto-fails on 4xx/5xx) | REQUIRED | `network-error-monitor.md` |\n| Authentication and token reuse | A login `setup` project writing `storageState`, re-login per run | `setAuthProvider(provider)` + `createAuthFixtures()`, `authToken` fixture | RECOMMENDED | `auth-session.md` |\n| Offline UI runs / backend-free E2E | Hand-maintained fixture JSON per endpoint | `networkRecorder.setup(context)` with HAR record/playback | RECOMMENDED | `network-recorder.md` |\n| Webhook / async event assertions | Custom polling of a mock server, ad hoc sleeps | `webhookTemplate` + `waitFor` / `waitForCount` / `getReceived` | RECOMMENDED | `webhook-*.md` |\n| Running only tests affected by a diff | `--only-changed`, hand-written CI grep filters | `runBurnIn({ configPath, baseBranch })` | RECOMMENDED | `burn-in.md` |\n\n## Banned Patterns\n\nWhen this mandate is active, these are defects in generated or reviewed code:\n\n- `import { test } from '@playwright/test'` in a spec file. Specs import `test` from the project's merged fixtures, which re-export Playwright's `expect` alongside it. playwright-utils exports no `expect` of its own, so importing `expect` from `@playwright/test` directly is correct too and never a violation.\n- `page.route(...)` or `page.waitForResponse(...)` used to spy on or stub an application API call.\n- `request.get/post/put/patch/delete` on the raw `APIRequestContext` for application endpoints.\n- `await response.json()` followed by manual status assertions, where `apiRequest` returns `{ status, body }` already parsed.\n- `page.waitForTimeout(...)` as a synchronization mechanism.\n- `console.log` for anything the test report should show.\n- A bespoke login helper or a `storageState`-producing setup project, where the project already has an auth provider configured.\n\n### Legitimate exceptions\n\nThese are not violations and need no deviation note:\n\n- `page.route` used to **block or stub non-API traffic** — third-party scripts, analytics beacons, fonts, images.\n- `page.waitForResponse` on a call the test does not own and cannot pattern-match by URL, where the response object itself is required.\n- `page.waitForTimeout` inside a debugging aid that is not committed.\n- Raw `request` inside the **auth provider implementation itself** — it runs before the fixtures exist.\n\n## Relationship to the Traditional Fragments\n\n`network-first.md` and `fixture-architecture.md` state principles that stay true under this mandate; only the mechanism changes.\n\n- `network-first` — \"intercept before you navigate\" still holds. The interception is `interceptNetworkCall`, declared before `page.goto`, not `page.route`.\n- `fixture-architecture` — \"pure function core, fixture shell, compose once\" still holds. The composition is `mergeTests` over the playwright-utils fixtures plus the project's own.\n\nWhen `tea_use_playwright_utils` is `true`, load these two fragments for the principles and take every code shape from the playwright-utils fragments. When the flag is `false`, both fragments govern mechanism as well.\n\n## Canonical Shapes\n\n### Merged fixtures — one per project\n\nThere is exactly one, and it lives under the project's configured `test_dir`. A workflow that hardcodes a different directory creates a second entry point, which is the one outcome this file exists to prevent.\n\n**The other fragments show `playwright/support/merged-fixtures.ts`.** That is the upstream playwright-utils repository's own layout in its examples, not a path to copy. `fixtures-composition.md`, `overview.md`, `network-error-monitor.md`, and `webhook-module-setup.md` all use it, and they are read for API shape rather than for where files go. In a TEA-scaffolded project the file is at `{test_dir}/support/merged-fixtures.ts`, and `{test_dir}` is whatever the project configured — `tests/`, `e2e/`, `playwright/`. Resolve it; do not assume it.\n\n```typescript\n// \u003ctest_dir>/support/merged-fixtures.ts (playwright/, tests/, or e2e/ — whatever the project's test_dir is)\nimport { mergeTests } from '@playwright/test';\nimport { log } from '@seontechnologies/playwright-utils';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as interceptFixture } from '@seontechnologies/playwright-utils/intercept-network-call/fixtures';\nimport { test as networkErrorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\n// Project-owned, built with setAuthProvider + createAuthFixtures\nimport { test as authFixture } from './auth-fixture';\n\nexport const test = mergeTests(apiRequestFixture, interceptFixture, networkErrorFixture, recurseFixture, authFixture);\n\nexport { expect } from '@playwright/test';\nexport { log };\n```\n\n### API test\n\n```typescript\nimport { test, expect, log } from '../support/merged-fixtures';\n\ntest.describe('Users API', () => {\n test('[P0] returns the created user', async ({ apiRequest, authToken }) => {\n await log.step('Create user');\n\n const { status, body } = await apiRequest\u003cUser>({\n method: 'POST',\n path: '/api/users',\n body: userFactory(),\n headers: { Authorization: `Bearer ${authToken}` },\n });\n\n expect(status).toBe(201);\n expect(body.email).toBe('...');\n });\n});\n```\n\n### UI test\n\n```typescript\nimport { test, expect } from '../support/merged-fixtures';\n\ntest('[P0] dashboard renders the user list', async ({ page, interceptNetworkCall }) => {\n const usersCall = interceptNetworkCall({ url: '**/api/users' });\n\n await page.goto('/dashboard');\n\n const { responseJson, status } = await usersCall;\n expect(status).toBe(200);\n\n await expect(page.getByRole('row')).toHaveCount(responseJson.length);\n});\n```\n\n## Self-Check Before Emitting a Test File\n\nRun this against every generated or edited spec. Any `yes` in the left column is a blocker.\n\n1. Does the file import `test` from `@playwright/test` instead of the merged fixtures?\n2. Does it call `page.route` or `page.waitForResponse` on an application API endpoint?\n3. Does it call `request.\u003cmethod>` on the raw request context for an application endpoint?\n4. Does it contain `page.waitForTimeout`?\n5. Does it contain `console.log`?\n6. Does it parse a downloaded file by hand?\n7. Does it re-authenticate inline instead of using the `authToken` fixture, with no note saying why?\n\nFix the file, or record a deviation. Do not emit it unresolved.\n\n## Deviation Protocol\n\nA vanilla implementation is allowed when the utility genuinely does not cover the case. When it happens:\n\n1. Add a one-line comment above the code: `// playwright-utils deviation: \u003creason>`\n2. List the deviation in the workflow's output summary under a `Playwright Utils deviations` heading, with file, line, and reason.\n\nAn unexplained vanilla implementation is a finding, not a deviation.\n\n## Review Behavior\n\nUnder `test-review`, with the flag `true`, each of the Banned Patterns above is a **maintainability** finding on the file where it appears, with the substitution named in the recommendation. Report adoption as a ratio (files using merged fixtures over files sampled) rather than a pass/fail, so partial migration is visible instead of collapsing to a single red mark.\n\n## Related Fragments\n\n- `library-integration-mandate.md` — the general contract this instantiates\n- `overview.md` — installation, design principles, the full utility table\n- `api-request.md`, `intercept-network-call.md`, `auth-session.md`, `recurse.md`, `log.md`, `file-utils.md`, `network-recorder.md`, `network-error-monitor.md`, `burn-in.md`\n- `fixtures-composition.md` — `mergeTests` patterns\n- `network-first.md`, `fixture-architecture.md` — the principles this mandate keeps\n- `confidence-gate.md` — stop and ask rather than invent an endpoint, selector, or schema\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13570,"content_sha256":"eb6e300cb76e0438d46d49862495dee6dd16836689aa9f2220b5301fe4f2e62b"},{"filename":"knowledge/probability-impact.md","content":"# Probability and Impact Scale\n\n## Principle\n\nRisk scoring uses a **probability × impact** matrix (1-9 scale) to prioritize testing efforts. Higher scores (6-9) demand immediate action; lower scores (1-3) require documentation only. This systematic approach ensures testing resources focus on the highest-value risks.\n\n## Rationale\n\n**The Problem**: Without quantifiable risk assessment, teams over-test low-value scenarios while missing critical risks. Gut feeling leads to inconsistent prioritization and missed edge cases.\n\n**The Solution**: Standardize risk evaluation with a 3×3 matrix (probability: 1-3, impact: 1-3). Multiply to derive risk score (1-9). Automate classification (DOCUMENT, MONITOR, MITIGATE, BLOCK) based on thresholds. This approach surfaces hidden risks early and justifies testing decisions to stakeholders.\n\n**Why This Matters**:\n\n- Consistent risk language across product, engineering, and QA\n- Objective prioritization of test scenarios (not politics)\n- Automatic gate decisions (score=9 → FAIL until resolved)\n- Audit trail for compliance and retrospectives\n\n## Pattern Examples\n\n### Example 1: Probability-Impact Matrix Implementation (Automated Classification)\n\n**Context**: Implement a reusable risk scoring system with automatic threshold classification\n\n**Implementation**:\n\n```typescript\n// src/testing/risk-matrix.ts\n\n/**\n * Probability levels:\n * 1 = Unlikely (standard implementation, low uncertainty)\n * 2 = Possible (edge cases or partial unknowns)\n * 3 = Likely (known issues, new integrations, high ambiguity)\n */\nexport type Probability = 1 | 2 | 3;\n\n/**\n * Impact levels:\n * 1 = Minor (cosmetic issues or easy workarounds)\n * 2 = Degraded (partial feature loss or manual workaround)\n * 3 = Critical (blockers, data/security/regulatory exposure)\n */\nexport type Impact = 1 | 2 | 3;\n\n/**\n * Risk score (probability × impact): 1-9\n */\nexport type RiskScore = 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9;\n\n/**\n * Action categories based on risk score thresholds\n */\nexport type RiskAction = 'DOCUMENT' | 'MONITOR' | 'MITIGATE' | 'BLOCK';\n\nexport type RiskAssessment = {\n probability: Probability;\n impact: Impact;\n score: RiskScore;\n action: RiskAction;\n reasoning: string;\n};\n\n/**\n * Calculate risk score: probability × impact\n */\nexport function calculateRiskScore(probability: Probability, impact: Impact): RiskScore {\n return (probability * impact) as RiskScore;\n}\n\n/**\n * Classify risk action based on score thresholds:\n * - 1-3: DOCUMENT (awareness only)\n * - 4-5: MONITOR (watch closely, plan mitigations)\n * - 6-8: MITIGATE (CONCERNS at gate until mitigated)\n * - 9: BLOCK (automatic FAIL until resolved or waived)\n */\nexport function classifyRiskAction(score: RiskScore): RiskAction {\n if (score >= 9) return 'BLOCK';\n if (score >= 6) return 'MITIGATE';\n if (score >= 4) return 'MONITOR';\n return 'DOCUMENT';\n}\n\n/**\n * Full risk assessment with automatic classification\n */\nexport function assessRisk(params: { probability: Probability; impact: Impact; reasoning: string }): RiskAssessment {\n const { probability, impact, reasoning } = params;\n\n const score = calculateRiskScore(probability, impact);\n const action = classifyRiskAction(score);\n\n return { probability, impact, score, action, reasoning };\n}\n\n/**\n * Generate risk matrix visualization (3x3 grid)\n * Returns markdown table with color-coded scores\n */\nexport function generateRiskMatrix(): string {\n const matrix: string[][] = [];\n const header = ['Impact \\\\ Probability', 'Unlikely (1)', 'Possible (2)', 'Likely (3)'];\n matrix.push(header);\n\n const impactLabels = ['Critical (3)', 'Degraded (2)', 'Minor (1)'];\n for (let impact = 3; impact >= 1; impact--) {\n const row = [impactLabels[3 - impact]];\n for (let probability = 1; probability \u003c= 3; probability++) {\n const score = calculateRiskScore(probability as Probability, impact as Impact);\n const action = classifyRiskAction(score);\n const emoji = action === 'BLOCK' ? '🔴' : action === 'MITIGATE' ? '🟠' : action === 'MONITOR' ? '🟡' : '🟢';\n row.push(`${emoji} ${score}`);\n }\n matrix.push(row);\n }\n\n return matrix.map((row) => `| ${row.join(' | ')} |`).join('\\n');\n}\n```\n\n**Key Points**:\n\n- Type-safe probability/impact (1-3 enforced at compile time)\n- Automatic action classification (DOCUMENT, MONITOR, MITIGATE, BLOCK)\n- Visual matrix generation for documentation\n- Risk score formula: `probability * impact` (max = 9)\n- Threshold-based decision rules (6-8 = MITIGATE, 9 = BLOCK)\n\n---\n\n### Example 2: Risk Assessment Workflow (Test Planning Integration)\n\n**Context**: Apply risk matrix during test design to prioritize scenarios\n\n**Implementation**:\n\n```typescript\n// tests/e2e/test-planning/risk-assessment.ts\nimport { assessRisk, generateRiskMatrix, type RiskAssessment } from '../../../src/testing/risk-matrix';\n\nexport type TestScenario = {\n id: string;\n title: string;\n feature: string;\n risk: RiskAssessment;\n testLevel: 'E2E' | 'API' | 'Unit';\n priority: 'P0' | 'P1' | 'P2' | 'P3';\n owner: string;\n};\n\n/**\n * Priority is a separate judgment, not a function of risk score.\n *\n * Risk score classifies the remediation ACTION required (DOCUMENT/MONITOR/MITIGATE/BLOCK, see\n * classifyRiskAction above). Priority (P0-P3) is assigned using the business-impact decision\n * tree in test-priorities-matrix.md, which the risk score informs but does not determine. Two\n * scenarios with the same score can carry different priorities depending on revenue impact,\n * user reach, and workaround availability.\n */\n\n/**\n * Example: Payment flow risk assessment\n * Priority below follows the test-priorities-matrix.md decision tree, informed by (not derived\n * from) the risk score.\n */\nexport const paymentScenarios: TestScenario[] = [\n {\n id: 'PAY-001',\n title: 'Valid credit card payment completes successfully',\n feature: 'Checkout',\n risk: assessRisk({\n probability: 2, // Possible (standard Stripe integration)\n impact: 3, // Critical (revenue loss if broken)\n reasoning: 'Core revenue flow, but Stripe is well-tested',\n }),\n priority: 'P0', // Revenue-critical core journey\n testLevel: 'E2E',\n owner: 'qa-team',\n },\n {\n id: 'PAY-002',\n title: 'Expired credit card shows user-friendly error',\n feature: 'Checkout',\n risk: assessRisk({\n probability: 3, // Likely (edge case handling often buggy)\n impact: 2, // Degraded (users see error, but can retry)\n reasoning: 'Error handling logic is custom and complex',\n }),\n priority: 'P1', // Core user journey, frequent error path\n testLevel: 'E2E',\n owner: 'qa-team',\n },\n {\n id: 'PAY-003',\n title: 'Payment confirmation email formatting is correct',\n feature: 'Email',\n risk: assessRisk({\n probability: 2, // Possible (template changes occasionally break)\n impact: 1, // Minor (cosmetic issue, email still sent)\n reasoning: 'Non-blocking, users get email regardless',\n }),\n priority: 'P2', // Customer-facing presentation, with email still delivered\n testLevel: 'Unit',\n owner: 'dev-team',\n },\n {\n id: 'PAY-004',\n title: 'Payment fails gracefully when Stripe is down',\n feature: 'Checkout',\n risk: assessRisk({\n probability: 1, // Unlikely (Stripe has 99.99% uptime)\n impact: 3, // Critical (complete checkout failure)\n reasoning: 'Rare but catastrophic, requires retry mechanism',\n }),\n priority: 'P0', // Revenue-critical checkout failure with no workaround\n testLevel: 'API',\n owner: 'qa-team',\n },\n];\n\n/**\n * Generate risk assessment report with priority distribution\n */\nexport function generateRiskReport(scenarios: TestScenario[]): string {\n const priorityCounts = scenarios.reduce(\n (acc, s) => {\n acc[s.priority] = (acc[s.priority] || 0) + 1;\n return acc;\n },\n {} as Record\u003cstring, number>,\n );\n\n const actionCounts = scenarios.reduce(\n (acc, s) => {\n acc[s.risk.action] = (acc[s.risk.action] || 0) + 1;\n return acc;\n },\n {} as Record\u003cstring, number>,\n );\n\n return `\n# Risk Assessment Report\n\n## Risk Matrix\n${generateRiskMatrix()}\n\n## Priority Distribution\n- **P0 (Critical)**: ${priorityCounts.P0 || 0} scenarios\n- **P1 (High)**: ${priorityCounts.P1 || 0} scenarios\n- **P2 (Medium)**: ${priorityCounts.P2 || 0} scenarios\n- **P3 (Low)**: ${priorityCounts.P3 || 0} scenarios\n\n## Action Required\n- **BLOCK**: ${actionCounts.BLOCK || 0} scenarios (auto-fail gate)\n- **MITIGATE**: ${actionCounts.MITIGATE || 0} scenarios (concerns at gate)\n- **MONITOR**: ${actionCounts.MONITOR || 0} scenarios (watch closely)\n- **DOCUMENT**: ${actionCounts.DOCUMENT || 0} scenarios (awareness only)\n\n## Scenarios by Risk Score (Highest First)\n${scenarios\n .sort((a, b) => b.risk.score - a.risk.score)\n .map((s) => `- **[${s.priority}]** ${s.id}: ${s.title} (Score: ${s.risk.score} - ${s.risk.action})`)\n .join('\\n')}\n`.trim();\n}\n```\n\n**Key Points**:\n\n- Risk score → remediation action mapping (DOCUMENT/MONITOR/MITIGATE/BLOCK, automated); priority\n is assigned separately via the test-priorities-matrix.md decision tree\n- Report generation with priority/action distribution\n- Scenarios sorted by risk score (highest first)\n- Visual matrix included in reports\n- Reusable across projects (extract to shared library)\n\n---\n\n### Example 3: Dynamic Risk Re-Assessment (Continuous Evaluation)\n\n**Context**: Recalculate risk scores as project evolves (requirements change, mitigations implemented)\n\n**Implementation**:\n\n```typescript\n// src/testing/risk-tracking.ts\nimport { type RiskAssessment, assessRisk, type Probability, type Impact } from './risk-matrix';\n\nexport type RiskHistory = {\n timestamp: Date;\n assessment: RiskAssessment;\n changedBy: string;\n reason: string;\n};\n\nexport type TrackedRisk = {\n id: string;\n title: string;\n feature: string;\n currentRisk: RiskAssessment;\n history: RiskHistory[];\n mitigations: string[];\n status: 'OPEN' | 'MITIGATED' | 'WAIVED' | 'RESOLVED';\n};\n\nexport class RiskTracker {\n private risks: Map\u003cstring, TrackedRisk> = new Map();\n\n /**\n * Add new risk to tracker\n */\n addRisk(params: {\n id: string;\n title: string;\n feature: string;\n probability: Probability;\n impact: Impact;\n reasoning: string;\n changedBy: string;\n }): TrackedRisk {\n const { id, title, feature, probability, impact, reasoning, changedBy } = params;\n\n const assessment = assessRisk({ probability, impact, reasoning });\n\n const risk: TrackedRisk = {\n id,\n title,\n feature,\n currentRisk: assessment,\n history: [\n {\n timestamp: new Date(),\n assessment,\n changedBy,\n reason: 'Initial assessment',\n },\n ],\n mitigations: [],\n status: 'OPEN',\n };\n\n this.risks.set(id, risk);\n return risk;\n }\n\n /**\n * Reassess risk (probability or impact changed)\n */\n reassessRisk(params: {\n id: string;\n probability?: Probability;\n impact?: Impact;\n reasoning: string;\n changedBy: string;\n }): TrackedRisk | null {\n const { id, probability, impact, reasoning, changedBy } = params;\n const risk = this.risks.get(id);\n if (!risk) return null;\n\n // Use existing values if not provided\n const newProbability = probability ?? risk.currentRisk.probability;\n const newImpact = impact ?? risk.currentRisk.impact;\n\n const newAssessment = assessRisk({\n probability: newProbability,\n impact: newImpact,\n reasoning,\n });\n\n risk.currentRisk = newAssessment;\n risk.history.push({\n timestamp: new Date(),\n assessment: newAssessment,\n changedBy,\n reason: reasoning,\n });\n\n this.risks.set(id, risk);\n return risk;\n }\n\n /**\n * Mark risk as mitigated (probability reduced)\n */\n mitigateRisk(params: { id: string; newProbability: Probability; mitigation: string; changedBy: string }): TrackedRisk | null {\n const { id, newProbability, mitigation, changedBy } = params;\n const risk = this.reassessRisk({\n id,\n probability: newProbability,\n reasoning: `Mitigation implemented: ${mitigation}`,\n changedBy,\n });\n\n if (risk) {\n risk.mitigations.push(mitigation);\n if (risk.currentRisk.action === 'DOCUMENT' || risk.currentRisk.action === 'MONITOR') {\n risk.status = 'MITIGATED';\n }\n }\n\n return risk;\n }\n\n /**\n * Get risks requiring action (MITIGATE or BLOCK)\n */\n getRisksRequiringAction(): TrackedRisk[] {\n return Array.from(this.risks.values()).filter(\n (r) => r.status === 'OPEN' && (r.currentRisk.action === 'MITIGATE' || r.currentRisk.action === 'BLOCK'),\n );\n }\n\n /**\n * Generate risk trend report (show changes over time)\n */\n generateTrendReport(riskId: string): string | null {\n const risk = this.risks.get(riskId);\n if (!risk) return null;\n\n return `\n# Risk Trend Report: ${risk.id}\n\n**Title**: ${risk.title}\n**Feature**: ${risk.feature}\n**Status**: ${risk.status}\n\n## Current Assessment\n- **Probability**: ${risk.currentRisk.probability}\n- **Impact**: ${risk.currentRisk.impact}\n- **Score**: ${risk.currentRisk.score}\n- **Action**: ${risk.currentRisk.action}\n- **Reasoning**: ${risk.currentRisk.reasoning}\n\n## Mitigations Applied\n${risk.mitigations.length > 0 ? risk.mitigations.map((m) => `- ${m}`).join('\\n') : '- None'}\n\n## History (${risk.history.length} changes)\n${risk.history\n .reverse()\n .map((h) => `- **${h.timestamp.toISOString()}** by ${h.changedBy}: Score ${h.assessment.score} (${h.assessment.action}) - ${h.reason}`)\n .join('\\n')}\n`.trim();\n }\n}\n```\n\n**Key Points**:\n\n- Historical tracking (audit trail for risk changes)\n- Mitigation impact tracking (probability reduction)\n- Status lifecycle (OPEN → MITIGATED → RESOLVED)\n- Trend reports (show risk evolution over time)\n- Re-assessment triggers (requirements change, new info)\n\n---\n\n### Example 4: Risk Matrix in Gate Decision (Integration with Trace Workflow)\n\n**Context**: Use probability-impact scores to drive gate decisions (PASS/CONCERNS/FAIL/WAIVED)\n\n**Implementation**:\n\n```typescript\n// src/testing/gate-decision.ts\nimport { type RiskScore, classifyRiskAction, type RiskAction } from './risk-matrix';\nimport { type TrackedRisk } from './risk-tracking';\n\nexport type GateDecision = 'PASS' | 'CONCERNS' | 'FAIL' | 'WAIVED';\n\nexport type GateResult = {\n decision: GateDecision;\n blockers: TrackedRisk[]; // Score=9, action=BLOCK\n concerns: TrackedRisk[]; // Score 6-8, action=MITIGATE\n monitored: TrackedRisk[]; // Score 4-5, action=MONITOR\n documented: TrackedRisk[]; // Score 1-3, action=DOCUMENT\n summary: string;\n};\n\n/**\n * Evaluate gate based on risk assessments\n */\nexport function evaluateGateFromRisks(risks: TrackedRisk[]): GateResult {\n const blockers = risks.filter((r) => r.currentRisk.action === 'BLOCK' && r.status === 'OPEN');\n const concerns = risks.filter((r) => r.currentRisk.action === 'MITIGATE' && r.status === 'OPEN');\n const monitored = risks.filter((r) => r.currentRisk.action === 'MONITOR');\n const documented = risks.filter((r) => r.currentRisk.action === 'DOCUMENT');\n\n let decision: GateDecision;\n\n if (blockers.length > 0) {\n decision = 'FAIL';\n } else if (concerns.length > 0) {\n decision = 'CONCERNS';\n } else {\n decision = 'PASS';\n }\n\n const summary = generateGateSummary({ decision, blockers, concerns, monitored, documented });\n\n return { decision, blockers, concerns, monitored, documented, summary };\n}\n\n/**\n * Generate gate decision summary\n */\nfunction generateGateSummary(result: Omit\u003cGateResult, 'summary'>): string {\n const { decision, blockers, concerns, monitored, documented } = result;\n\n const lines: string[] = [`## Gate Decision: ${decision}`];\n\n if (decision === 'FAIL') {\n lines.push(`\\n**Blockers** (${blockers.length}): Automatic FAIL until resolved or waived`);\n blockers.forEach((r) => {\n lines.push(`- **${r.id}**: ${r.title} (Score: ${r.currentRisk.score})`);\n lines.push(` - Probability: ${r.currentRisk.probability}, Impact: ${r.currentRisk.impact}`);\n lines.push(` - Reasoning: ${r.currentRisk.reasoning}`);\n });\n }\n\n if (concerns.length > 0) {\n lines.push(`\\n**Concerns** (${concerns.length}): Address before release`);\n concerns.forEach((r) => {\n lines.push(`- **${r.id}**: ${r.title} (Score: ${r.currentRisk.score})`);\n lines.push(` - Mitigations: ${r.mitigations.join(', ') || 'None'}`);\n });\n }\n\n if (monitored.length > 0) {\n lines.push(`\\n**Monitored** (${monitored.length}): Watch closely`);\n monitored.forEach((r) => lines.push(`- **${r.id}**: ${r.title} (Score: ${r.currentRisk.score})`));\n }\n\n if (documented.length > 0) {\n lines.push(`\\n**Documented** (${documented.length}): Awareness only`);\n }\n\n lines.push(`\\n---\\n`);\n lines.push(`**Next Steps**:`);\n if (decision === 'FAIL') {\n lines.push(`- Resolve blockers or request formal waiver`);\n } else if (decision === 'CONCERNS') {\n lines.push(`- Implement mitigations for high-risk scenarios (score 6-8)`);\n lines.push(`- Re-run gate after mitigations`);\n } else {\n lines.push(`- Proceed with release`);\n }\n\n return lines.join('\\n');\n}\n```\n\n**Key Points**:\n\n- Gate decision driven by risk scores (not gut feeling)\n- Automatic FAIL for score=9 (blockers)\n- CONCERNS for score 6-8 (requires mitigation)\n- PASS only when no blockers/concerns\n- Actionable summary with next steps\n- Integration with trace workflow (Phase 2)\n\n---\n\n## Probability-Impact Threshold Summary\n\n| Score | Action | Gate Impact | Typical Use Case |\n| ----- | -------- | -------------------- | -------------------------------------- |\n| 1-3 | DOCUMENT | None | Cosmetic issues, low-priority bugs |\n| 4-5 | MONITOR | None (watch closely) | Edge cases, partial unknowns |\n| 6-8 | MITIGATE | CONCERNS at gate | High-impact scenarios needing coverage |\n| 9 | BLOCK | Automatic FAIL | Critical blockers, must resolve |\n\n## Risk Assessment Checklist\n\nBefore deploying risk matrix:\n\n- [ ] **Probability scale defined**: 1 (unlikely), 2 (possible), 3 (likely) with clear examples\n- [ ] **Impact scale defined**: 1 (minor), 2 (degraded), 3 (critical) with concrete criteria\n- [ ] **Threshold rules documented**: Score → Action mapping (1-3 = DOCUMENT, 4-5 = MONITOR, 6-8 = MITIGATE, 9 = BLOCK)\n- [ ] **Gate integration**: Risk scores drive gate decisions (PASS/CONCERNS/FAIL/WAIVED)\n- [ ] **Re-assessment process**: Risks re-evaluated as project evolves (requirements change, mitigations applied)\n- [ ] **Audit trail**: Historical tracking for risk changes (who, when, why)\n- [ ] **Mitigation tracking**: Link mitigations to probability reduction (quantify impact)\n- [ ] **Reporting**: Risk matrix visualization, trend reports, gate summaries\n\n## Integration Points\n\n- **Used in workflows**: `*test-design` (initial risk assessment), `*trace` (gate decision Phase 2), `*nfr-assess` (security/performance risks)\n- **Related fragments**: `risk-governance.md` (risk scoring matrix, gate decision engine), `test-priorities-matrix.md` (P0-P3 mapping), `nfr-criteria.md` (impact assessment for NFRs)\n- **Tools**: TypeScript for type safety, markdown for reports, version control for audit trail\n\n_Source: Murat risk model summary, gate decision patterns from production systems, probability-impact matrix from risk governance practices._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":19390,"content_sha256":"3a73880070a9a1508a3058c7744aaf6b7fb78c844fe368ed6478e59e719cad10"},{"filename":"knowledge/recurse.md","content":"# Recurse (Polling) Utility\n\n## Principle\n\nUse Cypress-style polling with Playwright's `expect.poll` to wait for asynchronous conditions. Provides configurable timeout, interval, logging, and post-polling callbacks with enhanced error categorization. **Ideal for backend testing**: polling API endpoints for job completion, database eventual consistency, message queue processing, and cache propagation.\n\n## Rationale\n\nTesting async operations (background jobs, eventual consistency, webhook processing) requires polling:\n\n- Vanilla `expect.poll` is verbose\n- No built-in logging for debugging\n- Generic timeout errors\n- No post-poll hooks\n\nThe `recurse` utility provides:\n\n- **Clean syntax**: Inspired by cypress-recurse\n- **Enhanced errors**: Timeout vs command failure vs predicate errors\n- **Built-in logging**: Track polling progress\n- **Post-poll callbacks**: Process results after success\n- **Type-safe**: Full TypeScript generic support\n\n## Quick Start\n\n```typescript\nimport { expect, mergeTests } from '@playwright/test';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\n\nconst test = mergeTests(apiRequestFixture, recurseFixture);\n\ntest('wait for job completion', async ({ recurse, apiRequest }) => {\n const { body } = await apiRequest({\n method: 'POST',\n path: '/api/jobs',\n body: { type: 'export' },\n });\n\n // Poll until job completes\n const result = await recurse(\n () => apiRequest({ method: 'GET', path: `/api/jobs/${body.id}` }),\n (response) => response.body.status === 'completed',\n { timeout: 60000 },\n );\n\n expect(result.body.downloadUrl).toBeDefined();\n});\n```\n\n## Pattern Examples\n\n### Example 1: Basic Polling\n\n**Context**: Wait for async operation to complete with custom timeout and interval.\n\n**Implementation**:\n\n```typescript\nimport { expect, mergeTests } from '@playwright/test';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\n\nconst test = mergeTests(apiRequestFixture, recurseFixture);\n\ntest('should wait for job completion', async ({ recurse, apiRequest }) => {\n // Start job\n const { body } = await apiRequest({\n method: 'POST',\n path: '/api/jobs',\n body: { type: 'export' },\n });\n\n // Poll until ready\n const result = await recurse(\n () => apiRequest({ method: 'GET', path: `/api/jobs/${body.id}` }),\n (response) => response.body.status === 'completed',\n {\n timeout: 60000, // 60 seconds max\n interval: 2000, // Check every 2 seconds\n log: 'Waiting for export job to complete',\n },\n );\n\n expect(result.body.downloadUrl).toBeDefined();\n});\n```\n\n**Key Points**:\n\n- First arg: command function (what to execute)\n- Second arg: predicate function (when to stop)\n- Options: timeout, interval, log message\n- Returns the value when predicate returns true\n\n### Example 2: Working with Assertions\n\n**Context**: Use assertions directly in predicate for more expressive tests.\n\n**Implementation**:\n\n```typescript\ntest('should poll with assertions', async ({ recurse, apiRequest }) => {\n await apiRequest({\n method: 'POST',\n path: '/api/events',\n body: { type: 'user-created', userId: '123' },\n });\n\n // Poll with assertions in predicate - no return true needed!\n await recurse(\n async () => {\n const { body } = await apiRequest({ method: 'GET', path: '/api/events/123' });\n return body;\n },\n (event) => {\n // If all assertions pass, predicate succeeds\n expect(event.processed).toBe(true);\n expect(event.timestamp).toBeDefined();\n // No need to return true - just let assertions pass\n },\n { timeout: 30000 },\n );\n});\n```\n\n**Why no `return true` needed?**\n\nThe predicate checks for \"truthiness\" of the return value. But there's a catch - in JavaScript, an empty `return` (or no return) returns `undefined`, which is falsy!\n\nThe utility handles this by checking if:\n\n1. The predicate didn't throw (assertions passed)\n2. The return value was either `undefined` (implicit return) or truthy\n\nSo you can:\n\n```typescript\n// Option 1: Use assertions only (recommended)\n(event) => {\n expect(event.processed).toBe(true);\n};\n\n// Option 2: Return boolean (also works)\n(event) => event.processed === true;\n\n// Option 3: Mixed (assertions + explicit return)\n(event) => {\n expect(event.processed).toBe(true);\n return true;\n};\n```\n\n### Example 3: Error Handling\n\n**Context**: Understanding the different error types.\n\n**Error Types:**\n\n```typescript\n// RecurseTimeoutError - Predicate never returned true within timeout\n// Contains last command value and predicate error\ntry {\n await recurse(/* ... */);\n} catch (error) {\n if (error instanceof RecurseTimeoutError) {\n console.log('Timed out. Last value:', error.lastCommandValue);\n console.log('Last predicate error:', error.lastPredicateError);\n }\n}\n\n// RecurseCommandError - Command function threw an error\n// The command itself failed (e.g., network error, API error)\n\n// RecursePredicateError - Predicate function threw (not from assertions failing)\n// Logic error in your predicate code\n```\n\n**Custom Error Messages:**\n\n```typescript\ntest('custom error on timeout', async ({ recurse, apiRequest }) => {\n try {\n await recurse(\n () => apiRequest({ method: 'GET', path: '/api/status' }),\n (res) => res.body.ready === true,\n {\n timeout: 10000,\n error: 'System failed to become ready within 10 seconds - check background workers',\n },\n );\n } catch (error) {\n // Error message includes custom context\n expect(error.message).toContain('check background workers');\n throw error;\n }\n});\n```\n\n### Example 4: Post-Polling Callback\n\n**Context**: Process or log results after successful polling.\n\n**Implementation**:\n\n```typescript\ntest('post-poll processing', async ({ recurse, apiRequest }) => {\n const finalResult = await recurse(\n () => apiRequest({ method: 'GET', path: '/api/batch-job/123' }),\n (res) => res.body.status === 'completed',\n {\n timeout: 60000,\n post: (result) => {\n // Runs after successful polling\n console.log(`Job completed in ${result.body.duration}ms`);\n console.log(`Processed ${result.body.itemsProcessed} items`);\n return result.body;\n },\n },\n );\n\n expect(finalResult.itemsProcessed).toBeGreaterThan(0);\n});\n```\n\n**Key Points**:\n\n- `post` callback runs after predicate succeeds\n- Receives the final result\n- Can transform or log results\n- Return value becomes final `recurse` result\n\n### Example 5: UI Testing Scenarios\n\n**Context**: Wait for UI elements to reach a specific state through polling.\n\n**Implementation**:\n\n```typescript\ntest('table data loads', async ({ page, recurse }) => {\n await page.goto('/reports');\n\n // Poll for table rows to appear\n await recurse(\n async () => page.locator('table tbody tr').count(),\n (count) => count >= 10, // Wait for at least 10 rows\n {\n timeout: 15000,\n interval: 500,\n log: 'Waiting for table data to load',\n },\n );\n\n // Now safe to interact with table\n await page.locator('table tbody tr').first().click();\n});\n```\n\n### Example 6: Event-Based Systems (Kafka/Message Queues)\n\n**Context**: Testing eventual consistency with message queue processing.\n\n**Implementation**:\n\n```typescript\ntest('kafka event processed', async ({ recurse, apiRequest }) => {\n // Trigger action that publishes Kafka event\n await apiRequest({\n method: 'POST',\n path: '/api/orders',\n body: { productId: 'ABC123', quantity: 2 },\n });\n\n // Poll for downstream effect of Kafka consumer processing\n const inventoryResult = await recurse(\n () => apiRequest({ method: 'GET', path: '/api/inventory/ABC123' }),\n (res) => {\n // Assumes test fixture seeds inventory at 100; in production tests,\n // fetch baseline first and assert: expect(res.body.available).toBe(baseline - 2)\n expect(res.body.available).toBeLessThanOrEqual(98);\n },\n {\n timeout: 30000, // Kafka processing may take time\n interval: 1000,\n log: 'Waiting for Kafka event to be processed',\n },\n );\n\n expect(inventoryResult.body.lastOrderId).toBeDefined();\n});\n```\n\n### Example 7: Integration with API Request (Common Pattern)\n\n**Context**: Most common use case - polling API endpoints for state changes.\n\n**Implementation**:\n\n```typescript\nimport { expect, mergeTests } from '@playwright/test';\nimport { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';\nimport { test as recurseFixture } from '@seontechnologies/playwright-utils/recurse/fixtures';\n\nconst test = mergeTests(apiRequestFixture, recurseFixture);\n\ntest('end-to-end polling', async ({ apiRequest, recurse }) => {\n // Trigger async operation\n const { body: createResp } = await apiRequest({\n method: 'POST',\n path: '/api/data-import',\n body: { source: 's3://bucket/data.csv' },\n });\n\n // Poll until import completes\n const importResult = await recurse(\n () => apiRequest({ method: 'GET', path: `/api/data-import/${createResp.importId}` }),\n (response) => {\n const { status, rowsImported } = response.body;\n return status === 'completed' && rowsImported > 0;\n },\n {\n timeout: 120000, // 2 minutes for large imports\n interval: 5000, // Check every 5 seconds\n log: `Polling import ${createResp.importId}`,\n },\n );\n\n expect(importResult.body.rowsImported).toBeGreaterThan(1000);\n expect(importResult.body.errors).toHaveLength(0);\n});\n```\n\n**Key Points**:\n\n- Combine `apiRequest` + `recurse` for API polling\n- Compose both with `mergeTests`: `apiRequest` from `@seontechnologies/playwright-utils/api-request/fixtures`, `recurse` from `@seontechnologies/playwright-utils/recurse/fixtures`\n- Complex predicates with multiple conditions\n- Logging shows polling progress in test reports\n\n## API Reference\n\n### RecurseOptions\n\n| Option | Type | Default | Description |\n| ---------- | ------------------ | ----------- | ------------------------------------ |\n| `timeout` | `number` | `30000` | Maximum time to wait (ms) |\n| `interval` | `number` | `1000` | Time between polls (ms) |\n| `log` | `string` | `undefined` | Message logged on each poll |\n| `error` | `string` | `undefined` | Custom error message for timeout |\n| `post` | `(result: T) => R` | `undefined` | Callback after successful poll |\n| `delay` | `number` | `0` | Initial delay before first poll (ms) |\n\n### Error Types\n\n| Error Type | When Thrown | Properties |\n| ----------------------- | --------------------------------------- | ---------------------------------------- |\n| `RecurseTimeoutError` | Predicate never passed within timeout | `lastCommandValue`, `lastPredicateError` |\n| `RecurseCommandError` | Command function threw an error | `cause` (original error) |\n| `RecursePredicateError` | Predicate threw (not assertion failure) | `cause` (original error) |\n\n## Comparison with Vanilla Playwright\n\n| Vanilla Playwright | recurse Utility |\n| ----------------------------------------------------------------- | ------------------------------------------------------------------------- |\n| `await expect.poll(() => { ... }, { timeout: 30000 }).toBe(true)` | `await recurse(() => { ... }, (val) => val === true, { timeout: 30000 })` |\n| No logging | Built-in log option |\n| Generic timeout errors | Categorized errors (timeout/command/predicate) |\n| No post-poll hooks | `post` callback support |\n\n## When to Use\n\n**Use recurse for:**\n\n- Background job completion\n- Webhook/event processing\n- Database eventual consistency\n- Cache propagation\n- State machine transitions\n\n**Stick with vanilla expect.poll for:**\n\n- Simple UI element visibility (use `expect(locator).toBeVisible()`)\n- Single-property checks\n- Cases where logging isn't needed\n\n## Related Fragments\n\n- `api-testing-patterns.md` - Comprehensive pure API testing patterns\n- `api-request.md` - Combine for API endpoint polling\n- `overview.md` - Fixture composition patterns\n- `fixtures-composition.md` - Using with mergeTests\n- `contract-testing.md` - Contract testing with async verification\n\n## Anti-Patterns\n\n**DON'T use hard waits instead of polling:**\n\n```typescript\nawait page.click('#export');\nawait page.waitForTimeout(5000); // Arbitrary wait\nexpect(await page.textContent('#status')).toBe('Ready');\n```\n\n**DO poll for actual condition:**\n\n```typescript\nawait page.click('#export');\nawait recurse(\n () => page.textContent('#status'),\n (status) => status === 'Ready',\n { timeout: 10000 },\n);\n```\n\n**DON'T poll too frequently:**\n\n```typescript\nawait recurse(\n () => apiRequest({ method: 'GET', path: '/status' }),\n (res) => res.body.ready,\n { interval: 100 }, // Hammers API every 100ms!\n);\n```\n\n**DO use reasonable interval for API calls:**\n\n```typescript\nawait recurse(\n () => apiRequest({ method: 'GET', path: '/status' }),\n (res) => res.body.ready,\n { interval: 2000 }, // Check every 2 seconds (reasonable)\n);\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13771,"content_sha256":"cf6e8fa4f0d40098004b36b79262773ae1ebe058a8275b3a6ec3ec4b9c1f6e8c"},{"filename":"knowledge/risk-governance.md","content":"# Risk Governance and Gatekeeping\n\n## Principle\n\nRisk governance transforms subjective \"should we ship?\" debates into objective, data-driven decisions. By scoring risk (probability × impact), classifying by category (TECH, SEC, PERF, etc.), and tracking mitigation ownership, teams create transparent quality gates that balance speed with safety.\n\n## Rationale\n\n**The Problem**: Without formal risk governance, releases become political—loud voices win, quiet risks hide, and teams discover critical issues in production. \"We thought it was fine\" isn't a release strategy.\n\n**The Solution**: Risk scoring (1-3 scale for probability and impact, total 1-9) creates shared language. Scores ≥6 demand documented mitigation. Scores = 9 mandate gate failure. Every acceptance criterion maps to a test, and gaps require explicit waivers with owners and expiry dates.\n\n**Why This Matters**:\n\n- Removes ambiguity from release decisions (objective scores vs subjective opinions)\n- Creates audit trail for compliance (FDA, SOC2, ISO require documented risk management)\n- Identifies true blockers early (prevents last-minute production fires)\n- Distributes responsibility (owners, mitigation plans, deadlines for every risk >4)\n\n## Pattern Examples\n\n### Example 1: Risk Scoring Matrix with Automated Classification (TypeScript)\n\n**Context**: Calculate risk scores automatically from test results and categorize by risk type\n\n**Implementation**:\n\n```typescript\n// risk-scoring.ts - Risk classification and scoring system\nexport const RISK_CATEGORIES = {\n TECH: 'TECH', // Technical debt, architecture fragility\n SEC: 'SEC', // Security vulnerabilities\n PERF: 'PERF', // Performance degradation\n DATA: 'DATA', // Data integrity, corruption\n BUS: 'BUS', // Business logic errors\n OPS: 'OPS', // Operational issues (deployment, monitoring)\n} as const;\n\nexport type RiskCategory = keyof typeof RISK_CATEGORIES;\n\nexport type RiskScore = {\n id: string;\n category: RiskCategory;\n title: string;\n description: string;\n probability: 1 | 2 | 3; // 1=Low, 2=Medium, 3=High\n impact: 1 | 2 | 3; // 1=Low, 2=Medium, 3=High\n score: number; // probability × impact (1-9)\n owner: string;\n mitigationPlan?: string;\n deadline?: Date;\n status: 'OPEN' | 'MITIGATED' | 'WAIVED' | 'ACCEPTED';\n waiverReason?: string;\n waiverApprover?: string;\n waiverExpiry?: Date;\n};\n\n// Risk scoring rules\nexport function calculateRiskScore(probability: 1 | 2 | 3, impact: 1 | 2 | 3): number {\n return probability * impact;\n}\n\nexport function requiresMitigation(score: number): boolean {\n return score >= 6; // Scores 6-9 demand action\n}\n\nexport function isCriticalBlocker(score: number): boolean {\n return score === 9; // Probability=3 AND Impact=3 → FAIL gate\n}\n\nexport function classifyRiskLevel(score: number): 'LOW' | 'MEDIUM' | 'HIGH' | 'CRITICAL' {\n if (score === 9) return 'CRITICAL';\n if (score >= 6) return 'HIGH';\n if (score >= 4) return 'MEDIUM';\n return 'LOW';\n}\n\n// Example: Risk assessment from test failures\nexport function assessTestFailureRisk(failure: {\n test: string;\n category: RiskCategory;\n affectedUsers: number;\n revenueImpact: number;\n securityVulnerability: boolean;\n}): RiskScore {\n // Probability based on test failure frequency (simplified)\n const probability: 1 | 2 | 3 = 3; // Test failed = High probability\n\n // Impact based on business context\n let impact: 1 | 2 | 3 = 1;\n if (failure.securityVulnerability) impact = 3;\n else if (failure.revenueImpact > 10000) impact = 3;\n else if (failure.affectedUsers > 1000) impact = 2;\n else impact = 1;\n\n const score = calculateRiskScore(probability, impact);\n\n return {\n id: `risk-${Date.now()}`,\n category: failure.category,\n title: `Test failure: ${failure.test}`,\n description: `Affects ${failure.affectedUsers} users, ${failure.revenueImpact} revenue`,\n probability,\n impact,\n score,\n owner: 'unassigned',\n status: score === 9 ? 'OPEN' : 'OPEN',\n };\n}\n```\n\n**Key Points**:\n\n- **Objective scoring**: Probability (1-3) × Impact (1-3) = Score (1-9)\n- **Clear thresholds**: Score ≥6 requires mitigation, score = 9 blocks release\n- **Business context**: Revenue, users, security drive impact calculation\n- **Status tracking**: OPEN → MITIGATED → WAIVED → ACCEPTED lifecycle\n\n---\n\n### Example 2: Gate Decision Engine with Traceability Validation\n\n> **Illustrative only. The coverage gate is authoritative.** This example teaches a risk-driven\n> decision keyed directly on risk scores. TEA executes\n> the gate decision step of the `bmad-tea-trace` skill. That gate evaluates\n> coverage only when `allow_gate` is true and `collection_status` is `COLLECTED`; otherwise its\n> outcome is `NOT_EVALUATED`. Eligible coverage produces `PASS`, `CONCERNS`, or `FAIL` from P0,\n> P1, and overall percentages. Coverage resting only on recorded live verification cannot produce\n> `PASS` and is capped at `CONCERNS`. `WAIVED` remains a human override with its own approval\n> contract. When answering \"how does the gate decide\", use that coverage engine.\n\n**Context**: Automated gate decision based on risk scores and test coverage\n\n**Implementation**:\n\n```typescript\n// gate-decision-engine.ts\nexport type GateDecision = 'PASS' | 'CONCERNS' | 'FAIL' | 'WAIVED';\n\nexport type CoverageGap = {\n acceptanceCriteria: string;\n testMissing: string;\n reason: string;\n};\n\nexport type GateResult = {\n decision: GateDecision;\n timestamp: Date;\n criticalRisks: RiskScore[];\n highRisks: RiskScore[];\n coverageGaps: CoverageGap[];\n summary: string;\n recommendations: string[];\n};\n\nexport function evaluateGate(params: { risks: RiskScore[]; coverageGaps: CoverageGap[]; waiverApprover?: string }): GateResult {\n const { risks, coverageGaps, waiverApprover } = params;\n\n // Categorize risks\n const criticalRisks = risks.filter((r) => r.score === 9 && r.status === 'OPEN');\n const highRisks = risks.filter((r) => r.score >= 6 && r.score \u003c 9 && r.status === 'OPEN');\n const unresolvedGaps = coverageGaps.filter((g) => !g.reason);\n\n // Decision logic\n let decision: GateDecision;\n\n // FAIL: Critical blockers (score=9) or missing coverage\n if (criticalRisks.length > 0 || unresolvedGaps.length > 0) {\n decision = 'FAIL';\n }\n // WAIVED: All risks waived by authorized approver\n else if (risks.every((r) => r.status === 'WAIVED') && waiverApprover) {\n decision = 'WAIVED';\n }\n // CONCERNS: High risks (score 6-8) with mitigation plans\n else if (highRisks.length > 0 && highRisks.every((r) => r.mitigationPlan && r.owner !== 'unassigned')) {\n decision = 'CONCERNS';\n }\n // PASS: No critical issues, all risks mitigated or low\n else {\n decision = 'PASS';\n }\n\n // Generate recommendations\n const recommendations: string[] = [];\n if (criticalRisks.length > 0) {\n recommendations.push(`🚨 ${criticalRisks.length} CRITICAL risk(s) must be mitigated before release`);\n }\n if (unresolvedGaps.length > 0) {\n recommendations.push(`📋 ${unresolvedGaps.length} acceptance criteria lack test coverage`);\n }\n if (highRisks.some((r) => !r.mitigationPlan)) {\n recommendations.push(`⚠️ High risks without mitigation plans: assign owners and deadlines`);\n }\n if (decision === 'PASS') {\n recommendations.push(`✅ All risks mitigated or acceptable. Ready for release.`);\n }\n\n return {\n decision,\n timestamp: new Date(),\n criticalRisks,\n highRisks,\n coverageGaps: unresolvedGaps,\n summary: generateSummary(decision, risks, unresolvedGaps),\n recommendations,\n };\n}\n\nfunction generateSummary(decision: GateDecision, risks: RiskScore[], gaps: CoverageGap[]): string {\n const total = risks.length;\n const critical = risks.filter((r) => r.score === 9).length;\n const high = risks.filter((r) => r.score >= 6 && r.score \u003c 9).length;\n\n return `Gate Decision: ${decision}. Total Risks: ${total} (${critical} critical, ${high} high). Coverage Gaps: ${gaps.length}.`;\n}\n```\n\n**Usage Example**:\n\n```typescript\n// Example: Running gate check before deployment\nimport { assessTestFailureRisk, evaluateGate } from './gate-decision-engine';\n\n// Collect risks from test results\nconst risks: RiskScore[] = [\n assessTestFailureRisk({\n test: 'Payment processing with expired card',\n category: 'BUS',\n affectedUsers: 5000,\n revenueImpact: 50000,\n securityVulnerability: false,\n }),\n assessTestFailureRisk({\n test: 'SQL injection in search endpoint',\n category: 'SEC',\n affectedUsers: 10000,\n revenueImpact: 0,\n securityVulnerability: true,\n }),\n];\n\n// Identify coverage gaps\nconst coverageGaps: CoverageGap[] = [\n {\n acceptanceCriteria: 'User can reset password via email',\n testMissing: 'e2e/auth/password-reset.spec.ts',\n reason: '', // Empty = unresolved\n },\n];\n\n// Evaluate gate\nconst gateResult = evaluateGate({ risks, coverageGaps });\n\nconsole.log(gateResult.decision); // 'FAIL'\nconsole.log(gateResult.summary);\n// \"Gate Decision: FAIL. Total Risks: 2 (1 critical, 1 high). Coverage Gaps: 1.\"\n\nconsole.log(gateResult.recommendations);\n// [\n// \"🚨 1 CRITICAL risk(s) must be mitigated before release\",\n// \"📋 1 acceptance criteria lack test coverage\"\n// ]\n```\n\n**Key Points**:\n\n- **Automated decision**: No human interpretation required\n- **Clear criteria**: FAIL = critical risks or gaps, CONCERNS = high risks with plans, PASS = low risks\n- **Actionable output**: Recommendations drive next steps\n- **Audit trail**: Timestamp, decision, and context for compliance\n- **Not the executed gate**: see the callout above; `step-05-gate-decision.md` is the rule set that runs\n\n---\n\n### Example 3: Risk Mitigation Workflow with Owner Tracking\n\n**Context**: Track risk mitigation from identification to resolution\n\n**Implementation**:\n\n```typescript\n// risk-mitigation.ts\nexport type MitigationAction = {\n riskId: string;\n action: string;\n owner: string;\n deadline: Date;\n status: 'PENDING' | 'IN_PROGRESS' | 'COMPLETED' | 'BLOCKED';\n completedAt?: Date;\n blockedReason?: string;\n};\n\nexport class RiskMitigationTracker {\n private risks: Map\u003cstring, RiskScore> = new Map();\n private actions: Map\u003cstring, MitigationAction[]> = new Map();\n private history: Array\u003c{ riskId: string; event: string; timestamp: Date }> = [];\n\n // Register a new risk\n addRisk(risk: RiskScore): void {\n this.risks.set(risk.id, risk);\n this.logHistory(risk.id, `Risk registered: ${risk.title} (Score: ${risk.score})`);\n\n // Auto-assign mitigation requirements for score ≥6\n if (requiresMitigation(risk.score) && !risk.mitigationPlan) {\n this.logHistory(risk.id, `⚠️ Mitigation required (score ${risk.score}). Assign owner and plan.`);\n }\n }\n\n // Add mitigation action\n addMitigationAction(action: MitigationAction): void {\n const risk = this.risks.get(action.riskId);\n if (!risk) throw new Error(`Risk ${action.riskId} not found`);\n\n const existingActions = this.actions.get(action.riskId) || [];\n existingActions.push(action);\n this.actions.set(action.riskId, existingActions);\n\n this.logHistory(action.riskId, `Mitigation action added: ${action.action} (Owner: ${action.owner})`);\n }\n\n // Complete mitigation action\n completeMitigation(riskId: string, actionIndex: number): void {\n const actions = this.actions.get(riskId);\n if (!actions || !actions[actionIndex]) throw new Error('Action not found');\n\n actions[actionIndex].status = 'COMPLETED';\n actions[actionIndex].completedAt = new Date();\n\n this.logHistory(riskId, `Mitigation completed: ${actions[actionIndex].action}`);\n\n // If all actions completed, mark risk as MITIGATED\n if (actions.every((a) => a.status === 'COMPLETED')) {\n const risk = this.risks.get(riskId)!;\n risk.status = 'MITIGATED';\n this.logHistory(riskId, `✅ Risk mitigated. All actions complete.`);\n }\n }\n\n // Request waiver for a risk\n requestWaiver(riskId: string, reason: string, approver: string, expiryDays: number): void {\n const risk = this.risks.get(riskId);\n if (!risk) throw new Error(`Risk ${riskId} not found`);\n\n risk.status = 'WAIVED';\n risk.waiverReason = reason;\n risk.waiverApprover = approver;\n risk.waiverExpiry = new Date(Date.now() + expiryDays * 24 * 60 * 60 * 1000);\n\n this.logHistory(riskId, `⚠️ Waiver granted by ${approver}. Expires: ${risk.waiverExpiry}`);\n }\n\n // Generate risk report\n generateReport(): string {\n const allRisks = Array.from(this.risks.values());\n const critical = allRisks.filter((r) => r.score === 9 && r.status === 'OPEN');\n const high = allRisks.filter((r) => r.score >= 6 && r.score \u003c 9 && r.status === 'OPEN');\n const mitigated = allRisks.filter((r) => r.status === 'MITIGATED');\n const waived = allRisks.filter((r) => r.status === 'WAIVED');\n\n let report = `# Risk Mitigation Report\\n\\n`;\n report += `**Generated**: ${new Date().toISOString()}\\n\\n`;\n report += `## Summary\\n`;\n report += `- Total Risks: ${allRisks.length}\\n`;\n report += `- Critical (Score=9, OPEN): ${critical.length}\\n`;\n report += `- High (Score 6-8, OPEN): ${high.length}\\n`;\n report += `- Mitigated: ${mitigated.length}\\n`;\n report += `- Waived: ${waived.length}\\n\\n`;\n\n if (critical.length > 0) {\n report += `## 🚨 Critical Risks (BLOCKERS)\\n\\n`;\n critical.forEach((r) => {\n report += `- **${r.title}** (${r.category})\\n`;\n report += ` - Score: ${r.score} (Probability: ${r.probability}, Impact: ${r.impact})\\n`;\n report += ` - Owner: ${r.owner}\\n`;\n report += ` - Mitigation: ${r.mitigationPlan || 'NOT ASSIGNED'}\\n\\n`;\n });\n }\n\n if (high.length > 0) {\n report += `## ⚠️ High Risks\\n\\n`;\n high.forEach((r) => {\n report += `- **${r.title}** (${r.category})\\n`;\n report += ` - Score: ${r.score}\\n`;\n report += ` - Owner: ${r.owner}\\n`;\n report += ` - Deadline: ${r.deadline?.toISOString().split('T')[0] || 'NOT SET'}\\n\\n`;\n });\n }\n\n return report;\n }\n\n private logHistory(riskId: string, event: string): void {\n this.history.push({ riskId, event, timestamp: new Date() });\n }\n\n getHistory(riskId: string): Array\u003c{ event: string; timestamp: Date }> {\n return this.history.filter((h) => h.riskId === riskId).map((h) => ({ event: h.event, timestamp: h.timestamp }));\n }\n}\n```\n\n**Usage Example**:\n\n```typescript\nconst tracker = new RiskMitigationTracker();\n\n// Register critical security risk\ntracker.addRisk({\n id: 'risk-001',\n category: 'SEC',\n title: 'SQL injection vulnerability in user search',\n description: 'Unsanitized input allows arbitrary SQL execution',\n probability: 3,\n impact: 3,\n score: 9,\n owner: 'security-team',\n status: 'OPEN',\n});\n\n// Add mitigation actions\ntracker.addMitigationAction({\n riskId: 'risk-001',\n action: 'Add parameterized queries to user-search endpoint',\n owner: '[email protected]',\n deadline: new Date('2025-10-20'),\n status: 'IN_PROGRESS',\n});\n\ntracker.addMitigationAction({\n riskId: 'risk-001',\n action: 'Add WAF rule to block SQL injection patterns',\n owner: '[email protected]',\n deadline: new Date('2025-10-22'),\n status: 'PENDING',\n});\n\n// Complete first action\ntracker.completeMitigation('risk-001', 0);\n\n// Generate report\nconsole.log(tracker.generateReport());\n// Markdown report with critical risks, owners, deadlines\n\n// View history\nconsole.log(tracker.getHistory('risk-001'));\n// [\n// { event: 'Risk registered: SQL injection...', timestamp: ... },\n// { event: 'Mitigation action added: Add parameterized queries...', timestamp: ... },\n// { event: 'Mitigation completed: Add parameterized queries...', timestamp: ... }\n// ]\n```\n\n**Key Points**:\n\n- **Ownership enforcement**: Every risk >4 requires owner assignment\n- **Deadline tracking**: Mitigation actions have explicit deadlines\n- **Audit trail**: Complete history of risk lifecycle (registered → mitigated)\n- **Automated reports**: Markdown output for Confluence/GitHub wikis\n\n---\n\n### Example 4: Coverage Traceability Matrix (Test-to-Requirement Mapping)\n\n**Context**: Validate that every acceptance criterion maps to at least one test\n\n**Implementation**:\n\n```typescript\n// coverage-traceability.ts\nexport type AcceptanceCriterion = {\n id: string;\n story: string;\n criterion: string;\n priority: 'P0' | 'P1' | 'P2' | 'P3';\n};\n\nexport type TestCase = {\n file: string;\n name: string;\n criteriaIds: string[]; // Links to acceptance criteria\n};\n\nexport type CoverageMatrix = {\n criterion: AcceptanceCriterion;\n tests: TestCase[];\n covered: boolean;\n waiverReason?: string;\n};\n\nexport function buildCoverageMatrix(criteria: AcceptanceCriterion[], tests: TestCase[]): CoverageMatrix[] {\n return criteria.map((criterion) => {\n const matchingTests = tests.filter((t) => t.criteriaIds.includes(criterion.id));\n\n return {\n criterion,\n tests: matchingTests,\n covered: matchingTests.length > 0,\n };\n });\n}\n\nexport function validateCoverage(matrix: CoverageMatrix[]): {\n gaps: CoverageMatrix[];\n passRate: number;\n} {\n const gaps = matrix.filter((m) => !m.covered && !m.waiverReason);\n const passRate = ((matrix.length - gaps.length) / matrix.length) * 100;\n\n return { gaps, passRate };\n}\n\n// Example: Extract criteria IDs from test names\nexport function extractCriteriaFromTests(testFiles: string[]): TestCase[] {\n // Simplified: In real implementation, parse test files with AST\n // Here we simulate extraction from test names\n return [\n {\n file: 'tests/e2e/auth/login.spec.ts',\n name: 'should allow user to login with valid credentials',\n criteriaIds: ['AC-001', 'AC-002'], // Linked to acceptance criteria\n },\n {\n file: 'tests/e2e/auth/password-reset.spec.ts',\n name: 'should send password reset email',\n criteriaIds: ['AC-003'],\n },\n ];\n}\n\n// Generate Markdown traceability report\nexport function generateTraceabilityReport(matrix: CoverageMatrix[]): string {\n let report = `# Requirements-to-Tests Traceability Matrix\\n\\n`;\n report += `**Generated**: ${new Date().toISOString()}\\n\\n`;\n\n const { gaps, passRate } = validateCoverage(matrix);\n\n report += `## Summary\\n`;\n report += `- Total Criteria: ${matrix.length}\\n`;\n report += `- Covered: ${matrix.filter((m) => m.covered).length}\\n`;\n report += `- Gaps: ${gaps.length}\\n`;\n report += `- Waived: ${matrix.filter((m) => m.waiverReason).length}\\n`;\n report += `- Coverage Rate: ${passRate.toFixed(1)}%\\n\\n`;\n\n if (gaps.length > 0) {\n report += `## ❌ Coverage Gaps (MUST RESOLVE)\\n\\n`;\n report += `| Story | Criterion | Priority | Tests |\\n`;\n report += `|-------|-----------|----------|-------|\\n`;\n gaps.forEach((m) => {\n report += `| ${m.criterion.story} | ${m.criterion.criterion} | ${m.criterion.priority} | None |\\n`;\n });\n report += `\\n`;\n }\n\n report += `## ✅ Covered Criteria\\n\\n`;\n report += `| Story | Criterion | Tests |\\n`;\n report += `|-------|-----------|-------|\\n`;\n matrix\n .filter((m) => m.covered)\n .forEach((m) => {\n const testList = m.tests.map((t) => `\\`${t.file}\\``).join(', ');\n report += `| ${m.criterion.story} | ${m.criterion.criterion} | ${testList} |\\n`;\n });\n\n return report;\n}\n```\n\n**Usage Example**:\n\n```typescript\n// Define acceptance criteria\nconst criteria: AcceptanceCriterion[] = [\n { id: 'AC-001', story: 'US-123', criterion: 'User can login with email', priority: 'P0' },\n { id: 'AC-002', story: 'US-123', criterion: 'User sees error on invalid password', priority: 'P0' },\n { id: 'AC-003', story: 'US-124', criterion: 'User receives password reset email', priority: 'P1' },\n { id: 'AC-004', story: 'US-125', criterion: 'User can update profile', priority: 'P2' }, // NO TEST\n];\n\n// Extract tests\nconst tests: TestCase[] = extractCriteriaFromTests(['tests/e2e/auth/login.spec.ts', 'tests/e2e/auth/password-reset.spec.ts']);\n\n// Build matrix\nconst matrix = buildCoverageMatrix(criteria, tests);\n\n// Validate\nconst { gaps, passRate } = validateCoverage(matrix);\nconsole.log(`Coverage: ${passRate.toFixed(1)}%`); // \"Coverage: 75.0%\"\nconsole.log(`Gaps: ${gaps.length}`); // \"Gaps: 1\" (AC-004 has no test)\n\n// Generate report\nconst report = generateTraceabilityReport(matrix);\nconsole.log(report);\n// Markdown table showing coverage gaps\n```\n\n**Key Points**:\n\n- **Bidirectional traceability**: Criteria → Tests and Tests → Criteria\n- **Gap detection**: Automatically identifies missing coverage\n- **Priority awareness**: P0 gaps are critical blockers\n- **Waiver support**: Allow explicit waivers for low-priority gaps\n\n---\n\n## Risk Governance Checklist\n\nBefore deploying to production, ensure:\n\n- [ ] **Risk scoring complete**: All identified risks scored (Probability × Impact)\n- [ ] **Ownership assigned**: Every risk >4 has owner, mitigation plan, deadline\n- [ ] **Coverage validated**: Every acceptance criterion maps to at least one test\n- [ ] **Gate decision documented**: PASS/CONCERNS/FAIL/WAIVED with rationale\n- [ ] **Waivers approved**: All waivers have approver, reason, expiry date\n- [ ] **Audit trail captured**: Risk history log available for compliance review\n- [ ] **Traceability matrix**: Requirements-to-tests mapping up to date\n- [ ] **Critical risks resolved**: No score=9 risks in OPEN status\n\n## Integration Points\n\n- **Used in workflows**: `*trace` (Phase 2: gate decision), `*nfr-assess` (risk scoring), `*test-design` (risk identification)\n- **Related fragments**: `probability-impact.md` (scoring definitions), `test-priorities-matrix.md` (P0-P3 classification), `nfr-criteria.md` (non-functional risks)\n- **Tools**: Risk tracking dashboards (Jira, Linear), gate automation (CI/CD), traceability reports (Markdown, Confluence)\n\n_Source: Murat risk governance notes, gate schema guidance, enterprise production gate workflows, ISO 31000 risk management standards._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":21764,"content_sha256":"e697c39d6787a041fc8366d7f851eed6b081967bffa6d364b995d12d6cb754f7"},{"filename":"knowledge/selective-testing.md","content":"# Selective and Targeted Test Execution\n\n## Principle\n\nRun only the tests you need, when you need them. Use tags/grep to slice suites by risk priority (not directory structure), filter by spec patterns or git diff to focus on impacted areas, and combine priority metadata (P0-P3) with change detection to optimize pre-commit vs. CI execution. Document the selection strategy clearly so teams understand when full regression is mandatory.\n\n## Rationale\n\nRunning the entire test suite on every commit wastes time and resources. Smart test selection provides fast feedback (smoke tests in minutes, full regression in hours) while maintaining confidence. The \"32+ ways of selective testing\" philosophy balances speed with coverage: quick loops for developers, comprehensive validation before deployment. Poorly documented selection leads to confusion about when tests run and why.\n\n## Pattern Examples\n\n### Example 1: Tag-Based Execution with Priority Levels\n\n**Context**: Organize tests by risk priority and execution stage using grep/tag patterns.\n\n**Implementation**:\n\n```typescript\n// tests/e2e/checkout.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Tag-based test organization\n * - @smoke: Critical path tests (run on every commit, \u003c 5 min)\n * - @regression: Full test suite (run pre-merge, \u003c 30 min)\n * - @p0: Critical business functions (payment, auth, data integrity)\n * - @p1: Core features (primary user journeys)\n * - @p2: Secondary features (supporting functionality)\n * - @p3: Nice-to-have (cosmetic, non-critical)\n */\n\ntest.describe('Checkout Flow', () => {\n // P0 + Smoke: Must run on every commit\n test('@smoke @p0 should complete purchase with valid payment', async ({ page }) => {\n await page.goto('/checkout');\n await page.getByTestId('card-number').fill('4242424242424242');\n await page.getByTestId('submit-payment').click();\n\n await expect(page.getByTestId('order-confirmation')).toBeVisible();\n });\n\n // P0 but not smoke: Run pre-merge\n test('@regression @p0 should handle payment decline gracefully', async ({ page }) => {\n await page.goto('/checkout');\n await page.getByTestId('card-number').fill('4000000000000002'); // Decline card\n await page.getByTestId('submit-payment').click();\n\n await expect(page.getByTestId('payment-error')).toBeVisible();\n await expect(page.getByTestId('payment-error')).toContainText('declined');\n });\n\n // P1 + Smoke: Important but not critical\n test('@smoke @p1 should apply discount code', async ({ page }) => {\n await page.goto('/checkout');\n await page.getByTestId('promo-code').fill('SAVE10');\n await page.getByTestId('apply-promo').click();\n\n await expect(page.getByTestId('discount-applied')).toBeVisible();\n });\n\n // P2: Run in full regression only\n test('@regression @p2 should remember saved payment methods', async ({ page }) => {\n await page.goto('/checkout');\n await expect(page.getByTestId('saved-cards')).toBeVisible();\n });\n\n // P3: Low priority, run nightly or weekly\n test('@nightly @p3 should display checkout page analytics', async ({ page }) => {\n await page.goto('/checkout');\n const analyticsEvents = await page.evaluate(() => (window as any).__ANALYTICS__);\n expect(analyticsEvents).toBeDefined();\n });\n});\n```\n\n**package.json scripts**:\n\n```json\n{\n \"scripts\": {\n \"test\": \"playwright test\",\n \"test:smoke\": \"playwright test --grep '@smoke'\",\n \"test:p0\": \"playwright test --grep '@p0'\",\n \"test:p0-p1\": \"playwright test --grep '@p0|@p1'\",\n \"test:regression\": \"playwright test --grep '@regression'\",\n \"test:nightly\": \"playwright test --grep '@nightly'\",\n \"test:not-slow\": \"playwright test --grep-invert '@slow'\",\n \"test:critical-smoke\": \"playwright test --grep '@smoke.*@p0'\"\n }\n}\n```\n\n**Cypress equivalent**:\n\n```javascript\n// cypress/e2e/checkout.cy.ts\ndescribe('Checkout Flow', { tags: ['@checkout'] }, () => {\n it('should complete purchase', { tags: ['@smoke', '@p0'] }, () => {\n cy.visit('/checkout');\n cy.get('[data-cy=\"card-number\"]').type('4242424242424242');\n cy.get('[data-cy=\"submit-payment\"]').click();\n cy.get('[data-cy=\"order-confirmation\"]').should('be.visible');\n });\n\n it('should handle decline', { tags: ['@regression', '@p0'] }, () => {\n cy.visit('/checkout');\n cy.get('[data-cy=\"card-number\"]').type('4000000000000002');\n cy.get('[data-cy=\"submit-payment\"]').click();\n cy.get('[data-cy=\"payment-error\"]').should('be.visible');\n });\n});\n\n// cypress.config.ts\nexport default defineConfig({\n e2e: {\n env: {\n grepTags: process.env.GREP_TAGS || '',\n grepFilterSpecs: true,\n },\n setupNodeEvents(on, config) {\n require('@cypress/grep/src/plugin')(config);\n return config;\n },\n },\n});\n```\n\n**Usage**:\n\n```bash\n# Playwright\nnpm run test:smoke # Run all @smoke tests\nnpm run test:p0 # Run all P0 tests\nnpm run test -- --grep \"@smoke.*@p0\" # Run tests with BOTH tags\n\n# Cypress (with @cypress/grep plugin)\nnpx cypress run --env grepTags=\"@smoke\"\nnpx cypress run --env grepTags=\"@p0+@smoke\" # AND logic\nnpx cypress run --env grepTags=\"@p0 @p1\" # OR logic\n```\n\n**Key Points**:\n\n- **Multiple tags per test**: Combine priority (@p0) with stage (@smoke)\n- **AND/OR logic**: Grep supports complex filtering\n- **Clear naming**: Tags document test importance\n- **Fast feedback**: @smoke runs \u003c 5 min, full suite \u003c 30 min\n- **CI integration**: Different jobs run different tag combinations\n\n---\n\n### Example 2: Spec Filter Pattern (File-Based Selection)\n\n**Context**: Run tests by file path pattern or directory for targeted execution.\n\n**Implementation**:\n\n```bash\n#!/bin/bash\n# scripts/selective-spec-runner.sh\n# Run tests based on spec file patterns\n\nset -e\n\nPATTERN=${1:-\"**/*.spec.ts\"}\nTEST_ENV=${TEST_ENV:-local}\n\necho \"🎯 Selective Spec Runner\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"Pattern: $PATTERN\"\necho \"Environment: $TEST_ENV\"\necho \"\"\n\n# Pattern examples and their use cases\ncase \"$PATTERN\" in\n \"**/checkout*\")\n echo \"📦 Running checkout-related tests\"\n npx playwright test --grep-files=\"**/checkout*\"\n ;;\n \"**/auth*\"|\"**/login*\"|\"**/signup*\")\n echo \"🔐 Running authentication tests\"\n npx playwright test --grep-files=\"**/auth*|**/login*|**/signup*\"\n ;;\n \"tests/e2e/**\")\n echo \"🌐 Running all E2E tests\"\n npx playwright test tests/e2e/\n ;;\n \"tests/integration/**\")\n echo \"🔌 Running all integration tests\"\n npx playwright test tests/integration/\n ;;\n \"tests/component/**\")\n echo \"🧩 Running all component tests\"\n npx playwright test tests/component/\n ;;\n *)\n echo \"🔍 Running tests matching pattern: $PATTERN\"\n npx playwright test \"$PATTERN\"\n ;;\nesac\n```\n\n**Playwright config for file filtering**:\n\n```typescript\n// playwright.config.ts\nimport { defineConfig, devices } from '@playwright/test';\n\nexport default defineConfig({\n // ... other config\n\n // Project-based organization\n projects: [\n {\n name: 'smoke',\n testMatch: /.*smoke.*\\.spec\\.ts/,\n retries: 0,\n },\n {\n name: 'e2e',\n testMatch: /tests\\/e2e\\/.*\\.spec\\.ts/,\n retries: 2,\n },\n {\n name: 'integration',\n testMatch: /tests\\/integration\\/.*\\.spec\\.ts/,\n retries: 1,\n },\n {\n name: 'component',\n testMatch: /tests\\/component\\/.*\\.spec\\.ts/,\n use: { ...devices['Desktop Chrome'] },\n },\n ],\n});\n```\n\n**Advanced pattern matching**:\n\n```typescript\n// scripts/run-by-component.ts\n/**\n * Run tests related to specific component(s)\n * Usage: npm run test:component UserProfile,Settings\n */\n\nimport { execSync } from 'child_process';\n\nconst components = process.argv[2]?.split(',') || [];\n\nif (components.length === 0) {\n console.error('❌ No components specified');\n console.log('Usage: npm run test:component UserProfile,Settings');\n process.exit(1);\n}\n\n// Convert component names to glob patterns\nconst patterns = components.map((comp) => `**/*${comp}*.spec.ts`).join(' ');\n\nconsole.log(`🧩 Running tests for components: ${components.join(', ')}`);\nconsole.log(`Patterns: ${patterns}`);\n\ntry {\n execSync(`npx playwright test ${patterns}`, {\n stdio: 'inherit',\n env: { ...process.env, CI: 'false' },\n });\n} catch (error) {\n process.exit(1);\n}\n```\n\n**package.json scripts**:\n\n```json\n{\n \"scripts\": {\n \"test:checkout\": \"playwright test **/checkout*.spec.ts\",\n \"test:auth\": \"playwright test **/auth*.spec.ts **/login*.spec.ts\",\n \"test:e2e\": \"playwright test tests/e2e/\",\n \"test:integration\": \"playwright test tests/integration/\",\n \"test:component\": \"ts-node scripts/run-by-component.ts\",\n \"test:project\": \"playwright test --project\",\n \"test:smoke-project\": \"playwright test --project smoke\"\n }\n}\n```\n\n**Key Points**:\n\n- **Glob patterns**: Wildcards match file paths flexibly\n- **Project isolation**: Separate projects have different configs\n- **Component targeting**: Run tests for specific features\n- **Directory-based**: Organize tests by type (e2e, integration, component)\n- **CI optimization**: Run subsets in parallel CI jobs\n\n---\n\n### Example 3: Diff-Based Test Selection (Changed Files Only)\n\n**Context**: Run only tests affected by code changes for maximum speed.\n\n**Implementation**:\n\n```bash\n#!/bin/bash\n# scripts/test-changed-files.sh\n# Intelligent test selection based on git diff\n\nset -e\n\nBASE_BRANCH=${BASE_BRANCH:-main}\nTEST_ENV=${TEST_ENV:-local}\n\necho \"🔍 Changed File Test Selector\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"Base branch: $BASE_BRANCH\"\necho \"Environment: $TEST_ENV\"\necho \"\"\n\n# Get changed files\nCHANGED_FILES=$(git diff --name-only $BASE_BRANCH...HEAD)\n\nif [ -z \"$CHANGED_FILES\" ]; then\n echo \"✅ No files changed. Skipping tests.\"\n exit 0\nfi\n\necho \"Changed files:\"\necho \"$CHANGED_FILES\" | sed 's/^/ - /'\necho \"\"\n\n# Arrays to collect test specs\nDIRECT_TEST_FILES=()\nRELATED_TEST_FILES=()\nRUN_ALL_TESTS=false\n\n# Process each changed file\nwhile IFS= read -r file; do\n case \"$file\" in\n # Changed test files: run them directly\n *.spec.ts|*.spec.js|*.test.ts|*.test.js|*.cy.ts|*.cy.js)\n DIRECT_TEST_FILES+=(\"$file\")\n ;;\n\n # Critical config changes: run ALL tests\n package.json|package-lock.json|playwright.config.ts|cypress.config.ts|tsconfig.json|.github/workflows/*)\n echo \"⚠️ Critical file changed: $file\"\n RUN_ALL_TESTS=true\n break\n ;;\n\n # Component changes: find related tests\n src/components/*.tsx|src/components/*.jsx)\n COMPONENT_NAME=$(basename \"$file\" | sed 's/\\.[^.]*$//')\n echo \"🧩 Component changed: $COMPONENT_NAME\"\n\n # Find tests matching component name\n FOUND_TESTS=$(find tests -name \"*${COMPONENT_NAME}*.spec.ts\" -o -name \"*${COMPONENT_NAME}*.cy.ts\" 2>/dev/null || true)\n if [ -n \"$FOUND_TESTS\" ]; then\n while IFS= read -r test_file; do\n RELATED_TEST_FILES+=(\"$test_file\")\n done \u003c\u003c\u003c \"$FOUND_TESTS\"\n fi\n ;;\n\n # Utility/lib changes: run integration + unit tests\n src/utils/*|src/lib/*|src/helpers/*)\n echo \"⚙️ Utility file changed: $file\"\n RELATED_TEST_FILES+=($(find tests/unit tests/integration -name \"*.spec.ts\" 2>/dev/null || true))\n ;;\n\n # API changes: run integration + e2e tests\n src/api/*|src/services/*|src/controllers/*)\n echo \"🔌 API file changed: $file\"\n RELATED_TEST_FILES+=($(find tests/integration tests/e2e -name \"*.spec.ts\" 2>/dev/null || true))\n ;;\n\n # Type changes: run all TypeScript tests\n *.d.ts|src/types/*)\n echo \"📝 Type definition changed: $file\"\n RUN_ALL_TESTS=true\n break\n ;;\n\n # Documentation only: skip tests\n *.md|docs/*|README*)\n echo \"📄 Documentation changed: $file (no tests needed)\"\n ;;\n\n *)\n echo \"❓ Unclassified change: $file (running smoke tests)\"\n RELATED_TEST_FILES+=($(find tests -name \"*smoke*.spec.ts\" 2>/dev/null || true))\n ;;\n esac\ndone \u003c\u003c\u003c \"$CHANGED_FILES\"\n\n# Execute tests based on analysis\nif [ \"$RUN_ALL_TESTS\" = true ]; then\n echo \"\"\n echo \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\n echo \"🚨 Running FULL test suite (critical changes detected)\"\n echo \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\n npm run test\n exit $?\nfi\n\n# Combine and deduplicate test files\nALL_TEST_FILES=(${DIRECT_TEST_FILES[@]} ${RELATED_TEST_FILES[@]})\nUNIQUE_TEST_FILES=($(echo \"${ALL_TEST_FILES[@]}\" | tr ' ' '\\n' | sort -u))\n\nif [ ${#UNIQUE_TEST_FILES[@]} -eq 0 ]; then\n echo \"\"\n echo \"✅ No tests found for changed files. Running smoke tests.\"\n npm run test:smoke\n exit $?\nfi\n\necho \"\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\necho \"🎯 Running ${#UNIQUE_TEST_FILES[@]} test file(s)\"\necho \"━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\"\n\nfor test_file in \"${UNIQUE_TEST_FILES[@]}\"; do\n echo \" - $test_file\"\ndone\n\necho \"\"\nnpm run test -- \"${UNIQUE_TEST_FILES[@]}\"\n```\n\n**GitHub Actions integration**:\n\n```yaml\n# .github/workflows/test-changed.yml\nname: Test Changed Files\non:\n pull_request:\n types: [opened, synchronize, reopened]\n\njobs:\n detect-and-test:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n with:\n fetch-depth: 0 # Full history for accurate diff\n\n - name: Get changed files\n id: changed-files\n uses: tj-actions/changed-files@v40\n with:\n files: |\n src/**\n tests/**\n *.config.ts\n files_ignore: |\n **/*.md\n docs/**\n\n - name: Run tests for changed files\n if: steps.changed-files.outputs.any_changed == 'true'\n run: |\n echo \"Changed files: ${{ steps.changed-files.outputs.all_changed_files }}\"\n bash scripts/test-changed-files.sh\n env:\n BASE_BRANCH: ${{ github.base_ref }}\n TEST_ENV: staging\n```\n\n**Key Points**:\n\n- **Intelligent mapping**: Code changes → related tests\n- **Critical file detection**: Config changes = full suite\n- **Component mapping**: UI changes → component + E2E tests\n- **Fast feedback**: Run only what's needed (\u003c 2 min typical)\n- **Safety net**: Unrecognized changes run smoke tests\n\n---\n\n### Example 4: Promotion Rules (Pre-Commit → CI → Staging → Production)\n\n**Context**: Progressive test execution strategy across deployment stages.\n\n**Implementation**:\n\n```typescript\n// scripts/test-promotion-strategy.ts\n/**\n * Test Promotion Strategy\n * Defines which tests run at each stage of the development lifecycle\n */\n\nexport type TestStage = 'pre-commit' | 'ci-pr' | 'ci-merge' | 'staging' | 'production';\n\nexport type TestPromotion = {\n stage: TestStage;\n description: string;\n testCommand: string;\n timebudget: string; // minutes\n required: boolean;\n failureAction: 'block' | 'warn' | 'alert';\n};\n\nexport const TEST_PROMOTION_RULES: Record\u003cTestStage, TestPromotion> = {\n 'pre-commit': {\n stage: 'pre-commit',\n description: 'Local developer checks before git commit',\n testCommand: 'npm run test:smoke',\n timebudget: '2',\n required: true,\n failureAction: 'block',\n },\n 'ci-pr': {\n stage: 'ci-pr',\n description: 'CI checks on pull request creation/update',\n testCommand: 'npm run test:changed && npm run test:p0-p1',\n timebudget: '10',\n required: true,\n failureAction: 'block',\n },\n 'ci-merge': {\n stage: 'ci-merge',\n description: 'Full regression before merge to main',\n testCommand: 'npm run test:regression',\n timebudget: '30',\n required: true,\n failureAction: 'block',\n },\n staging: {\n stage: 'staging',\n description: 'Post-deployment validation in staging environment',\n testCommand: 'npm run test:e2e -- --grep \"@smoke\"',\n timebudget: '15',\n required: true,\n failureAction: 'block',\n },\n production: {\n stage: 'production',\n description: 'Production smoke tests post-deployment',\n testCommand: 'npm run test:e2e:prod -- --grep \"@smoke.*@p0\"',\n timebudget: '5',\n required: false,\n failureAction: 'alert',\n },\n};\n\n/**\n * Get tests to run for a specific stage\n */\nexport function getTestsForStage(stage: TestStage): TestPromotion {\n return TEST_PROMOTION_RULES[stage];\n}\n\n/**\n * Validate if tests can be promoted to next stage\n */\nexport function canPromote(currentStage: TestStage, testsPassed: boolean): boolean {\n const promotion = TEST_PROMOTION_RULES[currentStage];\n\n if (!promotion.required) {\n return true; // Non-required tests don't block promotion\n }\n\n return testsPassed;\n}\n```\n\n**Husky pre-commit hook**:\n\n```bash\n#!/bin/bash\n# .husky/pre-commit\n# Run smoke tests before allowing commit\n\necho \"🔍 Running pre-commit tests...\"\n\nnpm run test:smoke\n\nif [ $? -ne 0 ]; then\n echo \"\"\n echo \"❌ Pre-commit tests failed!\"\n echo \"Please fix failures before committing.\"\n echo \"\"\n echo \"To skip (NOT recommended): git commit --no-verify\"\n exit 1\nfi\n\necho \"✅ Pre-commit tests passed\"\n```\n\n**GitHub Actions workflow**:\n\n```yaml\n# .github/workflows/test-promotion.yml\nname: Test Promotion Strategy\non:\n pull_request:\n push:\n branches: [main]\n workflow_dispatch:\n\njobs:\n # Stage 1: PR tests (changed + P0-P1)\n pr-tests:\n if: github.event_name == 'pull_request'\n runs-on: ubuntu-latest\n timeout-minutes: 10\n steps:\n - uses: actions/checkout@v4\n - name: Run PR-level tests\n run: |\n npm run test:changed\n npm run test:p0-p1\n\n # Stage 2: Full regression (pre-merge)\n regression-tests:\n if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n runs-on: ubuntu-latest\n timeout-minutes: 30\n steps:\n - uses: actions/checkout@v4\n - name: Run full regression\n run: npm run test:regression\n\n # Stage 3: Staging validation (post-deploy)\n staging-smoke:\n if: github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 15\n steps:\n - uses: actions/checkout@v4\n - name: Run staging smoke tests\n run: npm run test:e2e -- --grep \"@smoke\"\n env:\n TEST_ENV: staging\n\n # Stage 4: Production smoke (post-deploy, non-blocking)\n production-smoke:\n if: github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-latest\n timeout-minutes: 5\n continue-on-error: true # Don't fail deployment if smoke tests fail\n steps:\n - uses: actions/checkout@v4\n - name: Run production smoke tests\n run: npm run test:e2e:prod -- --grep \"@smoke.*@p0\"\n env:\n TEST_ENV: production\n\n - name: Alert on failure\n if: failure()\n uses: 8398a7/action-slack@v3\n with:\n status: ${{ job.status }}\n text: '🚨 Production smoke tests failed!'\n webhook_url: ${{ secrets.SLACK_WEBHOOK }}\n```\n\n**Selection strategy documentation**:\n\n````markdown\n# Test Selection Strategy\n\n## Test Promotion Stages\n\n| Stage | Tests Run | Time Budget | Blocks Deploy | Failure Action |\n| ---------- | ------------------- | ----------- | ------------- | -------------- |\n| Pre-Commit | Smoke (@smoke) | 2 min | ✅ Yes | Block commit |\n| CI PR | Changed + P0-P1 | 10 min | ✅ Yes | Block merge |\n| CI Merge | Full regression | 30 min | ✅ Yes | Block deploy |\n| Staging | E2E smoke | 15 min | ✅ Yes | Rollback |\n| Production | Critical smoke only | 5 min | ❌ No | Alert team |\n\n## When Full Regression Runs\n\nFull regression suite (`npm run test:regression`) runs in these scenarios:\n\n- ✅ Before merging to `main` (CI Merge stage)\n- ✅ Nightly builds (scheduled workflow)\n- ✅ Manual trigger (workflow_dispatch)\n- ✅ Release candidate testing\n\nFull regression does NOT run on:\n\n- ❌ Every PR commit (too slow)\n- ❌ Pre-commit hooks (too slow)\n- ❌ Production deployments (deploy-blocking)\n\n## Override Scenarios\n\nSkip tests (emergency only):\n\n```bash\ngit commit --no-verify # Skip pre-commit hook\ngh pr merge --admin # Force merge (requires admin)\n```\n````\n\n```text\n\n**Key Points**:\n- **Progressive validation**: More tests at each stage\n- **Time budgets**: Clear expectations per stage\n- **Blocking vs. alerting**: Production tests don't block deploy\n- **Documentation**: Team knows when full regression runs\n- **Emergency overrides**: Documented but discouraged\n\n---\n\n## Test Selection Strategy Checklist\n\nBefore implementing selective testing, verify:\n\n- [ ] **Tag strategy defined**: @smoke, @p0-p3, @regression documented\n- [ ] **Time budgets set**: Each stage has clear timeout (smoke \u003c 5 min, full \u003c 30 min)\n- [ ] **Changed file mapping**: Code changes → test selection logic implemented\n- [ ] **Promotion rules documented**: README explains when full regression runs\n- [ ] **CI integration**: GitHub Actions uses selective strategy\n- [ ] **Local parity**: Developers can run same selections locally\n- [ ] **Emergency overrides**: Skip mechanisms documented (--no-verify, admin merge)\n- [ ] **Metrics tracked**: Monitor test execution time and selection accuracy\n\n## Integration Points\n\n- Used in workflows: `*ci` (CI/CD setup), `*automate` (test generation with tags)\n- Related fragments: `ci-burn-in.md`, `test-priorities-matrix.md`, `test-quality.md`\n- Selection tools: Playwright --grep, Cypress @cypress/grep, git diff\n\n_Source: 32+ selective testing strategies blog, Murat testing philosophy, enterprise CI optimization_\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":21850,"content_sha256":"99f72ba9117ecfdc8be51346b12d610bfd6e48ea6b6310575c7be7d0766cb5fd"},{"filename":"knowledge/selector-resilience.md","content":"# Selector Resilience\n\n## Principle\n\nRobust selectors follow a strict hierarchy: **data-testid > ARIA roles > text content > CSS/IDs** (last resort). Selectors must be resilient to UI changes (styling, layout, content updates) and remain human-readable for maintenance.\n\n## Rationale\n\n**The Problem**: Brittle selectors (CSS classes, nth-child, complex XPath) break when UI styling changes, elements are reordered, or design updates occur. This causes test maintenance burden and false negatives.\n\n**The Solution**: Prioritize semantic selectors that reflect user intent (ARIA roles, accessible names, test IDs). Use dynamic filtering for lists instead of nth() indexes. Validate selectors during code review and refactor proactively.\n\n**Why This Matters**:\n\n- Prevents false test failures (UI refactoring doesn't break tests)\n- Improves accessibility (ARIA roles benefit both tests and screen readers)\n- Enhances readability (semantic selectors document user intent)\n- Reduces maintenance burden (robust selectors survive design changes)\n\n## Pattern Examples\n\n### Example 1: Selector Hierarchy (Priority Order with Examples)\n\n**Context**: Choose the most resilient selector for each element type\n\n**Implementation**:\n\n```typescript\n// tests/selectors/hierarchy-examples.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Selector Hierarchy Best Practices', () => {\n test('Level 1: data-testid (BEST - most resilient)', async ({ page }) => {\n await page.goto('/login');\n\n // ✅ Best: Dedicated test attribute (survives all UI changes)\n await page.getByTestId('email-input').fill('[email protected]');\n await page.getByTestId('password-input').fill('password123');\n await page.getByTestId('login-button').click();\n\n await expect(page.getByTestId('welcome-message')).toBeVisible();\n\n // Why it's best:\n // - Survives CSS refactoring (class name changes)\n // - Survives layout changes (element reordering)\n // - Survives content changes (button text updates)\n // - Explicit test contract (developer knows it's for testing)\n });\n\n test('Level 2: ARIA roles and accessible names (GOOD - future-proof)', async ({ page }) => {\n await page.goto('/login');\n\n // ✅ Good: Semantic HTML roles (benefits accessibility + tests)\n await page.getByRole('textbox', { name: 'Email' }).fill('[email protected]');\n await page.getByRole('textbox', { name: 'Password' }).fill('password123');\n await page.getByRole('button', { name: 'Sign In' }).click();\n\n await expect(page.getByRole('heading', { name: 'Welcome' })).toBeVisible();\n\n // Why it's good:\n // - Survives CSS refactoring\n // - Survives layout changes\n // - Enforces accessibility (screen reader compatible)\n // - Self-documenting (role + name = clear intent)\n });\n\n test('Level 3: Text content (ACCEPTABLE - user-centric)', async ({ page }) => {\n await page.goto('/dashboard');\n\n // ✅ Acceptable: Text content (matches user perception)\n await page.getByText('Create New Order').click();\n await expect(page.getByText('Order Details')).toBeVisible();\n\n // Why it's acceptable:\n // - User-centric (what user sees)\n // - Survives CSS/layout changes\n // - Breaks when copy changes (forces test update with content)\n\n // ⚠️ Use with caution for dynamic/localized content:\n // - Avoid for content with variables: \"User 123\" (use regex instead)\n // - Avoid for i18n content (use data-testid or ARIA)\n });\n\n test('Level 4: CSS classes/IDs (LAST RESORT - brittle)', async ({ page }) => {\n await page.goto('/login');\n\n // ❌ Last resort: CSS class (breaks with styling updates)\n // await page.locator('.btn-primary').click()\n\n // ❌ Last resort: ID (breaks if ID changes)\n // await page.locator('#login-form').fill(...)\n\n // ✅ Better: Use data-testid or ARIA instead\n await page.getByTestId('login-button').click();\n\n // Why CSS/ID is last resort:\n // - Breaks with CSS refactoring (class name changes)\n // - Breaks with HTML restructuring (ID changes)\n // - Not semantic (unclear what element does)\n // - Tight coupling between tests and styling\n });\n});\n```\n\n**Key Points**:\n\n- Hierarchy: data-testid (best) > ARIA (good) > text (acceptable) > CSS/ID (last resort)\n- data-testid survives ALL UI changes (explicit test contract)\n- ARIA roles enforce accessibility (screen reader compatible)\n- Text content is user-centric (but breaks with copy changes)\n- CSS/ID are brittle (break with styling refactoring)\n\n---\n\n### Example 2: Dynamic Selector Patterns (Lists, Filters, Regex)\n\n**Context**: Handle dynamic content, lists, and variable data with resilient selectors\n\n**Implementation**:\n\n```typescript\n// tests/selectors/dynamic-selectors.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Dynamic Selector Patterns', () => {\n test('regex for variable content (user IDs, timestamps)', async ({ page }) => {\n await page.goto('/users');\n\n // ✅ Good: Regex pattern for dynamic user IDs\n await expect(page.getByText(/User \\d+/)).toBeVisible();\n\n // ✅ Good: Regex for timestamps\n await expect(page.getByText(/Last login: \\d{4}-\\d{2}-\\d{2}/)).toBeVisible();\n\n // ✅ Good: Regex for dynamic counts\n await expect(page.getByText(/\\d+ items in cart/)).toBeVisible();\n });\n\n test('partial text matching (case-insensitive, substring)', async ({ page }) => {\n await page.goto('/products');\n\n // ✅ Good: Partial match (survives minor text changes)\n await page.getByText('Product', { exact: false }).first().click();\n\n // ✅ Good: Case-insensitive (survives capitalization changes)\n await expect(page.getByText(/sign in/i)).toBeVisible();\n });\n\n test('filter locators for lists (avoid brittle nth)', async ({ page }) => {\n await page.goto('/products');\n\n // ❌ Bad: Index-based (breaks when order changes)\n // await page.locator('.product-card').nth(2).click()\n\n // ✅ Good: Filter by content (resilient to reordering)\n await page.locator('[data-testid=\"product-card\"]').filter({ hasText: 'Premium Plan' }).click();\n\n // ✅ Good: Filter by attribute\n await page\n .locator('[data-testid=\"product-card\"]')\n .filter({ has: page.locator('[data-status=\"active\"]') })\n .first()\n .click();\n });\n\n test('nth() only when absolutely necessary', async ({ page }) => {\n await page.goto('/dashboard');\n\n // ⚠️ Acceptable: nth(0) for first item (common pattern)\n const firstNotification = page.getByTestId('notification').nth(0);\n await expect(firstNotification).toContainText('Welcome');\n\n // ❌ Bad: nth(5) for arbitrary index (fragile)\n // await page.getByTestId('notification').nth(5).click()\n\n // ✅ Better: Use filter() with specific criteria\n await page.getByTestId('notification').filter({ hasText: 'Critical Alert' }).click();\n });\n\n test('combine multiple locators for specificity', async ({ page }) => {\n await page.goto('/checkout');\n\n // ✅ Good: Narrow scope with combined locators\n const shippingSection = page.getByTestId('shipping-section');\n await shippingSection.getByLabel('Address Line 1').fill('123 Main St');\n await shippingSection.getByLabel('City').fill('New York');\n\n // Scoping prevents ambiguity (multiple \"City\" fields on page)\n });\n});\n```\n\n**Key Points**:\n\n- Regex patterns handle variable content (IDs, timestamps, counts)\n- Partial matching survives minor text changes (`exact: false`)\n- `filter()` is more resilient than `nth()` (content-based vs index-based)\n- `nth(0)` acceptable for \"first item\", avoid arbitrary indexes\n- Combine locators to narrow scope (prevent ambiguity)\n\n---\n\n### Example 3: Selector Anti-Patterns (What NOT to Do)\n\n**Context**: Common selector mistakes that cause brittle tests\n\n**Problem Examples**:\n\n```typescript\n// tests/selectors/anti-patterns.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Selector Anti-Patterns to Avoid', () => {\n test('❌ Anti-Pattern 1: CSS classes (brittle)', async ({ page }) => {\n await page.goto('/login');\n\n // ❌ Bad: CSS class (breaks with design system updates)\n // await page.locator('.btn-primary').click()\n // await page.locator('.form-input-lg').fill('[email protected]')\n\n // ✅ Good: Use data-testid or ARIA role\n await page.getByTestId('login-button').click();\n await page.getByRole('textbox', { name: 'Email' }).fill('[email protected]');\n });\n\n test('❌ Anti-Pattern 2: Index-based nth() (fragile)', async ({ page }) => {\n await page.goto('/products');\n\n // ❌ Bad: Index-based (breaks when product order changes)\n // await page.locator('.product-card').nth(3).click()\n\n // ✅ Good: Content-based filter\n await page.locator('[data-testid=\"product-card\"]').filter({ hasText: 'Laptop' }).click();\n });\n\n test('❌ Anti-Pattern 3: Complex XPath (hard to maintain)', async ({ page }) => {\n await page.goto('/dashboard');\n\n // ❌ Bad: Complex XPath (unreadable, breaks with structure changes)\n // await page.locator('xpath=//div[@class=\"container\"]//section[2]//button[contains(@class, \"primary\")]').click()\n\n // ✅ Good: Semantic selector\n await page.getByRole('button', { name: 'Create Order' }).click();\n });\n\n test('❌ Anti-Pattern 4: ID selectors (coupled to implementation)', async ({ page }) => {\n await page.goto('/settings');\n\n // ❌ Bad: HTML ID (breaks if ID changes for accessibility/SEO)\n // await page.locator('#user-settings-form').fill(...)\n\n // ✅ Good: data-testid or ARIA landmark\n await page.getByTestId('user-settings-form').getByLabel('Display Name').fill('John Doe');\n });\n\n test('✅ Refactoring: Bad → Good Selector', async ({ page }) => {\n await page.goto('/checkout');\n\n // Before (brittle):\n // await page.locator('.checkout-form > .payment-section > .btn-submit').click()\n\n // After (resilient):\n await page.getByTestId('checkout-form').getByRole('button', { name: 'Complete Payment' }).click();\n\n await expect(page.getByText('Payment successful')).toBeVisible();\n });\n});\n```\n\n**Why These Fail**:\n\n- **CSS classes**: Change frequently with design updates (Tailwind, CSS modules)\n- **nth() indexes**: Fragile to element reordering (new features, A/B tests)\n- **Complex XPath**: Unreadable, breaks with HTML structure changes\n- **HTML IDs**: Not stable (accessibility improvements change IDs)\n\n**Better Approach**: Use selector hierarchy (testid > ARIA > text)\n\n---\n\n### Example 4: Selector Debugging Techniques (Inspector, DevTools, MCP)\n\n**Context**: Debug selector failures interactively to find better alternatives\n\n**Implementation**:\n\n```typescript\n// tests/selectors/debugging-techniques.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Selector Debugging Techniques', () => {\n test('use Playwright Inspector to test selectors', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Pause test to open Inspector\n await page.pause();\n\n // In Inspector console, test selectors:\n // page.getByTestId('user-menu') ✅ Works\n // page.getByRole('button', { name: 'Profile' }) ✅ Works\n // page.locator('.btn-primary') ❌ Brittle\n\n // Use \"Pick Locator\" feature to generate selectors\n // Use \"Record\" mode to capture user interactions\n\n await page.getByTestId('user-menu').click();\n await expect(page.getByRole('menu')).toBeVisible();\n });\n\n test('use locator.all() to debug lists', async ({ page }) => {\n await page.goto('/products');\n\n // Debug: How many products are visible?\n const products = await page.getByTestId('product-card').all();\n console.log(`Found ${products.length} products`);\n\n // Debug: What text is in each product?\n for (const product of products) {\n const text = await product.textContent();\n console.log(`Product text: ${text}`);\n }\n\n // Use findings to build better selector\n await page.getByTestId('product-card').filter({ hasText: 'Laptop' }).click();\n });\n\n test('use DevTools console to test selectors', async ({ page }) => {\n await page.goto('/checkout');\n\n // Open DevTools (manually or via page.pause())\n // Test selectors in console:\n // document.querySelectorAll('[data-testid=\"payment-method\"]')\n // document.querySelector('#credit-card-input')\n\n // Find robust selector through trial and error\n await page.getByTestId('payment-method').selectOption('credit-card');\n });\n\n test('MCP browser_generate_locator (if available)', async ({ page }) => {\n await page.goto('/products');\n\n // If Playwright MCP available, use browser_generate_locator:\n // 1. Click element in browser\n // 2. MCP generates optimal selector\n // 3. Copy into test\n\n // Example output from MCP:\n // page.getByRole('link', { name: 'Product A' })\n\n // Use generated selector\n await page.getByRole('link', { name: 'Product A' }).click();\n await expect(page).toHaveURL(/\\/products\\/\\d+/);\n });\n});\n```\n\n**Key Points**:\n\n- Playwright Inspector: Interactive selector testing with \"Pick Locator\" feature\n- `locator.all()`: Debug lists to understand structure and content\n- DevTools console: Test CSS selectors before adding to tests\n- MCP browser_generate_locator: Auto-generate optimal selectors (if MCP available)\n- Always validate selectors work before committing\n\n---\n\n### Example 2: Selector Refactoring Guide (Before/After Patterns)\n\n**Context**: Systematically improve brittle selectors to resilient alternatives\n\n**Implementation**:\n\n```typescript\n// tests/selectors/refactoring-guide.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Selector Refactoring Patterns', () => {\n test('refactor: CSS class → data-testid', async ({ page }) => {\n await page.goto('/products');\n\n // ❌ Before: CSS class (breaks with Tailwind updates)\n // await page.locator('.bg-blue-500.px-4.py-2.rounded').click()\n\n // ✅ After: data-testid\n await page.getByTestId('add-to-cart-button').click();\n\n // Implementation: Add data-testid to button component\n // \u003cbutton className=\"bg-blue-500 px-4 py-2 rounded\" data-testid=\"add-to-cart-button\">\n });\n\n test('refactor: nth() index → filter()', async ({ page }) => {\n await page.goto('/users');\n\n // ❌ Before: Index-based (breaks when users reorder)\n // await page.locator('.user-row').nth(2).click()\n\n // ✅ After: Content-based filter\n await page.locator('[data-testid=\"user-row\"]').filter({ hasText: '[email protected]' }).click();\n });\n\n test('refactor: Complex XPath → ARIA role', async ({ page }) => {\n await page.goto('/checkout');\n\n // ❌ Before: Complex XPath (unreadable, brittle)\n // await page.locator('xpath=//div[@id=\"payment\"]//form//button[contains(@class, \"submit\")]').click()\n\n // ✅ After: ARIA role\n await page.getByRole('button', { name: 'Complete Payment' }).click();\n });\n\n test('refactor: ID selector → data-testid', async ({ page }) => {\n await page.goto('/settings');\n\n // ❌ Before: HTML ID (changes with accessibility improvements)\n // await page.locator('#user-profile-section').getByLabel('Name').fill('John')\n\n // ✅ After: data-testid + semantic label\n await page.getByTestId('user-profile-section').getByLabel('Display Name').fill('John Doe');\n });\n\n test('refactor: Deeply nested CSS → scoped data-testid', async ({ page }) => {\n await page.goto('/dashboard');\n\n // ❌ Before: Deep nesting (breaks with structure changes)\n // await page.locator('.container .sidebar .menu .item:nth-child(3) a').click()\n\n // ✅ After: Scoped data-testid\n const sidebar = page.getByTestId('sidebar');\n await sidebar.getByRole('link', { name: 'Settings' }).click();\n });\n});\n```\n\n**Key Points**:\n\n- CSS class → data-testid (survives design system updates)\n- nth() → filter() (content-based vs index-based)\n- Complex XPath → ARIA role (readable, semantic)\n- ID → data-testid (decouples from HTML structure)\n- Deep nesting → scoped locators (modular, maintainable)\n\n---\n\n### Example 3: Selector Best Practices Checklist\n\n```typescript\n// tests/selectors/validation-checklist.spec.ts\nimport { test, expect } from '@playwright/test';\n\n/**\n * Selector Validation Checklist\n *\n * Before committing test, verify selectors meet these criteria:\n */\ntest.describe('Selector Best Practices Validation', () => {\n test('✅ 1. Prefer data-testid for interactive elements', async ({ page }) => {\n await page.goto('/login');\n\n // Interactive elements (buttons, inputs, links) should use data-testid\n await page.getByTestId('email-input').fill('[email protected]');\n await page.getByTestId('login-button').click();\n });\n\n test('✅ 2. Use ARIA roles for semantic elements', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Semantic elements (headings, navigation, forms) use ARIA\n await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();\n await page.getByRole('navigation').getByRole('link', { name: 'Settings' }).click();\n });\n\n test('✅ 3. Avoid CSS classes (except when testing styles)', async ({ page }) => {\n await page.goto('/products');\n\n // ❌ Never for interaction: page.locator('.btn-primary')\n // ✅ Only for visual regression: await expect(page.locator('.error-banner')).toHaveCSS('color', 'rgb(255, 0, 0)')\n });\n\n test('✅ 4. Use filter() instead of nth() for lists', async ({ page }) => {\n await page.goto('/orders');\n\n // List selection should be content-based\n await page.getByTestId('order-row').filter({ hasText: 'Order #12345' }).click();\n });\n\n test('✅ 5. Selectors are human-readable', async ({ page }) => {\n await page.goto('/checkout');\n\n // ✅ Good: Clear intent\n await page.getByTestId('shipping-address-form').getByLabel('Street Address').fill('123 Main St');\n\n // ❌ Bad: Cryptic\n // await page.locator('div > div:nth-child(2) > input[type=\"text\"]').fill('123 Main St')\n });\n});\n```\n\n**Validation Rules**:\n\n1. **Interactive elements** (buttons, inputs) → data-testid\n2. **Semantic elements** (headings, nav, forms) → ARIA roles\n3. **CSS classes** → Avoid (except visual regression tests)\n4. **Lists** → filter() over nth() (content-based selection)\n5. **Readability** → Selectors document user intent (clear, semantic)\n\n---\n\n## Selector Resilience Checklist\n\nBefore deploying selectors:\n\n- [ ] **Hierarchy followed**: data-testid (1st choice) > ARIA (2nd) > text (3rd) > CSS/ID (last resort)\n- [ ] **Interactive elements use data-testid**: Buttons, inputs, links have dedicated test attributes\n- [ ] **Semantic elements use ARIA**: Headings, navigation, forms use roles and accessible names\n- [ ] **No brittle patterns**: No CSS classes (except visual tests), no arbitrary nth(), no complex XPath\n- [ ] **Dynamic content handled**: Regex for IDs/timestamps, filter() for lists, partial matching for text\n- [ ] **Selectors are scoped**: Use container locators to narrow scope (prevent ambiguity)\n- [ ] **Human-readable**: Selectors document user intent (clear, semantic, maintainable)\n- [ ] **Validated in Inspector**: Test selectors interactively before committing (page.pause())\n\n## Integration Points\n\n- **Used in workflows**: `*atdd` (generate tests with robust selectors), `*automate` (healing selector failures), `*test-review` (validate selector quality)\n- **Related fragments**: `test-healing-patterns.md` (selector failure diagnosis), `fixture-architecture.md` (page object alternatives), `test-quality.md` (maintainability standards)\n- **Tools**: Playwright Inspector (Pick Locator), DevTools console, Playwright MCP browser_generate_locator (optional)\n\n_Source: Playwright selector best practices, accessibility guidelines (ARIA), production test maintenance patterns._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":19785,"content_sha256":"baab01dce68b97e87b27036b207ce10b7429a3e111cead0f1662ab726ec355ca"},{"filename":"knowledge/tea-index.csv","content":"id,name,description,tags,tier,fragment_file\nlibrary-integration-mandate,Library Integration Mandate,\"General contract for every TEA integration library flag: the two gates (flag true plus package installed), REQUIRED vs RECOMMENDED levels, deviation protocol, scope discipline, the flag-to-mandate registry, and the ten places a new library must be wired\",\"standards,governance,generation,review,integration,mandate\",core,library-integration-mandate.md\nfixture-architecture,Fixture Architecture,\"Composable fixture patterns (pure function → fixture → merge) and reuse rules\",\"fixtures,architecture,playwright,cypress\",core,fixture-architecture.md\nnetwork-first,Network-First Safeguards,\"Intercept-before-navigate workflow, HAR capture, deterministic waits, edge mocking\",\"network,stability,playwright,cypress,ui\",core,network-first.md\ndata-factories,Data Factories and API Setup,\"Factories with overrides, API seeding, cleanup discipline, and naming the domain literals a test hardcodes on purpose\",\"data,factories,setup,api,backend,seeding,magic-value,literals,naming,constants\",core,data-factories.md\ncomponent-tdd,Component TDD Loop,\"Red→green→refactor workflow, provider isolation, accessibility assertions, and user-level interaction over raw event dispatch\",\"component-testing,tdd,ui,user-event,fire-event,interaction\",extended,component-tdd.md\nplaywright-config,Playwright Config Guardrails,\"Environment switching, timeout standards, artifact outputs\",\"playwright,config,env\",extended,playwright-config.md\nci-burn-in,CI and Burn-In Strategy,\"Staged jobs, shard orchestration, burn-in loops, artifact policy\",\"ci,automation,flakiness\",extended,ci-burn-in.md\nselective-testing,Selective Test Execution,\"Tag/grep usage, spec filters, diff-based runs, promotion rules\",\"risk-based,selection,strategy\",extended,selective-testing.md\nfeature-flags,Feature Flag Governance,\"Enum management, targeting helpers, cleanup, release checklists\",\"feature-flags,governance,launchdarkly\",specialized,feature-flags.md\ncontract-testing,Contract Testing Essentials,\"Pact publishing, provider verification, short-lived branch coordination, resilience coverage, PactV4 four-rule determinism & FFI safety block (fileParallelism + pool:forks + singleFork + determinism gate)\",\"contract-testing,pact,api,backend,microservices,service-contract,vitest,ffi,determinism,pactv4,branch,can-i-deploy\",specialized,contract-testing.md\nemail-auth,Email Authentication Testing,\"Magic link extraction, state preservation, caching, negative flows\",\"email-authentication,security,workflow\",specialized,email-auth.md\nerror-handling,Error Handling Checks,\"Scoped exception handling, retry validation, telemetry logging\",\"resilience,error-handling,stability,api,backend\",extended,error-handling.md\nvisual-debugging,Visual Debugging Toolkit,\"Trace viewer usage, artifact expectations, accessibility integration\",\"debugging,dx,tooling,ui\",specialized,visual-debugging.md\nrisk-governance,Risk Governance,\"Scoring matrix, category ownership, gate decision rules\",\"risk,governance,gates\",core,risk-governance.md\nprobability-impact,Probability and Impact Scale,\"Shared definitions for scoring matrix and gate thresholds\",\"risk,scoring,scale\",core,probability-impact.md\ntest-quality,Test Quality Definition of Done,\"Execution limits, isolation rules, green criteria, committed skips and focus, assertions that cannot fail, and suite structure and naming\",\"quality,definition-of-done,tests,skip,only,focused,disabled,tautological,assertions,mocks,unreachable,concerns,grouping,nesting,naming,assertion-style\",core,test-quality.md\nnfr-criteria,NFR Review Criteria,\"Security, performance, reliability, maintainability status definitions\",\"nfr,assessment,quality\",extended,nfr-criteria.md\ntest-levels,Test Levels Framework,\"Guidelines for choosing unit, integration, or end-to-end coverage\",\"testing,levels,selection,api,backend,ui\",core,test-levels-framework.md\ntest-priorities,Test Priorities Matrix,\"P0–P3 criteria, coverage targets, execution ordering\",\"testing,prioritization,risk\",core,test-priorities-matrix.md\ntest-healing-patterns,Test Healing Patterns,\"Common failure patterns and automated fixes\",\"healing,debugging,patterns\",core,test-healing-patterns.md\nselector-resilience,Selector Resilience,\"Robust selector strategies and debugging techniques\",\"selectors,locators,debugging,ui\",core,selector-resilience.md\ntiming-debugging,Timing Debugging,\"Race condition identification, deterministic wait fixes, wall-clock fixtures and fake timers, and unawaited promises in test bodies\",\"timing,async,debugging,clock,fake-timers,expiry,ttl,unawaited,await,floating-promise\",extended,timing-debugging.md\noverview,Playwright Utils Overview,\"Installation, design principles, fixture patterns for API and UI testing\",\"playwright-utils,fixtures,api,backend,ui\",core,overview.md\nplaywright-utils-mandate,Playwright Utils Mandate,\"Binding rule when tea_use_playwright_utils is true: substitution table (page.route to interceptNetworkCall, raw request to apiRequest, waitForTimeout to recurse), REQUIRED vs RECOMMENDED levels, banned patterns, pre-emit self-check, deviation protocol\",\"playwright-utils,standards,generation,review,governance,mandate\",core,playwright-utils-mandate.md\napi-request,API Request,\"Typed HTTP client, schema validation, retry logic, operation-based overload for API and service testing\",\"api,backend,service-testing,api-testing,playwright-utils,openapi,codegen,operation\",core,api-request.md\nnetwork-recorder,Network Recorder,\"HAR record/playback, CRUD detection for offline UI testing\",\"network,playwright-utils,ui,har\",extended,network-recorder.md\nauth-session,Auth Session,\"Token persistence, multi-user, API and browser authentication\",\"auth,playwright-utils,api,backend,jwt,token\",core,auth-session.md\nintercept-network-call,Intercept Network Call,\"Network spy/stub, JSON parsing for UI tests\",\"network,playwright-utils,ui\",extended,intercept-network-call.md\nrecurse,Recurse Polling,\"Async polling for API responses, background jobs, eventual consistency\",\"polling,playwright-utils,api,backend,async,eventual-consistency\",extended,recurse.md\nlog,Log Utility,\"Report logging, structured output for API and UI tests\",\"logging,playwright-utils,api,ui\",extended,log.md\nfile-utils,File Utilities,\"CSV/XLSX/PDF/ZIP validation for API exports and UI downloads\",\"files,playwright-utils,api,backend,ui\",extended,file-utils.md\nburn-in,Burn-in Runner,\"Smart test selection, git diff for CI optimization\",\"ci,playwright-utils\",extended,burn-in.md\nnetwork-error-monitor,Network Error Monitor,\"HTTP 4xx/5xx detection for UI tests\",\"monitoring,playwright-utils,ui\",extended,network-error-monitor.md\nfixtures-composition,Fixtures Composition,\"mergeTests composition patterns for combining utilities\",\"fixtures,playwright-utils\",extended,fixtures-composition.md\napi-testing-patterns,API Testing Patterns,\"Pure API test patterns without browser: service testing, microservices, GraphQL\",\"api,backend,service-testing,api-testing,microservices,graphql,no-browser\",specialized,api-testing-patterns.md\npactjs-utils-overview,Pact.js Utils Overview,\"Installation, utility table, short-lived branch coordination, and breaking-change branch classification\",\"pactjs-utils,contract-testing,pact,api,backend,microservices,branch,short-lived-branch,breaking-change\",specialized,pactjs-utils-overview.md\npactjs-utils-mandate,Pact.js Utils Mandate,\"Binding rule when tea_use_pactjs_utils is true: substitution table (hand-cast given to createProviderState, literal VerifierOptions to buildVerifierOptions, bespoke auth middleware to createRequestFilter), relevance gate before scaffolding, banned patterns, pre-emit self-check, broker degradation\",\"pactjs-utils,contract-testing,pact,standards,generation,review,governance,mandate\",core,pactjs-utils-mandate.md\npactjs-utils-zod-to-pact,Pact.js Utils Zod to Pact,\"zodToPactMatchers for consumer-curated schemas, example precedence, Pact V3 matcher mapping, and anti-patterns\",\"pactjs-utils,zod,contract-testing,pact,consumer,schema,matchers,api\",specialized,pactjs-utils-zod-to-pact.md\npactjs-utils-consumer-helpers,Pact.js Utils Consumer Helpers,\"createProviderState, toJsonMap, setJsonContent, setJsonBody; PactV4 one-interaction-per-it() determinism rule\",\"pactjs-utils,consumer,contract-testing,pact,api,determinism,pactv4\",specialized,pactjs-utils-consumer-helpers.md\npactjs-utils-provider-verifier,Pact.js Utils Provider Verifier,\"Verifier builders, scoped consumerBranch selectors, provider revision metadata, isBreakingChangeTolerantBranch, and FFI-safe Vitest config\",\"pactjs-utils,provider,consumer,contract-testing,pact,api,backend,ci,vitest,ffi,consumer-branch,provider-branch,short-lived-branch,selectors,release-branch,breaking-change\",specialized,pactjs-utils-provider-verifier.md\npactjs-utils-request-filter,Pact.js Utils Request Filter,\"createRequestFilter, noOpRequestFilter for auth injection\",\"pactjs-utils,auth,contract-testing,pact\",specialized,pactjs-utils-request-filter.md\npact-mcp,Pact MCP Server,\"SmartBear MCP for PactFlow: generate tests, review, can-i-deploy, provider states\",\"pact,mcp,pactflow,contract-testing,broker\",specialized,pact-mcp.md\npact-consumer-framework-setup,Pact Consumer CDC Framework Setup,\"Directory structure, FFI-safe vitest config, deterministic publishing, PR-only provider branch detection, and additive branch-aware can-i-deploy\",\"pactjs-utils,consumer,contract-testing,pact,ci,framework,setup,vitest,shell-scripts,jq,pactv4,ffi,file-organization,one-file-per-pair,provider-branch,can-i-deploy,short-lived-branch\",specialized,pact-consumer-framework-setup.md\npact-broker-webhooks,Pact Broker Webhooks,\"PactFlow → GitHub repository_dispatch auth, exact registered provider target checkout, staleness monitoring, and PAT rotation\",\"pact,pactflow,broker,webhooks,github,auth,pat,ci,operations,security,provider-branch,short-lived-branch\",specialized,pact-broker-webhooks.md\nadr-quality-readiness-checklist,ADR Quality Readiness Checklist,\"8-category 29-criteria framework for ADR testability and NFR evidence audit\",\"nfr,testability,adr,quality,assessment,checklist\",extended,adr-quality-readiness-checklist.md\nplaywright-cli,Playwright CLI,\"Token-efficient CLI for AI coding agents: element refs, sessions, snapshots, trace analysis, debug=cli autonomous investigation\",\"cli,browser,agent,automation,snapshot,trace,debug\",core,playwright-cli.md\npact-consumer-di,Pact Consumer DI Pattern,\"Dependency injection pattern for Pact consumer tests — call actual source code instead of raw fetch by injecting mock server URL via optional baseUrl in context type\",\"contract-testing,pact,consumer,dependency-injection,api,backend,architecture\",extended,pact-consumer-di.md\nwebhook-fundamentals,Webhook Testing Fundamentals,\"Why webhook delivery is hard: async, parallel pollution, opaque timeouts, cleanup drift. playwright-utils approach with polling, typed matchers, rich errors, startedAt isolation\",\"webhook,async,playwright-utils,event-driven,eventually-consistent\",core,webhook-testing-fundamentals.md\nwebhook-setup,Webhook Module Setup,\"Fixture wiring for WireMock/MockServer/Mockoon providers, matched-only vs full-reset cleanup strategy, fullyParallel race condition fix\",\"webhook,fixtures,playwright-utils,wiremock,mockserver,mockoon,setup\",core,webhook-module-setup.md\nwebhook-matchers,Webhook Template Matchers,\"matchField (dot-path exact), matchPartial (deep subset), matchPredicate (arbitrary fn), AND semantics, template factories, clone, withTimeout, withInterval\",\"webhook,matchers,playwright-utils,templates,patterns\",core,webhook-template-matchers.md\nwebhook-waiting,Webhook Waiting and Querying,\"waitFor, waitForCount, getReceived, drain pattern for sequential events, parallel worker safety via ID-scoped templates\",\"webhook,async,playwright-utils,polling,patterns,eventually-consistent\",core,webhook-waiting-querying.md\nwebhook-timeout-error,WebhookTimeoutError Debugging,\"templateName, timeoutMs, totalReceived, receivedWebhooks, matcherDetails, toJSON — inspect what arrived vs what was expected\",\"webhook,debugging,errors,playwright-utils\",extended,webhook-timeout-error.md\nwebhook-providers,Webhook Provider Patterns,\"WireMock (deleteById supported), MockServer (deleteById no-op), Mockoon (deleteById no-op, 100-entry limit), custom WebhookProvider interface\",\"webhook,providers,playwright-utils,wiremock,mockserver,mockoon\",extended,webhook-providers.md\nwebhook-risk,Webhook Testing Risk Guidance,\"When webhook tests are required, P2×I3 default risk score, complete test checklist, failure patterns and mitigations, TA assessment checklist\",\"webhook,risk,assessment,event-driven,async,playwright-utils,governance\",core,webhook-risk-guidance.md\nconfidence-gate,Confidence Gate,\"1-10 confidence scoring with stop-and-ask rule below threshold for selectors, endpoints, risk classification, fixtures, schemas, and data factories — prevents agent fabrication\",\"reliability,agent-safety,generation,quality,governance\",core,confidence-gate.md\nmaestro-flows,Maestro Flow Patterns,\"Declarative mobile flow structure, selector hierarchy (id > text > scoped text), clearState isolation, synchronization without sleeps, subflow composition, text selectors as whole-element regex, taps that report COMPLETED without being handled, visible meaning inside the viewport, asserting the transition rather than a state that may already hold, anti-patterns\",\"mobile,maestro,ios,android,flows,selectors,ui\",specialized,maestro-flows.md\nmobile-test-strategy,Mobile Test Strategy,\"Mobile test level framework (unit/component/contract/device flow), what belongs in a device flow, mobile risk categories (permissions, lifecycle, connectivity, fragmentation, upgrade), device matrix with a gate profile matching local, CI shape, no live third-party flag evaluation in the run path\",\"mobile,maestro,strategy,risk,levels,ci,ios,android\",specialized,mobile-test-strategy.md\nevidence-integrity,Evidence Integrity,\"Falsifiability of checks (hollow green, optional assertions, continue-on-error, partial manifests, assertions already true before the action), three-state diagnostics (pass/fail/could-not-measure), probes that issue the client's own request, verifying the outcome rather than the act, verifying framework properties before use, proving claims from the side that can observe them, environment asymmetry including screen geometry and accumulated credentials, ranking hypotheses by the cost of the measurement that kills them, recording what a change did rather than what it was for\",\"quality,evidence,diagnostics,review,ci,gates,falsifiability\",core,evidence-integrity.md\nmobile-ci-device-lab,Mobile CI Device Lab,\"Build artifact selection (release build vs development shell vs debug-variant development build), native modules that degrade silently in a shell, deep links reachable through the shell routed URL form, development-server manifest signing in non-interactive CI, one device profile across local and CI, Android emulator snapshot caching and config.ini pitfalls, repairing locally created AVDs, per-device identity for sharded runs, runner version pinning, artifact layout and failure diagnosis, sharding measured on wall clock\",\"mobile,maestro,ci,android,ios,expo,emulator,artifacts\",specialized,mobile-ci-device-lab.md\n","content_type":"text/csv; charset=utf-8","language":"csv","size":15306,"content_sha256":"a9543ab8ccb3a201f74ff20fa24016344d1cfc3b3a88dcdc154a2b5b7df90305"},{"filename":"knowledge/test-healing-patterns.md","content":"# Test Healing Patterns\n\n## Principle\n\nRun generated tests, diagnose the cause from runtime and source evidence, repair confirmed test defects, and execute again. Acceptance criteria and assertions define the behavior to preserve. Error text suggests a hypothesis; it cannot establish that a test is wrong.\n\nThe merged `bmad-tea-automate` skill applies these patterns during Create in red and expand modes through `resources/run-and-heal.md`. Execution and healing default to enabled, with at most three repair rounds. Validate and Edit never heal. Expand seeks passing tests. Red verifies that the criterion executes and fails for the intended missing behavior, which remains unchanged.\n\n## Diagnose Before Editing\n\nFor each failure, read the generated test, its criterion, fixture setup, relevant application source, and fresh runner output. Inspect DOM/network snapshots or traces when needed. Record the file:line, class, supporting evidence and proposed repair. A timeout can mean a stale locator, unfinished setup, unavailable environment or absent product behavior. A numeric mismatch can be a product defect. Preserve uncertain failures and report what must be investigated.\n\n| Class | Confirm the cause | Repair boundary |\n| ------------------- | ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |\n| Selector | Intended element exists, generated locator misses it | Use its observed test ID, role/name or scoped locator |\n| Timing | Correct effect finishes after an unawaited operation or missed event | Await operation; register the observable wait before its trigger |\n| Data | Generated setup uses a stale/invalid value; authoritative factory has correct data | Fix setup input/identity and retain exact expected business values |\n| Network | Test URL, declared service or existing external double is configured incorrectly | Repair test wiring; preserve the real SUT boundary |\n| Hard wait | Sleep races a demonstrated state transition | Replace sleep with the matching event/state wait |\n| Syntax/import/setup | Generated code cannot parse/load or initialize its fixture | Repair generated code using actual project imports and fixture conventions |\n| Product defect | Valid setup reaches behavior that violates the criterion | Keep the failing assertion and report reproduction, expected/actual result and evidence |\n| Intended red | Assertion executes and fails for the criterion's unimplemented behavior | Keep the scaffold unchanged and record the verified missing behavior |\n| Unknown/environment | Required evidence, dependency, service or credential is unavailable | Report could not measure or investigation needed; preserve test and source |\n\n## Selector Repairs\n\nUse `selector-resilience.md`. Observe the same intended element in the live page or source first. Prefer its existing data-testid, then an accessible role/name, then scoped meaningful text. A class name does not establish a test ID. Adding production attributes belongs to product work outside the healing loop. An absent feature in red remains an intended missing-behavior failure.\n\n```typescript\n// Evidence: the checkout source and DOM expose this existing button.\n// Original generated locator: page.locator('.submit-payment-old')\nawait page.getByRole('button', { name: 'Complete Payment', exact: true }).click();\nawait expect(page.getByTestId('receipt-status')).toHaveText('Paid');\n```\n\nKeep the action target and receipt expectation. Ambiguous or unobserved replacement locators require investigation.\n\n## Timing and Hard-Wait Repairs\n\nUse `timing-debugging.md` and `network-first.md`. Await promise-returning setup and actions. Register the wait before the trigger so a fast event cannot be missed. Observe the exact request or state transition under test. A genuine slow or broken product path remains a product issue. Preserve its timeout budget.\n\n```typescript\n// The trace shows the response completes before the old wait was registered.\nconst responsePromise = page.waitForResponse(\n (response) => response.url().endsWith('/api/orders') && response.request().method() === 'POST',\n);\nawait page.getByRole('button', { name: 'Place Order' }).click();\nconst response = await responsePromise;\nexpect(response.status()).toBe(201);\nawait expect(page.getByTestId('order-status')).toHaveText('Confirmed');\n```\n\nWhen `tea_use_playwright_utils` is true, use the project's merged fixtures, `interceptNetworkCall` and `recurse` as required by `playwright-utils-mandate.md`. Register the helper before the triggering action and retain exact response assertions. Cypress uses a named intercept registered before navigation/action; backend suites await real operations through their own runner. Restore fake clocks in teardown when controlled time is relevant.\n\n```typescript\n// Replace a generated sleep with the observed terminal state.\nawait page.getByTestId('loading-spinner').waitFor({ state: 'detached' });\nawait expect(page.getByTestId('balance')).toHaveText('$42.00');\n```\n\n## Data Repairs\n\nUse `data-factories.md`. Trace identity to the factory or setup response, isolate fixture data and clean it up. Fix generated setup without changing the business expectation. Do not replace an exact value with a regex, substring, any-value matcher or the actual product output simply because it failed. Format-only matchers are appropriate when the original acceptance criterion explicitly specifies only a format.\n\n```typescript\n// Evidence: createOrder returns the seeded order's ID; 'order-123' was invented.\nconst seededOrder = await createOrder({ totalCents: 4200 });\nawait page.goto(`/orders/${seededOrder.id}`);\nawait expect(page.getByTestId('order-total')).toHaveText('$42.00');\n```\n\n## Network Repairs\n\nRead the configured base URL, service readiness, route method and actual response. Correct a generated typo or a missing declared test fixture. A 500 from a real application endpoint is evidence to investigate and reproduce. Keep the real endpoint and the expected status assertion intact.\n\nAn external double already specified by the scenario may have a wrong URL/method or may be registered after its request starts. Correct that wiring before the trigger within the declared boundary. Preserve response semantics. Never introduce an external double for the SUT endpoint under test or synthesize success to conceal a failure. Contract tests retain the real consumer and provider-source scrutiny, with `pactjs-utils-mandate.md` respected when enabled.\n\n```typescript\n// Evidence: the project config names '/api/orders'; the generated path was '/api/order'.\nconst response = await request.post('/api/orders', { data: { quantity: 1, sku: seededProduct.sku } });\nexpect(response.status()).toBe(201);\nexpect((await response.json()).totalCents).toBe(4200);\n```\n\nWhen `tea_use_playwright_utils` is true, use the project's `apiRequest` fixture for API calls. The raw runner example above applies when that flag is false.\n\n## Syntax, Import and Setup Repairs\n\nA parser error or unresolved import prevents the acceptance criterion from executing. Read the existing project exports, dependency declarations and fixture composition. Correct generated syntax, import paths, asynchronous fixture lifetime or setup order. Keep the scenario/assertion intact. Missing dependencies or unavailable credentials are environment blockers when they cannot be supplied within the run's authorized setup.\n\n## Red Verification\n\nPermanent Playwright acceptance scaffolds keep `test.skip('title', ...)`. Verify a disposable copy, preserving original files and every business assertion. Prefer the available `tea-atdd-red-check` for compatible browserless loopback projects: it uses a minimal worker environment, refuses worker child processes including browser launches, and supports `*.spec.ts` scaffolds. Derive its per-file process bound from the project's existing execution budgets; the default 15 seconds cannot replace a longer declared test budget. Its successful exit means the report exists.\n\nPermanent Vitest and Jest scaffolds keep `test.skip('title', ...)` or `it.skip('title', ...)`. Prefer the available `tea-unit-check --mode red`: it copies the project with linked dependencies, blanks only the generated leaves' `.skip` in the copy, runs each file alone with the installed runner and project config, and reports every attempt's failure as `assertion`, `timeout` or `error`, plus unhandled errors Vitest's JSON omits. A headless run accepts only this report as Vitest or Jest red evidence.\n\nFor browser tests, required project environment/services, non-loopback endpoints, another runner/format or a skill-only install with no verifier, use the actual project-native runner on the disposable activated copy with the original configuration and environment. Activate only the generated leaf scaffold wrappers. Preserve other skips and all test bodies. Run files separately when a load error would mask sibling results; capture fresh per-file JSON, every project/attempt's actual status, full assertion messages and load/setup errors. Compare production/config content and permissions before and after execution. Record the fallback reason and runner/budget in the summary. Keep the evaluation CLI's isolation unchanged.\n\nMap each result to the criterion's recorded failure signature and establish that the assertion executed. Skipped, empty, unmapped, timed-out and wrong-reason results cannot count as verified red. Repair wrong-reason syntax/import/locator/setup defects only until the expected missing-behavior failure is reached. Preserve that failure and keep production unchanged. A runner without usable per-test evidence reports could not measure.\n\n## Bounded Loop and Report\n\n1. Execute the generated scope and record fresh results.\n2. Classify with evidence and identify confirmed test defects.\n3. Persist the round count, apply scoped repairs and re-run.\n4. Stop when expand passes, red is verified, only preserved/blocked failures remain, or three repair rounds are used.\n5. Report commands, counts, healed file:line and change, criteria preserved, intended red failures, product defects and blockers.\n\nNever introduce `test.fixme()`, additional skip calls, expected-failure annotations, weakened assertions, arbitrary sleeps, enlarged timeout budgets or catch-and-ignore logic during healing. Unresolved expand tests stay active. Red retains its existing scaffold convention. Resume retains the round count. Report remaining failures without claiming green.\n\n## Healing Checklist\n\n- [ ] Generated scope, assertions, acceptance criteria and production baseline saved\n- [ ] Fresh execution evidence collected for every generated test\n- [ ] Each proposed repair supported by runtime/source evidence\n- [ ] Only generated tests and support files within this run repaired\n- [ ] Playwright Utils and Pact.js Utils mandates preserved when applicable\n- [ ] Correct red failures and real product defects left unchanged\n- [ ] At most three repair rounds, including rounds before interruption\n- [ ] Unresolved results reported with reproduction and next action\n\n## Integration Points\n\n- **Used in workflows**: `*automate` Create in red/expand modes; `*atdd` Create enters red mode through its existing command\n- **Related fragments**: `selector-resilience.md`, `timing-debugging.md`, `network-first.md`, `data-factories.md`, `test-quality.md`\n- **Tools**: project test runners, `tea-atdd-red-check`, runner traces and reports, optional browser evidence tools\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12152,"content_sha256":"90be2555085a996504d0848660892583b6c429442a7f4c223cf60d3651cce37a"},{"filename":"knowledge/test-levels-framework.md","content":"\u003c!-- Powered by BMAD-CORE™ -->\n\n# Test Levels Framework\n\nComprehensive guide for determining appropriate test levels (unit, integration, E2E) for different scenarios.\n\n## Test Level Decision Matrix\n\n### Unit Tests\n\n**When to use:**\n\n- Testing pure functions and business logic\n- Algorithm correctness\n- Input validation and data transformation\n- Error handling in isolated components\n- Complex calculations or state machines\n\n**Characteristics:**\n\n- Fast execution (immediate feedback)\n- No external dependencies (DB, API, file system)\n- Highly maintainable and stable\n- Easy to debug failures\n\n**Example scenarios:**\n\n```yaml\nunit_test:\n component: 'PriceCalculator'\n scenario: 'Calculate discount with multiple rules'\n justification: 'Complex business logic with multiple branches'\n mock_requirements: 'None - pure function'\n```\n\n### Integration Tests\n\n**When to use:**\n\n- Component interaction verification\n- Database operations and transactions\n- API endpoint contracts\n- Service-to-service communication\n- Middleware and interceptor behavior\n\n**Characteristics:**\n\n- Moderate execution time\n- Tests component boundaries\n- May use test databases or containers\n- Validates system integration points\n\n**Example scenarios:**\n\n```yaml\nintegration_test:\n components: ['UserService', 'AuthRepository']\n scenario: 'Create user with role assignment'\n justification: 'Critical data flow between service and persistence'\n test_environment: 'In-memory database'\n```\n\n### End-to-End Tests\n\n**When to use:**\n\n- Critical user journeys\n- Cross-system workflows\n- Visual regression testing\n- Compliance and regulatory requirements\n- Final validation before release\n\n**Characteristics:**\n\n- Slower execution\n- Tests complete workflows\n- Requires full environment setup\n- Most realistic but most brittle\n\n**Example scenarios:**\n\n```yaml\ne2e_test:\n journey: 'Complete checkout process'\n scenario: 'User purchases with saved payment method'\n justification: 'Revenue-critical path requiring full validation'\n environment: 'Staging with test payment gateway'\n```\n\n## Test Level Selection Rules\n\n### Favor Unit Tests When:\n\n- Logic can be isolated\n- No side effects involved\n- Fast feedback needed\n- High cyclomatic complexity\n\n### Favor Integration Tests When:\n\n- Testing persistence layer\n- Validating service contracts\n- Testing middleware/interceptors\n- Component boundaries critical\n\n### Favor E2E Tests When:\n\n- User-facing critical paths\n- Multi-system interactions\n- Regulatory compliance scenarios\n- Visual regression important\n\n## Anti-patterns to Avoid\n\n- E2E testing for business logic validation\n- Unit testing framework behavior\n- Integration testing third-party libraries\n- Duplicate coverage across levels\n\n## Duplicate Coverage Guard\n\n**Before adding any test, check:**\n\n1. Is this already tested at a lower level?\n2. Can a unit test cover this instead of integration?\n3. Can an integration test cover this instead of E2E?\n\n**Coverage overlap is only acceptable when:**\n\n- Testing different aspects (unit: logic, integration: interaction, e2e: user experience)\n- Critical paths requiring defense in depth\n- Regression prevention for previously broken functionality\n\n## Test Naming Conventions\n\n- Unit: `test_{component}_{scenario}`\n- Integration: `test_{flow}_{interaction}`\n- E2E: `test_{journey}_{outcome}`\n\n## Test ID Format\n\n`{EPIC}.{STORY}-{LEVEL}-{SEQ}`\n\nExamples:\n\n- `1.3-UNIT-001`\n- `1.3-INT-002`\n- `1.3-E2E-001`\n\n## Real Code Examples\n\n### Example 1: E2E Test (Full User Journey)\n\n**Scenario**: User logs in, navigates to dashboard, and places an order.\n\n```typescript\n// tests/e2e/checkout-flow.spec.ts\nimport { test, expect } from '@playwright/test';\nimport { createUser, createProduct } from '../test-utils/factories';\n\ntest.describe('Checkout Flow', () => {\n test('user can complete purchase with saved payment method', async ({ page, apiRequest }) => {\n // Setup: Seed data via API (fast!)\n const user = createUser({ email: '[email protected]', hasSavedCard: true });\n const product = createProduct({ name: 'Widget', price: 29.99, stock: 10 });\n\n await apiRequest.post('/api/users', { data: user });\n await apiRequest.post('/api/products', { data: product });\n\n // Network-first: Intercept BEFORE action\n const loginPromise = page.waitForResponse('**/api/auth/login');\n const cartPromise = page.waitForResponse('**/api/cart');\n const orderPromise = page.waitForResponse('**/api/orders');\n\n // Step 1: Login\n await page.goto('/login');\n await page.fill('[data-testid=\"email\"]', user.email);\n await page.fill('[data-testid=\"password\"]', 'password123');\n await page.click('[data-testid=\"login-button\"]');\n await loginPromise;\n\n // Assert: Dashboard visible\n await expect(page).toHaveURL('/dashboard');\n await expect(page.getByText(`Welcome, ${user.name}`)).toBeVisible();\n\n // Step 2: Add product to cart\n await page.goto(`/products/${product.id}`);\n await page.click('[data-testid=\"add-to-cart\"]');\n await cartPromise;\n await expect(page.getByText('Added to cart')).toBeVisible();\n\n // Step 3: Checkout with saved payment\n await page.goto('/checkout');\n await expect(page.getByText('Visa ending in 1234')).toBeVisible(); // Saved card\n await page.click('[data-testid=\"use-saved-card\"]');\n await page.click('[data-testid=\"place-order\"]');\n await orderPromise;\n\n // Assert: Order confirmation\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n await expect(page.getByText(/Order #\\d+/)).toBeVisible();\n await expect(page.getByText('$29.99')).toBeVisible();\n });\n});\n```\n\n**Key Points (E2E)**:\n\n- Tests complete user journey across multiple pages\n- API setup for data (fast), UI for assertions (user-centric)\n- Network-first interception to prevent flakiness\n- Validates critical revenue path end-to-end\n\n### Example 2: Integration Test (API/Service Layer)\n\n**Scenario**: UserService creates user and assigns role via AuthRepository.\n\n```typescript\n// tests/integration/user-service.spec.ts\nimport { test, expect } from '@playwright/test';\nimport { createUser } from '../test-utils/factories';\n\ntest.describe('UserService Integration', () => {\n test('should create user with admin role via API', async ({ request }) => {\n const userData = createUser({ role: 'admin' });\n\n // Direct API call (no UI)\n const response = await request.post('/api/users', {\n data: userData,\n });\n\n expect(response.status()).toBe(201);\n\n const createdUser = await response.json();\n expect(createdUser.id).toBeTruthy();\n expect(createdUser.email).toBe(userData.email);\n expect(createdUser.role).toBe('admin');\n\n // Verify database state\n const getResponse = await request.get(`/api/users/${createdUser.id}`);\n expect(getResponse.status()).toBe(200);\n\n const fetchedUser = await getResponse.json();\n expect(fetchedUser.role).toBe('admin');\n expect(fetchedUser.permissions).toContain('user:delete');\n expect(fetchedUser.permissions).toContain('user:update');\n\n // Cleanup\n await request.delete(`/api/users/${createdUser.id}`);\n });\n\n test('should validate email uniqueness constraint', async ({ request }) => {\n const userData = createUser({ email: '[email protected]' });\n\n // Create first user\n const response1 = await request.post('/api/users', { data: userData });\n expect(response1.status()).toBe(201);\n\n const user1 = await response1.json();\n\n // Attempt duplicate email\n const response2 = await request.post('/api/users', { data: userData });\n expect(response2.status()).toBe(409); // Conflict\n const error = await response2.json();\n expect(error.message).toContain('Email already exists');\n\n // Cleanup\n await request.delete(`/api/users/${user1.id}`);\n });\n});\n```\n\n**Key Points (Integration)**:\n\n- Tests service layer + database interaction\n- No UI involved—pure API validation\n- Business logic focus (role assignment, constraints)\n- Faster than E2E, more realistic than unit tests\n\n### Example 3: Component Test (Isolated UI Component)\n\n**Scenario**: Test button component in isolation with props and user interactions.\n\n```typescript\n// src/components/Button.cy.tsx (Cypress Component Test)\nimport { Button } from './Button';\n\ndescribe('Button Component', () => {\n it('should render with correct label', () => {\n cy.mount(\u003cButton label=\"Click Me\" />);\n cy.contains('Click Me').should('be.visible');\n });\n\n it('should call onClick handler when clicked', () => {\n const onClickSpy = cy.stub().as('onClick');\n cy.mount(\u003cButton label=\"Submit\" onClick={onClickSpy} />);\n\n cy.get('button').click();\n cy.get('@onClick').should('have.been.calledOnce');\n });\n\n it('should be disabled when disabled prop is true', () => {\n cy.mount(\u003cButton label=\"Disabled\" disabled={true} />);\n cy.get('button').should('be.disabled');\n cy.get('button').should('have.attr', 'aria-disabled', 'true');\n });\n\n it('should show loading spinner when loading', () => {\n cy.mount(\u003cButton label=\"Loading\" loading={true} />);\n cy.get('[data-testid=\"spinner\"]').should('be.visible');\n cy.get('button').should('be.disabled');\n });\n\n it('should apply variant styles correctly', () => {\n cy.mount(\u003cButton label=\"Primary\" variant=\"primary\" />);\n cy.get('button').should('have.class', 'btn-primary');\n\n cy.mount(\u003cButton label=\"Secondary\" variant=\"secondary\" />);\n cy.get('button').should('have.class', 'btn-secondary');\n });\n});\n\n// Playwright Component Test equivalent\nimport { test, expect } from '@playwright/experimental-ct-react';\nimport { Button } from './Button';\n\ntest.describe('Button Component', () => {\n test('should call onClick handler when clicked', async ({ mount }) => {\n let clicked = false;\n const component = await mount(\n \u003cButton label=\"Submit\" onClick={() => { clicked = true; }} />\n );\n\n await component.getByRole('button').click();\n expect(clicked).toBe(true);\n });\n\n test('should be disabled when loading', async ({ mount }) => {\n const component = await mount(\u003cButton label=\"Loading\" loading={true} />);\n await expect(component.getByRole('button')).toBeDisabled();\n await expect(component.getByTestId('spinner')).toBeVisible();\n });\n});\n```\n\n**Key Points (Component)**:\n\n- Tests UI component in isolation (no full app)\n- Props + user interactions + visual states\n- Faster than E2E, more realistic than unit tests for UI\n- Great for design system components\n\n### Example 4: Unit Test (Pure Function)\n\n**Scenario**: Test pure business logic function without framework dependencies.\n\n```typescript\n// src/utils/price-calculator.test.ts (Jest/Vitest)\nimport { calculateDiscount, applyTaxes, calculateTotal } from './price-calculator';\n\ndescribe('PriceCalculator', () => {\n describe('calculateDiscount', () => {\n it('should apply percentage discount correctly', () => {\n const result = calculateDiscount(100, { type: 'percentage', value: 20 });\n expect(result).toBe(80);\n });\n\n it('should apply fixed amount discount correctly', () => {\n const result = calculateDiscount(100, { type: 'fixed', value: 15 });\n expect(result).toBe(85);\n });\n\n it('should not apply discount below zero', () => {\n const result = calculateDiscount(10, { type: 'fixed', value: 20 });\n expect(result).toBe(0);\n });\n\n it('should handle no discount', () => {\n const result = calculateDiscount(100, { type: 'none', value: 0 });\n expect(result).toBe(100);\n });\n });\n\n describe('applyTaxes', () => {\n it('should calculate tax correctly for US', () => {\n const result = applyTaxes(100, { country: 'US', rate: 0.08 });\n expect(result).toBe(108);\n });\n\n it('should calculate tax correctly for EU (VAT)', () => {\n const result = applyTaxes(100, { country: 'DE', rate: 0.19 });\n expect(result).toBe(119);\n });\n\n it('should handle zero tax rate', () => {\n const result = applyTaxes(100, { country: 'US', rate: 0 });\n expect(result).toBe(100);\n });\n });\n\n describe('calculateTotal', () => {\n it('should calculate total with discount and taxes', () => {\n const items = [\n { price: 50, quantity: 2 }, // 100\n { price: 30, quantity: 1 }, // 30\n ];\n const discount = { type: 'percentage', value: 10 }; // -13\n const tax = { country: 'US', rate: 0.08 }; // +9.36\n\n const result = calculateTotal(items, discount, tax);\n expect(result).toBeCloseTo(126.36, 2);\n });\n\n it('should handle empty items array', () => {\n const result = calculateTotal([], { type: 'none', value: 0 }, { country: 'US', rate: 0 });\n expect(result).toBe(0);\n });\n\n it('should calculate correctly without discount or tax', () => {\n const items = [{ price: 25, quantity: 4 }];\n const result = calculateTotal(items, { type: 'none', value: 0 }, { country: 'US', rate: 0 });\n expect(result).toBe(100);\n });\n });\n});\n```\n\n**Key Points (Unit)**:\n\n- Pure function testing—no framework dependencies\n- Fast execution (milliseconds)\n- Edge case coverage (zero, negative, empty inputs)\n- High cyclomatic complexity handled at unit level\n\n## When to Use Which Level\n\n| Scenario | Unit | Integration | E2E |\n| ---------------------- | ------------- | ----------------- | ------------- |\n| Pure business logic | ✅ Primary | ❌ Overkill | ❌ Overkill |\n| Database operations | ❌ Can't test | ✅ Primary | ❌ Overkill |\n| API contracts | ❌ Can't test | ✅ Primary | ⚠️ Supplement |\n| User journeys | ❌ Can't test | ❌ Can't test | ✅ Primary |\n| Component props/events | ✅ Partial | ⚠️ Component test | ❌ Overkill |\n| Visual regression | ❌ Can't test | ⚠️ Component test | ✅ Primary |\n| Error handling (logic) | ✅ Primary | ⚠️ Integration | ❌ Overkill |\n| Error handling (UI) | ❌ Partial | ⚠️ Component test | ✅ Primary |\n\n## Anti-Pattern Examples\n\n### ❌ BAD: E2E test for business logic\n\n```typescript\n// DON'T DO THIS\ntest('calculate discount via UI', async ({ page }) => {\n await page.goto('/calculator');\n await page.fill('[data-testid=\"price\"]', '100');\n await page.fill('[data-testid=\"discount\"]', '20');\n await page.click('[data-testid=\"calculate\"]');\n await expect(page.getByText('$80')).toBeVisible();\n});\n// Problem: Slow, brittle, tests logic that should be unit tested\n```\n\n### ✅ GOOD: Unit test for business logic\n\n```typescript\ntest('calculate discount', () => {\n expect(calculateDiscount(100, 20)).toBe(80);\n});\n// Fast, reliable, isolated\n```\n\n_Source: Murat Testing Philosophy (test pyramid), existing test-levels-framework.md structure._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":14756,"content_sha256":"22b76acf805447f5968178e1a8c48f7e7e0d65965f1fe70a8f0f2a765ed0a216"},{"filename":"knowledge/test-priorities-matrix.md","content":"\u003c!-- Powered by BMAD-CORE™ -->\n\n# Test Priorities Matrix\n\nGuide for prioritizing test scenarios based on risk, criticality, and business impact.\n\n## Priority Levels\n\n### P0 - Critical (Must Test)\n\n**Criteria:**\n\n- Revenue-impacting functionality\n- Security-critical paths\n- Data integrity operations\n- Regulatory compliance requirements\n- Previously broken functionality (regression prevention)\n\n**Examples:**\n\n- Payment processing\n- Authentication/authorization\n- User data creation/deletion\n- Financial calculations\n- GDPR/privacy compliance\n\n**Testing Requirements:**\n\n- Comprehensive coverage at all levels\n- Both happy and unhappy paths\n- Edge cases and error scenarios\n- Performance under load\n\n### P1 - High (Should Test)\n\n**Criteria:**\n\n- Core user journeys\n- Frequently used features\n- Features with complex logic\n- Integration points between systems\n- Features affecting user experience\n\n**Examples:**\n\n- User registration flow\n- Search functionality\n- Data import/export\n- Notification systems\n- Dashboard displays\n\n**Testing Requirements:**\n\n- Primary happy paths required\n- Key error scenarios\n- Critical edge cases\n- Basic performance validation\n\n### P2 - Medium (Nice to Test)\n\n**Criteria:**\n\n- Secondary features\n- Admin functionality\n- Reporting features\n- Configuration options\n- UI polish and aesthetics\n\n**Examples:**\n\n- Admin settings panels\n- Report generation\n- Theme customization\n- Help documentation\n- Analytics tracking\n\n**Testing Requirements:**\n\n- Happy path coverage\n- Basic error handling\n- Can defer edge cases\n\n### P3 - Low (Test if Time Permits)\n\n**Criteria:**\n\n- Rarely used features\n- Nice-to-have functionality\n- Cosmetic issues\n- Non-critical optimizations\n\n**Examples:**\n\n- Advanced preferences\n- Legacy feature support\n- Experimental features\n- Debug utilities\n\n**Testing Requirements:**\n\n- Smoke tests only\n- Can rely on manual testing\n- Document known limitations\n\n## Risk-Based Priority Adjustments\n\n### Increase Priority When:\n\n- High user impact (affects >50% of users)\n- High financial impact (>$10K potential loss)\n- Security vulnerability potential\n- Compliance/legal requirements\n- Customer-reported issues\n- Complex implementation (>500 LOC)\n- Multiple system dependencies\n\n### Decrease Priority When:\n\n- Feature flag protected\n- Gradual rollout planned\n- Strong monitoring in place\n- Easy rollback capability\n- Low usage metrics\n- Simple implementation\n- Well-isolated component\n\n## Test Coverage by Priority\n\n| Priority | Unit Coverage | Integration Coverage | E2E Coverage |\n| -------- | ------------- | -------------------- | ------------------ |\n| P0 | >90% | >80% | All critical paths |\n| P1 | >80% | >60% | Main happy paths |\n| P2 | >60% | >40% | Smoke tests |\n| P3 | Best effort | Best effort | Manual only |\n\n## Priority Assignment Rules\n\n1. **Start with business impact** - What happens if this fails?\n2. **Consider probability** - How likely is failure?\n3. **Factor in detectability** - Would we know if it failed?\n4. **Account for recoverability** - Can we fix it quickly?\n\n## Priority Decision Tree\n\n```text\nIs it revenue-critical?\n├─ YES → P0\n└─ NO → Does it affect core user journey?\n ├─ YES → Is it high-risk?\n │ ├─ YES → P0\n │ └─ NO → P1\n └─ NO → Is it frequently used?\n ├─ YES → P1\n └─ NO → Is it customer-facing?\n ├─ YES → P2\n └─ NO → P3\n```\n\n## Test Execution Order\n\n1. Execute P0 tests first (fail fast on critical issues)\n2. Execute P1 tests second (core functionality)\n3. Execute P2 tests if time permits\n4. P3 tests only in full regression cycles\n\n## Continuous Adjustment\n\nReview and adjust priorities based on:\n\n- Production incident patterns\n- User feedback and complaints\n- Usage analytics\n- Test failure history\n- Business priority changes\n\n---\n\n## Automated Priority Classification\n\n### Example: Priority Calculator (Risk-Based Automation)\n\n```typescript\n// src/testing/priority-calculator.ts\n\nexport type Priority = 'P0' | 'P1' | 'P2' | 'P3';\n\nexport type PriorityFactors = {\n revenueImpact: 'critical' | 'high' | 'medium' | 'low' | 'none';\n userImpact: 'all' | 'majority' | 'some' | 'few' | 'minimal';\n securityRisk: boolean;\n complianceRequired: boolean;\n previousFailure: boolean;\n complexity: 'high' | 'medium' | 'low';\n usage: 'frequent' | 'regular' | 'occasional' | 'rare';\n};\n\n/**\n * Calculate test priority based on multiple factors\n * Mirrors the priority decision tree with objective criteria\n */\nexport function calculatePriority(factors: PriorityFactors): Priority {\n const { revenueImpact, userImpact, securityRisk, complianceRequired, previousFailure, complexity, usage } = factors;\n\n // P0: Revenue-critical, security, or compliance\n if (revenueImpact === 'critical' || securityRisk || complianceRequired || (previousFailure && revenueImpact === 'high')) {\n return 'P0';\n }\n\n // P0: High revenue + high complexity + frequent usage\n if (revenueImpact === 'high' && complexity === 'high' && usage === 'frequent') {\n return 'P0';\n }\n\n // P1: Core user journey (majority impacted + frequent usage)\n if (userImpact === 'all' || userImpact === 'majority') {\n if (usage === 'frequent' || complexity === 'high') {\n return 'P1';\n }\n }\n\n // P1: High revenue OR high complexity with regular usage\n if ((revenueImpact === 'high' && usage === 'regular') || (complexity === 'high' && usage === 'frequent')) {\n return 'P1';\n }\n\n // P2: Secondary features (some impact, occasional usage)\n if (userImpact === 'some' || usage === 'occasional') {\n return 'P2';\n }\n\n // P3: Rarely used, low impact\n return 'P3';\n}\n\n/**\n * Generate priority justification (for audit trail)\n */\nexport function justifyPriority(factors: PriorityFactors): string {\n const priority = calculatePriority(factors);\n const reasons: string[] = [];\n\n if (factors.revenueImpact === 'critical') reasons.push('critical revenue impact');\n if (factors.securityRisk) reasons.push('security-critical');\n if (factors.complianceRequired) reasons.push('compliance requirement');\n if (factors.previousFailure) reasons.push('regression prevention');\n if (factors.userImpact === 'all' || factors.userImpact === 'majority') {\n reasons.push(`impacts ${factors.userImpact} users`);\n }\n if (factors.complexity === 'high') reasons.push('high complexity');\n if (factors.usage === 'frequent') reasons.push('frequently used');\n\n return `${priority}: ${reasons.join(', ')}`;\n}\n\n/**\n * Example: Payment scenario priority calculation\n */\nconst paymentScenario: PriorityFactors = {\n revenueImpact: 'critical',\n userImpact: 'all',\n securityRisk: true,\n complianceRequired: true,\n previousFailure: false,\n complexity: 'high',\n usage: 'frequent',\n};\n\nconsole.log(calculatePriority(paymentScenario)); // 'P0'\nconsole.log(justifyPriority(paymentScenario));\n// 'P0: critical revenue impact, security-critical, compliance requirement, impacts all users, high complexity, frequently used'\n```\n\n### Example: Test Suite Tagging Strategy\n\n```typescript\n// tests/e2e/checkout.spec.ts\nimport { test, expect } from '@playwright/test';\n\n// Tag tests with priority for selective execution\ntest.describe('Checkout Flow', () => {\n test('valid payment completes successfully @p0 @smoke @revenue', async ({ page }) => {\n // P0: Revenue-critical happy path\n await page.goto('/checkout');\n await page.getByTestId('payment-method').selectOption('credit-card');\n await page.getByTestId('card-number').fill('4242424242424242');\n await page.getByRole('button', { name: 'Place Order' }).click();\n\n await expect(page.getByText('Order confirmed')).toBeVisible();\n });\n\n test('expired card shows user-friendly error @p1 @error-handling', async ({ page }) => {\n // P1: Core error scenario (frequent user impact)\n await page.goto('/checkout');\n await page.getByTestId('payment-method').selectOption('credit-card');\n await page.getByTestId('card-number').fill('4000000000000069'); // Test card: expired\n await page.getByRole('button', { name: 'Place Order' }).click();\n\n await expect(page.getByText('Card expired. Please use a different card.')).toBeVisible();\n });\n\n test('coupon code applies discount correctly @p2', async ({ page }) => {\n // P2: Secondary feature (nice-to-have)\n await page.goto('/checkout');\n await page.getByTestId('coupon-code').fill('SAVE10');\n await page.getByRole('button', { name: 'Apply' }).click();\n\n await expect(page.getByText('10% discount applied')).toBeVisible();\n });\n\n test('gift message formatting preserved @p3', async ({ page }) => {\n // P3: Cosmetic feature (rarely used)\n await page.goto('/checkout');\n await page.getByTestId('gift-message').fill('Happy Birthday!\\n\\nWith love.');\n await page.getByRole('button', { name: 'Place Order' }).click();\n\n // Message formatting preserved (linebreaks intact)\n await expect(page.getByTestId('order-summary')).toContainText('Happy Birthday!');\n });\n});\n```\n\n**Run tests by priority:**\n\n```bash\n# P0 only (smoke tests, 2-5 min)\nnpx playwright test --grep @p0\n\n# P0 + P1 (core functionality, 10-15 min)\nnpx playwright test --grep \"@p0|@p1\"\n\n# Full regression (all priorities, 30+ min)\nnpx playwright test\n```\n\n---\n\n## Integration with Risk Scoring\n\nPriority is **not derived from** risk score. Risk score (from `probability-impact.md`) classifies\nthe remediation action required (DOCUMENT/MONITOR/MITIGATE/BLOCK); priority is assigned separately\nusing the Priority Levels criteria and Priority Decision Tree above, which weigh business impact,\nuser reach, and workaround availability. The two axes correlate loosely, which is why the ranges\nbelow overlap: a score of 6-8 does not by itself resolve to P0 or P1.\n\n| Risk Score | Typical Priority | Rationale |\n| ---------- | ---------------- | ------------------------------------------ |\n| 9 | P0 | Critical blocker (probability=3, impact=3) |\n| 6-8 | P0 or P1 | High risk (requires mitigation) |\n| 4-5 | P1 or P2 | Medium risk (monitor closely) |\n| 1-3 | P2 or P3 | Low risk (document and defer) |\n\nTreat this table as a sanity check on a priority already assigned by the decision tree, not as an\nassignment rule: if a P3 scenario scores 8, that is a signal to revisit the priority call, not a\nmandate to overwrite it.\n\n**Example**: Risk score 9 (checkout API failure) is a BLOCK action; it also happens to be P0\nbecause it blocks a revenue-critical core journey with no workaround, per the decision tree.\n\n---\n\n## Priority Checklist\n\nBefore finalizing test priorities:\n\n- [ ] **Revenue impact assessed**: Payment, subscription, billing features → P0\n- [ ] **Security risks identified**: Auth, data exposure, injection attacks → P0\n- [ ] **Compliance requirements documented**: GDPR, PCI-DSS, SOC2 → P0\n- [ ] **User impact quantified**: >50% users → P0/P1, \u003c10% → P2/P3\n- [ ] **Previous failures reviewed**: Regression prevention → increase priority\n- [ ] **Complexity evaluated**: >500 LOC or multiple dependencies → increase priority\n- [ ] **Usage metrics consulted**: Frequent use → P0/P1, rare use → P2/P3\n- [ ] **Monitoring coverage confirmed**: Strong monitoring → can decrease priority\n- [ ] **Rollback capability verified**: Easy rollback → can decrease priority\n- [ ] **Priorities tagged in tests**: @p0, @p1, @p2, @p3 for selective execution\n\n## Integration Points\n\n- **Used in workflows**: `*automate` (priority-based test generation), `*test-design` (scenario prioritization), `*trace` (coverage validation by priority)\n- **Related fragments**: `risk-governance.md` (risk scoring), `probability-impact.md` (impact assessment), `selective-testing.md` (tag-based execution)\n- **Tools**: Playwright/Cypress grep for tag filtering, CI scripts for priority-based execution\n\n_Source: Risk-based testing practices, test prioritization strategies, production incident analysis._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12158,"content_sha256":"45dc87b018586c439d451b7b22c476e4ec70e8cd0ee28e9136dd2eac9fe34bef"},{"filename":"knowledge/test-quality.md","content":"# Test Quality Definition of Done\n\n## Principle\n\nTests must be deterministic, isolated, explicit, focused, and fast. Every test should execute in under 1.5 minutes, contain 1000 lines or fewer, avoid hard waits and branches that skip a step or an assertion, keep assertions visible in test bodies, and clean up after itself for parallel execution.\n\n## Rationale\n\nQuality tests provide reliable signal about application health. Flaky tests erode confidence and waste engineering time. Tests that use hard waits (`waitForTimeout(3000)`) are non-deterministic and slow. Tests with hidden assertions or conditional logic become unmaintainable. Large tests (>1000 lines) are hard to understand and debug. Slow tests (>1.5 min) block CI pipelines. Self-cleaning tests prevent state pollution in parallel runs.\n\n## Pattern Examples\n\n### Example 1: Deterministic Test Pattern\n\n**Context**: When writing tests, eliminate all sources of non-determinism: hard waits, branches that skip a step or an assertion, try-catch for flow control, and random data without seeds.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: Non-deterministic test with conditionals and hard waits\ntest('user can view dashboard - FLAKY', async ({ page }) => {\n await page.goto('/dashboard');\n await page.waitForTimeout(3000); // NEVER - arbitrary wait\n\n // Conditional flow control - test behavior varies\n if (await page.locator('[data-testid=\"welcome-banner\"]').isVisible()) {\n await page.click('[data-testid=\"dismiss-banner\"]');\n await page.waitForTimeout(500);\n }\n\n // Try-catch for flow control - hides real issues\n try {\n await page.click('[data-testid=\"load-more\"]');\n } catch (e) {\n // Silently continue - test passes even if button missing\n }\n\n // Random data without control\n const randomEmail = `user${Math.random()}@example.com`;\n await expect(page.getByText(randomEmail)).toBeVisible(); // Will fail randomly\n});\n\n// ✅ GOOD: Deterministic test with explicit waits\ntest('user can view dashboard', async ({ page, apiRequest }) => {\n const user = createUser({ email: '[email protected]', hasSeenWelcome: true });\n\n // Setup via API (fast, controlled)\n await apiRequest.post('/api/users', { data: user });\n\n // Network-first: Intercept BEFORE navigate\n const dashboardPromise = page.waitForResponse((resp) => resp.url().includes('/api/dashboard') && resp.status() === 200);\n\n await page.goto('/dashboard');\n\n // Wait for actual response, not arbitrary time\n const dashboardResponse = await dashboardPromise;\n const dashboard = await dashboardResponse.json();\n\n // Explicit assertions with controlled data\n await expect(page.getByText(`Welcome, ${user.name}`)).toBeVisible();\n await expect(page.getByTestId('dashboard-items')).toHaveCount(dashboard.items.length);\n\n // No conditionals - test always executes same path\n // No try-catch - failures bubble up clearly\n});\n\n// Cypress equivalent\ndescribe('Dashboard', () => {\n it('should display user dashboard', () => {\n const user = createUser({ email: '[email protected]', hasSeenWelcome: true });\n\n // Setup via task (fast, controlled)\n cy.task('db:seed', { users: [user] });\n\n // Network-first interception\n cy.intercept('GET', '**/api/dashboard').as('getDashboard');\n\n cy.visit('/dashboard');\n\n // Deterministic wait for response\n cy.wait('@getDashboard').then((interception) => {\n const dashboard = interception.response.body;\n\n // Explicit assertions\n cy.contains(`Welcome, ${user.name}`).should('be.visible');\n cy.get('[data-cy=\"dashboard-items\"]').should('have.length', dashboard.items.length);\n });\n });\n});\n```\n\n**Key Points**:\n\n- Replace `waitForTimeout()` with `waitForResponse()` or element state checks\n- Never branch on which actions run, and never let a branch leave an assertion unrun: tests should be deterministic\n- Avoid try-catch for flow control - let failures bubble up clearly\n- Use factory functions with controlled data, not `Math.random()`\n- Network-first pattern prevents race conditions\n\n### Example 2: Isolated Test with Cleanup\n\n**Context**: When tests create data, they must clean up after themselves to prevent state pollution in parallel runs. Use fixture auto-cleanup or explicit teardown.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: Test leaves data behind, pollutes other tests\ntest('admin can create user - POLLUTES STATE', async ({ page, apiRequest }) => {\n await page.goto('/admin/users');\n\n // Hardcoded email - collides in parallel runs\n await page.fill('[data-testid=\"email\"]', '[email protected]');\n await page.fill('[data-testid=\"name\"]', 'New User');\n await page.click('[data-testid=\"create-user\"]');\n\n await expect(page.getByText('User created')).toBeVisible();\n\n // NO CLEANUP - user remains in database\n // Next test run fails: \"Email already exists\"\n});\n\n// ✅ GOOD: Test cleans up with fixture auto-cleanup\n// playwright/support/fixtures/database-fixture.ts\nimport { test as base } from '@playwright/test';\nimport { deleteRecord, seedDatabase } from '../helpers/db-helpers';\n\ntype DatabaseFixture = {\n seedUser: (userData: Partial\u003cUser>) => Promise\u003cUser>;\n};\n\nexport const test = base.extend\u003cDatabaseFixture>({\n seedUser: async ({}, use) => {\n const createdUsers: string[] = [];\n\n const seedUser = async (userData: Partial\u003cUser>) => {\n const user = await seedDatabase('users', userData);\n createdUsers.push(user.id); // Track for cleanup\n return user;\n };\n\n await use(seedUser);\n\n // Auto-cleanup: Delete all users created during test\n for (const userId of createdUsers) {\n await deleteRecord('users', userId);\n }\n createdUsers.length = 0;\n },\n});\n\n// Use the fixture\ntest('admin can create user', async ({ page, seedUser }) => {\n // Create admin with unique data\n const admin = await seedUser({\n email: faker.internet.email(), // Unique each run\n role: 'admin',\n });\n\n await page.goto('/admin/users');\n\n const newUserEmail = faker.internet.email(); // Unique\n await page.fill('[data-testid=\"email\"]', newUserEmail);\n await page.fill('[data-testid=\"name\"]', 'New User');\n await page.click('[data-testid=\"create-user\"]');\n\n await expect(page.getByText('User created')).toBeVisible();\n\n // Verify in database\n const createdUser = await seedUser({ email: newUserEmail });\n expect(createdUser.email).toBe(newUserEmail);\n\n // Auto-cleanup happens via fixture teardown\n});\n\n// Cypress equivalent with explicit cleanup\ndescribe('Admin User Management', () => {\n const createdUserIds: string[] = [];\n\n afterEach(() => {\n // Cleanup: Delete all users created during test\n createdUserIds.forEach((userId) => {\n cy.task('db:delete', { table: 'users', id: userId });\n });\n createdUserIds.length = 0;\n });\n\n it('should create user', () => {\n const admin = createUser({ role: 'admin' });\n const newUser = createUser(); // Unique data via faker\n\n cy.task('db:seed', { users: [admin] }).then((result: any) => {\n createdUserIds.push(result.users[0].id);\n });\n\n cy.visit('/admin/users');\n cy.get('[data-cy=\"email\"]').type(newUser.email);\n cy.get('[data-cy=\"name\"]').type(newUser.name);\n cy.get('[data-cy=\"create-user\"]').click();\n\n cy.contains('User created').should('be.visible');\n\n // Track for cleanup\n cy.task('db:findByEmail', newUser.email).then((user: any) => {\n createdUserIds.push(user.id);\n });\n });\n});\n```\n\n**Key Points**:\n\n- Use fixtures with auto-cleanup via teardown (after `use()`)\n- Track all created resources in array during test execution\n- Use `faker` for unique data - prevents parallel collisions\n- Cypress: Use `afterEach()` with explicit cleanup\n- Never hardcode IDs or emails - always generate unique values\n\n### Example 3: Explicit Assertions in Tests\n\n**Context**: When validating test results, keep assertions visible in test bodies. Never hide assertions in helper functions - this obscures test intent and makes failures harder to diagnose.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: Assertions hidden in helper functions\n// helpers/api-validators.ts\nexport async function validateUserCreation(response: Response, expectedEmail: string) {\n const user = await response.json();\n expect(response.status()).toBe(201);\n expect(user.email).toBe(expectedEmail);\n expect(user.id).toBeTruthy();\n expect(user.createdAt).toBeTruthy();\n // Hidden assertions - not visible in test\n}\n\ntest('create user via API - OPAQUE', async ({ request }) => {\n const userData = createUser({ email: '[email protected]' });\n\n const response = await request.post('/api/users', { data: userData });\n\n // What assertions are running? Have to check helper.\n await validateUserCreation(response, userData.email);\n // When this fails, error is: \"validateUserCreation failed\" - NOT helpful\n});\n\n// ✅ GOOD: Assertions explicit in test\ntest('create user via API', async ({ request }) => {\n const userData = createUser({ email: '[email protected]' });\n\n const response = await request.post('/api/users', { data: userData });\n\n // All assertions visible - clear test intent\n expect(response.status()).toBe(201);\n\n const createdUser = await response.json();\n expect(createdUser.id).toBeTruthy();\n expect(createdUser.email).toBe(userData.email);\n expect(createdUser.name).toBe(userData.name);\n expect(createdUser.role).toBe('user');\n expect(createdUser.createdAt).toBeTruthy();\n expect(createdUser.isActive).toBe(true);\n\n // When this fails, error is: \"Expected role to be 'user', got 'admin'\" - HELPFUL\n});\n\n// ✅ ACCEPTABLE: Helper for data extraction, NOT assertions\n// helpers/api-extractors.ts\nexport async function extractUserFromResponse(response: Response): Promise\u003cUser> {\n const user = await response.json();\n return user; // Just extracts, no assertions\n}\n\ntest('create user with extraction helper', async ({ request }) => {\n const userData = createUser({ email: '[email protected]' });\n\n const response = await request.post('/api/users', { data: userData });\n\n // Extract data with helper (OK)\n const createdUser = await extractUserFromResponse(response);\n\n // But keep assertions in test (REQUIRED)\n expect(response.status()).toBe(201);\n expect(createdUser.email).toBe(userData.email);\n expect(createdUser.role).toBe('user');\n});\n\n// Cypress equivalent\ndescribe('User API', () => {\n it('should create user with explicit assertions', () => {\n const userData = createUser({ email: '[email protected]' });\n\n cy.request('POST', '/api/users', userData).then((response) => {\n // All assertions visible in test\n expect(response.status).to.equal(201);\n expect(response.body.id).to.exist;\n expect(response.body.email).to.equal(userData.email);\n expect(response.body.name).to.equal(userData.name);\n expect(response.body.role).to.equal('user');\n expect(response.body.createdAt).to.exist;\n expect(response.body.isActive).to.be.true;\n });\n });\n});\n\n// ✅ GOOD: Parametrized tests for soft assertions (bulk validation)\ntest.describe('User creation validation', () => {\n const testCases = [\n { field: 'email', value: '[email protected]', expected: '[email protected]' },\n { field: 'name', value: 'Test User', expected: 'Test User' },\n { field: 'role', value: 'admin', expected: 'admin' },\n { field: 'isActive', value: true, expected: true },\n ];\n\n for (const { field, value, expected } of testCases) {\n test(`should set ${field} correctly`, async ({ request }) => {\n const userData = createUser({ [field]: value });\n\n const response = await request.post('/api/users', { data: userData });\n const user = await response.json();\n\n // Parametrized assertion - still explicit\n expect(user[field]).toBe(expected);\n });\n }\n});\n```\n\n**Key Points**:\n\n- Never hide `expect()` calls in helper functions\n- Helpers can extract/transform data, but assertions stay in tests\n- Parametrized tests are acceptable for bulk validation (still explicit)\n- Explicit assertions make failures actionable: \"Expected X, got Y\"\n- Hidden assertions produce vague failures: \"Helper function failed\"\n\n### Example 4: Test Length Limits\n\n**Context**: When tests grow beyond 1000 lines, they become hard to understand, debug, and maintain. Refactor long tests by extracting setup helpers, splitting scenarios, or using fixtures.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: 1200-line monolithic test (truncated for example)\ntest('complete user journey - TOO LONG', async ({ page, request }) => {\n // 150 lines of setup\n const admin = createUser({ role: 'admin' });\n await request.post('/api/users', { data: admin });\n await page.goto('/login');\n await page.fill('[data-testid=\"email\"]', admin.email);\n await page.fill('[data-testid=\"password\"]', 'password123');\n await page.click('[data-testid=\"login\"]');\n await expect(page).toHaveURL('/dashboard');\n\n // 300 lines of user creation\n await page.goto('/admin/users');\n const newUser = createUser();\n await page.fill('[data-testid=\"email\"]', newUser.email);\n // ... 290 more lines of form filling, validation, etc.\n\n // 300 lines of permissions assignment\n await page.click('[data-testid=\"assign-permissions\"]');\n // ... 290 more lines\n\n // 300 lines of notification preferences\n await page.click('[data-testid=\"notification-settings\"]');\n // ... 290 more lines\n\n // 150 lines of cleanup\n await request.delete(`/api/users/${newUser.id}`);\n // ... 140 more lines\n\n // TOTAL: 1200 lines - impossible to understand or debug\n});\n\n// ✅ GOOD: Split into focused tests with shared fixture\n// playwright/support/fixtures/admin-fixture.ts\nexport const test = base.extend({\n adminPage: async ({ page, request }, use) => {\n // Shared setup: Login as admin\n const admin = createUser({ role: 'admin' });\n await request.post('/api/users', { data: admin });\n\n await page.goto('/login');\n await page.fill('[data-testid=\"email\"]', admin.email);\n await page.fill('[data-testid=\"password\"]', 'password123');\n await page.click('[data-testid=\"login\"]');\n await expect(page).toHaveURL('/dashboard');\n\n await use(page); // Provide logged-in page\n\n // Cleanup handled by fixture\n },\n});\n\n// Test 1: User creation (50 lines)\ntest('admin can create user', async ({ adminPage, seedUser }) => {\n await adminPage.goto('/admin/users');\n\n const newUser = createUser();\n await adminPage.fill('[data-testid=\"email\"]', newUser.email);\n await adminPage.fill('[data-testid=\"name\"]', newUser.name);\n await adminPage.click('[data-testid=\"role-dropdown\"]');\n await adminPage.click('[data-testid=\"role-user\"]');\n await adminPage.click('[data-testid=\"create-user\"]');\n\n await expect(adminPage.getByText('User created')).toBeVisible();\n await expect(adminPage.getByText(newUser.email)).toBeVisible();\n\n // Verify in database\n const created = await seedUser({ email: newUser.email });\n expect(created.role).toBe('user');\n});\n\n// Test 2: Permission assignment (60 lines)\ntest('admin can assign permissions', async ({ adminPage, seedUser }) => {\n const user = await seedUser({ email: faker.internet.email() });\n\n await adminPage.goto(`/admin/users/${user.id}`);\n await adminPage.click('[data-testid=\"assign-permissions\"]');\n await adminPage.check('[data-testid=\"permission-read\"]');\n await adminPage.check('[data-testid=\"permission-write\"]');\n await adminPage.click('[data-testid=\"save-permissions\"]');\n\n await expect(adminPage.getByText('Permissions updated')).toBeVisible();\n\n // Verify permissions assigned\n const response = await adminPage.request.get(`/api/users/${user.id}`);\n const updated = await response.json();\n expect(updated.permissions).toContain('read');\n expect(updated.permissions).toContain('write');\n});\n\n// Test 3: Notification preferences (70 lines)\ntest('admin can update notification preferences', async ({ adminPage, seedUser }) => {\n const user = await seedUser({ email: faker.internet.email() });\n\n await adminPage.goto(`/admin/users/${user.id}/notifications`);\n await adminPage.check('[data-testid=\"email-notifications\"]');\n await adminPage.uncheck('[data-testid=\"sms-notifications\"]');\n await adminPage.selectOption('[data-testid=\"frequency\"]', 'daily');\n await adminPage.click('[data-testid=\"save-preferences\"]');\n\n await expect(adminPage.getByText('Preferences saved')).toBeVisible();\n\n // Verify preferences\n const response = await adminPage.request.get(`/api/users/${user.id}/preferences`);\n const prefs = await response.json();\n expect(prefs.emailEnabled).toBe(true);\n expect(prefs.smsEnabled).toBe(false);\n expect(prefs.frequency).toBe('daily');\n});\n\n// TOTAL: 3 tests × 60 lines avg = 180 lines\n// Each test is focused, debuggable, and at or under 1000 lines\n```\n\n**Key Points**:\n\n- Split monolithic tests into focused scenarios (≤1000 lines each)\n- Extract common setup into fixtures (auto-runs for each test)\n- Each test validates one concern (user creation, permissions, preferences)\n- Failures are easier to diagnose: \"Permission assignment failed\" vs \"Complete journey failed\"\n- Tests can run in parallel (isolated concerns)\n\n### Example 5: Execution Time Optimization\n\n**Context**: When tests take longer than 1.5 minutes, they slow CI pipelines and feedback loops. Optimize by using API setup instead of UI navigation, parallelizing independent operations, and avoiding unnecessary waits.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: 4-minute test (slow setup, sequential operations)\ntest('user completes order - SLOW (4 min)', async ({ page }) => {\n // Step 1: Manual signup via UI (90 seconds)\n await page.goto('/signup');\n await page.fill('[data-testid=\"email\"]', '[email protected]');\n await page.fill('[data-testid=\"password\"]', 'password123');\n await page.fill('[data-testid=\"confirm-password\"]', 'password123');\n await page.fill('[data-testid=\"name\"]', 'Buyer User');\n await page.click('[data-testid=\"signup\"]');\n await page.waitForURL('/verify-email'); // Wait for email verification\n // ... manual email verification flow\n\n // Step 2: Manual product creation via UI (60 seconds)\n await page.goto('/admin/products');\n await page.fill('[data-testid=\"product-name\"]', 'Widget');\n // ... 20 more fields\n await page.click('[data-testid=\"create-product\"]');\n\n // Step 3: Navigate to checkout (30 seconds)\n await page.goto('/products');\n await page.waitForTimeout(5000); // Unnecessary hard wait\n await page.click('[data-testid=\"product-widget\"]');\n await page.waitForTimeout(3000); // Unnecessary\n await page.click('[data-testid=\"add-to-cart\"]');\n await page.waitForTimeout(2000); // Unnecessary\n\n // Step 4: Complete checkout (40 seconds)\n await page.goto('/checkout');\n await page.waitForTimeout(5000); // Unnecessary\n await page.fill('[data-testid=\"credit-card\"]', '4111111111111111');\n // ... more form filling\n await page.click('[data-testid=\"submit-order\"]');\n await page.waitForTimeout(10000); // Unnecessary\n\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n\n // TOTAL: ~240 seconds (4 minutes)\n});\n\n// ✅ GOOD: 45-second test (API setup, parallel ops, deterministic waits)\ntest('user completes order', async ({ page, apiRequest }) => {\n // Step 1: API setup (parallel, 5 seconds total)\n const [user, product] = await Promise.all([\n // Create user via API (fast)\n apiRequest\n .post('/api/users', {\n data: createUser({\n email: '[email protected]',\n emailVerified: true, // Skip verification\n }),\n })\n .then((r) => r.json()),\n\n // Create product via API (fast)\n apiRequest\n .post('/api/products', {\n data: createProduct({\n name: 'Widget',\n price: 29.99,\n stock: 10,\n }),\n })\n .then((r) => r.json()),\n ]);\n\n // Step 2: Auth setup via storage state (instant, 0 seconds)\n await page.context().addCookies([\n {\n name: 'auth_token',\n value: user.token,\n domain: 'localhost',\n path: '/',\n },\n ]);\n\n // Step 3: Network-first interception BEFORE navigation (10 seconds)\n const cartPromise = page.waitForResponse('**/api/cart');\n const orderPromise = page.waitForResponse('**/api/orders');\n\n await page.goto(`/products/${product.id}`);\n await page.click('[data-testid=\"add-to-cart\"]');\n await cartPromise; // Deterministic wait (no hard wait)\n\n // Step 4: Checkout with network waits (30 seconds)\n await page.goto('/checkout');\n await page.fill('[data-testid=\"credit-card\"]', '4111111111111111');\n await page.fill('[data-testid=\"cvv\"]', '123');\n await page.fill('[data-testid=\"expiry\"]', '12/25');\n await page.click('[data-testid=\"submit-order\"]');\n await orderPromise; // Deterministic wait (no hard wait)\n\n await expect(page.getByText('Order Confirmed')).toBeVisible();\n await expect(page.getByText(`Order #${product.id}`)).toBeVisible();\n\n // TOTAL: ~45 seconds (6x faster)\n});\n\n// Cypress equivalent\ndescribe('Order Flow', () => {\n it('should complete purchase quickly', () => {\n // Step 1: API setup (parallel, fast)\n const user = createUser({ emailVerified: true });\n const product = createProduct({ name: 'Widget', price: 29.99 });\n\n cy.task('db:seed', { users: [user], products: [product] });\n\n // Step 2: Auth setup via session (instant)\n cy.setCookie('auth_token', user.token);\n\n // Step 3: Network-first interception\n cy.intercept('POST', '**/api/cart').as('addToCart');\n cy.intercept('POST', '**/api/orders').as('createOrder');\n\n cy.visit(`/products/${product.id}`);\n cy.get('[data-cy=\"add-to-cart\"]').click();\n cy.wait('@addToCart'); // Deterministic wait\n\n // Step 4: Checkout\n cy.visit('/checkout');\n cy.get('[data-cy=\"credit-card\"]').type('4111111111111111');\n cy.get('[data-cy=\"cvv\"]').type('123');\n cy.get('[data-cy=\"expiry\"]').type('12/25');\n cy.get('[data-cy=\"submit-order\"]').click();\n cy.wait('@createOrder'); // Deterministic wait\n\n cy.contains('Order Confirmed').should('be.visible');\n cy.contains(`Order #${product.id}`).should('be.visible');\n });\n});\n\n// Additional optimization: Shared auth state (0 seconds per test)\n// playwright/support/global-setup.ts\nexport default async function globalSetup() {\n const browser = await chromium.launch();\n const page = await browser.newPage();\n\n // Create admin user once for all tests\n const admin = createUser({ role: 'admin', emailVerified: true });\n await page.request.post('/api/users', { data: admin });\n\n // Login once, save session\n await page.goto('/login');\n await page.fill('[data-testid=\"email\"]', admin.email);\n await page.fill('[data-testid=\"password\"]', 'password123');\n await page.click('[data-testid=\"login\"]');\n\n // Save auth state for reuse\n await page.context().storageState({ path: 'playwright/.auth/admin.json' });\n\n await browser.close();\n}\n\n// Use shared auth in tests (instant)\ntest.use({ storageState: 'playwright/.auth/admin.json' });\n\ntest('admin action', async ({ page }) => {\n // Already logged in - no auth overhead (0 seconds)\n await page.goto('/admin');\n // ... test logic\n});\n```\n\n**Key Points**:\n\n- Use API for data setup (10-50x faster than UI)\n- Run independent operations in parallel (`Promise.all`)\n- Replace hard waits with deterministic waits (`waitForResponse`)\n- Reuse auth sessions via `storageState` (Playwright) or `setCookie` (Cypress)\n- Skip unnecessary flows (email verification, multi-step signups)\n\n### Example 6: Committed Skips and Committed Focus\n\n**Context**: A suite reports green. Two of the ways it does that have nothing to do with the code being correct: a test that was turned off, and a test that turned every one of its siblings off.\n\nA skip is not automatically a defect. A skip whose reason nobody can read is. The reason belongs on the line or the line directly above it, it has to name the condition that will make the test runnable again, and it has to still be true. A `FIXME` pointing at a bug closed six months ago is a deleted test with extra steps. If you cannot write that reason, delete the test; a deleted test is honest about the coverage you no longer have, and a permanently skipped one is not.\n\nFocus is different, and worse. `.only` is a debugging tool that changes what the whole file runs. Committed, the file still passes, still reports as a passing file, and covers one test. Nothing in the output says the other nineteen did not run.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: the reason is gone, so nobody can tell whether this is still needed\ntest.skip('checkout applies the promo code', async ({ page }) => {\n /* ... */\n});\n\n// ✅ GOOD: the condition to remove the skip is on the line\n// FIXME(TEA-412): promo service returns 500 in staging; re-enable when TEA-412 lands\ntest.skip('checkout applies the promo code', async ({ page }) => {\n /* ... */\n});\n\n// ❌ BAD: the other tests in this file no longer run, and nothing says so\ntest.only('checkout applies the promo code', async ({ page }) => {\n /* ... */\n});\n```\n\n```python\n# ❌ BAD: skipped with no reason anyone can act on\[email protected]\ndef test_invoice_voids_after_payment():\n ...\n\n# ✅ GOOD: pytest carries the reason in the marker itself\[email protected](reason=\"void endpoint returns 500 upstream, see TEA-412\")\ndef test_invoice_voids_after_payment():\n ...\n```\n\n```java\n// ❌ BAD: JUnit's bare form records nothing\n@Disabled\nvoid settlementRetriesWithBackoff() { }\n\n// ✅ GOOD: the annotation takes the reason\n@Disabled(\"flaky against the shared broker; unblocked by TEA-412\")\nvoid settlementRetriesWithBackoff() { }\n```\n\n**Key Points**:\n\n- A skip with a documented, still-true reason is acceptable; a bare one is not\n- Prefer deleting over skipping indefinitely: coverage you admit losing beats coverage you pretend to have\n- `.only`, `fdescribe`, and `fit` must never be committed: they disable siblings silently\n- Guard both in CI, not only in review: a grep in the pipeline costs nothing and a committed `.only` costs a release\n\n### Example 7: Assertions That Cannot Fail\n\n**Context**: Example 3 keeps assertions visible. Visible is not the same as meaningful. Three shapes execute, look like assertions in the diff, and prove nothing: one compares a value to itself, one checks the mock instead of the system, and one never runs at all. All three are worse than having no test, because the suite reports green and the coverage number goes up.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: compares a value to itself; passes no matter what the app does\nexpect(true).toBe(true);\nexpect(user.id).toBe(user.id);\n\n// ✅ GOOD: the expectation could differ from the actual\nexpect(response.status()).toBe(201);\nexpect(user.email).toBe(userData.email);\n\n// ❌ BAD: the only assertion is against the mock this test configured.\n// Nothing called into the system between setting it up and checking it,\n// so this proves the mocking library works.\nconst repo = { save: vi.fn().mockResolvedValue({ id: 1 }) };\nrepo.save({ name: 'Ada' });\nexpect(repo.save).toHaveBeenCalledWith({ name: 'Ada' });\n\n// ✅ GOOD: the system under test is what calls the mock, and the assertion\n// is about what the system returned\nconst repo = { save: vi.fn().mockResolvedValue({ id: 1 }) };\nconst created = await createAccount(repo, { name: 'Ada' });\nexpect(created.id).toBe(1);\nexpect(repo.save).toHaveBeenCalledWith({ name: 'Ada' });\n\n// ❌ BAD: unreachable. The return happens first.\ntest('rejects an expired token', async () => {\n const result = await authorize(expiredToken);\n return;\n expect(result.ok).toBe(false);\n});\n\n// ❌ BAD: unreachable. The happy path never enters the catch, so a passing\n// run asserts nothing and a broken run is swallowed.\ntry {\n await authorize(expiredToken);\n} catch (error) {\n expect(error.code).toBe('EXPIRED');\n}\n\n// ✅ GOOD: assert on the rejection itself, against the same property the\n// catch block was checking. `rejects.toThrow('EXPIRED')` matches the error\n// MESSAGE, so swapping it in here would quietly assert something else.\nawait expect(authorize(expiredToken)).rejects.toMatchObject({ code: 'EXPIRED' });\n```\n\n```python\n# ❌ BAD: tautological; true for every possible value of total\nassert total == total\n\n# ✅ GOOD\nassert total == Decimal(\"42.00\")\n```\n\n**Key Points**:\n\n- If the assertion would pass against a completely broken implementation, it is not an assertion\n- Configuring a mock and then asserting on that same mock, with no call into the system between, tests the mocking library\n- An assertion after an unconditional `return`, or inside a `catch` the happy path never enters, or inside a callback the test never awaits, does not run\n- Prefer `rejects`/`raises` forms over `try`/`catch` around the thing you expect to throw: they fail when nothing throws\n\n**A branch in a test is a defect only when it can hide a wrong result.** An `if` that leaves a path with no assertion skips the check, whatever decides the branch: a guard on the system's own output skips it exactly when the system is wrong, and a parameter guard with no `else` skips it for every other case. An expected value picked by a branch on the system's output makes the test agree with whatever the system did, and so does a branch the system's output decides when each path asserts a literal: `if response.ok: assert body == order` with `else: assert response.status_code == 404` passes a regression that answers 404 for a valid order.\n\n```python\n# ❌ BAD: the assertion runs only when the system already returned a lower total\ntotal = apply_discount(cart, \"SAVE10\")\nif total \u003c cart.total():\n assert total == 18.0\n\n# ✅ GOOD: always asserted\nassert apply_discount(cart, \"SAVE10\") == 18.0\n\n# ✅ ACCEPTABLE, a readability note at most: a test input decides the branch and every path asserts the expected value\[email protected]((\"code\", \"expected\"), [(\"SAVE10\", 0.10), (\"NOPE\", None)])\ndef test_lookup(code, expected):\n rate = lookup_discount(code)\n if expected is None:\n assert rate is None\n else:\n assert rate == expected\n```\n\nThe branch runs a different line per case, but each case still checks the expected value, so a wrong rate fails either way. Drop the `else` and the cases that miss the `if` assert nothing, which is H3. Moving the expectation into the parameter table (`assert rate == expected`) makes the test easier to audit, and that is advice with no severity.\n\n**The assertion that can fail and still cannot tell you anything.** One step past the three above sits an assertion that would fail on a wrong type and passes on every wrong value of the right type. It is not an assertion that cannot fail, so none of the rows above reach it, and it is the most common way a test looks thorough and proves nothing.\n\n```typescript\n// ❌ BAD: every wrong id of the right type passes\nconst created = await service.createOrder(input);\nexpect(typeof created.id).toBe('string');\nexpect(created.total).toBeDefined();\n\n// ✅ GOOD: the value is constrained, not the shape\nexpect(created.id).toBe(input.idempotencyKey);\nexpect(created.total).toBe(4299);\n```\n\n`toBeDefined`, `toBeTruthy`, `not.toBeNull`, a bare `toBeInstanceOf`, and `toHaveProperty` with no expected value are the same shape wearing different names. A value test that accepts every wrong value of the right shape (`assert receipt.total is not None` where the total is the behavior) is the defect; a test whose purpose is that one type is assignable to another is a different test.\n\n```python\n# ❌ BAD: the total is the behavior, and any total passes\nassert receipt.total is not None\n\n# ✅ GOOD\nassert receipt.total == 20.0\n\n# ✅ NOT A VIOLATION when CI runs mypy: the typed assignment is the assertion\nresult: Result[Receipt] = wrap(receipt)\nwidened: Result[object] = result # fails the static check if covariance breaks\n```\n\nRead the project's CI and type-check configuration before judging such a test. With a static type check that CI runs over the test file (mypy, pyright or `tsc` invoked by the workflow or a script it calls, with the file inside its file list or `include` and no `exclude` or `ignore_errors` for it), the check is what fails when assignability breaks, and the runtime line beside it is incidental. A checker that is only configured (`[tool.mypy]`, a `tsconfig.json`) and never run does not count: the same test asserts nothing about the type and the shape-only finding stands. Three more cases are not violations: a value assertion standing beside the type check, a value the system generates that the test genuinely cannot predict such as a server-assigned id or a timestamp, where the line should say why the type is all there is to assert, and a presence assertion about a UI element such as `toBeVisible`, where being on the screen is the behavior under test.\n\n### Example 8: Suite Structure and One Dialect\n\n**Context**: These do not make a test wrong. They make a failure expensive to read, which is the same cost paid every time the suite goes red for the next several years.\n\nA test that asserts against three unrelated subjects does not localize: the failure says the test broke, not which behavior broke. Count subjects, not `expect` calls: three assertions about one response is one concern, and one assertion each about a response, a database row, and an email is three. An ungrouped file prints failures with no subject line. Nesting past three levels means the reader reconstructs the setup from four `beforeEach` blocks before they can read the test. And a file that mixes assertion dialects makes every reader translate between two styles for no benefit.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: three unrelated subjects; a failure does not say which one broke\ntest('checkout works correctly', async ({ page, request }) => {\n await checkout(page);\n expect(await orderCount(request)).toBe(1); // subject: the order API\n expect(await inventoryFor(request, 'sku-1')).toBe(9); // subject: inventory\n expect(await lastEmail()).toContain('Order confirmed'); // subject: email\n});\n\n// ✅ GOOD: one subject per test, grouped\ndescribe('checkout', () => {\n test('records the order', async ({ request }) => {\n /* one subject */\n });\n test('decrements inventory for the purchased sku', async ({ request }) => {\n /* one subject */\n });\n test('sends the confirmation email', async () => {\n /* one subject */\n });\n});\n\n// ❌ BAD: two dialects in one file\nexpect(response.status()).toBe(200);\nassert.equal(body.role, 'admin');\n\n// ✅ GOOD: pick the house dialect and keep it\nexpect(response.status()).toBe(200);\nexpect(body.role).toBe('admin');\n```\n\n**Key Points**:\n\n- One concern per test, counted by subject rather than by `expect` call\n- Group with `describe`/`context` once a file has three or more tests, so failures print with a subject\n- Keep `describe` nesting and block nesting at three levels or fewer\n- One assertion dialect per file, matching whatever the repo already uses\n\n## Integration Points\n\n- **Used in workflows**: `*atdd` (test generation quality), `*automate` (test expansion quality), `*test-review` (quality validation)\n- **Related fragments**:\n - `network-first.md` - Deterministic waiting strategies\n - `data-factories.md` - Isolated, parallel-safe data patterns\n - `fixture-architecture.md` - Setup extraction and cleanup\n - `test-levels-framework.md` - Choosing appropriate test granularity for speed\n - `confidence-gate.md` - Agent reliability gate that protects DoD compliance during LLM-assisted test generation\n\n## Core Quality Checklist\n\nEvery test must pass these criteria:\n\n- [ ] **No Hard Waits** - Use `waitForResponse`, `waitForLoadState`, or element state (not `waitForTimeout`)\n- [ ] **No Conditionals** - An assertion is never skipped, swallowed or selected from the system's own output (no `if` that leaves a path with no assertion, no try/catch for flow control); a branch that asserts the expected value on every path is a readability note at most\n- [ ] **≤ 1000 Lines** - Keep tests focused; split large tests or extract setup to fixtures\n- [ ] **\u003c 1.5 Minutes** - Optimize with API setup, parallel operations, and shared auth\n- [ ] **Self-Cleaning** - Use fixtures with auto-cleanup or explicit `afterEach()` teardown\n- [ ] **Explicit Assertions** - Keep `expect()` calls in test bodies, not hidden in helpers\n- [ ] **Unique Data** - Use `faker` for dynamic data; never hardcode IDs or emails\n- [ ] **Parallel-Safe** - Tests don't share state; run successfully with `--workers=4`\n- [ ] **No Committed Focus** - No `.only`, `fdescribe`, or `fit` reaches the branch\n- [ ] **Skips Documented** - Every skip carries a still-true reason naming what would re-enable it\n- [ ] **Assertions Can Fail** - No self-comparison, no assertion against only the mock the test configured, nothing after an unconditional `return`\n- [ ] **One Concern** - Counted by subject, not by `expect` call\n- [ ] **Grouped and Shallow** - `describe`/`context` once a file has three tests; nesting three levels or fewer\n- [ ] **One Dialect** - The file uses a single assertion style\n\n_Source: Murat quality checklist, Definition of Done requirements (lines 370-381, 406-422)._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":37022,"content_sha256":"82225c204ed84349d694381bb024213848366f32d7838093617d57362a36089d"},{"filename":"knowledge/timing-debugging.md","content":"# Timing Debugging and Race Condition Fixes\n\n## Principle\n\nRace conditions arise when tests make assumptions about asynchronous timing (network, animations, state updates). **Deterministic waiting** eliminates flakiness by explicitly waiting for observable events (network responses, element state changes) instead of arbitrary timeouts.\n\n## Rationale\n\n**The Problem**: Tests pass locally but fail in CI (different timing), or pass/fail randomly (race conditions). Hard waits (`waitForTimeout`, `sleep`) mask timing issues without solving them.\n\n**The Solution**: Replace all hard waits with event-based waits (`waitForResponse`, `waitFor({ state })`). Implement network-first pattern (intercept before navigate). Use explicit state checks (loading spinner detached, data loaded). This makes tests deterministic regardless of network speed or system load.\n\n**Why This Matters**:\n\n- Eliminates flaky tests (0 tolerance for timing-based failures)\n- Works consistently across environments (local, CI, production-like)\n- Faster test execution (no unnecessary waits)\n- Clearer test intent (explicit about what we're waiting for)\n\n## Pattern Examples\n\n### Example 1: Race Condition Identification (Network-First Pattern)\n\n**Context**: Prevent race conditions by intercepting network requests before navigation\n\n**Implementation**:\n\n```typescript\n// tests/timing/race-condition-prevention.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Race Condition Prevention Patterns', () => {\n test('❌ Anti-Pattern: Navigate then intercept (race condition)', async ({ page, context }) => {\n // BAD: Navigation starts before interception ready\n await page.goto('/products'); // ⚠️ Race! API might load before route is set\n\n await context.route('**/api/products', (route) => {\n route.fulfill({ status: 200, body: JSON.stringify({ products: [] }) });\n });\n\n // Test may see real API response or mock (non-deterministic)\n });\n\n test('✅ Pattern: Intercept BEFORE navigate (deterministic)', async ({ page, context }) => {\n // GOOD: Interception ready before navigation\n await context.route('**/api/products', (route) => {\n route.fulfill({\n status: 200,\n contentType: 'application/json',\n body: JSON.stringify({\n products: [\n { id: 1, name: 'Product A', price: 29.99 },\n { id: 2, name: 'Product B', price: 49.99 },\n ],\n }),\n });\n });\n\n const responsePromise = page.waitForResponse('**/api/products');\n\n await page.goto('/products'); // Navigation happens AFTER route is ready\n await responsePromise; // Explicit wait for network\n\n // Test sees mock response reliably (deterministic)\n await expect(page.getByText('Product A')).toBeVisible();\n });\n\n test('✅ Pattern: Wait for element state change (loading → loaded)', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Wait for loading indicator to appear (confirms load started)\n await page.getByTestId('loading-spinner').waitFor({ state: 'visible' });\n\n // Wait for loading indicator to disappear (confirms load complete)\n await page.getByTestId('loading-spinner').waitFor({ state: 'detached' });\n\n // Content now reliably visible\n await expect(page.getByTestId('dashboard-data')).toBeVisible();\n });\n\n test('✅ Pattern: Explicit visibility check (not just presence)', async ({ page }) => {\n await page.goto('/modal-demo');\n\n await page.getByRole('button', { name: 'Open Modal' }).click();\n\n // ❌ Bad: Element exists but may not be visible yet\n // await expect(page.getByTestId('modal')).toBeAttached()\n\n // ✅ Good: Wait for visibility (accounts for animations)\n await expect(page.getByTestId('modal')).toBeVisible();\n await expect(page.getByRole('heading', { name: 'Modal Title' })).toBeVisible();\n });\n\n test('❌ Anti-Pattern: waitForLoadState(\"networkidle\") in SPAs', async ({ page }) => {\n // ⚠️ Deprecated for SPAs (WebSocket connections never idle)\n // await page.goto('/dashboard')\n // await page.waitForLoadState('networkidle') // May timeout in SPAs\n\n // ✅ Better: Wait for specific API response\n const responsePromise = page.waitForResponse('**/api/dashboard');\n await page.goto('/dashboard');\n await responsePromise;\n\n await expect(page.getByText('Dashboard loaded')).toBeVisible();\n });\n});\n```\n\n**Key Points**:\n\n- Network-first: ALWAYS intercept before navigate (prevents race conditions)\n- State changes: Wait for loading spinner detached (explicit load completion)\n- Visibility vs presence: `toBeVisible()` accounts for animations, `toBeAttached()` doesn't\n- Avoid networkidle: Unreliable in SPAs (WebSocket, polling connections)\n- Explicit waits: Document exactly what we're waiting for\n\n---\n\n### Example 2: Deterministic Waiting Patterns (Event-Based, Not Time-Based)\n\n**Context**: Replace all hard waits with observable event waits\n\n**Implementation**:\n\n```typescript\n// tests/timing/deterministic-waits.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Deterministic Waiting Patterns', () => {\n test('waitForResponse() with URL pattern', async ({ page }) => {\n const responsePromise = page.waitForResponse('**/api/products');\n\n await page.goto('/products');\n await responsePromise; // Deterministic (waits for exact API call)\n\n await expect(page.getByText('Products loaded')).toBeVisible();\n });\n\n test('waitForResponse() with predicate function', async ({ page }) => {\n const responsePromise = page.waitForResponse((resp) => resp.url().includes('/api/search') && resp.status() === 200);\n\n await page.goto('/search');\n await page.getByPlaceholder('Search').fill('laptop');\n await page.getByRole('button', { name: 'Search' }).click();\n\n await responsePromise; // Wait for successful search response\n\n await expect(page.getByTestId('search-results')).toBeVisible();\n });\n\n test('waitForFunction() for custom conditions', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Wait for custom JavaScript condition\n await page.waitForFunction(() => {\n const element = document.querySelector('[data-testid=\"user-count\"]');\n return element && parseInt(element.textContent || '0') > 0;\n });\n\n // User count now loaded\n await expect(page.getByTestId('user-count')).not.toHaveText('0');\n });\n\n test('waitFor() element state (attached, visible, hidden, detached)', async ({ page }) => {\n await page.goto('/products');\n\n // Wait for element to be attached to DOM\n await page.getByTestId('product-list').waitFor({ state: 'attached' });\n\n // Wait for element to be visible (animations complete)\n await page.getByTestId('product-list').waitFor({ state: 'visible' });\n\n // Perform action\n await page.getByText('Product A').click();\n\n // Wait for modal to be hidden (close animation complete)\n await page.getByTestId('modal').waitFor({ state: 'hidden' });\n });\n\n test('Cypress: cy.wait() with aliased intercepts', async () => {\n // Cypress example (not Playwright)\n /*\n cy.intercept('GET', '/api/products').as('getProducts')\n cy.visit('/products')\n cy.wait('@getProducts') // Deterministic wait for specific request\n\n cy.get('[data-testid=\"product-list\"]').should('be.visible')\n */\n });\n});\n```\n\n**Key Points**:\n\n- `waitForResponse()`: Wait for specific API calls (URL pattern or predicate)\n- `waitForFunction()`: Wait for custom JavaScript conditions\n- `waitFor({ state })`: Wait for element state changes (attached, visible, hidden, detached)\n- Cypress `cy.wait('@alias')`: Deterministic wait for aliased intercepts\n- All waits are event-based (not time-based)\n\n---\n\n### Example 3: Timing Anti-Patterns (What NEVER to Do)\n\n**Context**: Common timing mistakes that cause flakiness\n\n**Problem Examples**:\n\n```typescript\n// tests/timing/anti-patterns.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest.describe('Timing Anti-Patterns to Avoid', () => {\n test('❌ NEVER: page.waitForTimeout() (arbitrary delay)', async ({ page }) => {\n await page.goto('/dashboard');\n\n // ❌ Bad: Arbitrary 3-second wait (flaky)\n // await page.waitForTimeout(3000)\n // Problem: Might be too short (CI slower) or too long (wastes time)\n\n // ✅ Good: Wait for observable event\n await page.waitForResponse('**/api/dashboard');\n await expect(page.getByText('Dashboard loaded')).toBeVisible();\n });\n\n test('❌ NEVER: cy.wait(number) without alias (arbitrary delay)', async () => {\n // Cypress example\n /*\n // ❌ Bad: Arbitrary delay\n cy.visit('/products')\n cy.wait(2000) // Flaky!\n\n // ✅ Good: Wait for specific request\n cy.intercept('GET', '/api/products').as('getProducts')\n cy.visit('/products')\n cy.wait('@getProducts') // Deterministic\n */\n });\n\n test('❌ NEVER: Multiple hard waits in sequence (compounding delays)', async ({ page }) => {\n await page.goto('/checkout');\n\n // ❌ Bad: Stacked hard waits (6+ seconds wasted)\n // await page.waitForTimeout(2000) // Wait for form\n // await page.getByTestId('email').fill('[email protected]')\n // await page.waitForTimeout(1000) // Wait for validation\n // await page.getByTestId('submit').click()\n // await page.waitForTimeout(3000) // Wait for redirect\n\n // ✅ Good: Event-based waits (no wasted time)\n await page.getByTestId('checkout-form').waitFor({ state: 'visible' });\n await page.getByTestId('email').fill('[email protected]');\n await page.waitForResponse('**/api/validate-email');\n await page.getByTestId('submit').click();\n await page.waitForURL('**/confirmation');\n });\n\n test('❌ NEVER: waitForLoadState(\"networkidle\") in SPAs', async ({ page }) => {\n // ❌ Bad: Unreliable in SPAs (WebSocket connections never idle)\n // await page.goto('/dashboard')\n // await page.waitForLoadState('networkidle') // Timeout in SPAs!\n\n // ✅ Good: Wait for specific API responses\n await page.goto('/dashboard');\n await page.waitForResponse('**/api/dashboard');\n await page.waitForResponse('**/api/user');\n await expect(page.getByTestId('dashboard-content')).toBeVisible();\n });\n\n test('❌ NEVER: Sleep/setTimeout in tests', async ({ page }) => {\n await page.goto('/products');\n\n // ❌ Bad: Node.js sleep (blocks test thread)\n // await new Promise(resolve => setTimeout(resolve, 2000))\n\n // ✅ Good: Playwright auto-waits for element\n await expect(page.getByText('Products loaded')).toBeVisible();\n });\n});\n```\n\n**Why These Fail**:\n\n- **Hard waits**: Arbitrary timeouts (too short → flaky, too long → slow)\n- **Stacked waits**: Compound delays (wasteful, unreliable)\n- **networkidle**: Broken in SPAs (WebSocket/polling never idle)\n- **Sleep**: Blocks execution (wastes time, doesn't solve race conditions)\n\n**Better Approach**: Use event-based waits from examples above\n\n---\n\n### Example 4: Fixtures Derived From the Live Clock\n\n**Context**: A hard wait makes a test slow and flaky. Reading the live clock makes it flaky on a schedule nobody can reproduce. The test that builds an expiry, a token lifetime, a TTL, or a scheduling boundary from `Date.now()` passes all day and fails at a month boundary, at midnight UTC, on the last day of February, or on the CI runner whose clock drifted four seconds.\n\nThis is a HIGH rather than a MEDIUM because of what these values usually govern. A token lifetime computed from the wall clock is a security boundary tested against a moving target: the test cannot distinguish \"the expiry logic is correct\" from \"the expiry has not happened yet.\"\n\nThe fix is to control time rather than to read it. Freeze it, then move it deliberately to the boundary the behavior is about.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: the boundary moves with the clock, and the failure lands on a Tuesday\nconst token = issueToken({ expiresAt: Date.now() + 3600_000 });\nawait sleep(1000);\nexpect(isExpired(token)).toBe(false); // proves nothing about expiry\n\n// ✅ GOOD: freeze, then step across the boundary on purpose.\n// Restore in afterEach, never as the last line of the test: an assertion that\n// throws would skip that line and leave every later test on a fake clock, which\n// is the unreset-shared-state defect wearing a different hat.\nafterEach(() => {\n vi.useRealTimers();\n});\n\ntest('the token expires at its TTL', () => {\n vi.useFakeTimers();\n vi.setSystemTime(new Date('2026-01-01T00:00:00Z'));\n const token = issueToken({ ttlSeconds: 3600 });\n expect(isExpired(token)).toBe(false);\n vi.advanceTimersByTime(3601_000);\n expect(isExpired(token)).toBe(true); // the boundary is what is under test\n});\n```\n\n```python\n# ❌ BAD: the fixture is different on every run\nexpires_at = time.time() + 3600\n\n# ✅ GOOD: pin the clock, then move it\nwith freeze_time(\"2026-01-01T00:00:00Z\") as frozen:\n token = issue_token(ttl_seconds=3600)\n assert not is_expired(token)\n frozen.tick(3601)\n assert is_expired(token)\n```\n\n**Key Points**:\n\n- A time-bounded value built from the live clock is not a fixture, it is a variable\n- Freeze the clock and advance it deliberately; the boundary is the behavior under test\n- Restore real timers in `afterEach`, so a failing assertion cannot leave a later test on a fake clock\n- Where the production code already takes an injectable clock, pass one. A production seam the application itself uses beats a test-only seam that only the suite knows about\n- A timestamp merely stamped into a record and never asserted against is not this defect; the row is about values that govern an expiry, a lifetime, a TTL, or a schedule\n\n### Example 5: Promises Nobody Awaited\n\n**Context**: The most common way for a race condition to be in the test rather than in the application. A promise-returning call that is neither awaited nor returned starts its work and hands control straight to the next line, so the assertion runs against the state from before the effect. Usually it still passes, because the effect is fast. It fails on the loaded CI runner, which is the one machine where the failure is least reproducible.\n\nAn unawaited rejection is the second half: it surfaces as an unhandled rejection attributed to whichever test happened to be running when it settled, so the reported test and the broken test are different tests.\n\n**Implementation**:\n\n```typescript\n// ❌ BAD: the click may not have landed when the assertion runs\ntest('adds the item to the cart', async ({ page }) => {\n page.getByRole('button', { name: 'Add to cart' }).click();\n await expect(page.getByTestId('cart-count')).toHaveText('1');\n});\n\n// ❌ BAD: the setup promise is still in flight\ntest('shows the seeded order', async ({ page, request }) => {\n seedOrder(request, { id: 'ord-1' });\n await page.goto('/orders/ord-1');\n await expect(page.getByText('ord-1')).toBeVisible();\n});\n\n// ✅ GOOD: await the effect before asserting on it\ntest('adds the item to the cart', async ({ page }) => {\n await page.getByRole('button', { name: 'Add to cart' }).click();\n await expect(page.getByTestId('cart-count')).toHaveText('1');\n});\n```\n\n**Key Points**:\n\n- Every promise-returning call in a test body is awaited or explicitly returned\n- The symptom is a test that passes locally and fails on a loaded runner, which reads as flake rather than as a missing `await`\n- An unhandled rejection is usually attributed to the wrong test, so treat one as a signal to audit `await` coverage across the file, not only in the named test\n- `no-floating-promises` in the linter catches this class before the suite ever runs; prefer that to catching it in review\n\n---\n\n## Async Debugging Techniques\n\n### Technique 1: Promise Chain Analysis\n\n```typescript\ntest('debug async waterfall with console logs', async ({ page }) => {\n console.log('1. Starting navigation...');\n await page.goto('/products');\n\n console.log('2. Waiting for API response...');\n const response = await page.waitForResponse('**/api/products');\n console.log('3. API responded:', response.status());\n\n console.log('4. Waiting for UI update...');\n await expect(page.getByText('Products loaded')).toBeVisible();\n console.log('5. Test complete');\n\n // Console output shows exactly where timing issue occurs\n});\n```\n\n### Technique 2: Network Waterfall Inspection (DevTools)\n\n```typescript\ntest('inspect network timing with trace viewer', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Generate trace for analysis\n // npx playwright test --trace on\n // npx playwright show-trace trace.zip\n\n // In trace viewer:\n // 1. Check Network tab for API call timing\n // 2. Identify slow requests (>1s response time)\n // 3. Find race conditions (overlapping requests)\n // 4. Verify request order (dependencies)\n});\n```\n\n### Technique 3: Trace Viewer for Timing Visualization\n\n```typescript\ntest('use trace viewer to debug timing', async ({ page }) => {\n // Run with trace: npx playwright test --trace on\n\n await page.goto('/checkout');\n await page.getByTestId('submit').click();\n\n // In trace viewer, examine:\n // - Timeline: See exact timing of each action\n // - Snapshots: Hover to see DOM state at each moment\n // - Network: Identify slow/failed requests\n // - Console: Check for async errors\n\n await expect(page.getByText('Success')).toBeVisible();\n});\n```\n\n---\n\n## Race Condition Checklist\n\nBefore deploying tests:\n\n- [ ] **Network-first pattern**: All routes intercepted BEFORE navigation (no race conditions)\n- [ ] **Explicit waits**: Every navigation followed by `waitForResponse()` or state check\n- [ ] **No hard waits**: Zero instances of `waitForTimeout()`, `cy.wait(number)`, `sleep()`\n- [ ] **Element state waits**: Loading spinners use `waitFor({ state: 'detached' })`\n- [ ] **Visibility checks**: Use `toBeVisible()` (accounts for animations), not just `toBeAttached()`\n- [ ] **Response validation**: Wait for successful responses (`resp.ok()` or `status === 200`)\n- [ ] **Trace viewer analysis**: Generate traces to identify timing issues (network waterfall, console errors)\n- [ ] **CI/local parity**: Tests pass reliably in both environments (no timing assumptions)\n\n## Integration Points\n\n- **Used in workflows**: `*automate` (healing timing failures), `*test-review` (detect hard wait anti-patterns), `*framework` (configure timeout standards)\n- **Related fragments**: `test-healing-patterns.md` (race condition diagnosis), `network-first.md` (interception patterns), `playwright-config.md` (timeout configuration), `visual-debugging.md` (trace viewer analysis)\n- **Tools**: Playwright Inspector (`--debug`), Trace Viewer (`--trace on`), DevTools Network tab\n\n_Source: Playwright timing best practices, network-first pattern from test-resources-for-ai, production race condition debugging._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":18740,"content_sha256":"fe0a57f14f154fe0acd898a90ca930197c8ddfb72761ab872c68fe13e669c361"},{"filename":"knowledge/visual-debugging.md","content":"# Visual Debugging and Developer Ergonomics\n\n## Principle\n\nFast feedback loops and transparent debugging artifacts are critical for maintaining test reliability and developer confidence. Visual debugging tools (trace viewers, screenshots, videos, HAR files) turn cryptic test failures into actionable insights, reducing triage time from hours to minutes.\n\n## Rationale\n\n**The Problem**: CI failures often provide minimal context—a timeout, a selector mismatch, or a network error—forcing developers to reproduce issues locally (if they can). This wastes time and discourages test maintenance.\n\n**The Solution**: Capture rich debugging artifacts **only on failure** to balance storage costs with diagnostic value. Modern tools like Playwright Trace Viewer, Cypress Debug UI, and HAR recordings provide interactive, time-travel debugging that reveals exactly what the test saw at each step.\n\n**Why This Matters**:\n\n- Reduces failure triage time by 80-90% (visual context vs logs alone)\n- Enables debugging without local reproduction\n- Improves test maintenance confidence (clear failure root cause)\n- Catches timing/race conditions that are hard to reproduce locally\n\n## Pattern Examples\n\n### Example 1: Playwright Trace Viewer Configuration (Production Pattern)\n\n**Context**: Capture traces for failures and retries so flaky runs can be compared directly. Prefer `retain-on-failure-and-retries` as the default policy so failed retries can be compared with passing runs.\n\n**Implementation**:\n\n```typescript\n// playwright.config.ts\nimport { defineConfig } from '@playwright/test';\n\nexport default defineConfig({\n use: {\n // Visual debugging artifacts (best signal for flaky triage)\n trace: 'retain-on-failure-and-retries', // Keep every failed attempt\n screenshot: 'only-on-failure', // Not on success\n video: 'retain-on-failure', // Delete on pass\n\n // Context for debugging\n baseURL: process.env.BASE_URL || 'http://localhost:3000',\n\n // Timeout context\n actionTimeout: 15_000, // 15s for clicks/fills\n navigationTimeout: 30_000, // 30s for page loads\n },\n\n // CI-specific artifact retention\n reporter: [\n ['html', { outputFolder: 'playwright-report', open: 'never' }],\n ['junit', { outputFile: 'results.xml' }],\n ['list'], // Console output\n ],\n\n // Failure handling\n retries: process.env.CI ? 2 : 0, // Retry in CI to capture trace\n workers: process.env.CI ? 1 : undefined,\n});\n```\n\n**Opening and Using Trace Viewer**:\n\n```bash\n# After test failure in CI, download trace artifact\n# Then inspect locally:\nnpx playwright trace open path/to/trace.zip\n\n# Filter to the failing expectation or action from the terminal\nnpx playwright trace actions path/to/trace.zip --grep=\"expect\"\nnpx playwright trace action path/to/trace.zip 9\nnpx playwright trace snapshot path/to/trace.zip 9 --name after\n\n# Or serve trace viewer:\nnpx playwright show-report\n```\n\n**Key Features to Use in Trace Viewer**:\n\n1. **Timeline**: See each action (click, navigate, assertion) with timing\n2. **Snapshots**: Hover over timeline to see DOM state at that moment\n3. **Network Tab**: Inspect all API calls, headers, payloads, timing\n4. **Console Tab**: View console.log/error messages\n5. **Source Tab**: See test code with execution markers\n6. **Metadata**: Browser, OS, test duration, screenshots\n\n**Why This Works**:\n\n- `retain-on-failure-and-retries` preserves enough history to compare the failing retry with a passing run\n- Screenshots + video give visual context without trace overhead\n- Interactive timeline makes timing issues obvious (race conditions, slow API)\n\n---\n\n### Example 2: HAR File Recording for Network Debugging\n\n**Context**: Capture all network activity for reproducible API debugging\n\n**Implementation**:\n\n```typescript\n// tests/e2e/checkout-with-har.spec.ts\nimport { test, expect } from '@playwright/test';\nimport path from 'path';\n\ntest.describe('Checkout Flow with HAR Recording', () => {\n test('should complete payment with full network capture', async ({ page, context }) => {\n // Start HAR recording BEFORE navigation\n await context.routeFromHAR(path.join(__dirname, '../fixtures/checkout.har'), {\n url: '**/api/**', // Only capture API calls\n update: true, // Update HAR if file exists\n });\n\n await page.goto('/checkout');\n\n // Interact with page\n await page.getByTestId('payment-method').selectOption('credit-card');\n await page.getByTestId('card-number').fill('4242424242424242');\n await page.getByTestId('submit-payment').click();\n\n // Wait for payment confirmation\n await expect(page.getByTestId('success-message')).toBeVisible();\n\n // HAR file saved to fixtures/checkout.har\n // Contains all network requests/responses for replay\n });\n});\n```\n\n**Using HAR for Deterministic Mocking**:\n\n```typescript\n// tests/e2e/checkout-replay-har.spec.ts\nimport { test, expect } from '@playwright/test';\nimport path from 'path';\n\ntest('should replay checkout flow from HAR', async ({ page, context }) => {\n // Replay network from HAR (no real API calls)\n await context.routeFromHAR(path.join(__dirname, '../fixtures/checkout.har'), {\n url: '**/api/**',\n update: false, // Read-only mode\n });\n\n await page.goto('/checkout');\n\n // Same test, but network responses come from HAR file\n await page.getByTestId('payment-method').selectOption('credit-card');\n await page.getByTestId('card-number').fill('4242424242424242');\n await page.getByTestId('submit-payment').click();\n\n await expect(page.getByTestId('success-message')).toBeVisible();\n});\n```\n\n**Key Points**:\n\n- **`update: true`** records new HAR or updates existing (for flaky API debugging)\n- **`update: false`** replays from HAR (deterministic, no real API)\n- Filter by URL pattern (`**/api/**`) to avoid capturing static assets\n- HAR files are human-readable JSON (easy to inspect/modify)\n\n**When to Use HAR**:\n\n- Debugging flaky tests caused by API timing/responses\n- Creating deterministic mocks for integration tests\n- Analyzing third-party API behavior (Stripe, Auth0)\n- Reproducing production issues locally (record HAR in staging)\n\n---\n\n### Example 3: Custom Artifact Capture (Console Logs + Network on Failure)\n\n**Context**: Capture additional debugging context automatically on test failure\n\n**Implementation**:\n\n```typescript\n// playwright/support/fixtures/debug-fixture.ts\nimport { test as base, type Request } from '@playwright/test';\nimport fs from 'fs';\nimport path from 'path';\n\ntype DebugFixture = {\n captureDebugArtifacts: () => Promise\u003cvoid>;\n};\n\nexport const test = base.extend\u003cDebugFixture>({\n captureDebugArtifacts: async ({ page }, use, testInfo) => {\n await use(async () => {\n // This function can be called manually in tests\n // But it also runs automatically on failure via afterEach\n });\n\n // After test completes, save artifacts if failed\n if (testInfo.status !== testInfo.expectedStatus) {\n const artifactDir = path.join(testInfo.outputDir, 'debug-artifacts');\n fs.mkdirSync(artifactDir, { recursive: true });\n\n const consoleLogs = (await page.consoleMessages()).map((msg) => `[${msg.type()} @ ${msg.timestamp().toISOString()}] ${msg.text()}`);\n const pageErrors = (await page.pageErrors()).map((error) => ({\n name: error.name,\n message: error.message,\n stack: error.stack,\n }));\n const networkRequests = await Promise.all(\n (await page.requests()).map(async (request: Request) => {\n const response = await request.response();\n return {\n url: request.url(),\n method: request.method(),\n status: response?.status() ?? 0,\n };\n }),\n );\n\n // Save console logs\n fs.writeFileSync(path.join(artifactDir, 'console.log'), consoleLogs.join('\\n'), 'utf-8');\n\n // Save page errors\n fs.writeFileSync(path.join(artifactDir, 'page-errors.json'), JSON.stringify(pageErrors, null, 2), 'utf-8');\n\n // Save network summary\n fs.writeFileSync(path.join(artifactDir, 'network.json'), JSON.stringify(networkRequests, null, 2), 'utf-8');\n\n console.log(`Debug artifacts saved to: ${artifactDir}`);\n }\n },\n});\n```\n\n**Usage in Tests**:\n\n```typescript\n// tests/e2e/payment-with-debug.spec.ts\nimport { test, expect } from '../support/fixtures/debug-fixture';\n\ntest('payment flow captures debug artifacts on failure', async ({ page, captureDebugArtifacts }) => {\n await page.goto('/checkout');\n\n // Test will automatically capture console + network on failure\n await page.getByTestId('submit-payment').click();\n await expect(page.getByTestId('success-message')).toBeVisible({ timeout: 5000 });\n\n // If this fails, console.log and network.json saved automatically\n});\n```\n\n**CI Integration (GitHub Actions)**:\n\n```yaml\n# .github/workflows/e2e.yml\nname: E2E Tests with Artifacts\non: [push, pull_request]\n\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n with:\n node-version-file: '.nvmrc'\n\n - name: Install dependencies\n run: npm ci\n\n - name: Run Playwright tests\n run: npm run test:e2e\n continue-on-error: true # Capture artifacts even on failure\n\n - name: Upload test artifacts on failure\n if: failure()\n uses: actions/upload-artifact@v4\n with:\n name: playwright-artifacts\n path: |\n test-results/\n playwright-report/\n retention-days: 30\n```\n\n**Key Points**:\n\n- Fixtures automatically capture context without polluting test code\n- Only saves artifacts on failure (storage-efficient)\n- CI uploads artifacts for post-mortem analysis\n- `continue-on-error: true` ensures artifact upload even when tests fail\n\n---\n\n### Example 4: Accessibility Debugging Integration (axe-core in Trace Viewer)\n\n**Context**: Catch accessibility regressions during visual debugging\n\n**Implementation**:\n\n```typescript\n// playwright/support/fixtures/a11y-fixture.ts\nimport { test as base } from '@playwright/test';\nimport AxeBuilder from '@axe-core/playwright';\n\ntype A11yFixture = {\n checkA11y: () => Promise\u003cvoid>;\n};\n\nexport const test = base.extend\u003cA11yFixture>({\n checkA11y: async ({ page }, use) => {\n await use(async () => {\n // Run axe accessibility scan\n const results = await new AxeBuilder({ page }).analyze();\n\n // Attach results to test report (visible in trace viewer)\n if (results.violations.length > 0) {\n console.log(`Found ${results.violations.length} accessibility violations:`);\n results.violations.forEach((violation) => {\n console.log(`- [${violation.impact}] ${violation.id}: ${violation.description}`);\n console.log(` Help: ${violation.helpUrl}`);\n });\n\n throw new Error(`Accessibility violations found: ${results.violations.length}`);\n }\n });\n },\n});\n```\n\n**Usage with Visual Debugging**:\n\n```typescript\n// tests/e2e/checkout-a11y.spec.ts\nimport { test, expect } from '../support/fixtures/a11y-fixture';\n\ntest('checkout page is accessible', async ({ page, checkA11y }) => {\n await page.goto('/checkout');\n\n // Verify page loaded\n await expect(page.getByRole('heading', { name: 'Checkout' })).toBeVisible();\n\n // Run accessibility check\n await checkA11y();\n\n // If violations found, test fails and trace captures:\n // - Screenshot showing the problematic element\n // - Console log with violation details\n // - Network tab showing any failed resource loads\n});\n```\n\n**Trace Viewer Benefits**:\n\n- **Screenshot shows visual context** of accessibility issue (contrast, missing labels)\n- **Console tab shows axe-core violations** with impact level and helpUrl\n- **DOM snapshot** allows inspecting ARIA attributes at failure point\n- **Network tab** reveals if icon fonts or images failed (common a11y issue)\n\n**Cypress Equivalent**:\n\n```javascript\n// cypress/support/commands.ts\nimport 'cypress-axe';\n\nCypress.Commands.add('checkA11y', (context = null, options = {}) => {\n cy.injectAxe(); // Inject axe-core\n cy.checkA11y(context, options, (violations) => {\n if (violations.length) {\n cy.task('log', `Found ${violations.length} accessibility violations`);\n violations.forEach((violation) => {\n cy.task('log', `- [${violation.impact}] ${violation.id}: ${violation.description}`);\n });\n }\n });\n});\n\n// tests/e2e/checkout-a11y.cy.ts\ndescribe('Checkout Accessibility', () => {\n it('should have no a11y violations', () => {\n cy.visit('/checkout');\n cy.injectAxe();\n cy.checkA11y();\n // On failure, Cypress UI shows:\n // - Screenshot of page\n // - Console log with violation details\n // - Network tab with API calls\n });\n});\n```\n\n**Key Points**:\n\n- Accessibility checks integrate seamlessly with visual debugging\n- Violations are captured in trace viewer/Cypress UI automatically\n- Provides actionable links (helpUrl) to fix issues\n- Screenshots show visual context (contrast, layout)\n\n---\n\n### Example 5: Time-Travel Debugging Workflow (Playwright Inspector)\n\n**Context**: Debug tests interactively with step-through execution\n\n**Implementation**:\n\n```typescript\n// tests/e2e/checkout-debug.spec.ts\nimport { test, expect } from '@playwright/test';\n\ntest('debug checkout flow step-by-step', async ({ page }) => {\n // Set breakpoint by uncommenting this:\n // await page.pause()\n\n await page.goto('/checkout');\n\n // Use Playwright Inspector to:\n // 1. Step through each action\n // 2. Inspect DOM at each step\n // 3. View network calls per action\n // 4. Take screenshots manually\n\n await page.getByTestId('payment-method').selectOption('credit-card');\n\n // Pause here to inspect form state\n // await page.pause()\n\n await page.getByTestId('card-number').fill('4242424242424242');\n await page.getByTestId('submit-payment').click();\n\n await expect(page.getByTestId('success-message')).toBeVisible();\n});\n```\n\n**Running with Inspector**:\n\n```bash\n# Open Playwright Inspector (GUI debugger)\nnpx playwright test --debug\n\n# Or use headed mode with slowMo\nnpx playwright test --headed --slow-mo=1000\n\n# Debug specific test\nnpx playwright test checkout-debug.spec.ts --debug\n\n# Set environment variable for persistent debugging\nPWDEBUG=1 npx playwright test\n```\n\n**Inspector Features**:\n\n1. **Step-through execution**: Click \"Next\" to execute one action at a time\n2. **DOM inspector**: Hover over elements to see selectors\n3. **Network panel**: See API calls with timing\n4. **Console panel**: View console.log output\n5. **Pick locator**: Click element in browser to get selector\n6. **Record mode**: Record interactions to generate test code\n\n**Common Debugging Patterns**:\n\n```typescript\n// Pattern 1: Debug selector issues\ntest('debug selector', async ({ page }) => {\n await page.goto('/dashboard');\n await page.pause(); // Inspector opens\n\n // In Inspector console, test selectors:\n // page.getByTestId('user-menu') ✅\n // page.getByRole('button', { name: 'Profile' }) ✅\n // page.locator('.btn-primary') ❌ (fragile)\n});\n\n// Pattern 2: Debug timing issues\ntest('debug network timing', async ({ page }) => {\n await page.goto('/dashboard');\n\n // Set up network listener BEFORE interaction\n const responsePromise = page.waitForResponse('**/api/users');\n await page.getByTestId('load-users').click();\n\n await page.pause(); // Check network panel for timing\n\n const response = await responsePromise;\n expect(response.status()).toBe(200);\n});\n\n// Pattern 3: Debug state changes\ntest('debug state mutation', async ({ page }) => {\n await page.goto('/cart');\n\n // Check initial state\n await expect(page.getByTestId('cart-count')).toHaveText('0');\n\n await page.pause(); // Inspect DOM\n\n await page.getByTestId('add-to-cart').click();\n\n await page.pause(); // Inspect DOM again (compare state)\n\n await expect(page.getByTestId('cart-count')).toHaveText('1');\n});\n```\n\n**Key Points**:\n\n- `page.pause()` opens Inspector at that exact moment\n- Inspector shows DOM state, network activity, console at pause point\n- \"Pick locator\" feature helps find robust selectors\n- Record mode generates test code from manual interactions\n\n---\n\n## Visual Debugging Checklist\n\nBefore deploying tests to CI, ensure:\n\n- [ ] **Artifact configuration**: `trace: 'retain-on-failure-and-retries'`, `screenshot: 'only-on-failure'`, `video: 'retain-on-failure'`\n- [ ] **CI artifact upload**: GitHub Actions/GitLab CI configured to upload `test-results/` and `playwright-report/`\n- [ ] **HAR recording**: Set up for flaky API tests (record once, replay deterministically)\n- [ ] **Custom debug fixtures**: Console logs + network summary captured on failure\n- [ ] **Accessibility integration**: axe-core violations visible in trace viewer\n- [ ] **Trace viewer docs**: README explains how to open traces locally (`npx playwright trace open`)\n- [ ] **Inspector workflow**: Document `--debug` flag for interactive debugging\n- [ ] **Storage optimization**: Artifacts deleted after 30 days (CI retention policy)\n\n## Integration Points\n\n- **Used in workflows**: `*framework` (initial setup), `*ci` (artifact upload), `*test-review` (validate artifact config)\n- **Related fragments**: `playwright-config.md` (artifact configuration), `ci-burn-in.md` (CI artifact upload), `test-quality.md` (debugging best practices)\n- **Tools**: Playwright Trace Viewer, Cypress Debug UI, axe-core, HAR files\n\n_Source: Playwright official docs, Murat testing philosophy (visual debugging manifesto), enterprise production debugging patterns._\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":17481,"content_sha256":"4c40d5173d5b933d650884c4206f7e12aea77e7ca6fc0eb22770865b55e6522d"},{"filename":"knowledge/webhook-module-setup.md","content":"# Webhook Module Setup\n\n## Principle\n\nWire the provider once in a central fixtures file using the `webhookProviderFixture + webhookFixture + mergeTests` pattern. Tests that request `webhookRegistry` get automatic setup and teardown; tests that don't pay nothing (Playwright lazy fixture evaluation).\n\n## Fixture Wiring Pattern\n\n### WireMock Provider (recommended for most setups)\n\nThe WireMock provider works with any backend that implements the `/__admin/requests` API format — not just actual WireMock. The playwright-utils sample app's Express backend uses this exact format.\n\n```typescript\n// playwright/support/merged-fixtures.ts\nimport { test as base, mergeTests } from '@playwright/test';\nimport { test as webhookFixture } from '@seontechnologies/playwright-utils/webhook/fixtures';\nimport { WireMockWebhookProvider } from '@seontechnologies/playwright-utils/webhook';\nimport { API_URL } from '../config/local.config';\n\n// Lazy-initialized by Playwright — no cost for tests that don't request webhookRegistry.\nconst webhookProviderFixture = base.extend\u003c{\n webhookProvider: WireMockWebhookProvider;\n}>({\n webhookProvider: async ({ request }, use) => {\n const provider = new WireMockWebhookProvider(API_URL, request);\n await use(provider);\n },\n});\n\nconst test = mergeTests(\n base,\n // ...your other fixtures...\n webhookFixture,\n webhookProviderFixture,\n);\n\n// Use matched-only cleanup project-wide: each test only deletes the webhooks it\n// matched, so a parallel worker's teardown cannot wipe the shared journal while\n// another test is still mid-flight (fullyParallel: true race condition).\ntest.use({ webhookConfig: { cleanupStrategy: 'matched-only' } });\n\nexport { test };\n```\n\nThis is the exact pattern used in the playwright-utils E2E suite (`playwright/support/merged-fixtures.ts`).\n\n### MockServer Provider\n\n```typescript\nimport { MockServerWebhookProvider } from '@seontechnologies/playwright-utils/webhook';\n\nconst webhookProviderFixture = base.extend\u003c{\n webhookProvider: MockServerWebhookProvider;\n}>({\n webhookProvider: async ({ request }, use) => {\n await use(new MockServerWebhookProvider(API_URL, request));\n },\n});\n\nconst test = mergeTests(base, /* ...other fixtures... */ webhookFixture, webhookProviderFixture);\n\n// MockServer has no delete-by-ID on log entries — use full-reset for explicit cleanup\ntest.use({ webhookConfig: { cleanupStrategy: 'full-reset' } });\n```\n\n### Mockoon Provider\n\n```typescript\nimport { MockoonWebhookProvider } from '@seontechnologies/playwright-utils/webhook';\n\nconst webhookProviderFixture = base.extend\u003c{\n webhookProvider: MockoonWebhookProvider;\n}>({\n webhookProvider: async ({ request }, use) => {\n await use(new MockoonWebhookProvider(API_URL, request));\n },\n});\n\nconst test = mergeTests(base, /* ...other fixtures... */ webhookFixture, webhookProviderFixture);\n\n// Mockoon has no delete-by-ID on log entries — use full-reset for explicit cleanup\ntest.use({ webhookConfig: { cleanupStrategy: 'full-reset' } });\n```\n\n## Cleanup Strategy Decision\n\n| Strategy | Behaviour | When to choose |\n| ------------------------ | ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------- |\n| `'full-reset'` (default) | Calls `provider.resetJournal()` — wipes the entire mock server journal | Safe only for serial execution or when each worker has an isolated provider instance |\n| `'matched-only'` | Calls `provider.deleteById(id)` for each webhook matched by `waitFor`/`waitForCount` | Required for `fullyParallel: true` with a shared journal **when the provider supports `deleteById`** (e.g. WireMock) |\n\n**The race condition under `fullyParallel: true`**: Worker A finishes and calls `resetJournal()`. Worker B is mid-poll waiting for its webhook. Worker A's reset just deleted Worker B's webhook — the poll times out with `WebhookTimeoutError`. Use `matched-only` to avoid this — but only when the provider supports `deleteById`.\n\n**MockServer and Mockoon limitation**: Neither supports `deleteById` — their implementations are no-ops. The `startedAt` timestamp filter isolates _reads_ inside `waitFor`/`waitForCount`, but `cleanup()` with `full-reset` still calls `resetJournal()`, which wipes the entire journal. This means the teardown race exists for these providers too under `fullyParallel: true`. For parallel suites with MockServer or Mockoon, either run serially (`workers: 1`) or provision an isolated mock server instance per worker.\n\n## Fixture Lifecycle\n\nThe fixture calls these in order:\n\n1. `provider.setup?.()` — optional health check or stub registration\n2. Tests run with `webhookRegistry` available\n3. `registry.cleanup()` — deletes matched webhooks (`matched-only`) or resets journal (`full-reset`)\n4. `provider.teardown?.()` — optional resource cleanup\n\nBoth cleanup and teardown failures are caught and logged as warnings — they don't mask actual test failures.\n\n## WebhookRegistryConfig Options\n\n```typescript\ntype WebhookRegistryConfig = {\n defaultTimeout?: number; // default: 30000 ms\n defaultInterval?: number; // default: 1000 ms\n cleanupStrategy?: 'matched-only' | 'full-reset'; // default: 'full-reset'\n};\n```\n\n## Related Fragments\n\n- `webhook-testing-fundamentals.md` — Why webhook tests are hard\n- `webhook-template-matchers.md` — Template building and matcher patterns\n- `webhook-providers.md` — WireMock, MockServer, Mockoon, custom provider details\n- `fixtures-composition.md` — mergeTests pattern\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5834,"content_sha256":"87dd6d7eb4dfd45691ccbb876a0bdf0b8a8b8622351be7539b8fb80504068048"},{"filename":"knowledge/webhook-providers.md","content":"# Webhook Provider Patterns\n\n## Principle\n\nThree built-in providers ship with playwright-utils. Each wraps a different mock server API. For any backend not covered, implement the `WebhookProvider` interface. The registry only cares about the contract — not the backend technology.\n\n## WireMockWebhookProvider\n\nUses `GET /__admin/requests` to fetch the webhook log and `DELETE /__admin/requests` to reset. Supports `deleteById` for `matched-only` cleanup.\n\n**Works with any backend implementing the `/__admin/requests` format** — not just actual WireMock. The playwright-utils sample app's Express backend uses this exact format.\n\n```typescript\nimport { WireMockWebhookProvider } from '@seontechnologies/playwright-utils/webhook';\nimport { API_URL } from '../config/local.config';\n\nconst webhookProviderFixture = base.extend\u003c{\n webhookProvider: WireMockWebhookProvider;\n}>({\n webhookProvider: async ({ request }, use) => {\n const provider = new WireMockWebhookProvider(API_URL, request);\n await use(provider);\n },\n});\n```\n\nSupports both cleanup strategies. Use `matched-only` when running `fullyParallel: true`.\n\n## MockServerWebhookProvider\n\nUses `PUT /mockserver/retrieve` to fetch logs with client-side `since` filtering.\n\n**Limitation**: `deleteById` is a no-op — MockServer does not support deleting individual log entries by ID. The `startedAt` timestamp filter handles per-test isolation. Use `full-reset` for explicit journal cleanup.\n\n```typescript\nimport { MockServerWebhookProvider } from '@seontechnologies/playwright-utils/webhook';\n\nconst webhookProviderFixture = base.extend\u003c{\n webhookProvider: MockServerWebhookProvider;\n}>({\n webhookProvider: async ({ request }, use) => {\n await use(new MockServerWebhookProvider(API_URL, request));\n },\n});\n\nconst test = mergeTests(base, /* ...other fixtures... */ webhookFixture, webhookProviderFixture);\n\n// MockServer has no delete-by-ID on log entries — use full-reset\ntest.use({ webhookConfig: { cleanupStrategy: 'full-reset' } });\n```\n\n## MockoonWebhookProvider\n\nUses `GET /mockoon-admin/logs` to fetch logs. The admin API is enabled by default in `@mockoon/cli`. Default log limit is 100 entries — increase with `--max-transaction-logs` if your suite generates more.\n\n**Limitation**: `deleteById` is a no-op for the same reason as MockServer. Use `full-reset`.\n\n```typescript\nimport { MockoonWebhookProvider } from '@seontechnologies/playwright-utils/webhook';\n\nconst webhookProviderFixture = base.extend\u003c{\n webhookProvider: MockoonWebhookProvider;\n}>({\n webhookProvider: async ({ request }, use) => {\n await use(new MockoonWebhookProvider(API_URL, request));\n },\n});\n\nconst test = mergeTests(base, /* ...other fixtures... */ webhookFixture, webhookProviderFixture);\n\n// Mockoon has no delete-by-ID on log entries — use full-reset\ntest.use({ webhookConfig: { cleanupStrategy: 'full-reset' } });\n```\n\nStart Mockoon with an increased log limit if needed:\n\n```bash\nmockoon-cli start --data ./mockoon-config.json --max-transaction-logs 500\n```\n\n## Custom Provider\n\nImplement `WebhookProvider` for any backend that exposes a queryable request log:\n\n```typescript\n// support/providers/custom-webhook-provider.ts\nimport type { WebhookProvider, ReceivedWebhook, WebhookQueryFilter } from '@seontechnologies/playwright-utils/webhook';\nimport type { APIRequestContext } from '@playwright/test';\n\nexport class CustomWebhookProvider implements WebhookProvider {\n constructor(\n private readonly baseUrl: string,\n private readonly request: APIRequestContext,\n ) {}\n\n async getReceivedWebhooks(filter?: WebhookQueryFilter): Promise\u003cReceivedWebhook[]> {\n const params = new URLSearchParams();\n if (filter?.since) params.set('since', filter.since.toISOString());\n if (filter?.method) params.set('method', filter.method);\n\n const response = await this.request.get(`${this.baseUrl}/webhooks/received?${params}`);\n const { webhooks } = await response.json();\n return webhooks.map((w: Record\u003cstring, unknown>) => ({\n id: String(w.id),\n url: String(w.url),\n method: String(w.method),\n headers: (w.headers as Record\u003cstring, string>) ?? {},\n body: w.body,\n receivedAt: new Date(String(w.receivedAt)),\n }));\n }\n\n async resetJournal(): Promise\u003cvoid> {\n await this.request.delete(`${this.baseUrl}/webhooks/received`);\n }\n\n async deleteById(id: string): Promise\u003cvoid> {\n await this.request.delete(`${this.baseUrl}/webhooks/received/${id}`);\n }\n\n async getCount(): Promise\u003cnumber> {\n const response = await this.request.get(`${this.baseUrl}/webhooks/count`);\n const { count } = await response.json();\n return count as number;\n }\n}\n```\n\n## WebhookProvider Interface\n\n```typescript\ninterface WebhookProvider {\n getReceivedWebhooks(filter?: WebhookQueryFilter): Promise\u003cReceivedWebhook[]>;\n resetJournal(): Promise\u003cvoid>;\n deleteById(id: string): Promise\u003cvoid>;\n getCount(criteria?: Record\u003cstring, unknown>): Promise\u003cnumber>;\n removeByCriteria?(criteria: Record\u003cstring, unknown>): Promise\u003cvoid>;\n setup?(): Promise\u003cvoid>; // optional — called before test\n teardown?(): Promise\u003cvoid>; // optional — called after test\n}\n```\n\n## Provider Comparison\n\n| Provider | deleteById | resetJournal | Parallel-safe (shared journal) | Recommended strategy | API endpoint |\n| ------------------------- | ---------- | ------------ | ----------------------------------- | ----------------------------------------------------- | ---------------------- |\n| WireMockWebhookProvider | ✅ Yes | ✅ Yes | ✅ Yes (`matched-only`) | `matched-only` | `/__admin/requests` |\n| MockServerWebhookProvider | ❌ No-op | ✅ Yes | ⚠️ No — serial or isolated instance | `full-reset` (serial or isolated provider per worker) | `/mockserver/retrieve` |\n| MockoonWebhookProvider | ❌ No-op | ✅ Yes | ⚠️ No — serial or isolated instance | `full-reset` (serial or isolated provider per worker) | `/mockoon-admin/logs` |\n| Custom | Depends | Depends | Depends on implementation | Depends | Your API |\n\n## Related Fragments\n\n- `webhook-module-setup.md` — Full fixture wiring for each provider\n- `webhook-testing-fundamentals.md` — Cleanup strategy rationale\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":6440,"content_sha256":"a71ba2c09c45541d4d262c4db85fb9b2aeb021bb09779c65454a5ed3bd546c63"},{"filename":"knowledge/webhook-risk-guidance.md","content":"# Webhook Testing Risk Guidance\n\n## Principle\n\nWebhook integration points are high-risk boundaries — they represent asynchronous side effects that cross service boundaries. A missing or malformed webhook means a downstream system never received its trigger. Default risk level: **P2 × I3** (medium probability, high impact = Risk Score 6) → must be covered by integration tests.\n\n## When Webhook Tests Are Required\n\nWebhook tests are **required** (not optional) when:\n\n| Condition | Rationale |\n| ------------------------------------------------------------------ | ---------------------------------------------------------------------- |\n| Application publishes events to external subscribers | External consumers depend on correct payload shape and delivery timing |\n| Event-driven architecture (Kafka/SQS/event bus → webhook delivery) | The delivery pipeline is a risk boundary; delivery failures are silent |\n| Payment, order, or notification side effects | Business-critical; missed webhooks = missed transactions |\n| Integration with third-party services via webhooks | Breaking payload changes won't surface in unit or component tests |\n| Any async side effect that a consumer polls-on or reacts-to | Polling tests (`recurse`) can mask webhook delivery failures entirely |\n\n## Risk Scoring\n\n```text\nRisk = Probability × Impact\n\nProbability factors (P1–P3):\n P1 (low): Webhook system is mature, well-tested, no history of failures\n P2 (medium): Kafka pipeline, multiple consumers, new integrations\n P3 (high): New delivery mechanism, external third-party webhooks, no retry logic\n\nImpact factors (I1–I3):\n I1 (low): Non-critical notifications (e.g. audit logs)\n I2 (medium): Feature-level side effects (e.g. search index updates)\n I3 (high): Business-critical events (payments, orders, compliance)\n```\n\nDefault webhook integrations: **P2 × I3 = 6** → High → must be tested.\n\n## What a Complete Webhook Test Looks Like\n\nA complete webhook test covers:\n\n1. **Happy path**: Action fires → webhook arrives with correct payload\n2. **Sequential events (drain pattern)**: Preceding event drained before asserting on next\n3. **Parallel isolation**: Template scoped by entity ID — workers don't cross-contaminate\n4. **Timeout/error shape**: `WebhookTimeoutError` tested for negative path coverage\n5. **Cleanup verification**: Fixture auto-cleans; no leaked webhooks after test\n\n**Minimal complete example** (from playwright-utils E2E suite):\n\n```typescript\n// Template factories scoped by ID — parallel safety\nconst movieCreated = (movieId: number) =>\n webhookTemplate\u003c{ event: string; data: { id: number } }>('movie.created')\n .matchField('event', 'movie.created')\n .matchField('data.id', movieId)\n .withTimeout(15_000)\n .withInterval(500)\n .build();\n\nconst movieDeleted = (movieId: number) =>\n webhookTemplate\u003c{ event: string; data: { id: number } }>('movie.deleted')\n .matchField('event', 'movie.deleted')\n .matchField('data.id', movieId)\n .withTimeout(15_000)\n .withInterval(500)\n .build();\n\ntest('movie deletion triggers a webhook with correct payload', async ({ authToken, addMovie, deleteMovie, webhookRegistry }) => {\n const movie = generateMovieWithoutId();\n const { body: createResponse } = await addMovie(authToken, movie);\n const movieId = createResponse.data.id;\n\n // Drain: consume the create webhook before testing the delete path\n await webhookRegistry.waitFor(movieCreated(movieId));\n\n await deleteMovie(authToken, movieId);\n const webhook = await webhookRegistry.waitFor(movieDeleted(movieId));\n\n expect(webhook.body).toMatchObject({\n event: 'movie.deleted',\n data: { id: movieId, name: movie.name },\n });\n});\n```\n\n## Common Failure Patterns\n\n| Failure pattern | Root cause | How the module addresses it |\n| -------------------------------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------- |\n| Test passes but webhook never verified | Test asserted on status endpoint, not delivery | `waitFor` forces assertion on actual webhook arrival |\n| Flaky under `fullyParallel: true` | `full-reset` cleanup deletes another worker's webhooks | `matched-only` strategy — only matched webhooks are deleted |\n| Timeout gives no useful information | No payload inspection on failure | `WebhookTimeoutError.receivedWebhooks` snapshot |\n| Template matches wrong test's webhook | Template not scoped by entity ID | Template factories accept ID parameter; `matchPredicate` for complex scoping |\n| Test hangs at 30s default timeout | Webhook not arriving; pipeline is slow | Use `withTimeout()` and `withInterval(500)` per template |\n| Journal grows unbounded | No cleanup strategy configured | Configure `cleanupStrategy` in `webhookConfig`; fixture auto-cleans |\n\n## Risk Mitigation Checklist (for TA assessment)\n\nWhen a system uses webhooks, verify the test suite covers:\n\n- [ ] Happy path for each event type that has an external subscriber\n- [ ] Template factories scoped by entity ID (parallel-safe)\n- [ ] Drain pattern applied to all sequential event assertions\n- [ ] Cleanup strategy matches provider capability: `matched-only` for providers that support `deleteById` (e.g. WireMock); `full-reset` with serial execution or an isolated provider instance per worker for MockServer/Mockoon\n- [ ] Timeout values appropriate for the delivery pipeline latency (Kafka pipelines need 15s+)\n- [ ] `WebhookTimeoutError` imported and tested in negative path coverage\n- [ ] Mock server (WireMock/MockServer/Mockoon) in Docker Compose / test infra\n\n## Related Fragments\n\n- `webhook-testing-fundamentals.md` — Why webhook tests are hard\n- `webhook-module-setup.md` — Fixture wiring for each provider\n- `webhook-template-matchers.md` — Template and matcher patterns\n- `risk-governance.md` — Risk scoring framework\n- `probability-impact.md` — P×I scale definitions\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":6478,"content_sha256":"ff8a5b79d3d4e5b9a98eaf346e7efcf685e40bf012e1a8a3e5e1b6c3635be26f"},{"filename":"knowledge/webhook-template-matchers.md","content":"# Webhook Template Matchers\n\n## Principle\n\nBuild typed templates with `webhookTemplate()` and compose matchers using `matchField`, `matchPartial`, and `matchPredicate`. All matchers on a template use AND semantics — every matcher must pass for a webhook to be considered a match. Templates are immutable value objects produced by a fluent builder.\n\n## Template Factory Pattern\n\nDefine template factories as pure functions that accept a test-scoped ID. This is the key pattern for parallel isolation — each factory call produces a template bound to a specific entity:\n\n```typescript\nimport { webhookTemplate } from '@seontechnologies/playwright-utils/webhook';\n\n// Template factories for movie webhooks\n// 15s timeout: the Kafka → HTTP webhook delivery pipeline can back up under\n// high CI concurrency (burn-in with many parallel workers). 10s was occasionally\n// not enough; 15s gives the pipeline headroom without slowing normal runs.\nconst movieCreated = (movieId: number) =>\n webhookTemplate\u003c{ event: string; data: { id: number } }>('movie.created')\n .matchField('event', 'movie.created')\n .matchField('data.id', movieId)\n .withTimeout(15_000)\n .withInterval(500)\n .build();\n\nconst movieDeleted = (movieId: number) =>\n webhookTemplate\u003c{ event: string; data: { id: number } }>('movie.deleted')\n .matchField('event', 'movie.deleted')\n .matchField('data.id', movieId)\n .withTimeout(15_000)\n .withInterval(500)\n .build();\n```\n\nThe ID parameter scopes each template to a specific entity, preventing parallel workers from matching each other's webhooks.\n\n## Matcher Reference\n\n### matchField — dot-path exact match\n\nTraverses dot-notation paths into the payload. Never throws if the path is missing — a missing path evaluates as non-matching.\n\n```typescript\nwebhookTemplate('order.created')\n .matchField('event', 'order.created') // top-level field\n .matchField('data.id', orderId) // nested path\n .matchField('data.status', 'pending') // nested string value\n .build();\n```\n\nMatcher detail output: `field(data.id=42)`\n\n### matchPartial — deep subset check\n\nChecks that the expected object is a subset of the received payload. Extra fields in the payload are ignored. Arrays use strict length matching.\n\n```typescript\nconst partialTemplate = webhookTemplate\u003c{\n event: string;\n data: { id: number; name: string };\n}>('movie.created.partial')\n .matchPartial({ event: 'movie.created', data: { id: movieId } })\n .withTimeout(10_000)\n .withInterval(500)\n .build();\n```\n\nMatcher detail output: `partial({\"event\":\"movie.created\",\"data\":{\"id\":42}})`\n\n### matchPredicate — arbitrary function\n\nAccepts any `(payload: T) => boolean` function. Always requires a human-readable description string — this appears in `WebhookTimeoutError.matcherDetails` for debugging.\n\n**ID-scoped parallel isolation** (prevents cross-worker contamination in `waitForCount`):\n\n```typescript\nconst batchTemplate = webhookTemplate\u003c{\n event: string;\n data: { id: number };\n}>('movie.created.batch')\n .matchField('event', 'movie.created')\n .matchPredicate(`data.id is ${id1} or ${id2}`, (p) => p.data.id === id1 || p.data.id === id2)\n .withTimeout(15_000)\n .withInterval(500)\n .build();\n```\n\n**Business data filtering**:\n\n```typescript\nconst highRatingTemplate = webhookTemplate\u003c{\n event: string;\n data: { id: number; rating: number };\n}>('movie.created.high-rating')\n .matchField('event', 'movie.created')\n .matchPredicate(`data.id is ${movieId} and data.rating >= 9`, (p) => p.data.id === movieId && p.data.rating >= 9)\n .withTimeout(10_000)\n .withInterval(500)\n .build();\n```\n\nMatcher detail output: `predicate(data.id is 42 and data.rating >= 9)`\n\n## Combining Matchers\n\nAll matchers use AND semantics — all must pass for the webhook to match:\n\n```typescript\n// Combined field + partial: both matchers must pass\nconst updateTemplate = webhookTemplate\u003c{\n event: string;\n data: { id: number; name: string };\n}>('movie.updated')\n .matchField('event', 'movie.updated')\n .matchPartial({ data: { id: movieId, name: nameUpdate.name } })\n .withTimeout(10_000)\n .withInterval(500)\n .build();\n```\n\n## Per-Template Timeout and Interval\n\nOverride the registry defaults on a per-template basis:\n\n```typescript\nwebhookTemplate('slow.pipeline.event')\n .matchField('event', 'slow.pipeline.event')\n .withTimeout(60_000) // 60s for slow delivery pipelines\n .withInterval(2_000) // poll every 2s\n .build();\n```\n\n## clone() for Base Template Variations\n\n> **Note**: `clone()` is available on the builder but is not used in the playwright-utils E2E suite. Use it when multiple tests share the same base template with slight field variations.\n\n```typescript\nconst base = webhookTemplate\u003cOrderPayload>('order').matchField('event', 'order.completed');\n\nconst forOrderA = base.clone().matchField('data.orderId', 'A').build();\nconst forOrderB = base.clone().matchField('data.orderId', 'B').build();\n```\n\n## Builder API Summary\n\n| Method | Description |\n| --------------------------- | ------------------------------------------------------ |\n| `webhookTemplate\u003cT>(name)` | Create a new builder with the given template name |\n| `.matchField(path, value)` | Add dot-path exact-match matcher |\n| `.matchPartial(expected)` | Add deep-subset matcher |\n| `.matchPredicate(desc, fn)` | Add arbitrary predicate matcher (description required) |\n| `.withTimeout(ms)` | Override registry default timeout |\n| `.withInterval(ms)` | Override registry default poll interval |\n| `.clone()` | Copy current builder state for variation |\n| `.build()` | Produce the immutable `WebhookTemplate\u003cT>` object |\n\n## Related Fragments\n\n- `webhook-waiting-querying.md` — waitFor, waitForCount, drain pattern\n- `webhook-timeout-error.md` — Reading matcherDetails in error output\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":6011,"content_sha256":"abf92e617b194199439c60fafe68cb0481b48e9ef09f69c01c8a424429a6009f"},{"filename":"knowledge/webhook-testing-fundamentals.md","content":"# Webhook Testing Fundamentals\n\n## Principle\n\nWebhook delivery is eventually consistent — your application fires HTTP callbacks asynchronously after events occur. Tests must poll until the expected webhook arrives or time out. The `@seontechnologies/playwright-utils` webhook module provides deterministic polling, typed matchers, rich timeout diagnostics, and cleanup strategies safe under `fullyParallel: true`.\n\n## Rationale\n\nWebhook tests fail for four structural reasons:\n\n- **Eventually consistent**: Webhook delivery happens asynchronously — you cannot assert immediately after triggering an event\n- **Parallel journal pollution**: When multiple workers share the same mock server, a fast worker's teardown can delete records a slow worker is still polling\n- **Opaque timeouts**: A bare timeout tells you only that the webhook didn't arrive — it shows you nothing about what did arrive\n- **Cleanup drift**: Resetting the full journal in `afterEach` creates a race condition under `fullyParallel: true`\n\nThe playwright-utils approach:\n\n- **Polling via `recurse`**: Uses Playwright's `expect.poll` under the hood — retries with configurable timeout and interval until a match is found\n- **Typed matchers**: `matchField`, `matchPartial`, `matchPredicate` — all must pass (AND semantics); matchers never throw on missing paths\n- **Rich timeout errors**: `WebhookTimeoutError` carries `totalReceived`, `receivedWebhooks`, and `matcherDetails` so you can see what arrived vs. what was expected\n- **Isolation via `startedAt`**: Each `WebhookRegistry` instance records its creation timestamp; polling only fetches webhooks received after that point, preventing leakage from prior tests\n- **Two cleanup strategies**: `full-reset` (resets entire journal) and `matched-only` (deletes only matched webhooks — parallel-safe when the provider supports delete-by-ID, e.g. WireMock)\n\n## When to Use Webhook Tests\n\n| Scenario | Use webhook tests |\n| ----------------------------------------------------------------- | ------------------------- |\n| Application publishes events to external subscribers | ✅ Required |\n| Event-driven architecture with Kafka/event bus → webhook delivery | ✅ Required |\n| Payment, order, or notification side effects via webhooks | ✅ Required |\n| Testing that a webhook was NOT delivered | ✅ Verify via timeout |\n| Polling a status endpoint for eventual consistency | ❌ Use `recurse` directly |\n| Frontend receiving push notifications (WebSocket) | ❌ Different mechanism |\n\n## Related Fragments\n\n- `webhook-module-setup.md` — Fixture wiring and cleanup strategies\n- `webhook-template-matchers.md` — matchField, matchPartial, matchPredicate\n- `webhook-waiting-querying.md` — waitFor, waitForCount, getReceived, drain pattern\n- `webhook-timeout-error.md` — WebhookTimeoutError debugging\n- `webhook-providers.md` — WireMock, MockServer, Mockoon, custom provider\n- `webhook-risk-guidance.md` — Risk-based guidance for TA and TD capabilities\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3185,"content_sha256":"cc1778b9aa95cb6fe00cfbb2115a2238636f71513cf159b7c02cbb3e091541c3"},{"filename":"knowledge/webhook-timeout-error.md","content":"# WebhookTimeoutError and Debugging\n\n## Principle\n\n`WebhookTimeoutError` is thrown when `waitFor` or `waitForCount` does not find a matching webhook within the configured timeout. It carries a snapshot of received webhooks from the last polling cycle — truncated to the last 10 entries — so you can inspect what arrived vs. what was expected. The full count of all received webhooks is available in `totalReceived`.\n\n## Error Properties\n\n```typescript\nclass WebhookTimeoutError extends Error {\n readonly name = 'WebhookTimeoutError';\n readonly templateName: string; // from webhookTemplate('...')\n readonly timeoutMs: number; // the timeout that was exceeded\n readonly totalReceived: number; // total webhooks seen in polling window\n readonly receivedWebhooks: ReceivedWebhook[]; // last ≤10 received webhooks\n readonly matcherDetails: string[]; // human-readable matcher summary\n\n toJSON(): Record\u003cstring, unknown>; // serialize all fields for CI logs\n}\n```\n\n`receivedWebhooks` is capped at the last 10 entries. If more than 10 webhooks arrived, `totalReceived` shows the full count but `receivedWebhooks` contains only the most recent 10.\n\n## Reading the Error\n\nThe error message format:\n\n```text\nWebhook \"movie.deleted\" not received within 15000ms.\n3 webhook(s) were received but none matched.\nMatchers: field(event=\"movie.deleted\"), field(data.id=42).\n```\n\nUse `matcherDetails` to confirm the matchers were configured correctly. Use `receivedWebhooks` to inspect actual payloads — compare field paths and values against what the matchers expect.\n\n## Validating the Error Shape in Tests\n\n```typescript\nimport { WebhookTimeoutError, webhookTemplate } from '@seontechnologies/playwright-utils/webhook';\n\nconst neverArrivingTemplate = webhookTemplate('never.arrives')\n .matchField('event', 'event.that.never.happens')\n .withTimeout(500)\n .withInterval(100)\n .build();\n\nconst [waitResult] = await Promise.allSettled([webhookRegistry.waitFor(neverArrivingTemplate)]);\n\nexpect(waitResult.status).toBe('rejected');\nif (waitResult.status !== 'rejected') {\n throw new Error('Expected webhook wait to reject with WebhookTimeoutError');\n}\n\nconst error = waitResult.reason as WebhookTimeoutError;\nexpect(error).toBeInstanceOf(WebhookTimeoutError);\nexpect(error.templateName).toBe('never.arrives');\nexpect(error.timeoutMs).toBe(500);\nexpect(error.toJSON()).toMatchObject({\n name: 'WebhookTimeoutError',\n templateName: 'never.arrives',\n timeoutMs: 500,\n totalReceived: expect.any(Number),\n matcherDetails: ['field(event=\"event.that.never.happens\")'],\n});\n```\n\n## Inspecting receivedWebhooks\n\nWhen a webhook arrives but doesn't match, `receivedWebhooks` shows you what actually came in:\n\n```typescript\n// Wait for create webhook first — puts it in the journal\nawait webhookRegistry.waitFor(movieCreated(movieId));\n\n// Wait for delete webhook that will never arrive — no delete was called\nconst undeliveredDelete = webhookTemplate\u003c{\n event: string;\n data: { id: number };\n}>('movie.deleted.not.delivered')\n .matchField('event', 'movie.deleted')\n .matchField('data.id', movieId)\n .withTimeout(2_000)\n .withInterval(200)\n .build();\n\nconst [waitResult] = await Promise.allSettled([webhookRegistry.waitFor(undeliveredDelete)]);\n\nexpect(waitResult.status).toBe('rejected');\nif (waitResult.status !== 'rejected') {\n throw new Error('Expected webhook wait to reject with WebhookTimeoutError');\n}\n\nconst error = waitResult.reason as WebhookTimeoutError;\nexpect(error).toBeInstanceOf(WebhookTimeoutError);\nexpect(error.totalReceived).toBeGreaterThanOrEqual(1);\n\n// The movie.created webhook that did arrive is visible in the error\nconst createdWebhook = error.receivedWebhooks.find((w) => (w.body as { data: { id: number } }).data.id === movieId);\nexpect(createdWebhook).toBeDefined();\nexpect((createdWebhook!.body as { event: string }).event).toBe('movie.created');\n```\n\n## Common Failure Patterns\n\n| What you see | Likely cause | Fix |\n| -------------------------------------- | ---------------------------------------------------- | ----------------------------------------------------------------- |\n| `totalReceived: 0` | Webhook not delivered; wrong URL or event not firing | Check application event publishing and webhook routing |\n| `totalReceived > 0`, none match | Webhooks arriving but matchers not matching | Inspect `receivedWebhooks[0].body` — check field paths and values |\n| `matcherDetails` shows wrong path | Template factory misconfigured | Print `error.toJSON()` and compare paths against actual payload |\n| `totalReceived: 0` with `matched-only` | Another worker claimed and deleted the webhook first | Ensure template is scoped by entity ID |\n| Parse error in body | Webhook body is not valid JSON | Check `receivedWebhooks[n].parseError` and `rawBody` |\n\n## matcherDetails Format per Matcher Type\n\n| Matcher | matcherDetails string |\n| ------------------------------- | --------------------- |\n| `matchField('event', 'x')` | `field(event=\"x\")` |\n| `matchPartial({ a: 1 })` | `partial({\"a\":1})` |\n| `matchPredicate('my desc', fn)` | `predicate(my desc)` |\n\n## Import\n\n```typescript\nimport { WebhookTimeoutError } from '@seontechnologies/playwright-utils/webhook';\n```\n\n## Related Fragments\n\n- `webhook-template-matchers.md` — matcherDetails string format per matcher type\n- `webhook-waiting-querying.md` — waitFor and waitForCount throw this error on timeout\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5730,"content_sha256":"98313793aac3314653d3e1f543bd6136206132e7cfb9ecb4bea3417587b6d09b"},{"filename":"migration-1.toml","content":"# TEA's first migration, v6 to v7. `bmad migrate` runs a module's `migration-\u003cn>.toml` files in `\u003cn>` order.\n\n[migration]\nmodule = \"tea\"\nfrom = \"6\"\nto = \"7\"\ntitle = \"Move v6 TEA settings to the v7 module config and skill customizations\"\nsummary = \"Keeps the answers the classic installer recorded, moves the two settings that only one skill reads into that skill's customization file, drops the settings no skill reads any more, and offers to remove the classic installer's leftovers under _bmad/tea/.\"\n\n# Read-only signals. Any one of them makes this migration worth offering.\ndetect = \"\"\"\n- `_bmad/tea/config.yaml` exists (the classic installer's TEA config).\n- `_bmad/tea/module-help.csv` exists.\n- `[modules.tea]` in `_bmad/config.toml`, `_bmad/custom/config.toml` or `_bmad/custom/config.user.toml` holds `ci_platform`, `tea_evaluations_folder`, `risk_threshold`, `test_design_output`, `test_review_output` or `trace_output`. The classic installer in bmad-method 6.12 already writes this table, so a project can have a `[modules.tea]` and still be on v6.\nA project with none of those signals, whose merged `[modules.tea]` holds only supported keys `bmad setup tea` asks, is on v7; say so and stop. Existing v7 configs may omit the four healing options; test generation applies the shipped defaults without requiring migration.\n\"\"\"\n\n# The state every rule below moves toward.\ntarget = \"\"\"\n- `_bmad/config.toml` `[modules.tea]` uses these supported keys: `test_artifacts`, `tea_use_playwright_utils`, `tea_use_pactjs_utils`, `tea_pact_mcp`, `tea_browser_automation`, `tea_execution_mode`, `tea_capability_probe`, `test_stack_type`, `test_framework`, `auto_validate`, `auto_heal_failures`, `max_healing_iterations`, `use_mcp_healing`. Every stored value is a string; v6 booleans (`true`, `false`) become `\"true\"` and `\"false\"`. Omitted healing options keep their runtime defaults: `auto_validate = \"true\"`, `auto_heal_failures = \"true\"`, `max_healing_iterations = \"3\"`, and `use_mcp_healing = \"true\"`.\n- A v6 `ci_platform` other than `auto` lives in `_bmad/custom/bmad-tea-ci.toml` as `ci_platform = \"\u003cvalue>\"` under `[workflow]`.\n- A v6 `tea_evaluations_folder` other than `evals` lives in `_bmad/custom/bmad-tea-evaluate.toml` as `evaluations_folder = \"\u003cfolder>\"` under `[workflow]`, relative to the project root.\n- `risk_threshold`, `test_design_output`, `test_review_output` and `trace_output` are gone; no skill reads them.\n- `_bmad/tea/config.yaml`, `_bmad/tea/module-help.csv` and any classic-installer copies of TEA's agents or workflows under `_bmad/tea/` are removed, so the CLI and the skills read only the v7 install.\n\"\"\"\n\nprecautions = \"\"\"\n- Read before writing. Show the user the plan (each key and where it goes) and wait for approval.\n- Edit only the `[modules.tea]` tables of `_bmad/config.toml`, `_bmad/custom/config.toml` and `_bmad/custom/config.user.toml`, and the two customization files `_bmad/custom/bmad-tea-ci.toml` and `_bmad/custom/bmad-tea-evaluate.toml`. Keep every other table, key and comment in those files as it is, and merge into a customization file that already exists instead of replacing it.\n- Never delete a file outside version control. Removing the leftovers is a commit the user chooses; offer it and stop there.\n- Preserve `_bmad/custom/bmad-testarch-atdd.toml`, `_bmad/custom/bmad-testarch-atdd.user.toml`, `_bmad/custom/bmad-testarch-automate.toml` and `_bmad/custom/bmad-testarch-automate.user.toml` until `tea-migrate-customizations` moves them. The combined skill resolves the selected mode's customizations.\n- Never run `bmad setup` from inside this migration; it owns the rest of `_bmad/`. When `[modules.tea]` is missing from every layer, stop and tell the user to run `bmad setup tea` first and to run this migration again afterwards.\n\"\"\"\n\nguide = \"\"\"\n1. **Inventory.** Read `[modules.tea]` in `_bmad/config.toml`, `_bmad/custom/config.toml` and `_bmad/custom/config.user.toml` (later files win, so the user layer is the value the skills read), `_bmad/tea/config.yaml`, and `_bmad/custom/bmad-tea-ci.toml` and `_bmad/custom/bmad-tea-evaluate.toml`, each when it exists. List every TEA key with its effective value and every file that holds it. A user who pinned a value in a custom layer chose it; that value is the one to keep.\n2. **Plan.** Build the table below for the user: one row per key found, naming its destination.\n\n | v6 key | v7 destination |\n |---|---|\n | `test_artifacts`, `tea_use_playwright_utils`, `tea_use_pactjs_utils`, `tea_pact_mcp`, `tea_browser_automation`, `tea_execution_mode`, `tea_capability_probe`, `test_stack_type`, `test_framework`, `auto_validate`, `auto_heal_failures`, `max_healing_iterations`, `use_mcp_healing` | stays in `[modules.tea]` of the layer that holds it, written as strings |\n | `ci_platform` | `_bmad/custom/bmad-tea-ci.toml`, `[workflow]` `ci_platform`, only when the effective value is not `auto`; removed from every `[modules.tea]` layer |\n | `tea_evaluations_folder` | `_bmad/custom/bmad-tea-evaluate.toml`, `[workflow]` `evaluations_folder`, only when the effective value is not `evals`; strip a leading `{project-root}/` when the value has one; removed from every `[modules.tea]` layer |\n | `risk_threshold`, `test_design_output`, `test_review_output`, `trace_output` | removed from every `[modules.tea]` layer |\n | any other key | ask the user; it is not a TEA setting v7 reads |\n\n Also list the classic-installer leftovers found under `_bmad/tea/` (`config.yaml`, `module-help.csv`, and any `agents/` or `workflows/` folder) as removal candidates.\n3. **Answers recorded only in `config.yaml`.** `bmad setup tea` writes the defaults it is given, so a project that had only a `_bmad/tea/config.yaml` can hold its v6 answers there while `[modules.tea]` holds defaults. For each of the thirteen kept keys where `config.yaml` differs from the effective `[modules.tea]` value and the effective value equals the default in `bmod.toml`, add a plan row that restores the `config.yaml` value in `_bmad/config.toml`. Show these rows apart from the rest and apply each only on approval; a value the user already changed from the default is never overwritten.\n4. **Apply the approved plan.** Rewrite the `[modules.tea]` values that are not strings, remove the keys the plan moves or drops from every layer that holds them, and write the two customization files. Create `_bmad/custom/` entries only for values that differ from the defaults named above.\n5. **Leftovers.** Offer to remove the candidates from step 2 as one commit. A `config.yaml` left beside `_bmad/config.toml` is ignored by the skills and read only by `tea-test-review` when `_bmad/config.toml` does not exist, so it is harmless but misleading, and this migration is offered again while it exists. Say so. On a rerun, offer the removal again and show a step 3 row only while its two values still differ.\n6. **Report.** Give the user the final `[modules.tea]` table, the two customization files that were written, and the leftovers that remain. Run `npx --package=bmad-method-test-architecture-enterprise@3 tea-migrate-customizations --project-root .` to move any other pre-3.0 customization filenames. Resolve a reported old/new value conflict before rerunning it.\n\"\"\"\n\nchecklist = [\n \"The merged `[modules.tea]` (`_bmad/config.toml` with the custom layers) holds only the thirteen supported v7 keys, each stored value a string; omitted healing options use the shipped defaults.\",\n \"A non-default v6 `ci_platform` is `ci_platform` under `[workflow]` in `_bmad/custom/bmad-tea-ci.toml`.\",\n \"A non-default v6 `tea_evaluations_folder` is `evaluations_folder` under `[workflow]` in `_bmad/custom/bmad-tea-evaluate.toml`, without the `{project-root}/` prefix.\",\n \"`risk_threshold`, `test_design_output`, `test_review_output` and `trace_output` are in no `[modules.tea]` layer.\",\n \"`ci_platform` and `tea_evaluations_folder` are in no `[modules.tea]` layer.\",\n \"`uv run _bmad/scripts/resolve_config.py --project-root . --key modules.tea` prints only supported v7 keys; omitted healing options use the shipped defaults.\",\n \"`uv run _bmad/scripts/resolve_customization.py --skill \u003cevaluate-skill-root> --project-root . --key workflow` prints the migrated Evaluate value; Framework CI scope merges `_bmad/custom/bmad-tea-ci.toml` and its user layer to read the migrated CI value.\",\n \"Every file the plan listed for removal is either removed in the commit the user chose or reported as still present.\",\n]\n","content_type":"text/plain; charset=utf-8","language":"toml","size":8483,"content_sha256":"e0bb466e4b911d5f88c70defbc1722ac29d17915b2ceead956cd7a9c0fda7231"},{"filename":"migration-2.toml","content":"# TEA's second migration: story outputs written under v6 story keys move onto v7 ticket keys. `bmad migrate` runs a module's `migration-\u003cn>.toml` files in `\u003cn>` order.\n\n[migration]\nmodule = \"tea\"\nfrom = \"3 with v6 story keys\"\nto = \"3 with v7 story keys\"\ntitle = \"Move TEA story outputs onto their v7 tickets\"\nsummary = \"Finds checklists, summaries, matrices and reports keyed by a v6 story file or a v7 leaf file name, maps each to its v7 ticket from exact evidence, and, once the user approves the plan, renames each to the ticket's key and rewrites its identity fields. Historical citations stay as written; a resolved-source list beside them names where each source lives now. Nothing outside the test artifacts folder changes.\"\n\n# Read-only signals. An unfinished plan, or all three of the rest.\ndetect = \"\"\"\n- `{test_artifacts}/migration-tea-v7.md` exists with a `status` other than `done`: an earlier run stopped part way; resume it (\"Resume\") before anything else.\n- That plan exists with `status: done` and its unresolved list names every pre-v7 story output still present: the project is already migrated. Report the plan's results and stop, writing nothing. A pre-v7 output the plan does not list is new work: run again with a new dated run section appended to the same plan, keeping its history and the first backup, and a new backup of the files that run touches.\n- `{project-root}/_bmad/method/scripts/tickets.py` exists, and the ticket tree is current v7: `uv run {project-root}/_bmad/method/scripts/tickets.py --project-root {project-root} status \u003cfolder>` succeeds for each `initiative-*/` folder and `backlog/` under the output folder (passing the folder works with no active initiative). When one errors with the earlier layout, or `{output_folder}/migration-v6-v7/migration-v6-v7.md` exists without `status: done`, the method migration comes first; say so and stop.\n- The test artifacts folder (`modules.tea.test_artifacts`) holds a story output whose key is not a v7 key. Story outputs are `atdd/atdd-checklist-\u003ckey>.md`, `automate/automation-summary-story-\u003ckey>.md`, `trace/traceability-matrix-story-\u003ckey>.md` with Trace's summary and gate files of the same run key, `nfr/nfr-assessment-story-\u003ckey>.md` and `test-review/test-review-story-\u003ckey>.md`, the same names at the folder's root, and the fixed root names `automation-summary.md`, `traceability-matrix.md`, `nfr-assessment.md` and `test-review.md` when their content names one story.\n- A v7 key contains `--`: every shape in `bmod-tea/contracts/story-identity.md` section 4 does (`shop--e1-s2`, `shop--epic-loyalty-s2`, `shop--s6a`, `backlog--story-x`), and no pre-v7 key can, since v6 basenames and leaf stems are slugs. A key without `--`, such as `1-2-user-authentication` or `story-guest-checkout`, is a pre-v7 key.\n\"\"\"\n\ntarget = \"\"\"\n\u003ctest_artifacts>/\n atdd/atdd-checklist-\u003cv7 key>.md\n automate/automation-summary-story-\u003cv7 key>.md\n trace/traceability-matrix-story-\u003cv7 key>.md # with its summary and gate files, renamed together\n nfr/nfr-assessment-story-\u003cv7 key>.md\n test-review/test-review-story-\u003cv7 key>.md\n migration-tea-v7.md # the plan, applied moves, and unresolved list\n\u003cproject-root>/.tea-v7-migration-backup-\u003cdatetime>/ # a copy of the test artifacts folder, made first\nEach moved file keeps its status, steps, evidence and gate decision. Its identity fields name the ticket: `storyId`, `storyKey`, `runKey = story-\u003cv7 key>`, `ticketRef`, target id and label, its own path fields, and links between the files of one Trace run. `migratedFrom` holds the old values. Its citations (`inputDocuments`, `oracleSources`, `oracleLedger` sources) stay verbatim, and `resolvedSources` lists, per citation, `original`, `current` and `status: resolved | unresolved`.\n\"\"\"\n\nprecautions = \"\"\"\n- Nothing changes before the user approves the plan. The only writes before approval are the backup and the plan file.\n- Never overwrite a file, and never leave two files with one story key in a workflow folder. An occupied destination, or two sources for one destination, is a conflict the user decides: keep the existing destination, or promote one source to it. Every file not kept at the destination moves unchanged to `\u003ctest_artifacts>/migration-tea-v7-preserved/\u003cits path relative to the test artifacts folder>`, outside every workflow's lookups; a root-level file and a folder file with one name therefore land apart. Every preserved destination is in the approved plan and is checked for an existing file before anything moves.\n- Never write outside the test artifacts folder and the backup. Ticket files, plans, stories and the method record are read only.\n- A key prefix or a similar title is never evidence. A file with no exact evidence stays where it is and is listed as unresolved.\n- When the files are under git, every move is a `git mv`, so history follows the file; an untracked file moves with `mv` and no overwrite.\n\"\"\"\n\nguide = \"\"\"\n## Sequence\n\n1. Check the signals in `detect`. An unfinished plan goes to \"Resume\"; otherwise stop when one fails.\n2. Backup (\"Backup\").\n3. Inventory (\"Inventory\").\n4. Map each set to a ticket (\"Map\").\n5. Write the plan and wait for approval (\"Plan\").\n6. Apply the approved moves (\"Apply\").\n7. Run every `checklist` item and record its result in the plan. Fix and recheck a failure, or report it; never skip one.\n8. Report (\"Report\").\n\nUnder git, commit before step 6 and after it, the message naming the step; never stage the backup. Never push.\n\n## Resume\n\nRead the unfinished plan. Its approved sets, mappings and conflict decisions stand; never write a new plan over it, and keep the first backup. The plan journals each file with its disposition and state: `adopted` (`pending`, `moved`, `rewritten`), `preserved` (`pending`, `moved`), `unresolved` or `left` (no operation). Resume only pending work: an `adopted` file still at its old path is moved, one at its new path but not `rewritten` gets its identity fields; a `preserved` file still at its old path moves unchanged and is never rewritten. When a file is at neither its old nor its planned path, or a source and its destination both exist, stop and report it. Then continue at step 7.\n\n## Backup\n\nRead `core.output_folder` and `modules.tea.test_artifacts` with `uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key core.output_folder --key modules.tea.test_artifacts`. Copy the whole test artifacts folder to `{project-root}/.tea-v7-migration-backup-\u003cdatetime>/` (`YYYYMMDD-HHMM`), write a `.gitignore` there holding `*`, and confirm the copy holds every file. On a rerun whose plan already exists, keep the first backup.\n\n## Inventory\n\nList every story output `detect` names that carries a pre-v7 key. Group the files of one run into a set: a Trace matrix with the summary and gate files of its run key; any other output alone. Read each set's frontmatter for `storyId`, `storyKey`, `storyFile`, `runKey` and its citations. A fixed root name whose content names no single story is listed as unresolved, not guessed.\n\n## Map\n\nMap each set to one ticket from every exact evidence that holds, checked in this order:\n\n1. Its `storyFile`. When the file still exists inside the ticket tree, pass its container folder and basename to `tickets.py find`. When it no longer exists, look the path up in the method record `{output_folder}/migration-v6-v7/migration-v6-v7.md`, which states what moved where and each story's v7 entry in prose; read the entry it names.\n2. An `[[entry]]` in any `tickets/tickets.toml` under the output folder whose `v6_key` equals the set's key exactly.\n3. The method record's statement of the story's v7 entry, for the set's `storyId`, the story id its H1 heading states (`Story 1.1: ...`), or its key read as a v6 story file name (for a spec-folder story, its original `stories/\u003cid>-\u003cslug>.md`). A title alone is never evidence.\n\nConfirm every mapped ticket with `uv run \"{project-root}/_bmad/method/scripts/tickets.py\" --project-root \"{project-root}\" find \"\u003ccontainer>\" \"\u003cref>\"`, always naming the container, so no active initiative is needed, and compute its key from the result by `bmod-tea/contracts/story-identity.md` section 4. Evidence that names no ticket, or pieces of evidence that name different tickets, leave the set unresolved, with each conflicting piece listed in the plan.\n\nFor each citation in a mapped set, resolve where its source lives now the same way: the record's moves, then `find`. A cited line number is kept as history; never claim it still points at the same text.\n\n## Plan\n\nWrite `{test_artifacts}/migration-tea-v7.md` with frontmatter `type: migration`, `status: draft`, `created`. It lists, per set: the old and new path of every file (a root-level file moves into its workflow's folder), the ticket `ref` and container, the evidence used, the identity fields that change with old and new values, and each citation's resolution. Then every conflict with its options, and every unresolved set with the reason. Show it and wait for approval and a disposition for each conflict.\n\n## Apply\n\nUnder git, make every approved move first and commit the moves alone, so git records each as a rename; then rewrite content in a second commit. Work through the approved sets in plan order. For each: move its files, rewrite the identity fields and `migratedFrom` as `target` says, add `resolvedSources`, and record each file's state in the plan's journal as it changes (`moved`, then `rewritten` for adopted files; `moved` is terminal for preserved ones). A restart goes through \"Resume\". Leave unresolved sets in place; move each file a conflict decision did not keep to the preserved folder unchanged.\n\n## Report\n\nSet the plan's `status` to `done`. Report the sets moved, the conflicts and how each was decided, the unresolved sets with their reasons, and the backup folder, which is the user's to delete once satisfied. Say that the workflows now name story outputs by ticket key, and that an unresolved file stays readable at its old path.\n\"\"\"\n\nchecklist = [\n \"Every approved set's files are at their planned paths, and no source file was overwritten: each destination that existed before the run still holds its original content or the user's chosen disposition.\",\n \"Under git, each moved tracked file keeps its history (`git log --follow` reaches its commits from before the move), and no commit includes the backup.\",\n \"No workflow folder holds two files with one story key; each file a conflict decision did not keep is unchanged under `migration-tea-v7-preserved/`. One story may have one set per workflow.\",\n \"Each adopted checkpoint's `runKey` is `story-\u003cits new key>`, its `storyKey`, `ticketRef` and path fields match its new location, and its status, steps, evidence and gate decision equal the backup copy's.\",\n \"Each adopted in-progress checkpoint resumes in its workflow by the ticket's ref and, with no story named, as the only candidate, under its new `runKey`.\",\n \"Every citation in a moved set is verbatim against the backup copy, and has a `resolvedSources` row marked `resolved` with a location that exists, or `unresolved`.\",\n \"Every preserved file is byte-identical to its backup copy and sits at `migration-tea-v7-preserved/\u003cits original relative path>`.\",\n \"Running this migration again changes no file, leaves the plan byte-identical, and reports the same unresolved sets; a run stopped after the moves and resumed finishes every file's pending operation without a new plan.\",\n \"`.tea-v7-migration-backup-\u003cdatetime>/` holds every file the test artifacts folder held before the first run.\",\n]\n","content_type":"text/plain; charset=utf-8","language":"toml","size":11625,"content_sha256":"bf1b930921ded752c47a4bc77ba5089a2f9f5d95f558579445a7c30250ee81b7"},{"filename":"roster.toml","content":"# Workflow modes and compatibility commands remain on the agent menu.\n# The people the tea module offers. A member with `skill` is an agent, present only while that\n# skill is installed; its name, title and icon follow the skill's customization.\n\n[[members]]\ncode = \"bmad-tea\"\nskill = \"bmad-tea\"\nname = \"Murat\"\nicon = \"🧪\"\ntitle = \"Master Test Architect and Quality Advisor\"\npersona = \"Risk-based testing strategy, fixture architecture, ATDD, API and UI automation (Playwright, Cypress, pytest, JUnit, Go test, xUnit, RSpec), mobile native automation (Maestro), consumer-driven contract testing (Pact), and performance/load/chaos testing (k6). Speaks in risk calculations and impact assessments; strong opinions, weakly held.\"\n","content_type":"text/plain; charset=utf-8","language":"toml","size":729,"content_sha256":"6e8862085332e361cbc2eb9c760f49475c11ce639a09f9d9873b329eefc12eaa"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"bmod-tea","type":"text"}]},{"type":"paragraph","content":[{"text":"This folder is the Test Architect module's record, not something to run. Invoke the ","type":"text"},{"text":"bmad","type":"text","marks":[{"type":"code_inline"}]},{"text":" skill with ","type":"text"},{"text":"setup tea","type":"text","marks":[{"type":"code_inline"}]},{"text":"; it sets the module up if it never was, and otherwise reports its state. If there is no ","type":"text"},{"text":"bmad","type":"text","marks":[{"type":"code_inline"}]},{"text":" skill, say so and offer ","type":"text"},{"text":"npx skills add bmad-code-org/BMAD-METHOD --skill bmad","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},"metadata":{"date":"2026-11-02","name":"bmod-tea","author":"@skillopedia","source":{"stars":115,"repo_name":"bmad-method-test-architecture-enterprise","origin_url":"https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/HEAD/skills/bmod-tea/SKILL.md","repo_owner":"bmad-code-org","body_sha256":"9c32b8f48d6f6b33ccbd9a3fac89ab43471f1c1a2ecf1e7a9e4039dcb82b4a0d","cluster_key":"996e9c8da7a524ac9ad28014bdde66a801c256cc88a3e759e9374ead777af209","clean_bundle":{"format":"clean-skill-bundle-v1","source":"bmad-code-org/bmad-method-test-architecture-enterprise/skills/bmod-tea/SKILL.md","text_bytes":1046267,"source_commit":"a2819f62fc1b56486d223895eaa50578b490f2f5","skills_sh_rank":19527,"attachment_count":66,"text_attachments":66,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/bmod-tea/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"general","category_label":"General"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"general","import_tag":"skills-sh-top20k-2026-11","description":"Required bmod metadata. Never invoke this skill."}},"renderedAt":1793751786176}

bmod-tea This folder is the Test Architect module's record, not something to run. Invoke the skill with ; it sets the module up if it never was, and otherwise reports its state. If there is no skill, say so and offer .