typescript-e2e-testing

E2E Testing Skill E2E testing validates complete workflows from user perspective, using real infrastructure via Docker. --- Workflows For comprehensive step-by-step guidance, use the appropriate workflow: | Workflow | When to Use | |----------|-------------| | Setup E2E Test | Setting up E2E infrastructure for a new or existing project | | Writing E2E Test | Creating new E2E test cases with proper GWT pattern | | Review E2E Test | Reviewing existing tests for quality and correctness | | Running E2E Test | Executing tests with proper verification | | Debugging E2E Test | Systematically fixing…

: 'ts-jest',\n },\n // ESM compatibility - map .js imports to .ts files\n moduleNameMapper: {\n '^(\\\\.{1,2}/.*)\\\\.js

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.

: '$1',\n },\n testTimeout: 25000,\n setupFilesAfterEnv: ['\u003crootDir>/setup.ts'],\n\n // CRITICAL: Run tests serially for database/messaging isolation\n maxWorkers: 1,\n\n // CRITICAL: Clean mock state between test files\n clearMocks: true,\n resetMocks: true,\n restoreMocks: true,\n\n // Prevent hanging connections\n forceExit: true,\n detectOpenHandles: true,\n};\n\nexport default config;\n```\n\n### package.json Scripts\n\n```json\n{\n \"scripts\": {\n \"test:e2e\": \"jest --config test/jest-e2e.config.ts --runInBand\",\n \"test:e2e:watch\": \"jest --config test/jest-e2e.config.ts --runInBand --watch\",\n \"test:e2e:debug\": \"node --inspect-brk node_modules/.bin/jest --config test/jest-e2e.config.ts --runInBand\",\n \"e2e:infra:up\": \"docker-compose -f docker-compose.e2e.yml up -d\",\n \"e2e:infra:down\": \"docker-compose -f docker-compose.e2e.yml down -v\",\n \"e2e:logs\": \"tail -f logs/e2e-test.log\"\n }\n}\n```\n\n### Running Tests (Context Efficient)\n\n**ALWAYS redirect output to temp files with unique session ID:**\n\n```bash\n# Initialize session (once at start)\nexport E2E_SESSION=$(date +%s)-$\n\n# Run all E2E tests (no console output)\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Run specific test (no console output)\nnpm run test:e2e -- -t \"test name\" > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Get failure details\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log\n\n# Debug mode (requires console for interactive debugging)\nnpm run test:e2e:debug\n\n# Cleanup when done\nrm -f /tmp/e2e-${E2E_SESSION}-*.log\n```\n\n## Test Setup Pattern\n\n### Global Setup (setup.ts)\n\n```typescript\nimport * as fs from 'fs';\n\n// Clean log file before test run\nbeforeAll(() => {\n const logFile = './logs/e2e-test.log';\n if (fs.existsSync(logFile)) {\n fs.unlinkSync(logFile);\n }\n // Ensure logs directory exists\n if (!fs.existsSync('./logs')) {\n fs.mkdirSync('./logs', { recursive: true });\n }\n});\n```\n\n### Test App Bootstrap Helper\n\n```typescript\nimport { Test, TestingModule } from '@nestjs/testing';\nimport { INestApplication, ValidationPipe } from '@nestjs/common';\nimport { ConfigModule, ConfigService } from '@nestjs/config';\nimport { HttpAdapterHost, Reflector } from '@nestjs/core';\nimport { ClsService } from 'nestjs-cls';\nimport { Transport } from '@nestjs/microservices';\n\nexport interface TestSetup {\n app: INestApplication;\n httpServer: any;\n moduleFixture: TestingModule;\n}\n\nexport async function createTestApp(\n modules: any[],\n options?: { kafka?: boolean }\n): Promise\u003cTestSetup> {\n const moduleFixture = await Test.createTestingModule({\n imports: [\n ConfigModule.forRoot({ isGlobal: true, envFilePath: '.env.e2e' }),\n ...modules,\n ],\n }).compile();\n\n const app = moduleFixture.createNestApplication();\n const { httpAdapter } = app.get(HttpAdapterHost);\n const reflector = app.get(Reflector);\n const cls = app.get(ClsService);\n const configService = app.get(ConfigService);\n\n // ✅ CRITICAL: Use custom logger (same as main.ts)\n app.useLogger(app.get(CustomLoggerService));\n\n // ✅ CRITICAL: Global interceptors (same as main.ts)\n app.useGlobalInterceptors(new HttpRequestLoggingInterceptor(cls, reflector));\n\n // ✅ CRITICAL: Global exception filters (same as main.ts)\n app.useGlobalFilters(new UnknownExceptionsFilter(httpAdapter));\n app.useGlobalFilters(new DefaultValidateExceptionFilter(httpAdapter));\n app.useGlobalFilters(new DefaultInternalExceptionFilter(httpAdapter));\n app.useGlobalFilters(new HttpExceptionFilter(httpAdapter));\n\n // ✅ CRITICAL: ValidationPipe BEFORE connectMicroservice\n app.useGlobalPipes(new ValidationPipe({\n whitelist: true,\n transform: true,\n forbidNonWhitelisted: true,\n }));\n\n if (options?.kafka) {\n const uniqueGroupId = `${configService.get('KAFKA_GROUP_ID')}-e2e-${Date.now()}`;\n\n app.connectMicroservice({\n transport: Transport.KAFKA,\n options: {\n client: {\n clientId: configService.get('KAFKA_CLIENT_ID'),\n brokers: [configService.get('KAFKA_BROKER')],\n },\n consumer: {\n groupId: uniqueGroupId,\n allowAutoTopicCreation: true,\n },\n subscribe: { fromBeginning: false },\n },\n }, { inheritAppConfig: true });\n\n await app.startAllMicroservices();\n }\n\n await app.init();\n\n return {\n app,\n httpServer: app.getHttpServer(),\n moduleFixture,\n };\n}\n```\n\n## Complete Test Suite Example\n\n```typescript\ndescribe('Feature E2E', () => {\n let app: INestApplication;\n let httpServer: any;\n let userRepository: Repository\u003cUser>;\n\n jest.setTimeout(25000);\n\n beforeAll(async () => {\n const setup = await createTestApp([AppModule], { kafka: true });\n app = setup.app;\n httpServer = setup.httpServer;\n userRepository = setup.moduleFixture.get(getRepositoryToken(User));\n\n // Wait for Kafka consumer to be ready\n await new Promise(r => setTimeout(r, 5000));\n }, 90000);\n\n afterAll(async () => {\n await app?.close();\n }, 30000);\n\n beforeEach(async () => {\n await new Promise(r => setTimeout(r, 1000));\n await userRepository.clear();\n });\n\n afterEach(async () => {\n await userRepository.clear();\n });\n\n it('should create user', async () => {\n // GIVEN: Valid user data\n const createUserDto = { email: '[email protected]', name: 'Test' };\n\n // WHEN: Creating user\n const response = await request(httpServer)\n .post('/users')\n .send(createUserDto)\n .expect(201);\n\n // THEN: User created\n expect(response.body.data.email).toBe('[email protected]');\n\n // THEN: Persisted in database\n const savedUser = await userRepository.findOne({\n where: { email: '[email protected]' },\n });\n expect(savedUser).toBeDefined();\n });\n});\n```\n\n## Common Issues and Solutions\n\n### Issue: Tests hang after completion\n\n```typescript\n// Solution 1: Force exit in jest config\nforceExit: true,\ndetectOpenHandles: true,\n\n// Solution 2: Proper cleanup in afterAll\nafterAll(async () => {\n await app?.close();\n}, 30000);\n```\n\n### Issue: Port conflicts between test runs\n\n```typescript\n// Solution: Use random ports\nconst app = await NestFactory.create(AppModule);\nawait app.listen(0); // Random available port\n```\n\n### Issue: Database state pollution\n\n```typescript\n// Solution: Clean before AND after each test\nbeforeEach(async () => {\n await connection.synchronize(true); // PostgreSQL\n await collection.deleteMany({}); // MongoDB\n});\n\nafterEach(async () => {\n await collection.deleteMany({});\n});\n```\n\n### Issue: Kafka consumer lag\n\n```typescript\n// Solution: Use unique consumer groups and skip old messages\nconst uniqueGroupId = `test-group-${Date.now()}-${Math.random()}`;\nsubscribe: { fromBeginning: false }\n```\n\n### Issue: Tests fail intermittently (flaky)\n\n```typescript\n// Solution: Ensure sequential execution\n// In jest config:\nmaxWorkers: 1\n// In npm script:\n\"test:e2e\": \"jest --runInBand\"\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":8194,"content_sha256":"237c7b456a917c47c3fe660393e4ec7ba39939e6d315b68076ca5991dfd7a11c"},{"filename":"references/common/rules.md","content":"# E2E Testing Rules\n\n## Core Rules\n\n| Rule | Requirement |\n|------|-------------|\n| Setup | Replicate `main.ts` configuration exactly |\n| Execution | `--runInBand` (sequential) - NEVER parallel |\n| Logger | File logging via `.env.e2e` - DO NOT mock, DO NOT console.log |\n| Log file | `logs/e2e-test.log` - clean before each test run |\n| Test structure | MUST follow GWT pattern (Given-When-Then) |\n| **Output** | **ALWAYS redirect to temp files** - prevents context bloat |\n\n## Context Efficiency Rule\n\n**CRITICAL**: All test execution MUST output to temp files with unique session ID.\n\n```bash\n# Initialize session (once at start)\nexport E2E_SESSION=$(date +%s)-$\n\n# Standard pattern - redirect to temp file only (no console output)\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Get failures\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log\n\n# Cleanup when done\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md\n```\n\n## Environment Configuration\n\nE2E tests MUST use `.env.e2e`:\n\n```bash\n# Logger configuration\nLOG_LEVEL=debug\nLOG_FILE=./logs/e2e-test.log\nLOG_SYNC_FILE=./logs/application.pid\n\n# Database\nDATABASE_URL=postgresql://test:test@localhost:5433/testdb\nMONGODB_URI=mongodb://localhost:27018/testdb\n\n# Kafka\nKAFKA_BROKER=localhost:9094\nKAFKA_CLIENT_ID=e2e-test-client\nKAFKA_GROUP_ID=e2e-test-group\n\n# Redis\nREDIS_URL=redis://localhost:6380\n\n# Application\nNODE_ENV=test\n```\n\n## Timeout Rules\n\n| Context | Value | Purpose |\n|---------|-------|---------|\n| `jest.setTimeout()` | 25000 | Default per-test timeout |\n| `beforeAll` | 90000 | Full app + infrastructure setup |\n| `afterAll` | 30000 | Graceful cleanup |\n| Kafka waits | 8-10s | Consumer group rebalancing |\n| MongoDB cleanup wait | 1000ms | In-flight message completion |\n\n## Test Isolation Rules\n\n1. **Clean Before AND After:** Clear data in both `beforeEach` and `afterEach`\n2. **Wait Before Cleanup:** Add 1s delay before cleanup for in-flight operations\n3. **Unique Identifiers:** Generate unique IDs per test (e.g., `test-${Date.now()}-${uuid}`)\n4. **Sequential Execution:** Use `--runInBand` to prevent conflicts\n\n## Logging Rules\n\n```typescript\n// ❌ BAD - NEVER DO\nconsole.log('debug info');\nmockLogger.log.mockImplementation(() => {});\n\n// ✅ GOOD - ALWAYS\napp.useLogger(app.get(CustomLoggerService));\n// Configure LOG_FILE in .env.e2e\n```\n\n**Viewing Logs:**\n```bash\n# View test output (from temp file)\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Get failure details\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log\n\n# View application logs (limited)\ntail -100 logs/e2e-test.log\ngrep -i error logs/e2e-test.log | tail -20\n```\n\n## Mock Rules\n\n**Mock ALL retry attempts:**\n```typescript\n// ❌ BAD - Retry succeeds on 2nd attempt\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 }));\n\n// ✅ GOOD - All 3 retry attempts fail\nmockHttpService.post.mockClear();\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 })); // attempt 1\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 })); // attempt 2\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 })); // attempt 3\n```\n\n## Exception Type Rules\n\n| Exception | HTTP Status | Use Case |\n|-----------|-------------|----------|\n| `ValidateException` | 400 | Input validation, malformed requests |\n| `InternalException` | 500 | Infrastructure failures, external service errors |\n| `UnauthorizedException` | 401 | Authentication failures |\n| `ForbiddenException` | 403 | Authorization failures |\n| `NotFoundException` | 404 | Resource not found |\n\n## NestJS Setup Rules\n\n**CRITICAL:** E2E tests MUST replicate production setup:\n\n```typescript\n// ✅ REQUIRED in beforeAll\napp.useLogger(app.get(CustomLoggerService));\napp.useGlobalFilters(new UnknownExceptionsFilter(httpAdapter));\napp.useGlobalFilters(new DefaultValidateExceptionFilter(httpAdapter));\napp.useGlobalFilters(new DefaultInternalExceptionFilter(httpAdapter));\napp.useGlobalFilters(new HttpExceptionFilter(httpAdapter));\napp.useGlobalInterceptors(new HttpRequestLoggingInterceptor(cls, reflector));\napp.useGlobalInterceptors(new KafkaRequestLoggingInterceptor(cls, reflector));\napp.useGlobalPipes(new ValidationPipe(DefaultValidationOptions));\n\n// ✅ REQUIRED for Kafka\napp.connectMicroservice(kafkaConfig, { inheritAppConfig: true });\n```\n\n## Failure Handling Rules\n\n**CRITICAL:** Fix ONE test at a time. NEVER run full suite repeatedly while debugging.\n\n```\n❌ WRONG: Run full suite → See 5 failures → Run full suite again → Still failures → ...\n✅ RIGHT: Run full suite → See 5 failures → Fix test 1 → Verify → Fix test 2 → ... → Full suite ONCE\n```\n\n**Workflow:**\n1. Create `/tmp/e2e-${E2E_SESSION}-failures.md` tracking file with ALL failing tests\n2. Select ONE failing test\n3. Run ONLY that test (never full suite, no console output):\n ```bash\n npm run test:e2e -- -t \"test name\" > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\n tail -50 /tmp/e2e-${E2E_SESSION}-debug.log\n ```\n4. Analyze failures: `grep -B 2 -A 15 \"FAIL\\|Error:\" /tmp/e2e-${E2E_SESSION}-debug.log`\n5. Fix and verify with 3-5 runs of SAME test:\n ```bash\n for i in {1..5}; do npm run test:e2e -- -t \"test name\" > /tmp/e2e-${E2E_SESSION}-run$i.log 2>&1 && echo \"Run $i: PASS\" || echo \"Run $i: FAIL\"; done\n ```\n6. Mark as FIXED in tracking file\n7. Move to next failing test - repeat steps 2-6\n8. Run full suite ONLY ONCE after ALL individual tests pass:\n ```bash\n npm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n ```\n9. Delete tracking file: `rm /tmp/e2e-${E2E_SESSION}-failures.md`\n\n**WHY**: Full suite runs waste time and context. Each failing test pollutes output.\n\n## Checklist\n\n**Environment:**\n- [ ] `.env.e2e` with file logging configuration\n- [ ] `logs/` directory in `.gitignore`\n- [ ] Global setup deletes previous log file\n\n**Setup:**\n- [ ] Use real logger via `app.useLogger()`\n- [ ] Global filters match `main.ts`\n- [ ] Global interceptors match `main.ts`\n- [ ] ValidationPipe BEFORE `connectMicroservice`\n- [ ] `inheritAppConfig: true` for Kafka\n- [ ] Unique consumer group ID\n- [ ] `--runInBand` in package.json\n\n**Test Structure:**\n- [ ] ALL tests follow GWT pattern\n- [ ] GWT sections marked: `// GIVEN:`, `// WHEN:`, `// THEN:`\n- [ ] One test, one behavior\n- [ ] No conditional assertions\n\n**Assertions:**\n- [ ] Assert specific values, not just existence\n- [ ] Tests fail when any field differs\n- [ ] Verify database state matches expected\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":6542,"content_sha256":"210390dfadeea1929569c6b1b30a94621ddac2f2fff49637adb798b229ce7dfe"},{"filename":"references/common/test-case-creation-guide.md","content":"# E2E Test Case Creation Guide\n\n## GWT Pattern (Given-When-Then)\n\n**CRITICAL**: ALL E2E tests MUST follow GWT pattern with explicit comments.\n\n```typescript\nit('should [expected behavior]', async () => {\n // GIVEN: [preconditions and context]\n // Setup test data, initial state, dependencies\n\n // WHEN: [single action being tested]\n // ONE primary action only\n\n // THEN: [expected outcomes]\n // Verify response, database state, side effects\n});\n```\n\n### GWT Rules\n\n| Rule | Description |\n|------|-------------|\n| Comments Required | Use `// GIVEN:`, `// WHEN:`, `// THEN:` (uppercase with colon) |\n| GIVEN | All test data setup MUST happen before WHEN |\n| WHEN | Contains ONLY ONE primary action |\n| THEN | Verify ALL relevant outcomes |\n| One Test = One Behavior | Split multiple scenarios into separate tests |\n\n---\n\n## Test Case Templates\n\n### REST API Success\n\n```typescript\nit('should create user and return 201', async () => {\n // GIVEN: Valid user data\n const createUserDto = {\n email: '[email protected]',\n name: 'Test User',\n password: 'SecurePass123!',\n };\n\n // WHEN: Creating a new user\n const response = await request(httpServer)\n .post('/users')\n .send(createUserDto)\n .expect(201);\n\n // THEN: User is created with correct data\n expect(response.body).toMatchObject({\n code: ErrorCode.SUCCESS,\n data: {\n email: '[email protected]',\n name: 'Test User',\n },\n });\n expect(response.body.data.id).toBeDefined();\n\n // THEN: User exists in database\n const savedUser = await userRepository.findOne({\n where: { email: '[email protected]' },\n });\n expect(savedUser).toBeDefined();\n expect(savedUser.name).toBe('Test User');\n});\n```\n\n### REST API Validation Error\n\n```typescript\nit('should return 400 for invalid email format', async () => {\n // GIVEN: User data with invalid email\n const invalidUserDto = {\n email: 'not-an-email',\n name: 'Test User',\n password: 'SecurePass123!',\n };\n\n // WHEN: Attempting to create user with invalid email\n const response = await request(httpServer)\n .post('/users')\n .send(invalidUserDto)\n .expect(400);\n\n // THEN: Validation error returned\n expect(response.body).toMatchObject({\n code: ErrorCode.VALIDATION_ERROR,\n message: expect.stringContaining('email'),\n });\n});\n```\n\n### REST API Not Found\n\n```typescript\nit('should return 404 when user not found', async () => {\n // GIVEN: No user exists with this ID\n const nonExistentId = 'user-does-not-exist';\n\n // WHEN: Fetching non-existent user\n const response = await request(httpServer)\n .get(`/users/${nonExistentId}`)\n .expect(404);\n\n // THEN: Not found error returned\n expect(response.body).toMatchObject({\n code: ErrorCode.NOT_FOUND,\n message: 'User not found',\n });\n});\n```\n\n### Database Query with Pagination\n\n```typescript\nit('should return paginated list of active users', async () => {\n // GIVEN: Multiple users in database\n await userRepository.save([\n { email: '[email protected]', name: 'User 1', status: 'active' },\n { email: '[email protected]', name: 'User 2', status: 'active' },\n { email: '[email protected]', name: 'User 3', status: 'inactive' },\n ]);\n\n // WHEN: Fetching active users with pagination\n const response = await request(httpServer)\n .get('/users')\n .query({ status: 'active', page: 1, limit: 10 })\n .expect(200);\n\n // THEN: Only active users returned with pagination metadata\n expect(response.body.data).toHaveLength(2);\n expect(response.body.data.every(u => u.status === 'active')).toBe(true);\n expect(response.body.meta).toMatchObject({\n page: 1,\n limit: 10,\n total: 2,\n });\n});\n```\n\n### Kafka Event Processing\n\n```typescript\nit('should process order created event and update inventory', async () => {\n // GIVEN: Product with available stock\n const product = await productRepository.save({\n id: 'prod-123',\n name: 'Test Product',\n stock: 100,\n });\n\n // GIVEN: Order created event\n const orderEvent = {\n orderId: 'order-456',\n productId: 'prod-123',\n quantity: 5,\n };\n\n // WHEN: Publishing order created event\n await kafkaHelper.publishEvent('order.created', orderEvent);\n\n // THEN: Wait for async processing (using smart polling)\n const updatedProduct = await waitFor(\n () => productRepository.findOne({ where: { id: 'prod-123', stock: 95 } }),\n 20000\n );\n\n // THEN: Inventory is updated\n expect(updatedProduct).toBeDefined();\n expect(updatedProduct.stock).toBe(95);\n});\n```\n\n### Kafka Event with Output Topic\n\n```typescript\nit('should process message and produce to output topic', async () => {\n // GIVEN: Input event\n const inputEvent = { id: 'test-1', data: 'test-data' };\n\n // WHEN: Publishing to input topic\n await kafkaHelper.publishEvent(inputTopic, inputEvent, inputEvent.id);\n\n // THEN: Output message received\n const outputMessages = await kafkaHelper.waitForMessages(outputTopic, 1, 20000);\n\n expect(outputMessages.length).toBe(1);\n expect(outputMessages[0].value).toMatchObject({\n id: 'test-1',\n processed: true,\n });\n});\n```\n\n### External API Mock\n\n```typescript\nit('should handle payment gateway timeout gracefully', async () => {\n // GIVEN: Payment request data\n const paymentDto = {\n orderId: 'order-123',\n amount: 99.99,\n currency: 'USD',\n };\n\n // GIVEN: Payment gateway configured to timeout\n mockPaymentGateway.onPost('/charge').timeout();\n\n // WHEN: Processing payment\n const response = await request(httpServer)\n .post('/payments')\n .send(paymentDto)\n .expect(503);\n\n // THEN: Graceful error response\n expect(response.body).toMatchObject({\n code: ErrorCode.PAYMENT_SERVICE_UNAVAILABLE,\n message: 'Payment service is temporarily unavailable',\n });\n\n // THEN: Order status updated to payment_failed\n const order = await orderRepository.findOne({\n where: { id: 'order-123' },\n });\n expect(order.paymentStatus).toBe('failed');\n});\n```\n\n### Authentication Flow\n\n```typescript\nit('should return 401 for expired JWT token', async () => {\n // GIVEN: User with expired token\n const expiredToken = generateJwt({ userId: 'user-123' }, { expiresIn: '-1h' });\n\n // WHEN: Accessing protected endpoint with expired token\n const response = await request(httpServer)\n .get('/users/me')\n .set('Authorization', `Bearer ${expiredToken}`)\n .expect(401);\n\n // THEN: Unauthorized error returned\n expect(response.body).toMatchObject({\n code: ErrorCode.TOKEN_EXPIRED,\n message: 'Token has expired',\n });\n});\n```\n\n---\n\n## Test Naming Conventions\n\n### Format\n\n```\nshould [action/outcome] when [condition]\n```\n\n### Examples by Category\n\n**Success Scenarios:**\n```typescript\n'should create user and return 201'\n'should return user by id'\n'should update user profile'\n'should delete user and cascade orders'\n```\n\n**Error Scenarios:**\n```typescript\n'should return 400 for invalid email format'\n'should return 404 when user not found'\n'should return 401 for expired token'\n'should return 409 for duplicate email'\n```\n\n**Business Logic:**\n```typescript\n'should calculate discount for premium users'\n'should apply tax based on shipping address'\n'should send notification after order completion'\n```\n\n**Edge Cases:**\n```typescript\n'should handle empty result set gracefully'\n'should process batch of 1000 items within timeout'\n'should recover from transient database error'\n```\n\n---\n\n## Anti-Patterns to Avoid\n\n### Multiple WHEN Actions\n\n```typescript\n// ❌ WRONG: Multiple actions in one test\nit('should create and update user', async () => {\n // WHEN: Creating user\n const createResponse = await request(httpServer).post('/users').send(userData);\n\n // WHEN: Updating user (WRONG - second action)\n const updateResponse = await request(httpServer)\n .patch(`/users/${createResponse.body.data.id}`)\n .send({ name: 'Updated' });\n});\n\n// ✅ CORRECT: Split into separate tests\nit('should create user', async () => { /* ... */ });\nit('should update existing user', async () => { /* ... */ });\n```\n\n### Conditional Assertions\n\n```typescript\n// ❌ WRONG: Conditional logic in assertions\nif (response.status === 200) {\n expect(response.body.data).toBeDefined();\n} else {\n expect(response.body.error).toBeDefined();\n}\n\n// ✅ CORRECT: Separate deterministic tests\nit('should return users on success', async () => { /* ... */ });\nit('should return error on failure', async () => { /* ... */ });\n```\n\n### Missing Specific Assertions\n\n```typescript\n// ❌ WRONG: Only checking existence\nexpect(response.body.data).toBeDefined();\n\n// ✅ CORRECT: Assert specific values\nexpect(response.body).toMatchObject({\n code: ErrorCode.SUCCESS,\n data: {\n id: expect.any(String),\n email: '[email protected]',\n },\n});\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":8675,"content_sha256":"b4c97c9a3b74e39bd0d7da235e8fe74692b227b233d135bda7bd630f31e99327"},{"filename":"references/kafka/docker-setup.md","content":"# Kafka Docker Setup for E2E Testing\n\n## Table of Contents\n- [Redpanda (Recommended)](#redpanda-recommended)\n- [Apache Kafka KRaft](#apache-kafka-kraft)\n- [Comparison](#comparison)\n- [Health Checks](#health-checks)\n- [Testcontainers Alternative](#testcontainers-alternative)\n\n---\n\n## Redpanda (Recommended)\n\nRedpanda is a Kafka-compatible streaming platform that starts 2x faster and requires no ZooKeeper.\n\n```yaml\n# docker-compose.e2e.yml\nversion: '3.8'\nservices:\n redpanda:\n image: redpandadata/redpanda:latest\n hostname: redpanda\n ports:\n - 9092:9092 # Internal broker port\n - 9094:9094 # External broker port (for test clients)\n - 18082:18082 # Pandaproxy (REST API)\n command:\n - redpanda start\n - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:9094\n - --advertise-kafka-addr internal://redpanda:9092,external://localhost:9094\n - --mode dev-container\n - --smp 1\n - --memory 512M\n - --overprovisioned\n healthcheck:\n test: [\"CMD-SHELL\", \"rpk cluster health | grep -q 'Healthy' || exit 1\"]\n interval: 5s\n timeout: 10s\n retries: 10\n start_period: 10s\n\n redpanda-console:\n image: redpandadata/console:latest\n ports:\n - 8082:8080\n environment:\n KAFKA_BROKERS: redpanda:9092\n depends_on:\n - redpanda\n\nvolumes:\n redpanda-data:\n driver: local\n```\n\n### Why Redpanda?\n- **Faster startup**: ~5.1s vs ~8.5s for Kafka\n- **No ZooKeeper**: Single binary, simpler setup\n- **Lower memory**: C++ implementation vs JVM\n- **Kafka-compatible**: Drop-in replacement API\n- **Built-in UI**: Console included\n\n---\n\n## Apache Kafka KRaft\n\nUse when you need exact Kafka compatibility or specific Kafka features.\n\n```yaml\n# docker-compose.e2e.yml\nversion: '3.8'\n\nservices:\n kafka:\n image: confluentinc/cp-kafka:7.5.0\n hostname: kafka\n container_name: kafka-e2e\n ports:\n - \"9092:9092\"\n - \"9094:9094\"\n environment:\n KAFKA_NODE_ID: 1\n KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'\n KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9094'\n KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1\n KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0\n KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1\n KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1\n KAFKA_PROCESS_ROLES: 'broker,controller'\n KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka:29093'\n KAFKA_LISTENERS: 'PLAINTEXT://kafka:29092,CONTROLLER://kafka:29093,PLAINTEXT_HOST://0.0.0.0:9094'\n KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'\n KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'\n KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'\n CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk'\n # Performance tuning for tests\n KAFKA_LOG_RETENTION_MS: 300000 # 5 minutes\n KAFKA_LOG_SEGMENT_BYTES: 1048576 # 1MB segments\n healthcheck:\n test: [\"CMD-SHELL\", \"kafka-broker-api-versions --bootstrap-server localhost:9092 || exit 1\"]\n interval: 5s\n timeout: 10s\n retries: 10\n start_period: 30s\n\nvolumes:\n kafka-e2e-data:\n driver: local\n```\n\n### Apache Kafka Native Image (Faster)\n\nFor 40% faster startup, use the native image:\n\n```yaml\nservices:\n kafka:\n image: apache/kafka-native:latest\n # ... same config as above\n```\n\n---\n\n## Comparison\n\n| Metric | Apache Kafka | Kafka Native | Redpanda |\n|--------|--------------|--------------|----------|\n| Container startup | ~8.5s | ~5.1s | ~5.1s |\n| Memory usage | High (JVM) | Medium | Low (C++) |\n| ZooKeeper required | No (KRaft) | No (KRaft) | No |\n| Kafka compatibility | 100% | 100% | ~99% |\n| Best for | Production parity | Faster CI | Fast dev/test |\n\n---\n\n## Health Checks\n\n### Redpanda Health Check Script\n\n```bash\n#!/bin/bash\n# wait-for-redpanda.sh\nTIMEOUT=${1:-30}\nHOST=${2:-localhost}\nPORT=${3:-9094}\n\necho \"Waiting for Redpanda at $HOST:$PORT...\"\nfor i in $(seq 1 $TIMEOUT); do\n if docker-compose -f docker-compose.e2e.yml exec redpanda \\\n rpk cluster health 2>/dev/null | grep -q \"Healthy\"; then\n echo \"Redpanda is ready!\"\n exit 0\n fi\n sleep 1\ndone\necho \"Timeout waiting for Redpanda\"\nexit 1\n```\n\n### Kafka Health Check Script\n\n```bash\n#!/bin/bash\n# wait-for-kafka.sh\nTIMEOUT=${1:-60}\nHOST=${2:-localhost}\nPORT=${3:-9092}\n\necho \"Waiting for Kafka at $HOST:$PORT...\"\nfor i in $(seq 1 $TIMEOUT); do\n if docker-compose -f docker-compose.e2e.yml exec kafka \\\n kafka-broker-api-versions --bootstrap-server localhost:9092 2>/dev/null; then\n echo \"Kafka is ready!\"\n exit 0\n fi\n sleep 1\ndone\necho \"Timeout waiting for Kafka\"\nexit 1\n```\n\n---\n\n## Testcontainers Alternative\n\nFor programmatic container management in tests:\n\n```typescript\nimport { KafkaContainer, StartedKafkaContainer } from '@testcontainers/kafka';\n\nlet kafkaContainer: StartedKafkaContainer;\n\nbeforeAll(async () => {\n kafkaContainer = await new KafkaContainer('redpandadata/redpanda:latest')\n .withExposedPorts(9092)\n .start();\n\n const bootstrapServers = kafkaContainer.getBootstrapServers();\n // Use bootstrapServers for KafkaTestHelper\n}, 60000);\n\nafterAll(async () => {\n await kafkaContainer?.stop();\n});\n```\n\n### Container Reuse (Development Only)\n\n```typescript\n// For faster local development - reuse containers between runs\nconst kafka = new KafkaContainer('redpandadata/redpanda:latest')\n .withReuse();\n\n// Requires ~/.testcontainers.properties:\n// testcontainers.reuse.enable=true\n```\n\n**Warning:** Don't use container reuse in CI - it can cause test pollution.\n\n### Parallel Container Startup\n\n```typescript\n// Start multiple containers in parallel for faster setup\nbeforeAll(async () => {\n const [kafkaContainer, mongoContainer, redisContainer] = await Promise.all([\n new KafkaContainer('redpandadata/redpanda:latest').start(),\n new MongoDBContainer('mongo:6').start(),\n new GenericContainer('redis:7').withExposedPorts(6379).start(),\n ]);\n\n // Sequential: 15-20s, Parallel: 6-8s\n}, 60000);\n```\n\n---\n\n## Broker Configuration for Tests\n\n### Recommended Settings\n\n```yaml\nenvironment:\n # Reduce rebalance delay for faster tests\n KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0\n\n # Faster log cleanup\n KAFKA_LOG_RETENTION_MS: 300000 # 5 minutes\n KAFKA_LOG_SEGMENT_BYTES: 1048576 # 1MB\n\n # Single replica for speed (test only!)\n KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1\n KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1\n```\n\n### Listener Configuration\n\nFor tests connecting from the host machine:\n\n```yaml\n# Internal (container-to-container): port 9092\n# External (host-to-container): port 9094\nKAFKA_LISTENERS: 'PLAINTEXT://kafka:29092,CONTROLLER://kafka:29093,PLAINTEXT_HOST://0.0.0.0:9094'\nKAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9094'\n```\n\nIn your tests, use `localhost:9094` as the broker address.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":6868,"content_sha256":"9a15f50b49c783a77eeb368fbff18ab9451124cd6669e84c381f39366cf5ee58"},{"filename":"references/kafka/examples.md","content":"# Kafka E2E Test Examples\n\n## Complete Test Suite Example\n\n```typescript\nimport { Test, TestingModule } from '@nestjs/testing';\nimport { INestApplication } from '@nestjs/common';\nimport { Transport, MicroserviceOptions } from '@nestjs/microservices';\nimport { KafkaTestHelper } from '../helpers/kafka-helper';\n\ndescribe('Kafka E2E Tests', () => {\n let app: INestApplication;\n let kafkaHelper: KafkaTestHelper;\n const inputTopic = 'test-input-topic';\n const outputTopic = 'test-output-topic';\n\n jest.setTimeout(30000);\n\n beforeAll(async () => {\n // Step 1: Connect test helper BEFORE app starts\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n\n // Step 2: Pre-subscribe to output topic (NOT fromBeginning)\n await kafkaHelper.subscribeToTopic(outputTopic, false);\n\n // Step 3: Create and start NestJS app\n const moduleFixture = await Test.createTestingModule({\n imports: [AppModule],\n }).compile();\n\n app = moduleFixture.createNestApplication();\n app.connectMicroservice\u003cMicroserviceOptions>({\n transport: Transport.KAFKA,\n options: {\n client: { clientId: 'test-app', brokers: ['localhost:9094'] },\n consumer: { groupId: `test-app-group-${Date.now()}` },\n subscribe: { fromBeginning: false },\n },\n }, { inheritAppConfig: true });\n\n await app.startAllMicroservices();\n await app.init();\n\n // Step 4: Wait for consumer to fully subscribe\n await new Promise(resolve => setTimeout(resolve, 5000));\n }, 90000);\n\n afterAll(async () => {\n await kafkaHelper?.disconnect();\n await app?.close();\n }, 30000);\n\n // CRITICAL: Clear buffer between tests\n beforeEach(async () => {\n kafkaHelper.clearMessages(outputTopic);\n await new Promise(resolve => setTimeout(resolve, 2000));\n });\n\n // Tests go here...\n});\n```\n\n---\n\n## Basic Message Processing\n\n```typescript\nit('should process message and produce output', async () => {\n // GIVEN: Input event\n const testEvent = {\n id: `event-${Date.now()}`,\n type: 'USER_CREATED',\n payload: { userId: 'user-123', email: '[email protected]' },\n };\n\n // WHEN: Publishing to input topic\n await kafkaHelper.publishEvent(inputTopic, testEvent, testEvent.id);\n\n // THEN: Wait for output message (smart polling)\n const outputMessages = await kafkaHelper.waitForMessages(outputTopic, 1, 20000);\n\n expect(outputMessages.length).toBe(1);\n expect(outputMessages[0].value).toMatchObject({\n eventId: testEvent.id,\n processed: true,\n timestamp: expect.any(String),\n });\n});\n```\n\n---\n\n## Event Updates Database\n\n```typescript\nit('should process order event and update inventory', async () => {\n // GIVEN: Product with available stock\n const productId = `prod-${Date.now()}`;\n const product = await productRepository.save({\n id: productId,\n name: 'Test Product',\n stock: 100,\n });\n\n // GIVEN: Order created event\n const orderEvent = {\n orderId: `order-${Date.now()}`,\n productId: productId,\n quantity: 5,\n timestamp: new Date().toISOString(),\n };\n\n // WHEN: Publishing order event\n await kafkaHelper.publishEvent('orders.created', orderEvent, orderEvent.orderId);\n\n // THEN: Inventory is updated (smart polling on database)\n const updatedProduct = await waitFor(\n () => productRepository.findOne({\n where: { id: productId, stock: 95 },\n }),\n 20000\n );\n\n expect(updatedProduct).toBeDefined();\n expect(updatedProduct.stock).toBe(95);\n});\n```\n\n---\n\n## Wait for Specific Message\n\n```typescript\nit('should process user registration and send welcome email', async () => {\n // GIVEN: User registration event\n const userId = `user-${Date.now()}`;\n const registrationEvent = {\n eventId: `evt-${Date.now()}`,\n type: 'USER_REGISTERED',\n userId: userId,\n email: '[email protected]',\n };\n\n // WHEN: Publishing registration event\n await kafkaHelper.publishEvent('user-events', registrationEvent);\n\n // THEN: Wait for specific notification event\n const notificationEvent = await kafkaHelper.waitForMessage(\n 'email-notifications',\n (msg) => msg.value.userId === userId && msg.value.type === 'WELCOME_EMAIL',\n 15000\n );\n\n expect(notificationEvent.value).toMatchObject({\n userId: userId,\n type: 'WELCOME_EMAIL',\n recipient: '[email protected]',\n template: 'welcome',\n });\n});\n```\n\n---\n\n## Batch Processing\n\n```typescript\nit('should process batch of 10 events within timeout', async () => {\n // GIVEN: Batch of events\n const events = Array.from({ length: 10 }, (_, i) => ({\n event: {\n id: `batch-event-${Date.now()}-${i}`,\n data: `data-${i}`,\n sequence: i,\n },\n key: `batch-key-${i}`,\n }));\n\n // WHEN: Publishing batch\n await kafkaHelper.publishBatch(inputTopic, events);\n\n // THEN: All events processed\n const outputMessages = await kafkaHelper.waitForMessages(outputTopic, 10, 30000);\n\n expect(outputMessages.length).toBe(10);\n expect(outputMessages.every(m => m.value.processed === true)).toBe(true);\n\n // THEN: Verify ordering preserved\n const sequences = outputMessages.map(m => m.value.originalSequence);\n expect(sequences).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);\n});\n```\n\n---\n\n## Error Handling\n\n### Malformed Message\n\n```typescript\nit('should send malformed message to DLQ', async () => {\n // GIVEN: Invalid message (missing required field)\n const invalidEvent = {\n id: `invalid-${Date.now()}`,\n // Missing required 'type' field\n payload: { data: 'test' },\n };\n\n // WHEN: Publishing invalid event\n await kafkaHelper.publishEvent(inputTopic, invalidEvent, invalidEvent.id);\n\n // THEN: Message sent to dead letter queue\n const dlqMessages = await kafkaHelper.waitForMessages(\n `${inputTopic}.DLQ`,\n 1,\n 20000\n );\n\n expect(dlqMessages[0].value).toMatchObject({\n originalMessage: invalidEvent,\n error: expect.stringContaining('type'),\n timestamp: expect.any(String),\n });\n});\n```\n\n### Processing Failure with Retry\n\n```typescript\nit('should retry failed processing and eventually succeed', async () => {\n // GIVEN: Event that fails twice then succeeds\n const eventId = `retry-test-${Date.now()}`;\n const testEvent = { id: eventId, data: 'test', failCount: 2 };\n\n // GIVEN: Mock external service to fail twice\n mockExternalService.process\n .mockRejectedValueOnce(new Error('Temporary failure'))\n .mockRejectedValueOnce(new Error('Temporary failure'))\n .mockResolvedValueOnce({ success: true });\n\n // WHEN: Publishing event\n await kafkaHelper.publishEvent(inputTopic, testEvent, eventId);\n\n // THEN: Eventually succeeds (after retries)\n const outputMessages = await kafkaHelper.waitForMessages(outputTopic, 1, 30000);\n\n expect(outputMessages[0].value).toMatchObject({\n eventId: eventId,\n status: 'success',\n retryCount: 2,\n });\n\n // THEN: External service called 3 times\n expect(mockExternalService.process).toHaveBeenCalledTimes(3);\n});\n```\n\n---\n\n## Domain Event Publishing\n\n```typescript\nit('should publish domain event when order created via API', async () => {\n // GIVEN: Valid order data\n const orderData = {\n customerId: 'cust-123',\n items: [{ productId: 'prod-456', quantity: 2 }],\n };\n\n // WHEN: Creating order via REST API\n const response = await request(httpServer)\n .post('/api/v1/orders')\n .send(orderData)\n .expect(201);\n\n const orderId = response.body.data.id;\n\n // THEN: Domain event published to Kafka\n const domainEvent = await kafkaHelper.waitForMessage(\n 'domain-events',\n (msg) => msg.value.aggregateId === orderId && msg.value.type === 'ORDER_CREATED',\n 15000\n );\n\n expect(domainEvent.value).toMatchObject({\n type: 'ORDER_CREATED',\n aggregateId: orderId,\n aggregateType: 'Order',\n payload: {\n customerId: 'cust-123',\n items: expect.arrayContaining([\n expect.objectContaining({ productId: 'prod-456', quantity: 2 }),\n ]),\n },\n timestamp: expect.any(String),\n version: 1,\n });\n});\n```\n\n---\n\n## Event Sourcing Pattern\n\n```typescript\nit('should rebuild aggregate state from events', async () => {\n // GIVEN: Series of events for an order\n const orderId = `order-${Date.now()}`;\n const events = [\n { type: 'ORDER_CREATED', orderId, items: [], total: 0 },\n { type: 'ITEM_ADDED', orderId, productId: 'p1', price: 100 },\n { type: 'ITEM_ADDED', orderId, productId: 'p2', price: 50 },\n { type: 'DISCOUNT_APPLIED', orderId, discount: 15 },\n ];\n\n // WHEN: Publishing all events\n for (const event of events) {\n await kafkaHelper.publishEvent('order-events', event, orderId);\n }\n\n // THEN: Wait for projection to be updated\n await waitFor(\n () => orderProjectionRepository.findOne({ where: { id: orderId } }),\n 15000\n );\n\n // THEN: Aggregate state is correct\n const projection = await orderProjectionRepository.findOne({\n where: { id: orderId },\n });\n\n expect(projection).toMatchObject({\n id: orderId,\n itemCount: 2,\n subtotal: 150,\n discount: 15,\n total: 135,\n eventCount: 4,\n });\n});\n```\n\n---\n\n## Performance Testing\n\n```typescript\nit('should process 100 messages within 60 seconds', async () => {\n // GIVEN: 100 events\n const events = Array.from({ length: 100 }, (_, i) => ({\n event: { id: `perf-${Date.now()}-${i}`, data: `data-${i}` },\n key: `key-${i % 10}`, // Distribute across 10 partitions\n }));\n\n const startTime = Date.now();\n\n // WHEN: Publishing batch\n await kafkaHelper.publishBatch(inputTopic, events);\n\n // THEN: All messages processed\n const outputMessages = await kafkaHelper.waitForMessages(outputTopic, 100, 60000);\n\n const duration = Date.now() - startTime;\n\n // THEN: Performance requirements met\n expect(outputMessages.length).toBe(100);\n expect(duration).toBeLessThan(60000);\n\n // THEN: Calculate throughput\n const throughput = (100 / duration) * 1000;\n console.log(`Throughput: ${throughput.toFixed(2)} messages/second`);\n expect(throughput).toBeGreaterThan(1); // At least 1 msg/s\n});\n```\n\n---\n\n## Multi-Consumer Scenario\n\n```typescript\nit('should handle messages from multiple producers', async () => {\n // GIVEN: Two different event types\n const userEvent = {\n id: `user-evt-${Date.now()}`,\n type: 'USER_UPDATED',\n userId: 'user-1',\n };\n\n const orderEvent = {\n id: `order-evt-${Date.now()}`,\n type: 'ORDER_COMPLETED',\n orderId: 'order-1',\n };\n\n // WHEN: Publishing from different \"producers\"\n await Promise.all([\n kafkaHelper.publishEvent('user-events', userEvent),\n kafkaHelper.publishEvent('order-events', orderEvent),\n ]);\n\n // THEN: Both events processed and aggregated\n const aggregatedMessages = await kafkaHelper.waitForMessages(\n 'analytics-events',\n 2,\n 20000\n );\n\n expect(aggregatedMessages).toHaveLength(2);\n expect(aggregatedMessages.map(m => m.value.sourceType).sort()).toEqual([\n 'ORDER_COMPLETED',\n 'USER_UPDATED',\n ]);\n});\n```\n\n---\n\n## Helper Function: waitFor\n\n```typescript\n// Utility for polling database state\nasync function waitFor\u003cT>(\n fn: () => Promise\u003cT | null>,\n timeout = 10000,\n interval = 200\n): Promise\u003cT> {\n const startTime = Date.now();\n while (Date.now() - startTime \u003c timeout) {\n const result = await fn();\n if (result) return result;\n await new Promise(r => setTimeout(r, interval));\n }\n throw new Error('Timeout waiting for condition');\n}\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":11290,"content_sha256":"81c8af96e1a394c3299288226c890fffe4ec75c6140af6513ba357ed48722439"},{"filename":"references/kafka/isolation.md","content":"# Kafka Test Isolation\n\n## Table of Contents\n- [Why Common Approaches Fail](#why-common-approaches-fail)\n- [The Pre-Subscription Pattern](#the-pre-subscription-pattern)\n- [Test Isolation Strategies](#test-isolation-strategies)\n- [Common Issues and Solutions](#common-issues-and-solutions)\n\n---\n\n## Why Common Approaches Fail\n\n| Common Approach | Why It Fails | Better Solution |\n|-----------------|--------------|-----------------|\n| `fromBeginning: true` | Reads ALL historical messages including from previous runs | Pre-subscribe with `fromBeginning: false`, clear buffer between tests |\n| Unique consumer groups per test | Complex setup, offset cleanup needed | Single test consumer, message buffer clearing |\n| Delete/recreate topics | Slow (1-3s per topic), admin permissions needed | Keep topics, clear in-memory buffers |\n| Aggressive retries + short timeouts | Flaky in CI, race conditions | Fixed long waits (deterministic) |\n\n### Key Insight\n\nThe commonly suggested `auto.offset.reset: 'earliest'` or `fromBeginning: true` does NOT provide test isolation because:\n\n1. **Reads ALL historical messages** - Including from previous test runs, causing false positives/negatives\n2. **Messages accumulate** - Multiple tests share the same topic\n3. **Offset tracking issues** - Consumer group offset tracking causes inconsistent behavior\n\n---\n\n## The Pre-Subscription Pattern\n\n### Core Principle\n\nSubscribe to topics BEFORE the application starts, accumulate messages in memory buffers, and clear buffers between tests.\n\n### Implementation Flow\n\n```\n1. beforeAll:\n ├── Connect KafkaTestHelper\n ├── Subscribe to output topics (fromBeginning: false)\n ├── Start NestJS app with Kafka microservice\n └── Wait 5s for consumer subscription\n\n2. beforeEach:\n ├── Clear message buffers\n └── Wait 2s grace period\n\n3. Test:\n ├── Publish to input topic\n ├── Poll for output messages (smart polling)\n └── Assert results\n\n4. afterAll:\n ├── Disconnect KafkaTestHelper\n └── Close app\n```\n\n### Why This Works\n\n| Approach | Problem | Pre-Subscription Solution |\n|----------|---------|---------------------------|\n| Ad-hoc subscription | Race condition - messages sent before consumer ready | Subscribe before app starts |\n| `fromBeginning: true` | Reads old messages from previous runs | Use `fromBeginning: false`, clear buffer |\n| Topic deletion | Slow, requires admin permissions | Keep topics, clear in-memory buffers only |\n| Callback-based assertions | Complex async handling | Polling-based with timeout |\n\n---\n\n## Test Isolation Strategies\n\n### Primary: Message Buffer Clearing\n\n```typescript\n// In beforeEach - the main isolation mechanism\nbeforeEach(async () => {\n kafkaHelper.clearMessages(outputTopic);\n await new Promise(r => setTimeout(r, 2000)); // Grace period\n});\n```\n\n### Secondary: Unique Consumer Groups (for parallel suites)\n\n```typescript\n// Only needed if running test suites in parallel\nconst uniqueGroupId = `test-${Date.now()}-${Math.random().toString(36).slice(2)}`;\n```\n\n### Avoid: Topic Deletion/Recreation\n\n```typescript\n// DON'T DO THIS - slow and error-prone\nawait admin.deleteTopics({ topics: [testTopic] }); // 1-3s\nawait admin.createTopics({ topics: [{ topic: testTopic }] }); // 1-2s\nawait new Promise(r => setTimeout(r, 3000)); // Wait for propagation\n```\n\n---\n\n## Test File Ordering (Cross-File Isolation)\n\n### Problem: librdkafka State Corruption\n\nWhen running multiple E2E test files in sequence, tests that connect to **invalid brokers** (for error handling validation) can corrupt librdkafka's internal state, causing subsequent tests to fail.\n\n**Key Insight:** This corruption persists within the Jest process and cannot be cleaned up via broker health checks alone.\n\n### Symptoms\n\n- Tests pass when run individually\n- Tests pass when run as a single file\n- Tests fail when run as part of the full E2E suite\n- Failures appear as timeouts or connection issues in tests that worked before\n\n### Root Cause\n\nTest files that contain tests connecting to invalid brokers (e.g., `localhost:19999`, `invalid-broker:9999`) leave librdkafka in a corrupted state:\n\n```typescript\n// These patterns corrupt librdkafka state for subsequent tests\n'bootstrap.servers': 'localhost:19999', // Non-existent port\n'bootstrap.servers': 'invalid-broker:9999', // Non-existent host\n```\n\n### Solution: Custom Jest Test Sequencer\n\nCreate a custom test sequencer to control test file execution order, ensuring tests with invalid broker connections run **LAST**.\n\n```javascript\n// test/e2e/test-sequencer.js\nimport Sequencer from '@jest/test-sequencer';\n\nconst TEST_ORDER = [\n // Group 1: Basic client tests (clean broker operations)\n 'confluent-kafka-client.e2e-spec.ts',\n 'confluent-kafka-client-status.e2e-spec.ts',\n\n // Group 2: Utility tests (standard producer/consumer operations)\n 'promisified-producer.e2e-spec.ts',\n 'promisified-consumer.e2e-spec.ts',\n\n // Group 3: Complex integration tests (sensitive to broker state)\n 'request-response.e2e-spec.ts',\n\n // Group 4: Tests that manipulate infrastructure\n 'self-healing.e2e-spec.ts',\n\n // Group 5: Tests with invalid broker connections (MUST RUN LAST)\n // These tests connect to non-existent brokers and may corrupt librdkafka state\n 'promisified-producer-connection.e2e-spec.ts',\n 'confluent-kafka-client-errors.e2e-spec.ts',\n];\n\nexport default class CustomSequencer extends Sequencer.default {\n sort(tests) {\n const orderMap = new Map(TEST_ORDER.map((name, index) => [name, index]));\n return [...tests].sort((a, b) => {\n const aName = a.path.split('/').pop();\n const bName = b.path.split('/').pop();\n const aOrder = orderMap.get(aName) ?? 999;\n const bOrder = orderMap.get(bName) ?? 999;\n return aOrder - bOrder;\n });\n }\n}\n```\n\n### Jest Configuration\n\n```javascript\n// test/jest-e2e.config.mjs\nconst config = {\n // ... other config\n testSequencer: '\u003crootDir>/e2e/test-sequencer.js',\n};\n```\n\n### Test File Grouping Strategy\n\n| Group | Test Type | Example Files | Order |\n|-------|-----------|---------------|-------|\n| 1 | Basic operations | `client.e2e-spec.ts` | First |\n| 2 | Standard producer/consumer | `promisified-*.e2e-spec.ts` | Early |\n| 3 | Complex integration | `request-response.e2e-spec.ts` | Middle |\n| 4 | Infrastructure manipulation | `self-healing.e2e-spec.ts` | Late |\n| 5 | Invalid broker connections | `*-connection.e2e-spec.ts`, `*-errors.e2e-spec.ts` | **LAST** |\n\n### Checklist for Test File Ordering\n\n- [ ] Tests with invalid broker connections run LAST\n- [ ] Tests that manipulate Docker/infrastructure run late\n- [ ] Complex integration tests run after basic tests stabilize\n- [ ] Custom test sequencer configured in `jest-e2e.config.mjs`\n- [ ] All test files listed in sequencer (unlisted files run last with default order)\n\n---\n\n## Common Issues and Solutions\n\n### Issue: `fromBeginning: true` reads old messages\n\n```typescript\n// WRONG: Reads ALL historical messages\nawait consumer.subscribe({ topic, fromBeginning: true });\n\n// CORRECT: Pre-subscribe with fromBeginning: false, clear buffer\nawait kafkaHelper.subscribeToTopic(outputTopic, false);\n// In beforeEach:\nkafkaHelper.clearMessages(outputTopic);\n```\n\n### Issue: Consumer not ready when test starts\n\n**Known Data Race:** There's a known issue where the consumer may not be fully up and running after calling `.run()`. This often only triggers in CI.\n\n```typescript\n// Solution: Wait after starting microservices\nawait app.startAllMicroservices();\nawait app.init();\nawait new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5s\n```\n\n### Issue: Tests interfere with each other\n\n```typescript\n// Solution: Clear message buffer between tests (NOT topics)\nbeforeEach(async () => {\n kafkaHelper.clearMessages(outputTopic);\n await new Promise(r => setTimeout(r, 2000)); // Grace period\n});\n```\n\n### Issue: Slow consumer rebalancing\n\n```typescript\n// Solution 1: Optimize consumer config for fast rebalancing\nconst consumer = kafka.consumer({\n groupId: 'test-consumer',\n sessionTimeout: 6000, // Reduce from 45s default\n heartbeatInterval: 2000, // Must be \u003c sessionTimeout/3\n rebalanceTimeout: 10000, // Reduce from 60s default\n});\n\n// Solution 2: Use Admin API to verify rebalance completion\n// This is MORE RELIABLE than blind waits\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\n```\n\n### Issue: Consumer not ready after dynamic subscription\n\nWhen subscribing to topics after consumer is already running, rebalance occurs and messages may be lost.\n\n```typescript\n// ❌ WRONG: Assume subscription is immediate\nawait client.subscribeToResponseOf('new-topic');\nawait publish('new-topic', event); // Message may be lost during rebalance!\n\n// ✅ CORRECT: Use Admin API to wait for rebalance completion\nawait client.subscribeToResponseOf('new-topic');\nconst groupId = client.getResponseConsumerGroupId();\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\nawait publish('new-topic', event); // Safe - consumer is stable\n```\n\n### Issue: Intermittent test failures (flaky tests)\n\nOften caused by race conditions where tests assume consumer is ready based on time, not state.\n\n```typescript\n// ❌ WRONG: Blind wait may not be long enough\nawait app.startAllMicroservices();\nawait new Promise(r => setTimeout(r, 5000));\n\n// ✅ CORRECT: State-based synchronization\nawait app.startAllMicroservices();\nconst state = await kafkaHelper.getConsumerGroupState(groupId);\nconsole.log(`Consumer group state: ${state}`); // Debug visibility\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\n```\n\n### Issue: Consumer doesn't receive messages in CI\n\nCommon causes and solutions:\n\n1. **Consumer not ready** - Add 5s wait after `startAllMicroservices()`\n2. **Wrong broker address** - Use `localhost:9094` for external access from host\n3. **Topic doesn't exist** - Use `ensureTopicExists()` before test\n4. **Messages sent before consumer ready** - Use pre-subscription pattern\n\n### Issue: Delivery reports not received\n\n```typescript\n// KafkaJS handles this automatically\n// For node-rdkafka, flush after sending:\nawait producer.send({ topic, messages: [...] });\n// producer.flush(10000);\n```\n\n---\n\n## Test Isolation Summary\n\n| Strategy | How It Works | When to Use |\n|----------|--------------|-------------|\n| **Admin API Synchronization** | Poll consumer group state until `Stable` | Always - replaces blind waits |\n| **Message Buffer Clearing** | Clear accumulated messages in `beforeEach()` | Always - primary mechanism |\n| **Pre-subscription Model** | Subscribe before app starts | Always - prevents race conditions |\n| **Unique Consumer Groups** | Per-suite unique group ID | Only for parallel suites |\n| **Environment-based Topics** | `env-${env}-topic-name` pattern | Multi-environment testing |\n| **Kafka Container Restart** | Restart Docker container in `beforeAll` | Request-response patterns, complex integration tests |\n\n---\n\n## Kafka State Accumulation Problem\n\n### The Hidden Issue\n\nEven with consumer group cleanup, message buffer clearing, and proper test isolation, **Kafka state accumulates** across test runs within the same Jest process. This causes intermittent failures, especially in:\n\n- Request-response pattern tests\n- Tests with high message throughput (10+ concurrent requests)\n- Tests run multiple times consecutively\n\n### Root Causes\n\n| Layer | State Type | Cleanup Method | Effectiveness |\n|-------|------------|----------------|---------------|\n| **Broker** | Consumer groups, offsets, topics | `deleteConsumerGroup()`, topic deletion | Partial |\n| **librdkafka** | Connection pools, metadata cache | None available | ❌ Cannot clean |\n| **Node.js process** | Client instances, internal state | Close clients | Partial |\n\n**Key Insight:** librdkafka (the underlying C library) maintains internal state that persists within the Node.js process. This cannot be cleaned up through the Kafka Admin API.\n\n### Symptoms\n\n| Observation | Likely Cause |\n|-------------|--------------|\n| Tests pass on run 1-2, fail on run 3+ | State accumulation in librdkafka |\n| First test in file passes, rest timeout | Consumer not receiving responses |\n| Works after `docker restart kafka-e2e` | Broker state needed reset |\n| Works in CI (fresh process each time) | Process isolation prevents accumulation |\n\n---\n\n## Solution: Kafka Container Restart in beforeAll\n\nFor complex tests that suffer from state accumulation, restart the Kafka container at the start of the test file.\n\n### Implementation\n\n```typescript\nimport { exec } from 'child_process';\nimport { promisify } from 'util';\n\nconst execAsync = promisify(exec);\n\n/**\n * Restart Kafka container to ensure clean state.\n * This eliminates all accumulated state (consumer groups, offsets, etc.)\n * that can cause test failures on subsequent runs.\n */\nasync function restartKafkaContainer(): Promise\u003cvoid> {\n console.log('[Setup] Restarting Kafka container for clean state...');\n await execAsync('docker restart kafka-e2e');\n // 20 seconds ensures broker, controller, and metadata are fully initialized\n await new Promise((resolve) => setTimeout(resolve, 20000));\n console.log('[Setup] Kafka container restarted');\n}\n\ndescribe('Request-Response E2E Tests', () => {\n beforeAll(async () => {\n // Step 0: Restart Kafka for clean state\n await restartKafkaContainer();\n\n // Step 1: Initialize helpers and verify broker health\n kafkaHelper = new KafkaTestHelper();\n kafkaHelper.initializeAdmin(BROKER);\n await kafkaHelper.waitForBrokerReady(30000);\n\n // Step 2: Create topics, start app, etc.\n // ...\n }, 120000); // Extended timeout for restart + setup\n});\n```\n\n### Results\n\n| Run | Without Restart | With Restart |\n|-----|-----------------|--------------|\n| Run 1 | ✅ Pass | ✅ Pass |\n| Run 2 | ❌ ~20% pass | ✅ Pass |\n| Run 3+ | ❌ ~0% pass | ⚠️ May fail (librdkafka state) |\n\n### When to Use Kafka Restart\n\n| Scenario | Recommendation |\n|----------|----------------|\n| Request-response pattern tests | ✅ Use restart |\n| High-concurrency tests (10+ parallel) | ✅ Use restart |\n| Tests that manipulate consumer groups | ✅ Use restart |\n| Simple produce/consume tests | ❌ Not needed |\n| CI/CD pipelines (fresh process) | ❌ Not needed |\n| Development with repeated runs | ✅ Use restart |\n\n### Limitations\n\nThe Kafka restart approach provides:\n- **100% reliability** for CI/CD (fresh process each run)\n- **2 consecutive successful runs** in development\n- **Diminishing reliability** on 3rd+ run in same process\n\nFor complete reliability across unlimited consecutive runs, you would need:\n1. Process isolation (run each test file in separate Node.js process)\n2. Or accept the limitation and restart Kafka manually when needed\n\n---\n\n## Admin API Synchronization Pattern\n\n**CRITICAL: Prefer Admin API state verification over blind waits.**\n\nThe Admin API `describeGroups` method provides real-time consumer group state. This enables deterministic test synchronization.\n\n### Consumer Group States\n\n| State | Numeric Value | Meaning | Test Action |\n|-------|---------------|---------|-------------|\n| `Unknown` | 0 | State unknown | ⏳ Retry query |\n| `PreparingRebalance` | 1 | Members joining/leaving | ⏳ Poll and wait |\n| `CompletingRebalance` | 2 | Partitions being assigned | ⏳ Poll and wait |\n| `Stable` | 3 | All partitions assigned, ready | ✅ Proceed with test |\n| `Dead` | 4 | Group has been deleted | ❌ Error - reinitialize |\n| `Empty` | 5 | No active members | ⚠️ Check consumer started |\n\n**Note:** KafkaJS uses string states ('Stable'), while @confluentinc/kafka-javascript (node-rdkafka) uses numeric states.\n\n### Implementation Pattern\n\n```typescript\n/**\n * Wait for consumer group to reach Stable state.\n * Use this instead of fixed setTimeout() calls.\n */\nasync waitForConsumerGroupStable(\n groupId: string,\n timeoutMs: number = 15000,\n pollIntervalMs: number = 500\n): Promise\u003cvoid> {\n const startTime = Date.now();\n const STABLE_STATE = 3; // numeric for node-rdkafka, use 'Stable' for KafkaJS\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const state = await this.getConsumerGroupState(groupId);\n const stateNum = typeof state === 'string' ? parseInt(state, 10) : state;\n\n if (stateNum === STABLE_STATE) {\n await new Promise(r => setTimeout(r, 500)); // Small buffer\n return; // Ready!\n }\n\n // Log state for debugging\n const stateNames = {\n 0: 'Unknown', 1: 'PreparingRebalance', 2: 'CompletingRebalance',\n 3: 'Stable', 4: 'Dead', 5: 'Empty'\n };\n console.log(`Group '${groupId}' state: ${stateNames[stateNum] || state}`);\n\n await new Promise(r => setTimeout(r, pollIntervalMs));\n }\n\n throw new Error(`Timeout waiting for consumer group ${groupId} to be Stable`);\n}\n```\n\n### When to Use Admin API Synchronization\n\n1. **After `startAllMicroservices()`** - Before running any tests\n2. **After dynamic topic subscription** - Before publishing to new topics\n3. **After consumer scaling** - When adding/removing consumers\n4. **Debugging flaky tests** - Replace blind waits with state verification\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":17109,"content_sha256":"4b88e7cd78f1c78e542e1d1cd90917e1ac446f5b92d3fa4341ab57b0263cd3d0"},{"filename":"references/kafka/knowledge.md","content":"# Kafka E2E Testing Knowledge\n\n## Overview\n\nE2E testing with Kafka requires a fundamentally different approach than other infrastructure. Common patterns like `fromBeginning: true` fail to provide test isolation.\n\n## Why Kafka Testing is Different\n\n| Challenge | Why It Matters |\n|-----------|----------------|\n| Async processing | Messages processed with variable latency |\n| Consumer groups | Offsets persist across test runs |\n| Rebalancing | Consumers need time to join groups |\n| Message retention | Old messages interfere with new tests |\n\n## Common Approaches That FAIL\n\n| Approach | Why It Fails |\n|----------|--------------|\n| `fromBeginning: true` | Reads ALL historical messages including from previous runs |\n| Unique consumer groups per test | Complex setup, offset cleanup needed |\n| Delete/recreate topics | Slow (1-3s per topic), admin permissions needed |\n| Aggressive retries + short timeouts | Flaky in CI, race conditions |\n\n**Key Insight:** `auto.offset.reset: 'earliest'` or `fromBeginning: true` does NOT provide test isolation because:\n1. Reads ALL historical messages including from previous test runs\n2. Multiple tests share the same topic - messages accumulate\n3. Consumer group offset tracking causes inconsistent behavior\n\n## The Proven Solution: Pre-Subscription + Buffer Clearing\n\n```\n┌─────────────────────────────────────────────────────────────────────────┐\n│ E2E Test Architecture │\n├─────────────────────────────────────────────────────────────────────────┤\n│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │\n│ │ Test │ │ NestJS │ │ Output │ │\n│ │ Helper │ │ App │ │ Topic │ │\n│ │ (Producer) │ │ (Consumer) │ │ Listener │ │\n│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │\n│ │ │ │ │\n│ │ 1. Publish │ 2. Consume │ 3. Produce │\n│ ▼ ▼ ▼ │\n│ ┌─────────────────────────────────────────────────────────────────┐ │\n│ │ KAFKA BROKER │ │\n│ │ ┌─────────────┐ ┌─────────────┐ │ │\n│ │ │ Input Topic │ ───────────────────▶ │ Output Topic│ │ │\n│ │ └─────────────┘ └─────────────┘ │ │\n│ └─────────────────────────────────────────────────────────────────┘ │\n│ │ │\n│ │ 4. Collect in Buffer │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────┐ │\n│ │ MESSAGE BUFFER (In-Memory) │ │\n│ │ [msg1, msg2, ...] ◀── Clear between tests │ │\n│ └──────────────────────────────────────────────────────────────────┘ │\n│ │ │\n│ │ 5. Poll & Assert │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────┐ │\n│ │ TEST ASSERTIONS │ │\n│ └──────────────────────────────────────────────────────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────┘\n```\n\n### How It Works\n\n1. **Test helper subscribes BEFORE the application starts**\n2. **Messages accumulate in memory buffers** per topic\n3. **Tests poll for expected messages** with timeout\n4. **Buffers are cleared between tests** (NOT topics)\n\n## Kafka-Specific Timing\n\n| Phase | Value | Purpose |\n|-------|-------|---------|\n| Setup timeout | 90s | Full app + Kafka initialization |\n| Consumer ready | Admin API | Use `waitForConsumerGroupStable()` instead of fixed wait |\n| Between-test delay | 2s | Buffer clearing grace period |\n| Polling interval | 50ms | Message check frequency |\n| Processing wait | 10-20s max | Async message handling timeout |\n| Test timeout | 30-60s | Account for variability |\n\n## Admin API for Test Synchronization\n\n**CRITICAL: Use Admin API to determine when tests are ready to proceed, instead of blind waits.**\n\n### Why Admin API?\n\n| Blind Wait Problems | Admin API Benefits |\n|--------------------|--------------------|\n| May wait too long (slow tests) | Proceeds as soon as ready |\n| May not wait long enough (flaky) | State-based, deterministic |\n| Guessing game in CI | Consistent across environments |\n| No visibility into consumer state | Can log/debug actual state |\n\n### Key Admin API Methods for E2E Tests\n\n| Method | Purpose |\n|--------|---------|\n| `describeGroups([groupId])` | Get consumer group state (Stable, PreparingRebalance, etc.) |\n| `listGroups()` | List all consumer groups |\n| `deleteGroups([groupId])` | Reset consumer group for clean start |\n\n### Consumer Group States\n\n```\nPreparingRebalance (1) → CompletingRebalance (2) → Stable (3)\n ↑ │\n └──────────── (new subscription) ────────────┘\n```\n\n| State | Numeric | String (KafkaJS) | Meaning |\n|-------|---------|------------------|---------|\n| Unknown | 0 | 'Unknown' | State unknown |\n| PreparingRebalance | 1 | 'PreparingRebalance' | Rebalance in progress ⏳ |\n| CompletingRebalance | 2 | 'CompletingRebalance' | Assigning partitions ⏳ |\n| **Stable** | **3** | **'Stable'** | **Ready to consume ✅** |\n| Dead | 4 | 'Dead' | Group deleted ❌ |\n| Empty | 5 | 'Empty' | No consumers ⚠️ |\n\n**Note:** @confluentinc/kafka-javascript (node-rdkafka) uses numeric states, KafkaJS uses strings.\n\n### Usage Pattern\n\n```typescript\n// After starting microservices or subscribing to topics\nconst groupId = client.getConsumerGroupId();\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\n// NOW safe to publish test messages\n```\n\n## Broker Options for E2E Tests\n\n| Broker | Startup Time | Memory | Recommendation |\n|--------|--------------|--------|----------------|\n| Redpanda | ~5.1s | Low (C++) | ✅ Recommended for tests |\n| Kafka Native | ~5.1s | Medium | Good for CI |\n| Confluent Kafka | ~8.5s | High (JVM) | Production parity |\n\n## Test Category Organization\n\n```\ntest/\n├── e2e/\n│ ├── 01-happy-path.e2e-spec.ts # Basic message flow\n│ ├── 02-error-handling.e2e-spec.ts # Malformed messages\n│ ├── 03-performance.e2e-spec.ts # Throughput, latency\n│ └── 04-data-integrity.e2e-spec.ts # Zero data loss\n├── fixtures/events/\n│ ├── valid-events.ts\n│ └── malformed-events.ts\n└── helpers/\n ├── kafka-helper.ts # KafkaTestHelper\n └── assertion-helpers.ts\n```\n\n## Cross-File Test Isolation\n\n### The librdkafka State Corruption Problem\n\nWhen running multiple E2E test files in a single Jest session, tests that connect to **invalid brokers** can corrupt librdkafka's internal state. This corruption:\n\n1. **Persists within the Jest process** - Cannot be cleaned up by broker health checks\n2. **Affects subsequent test files** - Tests that passed individually now fail\n3. **Manifests as timeouts** - Connection-related failures in previously working tests\n\n### Symptoms\n\n| Observation | Likely Cause |\n|-------------|--------------|\n| Tests pass individually, fail in suite | Cross-file state pollution |\n| Timeouts after certain test files run | Invalid broker connection tests corrupting state |\n| First test file passes, later files fail | Order-dependent pollution |\n\n### Solution: Custom Jest Test Sequencer\n\nControl test file execution order to ensure tests with invalid broker connections run **LAST**:\n\n```javascript\n// test/e2e/test-sequencer.js\nimport Sequencer from '@jest/test-sequencer';\n\nconst TEST_ORDER = [\n // Clean broker operations first\n 'standard-client.e2e-spec.ts',\n // Complex integrations middle\n 'request-response.e2e-spec.ts',\n // Invalid broker tests LAST\n 'error-handling.e2e-spec.ts',\n];\n\nexport default class CustomSequencer extends Sequencer.default {\n sort(tests) {\n const orderMap = new Map(TEST_ORDER.map((name, index) => [name, index]));\n return [...tests].sort((a, b) => {\n const aOrder = orderMap.get(a.path.split('/').pop()) ?? 999;\n const bOrder = orderMap.get(b.path.split('/').pop()) ?? 999;\n return aOrder - bOrder;\n });\n }\n}\n```\n\nSee [isolation.md](isolation.md) for detailed implementation.\n\n## Broker Health Checks\n\n### When to Use Broker Health Checks\n\n| Scenario | Method | Purpose |\n|----------|--------|---------|\n| Before any test file starts | `waitForBrokerReady()` | Ensure broker is available |\n| After infrastructure manipulation | `ensureBrokerHealthy()` | Verify recovery |\n| After self-healing/Docker tests | `ensureBrokerHealthy()` | Confirm broker stability |\n\n### Key Methods\n\n```typescript\n// Quick health check (lightweight)\nconst healthy = await kafkaHelper.isBrokerHealthy(5000);\n\n// Wait for broker with polling\nawait kafkaHelper.waitForBrokerReady(30000);\n\n// Comprehensive health verification\nawait kafkaHelper.ensureBrokerHealthy(60000);\n```\n\nSee [test-helper.md](test-helper.md) for implementation details.\n\n## Kafka State Accumulation\n\n### The Problem\n\nEven with proper cleanup, **Kafka state accumulates** across test runs within the same Jest process:\n\n| Layer | State Type | Can Clean? |\n|-------|------------|------------|\n| Broker | Consumer groups, offsets | ✅ Yes (Admin API) |\n| librdkafka | Connection pools, metadata | ❌ No |\n| Node.js | Client instances | ✅ Partial |\n\n### Impact on Tests\n\n| Run | Pass Rate (without restart) | Pass Rate (with restart) |\n|-----|----------------------------|--------------------------|\n| 1 | 100% | 100% |\n| 2 | ~80% | 100% |\n| 3+ | ~0-20% | ~60-80% |\n\n### Solution: Kafka Container Restart\n\nFor complex tests (request-response, high-concurrency), restart the Kafka Docker container in `beforeAll`:\n\n```typescript\nasync function restartKafkaContainer(): Promise\u003cvoid> {\n await execAsync('docker restart kafka-e2e');\n await new Promise(r => setTimeout(r, 20000)); // Wait for full initialization\n}\n\nbeforeAll(async () => {\n await restartKafkaContainer();\n // ... rest of setup\n}, 120000);\n```\n\nSee [isolation.md](isolation.md) for detailed implementation and when to use this pattern.\n\n---\n\n## Key Files in This Section\n\n| File | Purpose |\n|------|---------|\n| [rules.md](rules.md) | Kafka-specific testing rules |\n| [test-helper.md](test-helper.md) | KafkaTestHelper implementation |\n| [isolation.md](isolation.md) | Test isolation patterns including file ordering and state cleanup |\n| [docker-setup.md](docker-setup.md) | Redpanda/Kafka Docker configs |\n| [performance.md](performance.md) | Optimization techniques |\n| [examples.md](examples.md) | Complete test examples |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":12976,"content_sha256":"08a851eb4b560a2299de62ac531732cefa5e982bd8e0ed5ef57f681b6251abdd"},{"filename":"references/kafka/performance.md","content":"# Kafka E2E Test Performance Optimization\n\n## Table of Contents\n- [Where Time Is Spent](#where-time-is-spent)\n- [Smart Polling vs Fixed Waits](#smart-polling-vs-fixed-waits)\n- [Consumer Configuration](#consumer-configuration)\n- [Producer Configuration](#producer-configuration)\n- [Container Optimization](#container-optimization)\n- [Jest Configuration](#jest-configuration)\n- [Expected Savings](#expected-savings)\n\n---\n\n## Where Time Is Spent\n\n| Phase | Typical Time | Optimization Potential |\n|-------|--------------|------------------------|\n| Kafka/Redpanda container startup | 5-15s | High (use Redpanda, reuse containers) |\n| NestJS app initialization | 3-10s | Medium (singleton pattern) |\n| Consumer subscription ready | 3-5s | Medium (reduce timeouts) |\n| Message processing wait | 8-15s per test | High (tune Kafka configs, smart polling) |\n| Between-test cleanup | 2s | Low (buffer clearing is fast) |\n\n---\n\n## Smart Polling vs Fixed Waits\n\nThe biggest per-test optimization: return immediately when messages arrive.\n\n```typescript\n// BEFORE: Fixed wait (slow - always waits full time)\nawait new Promise(resolve => setTimeout(resolve, 8000));\nconst messages = kafkaHelper.getMessages(topic);\n\n// AFTER: Smart polling (fast - returns when ready)\nasync waitForMessages(\n topic: string,\n expectedCount: number,\n timeoutMs: number = 10000\n): Promise\u003cMessage[]> {\n const startTime = Date.now();\n const pollInterval = 50; // Poll every 50ms\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const messages = this.messageBuffers.get(topic) || [];\n if (messages.length >= expectedCount) {\n return messages; // Return immediately when ready\n }\n await new Promise(r => setTimeout(r, pollInterval));\n }\n\n throw new Error(`Timeout waiting for ${expectedCount} messages`);\n}\n```\n\n| Approach | Time for 1 message (fast case) | Time (slow case) |\n|----------|--------------------------------|------------------|\n| Fixed 8s wait | 8000ms | 8000ms |\n| Polling (50ms interval) | ~50-200ms | timeout |\n\n---\n\n## Consumer Configuration\n\n### Optimized for E2E Tests\n\n```typescript\nconst consumer = kafka.consumer({\n groupId: 'test-consumer',\n\n // FAST REBALANCING - reduce from 45s default to 6s\n sessionTimeout: 6000,\n heartbeatInterval: 2000, // Must be \u003c sessionTimeout/3\n\n // FAST FAILURE DETECTION\n rebalanceTimeout: 10000, // Reduce from 60s default\n\n // NO RETRIES IN TESTS - fail fast\n retry: {\n retries: 0,\n },\n\n // IMMEDIATE FETCH - don't wait for batch\n maxWaitTimeInMs: 100, // Max wait 100ms (default: 5000ms)\n});\n```\n\n### Impact\n\n| Setting | Default | Test Value | Impact |\n|---------|---------|------------|--------|\n| `sessionTimeout` | 45s | 6s | Faster dead consumer detection |\n| `heartbeatInterval` | 3s | 2s | More frequent liveness checks |\n| `rebalanceTimeout` | 60s | 10s | Faster group rebalancing |\n| `maxWaitTimeInMs` | 5000ms | 100ms | Faster message fetch |\n\n---\n\n## Producer Configuration\n\n### Optimized for E2E Tests\n\n```typescript\nconst producer = kafka.producer({\n allowAutoTopicCreation: true,\n metadataMaxAge: 10000, // 10s (default: 300s) - faster metadata refresh\n});\n\n// When sending messages\nawait producer.send({\n topic: 'test-topic',\n messages: [{ value: JSON.stringify(event) }],\n acks: 1, // Don't wait for all replicas (faster)\n timeout: 5000,\n});\n```\n\n### Impact\n\n| Setting | Default | Test Value | Impact |\n|---------|---------|------------|--------|\n| `linger.ms` | 5ms | 0ms | Send immediately, no batching |\n| `acks` | all | 1 | Don't wait for replica sync |\n| `metadataMaxAge` | 300s | 10s | Faster metadata refresh |\n\n---\n\n## Container Optimization\n\n### Use Redpanda Over Apache Kafka\n\n```yaml\n# Redpanda starts 2x faster\nservices:\n redpanda:\n image: redpandadata/redpanda:latest\n command:\n - redpanda start\n - --mode dev-container\n - --smp 1\n - --memory 512M\n - --overprovisioned\n```\n\n| Image | Startup Time |\n|-------|--------------|\n| `confluentinc/cp-kafka` | ~8.5s |\n| `apache/kafka-native` | ~5.1s |\n| `redpandadata/redpanda` | ~5.1s |\n\n### Container Reuse (Development Only)\n\n```typescript\n// Keep container running between test runs\nconst kafka = new KafkaContainer('redpandadata/redpanda:latest')\n .withReuse();\n\n// Requires ~/.testcontainers.properties:\n// testcontainers.reuse.enable=true\n```\n\n**Warning:** Only for local development. Don't use in CI.\n\n### Parallel Container Startup\n\n```typescript\n// Start multiple containers in parallel\nbeforeAll(async () => {\n const [kafkaContainer, mongoContainer] = await Promise.all([\n new KafkaContainer('redpandadata/redpanda:latest').start(),\n new MongoDBContainer('mongo:6').start(),\n ]);\n // Sequential: 15-20s, Parallel: 6-8s\n}, 60000);\n```\n\n### Singleton Container Pattern\n\n```typescript\n// test/setup/kafka-singleton.ts\nlet kafkaContainer: StartedKafkaContainer | null = null;\n\nexport async function getKafkaContainer(): Promise\u003cStartedKafkaContainer> {\n if (!kafkaContainer) {\n kafkaContainer = await new KafkaContainer('redpandadata/redpanda:latest')\n .withExposedPorts(9092)\n .start();\n }\n return kafkaContainer;\n}\n\n// In jest.config.js\nmodule.exports = {\n globalSetup: './test/setup/global-setup.ts',\n globalTeardown: './test/setup/global-teardown.ts',\n};\n```\n\n---\n\n## Jest Configuration\n\n```javascript\n// jest-e2e.config.js\nmodule.exports = {\n // Run tests sequentially for Kafka (avoid consumer group conflicts)\n maxWorkers: 1, // or --runInBand\n\n // Increase timeout for Kafka tests\n testTimeout: 30000,\n\n // Cache transformed files\n cache: true,\n cacheDirectory: './node_modules/.cache/jest',\n};\n```\n\n### Use `beforeAll` Instead of `beforeEach`\n\n```typescript\n// GOOD: Setup once for entire suite\nbeforeAll(async () => {\n await kafkaHelper.connect();\n await kafkaHelper.subscribeToTopic(outputTopic);\n await startApp();\n}, 90000);\n\n// GOOD: Only clear buffers between tests (fast)\nbeforeEach(() => {\n kafkaHelper.clearMessages(outputTopic);\n});\n\n// BAD: Full setup for each test (slow)\nbeforeEach(async () => {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n // ...\n});\n```\n\n### Reduce Between-Test Overhead\n\n```typescript\n// BEFORE: Topic deletion (slow - 1-3s per topic)\nbeforeEach(async () => {\n await kafkaHelper.deleteAllTopics();\n await kafkaHelper.createTopics();\n await new Promise(r => setTimeout(r, 2000));\n});\n\n// AFTER: Buffer clearing only (fast - instant)\nbeforeEach(async () => {\n kafkaHelper.clearMessages(outputTopic);\n // No wait needed - buffer clearing is synchronous\n});\n```\n\n---\n\n## Expected Savings\n\n| Optimization | Before | After | Savings |\n|--------------|--------|-------|---------|\n| Redpanda vs Kafka | 8.5s startup | 5.1s startup | ~3.4s |\n| Consumer timeouts | 45s session | 6s session | Faster rebalance |\n| Smart polling | 8s fixed wait | ~200ms avg | ~7.8s per test |\n| Buffer clearing vs topic deletion | 2-3s | \u003c10ms | ~2-3s per test |\n| Parallel container startup | 15-20s | 6-8s | ~10s |\n| Singleton containers | N × startup | 1 × startup | (N-1) × startup |\n\n**Total potential savings per test: 10-15 seconds**\n\n---\n\n## Complete Optimized Configuration\n\n```typescript\n// KafkaTestHelper with optimized configs\nconst kafka = new Kafka({\n clientId: 'test-helper',\n brokers: ['localhost:9094'],\n});\n\nconst consumer = kafka.consumer({\n groupId: 'test-consumer',\n sessionTimeout: 6000,\n heartbeatInterval: 2000,\n rebalanceTimeout: 10000,\n retry: { retries: 0 },\n maxWaitTimeInMs: 100,\n});\n\nconst producer = kafka.producer({\n allowAutoTopicCreation: true,\n metadataMaxAge: 10000,\n});\n\n// Test structure\ndescribe('E2E', () => {\n beforeAll(async () => { /* one-time setup */ }, 90000);\n beforeEach(() => { kafkaHelper.clearMessages(topic); });\n // Tests with smart polling...\n});\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":7802,"content_sha256":"a71a772525f6bf0423ab9c24c944f9d1662abf64c1965cad32ae0b6299af6a81"},{"filename":"references/kafka/rules.md","content":"# Kafka E2E Testing Rules\n\n## Core Rules\n\n| Rule | Requirement |\n|------|-------------|\n| Topic creation | Use Admin to create topics - NEVER rely on auto-create |\n| Kafka waits | Use smart polling (10-20s timeout, 50ms interval) |\n| Consumer isolation | Use `fromBeginning: false`, clear buffer between tests |\n| Consumer group | Unique per test suite: `${baseId}-e2e-${Date.now()}` |\n| Microservice config | `{ inheritAppConfig: true }` |\n| Consumer readiness | Use Admin API `waitForConsumerGroupStable()` - NOT blind waits |\n| Broker health | Use `waitForBrokerReady()` in beforeAll for reliability |\n| **Test file ordering** | Tests with invalid broker connections MUST run LAST |\n| **State cleanup** | For request-response tests, restart Kafka container in `beforeAll` |\n\n## Topic Management Rules\n\n### NEVER Rely on Auto-Create\n\nAuto-create causes intermittent test failures due to timing issues. Topics may not be ready when tests start.\n\n```typescript\n// ❌ WRONG: Relying on auto-create\nconst producer = kafka.producer({ allowAutoTopicCreation: true });\nawait producer.send({ topic: 'my-topic', messages: [...] }); // Topic may not exist!\n\n// ✅ CORRECT: Explicit topic creation with Admin\nbeforeAll(async () => {\n await kafkaHelper.connect();\n await kafkaHelper.createTopicIfNotExists(inputTopic, 1); // Create first\n await kafkaHelper.createTopicIfNotExists(outputTopic, 1);\n // Then subscribe and start app\n});\n```\n\n### Setup Order\n\n1. Initialize KafkaTestHelper and connect\n2. Create topics with Admin (createTopicIfNotExists)\n3. Subscribe to output topics\n4. Start application\n\n## Isolation Rules\n\n### Primary: Message Buffer Clearing\n\n```typescript\n// In beforeEach - the main isolation mechanism\nbeforeEach(async () => {\n kafkaHelper.clearMessages(outputTopic);\n await new Promise(r => setTimeout(r, 2000)); // Grace period\n});\n```\n\n### DO NOT Use\n\n```typescript\n// ❌ WRONG: fromBeginning reads ALL historical messages\nawait consumer.subscribe({ topic, fromBeginning: true });\n\n// ❌ WRONG: Topic deletion is slow and error-prone\nawait admin.deleteTopics({ topics: [testTopic] });\nawait admin.createTopics({ topics: [{ topic: testTopic }] });\n```\n\n### DO Use\n\n```typescript\n// ✅ CORRECT: Pre-subscribe with fromBeginning: false\nawait kafkaHelper.subscribeToTopic(outputTopic, false);\n\n// ✅ CORRECT: Clear buffer between tests\nbeforeEach(() => {\n kafkaHelper.clearMessages(outputTopic);\n});\n```\n\n## Consumer Configuration Rules\n\n### Optimized for E2E Tests\n\n```typescript\nconst consumer = kafka.consumer({\n groupId: `test-consumer-${Date.now()}`, // Unique per suite\n\n // FAST REBALANCING\n sessionTimeout: 6000, // Reduce from 45s default\n heartbeatInterval: 2000, // Must be \u003c sessionTimeout/3\n rebalanceTimeout: 10000, // Reduce from 60s default\n\n // NO RETRIES IN TESTS\n retry: { retries: 0 },\n\n // IMMEDIATE FETCH\n maxWaitTimeInMs: 100, // Max wait 100ms (default: 5000ms)\n});\n```\n\n## Producer Configuration Rules\n\n```typescript\nconst producer = kafka.producer({\n allowAutoTopicCreation: true,\n metadataMaxAge: 10000, // 10s (default: 300s)\n});\n\nawait producer.send({\n topic: 'test-topic',\n messages: [{ value: JSON.stringify(event) }],\n acks: 1, // Don't wait for all replicas\n timeout: 5000,\n});\n```\n\n## NestJS Kafka Setup Rules\n\n```typescript\n// ✅ REQUIRED: Create topics BEFORE starting app\nawait kafkaHelper.createTopicIfNotExists(inputTopic, 1);\nawait kafkaHelper.createTopicIfNotExists(outputTopic, 1);\n\n// ✅ REQUIRED: Generate unique consumer group\nconst uniqueGroupId = `${configService.get('KAFKA_GROUP_ID')}-e2e-${Date.now()}`;\n\n// ✅ REQUIRED: Connect microservice with inheritAppConfig\napp.connectMicroservice({\n transport: Transport.KAFKA,\n options: {\n client: {\n clientId: configService.get('KAFKA_CLIENT_ID'),\n brokers: [configService.get('KAFKA_BROKER')],\n },\n consumer: {\n groupId: uniqueGroupId,\n // ❌ DON'T use allowAutoTopicCreation - create topics with Admin instead\n },\n subscribe: { fromBeginning: false }, // NOT fromBeginning!\n },\n}, { inheritAppConfig: true }); // ← CRITICAL\n\n// ✅ REQUIRED: Wait for consumer subscription\nawait app.startAllMicroservices();\nawait app.init();\nawait new Promise(r => setTimeout(r, 5000)); // Wait 5s\n```\n\n## Consumer Group Synchronization Rules\n\n### CRITICAL: Use Admin API for Consumer Readiness\n\n**Never use blind waits to assume consumer readiness.** Use Admin API `describeGroups` to verify consumer group state.\n\n```typescript\n// ❌ WRONG: Blind wait (guessing consumer is ready)\nawait app.startAllMicroservices();\nawait new Promise(r => setTimeout(r, 5000)); // Hope 5s is enough\n\n// ✅ CORRECT: Admin API state verification\nawait app.startAllMicroservices();\nconst groupId = getConsumerGroupId(); // Expose from your client\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\n```\n\n### After Dynamic Topic Subscription\n\nWhen subscribing to topics after consumer is already running, a rebalance occurs. The consumer is NOT ready until rebalance completes.\n\n```typescript\n// ❌ WRONG: Assume subscription is immediate\nawait client.subscribeToResponseOf('new-topic');\nawait kafkaHelper.publishEvent('new-topic', event); // May be lost!\n\n// ✅ CORRECT: Wait for rebalance completion\nawait client.subscribeToResponseOf('new-topic');\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\nawait kafkaHelper.publishEvent('new-topic', event); // Safe\n```\n\n### Consumer Group State Reference\n\n| State | Value | Ready? | Action |\n|-------|-------|--------|--------|\n| `Stable` | 3 | ✅ Yes | Proceed with test |\n| `PreparingRebalance` | 1 | ❌ No | Wait |\n| `CompletingRebalance` | 2 | ❌ No | Wait |\n| `Empty` | 5 | ❌ No | Consumer not started |\n| `Dead` | 4 | ❌ No | Error - check setup |\n\n**Note:** @confluentinc/kafka-javascript uses numeric values, KafkaJS uses string names.\n\n---\n\n## Polling Rules\n\n### Smart Polling vs Fixed Waits\n\n```typescript\n// ❌ WRONG: Fixed wait (slow - always waits full time)\nawait new Promise(resolve => setTimeout(resolve, 8000));\nconst messages = kafkaHelper.getMessages(topic);\n\n// ✅ CORRECT: Smart polling (fast - returns when ready)\nconst messages = await kafkaHelper.waitForMessages(topic, 1, 20000);\n```\n\n### Polling Implementation\n\n```typescript\nasync waitForMessages(topic: string, count: number, timeoutMs: number = 10000) {\n const startTime = Date.now();\n const pollInterval = 50; // Check every 50ms\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const messages = this.messageBuffers.get(topic) || [];\n if (messages.length >= count) {\n return messages.slice(0, count);\n }\n await new Promise(r => setTimeout(r, pollInterval));\n }\n\n throw new Error(`Timeout: Expected ${count} messages`);\n}\n```\n\n## Error Handling Rules\n\n### Known Kafka Issues\n\n| Issue | Solution |\n|-------|----------|\n| Topic doesn't exist / timing issues | Use `createTopicIfNotExists()` in beforeAll - NEVER auto-create |\n| Consumer not ready | Wait 5s after `startAllMicroservices()` |\n| Wrong broker address | Use `localhost:9094` for external access |\n| Slow rebalancing | Optimize consumer config (sessionTimeout: 6000) |\n| Messages from previous runs | Use `fromBeginning: false`, clear buffer |\n\n### Retry Mock Rules\n\n```typescript\n// ❌ BAD: Only mocking one failure\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 }));\n\n// ✅ GOOD: Mock ALL retry attempts\nmockHttpService.post.mockClear();\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 })); // attempt 1\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 })); // attempt 2\nmockHttpService.post.mockReturnValueOnce(of({ status: 500 })); // attempt 3\n```\n\n## Test File Ordering Rules\n\n### CRITICAL: Invalid Broker Connection Tests Run LAST\n\nTests that connect to invalid brokers (for error handling validation) corrupt librdkafka's internal state. This corruption persists within the Jest process.\n\n```typescript\n// These patterns corrupt state for subsequent tests\n'bootstrap.servers': 'localhost:19999', // Non-existent port\n'bootstrap.servers': 'invalid-broker:9999', // Non-existent host\n```\n\n### Solution: Custom Test Sequencer\n\n```javascript\n// test/e2e/test-sequencer.js\nimport Sequencer from '@jest/test-sequencer';\n\nconst TEST_ORDER = [\n 'standard-tests.e2e-spec.ts', // Clean operations first\n 'integration-tests.e2e-spec.ts', // Complex tests middle\n 'error-handling.e2e-spec.ts', // Invalid brokers LAST\n];\n\nexport default class CustomSequencer extends Sequencer.default {\n sort(tests) {\n const orderMap = new Map(TEST_ORDER.map((name, index) => [name, index]));\n return [...tests].sort((a, b) => {\n const aOrder = orderMap.get(a.path.split('/').pop()) ?? 999;\n const bOrder = orderMap.get(b.path.split('/').pop()) ?? 999;\n return aOrder - bOrder;\n });\n }\n}\n```\n\n### Jest Configuration\n\n```javascript\n// test/jest-e2e.config.mjs\nconst config = {\n testSequencer: '\u003crootDir>/e2e/test-sequencer.js',\n // ... other config\n};\n```\n\n---\n\n## Broker Health Check Rules\n\n### Use Broker Health Checks in beforeAll\n\nAlways verify broker availability before running tests:\n\n```typescript\nbeforeAll(async () => {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n\n // CRITICAL: Wait for broker to be ready\n await kafkaHelper.waitForBrokerReady(30000);\n\n // Then create topics and subscribe\n await kafkaHelper.createTopicIfNotExists(inputTopic, 1);\n // ...\n}, 90000);\n```\n\n### After Infrastructure Manipulation Tests\n\nTests that stop/pause Docker containers need comprehensive health verification:\n\n```typescript\n// In self-healing test file\nafterAll(async () => {\n // Ensure broker is healthy before next test file runs\n await kafkaHelper.ensureBrokerHealthy(60000);\n await kafkaHelper.disconnect();\n}, 90000);\n```\n\n---\n\n## Kafka Container Restart Rules\n\n### When to Restart Kafka Container\n\nFor tests suffering from state accumulation, restart the Kafka Docker container in `beforeAll`:\n\n| Test Type | Restart Needed? | Reason |\n|-----------|-----------------|--------|\n| Request-response pattern | ✅ Yes | Consumer group state, pending responses |\n| High-concurrency (10+ parallel) | ✅ Yes | librdkafka connection pool state |\n| Consumer group manipulation | ✅ Yes | Offset tracking state |\n| Simple produce/consume | ❌ No | Minimal state accumulation |\n| CI/CD (fresh process) | ❌ No | Process isolation handles cleanup |\n\n### Implementation\n\n```typescript\nimport { exec } from 'child_process';\nimport { promisify } from 'util';\n\nconst execAsync = promisify(exec);\n\nasync function restartKafkaContainer(): Promise\u003cvoid> {\n console.log('[Setup] Restarting Kafka container for clean state...');\n await execAsync('docker restart kafka-e2e');\n // 20 seconds ensures broker fully initialized\n await new Promise((resolve) => setTimeout(resolve, 20000));\n console.log('[Setup] Kafka container restarted');\n}\n\nbeforeAll(async () => {\n await restartKafkaContainer();\n // ... rest of setup\n}, 120000); // Extended timeout\n```\n\n### Key Points\n\n- **Wait 20 seconds** after restart for Kafka to fully initialize\n- **Extend beforeAll timeout** to 120 seconds (2 minutes)\n- **Use in development** when running tests repeatedly\n- **Not needed in CI/CD** where each run is a fresh process\n\n### Limitations\n\n| Consecutive Runs | Result |\n|------------------|--------|\n| Run 1-2 | ✅ 100% pass |\n| Run 3+ | ⚠️ May fail (~60-80% pass) due to librdkafka internal state |\n\nFor complete isolation on unlimited runs, consider running each test file in a separate process.\n\n---\n\n## Checklist\n\n**Test File Ordering:**\n- [ ] Custom test sequencer configured in `jest-e2e.config.mjs`\n- [ ] Tests with invalid broker connections run LAST\n- [ ] Tests that manipulate Docker run late in sequence\n- [ ] All test files listed in sequencer\n\n**State Cleanup (Request-Response Tests):**\n- [ ] Restart Kafka container in `beforeAll` for complex tests\n- [ ] Wait 20 seconds after restart for full initialization\n- [ ] Extended `beforeAll` timeout (120 seconds)\n- [ ] Delete consumer groups in `afterAll`\n\n**Broker Health:**\n- [ ] Use `waitForBrokerReady()` in `beforeAll`\n- [ ] Use `ensureBrokerHealthy()` in `afterAll` for infrastructure tests\n- [ ] Handle broker unavailability gracefully\n\n**Topic Management:**\n- [ ] Create topics with Admin in `beforeAll` (createTopicIfNotExists)\n- [ ] NEVER rely on auto-create\n- [ ] Use unique topic names per suite: `topic-${Date.now()}`\n\n**Setup:**\n- [ ] Unique consumer group ID per suite\n- [ ] `fromBeginning: false` in subscribe options\n- [ ] `inheritAppConfig: true` in connectMicroservice\n- [ ] Pre-subscribe to output topics before app starts\n\n**Consumer Synchronization (Admin API):**\n- [ ] Use `waitForConsumerGroupStable()` instead of blind waits\n- [ ] Expose consumer group ID from your Kafka client\n- [ ] After dynamic subscription, wait for rebalance completion\n- [ ] Verify state is `Stable` before publishing test messages\n\n**Test Isolation:**\n- [ ] Clear message buffer in `beforeEach`\n- [ ] 2s grace period after buffer clearing\n- [ ] NO topic deletion/recreation between tests\n\n**Assertions:**\n- [ ] Use smart polling (waitForMessages)\n- [ ] 20s max timeout for message expectations\n- [ ] Verify message content with `toMatchObject`\n\n**Performance:**\n- [ ] Consumer optimized (sessionTimeout: 6000)\n- [ ] Producer optimized (acks: 1)\n- [ ] Polling interval: 50ms\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13412,"content_sha256":"bc410ca71e7a4b978f45f15b4924f1de398cd25d29e31b5514a4f344e75a656c"},{"filename":"references/kafka/test-helper.md","content":"# KafkaTestHelper Implementation\n\n## Table of Contents\n- [Topic Management with Admin](#topic-management-with-admin)\n- [Complete Implementation](#complete-implementation)\n- [Usage Patterns](#usage-patterns)\n- [API Reference](#api-reference)\n\n---\n\n## Topic Management with Admin\n\n**Never rely on topic auto-create in E2E tests.**\n\nAuto-create causes timing issues - tests may fail intermittently waiting for topics to be created. Use Kafka Admin to explicitly create topics before tests run.\n\n### Why Use Admin Instead of Auto-Create?\n\n| Auto-Create Problems | Admin Benefits |\n|---------------------|----------------|\n| Timing issues - topic may not be ready | Topic guaranteed ready before tests |\n| No control over partitions/replication | Explicit partition and replication control |\n| Intermittent test failures | Deterministic test setup |\n| Hidden dependencies on broker config | Self-contained test configuration |\n\n### Admin Initialization Pattern\n\n```typescript\n// In beforeAll - initialize admin and create topics FIRST\nbeforeAll(async () => {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n\n // Create topics explicitly - don't rely on auto-create\n await kafkaHelper.createTopicIfNotExists(inputTopic, 1);\n await kafkaHelper.createTopicIfNotExists(outputTopic, 1);\n\n // Then subscribe to output topics\n await kafkaHelper.subscribeToTopic(outputTopic, false);\n\n // Then start app\n}, 90000);\n\n// In afterAll - cleanup admin connection\nafterAll(async () => {\n await kafkaHelper?.disconnect();\n}, 30000);\n```\n\n---\n\n## Complete Implementation\n\n```typescript\n// test/helpers/kafka-helper.ts\nimport { Kafka, Producer, Consumer, Admin } from 'kafkajs';\n\nexport class KafkaTestHelper {\n private kafka: Kafka;\n private producer: Producer;\n private admin: Admin;\n private messageBuffers: Map\u003cstring, Array\u003c{ key: string | null; value: unknown }>> = new Map();\n private runningConsumers: Map\u003cstring, Consumer> = new Map();\n\n async connect(): Promise\u003cvoid> {\n this.kafka = new Kafka({\n clientId: 'test-helper',\n brokers: [process.env.KAFKA_BROKER_URL || 'localhost:9094'],\n });\n this.producer = this.kafka.producer();\n this.admin = this.kafka.admin();\n await this.producer.connect();\n await this.admin.connect();\n }\n\n /**\n * Subscribe BEFORE app starts - accumulate messages in buffer.\n * Use fromBeginning: false for isolation (NOT true!)\n */\n async subscribeToTopic(topic: string, fromBeginning: boolean = false): Promise\u003cvoid> {\n if (this.runningConsumers.has(topic)) {\n return; // Prevent duplicate subscriptions\n }\n\n const consumer = this.kafka.consumer({\n groupId: 'test-consumer-helper',\n sessionTimeout: 30000,\n });\n\n await consumer.connect();\n await consumer.subscribe({ topic, fromBeginning });\n\n // Initialize buffer\n this.messageBuffers.set(topic, []);\n\n // Start consuming - accumulate in buffer\n await consumer.run({\n eachMessage: async ({ message }) => {\n const value = message.value ? JSON.parse(message.value.toString()) : null;\n const buffer = this.messageBuffers.get(topic) || [];\n buffer.push({\n key: message.key?.toString() || null,\n value,\n });\n this.messageBuffers.set(topic, buffer);\n },\n });\n\n this.runningConsumers.set(topic, consumer);\n }\n\n /**\n * Poll for messages with timeout - key to async testing.\n * Returns as soon as expected count is reached (smart polling).\n */\n async waitForMessages(\n topic: string,\n count: number,\n timeoutMs: number = 10000\n ): Promise\u003cArray\u003c{ key: string | null; value: unknown }>> {\n if (!this.runningConsumers.has(topic)) {\n throw new Error(`No consumer subscribed to topic: ${topic}`);\n }\n\n const startTime = Date.now();\n const pollInterval = 50; // Check every 50ms for speed\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const messages = this.messageBuffers.get(topic) || [];\n if (messages.length >= count) {\n return messages.slice(0, count);\n }\n await new Promise(resolve => setTimeout(resolve, pollInterval));\n }\n\n const messages = this.messageBuffers.get(topic) || [];\n throw new Error(`Timeout: Expected ${count} messages but received ${messages.length}`);\n }\n\n /**\n * Wait for a specific message matching a predicate.\n */\n async waitForMessage\u003cT>(\n topic: string,\n predicate: (msg: { key: string | null; value: T }) => boolean,\n timeoutMs: number = 10000\n ): Promise\u003c{ key: string | null; value: T }> {\n if (!this.runningConsumers.has(topic)) {\n throw new Error(`No consumer subscribed to topic: ${topic}`);\n }\n\n const startTime = Date.now();\n const pollInterval = 50;\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const messages = this.messageBuffers.get(topic) || [];\n const found = messages.find(predicate as any);\n if (found) {\n return found as { key: string | null; value: T };\n }\n await new Promise(resolve => setTimeout(resolve, pollInterval));\n }\n\n throw new Error(`Timeout: Message matching predicate not found`);\n }\n\n /**\n * CRITICAL: Clear buffer between tests for isolation.\n * This is the primary isolation mechanism - NOT topic deletion.\n */\n clearMessages(topic: string): void {\n this.messageBuffers.set(topic, []);\n }\n\n /** Clear all message buffers */\n clearAllMessages(): void {\n for (const topic of this.messageBuffers.keys()) {\n this.messageBuffers.set(topic, []);\n }\n }\n\n /** Get current messages without waiting */\n getMessages(topic: string): Array\u003c{ key: string | null; value: unknown }> {\n return this.messageBuffers.get(topic) || [];\n }\n\n /** Publish single message */\n async publishEvent(topic: string, event: unknown, key?: string): Promise\u003cvoid> {\n await this.producer.send({\n topic,\n messages: [{\n key: key || null,\n value: JSON.stringify(event),\n }],\n });\n }\n\n /** Publish batch for performance tests */\n async publishBatch(\n topic: string,\n events: Array\u003c{ event: unknown; key?: string }>\n ): Promise\u003cvoid> {\n await this.producer.send({\n topic,\n messages: events.map(item => ({\n key: item.key || null,\n value: JSON.stringify(item.event),\n })),\n });\n }\n\n /**\n * Create topic if not exists - CRITICAL for test setup.\n * Call in beforeAll BEFORE subscribing or starting app.\n * Never rely on auto-create - it causes timing issues.\n */\n async createTopicIfNotExists(\n topic: string,\n numPartitions: number = 1,\n replicationFactor: number = 1\n ): Promise\u003cvoid> {\n const topics = await this.admin.listTopics();\n if (!topics.includes(topic)) {\n await this.admin.createTopics({\n topics: [{\n topic,\n numPartitions,\n replicationFactor,\n }],\n });\n }\n }\n\n /** Alias for backward compatibility */\n async ensureTopicExists(topic: string, numPartitions: number = 1): Promise\u003cvoid> {\n return this.createTopicIfNotExists(topic, numPartitions);\n }\n\n /**\n * Delete topic - use for cleanup or resetting test state.\n * Note: Topic deletion may take time to propagate.\n */\n async deleteTopic(topic: string): Promise\u003cvoid> {\n try {\n await this.admin.deleteTopics({ topics: [topic] });\n } catch {\n // Topic may not exist, ignore\n }\n }\n\n /**\n * Update topic configuration.\n * Use for testing config changes (retention, cleanup policy, etc.)\n */\n async updateTopicConfig(\n topic: string,\n configEntries: Array\u003c{ name: string; value: string }>\n ): Promise\u003cvoid> {\n await this.admin.alterConfigs({\n validateOnly: false,\n resources: [{\n type: 2, // TOPIC\n name: topic,\n configEntries,\n }],\n });\n }\n\n /** List all topics */\n async listTopics(): Promise\u003cstring[]> {\n return this.admin.listTopics();\n }\n\n /** Delete consumer group offsets (for complete reset) */\n async deleteConsumerGroupOffsets(groupId: string): Promise\u003cvoid> {\n try {\n await this.admin.deleteGroups([groupId]);\n } catch {\n // Group may not exist, ignore\n }\n }\n\n // ============================================================\n // CONSUMER GROUP STATE MONITORING (Admin API)\n // ============================================================\n // Use these methods to synchronize tests with Kafka consumer state\n // instead of blind waits. These provide deterministic test readiness.\n // ============================================================\n\n /**\n * Wait for consumer group to reach 'Stable' state.\n * CRITICAL: Use this after subscribing to topics to ensure consumer is ready.\n *\n * Consumer group states (from librdkafka/confluentinc ConsumerGroupStates enum):\n * - 0: Unknown\n * - 1: PreparingRebalance - Group is rebalancing, NOT ready\n * - 2: CompletingRebalance - Finishing rebalance, NOT ready\n * - 3: Stable - Ready to consume messages ✓\n * - 4: Dead - Group has been deleted\n * - 5: Empty - No active members\n *\n * Note: For KafkaJS, states are strings ('Stable', 'PreparingRebalance', etc.)\n * For @confluentinc/kafka-javascript (node-rdkafka), states are numeric.\n */\n async waitForConsumerGroupStable(\n groupId: string,\n timeoutMs: number = 30000,\n pollIntervalMs: number = 500\n ): Promise\u003cvoid> {\n const startTime = Date.now();\n const STABLE_STATE = 3; // For node-rdkafka/confluentinc\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const state = await this.getConsumerGroupState(groupId);\n const stateNum = typeof state === 'string' ? parseInt(state, 10) : state;\n\n if (stateNum === STABLE_STATE) {\n // Small buffer for consume loop to fully start\n await new Promise(r => setTimeout(r, 500));\n return; // Consumer ready\n }\n\n // Log for debugging\n if (state !== null) {\n const stateNames: Record\u003cnumber, string> = {\n 0: 'Unknown', 1: 'PreparingRebalance', 2: 'CompletingRebalance',\n 3: 'Stable', 4: 'Dead', 5: 'Empty'\n };\n console.log(`Consumer group '${groupId}' state: ${stateNames[stateNum] || state}`);\n }\n\n await new Promise(r => setTimeout(r, pollIntervalMs));\n }\n\n throw new Error(`Timeout waiting for consumer group '${groupId}' to reach Stable state`);\n }\n\n /**\n * Get current state of a consumer group using Admin API.\n * Returns null if group doesn't exist.\n *\n * For @confluentinc/kafka-javascript (node-rdkafka):\n */\n async getConsumerGroupState(groupId: string): Promise\u003cstring | null> {\n return new Promise((resolve, reject) => {\n this.admin.describeGroups([groupId], { timeout: 5000 }, (err, result) => {\n if (err) {\n reject(new Error(`Failed to describe group: ${err.message}`));\n return;\n }\n if (!result.groups || result.groups.length === 0) {\n resolve(null);\n return;\n }\n const groupInfo = result.groups[0];\n if (groupInfo.error) {\n resolve(null);\n return;\n }\n resolve(String(groupInfo.state));\n });\n });\n }\n\n /**\n * Wait for consumer group to have expected number of members.\n * Useful when testing consumer scaling or ensuring all consumers joined.\n */\n async waitForConsumerGroupMembers(\n groupId: string,\n expectedMembers: number,\n timeoutMs: number = 30000\n ): Promise\u003cvoid> {\n const startTime = Date.now();\n\n while (Date.now() - startTime \u003c timeoutMs) {\n const count = await this.getConsumerGroupMemberCount(groupId);\n if (count >= expectedMembers) {\n return;\n }\n await new Promise(r => setTimeout(r, 500));\n }\n\n const finalCount = await this.getConsumerGroupMemberCount(groupId);\n throw new Error(\n `Timeout waiting for ${expectedMembers} members in group ${groupId}. ` +\n `Current count: ${finalCount}`\n );\n }\n\n /**\n * Get current number of members in a consumer group.\n */\n async getConsumerGroupMemberCount(groupId: string): Promise\u003cnumber> {\n return new Promise((resolve, reject) => {\n this.admin.describeGroups([groupId], { timeout: 5000 }, (err, result) => {\n if (err) {\n reject(new Error(`Failed to describe group: ${err.message}`));\n return;\n }\n if (!result.groups || result.groups.length === 0) {\n resolve(0);\n return;\n }\n const groupInfo = result.groups[0];\n if (groupInfo.error) {\n resolve(0);\n return;\n }\n resolve(groupInfo.members?.length || 0);\n });\n });\n }\n\n async disconnect(): Promise\u003cvoid> {\n await this.producer.disconnect();\n await this.admin.disconnect();\n for (const consumer of this.runningConsumers.values()) {\n await consumer.disconnect();\n }\n this.runningConsumers.clear();\n this.messageBuffers.clear();\n }\n}\n```\n\n---\n\n## Usage Patterns\n\n### Basic Setup and Teardown\n\n```typescript\ndescribe('Kafka E2E Tests', () => {\n let kafkaHelper: KafkaTestHelper;\n const inputTopic = `test-input-${Date.now()}`;\n const outputTopic = `test-output-${Date.now()}`;\n\n beforeAll(async () => {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n\n // 1. Create topics first - NEVER rely on auto-create\n await kafkaHelper.createTopicIfNotExists(inputTopic, 1);\n await kafkaHelper.createTopicIfNotExists(outputTopic, 1);\n\n // 2. Subscribe to output topics\n await kafkaHelper.subscribeToTopic(outputTopic, false);\n\n // 3. Start app after topics are ready\n // ... start app\n }, 90000);\n\n afterAll(async () => {\n await kafkaHelper?.disconnect();\n });\n\n beforeEach(() => {\n kafkaHelper.clearMessages(outputTopic);\n });\n});\n```\n\n### Waiting for Specific Messages\n\n```typescript\nit('should process order and emit completion event', async () => {\n const orderId = `order-${Date.now()}`;\n\n await kafkaHelper.publishEvent('orders.created', { orderId, items: [] });\n\n // Wait for specific message\n const completionEvent = await kafkaHelper.waitForMessage(\n 'orders.completed',\n (msg) => msg.value.orderId === orderId,\n 15000\n );\n\n expect(completionEvent.value).toMatchObject({\n orderId,\n status: 'completed',\n });\n});\n```\n\n### Performance Testing\n\n```typescript\nit('should handle 100 messages within 30 seconds', async () => {\n const events = Array.from({ length: 100 }, (_, i) => ({\n event: { id: `event-${i}`, data: 'test' },\n key: `event-${i}`,\n }));\n\n const startTime = Date.now();\n await kafkaHelper.publishBatch('input-topic', events);\n\n const messages = await kafkaHelper.waitForMessages('output-topic', 100, 30000);\n const duration = Date.now() - startTime;\n\n expect(messages.length).toBe(100);\n expect(duration).toBeLessThan(30000);\n});\n```\n\n### Consumer Group State Synchronization (Admin API)\n\n**CRITICAL: Use Admin API for deterministic synchronization instead of blind waits.**\n\nThe Admin API `describeGroups` provides real-time consumer group state, enabling tests to proceed only when consumers are actually ready.\n\n```typescript\n// ❌ WRONG: Blind wait (may be too short or wastefully long)\nawait app.startAllMicroservices();\nawait new Promise(r => setTimeout(r, 5000)); // Guessing wait time\n\n// ✅ CORRECT: Admin API state-based synchronization\nawait app.startAllMicroservices();\nconst groupId = client.getResponseConsumerGroupId();\nif (groupId) {\n await kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\n}\n```\n\n#### After Subscribing to New Topics\n\nWhen dynamically subscribing to topics during tests, consumer group rebalancing occurs. Use Admin API to wait for stability:\n\n```typescript\n// Subscribe to multiple topics (triggers rebalance)\nawait client.subscribeToResponseOf('request-topic-1');\nawait client.subscribeToResponseOf('request-topic-2');\n\n// Wait for rebalance to complete using Admin API\nconst groupId = client.getResponseConsumerGroupId();\nawait kafkaHelper.waitForConsumerGroupStable(groupId, 15000);\n\n// NOW the consumer is ready to receive messages\nawait kafkaHelper.publishEvent('request-topic-1', { data: 'test' });\n```\n\n#### Consumer Group States Reference\n\n| State | Value | Meaning | Test Action |\n|-------|-------|---------|-------------|\n| `Unknown` | 0 | State unknown | ⏳ Retry query |\n| `PreparingRebalance` | 1 | Rebalancing in progress | ⏳ Wait |\n| `CompletingRebalance` | 2 | Finishing rebalance | ⏳ Wait |\n| `Stable` | 3 | Consumer ready, partitions assigned | ✅ Safe to proceed |\n| `Dead` | 4 | Group deleted | ❌ Error - reinitialize |\n| `Empty` | 5 | No active members | ⚠️ Check consumer started |\n\n**Note:** @confluentinc/kafka-javascript (node-rdkafka) uses numeric values. KafkaJS uses string names.\n\n#### Verifying Consumer Member Count\n\nFor tests involving multiple consumers or consumer scaling:\n\n```typescript\n// Ensure expected number of consumers have joined\nawait kafkaHelper.waitForConsumerGroupMembers('my-group', 3, 30000);\n\n// Get current member count\nconst memberCount = await kafkaHelper.getConsumerGroupMemberCount('my-group');\nexpect(memberCount).toBe(3);\n```\n\n---\n\n### Broker Health Verification\n\n**CRITICAL: Always verify broker health in beforeAll for reliable tests.**\n\nTests can fail if the broker is unavailable or recovering from previous test file operations. Use these methods to ensure broker readiness:\n\n```typescript\nbeforeAll(async () => {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n\n // CRITICAL: Wait for broker to be ready before any operations\n await kafkaHelper.waitForBrokerReady(30000);\n\n // Then proceed with topic creation and subscription\n await kafkaHelper.createTopicIfNotExists(inputTopic, 1);\n await kafkaHelper.subscribeToTopic(outputTopic, false);\n // ...\n}, 90000);\n```\n\n#### After Infrastructure Manipulation Tests\n\nTests that stop/pause Docker containers (e.g., self-healing tests) should ensure broker health in afterAll:\n\n```typescript\n// In self-healing.e2e-spec.ts or similar\nafterAll(async () => {\n // Ensure broker is fully healthy before next test file runs\n await kafkaHelper.ensureBrokerHealthy(60000);\n\n // Add small delay to allow librdkafka to stabilize\n await new Promise(r => setTimeout(r, 5000));\n\n await kafkaHelper.disconnect();\n}, 90000);\n```\n\n#### Broker Health Check Implementation\n\n```typescript\n/**\n * Quick health check using Admin API listTopics().\n * Returns true if broker responds within timeout.\n */\nasync isBrokerHealthy(timeoutMs: number = 5000): Promise\u003cboolean> {\n try {\n await Promise.race([\n this.admin.listTopics(),\n new Promise((_, reject) =>\n setTimeout(() => reject(new Error('Broker health check timeout')), timeoutMs)\n ),\n ]);\n return true;\n } catch {\n return false;\n }\n}\n\n/**\n * Wait for broker to be ready with polling.\n * Use in beforeAll before any Kafka operations.\n */\nasync waitForBrokerReady(\n timeoutMs: number = 30000,\n pollIntervalMs: number = 1000\n): Promise\u003cvoid> {\n const startTime = Date.now();\n\n while (Date.now() - startTime \u003c timeoutMs) {\n if (await this.isBrokerHealthy(5000)) {\n return; // Broker is ready\n }\n console.log('Waiting for broker to be ready...');\n await new Promise(r => setTimeout(r, pollIntervalMs));\n }\n\n throw new Error(`Broker not ready after ${timeoutMs}ms`);\n}\n\n/**\n * Comprehensive broker health verification.\n * Use after infrastructure manipulation (Docker stop/pause).\n */\nasync ensureBrokerHealthy(timeoutMs: number = 60000): Promise\u003cvoid> {\n await this.waitForBrokerReady(timeoutMs);\n\n // Verify can list topics (confirms Admin API working)\n await this.admin.listTopics();\n\n // Small buffer for full stability\n await new Promise(r => setTimeout(r, 1000));\n}\n```\n\n---\n\n## API Reference\n\n### Admin Methods (Broker Health)\n\n| Method | Description |\n|--------|-------------|\n| `isBrokerHealthy(timeoutMs)` | Quick health check - returns true if broker responds |\n| `waitForBrokerReady(timeoutMs, pollIntervalMs)` | Poll until broker is available |\n| `ensureBrokerHealthy(timeoutMs)` | Comprehensive health verification for infrastructure tests |\n\n### Admin Methods (Topic Management)\n\n| Method | Description |\n|--------|-------------|\n| `createTopicIfNotExists(topic, partitions, replication)` | Create topic if not exists - call in beforeAll |\n| `ensureTopicExists(topic, partitions)` | Alias for createTopicIfNotExists |\n| `deleteTopic(topic)` | Delete topic (for cleanup) |\n| `updateTopicConfig(topic, configEntries)` | Update topic configuration |\n| `listTopics()` | List all topics |\n| `deleteConsumerGroupOffsets(groupId)` | Reset consumer group |\n\n### Admin Methods (Consumer Group Cleanup)\n\n| Method | Description |\n|--------|-------------|\n| `deleteConsumerGroup(groupId)` | Delete a consumer group - use in afterAll for cleanup |\n| `deleteConsumerGroupsByPattern(pattern)` | Delete all groups matching RegExp pattern |\n\n#### Consumer Group Cleanup Example\n\n```typescript\nafterAll(async () => {\n // Store group IDs before closing clients\n const mainGroupId = client?.getResponseConsumerGroupId();\n const customGroupId = customClient?.getResponseConsumerGroupId();\n\n // Close clients first (consumers must be disconnected before group deletion)\n await client?.close();\n await customClient?.close();\n\n // Delete consumer groups to prevent state accumulation\n if (mainGroupId) {\n await kafkaHelper.deleteConsumerGroup(mainGroupId);\n }\n if (customGroupId) {\n await kafkaHelper.deleteConsumerGroup(customGroupId);\n }\n\n // Clean up orphaned groups from failed test runs\n await kafkaHelper.deleteConsumerGroupsByPattern(/^test-client-.*-response-consumer-/);\n\n kafkaHelper.disconnectAdmin();\n}, 30000);\n```\n\n#### Implementation\n\n```typescript\n/**\n * Delete a consumer group from the broker.\n * Use in afterAll to clean up test consumer groups.\n *\n * IMPORTANT: Consumers must be disconnected before deleting the group.\n */\nasync deleteConsumerGroup(groupId: string): Promise\u003cvoid> {\n return new Promise((resolve) => {\n this.admin.deleteGroups([groupId], { timeout: 10000 }, (err) => {\n if (err) {\n const errorCode = (err as LibrdKafkaError).code;\n // Error code 69 = GROUP_ID_NOT_FOUND, which is acceptable\n if (errorCode === 69) {\n resolve();\n } else {\n console.warn(`Could not delete consumer group '${groupId}': ${err.message}`);\n resolve(); // Don't fail tests due to cleanup issues\n }\n } else {\n console.log(`Deleted consumer group: ${groupId}`);\n resolve();\n }\n });\n });\n}\n\n/**\n * Delete all consumer groups matching a pattern.\n * Useful for cleaning up orphaned test consumer groups.\n */\nasync deleteConsumerGroupsByPattern(pattern: RegExp): Promise\u003cvoid> {\n return new Promise((resolve) => {\n this.admin.listGroups({ timeout: 10000 }, (err, result) => {\n if (err) {\n console.warn(`Could not list consumer groups: ${err.message}`);\n resolve();\n return;\n }\n\n const matchingGroups = (result.groups || [])\n .filter((g) => pattern.test(g.groupId))\n .map((g) => g.groupId);\n\n if (matchingGroups.length === 0) {\n resolve();\n return;\n }\n\n console.log(`Deleting ${matchingGroups.length} orphaned consumer groups...`);\n\n this.admin.deleteGroups(matchingGroups, { timeout: 10000 }, (deleteErr) => {\n if (deleteErr) {\n console.warn(`Could not delete some consumer groups: ${deleteErr.message}`);\n }\n resolve();\n });\n });\n });\n}\n```\n\n### Admin Methods (Consumer Group Monitoring)\n\n| Method | Description |\n|--------|-------------|\n| `waitForConsumerGroupStable(groupId, timeoutMs)` | Poll until consumer group state is 'Stable' |\n| `getConsumerGroupState(groupId)` | Get current state: 'Stable', 'PreparingRebalance', 'CompletingRebalance', 'Empty', 'Dead' |\n| `waitForConsumerGroupMembers(groupId, expectedMembers, timeoutMs)` | Wait for specific member count |\n| `getConsumerGroupMemberCount(groupId)` | Get current consumer count in group |\n\n### Connection Methods\n\n| Method | Description |\n|--------|-------------|\n| `connect()` | Initialize Kafka connections (producer + admin) |\n| `disconnect()` | Clean up all connections |\n\n### Consumer Methods\n\n| Method | Description |\n|--------|-------------|\n| `subscribeToTopic(topic, fromBeginning)` | Pre-subscribe to topic, accumulate in buffer |\n| `waitForMessages(topic, count, timeout)` | Poll for expected message count |\n| `waitForMessage(topic, predicate, timeout)` | Poll for specific message |\n| `clearMessages(topic)` | Clear buffer for topic (primary isolation) |\n| `clearAllMessages()` | Clear all buffers |\n| `getMessages(topic)` | Get current buffer contents |\n\n### Producer Methods\n\n| Method | Description |\n|--------|-------------|\n| `publishEvent(topic, event, key)` | Publish single message |\n| `publishBatch(topic, events)` | Publish multiple messages |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":24942,"content_sha256":"fd64a64694474ec0e1da21e0d9d6e97dc9545524e2f008c160c97780adb51401"},{"filename":"references/mongodb/docker-setup.md","content":"# MongoDB Docker Setup for E2E Testing\n\n## docker-compose.e2e.yml\n\n```yaml\nversion: '3.8'\n\nservices:\n mongodb-e2e:\n image: mongo:7.0\n container_name: mongodb-e2e\n ports:\n - \"27018:27017\" # Different port from dev\n environment:\n MONGO_INITDB_ROOT_USERNAME: test\n MONGO_INITDB_ROOT_PASSWORD: test\n MONGO_INITDB_DATABASE: testdb\n volumes:\n - mongodb-e2e-data:/data/db\n healthcheck:\n test: echo 'db.runCommand(\"ping\").ok' | mongosh localhost:27017/test --quiet\n interval: 5s\n timeout: 5s\n retries: 5\n # Performance: WiredTiger with limited cache\n command: mongod --wiredTigerCacheSizeGB 1\n\nvolumes:\n mongodb-e2e-data:\n```\n\n---\n\n## Health Check Script\n\n```bash\n#!/bin/bash\n# scripts/wait-for-mongodb.sh\n\nuntil docker-compose -f docker-compose.e2e.yml exec mongodb-e2e mongosh --eval \"db.adminCommand('ping')\"; do\n echo \"Waiting for MongoDB...\"\n sleep 1\ndone\necho \"MongoDB is ready!\"\n```\n\n---\n\n## Connection String\n\n```bash\n# .env.e2e\nMONGODB_URI=mongodb://test:test@localhost:27018/testdb?authSource=admin\n```\n\n---\n\n## Performance Tuning\n\n### Optimized for Test Speed\n\n```yaml\ncommand: >\n mongod\n --wiredTigerCacheSizeGB 0.5\n --storageEngine wiredTiger\n --nojournal # Disable journaling for tests (unsafe for production!)\n```\n\n### Settings Explained\n\n| Setting | Default | Test Value | Effect |\n|---------|---------|------------|--------|\n| `wiredTigerCacheSizeGB` | 50% RAM | 0.5-1 GB | Limit memory usage |\n| `nojournal` | on | off | Skip write-ahead logging (faster) |\n| `storageEngine` | wiredTiger | wiredTiger | Standard engine |\n\n**Warning:** These settings sacrifice durability for speed. Only use for tests.\n\n---\n\n## MongoDB Memory Server Alternative\n\nFor CI/CD environments without Docker:\n\n```typescript\n// test/setup-mongodb.ts\nimport { MongoMemoryServer } from 'mongodb-memory-server';\n\nlet mongod: MongoMemoryServer;\n\nexport async function setupMongoMemoryServer(): Promise\u003cstring> {\n mongod = await MongoMemoryServer.create({\n instance: {\n dbName: 'testdb',\n storageEngine: 'wiredTiger',\n },\n });\n return mongod.getUri();\n}\n\nexport async function teardownMongoMemoryServer(): Promise\u003cvoid> {\n if (mongod) {\n await mongod.stop();\n }\n}\n```\n\n### Global Setup\n\n```typescript\n// test/jest-e2e.setup.ts\nimport { setupMongoMemoryServer, teardownMongoMemoryServer } from './setup-mongodb';\n\nbeforeAll(async () => {\n const uri = await setupMongoMemoryServer();\n process.env.MONGODB_URI = uri;\n}, 60000);\n\nafterAll(async () => {\n await teardownMongoMemoryServer();\n});\n```\n\n---\n\n## Complete Docker Compose with All Services\n\n```yaml\nversion: '3.8'\n\nservices:\n mongodb-e2e:\n image: mongo:7.0\n container_name: mongodb-e2e\n ports:\n - \"27018:27017\"\n environment:\n MONGO_INITDB_ROOT_USERNAME: test\n MONGO_INITDB_ROOT_PASSWORD: test\n MONGO_INITDB_DATABASE: testdb\n healthcheck:\n test: echo 'db.runCommand(\"ping\").ok' | mongosh localhost:27017/test --quiet\n interval: 5s\n timeout: 5s\n retries: 5\n command: mongod --wiredTigerCacheSizeGB 1\n\n # Add other services as needed\n redis-e2e:\n image: redis:7-alpine\n ports:\n - \"6380:6379\"\n command: redis-server --save \"\" --appendonly no\n\n postgres-e2e:\n image: postgres:15-alpine\n ports:\n - \"5433:5432\"\n environment:\n POSTGRES_USER: test\n POSTGRES_PASSWORD: test\n POSTGRES_DB: testdb\n\nvolumes:\n mongodb-e2e-data:\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3464,"content_sha256":"7d73e8e148a31b4cb32d136b34e4990f3dc0008843cdb25189991db76fddca52"},{"filename":"references/mongodb/examples.md","content":"# MongoDB E2E Test Examples\n\n## Basic CRUD Operations\n\n### Create with Validation\n\n```typescript\nit('should create product with category reference', async () => {\n // GIVEN: Existing category\n const category = await categoryModel.create({ name: 'Electronics' });\n\n // WHEN: Creating product\n const response = await request(httpServer)\n .post('/products')\n .send({\n name: 'Laptop',\n categoryId: category._id.toString(),\n price: 999.99,\n })\n .expect(201);\n\n // THEN: Product created with reference\n await helper.assertDocumentExists(\n productModel,\n { name: 'Laptop' },\n {\n categoryId: category._id,\n price: 999.99,\n }\n );\n});\n```\n\n### Read with Population\n\n```typescript\nit('should return order with populated user and items', async () => {\n // GIVEN: User, products, and order\n const user = await userModel.create({ email: '[email protected]', name: 'Buyer' });\n const products = await productModel.insertMany([\n { name: 'Widget', price: 29.99 },\n { name: 'Gadget', price: 49.99 },\n ]);\n const order = await orderModel.create({\n userId: user._id,\n items: [\n { productId: products[0]._id, quantity: 2 },\n { productId: products[1]._id, quantity: 1 },\n ],\n status: 'completed',\n });\n\n // WHEN: Fetching order with population\n const response = await request(httpServer)\n .get(`/api/v1/orders/${order._id}`)\n .query({ populate: 'user,items.product' })\n .expect(200);\n\n // THEN: Order with populated relations\n expect(response.body.data).toMatchObject({\n status: 'completed',\n user: { email: '[email protected]' },\n items: expect.arrayContaining([\n expect.objectContaining({\n quantity: 2,\n product: { name: 'Widget' },\n }),\n ]),\n });\n});\n```\n\n---\n\n## Nested Document Updates\n\n```typescript\nit('should update nested document field', async () => {\n // GIVEN: Product with specifications\n const product = await productModel.create({\n name: 'Laptop',\n specs: { ram: '8GB', storage: '256GB', processor: 'Intel i5' },\n });\n\n // WHEN: Updating nested field\n const response = await request(httpServer)\n .patch(`/products/${product._id}`)\n .send({ 'specs.ram': '16GB' })\n .expect(200);\n\n // THEN: Only specified field updated\n const updated = await productModel.findById(product._id).lean();\n expect(updated.specs.ram).toBe('16GB');\n expect(updated.specs.storage).toBe('256GB'); // Unchanged\n expect(updated.specs.processor).toBe('Intel i5'); // Unchanged\n});\n```\n\n---\n\n## Text Search\n\n```typescript\nit('should search products by text', async () => {\n // GIVEN: Products with searchable content\n await helper.seedDocuments(productModel, [\n { name: 'Apple MacBook Pro', description: 'Powerful laptop' },\n { name: 'Samsung Galaxy', description: 'Android smartphone' },\n { name: 'Apple iPhone', description: 'iOS smartphone' },\n ]);\n\n // WHEN: Searching for \"Apple\"\n const response = await request(httpServer)\n .get('/products/search')\n .query({ q: 'Apple' })\n .expect(200);\n\n // THEN: Only Apple products returned\n expect(response.body.data).toHaveLength(2);\n expect(response.body.data.every(p => p.name.includes('Apple'))).toBe(true);\n});\n```\n\n---\n\n## Aggregation Pipeline\n\n```typescript\nit('should return sales statistics by category', async () => {\n // GIVEN: Orders with products in categories\n const electronics = await categoryModel.create({ name: 'Electronics' });\n const clothing = await categoryModel.create({ name: 'Clothing' });\n\n await productModel.insertMany([\n { name: 'Phone', categoryId: electronics._id, price: 500 },\n { name: 'Laptop', categoryId: electronics._id, price: 1000 },\n { name: 'Shirt', categoryId: clothing._id, price: 50 },\n ]);\n\n await orderItemModel.insertMany([\n { productId: 'phone-id', quantity: 2, price: 500 },\n { productId: 'laptop-id', quantity: 1, price: 1000 },\n { productId: 'shirt-id', quantity: 5, price: 50 },\n ]);\n\n // WHEN: Fetching aggregated stats\n const response = await request(httpServer)\n .get('/api/v1/reports/sales-by-category')\n .expect(200);\n\n // THEN: Aggregated statistics\n expect(response.body.data).toMatchObject({\n Electronics: { totalSales: 2000, itemCount: 3 },\n Clothing: { totalSales: 250, itemCount: 5 },\n });\n});\n```\n\n---\n\n## Pagination with Filtering\n\n```typescript\nit('should return filtered and paginated users', async () => {\n // GIVEN: Users with different statuses\n await helper.seedDocuments(userModel, [\n { email: '[email protected]', name: 'Active 1', status: 'active' },\n { email: '[email protected]', name: 'Active 2', status: 'active' },\n { email: '[email protected]', name: 'Active 3', status: 'active' },\n { email: '[email protected]', name: 'Inactive 1', status: 'inactive' },\n { email: '[email protected]', name: 'Pending 1', status: 'pending' },\n ]);\n\n // WHEN: Fetching active users, page 1\n const response = await request(httpServer)\n .get('/api/v1/users')\n .query({ status: 'active', page: 1, limit: 2, sortBy: 'name', order: 'asc' })\n .expect(200);\n\n // THEN: Correct pagination\n expect(response.body.data).toHaveLength(2);\n expect(response.body.data[0].name).toBe('Active 1');\n expect(response.body.data[1].name).toBe('Active 2');\n expect(response.body.meta).toMatchObject({\n page: 1,\n limit: 2,\n total: 3,\n totalPages: 2,\n });\n});\n```\n\n---\n\n## Unique Constraint Validation\n\n```typescript\nit('should return 409 for duplicate email', async () => {\n // GIVEN: Existing user\n await userModel.create({ email: '[email protected]', name: 'Existing' });\n\n // WHEN: Creating user with same email\n const response = await request(httpServer)\n .post('/api/v1/users')\n .send({ email: '[email protected]', name: 'New User' })\n .expect(409);\n\n // THEN: Conflict error\n expect(response.body).toMatchObject({\n code: 'EMAIL_ALREADY_EXISTS',\n message: expect.stringContaining('email'),\n });\n});\n```\n\n---\n\n## Soft Delete\n\n```typescript\nit('should soft delete user and exclude from queries', async () => {\n // GIVEN: Active user\n const user = await userModel.create({\n email: '[email protected]',\n name: 'To Be Deleted',\n deletedAt: null,\n });\n\n // WHEN: Deleting user\n await request(httpServer)\n .delete(`/api/v1/users/${user._id}`)\n .expect(204);\n\n // THEN: User has deletedAt set\n const deletedUser = await userModel.findById(user._id).lean();\n expect(deletedUser.deletedAt).not.toBeNull();\n\n // THEN: User not returned in list\n const response = await request(httpServer)\n .get('/api/v1/users')\n .expect(200);\n expect(response.body.data).toHaveLength(0);\n});\n```\n\n---\n\n## Embedded Array Operations\n\n```typescript\nit('should add item to embedded array', async () => {\n // GIVEN: Order with items\n const order = await orderModel.create({\n userId: 'user-123',\n items: [{ productId: 'prod-1', quantity: 1 }],\n status: 'pending',\n });\n\n // WHEN: Adding new item\n const response = await request(httpServer)\n .post(`/api/v1/orders/${order._id}/items`)\n .send({ productId: 'prod-2', quantity: 3 })\n .expect(200);\n\n // THEN: Item added to array\n const updated = await orderModel.findById(order._id).lean();\n expect(updated.items).toHaveLength(2);\n expect(updated.items[1]).toMatchObject({\n productId: 'prod-2',\n quantity: 3,\n });\n});\n\nit('should update item in embedded array', async () => {\n // GIVEN: Order with items\n const order = await orderModel.create({\n userId: 'user-123',\n items: [\n { productId: 'prod-1', quantity: 1 },\n { productId: 'prod-2', quantity: 2 },\n ],\n });\n\n // WHEN: Updating item quantity\n const response = await request(httpServer)\n .patch(`/api/v1/orders/${order._id}/items/prod-1`)\n .send({ quantity: 5 })\n .expect(200);\n\n // THEN: Only specified item updated\n const updated = await orderModel.findById(order._id).lean();\n expect(updated.items[0].quantity).toBe(5);\n expect(updated.items[1].quantity).toBe(2); // Unchanged\n});\n```\n\n---\n\n## Transaction Tests\n\n```typescript\ndescribe('MongoDB Transactions', () => {\n let session: ClientSession;\n\n beforeEach(async () => {\n session = await connection.startSession();\n });\n\n afterEach(async () => {\n session.endSession();\n });\n\n it('should transfer balance between users atomically', async () => {\n // GIVEN: Two users with balances\n const sender = await userModel.create({ email: '[email protected]', balance: 100 });\n const receiver = await userModel.create({ email: '[email protected]', balance: 50 });\n\n // WHEN: Transferring balance\n const response = await request(httpServer)\n .post('/api/v1/transfers')\n .send({\n fromUserId: sender._id.toString(),\n toUserId: receiver._id.toString(),\n amount: 30,\n })\n .expect(200);\n\n // THEN: Balances updated atomically\n const updatedSender = await userModel.findById(sender._id).lean();\n const updatedReceiver = await userModel.findById(receiver._id).lean();\n expect(updatedSender.balance).toBe(70);\n expect(updatedReceiver.balance).toBe(80);\n });\n\n it('should rollback on insufficient balance', async () => {\n // GIVEN: User with low balance\n const sender = await userModel.create({ email: '[email protected]', balance: 20 });\n const receiver = await userModel.create({ email: '[email protected]', balance: 50 });\n\n // WHEN: Attempting transfer exceeding balance\n const response = await request(httpServer)\n .post('/api/v1/transfers')\n .send({\n fromUserId: sender._id.toString(),\n toUserId: receiver._id.toString(),\n amount: 100,\n })\n .expect(400);\n\n // THEN: Error returned\n expect(response.body.code).toBe('INSUFFICIENT_BALANCE');\n\n // THEN: Balances unchanged\n const updatedSender = await userModel.findById(sender._id).lean();\n const updatedReceiver = await userModel.findById(receiver._id).lean();\n expect(updatedSender.balance).toBe(20);\n expect(updatedReceiver.balance).toBe(50);\n });\n});\n```\n\n---\n\n## Common Issues Handling\n\n### ObjectId Comparison\n\n```typescript\nit('should compare ObjectIds correctly', async () => {\n const category = await categoryModel.create({ name: 'Test' });\n const product = await productModel.create({\n name: 'Product',\n categoryId: category._id,\n });\n\n const found = await productModel.findById(product._id).lean();\n\n // CORRECT: Use toString()\n expect(found.categoryId.toString()).toBe(category._id.toString());\n});\n```\n\n### Virtual Fields with Lean\n\n```typescript\nit('should include virtuals in lean query', async () => {\n const user = await userModel.create({\n firstName: 'John',\n lastName: 'Doe',\n });\n\n // Use lean with virtuals option\n const found = await userModel.findById(user._id).lean({ virtuals: true });\n\n expect(found.fullName).toBe('John Doe');\n});\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":10859,"content_sha256":"87d6c30de5dc7249bea1ae11afabdce82cdef4f7fdbdb99861f9453cea9daa5d"},{"filename":"references/mongodb/knowledge.md","content":"# MongoDB E2E Testing Knowledge\n\n## Overview\n\nMongoDB E2E testing focuses on verifying document-based data persistence, schema validation, and query behaviors in a NoSQL context.\n\n## Key Concepts\n\n| Concept | Description |\n|---------|-------------|\n| Documents | JSON-like data structures with flexible schema |\n| Collections | Groups of documents (analogous to tables) |\n| ObjectId | Unique identifier for documents |\n| Embedded Documents | Nested objects within documents |\n| References | Links between documents using ObjectIds |\n| Indexes | Performance optimization structures |\n| Aggregation Pipeline | Multi-stage data processing |\n\n## MongoDB vs PostgreSQL for Testing\n\n| Aspect | MongoDB | PostgreSQL |\n|--------|---------|------------|\n| Schema | Flexible (schemaless) | Strict (defined tables) |\n| Transactions | Supported (4.0+) | Full ACID |\n| Cleanup | `deleteMany({})` | `TRUNCATE CASCADE` |\n| Relationships | References or embedding | Foreign keys |\n| Query Language | Document-based | SQL |\n| ID Type | ObjectId | UUID/SERIAL |\n\n## Test Isolation Strategies\n\n| Strategy | Speed | Complexity | Use Case |\n|----------|-------|------------|----------|\n| deleteMany | Fast | Low | Default - recommended |\n| Drop Collections | Medium | Low | Full reset needed |\n| Database Per Suite | Slow | Medium | Parallel test suites |\n| MongoDB Memory Server | Fast | Low | CI/CD without Docker |\n| Transactions | Fastest | Medium | When rollback needed |\n\n## Mongoose vs Native Driver\n\n| Feature | Mongoose | Native Driver |\n|---------|----------|---------------|\n| Schema Validation | Built-in | Manual |\n| Middleware (Hooks) | Yes | No |\n| Population | Built-in | Manual |\n| Type Safety | Schema-based | Manual |\n| Virtual Fields | Yes | No |\n| Query Builder | Chainable | Object-based |\n\n## Docker Configuration for Tests\n\n```yaml\nmongodb-e2e:\n image: mongo:7.0\n ports:\n - \"27018:27017\" # Different from dev\n environment:\n MONGO_INITDB_ROOT_USERNAME: test\n MONGO_INITDB_ROOT_PASSWORD: test\n MONGO_INITDB_DATABASE: testdb\n command: mongod --wiredTigerCacheSizeGB 1\n```\n\n## MongoDB Memory Server\n\nFor CI/CD environments without Docker or for faster local testing:\n\n```typescript\nimport { MongoMemoryServer } from 'mongodb-memory-server';\n\nlet mongod: MongoMemoryServer;\n\nexport async function setupMongoMemoryServer(): Promise\u003cstring> {\n mongod = await MongoMemoryServer.create({\n instance: { dbName: 'testdb', storageEngine: 'wiredTiger' },\n });\n return mongod.getUri();\n}\n\nexport async function teardownMongoMemoryServer(): Promise\u003cvoid> {\n if (mongod) await mongod.stop();\n}\n```\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| [rules.md](rules.md) | MongoDB-specific testing rules |\n| [test-helper.md](test-helper.md) | MongoDbTestHelper implementation |\n| [docker-setup.md](docker-setup.md) | Docker configuration |\n| [examples.md](examples.md) | Complete test examples |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2909,"content_sha256":"1f5446f94ceb55723405fb731570820c7b8956b9f8bc2faf524d096c84f79e41"},{"filename":"references/mongodb/rules.md","content":"# MongoDB E2E Testing Rules\n\n## Core Rules\n\n| Rule | Requirement |\n|------|-------------|\n| Port | Use different port from dev (e.g., 27018) |\n| Cleanup | `deleteMany({})` in beforeEach |\n| Connection | Single connection pool (`maxPoolSize: 1`) |\n| ObjectId | Compare using `.toString()` or `.equals()` |\n| Lean Queries | Use `.lean()` for read-only operations |\n\n## Cleanup Rules\n\n### deleteMany (Recommended)\n\n```typescript\nbeforeEach(async () => {\n await productModel.deleteMany({});\n await categoryModel.deleteMany({});\n await userModel.deleteMany({});\n});\n\n// Also clean after to prevent interference\nafterEach(async () => {\n await productModel.deleteMany({});\n});\n```\n\n### Clear All Collections\n\n```typescript\nbeforeEach(async () => {\n const collections = Object.keys(connection.collections);\n for (const collectionName of collections) {\n await connection.collections[collectionName].deleteMany({});\n }\n});\n```\n\n### Drop and Recreate (Full Reset)\n\n```typescript\nbeforeEach(async () => {\n const collections = await connection.db.collections();\n for (const collection of collections) {\n await collection.drop().catch(() => {\n // Ignore \"collection doesn't exist\" error\n });\n }\n});\n```\n\n## Connection Rules\n\n```typescript\n// Mongoose connection config for tests\nmongoose.connect(uri, {\n connectTimeoutMS: 30000,\n serverSelectionTimeoutMS: 30000,\n maxPoolSize: 1, // Single connection for tests\n});\n```\n\n## Assertion Rules\n\n### Assert Document Exists\n\n```typescript\n// GOOD: Assert specific values\nawait helper.assertDocumentExists(productModel, { name: 'Widget' }, {\n price: 29.99,\n stock: 100,\n});\n\n// BAD: Only check existence\nconst product = await productModel.findOne({ name: 'Widget' });\nexpect(product).toBeDefined();\n```\n\n### Assert Document Count\n\n```typescript\nawait helper.assertDocumentCount(Order, { userId: user.id }, 3);\n```\n\n### ObjectId Comparison\n\n```typescript\n// WRONG: Direct comparison\nexpect(product.categoryId).toBe(category._id);\n\n// CORRECT: Use toString()\nexpect(product.categoryId.toString()).toBe(category._id.toString());\n\n// CORRECT: Use equals()\nexpect(product.categoryId.equals(category._id)).toBe(true);\n```\n\n### Date Comparison\n\n```typescript\n// WRONG: Direct date comparison\nexpect(product.createdAt).toBe(expectedDate);\n\n// CORRECT: Compare timestamps\nexpect(product.createdAt.getTime()).toBe(expectedDate.getTime());\n\n// CORRECT: Use toISOString\nexpect(product.createdAt.toISOString()).toBe(expectedDate.toISOString());\n```\n\n## Query Rules\n\n### Use Lean Queries for Assertions\n\n```typescript\n// SLOW: Returns Mongoose documents\nconst products = await productModel.find();\n\n// FAST: Returns plain objects\nconst products = await productModel.find().lean();\n```\n\n### Use Projection for Needed Fields\n\n```typescript\n// SLOW: Fetch all fields\nconst user = await userModel.findById(id);\n\n// FAST: Only fetch needed fields\nconst user = await userModel.findById(id).select('email name');\n```\n\n## Performance Rules\n\n### Bulk Operations\n\n```typescript\n// SLOW: Individual inserts\nfor (const product of products) {\n await productModel.create(product);\n}\n\n// FAST: insertMany\nawait productModel.insertMany(products);\n\n// FASTEST: Unordered bulk\nawait productModel.insertMany(products, { ordered: false });\n```\n\n### Index Management\n\n```typescript\n// Ensure indexes once in beforeAll\nbeforeAll(async () => {\n await productModel.ensureIndexes();\n await categoryModel.ensureIndexes();\n});\n```\n\n## Transaction Rules (MongoDB 4.0+)\n\n```typescript\ndescribe('with transactions', () => {\n let session: ClientSession;\n\n beforeEach(async () => {\n session = await connection.startSession();\n session.startTransaction();\n });\n\n afterEach(async () => {\n await session.abortTransaction();\n session.endSession();\n });\n\n it('should rollback changes', async () => {\n await productModel.create([{ name: 'Test' }], { session });\n // Transaction will be rolled back after test\n });\n});\n```\n\n## Checklist\n\n**Setup:**\n- [ ] Different port from dev (e.g., 27018)\n- [ ] WiredTiger cache size limited\n- [ ] Single connection pool (`maxPoolSize: 1`)\n\n**Cleanup:**\n- [ ] `deleteMany({})` in beforeEach\n- [ ] Optional afterEach cleanup\n- [ ] Index recreation if using drop\n\n**Assertions:**\n- [ ] Use `.lean()` for read operations\n- [ ] Use `.toString()` for ObjectId comparison\n- [ ] Use `.getTime()` for Date comparison\n- [ ] Assert specific values, not just existence\n\n**Performance:**\n- [ ] Use `insertMany` for seeding\n- [ ] Use projection for needed fields only\n- [ ] Ensure indexes in beforeAll\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4559,"content_sha256":"670904b96dea0fe90f5ca1c7cae2d08b27e7600e9cc5eb5373135de1f3db8542"},{"filename":"references/mongodb/test-helper.md","content":"# MongoDB Test Helper\n\n## Complete Implementation\n\n```typescript\n// test/helpers/mongodb.helper.ts\nimport { Connection, Model, Document } from 'mongoose';\n\nexport class MongoDbTestHelper {\n constructor(private connection: Connection) {}\n\n /**\n * Clear all collections\n */\n async clearAllCollections(): Promise\u003cvoid> {\n const collections = Object.keys(this.connection.collections);\n for (const collectionName of collections) {\n const collection = this.connection.collections[collectionName];\n await collection.deleteMany({});\n }\n }\n\n /**\n * Clear specific collections\n */\n async clearCollections(collectionNames: string[]): Promise\u003cvoid> {\n for (const name of collectionNames) {\n const collection = this.connection.collections[name];\n if (collection) {\n await collection.deleteMany({});\n }\n }\n }\n\n /**\n * Drop entire database\n */\n async dropDatabase(): Promise\u003cvoid> {\n await this.connection.dropDatabase();\n }\n\n /**\n * Get model by name\n */\n getModel\u003cT>(name: string): Model\u003cT> {\n return this.connection.model\u003cT>(name);\n }\n\n /**\n * Assert document exists with expected values\n */\n async assertDocumentExists(\n model: Model\u003cany>,\n query: object,\n expected?: object\n ): Promise\u003cvoid> {\n const doc = await model.findOne(query).lean().exec();\n expect(doc).toBeDefined();\n expect(doc).not.toBeNull();\n if (expected) {\n expect(doc).toMatchObject(expected);\n }\n }\n\n /**\n * Assert document does not exist\n */\n async assertDocumentNotExists(\n model: Model\u003cany>,\n query: object\n ): Promise\u003cvoid> {\n const doc = await model.findOne(query).exec();\n expect(doc).toBeNull();\n }\n\n /**\n * Assert document count\n */\n async assertDocumentCount(\n model: Model\u003cany>,\n query: object,\n expectedCount: number\n ): Promise\u003cvoid> {\n const count = await model.countDocuments(query).exec();\n expect(count).toBe(expectedCount);\n }\n\n /**\n * Seed documents for testing\n */\n async seedDocuments\u003cT>(model: Model\u003cT>, documents: Partial\u003cT>[]): Promise\u003cT[]> {\n return model.insertMany(documents) as unknown as T[];\n }\n\n /**\n * Create indexes for test collections\n */\n async ensureIndexes(): Promise\u003cvoid> {\n const models = this.connection.modelNames();\n for (const modelName of models) {\n const model = this.connection.model(modelName);\n await model.ensureIndexes();\n }\n }\n}\n```\n\n---\n\n## Usage Example\n\n```typescript\ndescribe('Product API E2E', () => {\n let helper: MongoDbTestHelper;\n let productModel: Model\u003cProduct>;\n let categoryModel: Model\u003cCategory>;\n\n beforeAll(async () => {\n const setup = await createTestApp([ProductModule]);\n helper = new MongoDbTestHelper(setup.app.get(getConnectionToken()));\n productModel = helper.getModel\u003cProduct>('Product');\n categoryModel = helper.getModel\u003cCategory>('Category');\n });\n\n beforeEach(async () => {\n // Wait for in-flight operations\n await new Promise(r => setTimeout(r, 500));\n await helper.clearCollections(['products', 'categories']);\n });\n\n afterAll(async () => {\n await app.close();\n });\n\n it('should create product', async () => {\n // GIVEN: Category exists\n const category = await categoryModel.create({ name: 'Electronics' });\n\n // WHEN: Creating product\n const response = await request(httpServer)\n .post('/products')\n .send({\n name: 'Laptop',\n categoryId: category._id.toString(),\n price: 999.99,\n })\n .expect(201);\n\n // THEN: Product persisted\n await helper.assertDocumentExists(\n productModel,\n { name: 'Laptop' },\n { categoryId: category._id, price: 999.99 }\n );\n });\n});\n```\n\n---\n\n## API Reference\n\n| Method | Description |\n|--------|-------------|\n| `clearAllCollections()` | Delete all documents from all collections |\n| `clearCollections(names)` | Delete documents from specified collections |\n| `dropDatabase()` | Drop the entire test database |\n| `getModel(name)` | Get Mongoose model by name |\n| `assertDocumentExists(model, query, expected)` | Assert document exists with values |\n| `assertDocumentNotExists(model, query)` | Assert document does not exist |\n| `assertDocumentCount(model, query, count)` | Assert document count |\n| `seedDocuments(model, docs)` | Insert test documents |\n| `ensureIndexes()` | Ensure indexes on all models |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4382,"content_sha256":"4c09c41b92e10eab8492627c2b85a0b639c1c90ab1b4b1fec187b7c5b3bd5346"},{"filename":"references/postgres/examples.md","content":"# PostgreSQL E2E Test Examples\n\n## Basic CRUD Operations\n\n### Create with Validation\n\n```typescript\nit('should create user with hashed password', async () => {\n // GIVEN: Valid user data\n const createUserDto = {\n email: '[email protected]',\n name: 'New User',\n password: 'SecurePass123!',\n };\n\n // WHEN: Creating user\n const response = await request(httpServer)\n .post('/api/v1/users')\n .send(createUserDto)\n .expect(201);\n\n // THEN: User created with correct data\n expect(response.body.data).toMatchObject({\n id: expect.any(String),\n email: '[email protected]',\n name: 'New User',\n });\n expect(response.body.data.password).toBeUndefined(); // Not exposed\n\n // THEN: Password hashed in database\n const savedUser = await helper.assertRecordExists(User, {\n email: '[email protected]',\n });\n expect(savedUser.password).not.toBe('SecurePass123!');\n expect(savedUser.password).toMatch(/^\\$2[aby]\\$.{56}$/); // bcrypt format\n});\n```\n\n### Read with Relations\n\n```typescript\nit('should return order with user and items', async () => {\n // GIVEN: User, products, and order\n const user = await helper.seedData(User, [{\n email: '[email protected]',\n name: 'Buyer',\n }]);\n const products = await helper.seedData(Product, [\n { name: 'Widget', price: 29.99 },\n { name: 'Gadget', price: 49.99 },\n ]);\n const order = await helper.seedData(Order, [{\n userId: user[0].id,\n status: 'completed',\n }]);\n await helper.seedData(OrderItem, [\n { orderId: order[0].id, productId: products[0].id, quantity: 2, price: 29.99 },\n { orderId: order[0].id, productId: products[1].id, quantity: 1, price: 49.99 },\n ]);\n\n // WHEN: Fetching order with relations\n const response = await request(httpServer)\n .get(`/api/v1/orders/${order[0].id}`)\n .query({ include: 'user,items.product' })\n .expect(200);\n\n // THEN: Full order with relations\n expect(response.body.data).toMatchObject({\n id: order[0].id,\n status: 'completed',\n user: {\n id: user[0].id,\n email: '[email protected]',\n },\n items: expect.arrayContaining([\n expect.objectContaining({\n productId: products[0].id,\n quantity: 2,\n product: { name: 'Widget' },\n }),\n ]),\n total: 109.97,\n });\n});\n```\n\n---\n\n## Transaction Tests\n\n### Successful Transaction\n\n```typescript\nit('should transfer balance between users in transaction', async () => {\n // GIVEN: Two users with balances\n const [sender, receiver] = await helper.seedData(User, [\n { email: '[email protected]', balance: 100 },\n { email: '[email protected]', balance: 50 },\n ]);\n\n // WHEN: Transferring balance\n const response = await request(httpServer)\n .post('/api/v1/transfers')\n .send({\n fromUserId: sender.id,\n toUserId: receiver.id,\n amount: 30,\n })\n .expect(200);\n\n // THEN: Balances updated atomically\n await helper.assertRecordExists(User, { id: sender.id }, { balance: 70 });\n await helper.assertRecordExists(User, { id: receiver.id }, { balance: 80 });\n\n // THEN: Transfer record created\n await helper.assertRecordExists(Transfer, {\n fromUserId: sender.id,\n toUserId: receiver.id,\n }, {\n amount: 30,\n status: 'completed',\n });\n});\n```\n\n### Transaction Rollback on Failure\n\n```typescript\nit('should rollback order on payment failure', async () => {\n // GIVEN: User with balance\n const user = await helper.seedData(User, [{\n email: '[email protected]',\n balance: 100,\n }]);\n const product = await helper.seedData(Product, [{\n name: 'Expensive Item',\n price: 150, // More than balance\n stock: 10,\n }]);\n\n // WHEN: Creating order (should fail)\n const response = await request(httpServer)\n .post('/api/v1/orders')\n .send({\n userId: user[0].id,\n items: [{ productId: product[0].id, quantity: 1 }],\n })\n .expect(400);\n\n // THEN: Error returned\n expect(response.body.code).toBe('INSUFFICIENT_BALANCE');\n\n // THEN: No order created\n await helper.assertRecordCount(Order, { userId: user[0].id }, 0);\n\n // THEN: Stock unchanged\n await helper.assertRecordExists(Product, { id: product[0].id }, { stock: 10 });\n\n // THEN: Balance unchanged\n await helper.assertRecordExists(User, { id: user[0].id }, { balance: 100 });\n});\n```\n\n---\n\n## Complex Queries\n\n### Pagination with Filtering\n\n```typescript\nit('should return filtered and paginated users', async () => {\n // GIVEN: Users with different statuses\n await helper.seedData(User, [\n { email: '[email protected]', name: 'Active 1', status: 'active' },\n { email: '[email protected]', name: 'Active 2', status: 'active' },\n { email: '[email protected]', name: 'Active 3', status: 'active' },\n { email: '[email protected]', name: 'Inactive 1', status: 'inactive' },\n { email: '[email protected]', name: 'Pending 1', status: 'pending' },\n ]);\n\n // WHEN: Fetching active users, page 1\n const response = await request(httpServer)\n .get('/api/v1/users')\n .query({ status: 'active', page: 1, limit: 2, sortBy: 'name', order: 'ASC' })\n .expect(200);\n\n // THEN: Correct pagination\n expect(response.body.data).toHaveLength(2);\n expect(response.body.data[0].name).toBe('Active 1');\n expect(response.body.data[1].name).toBe('Active 2');\n expect(response.body.meta).toMatchObject({\n page: 1,\n limit: 2,\n total: 3,\n totalPages: 2,\n });\n});\n```\n\n### Aggregation Query\n\n```typescript\nit('should return sales statistics by category', async () => {\n // GIVEN: Orders with products in categories\n const electronics = await helper.seedData(Category, [{ name: 'Electronics' }]);\n const clothing = await helper.seedData(Category, [{ name: 'Clothing' }]);\n\n await helper.seedData(Product, [\n { name: 'Phone', categoryId: electronics[0].id, price: 500 },\n { name: 'Laptop', categoryId: electronics[0].id, price: 1000 },\n { name: 'Shirt', categoryId: clothing[0].id, price: 50 },\n ]);\n\n await helper.seedData(OrderItem, [\n { productId: 1, quantity: 2, price: 500 }, // 1000\n { productId: 2, quantity: 1, price: 1000 }, // 1000\n { productId: 3, quantity: 5, price: 50 }, // 250\n ]);\n\n // WHEN: Fetching sales by category\n const response = await request(httpServer)\n .get('/api/v1/reports/sales-by-category')\n .expect(200);\n\n // THEN: Aggregated statistics\n expect(response.body.data).toMatchObject({\n Electronics: { totalSales: 2000, itemCount: 3 },\n Clothing: { totalSales: 250, itemCount: 5 },\n });\n});\n```\n\n---\n\n## Constraint Validation\n\n### Unique Constraint\n\n```typescript\nit('should return 409 for duplicate email', async () => {\n // GIVEN: Existing user\n await helper.seedData(User, [{ email: '[email protected]', name: 'Existing' }]);\n\n // WHEN: Creating user with same email\n const response = await request(httpServer)\n .post('/api/v1/users')\n .send({ email: '[email protected]', name: 'New User' })\n .expect(409);\n\n // THEN: Conflict error\n expect(response.body).toMatchObject({\n code: 'EMAIL_ALREADY_EXISTS',\n message: expect.stringContaining('email'),\n });\n});\n```\n\n### Foreign Key Constraint\n\n```typescript\nit('should return 404 for non-existent category', async () => {\n // GIVEN: No categories exist\n\n // WHEN: Creating product with non-existent category\n const response = await request(httpServer)\n .post('/api/v1/products')\n .send({\n name: 'Test Product',\n categoryId: 'non-existent-uuid',\n price: 99.99,\n })\n .expect(404);\n\n // THEN: Category not found error\n expect(response.body.code).toBe('CATEGORY_NOT_FOUND');\n});\n```\n\n### Check Constraint\n\n```typescript\nit('should return 400 for negative price', async () => {\n // GIVEN: Category exists\n const category = await helper.seedData(Category, [{ name: 'Test' }]);\n\n // WHEN: Creating product with negative price\n const response = await request(httpServer)\n .post('/api/v1/products')\n .send({\n name: 'Test Product',\n categoryId: category[0].id,\n price: -10,\n })\n .expect(400);\n\n // THEN: Validation error\n expect(response.body.code).toBe('VALIDATION_ERROR');\n expect(response.body.errors).toContainEqual(\n expect.objectContaining({ field: 'price' })\n );\n});\n```\n\n---\n\n## Concurrent Updates\n\n### Optimistic Locking\n\n```typescript\nit('should handle concurrent updates with optimistic locking', async () => {\n // GIVEN: Product with stock\n const product = await helper.seedData(Product, [{\n name: 'Limited Item',\n stock: 1,\n version: 1,\n }]);\n\n // WHEN: Two concurrent purchase attempts\n const [response1, response2] = await Promise.all([\n request(httpServer).post('/api/v1/orders').send({\n items: [{ productId: product[0].id, quantity: 1 }],\n }),\n request(httpServer).post('/api/v1/orders').send({\n items: [{ productId: product[0].id, quantity: 1 }],\n }),\n ]);\n\n // THEN: One succeeds, one fails\n const statuses = [response1.status, response2.status].sort();\n expect(statuses).toEqual([201, 409]);\n\n // THEN: Only one order created\n await helper.assertRecordCount(Order, {}, 1);\n\n // THEN: Stock is zero\n await helper.assertRecordExists(Product, { id: product[0].id }, { stock: 0 });\n});\n```\n\n---\n\n## Soft Delete\n\n```typescript\nit('should soft delete user and exclude from queries', async () => {\n // GIVEN: Active user\n const user = await helper.seedData(User, [{\n email: '[email protected]',\n name: 'To Be Deleted',\n deletedAt: null,\n }]);\n\n // WHEN: Deleting user\n await request(httpServer)\n .delete(`/api/v1/users/${user[0].id}`)\n .expect(204);\n\n // THEN: User has deletedAt set\n const deletedUser = await helper.executeQuery(\n 'SELECT * FROM users WHERE id = $1',\n [user[0].id]\n );\n expect(deletedUser[0].deleted_at).not.toBeNull();\n\n // THEN: User not returned in list\n const response = await request(httpServer)\n .get('/api/v1/users')\n .expect(200);\n expect(response.body.data).toHaveLength(0);\n});\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":9895,"content_sha256":"5d326284314ba9bcfd90fd59f8d03e6041aa852bf5a1c95452cfb0c31a407922"},{"filename":"references/postgres/knowledge.md","content":"# PostgreSQL E2E Testing Knowledge\n\n## Overview\n\nPostgreSQL E2E testing focuses on verifying data persistence, transactions, and complex query behaviors in a relational database context.\n\n## Key Concepts\n\n| Concept | Description |\n|---------|-------------|\n| ACID Transactions | Atomicity, Consistency, Isolation, Durability |\n| Foreign Keys | Referential integrity constraints |\n| Indexes | Performance optimization structures |\n| Sequences | Auto-incrementing ID generators |\n| Schemas | Logical separation of tables |\n\n## Test Isolation Strategies\n\n| Strategy | Speed | Complexity | Use Case |\n|----------|-------|------------|----------|\n| TRUNCATE CASCADE | Fast | Low | Default - recommended |\n| Transaction Rollback | Fastest | Medium | When no commit needed |\n| Schema Per Suite | Medium | Medium | Parallel test suites |\n| Testcontainers | Slow | Low | Complete isolation |\n\n## PostgreSQL-Specific Challenges\n\n| Challenge | Solution |\n|-----------|----------|\n| Foreign key constraints | Clean in correct order or use CASCADE |\n| Sequence reset | `ALTER SEQUENCE ... RESTART WITH 1` |\n| Connection pool exhaustion | Single connection for tests |\n| Deadlocks in parallel tests | Run sequentially (`--runInBand`) |\n\n## TypeORM vs Prisma\n\n| Feature | TypeORM | Prisma |\n|---------|---------|--------|\n| Query Builder | Yes | No (raw SQL) |\n| Migrations | Built-in | Separate tool |\n| Transaction API | `QueryRunner` | `$transaction` |\n| Table Truncation | `synchronize(true)` | `$executeRaw` |\n\n## Docker Configuration for Tests\n\n```yaml\npostgres-e2e:\n image: postgres:15-alpine\n ports:\n - \"5433:5432\" # Different from dev\n environment:\n POSTGRES_USER: test\n POSTGRES_PASSWORD: test\n POSTGRES_DB: testdb\n # Performance tuning for tests\n command: >\n postgres\n -c shared_buffers=256MB\n -c fsync=off\n -c synchronous_commit=off\n -c full_page_writes=off\n```\n\n## Performance Settings Explained\n\n| Setting | Default | Test Value | Effect |\n|---------|---------|------------|--------|\n| `fsync` | on | off | Skip disk sync (faster) |\n| `synchronous_commit` | on | off | Async commits (faster) |\n| `full_page_writes` | on | off | Skip WAL writes (faster) |\n| `shared_buffers` | 128MB | 256MB | More memory cache |\n\n⚠️ **Warning:** These settings sacrifice durability for speed. Only use for tests.\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| [rules.md](rules.md) | PostgreSQL-specific testing rules |\n| [test-helper.md](test-helper.md) | PostgresTestHelper implementation |\n| [docker-setup.md](docker-setup.md) | Docker configuration |\n| [examples.md](examples.md) | Complete test examples |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2637,"content_sha256":"95dc5c227e8629e77447146e4627ddd6383e31b6ac1156fb110ea100b0e79bcb"},{"filename":"references/postgres/rules.md","content":"# PostgreSQL E2E Testing Rules\n\n## Core Rules\n\n| Rule | Requirement |\n|------|-------------|\n| Port | Use different port from dev (e.g., 5433) |\n| Cleanup | TRUNCATE with CASCADE in beforeEach |\n| Cleanup order | Children before parents (FK constraints) |\n| Sequences | Reset after TRUNCATE if predictable IDs needed |\n| Connections | Single connection for tests (`max: 1`) |\n\n## Cleanup Rules\n\n### TRUNCATE with CASCADE (Recommended)\n\n```typescript\nbeforeEach(async () => {\n await dataSource.query('SET session_replication_role = replica');\n await dataSource.query('TRUNCATE users, orders, products CASCADE');\n await dataSource.query('SET session_replication_role = DEFAULT');\n});\n```\n\n### Manual Order (Without CASCADE)\n\n```typescript\nbeforeEach(async () => {\n // Children first\n await orderItemRepository.delete({});\n await orderRepository.delete({});\n // Parents last\n await userRepository.delete({});\n await productRepository.delete({});\n});\n```\n\n### Sequence Reset\n\n```typescript\nafterEach(async () => {\n await dataSource.query(`ALTER SEQUENCE users_id_seq RESTART WITH 1`);\n await dataSource.query(`ALTER SEQUENCE orders_id_seq RESTART WITH 1`);\n});\n```\n\n## Connection Rules\n\n```typescript\n// TypeORM DataSource config for tests\n{\n type: 'postgres',\n url: process.env.DATABASE_URL,\n extra: {\n max: 1, // Single connection for tests\n },\n synchronize: false, // Don't auto-sync in tests\n}\n```\n\n## Transaction Rules\n\n### Transaction Rollback Pattern\n\n```typescript\ndescribe('with transaction rollback', () => {\n let queryRunner: QueryRunner;\n\n beforeEach(async () => {\n queryRunner = dataSource.createQueryRunner();\n await queryRunner.connect();\n await queryRunner.startTransaction();\n });\n\n afterEach(async () => {\n await queryRunner.rollbackTransaction();\n await queryRunner.release();\n });\n\n it('should not persist data', async () => {\n // Data created here will be rolled back\n });\n});\n```\n\n## Assertion Rules\n\n### Assert Record Exists\n\n```typescript\n// ✅ GOOD: Assert specific values\nawait helper.assertRecordExists(User, { email: '[email protected]' }, {\n name: 'Test User',\n status: 'active',\n});\n\n// ❌ BAD: Only check existence\nconst user = await userRepository.findOne({ where: { email: '[email protected]' } });\nexpect(user).toBeDefined();\n```\n\n### Assert Record Count\n\n```typescript\nawait helper.assertRecordCount(Order, { userId: user.id }, 3);\n```\n\n### Assert Record Not Exists\n\n```typescript\nawait helper.assertRecordNotExists(User, { email: '[email protected]' });\n```\n\n## Query Rules\n\n### Use Repository Methods\n\n```typescript\n// ✅ GOOD: Repository methods\nconst users = await userRepository.find({\n where: { status: 'active' },\n order: { createdAt: 'DESC' },\n take: 10,\n});\n\n// ❌ BAD: Raw SQL (unless necessary)\nconst users = await dataSource.query('SELECT * FROM users WHERE status = $1', ['active']);\n```\n\n### Use QueryBuilder for Complex Queries\n\n```typescript\nconst result = await userRepository\n .createQueryBuilder('user')\n .leftJoinAndSelect('user.orders', 'order')\n .where('user.status = :status', { status: 'active' })\n .andWhere('order.total > :total', { total: 100 })\n .getMany();\n```\n\n## Performance Rules\n\n### Batch Operations\n\n```typescript\n// ✅ GOOD: Batch insert\nawait userRepository.save(users);\n\n// ❌ BAD: Individual inserts\nfor (const user of users) {\n await userRepository.save(user);\n}\n```\n\n### Index Management for Large Data\n\n```typescript\n// Disable triggers during bulk load\nbeforeAll(async () => {\n await dataSource.query(`ALTER TABLE users DISABLE TRIGGER ALL`);\n});\n\nafterAll(async () => {\n await dataSource.query(`ALTER TABLE users ENABLE TRIGGER ALL`);\n await dataSource.query(`REINDEX TABLE users`);\n});\n```\n\n## Error Handling Rules\n\n### Foreign Key Constraint Errors\n\n```typescript\nit('should return error when deleting user with orders', async () => {\n // GIVEN: User with orders\n const user = await userRepository.save({ email: '[email protected]' });\n await orderRepository.save({ userId: user.id, total: 100 });\n\n // WHEN: Attempting to delete user\n const response = await request(httpServer)\n .delete(`/api/v1/users/${user.id}`)\n .expect(409);\n\n // THEN: Conflict error\n expect(response.body.code).toBe('USER_HAS_ORDERS');\n});\n```\n\n### Unique Constraint Errors\n\n```typescript\nit('should return error for duplicate email', async () => {\n // GIVEN: Existing user\n await userRepository.save({ email: '[email protected]', name: 'Existing' });\n\n // WHEN: Creating user with same email\n const response = await request(httpServer)\n .post('/api/v1/users')\n .send({ email: '[email protected]', name: 'New' })\n .expect(409);\n\n // THEN: Duplicate error\n expect(response.body.code).toBe('EMAIL_ALREADY_EXISTS');\n});\n```\n\n## Checklist\n\n**Setup:**\n- [ ] Different port from dev (e.g., 5433)\n- [ ] Performance settings enabled (fsync=off, etc.)\n- [ ] Single connection pool (`max: 1`)\n\n**Cleanup:**\n- [ ] TRUNCATE with CASCADE in beforeEach\n- [ ] Sequence reset if needed\n- [ ] Use `session_replication_role = replica` for CASCADE\n\n**Assertions:**\n- [ ] Assert specific values, not just existence\n- [ ] Verify all expected fields\n- [ ] Check record counts where applicable\n\n**Performance:**\n- [ ] Use batch operations for seeding\n- [ ] Avoid N+1 queries in test setup\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5316,"content_sha256":"458d23609b93910a6759a8caafb499c5f9547ba3456f2c2a784343dbe7a778e4"},{"filename":"references/postgres/test-helper.md","content":"# PostgreSQL Test Helper\n\n## Complete Implementation\n\n```typescript\n// test/helpers/postgres.helper.ts\nimport { DataSource, EntityTarget, FindOptionsWhere, DeepPartial } from 'typeorm';\n\nexport class PostgresTestHelper {\n constructor(private dataSource: DataSource) {}\n\n /**\n * Clear all tables with CASCADE to handle FK constraints.\n */\n async clearTables(tables: string[]): Promise\u003cvoid> {\n await this.dataSource.query('SET session_replication_role = replica');\n for (const table of tables) {\n await this.dataSource.query(`TRUNCATE TABLE \"${table}\" CASCADE`);\n }\n await this.dataSource.query('SET session_replication_role = DEFAULT');\n }\n\n /**\n * Clear all tables in the database (except migrations).\n */\n async clearAllTables(): Promise\u003cvoid> {\n const tables = await this.dataSource.query(`\n SELECT tablename FROM pg_tables\n WHERE schemaname = 'public' AND tablename != 'migrations'\n `);\n const tableNames = tables.map(t => t.tablename);\n await this.clearTables(tableNames);\n }\n\n /**\n * Assert record exists with expected values.\n */\n async assertRecordExists\u003cT>(\n entity: EntityTarget\u003cT>,\n where: FindOptionsWhere\u003cT>,\n expected?: Partial\u003cT>\n ): Promise\u003cT> {\n const repository = this.dataSource.getRepository(entity);\n const record = await repository.findOne({ where });\n\n expect(record).toBeDefined();\n expect(record).not.toBeNull();\n\n if (expected) {\n expect(record).toMatchObject(expected);\n }\n return record!;\n }\n\n /**\n * Assert record does not exist.\n */\n async assertRecordNotExists\u003cT>(\n entity: EntityTarget\u003cT>,\n where: FindOptionsWhere\u003cT>\n ): Promise\u003cvoid> {\n const repository = this.dataSource.getRepository(entity);\n const record = await repository.findOne({ where });\n expect(record).toBeNull();\n }\n\n /**\n * Assert count of records.\n */\n async assertRecordCount\u003cT>(\n entity: EntityTarget\u003cT>,\n where: FindOptionsWhere\u003cT>,\n expectedCount: number\n ): Promise\u003cvoid> {\n const repository = this.dataSource.getRepository(entity);\n const count = await repository.count({ where });\n expect(count).toBe(expectedCount);\n }\n\n /**\n * Seed data for testing.\n */\n async seedData\u003cT>(entity: EntityTarget\u003cT>, data: Partial\u003cT>[]): Promise\u003cT[]> {\n const repository = this.dataSource.getRepository(entity);\n return repository.save(data as DeepPartial\u003cT>[]);\n }\n\n /**\n * Reset sequences for predictable IDs.\n */\n async resetSequences(): Promise\u003cvoid> {\n const sequences = await this.dataSource.query(`\n SELECT sequence_name FROM information_schema.sequences\n WHERE sequence_schema = 'public'\n `);\n for (const { sequence_name } of sequences) {\n await this.dataSource.query(`ALTER SEQUENCE \"${sequence_name}\" RESTART WITH 1`);\n }\n }\n\n /**\n * Execute raw SQL query.\n */\n async executeQuery\u003cT = any>(sql: string, parameters?: any[]): Promise\u003cT> {\n return this.dataSource.query(sql, parameters);\n }\n\n /**\n * Begin transaction for test isolation.\n */\n async beginTransaction(): Promise\u003cvoid> {\n await this.dataSource.query('BEGIN');\n }\n\n /**\n * Rollback transaction.\n */\n async rollbackTransaction(): Promise\u003cvoid> {\n await this.dataSource.query('ROLLBACK');\n }\n\n /**\n * Get repository for entity.\n */\n getRepository\u003cT>(entity: EntityTarget\u003cT>) {\n return this.dataSource.getRepository(entity);\n }\n}\n```\n\n---\n\n## Usage Example\n\n```typescript\ndescribe('User API E2E', () => {\n let helper: PostgresTestHelper;\n let userRepository: Repository\u003cUser>;\n\n beforeAll(async () => {\n const setup = await createTestApp([UserModule]);\n helper = new PostgresTestHelper(setup.app.get(DataSource));\n userRepository = helper.getRepository(User);\n });\n\n beforeEach(async () => {\n await helper.clearTables(['users', 'orders', 'products']);\n await helper.resetSequences();\n });\n\n it('should create user', async () => {\n // GIVEN: No users exist\n\n // WHEN: Creating user\n const response = await request(httpServer)\n .post('/users')\n .send({ email: '[email protected]', name: 'Test' })\n .expect(201);\n\n // THEN: User persisted correctly\n await helper.assertRecordExists(User, { email: '[email protected]' }, {\n name: 'Test',\n status: 'active',\n });\n });\n\n it('should return users with pagination', async () => {\n // GIVEN: Multiple users exist\n await helper.seedData(User, [\n { email: '[email protected]', name: 'User 1' },\n { email: '[email protected]', name: 'User 2' },\n { email: '[email protected]', name: 'User 3' },\n ]);\n\n // WHEN: Fetching with pagination\n const response = await request(httpServer)\n .get('/users')\n .query({ page: 1, limit: 2 })\n .expect(200);\n\n // THEN: Paginated results returned\n expect(response.body.data).toHaveLength(2);\n expect(response.body.meta.total).toBe(3);\n });\n});\n```\n\n---\n\n## API Reference\n\n| Method | Description |\n|--------|-------------|\n| `clearTables(tables)` | TRUNCATE specified tables with CASCADE |\n| `clearAllTables()` | TRUNCATE all tables except migrations |\n| `assertRecordExists(entity, where, expected)` | Assert record exists with values |\n| `assertRecordNotExists(entity, where)` | Assert record does not exist |\n| `assertRecordCount(entity, where, count)` | Assert record count |\n| `seedData(entity, data)` | Insert test data |\n| `resetSequences()` | Reset all sequences to 1 |\n| `executeQuery(sql, params)` | Execute raw SQL |\n| `beginTransaction()` | Start transaction |\n| `rollbackTransaction()` | Rollback transaction |\n| `getRepository(entity)` | Get TypeORM repository |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5645,"content_sha256":"2112d5015a5bdabe9e370c710e8a9ba8d8092b1c8910dab3afaadcb139a8a369"},{"filename":"references/redis/docker-setup.md","content":"# Redis Docker Setup for E2E Testing\n\n## docker-compose.e2e.yml\n\n```yaml\nversion: '3.8'\n\nservices:\n redis-e2e:\n image: redis:7-alpine\n container_name: redis-e2e\n ports:\n - \"6380:6379\" # Different port from dev\n healthcheck:\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n interval: 5s\n timeout: 3s\n retries: 5\n # Performance: Disable persistence for tests\n command: redis-server --save \"\" --appendonly no --maxmemory 100mb --maxmemory-policy allkeys-lru\n\nvolumes:\n redis-e2e-data:\n```\n\n---\n\n## Health Check Script\n\n```bash\n#!/bin/bash\n# scripts/wait-for-redis.sh\n\nuntil docker-compose -f docker-compose.e2e.yml exec redis-e2e redis-cli ping; do\n echo \"Waiting for Redis...\"\n sleep 1\ndone\necho \"Redis is ready!\"\n```\n\n---\n\n## Connection String\n\n```bash\n# .env.e2e\nREDIS_URL=redis://localhost:6380\n```\n\n---\n\n## Performance Tuning\n\n### Optimized for Test Speed\n\n```yaml\ncommand: >\n redis-server\n --save \"\" # Disable RDB snapshots\n --appendonly no # Disable AOF\n --maxmemory 100mb # Limit memory\n --maxmemory-policy allkeys-lru # Evict old keys\n --tcp-backlog 128 # Smaller backlog for tests\n```\n\n### Settings Explained\n\n| Setting | Default | Test Value | Effect |\n|---------|---------|------------|--------|\n| `save \"\"` | periodic | disabled | Skip RDB snapshots |\n| `appendonly` | yes | no | Skip AOF persistence |\n| `maxmemory` | unlimited | 100mb | Limit memory usage |\n| `maxmemory-policy` | noeviction | allkeys-lru | Evict old keys |\n| `tcp-backlog` | 511 | 128 | Smaller backlog for tests |\n\n**Warning:** These settings sacrifice persistence for speed. Only use for tests.\n\n---\n\n## Connection Configuration\n\n### ioredis Configuration\n\n```typescript\nimport Redis from 'ioredis';\n\nconst redis = new Redis({\n host: 'localhost',\n port: 6380,\n db: 0,\n lazyConnect: true,\n retryDelayOnClusterDown: 100,\n retryDelayOnFailover: 100,\n maxRetriesPerRequest: 3,\n});\n```\n\n### node-redis Configuration\n\n```typescript\nimport { createClient } from 'redis';\n\nconst client = createClient({\n url: 'redis://localhost:6380',\n});\n\nawait client.connect();\n```\n\n---\n\n## Complete Docker Compose with All Services\n\n```yaml\nversion: '3.8'\n\nservices:\n redis-e2e:\n image: redis:7-alpine\n container_name: redis-e2e\n ports:\n - \"6380:6379\"\n healthcheck:\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n interval: 5s\n timeout: 3s\n retries: 5\n command: redis-server --save \"\" --appendonly no --maxmemory 100mb\n\n # Add other services as needed\n mongodb-e2e:\n image: mongo:7.0\n ports:\n - \"27018:27017\"\n command: mongod --wiredTigerCacheSizeGB 1\n\n postgres-e2e:\n image: postgres:15-alpine\n ports:\n - \"5433:5432\"\n environment:\n POSTGRES_USER: test\n POSTGRES_PASSWORD: test\n POSTGRES_DB: testdb\n```\n\n---\n\n## Redis with Sentinel (High Availability)\n\nFor testing Sentinel-based failover:\n\n```yaml\nservices:\n redis-master:\n image: redis:7-alpine\n ports:\n - \"6380:6379\"\n command: redis-server --save \"\" --appendonly no\n\n redis-sentinel:\n image: redis:7-alpine\n ports:\n - \"26379:26379\"\n depends_on:\n - redis-master\n command: >\n sh -c 'echo \"sentinel monitor mymaster redis-master 6379 1\" > /tmp/sentinel.conf &&\n echo \"sentinel down-after-milliseconds mymaster 5000\" >> /tmp/sentinel.conf &&\n redis-sentinel /tmp/sentinel.conf'\n```\n\n---\n\n## Redis Cluster (Sharding)\n\nFor testing cluster mode:\n\n```yaml\nservices:\n redis-cluster:\n image: grokzen/redis-cluster:7.0.0\n ports:\n - \"7000-7005:7000-7005\"\n environment:\n IP: 0.0.0.0\n INITIAL_PORT: 7000\n MASTERS: 3\n SLAVES_PER_MASTER: 0\n```\n\n**Note:** Cluster mode requires more complex setup and is typically not needed for most E2E tests.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3825,"content_sha256":"265c8b99d61201d0f42f8c72ecf5f611a70fab26bebf0d7a667a8fc05848e1e8"},{"filename":"references/redis/examples.md","content":"# Redis E2E Test Examples\n\n## Cache Testing\n\n### Cache on First Fetch\n\n```typescript\nit('should cache user profile on first fetch', async () => {\n // GIVEN: User exists in database\n await userRepository.save({ id: 'user-1', name: 'Test', email: '[email protected]' });\n\n // WHEN: Fetching user (triggers cache)\n const response = await request(httpServer)\n .get('/users/user-1')\n .expect(200);\n\n // THEN: Profile cached\n await helper.assertKeyExists('user:profile:user-1', {\n id: 'user-1',\n name: 'Test',\n email: '[email protected]',\n });\n\n // THEN: TTL is set correctly (1 hour = 3600 seconds)\n await helper.assertKeyTtl('user:profile:user-1', 3500, 3600);\n});\n```\n\n### Return Cached Data\n\n```typescript\nit('should return cached data on subsequent requests', async () => {\n // GIVEN: Cached user profile (different from DB)\n await helper.setTestData('user:profile:user-1', {\n id: 'user-1',\n name: 'Cached Name',\n email: '[email protected]',\n }, 3600);\n\n // WHEN: Fetching user\n const response = await request(httpServer)\n .get('/users/user-1')\n .expect(200);\n\n // THEN: Returns cached data (not from DB)\n expect(response.body.data.name).toBe('Cached Name');\n});\n```\n\n### Cache Invalidation\n\n```typescript\nit('should invalidate cache on user update', async () => {\n // GIVEN: User exists with cached profile\n await userRepository.save({ id: 'user-1', name: 'Original' });\n await helper.setTestData('user:profile:user-1', { id: 'user-1', name: 'Original' });\n\n // WHEN: Updating user\n await request(httpServer)\n .patch('/users/user-1')\n .send({ name: 'Updated' })\n .expect(200);\n\n // THEN: Cache invalidated\n await helper.assertKeyNotExists('user:profile:user-1');\n});\n```\n\n---\n\n## Session Management\n\n### Create Session on Login\n\n```typescript\nit('should create session on login', async () => {\n // GIVEN: User credentials\n const credentials = { email: '[email protected]', password: 'password123' };\n\n // WHEN: Logging in\n const response = await request(httpServer)\n .post('/auth/login')\n .send(credentials)\n .expect(200);\n\n // THEN: Session created in Redis\n const sessionKey = `session:${response.body.data.sessionId}`;\n await helper.assertKeyExists(sessionKey);\n\n // THEN: Session has correct TTL (24 hours)\n await helper.assertKeyTtl(sessionKey, 86000, 86400);\n});\n```\n\n### Extend Session TTL on Activity\n\n```typescript\nit('should extend session TTL on activity', async () => {\n // GIVEN: Existing session with short TTL\n const sessionId = 'sess-123';\n await helper.setTestData(`session:${sessionId}`, { userId: 'user-1' }, 1800); // 30 min\n\n // WHEN: Making authenticated request\n await request(httpServer)\n .get('/users/me')\n .set('X-Session-Id', sessionId)\n .expect(200);\n\n // THEN: TTL extended to full duration\n await helper.assertKeyTtl(`session:${sessionId}`, 86000, 86400);\n});\n```\n\n### Delete Session on Logout\n\n```typescript\nit('should delete session on logout', async () => {\n // GIVEN: Active session\n const sessionId = 'sess-123';\n await helper.setTestData(`session:${sessionId}`, { userId: 'user-1' }, 86400);\n\n // WHEN: Logging out\n await request(httpServer)\n .post('/auth/logout')\n .set('X-Session-Id', sessionId)\n .expect(200);\n\n // THEN: Session deleted\n await helper.assertKeyNotExists(`session:${sessionId}`);\n});\n```\n\n---\n\n## Rate Limiting\n\n### Allow Requests Within Limit\n\n```typescript\nit('should allow requests within limit', async () => {\n // GIVEN: Fresh rate limit state\n\n // WHEN: Making requests within limit (10 per minute)\n for (let i = 0; i \u003c 10; i++) {\n await request(httpServer)\n .get('/api/resource')\n .expect(200);\n }\n\n // THEN: All requests succeeded\n});\n```\n\n### Block Requests Exceeding Limit\n\n```typescript\nit('should block requests exceeding limit', async () => {\n // GIVEN: Rate limit nearly exhausted\n const limitKey = 'ratelimit:api:127.0.0.1';\n await redis.set(limitKey, '9');\n await redis.expire(limitKey, 60);\n\n // WHEN: Making request that reaches limit\n await request(httpServer)\n .get('/api/resource')\n .expect(200);\n\n // THEN: Next request blocked\n const response = await request(httpServer)\n .get('/api/resource')\n .expect(429);\n\n expect(response.body.code).toBe('RATE_LIMIT_EXCEEDED');\n});\n```\n\n### Reset Limit After Window Expires\n\n```typescript\nit('should reset limit after window expires', async () => {\n // GIVEN: Exhausted rate limit\n const limitKey = 'ratelimit:api:127.0.0.1';\n await redis.set(limitKey, '10');\n await redis.expire(limitKey, 1); // 1 second TTL\n\n // WHEN: Waiting for window to expire\n await new Promise(r => setTimeout(r, 1500));\n\n // THEN: Request succeeds\n await request(httpServer)\n .get('/api/resource')\n .expect(200);\n});\n```\n\n---\n\n## Pub/Sub Testing\n\n```typescript\ndescribe('Redis Pub/Sub E2E', () => {\n let subscriber: Redis;\n const receivedMessages: any[] = [];\n\n beforeAll(async () => {\n subscriber = new Redis(process.env.REDIS_URL);\n\n // Wait for subscription confirmation\n await new Promise\u003cvoid>((resolve) => {\n subscriber.once('subscribe', () => resolve());\n subscriber.subscribe('notifications');\n });\n\n subscriber.on('message', (channel, message) => {\n receivedMessages.push({ channel, message: JSON.parse(message) });\n });\n });\n\n beforeEach(() => {\n receivedMessages.length = 0;\n });\n\n afterAll(async () => {\n await subscriber.unsubscribe();\n await subscriber.quit();\n });\n\n it('should publish notification event', async () => {\n // GIVEN: User exists\n\n // WHEN: Creating order (triggers notification)\n await request(httpServer)\n .post('/orders')\n .send({ userId: 'user-1', items: [] })\n .expect(201);\n\n // THEN: Notification published (wait for pub/sub)\n await new Promise(r => setTimeout(r, 500));\n\n expect(receivedMessages).toContainEqual(\n expect.objectContaining({\n channel: 'notifications',\n message: expect.objectContaining({\n type: 'order_created',\n userId: 'user-1',\n }),\n })\n );\n });\n});\n```\n\n---\n\n## Hash Operations\n\n```typescript\nit('should store user preferences in hash', async () => {\n // WHEN: Saving user preferences\n await request(httpServer)\n .put('/users/user-1/preferences')\n .send({\n theme: 'dark',\n language: 'en',\n notifications: true,\n })\n .expect(200);\n\n // THEN: Preferences stored as hash\n await helper.assertHashField('user:prefs:user-1', 'theme', 'dark');\n await helper.assertHashField('user:prefs:user-1', 'language', 'en');\n await helper.assertHashField('user:prefs:user-1', 'notifications', true);\n});\n```\n\n---\n\n## List Operations (Queue)\n\n```typescript\nit('should add job to processing queue', async () => {\n // WHEN: Submitting job\n const response = await request(httpServer)\n .post('/jobs')\n .send({ type: 'email', data: { to: '[email protected]' } })\n .expect(201);\n\n // THEN: Job added to queue\n await helper.assertListLength('queue:jobs', 1);\n\n // THEN: Job data is correct\n const job = await redis.lindex('queue:jobs', 0);\n expect(JSON.parse(job!)).toMatchObject({\n type: 'email',\n data: { to: '[email protected]' },\n });\n});\n```\n\n---\n\n## Set Operations\n\n```typescript\nit('should track unique visitors', async () => {\n // WHEN: Multiple visits from different users\n await request(httpServer).get('/page').set('X-User-Id', 'user-1');\n await request(httpServer).get('/page').set('X-User-Id', 'user-2');\n await request(httpServer).get('/page').set('X-User-Id', 'user-1'); // Duplicate\n\n // THEN: Only unique visitors counted\n const count = await redis.scard('visitors:page:today');\n expect(count).toBe(2);\n\n // THEN: Set contains correct members\n await helper.assertSetContains('visitors:page:today', 'user-1');\n await helper.assertSetContains('visitors:page:today', 'user-2');\n});\n```\n\n---\n\n## Sorted Set (Leaderboard)\n\n```typescript\nit('should update leaderboard score', async () => {\n // GIVEN: Existing leaderboard\n await redis.zadd('leaderboard:game', 100, 'player-1', 200, 'player-2');\n\n // WHEN: Player scores more points\n await request(httpServer)\n .post('/games/score')\n .send({ playerId: 'player-1', points: 150 })\n .expect(200);\n\n // THEN: Score updated\n const score = await redis.zscore('leaderboard:game', 'player-1');\n expect(parseInt(score!)).toBe(250);\n\n // THEN: Rank calculated correctly\n const rank = await redis.zrevrank('leaderboard:game', 'player-1');\n expect(rank).toBe(0); // Top position (0-indexed)\n});\n```\n\n---\n\n## Distributed Lock\n\n```typescript\nit('should acquire and release distributed lock', async () => {\n const lockKey = 'lock:resource-1';\n\n // GIVEN: Lock is available\n\n // WHEN: Acquiring lock\n const response = await request(httpServer)\n .post('/resources/1/lock')\n .expect(200);\n\n // THEN: Lock acquired\n await helper.assertKeyExists(lockKey);\n await helper.assertKeyTtl(lockKey, 25, 30); // 30 second lock\n\n // WHEN: Releasing lock\n await request(httpServer)\n .delete('/resources/1/lock')\n .expect(204);\n\n // THEN: Lock released\n await helper.assertKeyNotExists(lockKey);\n});\n\nit('should prevent concurrent access with lock', async () => {\n // GIVEN: Lock already held\n await redis.set('lock:resource-1', 'holder-1', 'EX', 30);\n\n // WHEN: Trying to acquire same lock\n const response = await request(httpServer)\n .post('/resources/1/lock')\n .expect(409);\n\n // THEN: Conflict error\n expect(response.body.code).toBe('RESOURCE_LOCKED');\n});\n```\n\n---\n\n## Common Issues Handling\n\n### Connection Refused\n\n```typescript\nasync function waitForRedis(redis: Redis, maxRetries = 10): Promise\u003cvoid> {\n for (let i = 0; i \u003c maxRetries; i++) {\n try {\n await redis.ping();\n return;\n } catch {\n await new Promise(r => setTimeout(r, 1000));\n }\n }\n throw new Error('Redis not available');\n}\n```\n\n### TTL Unit Confusion\n\n```typescript\n// CORRECT: setex uses seconds\nawait redis.setex(key, 3600, value); // 1 hour\n\n// CORRECT: psetex uses milliseconds\nawait redis.psetex(key, 3600000, value); // 1 hour\n\n// WRONG: Using milliseconds with setex\nawait redis.setex(key, 3600000, value); // 41 days!\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":10205,"content_sha256":"07adf64152c10aaf16857b8f98fe9ac695a9384781af16f00720e0f9e92f4b7f"},{"filename":"references/redis/knowledge.md","content":"# Redis E2E Testing Knowledge\n\n## Overview\n\nRedis E2E testing focuses on verifying caching, session management, rate limiting, and pub/sub behaviors in an in-memory data store context.\n\n## Key Concepts\n\n| Concept | Description |\n|---------|-------------|\n| Key-Value Store | Basic string storage with keys |\n| TTL (Time To Live) | Automatic key expiration |\n| Data Structures | Strings, Hashes, Lists, Sets, Sorted Sets |\n| Pub/Sub | Publish/Subscribe messaging pattern |\n| Transactions | MULTI/EXEC atomic operations |\n| Pipelining | Batch command execution |\n\n## Common Use Cases\n\n| Use Case | Redis Feature | Test Focus |\n|----------|---------------|------------|\n| Caching | GET/SET with TTL | Cache hit/miss, TTL expiration |\n| Sessions | Hashes with TTL | Session creation/deletion/extend |\n| Rate Limiting | INCR with EXPIRE | Limit enforcement, window reset |\n| Real-time Updates | Pub/Sub | Message delivery, subscription |\n| Queues | Lists (LPUSH/RPOP) | Message ordering, processing |\n| Leaderboards | Sorted Sets | Score updates, ranking |\n\n## Test Isolation Strategies\n\n| Strategy | Speed | Use Case |\n|----------|-------|----------|\n| FLUSHDB | Fast | Clear current database |\n| FLUSHALL | Fast | Clear all databases |\n| DELETE by pattern | Medium | Selective cleanup |\n| Different DB index | N/A | Parallel test isolation |\n\n## Docker Configuration for Tests\n\n```yaml\nredis-e2e:\n image: redis:7-alpine\n ports:\n - \"6380:6379\" # Different from dev\n command: redis-server --save \"\" --appendonly no --maxmemory 100mb --maxmemory-policy allkeys-lru\n```\n\n## Performance Settings Explained\n\n| Setting | Default | Test Value | Effect |\n|---------|---------|------------|--------|\n| `save \"\"` | periodic | disabled | Skip RDB snapshots |\n| `appendonly no` | on | off | Skip AOF persistence |\n| `maxmemory` | unlimited | 100mb | Limit memory usage |\n| `maxmemory-policy` | noeviction | allkeys-lru | Evict old keys |\n\n**Warning:** These settings sacrifice persistence for speed. Only use for tests.\n\n## ioredis vs redis (node-redis)\n\n| Feature | ioredis | node-redis |\n|---------|---------|------------|\n| Cluster Support | Built-in | Built-in |\n| Sentinel Support | Built-in | Built-in |\n| Pipelining | Automatic | Manual |\n| Lua Scripts | Yes | Yes |\n| TypeScript | Yes | Yes |\n| Reconnection | Auto | Auto |\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| [rules.md](rules.md) | Redis-specific testing rules |\n| [test-helper.md](test-helper.md) | RedisTestHelper implementation |\n| [docker-setup.md](docker-setup.md) | Docker configuration |\n| [examples.md](examples.md) | Complete test examples |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2620,"content_sha256":"89023051de16418dde8430bfa4ee074bcc4504d33935a5cbd11fa5fe507232f4"},{"filename":"references/redis/rules.md","content":"# Redis E2E Testing Rules\n\n## Core Rules\n\n| Rule | Requirement |\n|------|-------------|\n| Port | Use different port from dev (e.g., 6380) |\n| Cleanup | `FLUSHDB` in beforeEach |\n| Connection | Single connection for tests |\n| TTL | Verify TTL is set correctly |\n| Close | Properly close connections in afterAll |\n\n## Cleanup Rules\n\n### FLUSHDB (Recommended)\n\n```typescript\nbeforeEach(async () => {\n await redis.flushdb();\n});\n```\n\n### Clear by Pattern\n\n```typescript\nbeforeEach(async () => {\n const keys = await redis.keys('test:*');\n if (keys.length > 0) {\n await redis.del(...keys);\n }\n});\n```\n\n### Different Database per Suite\n\n```typescript\n// Each test suite uses different DB index (0-15)\nconst dbIndex = parseInt(process.env.JEST_WORKER_ID || '0', 10) % 16;\nconst redis = new Redis({ db: dbIndex, ...otherConfig });\n```\n\n## Connection Rules\n\n```typescript\n// ioredis config for tests\nconst redis = new Redis({\n host: 'localhost',\n port: 6380,\n db: 0,\n lazyConnect: true,\n retryDelayOnClusterDown: 100,\n retryDelayOnFailover: 100,\n});\n\n// Properly close in afterAll\nafterAll(async () => {\n await redis.quit(); // Graceful close\n});\n```\n\n## TTL Assertion Rules\n\n### Verify TTL is Set\n\n```typescript\n// Assert TTL within range (accounting for timing)\nasync function assertKeyTtl(key: string, minTtl: number, maxTtl: number): Promise\u003cvoid> {\n const ttl = await redis.ttl(key);\n expect(ttl).toBeGreaterThanOrEqual(minTtl);\n expect(ttl).toBeLessThanOrEqual(maxTtl);\n}\n\n// Usage: Verify 1 hour TTL (allowing for test execution time)\nawait assertKeyTtl('user:profile:123', 3500, 3600);\n```\n\n### TTL Units\n\n```typescript\n// setex uses SECONDS\nawait redis.setex(key, 3600, value); // 1 hour\n\n// psetex uses MILLISECONDS\nawait redis.psetex(key, 3600000, value); // 1 hour\n\n// COMMON MISTAKE: Wrong unit\nawait redis.setex(key, 3600000, value); // Wrong! This is 41 days\n```\n\n## Pub/Sub Rules\n\n### Wait for Subscriber\n\n```typescript\n// WRONG: Subscribe and publish immediately\nsubscriber.subscribe('channel');\nawait redis.publish('channel', message); // May be missed!\n\n// CORRECT: Wait for subscription confirmation\nawait new Promise\u003cvoid>((resolve) => {\n subscriber.once('subscribe', () => resolve());\n subscriber.subscribe('channel');\n});\nawait redis.publish('channel', message);\n```\n\n### Clean Up Listeners\n\n```typescript\nbeforeEach(() => {\n receivedMessages.length = 0;\n});\n\nafterEach(() => {\n subscriber.removeAllListeners('message');\n});\n\nafterAll(async () => {\n await subscriber.unsubscribe();\n await subscriber.quit();\n});\n```\n\n## Assertion Rules\n\n### Assert Key Exists\n\n```typescript\n// GOOD: Assert specific value\nawait helper.assertKeyExists('user:123', { name: 'Test', email: '[email protected]' });\n\n// BAD: Only check existence\nconst exists = await redis.exists('user:123');\nexpect(exists).toBe(1);\n```\n\n### Assert Key Not Exists\n\n```typescript\nawait helper.assertKeyNotExists('cache:invalidated-key');\n```\n\n## Performance Rules\n\n### Use Pipeline for Multiple Operations\n\n```typescript\n// SLOW: Individual commands\nawait redis.set('key1', 'value1');\nawait redis.set('key2', 'value2');\nawait redis.set('key3', 'value3');\n\n// FAST: Pipeline\nconst pipeline = redis.pipeline();\npipeline.set('key1', 'value1');\npipeline.set('key2', 'value2');\npipeline.set('key3', 'value3');\nawait pipeline.exec();\n```\n\n### Use MGET/MSET for Batch Operations\n\n```typescript\n// SLOW: Individual gets\nconst values = [];\nfor (const key of keys) {\n values.push(await redis.get(key));\n}\n\n// FAST: MGET\nconst values = await redis.mget(...keys);\n\n// FAST: MSET\nawait redis.mset('key1', 'val1', 'key2', 'val2', 'key3', 'val3');\n```\n\n## Error Handling Rules\n\n### Connection Retry\n\n```typescript\nasync function waitForRedis(redis: Redis, maxRetries = 10): Promise\u003cvoid> {\n for (let i = 0; i \u003c maxRetries; i++) {\n try {\n await redis.ping();\n return;\n } catch {\n await new Promise(r => setTimeout(r, 1000));\n }\n }\n throw new Error('Redis not available');\n}\n```\n\n## Checklist\n\n**Setup:**\n- [ ] Different port from dev (e.g., 6380)\n- [ ] Persistence disabled (save \"\", appendonly no)\n- [ ] Memory limit set (maxmemory 100mb)\n\n**Cleanup:**\n- [ ] FLUSHDB in beforeEach\n- [ ] Remove message listeners in afterEach\n- [ ] Close connections in afterAll\n\n**Assertions:**\n- [ ] Verify key values, not just existence\n- [ ] Verify TTL within expected range\n- [ ] Use correct TTL units (seconds vs milliseconds)\n\n**Pub/Sub:**\n- [ ] Wait for subscription confirmation before publishing\n- [ ] Clear message buffers between tests\n- [ ] Remove listeners after each test\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4582,"content_sha256":"6db1c9c15cab8252810f982f2a5f096cf669a7755e9d6e949e8a782ca0d798f7"},{"filename":"references/redis/test-helper.md","content":"# Redis Test Helper\n\n## Complete Implementation\n\n```typescript\n// test/helpers/redis.helper.ts\nimport { Redis } from 'ioredis';\n\nexport class RedisTestHelper {\n constructor(private redis: Redis) {}\n\n /**\n * Clear all keys in current database\n */\n async flushDb(): Promise\u003cvoid> {\n await this.redis.flushdb();\n }\n\n /**\n * Clear all keys in all databases\n */\n async flushAll(): Promise\u003cvoid> {\n await this.redis.flushall();\n }\n\n /**\n * Set test data with optional TTL\n */\n async setTestData(key: string, value: any, ttl?: number): Promise\u003cvoid> {\n const serialized = JSON.stringify(value);\n if (ttl) {\n await this.redis.setex(key, ttl, serialized);\n } else {\n await this.redis.set(key, serialized);\n }\n }\n\n /**\n * Get and parse JSON data\n */\n async getTestData\u003cT>(key: string): Promise\u003cT | null> {\n const value = await this.redis.get(key);\n return value ? JSON.parse(value) : null;\n }\n\n /**\n * Assert key exists with expected value\n */\n async assertKeyExists(key: string, expectedValue?: any): Promise\u003cvoid> {\n const exists = await this.redis.exists(key);\n expect(exists).toBe(1);\n\n if (expectedValue !== undefined) {\n const value = await this.redis.get(key);\n expect(JSON.parse(value!)).toEqual(expectedValue);\n }\n }\n\n /**\n * Assert key does not exist\n */\n async assertKeyNotExists(key: string): Promise\u003cvoid> {\n const exists = await this.redis.exists(key);\n expect(exists).toBe(0);\n }\n\n /**\n * Get TTL for a key\n */\n async getKeyTtl(key: string): Promise\u003cnumber> {\n return this.redis.ttl(key);\n }\n\n /**\n * Assert TTL is within expected range\n */\n async assertKeyTtl(\n key: string,\n minTtl: number,\n maxTtl: number\n ): Promise\u003cvoid> {\n const ttl = await this.redis.ttl(key);\n expect(ttl).toBeGreaterThanOrEqual(minTtl);\n expect(ttl).toBeLessThanOrEqual(maxTtl);\n }\n\n /**\n * Assert hash field value\n */\n async assertHashField(\n key: string,\n field: string,\n expectedValue: any\n ): Promise\u003cvoid> {\n const value = await this.redis.hget(key, field);\n expect(JSON.parse(value!)).toEqual(expectedValue);\n }\n\n /**\n * Assert set contains value\n */\n async assertSetContains(key: string, value: string): Promise\u003cvoid> {\n const isMember = await this.redis.sismember(key, value);\n expect(isMember).toBe(1);\n }\n\n /**\n * Assert list length\n */\n async assertListLength(key: string, expectedLength: number): Promise\u003cvoid> {\n const length = await this.redis.llen(key);\n expect(length).toBe(expectedLength);\n }\n\n /**\n * Get all keys matching pattern\n */\n async getKeysByPattern(pattern: string): Promise\u003cstring[]> {\n return this.redis.keys(pattern);\n }\n\n /**\n * Delete keys by pattern\n */\n async deleteByPattern(pattern: string): Promise\u003cnumber> {\n const keys = await this.redis.keys(pattern);\n if (keys.length === 0) return 0;\n return this.redis.del(...keys);\n }\n}\n```\n\n---\n\n## Usage Example\n\n```typescript\ndescribe('Cache E2E', () => {\n let helper: RedisTestHelper;\n let redis: Redis;\n\n beforeAll(async () => {\n redis = new Redis(process.env.REDIS_URL);\n helper = new RedisTestHelper(redis);\n });\n\n beforeEach(async () => {\n await helper.flushDb();\n });\n\n afterAll(async () => {\n await redis.quit();\n });\n\n it('should cache user profile', async () => {\n // GIVEN: User exists in database\n await userRepository.save({ id: 'user-1', name: 'Test' });\n\n // WHEN: Fetching user (triggers cache)\n await request(httpServer)\n .get('/users/user-1')\n .expect(200);\n\n // THEN: Profile cached with TTL\n await helper.assertKeyExists('user:profile:user-1', {\n id: 'user-1',\n name: 'Test',\n });\n await helper.assertKeyTtl('user:profile:user-1', 3500, 3600);\n });\n});\n```\n\n---\n\n## API Reference\n\n| Method | Description |\n|--------|-------------|\n| `flushDb()` | Clear all keys in current database |\n| `flushAll()` | Clear all keys in all databases |\n| `setTestData(key, value, ttl?)` | Set JSON data with optional TTL |\n| `getTestData(key)` | Get and parse JSON data |\n| `assertKeyExists(key, expected?)` | Assert key exists with optional value check |\n| `assertKeyNotExists(key)` | Assert key does not exist |\n| `getKeyTtl(key)` | Get TTL for a key |\n| `assertKeyTtl(key, min, max)` | Assert TTL is within range |\n| `assertHashField(key, field, value)` | Assert hash field value |\n| `assertSetContains(key, value)` | Assert set contains value |\n| `assertListLength(key, length)` | Assert list length |\n| `getKeysByPattern(pattern)` | Get keys matching pattern |\n| `deleteByPattern(pattern)` | Delete keys matching pattern |\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4674,"content_sha256":"16d64e3e27487c293f3b975bcd2704123a6fae9f4d8ef7907a1c5b8fb09282ff"},{"filename":"workflows/debugging/steps/step-01-init.md","content":"---\nname: 'step-01-init'\ndescription: 'Initialize debugging workflow — set output path, load knowledge, detect continuation'\nnextStepFile: './step-02-capture-failure.md'\nreferenceFiles:\n - 'references/common/debugging.md'\n - 'references/common/best-practices.md'\n - 'references/common/rules.md'\n---\n\n# Step 1: Initialize E2E Debugging\n\n## STEP GOAL\n\nSet up the debugging session: identify failing tests, set the output path for the tracking document, load knowledge base, and check for existing output to resume.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/debugging.md` - VS Code config, log analysis, systematic resolution\n- `references/common/best-practices.md` - Common issues and solutions\n- `references/common/rules.md` - Verify test follows required patterns\n\n## EXECUTION\n\n### 1. Ask the User\n\nAsk the user:\n- **Which tests are failing?** (test names, file, or \"run suite to find out\")\n- **Output path** for the tracking document (suggest a default: `./e2e-debug-tracker-{{date}}.md`)\n- **Or provide path to an existing tracker** to resume\n\n### 2. Check for Existing Output\n\nIf the user provides a path to an existing tracker:\n- Read the file\n- Parse the YAML frontmatter\n- If `stepsCompleted` is non-empty → **STOP and load `step-01b-continue.md`**\n\n### 3. Fresh Workflow Setup\n\nIf starting fresh:\n1. Copy the template from `templates/tracking-template.md`\n2. Fill in the frontmatter:\n - `failingTests`: list of failing test names\n - `outputPath`: the chosen output path\n - `date`: current date\n3. Write the initialized tracker to the output path\n\n### 4. Initialize Session\n\n```bash\nexport E2E_SESSION=$(date +%s)-$\n```\n\n### 5. Load Technology-Specific References\n\nBased on failing tests:\n- **Kafka tests**: `references/kafka/isolation.md`, `references/kafka/performance.md`\n- **Database tests**: `references/{postgres|mongodb}/rules.md`\n- **API tests**: `references/api/mocking.md`\n\n### 6. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 1: Session Setup\n\n**Failing Tests**: {{list of failing tests}}\n**Session ID**: {{E2E_SESSION}}\n**References Loaded**: {{list of loaded references}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `1` to `stepsCompleted`\n- Set `currentTest` to first failing test\n\n## PRESENT TO USER\n\nShow confirmation of failing tests and session setup.\n\nThen ask: **[C] Continue to Step 2: Capture Failure**\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-02-capture-failure.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2511,"content_sha256":"115450377939f8b4cb5f003cf9c080db86e7151815e7f3a9a640bee33b8bb11c"},{"filename":"workflows/debugging/steps/step-01b-continue.md","content":"---\nname: 'step-01b-continue'\ndescription: 'Resume debugging from last completed step'\n---\n\n# Step 1b: Continue Previous Session\n\n## STEP GOAL\n\nResume a previously started debugging session by reading the existing tracker, determining progress, and routing to the next incomplete step.\n\n## EXECUTION\n\n### 1. Read Existing Output\n\n- Read the file at the output path provided by the user\n- Parse the YAML frontmatter\n- Extract `stepsCompleted` array, `failingTests`, `currentTest`, `iterations`\n\n### 2. Show Progress Summary\n\nDisplay to the user:\n\n```\nE2E Debugging Progress\n======================\nDate Started: {{date}}\nCurrent Test: {{currentTest}}\nIterations Completed: {{iterations.length}}\nSteps Completed: {{stepsCompleted}}\n\nStep Map:\n [1] Initialize & Setup {{done/pending}}\n [2] Capture Failure {{done/pending}}\n [3] Classify Failure {{done/pending}}\n [4] Isolate Failure {{done/pending}}\n [5] Root Cause {{done/pending}}\n [6] Apply Fix {{done/pending}}\n [7] Verify Fix {{done/pending}}\n [8] Next or Done {{done/pending}}\n\nFailing Tests:\n{{for each test: name - FIXED/INVESTIGATING/PENDING}}\n```\n\n### 3. Offer Options\n\nPresent:\n- **[R] Resume** from the next incomplete step\n- **[O] Overview** — re-read the existing tracker content before resuming\n- **[X] Start over** — create a fresh tracker (confirm: this will overwrite)\n\n### 4. Route to Next Step\n\nOn **[R]** or after **[O]**:\n\nDetermine the next step from `max(stepsCompleted) + 1` and load the corresponding file:\n\n| Next Step | File |\n|-----------|------|\n| 2 | `step-02-capture-failure.md` |\n| 3 | `step-03-classify.md` |\n| 4 | `step-04-isolate.md` |\n| 5 | `step-05-root-cause.md` |\n| 6 | `step-06-apply-fix.md` |\n| 7 | `step-07-verify-fix.md` |\n| 8 | `step-08-next-or-done.md` |\n\nOn **[X]**: Go back to `step-01-init.md` fresh workflow setup (section 3).\n\n## NEXT STEP\n\nLoad the step file determined above.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1998,"content_sha256":"57352cb6f4f6cf147bca1d34fd8f0794833efd1a530d0df894aabaf80b0dc245"},{"filename":"workflows/debugging/steps/step-02-capture-failure.md","content":"---\nname: 'step-02-capture-failure'\ndescription: 'Document the failure completely before investigating'\nnextStepFile: './step-03-classify.md'\n---\n\n# Step 2: Capture Failure Information\n\n## STEP GOAL\n\nDocument the failure completely before investigating.\n\n## EXECUTION\n\n### 1. Create or Update Tracking Entry\n\n```markdown\n## Test: \"{test name}\"\n- **File**: {file}:{line}\n- **Error Type**: {Timeout/Assertion/Connection/etc}\n- **Error Message**:\n ```\n {full error message}\n ```\n- **Stack Trace**:\n ```\n {relevant stack trace}\n ```\n- **Status**: INVESTIGATING\n- **Related Tests**: {list if part of a pattern}\n```\n\n### 2. Gather Context\n\n- When did it start failing?\n- Does it fail consistently or intermittently?\n- Does it fail alone or only with other tests?\n- What changed recently?\n\n### 3. Append to Output\n\nAppend failure details to the output document.\n\n## PRESENT FINDINGS\n\n```\nFailure Captured\n\nTest: \"{test name}\"\nFile: {file}:{line}\n\nError Type: {type}\nError: {message}\n\nQuestions:\n1. Is this a new failure or existing?\n2. Does it fail consistently?\n3. Did anything change recently (code, deps, config)?\n\n[C] Continue investigation / [P] Provide more context\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `2` to `stepsCompleted`\n- Update `currentTest`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-03-classify.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1350,"content_sha256":"e39114d5c4ffb0f448ff1d994d68eac44fd3ddd91276fcc50db9f3257b9ac5eb"},{"filename":"workflows/debugging/steps/step-03-classify.md","content":"---\nname: 'step-03-classify'\ndescription: 'Classify the failure type to guide debugging approach'\nnextStepFile: './step-04-isolate.md'\n---\n\n# Step 3: Classify the Failure\n\n## STEP GOAL\n\nDetermine the category of failure to guide debugging approach.\n\n## EXECUTION\n\n### Failure Categories\n\n1. **Timeout Failures**\n - Test didn't complete in time\n - Async operation never resolved\n - Infrastructure slow/unavailable\n\n2. **Assertion Failures**\n - Expected value doesn't match actual\n - Data not in expected state\n - Wrong error code returned\n\n3. **Connection Failures**\n - Cannot connect to database\n - Kafka broker unavailable\n - External API unreachable\n\n4. **Race Conditions**\n - Passes alone, fails with others\n - Intermittent failures\n - Order-dependent results\n\n5. **State Leakage**\n - Data from previous test interferes\n - Shared resources not cleaned\n - Global state modified\n\n### Append to Output\n\nAppend classification to the output document.\n\n## PRESENT FINDINGS\n\n```\nFailure Classification\n\nBased on the error, this appears to be a:\n\n**{Category}** failure\n\nCharacteristics:\n- {characteristic 1}\n- {characteristic 2}\n\nDebugging approach for this type:\n1. {step 1}\n2. {step 2}\n3. {step 3}\n\n[C] Continue with this approach / [R] Reclassify\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `3` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-04-isolate.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1431,"content_sha256":"1a6937e063dcc7457d6d745486a16b430d9a52f349f5cd7c8fd96f1cfb635ab7"},{"filename":"workflows/debugging/steps/step-04-isolate.md","content":"---\nname: 'step-04-isolate'\ndescription: 'Reproduce the failure in isolation'\nnextStepFile: './step-05-root-cause.md'\n---\n\n# Step 4: Isolate the Failure\n\n## STEP GOAL\n\nReproduce the failure in isolation to understand its nature.\n\n## EXECUTION\n\nAll commands redirect to temp files only (no console):\n\n### 1. Run Test Alone\n\n```bash\nnpm run test:e2e -- -t \"{exact test name}\" > /tmp/e2e-${E2E_SESSION}-isolation.log 2>&1\ntail -30 /tmp/e2e-${E2E_SESSION}-isolation.log\n```\n\n### 2. Run Multiple Times\n\n```bash\nfor i in {1..5}; do\n npm run test:e2e -- -t \"{test name}\" > /tmp/e2e-${E2E_SESSION}-run$i.log 2>&1\n if [ $? -eq 0 ]; then echo \"Run $i: PASS\"; else echo \"Run $i: FAIL\"; fi\ndone\n```\n\n### 3. Run in Different Contexts\n\n```bash\n# Just this file\nnpm run test:e2e -- test/e2e/{file}.e2e-spec.ts > /tmp/e2e-${E2E_SESSION}-isolation.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-isolation.log\n\n# With the test before it\nnpm run test:e2e -- -t \"{prev test name}\" -t \"{failing test name}\" > /tmp/e2e-${E2E_SESSION}-isolation.log 2>&1\ntail -30 /tmp/e2e-${E2E_SESSION}-isolation.log\n```\n\n### 4. Append to Output\n\nAppend isolation results to the output document.\n\n## PRESENT FINDINGS\n\n```\nIsolation Results\n\nTest Alone: {passes/fails}\n5 Consecutive Runs: {x}/5 passed\nWith Previous Test: {passes/fails}\nIn Full Suite: {passes/fails}\n\nAnalysis:\n- {conclusion based on results}\n\nLikely Cause: {hypothesis}\n\n[C] Continue to root cause / [T] Try different isolation\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `4` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-05-root-cause.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1611,"content_sha256":"b778a28fcd2e96c7ddc243cafa99bd5d2803cbee80285f05ed30058575c2cd3b"},{"filename":"workflows/debugging/steps/step-05-root-cause.md","content":"---\nname: 'step-05-root-cause'\ndescription: 'Investigate root cause based on failure category'\nnextStepFile: './step-06-apply-fix.md'\nreferenceFiles:\n - 'references/common/debugging.md'\n---\n\n# Step 5: Investigate Root Cause\n\n## STEP GOAL\n\nIdentify the specific cause of the failure.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/debugging.md` — systematic debugging approach\n\n## EXECUTION\n\n### Investigation by Category\n\n#### Timeout Failures\n\n```bash\n# Check infrastructure\ndocker-compose -f docker-compose.e2e.yml ps\n\n# Check specific service logs (limited output)\ndocker logs postgres-e2e --tail 50\ndocker logs kafka-e2e --tail 50\n\n# Run with extended timeout (no console output)\nnpm run test:e2e -- -t \"{test}\" --testTimeout=60000 > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-debug.log\n```\n\n**Common Causes**: Kafka consumer not subscribed before test, database query slow, external API mock not configured, async operation never completes\n\n#### Assertion Failures\n\n```typescript\n// Add debug logging to test\nit('should do something', async () => {\n // GIVEN\n console.log('=== GIVEN ===');\n console.log('Setup data:', JSON.stringify(data, null, 2));\n\n // WHEN\n console.log('=== WHEN ===');\n const response = await request(httpServer).post('/endpoint').send(data);\n console.log('Response:', JSON.stringify(response.body, null, 2));\n\n // THEN\n console.log('=== THEN ===');\n // assertions...\n});\n```\n\n**Common Causes**: Data not seeded correctly, wrong expected value, async state not ready, previous test left unexpected data\n\n#### Connection Failures\n\n```bash\n# Check service availability\ndocker exec postgres-e2e pg_isready\ndocker exec mongo-e2e mongosh --eval \"db.runCommand('ping')\"\ndocker exec redis-e2e redis-cli ping\n\n# Check ports\nlsof -i :5433 # PostgreSQL E2E port\nlsof -i :27018 # MongoDB E2E port\n```\n\n**Common Causes**: Docker service not running, port conflict, environment variables wrong, connection pool exhausted\n\n#### Race Conditions / State Leakage\n\n```typescript\n// Add delay to verify race condition\nbeforeEach(async () => {\n console.log('=== BEFORE EACH ===');\n console.log('Waiting for in-flight...');\n await new Promise(r => setTimeout(r, 2000)); // Extended wait\n\n console.log('Cleaning data...');\n await repository.clear();\n\n const count = await repository.count();\n console.log('After cleanup count:', count);\n});\n```\n\n**Common Causes**: Missing cleanup wait, shared consumer group (Kafka), global mock state, singleton service state\n\n### Append to Output\n\nAppend root cause analysis to the output document.\n\n## PRESENT FINDINGS\n\n```\nRoot Cause Analysis\n\nInvestigation:\n{what was checked}\n\nFindings:\n{what was discovered}\n\nRoot Cause:\n**{specific cause}**\n\nEvidence:\n{log output or observation}\n\n[F] Apply fix / [I] Investigate further\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `5` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[F]`, load `step-06-apply-fix.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3003,"content_sha256":"138dd9f2475cb617d0f71792014753a6765730fe36f9bf3e9b5c3b5d277fd555"},{"filename":"workflows/debugging/steps/step-06-apply-fix.md","content":"---\nname: 'step-06-apply-fix'\ndescription: 'Apply targeted fix — minimum change needed'\nnextStepFile: './step-07-verify-fix.md'\n---\n\n# Step 6: Apply Targeted Fix\n\n## STEP GOAL\n\nFix the specific root cause without broad changes.\n\n## EXECUTION\n\n### Fix Principles\n\n1. Make the MINIMUM change needed\n2. Don't refactor unrelated code\n3. Don't \"clean up\" while fixing\n4. Document what was changed\n\n### Common Fixes by Category\n\n#### Timeout Fixes\n```typescript\n// Increase specific timeout\nit('should process event', async () => {\n // ...\n}, 30000);\n\n// Add polling instead of fixed wait\nconst result = await waitFor(\n () => repository.findOne({ id }),\n 15000, // timeout\n 100 // poll interval\n);\n\n// Ensure Kafka subscription before test\nbeforeAll(async () => {\n await kafkaHelper.subscribeToTopic(outputTopic);\n await new Promise(r => setTimeout(r, 2000));\n}, 90000);\n```\n\n#### Assertion Fixes\n```typescript\n// Fix expected value\nexpect(response.body.code).toBe('SUCCESS');\n\n// Use partial matching for flexible assertions\nexpect(response.body).toMatchObject({\n code: 'SUCCESS',\n});\n\n// Verify precondition\nconst setupCount = await repository.count();\nexpect(setupCount).toBe(1);\n```\n\n#### Connection Fixes\n```typescript\n// Add retry logic\nconst connectWithRetry = async (retries = 3) => {\n for (let i = 0; i \u003c retries; i++) {\n try {\n await dataSource.initialize();\n return;\n } catch (e) {\n if (i === retries - 1) throw e;\n await new Promise(r => setTimeout(r, 1000));\n }\n }\n};\n```\n\n#### Race Condition Fixes\n```typescript\n// Ensure cleanup complete\nbeforeEach(async () => {\n await new Promise(r => setTimeout(r, 1000));\n await repository.clear();\n kafkaHelper.clearMessages(topic);\n jest.clearAllMocks();\n});\n\n// Use unique identifiers\nconst id = `test-${Date.now()}-${Math.random().toString(36).slice(2)}`;\n```\n\n### Append to Output\n\nAppend fix details to the output document.\n\n## PRESENT FINDINGS\n\n```\nProposed Fix\n\nFile: {file}\nLine: {line}\n\nChange:\n- {old code}\n+ {new code}\n\nRationale: {why this fixes the issue}\n\n[A] Apply fix / [M] Modify fix / [S] Suggest alternative\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `6` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter fix is applied, load `step-07-verify-fix.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2276,"content_sha256":"8468bd09c01c66c2912fa6817d965106431b64872c0d48c6bbf46e0f18043c0b"},{"filename":"workflows/debugging/steps/step-07-verify-fix.md","content":"---\nname: 'step-07-verify-fix'\ndescription: 'Verify fix with multiple runs and related tests'\nnextStepFile: './step-08-next-or-done.md'\n---\n\n# Step 7: Verify the Fix\n\n## STEP GOAL\n\nConfirm the fix works reliably.\n\n## EXECUTION\n\nAll commands redirect to temp files only (no console):\n\n### 1. Run Fixed Test Multiple Times\n\n```bash\nfor i in {1..5}; do\n npm run test:e2e -- -t \"{test name}\" > /tmp/e2e-${E2E_SESSION}-run$i.log 2>&1\n if [ $? -eq 0 ]; then echo \"Run $i: PASS\"; else echo \"Run $i: FAIL\"; fi\ndone\n```\n\n### 2. Run With Related Tests\n\n```bash\nnpm run test:e2e -- test/e2e/{file}.e2e-spec.ts > /tmp/e2e-${E2E_SESSION}-verify.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-verify.log\n```\n\n### 3. Check for Regressions\n\n```bash\n# Run full suite (only if all were passing before)\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n```\n\n### 4. Update Tracking\n\nMark the test as FIXED in the tracking document:\n\n```markdown\n## Test: \"{test name}\"\n- **Status**: FIXED ✅\n- **Root Cause**: {description}\n- **Fix Applied**: {description}\n- **Verified**: {date}\n```\n\n### 5. Append to Output\n\nAppend verification results to the output document.\n\n## PRESENT FINDINGS\n\n```\nFix Verification\n\nTest: \"{test name}\"\n\nIsolated Runs: {5}/5 passed ✅\nWith File: {passes/fails}\nFull Suite: {passes/fails}\n\n{if all pass}\nFix verified! Updating tracking...\n{/if}\n\n{if failures}\nFix incomplete. Additional issues:\n- {issue}\n\n[R] Retry fix / [I] Investigate new issue\n{/if}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `7` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter verification, load `step-08-next-or-done.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1660,"content_sha256":"ea514408193c29982bb4b0ddea499b250f3e1243e4937b0b79a1d774778dcbc1"},{"filename":"workflows/debugging/steps/step-08-next-or-done.md","content":"---\nname: 'step-08-next-or-done'\ndescription: 'Decide: loop back to step 2 for next failure, or mark complete'\n---\n\n# Step 8: Next Failure or Done\n\n## STEP GOAL\n\nDocument resolution of current test and decide whether to continue with the next failing test or mark the workflow complete.\n\n## DECISION\n\n### If More Failures Remain\n\n1. Add current fix to `iterations` array in frontmatter\n2. Remove steps 2-7 from `stepsCompleted` (so they can be re-added in next iteration)\n3. Keep step 1 and step 8 in `stepsCompleted`\n4. Update `currentTest` to the next failing test\n5. **Load `step-02-capture-failure.md`** to start the next debugging cycle\n\n### If All Failures Are Fixed\n\n1. Run final full suite verification:\n\n```bash\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n```\n\n2. Clean up temp files:\n\n```bash\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md\n```\n\n3. Set `status` to `'complete'`\n\n## PRESENT TO USER\n\n```\nProgress Update\n\nFixed: {n} tests\nRemaining: {m} tests\n\n{if remaining > 0}\nNext Failure: \"{next test name}\"\n\n[C] Continue to next / [S] Stop for now\n{/if}\n\n{if remaining == 0}\nAll failures resolved!\n\nFinal Verification:\nnpm run test:e2e\n\n[V] Run final verification / [D] Done\n{/if}\n```\n\n## FRONTMATTER UPDATE\n\n### If looping:\n- Add current iteration to `iterations`\n- Remove `[2, 3, 4, 5, 6, 7]` from `stepsCompleted`\n- Update `currentTest`\n\n### If complete:\n- Add `8` to `stepsCompleted`\n- Set `status` to `'complete'`\n\n### Post-Workflow Recommendations\n\nAfter debugging, review and consider updating:\n1. `references/common/debugging.md` - If new patterns discovered\n2. Technology-specific references - If configuration issues found\n3. Test helpers - If common fixes can be built in\n\n## NEXT STEP\n\n- If more failures → load `step-02-capture-failure.md`\n- If done → **WORKFLOW COMPLETE**. The debugging tracker is saved at the output path.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1933,"content_sha256":"1f8831fbe0e1991fa550dc3ef1aa23197bc45d0312fe68f0f0809925f0d3f135"},{"filename":"workflows/debugging/templates/tracking-template.md","content":"---\nstepsCompleted: []\nworkflowType: 'debugging'\nfailingTests: []\ncurrentTest: ''\ntechnologies: []\niterations: []\noutputPath: ''\ndate: ''\nstatus: 'in-progress'\n---\n\n# E2E Test Debugging Tracker\n\n**Date**: {{date}}\n**Failing Tests**: {{failingTests}}\n**Current Test**: {{currentTest}}\n**Technologies**: {{technologies}}\n**Status**: {{status}}\n\n---\n\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":348,"content_sha256":"9718bcc93a534976a6de2472b6ff84efeaf0a290ddfb17c3dac5b41c9d1534d3"},{"filename":"workflows/debugging/workflow.md","content":"---\nname: 'debugging'\ndescription: 'Systematically debug failing E2E tests by identifying root causes and applying targeted fixes'\nfirstStepFile: './steps/step-01-init.md'\ntemplateFile: './templates/tracking-template.md'\n---\n\n# Debugging E2E Test Workflow\n\nSystematically debug failing E2E tests by identifying root causes and applying targeted fixes without introducing new issues.\n\n## When to Use\n\n- Fix failing E2E tests\n- Debug flaky tests or test isolation issues\n- Troubleshoot connection errors (database, Kafka, Redis)\n- Fix timeout issues or async operation failures\n- Diagnose race conditions or state leakage\n- Debug Kafka message consumption issues\n\n## Prerequisites\n\nHave failing E2E tests to debug and access to Docker infrastructure.\n\n## Step-File Architecture\n\nThis workflow uses a **step-file architecture** for context-safe execution:\n\n- Each step is a separate file loaded sequentially\n- Progress is tracked via `stepsCompleted` in the output document's YAML frontmatter\n- If context is compacted mid-workflow, step-01 detects existing output and resumes from the last completed step via step-01b\n- **Loop pattern**: Steps 2-7 repeat for each failing test; step 8 decides to loop back or complete\n\n### Steps\n\n| Step | File | Description |\n|------|------|-------------|\n| 1 | `step-01-init.md` | Initialize workflow, set output path, detect continuation |\n| 1b | `step-01b-continue.md` | Resume from last completed step |\n| 2 | `step-02-capture-failure.md` | Document the failure completely |\n| 3 | `step-03-classify.md` | Classify the failure type |\n| 4 | `step-04-isolate.md` | Reproduce failure in isolation |\n| 5 | `step-05-root-cause.md` | Investigate root cause |\n| 6 | `step-06-apply-fix.md` | Apply targeted fix |\n| 7 | `step-07-verify-fix.md` | Verify fix with multiple runs |\n| 8 | `step-08-next-or-done.md` | Loop back to step 2 or mark complete |\n\n### Rules\n\n1. **Load one step at a time** - Read the step file, execute it, then load the next\n2. **Update frontmatter after each step** - Add the step number to `stepsCompleted`\n3. **Wait for user confirmation** - Present findings and wait for `[C]` before proceeding\n4. **Load reference files** - Each step specifies which reference files to load before action\n\n### Mandatory Execution Rules\n\n- **ALWAYS load debugging reference before starting**\n- **ALWAYS create tracking file for multiple failures**\n- **ALWAYS fix ONE test at a time** - Run only `-t \"test name\"`, never full suite\n- **NEVER run full suite while debugging** - Only after ALL individual tests pass\n- **NEVER change multiple things simultaneously**\n- **ALWAYS verify fix with multiple runs before moving on**\n- **ALWAYS output test results to temp files** - Avoids context bloat from verbose output\n\n### Critical: One-by-One Fixing Rule\n\n```\n❌ WRONG: Run full suite → See 5 failures → Run full suite again → Still 5 failures → ...\n✅ RIGHT: Run full suite → See 5 failures → Fix test 1 only → Verify → Fix test 2 only → ... → Run full suite ONCE\n```\n\n### Context Efficiency: Temp File Output\n\n**IMPORTANT**: Use unique session ID in filenames to prevent conflicts.\n\n```bash\n# Initialize session (once at start of debugging)\nexport E2E_SESSION=$(date +%s)-$\n\n# Run test and capture output (no console output)\nnpm run test:e2e -- -t \"{test name}\" > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\n\n# Read only summary\ntail -50 /tmp/e2e-${E2E_SESSION}-debug.log\n\n# Get failure details\ngrep -B 5 -A 20 \"FAIL\\|Error:\" /tmp/e2e-${E2E_SESSION}-debug.log\n\n# Cleanup when done\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md\n```\n\n### Debugging Tools Reference\n\n#### VS Code Debug Configuration\n```json\n{\n \"type\": \"node\",\n \"request\": \"launch\",\n \"name\": \"Debug Failing Test\",\n \"program\": \"${workspaceFolder}/node_modules/.bin/jest\",\n \"args\": [\n \"--config\", \"test/jest-e2e.config.ts\",\n \"--runInBand\",\n \"-t\", \"{test name}\"\n ],\n \"console\": \"integratedTerminal\"\n}\n```\n\n#### Command Line Debugging\n```bash\nnode --inspect-brk node_modules/.bin/jest --config test/jest-e2e.config.ts --runInBand -t \"{test}\"\nnpm run test:e2e -- -t \"{test}\" --verbose > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\nDEBUG=* npm run test:e2e -- -t \"{test}\" > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\n```\n\n## Begin\n\nLoad `steps/step-01-init.md` to start.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4303,"content_sha256":"ac0c90534ba9ddff71e77008c26c7bfb26474b69f5f39a1901bf11b2aa937b69"},{"filename":"workflows/optimize/steps/step-01-init.md","content":"---\nname: 'step-01-init'\ndescription: 'Initialize optimization workflow — set output path, load knowledge, detect continuation'\nnextStepFile: './step-02-baseline.md'\nreferenceFiles:\n - 'references/common/best-practices.md'\n - 'references/common/knowledge.md'\n - 'references/common/rules.md'\n---\n\n# Step 1: Initialize E2E Optimization\n\n## STEP GOAL\n\nSet up the optimization session: identify the test suite, set the output path, load knowledge base, and check for existing output to resume.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/best-practices.md` - Performance patterns\n- `references/common/knowledge.md` - Test lifecycle understanding\n- `references/common/rules.md` - Ensure optimizations don't violate rules\n\n## EXECUTION\n\n### 1. Ask the User\n\nAsk the user:\n- **Which test suite to optimize?** (all E2E tests, specific files)\n- **Output path** for the optimization report (suggest a default: `./e2e-optimization-report-{{date}}.md`)\n- **Or provide path to an existing report** to resume\n\n### 2. Check for Existing Output\n\nIf the user provides a path to an existing report:\n- Read the file\n- Parse the YAML frontmatter\n- If `stepsCompleted` is non-empty → **STOP and load `step-01b-continue.md`**\n\n### 3. Fresh Workflow Setup\n\nIf starting fresh:\n1. Copy the template from `templates/report-template.md`\n2. Fill in the frontmatter:\n - `testSuite`: the test suite scope\n - `outputPath`: the chosen output path\n - `date`: current date\n3. Write the initialized report to the output path\n\n### 4. Initialize Session\n\n```bash\nexport E2E_SESSION=$(date +%s)-$\n```\n\n### 5. Load Technology-Specific References\n\n- **Kafka**: `references/kafka/performance.md` - Kafka-specific optimizations\n- **Database**: `references/{postgres|mongodb}/rules.md` - Query optimization\n- **API**: `references/api/rules.md` - Request optimization\n\n### 6. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 1: Session Setup\n\n**Test Suite**: {{testSuite}}\n**Session ID**: {{E2E_SESSION}}\n**References Loaded**: {{list of loaded references}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `1` to `stepsCompleted`\n\n## PRESENT TO USER\n\nShow confirmation of test suite and session setup.\n\nThen ask: **[C] Continue to Step 2: Baseline Assessment**\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-02-baseline.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2380,"content_sha256":"f1f2606643ccff02cb86e5230c2facb20ec689a7e6acaba00cbba4b88bd24ffb"},{"filename":"workflows/optimize/steps/step-01b-continue.md","content":"---\nname: 'step-01b-continue'\ndescription: 'Resume optimization from last completed step'\n---\n\n# Step 1b: Continue Previous Session\n\n## STEP GOAL\n\nResume a previously started optimization session by reading the existing report, determining progress, and routing to the next incomplete step.\n\n## EXECUTION\n\n### 1. Read Existing Output\n\n- Read the file at the output path provided by the user\n- Parse the YAML frontmatter\n- Extract `stepsCompleted` array\n\n### 2. Show Progress Summary\n\nDisplay to the user:\n\n```\nE2E Optimization Progress\n=========================\nTest Suite: {{testSuite}}\nDate Started: {{date}}\nSteps Completed: {{stepsCompleted}}\n\nStep Map:\n [1] Initialize & Setup {{done/pending}}\n [2] Baseline Assessment {{done/pending}}\n [3] Identify Opportunities {{done/pending}}\n [4] Infrastructure Optimization {{done/pending}}\n [5] Setup Optimization {{done/pending}}\n [6] Execution Optimization {{done/pending}}\n [7] Code Organization {{done/pending}}\n [8] Verify Optimizations {{done/pending}}\n [9] Document Results {{done/pending}}\n```\n\n### 3. Offer Options\n\nPresent:\n- **[R] Resume** from the next incomplete step\n- **[O] Overview** — re-read the existing report content before resuming\n- **[X] Start over** — create a fresh report (confirm: this will overwrite)\n\n### 4. Route to Next Step\n\nOn **[R]** or after **[O]**:\n\nDetermine the next step from `max(stepsCompleted) + 1` and load the corresponding file:\n\n| Next Step | File |\n|-----------|------|\n| 2 | `step-02-baseline.md` |\n| 3 | `step-03-identify-opportunities.md` |\n| 4 | `step-04-infra-optimization.md` |\n| 5 | `step-05-setup-optimization.md` |\n| 6 | `step-06-execution-optimization.md` |\n| 7 | `step-07-code-organization.md` |\n| 8 | `step-08-verify.md` |\n| 9 | `step-09-document.md` |\n\nOn **[X]**: Go back to `step-01-init.md` fresh workflow setup (section 3).\n\n## NEXT STEP\n\nLoad the step file determined above.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1965,"content_sha256":"29fe30d879d7da0883fdd37089faf66e3fd9f3c1797bdc63d31c319a383d22b9"},{"filename":"workflows/optimize/steps/step-02-baseline.md","content":"---\nname: 'step-02-baseline'\ndescription: 'Measure current test suite performance'\nnextStepFile: './step-03-identify-opportunities.md'\n---\n\n# Step 2: Baseline Assessment\n\n## STEP GOAL\n\nMeasure current test suite performance to establish a baseline for comparison.\n\n## EXECUTION\n\nAll commands redirect to temp files only (no console):\n\n### 1. Run Full Suite with Timing\n\n```bash\n{ time npm run test:e2e > /tmp/e2e-${E2E_SESSION}-baseline.log 2>&1 ; } 2>> /tmp/e2e-${E2E_SESSION}-baseline.log\ntail -50 /tmp/e2e-${E2E_SESSION}-baseline.log\n```\n\n### 2. Capture Metrics\n\n```bash\n# Extract timing summary\ngrep -E \"Time:|Tests:|passed|failed|Duration|real|user|sys\" /tmp/e2e-${E2E_SESSION}-baseline.log\n```\n\n### 3. Identify Bottlenecks\n\n```bash\n# Run with verbose timing (no console output)\nnpm run test:e2e -- --verbose > /tmp/e2e-${E2E_SESSION}-baseline.log 2>&1\ntail -100 /tmp/e2e-${E2E_SESSION}-baseline.log\n\n# Profile specific file\nnpm run test:e2e -- test/e2e/{file}.e2e-spec.ts --verbose > /tmp/e2e-${E2E_SESSION}-timing.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-timing.log\n```\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 2: Baseline Performance\n\n**Total Duration**: {{X}} minutes\n**Test Count**: {{N}}\n**Average per Test**: {{Y}} seconds\n\n### Slowest Tests\n1. {{test name}} - {{time}}s\n2. {{test name}} - {{time}}s\n3. {{test name}} - {{time}}s\n\n### By Category\n- Setup (beforeAll): {{time}}s\n- Cleanup (beforeEach): {{time}}s\n- Assertions: {{time}}s\n```\n\n## PRESENT FINDINGS\n\n```\nBaseline Performance\n\nTotal Duration: {X} minutes\nTest Count: {N}\nAverage per Test: {Y} seconds\n\nSlowest Tests:\n1. {test name} - {time}s\n2. {test name} - {time}s\n3. {test name} - {time}s\n\nBy Category:\n- Setup (beforeAll): {time}s\n- Cleanup (beforeEach): {time}s\n- Assertions: {time}s\n\n[C] Continue to analysis / [D] Get more detail\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `2` to `stepsCompleted`\n- Set `baselineMetrics` in frontmatter\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-03-identify-opportunities.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2056,"content_sha256":"d7fac5dacdbd387c114abfcf1e800d9e39b6832c1cc13d19452ec445193b02f6"},{"filename":"workflows/optimize/steps/step-03-identify-opportunities.md","content":"---\nname: 'step-03-identify-opportunities'\ndescription: 'Categorize optimization opportunities by impact'\nnextStepFile: './step-04-infra-optimization.md'\nreferenceFiles:\n - 'references/common/best-practices.md'\n - 'references/kafka/performance.md'\n---\n\n# Step 3: Identify Optimization Opportunities\n\n## STEP GOAL\n\nCategorize potential optimizations by impact.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/best-practices.md` — performance patterns\n- `references/kafka/performance.md` — Kafka-specific optimizations (if applicable)\n\n## EXECUTION\n\n### Optimization Categories\n\n1. **Infrastructure Optimization** — Parallel container startup, container reuse, lighter alternatives (Redpanda vs Kafka)\n2. **Setup Optimization** — Reduce beforeAll time, minimize beforeEach cleanup, shared app instance\n3. **Test Execution Optimization** — Replace fixed waits with polling, batch database operations, optimize Kafka subscriptions\n4. **Code Organization** — Group related tests, share setup where safe, remove redundant assertions\n\n### Analysis Checklist\n\n- [ ] Are containers starting sequentially?\n- [ ] Is app recreated for each file?\n- [ ] Are there fixed `setTimeout` waits?\n- [ ] Is cleanup more aggressive than needed?\n- [ ] Are there redundant database queries?\n\n### Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 3: Optimization Opportunities\n\n### High Impact (>30% improvement potential)\n1. {{opportunity}} - Currently {{current}}, could be {{improved}}\n\n### Medium Impact (10-30%)\n1. {{opportunity}}\n\n### Low Impact (\u003c10%)\n1. {{opportunity}}\n\n### Recommended Priority\n1. {{most impactful change}}\n2. {{second most impactful}}\n```\n\n## PRESENT FINDINGS\n\nShow opportunities categorized by impact.\n\nThen ask: **[C] Continue with optimizations / [S] Select specific areas**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `3` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-04-infra-optimization.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2003,"content_sha256":"b511661ada9996f34853e87d47587f6c0979380aded384b0f52248e5388be92b"},{"filename":"workflows/optimize/steps/step-04-infra-optimization.md","content":"---\nname: 'step-04-infra-optimization'\ndescription: 'Optimize Docker and infrastructure setup'\nnextStepFile: './step-05-setup-optimization.md'\n---\n\n# Step 4: Infrastructure Optimization\n\n## STEP GOAL\n\nOptimize Docker and infrastructure setup for faster startup.\n\n## EXECUTION\n\n### Optimizations\n\n#### 1. Parallel Container Startup\n\n```typescript\n// ❌ Sequential (slow)\nawait postgresContainer.start();\nawait kafkaContainer.start();\nawait redisContainer.start();\n\n// ✅ Parallel (fast)\nconst [postgres, kafka, redis] = await Promise.all([\n new PostgresContainer().start(),\n new KafkaContainer().start(),\n new RedisContainer().start(),\n]);\n```\n\n#### 2. Use Lightweight Alternatives\n\n```yaml\n# docker-compose.e2e.yml\n# ❌ Full Kafka (slow startup)\nkafka:\n image: confluentinc/cp-kafka:latest\n\n# ✅ Redpanda (faster, Kafka-compatible)\nkafka:\n image: vectorized/redpanda:latest\n command: >\n redpanda start --smp 1 --memory 512M\n```\n\n#### 3. Pre-warm Containers\n\n```bash\n# Start containers before running tests\nnpm run docker:e2e && sleep 10 && npm run test:e2e\n```\n\n### Append to Output\n\nAppend infrastructure optimization details to the output document.\n\n## PRESENT FINDINGS\n\n```\nInfrastructure Optimization\n\nCurrent Setup Time: {X}s\n\nProposed Changes:\n1. Parallel container startup\n - Before: Sequential, {X}s\n - After: Parallel, ~{Y}s\n - Savings: {Z}s\n\n2. Use Redpanda instead of Kafka\n - Before: Kafka startup {X}s\n - After: Redpanda startup ~{Y}s\n - Savings: {Z}s\n\n[A] Apply all / [S] Select specific / [K] Skip\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `4` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms, load `step-05-setup-optimization.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1698,"content_sha256":"fd2d8bf0e87662a8b56ef45b61c4565dc2ab80a3f109bd281140655de6b57321"},{"filename":"workflows/optimize/steps/step-05-setup-optimization.md","content":"---\nname: 'step-05-setup-optimization'\ndescription: 'Reduce setup and teardown overhead'\nnextStepFile: './step-06-execution-optimization.md'\n---\n\n# Step 5: Test Setup Optimization\n\n## STEP GOAL\n\nReduce setup and teardown overhead.\n\n## EXECUTION\n\n### Optimizations\n\n#### 1. Reuse App Instance\n\n```typescript\n// ❌ App per file (slow)\ndescribe('Feature', () => {\n beforeAll(async () => {\n app = await createTestApp();\n });\n afterAll(async () => {\n await app.close();\n });\n});\n\n// ✅ Global app (if tests are isolated)\nlet app: INestApplication;\nexport const getTestApp = async () => {\n if (!app) {\n app = await createTestApp();\n }\n return app;\n};\n```\n\n#### 2. Optimize beforeEach\n\n```typescript\n// ❌ Over-aggressive cleanup\nbeforeEach(async () => {\n await new Promise(r => setTimeout(r, 2000)); // Too long\n await userRepository.clear();\n await orderRepository.clear();\n await productRepository.clear();\n await auditRepository.clear(); // Unnecessary if not used\n});\n\n// ✅ Targeted cleanup\nbeforeEach(async () => {\n await new Promise(r => setTimeout(r, 500)); // Shorter wait\n // Only clean tables used in this describe block\n await userRepository.clear();\n});\n```\n\n#### 3. Lazy Initialization\n\n```typescript\n// ❌ Always initialize everything\nbeforeAll(async () => {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n await kafkaHelper.subscribeToAllTopics(); // Expensive\n});\n\n// ✅ Initialize on demand\nlet kafkaHelper: KafkaTestHelper;\nconst getKafkaHelper = async () => {\n if (!kafkaHelper) {\n kafkaHelper = new KafkaTestHelper();\n await kafkaHelper.connect();\n }\n return kafkaHelper;\n};\n```\n\n### Append to Output\n\nAppend setup optimization details to the output document.\n\n## PRESENT FINDINGS\n\n```\nSetup Optimization\n\nCurrent beforeAll time: {X}s\nCurrent beforeEach time: {Y}s per test\n\nProposed Changes:\n1. {optimization 1}\n - Savings: {time}\n\n2. {optimization 2}\n - Savings: {time}\n\nEstimated Total Savings: {time}\n\n[A] Apply / [M] Modify / [S] Skip\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `5` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms, load `step-06-execution-optimization.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2186,"content_sha256":"8505fe5c3d27f7e5b90cdb092a8ea185e7fc445900068066c1fff46038c077d8"},{"filename":"workflows/optimize/steps/step-06-execution-optimization.md","content":"---\nname: 'step-06-execution-optimization'\ndescription: 'Speed up individual test execution'\nnextStepFile: './step-07-code-organization.md'\n---\n\n# Step 6: Test Execution Optimization\n\n## STEP GOAL\n\nSpeed up individual test execution.\n\n## EXECUTION\n\n### Optimizations\n\n#### 1. Replace Fixed Waits with Polling\n\n```typescript\n// ❌ Fixed wait (always waits full time)\nawait new Promise(r => setTimeout(r, 10000));\nconst result = await repository.findOne({ id });\n\n// ✅ Smart polling (returns as soon as ready)\nconst result = await waitFor(\n () => repository.findOne({ id }),\n 10000, // max wait\n 50 // poll interval\n);\n```\n\n#### 2. Batch Database Operations\n\n```typescript\n// ❌ Individual inserts\nfor (const item of items) {\n await repository.save(item);\n}\n\n// ✅ Batch insert\nawait repository.save(items);\n```\n\n#### 3. Optimize Kafka Waiting\n\n```typescript\n// ❌ Long fixed timeout\nconst messages = await kafkaHelper.waitForMessages(topic, 1, 30000);\n\n// ✅ Adaptive timeout based on expected load\nconst messages = await kafkaHelper.waitForMessages(\n topic,\n expectedCount,\n Math.max(5000, expectedCount * 1000) // Scale with count\n);\n```\n\n#### 4. Parallel Assertions (when independent)\n\n```typescript\n// ❌ Sequential verification\nconst dbRecord = await repository.findOne({ id });\nconst kafkaMessage = await kafkaHelper.getLastMessage(topic);\nconst redisValue = await redis.get(key);\n\n// ✅ Parallel verification\nconst [dbRecord, kafkaMessage, redisValue] = await Promise.all([\n repository.findOne({ id }),\n kafkaHelper.getLastMessage(topic),\n redis.get(key),\n]);\n```\n\n### Append to Output\n\nAppend execution optimization details to the output document.\n\n## PRESENT FINDINGS\n\n```\nExecution Optimization\n\nIdentified Slow Patterns:\n1. Fixed waits: {n} occurrences, ~{time}s wasted\n2. Sequential DB operations: {n} occurrences\n3. Suboptimal Kafka timeouts: {n} occurrences\n\nProposed Fixes:\n{list with code changes}\n\n[A] Apply all / [S] Select / [V] View details\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `6` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms, load `step-07-code-organization.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2144,"content_sha256":"785288a84c455aceeee26eb64894bac8185bc3d11ae4c415dcd51bf1e82a34cd"},{"filename":"workflows/optimize/steps/step-07-code-organization.md","content":"---\nname: 'step-07-code-organization'\ndescription: 'Improve test organization for better performance'\nnextStepFile: './step-08-verify.md'\n---\n\n# Step 7: Code Organization Optimization\n\n## STEP GOAL\n\nImprove test organization for better performance.\n\n## EXECUTION\n\n### Optimizations\n\n#### 1. Group Related Tests\n\n```typescript\n// ❌ Scattered setup\ndescribe('User API', () => {\n it('create user', async () => {\n const user = await createUser();\n // ...\n });\n it('get user', async () => {\n const user = await createUser(); // Duplicate setup\n // ...\n });\n});\n\n// ✅ Grouped with shared setup\ndescribe('User API', () => {\n describe('with existing user', () => {\n let user: User;\n beforeEach(async () => {\n user = await createUser();\n });\n\n it('should get user', async () => { /* uses shared user */ });\n it('should update user', async () => { /* uses shared user */ });\n it('should delete user', async () => { /* uses shared user */ });\n });\n});\n```\n\n#### 2. Extract Common Helpers\n\n```typescript\n// ❌ Repeated setup logic\nit('test 1', async () => {\n const token = await login('admin', 'password');\n const response = await request(httpServer)\n .get('/users')\n .set('Authorization', `Bearer ${token}`);\n});\n\n// ✅ Shared helper\nconst authenticatedRequest = async () => {\n const token = await authHelper.getToken('admin');\n return request(httpServer).set('Authorization', `Bearer ${token}`);\n};\n\nit('test 1', async () => {\n const response = await (await authenticatedRequest()).get('/users');\n});\n```\n\n#### 3. Remove Redundant Assertions\n\n```typescript\n// ❌ Redundant checks\nexpect(response).toBeDefined();\nexpect(response.body).toBeDefined();\nexpect(response.body.data).toBeDefined();\nexpect(response.body.data.id).toBe('123');\n\n// ✅ Single meaningful assertion\nexpect(response.body.data.id).toBe('123');\n```\n\n### Append to Output\n\nAppend code organization findings to the output document.\n\n## PRESENT FINDINGS\n\n```\nCode Organization Improvements\n\nFindings:\n- Duplicate setup: {n} occurrences\n- Redundant assertions: {n} occurrences\n- Ungrouped related tests: {n} cases\n\nRecommended Refactoring:\n{specific suggestions}\n\n[A] Apply / [V] View details / [S] Skip\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `7` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms, load `step-08-verify.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2365,"content_sha256":"2b427f1f04c717636e2cda22b5efd79f1b89c0a8bf7429ea30569be02ac56be8"},{"filename":"workflows/optimize/steps/step-08-verify.md","content":"---\nname: 'step-08-verify'\ndescription: 'Verify optimizations do not break tests'\nnextStepFile: './step-09-document.md'\n---\n\n# Step 8: Verify Optimizations\n\n## STEP GOAL\n\nEnsure optimizations don't break tests.\n\n## EXECUTION\n\nAll commands redirect to temp files only (no console):\n\n### 1. Run Full Suite\n\n```bash\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-optimized.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-optimized.log\n```\n\n### 2. Compare Performance\n\n```bash\n# Extract timing from both logs\necho \"=== BASELINE ===\" && grep -E \"Time:|Duration:|real|user|sys\" /tmp/e2e-${E2E_SESSION}-baseline.log\necho \"=== OPTIMIZED ===\" && grep -E \"Time:|Duration:|real|user|sys\" /tmp/e2e-${E2E_SESSION}-optimized.log\n```\n\n### 3. Run Multiple Times for Stability\n\n```bash\nfor i in {1..3}; do\n npm run test:e2e > /tmp/e2e-${E2E_SESSION}-run$i.log 2>&1\n if [ $? -eq 0 ]; then echo \"Run $i: PASS\"; else echo \"Run $i: FAIL\"; fi\ndone\n```\n\n### 4. Check for Flakiness\n\n```bash\n# Run 10 times to detect intermittent failures\nfor i in {1..10}; do\n npm run test:e2e > /tmp/e2e-${E2E_SESSION}-flaky$i.log 2>&1\n if [ $? -eq 0 ]; then echo \"Run $i: PASS\"; else echo \"Run $i: FAIL\"; fi\ndone\n```\n\n### 5. Append to Output\n\nAppend verification results to the output document.\n\n## PRESENT FINDINGS\n\n```\nOptimization Results\n\n### Performance Comparison\n| Metric | Before | After | Improvement |\n|--------|--------|-------|-------------|\n| Total Time | {X}m | {Y}m | {Z}% |\n| Avg per Test | {X}s | {Y}s | {Z}% |\n| Setup Time | {X}s | {Y}s | {Z}% |\n\n### Stability Check\nRuns: 10/10 passed ✅\n\n### Test Count\nBefore: {N} tests\nAfter: {N} tests (unchanged)\n\n[D] Done / [R] Rollback changes / [F] Further optimization\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `8` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter verification, load `step-09-document.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1835,"content_sha256":"01e3dba32bd23d183fa86d44459d13e2ff046194245376598116c2dc0d8f2712"},{"filename":"workflows/optimize/steps/step-09-document.md","content":"---\nname: 'step-09-document'\ndescription: 'Document optimizations and mark complete'\n---\n\n# Step 9: Document Optimizations\n\n## STEP GOAL\n\nRecord what was changed for future reference and mark the workflow complete.\n\n## EXECUTION\n\n### 1. Compile Documentation\n\nAppend to the output document:\n\n```markdown\n## Optimization Summary\n\n### Changes Applied\n1. **Infrastructure**: {{description}}\n - Impact: -{{X}}s setup time\n\n2. **Setup**: {{description}}\n - Impact: -{{Y}}s per test\n\n3. **Execution**: {{description}}\n - Impact: -{{Z}}s average per affected test\n\n4. **Code Organization**: {{description}}\n - Impact: {{description}}\n\n### Performance Results\n- Total time: {{before}} → {{after}} ({{improvement}}%)\n- All tests passing\n- Stability: {{X}}/{{X}} runs passed\n\n### Notes\n- {{any caveats or limitations}}\n```\n\n### 2. Clean Up Temp Files\n\n```bash\nrm -f /tmp/e2e-${E2E_SESSION}-*.log\n```\n\n## PRESENT TO USER\n\n```\nOptimization Complete\n\nSummary:\n- Total improvement: {X}%\n- Time saved: {Y} minutes per run\n\nChanges documented in: {output path}\n\nRecommendations for ongoing performance:\n1. Monitor for new fixed waits in PR reviews\n2. Re-run baseline monthly\n3. Consider parallel test execution if tests are fully isolated\n\n[D] Done\n```\n\n### Post-Workflow Recommendations\n\nAfter optimization, review and update:\n1. `references/common/best-practices.md` - Add new patterns discovered\n2. `references/kafka/performance.md` - Update Kafka-specific optimizations\n3. Test helpers - Incorporate optimized patterns\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `9` to `stepsCompleted`\n- Set `status` to `'complete'`\n\n## WORKFLOW COMPLETE\n\nThe optimization workflow is complete. The report is saved at the output path.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1744,"content_sha256":"979a320177a86eb509e7006401baa147baa52bcad65b14529fc515946bc14e4d"},{"filename":"workflows/optimize/templates/report-template.md","content":"---\nstepsCompleted: []\nworkflowType: 'optimize'\ntestSuite: ''\ntechnologies: []\nbaselineMetrics: {}\noutputPath: ''\ndate: ''\nstatus: 'in-progress'\n---\n\n# E2E Test Optimization Report\n\n**Date**: {{date}}\n**Test Suite**: {{testSuite}}\n**Technologies**: {{technologies}}\n**Status**: {{status}}\n\n---\n\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":295,"content_sha256":"4328827b0be8008bc4f0bfb5a62f08cded6ceea8d479ff7cf3f0a9019eb15cf6"},{"filename":"workflows/optimize/workflow.md","content":"---\nname: 'optimize'\ndescription: 'Improve E2E test suite performance, reliability, and maintainability without sacrificing quality'\nfirstStepFile: './steps/step-01-init.md'\ntemplateFile: './templates/report-template.md'\n---\n\n# Optimize E2E Test Workflow\n\nImprove E2E test suite performance, reliability, and maintainability without sacrificing test quality or coverage.\n\n## When to Use\n\n- Speed up slow E2E tests\n- Optimize Docker infrastructure startup\n- Replace fixed waits with smart polling\n- Reduce beforeEach cleanup time\n- Improve test parallelization where safe\n\n## Prerequisites\n\nHave a working E2E test suite to optimize.\n\n## Step-File Architecture\n\nThis workflow uses a **step-file architecture** for context-safe execution:\n\n- Each step is a separate file loaded sequentially\n- Progress is tracked via `stepsCompleted` in the output document's YAML frontmatter\n- If context is compacted mid-workflow, step-01 detects existing output and resumes from the last completed step via step-01b\n\n### Steps\n\n| Step | File | Description |\n|------|------|-------------|\n| 1 | `step-01-init.md` | Initialize workflow, set output path, detect continuation |\n| 1b | `step-01b-continue.md` | Resume from last completed step |\n| 2 | `step-02-baseline.md` | Measure current performance |\n| 3 | `step-03-identify-opportunities.md` | Categorize optimization opportunities |\n| 4 | `step-04-infra-optimization.md` | Optimize Docker and infrastructure |\n| 5 | `step-05-setup-optimization.md` | Reduce setup and teardown overhead |\n| 6 | `step-06-execution-optimization.md` | Speed up individual test execution |\n| 7 | `step-07-code-organization.md` | Improve test organization |\n| 8 | `step-08-verify.md` | Verify optimizations don't break tests |\n| 9 | `step-09-document.md` | Document optimizations, mark complete |\n\n### Rules\n\n1. **Load one step at a time** - Read the step file, execute it, then load the next\n2. **Update frontmatter after each step** - Add the step number to `stepsCompleted`\n3. **Wait for user confirmation** - Present findings and wait for `[C]` before proceeding\n4. **Load reference files** - Each step specifies which reference files to load before action\n\n### Mandatory Execution Rules\n\n- **ALWAYS load knowledge references before optimizing**\n- **ALWAYS measure baseline performance before changes**\n- **NEVER sacrifice test isolation for speed**\n- **ALWAYS verify tests still pass after optimization**\n- **NEVER reduce assertion quality for performance**\n- **ALWAYS output test results to temp files** - Avoids context bloat from verbose output\n\n### Context Efficiency: Temp File Output\n\n**IMPORTANT**: Redirect output to temp files only (NO console output). Use unique session ID.\n\n```bash\n# Initialize session (once at start of optimization)\nexport E2E_SESSION=$(date +%s)-$\n\n# Capture timing data (no console output)\n{ time npm run test:e2e > /tmp/e2e-${E2E_SESSION}-timing.log 2>&1 ; } 2>> /tmp/e2e-${E2E_SESSION}-timing.log\n\n# Read summary only\ntail -50 /tmp/e2e-${E2E_SESSION}-timing.log\n\n# Extract timing info\ngrep -E \"Time:|Duration:|passed|failed|real|user|sys\" /tmp/e2e-${E2E_SESSION}-timing.log\n\n# Cleanup when done\nrm -f /tmp/e2e-${E2E_SESSION}-*.log\n```\n\n### Quick Wins Checklist\n\nFast optimizations with high impact:\n\n- [ ] Parallel container startup\n- [ ] Reduce beforeEach wait time\n- [ ] Replace `setTimeout` with polling\n- [ ] Batch database operations\n- [ ] Remove redundant assertions\n- [ ] Use Redpanda instead of Kafka\n\n## Begin\n\nLoad `steps/step-01-init.md` to start.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3512,"content_sha256":"529b76fb46c41be831aa8a7cdf9cbd28504924131e7100200014163ca3743d71"},{"filename":"workflows/review/steps/step-01-init.md","content":"---\nname: 'step-01-init'\ndescription: 'Initialize review workflow — set output path, load knowledge, detect continuation'\nnextStepFile: './step-02-test-structure.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/best-practices.md'\n - 'references/common/examples.md'\n---\n\n# Step 1: Initialize E2E Test Review\n\n## STEP GOAL\n\nSet up the review session: identify the test files to review, set the output path for the review report, load knowledge base, and check for an existing report to resume.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/rules.md` - Mandatory patterns to enforce\n- `references/common/best-practices.md` - Quality standards\n- `references/common/examples.md` - Reference implementations\n\n## EXECUTION\n\n### 1. Ask the User\n\nAsk the user:\n- **What tests to review?** (specific test file(s), or review all E2E tests)\n- **Output path** for the review report (suggest a default: `./e2e-review-report-{{date}}.md`)\n- **Or provide path to an existing report** to resume a previous review\n\n### 2. Check for Existing Report\n\nIf the user provides a path to an existing report file:\n- Read the file\n- Parse the YAML frontmatter\n- If `stepsCompleted` is non-empty → **STOP and load `step-01b-continue.md`**\n\n### 3. Fresh Workflow Setup\n\nIf starting fresh:\n1. Copy the template from `templates/report-template.md`\n2. Fill in the frontmatter:\n - `targetTests`: the test file(s)/scope provided by the user\n - `technologies`: detected technologies (Kafka, PostgreSQL, MongoDB, Redis, API)\n - `outputPath`: the chosen output path\n - `date`: current date\n3. Write the initialized report to the output path\n\n### 4. Load Technology-Specific References\n\nBased on technologies detected in the test files:\n- **Kafka tests**: `references/kafka/rules.md`, `references/kafka/isolation.md`\n- **PostgreSQL tests**: `references/postgres/rules.md`\n- **MongoDB tests**: `references/mongodb/rules.md`\n- **Redis tests**: `references/redis/rules.md`\n- **API tests**: `references/api/rules.md`\n\n### 5. List Tests to Review\n\nList all test files and test cases in scope:\n\n```\nTests to Review:\n\n1. {file1.e2e-spec.ts} - {number} test cases\n2. {file2.e2e-spec.ts} - {number} test cases\n...\n\nTotal: {n} test files, {m} test cases\n```\n\n### 6. Append to Report\n\nAppend to the output document:\n\n```markdown\n## Step 1: Review Scope\n\n**Tests Under Review**:\n- {{file list with test counts}}\n\n**Technologies Detected**: {{technologies}}\n**References Loaded**: {{list of loaded references}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `1` to `stepsCompleted`\n\n## PRESENT TO USER\n\nShow the user:\n- Confirmation of tests to review and output path\n- Technologies detected\n- References loaded\n\nThen ask: **[C] Continue to Step 2: Test Structure**\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-02-test-structure.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2887,"content_sha256":"8c6fb83e59117d6470b39d6569a768199ad44dfa9dc78ce6a408afe3a6e8fb6e"},{"filename":"workflows/review/steps/step-01b-continue.md","content":"---\nname: 'step-01b-continue'\ndescription: 'Resume review from last completed step'\n---\n\n# Step 1b: Continue Previous Review\n\n## STEP GOAL\n\nResume a previously started E2E test review by reading the existing report, determining progress, and routing to the next incomplete step.\n\n## EXECUTION\n\n### 1. Read Existing Report\n\n- Read the file at the output path provided by the user\n- Parse the YAML frontmatter\n- Extract `stepsCompleted` array\n\n### 2. Show Progress Summary\n\nDisplay to the user:\n\n```\nE2E Test Review Progress\n========================\nTarget: {{targetTests}}\nDate Started: {{date}}\nSteps Completed: {{stepsCompleted}}\n\nStep Map:\n [1] Initialize & Scope {{done/pending}}\n [2] Test Structure {{done/pending}}\n [3] GWT Pattern Compliance {{done/pending}}\n [4] Assertion Quality {{done/pending}}\n [5] Test Isolation {{done/pending}}\n [6] Technology Patterns {{done/pending}}\n [7] Review Summary {{done/pending}}\n```\n\n### 3. Offer Options\n\nPresent:\n- **[R] Resume** from the next incomplete step\n- **[O] Overview** — re-read the existing report content before resuming\n- **[X] Start over** — create a fresh report (confirm: this will overwrite)\n\n### 4. Route to Next Step\n\nOn **[R]** or after **[O]**:\n\nDetermine the next step from `max(stepsCompleted) + 1` and load the corresponding file:\n\n| Next Step | File |\n|-----------|------|\n| 2 | `step-02-test-structure.md` |\n| 3 | `step-03-gwt-compliance.md` |\n| 4 | `step-04-assertion-quality.md` |\n| 5 | `step-05-test-isolation.md` |\n| 6 | `step-06-tech-patterns.md` |\n| 7 | `step-07-summary.md` |\n\nOn **[X]**: Go back to `step-01-init.md` fresh workflow setup (section 3).\n\n## NEXT STEP\n\nLoad the step file determined above.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1762,"content_sha256":"78de35b87123cf5f763097bbcab10ac0c551b3038b7402c20267a7dfeb55b440"},{"filename":"workflows/review/steps/step-02-test-structure.md","content":"---\nname: 'step-02-test-structure'\ndescription: 'Review test file organization and lifecycle hooks'\nnextStepFile: './step-03-gwt-compliance.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/nestjs-setup.md'\n---\n\n# Step 2: Review Test Structure\n\n## STEP GOAL\n\nVerify test file organization, lifecycle hooks (beforeAll/beforeEach/afterAll), and proper configuration.\n\n## REFERENCE LOADING\n\nBefore starting analysis, load and read:\n- `references/common/rules.md` — mandatory patterns\n- `references/common/nestjs-setup.md` — NestJS test setup patterns\n\nCite specific rules when reporting findings.\n\n## ANALYSIS PROCESS\n\nFor each test file, check:\n\n### Checklist\n- [ ] Test file uses `.e2e-spec.ts` suffix\n- [ ] Proper describe block organization\n- [ ] beforeAll: App initialization, helper setup\n- [ ] beforeEach: Cleanup and isolation\n- [ ] afterAll: Proper teardown\n- [ ] Appropriate timeouts set\n\n### Issues to Flag\n\n```typescript\n// ❌ Missing sequential execution\n// Fix: maxWorkers: 1 in jest config\n\n// ❌ Missing cleanup in beforeEach\nbeforeEach(async () => {\n // Must clean database\n await repository.clear();\n});\n\n// ❌ Missing timeout on long operations\nbeforeAll(async () => {\n // Add timeout\n}, 90000);\n\n// ❌ Not closing app properly\nafterAll(async () => {\n await app?.close(); // Must close\n}, 30000);\n```\n\n## PRESENT FINDINGS\n\nPresent findings to the user in this format:\n\n```\nStep 2: Structure Review\n========================\n\n[File: {filename}]\n\n✅ Test file naming correct\n❌ Missing beforeEach cleanup\n⚠️ No explicit timeout on beforeAll\n\nIssues Found: {n}\n\n{detailed findings with line numbers}\n\nSummary: {n} files reviewed, {m} structural issues found\n```\n\nThen ask: **[C] Continue to Step 3: GWT Compliance / [F] Fix issues first**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `2` to `stepsCompleted`\n- Append the findings section to the report\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-03-gwt-compliance.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1997,"content_sha256":"b478178ce6d14f33a3b0479f03ac0a6d32d3dec9e7c0ab18bca469f7dbba83f8"},{"filename":"workflows/review/steps/step-03-gwt-compliance.md","content":"---\nname: 'step-03-gwt-compliance'\ndescription: 'Verify all tests follow Given-When-Then pattern correctly'\nnextStepFile: './step-04-assertion-quality.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/test-case-creation-guide.md'\n---\n\n# Step 3: Review GWT Pattern Compliance\n\n## STEP GOAL\n\nVerify all tests follow the Given-When-Then pattern correctly with proper comments, single actions, and specific assertions.\n\n## REFERENCE LOADING\n\nBefore starting analysis, load and read:\n- `references/common/rules.md` — GWT pattern rules\n- `references/common/test-case-creation-guide.md` — GWT templates for all scenarios\n\nCite specific rules when reporting findings.\n\n## ANALYSIS PROCESS\n\nFor each test case, check:\n\n### 1. GIVEN Section\n- [ ] Has explicit GIVEN comment\n- [ ] Setup is clear and complete\n- [ ] Uses unique identifiers\n- [ ] Test data created via factories\n\n### 2. WHEN Section\n- [ ] Has explicit WHEN comment\n- [ ] EXACTLY ONE action\n- [ ] Action clearly matches test name\n\n### 3. THEN Section\n- [ ] Has explicit THEN comment\n- [ ] Specific assertions (not generic)\n- [ ] No conditional logic\n- [ ] Verifies all expected outcomes\n\n### Pattern Violations to Flag\n\n```typescript\n// ❌ Multiple WHEN actions\nit('should create and update user', async () => {\n const createRes = await request(httpServer).post('/users');\n const updateRes = await request(httpServer).patch(`/users/${id}`);\n // Split into separate tests!\n});\n\n// ❌ Generic assertion\nexpect(response.body.data).toBeDefined();\n// Fix: expect(response.body.data.email).toBe('[email protected]');\n\n// ❌ Conditional assertion\nif (response.status === 200) {\n expect(response.body.data).toBeDefined();\n}\n// Fix: Create separate tests for each scenario\n```\n\n## PRESENT FINDINGS\n\nPresent findings to the user in this format:\n\n```\nStep 3: GWT Pattern Review\n===========================\n\n[File: {filename}]\n\nTest: \"should {name}\"\n- GIVEN: ✅ Present / ❌ Missing\n- WHEN: ✅ Single action / ❌ Multiple actions\n- THEN: ✅ Specific / ❌ Generic\n\nIssues:\n{list issues with line numbers}\n\nRecommendations:\n{specific fixes}\n\nSummary: {n} tests reviewed, {m} GWT issues found\n```\n\nThen ask: **[C] Continue to Step 4: Assertion Quality / [F] Fix issues first**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `3` to `stepsCompleted`\n- Append the findings section to the report\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-04-assertion-quality.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2464,"content_sha256":"8103f3fcacc88c637bfd9570cfa45249dc683ae49d76a8b5fe425bdaab8d091d"},{"filename":"workflows/review/steps/step-04-assertion-quality.md","content":"---\nname: 'step-04-assertion-quality'\ndescription: 'Verify assertions are meaningful and specific'\nnextStepFile: './step-05-test-isolation.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/best-practices.md'\n---\n\n# Step 4: Review Assertion Quality\n\n## STEP GOAL\n\nVerify assertions are meaningful and specific, not just checking existence or types.\n\n## REFERENCE LOADING\n\nBefore starting analysis, load and read:\n- `references/common/rules.md` — assertion rules\n- `references/common/best-practices.md` — assertion best practices\n\nCite specific rules when reporting findings.\n\n## ANALYSIS PROCESS\n\n### Quality Checks\n- [ ] Uses `toMatchObject` for partial matching\n- [ ] Checks specific values, not just types\n- [ ] Verifies database state changes\n- [ ] Verifies async side effects\n- [ ] Error cases verify NO side effects\n\n### Common Issues to Flag\n\n```typescript\n// ❌ Type-only assertion\nexpect(typeof response.body.id).toBe('string');\n// ✅ Value assertion\nexpect(response.body.id).toMatch(/^user-/);\n\n// ❌ Existence-only assertion\nexpect(response.body.data).toBeDefined();\n// ✅ Content assertion\nexpect(response.body.data).toMatchObject({\n email: '[email protected]',\n status: 'active',\n});\n\n// ❌ Missing side effect verification in error case\nit('should reject invalid input', async () => {\n await request(httpServer).post('/users').send(invalid).expect(400);\n // Missing: verify no database record created\n});\n// ✅ With side effect check\nit('should reject invalid input', async () => {\n await request(httpServer).post('/users').send(invalid).expect(400);\n const count = await userRepository.count();\n expect(count).toBe(0);\n});\n```\n\n## PRESENT FINDINGS\n\nPresent findings to the user in this format:\n\n```\nStep 4: Assertion Quality\n=========================\n\n[File: {filename}]\n\nWeak Assertions Found: {n}\n- Line {x}: toBeDefined → should use toMatchObject\n- Line {y}: Missing database verification\n- Line {z}: Error case missing side effect check\n\nRecommendations:\n{specific improvements}\n\nSummary: {n} assertions reviewed, {m} quality issues found\n```\n\nThen ask: **[C] Continue to Step 5: Test Isolation / [F] Fix issues first**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `4` to `stepsCompleted`\n- Append the findings section to the report\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-05-test-isolation.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2384,"content_sha256":"7525ab4a359388f8fef47293b19cc457872e09e8b253d55a860ad86b572c52ea"},{"filename":"workflows/review/steps/step-05-test-isolation.md","content":"---\nname: 'step-05-test-isolation'\ndescription: 'Verify tests do not share state and are truly independent'\nnextStepFile: './step-06-tech-patterns.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/best-practices.md'\n---\n\n# Step 5: Review Test Isolation\n\n## STEP GOAL\n\nVerify tests don't share state and are truly independent.\n\n## REFERENCE LOADING\n\nBefore starting analysis, load and read:\n- `references/common/rules.md` — isolation rules\n- `references/common/best-practices.md` — isolation best practices\n\nCite specific rules when reporting findings.\n\n## ANALYSIS PROCESS\n\n### Isolation Checks\n- [ ] Unique identifiers per test (not hardcoded)\n- [ ] Database cleaned before each test\n- [ ] No global variables modified\n- [ ] Message buffers cleared (Kafka)\n- [ ] Mocks reset between tests\n\n### Isolation Issues to Flag\n\n```typescript\n// ❌ Hardcoded ID (causes conflicts)\nconst userId = 'user-123';\n// ✅ Unique ID\nconst userId = `user-${Date.now()}-${Math.random().toString(36).slice(2)}`;\n\n// ❌ Missing cleanup wait\nbeforeEach(async () => {\n await repository.clear();\n});\n// ✅ With wait for in-flight\nbeforeEach(async () => {\n await new Promise(r => setTimeout(r, 500));\n await repository.clear();\n});\n\n// ❌ Kafka: Using fromBeginning\nawait consumer.subscribe({ topic, fromBeginning: true });\n// ✅ Use pre-subscription + buffer clearing\n```\n\n## PRESENT FINDINGS\n\nPresent findings to the user in this format:\n\n```\nStep 5: Isolation Review\n========================\n\n[File: {filename}]\n\nIssues Found:\n- Hardcoded IDs: {lines}\n- Missing cleanup waits: {lines}\n- Shared state risk: {details}\n\nImpact: Tests may fail intermittently when run together\n\nFixes Required:\n{specific changes}\n\nSummary: {n} isolation issues found\n```\n\nThen ask: **[C] Continue to Step 6: Technology Patterns / [F] Fix issues first**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `5` to `stepsCompleted`\n- Append the findings section to the report\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-06-tech-patterns.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2050,"content_sha256":"99c69b77527f848c052e922177d06ac98ffac3fa069bf0a6df3a252c33442e1f"},{"filename":"workflows/review/steps/step-06-tech-patterns.md","content":"---\nname: 'step-06-tech-patterns'\ndescription: 'Verify correct usage of technology-specific test patterns'\nnextStepFile: './step-07-summary.md'\nreferenceFiles:\n - 'references/kafka/rules.md'\n - 'references/kafka/isolation.md'\n - 'references/postgres/rules.md'\n - 'references/mongodb/rules.md'\n - 'references/redis/rules.md'\n - 'references/api/rules.md'\n---\n\n# Step 6: Review Technology-Specific Patterns\n\n## STEP GOAL\n\nVerify correct usage of technology-specific test patterns by checking against the relevant reference files.\n\n## REFERENCE LOADING\n\nBefore starting analysis, load and read the reference files relevant to the technologies detected in the tests:\n- **Kafka**: `references/kafka/rules.md`, `references/kafka/isolation.md`\n- **PostgreSQL**: `references/postgres/rules.md`\n- **MongoDB**: `references/mongodb/rules.md`\n- **Redis**: `references/redis/rules.md`\n- **API**: `references/api/rules.md`\n\nAlso load the corresponding `test-helper.md` and `examples.md` files for technologies found.\n\nCite specific rules when reporting findings.\n\n## ANALYSIS PROCESS\n\n### Common Technology Issues\n\n**Kafka**:\n```typescript\n// ❌ Fixed wait instead of polling\nawait new Promise(r => setTimeout(r, 10000));\n// ✅ Smart polling\nconst messages = await kafkaHelper.waitForMessages(topic, 1, 20000);\n\n// ❌ fromBeginning: true\n// ✅ Pre-subscription with buffer clearing\n```\n\n**PostgreSQL**:\n```typescript\n// ❌ Wrong cleanup order (FK violations)\nawait orderRepository.clear();\nawait userRepository.clear();\n// ✅ Children first\nawait orderRepository.clear();\nawait userRepository.clear();\n```\n\n**MongoDB**:\n```typescript\n// ❌ Not waiting for indexes\nawait collection.createIndex({ field: 1 });\n// Immediately querying...\n// ✅ Wait or use ensureIndex in beforeAll\n```\n\n**External APIs**:\n```typescript\n// ❌ Not resetting mocks\n// ✅ afterEach: mockServer.resetHandlers();\n```\n\n## PRESENT FINDINGS\n\nPresent findings to the user in this format:\n\n```\nStep 6: Technology Pattern Review\n=================================\n\n### {Technology} Tests\n\nComparing against: references/{technology}/rules.md\n\nIssues Found:\n{list issues}\n\nCorrect Patterns:\n{show expected patterns from references}\n\nSummary: {n} technology-specific issues found\n```\n\nThen ask: **[C] Continue to Step 7: Summary / [F] Fix issues first**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `6` to `stepsCompleted`\n- Append the findings section to the report\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-07-summary.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2514,"content_sha256":"8ec7720b97f4663355b400a0da77a3514a05c965018a70da9d29c786f1ae3d56"},{"filename":"workflows/review/steps/step-07-summary.md","content":"---\nname: 'step-07-summary'\ndescription: 'Generate comprehensive review summary with prioritized fixes'\nreferenceFiles:\n - 'references/common/rules.md'\n---\n\n# Step 7: Generate Review Summary\n\n## STEP GOAL\n\nProvide comprehensive review summary with prioritized fixes, compiling all findings from steps 2-6.\n\n## REFERENCE LOADING\n\nBefore compiling summary, load and read:\n- `references/common/rules.md` — to verify all findings are properly categorized\n\n## COMPILATION PROCESS\n\n### 1. Gather All Findings\n\nRead through the output document and collect all issues found in steps 2-6.\n\n### 2. Categorize by Severity\n\nCategorize each issue:\n\n| Severity | Meaning |\n|----------|---------|\n| **Critical (Must Fix)** | Tests with multiple WHEN actions, conditional assertions, missing cleanup, hardcoded Kafka IDs, fromBeginning: true |\n| **Important (Should Fix)** | Generic assertions, missing side effect verification, hardcoded test IDs, missing GWT comments, inappropriate timeouts |\n| **Minor (Nice to Fix)** | Suboptimal organization, missing factory functions, verbose setup, missing error case tests |\n\n### 3. Write Summary\n\nAppend to the output document:\n\n```markdown\n## Review Summary\n\n### Files Reviewed\n- {file1}: {n} tests\n- {file2}: {m} tests\n\n### Critical Issues (Must Fix)\n1. {issue} - {file}:{line}\n2. {issue} - {file}:{line}\n\n### Important Issues (Should Fix)\n1. {issue} - {file}:{line}\n\n### Minor Issues (Nice to Fix)\n1. {issue} - {file}:{line}\n\n### Good Practices Found\n- {positive finding}\n- {positive finding}\n\n### Recommended Reading\n- references/common/{relevant}.md\n- references/{technology}/{relevant}.md\n\n### Next Steps\n1. Fix critical issues first\n2. Run tests to verify fixes\n3. Address important issues\n4. Consider refactoring for minor issues\n```\n\n## PRESENT TO USER\n\nShow the final summary. Then offer:\n\n```\nWould you like me to:\n[F] Fix critical issues now\n[D] Generate detailed fix instructions\n[E] Export review as markdown file\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `7` to `stepsCompleted`\n- Set `status` to `'complete'`\n\n### Post-Workflow Recommendations\n\nAfter review, recommend:\n1. Re-read `references/common/rules.md` if many GWT violations\n2. Re-read technology-specific guides if patterns incorrect\n3. Review `references/common/examples.md` for reference implementations\n\n## WORKFLOW COMPLETE\n\nThe review workflow is complete. The full report is saved at the output path.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2445,"content_sha256":"00d32619845d207e4c33f8c80b34b5090be5abdc35d5466d9d2d3006f3f583f8"},{"filename":"workflows/review/templates/report-template.md","content":"---\nstepsCompleted: []\nworkflowType: 'review'\ntargetTests: ''\ntechnologies: []\noutputPath: ''\ndate: ''\nstatus: 'in-progress'\n---\n\n# E2E Test Review Report\n\n**Date**: {{date}}\n**Target**: {{targetTests}}\n**Technologies**: {{technologies}}\n**Status**: {{status}}\n\n---\n\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":267,"content_sha256":"771174432986714350ac8176f7e4495069860a1f5b25dc5fec6cc36da02d4421"},{"filename":"workflows/review/workflow.md","content":"---\nname: 'review'\ndescription: 'Review existing E2E tests for quality, correctness, and adherence to best practices'\nfirstStepFile: './steps/step-01-init.md'\ntemplateFile: './templates/report-template.md'\n---\n\n# Review E2E Test Workflow\n\nReview existing E2E tests for quality, correctness, and adherence to best practices. Identify issues and provide actionable improvements.\n\n## When to Use\n\n- Reviewing existing E2E tests for quality\n- Checking test isolation and cleanup patterns\n- Auditing GWT pattern compliance\n- Evaluating assertion quality and specificity\n- Checking for anti-patterns\n\n## Prerequisites\n\nHave the E2E test file(s) to review accessible.\n\n## Step-File Architecture\n\nThis workflow uses a **step-file architecture** for context-safe execution:\n\n- Each step is a separate file loaded sequentially\n- Progress is tracked via `stepsCompleted` in the output document's YAML frontmatter\n- If context is compacted mid-workflow, step-01 detects existing output and resumes from the last completed step via step-01b\n\n### Steps\n\n| Step | File | Description |\n|------|------|-------------|\n| 1 | `step-01-init.md` | Initialize workflow, set output path, detect continuation |\n| 1b | `step-01b-continue.md` | Resume from last completed step |\n| 2 | `step-02-test-structure.md` | Review test file organization and lifecycle hooks |\n| 3 | `step-03-gwt-compliance.md` | Verify GWT pattern compliance |\n| 4 | `step-04-assertion-quality.md` | Verify assertions are meaningful and specific |\n| 5 | `step-05-test-isolation.md` | Verify tests don't share state |\n| 6 | `step-06-tech-patterns.md` | Verify technology-specific patterns |\n| 7 | `step-07-summary.md` | Generate review summary, mark complete |\n\n### Rules\n\n1. **Load one step at a time** - Read the step file, execute it, then load the next\n2. **Update frontmatter after each step** - Add the step number to `stepsCompleted`\n3. **Wait for user confirmation** - Present findings and wait for `[C]` before proceeding\n4. **Load reference files** - Each step specifies which reference files to load before analysis\n5. **Cite specific rules** - When reporting findings, cite the specific rule from the reference file\n\n### Mandatory Execution Rules\n\n- **ALWAYS load knowledge references before reviewing**\n- **ALWAYS check against mandatory rules from references/common/rules.md**\n- **ALWAYS provide specific, actionable feedback**\n- **NEVER approve tests with anti-patterns**\n\n### Review Severity Levels\n\n#### Critical (Must Fix)\n- Tests with multiple WHEN actions\n- Conditional assertions\n- Missing database cleanup\n- Hardcoded IDs in Kafka consumer groups\n- fromBeginning: true in Kafka tests\n\n#### Important (Should Fix)\n- Generic assertions (toBeDefined)\n- Missing side effect verification\n- Hardcoded test IDs\n- Missing GWT comments\n- Inappropriate timeouts\n\n#### Minor (Nice to Fix)\n- Suboptimal test organization\n- Missing factory functions\n- Verbose setup code\n- Missing error case tests\n\n## Begin\n\nLoad `steps/step-01-init.md` to start.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3003,"content_sha256":"ef960f1945ce363e1d7fe985f8f2aeb8601124daf23b5d89fc2528cfbe345919"},{"filename":"workflows/running/steps/step-01-init.md","content":"---\nname: 'step-01-init'\ndescription: 'Initialize running workflow — set output path, load knowledge, detect continuation'\nnextStepFile: './step-02-pre-run.md'\nreferenceFiles:\n - 'references/common/knowledge.md'\n - 'references/common/debugging.md'\n - 'references/common/nestjs-setup.md'\n---\n\n# Step 1: Initialize E2E Test Run\n\n## STEP GOAL\n\nSet up the test run session: set the output path for the run log, load knowledge base, and check for existing output to resume.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/knowledge.md` - Understand test lifecycle\n- `references/common/debugging.md` - Command line debugging and log analysis\n- `references/common/nestjs-setup.md` - Configuration verification\n\n## EXECUTION\n\n### 1. Ask the User\n\nAsk the user:\n- **What tests to run?** (all, specific file, specific test, pattern)\n- **Output path** for the run log (suggest a default: `./e2e-run-log-{{date}}.md`)\n- **Or provide path to an existing run log** to resume\n\n### 2. Check for Existing Output\n\nIf the user provides a path to an existing run log:\n- Read the file\n- Parse the YAML frontmatter\n- If `stepsCompleted` is non-empty → **STOP and load `step-01b-continue.md`**\n\n### 3. Fresh Workflow Setup\n\nIf starting fresh:\n1. Copy the template from `templates/output-template.md`\n2. Fill in the frontmatter:\n - `testScope`: the test scope\n - `outputPath`: the chosen output path\n - `date`: current date\n3. Write the initialized log to the output path\n\n### 4. Initialize Session\n\n```bash\nexport E2E_SESSION=$(date +%s)-$\n```\n\n### 5. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 1: Session Setup\n\n**Test Scope**: {{testScope}}\n**Session ID**: {{E2E_SESSION}}\n**References Loaded**: {{list of loaded references}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `1` to `stepsCompleted`\n\n## PRESENT TO USER\n\nShow confirmation of test scope and session setup.\n\nThen ask: **[C] Continue to Step 2: Pre-Run Verification**\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-02-pre-run.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2073,"content_sha256":"24f02faaac07b9a6db76513397eda0bb0dc1298cdbac8d4b2d46224d707272ad"},{"filename":"workflows/running/steps/step-01b-continue.md","content":"---\nname: 'step-01b-continue'\ndescription: 'Resume running workflow from last completed step'\n---\n\n# Step 1b: Continue Previous Session\n\n## STEP GOAL\n\nResume a previously started test run session by reading the existing run log, determining progress, and routing to the next incomplete step.\n\n## EXECUTION\n\n### 1. Read Existing Output\n\n- Read the file at the output path provided by the user\n- Parse the YAML frontmatter\n- Extract `stepsCompleted` array\n\n### 2. Show Progress Summary\n\nDisplay to the user:\n\n```\nE2E Test Run Progress\n=====================\nScope: {{testScope}}\nDate Started: {{date}}\nSteps Completed: {{stepsCompleted}}\n\nStep Map:\n [1] Initialize & Setup {{done/pending}}\n [2] Pre-Run Verification {{done/pending}}\n [3] Determine Scope {{done/pending}}\n [4] Execute Tests {{done/pending}}\n [5] Interpret Results {{done/pending}}\n [6] Handle Failures {{done/pending}}\n [7] Verify Fixes {{done/pending}}\n [8] Cleanup {{done/pending}}\n```\n\n### 3. Offer Options\n\nPresent:\n- **[R] Resume** from the next incomplete step\n- **[O] Overview** — re-read the existing run log content before resuming\n- **[X] Start over** — create a fresh run log (confirm: this will overwrite)\n\n### 4. Route to Next Step\n\nOn **[R]** or after **[O]**:\n\nDetermine the next step from `max(stepsCompleted) + 1` and load the corresponding file:\n\n| Next Step | File |\n|-----------|------|\n| 2 | `step-02-pre-run.md` |\n| 3 | `step-03-determine-scope.md` |\n| 4 | `step-04-execute.md` |\n| 5 | `step-05-interpret-results.md` |\n| 6 | `step-06-handle-failures.md` |\n| 7 | `step-07-verify-fixes.md` |\n| 8 | `step-08-cleanup.md` |\n\nOn **[X]**: Go back to `step-01-init.md` fresh workflow setup (section 3).\n\n## NEXT STEP\n\nLoad the step file determined above.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1843,"content_sha256":"321ece12c6483d34c2b4536e15e0048140d2b005162ee79a7e16c516b489f2be"},{"filename":"workflows/running/steps/step-02-pre-run.md","content":"---\nname: 'step-02-pre-run'\ndescription: 'Verify infrastructure is ready — Docker services, health checks'\nnextStepFile: './step-03-determine-scope.md'\n---\n\n# Step 2: Pre-Run Verification\n\n## STEP GOAL\n\nEnsure environment is ready for E2E tests.\n\n## EXECUTION\n\n### 1. Check Docker Infrastructure\n\n```bash\n# Verify Docker is running\ndocker info\n\n# Check if E2E services are up\ndocker-compose -f docker-compose.e2e.yml ps\n```\n\n### 2. Start Infrastructure if Needed\n\n```bash\n# Start E2E Docker services\nnpm run docker:e2e\n# OR\ndocker-compose -f docker-compose.e2e.yml up -d\n\n# Wait for services to be healthy\ndocker-compose -f docker-compose.e2e.yml ps\n```\n\n### 3. Verify Service Health\n\n```bash\n# PostgreSQL\ndocker exec postgres-e2e pg_isready\n\n# MongoDB\ndocker exec mongo-e2e mongosh --eval \"db.runCommand('ping')\"\n\n# Redis\ndocker exec redis-e2e redis-cli ping\n\n# Kafka/Redpanda\ndocker exec kafka-e2e rpk cluster info\n```\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 2: Pre-Run Verification\n\n**Docker Status**: {{running/stopped}}\n**Services**:\n- PostgreSQL: {{healthy/unhealthy/not running}}\n- MongoDB: {{healthy/unhealthy/not running}}\n- Redis: {{healthy/unhealthy/not running}}\n- Kafka: {{healthy/unhealthy/not running}}\n```\n\n## PRESENT FINDINGS\n\n```\nPre-Run Verification\n\nDocker Status: {running/stopped}\nServices:\n- PostgreSQL: {healthy/unhealthy/not running}\n- MongoDB: {healthy/unhealthy/not running}\n- Redis: {healthy/unhealthy/not running}\n- Kafka: {healthy/unhealthy/not running}\n\n{if issues}\nIssues Found:\n- {issue 1}\n\nFix: {command}\n{/if}\n\n[C] Continue with tests / [S] Start infrastructure / [F] Fix issues\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `2` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-03-determine-scope.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1817,"content_sha256":"3c3eca92526cd4038bbfb756311a098e1fdbec7e512b0bc787e66e71cfbe12e7"},{"filename":"workflows/running/steps/step-03-determine-scope.md","content":"---\nname: 'step-03-determine-scope'\ndescription: 'Determine which tests to run — full suite, file, test name, pattern'\nnextStepFile: './step-04-execute.md'\n---\n\n# Step 3: Determine Run Scope\n\n## STEP GOAL\n\nIdentify which tests to run.\n\n## EXECUTION\n\n### 1. Options\n\n1. **Full Suite**: Run all E2E tests\n2. **Specific File**: Run tests in one file\n3. **Specific Test**: Run a single test by name\n4. **By Pattern**: Run tests matching a pattern\n\n### 2. Commands\n\n```bash\n# Full suite\nnpm run test:e2e\n\n# Specific file\nnpm run test:e2e -- test/e2e/user.e2e-spec.ts\n\n# Specific test by name\nnpm run test:e2e -- -t \"should create user\"\n\n# Pattern matching\nnpm run test:e2e -- -t \"User API\"\n\n# Multiple files\nnpm run test:e2e -- test/e2e/user.e2e-spec.ts test/e2e/order.e2e-spec.ts\n```\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 3: Test Scope\n\n**Selected Scope**: {{scope description}}\n**Command**: {{command to run}}\n```\n\n## PRESENT FINDINGS\n\n```\nTest Run Scope\n\nAvailable Options:\n1. [A] All E2E tests ({n} files, {m} tests)\n2. [F] Specific file\n3. [T] Specific test by name\n4. [P] Tests matching pattern\n\nCurrent recommendation: {based on context}\n\nSelect option:\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `3` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms scope, load `step-04-execute.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1353,"content_sha256":"5d014eafef4f99e518b5f707b0913551bbe96cc518f1f5e60401170a417a9dce"},{"filename":"workflows/running/steps/step-04-execute.md","content":"---\nname: 'step-04-execute'\ndescription: 'Execute tests with appropriate configuration — redirect output to temp files'\nnextStepFile: './step-05-interpret-results.md'\n---\n\n# Step 4: Execute Tests\n\n## STEP GOAL\n\nRun tests with appropriate configuration, capturing output to temp files.\n\n## EXECUTION\n\n### 1. Run Tests\n\nAll commands redirect to temp file only (no console):\n\n```bash\n# Standard run\nnpm run test:e2e {scope} > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# Verbose output\nnpm run test:e2e {scope} -- --verbose > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# With coverage\nnpm run test:e2e {scope} -- --coverage > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# Force sequential (if not in config)\nnpm run test:e2e {scope} -- --runInBand > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# With specific timeout\nnpm run test:e2e {scope} -- --testTimeout=60000 > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n```\n\n### 2. Read Results\n\n```bash\n# Get summary (last 50 lines)\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# If failures exist, get failure details\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log > /tmp/e2e-${E2E_SESSION}-failures.log && cat /tmp/e2e-${E2E_SESSION}-failures.log\n\n# Count passed/failed\ngrep -c \"✓\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log\n```\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 4: Execution\n\n**Command**: {{command}}\n**Output File**: /tmp/e2e-${E2E_SESSION}-output.log\n**Result Summary**: {{summary from tail -50}}\n```\n\n## PRESENT FINDINGS\n\n```\nRunning Tests\n\nCommand: {command}\nScope: {scope description}\nOutput: /tmp/e2e-${E2E_SESSION}-output.log\n\nTest execution complete. Reading results...\n\n{summary from tail -50}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `4` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter execution, load `step-05-interpret-results.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1848,"content_sha256":"2a0549eafcd543ba7f56177ea35b271e9323099d93043879294a8e7cbc1794db"},{"filename":"workflows/running/steps/step-05-interpret-results.md","content":"---\nname: 'step-05-interpret-results'\ndescription: 'Analyze test results — all passed, some failed, or infrastructure error'\nnextStepFile: './step-06-handle-failures.md'\n---\n\n# Step 5: Interpret Results\n\n## STEP GOAL\n\nAnalyze test results and determine next steps.\n\n## EXECUTION\n\n### Possible Outcomes\n\n#### 1. All Passed\n\n```\n✅ All tests passed ({n} tests, {duration})\n\nSummary:\n- Test Files: {x} passed\n- Tests: {y} passed\n- Duration: {time}\n\n[D] Done / [R] Run again / [C] Run with coverage\n```\n\nIf all passed → skip to step 8 (cleanup).\n\n#### 2. Some Failed\n\n```\n❌ Tests Failed\n\nFailed Tests ({n}):\n1. {test name} - {file}:{line}\n Error: {message}\n\n2. {test name} - {file}:{line}\n Error: {message}\n\nPassed: {x}\nFailed: {y}\nDuration: {time}\n\n⚠️ STOP: Do NOT run full suite again!\n\nNext Steps:\n1. Create /tmp/e2e-${E2E_SESSION}-failures.md with ALL failing tests listed\n2. Select FIRST failing test\n3. Run ONLY that test: npm run test:e2e -- -t \"{test name}\" > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\n4. Fix and verify with 3-5 runs\n5. Move to next test\n6. Run full suite ONLY ONCE after all individual tests pass\n\n[I] Investigate first failure / [T] Create tracking file\n```\n\n#### 3. Infrastructure Error\n\n```\n⚠️ Infrastructure Error\n\nError: {connection/timeout/etc}\n\nLikely Causes:\n- Docker service not running\n- Port conflict\n- Service unhealthy\n\nFix Steps:\n1. {specific command}\n2. {specific command}\n\n[F] Fix and retry / [L] View Docker logs\n```\n\n### Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 5: Results\n\n**Outcome**: {{All Passed / Some Failed / Infrastructure Error}}\n**Passed**: {{x}}\n**Failed**: {{y}}\n**Duration**: {{time}}\n\n{{failure details if any}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `5` to `stepsCompleted`\n\n## NEXT STEP\n\n- If all passed → load `step-08-cleanup.md`\n- If failures → load `step-06-handle-failures.md`\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1910,"content_sha256":"ac6975c00a4da978fc36a8d47fcc3ac90a7e4f64fe3bee25d724b6450720b65a"},{"filename":"workflows/running/steps/step-06-handle-failures.md","content":"---\nname: 'step-06-handle-failures'\ndescription: 'Systematically address test failures ONE AT A TIME'\nnextStepFile: './step-07-verify-fixes.md'\nreferenceFiles:\n - 'references/common/debugging.md'\n---\n\n# Step 6: Handle Failures\n\n## STEP GOAL\n\nSystematically address test failures ONE AT A TIME.\n\n**CRITICAL**: Do NOT run full suite again. Fix each test individually.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/debugging.md` — detailed debugging steps\n\n## EXECUTION\n\n### 1. Create Tracking File\n\n```markdown\n\u003c!-- /tmp/e2e-${E2E_SESSION}-failures.md -->\n# E2E Test Failures - {date}\n\n## Test: \"{test name}\"\n- **File**: {file}:{line}\n- **Error**: {error message}\n- **Status**: INVESTIGATING\n\n### Analysis\n{notes}\n\n### Fix Applied\n{description}\n```\n\n### 2. Isolate Failing Test\n\n```bash\n# Run only the failing test (no console output)\nnpm run test:e2e -- -t \"{exact test name}\" > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# Read the result\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n```\n\n### 3. Analyze Logs\n\n```bash\n# Search for errors in test output\ngrep -i \"error\\|fail\\|exception\" /tmp/e2e-${E2E_SESSION}-output.log\n\n# View application logs (last 100 lines)\ntail -100 logs/e2e-test.log\n\n# View Docker logs (last 50 lines per service)\ndocker-compose -f docker-compose.e2e.yml logs --tail=50\n```\n\n### 4. Debug if Needed\n\n```bash\n# With Node inspector (requires console for interactive debugging)\nnode --inspect-brk node_modules/.bin/jest --config test/jest-e2e.config.ts --runInBand -t \"{test name}\"\n```\n\n### 5. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 6: Failure Investigation\n\n### Test: \"{test name}\"\n**Error**: {message}\n**Possible Causes**:\n1. {cause 1}\n2. {cause 2}\n\n**Fix Applied**: {description}\n```\n\n## PRESENT FINDINGS\n\n```\nFailure Investigation: \"{test name}\"\n\nError: {message}\n\nPossible Causes:\n1. {cause 1} - Check: {verification}\n2. {cause 2} - Check: {verification}\n\nRecommended Actions:\n1. {action}\n2. {action}\n\n[R] Re-run test / [D] Debug with inspector / [L] View logs / [F] Fix\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `6` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter fixes are applied, load `step-07-verify-fixes.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2216,"content_sha256":"8992b3a16d33152cd3890b4168c037fbb902058b76897ed4bdbde3c3f3a682ee"},{"filename":"workflows/running/steps/step-07-verify-fixes.md","content":"---\nname: 'step-07-verify-fixes'\ndescription: 'Verify fixes are stable with multiple runs'\nnextStepFile: './step-08-cleanup.md'\n---\n\n# Step 7: Verify Fixes\n\n## STEP GOAL\n\nConfirm fixes are stable.\n\n## EXECUTION\n\n### 1. Run Fixed Test Multiple Times\n\n```bash\n# Run 5 times to verify stability (no console output)\nfor i in {1..5}; do\n npm run test:e2e -- -t \"{test name}\" > /tmp/e2e-${E2E_SESSION}-run$i.log 2>&1\n if [ $? -eq 0 ]; then echo \"Run $i: PASS\"; else echo \"Run $i: FAIL\"; fi\ndone\n```\n\n### 2. Run Related Tests\n\n```bash\n# Run the entire file (no console output)\nnpm run test:e2e -- {file} > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n```\n\n### 3. Run Full Suite (Only When All Individual Tests Pass)\n\n```bash\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n```\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 7: Fix Verification\n\n### Test: \"{test name}\"\n- Run 1: {{PASS/FAIL}}\n- Run 2: {{PASS/FAIL}}\n- Run 3: {{PASS/FAIL}}\n- Run 4: {{PASS/FAIL}}\n- Run 5: {{PASS/FAIL}}\n\n**Full Suite**: {{PASS/FAIL}}\n```\n\n## PRESENT FINDINGS\n\n```\nFix Verification\n\nTest: \"{test name}\"\n\nRun 1: ✅ Passed\nRun 2: ✅ Passed\nRun 3: ✅ Passed\nRun 4: ✅ Passed\nRun 5: ✅ Passed\n\nFix is stable!\n\n[N] Fix next failure / [F] Run full suite / [U] Update tracking file\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `7` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter verification, load `step-08-cleanup.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1528,"content_sha256":"2b3a604c390f6f38110ae323c95f56a1741cbb58d72b846b7ffbd3170937c426"},{"filename":"workflows/running/steps/step-08-cleanup.md","content":"---\nname: 'step-08-cleanup'\ndescription: 'Post-run cleanup — stop Docker, delete tracking files, mark complete'\n---\n\n# Step 8: Post-Run Cleanup\n\n## STEP GOAL\n\nClean up after test run.\n\n## EXECUTION\n\n### 1. Stop Docker if Not Needed\n\n```bash\ndocker-compose -f docker-compose.e2e.yml down\n```\n\n### 2. Delete Tracking Files\n\n```bash\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md\n```\n\n### 3. Review and Commit Fixes\n\nIf test fixes were made, present for review.\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 8: Cleanup\n\n**Docker**: {{stopped/kept running}}\n**Temp Files**: {{cleaned/kept}}\n**Results Summary**:\n- Passed: {{x}}\n- Failed: {{y}}\n- Fixed: {{z}}\n```\n\n## PRESENT TO USER\n\n```\nTest Run Complete\n\nResults Summary:\n- Passed: {x}\n- Failed: {y}\n- Fixed: {z}\n\nCleanup Options:\n[D] Stop Docker services\n[K] Keep Docker running\n[C] Commit test fixes\n```\n\n### Post-Workflow Recommendations\n\nAfter running tests, if failures occurred:\n1. Review `references/common/debugging.md` for systematic debugging\n2. Review technology-specific guides for common issues\n3. Check `references/common/examples.md` for correct patterns\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `8` to `stepsCompleted`\n- Set `status` to `'complete'`\n\n## WORKFLOW COMPLETE\n\nThe running workflow is complete. The run log is saved at the output path.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1393,"content_sha256":"c0f154b0c10e7fa1f8270945b542355b0ac1ef55cdc9a23b92666f0e41a0037b"},{"filename":"workflows/running/templates/output-template.md","content":"---\nstepsCompleted: []\nworkflowType: 'running'\ntestScope: ''\ntechnologies: []\noutputPath: ''\ndate: ''\nstatus: 'in-progress'\n---\n\n# E2E Test Run Log\n\n**Date**: {{date}}\n**Scope**: {{testScope}}\n**Technologies**: {{technologies}}\n**Status**: {{status}}\n\n---\n\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":257,"content_sha256":"a4f791129f370e7154c79506024bf6058ec9101b698114b13d2863899f387629"},{"filename":"workflows/running/workflow.md","content":"---\nname: 'running'\ndescription: 'Execute E2E tests reliably with proper infrastructure setup, environment configuration, and result interpretation'\nfirstStepFile: './steps/step-01-init.md'\ntemplateFile: './templates/output-template.md'\n---\n\n# Running E2E Test Workflow\n\nExecute E2E tests reliably with proper infrastructure setup, environment configuration, and result interpretation.\n\n## When to Use\n\n- Running E2E tests\n- Starting/stopping Docker infrastructure for testing\n- Analyzing E2E test results\n- Verifying Docker services are healthy\n- Interpreting test output and failures\n\n## Prerequisites\n\nHave E2E tests written and Docker infrastructure configured.\n\n## Step-File Architecture\n\nThis workflow uses a **step-file architecture** for context-safe execution:\n\n- Each step is a separate file loaded sequentially\n- Progress is tracked via `stepsCompleted` in the output document's YAML frontmatter\n- If context is compacted mid-workflow, step-01 detects existing output and resumes from the last completed step via step-01b\n\n### Steps\n\n| Step | File | Description |\n|------|------|-------------|\n| 1 | `step-01-init.md` | Initialize workflow, set output path, detect continuation |\n| 1b | `step-01b-continue.md` | Resume from last completed step |\n| 2 | `step-02-pre-run.md` | Verify infrastructure is ready |\n| 3 | `step-03-determine-scope.md` | Determine which tests to run |\n| 4 | `step-04-execute.md` | Execute tests with appropriate configuration |\n| 5 | `step-05-interpret-results.md` | Analyze test results |\n| 6 | `step-06-handle-failures.md` | Systematically address failures (if any) |\n| 7 | `step-07-verify-fixes.md` | Verify fixes are stable |\n| 8 | `step-08-cleanup.md` | Post-run cleanup, mark complete |\n\n### Rules\n\n1. **Load one step at a time** - Read the step file, execute it, then load the next\n2. **Update frontmatter after each step** - Add the step number to `stepsCompleted`\n3. **Wait for user confirmation** - Present findings and wait for `[C]` before proceeding\n4. **Load reference files** - Each step specifies which reference files to load before action\n\n### Mandatory Execution Rules\n\n- **ALWAYS verify infrastructure is running before tests**\n- **ALWAYS run tests sequentially (maxWorkers: 1)**\n- **NEVER run full suite when individual tests are failing**\n- **NEVER run full suite repeatedly while debugging** - Fix one test at a time\n- **ALWAYS review logs when tests fail**\n- **ALWAYS output test results to temp files** - Avoids context bloat from verbose output\n\n### Context Efficiency: Temp File Output\n\n**IMPORTANT**: Redirect output to temp files only (NO console output). Use unique session ID to prevent conflicts.\n\n```bash\n# Initialize session (once at start of E2E work)\nexport E2E_SESSION=$(date +%s)-$\n\n# Standard pattern for all test runs\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# Then read only what's needed\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Cleanup when done\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md\n```\n\n### Quick Command Reference\n\n```bash\nnpm run docker:e2e # Start infrastructure\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log # Run all\nnpm run test:e2e -- test/e2e/{file}.e2e-spec.ts > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log # Run file\nnpm run test:e2e -- -t \"{test name}\" > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log # Run test\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log # Get failures\ndocker-compose -f docker-compose.e2e.yml down # Stop infrastructure\n```\n\n### Common Issues and Solutions\n\n| Issue | Cause | Solution |\n|-------|-------|----------|\n| Connection refused | Docker not running | `npm run docker:e2e` |\n| Test timeout | Slow infrastructure | Increase timeout or check service health |\n| Flaky tests | Race condition | Add wait in beforeEach, use polling |\n| Port conflict | Another service | Change E2E ports in docker-compose |\n| Memory issues | Too many containers | Restart Docker, increase memory |\n\n## Begin\n\nLoad `steps/step-01-init.md` to start.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4292,"content_sha256":"faf9f376986bdeea43ae6d76d182b67c122d9b3aa5b3e82c76e95ef5d073307c"},{"filename":"workflows/setup/steps/step-01-init.md","content":"---\nname: 'step-01-init'\ndescription: 'Initialize setup workflow — set output path, load knowledge, detect continuation'\nnextStepFile: './step-02-project-analysis.md'\nreferenceFiles:\n - 'references/common/knowledge.md'\n - 'references/common/nestjs-setup.md'\n - 'references/common/rules.md'\n---\n\n# Step 1: Initialize E2E Setup\n\n## STEP GOAL\n\nSet up the setup session: identify the project, set the output path for the setup checklist, load knowledge base, and check for existing output to resume.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/knowledge.md` - Understand E2E fundamentals and test pyramid\n- `references/common/nestjs-setup.md` - Project structure and Jest configuration\n- `references/common/rules.md` - Mandatory patterns (GWT, timeouts, isolation)\n\n## EXECUTION\n\n### 1. Ask the User\n\nAsk the user:\n- **Project path** to set up E2E testing for\n- **Output path** for the setup checklist (suggest a default: `./e2e-setup-checklist-{{date}}.md`)\n- **Or provide path to an existing checklist** to resume\n\n### 2. Check for Existing Output\n\nIf the user provides a path to an existing checklist file:\n- Read the file\n- Parse the YAML frontmatter\n- If `stepsCompleted` is non-empty → **STOP and load `step-01b-continue.md`**\n\n### 3. Fresh Workflow Setup\n\nIf starting fresh:\n1. Copy the template from `templates/output-template.md`\n2. Fill in the frontmatter:\n - `projectPath`: the project path\n - `outputPath`: the chosen output path\n - `date`: current date\n3. Write the initialized checklist to the output path\n\n### 4. Identify Technology References\n\nBased on project, identify which technology-specific references to load:\n- **Kafka**: `references/kafka/docker-setup.md`, `references/kafka/knowledge.md`\n- **PostgreSQL**: `references/postgres/knowledge.md`\n- **MongoDB**: `references/mongodb/docker-setup.md`, `references/mongodb/knowledge.md`\n- **Redis**: `references/redis/docker-setup.md`, `references/redis/knowledge.md`\n- **API**: `references/api/knowledge.md`\n\n### 5. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 1: Session Setup\n\n**Project**: {{projectPath}}\n**References Loaded**: {{list of loaded references}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `1` to `stepsCompleted`\n\n## PRESENT TO USER\n\nShow the user:\n- Confirmation of project path and output path\n- References loaded\n\nThen ask: **[C] Continue to Step 2: Project Analysis**\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-02-project-analysis.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2530,"content_sha256":"631e7cc10c5b8c7e8b945bb2ddf18b864eb2412cc44ce854ec723209dede3e1c"},{"filename":"workflows/setup/steps/step-01b-continue.md","content":"---\nname: 'step-01b-continue'\ndescription: 'Resume setup from last completed step'\n---\n\n# Step 1b: Continue Previous Session\n\n## STEP GOAL\n\nResume a previously started E2E setup session by reading the existing checklist, determining progress, and routing to the next incomplete step.\n\n## EXECUTION\n\n### 1. Read Existing Output\n\n- Read the file at the output path provided by the user\n- Parse the YAML frontmatter\n- Extract `stepsCompleted` array\n\n### 2. Show Progress Summary\n\nDisplay to the user:\n\n```\nE2E Setup Progress\n===================\nProject: {{projectPath}}\nDate Started: {{date}}\nSteps Completed: {{stepsCompleted}}\n\nStep Map:\n [1] Initialize & Setup {{done/pending}}\n [2] Project Analysis {{done/pending}}\n [3] Directory Structure {{done/pending}}\n [4] Jest Configuration {{done/pending}}\n [5] Docker Compose {{done/pending}}\n [6] Test Helpers {{done/pending}}\n [7] Global Setup {{done/pending}}\n [8] Example Test {{done/pending}}\n [9] Verification {{done/pending}}\n```\n\n### 3. Offer Options\n\nPresent:\n- **[R] Resume** from the next incomplete step\n- **[O] Overview** — re-read the existing checklist content before resuming\n- **[X] Start over** — create a fresh checklist (confirm: this will overwrite)\n\n### 4. Route to Next Step\n\nOn **[R]** or after **[O]**:\n\nDetermine the next step from `max(stepsCompleted) + 1` and load the corresponding file:\n\n| Next Step | File |\n|-----------|------|\n| 2 | `step-02-project-analysis.md` |\n| 3 | `step-03-directory-structure.md` |\n| 4 | `step-04-jest-config.md` |\n| 5 | `step-05-docker-compose.md` |\n| 6 | `step-06-test-helpers.md` |\n| 7 | `step-07-global-setup.md` |\n| 8 | `step-08-example-test.md` |\n| 9 | `step-09-verification.md` |\n\nOn **[X]**: Go back to `step-01-init.md` fresh workflow setup (section 3).\n\n## NEXT STEP\n\nLoad the step file determined above.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1939,"content_sha256":"70d23f0b5409af44b5c2de83a3725a6cc8878271aa753df366a00563dcd6c048"},{"filename":"workflows/setup/steps/step-02-project-analysis.md","content":"---\nname: 'step-02-project-analysis'\ndescription: 'Analyze project structure — package.json, existing tests, infrastructure dependencies'\nnextStepFile: './step-03-directory-structure.md'\n---\n\n# Step 2: Project Analysis\n\n## STEP GOAL\n\nUnderstand the existing project structure and testing requirements.\n\n## EXECUTION\n\n### 1. Analyze package.json\n\nCheck for:\n- Existing test frameworks (Jest, Vitest, etc.)\n- Database drivers (pg, mongoose, ioredis, kafkajs, etc.)\n- NestJS version and dependencies\n\n### 2. Check Existing Test Configuration\n\n- `jest.config.ts` or `jest.config.js`\n- `test/` directory structure\n- Existing E2E tests\n\n### 3. Identify Infrastructure Dependencies\n\nFrom source code, identify:\n- Database connections (TypeORM, Mongoose, Prisma)\n- Message brokers (Kafka, RabbitMQ)\n- Cache systems (Redis)\n- External APIs\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 2: Project Analysis\n\n**Framework**: NestJS v{{version}}\n**Existing Tests**: {{yes/no}}\n\n**Infrastructure Detected**:\n- Database: {{PostgreSQL/MongoDB/None}}\n- Message Broker: {{Kafka/None}}\n- Cache: {{Redis/None}}\n- External APIs: {{list or None}}\n\n**Recommended Setup**:\n- [ ] Jest E2E configuration\n- [ ] Docker Compose for {{services}}\n- [ ] Test helpers for {{technologies}}\n```\n\n## PRESENT FINDINGS\n\nPresent analysis to the user.\n\nThen ask: **[C] Continue to Step 3: Directory Structure / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `2` to `stepsCompleted`\n- Update `technologies` in frontmatter\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-03-directory-structure.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1620,"content_sha256":"929a32ba4f17771c3793edb93773ead70c97d55a5aa109f2d5dca7071a79d5b9"},{"filename":"workflows/setup/steps/step-03-directory-structure.md","content":"---\nname: 'step-03-directory-structure'\ndescription: 'Create E2E test directory structure'\nnextStepFile: './step-04-jest-config.md'\nreferenceFiles:\n - 'references/common/nestjs-setup.md'\n---\n\n# Step 3: Create Directory Structure\n\n## STEP GOAL\n\nSet up the E2E test directory structure.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/nestjs-setup.md` — project structure patterns\n\n## EXECUTION\n\n### 1. Plan Structure\n\n```\ntest/\n├── e2e/\n│ ├── setup.ts # Global test setup\n│ └── helpers/ # Test utilities\n│ ├── test-app.helper.ts\n│ └── {technology}.helper.ts\n└── jest-e2e.config.ts # E2E Jest configuration\n```\n\n### 2. Present to User for Confirmation\n\nShow the planned structure before creating.\n\n### 3. Create Directories\n\nAfter user confirms, create the directory structure.\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 3: Directory Structure\n\n**Created**:\n{{show tree structure}}\n```\n\n## PRESENT FINDINGS\n\n```\nI'll create this directory structure:\n\n{show tree structure}\n\n[C] Continue / [S] Skip (keep existing) / [M] Modify\n```\n\nThen ask: **[C] Continue to Step 4: Jest Config**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `3` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-04-jest-config.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1385,"content_sha256":"69e18105bd33f4ab5c7806fcef6ca04e2bebe2eeb20fb587133e27bd71b6ec86"},{"filename":"workflows/setup/steps/step-04-jest-config.md","content":"---\nname: 'step-04-jest-config'\ndescription: 'Create or update Jest E2E configuration'\nnextStepFile: './step-05-docker-compose.md'\nreferenceFiles:\n - 'references/common/nestjs-setup.md'\n - 'references/common/rules.md'\n---\n\n# Step 4: Configure Jest for E2E\n\n## STEP GOAL\n\nCreate or update Jest E2E configuration with proper settings for sequential execution, timeouts, and test matching.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/nestjs-setup.md` — Jest configuration patterns\n- `references/common/rules.md` — mandatory configuration rules\n\n## EXECUTION\n\n### 1. Create jest-e2e.config.ts\n\nCreate `test/jest-e2e.config.ts` with:\n- `preset: 'ts-jest'`\n- `testEnvironment: 'node'`\n- `testMatch: ['**/*.e2e-spec.ts']`\n- `testTimeout: 25000`\n- `maxWorkers: 1` (CRITICAL for E2E isolation)\n- `setupFilesAfterEnv` if needed\n- `forceExit: true`\n- `detectOpenHandles: true`\n\n### 2. Update package.json\n\nAdd/update scripts:\n- `\"test:e2e\": \"jest --config test/jest-e2e.config.ts\"`\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 4: Jest Configuration\n\n**File**: test/jest-e2e.config.ts\n{{show configuration}}\n\n**package.json scripts updated**:\n- test:e2e: jest --config test/jest-e2e.config.ts\n```\n\n## PRESENT FINDINGS\n\nShow the configuration to the user.\n\nThen ask: **[C] Continue to Step 5: Docker Compose / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `4` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-05-docker-compose.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1539,"content_sha256":"5fee4897bd3f55ebdd77d05129ac8cd1924bbb4ed770ebce3ca5be2a65c00030"},{"filename":"workflows/setup/steps/step-05-docker-compose.md","content":"---\nname: 'step-05-docker-compose'\ndescription: 'Create Docker Compose for E2E test infrastructure'\nnextStepFile: './step-06-test-helpers.md'\nreferenceFiles:\n - 'references/kafka/docker-setup.md'\n - 'references/mongodb/docker-setup.md'\n - 'references/redis/docker-setup.md'\n---\n\n# Step 5: Create Docker Compose\n\n## STEP GOAL\n\nSet up Docker infrastructure for E2E tests with non-conflicting ports.\n\n## REFERENCE LOADING\n\nBefore starting, load and read the docker-setup references for detected technologies:\n- `references/kafka/docker-setup.md` — Redpanda/Kafka Docker configs\n- `references/mongodb/docker-setup.md` — MongoDB Docker config\n- `references/redis/docker-setup.md` — Redis Docker config\n\n## EXECUTION\n\n### 1. Create docker-compose.e2e.yml\n\nBased on detected technologies, create with non-conflicting ports:\n- PostgreSQL: Port 5433\n- MongoDB: Port 27018\n- Redis: Port 6380\n- Kafka/Redpanda: Ports 9093/19093\n\n### 2. Create .env.e2e\n\nCreate `.env.e2e` with test-specific environment variables.\n\n### 3. Add npm Script\n\nAdd: `\"docker:e2e\": \"docker-compose -f docker-compose.e2e.yml up -d\"`\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 5: Docker Compose\n\n**File**: docker-compose.e2e.yml\n{{show docker-compose.e2e.yml}}\n\n**Environment Variables** (.env.e2e):\n{{show variables}}\n\n**npm script**: docker:e2e\n```\n\n## PRESENT FINDINGS\n\nShow the Docker Compose configuration and environment variables.\n\nThen ask: **[C] Continue to Step 6: Test Helpers / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `5` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-06-test-helpers.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1665,"content_sha256":"4456a9489fe225f31541be88bcd7511ae38bfe9074e826f963a729ccb648679d"},{"filename":"workflows/setup/steps/step-06-test-helpers.md","content":"---\nname: 'step-06-test-helpers'\ndescription: 'Create reusable test helper classes for each technology'\nnextStepFile: './step-07-global-setup.md'\nreferenceFiles:\n - 'references/kafka/test-helper.md'\n - 'references/postgres/test-helper.md'\n - 'references/mongodb/test-helper.md'\n - 'references/redis/test-helper.md'\n - 'references/api/test-helper.md'\n---\n\n# Step 6: Create Test Helpers\n\n## STEP GOAL\n\nCreate reusable test helper classes for each detected technology.\n\n## REFERENCE LOADING\n\nBefore starting, load and read the test-helper references for detected technologies:\n- `references/kafka/test-helper.md` — KafkaTestHelper implementation\n- `references/postgres/test-helper.md` — PostgresTestHelper implementation\n- `references/mongodb/test-helper.md` — MongoDbTestHelper implementation\n- `references/redis/test-helper.md` — RedisTestHelper implementation\n- `references/api/test-helper.md` — Auth and Supertest helpers\n\n## EXECUTION\n\n### 1. Create test-app.helper.ts\n\nCreate `test/e2e/helpers/test-app.helper.ts`:\n- NestJS app bootstrap\n- HTTP server access\n- Cleanup utilities\n\n### 2. Create Technology-Specific Helpers\n\nFor each detected technology, create `{technology}.helper.ts` based on the reference file.\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 6: Test Helpers\n\n### test-app.helper.ts\n{{show code preview}}\n\n### {technology}.helper.ts\n{{show code preview}}\n```\n\n## PRESENT FINDINGS\n\nShow the test helpers to the user.\n\nThen ask: **[C] Continue to Step 7: Global Setup / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `6` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-07-global-setup.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1704,"content_sha256":"b8d1c9f52b2d135bce0c8bcd5b238aef3728b1b75d3a4f612ff8a262f9915656"},{"filename":"workflows/setup/steps/step-07-global-setup.md","content":"---\nname: 'step-07-global-setup'\ndescription: 'Create global E2E test setup file'\nnextStepFile: './step-08-example-test.md'\nreferenceFiles:\n - 'references/common/nestjs-setup.md'\n---\n\n# Step 7: Create Global Setup\n\n## STEP GOAL\n\nCreate the global E2E test setup file with environment loading, timeout configuration, and global hooks.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/nestjs-setup.md` — global setup patterns\n\n## EXECUTION\n\n### 1. Create setup.ts\n\nCreate `test/e2e/setup.ts`:\n- Environment variable loading\n- Test timeout configuration\n- Global beforeAll/afterAll hooks\n\n### 2. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 7: Global Setup\n\n**File**: test/e2e/setup.ts\n{{show code}}\n```\n\n## PRESENT FINDINGS\n\nShow the global setup code.\n\nThen ask: **[C] Continue to Step 8: Example Test / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `7` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-08-example-test.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1027,"content_sha256":"42358bc5b5fc83a976f5d09bdb460de399d99b27c0cf4295efee639e9ab372ff"},{"filename":"workflows/setup/steps/step-08-example-test.md","content":"---\nname: 'step-08-example-test'\ndescription: 'Create a sample E2E test file demonstrating patterns'\nnextStepFile: './step-09-verification.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/examples.md'\n---\n\n# Step 8: Create Example Test\n\n## STEP GOAL\n\nCreate a sample E2E test file demonstrating GWT pattern, lifecycle hooks, helper usage, and cleanup patterns.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/rules.md` — GWT and test rules\n- `references/common/examples.md` — reference implementations\n\n## EXECUTION\n\n### 1. Create Example Test\n\nCreate `test/e2e/example.e2e-spec.ts`:\n- Proper GWT pattern\n- beforeAll/beforeEach/afterAll lifecycle\n- Helper usage examples\n- Cleanup patterns\n\n### 2. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 8: Example Test\n\n**File**: test/e2e/example.e2e-spec.ts\n{{show code}}\n\nThis demonstrates:\n- GWT pattern usage\n- Test isolation\n- Helper integration\n- Proper cleanup\n```\n\n## PRESENT FINDINGS\n\nShow the example test code and what it demonstrates.\n\nThen ask: **[C] Continue to Step 9: Verification / [S] Skip**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `8` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-09-verification.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1290,"content_sha256":"437d89567b161a1bc105f5440bc6e2d0a5190a7b341a38e530ebf89d21b8ba2d"},{"filename":"workflows/setup/steps/step-09-verification.md","content":"---\nname: 'step-09-verification'\ndescription: 'Verify setup works correctly — start Docker, run example test, mark complete'\nreferenceFiles:\n - 'references/common/debugging.md'\n---\n\n# Step 9: Verification\n\n## STEP GOAL\n\nVerify the setup works correctly by starting Docker infrastructure and running the example test.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/debugging.md` — for troubleshooting if verification fails\n\n## EXECUTION\n\n### 1. Start Docker Infrastructure\n\n```bash\nnpm run docker:e2e\n```\n\n### 2. Wait for Services to be Healthy\n\nVerify each service is responsive.\n\n### 3. Run Example Test\n\n```bash\nnpm run test:e2e -- test/e2e/example.e2e-spec.ts\n```\n\n### 4. Report Results\n\n### 5. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 9: Verification\n\n**Docker Services**: {{status of each service}}\n**Example Test**: {{passed/failed}}\n\n### Next Steps\n1. Review references/common/rules.md for mandatory patterns\n2. Use references/common/test-case-creation-guide.md for writing tests\n3. Check technology-specific examples in references/{tech}/examples.md\n```\n\n## PRESENT TO USER\n\n```\nSetup Verification:\n\n✅ Docker services started\n✅ Database connection successful\n✅ Example test passed\n\nYour E2E testing infrastructure is ready!\n\nNext steps:\n1. Review references/common/rules.md for mandatory patterns\n2. Use references/common/test-case-creation-guide.md for writing tests\n3. Check technology-specific examples in references/{tech}/examples.md\n```\n\n### Post-Workflow Recommendations\n\nAfter completing setup, recommend the user review:\n1. `references/common/rules.md` - Mandatory patterns they must follow\n2. `references/common/best-practices.md` - Test design guidelines\n3. Technology-specific `rules.md` files for their stack\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `9` to `stepsCompleted`\n- Set `status` to `'complete'`\n\n## WORKFLOW COMPLETE\n\nThe setup workflow is complete. The setup checklist is saved at the output path.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2028,"content_sha256":"0a9030ca666bed57ce3a292400750d0d9951f46c12b0859f9a4eee8ec09e8259"},{"filename":"workflows/setup/templates/output-template.md","content":"---\nstepsCompleted: []\nworkflowType: 'setup'\nprojectPath: ''\ntechnologies: []\noutputPath: ''\ndate: ''\nstatus: 'in-progress'\n---\n\n# E2E Setup Checklist\n\n**Date**: {{date}}\n**Project**: {{projectPath}}\n**Technologies**: {{technologies}}\n**Status**: {{status}}\n\n---\n\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":264,"content_sha256":"d0b182f78cd20f0eee3df3b4335ee1d88a89b8ca2a94a0bf41af2d664dbdd4c9"},{"filename":"workflows/setup/workflow.md","content":"---\nname: 'setup'\ndescription: 'Set up a complete E2E testing infrastructure for a TypeScript/NestJS project'\nfirstStepFile: './steps/step-01-init.md'\ntemplateFile: './templates/output-template.md'\n---\n\n# Setup E2E Test Workflow\n\nSet up a complete E2E testing infrastructure for a TypeScript/NestJS project with all necessary configuration, helpers, and Docker infrastructure.\n\n## When to Use\n\n- Setting up E2E testing for a new project\n- Configuring docker-compose for testing (Kafka, PostgreSQL, MongoDB, Redis)\n- Creating jest-e2e.config.ts or E2E Jest configuration\n- Setting up test helpers for database, Kafka, or Redis\n- Configuring .env.e2e environment variables\n- Creating test/e2e directory structure\n\n## Prerequisites\n\nHave access to the project's source code and understand its infrastructure dependencies.\n\n## Step-File Architecture\n\nThis workflow uses a **step-file architecture** for context-safe execution:\n\n- Each step is a separate file loaded sequentially\n- Progress is tracked via `stepsCompleted` in the output document's YAML frontmatter\n- If context is compacted mid-workflow, step-01 detects existing output and resumes from the last completed step via step-01b\n\n### Steps\n\n| Step | File | Description |\n|------|------|-------------|\n| 1 | `step-01-init.md` | Initialize workflow, set output path, detect continuation |\n| 1b | `step-01b-continue.md` | Resume from last completed step |\n| 2 | `step-02-project-analysis.md` | Analyze project structure and dependencies |\n| 3 | `step-03-directory-structure.md` | Create E2E test directory structure |\n| 4 | `step-04-jest-config.md` | Configure Jest for E2E testing |\n| 5 | `step-05-docker-compose.md` | Create Docker Compose for test infrastructure |\n| 6 | `step-06-test-helpers.md` | Create reusable test helper classes |\n| 7 | `step-07-global-setup.md` | Create global E2E test setup file |\n| 8 | `step-08-example-test.md` | Create example E2E test demonstrating patterns |\n| 9 | `step-09-verification.md` | Verify setup works correctly, mark complete |\n\n### Rules\n\n1. **Load one step at a time** - Read the step file, execute it, then load the next\n2. **Update frontmatter after each step** - Add the step number to `stepsCompleted`\n3. **Wait for user confirmation** - Present findings and wait for `[C]` before proceeding\n4. **Load reference files** - Each step specifies which reference files to load before action\n5. **Ask before creating files** - Always confirm with user before creating/modifying files\n\n### Mandatory Execution Rules\n\n- **NEVER proceed without understanding project context first**\n- **ALWAYS load and review knowledge references before taking action**\n- **ALWAYS ask user for confirmation before creating files**\n- **ALWAYS verify Docker and dependencies are available**\n\n### Failure Recovery\n\nIf any step fails:\n1. Report the specific error\n2. Reference relevant documentation from `references/`\n3. Offer manual troubleshooting steps\n4. Ask user if they want to retry or skip\n\n## Begin\n\nLoad `steps/step-01-init.md` to start.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3025,"content_sha256":"49277713b85b553bed78a8976f06fbbdd55d6aa675ce169ea3aa541b9ceb38f5"},{"filename":"workflows/writing/steps/step-01-init.md","content":"---\nname: 'step-01-init'\ndescription: 'Initialize writing workflow — set output path, load knowledge, detect continuation'\nnextStepFile: './step-02-requirements.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/test-case-creation-guide.md'\n - 'references/common/best-practices.md'\n---\n\n# Step 1: Initialize E2E Test Writing\n\n## STEP GOAL\n\nSet up the test writing session: identify the feature to test, set the output path, load knowledge base, and check for existing output to resume.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/rules.md` - GWT pattern, timeout rules, isolation requirements\n- `references/common/test-case-creation-guide.md` - Templates for all scenarios\n- `references/common/best-practices.md` - Test design patterns\n\n## EXECUTION\n\n### 1. Ask the User\n\nAsk the user:\n- **What feature/flow to test?** (endpoint, workflow, event handler)\n- **Output path** for the test plan document (suggest a default: `./e2e-test-plan-{{date}}.md`)\n- **Or provide path to an existing plan** to resume\n\n### 2. Check for Existing Output\n\nIf the user provides a path to an existing plan file:\n- Read the file\n- Parse the YAML frontmatter\n- If `stepsCompleted` is non-empty → **STOP and load `step-01b-continue.md`**\n\n### 3. Fresh Workflow Setup\n\nIf starting fresh:\n1. Copy the template from `templates/output-template.md`\n2. Fill in the frontmatter:\n - `targetFeature`: the feature/flow to test\n - `technologies`: detected technologies\n - `testFilePath`: planned test file path\n - `outputPath`: the chosen output path\n - `date`: current date\n3. Write the initialized plan to the output path\n\n### 4. Load Technology-Specific References\n\nBased on technologies involved:\n- **Kafka tests**: `references/kafka/rules.md`, `references/kafka/isolation.md`, `references/kafka/examples.md`\n- **PostgreSQL tests**: `references/postgres/rules.md`, `references/postgres/examples.md`\n- **MongoDB tests**: `references/mongodb/rules.md`, `references/mongodb/examples.md`\n- **Redis tests**: `references/redis/rules.md`, `references/redis/examples.md`\n- **API tests**: `references/api/rules.md`, `references/api/examples.md`, `references/api/mocking.md`\n\n### 5. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 1: Session Setup\n\n**Feature to Test**: {{targetFeature}}\n**Technologies**: {{technologies}}\n**Test File**: {{testFilePath}}\n**References Loaded**: {{list of loaded references}}\n```\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `1` to `stepsCompleted`\n\n## PRESENT TO USER\n\nShow the user:\n- Confirmation of feature, technologies, and test file path\n- References loaded\n\nThen ask: **[C] Continue to Step 2: Requirements**\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-02-requirements.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2802,"content_sha256":"1d983f791e43ac02c781774e15bf6dcb5220c6b4fa50ad6e0bf4e52affebdd3d"},{"filename":"workflows/writing/steps/step-01b-continue.md","content":"---\nname: 'step-01b-continue'\ndescription: 'Resume writing workflow from last completed step'\n---\n\n# Step 1b: Continue Previous Session\n\n## STEP GOAL\n\nResume a previously started test writing session by reading the existing plan, determining progress, and routing to the next incomplete step.\n\n## EXECUTION\n\n### 1. Read Existing Output\n\n- Read the file at the output path provided by the user\n- Parse the YAML frontmatter\n- Extract `stepsCompleted` array\n\n### 2. Show Progress Summary\n\nDisplay to the user:\n\n```\nE2E Test Writing Progress\n=========================\nFeature: {{targetFeature}}\nDate Started: {{date}}\nSteps Completed: {{stepsCompleted}}\n\nStep Map:\n [1] Initialize & Setup {{done/pending}}\n [2] Requirements {{done/pending}}\n [3] Code Analysis {{done/pending}}\n [4] Test Structure {{done/pending}}\n [5] Test Setup (Lifecycle) {{done/pending}}\n [6] Happy Path Tests {{done/pending}}\n [7] Error Case Tests {{done/pending}}\n [8] Async Tests {{done/pending}}\n [9] Final Review {{done/pending}}\n```\n\n### 3. Offer Options\n\nPresent:\n- **[R] Resume** from the next incomplete step\n- **[O] Overview** — re-read the existing plan content before resuming\n- **[X] Start over** — create a fresh plan (confirm: this will overwrite)\n\n### 4. Route to Next Step\n\nOn **[R]** or after **[O]**:\n\nDetermine the next step from `max(stepsCompleted) + 1` and load the corresponding file:\n\n| Next Step | File |\n|-----------|------|\n| 2 | `step-02-requirements.md` |\n| 3 | `step-03-code-analysis.md` |\n| 4 | `step-04-test-structure.md` |\n| 5 | `step-05-test-setup.md` |\n| 6 | `step-06-happy-path.md` |\n| 7 | `step-07-error-cases.md` |\n| 8 | `step-08-async-tests.md` |\n| 9 | `step-09-final-review.md` |\n\nOn **[X]**: Go back to `step-01-init.md` fresh workflow setup (section 3).\n\n## NEXT STEP\n\nLoad the step file determined above.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1938,"content_sha256":"435db05f8411f291a9977635c6ded37dd376a381b92a9e5978ef811639f1b09f"},{"filename":"workflows/writing/steps/step-02-requirements.md","content":"---\nname: 'step-02-requirements'\ndescription: 'Understand test requirements — clarify behavior, error cases, technologies'\nnextStepFile: './step-03-code-analysis.md'\nreferenceFiles:\n - 'references/common/test-case-creation-guide.md'\n---\n\n# Step 2: Understand Test Requirements\n\n## STEP GOAL\n\nClarify what behavior needs to be tested, including happy paths, error cases, and technologies involved.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/test-case-creation-guide.md` — Templates for all test scenarios\n\n## EXECUTION\n\n### 1. Questions to Ask User\n\n1. What feature/flow are you testing?\n2. What is the expected behavior (happy path)?\n3. What error cases should be covered?\n4. What technologies are involved (DB, Kafka, external APIs)?\n\n### 2. Document Requirements\n\nBased on user answers, document:\n- Feature description\n- Happy path scenarios\n- Error scenarios\n- Technologies and their roles\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 2: Test Requirements\n\n**Feature**: {{feature description}}\n\n**Involved Technologies**:\n- {{list technologies and their roles}}\n\n**Test Cases to Write**:\n1. {{happy path case}}\n2. {{error case 1}}\n3. {{error case 2}}\n...\n```\n\n## PRESENT FINDINGS\n\nPresent to the user:\n\n```\nI understand you want to test: {feature}\n\nInvolved Technologies:\n- {list technologies}\n\nTest Cases to Write:\n1. {happy path case}\n2. {error case 1}\n3. {error case 2}\n...\n\nLoading relevant references for {technologies}...\n```\n\nThen ask: **[C] Continue to Step 3: Code Analysis / [M] Modify test cases**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `2` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-03-code-analysis.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1740,"content_sha256":"c40766966fa4c954cedcf1ca6f3eb2e7fe7885d806dee5b760fa53511f007046"},{"filename":"workflows/writing/steps/step-03-code-analysis.md","content":"---\nname: 'step-03-code-analysis'\ndescription: 'Analyze existing code — identify inputs, DB ops, async operations, error paths'\nnextStepFile: './step-04-test-structure.md'\n---\n\n# Step 3: Analyze Existing Code\n\n## STEP GOAL\n\nUnderstand the implementation being tested to inform test design.\n\n## EXECUTION\n\n### 1. Read Source Code\n\nRead the source file(s) being tested and identify:\n- Input parameters and validation rules\n- Database operations (what gets created/updated/deleted)\n- External API calls (what needs mocking)\n- Async operations (Kafka events, webhooks)\n- Error handling paths\n\n### 2. Document Analysis\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 3: Code Analysis\n\n**Endpoint/Function**: {{name}}\n**Input Validation**: {{rules}}\n\n**Database Operations**:\n- Creates: {{entities}}\n- Updates: {{entities}}\n- Queries: {{entities}}\n\n**Async Operations**:\n- Publishes to: {{topics}}\n- Consumes from: {{topics}}\n\n**External Dependencies** (need mocking):\n- {{API 1}}\n- {{API 2}}\n```\n\n## PRESENT FINDINGS\n\nPresent to the user:\n\n```\nCode Analysis:\n\n**Endpoint/Function**: {name}\n**Input Validation**: {rules}\n**Database Operations**:\n- Creates: {entities}\n- Updates: {entities}\n- Queries: {entities}\n\n**Async Operations**:\n- Publishes to: {topics}\n- Consumes from: {topics}\n\n**External Dependencies** (need mocking):\n- {API 1}\n- {API 2}\n```\n\nThen ask: **[C] Continue to Step 4: Test Structure / [Q] Ask questions**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `3` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-04-test-structure.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1611,"content_sha256":"7694bac7e376a95f8ff1bca3de889ec02747122ce7d7eb0f496b4dfbdab7f59f"},{"filename":"workflows/writing/steps/step-04-test-structure.md","content":"---\nname: 'step-04-test-structure'\ndescription: 'Design test file structure — describe blocks, test cases, setup/cleanup plan'\nnextStepFile: './step-05-test-setup.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/examples.md'\n---\n\n# Step 4: Design Test Structure\n\n## STEP GOAL\n\nPlan the test file structure following patterns from references.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/rules.md` — structural rules\n- `references/common/examples.md` — reference implementations\n\n## EXECUTION\n\n### 1. Design Structure\n\n1. Design describe blocks based on feature organization\n2. Plan test cases with descriptive names\n3. Identify shared setup vs per-test setup\n4. Determine cleanup requirements\n\n### 2. Test File Template\n\n```typescript\ndescribe('{Feature} E2E', () => {\n let app: INestApplication;\n let httpServer: any;\n // Technology-specific helpers\n\n beforeAll(async () => {\n // App initialization\n // Helper initialization (pre-subscription for Kafka)\n }, 90000);\n\n afterAll(async () => {\n // Cleanup connections\n // Close app\n }, 30000);\n\n beforeEach(async () => {\n // Wait for in-flight operations\n // Clear database/message buffers\n });\n\n describe('{Sub-feature}', () => {\n it('should {expected behavior}', async () => {\n // GIVEN: {setup}\n // WHEN: {action}\n // THEN: {assertions}\n });\n });\n});\n```\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 4: Test Structure\n\n**Test File**: {{testFilePath}}\n\n**Structure**:\n{{show describe block hierarchy}}\n\n**Test Cases**:\n1. should {{case 1}} - Tests {{scenario}}\n2. should {{case 2}} - Tests {{scenario}}\n...\n```\n\n## PRESENT FINDINGS\n\nPresent the proposed structure to the user.\n\nThen ask: **[C] Continue to Step 5: Test Setup / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `4` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-05-test-setup.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1991,"content_sha256":"3220134a8a71d9c116a32b06af6b978ac5eccbcc4f787632e4a0794cfd6535df"},{"filename":"workflows/writing/steps/step-05-test-setup.md","content":"---\nname: 'step-05-test-setup'\ndescription: 'Write test lifecycle hooks — beforeAll, beforeEach, afterAll'\nnextStepFile: './step-06-happy-path.md'\nreferenceFiles:\n - 'references/common/nestjs-setup.md'\n---\n\n# Step 5: Write Test Setup\n\n## STEP GOAL\n\nWrite proper test lifecycle hooks (beforeAll/beforeEach/afterAll).\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/nestjs-setup.md` — NestJS test app bootstrap\n\nAlso load technology-specific test-helper files as needed:\n- `references/kafka/test-helper.md`\n- `references/postgres/test-helper.md`\n- `references/mongodb/test-helper.md`\n- `references/redis/test-helper.md`\n- `references/api/test-helper.md`\n\n## EXECUTION\n\n### 1. Write beforeAll\n\n- Create NestJS test module\n- Initialize app with proper config\n- Set up technology-specific helpers\n- For Kafka: Pre-subscribe to output topics\n\n### 2. Write beforeEach\n\n- Wait for in-flight operations (500-1000ms)\n- Clear database tables/collections\n- Clear message buffers\n\n### 3. Write afterAll\n\n- Disconnect helpers\n- Close app\n\n### 4. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 5: Test Setup\n\n### beforeAll\n{{show beforeAll code}}\n\n### beforeEach\n{{show beforeEach code}}\n\n### afterAll\n{{show afterAll code}}\n\n**Patterns from**: references/common/nestjs-setup.md, references/{tech}/test-helper.md\n```\n\n## PRESENT FINDINGS\n\nPresent the setup code to the user.\n\nThen ask: **[C] Continue to Step 6: Happy Path Tests / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `5` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-06-happy-path.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1639,"content_sha256":"98b296bf4da595eda073d4ddedf43ba7adbcb2758f82107786a93155d017b9d2"},{"filename":"workflows/writing/steps/step-06-happy-path.md","content":"---\nname: 'step-06-happy-path'\ndescription: 'Write happy path test cases following GWT pattern'\nnextStepFile: './step-07-error-cases.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/test-case-creation-guide.md'\n---\n\n# Step 6: Write Happy Path Tests\n\n## STEP GOAL\n\nWrite each happy path test case following GWT pattern.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/rules.md` — GWT rules\n- `references/common/test-case-creation-guide.md` — test case templates\n\n## EXECUTION\n\nFor each happy path test case:\n\n### 1. GIVEN Section (Setup)\n- Create test data using factories\n- Seed database if needed\n- Configure mocks for external APIs\n- Use unique identifiers (timestamp + random)\n\n### 2. WHEN Section (Action)\n- Execute EXACTLY ONE action\n- Capture response/result\n- For async: Trigger event and wait for result\n\n### 3. THEN Section (Assertions)\n- Assert specific values, not just existence\n- Use `toMatchObject` for partial matching\n- Verify database state changes\n- Verify async side effects (Kafka messages)\n\n### GWT Template\n\n```typescript\nit('should {expected behavior}', async () => {\n // GIVEN: {describe setup}\n const testData = createTestData({\n id: `test-${Date.now()}-${Math.random().toString(36).slice(2)}`,\n });\n await repository.save(testData);\n\n // WHEN: {describe action}\n const response = await request(httpServer)\n .post('/endpoint')\n .send(requestBody)\n .expect(201);\n\n // THEN: {describe expectations}\n expect(response.body).toMatchObject({\n code: 'SUCCESS',\n data: {\n id: expect.any(String),\n field: expectedValue,\n },\n });\n\n // Verify database state\n const saved = await repository.findOne({ where: { id: response.body.data.id } });\n expect(saved).toMatchObject({ field: expectedValue });\n});\n```\n\n### Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 6: Happy Path Tests\n\n### Test: should {{description}}\n{{show code}}\n\nVerifies:\n- {{verification 1}}\n- {{verification 2}}\n\nPattern Reference: references/common/test-case-creation-guide.md\n```\n\n## PRESENT FINDINGS\n\nFor each test case, present the code and what it verifies.\n\nThen ask: **[C] Continue to Step 7: Error Cases / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `6` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-07-error-cases.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2387,"content_sha256":"497e8dbdde874678214255ee58aa36ea884445631b339e1a157fc81530679a87"},{"filename":"workflows/writing/steps/step-07-error-cases.md","content":"---\nname: 'step-07-error-cases'\ndescription: 'Write error case tests — validation, not found, authorization, no side effects'\nnextStepFile: './step-08-async-tests.md'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/test-case-creation-guide.md'\n---\n\n# Step 7: Write Error Case Tests\n\n## STEP GOAL\n\nTest error handling paths with proper verification that no side effects occur.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/common/rules.md` — error test rules\n- `references/common/test-case-creation-guide.md` — error test templates\n\n## EXECUTION\n\n### 1. Error Cases to Cover\n\n1. Test validation errors (400)\n2. Test not found errors (404)\n3. Test authorization errors (401/403)\n4. Test business logic errors\n5. Verify NO side effects occur on error\n\n### 2. Error Test Template\n\n```typescript\nit('should return 400 when {invalid condition}', async () => {\n // GIVEN: Invalid input\n const invalidData = { field: 'invalid-value' };\n\n // WHEN: Attempting action\n const response = await request(httpServer)\n .post('/endpoint')\n .send(invalidData)\n .expect(400);\n\n // THEN: Validation error returned\n expect(response.body).toMatchObject({\n code: 'VALIDATION_ERROR',\n errors: expect.arrayContaining([\n expect.objectContaining({ field: 'field' }),\n ]),\n });\n\n // AND: No side effects\n const count = await repository.count();\n expect(count).toBe(0);\n});\n```\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 7: Error Case Tests\n\n### Test: should return {{status}} when {{condition}}\n{{show code}}\n\nVerifies:\n- Correct error response\n- No database changes\n- No events published\n```\n\n## PRESENT FINDINGS\n\nFor each error case, present the code and verifications.\n\nThen ask: **[C] Continue to Step 8: Async Tests / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `7` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-08-async-tests.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1983,"content_sha256":"3d688c187ecb94573a8809456dc168f0d74c61e377908983a0be4b06cd2c8a56"},{"filename":"workflows/writing/steps/step-08-async-tests.md","content":"---\nname: 'step-08-async-tests'\ndescription: 'Write async tests — Kafka events, webhooks, with proper polling'\nnextStepFile: './step-09-final-review.md'\nreferenceFiles:\n - 'references/kafka/isolation.md'\n - 'references/kafka/rules.md'\n - 'references/kafka/examples.md'\n---\n\n# Step 8: Write Async Tests (If Applicable)\n\n## STEP GOAL\n\nTest asynchronous operations (Kafka, webhooks) with proper pre-subscription and smart polling.\n\n## REFERENCE LOADING\n\nBefore starting, load and read:\n- `references/kafka/isolation.md` — pre-subscription pattern details\n- `references/kafka/rules.md` — Kafka-specific testing rules\n- `references/kafka/examples.md` — Kafka test examples\n\n## EXECUTION\n\n**If no async operations are involved, skip this step and proceed to step 9.**\n\n### 1. Kafka Test Patterns\n\nUse smart polling instead of fixed waits. Verify message content with specific assertions.\n\n### 2. Async Test Template\n\n```typescript\nit('should process event and produce output', async () => {\n // GIVEN: Valid event\n kafkaHelper.clearMessages(outputTopic);\n const event = createTestEvent({ id: `event-${Date.now()}` });\n\n // WHEN: Publishing event\n await kafkaHelper.publishEvent(inputTopic, event, event.id);\n\n // THEN: Output event published with correct data\n const messages = await kafkaHelper.waitForMessages(outputTopic, 1, 20000);\n expect(messages).toHaveLength(1);\n expect(messages[0].value).toMatchObject({\n id: event.id,\n status: 'PROCESSED',\n });\n\n // AND: Database updated\n const record = await repository.findOne({ where: { eventId: event.id } });\n expect(record).toMatchObject({ status: 'COMPLETED' });\n});\n```\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 8: Async Tests\n\n### Test: should {{description}}\n{{show code}}\n\nPattern: Pre-subscription + smart polling (from references/kafka/isolation.md)\n```\n\n## PRESENT FINDINGS\n\nFor each async test, present the code and pattern used.\n\nThen ask: **[C] Continue to Step 9: Final Review / [M] Modify**\n\n## FRONTMATTER UPDATE\n\nUpdate the output document:\n- Add `8` to `stepsCompleted`\n\n## NEXT STEP\n\nAfter user confirms `[C]`, load `step-09-final-review.md`.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2171,"content_sha256":"ec520e131703038f99ac16fc204420419c97ec255c7dcf9efe202aec21852a3f"},{"filename":"workflows/writing/steps/step-09-final-review.md","content":"---\nname: 'step-09-final-review'\ndescription: 'Final review against rules — verify GWT, isolation, assertions, mark complete'\nreferenceFiles:\n - 'references/common/rules.md'\n - 'references/common/examples.md'\n---\n\n# Step 9: Final Review\n\n## STEP GOAL\n\nReview the complete test file against all rules and patterns before saving.\n\n## REFERENCE LOADING\n\nBefore starting review, load and read:\n- `references/common/rules.md` — verify compliance\n- `references/common/examples.md` — compare with reference implementations\n\n## EXECUTION\n\n### 1. Review Checklist\n\n- [ ] All tests follow GWT pattern\n- [ ] Each test has exactly ONE action in WHEN\n- [ ] No conditional assertions\n- [ ] Unique identifiers used (no hardcoded IDs)\n- [ ] Database cleaned in beforeEach\n- [ ] Specific assertions (not just `toBeDefined`)\n- [ ] Error cases verify no side effects\n- [ ] Timeouts appropriate for operations\n- [ ] Async tests use polling, not fixed waits\n\n### 2. Present Complete Test File\n\nShow the complete test file for final confirmation.\n\n### 3. Append to Output\n\nAppend to the output document:\n\n```markdown\n## Step 9: Final Review\n\n### Compliance Check\n{{checklist results}}\n\n### Complete Test File\n{{show final code}}\n```\n\n## PRESENT TO USER\n\n```\nTest File Review:\n\n✅ {check 1}\n✅ {check 2}\n⚠️ {warning if any}\n\nComplete test file:\n{show final code}\n\n[C] Confirm and save / [M] Modify\n```\n\n### Post-Workflow Recommendations\n\nAfter writing tests, recommend reviewing:\n1. `references/common/examples.md` - Verify patterns match examples\n2. Technology-specific `examples.md` - Compare with reference implementations\n3. `references/common/debugging.md` - For when tests fail\n\n## FRONTMATTER UPDATE\n\nUpdate the output document frontmatter:\n- Add `9` to `stepsCompleted`\n- Set `status` to `'complete'`\n\n## WORKFLOW COMPLETE\n\nThe test writing workflow is complete. The test plan and code are saved at the output path.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1915,"content_sha256":"ba4fb09a7df0aa71d8d38f5e9d6db32e9090a81b65bb2548931a4d1ff0979e09"},{"filename":"workflows/writing/workflow.md","content":"---\nname: 'writing'\ndescription: 'Write high-quality E2E tests following GWT pattern with proper isolation, assertions, and cleanup'\nfirstStepFile: './steps/step-01-init.md'\ntemplateFile: './templates/output-template.md'\n---\n\n# Writing E2E Test Workflow\n\nWrite high-quality E2E tests following GWT pattern with proper isolation, assertions, and cleanup for TypeScript/NestJS applications.\n\n## When to Use\n\n- Creating new E2E test cases with proper GWT pattern\n- Testing API endpoints, workflows, or complete features end-to-end\n- Testing with real databases, message brokers, or external services\n- Testing Kafka consumers/producers, event-driven workflows\n\n## Prerequisites\n\nHave the source code to test accessible and understand what behavior needs to be tested.\n\n## Step-File Architecture\n\nThis workflow uses a **step-file architecture** for context-safe execution:\n\n- Each step is a separate file loaded sequentially\n- Progress is tracked via `stepsCompleted` in the output document's YAML frontmatter\n- If context is compacted mid-workflow, step-01 detects existing output and resumes from the last completed step via step-01b\n\n### Steps\n\n| Step | File | Description |\n|------|------|-------------|\n| 1 | `step-01-init.md` | Initialize workflow, set output path, detect continuation |\n| 1b | `step-01b-continue.md` | Resume from last completed step |\n| 2 | `step-02-requirements.md` | Understand test requirements |\n| 3 | `step-03-code-analysis.md` | Analyze existing code to be tested |\n| 4 | `step-04-test-structure.md` | Design test file structure |\n| 5 | `step-05-test-setup.md` | Write test lifecycle hooks |\n| 6 | `step-06-happy-path.md` | Write happy path test cases |\n| 7 | `step-07-error-cases.md` | Write error case tests |\n| 8 | `step-08-async-tests.md` | Write async tests (if applicable) |\n| 9 | `step-09-final-review.md` | Final review against rules, mark complete |\n\n### Rules\n\n1. **Load one step at a time** - Read the step file, execute it, then load the next\n2. **Update frontmatter after each step** - Add the step number to `stepsCompleted`\n3. **Wait for user confirmation** - Present findings and wait for `[C]` before proceeding\n4. **Load reference files** - Each step specifies which reference files to load before analysis\n5. **Cite specific rules** - When writing tests, follow patterns from reference files\n\n### Mandatory Execution Rules\n\n- **ALWAYS load knowledge references before writing any test**\n- **ALWAYS follow GWT (Given-When-Then) pattern**\n- **ALWAYS ensure test isolation (no shared state between tests)**\n- **NEVER write tests with conditional assertions**\n- **NEVER mock databases or message brokers in E2E tests**\n\n### Anti-Patterns to Avoid\n\n1. Multiple WHEN actions in one test\n2. Conditional assertions (`if/else` in THEN)\n3. Hardcoded test IDs\n4. Mocking databases\n5. Generic assertions (`toBeDefined`, `toBeTruthy`)\n6. Fixed waits for async operations\n7. Shared state between tests\n8. fromBeginning: true for Kafka consumers\n\n## Begin\n\nLoad `steps/step-01-init.md` to start.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3027,"content_sha256":"c4bb8475bee88aa46e842e55a093b87b2090c67331cbdc23357a3115946aba0b"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"E2E Testing Skill","type":"text"}]},{"type":"paragraph","content":[{"text":"E2E testing validates complete workflows from user perspective, using real infrastructure via Docker.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Workflows","type":"text"}]},{"type":"paragraph","content":[{"text":"For comprehensive step-by-step guidance, use the appropriate workflow:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Workflow","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"When to Use","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Setup E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/setup/workflow.md","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Setting up E2E infrastructure for a new or existing project","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Writing E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/writing/workflow.md","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Creating new E2E test cases with proper GWT pattern","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Review E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/review/workflow.md","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Reviewing existing tests for quality and correctness","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Running E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/running/workflow.md","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Executing tests with proper verification","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Debugging E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/debugging/workflow.md","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Systematically fixing failing tests","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Optimize E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/optimize/workflow.md","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Improving test suite performance","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Workflow Selection Guide","type":"text"}]},{"type":"paragraph","content":[{"text":"IMPORTANT","type":"text","marks":[{"type":"strong"}]},{"text":": Before starting any E2E testing task, identify the user's intent and load the appropriate workflow.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Detect User Intent → Select Workflow","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"User Says / Wants","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Workflow to Load","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"File","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\"Set up E2E tests\", \"configure docker-compose\", \"add E2E to project\", \"create test helpers\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Setup","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"workflows/setup/workflow.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\"Write E2E tests\", \"add integration tests\", \"test this endpoint\", \"create e2e-spec\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Writing","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"workflows/writing/workflow.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\"Review E2E tests\", \"check test quality\", \"audit tests\", \"is this test correct?\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Reviewing","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"workflows/review/workflow.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\"Run E2E tests\", \"execute tests\", \"start docker and test\", \"check if tests pass\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Running","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"workflows/running/workflow.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\"Fix E2E tests\", \"debug tests\", \"tests are failing\", \"flaky test\", \"connection error\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Debugging","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"workflows/debugging/workflow.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\"Speed up E2E tests\", \"optimize tests\", \"tests are slow\", \"reduce test time\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Optimizing","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"workflows/optimize/workflow.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Workflow Execution Protocol","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"ALWAYS load the workflow file first","type":"text","marks":[{"type":"strong"}]},{"text":" - Read the full workflow before taking action","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Follow each step in order","type":"text","marks":[{"type":"strong"}]},{"text":" - Complete checkpoints before proceeding","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Load knowledge files as directed","type":"text","marks":[{"type":"strong"}]},{"text":" - Each workflow specifies which ","type":"text"},{"text":"references/","type":"text","marks":[{"type":"code_inline"}]},{"text":" files to read","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Verify compliance after completion","type":"text","marks":[{"type":"strong"}]},{"text":" - Re-read relevant reference files to ensure quality","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Important","type":"text","marks":[{"type":"strong"}]},{"text":": Each workflow includes instructions to load relevant knowledge from the ","type":"text"},{"text":"references/","type":"text","marks":[{"type":"code_inline"}]},{"text":" folder before and after completing tasks.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Knowledge Base Structure","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"references/\n├── common/ # Shared testing fundamentals\n│ ├── knowledge.md # Core E2E concepts and test pyramid\n│ ├── rules.md # Mandatory testing rules (GWT, timeouts, logging)\n│ ├── best-practices.md # Test design and cleanup patterns\n│ ├── test-case-creation-guide.md # GWT templates for all scenarios\n│ ├── nestjs-setup.md # NestJS app bootstrap and Jest config\n│ ├── debugging.md # VS Code config and log analysis\n│ └── examples.md # Comprehensive examples by category\n│\n├── kafka/ # Kafka-specific testing\n│ ├── knowledge.md # Why common approaches fail, architecture\n│ ├── rules.md # Kafka-specific testing rules\n│ ├── test-helper.md # KafkaTestHelper implementation\n│ ├── docker-setup.md # Redpanda/Kafka Docker configs\n│ ├── performance.md # Optimization techniques\n│ ├── isolation.md # Pre-subscription pattern details\n│ └── examples.md # Kafka test examples\n│\n├── postgres/ # PostgreSQL-specific testing\n│ ├── knowledge.md # PostgreSQL testing concepts\n│ ├── rules.md # Cleanup, transaction, assertion rules\n│ ├── test-helper.md # PostgresTestHelper implementation\n│ └── examples.md # CRUD, transaction, constraint examples\n│\n├── mongodb/ # MongoDB-specific testing\n│ ├── knowledge.md # MongoDB testing concepts\n│ ├── rules.md # Document cleanup and assertion rules\n│ ├── test-helper.md # MongoDbTestHelper implementation\n│ ├── docker-setup.md # Docker and Memory Server setup\n│ └── examples.md # Document and aggregation examples\n│\n├── redis/ # Redis-specific testing\n│ ├── knowledge.md # Redis testing concepts\n│ ├── rules.md # TTL and pub/sub rules\n│ ├── test-helper.md # RedisTestHelper implementation\n│ ├── docker-setup.md # Docker configuration\n│ └── examples.md # Cache, session, rate limit examples\n│\n└── api/ # API testing (REST, GraphQL, gRPC)\n ├── knowledge.md # API testing concepts\n ├── rules.md # Request/response assertion rules\n ├── test-helper.md # Auth and Supertest helpers\n ├── examples.md # REST, GraphQL, validation examples\n └── mocking.md # MSW and Nock external API mocking","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Quick Reference by Task","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Tip","type":"text","marks":[{"type":"strong"}]},{"text":": For detailed step-by-step guidance, use the ","type":"text"},{"text":"Workflows","type":"text","marks":[{"type":"link","attrs":{"href":"#workflows","title":null}}]},{"text":" section above.","type":"text"}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Setup New E2E Structure","type":"text"}]},{"type":"paragraph","content":[{"text":"Workflow","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"Setup E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/setup/workflow.md","title":null}}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/knowledge.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Understand E2E fundamentals","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/nestjs-setup.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Project setup","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read technology-specific ","type":"text"},{"text":"docker-setup.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" files as needed","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Write Test Cases","type":"text"}]},{"type":"paragraph","content":[{"text":"Workflow","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"Writing E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/writing/workflow.md","title":null}}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"MANDATORY","type":"text","marks":[{"type":"strong"}]},{"text":": Read ","type":"text"},{"text":"references/common/rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - GWT pattern, timeouts","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/test-case-creation-guide.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Templates","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read technology-specific files:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Kafka","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"references/kafka/knowledge.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"test-helper.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"isolation.md","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"PostgreSQL","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"references/postgres/rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"test-helper.md","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"MongoDB","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"references/mongodb/rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"test-helper.md","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Redis","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"references/redis/rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"test-helper.md","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"API","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"references/api/rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"test-helper.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Review Test Quality","type":"text"}]},{"type":"paragraph","content":[{"text":"Workflow","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"Review E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/review/workflow.md","title":null}}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Check against mandatory patterns","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/best-practices.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Quality standards","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read technology-specific ","type":"text"},{"text":"rules.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" files","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Run E2E Tests","type":"text"}]},{"type":"paragraph","content":[{"text":"Workflow","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"Running E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/running/workflow.md","title":null}}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Verify Docker infrastructure is running","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Run tests sequentially with ","type":"text"},{"text":"npm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Follow failure protocol if tests fail","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Debug Failing Tests","type":"text"}]},{"type":"paragraph","content":[{"text":"Workflow","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"Debugging E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/debugging/workflow.md","title":null}}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/debugging.md","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Create ","type":"text"},{"text":"/tmp/e2e-${E2E_SESSION}-failures.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" tracking file","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Fix ONE test at a time","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Optimize Test Performance","type":"text"}]},{"type":"paragraph","content":[{"text":"Workflow","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"Optimize E2E Test","type":"text","marks":[{"type":"link","attrs":{"href":"workflows/optimize/workflow.md","title":null}}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/best-practices.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Performance patterns","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/kafka/performance.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for Kafka tests","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Measure baseline before making changes","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Examples","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"references/common/examples.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for general patterns","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read technology-specific ","type":"text"},{"text":"examples.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for detailed scenarios","type":"text"}]}]}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Core Principles","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"0. Context Efficiency (Temp File Output)","type":"text"}]},{"type":"paragraph","content":[{"text":"ALWAYS redirect E2E test output to temp files, NOT console","type":"text","marks":[{"type":"strong"}]},{"text":". E2E output is verbose and bloats agent context.","type":"text"}]},{"type":"paragraph","content":[{"text":"IMPORTANT","type":"text","marks":[{"type":"strong"}]},{"text":": Redirect output to temp files only (NO console output). Use unique session ID to prevent conflicts.","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Generate unique session ID at start of debugging session\nexport E2E_SESSION=$(date +%s)-$\n\n# Standard pattern - redirect to file only (no console output)\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1\n\n# Read summary only (last 50 lines)\ntail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Get failure details\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log\n\n# Cleanup when done\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md","type":"text"}]},{"type":"paragraph","content":[{"text":"Temp Files","type":"text","marks":[{"type":"strong"}]},{"text":" (with ","type":"text"},{"text":"${E2E_SESSION}","type":"text","marks":[{"type":"code_inline"}]},{"text":" unique per agent):","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"/tmp/e2e-${E2E_SESSION}-output.log","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Full test output","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"/tmp/e2e-${E2E_SESSION}-failures.log","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Filtered failure output","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"/tmp/e2e-${E2E_SESSION}-failures.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Tracking file for one-by-one fixing","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"/tmp/e2e-${E2E_SESSION}-debug.log","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Debug runs","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"/tmp/e2e-${E2E_SESSION}-verify.log","type":"text","marks":[{"type":"code_inline"}]},{"text":" - Verification runs","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"1. Real Infrastructure","type":"text"}]},{"type":"paragraph","content":[{"text":"Test against actual services via Docker. Never mock databases or message brokers for E2E tests.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"2. GWT Pattern (Mandatory)","type":"text"}]},{"type":"paragraph","content":[{"text":"ALL E2E tests MUST follow Given-When-Then:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"it('should create user and return 201', async () => {\n // GIVEN: Valid user data\n const userData = { email: '[email protected]', name: 'Test' };\n\n // WHEN: Creating user\n const response = await request(httpServer)\n .post('/users')\n .send(userData)\n .expect(201);\n\n // THEN: User created with correct data\n expect(response.body.data.email).toBe('[email protected]');\n});","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"3. Test Isolation","type":"text"}]},{"type":"paragraph","content":[{"text":"Each test MUST be independent:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Clean database state in ","type":"text"},{"text":"beforeEach","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Use unique identifiers (consumer groups, topics)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Wait for async operations to complete","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"4. Specific Assertions","type":"text"}]},{"type":"paragraph","content":[{"text":"Assert exact values, not just existence:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"// WRONG\nexpect(response.body.data).toBeDefined();\n\n// CORRECT\nexpect(response.body).toMatchObject({\n code: 'SUCCESS',\n data: { email: '[email protected]', name: 'Test' }\n});","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Project Structure","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"project-root/\n├── src/\n├── test/\n│ ├── e2e/\n│ │ ├── feature.e2e-spec.ts\n│ │ ├── setup.ts\n│ │ └── helpers/\n│ │ ├── test-app.helper.ts\n│ │ ├── postgres.helper.ts\n│ │ ├── mongodb.helper.ts\n│ │ ├── redis.helper.ts\n│ │ └── kafka.helper.ts\n│ └── jest-e2e.config.ts\n├── docker-compose.e2e.yml\n├── .env.e2e\n└── package.json","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Essential Jest Configuration","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"// test/jest-e2e.config.ts\nconst config: Config = {\n preset: 'ts-jest',\n testEnvironment: 'node',\n testMatch: ['**/*.e2e-spec.ts'],\n testTimeout: 25000,\n maxWorkers: 1, // CRITICAL: Sequential execution\n clearMocks: true,\n forceExit: true,\n detectOpenHandles: true,\n};","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Technology-Specific Timeouts","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Technology","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Wait Time","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Strategy","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Kafka","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"10-20s max (polling)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Smart polling with 50ms intervals","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"PostgreSQL","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\u003c1s","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Direct queries","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"MongoDB","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\u003c1s","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Direct queries","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Redis","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"\u003c100ms","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"In-memory operations","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"External API","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"1-5s","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Network latency","type":"text"}]}]}]}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Failure Resolution Protocol","type":"text"}]},{"type":"paragraph","content":[{"text":"CRITICAL: Fix ONE test at a time. NEVER run full suite repeatedly while fixing.","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"When E2E tests fail:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Initialize session","type":"text","marks":[{"type":"strong"}]},{"text":" (once at start):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"export E2E_SESSION=$(date +%s)-$","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Create tracking file","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"/tmp/e2e-${E2E_SESSION}-failures.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" with all failing tests","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Select ONE failing test","type":"text","marks":[{"type":"strong"}]},{"text":" - work on only this test","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Run ONLY that test","type":"text","marks":[{"type":"strong"}]},{"text":" (never full suite):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"npm run test:e2e -- -t \"test name\" > /tmp/e2e-${E2E_SESSION}-debug.log 2>&1\ntail -50 /tmp/e2e-${E2E_SESSION}-debug.log","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Fix the issue","type":"text","marks":[{"type":"strong"}]},{"text":" - analyze error, make targeted fix","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Verify fix","type":"text","marks":[{"type":"strong"}]},{"text":" - run same test 3-5 times:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"for i in {1..5}; do npm run test:e2e -- -t \"test name\" > /tmp/e2e-${E2E_SESSION}-run$i.log 2>&1 && echo \"Run $i: PASS\" || echo \"Run $i: FAIL\"; done","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Mark as FIXED","type":"text","marks":[{"type":"strong"}]},{"text":" in tracking file","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Move to next failing test","type":"text","marks":[{"type":"strong"}]},{"text":" - repeat steps 3-7","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Run full suite ONLY ONCE","type":"text","marks":[{"type":"strong"}]},{"text":" after ALL individual tests pass","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Cleanup","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"rm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"paragraph","content":[{"text":"WHY","type":"text","marks":[{"type":"strong"}]},{"text":": Running full suite wastes time and context. Each failing test pollutes output, making debugging harder.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Common Patterns","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Database Cleanup (PostgreSQL/MongoDB)","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"beforeEach(async () => {\n await new Promise(r => setTimeout(r, 500)); // Wait for in-flight\n await repository.clear(); // PostgreSQL\n // OR\n await model.deleteMany({}); // MongoDB\n});","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Kafka Test Helper Pattern","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"// Use pre-subscription + buffer clearing (NOT fromBeginning: true)\nconst kafkaHelper = new KafkaTestHelper();\nawait kafkaHelper.subscribeToTopic(outputTopic, false);\n// In beforeEach: kafkaHelper.clearMessages(outputTopic);","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Redis Cleanup","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"beforeEach(async () => {\n await redis.flushdb();\n});","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"External API Mock (MSW)","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"mockServer.use(\n http.post('https://api.external.com/endpoint', () => {\n return HttpResponse.json({ status: 'success' });\n })\n);","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Async Event Verification (Kafka)","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"typescript"},"content":[{"text":"// Use smart polling instead of fixed waits\nawait kafkaHelper.publishEvent(inputTopic, event, event.id);\nconst messages = await kafkaHelper.waitForMessages(outputTopic, 1, 20000);\nexpect(messages[0].value).toMatchObject({ id: event.id });","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Debugging Commands","type":"text"}]},{"type":"paragraph","content":[{"text":"All commands redirect output to temp files only (no console output).","type":"text","marks":[{"type":"strong"}]}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Initialize session (once at start)\nexport E2E_SESSION=$(date +%s)-$\n\n# Run specific test (no console output)\nnpm run test:e2e -- -t \"should create user\" > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Run specific file\nnpm run test:e2e -- test/e2e/user.e2e-spec.ts > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Run full suite\nnpm run test:e2e > /tmp/e2e-${E2E_SESSION}-output.log 2>&1 && tail -50 /tmp/e2e-${E2E_SESSION}-output.log\n\n# Get failure details from last run\ngrep -B 2 -A 15 \"FAIL\\|✕\" /tmp/e2e-${E2E_SESSION}-output.log\n\n# Debug with breakpoints (requires console for interactive debugging)\nnode --inspect-brk node_modules/.bin/jest --config test/jest-e2e.config.ts --runInBand\n\n# View application logs (limited)\ntail -100 logs/e2e-test.log\ngrep -i error logs/e2e-test.log | tail -50\n\n# Cleanup session files\nrm -f /tmp/e2e-${E2E_SESSION}-*.log /tmp/e2e-${E2E_SESSION}-*.md","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Anti-Patterns to Avoid","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Multiple WHEN actions","type":"text","marks":[{"type":"strong"}]},{"text":" - Split into separate tests","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Conditional assertions","type":"text","marks":[{"type":"strong"}]},{"text":" - Create deterministic test cases","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Shared state between tests","type":"text","marks":[{"type":"strong"}]},{"text":" - Clean in beforeEach","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Mocking databases","type":"text","marks":[{"type":"strong"}]},{"text":" - Use real connections","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Skipping cleanup","type":"text","marks":[{"type":"strong"}]},{"text":" - Always clean before AND after","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Fixing multiple tests at once","type":"text","marks":[{"type":"strong"}]},{"text":" - Fix one at a time","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Generic assertions","type":"text","marks":[{"type":"strong"}]},{"text":" - Assert specific values","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"fromBeginning: true for Kafka","type":"text","marks":[{"type":"strong"}]},{"text":" - Use pre-subscription + buffer clearing","type":"text"}]}]}]}]},"metadata":{"date":"2026-11-02","name":"typescript-e2e-testing","author":"@skillopedia","source":{"stars":19,"repo_name":"skills","origin_url":"https://github.com/bmad-labs/skills/blob/HEAD/skills/typescript-e2e-testing/SKILL.md","repo_owner":"bmad-labs","body_sha256":"a3cb740fa6e717af388f7afc4b9f98808df3bbf9a5fef37b91ab340d59747f83","cluster_key":"e92143387c16493ceb1fed32060d4c743d3870d289192c1abfbb743c28571544","clean_bundle":{"format":"clean-skill-bundle-v1","source":"bmad-labs/skills/skills/typescript-e2e-testing/SKILL.md","text_bytes":400197,"source_commit":"c7496dd1e96b8f792489e8f9a06200f2f741a709","skills_sh_rank":7822,"attachment_count":100,"text_attachments":100,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/typescript-e2e-testing/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"testing-qa","category_label":"Testing"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"testing-qa","import_tag":"skills-sh-top20k-2026-11","description":"E2E and integration testing for TypeScript/NestJS projects using Jest, supertest, and real infrastructure via Docker (Kafka, PostgreSQL, MongoDB, Redis) with the Given-When-Then pattern. Use whenever the user is working on `.e2e-spec.ts` files or anything under `test/e2e/`, or asks to set up, write, review, run, debug, or optimize E2E or integration tests — including flaky tests, docker-compose for tests, Kafka/Redpanda consumers, test isolation, or GWT compliance."}},"renderedAt":1793671073973}

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.