loop-benchmarking

GAMEPLAY_BOT_SPEC.md (8318B)

---

 # Tetris Gameplay Bot Spec
 
 ## Purpose
 
 A Playwright-based bot that can load any Tetris implementation, figure out how to interact with it, play the game, and report which game mechanics work and which don't. It must handle wildly different implementations -- different DOM structures, canvas vs DOM rendering, different control schemes, start buttons vs auto-start, etc.
 
 ## Architecture
 
 Three phases: **Calibration**, **Play**, **Report**.
 
 ### Phase 1: Calibration
 
 The bot loads the page and figures out how to interact with this specific implementation.
 
 **1a. Start the game**
 
 Try multiple start mechanisms in order, checking after each if the game state changed:
 1. Wait 3 seconds (some games auto-start)
 2. Click the canvas or game container
 3. Press Enter
 4. Press Space
 5. Look for a button with text matching /start|play|begin|new game/i and click it
 6. Press any key
 
 After each attempt, take a screenshot and compare to the previous one. If pixels changed, the game has started.
 
 **1b. Locate the game grid**
 
 The grid could be:
 - A `<canvas>` element
 - A grid of `<div>` or `<td>` elements
 - An SVG
 
 Detection strategy:
 1. Check for a `<canvas>` element. If found, use `getImageData()` to read pixels.
 2. If no canvas, look for a grid-like DOM structure (many sibling elements in a container with grid/flex layout, or a table).
 3. Take a screenshot and look for a rectangular region with a grid pattern.
 
 Once found, determine:
 - Grid pixel bounds (x, y, width, height)
 - Cell size (width / 10, height / 20 for standard Tetris)
 - Sample one pixel per cell to build a 10x20 boolean matrix
 
 **1c. Detect controls**
 
 Default to standard controls: ArrowLeft, ArrowRight, ArrowDown, ArrowUp (rotate), Space (hard drop).
 
 Verify by:
 1. Read the page text/HTML for control instructions (look for "arrow", "wasd", "z", "x", "space", "rotate" etc.)
 2. Press ArrowLeft, take screenshot, check if a piece moved. If not, try "a".
 3. Press ArrowUp, take screenshot, check if a piece rotated. If not, try "z" or "x".
 
 Store the working key mappings.
 
 **1d. Locate score display**
 
 Scan the page for elements containing the text "score" (case insensitive) or elements that contain only a number that changes during gameplay.
 
 ### Phase 2: Play
 
 A deterministic play session that exercises all game mechanics. Not trying to play well -- trying to test everything.
 
 **2a. Test Suite (sequential, do not stop on failure)**
 
 Each test captures before/after state and reports pass/fail independently.
 
 | # | Test | Method | Pass condition |
 |---|------|--------|----------------|
 | 1 | Game loads | Page loads without console errors | No uncaught exceptions in first 3s |
 | 2 | Game starts | Run calibration start sequence | Screenshot changes after start |
 | 3 | Auto-drop | Wait 5s with no input after start | Grid state changes (piece fell) |
 | 4 | Move left | Press left key | Grid state differs from before |
 | 5 | Move right | Press right key | Grid state differs from before |
 | 6 | Move down | Press down key | Grid state differs from before |
 | 7 | Rotate | Press rotate key | Grid state differs, piece shape changed |
 | 8 | Hard drop | Press hard drop key | Piece immediately at bottom, new piece appears |
 | 9 | Piece locks | Wait for a piece to reach bottom via auto-drop (no input for ~15s) | Grid has filled cells at bottom that persist |
 | 10 | New piece spawns | After piece locks, check top of grid | New piece appears at top |
 | 11 | Multiple pieces | Play 10 pieces (hard drop each) | Grid accumulates filled cells |
 | 12 | Line clear | Fill a complete row by strategic placement | At least one row disappears, cells above shift down |
 | 13 | Score changes | Check score element before and after line clear | Score value increased |
 | 14 | Game over | Stack pieces to the top rapidly | Game stops, some game-over indication |
 | 15 | Playable for 30s | Play normally for 30 seconds | No crashes, console errors, or freezes |
 
 **2b. Playing Strategy**
 
 For tests that require actual gameplay (11, 12, 15), use the 4-heuristic algorithm:
 
 ```
 score = -0.51 * aggregateHeight + 0.76 * completeLines - 0.36 * holes - 0.18 * bumpiness
 ```
 
 For each piece:
 1. Read current grid state (10x20 boolean matrix)
 2. Read current piece (detect from grid -- the moving cells)
 3. Try all (rotation, column) placements
 4. Score each resulting board
 5. Execute: rotate N times, move left/right, hard drop
 
 If the bot can't read the grid reliably, fall back to random inputs: cycle through left, right, rotate, down in a fixed pattern.
 
 **2c. Grid Reading**
 
 For canvas-based games:
 ```js
 async function readGrid(page, bounds, cellW, cellH) {
   return await page.evaluate(({ x, y, cellW, cellH }) => {
     const canvas = document.querySelector('canvas');
     const ctx = canvas.getContext('2d');
     const grid = [];
     for (let row = 0; row < 20; row++) {
       const rowData = [];
       for (let col = 0; col < 10; col++) {
         const px = x + col * cellW + cellW / 2;
         const py = y + row * cellH + cellH / 2;
         const pixel = ctx.getImageData(px, py, 1, 1).data;
         // Consider a cell filled if it's not the background color
         const brightness = pixel[0] + pixel[1] + pixel[2];
         rowData.push(brightness > 100); // threshold
       }
       grid.push(rowData);
     }
     return grid;
   }, { x: bounds.x, y: bounds.y, cellW, cellH });
 }
 ```
 
 For DOM-based games:
 ```js
 // Find cells by their grid position, check background color or class
 ```
 
 The background color threshold should be calibrated during Phase 1 by reading the empty grid.
 
 ### Phase 3: Report
 
 Output a JSON report:
 
 ```json
 {
   "implementation": {
     "renderer": "canvas|dom|svg",
     "grid_detected": true,
     "grid_bounds": { "x": 0, "y": 0, "width": 300, "height": 600 },
     "controls": { "left": "ArrowLeft", "right": "ArrowRight", "rotate": "ArrowUp", "drop": "Space" },
     "start_mechanism": "button|auto|keypress",
     "score_element_found": true
   },
   "tests": [
     { "name": "game_loads", "pass": true, "detail": "no console errors" },
     { "name": "game_starts", "pass": true, "detail": "started via button click" },
     { "name": "auto_drop", "pass": false, "detail": "piece did not move in 5 seconds" },
     ...
   ],
   "summary": {
     "total": 15,
     "passed": 12,
     "failed": 3,
     "score": 0.80
   },
   "gameplay": {
     "pieces_placed": 47,
     "lines_cleared": 3,
     "max_score_observed": 400,
     "play_duration_seconds": 30,
     "errors_during_play": 0
   }
 }
 ```
 
 ## Error Handling
 
 - NEVER crash on a single test failure. Each test is independent.
 - If grid detection fails, skip grid-dependent tests but still test basic page load, console errors, and input response via screenshots.
 - If a test times out (e.g., waiting for auto-drop), mark it as failed and move on.
 - Capture all console errors throughout the session and include them in the report.
 - If the game page itself fails to load, report all tests as failed with the error.
 
 ## File Structure
 
 ```
 tasks/tetris/eval/
   gameplay-bot/
     index.ts          # Main entry point, orchestrates calibration + play + report
     calibrate.ts      # Phase 1: detect grid, controls, start mechanism
     grid-reader.ts    # Read grid state from canvas or DOM
     player.ts         # Phase 2: heuristic AI + move execution
     tests.ts          # Individual test implementations
     types.ts          # Shared types
   playwright.config.ts
 ```
 
 ## Dependencies
 
 - `@playwright/test` (already in the project)
 - No other dependencies. Pure Playwright + vanilla JS evaluation.
 
 ## Integration
 
 The harness calls:
 ```bash
 npx playwright test --config=tasks/tetris/eval/playwright.config.ts
 ```
 
 The Playwright test:
 1. Starts an HTTP server for the workspace (serve static files)
 2. Runs the bot against the served game
 3. Writes the JSON report to a specified output path
 4. Exit code 0 regardless of test results (the report contains pass/fail)
 
 ## Constraints
 
 - Must work with canvas-based AND DOM-based Tetris implementations
 - Must handle games that auto-start and games with start buttons
 - Must handle different control schemes
 - Must not depend on any specific DOM structure, class names, or IDs
 - Each test has a timeout (default 10 seconds per test, 30 seconds for the play test)
 - Total bot runtime should be under 2 minutes per game