I Let Claude & GPT-4 Play My Board Game Until It Was Bug-Free

The Problem With Friends as Testers

The itch started the way most hobby projects do — a rainy weekend, a group chat full of "we should meet up" messages that never materialised into plans, and a half-formed idea: what if we could play Monopoly Deal online, just the four of us?

Months later I had working web implementations of both Monopoly Deal (the fast-paced card game version) and Scotland Yard (the asymmetric hidden-movement deduction game). The core mechanics worked. The UI was clean. I had even written a game simulator that ran thousands of automated playthroughs to verify turn sequencing and win conditions.

But every time we played with real people, something broke. Not dramatically — no crashes, no data corruption. It was subtler: a rent card applied twice because a player double-tapped on a laggy connection; a Scotland Yard move that was technically valid but left the board in a state the game had never anticipated; a "Just Say No" counter-play card chain that produced an outcome none of us could agree was correct.

The simulator couldn't catch these because it played ideally. Real players don't play ideally. They probe the edges. They do unusual sequences. They find the seams.

I needed testers who would do that systematically — and I needed them available at 2am when I had a free hour to code.

AI as Adversarial Playtester

The insight was conceptually simple: large language models like Claude and GPT-4 can reason about rules, maintain game state in context, and make strategic decisions. They're also infinitely patient, available via API at any hour, and — critically — they don't play the same way twice.

The plan had three parts:

1. Give each AI model the full rulebook as a system prompt, establishing it as a named player with the explicit goal of winning while strictly following the rules.
2. Build an orchestrator that managed game state, serialised it after every action, sent it to the appropriate AI, parsed its move, applied it to the game engine, and looped.
3. Hook in an edge-case tracker — whenever the game engine threw an unexpected state, a validation failed, or an AI attempted an action the engine didn't know how to handle, that situation was logged automatically into a structured bug document.

Run this for long enough, with two AI players who are genuinely trying to outmanoeuvre each other, and you'd surface every weird rule interaction the game could produce.

How the System Was Actually Built

The Playwright Question

The original instinct was to have the AI models control the actual browser UI via Playwright — literally clicking buttons, selecting cards, dragging pieces, the way a human would. On paper this feels elegant: if the AI can navigate the UI, you're testing the full stack including the frontend interaction layer.

In practice, this approach has a fundamental friction problem. LLMs don't produce Playwright commands natively — you'd need an intermediate layer that converts the AI's natural-language decision ("I want to play the Rent card targeting blue properties against Player 2") into a precise sequence of DOM interactions. That translation layer becomes its own source of bugs, and any frontend change breaks the automation.

The result was a three-layer architecture:

On each turn, the orchestrator serialised the full game state — every player's hand, the board, discard piles, whose turn it was, what phase of the turn was active — into a structured JSON payload. That payload became the user message to the active AI player (alternating between Claude and GPT-4).

The System Prompt: Giving the AI a Rulebook

The system prompt was doing serious heavy lifting. For Monopoly Deal, it was around 1,800 words covering every card type, the property set completion rules, action card interactions, the "Just Say No" chain limit, what constitutes a legal move at each phase, and what format to use when responding.

Crucially, the AI was instructed to respond in strict JSON — not prose. This eliminated parsing ambiguity entirely. A valid move response looked like this:

{
  "action": "PLAY_CARD",
  "cardId": "rent_blue_green_007",
  "targets": ["player_2"],
  "targetProperties": ["prop_blue_mayfair", "prop_blue_park_lane"],
  "useDoubleRent": true,
  "reasoning": "Player 2 has full blue set. Double rent gives 8M."
}

The reasoning field was not used by the engine — it was logged for human review and turned out to be invaluable for understanding why an AI took a move that caused an unexpected state.

The Playwright Layer

Once the orchestrator had a validated, parsed move from the AI, Playwright executed it against the live web application running in a headless Chromium instance. This meant every single game interaction — clicking a card, confirming a payment, moving a detective token — went through the actual frontend code, triggering all the same event handlers, state updates, and websocket messages that a real player would produce.

This was the key insight that made the approach worth the extra complexity: you weren't just testing the game logic engine. You were testing the entire stack, continuously, for days.

async function runSimulationTurn(gameState, activePlayer) {
  // 1. Serialise state and ask the right AI
  const prompt = buildPrompt(gameState);
  const aiResponse = activePlayer === 'claude'
    ? await askClaude(prompt)
    : await askGPT4(prompt);

  // 2. Parse and validate the move
  const move = parseMove(aiResponse);
  const validation = validateMove(gameState, move);

  if (!validation.valid) {
    // Illegal move attempt → log it and request retry
    logEdgeCase('ILLEGAL_MOVE', { move, state: gameState, reason: validation.reason });
    return runSimulationTurn(gameState, activePlayer); // retry
  }

  // 3. Execute via Playwright against real web UI
  await playwrightExecuteMove(move);

  // 4. Capture new state from UI / server
  const newState = await captureGameState();

  // 5. Check for state anomalies
  detectAnomalies(gameState, newState, move);

  return newState;
}

The Edge-Case Tracker: Your Living Bug Document

Every unusual event was written to a structured tracker — not just crashes and errors, but a broader taxonomy of interesting events that warranted review:

1. Hard failures — the game engine threw an unhandled exception or the frontend crashed. These were P0 bugs fixed immediately.
2. Soft failures — the engine produced a valid state that was logically impossible given the rules. For example: a player ending their turn with four cards in hand instead of the legal maximum of three after drawing.
3. AI rule disagreements — Claude and GPT-4 interpreted the same rule differently and attempted incompatible moves in the same scenario. These were flagged for manual rulebook review, because ambiguity in the AI's interpretation usually reflected ambiguity in the implementation.
4. Retry storms — if the same AI player attempted three illegal moves in a row in the same game state, it indicated the engine was presenting a state from which no legal move was possible. A deadlock bug.
5. Performance anomalies — Playwright recorded timing data. Any turn taking over 2 seconds to execute against the UI was logged. These revealed rendering bottlenecks under complex board states.

Each entry in the tracker included the full game state at the time of the anomaly (as a JSON snapshot), the AI's move and reasoning string, the expected outcome, the actual outcome, and an auto-generated severity label. This gave whoever was fixing the bug everything they needed to reproduce it in one place.

What the Simulation Found: Monopoly Deal

Monopoly Deal is deceptively complex for a card game. The core rules are simple, but the interactions between action cards create a combinatorial explosion of states. Here are the most instructive bugs the simulation surfaced:

The Double "Just Say No" Deadlock

This was the first major bug. The rules allow a target to play a "Just Say No" card to block an action, and the original player can counter with their own "Just Say No" to re-activate the action. This creates a chain of up to two blocks. The question is: does the chain end after two "Just Say No" plays, or can it continue indefinitely?

The implementation had no limit. GPT-4 and Claude, both holding "Just Say No" cards, entered a loop. The game never progressed. The fix required implementing a maximum chain depth of two (as per the official rules) — but the simulation discovered this in game 7. A human tester might have taken weeks to construct that specific hand state.

Wild Card Property Set Completion Race

A wild property card can be placed on any colour set. When a player has a near-complete set containing a wild card, and the opponent plays a "Deal Breaker" to steal that set, the question becomes: does the stolen set retain the wild card's current colour assignment, or does it revert to unassigned?

The simulation found this because Claude played a Deal Breaker on a set containing a wild card in game 31, then immediately tried to use the stolen set's wild card in a different colour. The engine allowed it, creating a free colour reassignment exploit. This was fixed by locking the wild card's colour on steal.

Rent Card + Forced Payment + Empty Hand

When a player must pay rent but has no money cards and no property cards to hand over, the rules state they pay what they can and the debt is cleared. The engine was instead entering a payment-pending state indefinitely, blocking the turn. This deadlock showed up in roughly 1 in 400 games — rare enough that human testing had never caught it.

What the Simulation Found: Scotland Yard

Scotland Yard presented a different class of challenges. The game is asymmetric — one player is the fugitive Mr. X, the others are detectives trying to corner him on a map of London. The hidden information mechanic (Mr. X's position is only revealed on specific turns) created edge cases that were genuinely hard to anticipate.

The AI as Mr. X: A Strategic Surprise

When Claude played Mr. X, it played exceptionally conservatively — prioritising board positions with the maximum number of escape routes. GPT-4 as Mr. X played more aggressively, sometimes doubling back deliberately to confuse detectives. Both strategies were valid, but they produced very different board states. The conservative strategy occasionally found moves that exploited an unchecked edge in the transport graph: a taxi route that connected two zones through an intermediary node that hadn't been added to the connectivity validation. Mr. X could "move" to a location the detectives couldn't legally reach, making capture impossible.

Detective Collision Handling

The rules are ambiguous about whether two detectives can occupy the same node simultaneously. The implementation had not made a decision — it allowed it silently, but the UI hadn't been designed for it. When the simulation put two detectives on the same node, the frontend rendered them overlapping and the click target became unpredictable. Minor UI bug, major gameplay confusion.

Double Ticket Exhaustion

Mr. X has a limited supply of "double move" tickets allowing two moves in one turn. The simulation discovered that the engine correctly decremented the ticket count for a double move — but didn't validate that a double move was only legal when the player actually had tickets remaining. Claude attempted a double move with zero tickets; the engine allowed it. The ticket count went negative.

What the AI Got Wrong (And Why That Was Also Useful)

The AI players weren't perfect rule-followers. They made mistakes — and those mistakes were informative in their own right.

Context Window Decay in Long Games

In long Scotland Yard games (25+ turns), both models began making suboptimal moves that suggested they were losing track of information from earlier in the conversation. A detective who had definitively seen Mr. X at a specific node 15 turns ago would sometimes move as if that information hadn't been established. This wasn't a bug in the game — it was a limitation of the AI's effective context window under a large game-state payload. The fix was a "memory compression" step where the orchestrator summarised historical game facts into a compact structured header sent at the start of each prompt.

Rule Hallucination

On three occasions during Monopoly Deal testing, GPT-4 attempted a move that didn't correspond to any card in the game — referencing a card type that doesn't exist in Monopoly Deal but does exist in the original Monopoly board game. The move parser correctly rejected it and logged it, but it underscored that the models were drawing on broad game knowledge, not only the rulebook in the system prompt. Tighter JSON schemas and explicit "only reference card IDs from the following list" instructions reduced this to zero occurrences.

The Simulation Timeline

The simulation ran in phases rather than one continuous block. API rate limits, cost management, and the need to deploy fixes between sessions shaped the schedule:

1. Days 1–2 · Monopoly Deal Phase 1. Initial run, high crash rate. The first 50 games surfaced the most severe bugs — the "Just Say No" loop, the rent deadlock, and three UI rendering failures. These were fixed before continuing.
2. Days 3–4 · Monopoly Deal Phase 2. Re-run with fixes deployed. Significantly more stable. Edge cases became increasingly subtle — timing issues, rare card combinations, payment calculation edge cases with mixed-denomination hands.
3. Days 5–6 · Scotland Yard Phase 1. The graph traversal bug and double-ticket issue appeared in the first day. Mr. X AI revealed the transport graph hole in game 12.
4. Day 7 · Scotland Yard Phase 2 + Combined Regression. Final simulation run covering both games with all fixes applied. Both games completed 200+ consecutive games without a single hard failure or new edge case.

Total cost for API calls across the full simulation: approximately $40–60, split roughly evenly between Anthropic and OpenAI. Cheap by any professional QA standard.

What's Actually Feasible vs. What Needs Adjustment

Let's be direct about where this approach works exactly as described, and where you'd need to adapt it for your own project:

✓ Fully Feasible

LLM-driven game logic simulation works reliably for turn-based card and board games. The models understand rule text well, produce structured JSON moves consistently with appropriate prompting, and genuinely play strategically rather than randomly. This is not a theoretical approach — it produces meaningful test coverage.

Playwright as execution layer is practical and adds real value. Running moves through the actual web UI catches frontend bugs that a pure engine simulation would miss entirely. The key is using Playwright as a controlled executor, not as a natural-language browser agent.

Automated edge-case logging is straightforwardly implementable. The combination of move validation, state anomaly detection, and AI reasoning strings gives you rich, actionable bug reports without manual intervention.

~ Needs Calibration

Context management in long games requires deliberate engineering. You cannot simply append the full game history to every prompt — token costs and model attention limits make this impractical for games running 30+ turns. A structured state summary with only relevant historical facts is essential.

Scotland Yard's hidden information requires careful prompt engineering. The AI must receive only the information its character would legitimately know. Giving Mr. X the detectives' full state (or vice versa) defeats the test's validity and lets the AI exploit information it shouldn't have.

Simulation speed is constrained by API rate limits. A single Monopoly Deal game with Claude and GPT-4 alternating turns takes 3–5 minutes. Plan for asynchronous batch runs, not real-time simulation.

✗ Not Directly Possible

Having the AI control Playwright natively (i.e., writing its own selectors and click sequences) is technically possible via computer-use models but introduces too much translation-layer fragility for this use case. The inversion — orchestrator controls Playwright, AI controls game logic — is far more robust.

Full-stack testing without a real server is a prerequisite. The Playwright layer requires the actual web application running and accessible. This means you need a stable local or staging environment with persistent game state between API calls — not a blocker, but something to set up before the simulation starts.

What I'd Do Differently — and Would Absolutely Do Again

The approach worked. Both games shipped with meaningfully fewer bugs than they would have without the simulation. More importantly, the bugs it caught were the right kind — rare-state logic errors and rule-interaction edge cases that would have surfaced during real play with friends and produced that most frustrating of moments: a disputed game outcome with no clear answer.

If I were starting this today, I'd make three changes. I'd invest more time upfront in the JSON schema design — rigid, versioned move schemas make the parser trivially simple and eliminate an entire category of failures. I'd run the simulation from the very first week of development rather than at the end — many of the bugs fixed near launch could have been caught (and fixed cheaply) much earlier. And I'd add a third AI player, not for the gameplay mechanics, but as a neutral "rules arbiter" model that the other two could query when they attempted a move they were uncertain was legal.

The deeper lesson, though, is about the nature of AI as a testing tool. It isn't a replacement for thoughtful human playtesters. It's something different: a tireless, strategic, rules-aware adversary that will play your game hundreds of times across every combination of decisions, and quietly log every moment where the game doesn't know what to do next. That's not something any human tester will give you — at least, not without significantly more pizza.