GambleBench

Scenario Generation

Scenarios are programmatically generated using deterministic card-dealing algorithms across multiple deck configurations. The generator creates 493 unique situations spanning:

• Single & double deck card counting scenarios with targeted true counts
• Multi-player tables (3-player and 6-player) with visible card information
• Strategy deviations based on Hi-Lo card counting system (running count, true count, deck penetration)
• All difficulty levels from basic strategy to complex count-dependent decisions

Card Counting Integration

Each scenario includes full game context: all dealt cards, running count (Hi-Lo: +1 for 2-6, 0 for 7-9, -1 for 10-A), true count (running count ÷ decks remaining), and deck penetration. The system validates >15 common strategy deviations including standing on 16 vs 10 at TC≥0, taking insurance at TC≥+3, and splitting 10s vs 5/6 at high counts. Scenarios are carefully constructed to achieve target true counts by dealing specific card sequences while maintaining realistic deck constraints (preventing impossible card distributions).

Evaluation & Scoring

Models are evaluated using a custom blackjack evaluator with partial credit scoring:

• 1.0 points: Optimal action (matches precomputed card-counting strategy)
• 0.5 points: Basic strategy action (correct for basic strategy but misses count deviation)
• 0.0 points: Incorrect action or invalid response

This scoring system recognizes that basic strategy decisions are still valuable even when count deviations are missed, providing nuanced evaluation of model capabilities. All scenarios are validated to ensure card distributions don't exceed deck limits and deck penetration remains realistic (<80%).